Skip to content

Add support for pre-multiplied alpha #3391

@neilcsmith-net

Description

@neilcsmith-net

At the bottom of this pull request (#2601) @benfry mentioned opening an issue about implementing pre-multiplied alpha, which I don't think was ever done? I certainly dropped the ball on following up. It would be great if an official image mode for this was added (I'm currently sub-classing the GL renderer to achieve this). Otherwise multiple layering (as well as convolution, etc) don't work correctly.

For reference, a long and admittedly opinionated extract of an email sent to @JakubValtar last year with a suggestion for handling this internally in the OpenGL renderer - feel free to TL;DR, although the NVIDIA link is worth a read.


OK, you asked me how Processing might use pre-multiplied alpha without
breaking the API. So to start, a possibly controversial statement -
blending in Processing is completely broken. It was one of the
reasons I stopped using Processing the first time around (~2006) - I
like the API but in my opinion there are some issues with the
implementation. Praxis LIVE v1 deliberately made use of aspects of
the Processing API, but re-implemented on top of a custom software and
OpenGL renderer. Fast forward to a year ago, and I decided to stop
trying to re-implement the API - the new Processing OpenGL is too good
to miss! So Praxis LIVE v2 now embeds Processing as a library -
however, there's no way I'm breaking blending in the process - luckily
it isn't that difficult to work around.

There are many sources around about pre-multipled alpha. I found this
page recently looking into GL blend modes
(http://www.slideshare.net/Mark_Kilgard/blend-modes-for-opengl) - it's
about the new NVIDIA blend mode extensions - note that slide 16 talks
about the fact that blend modes assume pre-multiplied alpha. It
mentions one of the key issues (one that affects me) - drawing into
surfaces with alpha channels - sometimes you can get away with it if
the destination is opaque. He also mentions the Porter-Duff model and
how it relies on pre-multiplied alpha - interesting that the default
blending in the Java2D renderer works fine - because the JRE
AlphaComposite pre-multiplies, blends and un-premultiplies.

Incidentally, that slide also touches on another issue -
pre-multiplied alpha is necessary for performing various filtering
effects (eg. blur) on surfaces with alpha. I haven't looked at the
PImage filters but I imagine there might be an issue there (note the
ConvolveOp in the JRE multiplies, filters and unmultiplies).

Now, all that multiplying and unmultiplying is a bit pointless! ;-)
Not to mention it's not easily achievable with OpenGL. So, how to use
pre-multiplied alpha without changing the API?

The important thing in my opinion is separating the external format of
images to the format they're stored internally. Take managed images
in the Java2D backend - cached textures are not stored in the format
of the BufferedImage, but the optimum format for rendering. Just
because the format to the user is un-premultiplied ARGB doesn't mean
that the underlying storage has to be. Processing could easily switch
to storing the underlying pixel data, in all modes, as pre-multiplied.
Or, simpler, though not fixing issues elsewhere, it could just store
pre-multiplied data in the OpenGL textures.

The key things to handle are times when the user is setting colours,
times when the user is retrieving colours, and access to the pixel
array.

Processing already does colour calculations to match a colour format
set by the user and convert into internal data. What I've done in
Praxis LIVE's PGraphics sub-class is override the various colorCalc()
methods to multiply in the alpha. That's pretty much all that I'm
using right now, but then I don't have 100% API coverage! It would be
possible to handle multiply and unpremultiply fairly easily in set()
and get(), which leaves the pixel array - I'd suggest running a
un-premultiply <> premultiply during loadPixels() and updatePixels(),
with an extra hint to leave it premultiplied and save any hit (of
course RGB images wouldn't have any anyway).

I'm not quite sure if there would be any implications to PShape that
would need looking at - I guess I'll find out when I come to
integrating support for them! ;-)

The one aspect of the API that would change would be GLSL - mind you
I'd assert that's fixing it rather than breaking it - however, it's
the one place that might affect the behaviour of user code.

Incidentally, if it's of interest - some alternate software blend
modes that I and another person on the Java Gaming forum worked on a
few years back - all using pre-multiplied alpha, obviously -
http://www.java-gaming.org/topics/some-new-blendmodes-add-multiply-overlay-etc/24529/view.html

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions