Repository navigation
Incorrect alpha compositing using "matplotlib.pyplot.imshow". #9906
Description
Bug report
Bug summary
matplotlib.pyplot.imshow definition alpha compositing is incorrect, references #3343.
Code for reproduction
from matplotlib import pyplot as plt
plt.figure(figsize=(8, 8))
RGBA = plt.imread('RGB_Droplet.png')
plt.imshow(RGBA)
plt.show()Actual outcome
Expected outcome
Composited in The Foundry Nuke:
The images used are available here:
Matplotlib version
- Operating system:
macOS - Matplotlib version:
'2.1.0' - Matplotlib backend (
print(matplotlib.get_backend())):'module://ipykernel.pylab.backend_inline' - Python version:
'2.7.13 |Continuum Analytics, Inc.| (default, Dec 20 2016, 23:05:08) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]' - Jupyter version (if applicable):
notebook 5.0.0 py27_0
Activity
anntzer commentedon Dec 1, 2017
Possibly related to #8847?
QuLogic commentedon Dec 5, 2017
Here's how it looks in the Drive preview (albeit with checkerboard background):
and here's how it looks in eog with white background:
or in gimp with a white layer below it:
So I'm not sure this is a bug per se. It is possible that one is using premultiplied alpha and the other is not (when the incoming image is the opposite.)
thomasmansencal commentedon Dec 5, 2017
My image has alpha premultiplied, so I guess what Matplotlib doing is multiplying again RGB by alpha, this should probably be an optional behavior.
thomasmansencal commentedon Dec 5, 2017
Seems to be it:
Using:
RGB_Droplet_UnPremultiplied.png
efiring commentedon Dec 7, 2017
It's not clear to me whether there is a proposed change for mpl here. If not, this should be closed. If there is something that needs to be done, perhaps it should be stated at the top of a new issue, with a reference to this one, and then close this one.
thomasmansencal commentedon Dec 7, 2017
I don't know if it is at image read time or display time, but MPL should offer an option for different behaviour whether the RGB channel has alpha pre-multiplied or not. Currently the RGB channel is getting multiplied potentially two times, so this is an issue.
efiring commentedon Dec 7, 2017
I think the option would have to be at image read time; internally, mpl has to consistently use one form or the other, and presently it is using straight RGBA. It seems like the underlying problem here is that RGBA is not well-defined. Therefore imread could gain a kwarg, defaulting to
premultiplied=False; when set to True it would cause the input RGB values to be divided by the alpha value. Is this correct? If so, would you like to submit a PR for this?jklymak commentedon Dec 7, 2017
PNG spec says not to premultiply alpha, so I don't think we are doing anything incorrect by reading it that way: https://www.w3.org/TR/PNG-Rationale.html If the user wants to divide by alpha after reading, they are welcome to do so.
thomasmansencal commentedon Dec 7, 2017
Just to be clear, I have the same issue if I pass an image generated directly by code to
imshow.I could not find anywhere in MPL
imshowdocumentation a statement that image must be either premultiplied or un-premultiplied, this should be clarified.jklymak commentedon Dec 7, 2017
I'm not following your first statement. Do you have an example?
I think the documentation is pretty clear. If you want to specify RGB, you pass RGB, if you want to specify RGBA, you pass RGBA. I'm not clear why you would you pass RGBA to
imshowif you've already applied the alpha to the RGB data. What wouldimshowdo with the A channel?thomasmansencal commentedon Dec 7, 2017
The question is not whether you pass alpha or not, but whether your RGB has been premultiplied with Alpha or not. The capability of doing A over B through an alpha is totally independent of the state of RGB.
thomasmansencal commentedon Dec 7, 2017
Defining the following variables:
Mask: the black and white droplet alpha / maskA: the RGB droplet, already premultiplied byMaskB: the grid backgroundOn the left,
A, is composited overB, usingMaskas an alpha in the Merge1 (Over) node.On the right,
Ais divided byMaskin the Merge3 (Divide) node and is composited overB, usingMaskas an alpha in the Merge2 (Over) node.As you can see A over B is possible in both cases, the problem is that depending whether your RGB values have been premultiplied, you have to account for it, and thus at the very least, indicate clearly in the document what MPL expect, which seems to be un-premultiplied RGB data to be sent to
imshow.jklymak commentedon Dec 7, 2017
Matplotlib
imshowhas no concept of premultiplied. It takes an MxNx3 RGB array of floats and displays those values. Matplotlib doesn't care how those values were produced. If you pass it an MxNx4 RGBA array it displays those RGB values with an alpha given by A. The image is displayed with the alpha mixed with whatever background exists on the figure behind the image.The image you supplied has already had A applied against black background. Passing A again, as you are doing above, masks the black background, but of course still allows some of the side data to leak in and look greyish.
I really don't think this is a bug, and I really think the documentation is clear.
That there is a community out there that pre-multiplies alpha into their images and then still supplies that alpha in the PNG isn't really our business, but in my opinion, they are going against the spec and missing the point of having an alpha channel.
thomasmansencal commentedon Dec 7, 2017
PNG is probably not the good file format to discuss about since the spec seems to advocate straight / un-premultiplied alpha but a lot of other formats don't specify it.
I would argue the opposite considering:
imshowand RGBA values #3343I don't think a mention to straight un-premultiplied and premultiplied alpha compositing, matplotlib behavior in that regard along a possible reference to https://en.wikipedia.org/wiki/Alpha_compositing would be bad in the
imshowdocumentation.I have scratched my head a few hours trying to understand what is happening and I'm not alone.
jklymak commentedon Dec 7, 2017
#3343 and the S/O question are the same thing and both were fixed by #5718. If you run the code in #5718 you get the expected result (all white). The state of affairs before #5718 was indeed incorrect and a bug.
Please provide a minimal self-contained example that does not require opening an image that displays the problem you think still exists.
thomasmansencal commentedon Dec 7, 2017
As of this writing I'm leaning on the side that the documentation is scarce in term of describing what is the alpha compositing process in MPL not so much on there is an issue. With the myriad of image manipulation packages, image file formats and way of writing those images, I think MPL should describe its alpha compositing process. As a matter of fact,
imshowdocumentation does not describe how it performs its over operation, it is a black box and one would have to dive into the code to understand it.jklymak commentedon Dec 7, 2017
If alpha is not documented anywhere, then that is indeed an oversight.
99% sure that its just: RGB_back is the background and RGB_fore is the foreground, its
RGB = RBG_back * (1 - alpha) + RGB_fore * alpha. Thezordersets what is the foreground.thomasmansencal commentedon Dec 7, 2017
Thanks @jklymak!
jklymak commentedon Dec 7, 2017
No problem - I know next to nothing about "formal" image processing, so I only have the "basic" intuition about what alpha could mean. Fortunately for me, I think thats the same as how Matplotlib thinks about it. You've obviously used it in different contexts, so are approaching it from a different point of view. Its definitely helpful to get different points of view so we can make the docs as universal as possible.
I'd suggest https://matplotlib.org/tutorials/colors/colors.html#sphx-glr-tutorials-colors-colors-py as a decent place to put the appropriate definitions, though your suggestion as to where else it should go would be appreciated. However, bear in mind that Matplotlib uses alpha for most objects, so we aren't going to explain it for every function call!
Thanks!
anntzer commentedon Jan 11, 2018
Closed by #9959 and #9981, afaict.