You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There seems to be a discrepancy in how matplotlib's imshow displays rasters which contain NaN areas. The NaN areas of the raster appear larger than they really are.
I assume the problem might be that the raw raster gets resampled/interpolated during display (but before converting to RGBA) and the NaN values propagate during that interpolation (since the ordinary mean of a list containing at least one nonfinite value is also not finite). If so, a fix would be to ensure conversion to RGBA always precedes any image resampling/rescaling.
Here is an example. The first subplot shows a float32 raster with values of zero and one. The second subplot exactly replaces the nonzero pixels with NaN, however the resulting shapes look different (making narrow non-NaN features invisible). The third subplot combines both, showing that the non-NaN area (red) is eroded, resulting in a black outline.
The expected correct behaviour would be that the red layer perfectly covers the black layer, leaving no border visible.
importmatplotlib.pyplotasplt, numpyasnpraster=np.full(shape=[200]*2, fill_value=np.nan, dtype=np.float32) # example imageraster[100:150, 100:150] =0foriinrange(10):
raster[20*i:20*i+i, :] =0# lines of varying thicknessplt.figure(figsize=(8,8)) # small figureplt.subplot(1,3,1).imshow(np.isnan(raster).astype(np.float32)) # 0 vs 1plt.subplot(1,3,2).imshow(raster) # 0 vs NaNax=plt.subplot(1,3,3)
ax.imshow(np.isnan(raster), cmap='gray') # black border/underlayerax.imshow(raster, cmap='autumn') # red interior
Tested on matplotlib 3.2.1, in a jupyter 1.0.0 sandbox using ipykernel.pylab.backend_inline backend, ipykernel 5.3.4, python 3.6.9, Linux 4.14 amzn2 x86_64.
By default we apply an antialiasing filter if subsampling. One consequence is that NaNs will be smeared with the data. You can turn off the antialiasing or you can use more resolution so you don't need to subsample (i.e. in the above you cite a "small figure" so basically this problem goes away for a big figure).
You could argue that we should interpolate in RGB space, but that leads to colors that are not even in the colormap. I'm personally not sure thats a problem for the anti-aliasing filter, but it is disconcerting to see red in an image that is not supposed to have any red in it.
I strongly argue interpolation should always be in RGBA space.
For example, consider categorical data. Interpolation in the raw data space results in colours that correspond to value ranges that do not exist in the dataset (such as orange in this image below). This causes misleading visualisations.
defsample(n):
rnd=np.random.random([n]*2)
categorical=np.zeros_like(rnd, dtype=np.uint8) # 49% category A (Blue)categorical[rnd>np.linspace(0, 1, n)[:,None]] =10# 46% category B (Grey)categorical[rnd>0.95] =5# 5% category C (Green)returncategorical# 0% category D (Orange)fig, (ax1, ax2) =plt.subplots(1, 2)
im1=ax1.imshow(sample(20), cmap='tab20c') # coarse featuresim2=ax2.imshow(sample(100), cmap='tab20c') # fine featuresplt.colorbar(im2, ax=(ax1, ax2), location='bottom')
Instead if we interpolate in RGBA space, the result will be the same as if we generated the image at a very high resolution, and then viewed that image from a distance. This is exactly what we should want to see.
We can get the desired behaviour by using imsave to convert to RGBA in advance:
If we need to interpolate, we want the image to look the same as if we were looking at a high resolution version from closer or further away. Zooming in and out (or changing resolution) shouldn't create perceptually different colours.
For example, consider categorical data. Interpolation in the raw data space results in colours that correspond to value ranges that do not exist in the dataset (such as orange in this image below).
If you have categorical data use interpolation='nearest' so that we do not try to interpolate (as unlike continuous data categorical data can not be interpolated). Our default assumption is that the input data to imshow is continuous like (e.g. an image) even if it comes in as integers so the AA default interpolation makes sense.
Zooming in and out (or changing resolution) shouldn't create perceptually different colours.
Yes, that is why we resample in data space not in RGBA space. We used to do it that way and it lead to significant artifacts in the continious data case. This image was generated with mpl1.3 (iirc) using viridis (which by design contains no red)
This effect will also happen in the categorical case but because it is getting in wrong in a less obvious way (the average of blue and gray stands out from blue and gray less than orange).
Similarly, if data that is nan is participating in the computation to get the resampled value, then that value must also be poisoned to be "bad" which leads to the apparent erosion. The size of the eroded area is going to depend on the size of the interpolation kernel, to avoid it use "nearest" or "none" (which again, if you have categorical data you should be doing anyway).
We can get the desired behaviour by using imsave to convert to RGBA in advance:
You can also call the color map directly rather than having to go through disk:
Just to put my opinion here, I would argue that antialiasing is an image-space operation. If it introduces artifacts that the user does not like, then they don't want antialiasing, they want data interpolation.
That said, to my (naive) eyes, having not thought about this problem nearly as much as many other people here, the best default behavior would seem to be to alias in norm'd-but-not-cmap'd space.
OK, there are two conflicting desires here, but they both are correct. But if we don't delineate them we will continue to argue in circles.
Data smoothing:
The example in @tacaswell comment is smoothing the data - the user has coarse data and they would like it to not look "blocky". In this instance I agree that smoothing in data space is absolutely correct. Whether Matplotlib should do this, or whether the user should do this before they send to matplotlib is a reasonable question, but currently we do it, and so be it.
The user may also want to smooth in data space to reduce subsampling artifacts if they know the data will be subsampled in their output image. i.e. they have 50,000x50,000 data points, they should probably consider smoothing the data in some way before trying to publish it. Again, I'm not really sure matplotlib should do this, but...
Image resampling
We currently smooth and resample in one step. However, all the above could be done without changing the size of the data, and a post-colormap resampling step be added that resamples an image in RGBA space. While this is "image processing" and could also be argued that it is not something Matplotlib should do, the fact of the matter is that we output raster data at certain dpi, and hence always need to resample.
So, in the plots above, the red regions are egregious if we are smoothing blocky data. Of course we don't want that. However, if we are subsampling, and those are individual pixels that you have zoomed in on, then they are probably perceptually correct.
So, a possible algorithm would be
smooth the data with a smoothing kernel, but keep the data the same size.
map to RGBA
subsample the RGBA image with a different kernel that does anti-aliasing.
i.e. imshow(X, interpolation='nearest', rgba_subsample='hanning') would give the user above what they want, and imshow(X, intepolation='sinc', rgba_subsample='nearest') would be what we currently do.
The cost would be potentially making a very big RGBA image and two interpolation steps, and of course extra documentation.
If we had a checker-board of 0, 1, and then we down-sampled by averaging in 2x2 grids, and put a color bar on the image: should the image be a uniform reddish-brown or a uniform teal-ish?
@brunobeltran We currently do the resampling on a (linearly normalized) version of the data, the then apply the user norm and then run the result of that through the colormap (because (log(a) + log(b))/2 != log((a + b) / 2) and it is not clear which which order is "right").
If we had a checker-board of 0, 1, and then we down-sampled by averaging in 2x2 grids, and put a color bar on the image: should the image be a uniform reddish-brown or a uniform teal-ish?
Either is wrong with no context. If there is no data that is 0.5 in the dataset, whether its OK to make it be 0.5 depends on the application. Making it brown is equally valid because it is a visual mix of this blue and this yellow.
Redo the above with RdBu. You get purple in between. Is white more "correct"? Like I'm sympathetic that it is in many situations, but from an antialiasing point of view, it will actually be a disaster.
imshow when you are in the single channel input mode it has the assumption baked into it that the input data is a uniform rectangular sampling of a continuous function who's values are a field (in the math sense aka the Reals). To come back to @benjimin 's issue, categorical values do not satisfy that last condition (cat1 + cat2 is probably not defined and cat1 / 2 is definitely not), however because the values are being encoded as a numpy array, we happy do it (even though it is nonsense) because there is no way to express to Matplotlib that "these are categoricals, you can't add them or divide by the reals!" (being able to express this sort of thing is what @story645 is working on with the CZI funding).
The whole reason we went down the perceptually uniform colormap path was so that the user gets a reliable understanding of the underlying data, doing RGB interpolation makes most of that moot. In these cases we are generating colors that are not in the color map so it is clear that "something is wrong", I suspect that there is a color map (maybe a pathological one) where the RGB averaging takes you to a valid, but wrong color.
Either is wrong with no context. If there is no data that is 0.5 in the dataset, whether its OK to make it be 0.5 depends on the application.
and not with the following one
in the single channel input mode it has the assumption baked into it that the input data is a uniform rectangular sampling of a continuous function who's values are a field
In general we get in a MxN array from the user and need to use that data to color a PxQ set of pixels. In all but the case where the user data size exactly matches the output size (which is the situation we fabricate when you use mimage.imsave or let fig.figimage resize its self) we have to do some resampling.
To do that re-sampling you can not start on the user side and "push" forward to the result side (because depending on the transform and kernel etc you a naive loop over the input space will map to every point in the output space), you have to start from the result side and "pull" results forward. That is for each pixel in the target space you say "given my target position, the transform between the two spaces, and my kernel I need to pull the following pixels from the input space and do some math on them". In the case of "nearest", you sort out what pixel is nearest to where the output pixel projects back to and grab that value. In the case of the non-trivial kernels you sort out where the target pixel maps back to then grab the window of input pixels around that and produce a value. This process is mathematically identical if the target space is bigger or smaller than the source space (but in one case we call it upsampling and in the other smoothing).
The default interpolation mode when down-sampling prevents getting aliases patterns when you have features in the data that are beating against the output pixels by smoothing at the size of the kernel (and we probably should scale the kernel radius based on the pixel size as even with the current version of AA you can construct a pathological dataset that will show the beating).
If 0.5 is not a valid value in the data set, then the RGB average still maps to something that is not in the input data ( and is not in the colormap so it also is not valid for the output space).
@anntzer If you do not agree that those are the assumptions baked into imshow, what do you think the assumptions on the data are? I do not disagree it gets used for things that do not quite match those assumptions, but that is how we treat the data inside.
Just to be clear, I am only talking about the downsampling case here; I think we are all in agreement in the upsampling case.
We explicitly support categorical colormaps: there is a large amount of code/complexity in colorbar.py mostly present to make them work. In the case of catgorical colormaps your assumptions completely break down (as you stated yourself).
Actually I personally rarely use categoricals, so let's look at the continuous case. Your argument seems to be that mapping to a value that is both not in the colormap nor in the input data is worse than mapping to a value not in the input data, but still in the colorbar. As stated by @jklymak above, I believe this is highly application-dependent and not always true (and it is incumbent on the user to pick the right colormap and sampling level to make the display suitably reflect the data). To reuse the example of the RdBu colormap used by @jklymak (assume white = zero), one could well be happy to be able to distinguish between white (a region where the values are really actually close to zero) and purple (a region where the values vary quickly between negative and positive values). Or perhaps one does want to treat such a mixture of values as if it was a flat region zero (white) -- it depends on the application.
colorbar applies to more than images (scatter, contours, pcolormesh, etc), my point is only about the assumptions on the input data to imshow not to all Colormappable objects. If the OP had used something in the pcolor family we would not have had this report :)
importmatplotlib.pyplotasplt, numpyasnpraster=np.full(shape=[200]*2, fill_value=np.nan, dtype=np.float32) # example imageraster[100:150, 100:150] =0foriinrange(10):
raster[20*i:20*i+i, :] =0# lines of varying thicknessplt.figure(figsize=(8,8)) # small figureplt.subplot(1,3,1).pcolormesh(np.isnan(raster).astype(np.float32)) # 0 vs 1plt.subplot(1,3,2).pcolormesh(raster) # 0 vs NaNax=plt.subplot(1,3,3)
ax.pcolormesh(np.isnan(raster), cmap='gray') # black border/underlayerax.pcolormesh(raster, cmap='autumn') # red interior
defsample(n):
rnd=np.random.random([n]*2)
categorical=np.zeros_like(rnd, dtype=np.uint8) # 49% category A (Blue)categorical[rnd>np.linspace(0, 1, n)[:,None]] =10# 46% category B (Grey)categorical[rnd>0.95] =5# 5% category C (Green)returncategorical# 0% category D (Orange)fig, (ax1, ax2) =plt.subplots(1, 2)
im1=ax1.pcolormesh(sample(20), cmap='tab20c') # coarse featuresim2=ax2.pcolormesh(sample(100), cmap='tab20c') # fine featuresplt.colorbar(im2, ax=(ax1, ax2), location='bottom')
I would argue that in addition to dealing with the initial nan erosion, it also gives significantly crisper looking plots as the edges of the boxes are not rounded off and the halo around the edges of the first image is gone.
BoundryNorm is also a bit of a red-herring because it is applying the categories after we have done the re-sampling and might be a better argument of why it needs to be resample -> norm -> colormap than LogNorm.
Relying on a colormap and interpolation implementation details to detect noisy parts of your data seems cute but super brittle...
Again, great example! But the bottom right one is what imagemagic would give you if you gave it a high-res version, and it is the true perceptual blend of those colors:
e.g. (I made a 1000-dpi 8x8 image and then ran convert Boo.png -resize 200x200 Magick50.png)
imagemagick of course uses an anti-aliasing filter. I'm suggesting that we could add an anti-aliasing step that is equivalent.
This thread seems to be full of people that agree with each other presenting increasingly clever examples of when one of antialiasing or data interpolation works better than the other.
I largely agree with what @jklymak is saying, which, as I read it, is simply that we should just let the user choose whether to do any interpolation before cmap, after, neither, or both. I understand that there needs to be a default, but @tacaswell is there anything wrong with offering both options in principle? Obviously both are useful in different domains.
My only point earlier was that if we do offer both options...only one of these should be called antialiasing....
We can not not interpolate (except for the vanishingly small case when the size of the image exactly matches the size on the screen). Interpolating to before color mapping leads to technically correct (based on the assumptions that imshow has baked into it) but in some cases undesirable effects. Interpolating after color mapping leads colors being in the image that are not in the color map, which I think violates a rather core aspect of using color mappables (that you can go from the color -> colorbar -> value).
I posted an image in #18749 , if you use interpolation='nearest' with the disk example you get both no white ring and no purple pixels which seems strictly better to me.
I am 👍 on documenting that you can do the color mapping yourself, hand imshow an RGBA array and then it will do the interpolation in RGBA space, however I am still very skeptical that you ever actually want this (because you can get the same or better effect some other way).
that you can go from the color -> colorbar -> value
You lose that ability as soon as you subsample. You can't go back to the original data from the image.
So you have two choices: do you want the "fake" pixel to look like it has a certain data value between the adjacent original data? Or do you want the "fake" pixel to look like its between the two (or more) colors from the upsampled image that make it up? Both are completely valid.
In terms of looking things up in colormaps, If I have red/blue/red/blue, making the picture purple makes it pretty clear to me that the data is a combination of many red-blues all smeared together. If I make the field all white, I might naturally assume my data is all zero:
If you zoom on the white figure, imagine your surprise when solid blue and red stripes emerge, whereas its not at all surprising when they emerge on the purple figure. Which one tells you more about the (hidden) aspects of your subsampled image?
however I am still very skeptical that you ever actually want this
This is basic image anti-aliasing and downsampling, which every image-editing software package in the world does. It is a very common thing to want to do when subsampling an image. You can readily claim its not always advisable and I agree, particularly when dealing with quantitative data, but people clearly "actually want this".
Activity
jklymak commentedon Oct 15, 2020
By default we apply an antialiasing filter if subsampling. One consequence is that NaNs will be smeared with the data. You can turn off the antialiasing or you can use more resolution so you don't need to subsample (i.e. in the above you cite a "small figure" so basically this problem goes away for a big figure).
For more discussion, you can look at https://matplotlib.org/gallery/images_contours_and_fields/image_antialiasing.html
You could argue that we should interpolate in RGB space, but that leads to colors that are not even in the colormap. I'm personally not sure thats a problem for the anti-aliasing filter, but it is disconcerting to see red in an image that is not supposed to have any red in it.
benjimin commentedon Oct 15, 2020
I strongly argue interpolation should always be in RGBA space.
For example, consider categorical data. Interpolation in the raw data space results in colours that correspond to value ranges that do not exist in the dataset (such as orange in this image below). This causes misleading visualisations.
Instead if we interpolate in RGBA space, the result will be the same as if we generated the image at a very high resolution, and then viewed that image from a distance. This is exactly what we should want to see.
We can get the desired behaviour by using
imsaveto convert to RGBA in advance:If we need to interpolate, we want the image to look the same as if we were looking at a high resolution version from closer or further away. Zooming in and out (or changing resolution) shouldn't create perceptually different colours.
jklymak commentedon Oct 15, 2020
ping @tacaswell who is the current custodian behind how we currently do it.
tacaswell commentedon Oct 15, 2020
If you have categorical data use
interpolation='nearest'so that we do not try to interpolate (as unlike continuous data categorical data can not be interpolated). Our default assumption is that the input data to imshow is continuous like (e.g. an image) even if it comes in as integers so the AA default interpolation makes sense.Yes, that is why we resample in data space not in RGBA space. We used to do it that way and it lead to significant artifacts in the continious data case. This image was generated with mpl1.3 (iirc) using viridis (which by design contains no red)
This effect will also happen in the categorical case but because it is getting in wrong in a less obvious way (the average of blue and gray stands out from blue and gray less than orange).
Similarly, if data that is
nanis participating in the computation to get the resampled value, then that value must also be poisoned to be "bad" which leads to the apparent erosion. The size of the eroded area is going to depend on the size of the interpolation kernel, to avoid it use"nearest"or"none"(which again, if you have categorical data you should be doing anyway).You can also call the color map directly rather than having to go through disk:
which is faster (no I/O) and does not do any resampling of any sort under the hood for you.
brunobeltran commentedon Oct 15, 2020
Just to put my opinion here, I would argue that
antialiasingis an image-space operation. If it introduces artifacts that the user does not like, then they don't want antialiasing, they want data interpolation.That said, to my (naive) eyes, having not thought about this problem nearly as much as many other people here, the best default behavior would seem to be to alias in
norm'd-but-not-cmap'd space.jklymak commentedon Oct 15, 2020
OK, there are two conflicting desires here, but they both are correct. But if we don't delineate them we will continue to argue in circles.
Data smoothing:
The example in @tacaswell comment is smoothing the data - the user has coarse data and they would like it to not look "blocky". In this instance I agree that smoothing in data space is absolutely correct. Whether Matplotlib should do this, or whether the user should do this before they send to matplotlib is a reasonable question, but currently we do it, and so be it.
The user may also want to smooth in data space to reduce subsampling artifacts if they know the data will be subsampled in their output image. i.e. they have 50,000x50,000 data points, they should probably consider smoothing the data in some way before trying to publish it. Again, I'm not really sure matplotlib should do this, but...
Image resampling
We currently smooth and resample in one step. However, all the above could be done without changing the size of the data, and a post-colormap resampling step be added that resamples an image in RGBA space. While this is "image processing" and could also be argued that it is not something Matplotlib should do, the fact of the matter is that we output raster data at certain dpi, and hence always need to resample.
So, in the plots above, the red regions are egregious if we are smoothing blocky data. Of course we don't want that. However, if we are subsampling, and those are individual pixels that you have zoomed in on, then they are probably perceptually correct.
So, a possible algorithm would be
i.e.
imshow(X, interpolation='nearest', rgba_subsample='hanning')would give the user above what they want, andimshow(X, intepolation='sinc', rgba_subsample='nearest')would be what we currently do.The cost would be potentially making a very big RGBA image and two interpolation steps, and of course extra documentation.
tacaswell commentedon Oct 15, 2020
To be very concrete, averaging / interpolating in RGB space produces colors that are not in the color map:
If we had a checker-board of 0, 1, and then we down-sampled by averaging in 2x2 grids, and put a color bar on the image: should the image be a uniform reddish-brown or a uniform teal-ish?
A version of this code needs to end up in #18487
@brunobeltran We currently do the resampling on a (linearly normalized) version of the data, the then apply the user norm and then run the result of that through the colormap (because
(log(a) + log(b))/2 != log((a + b) / 2)and it is not clear which which order is "right").jklymak commentedon Oct 15, 2020
Great example.
Either is wrong with no context. If there is no data that is 0.5 in the dataset, whether its OK to make it be 0.5 depends on the application. Making it brown is equally valid because it is a visual mix of this blue and this yellow.
Redo the above with
RdBu. You get purple in between. Is white more "correct"? Like I'm sympathetic that it is in many situations, but from an antialiasing point of view, it will actually be a disaster.tacaswell commentedon Oct 15, 2020
imshowwhen you are in the single channel input mode it has the assumption baked into it that the input data is a uniform rectangular sampling of a continuous function who's values are a field (in the math sense aka the Reals). To come back to @benjimin 's issue, categorical values do not satisfy that last condition (cat1 + cat2is probably not defined andcat1 / 2is definitely not), however because the values are being encoded as a numpy array, we happy do it (even though it is nonsense) because there is no way to express to Matplotlib that "these are categoricals, you can't add them or divide by the reals!" (being able to express this sort of thing is what @story645 is working on with the CZI funding).The whole reason we went down the perceptually uniform colormap path was so that the user gets a reliable understanding of the underlying data, doing RGB interpolation makes most of that moot. In these cases we are generating colors that are not in the color map so it is clear that "something is wrong", I suspect that there is a color map (maybe a pathological one) where the RGB averaging takes you to a valid, but wrong color.
jklymak commentedon Oct 15, 2020
@tacaswell, again two different use cases (#18735 (comment)) which should have different interpolation.
anntzer commentedon Oct 15, 2020
FWIW I agree with the following statement
and not with the following one
tacaswell commentedon Oct 15, 2020
Those are not different things.
In general we get in a MxN array from the user and need to use that data to color a PxQ set of pixels. In all but the case where the user data size exactly matches the output size (which is the situation we fabricate when you use
mimage.imsaveor letfig.figimageresize its self) we have to do some resampling.To do that re-sampling you can not start on the user side and "push" forward to the result side (because depending on the transform and kernel etc you a naive loop over the input space will map to every point in the output space), you have to start from the result side and "pull" results forward. That is for each pixel in the target space you say "given my target position, the transform between the two spaces, and my kernel I need to pull the following pixels from the input space and do some math on them". In the case of "nearest", you sort out what pixel is nearest to where the output pixel projects back to and grab that value. In the case of the non-trivial kernels you sort out where the target pixel maps back to then grab the window of input pixels around that and produce a value. This process is mathematically identical if the target space is bigger or smaller than the source space (but in one case we call it upsampling and in the other smoothing).
The default interpolation mode when down-sampling prevents getting aliases patterns when you have features in the data that are beating against the output pixels by smoothing at the size of the kernel (and we probably should scale the kernel radius based on the pixel size as even with the current version of AA you can construct a pathological dataset that will show the beating).
If 0.5 is not a valid value in the data set, then the RGB average still maps to something that is not in the input data ( and is not in the colormap so it also is not valid for the output space).
@anntzer If you do not agree that those are the assumptions baked into
imshow, what do you think the assumptions on the data are? I do not disagree it gets used for things that do not quite match those assumptions, but that is how we treat the data inside.anntzer commentedon Oct 15, 2020
Just to be clear, I am only talking about the downsampling case here; I think we are all in agreement in the upsampling case.
We explicitly support categorical colormaps: there is a large amount of code/complexity in colorbar.py mostly present to make them work. In the case of catgorical colormaps your assumptions completely break down (as you stated yourself).
Actually I personally rarely use categoricals, so let's look at the continuous case. Your argument seems to be that mapping to a value that is both not in the colormap nor in the input data is worse than mapping to a value not in the input data, but still in the colorbar. As stated by @jklymak above, I believe this is highly application-dependent and not always true (and it is incumbent on the user to pick the right colormap and sampling level to make the display suitably reflect the data). To reuse the example of the RdBu colormap used by @jklymak (assume white = zero), one could well be happy to be able to distinguish between white (a region where the values are really actually close to zero) and purple (a region where the values vary quickly between negative and positive values). Or perhaps one does want to treat such a mixture of values as if it was a flat region zero (white) -- it depends on the application.
tacaswell commentedon Oct 15, 2020
colorbar applies to more than images (scatter, contours, pcolormesh, etc), my point is only about the assumptions on the input data to
imshownot to allColormappableobjects. If the OP had used something in the pcolor family we would not have had this report :)I would argue that in addition to dealing with the initial nan erosion, it also gives significantly crisper looking plots as the edges of the boxes are not rounded off and the halo around the edges of the first image is gone.
BoundryNormis also a bit of a red-herring because it is applying the categories after we have done the re-sampling and might be a better argument of why it needs to be resample -> norm -> colormap thanLogNorm.Relying on a colormap and interpolation implementation details to detect noisy parts of your data seems cute but super brittle...
2 remaining items
jklymak commentedon Oct 15, 2020
Again, great example! But the bottom right one is what imagemagic would give you if you gave it a high-res version, and it is the true perceptual blend of those colors:
e.g. (I made a 1000-dpi 8x8 image and then ran

convert Boo.png -resize 200x200 Magick50.png)imagemagick of course uses an anti-aliasing filter. I'm suggesting that we could add an anti-aliasing step that is equivalent.
jklymak commentedon Oct 15, 2020
At the minumium this whole issue could be added to the discussion at https://matplotlib.org/devdocs/gallery/images_contours_and_fields/image_antialiasing.html Certainly the trick of applying the norm and filtering RGBA is a cool one I didn't realize would work and could be documented there.
tacaswell commentedon Oct 16, 2020
I would have called this "a data destroying artifact due to down-sampling the after rendering" ;)
@benjimin Thank you for asking an interesting question, hopefully the high volume and intensity of this discussion has not been offputting ;)
jklymak commentedon Oct 16, 2020
All subsampling destroys data, by definition.
jklymak commentedon Oct 16, 2020
From the example in #18749, using
RdBucolormap - obviously the purple colors are not in the colormap:brunobeltran commentedon Oct 16, 2020
This thread seems to be full of people that agree with each other presenting increasingly clever examples of when one of antialiasing or data interpolation works better than the other.
I largely agree with what @jklymak is saying, which, as I read it, is simply that we should just let the user choose whether to do any interpolation before cmap, after, neither, or both. I understand that there needs to be a default, but @tacaswell is there anything wrong with offering both options in principle? Obviously both are useful in different domains.
My only point earlier was that if we do offer both options...only one of these should be called
antialiasing....tacaswell commentedon Oct 16, 2020
We can not not interpolate (except for the vanishingly small case when the size of the image exactly matches the size on the screen). Interpolating to before color mapping leads to technically correct (based on the assumptions that imshow has baked into it) but in some cases undesirable effects. Interpolating after color mapping leads colors being in the image that are not in the color map, which I think violates a rather core aspect of using color mappables (that you can go from the color -> colorbar -> value).
I posted an image in #18749 , if you use
interpolation='nearest'with the disk example you get both no white ring and no purple pixels which seems strictly better to me.I am 👍 on documenting that you can do the color mapping yourself, hand
imshowan RGBA array and then it will do the interpolation in RGBA space, however I am still very skeptical that you ever actually want this (because you can get the same or better effect some other way).jklymak commentedon Oct 16, 2020
You lose that ability as soon as you subsample. You can't go back to the original data from the image.
So you have two choices: do you want the "fake" pixel to look like it has a certain data value between the adjacent original data? Or do you want the "fake" pixel to look like its between the two (or more) colors from the upsampled image that make it up? Both are completely valid.
In terms of looking things up in colormaps, If I have red/blue/red/blue, making the picture purple makes it pretty clear to me that the data is a combination of many red-blues all smeared together. If I make the field all white, I might naturally assume my data is all zero:
If you zoom on the white figure, imagine your surprise when solid blue and red stripes emerge, whereas its not at all surprising when they emerge on the purple figure. Which one tells you more about the (hidden) aspects of your subsampled image?
This is basic image anti-aliasing and downsampling, which every image-editing software package in the world does. It is a very common thing to want to do when subsampling an image. You can readily claim its not always advisable and I agree, particularly when dealing with quantitative data, but people clearly "actually want this".
jklymak commentedon Oct 21, 2020
@benjimin #18782 is meant to allow you to get what you want directly.