Repository navigation
[ENH]: directional antialiasing filter #29711
Description
Problem
Currently, the antialiasing filter used by imshow() is applied both in the x and the y direction. There are cases where "image-like" data is highly sampled in one direction (requiring antialiasing) but not in the other, and also has some nans; an example is a kymograph like subfigure 1c at https://www.nature.com/articles/s41467-017-01462-y/figures/1
For such data, the current default interpolation settings (antialiased + rgba (because one of the directions is oversampled) can yield not nice artefacts, in particular at the boundary between "data" and "nan":
from pylab import *
vals = np.full((50, 500), np.nan)
for i, row in enumerate(vals):
n = 200 + np.random.randint(300)
row[:n] = np.sin(np.arange(n) / 10 + i / 5) # make some fake data
axs = figure(layout="constrained", figsize=(10, 10)).subplots(
3, 3, subplot_kw={"xticks": [], "yticks": []})
for i, interpolation in enumerate([None, "none", "antialiased"]):
for j, stage in enumerate([None, "data", "rgba"]):
axs[i, j].text(0, 0, f"{interpolation=}, {stage=}", bbox={"fc": "w"})
axs[i, j].imshow(vals.T, aspect="auto", origin="lower",
interpolation=interpolation, interpolation_stage=stage)
show()
For this specific dataset, the best settings currently available is probably to go back to interpolation="data". But ideally, it would be nice if it was possible to have a separable antialiasing filter (I don't actually know if the current one is separable), so that one can apply it (here) only in the vertical direction (ideally in rgba space).
Proposed solution
No response
Activity
timhoffm commentedon Mar 6, 2025
IMHO this feels quite niche. Is there any precedence that this is done in other libraries or applications? I'd say for very special cases like a publication where you care a lot about the visual quality, you could resample the data appropriately.
jklymak commentedon Mar 6, 2025
I actually don't understand the request. If you interpolate in rgba space, and your x-dimension is well-resolved by the dpi, it should not matter if there is an antialiasing filter?
jklymak commentedon Mar 6, 2025
OK, looking at this - if I save your image above at 600 dpi and then rely on my image viewer to downscale to 100 dpi, it does a better job of the antialiasing on the NaN's because it knows the background is white. I guess I don't know what color the matplotlib "RGBA" antialiasing is assuming for NaN, but maybe that is an easy to fix bug.
600dpi downsampled to 144 dpi and then screen-shotted:
anntzer commentedon Mar 6, 2025
It does, because (as you have noticed in your followup comment) the vertical edges between data and nans become needlessly blurry.
Good question as to whether other libraries do anything special here for antialiasing, but that kind of plot ("kymograph") is very common in my field; another example is figure 5 of https://pubmed.ncbi.nlm.nih.gov/27099372/
jklymak commentedon Mar 6, 2025
OK, but just to be clear, if you save this at publication-quality resolutions, this is not really a problem. If you are worried about publication quality antialiasing, you can also save at 600 dpi or so, and then down sample if needed.
Regardless, we should see what color is being used for the antialiasing.. It seems to me the right solution is to allow the user some control over that.
WeatherGod commentedon Mar 6, 2025
anntzer commentedon Mar 6, 2025
Sure, we are probably fine at high enough (but reasonable) dpi for real use cases. I just noticed the problem at the default, low on-screen dpi looking at some real data.
jklymak commentedon Mar 6, 2025
OK, so the issue is that the "bad" data is coloured black by default in viridis. If you do
then the antialiasing with the NaN regions looks "fine".
anntzer commentedon Mar 7, 2025
It's definitely better, but still unnecessarily blurry?
jklymak commentedon Mar 7, 2025
It's not anti-aliasing for the edges only, it smooths everything.
anntzer commentedon Mar 7, 2025
Yes. But ideally I'd like it to smooth only vertically, not horizontally.
jklymak commentedon Mar 7, 2025
Time to invest in a high-DPI monitor ;-). But sure, antialiasing of images comes at a cost of sharpness, in exchange for the benefit of not showing erroneous data.
Edit: I'm not sure we should supply a directional image anti aliasing feature. Note that our current interpolation just uses the AGG functions which are isotropic. Allowing the user to specify the kernel might be possible but it'd be a fair bit of work.
timhoffm commentedon Mar 7, 2025
If it's reasonably simple to integrate and @anntzer wants to look into this, I'd be fine with that. I trust that the solution is viable if he comes up with one. I'd be very hesitant if a non-core dev would try to get such a feature in.
anntzer commentedon Mar 7, 2025
Actually, there's really two different problems there:
"#0000") rather than "transparent white" (#fff0), which leads to the grayish borders between data and nan in the originally posted example image; at least, antialiasing against "transparent white" (or rather "transparent background") leaves the bluriness but removes the grayish borders. It's not actually clear whether this is something for which we can have a general & automatic solution (because it depends on the background color behind the image), though.jklymak commentedon Mar 7, 2025
Colormaps have an "invalid" colour. However, then plotting methods turn NaN into masked regions that don't have any colour. I'm not sure if there are any methods that ever plot the "invalid" color?
If not, or it it is rare, one could imagine setting the "invalid" colors to white by default, though users would still want to be able to change that in case their background is gray or green or whatever. even if we dont' do that, documenting the filter's behaviour with the invalid color is useful so users know to change it.
One could also imagine "invalid" defaulting to masking, and then the imshow picking up the parent axes facecolor for invalid.
anntzer commentedon Mar 17, 2025
Looking at it again, I think the proper fix for the "transparent black" problem is likely to switch (at least the antialiasing pipeline) to using premultiplied alpha, see e.g. https://www.adriancourreges.com/blog/2017/05/09/beware-of-transparent-pixels/ (Likely, we can also switch the antialiasing to handle both RGB and A at the same time rather than splitting them as we do right now.)
TL;DR: the current formula for equal blending of two colors with e.g. red channel r0 & r1 and alpha channels a0 & a1 (I skip blue and green for simplicity) is that the blended red channel is
0.5 r0 + 0.5 r1; with premultiplied alpha it is (after backconversion to straight alpha)(0.5 a0 * r0 + 0.5 a1 * r1) / (0.5 a0 + 0.5 a1)i.e. the color channels are further weighted by their alpha values; in particular, fully transparent colors don't have any weight at all.I don't think this is overly hard to do (... if you understand blending models and the interfacing with agg filters...) so I'll keep it on my todo (but of course if anyone else wants to go for it they are welcome to ;-))
Edit: likely related to #12062 -- would be nice to figure out why it doesn't help here; also if we do switch the filtering step to work on premultiplied alpha then we can probably revert that "split alpha and rgb handling".
jklymak commentedon Mar 17, 2025
Yes that makes good sense. Agree that it is straight forward as well.