Repository navigation
PGF Backend: Support interpolation='none'? #6740
Description
I'm currently trying to figure out the best way to unify the figures for my thesis, and intend to use the PGF backend to directly \include figures in my LaTeX document.
When creating PGF figures that include raster graphics (e.g., via imshow), the graphic gets stored as a PNG file and imported via \pgfimage, which is great. Unfortunately, it doesn't support interpolation='none' as the PDF backend does:
import numpy as np
import matplotlib as mpl
import matplotlib.pyplot as plt
np.random.seed(42)
data = np.random.randn(5,6)
plt.imshow(data, cmap='hot', interpolation='none')
plt.savefig('test.pdf')
mpl.rcParams['pgf.texsystem'] = 'pdflatex'
plt.savefig('test.pgf')Checking the created PDF file in Evince, it indeed includes a PNG graphic of 5x6 pixels. The graphic for the PGF file is blown up, though:
$ file test-img0.png
test-img0.png: PNG image data, 577 x 481, 8-bit/color RGBA, non-interlaced
I can manually fix this by replacing it with a 5x6 PNG file and tweaking the PGF output as follows:
59c59
< \pgftext[at=\pgfqpoint{1.220000in}{0.600000in},left,bottom]{\pgfimage[interpolate=true,width=5.770000in,height=4.810000in]{test-img0.png}}%
---
> \pgftext[at=\pgfqpoint{1.220000in}{0.600000in},left,bottom]{\pgfimage[interpolate=false,width=5.770000in,height=4.810000in]{test-img0.png}}%Can we modify the PGF backend such that it emits the unscaled raster graphic and uses interpolate=false?
I'm willing to work on this myself, but would appreciate some guidance. Looking through the code, the image is written in https://github.com/matplotlib/matplotlib/blob/f1bab50/lib/matplotlib/backends/backend_pgf.py#L612.
The corresponding code in the PDF backend starts at https://github.com/matplotlib/matplotlib/blob/f1bab50/lib/matplotlib/backends/backend_pdf.py#L1600.
It overrides the option_scale_image() function to return True, and it takes an additional transform parameter in draw_image(). I guess this is where the magic happens -- if there's a transformation to apply, it will get the unmodified image and construct the PDF so the image is scaled appropriately.
The documentation on that parameter is scarce (https://github.com/matplotlib/matplotlib/blob/f1bab50/lib/matplotlib/backend_bases.py#L511), but it seems you can convert it into six values giving an affine transformation.
If I want the PGF backend to handle scaling, will the backend have to handle the full range of possible affine transformations (scaling would suffice for me)? Is the transform guaranteed to only be used for interpolation='none', so I can set interpolate=false in this case? Is there any better documentation on the transform?
Activity
f0k commentedon Jul 14, 2016
I found that in the case of a simple scaling,
draw_image()receives as additional argumentsdxanddy, the target width and height of the image, andtransformis set tomatplotlib.transforms.IdentityTransform. This is enough for my use case, and simple to implement. I've created a custom backend that does the necessary overrides: https://gist.github.com/f0k/f8c87ffd82488c2ed303989592bf4ebdThis implementation does not support any other transformation than
IdentityTransformnow, so it's probably not eligible to be merged. I'll leave this issue open as a feature request, but I don't have the time to work on a feature-complete implementation myself -- I'm fine with my workaround. Feel free to close the issue if you think there's not enough demand.Another solution might be to extend the backend API so a backend can declare "I can handle scaling myself, but anything else is above my limits".
Side note: The original backend received a pre-scaled image of 577 x 481 pixels, my modified backend (with
option_scale_image()returningTrue) receives(dy, dx)of(570.0, 480.0). This may indicate a rounding issue somewhere.tacaswell commentedon Jul 15, 2016
at @pwuertz
@f0k Can you put in a PR making that change to the backend?
f0k commentedon Jul 15, 2016
I could, but then the backend cannot cope with arbitrary affine image transforms any longer. I guess this is not what we want. Let's spell out the options (sorry for the wall of text above):
interpolation='none'work for simple scaling, but break for rotation, shearing etc. (that'd be a regression)Option E could be realized as an extension of
option_scale_image()so instead of returningTrueorFalse, the backend could specifically tell matplotlib whether it supports scaling (simplest case), scaling + rotation (harder) or every affine transform (hardest).f0k commentedon Jul 15, 2016
Option C would probably be solvable with
\pgfsys@transformcm#1#2#3#4#5#6. It would still require a lot of tinkering (and writing/finding some good test cases), but not be impossible./edit: Okay, at least one test case is there: http://matplotlib.org/examples/api/demo_affine_image.html.
f0k commentedon Jul 15, 2016
Oh, and another thing: I based my modified backend on matplotlib 1.5.2, which has
draw_image(self, gc, x, y, im, dx=None, dy=None, transform=None).The current master (specifically, #5718) reduced this to
draw_image(self, gc, x, y, im, dx=None, dy=None, transform=None), so basically this removes options B, D, E because the scaling shortcut is not available any more. I just got lucky it was still there in the version I have installed.Maybe Option C would be the best solution then. Option E would require peeling out the scaling factors from the transform, which wouldn't be too hard either if we know it's a scale transform.
pwuertz commentedon Jul 15, 2016
I don't remember the exact details when I implemented the function back then, but I vaguely remember some thoughts concerning image transformation within PGF.. unfortunately I didn't document them.
First of all, the lack of documentation on how
draw_image()should handle thetransformparameter and how capabilities are announced / obeyed is a problem of course. But I think I also remember that handling the scaling externally lead to results that didn't match the original figure. Maybe some half-pixel mismatch that causes distortions for very small images? @f0k also mentioned some 'rounding' issues between pre-scaled and non-scaled images, maybe I observed something like that and decided not to chase it down and to go with the pre-scaled image for pixel-perfect matching with other backends.I agree that the best solution is to move all transformations to the PGF layer. Thanks for the
\pgfsys@transformcm#1#2#3#4#5#6hint! But as @f0k already mentioned this might need some more work and we need a lot of test coverage to ensure perfect alignment.But I'd also like to ask the question "is it really worth it?". We do have perfect matching with other backends, and I don't believe that 10x or 100x upscaled images grow considerably in size after being PNG compressed. Can't be more than a few kB, right?
Also remember that there is
pcolor(), which emits vector graphics instead of pixel images. If your graph consists of only few points, I think this is technically the better solution.f0k commentedon Jul 15, 2016
It's difficult to hit an exact integer upscaling factor, though -- I can influence it via the dpi setting, but if it was easy to hit a certain factor, I could also just set the dpi so it hits a factor of 1. So most of the time,
interpolation='nearest'will lead to some pixel blocks being larger and some being smaller. To avoid the picture looking funny, I'd have to crank up the dpi to a very large value.Now I agree that for a 5x6 pixels image, the gain is negligible. I can blow it up to about 500x600 and rest assured that nobody will notice whether some of the 100x100 pixels are actually 99x99 or 101x101, without wasting too much space.
However, this was just a showcase, I'm actually plotting spectrogram excerpts. When they're blown up by a factor of 10 or 100, they get unmanageably large. When I don't blow them up, they get awkwardly distorted in the process of plotting. When I set
interpolationto another interpolation algorithm, they get blurry.Good to know! (Not for this case, but in general.)
Is this tested anywhere?