PGF Backend: "Dimension too large" error while processing log-scale plot

The PGF backend of MPL 1.4.3 causes a "Dimension too large" LaTeX error when processing a log-scale plot with a point at zero. The error is

[...]
! Dimension too large.
<to be read again> 
                   \pgf@y 
l.69 ...\pgfqpoint{-2148537.332542in}{0.600000in}}
                                                  %
No pages of output.
Transcript written on figure.log.

Obviously, the x coordinate of the point is way out of bounds. The script triggering the error is

import matplotlib as mpl
mpl.use('pgf')
import matplotlib.pyplot as plt

fig, ax = plt.subplots()
ax.plot([0, 1], [.5, 1])
ax.set_xscale('log')
fig.savefig('fig.pdf')

Adding nonposx='mask' to the call to set_xscale prevents the error but produces a plot consisting of only a single point. The PGF version installed is 2.10 but the error also occurs with PGF 3.00.

@pwuertz

attn @pwuertz

With that data and a logscale on the x-axis a plot with a single point is correct. log(0) is -∞ which can not be plotted.

I don't know the details, but from a quick search I found a comment in the changelog, saying that nonposx='mask' should be the default behaviour, thus the point that is causing trouble shouldn't be emitted to the backend in the first place.
Other than that, everything seems to be in order, right? So the invalid point is discarded and you end up with a plot containing the remaining point.

The current situation precludes users from using nonposx='clip', which is not ideal. It would be nice if the backend handled these "points at infinity" gracefully, just like the other backends where the above script produces a horizontal line (I checked that with the Qt and Tk backends).

Couldn't one just clamp these dimensions to some large value less than TeX's \maxdimen? My TeX distro gives me a value slightly below 16384 pt which corresponds to 226.7 in, so something of the order of 100 in should be a reasonable choice. I can try and hack something together that passes all coordinates that are emitted to TeX through some kind of sanitizer function.

I don't think hardcoding some clamping behaviour inside matplotlib does away with this limitation.
For instance, what happens if you clamped all emitted values to 100in and the user expects a 1000in point to be placed correctly after placing the figure in a scale-box of 1/10? Or you clamp all points to 100in, the user upscales the figure 3x and runs into this error anyways? Or the user really tries to create a figure of 200in size and instead of an error, everything is displaced in a strange way?

TeX has limitations, alot of them. TeX could (should?) do the clamping you suggested itself and post a warning instead of an error, but for some reason they decided not to do that. I don't think trying to hide this fact is the best course of action.

Other than that, matplotlib could simply clip/omit points that are outside the viewable space. We had the same problem when drawing lines that happened to be way outside the plotting area and used a clip/optimize option of the path iterator to fix that. Something like this should be possible for points as well.

I don't have time to test it right now, but does that commit fix the problem for you?

Thanks for the effort, but this doesn't fix the problem. The code you inserted was missing there but I don't think it gets called when the script is executed because no markers are requested. The "offending" point is part of a path that gets drawn by _print_pgf_path, which should get properly clipped already. The path just consists of two operations: MOVETO (-∞,y0), LINETO (x1,y1), or from the script:

[([-2.14853733e+08, 6.00000000e+01], 1), ([ 410.15500003, 460.], 2)]

I went through the code and I think the problem is in _path.cleanup_path. During the call to that function a PathClipper is instantiated that does the clipping via some agg function, but only if the operation was a line-to. The move-to that contains the offending point is passed on verbatim and a new move-to is inserted as the next command as a result of the clipping operation. After the clipping the above path looks like

[([-2.14853733e+08, 6.00000000e+01], 1), ([99., 459.99942071], 1), ([410.15500003, 460.], 2)]

I guess that second move-to is the correct one and one has to somehow detect this situation and discard the first one.

I haven't had time to wrap my head around the code in PathClipper<T>::vertex but there is a private member m_broke_path that seems to be set in exactly that situation. However, that member is never read.

Also, the draw_markers function should probably iterate over the unclipped path and just exclude markers whose bounding box does not intersect the clip rectangle. If I understand correctly your patch would put such markers at the point of intersection of the path segment and the clip box.

Oh ok, I thought the problem was related to points since we talked about two of them, one that is invalid, and the one left. Concerning the markers, true, points should only be clipped if they are further away from the scene than half the markersize, so this presumed fix might be dangerous.

So apparently the code for optimizing paths in terms of clipping needs some work. If I remember correctly, this topic came up from time to time. The current path clipping solution can't handle filled paths for example, and as it seems it doesn't split path segments for achieving true clipping. There have been suggestions for using a more complete solution, I think someone pointed out Clipper could do the job.

So I also ran into this just now. I think, that the main problems in path clipping were already resolved in #5895. The remaining issue seems to be the markers. Having no glue about the inner workings of mpl, I tried skipping markers outside the canvas, see https://github.com/mabl/matplotlib/tree/fix_mpl_4482. It looks like this fixes the issue for me.

@mabl

I had this issue now as well when plotting with markers. The fix by @mabl solved it.

@mabl

@mabl Could you open a PR with that fix? It would be great to finally fix this.

@mabl

I just found that even after the @mabl fix, a problem lingers for certain markers, such as x, while others such as o are ok. After applying the fix the following code works for o but not for x:

import matplotlib
matplotlib.use('pgf')
import matplotlib.pyplot as plt

x=[1, 2, 0]
y=[1, 2, 1]

for marker in 'ox':
    fig, ax = plt.subplots()
    
    ax.scatter(x, y, marker=marker)
    ax.set_xscale('log')

    plt.savefig('plot_' + marker +'.pgf')
    plt.close()

I was not able to patch _print_pgf_path method, which is used here, in the same way as the @mabl fix.

In my case I can sanitise the data by changing x[2] to a non-zero value so I will be doing this for now instead of trying to fix this bug.

Any updates here? I have the same issue with a broken y-axis graph.

fig, (ax1, ax2) = plt.subplots(2, 1, sharex=True, gridspec_kw={'height_ratios': [1,5]}, figsize=(8,6))

ax1.bar(langs["language"], langs["count"])
ax2.bar(langs["language"], langs["count"])

ax1.set_ylim(119150, 119200)  # outliers only
ax2.set_ylim(0, 1200)  # most of the data

ax2.set_xticklabels(langs["language"], rotation=60, ha="right")
plt.tight_layout()
plt.savefig('language_distribution.pgf', format='pgf')

When using the above code to save a figure as pgf, LaTeX always crashes and says the dimension is too large.

I think a workaround would be to postprocess the output pgf to remove/clip any number above -226in (see my comments above).

I don't know how matplotlib works internally, but is it converting the values of the very large data from your ax1.set_ylim(119150, 119200) # outliers only as inches in the resulting plot? If so, maybe another workaround would be to scale the data down to smaller numbers, and manually adjust the y-axis labels to compensate (I haven't tested this, it's just a theory).

Really there's no need for a plot to produce a pgf with elements on the order of 226 inches (~18 feet) (TeX is thinking about literal inches on a printed page). Everything should be scaled down to reasonable values, or clipped if the value in question renders something way off the resulting page. But someone who actually understands how this stuff works would need to comment on how reasonable that would be to actually do.

Most vector viewers do the clipping for you, so if we put a data point at -1e6 inches, it doesn't matter so long as our clip mask is finite. We don't remove points in general because that is a difficult problem that truly is better handled by the downstream viewer.

PGF/tikz have clip boxes defined. It really seems to me that this is a PGF bug rather than something matplotlib should fix. i.e. they should only complain about out-of-range data if its not clipped.

PGF: Clip lines/markers to maximum LaTeX dimensions.

A simple fix for me was to change y-scale to symlog. Maybe if this error is produced this could be suggested to the user?

Navigation Menu

Repository navigation

PGF Backend: "Dimension too large" error while processing log-scale plot #4482

35 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

PGF Backend: "Dimension too large" error while processing log-scale plot #4482

Description

Activity

tacaswell commented on May 28, 2015

tacaswell commented on May 28, 2015

pwuertz commented on May 28, 2015

rwirth commented on May 29, 2015

pwuertz commented on May 29, 2015

pwuertz commented on May 29, 2015

rwirth commented on May 29, 2015

rwirth commented on May 29, 2015

pwuertz commented on May 29, 2015

35 remaining items

mabl commented on Aug 19, 2020

svaberg commented on Sep 29, 2020

maxnoe commented on Sep 29, 2020

svaberg commented on Oct 1, 2020

nikchha commented on Feb 14, 2021

asmeurer commented on Feb 15, 2021

jklymak commented on Feb 16, 2021

randolf-scholz commented on Jan 25, 2022

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions