Repository navigation
[MNT]: Python 3.14.0a7 test failures #29959
Description
Summary
I ran tox on the main branch of matplotlib with the fix from #29393 applied on top. I ran the same set of tests which we run during the Fedora matplotlib build, so invocation looked like this:
tox -e py314 -- -k 'not test_invisible_Line_rendering and not test_form_widget_get_with_datetime_and_date_fields' --pyargs matplotlib mpl_toolkits.axes_grid1 mpl_toolkits.axisartist mpl_toolkits.mplot3d
And the results:
________________________________________________ test_hexbin_log[png] _________________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_hexbin_log[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 35.934):
E result_images/test_axes/hexbin_log.png
E result_images/test_axes/hexbin_log-expected.png
E result_images/test_axes/hexbin_log-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
_______________________________________________ test_hexbin_linear[png] _______________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_hexbin_linear[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 20.427):
E result_images/test_axes/hexbin_linear.png
E result_images/test_axes/hexbin_linear-expected.png
E result_images/test_axes/hexbin_linear-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
__________________________________________________ test_symlog2[pdf] __________________________________________________
args = (), kwds = {'extension': 'pdf', 'request': <FixtureRequest for <Function test_symlog2[pdf]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 12.445):
E result_images/test_axes/symlog2_pdf.png
E result_images/test_axes/symlog2-expected_pdf.png
E result_images/test_axes/symlog2_pdf-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
_______________________________________ test_mollweide_forward_inverse_closure ________________________________________
def test_mollweide_forward_inverse_closure():
# test that the round-trip Mollweide forward->inverse transformation is an
# approximate identity
fig = plt.figure()
ax = fig.add_subplot(projection='mollweide')
# set up 1-degree grid in longitude, latitude
lon = np.linspace(-np.pi, np.pi, 360)
# The poles are degenerate and thus sensitive to floating point precision errors
lat = np.linspace(-np.pi / 2.0, np.pi / 2.0, 180)[1:-1]
lon, lat = np.meshgrid(lon, lat)
ll = np.vstack((lon.flatten(), lat.flatten())).T
# perform forward transform
> xy = ax.transProjection.transform(ll)
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_axes.py:5021:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/ksurma/dev/matplotlib/lib/matplotlib/transforms.py:1523: in transform
res = self.transform_affine(self.transform_non_affine(values))
/home/ksurma/dev/matplotlib/lib/matplotlib/projections/geo.py:372: in transform_non_affine
delta, large_delta = d(theta)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
theta = array([ 3.22129331e+27, 3.22129331e+27, 3.22129331e+27, ...,
-3.22129331e+27, -3.22129331e+27, -3.22129331e+27], shape=(61200,))
def d(theta):
> delta = (-(theta + np.sin(theta) - pi_sin_l)
/ (1 + np.cos(theta)))
E RuntimeWarning: overflow encountered in divide
/home/ksurma/dev/matplotlib/lib/matplotlib/projections/geo.py:355: RuntimeWarning
_______________________________________ test_mollweide_inverse_forward_closure ________________________________________
def test_mollweide_inverse_forward_closure():
# test that the round-trip Mollweide inverse->forward transformation is an
# approximate identity
fig = plt.figure()
ax = fig.add_subplot(projection='mollweide')
# set up grid in x, y
x = np.linspace(0, 1, 500)
x, y = np.meshgrid(x, x)
xy = np.vstack((x.flatten(), y.flatten())).T
# perform inverse transform
ll = ax.transProjection.inverted().transform(xy)
# perform forward transform
> xy2 = ax.transProjection.transform(ll)
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_axes.py:5045:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/home/ksurma/dev/matplotlib/lib/matplotlib/transforms.py:1523: in transform
res = self.transform_affine(self.transform_non_affine(values))
/home/ksurma/dev/matplotlib/lib/matplotlib/projections/geo.py:372: in transform_non_affine
delta, large_delta = d(theta)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
theta = array([ 0. , 0. , 0. , ..., -3.14323857,
-3.14323857, -3.14323857], shape=(250000,))
def d(theta):
> delta = (-(theta + np.sin(theta) - pi_sin_l)
/ (1 + np.cos(theta)))
E RuntimeWarning: divide by zero encountered in divide
/home/ksurma/dev/matplotlib/lib/matplotlib/projections/geo.py:355: RuntimeWarning
______________________________________________ test_specgram_angle[png] _______________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_specgram_angle[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 23.460):
E result_images/test_axes/specgram_phase_freqs.png
E result_images/test_axes/specgram_phase_freqs-expected.png
E result_images/test_axes/specgram_phase_freqs-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
__________________________________________ test_proportional_colorbars[png] ___________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_proportional_colorbars[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 47.976):
E result_images/test_colorbar/proportional_colorbars.png
E result_images/test_colorbar/proportional_colorbars-expected.png
E result_images/test_colorbar/proportional_colorbars-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
_______________________________________________ test_rgb_hsv_round_trip _______________________________________________
x = array([0.5488135 , 0.71518937, 0.60276338, ..., 0.31486942, 0.95593047,
0.76646949], shape=(750000,))
y = array([], dtype=float64)
def compare(x, y):
try:
if npany(isinf(x)) or npany(isinf(y)):
xinfid = isinf(x)
yinfid = isinf(y)
if not (xinfid == yinfid).all():
return False
# if one item, x and y is +- inf
if x.size == y.size == 1:
return x == y
x = x[~xinfid]
y = y[~yinfid]
except (TypeError, NotImplementedError):
pass
# make sure y is an inexact type to avoid abs(MIN_INT); will cause
# casting of x later.
dtype = result_type(y, 1.)
y = np.asanyarray(y, dtype)
> z = abs(x - y)
E ValueError: operands could not be broadcast together with shapes (750000,) (0,)
/home/ksurma/dev/matplotlib/.tox/py314/lib64/python3.14/site-packages/numpy/testing/_private/utils.py:1165: ValueError
During handling of the above exception, another exception occurred:
def test_rgb_hsv_round_trip():
for a_shape in [(500, 500, 3), (500, 3), (1, 3), (3,)]:
np.random.seed(0)
tt = np.random.random(a_shape)
> assert_array_almost_equal(
tt, mcolors.hsv_to_rgb(mcolors.rgb_to_hsv(tt)))
E ValueError:
E error during assertion:
E
E Traceback (most recent call last):
E File "/home/ksurma/dev/matplotlib/.tox/py314/lib64/python3.14/site-packages/numpy/testing/_private/utils.py", line 851, in assert_array_compare
E val = comparison(x, y)
E File "/home/ksurma/dev/matplotlib/.tox/py314/lib64/python3.14/site-packages/numpy/testing/_private/utils.py", line 1165, in compare
E z = abs(x - y)
E ~~^~~
E ValueError: operands could not be broadcast together with shapes (750000,) (0,)
E
E
E Arrays are not almost equal to 6 decimals
E ACTUAL: array([0.548814, 0.715189, 0.602763, ..., 0.314869, 0.95593 , 0.766469],
E shape=(750000,))
E DESIRED: array([], dtype=float64)
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_colors.py:947: ValueError
_________________________________________ test_light_source_topo_surface[png] _________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_light_source_topo_surface[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 31.976):
E result_images/test_colors/light_source_shading_topo.png
E result_images/test_colors/light_source_shading_topo-expected.png
E result_images/test_colors/light_source_shading_topo-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
__________________________________________________ test_labels[png] ___________________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_labels[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 30.608):
E result_images/test_contour/contour_test_label_transforms.png
E result_images/test_contour/contour_test_label_transforms-expected.png
E result_images/test_contour/contour_test_label_transforms-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
___________________________________________ test_mask_image_over_under[png] ___________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_mask_image_over_under[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 41.585):
E result_images/test_image/mask_image_over_under.png
E result_images/test_image/mask_image_over_under-expected.png
E result_images/test_image/mask_image_over_under-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
______________________________________________ test_rgba_antialias[png] _______________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_rgba_antialias[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 59.938):
E result_images/test_image/rgba_antialias.png
E result_images/test_image/rgba_antialias-expected.png
E result_images/test_image/rgba_antialias-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
_______________________________________________ test_downsampling[png] ________________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_downsampling[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 71.138):
E result_images/test_image/downsampling.png
E result_images/test_image/downsampling-expected.png
E result_images/test_image/downsampling-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
_______________________________________________ test_quiver_memory_leak _______________________________________________
@pytest.mark.skipif(platform.python_implementation() != 'CPython',
reason='Requires CPython')
def test_quiver_memory_leak():
fig, ax = plt.subplots()
Q = draw_quiver(ax)
ttX = Q.X
Q.remove()
del Q
> assert sys.getrefcount(ttX) == 2
E AssertionError
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_quiver.py:33: AssertionError
_____________________________________________ test_quiver_key_memory_leak _____________________________________________
@pytest.mark.skipif(platform.python_implementation() != 'CPython',
reason='Requires CPython')
def test_quiver_key_memory_leak():
fig, ax = plt.subplots()
Q = draw_quiver(ax)
qk = ax.quiverkey(Q, 0.5, 0.92, 2, r'$2 \frac{m}{s}$',
labelpos='W',
fontproperties={'weight': 'bold'})
> assert sys.getrefcount(qk) == 3
E AssertionError
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_quiver.py:46: AssertionError
_________________________________________________ test_fft_peaks[png] _________________________________________________
@image_comparison(['fft_peaks'], remove_text=True)
def test_fft_peaks():
fig, ax = plt.subplots()
t = np.arange(65536)
p1 = ax.plot(abs(np.fft.fft(np.sin(2*np.pi*.01*t)*np.blackman(len(t)))))
# Ensure that the path's transform takes the new axes limits into account.
fig.canvas.draw()
path = p1[0].get_path()
transform = p1[0].get_transform()
path = transform.transform_path(path)
simplified = path.cleaned(simplify=True)
> assert simplified.vertices.size == 36
E AssertionError
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_simplification.py:421: AssertionError
_________________________________________________ test_fft_peaks[pdf] _________________________________________________
@image_comparison(['fft_peaks'], remove_text=True)
def test_fft_peaks():
fig, ax = plt.subplots()
t = np.arange(65536)
p1 = ax.plot(abs(np.fft.fft(np.sin(2*np.pi*.01*t)*np.blackman(len(t)))))
# Ensure that the path's transform takes the new axes limits into account.
fig.canvas.draw()
path = p1[0].get_path()
transform = p1[0].get_transform()
path = transform.transform_path(path)
simplified = path.cleaned(simplify=True)
> assert simplified.vertices.size == 36
E AssertionError
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_simplification.py:421: AssertionError
_________________________________________________ test_fft_peaks[svg] _________________________________________________
@image_comparison(['fft_peaks'], remove_text=True)
def test_fft_peaks():
fig, ax = plt.subplots()
t = np.arange(65536)
p1 = ax.plot(abs(np.fft.fft(np.sin(2*np.pi*.01*t)*np.blackman(len(t)))))
# Ensure that the path's transform takes the new axes limits into account.
fig.canvas.draw()
path = p1[0].get_path()
transform = p1[0].get_transform()
path = transform.transform_path(path)
simplified = path.cleaned(simplify=True)
> assert simplified.vertices.size == 36
E AssertionError
/home/ksurma/dev/matplotlib/lib/matplotlib/tests/test_simplification.py:421: AssertionError
___________________________________________ test_tri_smooth_contouring[png] ___________________________________________
args = (), kwds = {'extension': 'png', 'request': <FixtureRequest for <Function test_tri_smooth_contouring[png]>>}
@wraps(func)
def inner(*args, **kwds):
with self._recreate_cm():
> return func(*args, **kwds)
E matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 123.603):
E result_images/test_triangulation/tri_smooth_contouring.png
E result_images/test_triangulation/tri_smooth_contouring-expected.png
E result_images/test_triangulation/tri_smooth_contouring-failed-diff.png
/usr/lib64/python3.14/contextlib.py:85: ImageComparisonFailure
FAILED tests/test_axes.py::test_hexbin_log[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 35.934):
FAILED tests/test_axes.py::test_hexbin_linear[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 20.427):
FAILED tests/test_axes.py::test_symlog2[pdf] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 12.445):
FAILED tests/test_axes.py::test_mollweide_forward_inverse_closure - RuntimeWarning: overflow encountered in divide
FAILED tests/test_axes.py::test_mollweide_inverse_forward_closure - RuntimeWarning: divide by zero encountered in divide
FAILED tests/test_axes.py::test_specgram_angle[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 23.460):
FAILED tests/test_colorbar.py::test_proportional_colorbars[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 47.976):
FAILED tests/test_colors.py::test_rgb_hsv_round_trip - ValueError:
FAILED tests/test_colors.py::test_light_source_topo_surface[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 31.976):
FAILED tests/test_contour.py::test_labels[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 30.608):
FAILED tests/test_image.py::test_mask_image_over_under[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 41.585):
FAILED tests/test_image.py::test_rgba_antialias[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 59.938):
FAILED tests/test_image.py::test_downsampling[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 71.138):
FAILED tests/test_quiver.py::test_quiver_memory_leak - AssertionError
FAILED tests/test_quiver.py::test_quiver_key_memory_leak - AssertionError
FAILED tests/test_simplification.py::test_fft_peaks[png] - AssertionError
FAILED tests/test_simplification.py::test_fft_peaks[pdf] - AssertionError
FAILED tests/test_simplification.py::test_fft_peaks[svg] - AssertionError
FAILED tests/test_triangulation.py::test_tri_smooth_contouring[png] - matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 123.603):
================= 19 failed, 9423 passed, 507 skipped, 2 deselected, 30 xfailed in 330.41s (0:05:30) ==================
Proposed fix
No response
Activity
ngoldbaum commentedon Apr 22, 2025
FYI, a NumPy optimization that elides temporaries is broken on 3.14. Probably best to wait until numpy/numpy#28748 (comment) is merged to work on downstream libraries, otherwise you'll be dealing with deeply confusing test failures.
befeleme commentedon Apr 23, 2025
FYI, I jumped to test matplotlib on numpy 2.2.5 with that PR patched on top - it fixed most of the problems, remaining ones seem to be genuine for matplotlib:
tacaswell commentedon Apr 23, 2025
It looks like the refcount is coming in one better than expected (I suspect this is due to an optimization in refcounting where it changes using "borrow" bytecodes when it decides it is safe so the refcount inside of
getrefcount()is not added), we probably need to make the expected number python dependent.ngoldbaum commentedon Apr 23, 2025
In NumPy I handled updating tests that depend on refcounts to check the change in the refcount rather than a constant refcount value. That should be the same in all Python versions. There was only one other obscure case in NumPy where that didn’t work because we happened to trigger another new refcount optimization on top of the stackref optimization.
QuLogic commentedon Sep 17, 2025
This was fixed by #29970