Repository navigation
Description
Going through the logs from @mattip 's logs in #19154 I think the failures we are seeing can be classified as below.
finalizer / weakrefs different in pypy
We probably should just detect and skip pypy on these.
- FAILED lib/matplotlib/tests/test_animation.py::test_animation_delete[anim0]
- FAILED lib/matplotlib/tests/test_animation.py::test_animation_repr_html[anim0-none-None-imagemagick]
- FAILED lib/matplotlib/tests/test_animation.py::test_animation_repr_html[anim0-none-None-ffmpeg]
- FAILED lib/matplotlib/tests/test_animation.py::test_funcanimation_cache_frame_data[False]
- FAILED lib/matplotlib/tests/test_cbook.py::Test_callback_registry::test_callback_complete
pypy does not implement a working sys.getsizeof
Skip this test on pypy, we are only using it to make sure that the output of rendering makes sense.
- FAILED lib/matplotlib/tests/test_backend_pdf.py::test_multipage_properfinalize
stack walking different?
The failures due to raising an unexpected warning are a bit more concerning. We are doing some frame walking / inspection to sort out if we should warn or not in
matplotlib/lib/matplotlib/backend_bases.py
Lines 1624 to 1665 in 09ab2c2
| @functools.wraps(func) | |
| def wrapper(*args, **kwargs): | |
| name = 'savefig' # Reasonable default guess. | |
| public_api = re.compile(r'^savefig|print_[A-Za-z0-9]+$') | |
| seen_print_figure = False | |
| for frame, line in traceback.walk_stack(None): | |
| if frame is None: | |
| # when called in embedded context may hit frame is None. | |
| break | |
| if re.match(r'\A(matplotlib|mpl_toolkits)(\Z|\.(?!tests\.))', | |
| # Work around sphinx-gallery not setting __name__. | |
| frame.f_globals.get('__name__', '')): | |
| if public_api.match(frame.f_code.co_name): | |
| name = frame.f_code.co_name | |
| if name == 'print_figure': | |
| seen_print_figure = True | |
| else: | |
| break | |
| accepted_kwargs = {*old_sig.parameters, *extra_kwargs} | |
| if seen_print_figure: | |
| for kw in ['dpi', 'facecolor', 'edgecolor', 'orientation', | |
| 'bbox_inches_restore']: | |
| # Ignore keyword arguments that are passed in by print_figure | |
| # for the use of other renderers. | |
| if kw not in accepted_kwargs: | |
| kwargs.pop(kw, None) | |
| for arg in list(kwargs): | |
| if arg in accepted_kwargs: | |
| continue | |
| cbook.warn_deprecated( | |
| '3.3', name=name, | |
| message='%(name)s() got unexpected keyword argument "' | |
| + arg + '" which is no longer supported as of ' | |
| '%(since)s and will become an error ' | |
| '%(removal)s') | |
| kwargs.pop(arg) | |
| return func(*args, **kwargs) | |
| return wrapper |
If someone who understands pypy can fix that to work on both that would be best, but if not we should detect if we are on pypy and fall back to never warning. It is operating in a slightly degraded mode (users may not get warnings they should see), but the status quo is going to warn for things the user can not do anything about (I think every time we use tightlayout of bbox_inches='tight').
This is probably the only failures I would hold 3.4 over.
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[pdf]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_clipping[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_clipping[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_raster[pdf]
- FAILED lib/matplotlib/tests/test_backend_pgf.py::test_bbox_inches_tight - mat...
- FAILED lib/matplotlib/tests/test_backend_pgf.py::test_mixedmode[pdf] - matplo...
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight[pdf]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_suptile_legend[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_clipping[pdf]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_raster[png]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_bbox_inches_tight_raster[svg]
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_tight_pcolorfast - matpl...
- FAILED lib/matplotlib/tests/test_figure.py::test_tightbbox_box_aspect[svg] - ...
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_only_on_non_finite_bbox
- FAILED lib/matplotlib/tests/test_bbox_tight.py::test_noop_tight_bbox - matplo...
- FAILED lib/matplotlib/tests/test_offsetbox.py::test_annotationbbox_extents - ...
- FAILED lib/mpl_toolkits/tests/test_axes_grid1.py::test_image_grid[png] - matp...
segfaults in tk
We should either track this down or just skip the tests.
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolbar2-tkagg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolmanager-gtk3agg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolmanager-tkagg]
subprocess hung
We should either track this down or just skip the tests.
- FAILED lib/matplotlib/tests/test_backend_tk.py::test_never_update - Failed: S...
- FAILED lib/matplotlib/tests/test_backend_tk.py::test_figuremanager_cleans_own_mainloop
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_backend[toolbar2-gtk3agg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_thread_safety[gtk3agg]
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_interactive_thread_safety[tkagg]
not matching numpy?
These both look like they are upstream bugs in pypy? We should probably just skip these.
- FAILED lib/matplotlib/tests/test_cbook.py::test_step_fails[args1] - SystemErr...
- FAILED lib/matplotlib/tests/test_matplotlib.py::test_importable_with__OO - As...
checking refcounts
These tests do not make sense for pypy, we should skip them.
- FAILED lib/matplotlib/tests/test_quiver.py::test_quiver_memory_leak - Attribu...
- FAILED lib/matplotlib/tests/test_quiver.py::test_quiver_key_memory_leak - Att...
sockets not working
The comments around this suggest we never got this working on azure (which I assume is what is hosting GH actions) so we should sort out why the code to not run these tests in that case is not being triggered here.
- FAILED lib/matplotlib/tests/test_backends_interactive.py::test_webagg - Faile...
Activity
anntzer commentedon Dec 21, 2020
I don't have an easy way to check, but I guess the animation failures can be fixed by just adding a call to
gc.collect()afterdel animor beforefor f in frames_generated?QuLogic commentedon Aug 14, 2021
I don't know how to fix these animation warning tests, because adding
gc.collectdoes not help:Clearly, the warning is happening, because it is printed out to stderr:
but it does not seem to get captures by
pytest.warns.Otherwise, I've managed to fixed all the other failures (except maybe Qt, since I couldn't install the deps).
mattip commentedon Aug 14, 2021
I think you need to call gc.collect() at least 3 times in a loop. NumPy has a
break_cyclesfunction for thisQuLogic commentedon Aug 16, 2021
Thanks, that's even public too. However, it doesn't work, even calling it twice (so
gc.collect8 times), the Animation is not cleaned up in thepytest.warns(but it is when the test exits).QuLogic commentedon Aug 16, 2021
Or maybe the
Animationis cleaned up there, but pytest is unable to capture the warning that's raised in__del__?QuLogic commentedon Aug 17, 2021
So I added a
weakrefon the object, and checked it before and afterbreak_cycles, and the object is gone. Looking atpytest.warns, it's really just a wrapper aroundwarnings.catch_warnings, so I changed the test to:and still no warnings are recorded, but are printed to stderr as an 'ignored Exception'. It seems that PyPy does not support capturing warnings emitted in
__del__.QuLogic commentedon Aug 17, 2021
The closest I can get is to mark the test with
@pytest.mark.filterwarnings('always'), which converts the warning from an unhandled exception (because we set warnings to error) in a destructor to a warning captured at the exit of the test, but the change to warning filtering within the test does nothing.mattip commentedon Aug 17, 2021
It might be best just to skip this test on PyPy so you don't spend too much time with it. As I understand it, the warning is meant to inform the user that the animation has never been run, which could be considered a bit of an edge case. PyPy does catch warnings in other cases. Just yesterday I added a single
gc.collect()to some lib-python tests to get a warning printed when the object is destroyed, so I am not sure what is going on here specifically.QuLogic commentedon Aug 17, 2021
Too late; I spent yesterday whittling it down from tests, fixtures, and Matplotlib code to a small script: https://foss.heptapod.net/pypy/pypy/-/issues/3536