Repository navigation
check_figures_equal decorator reuses figures if called multiple times inside a single test. #15079
Description
Bug report
Bug summary
When running a test suite with multiple tests using check_figures_equal, figures are re-used, causing plot elements to become "stacked", causing tests to unexpectedly fail.
Code for reproduction
from matplotlib.testing.decorators import check_figures_equal
from matplotlib import pyplot as plt
@check_figures_equal(extensions=["png"])
def test_red_line_first(test_figure, reference_figure):
red_circle = plt.Circle((0, 0), 0.2, color='r', clip_on=False)
reference_figure.add_artist(red_circle)
reference_figure.subplots().plot([0,1,2], [3,4,5], color='red')
@check_figures_equal(extensions=["png"])
def test_blue_line_second(test_figure, reference_figure):
blue_circle = plt.Circle((1, 1), 0.2, color='b', clip_on=False)
reference_figure.add_artist(blue_circle)
reference_figure.subplots().plot([0,1,2], [4,4,4], color='blue')
@check_figures_equal(extensions=["png"])
def test_green_line_third(test_figure, reference_figure):
green_circle = plt.Circle((1, 0), 0.2, color='g', clip_on=False)
reference_figure.add_artist(green_circle)
reference_figure.subplots().plot([0,1,2], [5,4,3], color='green')running as:
pytest .\mpl_figure_equal_test.py
Actual outcome
Reference Screenshots:
Expected outcome
Reference Screenshots:
Matplotlib version
- Operating system:
Windows 10 - Matplotlib version:
matplotlib 3.1.0 - Matplotlib backend (
print(matplotlib.get_backend())):TkAgg - Python version:
3.7.3 - Jupyter version (if applicable):
- Other libraries:
pytest 4.5.0
everything installed via pip
matplotlib/lib/matplotlib/testing/decorators.py
Lines 394 to 395 in e86c9cd
I'm guessing these lines are at fault, and might need some kind of other information to distinguish them?
Activity
anntzer commentedon Aug 18, 2019
Probably one should just wrap the whole thing in a
try... finally: plt.close(fig_test); plt.close(fitg_ref)? In our own test suite this is done by an autouse fixture but I guess adding the close() to check_figures_equal makes it more usable by third-party packages.QuLogic commentedon Aug 18, 2019
All our tests autouse a fixture from
matplotlib.testing.conftestthat closes all figure at the end of a test. If you're not running these as Matplotlib tests, you'll need to replicate that behaviour.andrewzwicky commentedon Aug 18, 2019
Both of those make sense. Is
check_figures_equalintended for use in third-party projects? It seems quite useful, so it might make sense to either update the documentation re: extra testing steps for closing figures or include that functionality to close just those two figures. Do maintainers have a preference? It seems like a relatively small thing I can contribute.I just tested a small fix with
try/finallyand it worked as I expected. I think having to add extra figure closing stuff in my own test suite would make me scratch my head a bit, so I'd prefer theplt.closeincluded in the wrapper, but I understand if that's not desirable for some other reason.anntzer commentedon Aug 18, 2019
I'm personally fine with a pr adding the call to close.