Skip to content

Matplotlib git master version fails to pass several pytest's tests #17310

Closed

Description

@hongyi-zhao

Hi,

The environment is as follows: Ubuntu 19.10, python 3.8.2 managed by pyenv and matplotlib git master version. See the following for the testing steps and results:

$ sudo apt-get build-dep python3-matplotlib python3-matplotlib-venn
$ pip install -e .[dev,extra,all]
$ pytest
========================================== test session starts ==========================================
platform linux -- Python 3.8.2, pytest-5.4.1, py-1.5.4, pluggy-0.13.1
rootdir: /home/werner/Public/hpc/tools/matplotlib.git, inifile: pytest.ini, testpaths: lib
plugins: cov-2.8.1, datadir-1.3.1, regressions-1.0.6, timeout-1.3.4
collected 7591 items     
[...]
================================================ FAILURES =================================================
___________________________________________ test_pdflatex[pdf] ____________________________________________

expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_pdflatex-expected.pdf'
actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_pdflatex.pdf')
tol = 0

    def _raise_on_image_difference(expected, actual, tol):
        __tracebackhide__ = True
    
        err = compare_images(expected, actual, tol, in_decorator=True)
        if err:
            for key in ["actual", "expected"]:
                err[key] = os.path.relpath(err[key])
>           raise ImageComparisonFailure(
                'images not close (RMS %(rms).3f):\n\t%(actual)s\n\t%(expected)s '
                 % err)
E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 11.669):
E           	result_images/test_backend_pgf/pgf_pdflatex_pdf.png
E           	result_images/test_backend_pgf/pgf_pdflatex-expected_pdf.png

lib/matplotlib/testing/decorators.py:135: ImageComparisonFailure
______________________________________________ test_rcupdate ______________________________________________

    @needs_xelatex
    @needs_pdflatex
    @pytest.mark.skipif(not _has_sfmath(), reason='needs sfmath.sty')
    @pytest.mark.style('default')
    @pytest.mark.backend('pgf')
    def test_rcupdate():
        rc_sets = [{'font.family': 'sans-serif',
                    'font.size': 30,
                    'figure.subplot.left': .2,
                    'lines.markersize': 10,
                    'pgf.rcfonts': False,
                    'pgf.texsystem': 'xelatex'},
                   {'font.family': 'monospace',
                    'font.size': 10,
                    'figure.subplot.left': .1,
                    'lines.markersize': 20,
                    'pgf.rcfonts': False,
                    'pgf.texsystem': 'pdflatex',
                    'pgf.preamble': ('\\usepackage[utf8x]{inputenc}'
                                     '\\usepackage[T1]{fontenc}'
                                     '\\usepackage{sfmath}')}]
        tol = [6, 0]
        for i, rc_set in enumerate(rc_sets):
            with mpl.rc_context(rc_set):
                create_figure()
>               compare_figure('pgf_rcupdate%d.pdf' % (i + 1), tol=tol[i])

lib/matplotlib/tests/test_backend_pgf.py:157: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

fname = 'pgf_rcupdate2.pdf', savefig_kwargs = {}, tol = 0

    def compare_figure(fname, savefig_kwargs={}, tol=0):
        actual = os.path.join(result_dir, fname)
        plt.savefig(actual, **savefig_kwargs)
    
        expected = os.path.join(result_dir, "expected_%s" % fname)
        shutil.copyfile(os.path.join(baseline_dir, fname), expected)
        err = compare_images(expected, actual, tol=tol)
        if err:
>           raise ImageComparisonFailure(err)
E           matplotlib.testing.exceptions.ImageComparisonFailure: Error: Image files did not match.
E             RMS Value: 13.166575390337284
E             Expected:  
E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/expected_pgf_rcupdate2_pdf.png
E             Actual:    
E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_rcupdate2_pdf.png
E             Difference:
E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_rcupdate2_pdf-failed-diff.png
E             Tolerance: 
E               0

lib/matplotlib/tests/test_backend_pgf.py:63: ImageComparisonFailure
____________________________________________ test_usetex[png] _____________________________________________

expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_usetex/test_usetex-expected.png'
actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_usetex/test_usetex.png')
tol = 0

    def _raise_on_image_difference(expected, actual, tol):
        __tracebackhide__ = True
    
        err = compare_images(expected, actual, tol, in_decorator=True)
        if err:
            for key in ["actual", "expected"]:
                err[key] = os.path.relpath(err[key])
>           raise ImageComparisonFailure(
                'images not close (RMS %(rms).3f):\n\t%(actual)s\n\t%(expected)s '
                 % err)
E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 12.877):
E           	result_images/test_usetex/test_usetex.png
E           	result_images/test_usetex/test_usetex-expected.png

lib/matplotlib/testing/decorators.py:135: ImageComparisonFailure
========================================= short test summary info =========================================
FAILED lib/matplotlib/tests/test_backend_pgf.py::test_pdflatex[pdf] - matplotlib.testing.exceptions.Imag...
FAILED lib/matplotlib/tests/test_backend_pgf.py::test_rcupdate - matplotlib.testing.exceptions.ImageComp...
FAILED lib/matplotlib/tests/test_usetex.py::test_usetex[png] - matplotlib.testing.exceptions.ImageCompar...
=================== 3 failed, 7517 passed, 60 skipped, 11 xfailed in 630.55s (0:10:30) ====================

Any hints for solving this problems?

Regards

Activity

  1. changed the title [-]pytest failed for some tests.[/-] [+]Matploglib git master version fails to pass serveral pytest's tests.[/+] on May 3, 2020
  2. changed the title [-]Matploglib git master version fails to pass serveral pytest's tests.[/-] [+]Matplotlib git master version fails to pass serveral pytest's tests.[/+] on May 3, 2020
  3. timhoffm commented on May 3, 2020

    @timhoffm
    Member

    Compare the actual and expected images in the result_images folder. That tells you what's unexpected and will give a hint what's causing the failure.

  4. hongyi-zhao commented on May 3, 2020

    @hongyi-zhao
    Author

    I find all of the failed-diff files as following:

    werner@ubuntu-01:~/Public/hpc/tools/matplotlib.git/result_images$ find . -type f -name '*failed-diff*'
    ./test_compare_images/all127-failed-diff.png
    ./test_compare_images/all128-failed-diff.png
    ./test_compare_images/basn3p02-failed-diff.png
    ./test_backend_pgf/pgf_rcupdate2_pdf-failed-diff.png
    ./test_backend_pgf/pgf_pdflatex_pdf-failed-diff.png
    ./test_usetex/test_usetex-failed-diff.png
    

    But some of them are "empty" without content, others have something like garbled. I still cannot figure out the reason. See the following for you info:

    image
    image
    image
    image

  5. anntzer commented on May 3, 2020

    @anntzer
    Contributor

    What is your version of dvipng? If it is 1.16 there's a known issue about that (#16476 (comment)). Otherwise please delete your ~/.cache/matplotlib/tex.cache and try again.

  6. hongyi-zhao commented on May 4, 2020

    @hongyi-zhao
    Author

    Now, I updated to dvipng 1.17 shipped with TeX Live 2020, see following:

    $ dvipng --version
    This is dvipng 1.17 Copyright 2002-2015, 2019 Jan-Ake Larsson
    dvipng (TeX Live) 1.17
    kpathsea version 6.3.2
    Compiled with Freetype 2.10.1
    Using libft 2.10.1
    Copyright (C) 2002-2015, 2019 Jan-Ake Larsson.
    There is NO warranty.  You may redistribute this software
    under the terms of the GNU Lesser General Public License
    version 3, see the COPYING file in the dvipng distribution
    or <http://www.gnu.org/licenses/>.
    

    But pytest still throws errors, see following:

    
    $ pytest
    =============================== test session starts ===============================
    platform linux -- Python 3.8.2, pytest-5.4.1, py-1.5.4, pluggy-0.13.1
    rootdir: /home/werner/Public/hpc/tools/matplotlib.git, inifile: pytest.ini, testpaths: lib
    plugins: cov-2.8.1, datadir-1.3.1, regressions-1.0.6, timeout-1.3.4
    collected 7593 items                                                  
    [...]
    ==================================== FAILURES =====================================
    _______________________________ test_pdflatex[pdf] ________________________________
    
    expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_pdflatex-expected.pdf'
    actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_pdflatex.pdf')
    tol = 0
    
        def _raise_on_image_difference(expected, actual, tol):
            __tracebackhide__ = True
        
            err = compare_images(expected, actual, tol, in_decorator=True)
            if err:
                for key in ["actual", "expected"]:
                    err[key] = os.path.relpath(err[key])
    >           raise ImageComparisonFailure(
                    'images not close (RMS %(rms).3f):\n\t%(actual)s\n\t%(expected)s '
                     % err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 11.669):
    E           	result_images/test_backend_pgf/pgf_pdflatex_pdf.png
    E           	result_images/test_backend_pgf/pgf_pdflatex-expected_pdf.png
    
    lib/matplotlib/testing/decorators.py:135: ImageComparisonFailure
    __________________________________ test_rcupdate __________________________________
    
        @needs_xelatex
        @needs_pdflatex
        @pytest.mark.skipif(not _has_sfmath(), reason='needs sfmath.sty')
        @pytest.mark.style('default')
        @pytest.mark.backend('pgf')
        def test_rcupdate():
            rc_sets = [{'font.family': 'sans-serif',
                        'font.size': 30,
                        'figure.subplot.left': .2,
                        'lines.markersize': 10,
                        'pgf.rcfonts': False,
                        'pgf.texsystem': 'xelatex'},
                       {'font.family': 'monospace',
                        'font.size': 10,
                        'figure.subplot.left': .1,
                        'lines.markersize': 20,
                        'pgf.rcfonts': False,
                        'pgf.texsystem': 'pdflatex',
                        'pgf.preamble': ('\\usepackage[utf8x]{inputenc}'
                                         '\\usepackage[T1]{fontenc}'
                                         '\\usepackage{sfmath}')}]
            tol = [6, 0]
            for i, rc_set in enumerate(rc_sets):
                with mpl.rc_context(rc_set):
                    create_figure()
    >               compare_figure('pgf_rcupdate%d.pdf' % (i + 1), tol=tol[i])
    
    lib/matplotlib/tests/test_backend_pgf.py:157: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    fname = 'pgf_rcupdate2.pdf', savefig_kwargs = {}, tol = 0
    
        def compare_figure(fname, savefig_kwargs={}, tol=0):
            actual = os.path.join(result_dir, fname)
            plt.savefig(actual, **savefig_kwargs)
        
            expected = os.path.join(result_dir, "expected_%s" % fname)
            shutil.copyfile(os.path.join(baseline_dir, fname), expected)
            err = compare_images(expected, actual, tol=tol)
            if err:
    >           raise ImageComparisonFailure(err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: Error: Image files did not match.
    E             RMS Value: 13.166575390337284
    E             Expected:  
    E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/expected_pgf_rcupdate2_pdf.png
    E             Actual:    
    E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_rcupdate2_pdf.png
    E             Difference:
    E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_rcupdate2_pdf-failed-diff.png
    E             Tolerance: 
    E               0
    
    lib/matplotlib/tests/test_backend_pgf.py:63: ImageComparisonFailure
    ============================= short test summary info =============================
    FAILED lib/matplotlib/tests/test_backend_pgf.py::test_pdflatex[pdf] - matplotlib...
    FAILED lib/matplotlib/tests/test_backend_pgf.py::test_rcupdate - matplotlib.test...
    ======= 2 failed, 7520 passed, 60 skipped, 11 xfailed in 627.69s (0:10:27) ========
    
    werner@ubuntu-01:~/Public/hpc/tools/matplotlib.git/result_images$ find . -type f -name '*failed-diff*'
    ./test_compare_images/all127-failed-diff.png
    ./test_compare_images/all128-failed-diff.png
    ./test_compare_images/basn3p02-failed-diff.png
    ./test_backend_pgf/pgf_rcupdate2_pdf-failed-diff.png
    ./test_backend_pgf/pgf_pdflatex_pdf-failed-diff.png
    
    
  7. tacaswell commented on May 4, 2020

    @tacaswell
    Member

    But some of them are only blank without content,

    Almost all of our tests are set to 0 difference thresholds, some of the "empty" images make have only a few pixels off by a few bits which is extremely hard to see by eye in the difference images. That said, the three images in test_compare_images/ are red-herrings, they are testing that things do fail when expected.

  8. added this to the v3.3.0 milestone on May 4, 2020
  9. hongyi-zhao commented on May 4, 2020

    @hongyi-zhao
    Author

    But what's the meaning of this from my testing results:

    60 skipped, 11 xfailed

    Regards

  10. tacaswell commented on Jun 2, 2020

    @tacaswell
    Member

    Sorry for the very slow response

    60 skipped, 11 xfailed

    That means 60 tests were skipped (usually because you are missing an optional dependancy) and 11 tests that we expect to fail failed (some are tests that are documenting bugs, at least one is testing that the tests fail when we think they should).

  11. hongyi-zhao commented on Jun 2, 2020

    @hongyi-zhao
    Author

    Sorry for the very slow response

    60 skipped, 11 xfailed

    That means 60 tests were skipped (usually because you are missing an optional dependancy)

    How to check out the full list for these dependencies?

    I install the git master version of matplotlib using the following command in its local repo folder:
    $ pip install -e .

    and 11 tests that we expect to fail failed (some are tests that are documenting bugs, at least one is testing that the tests fail when we think they should).

  12. tacaswell commented on Jun 2, 2020

    @tacaswell
    Member

    Look at the files https://github.com/matplotlib/matplotlib/tree/master/requirements/testing for what we install on CI.

    I think we have some platform specific tests so it may not be possible to run every test on your system (this is why we test on windows, linux, and OSX as part of CI).

    I suspect you are missing one or more of the GUI frameworks.

  13. hongyi-zhao commented on Jun 2, 2020

    @hongyi-zhao
    Author

    Look at the files https://github.com/matplotlib/matplotlib/tree/master/requirements/testing for what we install on CI.

    Still no so clear. There are several files and do you mean I should install all the packages listed in these files for doing the tests?

    I think we have some platform specific tests so it may not be possible to run every test on your system (this is why we test on windows, linux, and OSX as part of CI).

    I suspect you are missing one or more of the GUI frameworks.

  14. hongyi-zhao commented on Jun 2, 2020

    @hongyi-zhao
    Author

    I installed the requirements list here with the following commands:

    $ pip install -r flake8.txt
    $ pip install -r travis_all.txt
    $ pip install -r travis_extra.txt
    

    Then run pytest and obtain the following results:

    ===================================== FAILURES ======================================
    _____________________________ test_use14corefonts[pdf] ______________________________
    
    expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pdf/pdf_use14corefonts-expected.pdf'
    actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pdf/pdf_use14corefonts.pdf')
    tol = 0
    
        def _raise_on_image_difference(expected, actual, tol):
            __tracebackhide__ = True
        
            err = compare_images(expected, actual, tol, in_decorator=True)
            if err:
                for key in ["actual", "expected", "diff"]:
                    err[key] = os.path.relpath(err[key])
    >           raise ImageComparisonFailure(
                    ('images not close (RMS %(rms).3f):'
                        '\n\t%(actual)s\n\t%(expected)s\n\t%(diff)s') % err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 3.233):
    E           	result_images/test_backend_pdf/pdf_use14corefonts_pdf.png
    E           	result_images/test_backend_pdf/pdf_use14corefonts-expected_pdf.png
    E           	result_images/test_backend_pdf/pdf_use14corefonts_pdf-failed-diff.png
    
    lib/matplotlib/testing/decorators.py:139: ImageComparisonFailure
    _________________________________ test_xelatex[pdf] _________________________________
    
    expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_xelatex-expected.pdf'
    actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_xelatex.pdf')
    tol = 0
    
        def _raise_on_image_difference(expected, actual, tol):
            __tracebackhide__ = True
        
            err = compare_images(expected, actual, tol, in_decorator=True)
            if err:
                for key in ["actual", "expected", "diff"]:
                    err[key] = os.path.relpath(err[key])
    >           raise ImageComparisonFailure(
                    ('images not close (RMS %(rms).3f):'
                        '\n\t%(actual)s\n\t%(expected)s\n\t%(diff)s') % err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 1.043):
    E           	result_images/test_backend_pgf/pgf_xelatex_pdf.png
    E           	result_images/test_backend_pgf/pgf_xelatex-expected_pdf.png
    E           	result_images/test_backend_pgf/pgf_xelatex_pdf-failed-diff.png
    
    lib/matplotlib/testing/decorators.py:139: ImageComparisonFailure
    ________________________________ test_pdflatex[pdf] _________________________________
    
    expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_pdflatex-expected.pdf'
    actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_pdflatex.pdf')
    tol = 0
    
        def _raise_on_image_difference(expected, actual, tol):
            __tracebackhide__ = True
        
            err = compare_images(expected, actual, tol, in_decorator=True)
            if err:
                for key in ["actual", "expected", "diff"]:
                    err[key] = os.path.relpath(err[key])
    >           raise ImageComparisonFailure(
                    ('images not close (RMS %(rms).3f):'
                        '\n\t%(actual)s\n\t%(expected)s\n\t%(diff)s') % err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 11.704):
    E           	result_images/test_backend_pgf/pgf_pdflatex_pdf.png
    E           	result_images/test_backend_pgf/pgf_pdflatex-expected_pdf.png
    E           	result_images/test_backend_pgf/pgf_pdflatex_pdf-failed-diff.png
    
    lib/matplotlib/testing/decorators.py:139: ImageComparisonFailure
    ___________________________________ test_rcupdate ___________________________________
    
        @needs_xelatex
        @needs_pdflatex
        @pytest.mark.skipif(not _has_sfmath(), reason='needs sfmath.sty')
        @pytest.mark.style('default')
        @pytest.mark.backend('pgf')
        def test_rcupdate():
            rc_sets = [{'font.family': 'sans-serif',
                        'font.size': 30,
                        'figure.subplot.left': .2,
                        'lines.markersize': 10,
                        'pgf.rcfonts': False,
                        'pgf.texsystem': 'xelatex'},
                       {'font.family': 'monospace',
                        'font.size': 10,
                        'figure.subplot.left': .1,
                        'lines.markersize': 20,
                        'pgf.rcfonts': False,
                        'pgf.texsystem': 'pdflatex',
                        'pgf.preamble': ('\\usepackage[utf8x]{inputenc}'
                                         '\\usepackage[T1]{fontenc}'
                                         '\\usepackage{sfmath}')}]
            tol = [6, 0]
            for i, rc_set in enumerate(rc_sets):
                with mpl.rc_context(rc_set):
                    create_figure()
    >               compare_figure('pgf_rcupdate%d.pdf' % (i + 1), tol=tol[i])
    
    lib/matplotlib/tests/test_backend_pgf.py:157: 
    _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
    
    fname = 'pgf_rcupdate2.pdf', savefig_kwargs = {}, tol = 0
    
        def compare_figure(fname, savefig_kwargs={}, tol=0):
            actual = os.path.join(result_dir, fname)
            plt.savefig(actual, **savefig_kwargs)
        
            expected = os.path.join(result_dir, "expected_%s" % fname)
            shutil.copyfile(os.path.join(baseline_dir, fname), expected)
            err = compare_images(expected, actual, tol=tol)
            if err:
    >           raise ImageComparisonFailure(err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: Error: Image files did not match.
    E             RMS Value: 13.166575390337284
    E             Expected:  
    E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/expected_pgf_rcupdate2_pdf.png
    E             Actual:    
    E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_rcupdate2_pdf.png
    E             Difference:
    E               /home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_rcupdate2_pdf-failed-diff.png
    E             Tolerance: 
    E               0
    
    lib/matplotlib/tests/test_backend_pgf.py:63: ImageComparisonFailure
    ________________________________ test_mixedmode[pdf] ________________________________
    
    expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_mixedmode-expected.pdf'
    actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_backend_pgf/pgf_mixedmode.pdf')
    tol = 1.086
    
        def _raise_on_image_difference(expected, actual, tol):
            __tracebackhide__ = True
        
            err = compare_images(expected, actual, tol, in_decorator=True)
            if err:
                for key in ["actual", "expected", "diff"]:
                    err[key] = os.path.relpath(err[key])
    >           raise ImageComparisonFailure(
                    ('images not close (RMS %(rms).3f):'
                        '\n\t%(actual)s\n\t%(expected)s\n\t%(diff)s') % err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 5.063):
    E           	result_images/test_backend_pgf/pgf_mixedmode_pdf.png
    E           	result_images/test_backend_pgf/pgf_mixedmode-expected_pdf.png
    E           	result_images/test_backend_pgf/pgf_mixedmode_pdf-failed-diff.png
    
    lib/matplotlib/testing/decorators.py:139: ImageComparisonFailure
    ______________________________ test_font_scaling[pdf] _______________________________
    
    expected = '/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_text/font_scaling-expected.pdf'
    actual = PosixPath('/home/werner/Public/hpc/tools/matplotlib.git/result_images/test_text/font_scaling.pdf')
    tol = 0
    
        def _raise_on_image_difference(expected, actual, tol):
            __tracebackhide__ = True
        
            err = compare_images(expected, actual, tol, in_decorator=True)
            if err:
                for key in ["actual", "expected", "diff"]:
                    err[key] = os.path.relpath(err[key])
    >           raise ImageComparisonFailure(
                    ('images not close (RMS %(rms).3f):'
                        '\n\t%(actual)s\n\t%(expected)s\n\t%(diff)s') % err)
    E           matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 2.177):
    E           	result_images/test_text/font_scaling_pdf.png
    E           	result_images/test_text/font_scaling-expected_pdf.png
    E           	result_images/test_text/font_scaling_pdf-failed-diff.png
    
    lib/matplotlib/testing/decorators.py:139: ImageComparisonFailure
    ================================= warnings summary ==================================
    lib/matplotlib/widgets.py:1911
      /home/werner/Public/hpc/tools/matplotlib.git/lib/matplotlib/widgets.py:1911: DeprecationWarning: invalid escape sequence \s
        """
    
    -- Docs: https://docs.pytest.org/en/latest/warnings.html
    ============================== short test summary info ==============================
    FAILED lib/matplotlib/tests/test_backend_pdf.py::test_use14corefonts[pdf] - matplo...
    FAILED lib/matplotlib/tests/test_backend_pgf.py::test_xelatex[pdf] - matplotlib.te...
    FAILED lib/matplotlib/tests/test_backend_pgf.py::test_pdflatex[pdf] - matplotlib.t...
    FAILED lib/matplotlib/tests/test_backend_pgf.py::test_rcupdate - matplotlib.testin...
    FAILED lib/matplotlib/tests/test_backend_pgf.py::test_mixedmode[pdf] - matplotlib....
    FAILED lib/matplotlib/tests/test_text.py::test_font_scaling[pdf] - matplotlib.test...
    == 6 failed, 6979 passed, 666 skipped, 11 xfailed, 1 warning in 574.61s (0:09:34) ===
    
    
  15. modified the milestones: v3.3.0, v3.4.0 on Jun 9, 2020
  16. tacaswell commented on Jun 9, 2020

    @tacaswell
    Member

    Moved this to 3.4 as I think that the pdf / pgf failures are due to the latex changing under us.

  17. hongyi-zhao commented on Jun 9, 2020

    @hongyi-zhao
    Author

    @tacaswell Have you tested them?

  18. QuLogic commented on Jun 30, 2020

    @QuLogic
    Member

    For test_usetex, I can always reproduce that failure on Fedora 31+ (it used to be okay in 30, I think.) For the pgf tests, that failure can be reproduce on Fedora Rawhide only, because it has new texlive. Somewhere in the latex stack, the degree glyph was switched from Computer Modern to Latin Modern (which matches the rest of the text), so the line is a bit different now.

  19. modified the milestones: v3.4.0, v3.5.0 on Jan 27, 2021
  20. QuLogic commented on Jun 3, 2021

    @QuLogic
    Member

    Those remaining tests have now got updated images, because we no longer run CI on Ubuntu 16.04 that needed the old ones. So I'm closing this.

  21. changed the title [-]Matplotlib git master version fails to pass serveral pytest's tests.[/-] [+]Matplotlib git master version fails to pass several pytest's tests[/+] on Jun 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions