Skip to content

[MNT]: macos-latest memory leak over threshold #22959

Closed

Description

@andrew-fennell

Summary

macos-latest test is failing. Memory leakage is slightly over the threshold.

Failing test:
lib/matplotlib/tests/test_backends_interactive.py:559: AssertionError

Exact assertion error:
assert growth <= acceptable_memory_leakage
assert 30457856 <= 30000000

Example of test failing: macos-latest failing

Proposed fix

Raise threshold.

Activity

  1. andrew-fennell commented on May 2, 2022

    @andrew-fennell
    ContributorAuthor

    Threshold can be raised by 1000000 (double 457856) to account for fluctuation.

  2. added this to the v3.5.2 milestone on May 2, 2022
  3. andrew-fennell commented on May 5, 2022

    @andrew-fennell
    ContributorAuthor

    @oscargus I think my original solution (#22961) didn't cover the whole problem. The original examples of this test failing was specifically with macosx and the threshold was less than 1000000 off.

    The test that is failing:

    @pytest.mark.parametrize("env", _get_testable_interactive_backends())
    @pytest.mark.parametrize("time_mem", [(0.0, 2_000_000), (0.1, 30_000_000)])
    def test_figure_leak_20490(env, time_mem):
    pytest.importorskip("psutil", reason="psutil needed to run this test")
    # We haven't yet directly identified the leaks so test with a memory growth
    # threshold.
    pause_time, acceptable_memory_leakage = time_mem
    if env["MPLBACKEND"] == "macosx":
    acceptable_memory_leakage += 11_000_000
    result = _run_helper(
    _test_figure_leak, str(pause_time), timeout=_test_timeout, **env
    )

    Here is an example of this tests still failing: https://github.com/matplotlib/matplotlib/runs/6246645151?check_suite_focus=true

    Should I make this adjustment?
    @pytest.mark.parametrize("time_mem", [(0.0, 2_000_000), (0.1, 35_000_000)])

  4. andrew-fennell commented on May 5, 2022

    @andrew-fennell
    ContributorAuthor
  5. tacaswell commented on May 6, 2022

    @tacaswell
    Member

    Looking at this in detail, I was wrong about how the tests are run. They are in their own process so the inter-test interaction should not be a problem. It seems there is just a lot of variation, but we do not know why.

    I think that in addition to the OSX backend on osx, we also need to increase the threshold when running with tkagg on osx. I'm going to open a PR with that stage and also write up an issue starting the discussion about how we may need to find a more stable way to benchmark these memory issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions