Memory leak when plotting multiple figures with the macOS backend

Bug report

Bug summary

Hi,

I believe I have found a memory issue when plotting multiple figures with the macOS backend.

Here is a minimal working example:

import sys
import matplotlib.pyplot as plt
import numpy as np

import matplotlib
#matplotlib.use('agg')

data = np.random.rand(60, 60)

def plot_expr(plot_name):

    plt.figure(figsize=(9, 8))

    for i in range(60):
        plt.plot(data[i, :])

    plt.savefig(plot_name)

    plt.close('all')


if __name__ == "__main__":

    print(f'python: {sys.version}')
    print(f'matplotlib: {matplotlib.__version__}')
    print(f'backend: {matplotlib.get_backend()}')

    for i in range(50):
        print(f'{i}...')
        plot_expr(f'leak_{i}')

When running the code with memory-profiler, I obtain something like:

Even though I am explicitly calling the plt.close() function.

When using the 'agg' backend instead of the 'MacOSX' backend by uncommenting line 6, I get:

Which I would expect to be the normal behavior.

As I have been unable to reproduce this on Linux, I assume that the issue is with the macOS backend.

Matplotlib version

Operating system: macOS 11.2.1
Matplotlib version 3.3.4
Matplotlib backend MacOSX
Python version: 3.7.9

I installed Python via venv and matplotlib via pip.

Cheers,

Théotime

does this persist when you use Qt5Agg?
does it persist when you manually invoke garbage collection?

No, the leak does not seem to persist with Qt5Agg (with or without manual GC).
Yes, I have the issue even after adding gc.collect() after the plt.close() call.

If you throw a plt.pause(.1) into the loop does the memory get cleared? We have seen issues where if you are creating figures that in turn create GUI objects at the c/c++/objectivec level. Depending exactly how the event loops work under the hood we can have the situation:

create figure (allocate memory)
destroy the figure (do not actually destroy the GUI objects, but just tell the event loop you would like to destroy those objects)
the event loop notices your request, makes sure it is fully safe, tears down the GUI object and releases the memory

but, if you are running this in a tight loop like the example you are never letting the event loop run so you have N GUI windows hanging out waiting to be cleared.

Hey,

Nope, even with a plt.pause(1.0) thrown in I still get the same behavior.

Perhaps try using objgraph.show_growth() (https://mg.pov.lt/objgraph/#memory-leak-example) to see whether it's Python objects leaking or (Obj)C ones?

I certainly can do that.

Here's the result for the last two iterations of the loops (deltas are the same from one iteration to the next), with limit=25:

48...
dict                          87426     +1680
weakref                       47218      +922
tuple                         43318      +801
function                      50804      +799
builtin_function_or_method    37394      +735
list                          27544      +472
cell                          14544      +257
method                        12511      +254
_XYPair                        9359      +191
Path                           8938      +182
CompositeGenericTransform      8183      +167
Bbox                           7694      +156
WeakMethod                     6223      +127
Affine2D                       5733      +117
Line2D                         5733      +117
MarkerStyle                    5733      +117
IdentityTransform              4362       +89
BboxTransformTo                3822       +78
TransformedBbox                3773       +77
TransformedPath                3577       +73
FontProperties                 2205       +45
Text                           2205       +45
ScaledTranslation               637       +13
XTick                           490       +10
YTick                           441        +9

49...
dict                          89106     +1680
weakref                       48140      +922
tuple                         44119      +801
function                      51603      +799
builtin_function_or_method    38129      +735
list                          28016      +472
cell                          14801      +257
method                        12765      +254
_XYPair                        9550      +191
Path                           9120      +182
CompositeGenericTransform      8350      +167
Bbox                           7850      +156
WeakMethod                     6350      +127
Affine2D                       5850      +117
Line2D                         5850      +117
MarkerStyle                    5850      +117
IdentityTransform              4451       +89
BboxTransformTo                3900       +78
TransformedBbox                3850       +77
TransformedPath                3650       +73
FontProperties                 2250       +45
Text                           2250       +45
ScaledTranslation               650       +13
XTick                           500       +10
YTick                           450        +9

It does look like matplotlib objects to me.

I did a bit of digging, and the back reference graphs I get tracing from Figure have only one difference between Agg and macosx. FigureManagerMac has a ref count of 3, whereas FigureManagerBase a ref count of 2.

So I suspect that a reference to FigureManagerMac is hanging around somewhere in the C layer, because it's not showing up on the python reference graphs. I guess this is probably somewhere in _macosx.FigureManager.

Here's the code I'm using if anyone's interested:

import gc
import random

import matplotlib.pyplot as plt
import objgraph

import matplotlib
matplotlib.use('macosx')


def plot_expr(plot_name):

    fig = plt.figure(figsize=(9, 8))
    plt.close('all')
    objgraph.show_growth(limit=50)

    if plot_name == 'leak_2':
        objgraph.show_backrefs(
            random.choice(objgraph.by_type('Figure')),
            filename='chain_mac.png', max_depth=50, refcounts=True)


if __name__ == "__main__":
    for i in range(3):
        print(f'{i}...')
        plot_expr(f'leak_{i}')

I don't think I have the knowledge to track this down in the C layer (if that's indeed where it is), but FWIW this issue is present at least back to Matplotlib v3.2.2.

Is the reference count the same with Qt backends?

There is a reference held in the ObjC side on the Window class. It is taken in initWithContentRect and removed in dealloc. Maybe there is something related to the lifecycle issue in #21788.

Coming back to this, the growth I'm getting with the code in #19769 (comment) is

dict                           6635       +50
list                           4643       +19
ReferenceType                  3711       +13
tuple                          7059       +12
cell                           4414       +12
function                      12858       +11
_StrongRef                       27        +9
method                           81        +6
count                            22        +6
set                             701        +5
CallbackRegistry                 15        +5
WeakMethod                       15        +5
builtin_function_or_method     2827        +3
Grouper                          10        +2
_XYPair                           6        +2
FigureManagerMac                  3        +1
TimerMac                          3        +1
NavigationToolbar2Mac             3        +1
Stack                             3        +1
FigureCanvasMac                   3        +1
LockDraw                          3        +1
Figure                            3        +1
Bbox                              3        +1
Affine2D                          3        +1
TransformedBbox                   3        +1
BboxTransformTo                   3        +1
Rectangle                         3        +1
SubplotParams                     3        +1
_AxesStack                        3        +1

And the chain for the canvas is below. It looks like there might be an issue with TimerMac being in a circular reference with a __del__ defined (in red below)? Again I'm far from an expert, but might be worth someone who understands the mac code checking that TimerMac (fully implemented in the C layer) doesn't have any memory issues?

This was fixed by: #23059

Navigation Menu

Repository navigation

Memory leak when plotting multiple figures with the macOS backend #19769

Bug report

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Memory leak when plotting multiple figures with the macOS backend #19769

Description

Bug report

Activity

jklymak commented on Mar 25, 2021

tgrohens commented on Mar 25, 2021

tacaswell commented on Mar 25, 2021

tgrohens commented on Mar 26, 2021

anntzer commented on Mar 26, 2021

tgrohens commented on Mar 26, 2021

dstansby commented on Dec 21, 2021

dstansby commented on Dec 21, 2021

QuLogic commented on Dec 22, 2021

dstansby commented on Jan 27, 2023

greglucas commented on Apr 27, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions