FigureCanvasTkAgg memory leak #24820

Activity

1. dstansby

   added

   Performance

   on Dec 27, 2022
2. richardsheridan commented on Jan 27, 2023

   

   richardsheridan

   on Jan 27, 2023

   Last edited by richardsheridan

   Contributor

   More actions

   This version of the tk memory leak is even more fundamental than some of the other open issues, because it doesn't involve FigureManagerTk at all.

   If we write the example in a more conventional tkinter style:

   Longish code

   import itertools
   import tkinter as tk
   from tkinter import ttk
   
   from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
   from matplotlib.figure import Figure
   import numpy as np
   
   counter = itertools.count()
   
   
   def DisplayGraph(
       passed_frame, x_limit=(), delta_x=0.0, y_input="", figsize_x=6, figsize_y=5
   ):
       fig = Figure(figsize=(figsize_x, figsize_y), dpi=100)
       ax = fig.add_subplot(111)
       ax.grid()
       x = np.arange(x_limit[0], x_limit[1], delta_x)
       y = eval(y_input)
       ax.plot(x, y)
       canvas_mat = FigureCanvasTkAgg(fig, master=passed_frame)
       canvas_mat.get_tk_widget().pack(anchor="center", expand=True, padx=2, pady=2)
       canvas_mat.count = counter.__next__()
       # canvas_mat.draw()  #  redundant!
   
   
   def update_graph():
       for widget in bottom_frame.winfo_children():
           widget.destroy()
       DisplayGraph(bottom_frame, x_limit=(-10, 10), delta_x=0.1, y_input="2*x+2")
       root.after_idle(report)
   
   
   def report():
       import gc, psutil, objgraph
   
       print(gc.collect())
   
       def filt(o):
           return isinstance(o, FigureCanvasTkAgg)
   
       objgraph.show_growth(filter=filt)
       objgraph.show_backrefs(
           [x for x in gc.get_objects() if filt(x) and not x.count],
           max_depth=10,
           filename="stuff.png",
       )
       print(psutil.Process().memory_full_info().uss)
   
   
   root = tk.Tk()
   root.geometry("500x500")
   top_frame = tk.Frame(root)
   ttk.Button(top_frame, text="Update graph", command=update_graph).pack()
   top_frame.pack()
   bottom_frame = tk.Frame(root)
   DisplayGraph(bottom_frame, x_limit=(-10, 10), delta_x=0.1, y_input="2*x+2")
   bottom_frame.pack()
   
   root.mainloop()

   We can clearly see that there are no references to the figure or canvas left after leaving the `DisplayGraph` function, so you'd think all matplotlib components would be gone before tkinter has a chance to render them. Yet, the interactive updates of the resize callback are still working, so they have not been garbage collected. We can see that instances of `FigureCanvasTkAgg` are surviving the garbage collection and growing by one instance each cycle, which just about accounts for the size of the memory leak.

   It's no secret that there is a reference cycle here, namely fig.canvas <-> canvas.figure, but the confusing thing is how they are surviving gc.collect. Objgraph (isolating the first-ever FigureCanvasTkAgg) points to two culprits. First, the canvas is referenced in the MouseEvent sitting in LocationEvent.lastevent, which is a long-lived, class-attribute reference in matplotlib.backend_bases:

   Big object graph image

   (likely better viewed in its own window. Look for the red arrow!)
   

   It would be better if this reference didn't live so long, but we can overwrite it by clicking somewhere in the plot before hitting the update button again. This cleans up the graph and reveals the only other things holding FigureCanvasTkAgg alive:

   Less big object graph image

   

   From the perspective of the python object graph, these are "leaks" since they have no referents. but what it actually means in this specific case is that the references are "owned" by the tkinter c code.

   Fortunately, we can see which methods are referenced by the leaks specifically, directing us to the culprit binds:

   matplotlib/lib/matplotlib/backends/_backend_tk.py

   Lines 201 to 208 in 84cc898

   	root.bind("<MouseWheel>", self.scroll_event_windows, "+")
   	# Can't get destroy events by binding to _tkcanvas. Therefore, bind
   	# to the window and filter.
   	def filter_destroy(event):
   	if event.widget is self._tkcanvas:
   	CloseEvent("close_event", self)._process()
   	root.bind("<Destroy>", filter_destroy, "+")

   These are not cleaned up by the widget.destroy() logic because they are bound to the toplevel window rather than the canvas. If you comment them out, the leaks shrink by 99%.

   But at this point, I'm stuck, because we sorta need those?! and it's not obvious how we could sneak an unbind into the logic, at least not without having users make an extra method call they never had to make before.

   CC @tacaswell @anntzer @oscargus
3. tacaswell commented on Jan 27, 2023

   

   tacaswell

   on Jan 27, 2023

   Member

   More actions

   Can we use a weakref in the callbacks rather than self?
4. richardsheridan commented on Jan 27, 2023

   

   richardsheridan

   on Jan 27, 2023

   Contributor

   More actions

   This works but it's a bit ugly? weakref.proxy gave various errors. is there a better way?

           root = self._tkcanvas.winfo_toplevel()
           weakself = weakref.ref(self)
           def scroll_event_windows(event):
               self = weakself()
               if self is None:
                   return
               return self.scroll_event_windows(event)
           root.bind("<MouseWheel>", scroll_event_windows, "+")
   
           # Can't get destroy events by binding to _tkcanvas. Therefore, bind
           # to the window and filter.
           def filter_destroy(event):
               self = weakself()
               if self is None:
                   return
               if event.widget is self._tkcanvas:
                   CloseEvent("close_event", self)._process()
           root.bind("<Destroy>", filter_destroy, "+")

   Also it would be good to perform an unbind rather than just short-circuiting the existing bind function.
5. tacaswell commented on Jan 27, 2023

   

   tacaswell

   on Jan 27, 2023

   Member

   More actions

   un-bind as part of the short-circuit so we will run those at most once?
6. ksunden

   closed this as completedin #25097on Feb 7, 2023
7. QuLogic

   added this to the v3.7.0 milestone on Feb 21, 2023