Repository navigation
matplotlib 3.5 breaks yt #20520
Description
Bug report
Bug summary
I found a regression in yt (https://github.com/yt-project/yt) by running our CI against matplotlib's master branch.
Code for reproduction
Here's a "minimal" script using yt's api
from yt.testing import fake_sph_grid_ds
from yt.visualization.plot_window import ProjectionPlot
ds = fake_sph_grid_ds()
plot = ProjectionPlot(ds, "z", ("gas", "density"))
plot.save()I apologise for not being able to provide a pure mpl minimal example at the moment. The data flow is unfortunately very hard to keep track of on the yt side, but I'm working on it.
I think a better approach might be to go through the diff from the breaking commit and attempt a patch from there. I will attempt this and report here.
Actual outcome
Here's the traceback (using rich)
yt : [INFO ] 2021-06-25 17:00:25,961 Parameters: current_time = 0.0
yt : [INFO ] 2021-06-25 17:00:25,961 Parameters: domain_dimensions = [1 1 1]
yt : [INFO ] 2021-06-25 17:00:25,961 Parameters: domain_left_edge = [0. 0. 0.]
yt : [INFO ] 2021-06-25 17:00:25,961 Parameters: domain_right_edge = [3. 3. 3.]
yt : [INFO ] 2021-06-25 17:00:25,962 Parameters: cosmological_simulation = 0
yt : [INFO ] 2021-06-25 17:00:25,962 Allocating for 2.700e+01 particles
yt : [INFO ] 2021-06-25 17:00:28,828 xlim = 0.000000 3.000000
yt : [INFO ] 2021-06-25 17:00:28,828 ylim = 0.000000 3.000000
yt : [INFO ] 2021-06-25 17:00:28,828 xlim = 0.000000 3.000000
yt : [INFO ] 2021-06-25 17:00:28,828 ylim = 0.000000 3.000000
yt : [INFO ] 2021-06-25 17:00:28,836 Making a fixed resolution buffer of (('gas', 'density')) 800 by 800
> /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/plot_window.py(1072)_setup_plots()
1070 #balise 3
1071 # THE ISSUE IS HERE
-> 1072 self.plots = WindowPlotMPL(
1073 ia,
1074 self._field_transform.name,
(Pdbr) c
> /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/base_plot_types.py(238)_init_image()
236 # tuple colormaps are from palettable (or brewer2mpl)
237 breakpoint()
--> 238 if isinstance(cmap, tuple):
239 cmap = get_brewer_cmap(cmap)
240
(Pdbr) c
> /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/plot_window.py(1072)_setup_plots()
1070 #balise 3
1071 # THE ISSUE IS HERE
-> 1072 self.plots = WindowPlotMPL(
1073 ia,
1074 self._field_transform.name,
(Pdbr) c
> /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/base_plot_types.py(238)_init_image()
236 # tuple colormaps are from palettable (or brewer2mpl)
237 breakpoint()
--> 238 if isinstance(cmap, tuple):
239 cmap = get_brewer_cmap(cmap)
240
(Pdbr) c
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /Users/robcleme/dev/yt-project/yt-py310/repro_mpl_bug.py:49 in <module> │
│ │
│ 46 │ return 0 │
│ 47 │
│ 48 if __name__ == "__main__": │
│ ❱ 49 │ sys.exit(main()) │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/repro_mpl_bug.py:25 in main │
│ │
│ 22 │ │
│ 23 │ ds = fake_sph_grid_ds() │
│ 24 │ plot = ProjectionPlot(ds, "z", ("gas", "density")) │
│ ❱ 25 │ plot.save() │
│ 26 │ return 0 │
│ 27 │
│ 28 def main2() -> int: │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/plot_container.py:106 in newfunc │
│ │
│ 103 │ │ if not args[0]._plot_valid: │
│ 104 │ │ │ # it is the responsibility of _setup_plots to │
│ 105 │ │ │ # call args[0].run_callbacks() │
│ ❱ 106 │ │ │ args[0]._setup_plots() │
│ 107 │ │ rv = f(*args, **kwargs) │
│ 108 │ │ return rv │
│ 109 │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/plot_window.py:1072 in _setup_plots │
│ │
│ 1069 │ │ │ breakpoint() │
│ 1070 │ │ │ #balise 3 │
│ 1071 │ │ │ # THE ISSUE IS HERE │
│ ❱ 1072 │ │ │ self.plots[f] = WindowPlotMPL( │
│ 1073 │ │ │ │ ia, │
│ 1074 │ │ │ │ self._field_transform[f].name, │
│ 1075 │ │ │ │ self._field_transform[f].func, │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/plot_window.py:2268 in __init__ │
│ │
│ 2265 │ │ │
│ 2266 │ │ super().__init__(size, axrect, caxrect, zlim, figure, axes, cax) │
│ 2267 │ │ │
│ ❱ 2268 │ │ self._init_image(data, cbname, cblinthresh, cmap, extent, aspect) │
│ 2269 │ │ │
│ 2270 │ │ # In matplotlib 2.1 and newer we'll be able to do this using │
│ 2271 │ │ # self.image.axes.ticklabel_format │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/yt/visualization/base_plot_types.py:327 in _init_image │
│ │
│ 324 │ │ else: │
│ 325 │ │ │ # this is where it breaks (but only on second pass) │
│ 326 │ │ │ #breakpoint() │
│ ❱ 327 │ │ │ self.cb = self.figure.colorbar(self.image, self.cax) │
│ 328 │ │ for which in ["major", "minor"]: │
│ 329 │ │ │ self.cax.tick_params(which=which, axis="y", direction="in") │
│ 330 │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/matplotlib/lib/matplotlib/figure.py:1156 in colorbar │
│ │
│ 1153 │ │ │ │ │ │ │ 'panchor'] │
│ 1154 │ │ cb_kw = {k: v for k, v in kw.items() if k not in NON_COLORBAR_KEYS} │
│ 1155 │ │ │
│ ❱ 1156 │ │ cb = cbar.Colorbar(cax, mappable, **cb_kw) │
│ 1157 │ │ │
│ 1158 │ │ if not kw['userax']: │
│ 1159 │ │ │ self.sca(current_ax) │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/matplotlib/lib/matplotlib/colorbar.py:1196 in __init__ │
│ │
│ 1193 │ │ │ │ kwargs.setdefault('extend', mappable.cmap.colorbar_extend) │
│ 1194 │ │ │ if isinstance(mappable, martist.Artist): │
│ 1195 │ │ │ │ _add_disjoint_kwargs(kwargs, alpha=mappable.get_alpha()) │
│ ❱ 1196 │ │ │ super().__init__(ax, **kwargs) │
│ 1197 │ │ │
│ 1198 │ │ mappable.colorbar = self │
│ 1199 │ │ mappable.colorbar_cid = mappable.callbacksSM.connect( │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/matplotlib/lib/matplotlib/_api/deprecation.py:489 in │
│ wrapper │
│ │
│ 486 │ │ │ │ "positionally is deprecated since Matplotlib %(since)s; the " │
│ 487 │ │ │ │ "parameter will become keyword-only %(removal)s.", │
│ 488 │ │ │ │ name=name, obj_type=f"parameter of {func.__name__}()") │
│ ❱ 489 │ │ return func(*args, **kwargs) │
│ 490 │ │
│ 491 │ return wrapper │
│ 492 │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/matplotlib/lib/matplotlib/colorbar.py:421 in __init__ │
│ │
│ 418 │ │ │ ['uniform', 'proportional'], spacing=spacing) │
│ 419 │ │ │
│ 420 │ │ # wrap the axes so that it can be positioned as an inset axes: │
│ ❱ 421 │ │ ax = ColorbarAxes(ax, userax=userax) │
│ 422 │ │ self.ax = ax │
│ 423 │ │ ax.set(navigate=False) │
│ 424 │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/matplotlib/lib/matplotlib/colorbar.py:260 in __init__ │
│ │
│ 257 │ │ │ │ parent._axes.add_child_axes(outer_ax) │
│ 258 │ │ │ │ outer_ax._axes.child_axes.remove(parent) │
│ 259 │ │ │ else: │
│ ❱ 260 │ │ │ │ parent.remove() │
│ 261 │ │ else: │
│ 262 │ │ │ outer_ax = parent │
│ 263 │
│ │
│ /Users/robcleme/dev/yt-project/yt-py310/matplotlib/lib/matplotlib/artist.py:163 in remove │
│ │
│ 160 │ │ # protected attribute if Python supported that sort of thing. The │
│ 161 │ │ # callback has one parameter, which is the child to be removed. │
│ 162 │ │ if self._remove_method is not None: │
│ ❱ 163 │ │ │ self._remove_method(self) │
│ 164 │ │ │ # clear stale callback │
│ 165 │ │ │ self.stale_callback = None │
│ 166 │ │ │ _ax_flag = False │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: list.remove(x): x not in list
A CI log with multiple such failures can be found here https://github.com/yt-project/yt/runs/2899209479?check_suite_focus=true
Expected outcome
The script is supposed to output a file named "ParticleData_Projection_z_density.png".
It works fine with matplotlib 3.4.0, and I was able to identify (git bisect) the breaking commit in matplotilb: 146856b
Matplotlib version
- Operating system: OsX
- Matplotlib version (
import matplotlib; print(matplotlib.__version__)): 3.4.2.post863+g146856b03 - Matplotlib backend (
print(matplotlib.get_backend())): - Python version: I used 3.10.0b2 but I don't believe it is relevant
- Jupyter version (if applicable):
- Other libraries: yt (installed from source, main branch, commit c3054df545b51871ae22ef900a4d0e56cf0f2428) https://github.com/yt-project/yt
I installed matplotlib with
pip install git+https://github.com/matplotlib/matplotlib.git
and reinstalled on every iteration of my git bisect run.
Activity
greglucas commentedon Jun 25, 2021
Looks like that was because of the ColorbarAxes redesign. There is a proposal to change that back again in #20501. Any chance you can run yt's CI against that branch and see if that fixes things?
tacaswell commentedon Jun 25, 2021
And thank you @neutrinoceros for both running against the default branch and for reporting the regression!
neutrinoceros commentedon Jun 25, 2021
You got it.
neutrinoceros commentedon Jun 25, 2021
@greglucas I've rerun the simple example I gave in the report against #20501 and it ran smoothly. I'll try running the full test suite too just in case !
edit: not the actual full test suite, it's a little too complicated to set up, but I'll run a subset as a first line of defense :)
neutrinoceros commentedon Jun 25, 2021
I don't believe I got any other matplotlib-related failures running against #20501, and it definitely solves all of the known ones !
greglucas commentedon Jun 26, 2021
Excellent, thanks for checking on that @neutrinoceros!
jklymak commentedon Jul 8, 2021
@neutrinoceros #20501 should fix yt's colorbar issue, but please let us know if not...
neutrinoceros commentedon Jul 8, 2021
Awesome, thank you so much !
I'll keep you updated.
neutrinoceros commentedon Jul 8, 2021
I ran our CI again with the latest version of matplotlib's master branch and it's all green ✅
Thanks again @jklymak, @greglucas and @tacaswell. We really appreciate your hard work down hill !