Repository navigation
Closed
Labels
Release criticalFor bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.topic: units and array ducktypes
Milestone
Description
Bug summary
Discovered this in unyt's daily CI jobs: https://github.com/yt-project/unyt/actions
Code for reproduction
import matplotlib.pyplot as plt
import unyt as un
un.matplotlib_support.enable()
fig, ax = plt.subplots()
x = un.unyt_array([1, 2], "cm")
y = un.unyt_array([3, 4], "kg")
yerr = un.unyt_array([0.1, 0.2], "kg")
ax.errorbar(x, y, yerr=(yerr, yerr))Actual outcome
UnitOperationError: The <ufunc 'subtract'> operator for unyt_arrays with units "kg"
(dimensions "(mass)") and "dimensionless" (dimensions "1") is not well defined.
detailed raceback
╭──────────────────────────── Traceback (most recent call last) ────────────────────────────╮
│ │
│ /private/tmp/matplotlib/t.py:10 in <module> │
│ │
│ 7 x = un.unyt_array([1, 2], "cm") │
│ 8 y = un.unyt_array([3, 4], "kg") │
│ 9 yerr = un.unyt_array([0.1, 0.2], "kg") │
│ ❱ 10 ax.errorbar(x, y, yerr=yerr) │
│ 11 │
│ /private/tmp/matplotlib/lib/matplotlib/__init__.py:1348 in inner │
│ │
│ 1345 │ @functools.wraps(func) │
│ 1346 │ def inner(ax, *args, data=None, **kwargs): │
│ 1347 │ │ if data is None: │
│ ❱ 1348 │ │ │ return func(ax, *map(sanitize_sequence, args), **kwargs) │
│ 1349 │ │ │
│ 1350 │ │ bound = new_sig.bind(ax, *args, **kwargs) │
│ 1351 │ │ auto_label = (bound.arguments.get(label_namer) │
│ │
│ /private/tmp/matplotlib/lib/matplotlib/axes/_axes.py:3474 in errorbar │
│ │
│ 3471 │ │ │ │ │ │ xo, yo, marker='|', **eb_cap_style)) │
│ 3472 │ │ │
│ 3473 │ │ if yerr is not None: │
│ ❱ 3474 │ │ │ lower, upper = extract_err('y', yerr, y, lolims, uplims) │
│ 3475 │ │ │ barcols.append(self.vlines( │
│ 3476 │ │ │ │ *apply_mask([x, lower, upper], everymask), **eb_lines_style)) │
│ 3477 │ │ │ # select points without upper/lower limits in y and │
│ │
│ /private/tmp/matplotlib/lib/matplotlib/axes/_axes.py:3434 in extract_err │
│ │
│ 3431 │ │ │ │ │ f"'{name}err' (shape: {np.shape(err)}) must be a scalar " │
│ 3432 │ │ │ │ │ f"or a 1D or (2, n) array-like whose shape matches " │
│ 3433 │ │ │ │ │ f"'{name}' (shape: {np.shape(data)})") from None │
│ ❱ 3434 │ │ │ return data - low * ~lolims, data + high * ~uplims # low, high │
│ 3435 │ │ │
│ 3436 │ │ if xerr is not None: │
│ 3437 │ │ │ left, right = extract_err('x', xerr, x, xlolims, xuplims) │
│ │
│ /Users/robcleme/.pyenv/versions/3.9.7/envs/tmp_unit/lib/python3.9/site-packages/unyt/arra │
│ y.py:1764 in __array_ufunc__ │
│ │
│ 1761 │ │ │ │ │ │ │ │ else: │
│ 1762 │ │ │ │ │ │ │ │ │ raise UnitOperationError(ufunc, u0, u1) │
│ 1763 │ │ │ │ │ │ else: │
│ ❱ 1764 │ │ │ │ │ │ │ raise UnitOperationError(ufunc, u0, u1) │
│ 1765 │ │ │ │ │ conv, offset = u1.get_conversion_factor(u0, inp1.dtype) │
│ 1766 │ │ │ │ │ new_dtype = np.dtype("f" + str(inp1.dtype.itemsize)) │
│ 1767 │ │ │ │ │ conv = new_dtype.type(conv) │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
UnitOperationError: The <ufunc 'subtract'> operator for unyt_arrays with units "kg"
(dimensions "(mass)") and "dimensionless" (dimensions "1") is not well defined.
Expected outcome
This is the output figure with matplotlib 3.4.3

Additional information
The regression bisects to #19526 (8cd22b4)
I note that if replace the last line with
ax.errorbar(x, y, yerr=yerr)
(passing a single array instead of a tuple), it doesn't break on the main branch.
Operating system
Unbuntu
Matplotlib Version
3.5.0
Matplotlib Backend
not relevant (I get an identical error with the 'agg' backend)
Python version
tested on 3.7 to 3.9
Jupyter version
No response
Installation
pip
Metadata
Metadata
Assignees
Labels
Release criticalFor bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.topic: units and array ducktypes
Activity
neutrinoceros commentedon Nov 18, 2021
(updated because I saw that that my script was overly simplified to a point that it was actually not broken on the main branch)
neutrinoceros commentedon Nov 18, 2021
I think I got a fix, will open a PR shortly !
neutrinoceros commentedon Nov 18, 2021
After digging a bit I'm actually stuck on this comment:
matplotlib/lib/matplotlib/axes/_axes.py
Line 3281 in 552e078
It seems crucial to understand why this assumption was made in matplotlib, and wether it complies with NEP 18 (array_function protocol).
It is entirely possible at this stage that the actual issue is on the unyt side, because it's not up to date with NEP 18.
Also note that unyt's integration is still considered experimental (to the best of my knowledge).
anntzer commentedon Nov 18, 2021
I think(??) I got this belief from pint, but not sure either. OTOH, is there any thing we can do to convert unitful arraylikes to actual arrays that 1) works across unit libs and 2) does not destroy units?
Possibly, it may make sense for numpy to rule on an expected behavior here...
ngoldbaum commentedon Nov 18, 2021
If you’re handed an
ndarraysubclass, I think casting to a base ndarray would work across libraries? Pint doesn’t implement anndarraysubclass so expecting all unit libraries to work like it does may not be a good expectation. Both unyt and astropy implementndarraysubclasses.It might also help to have mock unit libraries (or even directly testing extant unit libraries) in matplotlib’s test suite as this kind of change seems to happen relatively often and you’re not depending on downstream libraries to alert you to breakage.
anntzer commentedon Nov 18, 2021
The case pointed out by @neutrinoceros is already (should?) only done when the input is not a ndarray or subclass thereof.
neutrinoceros commentedon Nov 18, 2021
Correct, but the way it's implemented currently behaves weirdly when the input is a tuple of arrays
anntzer commentedon Nov 18, 2021
Ah, fair enough.
jklymak commentedon Nov 18, 2021
We do have a mock unit library. If downstream libraries want to add new ones (and tests) I'm sure we would entertain that. I'm not sure we will accept testing every possible downstream library that may have decided to add units. OTOH we should specify exactly what we will accept in terms of units so downstream libraries have consistent rules. This has been a problem for years, and no one has fixed it, so we play whack-a-mole. Hopefully there is light at the end of the tunnel with the upcoming NASA grant perhaps supporting a specification and interface overhaul. Please
neutrinoceros commentedon Nov 18, 2021
Well unyt isn't new to this game, I think it's maybe the third biggest player after pint and astropy, but clearly we've missed an opportunity to catch this much earlier by testing only against final releases, so I'll try to fix this on our end to try and improve the situation for everyone in the future.
ngoldbaum commentedon Nov 18, 2021
Ah then I guess a concrete improvement would be a test that directly exercises the tuple of unit arrays case for errorbar. Sorry for the off-the-cuff peanut gallery advice.
anntzer commentedon Nov 18, 2021
Going back to the specific problem here: is there a way to cast a 2-tuple of unyt arrays into a (2, N) unyt array? (... that is not unyt-specific...)
ngoldbaum commentedon Nov 18, 2021
@anntzer, perhaps something like this?
anntzer commentedon Nov 18, 2021
Ah, thanks for the suggestion.
neutrinoceros commentedon Nov 18, 2021
Note that this doesn't work with two unyt arrays unless they have the exact same unit (not just coercible ones), but other than that this was also my best guess, and it's probably good enough indeed.
neutrinoceros commentedon Nov 19, 2021
Update: @ngoldbaum has a working patch for the problem I mentioned in my previous reply here yt-project/unyt#211, so now I have no reserves left to express on this approach
jklymak commentedon Nov 19, 2021
Do we need the errors to be a 2, N array, versus two N-arrays? I assume the unyt 1-D arrays cast, so isn't the solution from our side to just have
errloanderrhi?anntzer commentedon Nov 19, 2021
Technically, the docs state that
yerrcan be a(2, N)array-like which I think means "can be cast to a (2, N) array" (and it should be possible to cast a tuple[1d_unyt_array, 1d_unyt_array] into that). Trying to unpack immediately into errlo, errhi is probably also possible, but just requires much more wrangling than a straightforwardnp.broadcast_to(err, (2, N))...FIX: be more careful about coercing unit-full containers to ndarray
FIX: be more careful about coercing unit-full containers to ndarray