Repository navigation
Closed
Labels
Description
Bug summary
Pandas uses mypy/pyright to check its internal code. We currently pin matplotlib <3.8 to avoid many type errors pandas-dev/pandas#55210 Many of these errors are definitely on pandas's side :) but a few might be bugs in matplotlib.
Code for reproduction
# Simplified examples from pandas that trigger type errors
def format_date_labels(ax: Axes, rot) -> None:
for label in ax.get_xticklabels():
label.set_ha("right") # error: "Text" has no attribute "set_ha" [attr-defined]
def test(cellText: np.ndarray):
# error: Argument "cellText" to "table" has incompatible type "ndarray[Any, Any]"; expected "Sequence[Sequence[str]] | None" [arg-type]
# maybe use a Protocol such as Iterable?
matplotlib.table.table(ax, cellText=cellText **kwargs)
def decorate_axes(ax: Axes, freq, kwargs) -> None:
# The following might just be bad pandas code
# error: "Axes" has no attribute "freq" [attr-defined]
ax.freq = freq
xaxis = ax.get_xaxis()
xaxis.freq = freq
if not hasattr(ax, "legendlabels"):
ax.legendlabels = [kwargs.get("label", None)]
else:
ax.legendlabels.append(kwargs.get("label", None))
# error: Argument 1 to "FixedLocator" has incompatible type "ndarray[Any, Any]"; expected "Sequence[float]" [arg-type]
FixedLocator(ax.get_xticks())
# same also for set_ylabel, maybe use a protocol for str-coercible types instead of requiring str
def test(ax: Axes, label: Hashable): # Hashable might be too wide, but str seems to strict
ax.set_xlabel(label)
def test(ax: Axes):
axes = np.array([ax])
# error: Argument 1 to "set_ticks_props" has incompatible type "ndarray[Any, dtype[Any]]"; expected "Axes | Sequence[Axes]" [arg-type]
set_ticks_props(axes, **kwargs)Actual outcome
the above type errors
Expected outcome
Fewer errors, but still some errors - some of the pandas code is definitely to be blamed!
Additional information
No response
Operating system
No response
Matplotlib Version
3.8
Matplotlib Backend
No response
Python version
No response
Jupyter version
No response
Installation
None
Activity
tacaswell commentedon Sep 21, 2023
These should be
AxesnotAxis?twoertwein commentedon Sep 21, 2023
Thank you - that might be my copy&past mistakes. The main point is that it doesn't work with an np.ndarray that contains the appropriate type.
(Sorry, not familiar with matplotlib and how pandas uses matplotlib)
edit: updated the examples
oscargus commentedon Sep 21, 2023
This is documented as
strall the way down to theTextobject, whereset_textwill applystron whatever comes in. So this can be changed, both in the documentation and in the typing stubs.I'd suspect the same thing holds for
set_*label.The natural thing is to provide
strhere, but, as you point out, anything coercible to astrcan be used. Which then is everything, soAnyis correct, but still maybe not what we would like to encourage...ksunden commentedon Sep 21, 2023
set_hais a dynamically generated alias ofset_horizontalalignment, so type-checked code should probably use the statically defined canonical version. (Without effectively undoing the reasons for having the dynamic alias helpers, we can't type hint aliases appropriately)np.ndarraynot being compatible withSequence[Sequence[...]], maybe we can add an overload or something, but the key to when the latter is used is that ragged data structures are allowed, soArrayLikeand other nd-array type hints are insufficient themselves. Not 100% clear how to square that circle here.ax.freqis not an attribute that is present onAxesorAxisobjects by default, I don't see anywhere in the matplotlib codebase where that is added, I think that one is correct to flag as a type checker. Pandas is adding a dynamic attribute, here, I think. (same withlegendlabels, actually)FixedLocator, is partially that we do not specify the dtype on return of
get_xticks, which we probably can do, but alsonp.NDArray[np.float64], still doesn't fully qualify as aSequence, I guess, so similar to theSequence[Sequence[...]]ones, may need to overload/union type (which is not ideal for maintainability) or figure out if there is a way to enforce that NDArray is a sequence (I think zero-D arrays are what throws that for a bit of a loop)... I thought NDArray met the definition of Sequence when I was writing the type hints initially, though, so I'm slightly confused by this.Labels, I think the actual "proper" type hint might be:
(the stdlib typing module provides similar protocols for
int,float, etc, but not actuallystr, probably becauseobjectimplements__str__)Which effectively reduces to
Any(aside from things that actively remove__str__, sinceobjectdefines__str__)I will note that in the case of
set_xlabel, the docstring actually does declare the officially supported type asstr, though (as doesAxis.set_label_text), though it is eventually handed toText.set_textwhich is actually type hinted asAny/docstring defined asobject. So I was following the docstring here, but there is a reasonable argument that those could be expanded.numpy.object_numpy/numpy#24738 on top of that... but yeah, from a typing perspective the contents of the sequence is important to that method, so not sure we should just acceptNDArray[Any]... at which point, not sure we can type hintNDArrayat all at present...ksunden commentedon Sep 21, 2023
ndarrayseems to not qualify underSequencebecause it does not implementindexorcount, which are methods defined forSequence, but when used as a parent class are implemented for you.There does not seem to be another good ABC to use between
Collection(which doesn't define__getitem__) andSequence(which requiresindex/count)I know numpy is trying to reduce the number of methods rather than increase it, but perhaps there is a compelling case to be made for these... though not fully sure
indexat least can be made sensibly such that type checkers will agree that it is implemented according toSequence(though at least at runtime, no actual annotations are reported forSequencemethods, so who knows?)ksunden commentedon Sep 21, 2023
@twoertwein Can you help me to run the type checks against pandas locally?
I had tried quite a bit earlier in the process running against
pandas-stubsand it did not flag anything from mpl, so I moved on (and therefore didn't give you all as much of a warning as I did to some other projects where I did find things flagged, so sorry to have missed it, I just got a little hopeful that things just ended up working out even better than I thought)Even now, if I do
mypy pandas-stubsfrom that repo (with mpl git version), I don't see anything, and if I domypy pandasfrom the main repo (with stubs installed) I see a lot of unrelated errors, mostly forNeed type annotation, though I do additionally see some things related toAxes, though they all seem like legitimate flags for a static type checker (mostly that dynamic attributes seem to be added to them, like the examples regardingfreqabove.I also tried
mypy -m pandasto see if that changed anything, and didn't get any flags?Edit: after writing this, I noticed the
testsfolder in the pandas-stubs repo, but even running those gave nothing related to mpl (some failures due to lack of optional dependenciees and somenp.bool_vsboolbuiltin errors)Double edit: I think I got there, turns out
git pullhelps reduce the unrelated things. (I had pulled on stubs but not the main repo...) I guess I'm still a bit surprised that mypy doesn't flag anything when run on the stubs.twoertwein commentedon Sep 22, 2023
The pandas mypy/pyright checks do not use pandas-stubs. It is best to follow the instructions here to set the development environment up. After that you can simply call mypy.
Thank you for spending much time on this summary. Sorry, I didn't intend to consume your time by debugging the issues over at pandas - I'm happy to cc you for targeted questions though :)
edit: also feel free to ping me if you like me to test the impact of important/big matplotlib type changes on internal pandas code.
twoertwein commentedon Sep 22, 2023
Thank you very much for the quick and active response! I will work on pandas PRs to get the internal pandas code (more) compliant with the matplotlib annotations (might take a bit).
Sorry, I'm not sure how actionable this issue is on the matplotlib side (obviously many todos on the pandas side). Feel free to close it.
twoertwein commentedon Sep 22, 2023
About ndarray and Sequence: Sequence is a concrete class, not a protocol. Unless ndarray inherits from it, it will never match it.
ksunden commentedon Sep 22, 2023
I mean, Sequence is very much an Abstract class, but you are correct that it does require inheritance to be considered (well, runtime you can
registerit... and in factlist/tuple/etc do not actually inherit from it, though they count for mypy purposes... I had been thinking that mypy had special handling that effectively treated it as a protocol, even though I knew it wasn't actually one at runtime, but I was in fact mistaken)BvB93 commentedon Sep 25, 2023
I recall some discussions about changing this way back in the past, though nothing has come for it. If you do really want to express
ndarrayas a sequence you could consider using a slightly different definition: that of the sequence protocol as defined by the Python C API: https://docs.python.org/3/c-api/sequence.html#c.PySequence_Check. Not sure to what extent this would be practically useable in matplotlib, but this should take care of the whole "ndarray is not anabc.Sequencemember"-problem.twoertwein commentedon Sep 25, 2023
There is also a Sequence-like protocol that disallows strings hauntsaninja/useful_types#14 It could probably be adjusted to also fit ndarray.
(Pandas has the same issue: Index and Series are incompatible with Sequence.)
QuLogic commentedon Sep 17, 2025
Is there still some specific thing that should be implemented in Matplotlib here?
twoertwein commentedon Sep 18, 2025
Happy to close it