You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Cannot plot fully masked array against datetimes #11337
Bug summary
Cannot plot a fully masked array agaisnt an array of datetimes. A ValueError is raised for a seemingly unrelated reason. Surprisingly, matplotlib has no problem plotting a fully masked array by itself.
[super big traceback, only showing the most recent to save space...]
/usr/lib64/python3.5/dist-packages/matplotlib/dates.py in _from_ordinalf(x, tz)
258
259 ix = int(x)
--> 260 dt = datetime.datetime.fromordinal(ix).replace(tzinfo=UTC)
261
262 remainder = float(x) - ix
ValueError: ordinal must be >= 1
Expected outcome
An empty graph. If you comment out x and don't plot against it, then matplotlib has no problem displaying an empty graph. It is only when you plot against a datetime array this is an issue.
The problem seems to be that a fully masked array is "plotted" around (0, 0) independent of the x values and this doesn't work with datetimes because they should always be larger than 1.
I get the following value error in 2.2.2 that explains more. ValueError: view limit minimum -0.001 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime units
In the second case, even though the artist has no extent on the axes, the axes limits are changed. I would argue that this shouldn't be the case - and if fixed, would also solve this issue here automagically.
I think @ImportanceOfBeingErnest diagnosed this right. The bug is someplace in the auto-scale logic for Line2D objects with no valid data defaulting to min/max of 0 (which is then getting expanded to +/- 0.05) rather than being skipped.
I just checked that the array was not completely masked with an if
statement before I plotted the data. If it was fully masked, it just didn't
plot the data and displayed a small graphic saying the data was fully
masked for that particular graph.
I think that path does the "right" thing, and doesn't update the view limits based on points that won't plot.
I think its possible to argue that we should be able to handle num2date(0) better than we do. Right now we use datetime.datetime.toordinal(0), and that fails because datetime ordinals cannot be less than 0. However, numpy.datetime64 has no such restriction. I don't think its ridiculously hard to rewrite dates.py to make the internal representation of dates datetime64 objects. However, there may b pitfalls. I know @efiring and @pganssle have opinions on date times - maybe they have some thoughts here. But I don't see fundamentally why matplotlib should not be able to plot dates before 0000.
But I don't see fundamentally why matplotlib should not be able to plot dates before 0000.
I can think of many reasons why you'd want to plot very old dates, but it's pretty unusual to want to represent them in a proleptic Gregorian calendar (and even for those when you do want that, it's pretty rare to care about the time portion of it). Most of the functionality you gain by using a datetime is essentially useless or misleading for very old datetimes, but there's no special transition at 1 AD, as you go back in time the datetime class just sort of progressively decays into something of dubious value because it doesn't match contemporary timekeeping systems. I think the main reason datetime itself drops support there is that it's a natural point to drop support so that you can make assumptions about the sign of the components.
So what I will say is that support for out-of-bounds datetime objects should probably be as tightly scoped as possible. There are a number of bugs reported to dateutil and other projects relating to old datetimes, and I don't think I've ever seen an example of one where someone had a specific use case for these things. Most of the bugs come from people using hypothesis in their test suite or situations like this, where some default value doesn't totally fit in with the way datetime is intended to be used. I'd be willing to change my mind if I could identify a set of users who have an actual use for very old proleptic Gregorian datetimes.
In this particular situation, I would want to avoid a solution that too-eagerly converts from datetime to some ordinal number, as I was concerned about in (I think) #12863. I haven't looked at the relevant code paths, but I think maybe either changing the default value to 1970-01-01T00:00:00 or a somewhat more complicated mechanism that uses ordinal dates only when a datetime value can't be represented as datetime would be the right way to go here.
The problem here is the converse - the default limits are 0-1 but the user supplied datetime to the x-axis so the converter is datetime, and then when ticking, it tries to convert 0 to a datetime and that fails.
I guess the correct fix is when the axis first gets registered as datetime its default limits get changed to something other than 0-1.
Activity
logdog commentedon May 29, 2018
Here is the full traceback if it is needed:
fredrik-1 commentedon May 30, 2018
The problem seems to be that a fully masked array is "plotted" around (0, 0) independent of the x values and this doesn't work with datetimes because they should always be larger than 1.
I get the following value error in 2.2.2 that explains more.
ValueError: view limit minimum -0.001 is less than 1 and is an invalid Matplotlib date value. This often happens if you pass a non-datetime value to an axis that has datetime unitsImportanceOfBeingErnest commentedon May 30, 2018
I think the problem is more that an artist without any visual representation is attempted to be drawn.
I would say this tracks down to the question of why the two below codes give different xlimits:
In the second case, even though the artist has no extent on the axes, the axes limits are changed. I would argue that this shouldn't be the case - and if fixed, would also solve this issue here automagically.
fredrik-1 commentedon May 30, 2018
I believe that is consistent with my thoughts. plt.plot([]) sets axis around 0 and that does not work with a datetime x axis.
I also saw that loglog with fully masked data worked and that the limits where not around zero in that case.
tacaswell commentedon May 30, 2018
I think @ImportanceOfBeingErnest diagnosed this right. The bug is someplace in the auto-scale logic for
Line2Dobjects with no valid data defaulting to min/max of 0 (which is then getting expanded to +/- 0.05) rather than being skipped.logdog commentedon Oct 3, 2018
jonnyhtw commentedon Oct 12, 2018
I'm also seeing this error using this conda environment...
jklymak commentedon Mar 4, 2019
Note this will still error if xmin=0. xmin>=1 for
num2dateto work.jklymak commentedon Mar 4, 2019
I think that path does the "right" thing, and doesn't update the view limits based on points that won't plot.
I think its possible to argue that we should be able to handle
num2date(0)better than we do. Right now we usedatetime.datetime.toordinal(0), and that fails because datetime ordinals cannot be less than 0. However,numpy.datetime64has no such restriction. I don't think its ridiculously hard to rewritedates.pyto make the internal representation of datesdatetime64objects. However, there may b pitfalls. I know @efiring and @pganssle have opinions on date times - maybe they have some thoughts here. But I don't see fundamentally why matplotlib should not be able to plot dates before 0000.pganssle commentedon Mar 4, 2019
I can think of many reasons why you'd want to plot very old dates, but it's pretty unusual to want to represent them in a proleptic Gregorian calendar (and even for those when you do want that, it's pretty rare to care about the time portion of it). Most of the functionality you gain by using a
datetimeis essentially useless or misleading for very old datetimes, but there's no special transition at 1 AD, as you go back in time thedatetimeclass just sort of progressively decays into something of dubious value because it doesn't match contemporary timekeeping systems. I think the main reasondatetimeitself drops support there is that it's a natural point to drop support so that you can make assumptions about the sign of the components.So what I will say is that support for out-of-bounds
datetimeobjects should probably be as tightly scoped as possible. There are a number of bugs reported todateutiland other projects relating to old datetimes, and I don't think I've ever seen an example of one where someone had a specific use case for these things. Most of the bugs come from people usinghypothesisin their test suite or situations like this, where some default value doesn't totally fit in with the waydatetimeis intended to be used. I'd be willing to change my mind if I could identify a set of users who have an actual use for very old proleptic Gregorian datetimes.In this particular situation, I would want to avoid a solution that too-eagerly converts from
datetimeto some ordinal number, as I was concerned about in (I think) #12863. I haven't looked at the relevant code paths, but I think maybe either changing the default value to1970-01-01T00:00:00or a somewhat more complicated mechanism that uses ordinal dates only when a datetime value can't be represented asdatetimewould be the right way to go here.jklymak commentedon Mar 4, 2019
The problem here is the converse - the default limits are 0-1 but the user supplied datetime to the x-axis so the converter is datetime, and then when ticking, it tries to convert 0 to a datetime and that fails.
I guess the correct fix is when the axis first gets registered as datetime its default limits get changed to something other than 0-1.
anntzer commentedon Mar 4, 2019
Hummm... possibly #11337 (comment) is related to #13463 (the comment mentioning that dates Locators likely need view_limits)?
jklymak commentedon Mar 4, 2019
Yeah I was just looking at it... The default view interval gets set by
dates, but not used in the ticking.