Repository navigation
Closed
Description
Trying to plot the NaT (not a time) value of pandas gives an error. Feature request: treat NaT just like NaN from numpy.
from pylab import *
import pandas as pd
plot(pd.NaT,5)---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-3-60addc339c2f> in <module>()
----> 1 plot(pd.NaT,5)
C:\Anaconda\lib\site-packages\matplotlib\pyplot.pyc in plot(*args, **kwargs)
3091 ax.hold(hold)
3092 try:
-> 3093 ret = ax.plot(*args, **kwargs)
3094 draw_if_interactive()
3095 finally:
C:\Anaconda\lib\site-packages\matplotlib\axes\_axes.pyc in plot(self, *args, **kwargs)
1372
1373 for line in self._get_lines(*args, **kwargs):
-> 1374 self.add_line(line)
1375 lines.append(line)
1376
C:\Anaconda\lib\site-packages\matplotlib\axes\_base.pyc in add_line(self, line)
1484 line.set_clip_path(self.patch)
1485
-> 1486 self._update_line_limits(line)
1487 if not line.get_label():
1488 line.set_label('_line%d' % len(self.lines))
C:\Anaconda\lib\site-packages\matplotlib\axes\_base.pyc in _update_line_limits(self, line)
1495 Figures out the data limit of the given line, updating self.dataLim.
1496 """
-> 1497 path = line.get_path()
1498 if path.vertices.size == 0:
1499 return
C:\Anaconda\lib\site-packages\matplotlib\lines.pyc in get_path(self)
863 """
864 if self._invalidy or self._invalidx:
--> 865 self.recache()
866 return self._path
867
C:\Anaconda\lib\site-packages\matplotlib\lines.pyc in recache(self, always)
564 x = ma.asarray(xconv, np.float_)
565 else:
--> 566 x = np.asarray(xconv, np.float_)
567 x = x.ravel()
568 else:
C:\Anaconda\lib\site-packages\numpy\core\numeric.pyc in asarray(a, dtype, order)
460
461 """
--> 462 return array(a, dtype, copy=False, order=order)
463
464 def asanyarray(a, dtype=None, order=None):
TypeError: float() argument must be a string or a number
Calling plot with a np.nan results in the point being silently ignored, which I believe should also happen for pd.NaT.
plot(np.nan,5)
Activity
WeatherGod commentedon Jan 14, 2015
how does pandas implement NaT? We don't have a dependency upon pandas, so
we can't import it directly to see if they are the same.
On Wed, Jan 14, 2015 at 11:27 AM, jvspl notifications@github.com wrote:
jvspl commentedon Jan 14, 2015
How pandas implements NaT is beyond my knowledge. I cross-posted this issue on the pandas tracker as pandas-dev/pandas#9253
tacaswell commentedon Jan 14, 2015
internally we convert input to float (because that makes sense for a plotting library) and the code is blowing up because
float(pd.NaT)blows up.Surprisingly
int(pd.NaT)does work, but return a very negative number which is probably more surprising than the code blowing up, so adding atry...exceptblock to fall back to converting tointwould be worse.The ideal thing would be for
float(pd.NaT)to returnnp.nan, but I am not sure if that is possible.pandasimportsmatplotlibso we can not import pandas. The real solution to a lot of these problems is that all of the plotting should be pulled out of pandas and then either pandas becomes one of our dependencies or there is a third project which is a pandas-aware extension of mpl.In the mean time, I would suggest using pandas to either drop your
NaTrows or replace them withnp.nan.@jreback who should this get this pointed at on your side?
jreback commentedon Jan 15, 2015
pd.NaTis a sub-class ofdatetime.datetime, that is treated like a missing value (it has similar properties asnp.datetime64('NaT'). It isnot-a-time. It is implemented similarly tonp.datetime64('NaT')in that it is defined asnp.iinfo(np.int64).min. This was a work-around a long time ago by @wesm to add missing value support to datetimes.It does not make sense to treat it as anything other than a datetime-like (which are represented by int64's under the hood).
You can simply treat it as a
datetime.datetimeshoyer commentedon Jan 15, 2015
@tacaswell It's not super-clean, but it definitely is an option to make pandas an optional dependency of matplotlib, with
import pandasnever found at the top level of any modules but rather at the top of functions. In fact, this is exactly what pandas does (in reverse) with matplotlib.efiring commentedon Jan 15, 2015
@shoyer, I don't understand why we would want to do that. In which matplotlib functions would you bury this dependency?
I wonder whether the problem of pandas inputs could be handled via our units support; could pandas classes be optionally added to the units.registry, just as datetime.date and datetime.datetime are?
shoyer commentedon Jan 15, 2015
@efiring I did not mean to suggest that that was a good idea, merely that it was possible! It does look like the matplotlib units library would be reasonable way to handle this -- probably worth a try.
pelson commentedon Feb 3, 2015
Does the NaT not just need registering at https://github.com/pydata/pandas/blob/722fc778654f36092eccf9c0c654b03e97617aeb/pandas/tseries/converter.py#L27?
For the record (and because I guess @shoyer will be interested) - I recently delved into the units code for the first time (in depth at least) - it is a pretty powerful (if perhaps over-engineered) piece of code, and I used it to implement support for netcdftime phony datetime objects so that I could plot 360-day calendars. Work in progress at https://gist.github.com/pelson/786ce4827aa1cd40b3f6 - eventually this will find its way to matplotlib/netcdftime/Iris/somewhere once I've rounded off the corners.
shoyer commentedon Feb 4, 2015
@pelson In theory, yes, I think that would do the trick. But I spent 15 minutes on this and it didn't work -- not quite sure why. See shoyer/pandas@62895af if you're curious or if anyone else is interested in following up on this.
Also, glad to hear you're making progress on plotting netcdftime.datetime objects!
petehuang commentedon Jan 6, 2017
Hi,
This hasn't been updated in a while but I am confirming in 1.5.3 that this still is not supported. It sounds like there's some work put in here though.
Recommend removing
upstream fix requiredas it doesn't sound like the burden is on pandas for the fixefiring commentedon Jan 6, 2017
@petehuang, my understanding of this issue is that it really is an upstream bug. When pandas is imported, it registers its own unit converter with matplotlib, and that converter is responsible for turning pd.NaT into a NaN. It's out of our hands. The converter does not seem to be handling either pd.NaT (which is a datetime.datetime) or np.datetime64('NaT') correctly. (Pandas registers a converter for np.datetime64, which matplotlib lacks, in addition to one for datetime.datetime.)
The pd.NaT is a very strange beast. It is indeed a subclass of datetime.datetime, but instead of ints for its central data elements like the year, it uses floats, which allows those elements to be NaN.
petehuang commentedon Jan 6, 2017
Perfect, thank you for the explanation!
anntzer commentedon Jun 21, 2017
Closing, as the general agreement seems to be that this should be fixed on pandas' side. pandas devs should feel free to request a reopen if this changes.