Skip to content

time series plotting gap causes strange jump #16266

Closed as not planned
Closed as not planned

Description

@Fealthas

Bug report

Gap in time series data causes MPL to jump to a very small datetime that doesnt exist in the dataset.
image

If I don't manually set the axis, it tries to autofit.

ax1.set_xlim( [ min(datana.date.dt.to_pydatetime()) , max(datana.date.dt.to_pydatetime()) ])

Autofitting looks like this:
image

Note that the minimum x datetime in the dataset is in 2019! Not sure where the 1680ish time comes from. I've manually checked what actually gets passed into mpl, a numpy object array of datetime objects, does not have a time value below 2019.

Abbreviated code:

import datetime
import time
import matplotlib
matplotlib.use("agg")
import matplotlib.pyplot as plt
import pandas as pd

fig1, ax1 = plt.subplots( figsize=(15,5) )

    data = pd.read_csv("temps.dat",header=None, names=['date','temp'])
    data.temp = data.temp/1000
    data.date = pd.to_datetime(data.date)
    data.date = data.date.dt.ceil(freq='s') #strip microseconds
    data['roll'] = data['temp'].rolling(window=722 ).mean()
    data['roll2'] = data['temp'].rolling(window=14220  ).mean()
    datana = data.dropna(subset=['roll'])
    datana2 = data.dropna(subset=['roll2'])

    ax1.cla()
 
    ax1.plot(datana2.date.dt.to_pydatetime()  ,datana2.roll2, lw=3, alpha=.8)
    ax1.plot(datana.date.dt.to_pydatetime()  ,datana.roll, lw=.7)

    ax1.set_xlim( [ min(datana.date.dt.to_pydatetime()) , max(datana.date.dt.to_pydatetime()) ])
fig1.autofmt_xdate()
fig1.savefig("fig1.png",format='png' ,bbox_inches='tight', dpi=500)

temps.zip
Matplotlib version

  • Operating system: debian (raspbian)

  • Matplotlib version: 3.1.2 , in a venv, installed by pip

  • Matplotlib backend (print(matplotlib.get_backend())): agg

  • Python version: 3.7

  • Other libraries: pandas, Pillow

Activity

  1. jklymak commented on Jan 19, 2020

    @jklymak
    Member

    You will get faster turnaround on this if you can provide something more minimal with a data set you can type out rather than providing it as a zip file. In producing that minimal example you may discover the specific problem and narrow down if the problem is in matplotlib or your data. Even better would be to not require pandas. Thanks!

  2. anntzer commented on Jan 20, 2020

    @anntzer
    Contributor

    There are some NaT rows in your dataset. I guess old pandas (which I don't have around) would convert these to some nonsensical value when using toordinal() (which is what we use internally in dates._to_ordinalf); pandas 1.0(pre) now raises a ValueError here. I guess we could detect NaT and convert it to nan here... (pandas-dev/pandas#10372?)

    Edit: Actually I think the change is due to pandas-dev/pandas#18720.

  3. Fealthas commented on Jan 25, 2020

    @Fealthas
    Author

    Fixing the offending NaT fixed the issue.

    I guess the real bug here is that neither pandas nor mpl threw an error for an invalid value and just attempted to finish what it started no matter what.

  4. github-actions commented on Jul 7, 2023

    @github-actions

    This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

  5. added
    status: closed as inactiveIssues closed by the "Stale" Github Action. Please comment on any you think should still be open.
    on Aug 7, 2023
  6. tacaswell commented on Aug 7, 2023

    @tacaswell
    Member

    I am comfortable leaving this closed as it was fixed upstream by pandas.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions