Skip to content

IndexError: tuple index out of range with pandas 0.25. #14992

Closed

Description

@Foadsf

Bug report

Bug summary

I have reported the issue in details here
basically after an upgrade to 3.1.1 it drops the error:

IndexError: tuple index out of range

all other details inclduing the CSV file and code are in the above link.

Activity

  1. jamespreed commented on Aug 6, 2019

    @jamespreed

    Looks like this commit changed how the indices were assigned by removing the _xy_from_xy method. This works as of 3.1.0, but not 3.1.1 (on Windows).

  2. Foadsf commented on Aug 6, 2019

    @Foadsf
    Author

    Downgrading to the previous versions doesn't help! There might be more issues IMHO ¯\_(ツ)_/¯

  3. anntzer commented on Aug 6, 2019

    @anntzer
    Contributor

    This is actually due to the upgrade to pandas 0.25:

    import pandas as pd
    df = pd.DataFrame({"a": [1, 2, 3]})
    print(df.index.shape, df.index[:, None].shape)
    

    This now prints (3,) (3,), but with pandas 0.24 used to print (3,) (3, 1) which we relied on to convert input to 2D.

  4. Foadsf commented on Aug 6, 2019

    @Foadsf
    Author

    @anntzer downgrading to Pandas 0.24 solved the problem!

  5. Foadsf commented on Aug 6, 2019

    @Foadsf
    Author

    I reported an issue on pandas repo here

  6. changed the title [-]IndexError: tuple index out of range after upgrade to 3.1.1[/-] [+]IndexError: tuple index out of range with pandas 0.25.[/+] on Aug 6, 2019
  7. jorisvandenbossche commented on Aug 6, 2019

    @jorisvandenbossche

    @story645 I am not fully sure we can provide an upstream fix (or at least not long term).

    df.index is a one-dimensional object, so df.index[:, None] actually giving an Index that is 2D, could have been considered a bug in pandas.

  8. tacaswell commented on Aug 6, 2019

    @tacaswell
    Member

    @Foadsf Can you please paste a minimal copy-pastable example into this issue? SO links rot over time.

  9. story645 commented on Aug 6, 2019

    @story645
    Member

    @jorisvandenbossche wondering then if pandas should be throwing a warning or something since
    array[:,None] syntax is the numpy recommended way of expanding dims and so the expectation there is to return a 2D item.

  10. jklymak commented on Aug 7, 2019

    @jklymak
    Member

    What does np.newaxis give if we use that instead?

  11. Foadsf commented on Aug 7, 2019

    @Foadsf
    Author

    Dear @tacaswell the Pandas guys were kind to edit my post here and add clear and concise examples demonstrating the issue.

  12. jorisvandenbossche commented on Aug 7, 2019

    @jorisvandenbossche

    @Foadsf that's a reproducible example for just the pandas side, but I think @tacaswell is looking for an example that shows the error with matplotlib.

    Something like

    s = pd.Series([1, 2, 3])
    plt.plot(s.index, s.values)
    
  13. tacaswell commented on Aug 7, 2019

    @tacaswell
    Member

    Thanks @jorisvandenbossche

    @Foadsf in general, you will get much better responses from projects if you provide an easily understandable and testable examples (see http://www.sscce.org/ for a longer discussion as to why). The information we ask for in the issue template to guide you so we can help you as easily as possible.

  14. QuLogic commented on Aug 7, 2019

    @QuLogic
    Member

    @jklymak np.newaxis is None; there is no difference between the two other than programmer clarity.

  15. ImportanceOfBeingErnest commented on Aug 7, 2019

    @ImportanceOfBeingErnest
    Member
    df = pd.DataFrame({"XX" : [4,5,6], "YY" : [7,1,2]})
    print(df.index[:,None].shape)
    print(df["YY"][:,None].shape)
    

    prints

    (3,)
    (3, 1)
    

    This seems rather inconsistent.

    Marking as release critical, because something needs to be done here.

  16. added this to the v3.2.0 milestone on Aug 7, 2019
  17. added
    Release criticalFor bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.
    on Aug 7, 2019
  18. modified the milestones: v3.2.0, v2.2.5 on Aug 8, 2019
  19. added a commit that references this issue on Aug 8, 2019
    d33d654
  20. jorisvandenbossche commented on Aug 8, 2019

    @jorisvandenbossche

    This seems rather inconsistent.

    @ImportanceOfBeingErnest yes, that is certainly inconsistent, and "kind of" a regression in pandas (which I also propose to fix in pandas itself for 0.25.1, see pandas-dev/pandas#27775 (comment))
    I am saying "kind of" because we should decide what to do when you end up with a 2D Index (which is invalid, but currently you can get that with the indexing operation as idx[:, None]): return a 2D array or raise an error.
    series[:, None] does return a 2D numpy array, hence you get the correct shape.

    But in any case, I think it would be good for matplotlib to not rely on this indexing aspect to convert a pandas Index into a 2D array-like.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions