Skip to content

fill_between interpolation & nan issue #11781

Closed

Description

@Phlya

The issue was discovered here
deeptools/pyGenomeTracks#37

Basically, when fill_between encounters a NaN in the y values, when interpolate=True it produces a strange output.

This is without interpolation:

import matplotlib.pyplot as plt
import numpy as np

x = np.arange(107).astype(float)
y = np.append(np.zeros((100))+0.01, [0.5, 0.75, 1, np.nan, 1, 0.75, 0.5])

plt.figure(figsize=(20, 5))
plt.fill_between(x, y, interpolate=False)

image

If I turn interpolation on (which I wouldn't expect to do anything in this case, since there are not zero crossings), this is what I get:

plt.figure(figsize=(20, 5))
plt.fill_between(x, y, interpolate=True)

image

Even if I am misunderstanding something about how interpolation is supposed to work, clearly this is not what it should do...

mpl 2.1.2, python 3.5.2

Activity

  1. Phlya commented on Jul 27, 2018

    @Phlya
    ContributorAuthor

    Another example to clarify what's going on

    x = np.arange(107).astype(float)
    y = np.append(np.zeros((100))+0.01, [2, 0.75, 1, np.nan, 1, 0.75, 0.5])
    
    plt.figure(figsize=(20, 5))
    plt.fill_between(x, y, interpolate=True)

    image

  2. changed the title [-]fill_between interpolation issue[/-] [+]fill_between interpolation & nan issue[/+] on Jul 27, 2018
  3. Phlya commented on Jul 27, 2018

    @Phlya
    ContributorAuthor

    Also possible to make an example where this fails when there actually are negative values and a where=y>0 condition:

    x = np.arange(101).astype(float)
    y = np.append(np.zeros((100))+np.random.random(100)-0.05, [np.nan])
    
    plt.figure(figsize=(20, 5))
    plt.fill_between(x, y, interpolate=True, where=y>0)

    image

  4. jklymak commented on Jul 27, 2018

    @jklymak
    Member

    While you are testing, have you tried masking instead of passing NaN?

  5. Phlya commented on Jul 27, 2018

    @Phlya
    ContributorAuthor

    No, sorry - but can try tomorrow.

  6. timhoffm commented on Jul 27, 2018

    @timhoffm
    Member

    Part of the problem is that

    _axes.py l.5039:

    x, y1, y2 = np.broadcast_arrays(np.atleast_1d(x), y1, y2)
    

    casts the masked arrays back to regular arrays.

    _axes.py l.5063ff:

                        if len(diff_values) == 2:
                            if np.ma.is_masked(diff_values[1]):
                                return x[im1], y1[im1]
                            elif np.ma.is_masked(diff_values[0]):
                                return x[ind], y1[ind]
    

    does not have an effect anymore (the invalid values are not masked anymore). It works if you replace np.ma.is_masked()by ~np.isfinite(). No sure if this is the right way to go, though.

  7. jklymak commented on Jul 27, 2018

    @jklymak
    Member

    Good find; seems like thats definitely the issue.

    Marking as "units", because how we normalize arrays after they come to us is tied up in the units discussion. It'd be nice if every x/y data array input to axes methods got the same consistent conversion, and that would certainly include how to handle NaN versus masked....

  8. added this to the v3.1 milestone on Jul 27, 2018
  9. Phlya commented on Jul 28, 2018

    @Phlya
    ContributorAuthor

    Just in case, masking the NaN value doesn't help.

  10. jklymak commented on Mar 3, 2019

    @jklymak
    Member

    This looks to have been introduced in #7562... ping @anntzer.

  11. anntzer commented on Mar 3, 2019

    @anntzer
    Contributor

    I can't repro the issue, neither with mpl2.1.2 (original report) or master...

  12. jklymak commented on Mar 4, 2019

    @jklymak
    Member

    You can't reproduce with interpolate=True?

  13. anntzer commented on Mar 4, 2019

    @anntzer
    Contributor

    This is what I get when running the example copy-pasted from #11781 (comment) with master:
    test

  14. jklymak commented on Mar 4, 2019

    @jklymak
    Member
  15. anntzer commented on Mar 4, 2019

    @anntzer
    Contributor

    test

  16. jklymak commented on Mar 4, 2019

    @jklymak
    Member
    import matplotlib.pyplot as plt
    import numpy as np
    
    x = np.arange(107).astype(float)
    y = np.append(np.zeros((100))+0.01, [0.5, 0.75, 1, np.nan, 1, 0.75, 0.5])
    
    plt.figure()
    plt.fill_between(x, y, interpolate=True)
    plt.show()
    

    figure_1

  17. anntzer commented on Mar 4, 2019

    @anntzer
    Contributor

    What version of numpy? I'm on 1.16.2.
    I can't think of anything else...

  18. jklymak commented on Mar 4, 2019

    @jklymak
    Member

    1.14.6... Upgrading to 1.16 fixes. So, I guess we can close this....

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions