Skip to content

[Bug]: Hist2d is less sensitive to narrow bins in new version, resulting in different plots #23505

New issue
New issue
Closed as not planned
Closed as not planned
[Bug]: Hist2d is less sensitive to narrow bins in new version, resulting in different plots#23505

Description

@mmajewsk
mmajewsk
opened on Jul 28, 2022

Bug summary

I was redoing old analysis, which earlier has been showing narrow bands in histogram:

(matplotlib 3.3.4)
MicrosoftTeams-image (2)

and when i tried it again, with exactly the same code, the same data the bands are gone:
(matplotlib 3.5.1)
MicrosoftTeams-image (1)

Code for reproduction

axe.hist2d(px, py,range=[[0,2047],[0,30]], bins=[2048,30],cmin=1)

Actual outcome

(matplotlib 3.5.1)
MicrosoftTeams-image (1)

Expected outcome

(matplotlib 3.3.4)
MicrosoftTeams-image (2)

Additional information

No response

Operating system

No response

Matplotlib Version

3.5.1

Matplotlib Backend

No response

Python version

No response

Jupyter version

No response

Installation

No response

Activity

  1. jklymak commented on Jul 28, 2022

    @jklymak
    jklymak
    on Jul 28, 2022
    Member

    Please make a reproducible minimal example. We can't test this as-is.

  2. jklymak
    added
    status: needs clarificationIssues that need more information to resolve.
    on Jul 28, 2022

  3. tacaswell commented on Jul 28, 2022

    @tacaswell
    tacaswell
    on Jul 28, 2022
    Member

    If you zoom in do the bands come back? Can you also check than np.histogram2d behaves the same with both numpy versions?

    My knee-jerk diagnosis is that either something changed with how we are doing anti-aliasing on pcolormesh or have shifted something in the rendering by a few pixels so that it is hitting the AA code differently. If I am reading that right, there is something like 2-3x as many bins in the x-direction as there pixels available.

  4. mmajewsk commented on Jul 28, 2022

    @mmajewsk
    mmajewsk
    on Jul 28, 2022
    Author

    @jklymak

    minimal code sample:

    import matplotlib.pyplot as plt
import numpy as np
size = 2048

px = []
py = []
std=3

cases = 100
for i in range(size):
    for j in range(cases):
        px.append(i)
        m = 10*i/size
        if i%32 == 0:
            py.append(np.random.normal(1.5*m,std*2))
        else:
            py.append(np.random.normal(m,std))

fig, axe = plt.subplots(1,1,figsize=(15,7))
axe.hist2d(px, py,range=[[0,2047],[0,30]], bins=[2048,30],cmin=1)

    matplotlib 3.3.4

    bugmplold

    matplotlib 3.5.1

    bugmplnew

  5. mmajewsk commented on Jul 28, 2022

    @mmajewsk
    mmajewsk
    on Jul 28, 2022
    Author

    @tacaswell

    If you zoom in do the bands come back? Can you also check than np.histogram2d behaves the same with both numpy versions?

    No they do not, at least not with png and in notebook.
    That would actually be hard for me to do right now, but i think this can be excluded because:

    My knee-jerk diagnosis is that either something changed with how we are doing anti-aliasing on pcolormesh or have shifted something in the rendering by a few pixels so that it is hitting the AA code differently. If I am reading that right, there is something like 2-3x as many bins in the x-direction as there pixels available.

    It seems it must be the rendereing since when written to pdf both plots are roughly the same.

  6. jklymak commented on Jul 28, 2022

    @jklymak
    jklymak
    on Jul 28, 2022
    Member

    I think we just need a FAQ https://matplotlib.org/stable/users/faq/index.html for these sorts of issues. Basically if you have too many pixels we (or any rendering engine) have to decide which ones to show. We don't anti-alias pcolormesh cells like we do imshow.

  7. mmajewsk commented on Jul 28, 2022

    @mmajewsk
    mmajewsk
    on Jul 28, 2022
    Author

    @jklymak
    I literally wouldn't have published a few papers If I would see the version from new matplotlib.
    I don't know how this can be solved within the code, but i would strongly suggest going back to the previous rendering, since:

    1. The new way hurts reproducibility of plots with matplotlib
    2. Very crutial things on the plot can be missed in the way that the new plots are made.
    3. AFAIK the anti-aliasing is for the plots to look nice, and if i would care for them looking nice i would opt for it, but good representation of the underlying data should take precedence.

    It doesn't help matplotlib to be serious plotting library missing information from a plot because of a rendering issue (which wasn't there before).

  8. jklymak commented on Jul 28, 2022

    @jklymak
    jklymak
    on Jul 28, 2022
    Member

    The contention is you had a rendering issue in both versions, just that it was different.

  9. jklymak commented on Jul 28, 2022

    @jklymak
    jklymak
    on Jul 28, 2022
    Member

    For instance in the above, at least on my screen, the 3.5.1 version has more detail than the 3.3.4 version. Maybe you got them backwards, but regardless, you ideally will have two pixels for each bin, which for your plot means at least 275 dpi (4096 dots /15 inches). And practically you need more because your axes don't reach both sides of the figure, so round up to 300 dpi. Anything less and you will get aliasing.

    If you use a pdf viewer it will anti-alias for you and do a better job of dealing with the singletons by averaging visual pixels. We do a version of that for imshow (https://matplotlib.org/stable/gallery/images_contours_and_fields/image_antialiasing.html) but we do not do that for pcolormesh rasters.

  10. tacaswell commented on Jul 28, 2022

    @tacaswell
    tacaswell
    on Jul 28, 2022
    Member

    Did we change something about drawing edges by default on pcolormesh?

  11. jklymak commented on Jul 28, 2022

    @jklymak
    jklymak
    on Jul 28, 2022
    Member

    Maybe? All I remember us working of for pcolormesh was the shading, which shouldn't affect this...

  12. greglucas commented on Jul 28, 2022

    @greglucas
    greglucas
    on Jul 28, 2022
    Contributor

    Snapping was changed, so maybe something related to that? #16090

  13. jklymak commented on Jul 29, 2022

    @jklymak
    jklymak
    on Jul 29, 2022
    Member

    @mmajewsk try setting snap=False to get the old (still aliased) behaviour.

  14. jklymak commented on Jul 29, 2022

    @jklymak
    jklymak
    on Jul 29, 2022
    Member

    I can confirm that snap=False gives the 3.3.4 behaviour.

    The fundamental issue still remains that pcolormesh (or hist2d) with more bins than pixels is going to be aliased. I strongly recommend that folks who want to do raster outputs save in a quite high dpi and then reduce visually using an external package. Theoretically Matplotlib could do that, but currently we do not. For it to work properly you need the whole image to be composed in high definition and no matter what high definition you use, it will be too low for someone's plot.

  15. jklymak
    removed
    status: needs clarificationIssues that need more information to resolve.
    on Oct 26, 2022

  16. jklymak commented on Oct 26, 2022

    @jklymak
    jklymak
    on Oct 26, 2022
    Member

    I'm going to close this, because I don't think there is much we can do about this issue...

  17. jklymak
    closed this as not plannedon Oct 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions