Skip to content

Travis error is MacOS image tolerance of 0.005 for test_constrained_layout.py::test_colorbar_location #12433

Closed

Description

@jklymak

Originally posted by @jklymak in #12431

Travis error is MacOS image tolerance of 0.005 changing for test_constrained_layout.py::test_colorbar_location. Not sure what changed on macOS to cause this, but it doesn't affect this PR.

Activity

  1. added this to the v3.0.x milestone on Oct 7, 2018
  2. jklymak commented on Oct 15, 2018

    @jklymak
    MemberAuthor

    This seems to have fixed itself 🤷‍♂️

  3. QuLogic commented on Oct 31, 2018

    @QuLogic
    Member

    I assume you don't know what the result looks like; I just hit this trying to build a package locally, so here's what it looks like:
    test_colorbar_location
    Note that in this case, the RMS is much higher: 78.990.

  4. modified the milestones: v3.0.1, v3.0.x on Oct 31, 2018
  5. jklymak commented on Oct 31, 2018

    @jklymak
    MemberAuthor

    Yeah, its just a flaky test. 90% of the time, if you rerun it, it'll look correct...

    I'm guessing the constraint solver gets in a bad loop sometimes or finds an incorrect minima?

    Does it really take a long time if this happens for you?

  6. jklymak commented on Oct 31, 2018

    @jklymak
    MemberAuthor

    Also open to suggestions - happy to simplify the test - its a bit of a crazy figure 😉

  7. jklymak commented on Oct 31, 2018

    @jklymak
    MemberAuthor

    i.e. #12678 it failed, and I reran the CI it failed on and it passed...

  8. QuLogic commented on Oct 31, 2018

    @QuLogic
    Member

    I mean, it was the full test suite, so whether it was slower wasn't particularly noticeable. I also re-ran just the one test 100 times over and it never failed, so...

    Should we just mark it as flaky then?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions