Skip to content

Default legend location 'best' is extremely slow #8108

Closed

Description

@MaartenBaert

Matplotlib 2.0.0 seems to have changed the default legend location from "upper right" to "best" (although the documentation doesn't say this). I have noticed that this results in a severe performance penalty: several plots which would normally render in a fraction of a second now need 5-10 seconds. To make things worse, the legend location is recalculated whenever the plot is moved or zoomed. The sharex/sharey functionality which I use quite a lot makes this even slower, since all subplots are moved or zoomed simultaneously.

Code for reproduction

from pylab import *

x = linspace(0, 100, 100000)

close("all")
figure("Test", figsize=(12, 8))
for j in range(9):
	if j == 0:
		ax = subplot(3, 3, 1)
	else:
		subplot(3, 3, j + 1, sharex=ax, sharey=ax)
	for i in range(8):
		plot(x, sin(x + i * 0.5), color=cm.magma(i/8), label="test")
	legend(fontsize="small")

tight_layout()
show()

Matplotlib version

  • Matplotlib 2.0.0 (installed with pip)
  • Python 3.6.0 (compiled from source)
  • Platform: Linux (CentOS 6)

Activity

  1. tacaswell commented on Feb 20, 2017

    @tacaswell
    Member

    You can revert this change by setting rcParams['legend.loc'] = 'upper right'] (see http://matplotlib.org/users/dflt_style_changes.html#legends ).

    It looks like what is happening here is that for plotting the path simplification is kicking in so rendering is relatively fast, but the hit-testing is in python and is probably by-passing the simplification code so ends up being slow.

    I still think this is a reasonable trade-off overall, but we should look into if we can make hit-testing faster.

  2. anntzer commented on Feb 20, 2017

    @anntzer
    Contributor

    Hit testing is also done in C (it eventually resolves to _path.path_intersects_path) so I guess path simplification may be hooked into it as well (with "some" effort).

  3. modified the milestones: , on Feb 20, 2017
  4. added
    API: changesChanges to the public API, typically requiring deprecation.
    on Feb 20, 2017
  5. MaartenBaert commented on Feb 20, 2017

    @MaartenBaert
    ContributorAuthor

    Since the legend is an axis-aligned rectangle, it should be possible to write hit-testing code that is much faster than a generic path intersection algorithm. I just looked at path_intersects_path, I think I can write a much faster path_intersects_rectangle if there's interest in that, but I don't really know how to integrate this with the existing matplotlib internals.

  6. anntzer commented on Feb 21, 2017

    @anntzer
    Contributor

    Ah, good point that we don't need the general version here. I thought this would be easy, but can you actually also detect the case where a segment starts outside of (e.g., above) a rectangle, "cuts through a corner" and ends again out of (e.g., to the right of) the rectangle?

    If this can be done with numpy I'd suggest just adding it as a private function to legend.py -- we can always hoist it out later (possibly overloading Bbox.overlaps/Bbox.count_overlaps?).

  7. MaartenBaert commented on Feb 21, 2017

    @MaartenBaert
    ContributorAuthor

    Based on Minkowski difference theory (as used by the Gilbert–Johnson–Keerthi distance algorithm), I got the following conditions:

    // returns whether the segment from (x1,y1) to (x2,y2)
    // intersects the rectangle centered at (cx,cy) with size (w,h)
    bool segment_intersects_rectangle(double x1, double y1, double x2, double y2,
                                      double cx, double cy, double w, double h) {
        return fabs(x1 + x2 - 2.0 * cx) < fabs(x1 - x2) + w &&
               fabs(y1 + y2 - 2.0 * cy) < fabs(y1 - y2) + h &&
               2.0 * fabs((x1 - cx) * (y1 - y2) - (y1 - cy) * (x1 - x2)) <
                   w * fabs(y1 - y2) + h * fabs(x1 - x2);
    }

    I haven't tested this yet. Since it is based on the Minkowski difference, it should handle all possible edge cases without issue, assuming that I did the derivation correctly. It should be much faster than the current implementation which uses four segment intersection tests. It takes advantage of the inherent symmetry of the problem by using fabs() to reduce the number of checks. fabs() should be very fast on most architectures. Also, no divisions :).

    A C implementation is probably best since this is already a performance issue. It also makes it easy to add some 'early out' tests that can probably skip the vast majority of the calculations in the most common cases.

  8. anntzer commented on Feb 21, 2017

    @anntzer
    Contributor

    Very nice, I wasn't aware of this algorithm at all.

    It is up to you whether you want to add a numpy or a C implementation (which could probably go to _path_wrapper.cpp).

  9. MaartenBaert commented on Feb 21, 2017

    @MaartenBaert
    ContributorAuthor

    Here's a quick demo to show that it actually works: gist

    I will try to write an optimized C implementation later. Suppose I add my function to _path.h right below path_intersects_path, which other files should I change in order to make that function accessible from Python?

  10. anntzer commented on Feb 21, 2017

    @anntzer
    Contributor

    You want to add an entry to the

        static PyMethodDef module_functions[] = {
            {"point_in_path", (PyCFunction)Py_point_in_path, METH_VARARGS, Py_point_in_path__doc__},
            // etc.
    

    table at the end of _path_wrapper.cpp.

    The function will then show up in the (private) matplotlib._path module, which I'd suggest directly importing into legend.py (I don't think we need to expose this on the Path class... at least for now).

    PS: Cute example :-)

  11. tacaswell commented on Feb 21, 2017

    @tacaswell
    Member

    That is a nice example!

  12. stmax82 commented on Feb 21, 2017

    @stmax82

    I just ran into the same problem after upgrading to 2.0.0.. looks like it's already been reported. fwiw here's my gist that shows the problem including timings and a comparison between 1.5.3 (t=2.6s) and 2.0.0 (t=1m47s):

    https://gist.github.com/stmax82/7c1322e5d10e0395be06e6418cad735e

  13. MaartenBaert commented on Feb 22, 2017

    @MaartenBaert
    ContributorAuthor

    While reading the code I noticed that Path already has a intersects_bbox function which is used by legend, so I decided to replace that one instead of adding the code to legend.py.

    I've managed to get a 3x speed improvement by replacing path.intersects_bbox with my C++ implementation (I will send a pull request soon). It's better than the current situation but still unacceptably slow for something that's enabled by default IMHO:

    • Original with loc="upper right": 0.5 s
    • Original with loc="best": 12.3 s
    • Optimized with loc="best": 4.0 s

    Based on some simple profiling it looks like I can't get much better results with this function. After my optimizations, path.intersects_bbox accounts for just 30% of the total processing time, the remaining 70% is now mostly spent in bbox.count_contains. So that one probably needs a C++ implementation too.

    The problem is that my C++ implementation is actually wasting 95% of its time just getting the vertices of the path out of the AGG curve, the actual intersection math is just 5% of the total processing time. A faster implementation of bbox.count_contains would hit the same limit. This is pretty stupid, I know that the points I need are stored in a simple numpy array somewhere, there should be a much faster way to access them in C++. Does anyone know how?

  14. 2 remaining items

  15. anntzer commented on Feb 22, 2017

    @anntzer
    Contributor

    It wouldn't be the first time "nonstandard" axes need a slower handling codepath, see e.g. #7844. Orthogonal axes are still a common enough case that it's worth optimizing them, I'd say.

  16. anntzer commented on Mar 5, 2017

    @anntzer
    Contributor

    Also, any chance you can try profiling the agg part of the code to see what is so slow there? If on Linux, you may be able to use e.g. perf.

  17. modified the milestones: , on Mar 9, 2017
  18. ibnIrshad commented on Mar 9, 2017

    @ibnIrshad

    Might wanna close this issue since the implementation has been merged in PR #8224

  19. anntzer commented on Mar 9, 2017

    @anntzer
    Contributor

    There are clearly further improvements possibly possible (investigate count_contains, investigate whether vertex retrieval can be made faster) so let's just keep this open for now.

  20. modified the milestones: , on Aug 14, 2017
  21. removed
    API: changesChanges to the public API, typically requiring deprecation.
    on Aug 14, 2017
  22. github-actions commented on Apr 7, 2023

    @github-actions

    This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!

  23. added
    keepItems to be ignored by the “Stale” Github Action
    and removed
    status: inactiveMarked by the “Stale” Github Action
    on Apr 7, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    PerformancekeepItems to be ignored by the “Stale” Github Action

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions