You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Default legend location 'best' is extremely slow #8108
Matplotlib 2.0.0 seems to have changed the default legend location from "upper right" to "best" (although the documentation doesn't say this). I have noticed that this results in a severe performance penalty: several plots which would normally render in a fraction of a second now need 5-10 seconds. To make things worse, the legend location is recalculated whenever the plot is moved or zoomed. The sharex/sharey functionality which I use quite a lot makes this even slower, since all subplots are moved or zoomed simultaneously.
It looks like what is happening here is that for plotting the path simplification is kicking in so rendering is relatively fast, but the hit-testing is in python and is probably by-passing the simplification code so ends up being slow.
I still think this is a reasonable trade-off overall, but we should look into if we can make hit-testing faster.
Hit testing is also done in C (it eventually resolves to _path.path_intersects_path) so I guess path simplification may be hooked into it as well (with "some" effort).
Since the legend is an axis-aligned rectangle, it should be possible to write hit-testing code that is much faster than a generic path intersection algorithm. I just looked at path_intersects_path, I think I can write a much faster path_intersects_rectangle if there's interest in that, but I don't really know how to integrate this with the existing matplotlib internals.
Ah, good point that we don't need the general version here. I thought this would be easy, but can you actually also detect the case where a segment starts outside of (e.g., above) a rectangle, "cuts through a corner" and ends again out of (e.g., to the right of) the rectangle?
If this can be done with numpy I'd suggest just adding it as a private function to legend.py -- we can always hoist it out later (possibly overloading Bbox.overlaps/Bbox.count_overlaps?).
Based on Minkowski difference theory (as used by the Gilbert–Johnson–Keerthi distance algorithm), I got the following conditions:
// returns whether the segment from (x1,y1) to (x2,y2)// intersects the rectangle centered at (cx,cy) with size (w,h)boolsegment_intersects_rectangle(doublex1, doubley1, doublex2, doubley2,
doublecx, doublecy, doublew, doubleh) {
returnfabs(x1+x2-2.0*cx) <fabs(x1-x2) +w&&fabs(y1+y2-2.0*cy) <fabs(y1-y2) +h&&2.0*fabs((x1-cx) * (y1-y2) - (y1-cy) * (x1-x2)) <w*fabs(y1-y2) +h*fabs(x1-x2);
}
I haven't tested this yet. Since it is based on the Minkowski difference, it should handle all possible edge cases without issue, assuming that I did the derivation correctly. It should be much faster than the current implementation which uses four segment intersection tests. It takes advantage of the inherent symmetry of the problem by using fabs() to reduce the number of checks. fabs() should be very fast on most architectures. Also, no divisions :).
A C implementation is probably best since this is already a performance issue. It also makes it easy to add some 'early out' tests that can probably skip the vast majority of the calculations in the most common cases.
Here's a quick demo to show that it actually works: gist
I will try to write an optimized C implementation later. Suppose I add my function to _path.h right below path_intersects_path, which other files should I change in order to make that function accessible from Python?
The function will then show up in the (private) matplotlib._path module, which I'd suggest directly importing into legend.py (I don't think we need to expose this on the Path class... at least for now).
I just ran into the same problem after upgrading to 2.0.0.. looks like it's already been reported. fwiw here's my gist that shows the problem including timings and a comparison between 1.5.3 (t=2.6s) and 2.0.0 (t=1m47s):
While reading the code I noticed that Path already has a intersects_bbox function which is used by legend, so I decided to replace that one instead of adding the code to legend.py.
I've managed to get a 3x speed improvement by replacing path.intersects_bbox with my C++ implementation (I will send a pull request soon). It's better than the current situation but still unacceptably slow for something that's enabled by default IMHO:
Original with loc="upper right": 0.5 s
Original with loc="best": 12.3 s
Optimized with loc="best": 4.0 s
Based on some simple profiling it looks like I can't get much better results with this function. After my optimizations, path.intersects_bbox accounts for just 30% of the total processing time, the remaining 70% is now mostly spent in bbox.count_contains. So that one probably needs a C++ implementation too.
The problem is that my C++ implementation is actually wasting 95% of its time just getting the vertices of the path out of the AGG curve, the actual intersection math is just 5% of the total processing time. A faster implementation of bbox.count_contains would hit the same limit. This is pretty stupid, I know that the points I need are stored in a simple numpy array somewhere, there should be a much faster way to access them in C++. Does anyone know how?
It wouldn't be the first time "nonstandard" axes need a slower handling codepath, see e.g. #7844. Orthogonal axes are still a common enough case that it's worth optimizing them, I'd say.
There are clearly further improvements possibly possible (investigate count_contains, investigate whether vertex retrieval can be made faster) so let's just keep this open for now.
This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!
Activity
tacaswell commentedon Feb 20, 2017
You can revert this change by setting
rcParams['legend.loc'] = 'upper right'](see http://matplotlib.org/users/dflt_style_changes.html#legends ).It looks like what is happening here is that for plotting the path simplification is kicking in so rendering is relatively fast, but the hit-testing is in python and is probably by-passing the simplification code so ends up being slow.
I still think this is a reasonable trade-off overall, but we should look into if we can make hit-testing faster.
anntzer commentedon Feb 20, 2017
Hit testing is also done in C (it eventually resolves to
_path.path_intersects_path) so I guess path simplification may be hooked into it as well (with "some" effort).MaartenBaert commentedon Feb 20, 2017
Since the legend is an axis-aligned rectangle, it should be possible to write hit-testing code that is much faster than a generic path intersection algorithm. I just looked at path_intersects_path, I think I can write a much faster path_intersects_rectangle if there's interest in that, but I don't really know how to integrate this with the existing matplotlib internals.
anntzer commentedon Feb 21, 2017
Ah, good point that we don't need the general version here. I thought this would be easy, but can you actually also detect the case where a segment starts outside of (e.g., above) a rectangle, "cuts through a corner" and ends again out of (e.g., to the right of) the rectangle?
If this can be done with numpy I'd suggest just adding it as a private function to
legend.py-- we can always hoist it out later (possibly overloadingBbox.overlaps/Bbox.count_overlaps?).MaartenBaert commentedon Feb 21, 2017
Based on Minkowski difference theory (as used by the Gilbert–Johnson–Keerthi distance algorithm), I got the following conditions:
I haven't tested this yet. Since it is based on the Minkowski difference, it should handle all possible edge cases without issue, assuming that I did the derivation correctly. It should be much faster than the current implementation which uses four segment intersection tests. It takes advantage of the inherent symmetry of the problem by using fabs() to reduce the number of checks. fabs() should be very fast on most architectures. Also, no divisions :).
A C implementation is probably best since this is already a performance issue. It also makes it easy to add some 'early out' tests that can probably skip the vast majority of the calculations in the most common cases.
anntzer commentedon Feb 21, 2017
Very nice, I wasn't aware of this algorithm at all.
It is up to you whether you want to add a numpy or a C implementation (which could probably go to
_path_wrapper.cpp).MaartenBaert commentedon Feb 21, 2017
Here's a quick demo to show that it actually works: gist
I will try to write an optimized C implementation later. Suppose I add my function to
_path.hright belowpath_intersects_path, which other files should I change in order to make that function accessible from Python?anntzer commentedon Feb 21, 2017
You want to add an entry to the
table at the end of
_path_wrapper.cpp.The function will then show up in the (private)
matplotlib._pathmodule, which I'd suggest directly importing intolegend.py(I don't think we need to expose this on the Path class... at least for now).PS: Cute example :-)
tacaswell commentedon Feb 21, 2017
That is a nice example!
stmax82 commentedon Feb 21, 2017
I just ran into the same problem after upgrading to 2.0.0.. looks like it's already been reported. fwiw here's my gist that shows the problem including timings and a comparison between 1.5.3 (t=2.6s) and 2.0.0 (t=1m47s):
https://gist.github.com/stmax82/7c1322e5d10e0395be06e6418cad735e
MaartenBaert commentedon Feb 22, 2017
While reading the code I noticed that Path already has a intersects_bbox function which is used by legend, so I decided to replace that one instead of adding the code to
legend.py.I've managed to get a 3x speed improvement by replacing path.intersects_bbox with my C++ implementation (I will send a pull request soon). It's better than the current situation but still unacceptably slow for something that's enabled by default IMHO:
Based on some simple profiling it looks like I can't get much better results with this function. After my optimizations, path.intersects_bbox accounts for just 30% of the total processing time, the remaining 70% is now mostly spent in bbox.count_contains. So that one probably needs a C++ implementation too.
The problem is that my C++ implementation is actually wasting 95% of its time just getting the vertices of the path out of the AGG curve, the actual intersection math is just 5% of the total processing time. A faster implementation of bbox.count_contains would hit the same limit. This is pretty stupid, I know that the points I need are stored in a simple numpy array somewhere, there should be a much faster way to access them in C++. Does anyone know how?
2 remaining items
anntzer commentedon Feb 22, 2017
It wouldn't be the first time "nonstandard" axes need a slower handling codepath, see e.g. #7844. Orthogonal axes are still a common enough case that it's worth optimizing them, I'd say.
anntzer commentedon Mar 5, 2017
Also, any chance you can try profiling the agg part of the code to see what is so slow there? If on Linux, you may be able to use e.g.
perf.ibnIrshad commentedon Mar 9, 2017
Might wanna close this issue since the implementation has been merged in PR #8224
anntzer commentedon Mar 9, 2017
There are clearly further improvements possibly possible (investigate
count_contains, investigate whether vertex retrieval can be made faster) so let's just keep this open for now.github-actions commentedon Apr 7, 2023
This issue has been marked "inactive" because it has been 365 days since the last comment. If this issue is still present in recent Matplotlib releases, or the feature request is still wanted, please leave a comment and this label will be removed. If there are no updates in another 30 days, this issue will be automatically closed, but you are free to re-open or create a new issue if needed. We value issue reports, and this procedure is meant to help us resurface and prioritize issues that have not been addressed yet, not make them disappear. Thanks for your help!