Replace warnings.warn with cbook._warn_external or logging.warning by hershen · Pull Request #12006 · matplotlib/matplotlib

hershen · 2018-09-02T22:01:05Z

PR Summary

Fix issue #10643

Added stacklevel=2 to all warnings.warn calls that didn't have stacklevel set.
Added new section to contributing.rst advising to set stacklevel when using warnings.warn

Note:
I liked @ImportanceOfBeingErnest 's suggestion of advising to set stacklevel in the documentation. I didn't find a good place for it, so I created a new section in contributing.rst. Please let me know if there's a better place or if it's too long.

PR Checklist

Has Pytest style unit tests
Code is Flake 8 compliant
New features are documented, with examples if plot related
Documentation is sphinx and numpydoc compliant
Added an entry to doc/users/next_whats_new/ if major new feature (follow instructions in README.rst there)
Documented in doc/api/api_changes.rst if API changed in a backward-incompatible way

jklymak · 2018-09-02T22:18:37Z

Wow, looks great! You are going to have to rebase __init__.py, but I hope that is straight forward.

However, is there no way for us to automate this? Does it have to be set manually for every warnings call? I admit that a quick google search didn't turn up anything hopeful, but thought I'd ask...

jklymak · 2018-09-02T22:19:33Z

We also don't allow lines >79 characters, so you'll have to sort out all the overspilling lines (see the flake8 errors in travis-ci/py3.6 build)

anntzer · 2018-09-03T06:01:28Z

In general, this is an improvement. But an even better improvement would be to just use _warn_external everywhere -- see #11298 (and adapt the (nice) explanation for it). (See for example (not in this PR) backend_pdf's writeInfoDict which currently warns with stacklevel=2 on invalid keywords, but that'll point to some other internal method of backend_pdf which only makes things more (not less) confusing for the end user.)

However, it would be even better not to apply either stacklevel=2, or _warn_external, indiscriminately. For example, the change in backend_bases.py in this PR is in key_press_handler, which is invoked asynchronously by the GUI event loop (well, I don't know if there's a good value for stacklevel in that case, so perhaps we can just live with it...).

hershen · 2018-09-05T00:12:37Z

Oh, that's a nice idea in _warn_external.

Sorry, I didn't follow your key_press_handler example. Why would _warn_external not work well in that case?

And a git question - is there a way to continue working on this branch without polluting the history (for example changing warnings.warn(___, stacklevel=2) to _warn_external)? Or is it better to just create a new branch from master?

anntzer · 2018-09-05T22:19:04Z

Perhaps key_press_handler is not the best example but I remember looking into doing a global replace and thinking that some of the warn calls would become more confusing if using _warn_external. I don't have a case right now, someone just needs to do a careful review of the changes.
re: git: you can create a throwaway branch here if you don't want to lose your work, then you can git reset --hard HEAD~ this first branch and rework from there.

hershen · 2018-09-06T10:21:50Z

Ok, thanks.

I'm away from a computer until the middle of next week. When I return, I can do a global replace, as I don't know the code well enough to identify when _warn_external isn't appropriate. A more knowledgeable reviewer can then hopefully identify the completed cases. Will that be helpful, or should I pick a different issue to work on?

Thanks for the git tip. I'll play around with it.

anntzer · 2018-09-06T10:26:39Z

Sounds good.

anntzer · 2018-09-06T13:27:35Z

            json.dump(data, fh, cls=JSONEncoder, indent=2)
        except OSError as e:
-            warnings.warn('Could not save font_manager cache {}'.format(e))
+            warnings.warn('Could not save font_manager cache {}'.format(e), stacklevel=2)


For example, I don't think that this warning would get clearer if using _warn_external (or even stacklevel=2).

OK, I see what you mean.

Is it safe to assume that this warning will be "triggered" when importing pyplot? If so, then does it make sense to print the user line of code that imports pyplot i.e. import matplotlib.pyplot as plt?

I tried to manipulate _warn_external to do this without success. If this solution is acceptable, any ideas how to achieve that?

Is it safe to assume that this warning will be "triggered" when importing pyplot?

Yes

If so, then does it make sense to print the user line of code that imports pyplot i.e. import matplotlib.pyplot as plt?

I wouldn't overthink it (there isn't anything really smart to do in this case IMO) and would just trigger a warning at this line; or perhaps is actually makes sense to use logging.warning instead of warnings.warn here in fact, now that I think of it...

After further thought, I convinced myself that that should exactly be the criterion for warnings vs logging: by showing the line of offending code, does that point the user to something they should modify?

Sounds good, thanks.

timhoffm · 2018-10-13T06:51:26Z

@hershen still interested to work on this?

hershen · 2018-10-13T16:56:30Z

I am. Sorry it got set aside for so long. I'll try to finish it this weekend.

timhoffm · 2018-10-13T17:53:00Z

No hurry. Just wanted to make sure this is not orphaned.

hershen · 2018-10-16T22:41:19Z

I (finally) switched the warnings.warn to either logging.warning or cbook._warn_external. I followed @anntzer's advice and tried to choose cbook._warn_external when I thought the reason for the warning is the user's code or if the warning will help the user find the instance in the code where the warning occurred and logging.warning when the user can't fix his code. Please have a look if the choice of function makes sense. Please pay particular attention to the following places where I felt even less sure which function to choose:

lib/matplotlib/artist.py (lines 404, 855)
lib/matplotlib/backend_manager.py (263)
lib/matplotlib/font_manager.py (1230, 1239)
lib/matplotlib/lines.py (475)
lib/matplotlib/mathtext.py (all)
lib/matplotlib/textpath.py (321, 370)
lib/matplotlib/tight_layout.py (230)
lib/matplotlib/axes/_base.py (3912)
lib/matplotlib/backends/backend_pgf.py (402)
lib/matplotlib/backends/backend_webagg_core.py (243)
lib/matplotlib/backends/qt_editor/formlayout.py (97, 268)
lib/matplotlib/style/core.py (193)
lib/mplt_toolkits/mplot3d/axes3d.py (1081)

anntzer · 2018-10-17T09:59:43Z

                    ax._set_position(newpos, which='original')
        else:
-            warnings.warn('constrained_layout not applied.  At least '
+            cbook._warn_ext('constrained_layout not applied.  At least '


Good catch, thanks!

anntzer · 2018-10-17T10:00:42Z

        """
        if rasterized and not hasattr(self.draw, "_supports_rasterization"):
-            warnings.warn("Rasterization of '%s' will be ignored" % self)
+            logging.warning("Rasterization of '%s' will be ignored" % self)


looks like cbook._warn_external

anntzer · 2018-10-17T10:02:58Z

    def handle_unknown_event(self, event):
-        warnings.warn('Unhandled message type {0}. {1}'.format(
-            event['type'], event), stacklevel=2)
+        cbook._warn_external('Unhandled message type {0}. {1}'.format(


I think this should use logging (it's basically an event handler and thus likely to be called asynchronously anyways).

anntzer · 2018-10-17T10:03:47Z

                # looks like they forgot to set the image type drop
                # down, going with the extension.
-                warnings.warn(
+                cbook._warn_external(


logging
(this is most likely called throgh gui interaction)

anntzer · 2018-10-17T10:03:53Z

            # looks like they forgot to set the image type drop
            # down, going with the extension.
-            warnings.warn(
+            cbook._warn_external(


same as above

anntzer · 2018-10-17T10:04:18Z

                ax.set_yscale('log')
            except ValueError as exc:
-                warnings.warn(str(exc))
+                cbook._warn_external(str(exc))


logging (most likely called through gui interaction)

anntzer · 2018-10-17T10:04:28Z

                ax.set_xscale('log')
            except ValueError as exc:
-                warnings.warn(str(exc))
+                cbook._warn_external(str(exc))


same as above

anntzer · 2018-10-17T10:08:32Z

        # Convert pick radius from points to pixels
        if self.figure is None:
-            warnings.warn('no figure set when check if mouse is on line')
+            logging.warning('no figure set when check if mouse is on line')


I think warn_external here (can be callback or direct call).

Isn't this gui interaction?

I was thinking that one can also call contains() directly, but actually that's almost certainly much rarer, so logging is fine.

anntzer · 2018-10-17T10:09:18Z

        if o == 0:
            if len(self.children):
-                warnings.warn(
+                cbook._warn_external(


also logging? a bit borderline but consistent with the rest of the mathtext module

anntzer · 2018-10-17T10:13:24Z

                # if this fails, we will just write to stdout
            except IOError:
-                warnings.warn('could not open log file "{0}"'
+                logging.warning('could not open log file "{0}"'


throughout the PR:

You should use the module-level logger _log = logging.getLogger(__name__) then _log.warning(...).

Alignment of subsequent lines are a bit messed up sometimes.

Ah, right. I even read that I should do that, but for some reason it didn't register.

Thanks for pointing that out. I wasn't really paying attention to that.

anntzer · 2018-10-17T10:14:10Z

Thanks for the thorough changes! Left some comments.

anntzer · 2018-10-19T08:18:12Z

 ##############################################################################
 # FONTS

+


no empty line here?

I added it because I got E302 expected 2 blank lines, found 1 from flake8 when it wasn't there.
Is there a better solution?

Putting the blank line before the ##### seems more logical? Should hopefully also placate the style checker.

You're right. That fixed it.

anntzer · 2018-10-19T08:19:33Z

-        matplotlib.cbook._warn_external(
-            'The left and right margins cannot be made large enough to '
-            'accommodate all axes decorations. ')
+        matplotlib.cbook._warn_external('The left and right margins cannot '


just do from matplotlib import cbook at the top?

Fixed, thanks.

anntzer

Only minor formatting points left.

hershen · 2018-10-19T17:46:13Z

Squashed.

QuLogic

A few typos; I mostly assume @anntzer has looked over the other bits.

QuLogic · 2018-10-20T05:08:02Z

+`warnings.warn`) is that `cbook._warn_external` should be used for things the
+user must change to stop the warning (typically in the source), whereas
+`logging.warning` can be more persistent.  Moreover, note that
+`cbook._warn_extrenal` will by default only emit a given warning *once* for


Typo: extrenal

QuLogic · 2018-10-20T05:08:38Z

-layouting or rendering) should only log at this level.
+By default, `warnings.warn` displays the line of code that has the `warn` call.
+This usually isn't more informative than the warning message itself. Therefore,
+Matplotlib uses `cbook._warn_external` which uses `warnings.warn`, but it goes


Remove 'it'

QuLogic · 2018-10-20T05:09:01Z

+up the stack and displays the first line of code outside of Matplotlib.
+For example, for the module::
+
+    #in my_matplotlib_module.py


Space after #

QuLogic · 2018-10-20T05:09:22Z

+running the script::
+
+    from matplotlib import my_matplotlib_module
+    my_matplotlib_module.set_range(0,0) #set range


Space after , and #; two spaces before #.

QuLogic · 2018-10-20T05:09:48Z

+    UserWarning: Attempting to set identical bottom==top
+    warnings.warn('Attempting to set identical bottom==top')
+
+Modifiying the module to use `cbook._warn_extermal`::


Typo: Modifiying; extermal.

QuLogic · 2018-10-20T05:22:59Z

 """

-import warnings
+import matplotlib.cbook as cbook


Move down to below numpy (can also add a blank line after numpy)

QuLogic · 2018-10-20T05:23:28Z


 import logging
-import warnings
+import matplotlib.cbook as cbook


Move down to matplotlib section below.

QuLogic · 2018-10-20T05:23:39Z

 from matplotlib import cbook, rcParams, backend_tools

 import wx
+import logging


Move up to stdlib section of imports.

QuLogic · 2018-10-20T05:25:44Z

+                                     'range. It may be necessary to add an '
+                                     'interval value to the '
+                                     'AutoDateLocator\'s intervald '
+                                     'dictionary. Defaulting to {'


Funny place to break; better to keep {0} together.

QuLogic · 2018-10-20T05:27:21Z

-                            warnings.warn(
-                                "Casting input data from '{0}' to 'float64'"
-                                "for imshow".format(A.dtype))
+                            cbook._warn_external("Casting input data from '{"


Same here; keep {0} together.

hershen · 2018-10-21T05:02:50Z

Thanks for the thorough review @QuLogic. I addressed all your comments (and fixed a few other places where I had the same issues).

I'll squash again when you're happy.

hershen · 2018-10-21T05:36:14Z

Sorry, but what did I do to make the travis/no language test angry?

QuLogic · 2018-10-21T05:40:14Z

Seems to be a bug in Homebrew's NumPy package. Feel free to squash, but it probably won't fix it.

hershen · 2018-10-22T05:00:47Z

OK. Squashed.

The travis issues here seem to be common to all the commits in PRs in the last ~15 hours, so I assume it's not something I'm did wrong.

hershen · 2018-10-27T22:05:24Z

Rebased.

timhoffm

Thanks for working through all the warnings.

timhoffm · 2018-10-28T18:35:08Z

-                warnings.warn("streamed pgf-code does not support raster "
-                              "graphics, consider using the pgf-to-pdf option",
-                              UserWarning, stacklevel=2)
+                _log.warning("streamed pgf-code does not support raster "


Isn't this of the category "things the that the user must change to stop the warning" and thus should be _warn_external?

timhoffm · 2018-10-28T18:36:40Z

    def handle_unknown_event(self, event):
-        warnings.warn('Unhandled message type {0}. {1}'.format(
-            event['type'], event), stacklevel=2)
+        _log.warning('Unhandled message type {0}. {1}'.format(


also _warn_external?

This was changed to _log.warning following @anntzer's suggestion.

In practice this will almost always get called asynchronously by the webserver event handler so "external" will point to the call to mainloop(), which is not really helpful.

timhoffm · 2018-10-28T18:37:47Z

                # looks like they forgot to set the image type drop
                # down, going with the extension.
-                warnings.warn(
+                _log.warning(


improper use -> _warn_external?

This was changed to _log.warning following @anntzer's suggestion.

In practice this will almost always get called through a wx GUI event handler so "external" will point to the call to mainloop(), which is not really helpful.

timhoffm · 2018-10-28T18:37:57Z

            # looks like they forgot to set the image type drop
            # down, going with the extension.
-            warnings.warn(
+            _log.warning(


improper use -> _warn_external?

This was changed to _log.warning following @anntzer's suggestion.

In practice this will almost always get called through a wx GUI event handler so "external" will point to the call to mainloop(), which is not really helpful.

timhoffm · 2018-10-28T18:38:30Z

        rgba = mcolors.to_rgba(color)
    except ValueError:
-        warnings.warn('Ignoring invalid color %r' % color, stacklevel=2)
+        _log.warning('Ignoring invalid color %r' % color)


improper value passed -> _warn_external?

timhoffm · 2018-10-28T18:40:55Z

-            warnings.warn('Matplotlib is currently using %s, which is a '
-                          'non-GUI backend, so cannot show the figure.'
-                          % get_backend())
+                    and warn):


Why the additional spaces? PEP-8 example suggests:

if (this_is_one_thing and that_is_another_thing): do_something()

Good catch!

timhoffm · 2018-10-28T18:47:50Z

-                          "Using bbox_to_anchor=(0,0,1,1) now.")
+            cbook._warn_external("Using the axes or figure transform "
+                                 "requires a bounding box in the respective "
+                                 "coordinates. Using bbox_to_anchor=(0,0,1,"


I wouldn't break the parenthesis for better readability in the code. Would break befrore "Using" here even though the line will get quite short.

timhoffm · 2018-10-28T18:49:05Z

            self._cids = [c1, c2, c3]
        else:
-            warnings.warn(
+            _log.warning(


Is this not also a case the user should do something about (and thus _warn_external).

timhoffm · 2018-10-28T18:49:37Z

        if not has_include_file(
                ext.include_dirs, os.path.join("numpy", "arrayobject.h")):
-            warnings.warn(
+            _log.warning(


User action required -> _warn_extenal.

This doesn't live in the matplotlib package so "external" will point to this very file; showing the source won't help.

hershen · 2018-10-28T19:14:36Z

Thanks @timhoffm for the review!
Some of your suggestions were what I had originally, but @anntzer suggested I change them.
As I'm not familiar enough with the codebase, I'll wait for you two to agree.

anntzer · 2018-10-29T11:09:02Z

Agreed with @timhoffm on some, left comments on others.

…_external. 2) Updated contributions guidelines to use cbook._warn_external.

hershen · 2018-10-29T18:59:12Z

Added @timhoffm's suggestions.

jklymak · 2018-10-29T20:54:26Z

~~@anntzer, can you merge if your review still holds?~~ !!

kbrose · 2018-11-15T00:54:05Z

Can the title of this PR be updated to reflect its contents? I had a hard time finding it because its title ("Added stacklevel=2 to all warnings.warn calls (issue 10643)") leaves out the important part of changing every warning.warn call.

hershen force-pushed the add-stacklevel-to-warnings-fixes-issue-10643 branch from 217e695 to 3c71722 Compare September 3, 2018 02:58

anntzer reviewed Sep 6, 2018

View reviewed changes

hershen force-pushed the add-stacklevel-to-warnings-fixes-issue-10643 branch 2 times, most recently from aa8420c to b00921b Compare October 16, 2018 21:37

anntzer reviewed Oct 17, 2018

View reviewed changes

hershen force-pushed the add-stacklevel-to-warnings-fixes-issue-10643 branch from 416fed2 to 1bece33 Compare October 19, 2018 05:11

anntzer reviewed Oct 19, 2018

View reviewed changes

anntzer approved these changes Oct 19, 2018

View reviewed changes

hershen force-pushed the add-stacklevel-to-warnings-fixes-issue-10643 branch from 6e355a7 to bd01a00 Compare October 19, 2018 17:11

QuLogic reviewed Oct 20, 2018

View reviewed changes

hershen force-pushed the add-stacklevel-to-warnings-fixes-issue-10643 branch 2 times, most recently from 6cf17f3 to 9ecdec3 Compare October 22, 2018 04:18

anntzer added the status: needs rebase label Oct 25, 2018

hershen force-pushed the add-stacklevel-to-warnings-fixes-issue-10643 branch from 9ecdec3 to 226f1d4 Compare October 27, 2018 21:22

timhoffm reviewed Oct 28, 2018

View reviewed changes

timhoffm removed the status: needs rebase label Oct 28, 2018

1) Replaced warnings.warn with either logging.warnings or cbook._warn…

67e57b2

…_external. 2) Updated contributions guidelines to use cbook._warn_external.

hershen force-pushed the add-stacklevel-to-warnings-fixes-issue-10643 branch from 226f1d4 to 67e57b2 Compare October 29, 2018 18:29

timhoffm approved these changes Oct 29, 2018

View reviewed changes

timhoffm added this to the v3.1 milestone Oct 29, 2018

timhoffm merged commit ad779b4 into matplotlib:master Oct 29, 2018

hershen mentioned this pull request Oct 29, 2018

Most warnings calls do not set the stacklevel #10643

Closed

hershen deleted the add-stacklevel-to-warnings-fixes-issue-10643 branch October 29, 2018 21:50

kbrose mentioned this pull request Nov 15, 2018

Warn when "best" loc of legend is used with lots of data #12455

Merged

6 tasks

timhoffm changed the title ~~Added stacklevel=2 to all warnings.warn calls (issue 10643)~~ Change most warnings.warn to - Nov 15, 2018

timhoffm changed the title ~~Change most warnings.warn to -~~ Change most warnings.warn to cook._warn_external or logging.warning Nov 15, 2018

hershen changed the title ~~Change most warnings.warn to cook._warn_external or logging.warning~~ Replace warnings.warn with cbook._warn_external or logging.warning Nov 15, 2018

		##############################################################################
		# FONTS

Uh oh!

Conversation

hershen commented Sep 2, 2018

PR Summary

PR Checklist

Uh oh!

jklymak commented Sep 2, 2018

Uh oh!

jklymak commented Sep 2, 2018

Uh oh!

anntzer commented Sep 3, 2018

Uh oh!

hershen commented Sep 5, 2018

Uh oh!

anntzer commented Sep 5, 2018

Uh oh!

hershen commented Sep 6, 2018

Uh oh!

anntzer commented Sep 6, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

timhoffm commented Oct 13, 2018

Uh oh!

hershen commented Oct 13, 2018

Uh oh!

timhoffm commented Oct 13, 2018

Uh oh!

hershen commented Oct 16, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzer Oct 17, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzer commented Oct 17, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anntzer Oct 17, 2018 •

edited

Loading