Repository navigation
Closed
Labels
Release criticalFor bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.status: upstream fix requiredtopic: text/mathtext
Milestone
Description
Bug summary
Pyparsing 3.1.0 was released today and something has changed which causes quite a bit of test failures.
List of changes https://github.com/pyparsing/pyparsing/releases/tag/3.1.0 (not obvious to me what caused it).
Code for reproduction
Run the testsActual outcome
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[hspace without value]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[hspace with invalid value]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[function without space]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[accent without space]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[frac without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[frac with empty parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[binom without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[binom with empty parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[genfrac without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[genfrac with empty parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[sqrt without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[sqrt with invalid value]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[overline without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[overline with empty parameter]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[left with invalid delimiter]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[right with invalid delimiter]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[unclosed parentheses with sizing]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[unclosed parentheses without sizing]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[dfrac without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[dfrac with empty parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[overset without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[underset without parameters]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[unknown symbol]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[double superscript]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[double subscript]
FAILED lib/matplotlib/tests/test_mathtext.py::test_mathtext_exceptions[super on sub without braces]
FAILED lib/matplotlib/tests/test_text.py::test_parse_math - AssertionError: R...
FAILED lib/matplotlib/tests/test_text.py::test_parse_math_rcparams - Assertio...
Expected outcome
No failures
Additional information
I guess a two-stage fix:
- Pin the version to
<3.1 - Fix the actual problem and unpin
Operating system
No response
Matplotlib Version
main-branch
Matplotlib Backend
No response
Python version
No response
Jupyter version
No response
Installation
None
Metadata
Metadata
Assignees
Labels
Release criticalFor bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.For bugs that make the library unusable (segfaults, incorrect plots, etc) and major regressions.status: upstream fix requiredtopic: text/mathtext
Activity
ksunden commentedon Jun 19, 2023
Possibly related to pyparsing/pyparsing#474
Have not proven it, but that has to do with handling escape sequences and the errors appear to all be related to
\I'll also note that pyparsing handling of
\is (or at least has been) problematic for python 3.12 support, so could be a different change related to that as well...ksunden commentedon Jun 24, 2023
pyparsing/pyparsing@4cd691f#diff-651b49e05968bd7f42614f05b7e9d94b30f52db27eb692ce20ef87e9799c624fR4449-R4453
Okay, I have bisected/tested that this change is what is causing our test failures.
I will note that it is only the error messages that change (and so anything that does not raise will still work)
Essentially we were relying on the "pass through" of error messages to provide actually useful error messages (and testing that we got them)
@ptmcg, perhaps you can advise us on how to respond? It is unclear to me (at least quickly) if there is a good way to get the errors from further down when we want it.
ptmcg commentedon Jun 24, 2023
I am not able to readily set up my own development environment for running these tests. Can someone please post (or link to a pastebin) a full test output, including expected and observed values?
I think I was striving for improving the error messages as part of this effort.
oscargus commentedon Jun 24, 2023
These are the failing tests:
matplotlib/lib/matplotlib/tests/test_mathtext.py
Lines 279 to 346 in 060992a
Here is the full traceback of one of them:
All other looks very similar.
oscargus commentedon Jun 24, 2023
That particular exception we raise here:
matplotlib/lib/matplotlib/_mathtext.py
Line 2294 in 060992a
ksunden commentedon Jun 24, 2023
Essentially we had been getting informative error messages and now just get "Expected token (at char 0)" for any failure to parse, which is not informative. Reverting the try/except linked above fixes our tests
ptmcg commentedon Jun 25, 2023
I'm still trying to setup a development environment of my own following the matplotlib dev guide, but now getting TypeErrors with FT2Font something something.
What happens if you stop calling
set_name("token")onp.token? Are there other p attributes that are also emitting unhelpful names? Something like this?Or possibly change the if statement to any attribute whose value is a pyparsing Literal or Keyword, which would probably give clearer-looking exceptions than using your internal variable name:
ptmcg commentedon Jun 26, 2023
tl;dr - This change has been in staging and pre-releases for over a year before being released, and I announced (in Twitter and comp.lang.python.announce) the coming 3.1.0 release of pyparsing in alpha and beta releases in March, April, and May of this year. In general, I feel that this particular change improves logging for expressions that contain other expressions when the container expression has been assigned a custom name, so I do not plan on reverting this change. It saddens me that there is this post-release surprise issue with a pyparsing user as significant as matplotlib.
As a workaround, you can see what the effects are by not calling
setNameon selected expressions in your parser. Or as you have done for now, pin to a 3.0.x version of pyparsing.I also have tried to spin up my own development environment of matplotlib, so that I could run your tests against my in-development pyparsing, but with no success (both Windows and Ubuntu). I welcome any assistance in doing this, so that I can run matplotlib regression tests for myself before releasing new versions of pyparsing.
Some possible changes I could make in a future pyparsing release:
ParserElementto adjust the verbosity of error messages, so that you could get an error message that included the contained element's custom name or default name as well, something like "Expected token ('\hspace{space}')" - I am most optimistic that this would be a good addition, and would serve to address the issue you have with your tests.setNameby callingsetNamewith aNonevalue (I have looked briefly at this, I am not sure this will have a very clean implementation, but I'll be the first to admit it would not be the first unclean thing in pyparsing)I have pressing deadlines at work for the next month, so this future release will not be for several weeks yet. Also, I would like to give 3.1.0 some more time in the field, to get any other fallout from other users.
@ksunden - I tested this version of pyparsing with Python 3.12, so I believe I've addressed any issues with
\. Note that\inQuotedStringis parsed a little differently (and to my mind more sanely) in this new version. If there are still problems with\in 3.12, it points to gaps in my test suite, which I would be happy to learn about.ksunden commentedon Jun 26, 2023
The guess at
QuotedString/\behavior was an initial guess upon reading the changelog without looking at the code, I suspect that is not an issue (though will check in the relatively near term on python 3.12 to be sure...) certainly I'm convinced it is not actually the cause for what we are seeing on python <= 3.11 nowI will focus discussion in the interim on one particular path, as I suspect that once we find a path for one, the rest will fall into line with relatively trivial modifications.
So let's focus on
unknown_symbol(the test case for which isr"$\sinx$")Unknown symbol is
Regex(r"\\[A-Za-z]*)("name")(i.e. a alpha word preceded by a\, such that it looks like a LaTeX command, but is not one we support in Mathtext)It's parsing function always raises:
As such, our expected behavior is that when such a symbol is parsed, we should get an error message that points directly to the unknown symbol and says
Unknown symbolin the error message explanation.A simple test case, extracting from the pytest markup:
This should raise a
ValueErrorwhich prints out the explanation including the phraseUnknown symbol. This is what it does with pyparsing 3.0.9:With pyparsing 3.1.0 and no modifications to matplotlib this gives:
Which does not include the key phrase we are looking for in out tests, and is generally not a helpful error message in our usage.
Excluding
tokenfrom havingsetNamecalled it gets better, but does not go all the way:That is the internal definition of
token, which at least is progress, but does not get us to where we are seeing the error message we were looking for (pretty much any of the other suggestions such as keying off of Literal/Keyword etc had similar results)If I modify pyparsing a little more minimally such that instead of:
I do (i.e. do not reset the error message for Forward instances):
Then the matplotlib tests pass (even without the modification so that
tokendoes not getsetNamecalled). If this would be an acceptable upstream change, I will happily PR it, make sure our pins are just!=3.1.0and call this resolved. Though I do understand if that is still not ideal on your end. It does not seem to cause any test failures that I can tell (I am having some trouble regarding the diagrams portions of pyparsing which are erroring the full test suite, so ran the non-diagram tests as individual files, had some diagram-related failures still, so may have missed something)The "depth" argument for "explain" does not seem to affect the output of these messages (well, other than adding a trailing line of
matplotlib._mathtext.Parserto the output) I could see that potentially giving the best of both worlds... though have not looked into it closely enough to know if that is easy or not (certainly would still require us to change the explain call).RE your dev setup for matplotlib, happy to help if I can, but not sure what could be going wrong.
We have some docs for getting started that may be of help: https://matplotlib.org/stable/devel/development_setup.html
RE preventing surprises post-release in the future, we do have a test setup which pulls nightly builds of numpy/pandas (from here). We could probably modify that to get either at least
--prefrom pypi.org or get you in touch with the people who run those nightlies and get your CI set up to upload there, so that we (and other downstreams, potentially) can install from there (which would allow potentially even further advanced warning... I'm happy to PR it if you want to go that route and we get the green light from them, though you will probably need to handle generating an upload token/adding it as a github secret)ksunden commentedon Jun 26, 2023
Also, we newly have github codespaces set up to get a dev setup with VSCode through the browser, while we are still working on some of the docs for using it, the config files are there, you may be able to get something good enough to run some quick tests on through that.
ksunden commentedon Jun 26, 2023
Perhaps the other heuristic which may be more in line with what you were thinking would happen by not calling
setNamewould be to do:Which would prevent autogen names from showing (unless they are the leaf).
ptmcg commentedon Jun 26, 2023
Your code fragment also leads me to another thought, since the issue seems not to be so much about set_name (as I originally thought), but about the custom message in your raised exception. What if you raised your ParseFatalExceptions as a subclass of ParseFatalException, and I only do the substitution if the exception is of a type known to pyparsing (i.e., ParseBaseException, ParseException, ParseFatalExecption, or ParseSyntaxException). I've seen this exception style (don't raise library exceptions, raise your own subclass of them) recommended in other sources.
7 remaining items
graphics/py-matplotlib: Unbreak via pyparsing dep hack
jklymak commentedon Jul 27, 2023
This is marked 3.7.2 RC. Is there a path forward, and if so, from whose side does this need to be fixed?
ksunden commentedon Jul 27, 2023
I have pyparsing/pyparsing#493 open. Once that is merged I'll adjust the pinning to either
!=3.1.0or allow it and xfail the error message tests with that particular version of pyparsing (it works just gives less informative error messages than the ones we test for)ptmcg commentedon Jul 29, 2023
I was working with 3.1.0 today on a new parser for work, and the exception messages are really pretty bad! I'll definitely get on this over the weekend.
ptmcg commentedon Jul 29, 2023
Merged your PR (and a few others, and addressing some other reported issues). Thanks for your work on this.
Update pyparsing from 2.4.7 to 3.0.9
ptmcg commentedon Jul 30, 2023
PR merged and released in pyparsing 3.1.1 - let me know if there are further issues
sin-ack commentedon Aug 24, 2023
It appears that this is still broken, unfortunately :( I ran tests with pyparsing 3.1.1 on Gentoo through Portage, and got the following failure log:
pytest-failures.log
ksunden commentedon Aug 24, 2023
@sin-ack, this did require some changes on our end, but is fixed on main/3.8.0rc1/3.7.x backport branch, just we haven't got 3.7.3 or 3.8.0 (final) out yet.
Ref #26432 (for the 3.7 backport)
ptmcg commentedon Sep 9, 2024
FYI - I've updated pyparsing's test suite to include tests from matplotlib's test suite. Just trying to stay ahead of incidents like this.
graphics/py-matplotlib: Unbreak via pyparsing dep hack
graphics/py-matplotlib: Unbreak via pyparsing dep hack
graphics/py-matplotlib: Unbreak via pyparsing dep hack