Speed up parser.parse() (ISO-8601 fast path, faster tokenizer) with no behaviour change - #1624

Summary of changes

Speeds up parser.parse() without changing its behaviour, and adds tests that check this against a frozen copy of the current parser.

 parse(timestr)
-  default = now()                      # every call
   _parse(timestr)
+    if ISO-8601 and stock parserinfo and not dayfirst/fuzzy:
+      return _try_parse_iso(timestr)   # regex → the same _result
-    _timelex.split(timestr)            # StringIO.read(1) per char
+    _timelex.split(timestr)            # same state machine, over the str
     ... unchanged ...
+  default = now() only if the string lacks year, month or day
   _build_naive / _build_tzaware        # unchanged

• ISO-8601 fast path: strict YYYY-MM-DD[( |T)HH:MM[:SS[.ffffff]][Z|±HH[[:]MM]]] strings build the _result the general path would build, then go through the unchanged _build_naive / _build_tzaware. Out-of-range values are not checked separately; datetime rejects them with the same messages as before. Custom parserinfo subclasses, dayfirst, fuzzy and non-str input always take the general path.
• Tokenizer: _timelex.split() runs the existing get_token() state machine directly over the string. _timelex subclasses and streams keep the old path.
• Smaller costs: datetime.now() is only called when the string doesn't give a full date, and _resultbase.__len__ no longer sums a generator.

This covers the ISO fast path and tokenizer work proposed in #1519. It overlaps with #1609 and #1613 (opened 14 Oct). The difference here is how the tests check that behaviour stays the same.

Evidence

parse(), µs per call (CPython 3.12, macOS arm64):

Behaviour preservation:

• tests/_reference_parser.py is a verbatim copy of _parser.py at 2642afa (only its two relative imports changed). It is test-only.
• tests/test_parser_equivalence.py uses Hypothesis to check that _timelex.split tokens (str, bytes and streams) and full parse() results match the reference. It compares values, utcoffset(), exception type and args, and fuzzy_with_tokens output. Inputs are real-world strftime formats, random fragments and Unicode digits/letters. Options vary over dayfirst, yearfirst, ignoretz, fuzzy, fuzzy_with_tokens, default, tzinfos and parserinfo.
• tests/test_parser_fastpath.py checks parse() with and without the fast path on inputs built around its grammar boundaries, including a parserinfo subclass with a different JUMP list. It also checks that common ISO strings skip the tokenizer.
• I checked that the tests can catch bugs by injecting 24 deliberate bugs into the new code; all 24 were caught. A separate run of 100,000 cases per check found no difference from the reference.

Full suite: CPython 2.7, 3.5, 3.6, 3.7, 3.12 and PyPy 2.7 all pass. Python 2.7 needed one test-only setting: Hypothesis 4.x (the last release for Python 2) can spend minutes in its target phase, so these tests skip that phase. They don't call target(), so nothing is lost.

Merge Danger

Door: two-way. The changes are internal to _parser.py with no API change, and a revert is clean.

Blast Radius: wide. parse() is used throughout the ecosystem (pandas calls _timelex.split directly), which is why the tests compare against the old implementation rather than against hand-picked cases.

The reference copy adds about 1,600 lines to tests/. If you would rather not carry it, the equivalence tests could compare against an installed release instead.

Pull Request Checklist

• Changes have tests
• Authors have been added to AUTHORS.md
• News fragment added in changelog.d. See CONTRIBUTING.md for details

🤖 Generated with Claude Code