Skip to content

[3.14] Revert "gh-140797: Forbid capturing groups in re.Scanner lexicon patterns (GH-140944) (GH-140982)" #142309

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hugovk merged 2 commits into from
Dec 5, 2025
Merged

[3.14] Revert "gh-140797: Forbid capturing groups in re.Scanner lexicon patterns (GH-140944) (GH-140982)" #142309

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
e149e66
Revert "[3.14] gh-140797: Forbid capturing groups in re.Scanner lexic…
hugovk Dec 5, 2025
dab5629
Add blurb
hugovk Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions Lib/re/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -399,12 +399,9 @@ def __init__(self, lexicon, flags=0):
s = _parser.State()
s.flags = flags
for phrase, action in lexicon:
sub_pattern = _parser.parse(phrase, flags)
if sub_pattern.state.groups != 1:
raise ValueError("Cannot use capturing groups in re.Scanner")
gid = s.opengroup()
p.append(_parser.SubPattern(s, [
(SUBPATTERN, (gid, 0, 0, sub_pattern)),
(SUBPATTERN, (gid, 0, 0, _parser.parse(phrase, flags))),
]))
s.closegroup(gid, p[-1])
p = _parser.SubPattern(s, [(BRANCH, (None, p))])
Expand Down
18 changes: 0 additions & 18 deletions Lib/test/test_re.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1639,24 +1639,6 @@ def s_int(scanner, token): return int(token)
(['sum', 'op=', 3, 'op*', 'foo', 'op+', 312.5,
'op+', 'bar'], ''))

def test_bug_gh140797(self):
# gh140797: Capturing groups are not allowed in re.Scanner

msg = r"Cannot use capturing groups in re\.Scanner"
# Capturing group throws an error
with self.assertRaisesRegex(ValueError, msg):
Scanner([("(a)b", None)])

# Named Group
with self.assertRaisesRegex(ValueError, msg):
Scanner([("(?P<name>a)", None)])

# Non-capturing groups should pass normally
s = Scanner([("(?:a)b", lambda scanner, token: token)])
result, rem = s.scan("ab")
self.assertEqual(result,['ab'])
self.assertEqual(rem,'')

def test_bug_448951(self):
# bug 448951 (similar to 429357, but with single char match)
# (Also test greedy matches.)
Expand Down
3 changes: 3 additions & 0 deletions Misc/NEWS.d/next/Library/2025-12-05-17-58-29.gh-issue-140797.YxB27u.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Revert changes to the undocumented :class:`!re.Scanner` class. Capturing
groups are still allowed for backward compatibility, although using them can
lead to incorrect result. They will be forbidden in future Python versions.
Loading