Skip to content

[3.14] gh-140797: Forbid capturing groups in re.Scanner lexicon patterns (GH-140944) #140982

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
serhiy-storchaka merged 1 commit into from
Nov 4, 2025
Merged

[3.14] gh-140797: Forbid capturing groups in re.Scanner lexicon patterns (GH-140944) #140982

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
gh-140797: Forbid capturing groups in re.Scanner lexicon patterns (GH… 
…-140944)

(cherry picked from commit fa9c3ee)

Co-authored-by: Abhishek Tiwari <Abhi210@users.noreply.github.com>
  • Loading branch information
@Abhi210 @miss-islington
Abhi210 authored and miss-islington committed Nov 4, 2025
commit 7b4aa48dbc08bfd775d21ec45c5ac02c779aa8d2
5 changes: 4 additions & 1 deletion Lib/re/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -399,9 +399,12 @@ def __init__(self, lexicon, flags=0):
s = _parser.State()
s.flags = flags
for phrase, action in lexicon:
sub_pattern = _parser.parse(phrase, flags)
if sub_pattern.state.groups != 1:
raise ValueError("Cannot use capturing groups in re.Scanner")
gid = s.opengroup()
p.append(_parser.SubPattern(s, [
(SUBPATTERN, (gid, 0, 0, _parser.parse(phrase, flags))),
(SUBPATTERN, (gid, 0, 0, sub_pattern)),
]))
s.closegroup(gid, p[-1])
p = _parser.SubPattern(s, [(BRANCH, (None, p))])
Expand Down
18 changes: 18 additions & 0 deletions Lib/test/test_re.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1639,6 +1639,24 @@ def s_int(scanner, token): return int(token)
(['sum', 'op=', 3, 'op*', 'foo', 'op+', 312.5,
'op+', 'bar'], ''))

def test_bug_gh140797(self):
# gh140797: Capturing groups are not allowed in re.Scanner

msg = r"Cannot use capturing groups in re\.Scanner"
# Capturing group throws an error
with self.assertRaisesRegex(ValueError, msg):
Scanner([("(a)b", None)])

# Named Group
with self.assertRaisesRegex(ValueError, msg):
Scanner([("(?P<name>a)", None)])

# Non-capturing groups should pass normally
s = Scanner([("(?:a)b", lambda scanner, token: token)])
result, rem = s.scan("ab")
self.assertEqual(result,['ab'])
self.assertEqual(rem,'')

def test_bug_448951(self):
# bug 448951 (similar to 429357, but with single char match)
# (Also test greedy matches.)
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Library/2025-11-03-16-23-54.gh-issue-140797.DuFEeR.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
The undocumented :class:`!re.Scanner` class now forbids regular expressions containing capturing groups in its lexicon patterns. Patterns using capturing groups could
previously lead to crashes with segmentation fault. Use non-capturing groups (?:...) instead.
Loading