Skip to content

gh-151238: Check for _get_resized_exprs failure in _PyPegen_{joined,template}_str#151259

Merged
StanFromIreland merged 1 commit into
python:mainfrom
StanFromIreland:_get_resized_exprs
Jun 11, 2026
Merged

gh-151238: Check for _get_resized_exprs failure in _PyPegen_{joined,template}_str#151259
StanFromIreland merged 1 commit into
python:mainfrom
StanFromIreland:_get_resized_exprs

Conversation

@StanFromIreland

@StanFromIreland StanFromIreland commented Jun 10, 2026 •
edited by bedevere-app Bot
Loading

Copy link
Copy Markdown
Member

There are several cases where _get_resized_exprs can fail and return NULL (all of which are quite rare, however):

cpython/Parser/action_helpers.c

Lines 1341 to 1344 in e2bd50d

const char* quote_str = PyBytes_AsString(a->bytes);
if (quote_str == NULL) {
return NULL;
}

cpython/Parser/action_helpers.c

Lines 1362 to 1368 in e2bd50d

if (asdl_seq_LEN(values) != 2) {
PyErr_Format(PyExc_SystemError,
string_kind == TSTRING
? "unexpected TemplateStr node without debug data in t-string at line %d"
: "unexpected JoinedStr node without debug data in f-string at line %d",
item->lineno);
return NULL;

cpython/Parser/action_helpers.c

Lines 1347 to 1351 in e2bd50d

asdl_expr_seq *seq = _Py_asdl_expr_seq_new(total_items, p->arena);
if (seq == NULL) {
return NULL;
}

cpython/Parser/action_helpers.c

Lines 1401 to 1404 in e2bd50d

resized_exprs = _Py_asdl_expr_seq_new(index, p->arena);
if (resized_exprs == NULL) {
return NULL;
}

And the one triggered by the reproducer in the issue (which I test):

cpython/Parser/action_helpers.c

Lines 1383 to 1386 in e2bd50d

item = _PyPegen_decode_fstring_part(p, is_raw, item, b);
if (item == NULL) {
return NULL;
}

_PyPegen_decode_fstring_part returns NULL because it calls _PyPegen_decode_string which in turn calls warn_invalid_escape_sequence which triggers the failure as it tries to import modules to issue a warning, but gets a TypeError.

@StanFromIreland StanFromIreland changed the title Check for _get_resized_exprs failure in _PyPegen_{joined,template}_str gh-151238: Check for _get_resized_exprs failure in _PyPegen_{joined,template}_str Jun 10, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Jun 10, 2026
Closed
@pablogsal pablogsal added awaiting core review needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes needs backport to 3.15 pre-release feature fixes, bugs and security fixes and removed awaiting merge labels Jun 11, 2026
@pablogsal

pablogsal commented Jun 11, 2026

Copy link
Copy Markdown
Member

Thanks @StanFromIreland

@StanFromIreland StanFromIreland merged commit 4b44b1e into python:main Jun 11, 2026
70 checks passed
@miss-islington-app

miss-islington-app Bot commented Jun 11, 2026

Copy link
Copy Markdown

Thanks @StanFromIreland for the PR 🌮🎉.. I'm working now to backport this PR to: 3.13, 3.14, 3.15.
🐍🍒⛏🤖

@StanFromIreland StanFromIreland deleted the _get_resized_exprs branch June 11, 2026 14:26
@miss-islington-app

miss-islington-app Bot commented Jun 11, 2026

Copy link
Copy Markdown

Sorry, @StanFromIreland, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 4b44b1e1fd654f3a3fefb02ae7fb26456fe33dc3 3.13

@bedevere-app

bedevere-app Bot commented Jun 11, 2026

Copy link
Copy Markdown

GH-151344 is a backport of this pull request to the 3.15 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.15 pre-release feature fixes, bugs and security fixes label Jun 11, 2026
@bedevere-app

bedevere-app Bot commented Jun 11, 2026

Copy link
Copy Markdown

GH-151345 is a backport of this pull request to the 3.14 branch.

@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Jun 11, 2026
@StanFromIreland

StanFromIreland commented Jun 11, 2026

Copy link
Copy Markdown
Member Author

This doesn’t need a backport to 3.13, this was introduced as part of the t-strings implementation. The _get_resized_exprs function is inline there:

cpython/Parser/action_helpers.c

Lines 1359 to 1368 in 1d74300

}
}
else {
resized_exprs = seq;
}
return _PyAST_JoinedStr(resized_exprs, a->lineno, a->col_offset,
b->end_lineno, b->end_col_offset,
p->arena);
}

@StanFromIreland StanFromIreland removed the needs backport to 3.13 bugs and security fixes label Jun 11, 2026
StanFromIreland added a commit that referenced this pull request Jun 11, 2026
@miss-islington @StanFromIreland
[3.14] gh-151238: Check for _get_resized_exprs failure in `_PyPegen…
7c3c0d1 
…_{joined,template}_str` (GH-151259)

(cherry picked from commit 4b44b1e)

Co-authored-by: Stan Ulbrych <stan@python.org>
StanFromIreland added a commit that referenced this pull request Jun 11, 2026
@miss-islington @StanFromIreland
[3.15] gh-151238: Check for _get_resized_exprs failure in `_PyPegen…
f3532d6 
…_{joined,template}_str` (GH-151259) (#151344)

(cherry picked from commit 4b44b1e)

Co-authored-by: Stan Ulbrych <stan@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pablogsal pablogsal pablogsal approved these changes

@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou lysnikolaou is a code owner

Assignees

@StanFromIreland StanFromIreland

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@StanFromIreland @pablogsal