Skip to content

[3.6] bpo-31852: Fix segfault caused by using the async soft keyword #4122

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 4 commits into from
Oct 31, 2017
+17 −0
Merged

[3.6] bpo-31852: Fix segfault caused by using the async soft keyword #4122

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
aab5c79
Fix a segfault caused by using continuation lines and the async soft …
pablogsal Oct 25, 2017
7684af5
Reset previous value of tok->async_def if next token is invalid
pablogsal Oct 25, 2017
7518077
Fix spacing
pablogsal Oct 26, 2017
e5b1993
Add News entry
pablogsal Oct 27, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions Lib/test/test_tokenize.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -630,6 +630,11 @@ def test_async(self):
NAME 'async' (1, 0) (1, 5)
OP '=' (1, 6) (1, 7)
NUMBER '1' (1, 8) (1, 9)
""")

self.check_tokenize("async\\", """\
ERRORTOKEN '\\\\' (1, 5) (1, 6)
NAME 'async' (1, 0) (1, 5)
""")

self.check_tokenize("a = (async = 1)", """\
Expand Down
2 changes: 2 additions & 0 deletions Misc/NEWS.d/next/Core and Builtins/2017-10-27-19-18-44.bpo-31852.P_4cVr.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Fix a segmentation fault caused by a combination of the async soft keyword
and continuation lines.
10 changes: 10 additions & 0 deletions Parser/tokenizer.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1563,6 +1563,9 @@ tok_get(struct tok_state *tok, char **p_start, char **p_end)
/* The current token is 'async'.
Look ahead one token.*/

int async_def_prev = tok->async_def;
tok->async_def = 2;

struct tok_state ahead_tok;
char *ahead_tok_start = NULL, *ahead_tok_end = NULL;
int ahead_tok_kind;
Expand All @@ -1581,6 +1584,9 @@ tok_get(struct tok_state *tok, char **p_start, char **p_end)
tok->async_def = 1;
return ASYNC;
}
else{
tok->async_def = async_def_prev;
}
}
}

Expand Down Expand Up @@ -1844,6 +1850,10 @@ tok_get(struct tok_state *tok, char **p_start, char **p_end)
/* Line continuation */
if (c == '\\') {
c = tok_nextc(tok);
if (tok->async_def == 2) {
tok->done = E_SYNTAX;
return ERRORTOKEN;
}
if (c != '\n') {
tok->done = E_LINECONT;
tok->cur = tok->inp;
Expand Down