Skip to content

Commit ee94b5b

Browse files
authored
sre_engine: resume the tail match at prefix_skip, not past the whole prefix (#8473)
`search_info_literal`'s len>1 arm reset the tail-match cursor by advancing one character past the matched prefix. That position equals the INFO block's prefix_skip boundary only when the prefix ends where the skip does; for a pattern like `ab(cd)`, `_get_literal_prefix` reports prefix_len=4 with prefix_skip=2, so the tail resumed two characters too far. `state.cursor` already holds `req.start + skip`, which is what `sre_lib.h` SRE(search) computes as `ptr - (prefix_len - prefix_skip - 1)`, and which the len==1 arm above already uses. Take it unconditionally. Searching "xabcdcd" for `ab(cd)` returned span (1, 7) instead of (1, 5); over a 600-case corpus stamped with CPython's own answers, 94 cases disagreed before this change and none after. The wrong resume also dropped matches outright, not only widened them. Assisted-by: Claude
1 parent 15f8590 commit ee94b5b

2 files changed

Lines changed: 20 additions & 5 deletions

File tree

‎crates/sre_engine/src/engine.rs‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -998,12 +998,12 @@ fn search_info_literal<const LITERAL: bool, S: StrDrive>(
998998
return true;
999999
}
10001000

1001+
// `state.cursor` is `req.start + skip`, the position the
1002+
// tail match resumes from; advancing past the prefix
1003+
// instead would resume at `req.start + len` and only
1004+
// agree when the prefix ends at the skip boundary.
10011005
let mut next_ctx = ctx;
1002-
if skip != 0 {
1003-
next_ctx.advance_char::<S>();
1004-
} else {
1005-
next_ctx.cursor = state.cursor;
1006-
}
1006+
next_ctx.cursor = state.cursor;
10071007

10081008
if _match(req, state, next_ctx) {
10091009
return true;

‎crates/sre_engine/tests/tests.rs‎

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -252,4 +252,19 @@ mod tests {
252252
#[rustfmt::skip] let p = Pattern { pattern: "\u{e0}+", code: &[14, 4, 0, 1, 4294967295, 24, 6, 1, 4294967295, 16, 224, 1, 1] };
253253
// END GENERATED
254254
}
255+
256+
#[test]
257+
fn search_literal_prefix_longer_than_skip() {
258+
// The INFO block carries prefix_len=4 and prefix_skip=2, so the tail
259+
// match has to resume at the skip boundary rather than past the whole
260+
// prefix.
261+
// pattern p = re.compile('ab(cd)')
262+
// START GENERATED by generate_tests.py
263+
#[rustfmt::skip] let p = Pattern { pattern: "ab(cd)", code: &[14, 14, 1, 4, 4, 4, 2, 97, 98, 99, 100, 0, 0, 0, 0, 16, 97, 16, 98, 17, 0, 16, 99, 16, 100, 17, 1, 1] };
264+
// END GENERATED
265+
let (req, mut state) = p.state("xabcdcd");
266+
assert!(state.search(req));
267+
assert_eq!(state.start, 1);
268+
assert_eq!(state.cursor.position, 5);
269+
}
255270
}

0 commit comments

Comments
 (0)