mruby-regexp: match \Z under the backtracking engine - #7257
Conversation
`\Z` compiles to `RE_EOTNL`, and only the Pike VM executed it. `bt_match()` had no case for the opcode, so it fell to `default:` and failed, and `\Z` never matched once anything in the pattern routed it to the backtracking engine, whether or not the subject ended in a newline: ```ruby "a" =~ /a\Za*?/ # CRuby: 0, mruby: nil "a\n" =~ /a\Z.*?/ # CRuby: 0, mruby: nil "a\n" =~ /(a)\1*\Z/ # CRuby: 0, mruby: nil "ab\n".sub(/b\Z(?=)/, "X") # CRuby: "aX\n", mruby: "ab\n" "a\n" =~ /a\Z/ # 0 in both, the Pike VM has the case ``` A lazy quantifier, a backreference, or a lookaround is enough to get there. It has been so since the backtracking engine landed in 23b2d24, and no test in the gem used `\Z`, so nothing noticed. `RE_EOTNL` was the one opcode the Pike VM handled that `bt_match()` did not; the ones the other way round are the backreference and the lookarounds, which only the backtracker runs. The new case sits beside `RE_EOT` and tests what `add_thread()` tests: the string end, or the position just before a newline that is the last byte. The test exercises `\Z` under both engines, with the lazy quantifier as the shortest way onto the backtracker.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Included review availability: Your plan includes up to 8 reviews per rolling hour; 5 remain after this review. 📝 WalkthroughWalkthroughThe backtracking regexp engine now supports ChangesRegexp end-of-text assertions
Estimated code review effort: 2 (Simple) | ~10 minutes Merge Risk: ⚪ Minimal · up to This change restores expected 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
2 participants
\Zcompiles toRE_EOTNL, and only the Pike VM executed it.bt_match()had no case for the opcode, so it fell to
default:and failed, and\Znever matched once anything in the pattern routed it to the backtracking
engine, whether or not the subject ended in a newline:
A lazy quantifier, a backreference, or a lookaround is enough to get there,
and with #7256 an atomic group. It has been so since the backtracking engine
landed in 23b2d24, and no test in the gem used
\Z, so nothing noticed.RE_EOTNLwas the one opcode the Pike VM handled thatbt_match()did not;the ones the other way round are the backreference and the lookarounds, which
only the backtracker runs.
The fix
One case beside
RE_EOTinbt_match(), testing whatadd_thread()tests:the string end, or the position just before a newline that is the last byte.
The test exercises
\Zunder both engines, with the lazy quantifier as theshortest way onto the backtracker, and pins the two positions it must not
match: before a newline that is not the last byte, and before any other byte.
Testing
build_config/ci/gcc-clang.rbandbuild_config/gcc-asan.rb, run per build sothe counts are attributable:
full-debugbintestcxx_abibyte-stringascii-casegcc-asanThe binary tests pass 122 of 123 under
ci/gcc-clangand 84 of 85 undergcc-asan, one skip each. The only warning any build prints is the-Wreturn-typelinegcc-asangives forsrc/vm.c:4073at-O0, and masterprints the same; this branch does not touch
src/vm.c.Against a
bt_match()without the case, the test block turns red:Fail: Regexp - \Z matches before a trailing newline under both engines (mrbgems: mruby-regexp)Size
Summed
.textoverlibmruby.a,full-coreat-O3, each build from anempty build directory:
libmruby.a.textre_exec.o.textThe whole of it is the new case in
bt_match().mruby-regexpis not in thedefault gembox, so
build_config/default.rbdoes not move.Environment
Details
The size rows use a build config of their own, so that
full-coreat-O3carries no test or debug options:
Compile lines for
mrbgems/mruby-regexp/src/re_exec.cin the builds quotedabove, paths shortened:
Summary by CodeRabbit
\Zregular-expression matching so it recognizes both the true end of a string and the position immediately before a single trailing newline.