mruby-regexp: write the disabled flags in Regexp#to_s - #7062
Merged
Conversation
`Regexp#to_s` named only the flags that were on, so the form it produced
did not carry the ones that were off. Interpolating it into another
pattern therefore let the enclosing flags reach the embedded source, which
is the reason CRuby spells them out. Both `to_s` and `Regexp#inspect` also
wrote the letters in the `i`, `m`, `x` order rather than Ruby's `m`, `i`,
`x`.
```ruby
/a/i.to_s # CRuby: "(?i-mx:a)", mruby: "(?i:a)"
/a/im.inspect # CRuby: "/a/mi", mruby: "/a/im"
/#{/a/}b/i.match?("Ab") # CRuby: false, mruby: true
```
`to_s` now emits the flags that are off after a `-`, and drops that run
only when none of them are. The letters of both forms come from one shared
table, so the two orders cannot drift apart again.
The new form has to recompile, and `parse_inline_flags()` rejected it:
every `to_s` of a pattern that is not extended now carries a `-x`, and an
`x` raised `inline extended mode (?x) is not supported` wherever it stood.
A disabled `x` is accepted and dropped instead. That is exact when the
enclosing pattern is not extended, since the flag is already off there.
Inside a pattern that is extended it is not, because `preprocess_pattern()`
removes the whitespace before the parser runs and a scoped `-x` cannot
bring it back: `Regexp.new("(?-x:a b)", Regexp::EXTENDED)` matches "ab"
where CRuby matches "a b". An enabled `x` keeps raising. The gem's
Limitations section now records both.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
📝 WalkthroughWalkthroughRegexp inline option parsing now accepts ChangesRegexp option handling and serialization
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Regexp#to_swrites only the flags that are on (regexp.c:523-525), so the(?...)form it produces says nothing about the ones that are off. CRuby namesthose after a
-, and that is what keeps the form meaningful once it isinterpolated into another pattern: without it, the enclosing pattern's flags
reach the embedded source.
Regexp#inspect(regexp.c:541-543) repeats thesame run of tests and writes the letters in the
i,m,xorder, where Rubywrites
m,i,x.Nothing raises. The missing part turns into a different pattern only once the
result is used, which is where it is hardest to attribute:
Interpolation goes through
to_s: the compiler concatenates the parts of aninterpolated literal with
OP_STRCATand hands the result toRegexp.compile,so
/#{/a/}b/icompiles the source(?:a)bunderRegexp::IGNORECASEand theais matched case-insensitively.Regexp.new(re.to_s)loses the flags thesame way.
Fix
Emit the flags that are off after a
-, dropping that run only when none ofthem are, and take the letters of both forms from one shared table so the two
orders cannot drift apart again.
The new form has to recompile, and
parse_inline_flags()rejected it. Everyto_sof a pattern that is not extended now carries a-x, while thexbranch (
re_compile.c:607-610) raisedinline extended mode (?x) is not supportedbefore it consultednegate, so it rejected anxin the disabledrun exactly like an enabled one. With the
to_schange alone, everyinterpolation of a Regexp would start raising
RegexpError.So a disabled
xis accepted and dropped. That is exact whenever the enclosingpattern is not extended, since the flag is already off there, which is the case
for every string
to_sproduces for such a pattern. Inside a pattern that isitself extended it is not exact:
preprocess_pattern()strips the whitespacebefore the parser runs, so a scoped
-xcannot bring it back.An enabled
xkeeps raising, as it must: extended mode is applied to the wholepattern before it is parsed and cannot be scoped inline at all. Both limits are
now in the gem's Limitations section, which said nothing about inline extended
mode before.
This does not repair the round trip for an extended Regexp, which was already
broken:
/a/x.to_swas"(?x:a)"and is now"(?x-mi:a)", and neitherrecompiles.
Tests
mrbgems/mruby-regexp/test/regexp.rb.Regexp#to_sgains the disabled run in all four existing assertions, plus theall-flags case where the
-run is dropped, a round trip throughRegexp.new, and a case pinning that the flags do not leak in eitherdirection.
Regexp#inspectgains two multi-flag cases, which are what theorder change is visible in. A new
Regexp#to_s - interpolationblock coversthe embedded Regexp keeping its own flags and picking up none of the outer
ones.
Regexp - inline options (?i) / (?i:...)gains the disabledxin both thescoped and the toggle position, an enabled
xin the scoped form next to thetoggle form already asserted there, and the inexact case inside an extended
pattern.
Regexp extended mode (x flag)has itsto_sexpectation updated.Verified on
x86_64-linux:over the eight flag combinations, each checked for
to_s,inspect, sixsubjects, and a round trip through
Regexp.new(re.to_s); every pair ofinner and outer flag combinations for an interpolated Regexp; and 24
hand-written inline flag runs, on and off, scoped and toggled, over the same
eight combinations. 366 cases disagreed before this change and 268 after,
with no case disagreeing that did not disagree before.
mode, which is the limitation above, and 8 are
(?-:a b), an empty disabledrun that mruby rejects as
undefined (?...) sequenceand CRuby accepts.That last one is an unrelated pre-existing gap and is left as is:
to_salways names at least one flag, so it never produces that form.
rake test: 1988 total, 1969 OK, 0 KO, 0 crash, and bintest 105 OK.MRB_INT32build with clang and-Wall -Wextra: 1890 total, 1879 OK,0 KO, 0 crash, bintest 105 OK, and no new warning from any
mruby-regexpfile (
regexp.calready emits four-Wunused-parameter).Summary by CodeRabbit
New Features
?-x).Bug Fixes
Documentation