mruby-regexp: add the regexp form of String#[] and String#slice - #7006
mruby-regexp: add the regexp form of String#[] and String#slice#7006takumin wants to merge 3 commits into
String#[] and String#slice#7006Conversation
`str[re]` and `str.slice(re)` are among the most common ways to pull a substring out of a match, and both raised a `TypeError` naming `Integer`, which says nothing about what is missing. ```ruby "hello"[/l+/] # CRuby: "ll", mruby: TypeError "hello"[/(l+)(o)/, 1] # CRuby: "ll", mruby: TypeError "hello"[/(?<x>l+)/, :x] # CRuby: "ll", mruby: TypeError "hello"[/z/] # CRuby: nil, mruby: TypeError ``` Override `[]` in mrblib and alias `slice` to it, following the shape `split` already uses in this file: the C-defined method is captured as `__aref` first, and every argument list that does not start with a Regexp goes straight back to it, so the non-regexp forms keep their behaviour and their arity errors. The regexp branch goes through `Regexp#match` rather than `match?` even when no capture is asked for, because `$~` has to be set on a failed match too. The capture argument is handed to `MatchData#[]` unchanged, which answers nil for an index past the last group and raises `IndexError` for a name that resolves to no group, matching `rb_reg_nth_match()` and `rb_reg_backref_number()`. `OP_GETIDX` answers an Integer, String or Range index for a String receiver from C without consulting the method table, so those three forms never reach the override. That is only safe because the override delegates them unchanged, and the method comment records it. The same shortcut does not apply to a String receiver reached by an explicit send, which is why the loops in `gsub` and `split` now spell their one-character reads `__aref(0)` rather than paying for a Ruby frame per iteration. `Symbol#[]` and `Symbol#slice` come from `mruby-symbol-ext` and delegate to the String methods, so `sym[re]` follows without an implementation of its own. This gem does not depend on that one, so the test skips where it is absent.
📝 WalkthroughWalkthroughThe regexp gem adds regexp-aware ChangesRegexp indexing APIs
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant String
participant Regexp
participant MatchState
String->>Regexp: match regexp against string
Regexp-->>String: return match and captures
String->>MatchState: update match globals
String-->>String: return match or selected capture
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The previous commit added the regexp form of `String#[]` and took the Limitations item that named it along, which reads as though the regexp form of element reference is now complete in both directions. It is not: the write side still funnels a Regexp through an Integer conversion. ```ruby s = +"hello" s[/l+/] = "X" # CRuby: "heXo", mruby: TypeError s.slice!(/l+/) # CRuby: "ll", mruby: TypeError ``` `String#[]=` is core (`mrb_str_aset_m`) and `String#slice!` is mruby-string-ext (`mrb_str_slice_bang`, registered as `MRB_SYM_B(slice)`, which is why grepping for the method name by hand misses it). Both are left as they are here. The capture form of `str[re, capture] = repl` needs the byte range of a named group, and `MatchData#__byte_begin` takes an Integer only, so covering it means adding C rather than another mrblib override; the destructive side also brings the frozen check and the live reference a MatchData keeps to the string it matched, which belong with the rest of the destructive-method work. State the narrowed limitation instead, so the section says what is missing rather than nothing at all.
The comment above the override says an Integer, String or Range index never reaches it, because OP_GETIDX answers those from C without consulting the method table. That is true of `OP_GETIDX`, and it leaves out `OP_GETIDX0`. A literal zero index compiles to the second opcode, which has a fast path for Array and Hash and none for String, so `str[0]` falls back to a send and does arrive at the override. It answers the same value as before through `__aref`, one Ruby frame and one argument array later: ```console $ ./build/host/bin/mruby -e 's="hello"; ... 1000000 iterations of s[0] ...' 193 ms $ ./build/host/bin/mruby -e 's="hello"; ... 1000000 iterations of s.__aref(0) ...' 48 ms ``` `__aref(0)` is the path `str[0]` took before this branch, so that is the whole of the difference. It is also why the loops in `gsub` and `split` already read their one character that way. Comment only. No behaviour changes.
|
Thank you for taking a look. I’m going to close this PR for now. I’ll reorganize the individual changes, make their dependencies and intended submission order explicit, and then resubmit them as appropriately scoped PRs. Sorry for the churn. |
Problem
String#[]and its aliasString#sliceaccept an Integer, a Range and aString, but not a Regexp. Every regexp form fails with a
TypeErrornamingInteger, which does not hint at what is actually missing.mrb_str_aref()funnels every non-String, non-Range argument through anInteger conversion, and
mruby-regexpnever overrode[]orslice.Solution
Override
[]inmrbgems/mruby-regexp/mrblib/string_regexp.rband aliassliceto it, in the shapesplitalready uses in that file: capture theC-defined method as
__aref, check the arity, then delegate every argumentlist that does not start with a Regexp straight back to C.
The type test is
Regexp === args[0]rather thanis_a?, for the reasonthe
splitand=~comments already give:is_a?is redefinable, so anargument could deny its own type and be read as an index instead, or claim
a type it does not have.
Module#===reads the real type.Behaviour notes
$~is set throughRegexp#match, including to nil on a failed match,which is why
match?is not used even in the no-capture case.MatchData#[]unchanged. A negative indexis normalized, an index past the last group answers nil, and a name that
resolves to no group raises
IndexError. This matches CRuby, whererb_reg_nth_match()answers nil andrb_reg_backref_number()raises.mruby-regexp: fix
MatchData#[]for a negative index and an unknown name #7000 is what made this delegation correct on its own.rb_str_subpat()does."a"[/a/, 1, 2]raisesArgumentError (given 3, expected 1..2)ratherthan quietly ignoring the extra argument.
MatchData#[]reads it throughmrb_as_int(). This is the same latitudethe surrounding gem already takes.
OP_GETIDXinteractionvm_op_getidx()answers an Integer, String or Range index for a receiverwhose class is exactly String directly from
mrb_str_aref(), withoutconsulting the method table, so an override of
String#[]is not visiblefor those three forms. A Regexp index falls through to a send and does
reach the override.
This is safe here because the override only delegates those forms, and the
method comment records the constraint for whoever changes it next.
str[0]is the exception, and the one place this change costs anything.A literal zero index compiles to
OP_GETIDX0, which has a fast path forArray and Hash and none for String, so it always sends and therefore does
reach the override. Measured over 1000000 iterations on the built binary,
s[0]goes from 48 ms to 193 ms, which is the Ruby frame and the argumentarray; 48 ms is what
s.__aref(0)still costs, and__arefis the same Cmethod
str[0]reached before. Every other form is unaffected:str[i]with a non-zero or non-literal index is answered by
OP_GETIDXfrom C asbefore, and
str[i, len],str.slice(...), an explicit send and a Stringsubclass receiver were already sends.
The loops in
gsubandsplitread one character withstr[0]on everyiteration, so they now spell it
__aref(0)and stay on the old path.Symbol
Symbol#[]andSymbol#sliceare defined inmruby-symbol-ext, not incore, and delegate to the String methods, so
sym[re]follows from thischange with no implementation of its own.
mruby-regexpdoes not depend onmruby-symbol-ext, so the Symbol test skips itself where that gem isabsent, the same way the
to_enumtest in this gem skips withoutmruby-enumerator.The write side is left alone
str[re] = replandstr.slice!(re)still raise theTypeErrorthis PRremoves from the read side.
String#[]=is core (mrb_str_aset_m) andString#slice!is mruby-string-ext (mrb_str_slice_bang, registered asMRB_SYM_B(slice)). Covering the capture form of the assignment would meanadding C, since
MatchData#__byte_begintakes an Integer and the name togroup resolution is private to
matchdata_aref(), and the destructive sidebrings the frozen check and the live reference a MatchData keeps to the
string it matched. Both belong with the rest of the destructive-method work,
so the README says so rather than claiming the regexp form is complete.
Tests
mrbgems/mruby-regexp/test/regexp.rbcovers the regexp form and itsslicetwin, a subclass receiver, capture access by index and by name,negative and out of range indices, an unknown name, a failed match, the
$~and$1globals on success and on failure, the delegated non-regexpforms including the subclass path that always goes through the override,
the arity errors, and an argument that lies in
is_a?.mrbgems/mruby-regexp/test/symbol_regexp.rbcoverssym[re]behind themruby-symbol-extguard.rake testpasses.Documentation
Narrows the No regexp form of
String#[]item in the gem README'sLimitations section to the write side, adds the forms to the usage examples,
and rewrites the clause in the
symbol_regexp.rbheader comment thatrecorded
sym[/re/]as still missing.Summary by CodeRabbit
New Features
String#[]andString#slice, including capture and named-capture extraction.Documentation
Tests