mruby-regexp: accept a Regexp in String#index, #partition, #start_with? and their siblings - #7075
Merged
Merged
Conversation
Both reach a C implementation that converts the argument to a String, so a Regexp is a `TypeError` where CRuby searches with it: ```ruby "abc".index(/b/) # CRuby: 1, mruby: TypeError "abcabc".rindex(/b/) # CRuby: 4, mruby: TypeError "abcabc".rindex(/bca/, 1) # CRuby: 1, mruby: TypeError ``` Override both in the gem's mrblib, alongside `String#[]` and `#[]=` and with the same guard: `Regexp === args[0]` reads the real type where `is_a?` is redefinable, and every other argument form goes back to the C method under a private alias, keeping the C arity check and the C error messages. `index` hands its position argument to `Regexp#match` unexamined. The two normalize a position the same way and read it with the same `mrb_get_args()` conversion, so a negative one counts back from the end and one that lands outside the subject answers nil. `rindex` wants the last match that starts at or before its position, and the match may run past that position, which is why `"abcabc".rindex(/bca/, 1)` is 1. The engine has no backward search, so `__regexp_rsearch` walks the subject from the start and keeps the last match that qualifies: linear in the number of positions a match starts at, where the backward search CRuby hands to Onig is not. Each step resumes one character past the match start rather than at the match end, without which overlapping matches stay invisible and `"aaa".rindex(/aa/)` answers 0 instead of 1. Both search through `match` rather than `match?` so that `$~` and the names derived from it are published, including the clearing a failed match does. CRuby sets them for both methods and clears them after a miss. The walk in `__regexp_rsearch` ends on a failed match or on one past its limit, so it republishes the match it settled on with `__set_globals`.
The byte-offset half of the pair `index` and `rindex` just gained, rejecting a Regexp for the same reason and answering the same two searches read in the other space: ```ruby "abc".byteindex(/b/) # CRuby: 1, mruby: TypeError "abc".byterindex(/b/) # CRuby: 1, mruby: TypeError ``` `MatchData#begin` reports character offsets, which is the convention the rest of the gem follows, so these two read `__byte_begin` instead. On a build without MRB_UTF8_STRING the two spaces coincide and the pairs answer the same number, which is what makes these nearly free once `index` and `rindex` exist; on one with it they part company, and a position argument is bytes here where it is characters there. `byteindex` searches through `Regexp#__byte_match`, whose position is already a byte offset. That method does no range check, so the ends of the subject are checked here: both are a miss, as they are for `mrb_str_byteindex_m()`. An offset that lands inside a character is not an error, because the C method does not check for one either. `byterindex` shares `__regexp_rsearch` with `rindex` and differs only in the space its limit is compared in.
Both come from mruby-string-ext, which this gem depends on, and both convert their argument to a String: ```ruby "abc".partition(/b/) # CRuby: ["a", "b", "c"], mruby: TypeError "abc".rpartition(/b/) # CRuby: ["a", "b", "c"], mruby: TypeError ``` The three pieces come straight from the match, so `pre_match`, `[0]` and `post_match` are the whole of the matched case. The row worth naming is the unmatched one, where the subject stays whole and the two methods put it at opposite ends: ```ruby "abc".partition(/z/) # ["abc", "", ""] "abc".rpartition(/z/) # ["", "", "abc"] ``` That copy is a plain String even when the receiver is a String subclass, as `mrb_str_dup()` in the C implementation and `str_duplicate(rb_cString, str)` in CRuby both hand back. `rpartition` wants the last match, overlapping ones included, so it shares `__regexp_rsearch` with `rindex` and passes the end of the subject as its limit.
The last of the family, and the only one that takes several patterns:
```ruby
"abc".start_with?(/a/) # CRuby: true, mruby: TypeError
"abc".start_with?("x", /a/) # CRuby: true, mruby: TypeError
```
The override reads the arguments left to right and hands each non-regexp one
to the C method under a private alias, one at a time, so a String keeps the C
comparison and its error and the arguments are still answered in order. An
argument after the one that answers is never looked at, as in CRuby.
A regexp is anchored at the start rather than searched for, while
`Regexp#match` searches forward from its position, so a successful match is
not the answer on its own and the override checks `begin(0) == 0`. The engine
matches leftmost, so a pattern that can match at 0 does, which makes that
check the anchored answer rather than an approximation of it:
```ruby
"abc".start_with?(/b/) # false, though /b/ matches at 1
"abc\ndef".start_with?(/^d/) # false, though /^d/ matches at 4
```
CRuby leaves no match behind for either of those, so a match that starts
further along is cleared rather than published. A non-regexp argument does
not touch the match globals at all, which falls out of leaving it to C.
`String#end_with?` gets nothing here: CRuby rejects a Regexp there too, and
so does this build, so the two already agree.
📝 WalkthroughWalkthrough
ChangesRegexp-aware String APIs
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant String
participant Regexp
participant MatchGlobals
String->>Regexp: Search with normalized position
Regexp-->>String: Return match position and length
String->>MatchGlobals: Update or restore match globals
String-->>String: Return character or byte offset
Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The `Ruby API` section of the gem's README lists the String methods that take a Regexp here, and the seven added by the commits before this one were missing from it. Add them, and a limitation entry for the one cost the addition carries. `rindex`, `byterindex` and `rpartition` share `__regexp_rsearch`, which walks the subject from the start because the engine has no backward search. That is linear in the number of positions a match starts at, where the backward search CRuby hands to Onig is not, so it belongs next to the other limitations rather than only in a comment in the mrblib file.
This was referenced Aug 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Seven
Stringmethods take a Regexp in CRuby and reach a C implementation herethat converts the argument to a String, so a Regexp is a
TypeError. Four arecore (
index,rindex,byteindexandbyterindex, insrc/string.c); theother three come from mruby-string-ext (
partition,rpartitionandstart_with?), which mruby-regexp already depends on, so an override in thegem's mrblib reaches both sets.
Nothing is unreachable today, since
=~andString#[]cover the same ground.What is missing is the spelling CRuby code uses, which means code carried over
raises instead of running.
This finishes the family.
String#[]and#slicearrived in 46d88a8,#sub!and#gsub!in #7061,#[]=and#slice!in #7063; none of the threetouched any of these seven.
String#end_with?is deliberately not in the list:CRuby rejects a Regexp there too, and so does this build, so the two already
agree.
Shape of the change
mrbgems/mruby-regexp/mrblib/string_regexp.rbalready had the pattern to copy.Each C method is captured under a private alias before the override replaces
it, and the override tests the argument with
Regexp === argrather thanis_a?, which is redefinable: anything that is not a Regexp goes back to thecaptured C method untouched, so it keeps the C arity check and the C error
messages. The note at the top of that file sets out how far that guard goes and
where it stops.
One commit per pair, in the order the methods depend on each other, and a
last one for the README: the seven join the list of String methods that take
a Regexp here, and the walk described below joins the limitations.
indexhands its position argument toRegexp#matchunexamined: the twonormalize a position the same way and read it with the same
mrb_get_args()conversion.
rindexwants the last match that starts at or before its position, and thematch may run past that position, so
"abcabc".rindex(/bca/, 1)is 1. Theengine has no backward search, so
__regexp_rsearchwalks the subject from thestart and keeps the last match that qualifies. That is linear in the number of
positions a match starts at, where the backward search CRuby hands to Onig is
not, which is the one cost worth naming here. Each step resumes one character
past the match start rather than at the match end, without which overlapping
matches stay invisible and
"aaa".rindex(/aa/)answers 0 instead of 1.byteindexandbyterindexare the same two searches read in byte space.MatchData#beginreports character offsets, which is the convention the rest ofthe gem follows, so these two read
__byte_begininstead. On a build withoutMRB_UTF8_STRINGthe two spaces coincide and each pair answers the same number.partitionandrpartitionbuild their three pieces from the match, so the rowworth naming is the one with no match, where the subject stays whole and the two
put it at opposite ends. The copy is a plain String even for a String subclass
receiver, as
mrb_str_dup()and CRuby'sstr_duplicate(rb_cString, str)bothhand back.
rpartitionshares__regexp_rsearchwithrindex.start_with?reads its arguments left to right and hands each non-regexp one tothe C method one at a time, so a String keeps the C comparison and its error. A
regexp is anchored at the start rather than searched for, so the override checks
begin(0) == 0; the engine matches leftmost, so a pattern that can match at 0does, which makes that check the anchored answer rather than an approximation of
it.
"abc".start_with?(/b/)and"abc\ndef".start_with?(/^d/)are both false.Two details the existing overrides settle already, and these follow:
matchand notmatch?, so$~and the names derived from it are published, including the clearing a failed
match does. CRuby sets them for all seven and clears them after a miss. That
extends to the misses that are not a failed search: a position outside the
subject, a match that starts past what
rindexasked for, and a matchstart_with?refuses for starting later than 0 all leave the globals cleared.A non-regexp argument does not touch them, which falls out of leaving it to C.
Tests
mrbgems/mruby-regexp/test/regexp.rb, after theString#[],#[]=and#slice!regexp groups at the end of the file, in the same three groups permethod pair: the search itself, the match globals, and the delegation of every
non-regexp argument. The multibyte rows sit behind
__ENCODING__ == "UTF-8",which is what the file already uses for the character-offset cases.
Checked
Every row above, and every row in the new tests, was run against CRuby 4.0.6 and
compared: a table of 93 calls covering all seven methods, their positions at
both ends of the subject, their globals after a hit and after each kind of miss,
and the errors raised, is identical between the two, on the default build and on
one with
MRB_UTF8_STRING. A second table of 37 multibyte calls is identicaltoo.
rake testpasses on both builds.Summary by CodeRabbit
New Features
Bug Fixes
Documentation