Skip to content

mruby-regexp: accept a Regexp in String#index, #partition, #start_with? and their siblings - #7075

Merged
matz merged 5 commits into
mruby:masterfrom
takumin:regexp-string-search-family
Aug 10, 2026
Merged

mruby-regexp: accept a Regexp in String#index, #partition, #start_with? and their siblings#7075
matz merged 5 commits into
mruby:masterfrom
takumin:regexp-string-search-family

Conversation

@takumin

@takumin takumin commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Seven String methods take a Regexp in CRuby and reach a C implementation here
that converts the argument to a String, so a Regexp is a TypeError. Four are
core (index, rindex, byteindex and byterindex, in src/string.c); the
other three come from mruby-string-ext (partition, rpartition and
start_with?), which mruby-regexp already depends on, so an override in the
gem's mrblib reaches both sets.

"abc".index(/b/)        # CRuby: 1,    mruby: TypeError
"abc".rindex(/b/)       # CRuby: 1,    mruby: TypeError
"abc".byteindex(/b/)    # CRuby: 1,    mruby: TypeError
"abc".byterindex(/b/)   # CRuby: 1,    mruby: TypeError
"abc".partition(/b/)    # CRuby: ["a", "b", "c"], mruby: TypeError
"abc".rpartition(/b/)   # CRuby: ["a", "b", "c"], mruby: TypeError
"abc".start_with?(/a/)  # CRuby: true, mruby: TypeError

Nothing is unreachable today, since =~ and String#[] cover the same ground.
What is missing is the spelling CRuby code uses, which means code carried over
raises instead of running.

This finishes the family. String#[] and #slice arrived in 46d88a8,
#sub! and #gsub! in #7061, #[]= and #slice! in #7063; none of the three
touched any of these seven. String#end_with? is deliberately not in the list:
CRuby rejects a Regexp there too, and so does this build, so the two already
agree.

Shape of the change

mrbgems/mruby-regexp/mrblib/string_regexp.rb already had the pattern to copy.
Each C method is captured under a private alias before the override replaces
it, and the override tests the argument with Regexp === arg rather than
is_a?, which is redefinable: anything that is not a Regexp goes back to the
captured C method untouched, so it keeps the C arity check and the C error
messages. The note at the top of that file sets out how far that guard goes and
where it stops.

One commit per pair, in the order the methods depend on each other, and a
last one for the README: the seven join the list of String methods that take
a Regexp here, and the walk described below joins the limitations.

index hands its position argument to Regexp#match unexamined: the two
normalize a position the same way and read it with the same mrb_get_args()
conversion.

rindex wants the last match that starts at or before its position, and the
match may run past that position, so "abcabc".rindex(/bca/, 1) is 1. The
engine has no backward search, so __regexp_rsearch walks the subject from the
start and keeps the last match that qualifies. That is linear in the number of
positions a match starts at, where the backward search CRuby hands to Onig is
not, which is the one cost worth naming here. Each step resumes one character
past the match start rather than at the match end, without which overlapping
matches stay invisible and "aaa".rindex(/aa/) answers 0 instead of 1.

byteindex and byterindex are the same two searches read in byte space.
MatchData#begin reports character offsets, which is the convention the rest of
the gem follows, so these two read __byte_begin instead. On a build without
MRB_UTF8_STRING the two spaces coincide and each pair answers the same number.

partition and rpartition build their three pieces from the match, so the row
worth naming is the one with no match, where the subject stays whole and the two
put it at opposite ends. The copy is a plain String even for a String subclass
receiver, as mrb_str_dup() and CRuby's str_duplicate(rb_cString, str) both
hand back. rpartition shares __regexp_rsearch with rindex.

start_with? reads its arguments left to right and hands each non-regexp one to
the C method one at a time, so a String keeps the C comparison and its error. A
regexp is anchored at the start rather than searched for, so the override checks
begin(0) == 0; the engine matches leftmost, so a pattern that can match at 0
does, which makes that check the anchored answer rather than an approximation of
it. "abc".start_with?(/b/) and "abc\ndef".start_with?(/^d/) are both false.

Two details the existing overrides settle already, and these follow:

  • The match globals. Each search goes through match and not match?, so $~
    and the names derived from it are published, including the clearing a failed
    match does. CRuby sets them for all seven and clears them after a miss. That
    extends to the misses that are not a failed search: a position outside the
    subject, a match that starts past what rindex asked for, and a match
    start_with? refuses for starting later than 0 all leave the globals cleared.
    A non-regexp argument does not touch them, which falls out of leaving it to C.
  • Character offsets against byte offsets, as above.

Tests

mrbgems/mruby-regexp/test/regexp.rb, after the String#[], #[]= and
#slice! regexp groups at the end of the file, in the same three groups per
method pair: the search itself, the match globals, and the delegation of every
non-regexp argument. The multibyte rows sit behind __ENCODING__ == "UTF-8",
which is what the file already uses for the character-offset cases.

Checked

Every row above, and every row in the new tests, was run against CRuby 4.0.6 and
compared: a table of 93 calls covering all seven methods, their positions at
both ends of the subject, their globals after a hit and after each kind of miss,
and the errors raised, is identical between the two, on the default build and on
one with MRB_UTF8_STRING. A second table of 37 multibyte calls is identical
too. rake test passes on both builds.

Summary by CodeRabbit

  • New Features

    • Added regular-expression support to string search, partition, and prefix-checking methods.
    • Added forward and reverse searches with character- and byte-based position handling.
    • Added support for multiple patterns, anchored prefix checks, and match result tracking.
  • Bug Fixes

    • Improved validation and handling of positions, empty matches, multibyte text, and invalid arguments.
    • Preserved existing behavior for non-regular-expression arguments.
  • Documentation

    • Documented regular-expression support and reverse-search behavior.

Both reach a C implementation that converts the argument to a String, so a
Regexp is a `TypeError` where CRuby searches with it:

```ruby
"abc".index(/b/)         # CRuby: 1, mruby: TypeError
"abcabc".rindex(/b/)     # CRuby: 4, mruby: TypeError
"abcabc".rindex(/bca/, 1) # CRuby: 1, mruby: TypeError
```

Override both in the gem's mrblib, alongside `String#[]` and `#[]=` and with
the same guard: `Regexp === args[0]` reads the real type where `is_a?` is
redefinable, and every other argument form goes back to the C method under a
private alias, keeping the C arity check and the C error messages.

`index` hands its position argument to `Regexp#match` unexamined. The two
normalize a position the same way and read it with the same `mrb_get_args()`
conversion, so a negative one counts back from the end and one that lands
outside the subject answers nil.

`rindex` wants the last match that starts at or before its position, and the
match may run past that position, which is why `"abcabc".rindex(/bca/, 1)` is
1. The engine has no backward search, so `__regexp_rsearch` walks the subject
from the start and keeps the last match that qualifies: linear in the number
of positions a match starts at, where the backward search CRuby hands to Onig
is not. Each step resumes one character past the match start rather than at
the match end, without which overlapping matches stay invisible and
`"aaa".rindex(/aa/)` answers 0 instead of 1.

Both search through `match` rather than `match?` so that `$~` and the names
derived from it are published, including the clearing a failed match does.
CRuby sets them for both methods and clears them after a miss. The walk in
`__regexp_rsearch` ends on a failed match or on one past its limit, so it
republishes the match it settled on with `__set_globals`.
The byte-offset half of the pair `index` and `rindex` just gained, rejecting a
Regexp for the same reason and answering the same two searches read in the
other space:

```ruby
"abc".byteindex(/b/)   # CRuby: 1, mruby: TypeError
"abc".byterindex(/b/)  # CRuby: 1, mruby: TypeError
```

`MatchData#begin` reports character offsets, which is the convention the rest
of the gem follows, so these two read `__byte_begin` instead. On a build
without MRB_UTF8_STRING the two spaces coincide and the pairs answer the same
number, which is what makes these nearly free once `index` and `rindex` exist;
on one with it they part company, and a position argument is bytes here where
it is characters there.

`byteindex` searches through `Regexp#__byte_match`, whose position is already
a byte offset. That method does no range check, so the ends of the subject are
checked here: both are a miss, as they are for `mrb_str_byteindex_m()`. An
offset that lands inside a character is not an error, because the C method
does not check for one either.

`byterindex` shares `__regexp_rsearch` with `rindex` and differs only in the
space its limit is compared in.
Both come from mruby-string-ext, which this gem depends on, and both convert
their argument to a String:

```ruby
"abc".partition(/b/)   # CRuby: ["a", "b", "c"], mruby: TypeError
"abc".rpartition(/b/)  # CRuby: ["a", "b", "c"], mruby: TypeError
```

The three pieces come straight from the match, so `pre_match`, `[0]` and
`post_match` are the whole of the matched case. The row worth naming is the
unmatched one, where the subject stays whole and the two methods put it at
opposite ends:

```ruby
"abc".partition(/z/)   # ["abc", "", ""]
"abc".rpartition(/z/)  # ["", "", "abc"]
```

That copy is a plain String even when the receiver is a String subclass, as
`mrb_str_dup()` in the C implementation and `str_duplicate(rb_cString, str)`
in CRuby both hand back.

`rpartition` wants the last match, overlapping ones included, so it shares
`__regexp_rsearch` with `rindex` and passes the end of the subject as its
limit.
The last of the family, and the only one that takes several patterns:

```ruby
"abc".start_with?(/a/)       # CRuby: true, mruby: TypeError
"abc".start_with?("x", /a/)  # CRuby: true, mruby: TypeError
```

The override reads the arguments left to right and hands each non-regexp one
to the C method under a private alias, one at a time, so a String keeps the C
comparison and its error and the arguments are still answered in order. An
argument after the one that answers is never looked at, as in CRuby.

A regexp is anchored at the start rather than searched for, while
`Regexp#match` searches forward from its position, so a successful match is
not the answer on its own and the override checks `begin(0) == 0`. The engine
matches leftmost, so a pattern that can match at 0 does, which makes that
check the anchored answer rather than an approximation of it:

```ruby
"abc".start_with?(/b/)        # false, though /b/ matches at 1
"abc\ndef".start_with?(/^d/)  # false, though /^d/ matches at 4
```

CRuby leaves no match behind for either of those, so a match that starts
further along is cleared rather than published. A non-regexp argument does
not touch the match globals at all, which falls out of leaving it to C.

`String#end_with?` gets nothing here: CRuby rejects a Regexp there too, and
so does this build, so the two already agree.
@takumin
takumin requested a review from matz as a code owner August 10, 2026 12:43
@coderabbitai

coderabbitai Bot commented Aug 10, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

String now accepts real Regexp arguments for search, partition, reverse partition, and prefix methods. Core implementations remain available for other argument types. Tests cover offsets, match globals, validation, delegation, and multibyte behavior.

Changes

Regexp-aware String APIs

Layer / File(s) Summary
Regexp search methods
mrbgems/mruby-regexp/mrblib/string_regexp.rb, mrbgems/mruby-regexp/test/regexp.rb, mrbgems/mruby-regexp/README.md
Preserves core methods and adds Regexp-aware index, rindex, byteindex, and byterindex behavior. Reverse searches scan forward and retain the last match. Tests and documentation cover offsets, byte semantics, match globals, delegation, validation, and multibyte behavior.
Partition and prefix methods
mrbgems/mruby-regexp/mrblib/string_regexp.rb, mrbgems/mruby-regexp/test/regexp.rb
Adds Regexp-aware partition, rpartition, and start_with? behavior. Tests cover anchoring, mixed arguments, empty and missing matches, match globals, delegation, and errors.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant String
  participant Regexp
  participant MatchGlobals
  String->>Regexp: Search with normalized position
  Regexp-->>String: Return match position and length
  String->>MatchGlobals: Update or restore match globals
  String-->>String: Return character or byte offset
Loading

Possibly related PRs

  • mruby/mruby#6994: Extends related mruby-regexp String pattern handling and validation.
  • mruby/mruby#7054: Modifies the same string_regexp.rb implementation and preserves delegation to core methods.
  • mruby/mruby#7063: Adds related Regexp-aware String method behavior in the same implementation area.

Suggested reviewers: matz

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding Regexp support to String search, partition, prefix, and related methods.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

The `Ruby API` section of the gem's README lists the String methods that
take a Regexp here, and the seven added by the commits before this one were
missing from it. Add them, and a limitation entry for the one cost the
addition carries.

`rindex`, `byterindex` and `rpartition` share `__regexp_rsearch`, which walks
the subject from the start because the engine has no backward search. That is
linear in the number of positions a match starts at, where the backward search
CRuby hands to Onig is not, so it belongs next to the other limitations rather
than only in a comment in the mrblib file.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants