Skip to content

mruby-regexp: add the regexp form of String#[] and String#slice - #7006

Closed
takumin wants to merge 3 commits into
mruby:masterfrom
takumin:string-aref-regexp
Closed

mruby-regexp: add the regexp form of String#[] and String#slice#7006
takumin wants to merge 3 commits into
mruby:masterfrom
takumin:string-aref-regexp

Conversation

@takumin

@takumin takumin commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Problem

String#[] and its alias String#slice accept an Integer, a Range and a
String, but not a Regexp. Every regexp form fails with a TypeError naming
Integer, which does not hint at what is actually missing.

"hello"[/l+/]            # CRuby: "ll",  mruby: TypeError (Regexp cannot be converted to Integer)
"hello".slice(/l+/)      # CRuby: "ll",  mruby: TypeError
"hello"[/(l+)(o)/, 1]    # CRuby: "ll",  mruby: TypeError
"hello"[/(?<x>l+)/, :x]  # CRuby: "ll",  mruby: TypeError
"hello"[/z/]             # CRuby: nil,   mruby: TypeError
:hello[/l+/]             # CRuby: "ll",  mruby: TypeError

mrb_str_aref() funnels every non-String, non-Range argument through an
Integer conversion, and mruby-regexp never overrode [] or slice.

Solution

Override [] in mrbgems/mruby-regexp/mrblib/string_regexp.rb and alias
slice to it, in the shape split already uses in that file: capture the
C-defined method as __aref, check the arity, then delegate every argument
list that does not start with a Regexp straight back to C.

The type test is Regexp === args[0] rather than is_a?, for the reason
the split and =~ comments already give: is_a? is redefinable, so an
argument could deny its own type and be read as an index instead, or claim
a type it does not have. Module#=== reads the real type.

Behaviour notes

  • $~ is set through Regexp#match, including to nil on a failed match,
    which is why match? is not used even in the no-capture case.
  • The capture argument reaches MatchData#[] unchanged. A negative index
    is normalized, an index past the last group answers nil, and a name that
    resolves to no group raises IndexError. This matches CRuby, where
    rb_reg_nth_match() answers nil and rb_reg_backref_number() raises.
    mruby-regexp: fix MatchData#[] for a negative index and an unknown name #7000 is what made this delegation correct on its own.
  • A failed match answers nil without inspecting the capture argument, as
    rb_str_subpat() does.
  • The arity check comes before the argument is inspected, so
    "a"[/a/, 1, 2] raises ArgumentError (given 3, expected 1..2) rather
    than quietly ignoring the extra argument.
  • One known divergence remains: a Float capture argument is accepted, since
    MatchData#[] reads it through mrb_as_int(). This is the same latitude
    the surrounding gem already takes.

OP_GETIDX interaction

vm_op_getidx() answers an Integer, String or Range index for a receiver
whose class is exactly String directly from mrb_str_aref(), without
consulting the method table, so an override of String#[] is not visible
for those three forms. A Regexp index falls through to a send and does
reach the override.

This is safe here because the override only delegates those forms, and the
method comment records the constraint for whoever changes it next.

str[0] is the exception, and the one place this change costs anything.
A literal zero index compiles to OP_GETIDX0, which has a fast path for
Array and Hash and none for String, so it always sends and therefore does
reach the override. Measured over 1000000 iterations on the built binary,
s[0] goes from 48 ms to 193 ms, which is the Ruby frame and the argument
array; 48 ms is what s.__aref(0) still costs, and __aref is the same C
method str[0] reached before. Every other form is unaffected: str[i]
with a non-zero or non-literal index is answered by OP_GETIDX from C as
before, and str[i, len], str.slice(...), an explicit send and a String
subclass receiver were already sends.

The loops in gsub and split read one character with str[0] on every
iteration, so they now spell it __aref(0) and stay on the old path.

Symbol

Symbol#[] and Symbol#slice are defined in mruby-symbol-ext, not in
core, and delegate to the String methods, so sym[re] follows from this
change with no implementation of its own. mruby-regexp does not depend on
mruby-symbol-ext, so the Symbol test skips itself where that gem is
absent, the same way the to_enum test in this gem skips without
mruby-enumerator.

The write side is left alone

str[re] = repl and str.slice!(re) still raise the TypeError this PR
removes from the read side. String#[]= is core (mrb_str_aset_m) and
String#slice! is mruby-string-ext (mrb_str_slice_bang, registered as
MRB_SYM_B(slice)). Covering the capture form of the assignment would mean
adding C, since MatchData#__byte_begin takes an Integer and the name to
group resolution is private to matchdata_aref(), and the destructive side
brings the frozen check and the live reference a MatchData keeps to the
string it matched. Both belong with the rest of the destructive-method work,
so the README says so rather than claiming the regexp form is complete.

Tests

mrbgems/mruby-regexp/test/regexp.rb covers the regexp form and its
slice twin, a subclass receiver, capture access by index and by name,
negative and out of range indices, an unknown name, a failed match, the
$~ and $1 globals on success and on failure, the delegated non-regexp
forms including the subclass path that always goes through the override,
the arity errors, and an argument that lies in is_a?.
mrbgems/mruby-regexp/test/symbol_regexp.rb covers sym[re] behind the
mruby-symbol-ext guard.

rake test passes.

Documentation

Narrows the No regexp form of String#[] item in the gem README's
Limitations section to the write side, adds the forms to the usage examples,
and rewrites the clause in the symbol_regexp.rb header comment that
recorded sym[/re/] as still missing.

Summary by CodeRabbit

  • New Features

    • Added regular expression support to String#[] and String#slice, including capture and named-capture extraction.
    • Added regular expression slicing support for symbols where symbol slicing is available.
    • Preserved existing behavior for non-regexp indexing and slicing.
  • Documentation

    • Documented regexp-based string and symbol slicing, including supported reads and unsupported writes.
  • Tests

    • Added coverage for captures, match globals, validation, subclasses, unmatched patterns, and edge cases.

`str[re]` and `str.slice(re)` are among the most common ways to pull a
substring out of a match, and both raised a `TypeError` naming `Integer`,
which says nothing about what is missing.

```ruby
"hello"[/l+/]            # CRuby: "ll",  mruby: TypeError
"hello"[/(l+)(o)/, 1]    # CRuby: "ll",  mruby: TypeError
"hello"[/(?<x>l+)/, :x]  # CRuby: "ll",  mruby: TypeError
"hello"[/z/]             # CRuby: nil,   mruby: TypeError
```

Override `[]` in mrblib and alias `slice` to it, following the shape
`split` already uses in this file: the C-defined method is captured as
`__aref` first, and every argument list that does not start with a Regexp
goes straight back to it, so the non-regexp forms keep their behaviour and
their arity errors.

The regexp branch goes through `Regexp#match` rather than `match?` even
when no capture is asked for, because `$~` has to be set on a failed match
too. The capture argument is handed to `MatchData#[]` unchanged, which
answers nil for an index past the last group and raises `IndexError` for a
name that resolves to no group, matching `rb_reg_nth_match()` and
`rb_reg_backref_number()`.

`OP_GETIDX` answers an Integer, String or Range index for a String receiver
from C without consulting the method table, so those three forms never
reach the override. That is only safe because the override delegates them
unchanged, and the method comment records it. The same shortcut does not
apply to a String receiver reached by an explicit send, which is why the
loops in `gsub` and `split` now spell their one-character reads `__aref(0)`
rather than paying for a Ruby frame per iteration.

`Symbol#[]` and `Symbol#slice` come from `mruby-symbol-ext` and delegate to
the String methods, so `sym[re]` follows without an implementation of its
own. This gem does not depend on that one, so the test skips where it is
absent.
@takumin
takumin requested a review from matz as a code owner August 3, 2026 10:10
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

The regexp gem adds regexp-aware String#[] and String#slice, preserves non-regexp indexing, documents Symbol delegation, and adds tests for captures, match state, validation, and subclass behavior.

Changes

Regexp indexing APIs

Layer / File(s) Summary
String regexp indexing API
mrbgems/mruby-regexp/mrblib/string_regexp.rb, mrbgems/mruby-regexp/mrblib/symbol_regexp.rb, mrbgems/mruby-regexp/README.md
String#[] matches regular expressions, returns captures, updates match globals, and aliases String#slice. Non-regexp calls use the original implementation. Internal gsub and split access also uses the original implementation. Documentation describes String and Symbol regexp indexing and the remaining write limitations.
Regexp indexing validation
mrbgems/mruby-regexp/test/regexp.rb, mrbgems/mruby-regexp/test/symbol_regexp.rb
Tests cover matches, captures, named and negative indexes, failed matches, match globals, delegation, argument validation, subclass receivers, spoofed type checks, and Symbol behavior.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant String
  participant Regexp
  participant MatchState
  String->>Regexp: match regexp against string
  Regexp-->>String: return match and captures
  String->>MatchState: update match globals
  String-->>String: return match or selected capture
Loading

Possibly related PRs

  • mruby/mruby#6993: Both PRs modify mrbgems/mruby-regexp Symbol regexp behavior and tests, but implement different methods.
  • mruby/mruby#6995: This PR adds Symbol/String regexp slicing, while the related PR adds Symbol operands to Regexp matching methods.

Suggested reviewers: matz

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the main change: adding Regexp support to String#[] and String#slice.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

takumin added 2 commits August 3, 2026 19:23
The previous commit added the regexp form of `String#[]` and took the
Limitations item that named it along, which reads as though the regexp
form of element reference is now complete in both directions. It is not:
the write side still funnels a Regexp through an Integer conversion.

```ruby
s = +"hello"
s[/l+/] = "X"   # CRuby: "heXo", mruby: TypeError
s.slice!(/l+/)  # CRuby: "ll",   mruby: TypeError
```

`String#[]=` is core (`mrb_str_aset_m`) and `String#slice!` is
mruby-string-ext (`mrb_str_slice_bang`, registered as `MRB_SYM_B(slice)`,
which is why grepping for the method name by hand misses it). Both are
left as they are here. The capture form of `str[re, capture] = repl` needs
the byte range of a named group, and `MatchData#__byte_begin` takes an
Integer only, so covering it means adding C rather than another mrblib
override; the destructive side also brings the frozen check and the live
reference a MatchData keeps to the string it matched, which belong with
the rest of the destructive-method work.

State the narrowed limitation instead, so the section says what is missing
rather than nothing at all.
The comment above the override says an Integer, String or Range index never
reaches it, because OP_GETIDX answers those from C without consulting the
method table. That is true of `OP_GETIDX`, and it leaves out `OP_GETIDX0`.

A literal zero index compiles to the second opcode, which has a fast path
for Array and Hash and none for String, so `str[0]` falls back to a send and
does arrive at the override. It answers the same value as before through
`__aref`, one Ruby frame and one argument array later:

```console
$ ./build/host/bin/mruby -e 's="hello"; ... 1000000 iterations of s[0] ...'
193 ms
$ ./build/host/bin/mruby -e 's="hello"; ... 1000000 iterations of s.__aref(0) ...'
48 ms
```

`__aref(0)` is the path `str[0]` took before this branch, so that is the
whole of the difference. It is also why the loops in `gsub` and `split`
already read their one character that way.

Comment only. No behaviour changes.

takumin commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Thank you for taking a look. I’m going to close this PR for now. I’ll reorganize the individual changes, make their dependencies and intended submission order explicit, and then resubmit them as appropriately scoped PRs. Sorry for the churn.

@takumin

takumin commented Aug 9, 2026

Copy link
Copy Markdown
Contributor Author

Resubmitted as #7054, with the OP_GETIDX0 performance concern split out and merged separately as #7040.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant