Skip to content

string.c: search bytes in String#byterindex and in String#rindex on a binary string - #7099

Merged
matz merged 3 commits into
mruby:masterfrom
takumin:string-byterindex-by-byte
Aug 12, 2026
Merged

string.c: search bytes in String#byterindex and in String#rindex on a binary string#7099
matz merged 3 commits into
mruby:masterfrom
takumin:string-byterindex-by-byte

Conversation

@takumin

@takumin takumin commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

A backward search reads a string one way while the forward search reads it
another. str_index() scans bytes, but str_rindex() walked character
boundaries with char_adjust() / char_backtrack(), and both String#rindex
and String#byterindex went through it. Two pairs of methods disagree as a
result, and neither disagreement depends on malformed input.

byterindex against byteindex

byterindex reports byte positions, so it has to be able to name a position
inside a multi-byte sequence. Walking characters, it never visited one:

s = "aあb"
s.byteindex("\x81")   # => 2
s.byterindex("\x81")  # => nil, CRuby answers 2 for a byte-indexed subject

str_byterindex() now scans one byte at a time, the mirror of str_index(),
and backs String#byterindex. str_char_rindex() keeps the boundary walk and
backs String#rindex, where a match starting inside a character must be
passed over so the search continues to the character before it. The existing
assert_nil broken.rindex("\x81") in test/t/string.rb pins that half down
and still holds. A non-UTF-8 build reaches only the byte scan, so the
char_adjust() and char_backtrack() identity macros defined there are gone.

rindex against index on a binary string

rindex dispatches to the byte search when the single-byte flag is set, but a
binary string that has not had its flag computed went on to the UTF-8 path, so
the walk moved over character boundaries and a negative position was counted
in characters. index counts bytes for the same string:

s = "aあb".b
s.index("\x81".b)      # => 2
s.rindex("\x81".b)     # => nil, CRuby answers 2
s.rindex("\x81".b, -2) # => nil, CRuby answers 2
s.length               # marks the string single-byte
s.rindex("\x81".b)     # => 2

Adding RSTR_BINARY_P to that dispatch reads the string the way
mrb_str_char_to_byte() and mrb_str_byte_to_char() already read it. Every
value above now matches CRuby 4.0.6, before and after the flag is set.

Testing

  • rake test with build_config/host-debug.rb (full-core, MRB_UTF8_STRING):
    mrbtest 2235 OK / 0 KO, bintest 116 OK.
  • rake test with the default config (byte strings): mrbtest 2033 OK / 0 KO,
    bintest 105 OK.
  • Both new assertions were confirmed to fail against the implementation each
    commit replaces.

Summary by CodeRabbit

  • Bug Fixes

    • Improved reverse string searches for binary and single-byte strings by consistently using byte offsets.
    • Preserved character-aware reverse searching for standard UTF-8 strings.
    • Corrected handling of negative positions, bounds, empty searches, missing matches, and multibyte content.
    • Ensured String#byterindex correctly searches by byte position, including within multibyte UTF-8 sequences.
  • Tests

    • Added coverage for byte-based reverse searches across UTF-8, binary, and ASCII strings.

`str_rindex()` walked backward over character boundaries with
`char_adjust()` and `char_backtrack()`, and both `String#rindex` and
`String#byterindex` went through it. A byte position inside a multi-byte
sequence was therefore never visited, so `byterindex` disagreed with
`byteindex`, which scans bytes:

```ruby
s = "aあb"
s.byteindex("\x81")   # => 2
s.byterindex("\x81")  # => nil, CRuby answers 2 for a byte-indexed subject
```

Split the search in two. `str_byterindex()` scans one byte at a time and
backs `String#byterindex`; `str_char_rindex()` keeps the boundary walk and
backs `String#rindex`, where a match that starts inside a character must be
passed over so the search continues to the character before it. Only the
byte scan is reachable on a non-UTF-8 build, so the `char_adjust()` and
`char_backtrack()` identity macros defined there are gone.
`rindex` sends a string to the byte search when the single-byte flag is
set, but a binary string without that flag went on to the UTF-8 path: the
backward walk moved over character boundaries, and a negative position was
counted in characters. `index` counts bytes for the same string, so the two
disagreed until something happened to set the flag:

```ruby
s = "aあb".b
s.index("\x81".b)   # => 2
s.rindex("\x81".b)  # => nil, CRuby answers 2
```

Send a binary string to the byte search as well, which is how
`mrb_str_char_to_byte()` and `mrb_str_byte_to_char()` already read it.
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 636bb542-ed8a-4c70-9189-2d1ab2a3fba9

📥 Commits

Reviewing files that changed from the base of the PR and between d055fcc and 63df142.

📒 Files selected for processing (1)
  • src/string.c
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/string.c

📝 Walkthrough

Walkthrough

The change separates byte-oriented and UTF-8 character-oriented reverse searches. Binary and single-byte strings now use byte indexing for rindex. Tests cover byterindex and binary-string offsets.

Changes

String reverse indexing

Layer / File(s) Summary
Reverse-search implementation
src/string.c
Adds separate byte and UTF-8 character reverse-search helpers. byterindex uses byte searching, while rindex selects byte searching for binary and single-byte strings.
Reverse-search tests
test/t/string.rb, mrbgems/mruby-string-ext/test/string.rb
Adds coverage for byte offsets, bounds, empty and missing matches, negative positions, ASCII matches, and binary strings after length.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

  • mruby/mruby#7075: Modifies and tests related String#rindex and String#byterindex byte-search behavior.
  • mruby/mruby#7080: Updates string handling for binary and single-byte indexing.
  • mruby/mruby#7081: Updates binary-string handling and byte-versus-character indexing in src/string.c.

Suggested reviewers: matz

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: byte-based reverse searches for String#byterindex and binary String#rindex.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/string.c`:
- Around line 984-989: Update the backward-search loop around the visible memcmp
logic so it stops after checking s == sbeg, without decrementing s below sbeg.
Preserve the existing match return and ensure the no-match path exits cleanly
before the next condition evaluates an out-of-bounds pointer.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b81f6c96-ac12-4497-a944-5ae211cbf1dd

📥 Commits

Reviewing files that changed from the base of the PR and between e24e810 and d055fcc.

📒 Files selected for processing (3)
  • mrbgems/mruby-string-ext/test/string.rb
  • src/string.c
  • test/t/string.rb

Comment thread src/string.c Outdated
`str_byterindex()` stepped its pointer one past the first byte to end a
failed search, and `str_char_rindex()` did the same through
`char_backtrack()`, which answers the byte before the one it is given. C
lets a pointer sit one past the end of an object but not one before its
start, even where nothing reads through it.

Count an index down in the byte search, and stop the character search once
it has looked at the first character.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants