string.c: search bytes in String#byterindex and in String#rindex on a binary string - #7099
Conversation
`str_rindex()` walked backward over character boundaries with
`char_adjust()` and `char_backtrack()`, and both `String#rindex` and
`String#byterindex` went through it. A byte position inside a multi-byte
sequence was therefore never visited, so `byterindex` disagreed with
`byteindex`, which scans bytes:
```ruby
s = "aあb"
s.byteindex("\x81") # => 2
s.byterindex("\x81") # => nil, CRuby answers 2 for a byte-indexed subject
```
Split the search in two. `str_byterindex()` scans one byte at a time and
backs `String#byterindex`; `str_char_rindex()` keeps the boundary walk and
backs `String#rindex`, where a match that starts inside a character must be
passed over so the search continues to the character before it. Only the
byte scan is reachable on a non-UTF-8 build, so the `char_adjust()` and
`char_backtrack()` identity macros defined there are gone.
`rindex` sends a string to the byte search when the single-byte flag is
set, but a binary string without that flag went on to the UTF-8 path: the
backward walk moved over character boundaries, and a negative position was
counted in characters. `index` counts bytes for the same string, so the two
disagreed until something happened to set the flag:
```ruby
s = "aあb".b
s.index("\x81".b) # => 2
s.rindex("\x81".b) # => nil, CRuby answers 2
```
Send a binary string to the byte search as well, which is how
`mrb_str_char_to_byte()` and `mrb_str_byte_to_char()` already read it.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughThe change separates byte-oriented and UTF-8 character-oriented reverse searches. Binary and single-byte strings now use byte indexing for ChangesString reverse indexing
Estimated code review effort: 3 (Moderate) | ~20 minutes Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@src/string.c`:
- Around line 984-989: Update the backward-search loop around the visible memcmp
logic so it stops after checking s == sbeg, without decrementing s below sbeg.
Preserve the existing match return and ensure the no-match path exits cleanly
before the next condition evaluates an out-of-bounds pointer.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: b81f6c96-ac12-4497-a944-5ae211cbf1dd
📒 Files selected for processing (3)
mrbgems/mruby-string-ext/test/string.rbsrc/string.ctest/t/string.rb
`str_byterindex()` stepped its pointer one past the first byte to end a failed search, and `str_char_rindex()` did the same through `char_backtrack()`, which answers the byte before the one it is given. C lets a pointer sit one past the end of an object but not one before its start, even where nothing reads through it. Count an index down in the byte search, and stop the character search once it has looked at the first character.
A backward search reads a string one way while the forward search reads it
another.
str_index()scans bytes, butstr_rindex()walked characterboundaries with
char_adjust()/char_backtrack(), and bothString#rindexand
String#byterindexwent through it. Two pairs of methods disagree as aresult, and neither disagreement depends on malformed input.
byterindexagainstbyteindexbyterindexreports byte positions, so it has to be able to name a positioninside a multi-byte sequence. Walking characters, it never visited one:
str_byterindex()now scans one byte at a time, the mirror ofstr_index(),and backs
String#byterindex.str_char_rindex()keeps the boundary walk andbacks
String#rindex, where a match starting inside a character must bepassed over so the search continues to the character before it. The existing
assert_nil broken.rindex("\x81")intest/t/string.rbpins that half downand still holds. A non-UTF-8 build reaches only the byte scan, so the
char_adjust()andchar_backtrack()identity macros defined there are gone.rindexagainstindexon a binary stringrindexdispatches to the byte search when the single-byte flag is set, but abinary string that has not had its flag computed went on to the UTF-8 path, so
the walk moved over character boundaries and a negative position was counted
in characters.
indexcounts bytes for the same string:Adding
RSTR_BINARY_Pto that dispatch reads the string the waymrb_str_char_to_byte()andmrb_str_byte_to_char()already read it. Everyvalue above now matches CRuby 4.0.6, before and after the flag is set.
Testing
rake testwithbuild_config/host-debug.rb(full-core,MRB_UTF8_STRING):mrbtest 2235 OK / 0 KO, bintest 116 OK.
rake testwith the default config (byte strings): mrbtest 2033 OK / 0 KO,bintest 105 OK.
commit replaces.
Summary by CodeRabbit
Bug Fixes
String#byterindexcorrectly searches by byte position, including within multibyte UTF-8 sequences.Tests