string.c: step String#rindex by the characters String#length counts - #7107

String#rindex walks a character-indexed receiver backward, and two things
about that walk were wrong. Both show only there: an ASCII or binary receiver
is handed to String#byterindex, which does not walk.

Reaching the first character

A negative pos counts characters back from the end, so minus the length names
the first character. The walk stepped back that far and then read arriving at
the string start as having run out of string, which made -length and one step
past it answer alike:

"あいう".rindex("あ", -3)  #=> nil  (CRuby: 0)
"あいう".rindex("あ", -4)  #=> nil

String#byterindex adds the length to pos rather than walking, so it was
already right: "abc".rindex("a", -3) is 0.

Where a character starts

char_adjust() and char_backtrack() found a character start by looking for a
byte that is not a continuation byte, and never asked whether the byte they
found reaches that far. That is the whole of what mrb_utf8len() goes on to
check, so the walk and the character count disagreed wherever a lead byte
promises more than it delivers:

"あ\x80x".length            #=> 3
"あ\x80x".chars             #=> ["あ", "\x80", "x"]
"あ\x80x".index("\x80")     #=> 1
"あ\x80x".rindex("\x80")    #=> nil
"あ\x80x".rindex("あ", -2)  #=> nil  (CRuby: 0)

\x80 follows no lead byte that reaches it, so it is a character of its own,
which is what #length and #chars count it as. The walk stepped over it into
あ instead.

char_adjust() also rounded forward. rindex wants the last start at or
before pos, so a pos that is not a boundary belongs rounded toward the
start; rounding the other way let a backward search answer a position after the
one it was given:

"あ\x80x".rindex("x", 1)  #=> 2  (CRuby: nil)

One helper

str_char_head(beg, p, end) replaces both: the byte the character covering p
starts at, and p itself when p is already a boundary. It reads back at most
three bytes to a lead byte and keeps that lead only when mrb_utf8len() says it
reaches. Rounding is then always toward the start, and the character before p
is the head of p-1.

The bounded lookback is the forward walk rather than an approximation of it. If
a character covers p and starts back bytes earlier, every byte in between is
a continuation byte, so the nearest non-continuation byte going back is that
character's lead; and the lead cannot belong to some earlier character in turn,
since the lead byte of a multi-byte sequence is never a continuation byte. Three
bytes is the whole of the range, because nothing longer than four bytes spells a
character.

This is the rule mruby-regexp already applies in mrb_re_utf8_interior_p(), so
core and the regexp engine now agree on where a character starts.

Against CRuby

Over rindex with eleven subjects holding malformed sequences of each kind,
seven needles that are valid UTF-8, and every position from -length-1 to
length+1, the five differences from CRuby 4.0.6 are the ones above. After this
change there are none.

Differences remain for a needle that is itself malformed: CRuby answers nil
for those regardless of the subject, while mruby searches for the bytes.
String#index answers the same way, so that is a separate question about what a
malformed argument means, and nothing here changes it.

Verified

• rake test on a full-core build (MRB_UTF8_STRING through mruby-encoding):
  2248 tests, all green
• rake test on the default gembox (no MRB_UTF8_STRING): 2057 tests, all
  green
• each commit is green on its own, and the test added by each fails on the
  commit before it
• prek run --all-files passes, except that markdownlint could not install
  locally (npm engine mismatch); no Markdown is touched here

Summary by CodeRabbit

• Bug Fixes

  • Improved reverse substring searches for UTF-8 text, including searches starting at negative positions.
  • Corrected character-boundary handling for multibyte and invalid UTF-8 bytes.
  • Enforced position limits more consistently during backward searches.

• Tests

  • Added coverage for UTF-8 reverse searches at character boundaries and with invalid byte sequences.