Move the String#valid_encoding? body into core as mrb_str_valid_encoding_p - #7102

String#valid_encoding? in mruby-encoding walks the string with
mrb_utf8len and counts the characters. mrb_utf8_strlen() in string.c,
reached through utf8_strlen(), walks a string for the same purpose. The
two loops make the same decisions about how a UTF-8 string is read:

• the byte length of a character comes from mrb_utf8len
• the walk stops at the end of the range it was given, rather than at
  whatever bytes follow it
• a binary string is answered without walking it at all
• a string that turns out to hold as many characters as it has bytes is
  marked MRB_STR_SINGLE_BYTE on the way out

Only two steps are not shared: the gem tests each character for a byte that
stands for no character, and core skips runs of ASCII a word at a time.

This moves the body into string.c as mrb_str_valid_encoding_p and
leaves the gem method as a wrapper. The one walk that remains does both:
it skips the ASCII a word at a time, and it reports the first byte that
stands for no character. Neither step can hide the other, since a skipped
byte is ASCII and therefore valid, and a continuation byte is never ASCII.

Why this belongs in core

The method stays in the gem. String#valid_encoding? is defined by
mruby-encoding exactly as before, and a build without the gem gains no
method. What moves is the walk that core already performs for String#size
and character indexing, and the single byte flag that utf8_strlen() sets
while doing it.

string.c

The body of mrb_utf8_strlen becomes a static utf8_strlen_check(str, byte_len, validp). With validp given, the walk stops at the first
non-ASCII byte that mrb_utf8len measures as one byte and sets *validp
to FALSE; the returned count is a character count only while *validp
stays TRUE. mrb_utf8_strlen calls it with NULL, so nothing changes for
its existing callers.

mrb_str_valid_encoding_p(mrb, str) sits on top of that worker with the
rules the gem has: a binary string is valid without a walk, and a string
whose character count equals its byte count gets the single byte flag. It
does not read that flag on the way in, keeping what 470396b established:
the flag says one byte per character, which a string of stray bytes
satisfies as well as an ASCII one.

internal.h

The declaration sits with mrb_str_char_to_byte and mrb_str_byte_to_char,
outside the MRB_UTF8_STRING guard, so a caller does not need to guard the
call. Without UTF-8 support a string is a sequence of bytes with no encoding
to disagree with, and the function answers TRUE.

mruby-encoding

str_valid_enc_p is now one line, and the gem no longer reaches into
RSTR_LEN, RSTR_PTR and RSTR_SET_SINGLE_BYTE_FLAG to answer this
question itself.

Behavior

No change to what String#valid_encoding? answers, and no change to when
it sets the single byte flag. What changes is the cost of the walk, since
core skips runs of ASCII a word at a time through search_nonascii()
instead of calling mrb_utf8len on every byte. Measured with the default
gembox plus mruby-encoding, gcc -O3, 200 calls per case, best of five, both
binaries built with the same config on the same machine and run one after
the other:

A string that is all multi-byte gains nothing, having no run of ASCII to
skip, and nothing measured got slower. String#size reaches the same walk
now, through utf8_strlen_check(str, byte_len, NULL), and measures the same
as before.

Two tests are added for shapes the existing cases do not reach: a broken
byte found after a run of ASCII, and a substring long enough to share its
parent's buffer, whose walk has to stop at its own length.

Testing

The gem is in no gembox, so a default rake test builds neither the gem
nor MRB_UTF8_STRING. Both sides were run from a clean build/:

• MRUBY_CONFIG=host-debug rake test (full-core, so the gem is in and
  MRB_UTF8_STRING is on): 2237 assertions, 0 KO, 0 crash, plus 116
  bintest.
• rake test with the default config: 2053 assertions, 0 KO, 0 crash, plus
  105 bintest. This is the build where mrb_str_valid_encoding_p answers
  TRUE without a walk.
• MRUBY_CONFIG=minimal rake, to confirm the function compiles where UTF-8
  support is absent.

prek run --all-files passes, except that markdownlint could not install
locally; no Markdown is touched here.

Summary by CodeRabbit

• New Features

  • Added string encoding validation support through mrb_str_valid_encoding_p.
  • String#valid_encoding? now reliably handles UTF-8, binary strings, invalid trailing bytes, truncated sequences, and shared substrings.

• Bug Fixes

  • Improved detection of invalid UTF-8 sequences, including cases following long ASCII content.