mruby-regexp: fold a byte-indexed subject by ASCII under a /i backreference - #7264

A backreference under /i folded a byte-indexed subject (String#b) as if each byte were the codepoint of the same value. On a build with the Unicode case table 0xC0 folded to 0xE0 the way U+00C0 folds to U+00E0, so two bytes that spell no letter in common compared equal:

"\xC0\xE0".b =~ /(.)\1/i      # CRuby: nil, mruby before: 0, mruby after: nil
"\xC0a\xE0A".b =~ /(..)\1/i   # CRuby: nil, mruby before: 0, mruby after: nil
"\xC0\xC0".b =~ /(.)\1/i      # CRuby: 0,   mruby before: 0, mruby after: 0
"aA".b =~ /(.)\1/i            # CRuby: 0,   mruby before: 0, mruby after: 0
"sſ" =~ /(.)\1/i              # CRuby: 0,   mruby before: 0, mruby after: 0
"ſs".b =~ /(.)\1/i            # CRuby: nil, mruby before: nil, mruby after: nil

memcmp_ci in re_exec.c compares the captured span and the text at the cursor by folding both sides with mrb_re_case_fold. mrb_re_decode_char hands out the raw byte for a byte-indexed subject, and the fold took that byte for a codepoint. The class side of the same report ("\xE5".b =~ /[\xC5]/i) was already answered by class_match, which keeps a byte member apart from the codepoint of the same number; the backreference was the one path that still folded a subject at match time.

Fix

memcmp_ci folds through a new subject_fold(c, binary), which returns a byte above 127 unchanged when the subject is byte-indexed and folds everything else as before. The letters a byte can spell are the ASCII ones, and those still fold; a subject read as characters is untouched. Builds without the Unicode table were never affected (their fold leaves 0xC0 alone), so the fix changes nothing there.

Testing

New test Regexp - a backreference under /i folds a byte-indexed subject by ASCII in mrbgems/mruby-regexp/test/regexp_utf8.rb. The byte-indexed lines hold on every build and run unconditionally; the contrast with a subject read as characters ("sſ" =~ /(.)\1/i folds U+017F to s, "sſ".b does not) sits in an if __ENCODING__ == "UTF-8" branch rather than behind a skip, so a failure in the lines before it is not swallowed on the byte build. The test fails on master (Expected 0 to be nil) and passes with the fix.

Full suite green at every commit (one commit).

One divergence from CRuby is left alone, since it is not about byte-indexed subjects: CRuby answers "ſs" =~ /(.)\1/i with nil while "sſ" =~ /(.)\1/i is 0, because Onigmo requires the remaining subject to hold at least the captured span's byte length before it folds. mruby answers 0 for both orders.

Environment

# full-debug
gcc -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -g3 -O0 -DMRB_GC_STRESS -DMRB_USE_DEBUG_HOOK -DMRB_DEBUG -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER src/string.c
# bintest
gcc -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRB_GC_FIXED_ARENA -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -DMRB_USE_DEBUG_HOOK src/string.c
# cxx_abi
gcc -g -O3 -Wall -Wundef -Wwrite-strings -x c++ -std=gnu++03 -DMRB_GC_FIXED_ARENA -DMRB_USE_CXX_EXCEPTION -DMRB_USE_CXX_ABI -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER src/string.c
# byte-string
gcc -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER src/string.c
# ascii-case
gcc -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRB_USE_ASCII_CASE -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER src/string.c

Summary by CodeRabbit

• Bug Fixes

  • Improved case-insensitive regular expression matching for binary data.
  • ASCII byte values are folded correctly without incorrectly applying Unicode mappings to non-ASCII bytes.
  • Case-insensitive backreferences now behave consistently for byte-indexed subjects and valid UTF-8 text.

• Tests

  • Added regression coverage for binary and UTF-8 case-insensitive matching scenarios.