mruby-regexp: say which \k reference failure it was - #7228

A \k<...> reference can fail in four ways, and CRuby names each one. mruby
answers the first three with a single message and the fourth with another, so
a pattern that misspelled a name and a pattern that named a group it never
opened read the same:

Why the three collapse

The number was read and bounded in one loop:

/* mrbgems/mruby-regexp/src/re_compile.c */
for (uint32_t i = (relative ? 1 : 0); i < name_len; i++) {
  if (name[i] < '0' || name[i] > '9') compile_error(c, "invalid backreference");
  n = n * 10 + (name[i] - '0');
  if (n > (int)c->num_captures - 1) compile_error(c, "undefined group name reference");
}

The bound on the partial value is what keeps the accumulator from wrapping, and
it is also the only thing that answers whether the group exists. So the loop
stops before the number is whole, and a number too large to be one at all
arrives at the same site as a number that simply names a group the pattern does
not have. The final group < 1 || group >= num_captures check outside the loop
carried the third case, an undefined name, to that same message.

Reading the name in two passes

The first pass says whether the name is -? followed by digits; the second
converts it and stops at RE_MAX_BACKREF_NUM. Each failure then has its own
site: a name that is not a number, a number past the bound, and a number within
it that resolves to no group. The name lookup keeps its own.

Reading the name whole before converting it is what CRuby does, and it decides
one row on its own: \k<99999999999999999999x> is a malformed name, not an
oversized number, since the digits are never converted.

Two more rows fall out. \k<0> names the whole match, which no reference can
name, and \k<-> has no digits at all; both are malformed names rather than
references to a missing group. And the refusal a named pattern gives a numbered
reference now comes after the name is read as a number at all, so
(a)(?<b>b)\k<1x> reports the malformed name rather than the refusal, which is
again CRuby's order.

RE_MAX_BACKREF_NUM is 2147483647, where CRuby's scanner stops. The bound is
not a capacity, it is where two messages part: \k<2147483647> is
invalid backref number/name and \k<2147483648> is too big number.

Carrying the name in the message

Two of the four messages quote the name. compile_error() takes the message as
a C string and formats it with %s, which cannot carry one: the name is a
length-counted slice of the pattern, and a name holding a NUL would be cut
short. The first commit splits the function, leaving compile_error_str() to
take an mrb_value and compile_error() a wrapper over it. Every existing
caller keeps the message it already passes, so that commit changes no message.

What still differs from CRuby

CRuby quotes one byte too many for the group-0 case, invalid group name <0>>
for \k<0> and <0'> for \k'0'. The name is quoted here without the
delimiter that closed it.

A name holding a NUL is quoted whole here. CRuby builds these messages through
a C string and stops at the NUL, so (a)\k<a\0b> is undefined name <a there
and undefined name <a\0b> reference here. This is the row the message carrier
above buys, and the tests pin it.

\k<1-1> is Onigmo's nesting-level backreference, which this engine does not
implement. It was invalid backreference and is now invalid group name <1-1>; a spelling the engine cannot read is what both messages report.

Testing

build_config/ci/gcc-clang.rb and build_config/gcc-asan.rb, run per build so
the counts are attributable:

The binary tests pass 122 of 122 under ci/gcc-clang and 84 of 84 under
gcc-asan. No build warns.

Against a parser that still bounds the partial value, both test blocks turn
red:

Fail: Regexp - numeric \k backreference out of int range (mrbgems: mruby-regexp)
Fail: Regexp - \k group reference errors say which failure it was (mrbgems: mruby-regexp)

Size

Summed .text over libmruby.a, full-core at -O3, each build from an
empty build directory:

The whole of it is the second pass over the name and the extra call sites. The
strings move by +43 bytes across .rodata.str1.1 and .rodata.str1.8.
mruby-regexp is not in the default gembox, so build_config/default.rb does
not move.

Environment

MRuby::Build.new('size') do |conf|
  conf.toolchain
  conf.gembox 'full-core'
end

# size, the -O3 full-core rows
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/size/include" -o "build/size/mrbgems/mruby-regexp/src/re_compile.o" "mrbgems/mruby-regexp/src/re_compile.c"

# ci/gcc-clang full-debug
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -g3 -O0 -DMRB_GC_STRESS -DMRB_USE_DEBUG_HOOK -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_DEBUG -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/full-debug/include" -o "build/full-debug/mrbgems/mruby-regexp/src/re_compile.o" "mrbgems/mruby-regexp/src/re_compile.c"

# ci/gcc-clang bintest
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRB_GC_FIXED_ARENA -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -DMRB_USE_DEBUG_HOOK -I"include" -I"mrbgems/mruby-regexp/include" -I"build/bintest/include" -o "build/bintest/mrbgems/mruby-regexp/src/re_compile.o" "mrbgems/mruby-regexp/src/re_compile.c"

# ci/gcc-clang cxx_abi
gcc -MMD -c -g -O3 -Wall -Wundef -Wwrite-strings -x c++ -std=gnu++03 -DMRB_GC_FIXED_ARENA -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_CXX_EXCEPTION -DMRB_USE_CXX_ABI -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/cxx_abi/include" -o "build/cxx_abi/mrbgems/mruby-regexp/src/re_compile.o" "mrbgems/mruby-regexp/src/re_compile.c"

# ci/gcc-clang byte-string
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/byte-string/include" -o "build/byte-string/mrbgems/mruby-regexp/src/re_compile.o" "mrbgems/mruby-regexp/src/re_compile.c"

# ci/gcc-clang ascii-case
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRB_USE_ASCII_CASE -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/ascii-case/include" -o "build/ascii-case/mrbgems/mruby-regexp/src/re_compile.o" "mrbgems/mruby-regexp/src/re_compile.c"

# gcc-asan
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -fsanitize=address,undefined -g3 -O0 -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_DEBUG -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/gcc-asan/include" -o "build/gcc-asan/mrbgems/mruby-regexp/src/re_compile.o" "mrbgems/mruby-regexp/src/re_compile.c"

🤖 Generated with Claude Code

https://claude.ai/code/session_01EtQ1ZeRkWXncrDed7qpJyY

Summary by CodeRabbit

• Bug Fixes

  • Improved regular expression error messages for invalid named and numeric backreferences.
  • Preserved embedded characters in group names and the original pattern text in diagnostics.
  • Clearly distinguishes malformed, undefined, unavailable, and oversized references.
  • Accepts valid references with leading zeros while detecting numeric overflow.

• Tests

  • Expanded coverage for backreference validation, embedded characters, overflow handling, and exact error messages.