mruby-regexp: bound the numeric \k backreference accumulator - #7019

compile_atom() parses the numeric form of \k by accumulating digits into an int
with no bound. A number too large for the type wraps, and the wrapped value is what the
range check below it sees, so a backreference to a group that does not exist compiles as
a backreference to one that does.

Regexp.new("(a)\\k<4294967297>").match("aa")      # CRuby: RegexpError (too big number)
                                                  # mruby: ["aa", "a"]
Regexp.new("(a)\\k<-4294967297>").match("aa")     # CRuby: RegexpError
                                                  # mruby: ["aa", "a"]
Regexp.new("(a)(b)\\k<4294967298>").match("abb")  # CRuby: RegexpError
                                                  # mruby: ["abb", "a", "b"]

The range check is not bypassed; it is handed a number that is no longer the number in
the pattern. 4294967297 overflows on the tenth digit and wraps modulo 2^32 to 1,
which is in range for /(a)/, so the check passes and RE_BACKREF is emitted against
group 1. Which group a given number lands on is a property of the wrap, not of the
pattern: \k<10000000000000000000000> raises only because its wrapped value happens to
fall outside the range.

The accumulation is also signed integer overflow, undefined behaviour under C99 6.5/5
independently of the wrong answer it produces here:

$ ./build/asan/bin/mruby -e 'p Regexp.new("(a)\\k<4294967297>").match("aa").to_a'
re_compile.c:823:17: runtime error: signed integer overflow: 429496729 * 10
  cannot be represented in type 'int'
["aa", "a"]

Fix

Bound the accumulator inside the loop against c->num_captures - 1, the same quantity
the range check below it uses. The bound is tight and correct for both forms: an
absolute reference needs group < c->num_captures, and a relative one needs
c->num_captures - n >= 1. It rejects nothing the range check would have accepted, so
it carries that check's message and no pattern that compiles today changes its error.

Testing after the addition rather than before it keeps the check to one line, and n
cannot overflow between two iterations: it is at most c->num_captures - 1 on entry,
itself at most RE_MAX_CAPTURES - 1 (32), so n * 10 + 9 is at most 319.

Nothing else changes. re_exec.c is not involved: RE_BACKREF carries a resolved group
number in re_inst.a, so the matcher never sees the digits. The named branch resolves a
name through the capture table and does no arithmetic.

Test

Three cases in mrbgems/mruby-regexp/test/regexp.rb, next to the existing
Regexp - named backreference \k assertion. Each one binds to a group without this
change and raises with it, so each one fails on master.

The in-range direction is already pinned by that neighbouring assertion: /(a)\k<1>/
has n == c->num_captures - 1 == 1 and /(.)(.)\k<-1>\k<-2>/ reaches
n == c->num_captures - 1 == 2, which are exactly the two boundary values the new
comparison is written against.

Checked against CRuby 4.0.6. rake test passes, and the UBSan report above is gone on a
build_config/asan.rb build.

CodeRabbit raised this overflow while reviewing #7007, where it is pre-existing rather
than introduced; it goes out on its own per CONTRIBUTING.md's one bugfix per pull
request.

Summary by CodeRabbit

• Bug Fixes

  • Invalid numeric regular-expression backreferences now raise RegexpError immediately instead of being interpreted as unintended capture groups.
  • Backreferences exceeding the supported capture-group range are handled consistently.

• Tests

  • Added regression coverage for oversized numeric backreference values.