mruby-regexp: take every search's capture buffer from the stack - #7289

A search fills a capture buffer pat->num_captures * 2 ints wide, and that width has a
compile-time bound. num_captures starts at 1 for group 0 and the compiler refuses the group
that would take it past RE_MAX_CAPTURES (re_compile.c:1560), so no pattern reaches the
engine asking for more than RE_MAX_CAPTURES * 2 ints, 256 bytes.

Most of the code that holds such a buffer says exactly that:

The last row is what this changes. Those three reached for the heap to hold a buffer of that
same fixed width, so the allocation stood for a variability the width does not have. They
declare it like the rest now.

regexp_s_scan() carried the split inside one function: the buffer it searched into was
allocated, while last_captures, the copy it keeps of the match to publish at the end, was
already the array on the stack beside it.

What the allocation carried

mrb_malloc() is more than the allocator here. A fresh allocation adds its size to
gc.malloc_increase and may drive an incremental step at that point (src/gc.c:309-333), so
every search paced the collector by its pattern's capture count, for a block the collector
never sees. Under MRB_GC_STRESS with MRB_DEBUG each allocation runs a full collection
first (src/gc.c:275-279), which is where the full-debug figures below come from.

The two functions that answer with a MatchData also had two exits each to free the buffer
on. Both are plain returns now, so a return added later has nothing to remember, and a raise
from inside create_matchdata() cannot strand the block.

Behaviour

Unchanged. exec_match() and regexp_s_scan() clear the same cap_size entries they cleared
before, and regexp_s_byte_rsearch() still clears none, because mrb_re_rexec() fills the
buffer before anything reads it (re_exec.c:1240, :1249, :1264).

The three frames grow by 256 bytes, and regexp_s_scan() reaches 512 with the array it
already had. None of the three recurses, and the engine they call recurses to
MRB_REGEXP_RECURSION_LIMIT (1,000) inside bt_match(), so what a regexp search costs in
stack is unchanged in the term that decides it.

Speed

Not the point of the change, but it is where the collector shows. Instruction counts from
callgrind, Ir(2N) - Ir(N) so process startup cancels, per call. The subject is a 880 byte
ASCII string; match is /q(u)ick/, rindex is /l(a)zy/, scan is /(\w+)o(\w*)/ with
60 matches.

The three full-debug rows land on the same figure per call, near 390,000 instructions, which
is the full collection each search no longer draws under MRB_GC_STRESS. scan gains the
same amount but spends it against a call that is 155M instructions wide, since it allocated
once per call rather than once per match.

scan costs 641 instructions more in the optimised builds, reproducibly and in every
configuration measured. It is codegen around the loop rather than anything the change asks
for, and I did not chase it further at 0.07%.

Size

.text of bin/mruby, build_config/ci/gcc-clang.rb, each side built from a clean build
directory at the same path. regexp.o is the only object that changes.

regexp.o itself: 28,917 → 28,837 in bintest and ascii-ctype, 28,437 → 28,357 in
byte-string, 29,013 → 28,949 in cxx_abi, and 30,710 → 30,657 at -O0, where the binary
moves 11 bytes more than the object by the padding the linker drops.

Verification

No test accompanies the change: the three functions answer with what they answered with
before, and the existing suites already drive all three (Regexp#match and #=~,
String#index, #rindex and #rpartition, String#scan).

rake test, build_config/ci/gcc-clang.rb, no compiler warning in any of the ten builds
this table and the size table above were taken from:

The default configuration: 2,174 total, 0 KO, 0 crash, and 112 bintests.

Environment

# ci/gcc-clang full-debug
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -g3 -O0 -DMRB_GC_STRESS -DMRB_USE_DEBUG_HOOK -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_DEBUG -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/full-debug/include" -o "build/full-debug/mrbgems/mruby-regexp/src/regexp.o" "mrbgems/mruby-regexp/src/regexp.c"

# ci/gcc-clang bintest
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRB_GC_FIXED_ARENA -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -DMRB_USE_DEBUG_HOOK -I"include" -I"mrbgems/mruby-regexp/include" -I"build/bintest/include" -o "build/bintest/mrbgems/mruby-regexp/src/regexp.o" "mrbgems/mruby-regexp/src/regexp.c"

# ci/gcc-clang cxx_abi
gcc -MMD -c -g -O3 -Wall -Wundef -Wwrite-strings -x c++ -std=gnu++03 -DMRB_GC_FIXED_ARENA -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_CXX_EXCEPTION -DMRB_USE_CXX_ABI -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/cxx_abi/include" -o "build/cxx_abi/mrbgems/mruby-regexp/src/regexp.o" "mrbgems/mruby-regexp/src/regexp.c"

# ci/gcc-clang byte-string
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/byte-string/include" -o "build/byte-string/mrbgems/mruby-regexp/src/regexp.o" "mrbgems/mruby-regexp/src/regexp.c"

# ci/gcc-clang ascii-ctype
gcc -MMD -c -std=gnu99 -g -O3 -Wall -Wundef -Werror-implicit-function-declaration -Wwrite-strings -DMRB_USE_ASCII_CTYPE -DMRBGEM_MRUBY_REGEXP_VERSION=0.0.0 -DMRB_USE_BIGINT -DMRB_USE_COMPLEX -DHAVE_MRUBY_ENCODING_GEM -DMRB_UTF8_STRING -DHAVE_MRUBY_IO_GEM -DMRB_USE_RATIONAL -DHAVE_MRUBY_REGEXP_GEM -DMRB_USE_SET -DMRB_USE_TASK_SCHEDULER -I"include" -I"mrbgems/mruby-regexp/include" -I"build/ascii-ctype/include" -o "build/ascii-ctype/mrbgems/mruby-regexp/src/regexp.o" "mrbgems/mruby-regexp/src/regexp.c"

Summary by CodeRabbit

• Performance
  • Improved regular expression matching and scanning by reducing temporary memory allocations.
• Reliability
  • Preserved existing match creation and failure handling behavior.