string.c: record the source overlap before str_modify_cat() runs - #7043

Summary

mrb_str_cat() has to know whether ptr points into the string it is appending to, so
that the source can be followed if the destination moves. It asks that question after
str_modify_cat() has run:

  mrb_int capa = str_modify_cat(mrb, s, (mrb_int)len);
  if (ptr >= RSTR_PTR(s) && ptr <= RSTR_PTR(s) + (size_t)RSTR_LEN(s)) {
      off = ptr - RSTR_PTR(s);
  }

By then s may hold a different buffer. str_modify_cat() takes one of two paths. It
appends inside the shared allocation, which leaves the buffer where it is and ptr valid;
or it detaches s through mrb_str_modify(), which copies the string into a fresh
allocation and frees the old one when the copy takes the last reference to it. On the
second path ptr is outside the new buffer, the comparison records no overlap, off
stays -1, and the memcpy() at the end of the function reads through ptr regardless.

Reachability

Not from inside mruby. It needs a ptr into a shared buffer whose only remaining
reference is s itself, and the paths that pass an interior pointer do not produce that:
mrb_str_cat_str() on two strings over one buffer means two references, so the buffer
outlives the copy, and on one string it modifies up front. An extension that kept a
RSTRING_PTR() across an operation that dropped the other sharer can produce it, which is
the same class of caller the length of mrb_str_cat() already comes from.

That makes this a fix for the public API contract rather than a live bug.

The change

Compare before the call rather than after it. The offset is what the rest of the function
wants on either path, and it stays valid across the copy, since a copy preserves the
contents and the offsets into them alike, whether the string ends up in a fresh allocation
or embedded.

Where the buffer stays put, the offset is the same answer reached another way, and nothing
changes. Where it moves, off is now recorded instead of lost, so ptr is re-derived
from the new buffer at the memcpy() rather than left pointing at the old one. Recording
ahead of the call covers both without having to know which path ran.

Rebase

Rebased onto 9360b3fd0. #7038 and #7039 have both landed and both touch the lines above
the overlap check, so two things move. The size check from #7038 stays ahead of
everything, since it is the one that leaves the string untouched when it raises. And what
the overlap check has to sit in front of is now str_modify_cat() rather than
mrb_str_modify(), which is also why the reason for recording first is broader than it
was: the in-place path leaves ptr valid, so there the offset is merely correct rather
than necessary.

Testing

rake test with build_config/default.rb, MRUBY_CONFIG=asan rake test
(address,undefined), and MRUBY_CONFIG=ci/gcc-clang rake test, which covers
MRB_GC_STRESS with MRB_USE_DEBUG_HOOK, MRB_GC_FIXED_ARENA, and the C++ ABI build.
KO: 0 and Crash: 0 on all of them.

No test comes with this. Reproducing it takes an extension holding a stale RSTRING_PTR(),
which is not something the Ruby level suite can express.

Summary by CodeRabbit

• Bug Fixes
  • Fixed string concatenation when the source and destination overlap.
  • Self-appending and overlapping appends now remain correct even when the string buffer is resized.