mruby-regexp: make an escaped multibyte literal one atom - #7066

bcacba1e1 made a multibyte literal a single atom, so /Ā+/ repeats the whole
character instead of its last byte. A backslash in front of the same character
takes a different branch of compile_atom(), which never got the same
treatment: the escape path emits only the lead byte and lets the parse loop pick
up the continuation bytes as atoms of their own. The quantifier then binds to
the last of those. The same backslash inside [...] goes through
read_class_atom(), which splits the character the same way, so the class ends
up holding two wrong codepoints.

Regexp.new("\\Ā+").match("ĀĀ")[0].bytesize  # CRuby: 4,     mruby: 2
Regexp.new("\\Ā{2}").match?("ĀĀ")           # CRuby: true,  mruby: false
Regexp.new("[\\Ā]").match?("Ā")             # CRuby: true,  mruby: false
Regexp.new("[\\Ā]").match?("Ä")             # CRuby: false, mruby: true
Regexp.new("\\Ā").match?("Ā")               # CRuby: true,  mruby: true

In CRuby /\Ā/ and /Ā/ are the same pattern: a backslash before a character
with no special meaning is just that character. Here they differ as soon as a
quantifier follows or a character class encloses them, which is exactly the bug
bcacba1e1 fixed for the unescaped spelling.

The /.../ literal spelling hides this, because the lexer strips the backslash
before the gem ever sees the pattern: /\Ā/.source is the two bytes of Ā
alone, so both /\Ā+/ and /[\Ā]/ behave. Regexp.new with a string, which is
how a pattern built at runtime arrives, does not.

Cause

compile_atom() has two paths that can see the bytes of a multibyte character.
The default path asks mrb_re_utf8_charlen() how long the character is and
emits every byte of it before returning, so the atom the quantifier sees is the
whole character. The escape path calls parse_escape(), whose default arm
returns the single byte it read, and emits that one byte as RE_CHAR.
compile_quantified() measures the atom as what was emitted between start and
code_len, which is now one byte, and the remaining 0x80 is left for the next
turn of compile_seq() to emit as its own atom. So \Ā+ compiles as \xC4
followed by (\x80)+, which matches C4 80 and stops.

read_class_atom() carries the same split. Its own multibyte path decodes a
whole codepoint through mrb_re_utf8_decode(), but the escape path hands the
backslash to parse_escape() and gets one byte back. The continuation byte is
then read as a class atom of its own, so [\Ā] enumerates U+00C4 and U+0080
where [Ā] enumerates U+0100.

Fix

A backslash before a byte at or above 0xC0 has no escape meaning in this
parser, so route it away from parse_escape() in both places, before that
function consumes the letter.

• compile_atom() emits the whole character through emit_char_bytes(),
  extracted from the default branch that already did this for the unescaped
  spelling, so there is one description of what a character atom is instead of
  two.
• read_class_atom() falls through to the mrb_re_utf8_decode() path that
  [Ā] already takes.

The invariant the two share, and the one compile_quantified() depends on, is
that an atom that consumes a character has to emit all of that character's bytes
before it returns.

parse_escape() itself is unchanged. Returning an int byte is right for every
escape it is meant to serve; the multibyte case just should not reach it.

Byte escapes are left as they are

\xNN and octal \NNN name a byte rather than a character, and the distinction
lives in the pattern text rather than in the value, which is why the dispatch has
to happen before parse_escape() reads the letter. They keep taking the old
path, so Regexp.new("\\xC4\\x80+") still repeats \x80 alone. CRuby joins byte
escapes that spell a valid UTF-8 sequence into one character and matches four
bytes there. That is a pre-existing difference and closing it belongs to a
separate change; the test added here pins the current behaviour so the next
person sees which way it goes.

Tests

mrbgems/mruby-regexp/test/regexp.rb gets a group next to the one bcacba1e1
added for the unescaped form, so the two spellings sit together. It covers the
quantifier and {n} forms, three and four byte characters, a quantified escape
after another atom, the non-greedy form, [\Ā] from both sides, an escaped range
inside a class, and one \xNN row for the byte escape above. Every assertion
except that last one matches CRuby 4.0.6.

rake test: 1991 OK, 0 KO.

Summary by CodeRabbit

• Bug Fixes

  • Fixed regular expressions so escaped UTF-8 characters are matched as complete characters.
  • Corrected quantifier behavior for escaped multibyte characters, including optional, bounded, greedy, and nongreedy patterns.
  • Preserved byte-oriented behavior for raw byte escapes.

• Tests

  • Added coverage for escaped two-, three-, and four-byte characters, character classes, ranges, and quantifier combinations.