Skip to content

mruby-regexp: fold a byte-indexed subject by ASCII under a /i backreference #7264

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
matz merged 1 commit into from
Aug 18, 2026
+34 −1
Merged

mruby-regexp: fold a byte-indexed subject by ASCII under a /i backreference #7264

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion mrbgems/mruby-regexp/src/re_exec.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,18 @@ class_match(const re_charclass *cc, uint32_t cp, mrb_bool raw)
return cc->utf8_any;
}

/* The fold of one unit of the subject. A byte-indexed subject hands out
bytes, and a byte above 127 is not the codepoint of the same value: 0xC0 is
not U+00C0, so folding it to 0xE0 would pair two bytes that spell no letter
in common. The letters a byte can spell are the ASCII ones, and those fold
as they do everywhere. */
static inline uint32_t
subject_fold(uint32_t c, mrb_bool binary)
{
if (binary && c >= 128) return c;
return mrb_re_case_fold(c);
}

/* Compare two spans ignoring case. Returns how many bytes of `a` were
consumed, or -1 when they differ. The count is not always the length of
`b`: with Unicode folding a counterpart can be a different width (U+212A
Expand All @@ -77,7 +89,7 @@ memcmp_ci(const char *a, const char *a_end, const char *b, const char *b_end,
int alen = 0, blen = 0;
uint32_t ca = mrb_re_decode_char(a, a_end, &alen, binary);
uint32_t cb = mrb_re_decode_char(b, b_end, &blen, binary);
if (mrb_re_case_fold(ca) != mrb_re_case_fold(cb)) return -1;
if (subject_fold(ca, binary) != subject_fold(cb, binary)) return -1;
a += alen;
b += blen;
}
Expand Down
21 changes: 21 additions & 0 deletions mrbgems/mruby-regexp/test/regexp_utf8.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,27 @@
assert_equal 0, (Regexp.new(lead, Regexp::IGNORECASE) =~ lead.b)
end

assert("Regexp - a backreference under /i folds a byte-indexed subject by ASCII") do
# A byte-indexed subject hands the folded comparison bytes, and a byte above
# 127 is not the codepoint of the same value: 0xC0 is not U+00C0. The
# comparison folded it as if it were, so a build with the Unicode table
# paired 0xC0 with 0xE0 the way it pairs "À" with "à". The letters a byte
# can spell are the ASCII ones, and those still fold.
assert_nil ("\xC0\xE0".b =~ /(.)\1/i)
assert_nil ("\xC0a\xE0A".b =~ /(..)\1/i)
assert_equal 0, ("\xC0\xC0".b =~ /(.)\1/i)
assert_equal 0, ("\xC0a\xC0A".b =~ /(..)\1/i)
assert_equal 0, ("aA".b =~ /(.)\1/i)
# Nor does a byte take the fold of the character it is part of. Read as
# characters "s" and "ſ" fold alike (U+017F to 's', which every build
# carries), and the same bytes read one at a time have no character to fold.
# A skip here would drop the assertions above, so this is a branch.
if __ENCODING__ == "UTF-8"
assert_equal 0, ("sſ" =~ /(.)\1/i)
assert_nil ("sſ".b =~ /(.)\1/i)
end
end

assert("Regexp - quantifier on a multibyte literal") do
# The bytes of a multibyte literal used to be separate atoms, so a
# quantifier bound to the last one: /Ā+/ was \xC4(\x80)+ and stopped after
Expand Down
Loading