Skip to content

mruby-regexp: reject overlong and out-of-range UTF-8 sequences #7068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
matz merged 1 commit into from
Aug 10, 2026
Merged

mruby-regexp: reject overlong and out-of-range UTF-8 sequences #7068

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 18 additions & 6 deletions mrbgems/mruby-regexp/src/re_utf8.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,23 +8,35 @@

/* Return byte length of UTF-8 character at s.
Returns 1 for invalid sequences (treat as single byte): a byte that starts
no sequence, a sequence cut short by end, and one whose continuation bytes
are not 10xxxxxx. */
no sequence, a sequence cut short by end, one whose continuation bytes are
not 10xxxxxx, and one that spells a codepoint a shorter sequence already
holds, a surrogate, or a value above U+10FFFF. The last three are decided
by the lead byte together with the first continuation byte. */
int
mrb_re_utf8_charlen(const char *s, const char *end)
{
uint8_t c = (uint8_t)*s;
uint8_t lo = 0x80, hi = 0xbf; /* range the first continuation byte must fall in */
int len;

if (c < 0x80) return 1;
else if (c < 0xc0) return 1; /* invalid continuation */
else if (c < 0xc2) return 1; /* continuation byte, or an overlong two-byte lead */
else if (c < 0xe0) len = 2;
else if (c < 0xf0) len = 3;
else if (c < 0xf8) len = 4;
else if (c < 0xf0) {
len = 3;
if (c == 0xe0) lo = 0xa0; /* below U+0800 */
else if (c == 0xed) hi = 0x9f; /* U+D800 to U+DFFF */
}
else if (c < 0xf5) {
len = 4;
if (c == 0xf0) lo = 0x90; /* below U+10000 */
else if (c == 0xf4) hi = 0x8f; /* above U+10FFFF */
}
else return 1; /* invalid */

if (s + len > end) return 1; /* truncated */
for (int i = 1; i < len; i++) {
if ((uint8_t)s[1] < lo || (uint8_t)s[1] > hi) return 1;
for (int i = 2; i < len; i++) {
if (((uint8_t)s[i] & 0xc0) != 0x80) return 1; /* invalid continuation */
}
return len;
Expand Down
31 changes: 31 additions & 0 deletions mrbgems/mruby-regexp/test/regexp.rb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2168,6 +2168,37 @@ def -(other)
assert_equal 0, ("ab\xf0" =~ /[^cd]+$/)
end

assert("Regexp - overlong UTF-8 is not the character it spells") do
# C0 BC is the two-byte overlong spelling of "<" and E0 84 80 the three-byte
# spelling of "Ā". A class compares the decoded codepoint and a literal
# compares bytes, so a decoder that hands out a codepoint for these makes the
# two disagree about the same subject: assert them together.
assert_nil ("\xC0\xBC" =~ /[<]/)
assert_nil ("\xC0\xBC" =~ /</)
assert_equal 0, ("\xC0\xBC" =~ /[^<]/)
assert_equal "\xC0\xBC", "\xC0\xBC".gsub(/[<]/, "&lt;")
assert_nil ("\xE0\x80\xBC" =~ /[<]/)
assert_false Regexp.new("[Ā]").match?("\xE0\x84\x80")
assert_false (/Ā/.match?("\xE0\x84\x80"))
# the pattern side decodes through the same helper
assert_false Regexp.new("[\xC0\xBC]").match?("<")
# surrogates and codepoints above U+10FFFF encode no character either, so
# each byte stands on its own
assert_equal 2, "\xC0\xBC".scan(/./).size
assert_equal 3, "\xED\xA0\x80".scan(/./).size
assert_equal 4, "\xF0\x80\x80\xBC".scan(/./).size
assert_equal 4, "\xF5\x80\x80\x80".scan(/./).size
Comment thread
coderabbitai[bot] marked this conversation as resolved.
# the shortest spelling on each side of those bounds is still one character
assert_equal 1, "\u{0080}".scan(/./).size # C2 80
assert_equal 1, "\u{0800}".scan(/./).size # E0 A0 80
assert_equal 1, "\u{D7FF}".scan(/./).size # ED 9F BF
assert_equal 1, "\u{E000}".scan(/./).size # EE 80 80
assert_equal 1, "\u{10000}".scan(/./).size # F0 90 80 80
assert_equal 1, "\u{10FFFF}".scan(/./).size # F4 8F BF BF
assert_equal 0, ("\u{0800}" =~ Regexp.new("[\u{0800}]"))
assert_equal 0, ("\u{10FFFF}" =~ Regexp.new("[\u{10FFFF}]"))
end

assert("Regexp - pattern too large for its jump targets is refused") do
# Jump targets live in a 16-bit field, so a program that outgrows the field
# used to wrap them and jump to an unrelated instruction: the pattern then
Expand Down
Loading