Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
159 changes: 83 additions & 76 deletions mrbgems/mruby-regexp/mrblib/string_regexp.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,32 +9,27 @@
# check to reject everything that is not a Regexp. `[]`, `[]=` and `slice!`
# read the real class with `Regexp === pattern` and leave anything else to the
# built-in method they aliased. `=~` rejects a String, which would recurse
# back into this method, and hands anything else to the argument's own `=~`,
# as CRuby does.
# back into this method, and hands anything that is not a Regexp to the
# argument's own `=~`, as CRuby does.
#
# That is where the guarantee ends. With the type established, each override
# calls ordinary methods on the pattern (`match`, `match?`, `=~`,
# `__byte_match`, `__sub_str`, `__gsub_str`, `__scan`) and on the MatchData it
# hands back (`[]`, `pre_match`, `post_match`, `begin`, `end`, `size`,
# `length`, `__byte_begin`, `__byte_end`, `__set_globals`). Every one of
# those is defined with `mrb_define_method()`, so a singleton method on the
# pattern replaces it and the override follows the replacement; the `__`
# prefix is a naming convention, not a protection.
# With the type established, each override reaches the engine through class
# methods that take the pattern as an argument (`Regexp.__search`,
# `__byte_search`, `__search_p`, `__sub_str`, `__gsub_str`, `__scan`), so
# nothing rewritten on the pattern instance is consulted on the way: the C
# side searches, and the loops and blocks stay here. The MatchData those
# searches answer is built in C too, so what the overrides read from it
# (`[]`, `pre_match`, `begin` and the rest) cannot have been planted by an
# argument. What remains reachable is a class-wide redefinition of a
# MatchData method, which is the same category as redefining `String#sub`
# itself and is not steered by the argument.
#
# CRuby is open the same way in `rb_str_match_m()`, which dispatches `match` to
# the pattern on purpose, and closed everywhere else: `rb_str_sub_bang()`,
# `rb_str_subpat()`, `rb_str_subpat_set()` and `rb_str_slice_bang()` call
# `rb_reg_search()` directly, and `String#match?` and `String#=~` search a real
# Regexp without asking it anything. The overrides here dispatch on every
# path, so a rewritten pattern reaches results CRuby would not give it.
#
# The gem accepts that rather than closing it. Reaching it takes rewriting a
# method on a Regexp instance and then passing that instance to a String
# method; nothing here is reachable from ordinary input, because an argument
# that is not a Regexp is rejected, compiled into a Regexp built here, or left
# to the built-in method the override aliased; `=~` alone forwards it to the
# argument, which CRuby does too. The type checks in front of the searches
# are claims about the argument, not about a pattern its owner has rewritten.
# `String#match` is the one deliberate exception: it dispatches `match` to
# the pattern because CRuby's `rb_str_match_m()` does so on purpose, and
# following that is the correct behaviour, not a hole. The `=~` forward for
# a non-Regexp argument is the same shape, from `rb_str_match()`. Everywhere
# else the gem is closed the way CRuby is: `rb_str_sub_bang()`,
# `rb_str_subpat()` and the rest reach the engine without asking the pattern
# anything.
class String
# Capture the C-defined String#split under `__split` before the override
# below replaces it, so the override can delegate non-regexp patterns
Expand Down Expand Up @@ -78,10 +73,13 @@ def match(re, pos = 0, &block)
re.match(self, pos, &block)
end

# Unlike `match`, the search does not dispatch on the pattern: CRuby's
# `rb_str_match_m_p()` resolves the argument and searches it directly,
# where `rb_str_match_m()` sends `match` to it on purpose.
def match?(re, pos = 0)
re = Regexp.__check_pattern(re)
re = Regexp.new(re) if String === re
re.match?(self, pos)
Regexp.__search_p(re, self, pos)
end

def =~(re)
Expand All @@ -90,6 +88,13 @@ def =~(re)
# redefinable, so a String subclass denying its own type would slip past
# the guard and recurse anyway; `Module#===` reads the real type.
raise TypeError, "type mismatch: String given" if String === re
# A real Regexp is searched here rather than asked, as CRuby's
# `rb_str_match()` does: it sends `=~` to the argument only when the
# argument is not a Regexp, which is what the tail below keeps doing.
if Regexp === re
md = Regexp.__search(re, self)
return md && md.begin(0)
end
re =~ self
end

Expand All @@ -113,9 +118,9 @@ def sub(*args, &block)
pattern = Regexp.new(Regexp.escape(pattern)) if String === pattern
# A replacement argument wins over the block, as in CRuby.
if args.length == 2
return pattern.__sub_str(self, replacement.to_s)
return Regexp.__sub_str(pattern, self, replacement.to_s)
end
md = pattern.match(self)
md = Regexp.__search(pattern, self)
return self.dup unless md
md.pre_match + block.call(md[0]).to_s + md.post_match
end
Expand All @@ -140,8 +145,8 @@ def sub!(*args, &block)
raise FrozenError, "can't modify frozen String" if frozen?
# Whether a substitution happened is a question about the match, not about
# the result: `"aaa".sub!(/a/, "a")` returns self even though the string is
# unchanged. `match` and not `match?`, so a failed match clears $~.
return nil unless pattern.match(self)
# unchanged. A full search and not `match?`, so a failed match clears $~.
return nil unless Regexp.__search(pattern, self)
# `sub` matches again and publishes its own $~ over this one, leaving the
# caller the match `sub` would have left, a block's own matches included.
# The resolved pattern takes the place of the original argument so that a
Expand All @@ -167,20 +172,20 @@ def gsub(*args, &block)
pattern = Regexp.new(Regexp.escape(pattern)) if String === pattern
# A replacement argument wins over the block, as in CRuby.
if args.length == 2
return pattern.__gsub_str(self, replacement.to_s)
return Regexp.__gsub_str(pattern, self, replacement.to_s)
end
# block case: keep in Ruby to avoid VM callback from C
parts = []
pos = 0
len = self.bytesize
binary = Regexp.__binary_string?(self)
# The loop normally ends on a failed __byte_match, which clears $~ and the
# thirteen names that go with it. CRuby leaves the last match behind, so
# keep it and republish it below. A gsub that matched nothing has nothing
# to restore and keeps the cleared state, as CRuby does.
# The loop normally ends on a failed __byte_search, which clears $~ and
# the thirteen names that go with it. CRuby leaves the last match behind,
# so keep it and republish it below. A gsub that matched nothing has
# nothing to restore and keeps the cleared state, as CRuby does.
last = nil
while pos <= len
md = pattern.__byte_match(self, pos)
md = Regexp.__byte_search(pattern, self, pos)
break unless md
last = md
# gsub works in byte space (match pos, byteslice). begin/end report
Expand Down Expand Up @@ -223,18 +228,18 @@ def gsub!(*args, &block)
return to_enum(:gsub!, *args) if args.length == 1 && !block
pattern = Regexp.__check_pattern(args[0])
pattern = Regexp.new(Regexp.escape(pattern)) if String === pattern
# As in `sub!`: the match decides the return value, and `match` clears $~
# when it fails. What it publishes on success is replaced right away by
# the last match of the `gsub` below, which is the one CRuby leaves behind.
return nil unless pattern.match(self)
# As in `sub!`: the match decides the return value, and a failed search
# clears $~. What it publishes on success is replaced right away by the
# last match of the `gsub` below, which is the one CRuby leaves behind.
return nil unless Regexp.__search(pattern, self)
str = args.length == 2 ? self.gsub(pattern, args[1], &block) : self.gsub(pattern, &block)
self.replace(str)
end

def scan(pattern)
pattern = Regexp.__check_pattern(pattern)
pattern = Regexp.new(Regexp.escape(pattern)) if String === pattern
result = pattern.__scan(self)
result = Regexp.__scan(pattern, self)
if block_given?
result.each { |m| yield m }
self
Expand Down Expand Up @@ -283,7 +288,7 @@ def split(pattern = nil, *args)
result << (self.byteslice(field_start..-1) || "")
return result
end
md = pattern.__byte_match(self, search_pos)
md = Regexp.__byte_search(pattern, self, search_pos)
break unless md
match_start = md.__byte_begin(0)
match_end = md.__byte_end(0)
Expand Down Expand Up @@ -350,11 +355,11 @@ def [](*args)
if args.length > 2
raise ArgumentError, "wrong number of arguments (given #{args.length}, expected 1..2)"
end
# `match` and not `match?`: the match globals have to be published here,
# including the clearing a failed match does, and `match?` leaves them
# alone. That is why the MatchData is fetched even with no capture
# A full search and not `match?`: the match globals have to be published
# here, including the clearing a failed match does, and `match?` leaves
# them alone. That is why the MatchData is fetched even with no capture
# argument, where only its `[0]` is used.
md = args[0].match(self)
md = Regexp.__search(args[0], self)
return nil unless md
# The capture argument reaches `MatchData#[]` untouched: it already
# normalizes a negative index, answers nil for an index past the last
Expand Down Expand Up @@ -383,13 +388,13 @@ def []=(*args)
unless args.length == 2 || args.length == 3
raise ArgumentError, "wrong number of arguments (given #{args.length}, expected 2..3)"
end
# `match` and not `match?`, so that the match globals are published here
# including the clearing a failed match does. CRuby searches before it
# checks the receiver for modification, which makes the order observable:
# a frozen receiver still leaves the match behind, and a pattern that does
# not match raises IndexError rather than FrozenError. Letting the
# mutation below be what raises reproduces both.
md = args[0].match(self)
# A full search and not `match?`, so that the match globals are published
# here including the clearing a failed match does. CRuby searches before
# it checks the receiver for modification, which makes the order
# observable: a frozen receiver still leaves the match behind, and a
# pattern that does not match raises IndexError rather than FrozenError.
# Letting the mutation below be what raises reproduces both.
md = Regexp.__search(args[0], self)
raise IndexError, "regexp not matched" unless md
group = args.length > 2 ? args[1] : 0
if Integer === group
Expand Down Expand Up @@ -434,7 +439,7 @@ def slice!(*args)
# the C check is not, but no other route to that check leaves the string
# alone on the way.
raise FrozenError, "can't modify frozen String" if frozen?
md = args[0].match(self)
md = Regexp.__search(args[0], self)
return nil unless md
group = args.length > 1 ? args[1] : 0
if Integer === group
Expand Down Expand Up @@ -467,12 +472,13 @@ def index(*args)
if args.length > 2
raise ArgumentError, "wrong number of arguments (given #{args.length}, expected 1..2)"
end
# `Regexp#match` normalizes a position the way `index` does and reads it
# with the same `mrb_get_args()` conversion, so the argument goes over
# `Regexp.__search` normalizes a position the way `index` does and reads
# it with the same `mrb_get_args()` conversion, so the argument goes over
# unexamined: a negative one counts back from the end, and one that lands
# outside the subject answers nil after clearing the match globals.
# `match` and not `match?`, because those globals are part of the answer.
md = args.length > 1 ? args[0].match(self, args[1]) : args[0].match(self)
# A full search and not `match?`, because those globals are part of the
# answer.
md = args.length > 1 ? Regexp.__search(args[0], self, args[1]) : Regexp.__search(args[0], self)
# `begin` reports character offsets, which is the space `index` answers
# in; `byteindex` below is the same search read in the other space.
md && md.begin(0)
Expand All @@ -492,15 +498,15 @@ def index(*args)
def __regexp_rsearch(pattern, limit, bytes)
found = nil
pos = 0
while (md = pattern.match(self, pos))
while (md = Regexp.__search(pattern, self, pos))
break if (bytes ? md.__byte_begin(0) : md.begin(0)) > limit
found = md
pos = md.begin(0) + 1
end
# The loop leaves behind whatever its last `match` published: the clear a
# The loop leaves behind whatever its last search published: the clear a
# failed one does, or a match past `limit` that is not the answer. Both
# have to give way to what the search found.
found ? found.__set_globals : pattern.match(nil)
found ? found.__set_globals : Regexp.__search(pattern, nil)
found
end

Expand All @@ -523,7 +529,7 @@ def rindex(*args)
pos += len
# Out of the subject at the negative end is a miss, and a miss
# clears the match globals.
return args[0].match(nil) if pos < 0
return Regexp.__search(args[0], nil) if pos < 0
elsif pos > len
# Past the other end is not: `rindex` searches back from the end of
# the subject, and `mrb_str_byterindex_m()` clamps for the same
Expand All @@ -548,14 +554,14 @@ def byteindex(*args)
pos = Integer.__ensure(args[1])
pos += len if pos < 0
end
# `__byte_match` takes the position as given and does not range check it,
# where `Regexp#match` answers nil for one outside the subject. Both
# ends are a miss here, as they are for `mrb_str_byteindex_m()`. An
# `__byte_search` takes the position as given and does not range check
# it, where `Regexp.__search` answers nil for one outside the subject.
# Both ends are a miss here, as they are for `mrb_str_byteindex_m()`. An
# offset that lands inside a character is not an error: the C method does
# not check for one either, and on a build without MRB_UTF8_STRING there
# is nothing to check.
return args[0].match(nil) if pos < 0 || pos > len
md = args[0].__byte_match(self, pos)
return Regexp.__search(args[0], nil) if pos < 0 || pos > len
md = Regexp.__byte_search(args[0], self, pos)
md && md.__byte_begin(0)
end

Expand All @@ -572,7 +578,7 @@ def byterindex(*args)
pos = Integer.__ensure(args[1])
if pos < 0
pos += len
return args[0].match(nil) if pos < 0
return Regexp.__search(args[0], nil) if pos < 0
elsif pos > len
pos = len
end
Expand All @@ -585,7 +591,7 @@ def byterindex(*args)
# (aliased as `__partition` above) for every other argument.
def partition(sep)
return __partition(sep) unless Regexp === sep
md = sep.match(self)
md = Regexp.__search(sep, self)
# No match leaves the whole subject in the head, and the copy is a plain
# String even when the receiver is a subclass, as `mrb_str_dup()` and
# CRuby's `str_duplicate(rb_cString, str)` both hand back.
Expand Down Expand Up @@ -615,16 +621,17 @@ def start_with?(*args)
while i < args.length
arg = args[i]
if Regexp === arg
# A regexp is anchored at the start, not searched for, while `match`
# searches forward from its position. The engine matches leftmost,
# so a pattern that can match at 0 does, which makes `begin(0) == 0`
# the anchored answer rather than an approximation of it.
md = arg.match(self)
# A regexp is anchored at the start, not searched for, while the
# search runs forward from its position. The engine matches
# leftmost, so a pattern that can match at 0 does, which makes
# `begin(0) == 0` the anchored answer rather than an approximation
# of it.
md = Regexp.__search(arg, self)
return true if md && md.begin(0) == 0
# A match further along is not an answer and CRuby leaves none
# behind for one, so clear what the search published. Matching
# against nil is how a Regexp clears the globals.
arg.match(nil) if md
# behind for one, so clear what the search published. Searching
# nil is how the globals are cleared.
Regexp.__search(arg, nil) if md
elsif __start_with?(arg)
return true
end
Expand Down
Loading
Loading