Read the pieces partition cuts the way the whole was read - #7146
Conversation
A piece cut out of a byte-read string carries the byte reading with it since 7806a13, and `split`, `chars`, `scan` and `byteslice` all hand back pieces that answer as bytes. `partition` and `rpartition` cut the same bytes and their pieces answer UTF-8, over bytes that refuse to read as it, one derivation away from the state `Integer#chr` stopped handing out. The three pieces do not all come from one place. The head and the tail are subranges of the receiver. The middle piece is the separator that was handed in, except where none is found: then there is none to hand back, and the two empty pieces stand for places in the receiver instead. Only the separator carries a reading of its own today, being handed back through `mrb_str_dup`. Pin where every answer stands before any of it moves: a separator found, a separator found nowhere, an empty separator, a separator read as bytes in a receiver read as UTF-8, and a receiver that stays UTF-8 throughout.
…was read The head and the tail are subranges of the receiver's bytes, so they carry the receiver's reading, the way every other cut of a byte-read string already does. The empty pieces left where a separator is found nowhere stand for places in the receiver and carry it too. The separator piece is the argument itself and goes on being read the way the argument was, which `mrb_str_dup` already gives it; an empty separator now comes back through that same dup rather than as a fresh literal, so it too answers the way it was handed in. `str_cut_piece` says the rule once for both methods: cut the bytes, then read them the way the string they were cut from was read. An empty piece goes through it as a cut of no bytes, which is what it is, rather than through `mrb_str_new_lit` where no reading is at hand to carry.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
📝 WalkthroughWalkthrough
ChangesPartition binary-state preservation
Merge Risk: ⚪ Minimal · up to This localized change makes partitioned string pieces preserve their intended encoding behavior, with the stated test suite passing; no actionable merge-blocking risk remains beyond normal checks and review. Possibly related PRs
Suggested reviewers: 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Follow-up to #7136, which carried the byte reading into every cut of a string but this one.
String#partitionandString#rpartitionbuild their pieces withmrb_str_new()and never look at what the string they cut was read as, so the pieces come back reading UTF-8 over bytes that refuse to read as it:Since 80c781b refuses a subject whose bytes do not read as UTF-8, this is no longer a mis-indexing but a refusal:
b[1, 2] =~ /b/answers, whileb.partition("a")[2], the same two bytes, raisesArgumentError.The rule
The three pieces do not all come from one place, and CRuby reads each accordingly.
mrb_str_dup()already gives it. An empty separator now comes back through that same dup rather than as a fresh literal, so it too answers the way it was handed in.str_cut_piece()says the first and the last of these once for both methods: cut the bytes, then read them the way the string they were cut from was read. An empty piece goes through it as a cut of no bytes, which is what it is, rather than throughmrb_str_new_lit()where no reading is at hand to carry.This reproduces CRuby's answer for every pair it accepts. The pairs CRuby refuses with
Encoding::CompatibilityErrorare separators found nowhere here, as they already were, so"aあb".partition(171.chr)hands the receiver back whole and says nothing about the reading rather than something false, the same choice #7137 made.Tests
The first commit pins where every answer stands before any of it moves; the second flips exactly the pins it changes. Both sit in mruby-encoding's test file next to the cuts #7136 pinned, since that gem already takes mruby-string-ext as a test dependency and the reading is only visible through it.
Verification
Full suite green at every commit, on full-core with
MRB_UTF8_STRING(2281 tests), the same with the C++ ABI (2281), and the default gembox, where mruby-encoding is absent and the new test skips (2066). 0 failures, 0 crashes, no new compiler warnings.Still left out
Array#pack/String#unpackand the Symbol round trip, the two #7143 named. Each is its own change.Summary by CodeRabbit
String#partitionandString#rpartitionto preserve byte-oriented string behavior across extracted pieces.