Skip to content

Read the pieces partition cuts the way the whole was read - #7146

Merged
matz merged 2 commits into
mruby:masterfrom
takumin:binary-flag-through-partition
Aug 14, 2026
Merged

Read the pieces partition cuts the way the whole was read#7146
matz merged 2 commits into
mruby:masterfrom
takumin:binary-flag-through-partition

Conversation

@takumin

@takumin takumin commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Follow-up to #7136, which carried the byte reading into every cut of a string but this one.

String#partition and String#rpartition build their pieces with mrb_str_new() and never look at what the string they cut was read as, so the pieces come back reading UTF-8 over bytes that refuse to read as it:

b = "a\xABb".b

b.partition("a")   #=> ["", "a", "\xABb"]  head and tail UTF-8, invalid  (CRuby: ASCII-8BIT)
b.rpartition("b")  #=> ["a\xAB", "b", ""]  head and tail UTF-8, invalid  (CRuby: ASCII-8BIT)
b.partition("x")   #=> the two empty pieces UTF-8                        (CRuby: ASCII-8BIT)

b.split("a")       #=> ASCII-8BIT   (since #7136)
b.chars            #=> ASCII-8BIT
b[1, 2]            #=> ASCII-8BIT

Since 80c781b refuses a subject whose bytes do not read as UTF-8, this is no longer a mis-indexing but a refusal: b[1, 2] =~ /b/ answers, while b.partition("a")[2], the same two bytes, raises ArgumentError.

The rule

The three pieces do not all come from one place, and CRuby reads each accordingly.

  • The head and the tail are cut out of the receiver. They hold nothing but bytes of it, so they are read the way it is, ASCII bytes and all, which is what every other cut already does.
  • The middle piece is the separator that was handed in. It is read the way that was, which mrb_str_dup() already gives it. An empty separator now comes back through that same dup rather than as a fresh literal, so it too answers the way it was handed in.
  • Where the separator is found nowhere there is none to hand back. The two empty pieces stand for places in the receiver, so they carry the receiver's reading.

str_cut_piece() says the first and the last of these once for both methods: cut the bytes, then read them the way the string they were cut from was read. An empty piece goes through it as a cut of no bytes, which is what it is, rather than through mrb_str_new_lit() where no reading is at hand to carry.

This reproduces CRuby's answer for every pair it accepts. The pairs CRuby refuses with Encoding::CompatibilityError are separators found nowhere here, as they already were, so "aあb".partition(171.chr) hands the receiver back whole and says nothing about the reading rather than something false, the same choice #7137 made.

Tests

The first commit pins where every answer stands before any of it moves; the second flips exactly the pins it changes. Both sit in mruby-encoding's test file next to the cuts #7136 pinned, since that gem already takes mruby-string-ext as a test dependency and the reading is only visible through it.

Verification

Full suite green at every commit, on full-core with MRB_UTF8_STRING (2281 tests), the same with the C++ ABI (2281), and the default gembox, where mruby-encoding is absent and the new test skips (2066). 0 failures, 0 crashes, no new compiler warnings.

Still left out

Array#pack / String#unpack and the Symbol round trip, the two #7143 named. Each is its own change.

Summary by CodeRabbit

  • Bug Fixes
    • Improved String#partition and String#rpartition to preserve byte-oriented string behavior across extracted pieces.
    • Corrected encoding propagation for found, missing, and empty separators.
    • Ensured separator encoding affects only the appropriate partition result.

A piece cut out of a byte-read string carries the byte reading with it since
7806a13, and `split`, `chars`, `scan` and `byteslice` all hand back pieces
that answer as bytes. `partition` and `rpartition` cut the same bytes and
their pieces answer UTF-8, over bytes that refuse to read as it, one
derivation away from the state `Integer#chr` stopped handing out.

The three pieces do not all come from one place. The head and the tail are
subranges of the receiver. The middle piece is the separator that was handed
in, except where none is found: then there is none to hand back, and the two
empty pieces stand for places in the receiver instead. Only the separator
carries a reading of its own today, being handed back through `mrb_str_dup`.

Pin where every answer stands before any of it moves: a separator found, a
separator found nowhere, an empty separator, a separator read as bytes in a
receiver read as UTF-8, and a receiver that stays UTF-8 throughout.
…was read

The head and the tail are subranges of the receiver's bytes, so they carry the
receiver's reading, the way every other cut of a byte-read string already
does. The empty pieces left where a separator is found nowhere stand for
places in the receiver and carry it too. The separator piece is the argument
itself and goes on being read the way the argument was, which `mrb_str_dup`
already gives it; an empty separator now comes back through that same dup
rather than as a fresh literal, so it too answers the way it was handed in.

`str_cut_piece` says the rule once for both methods: cut the bytes, then read
them the way the string they were cut from was read. An empty piece goes
through it as a cut of no bytes, which is what it is, rather than through
`mrb_str_new_lit` where no reading is at hand to carry.
@takumin
takumin requested a review from matz as a code owner August 14, 2026 04:16
@coderabbitai

coderabbitai Bot commented Aug 14, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: b6451739-0ba2-47cb-b432-0ff6f1e96010

📥 Commits

Reviewing files that changed from the base of the PR and between 7fcad02 and 3aa8f39.

📒 Files selected for processing (2)
  • mrbgems/mruby-encoding/test/string.rb
  • mrbgems/mruby-string-ext/src/string.c

📝 Walkthrough

Walkthrough

String#partition and String#rpartition now preserve binary state on derived pieces. Tests cover byte-read and UTF-8 strings, empty and missing separators, and separator encoding behavior.

Changes

Partition binary-state preservation

Layer / File(s) Summary
Binary-aware substring helper
mrbgems/mruby-string-ext/src/string.c
Adds str_cut_piece to copy a byte range and preserve the source string’s binary flag.
Partition and rpartition handling
mrbgems/mruby-string-ext/src/string.c, mrbgems/mruby-encoding/test/string.rb
Updates matched, missing, and empty-separator pieces for both methods. Tests validate contents, encodings, and validity.
Estimated code review effort: 3 (Moderate) ~20 minutes

Merge Risk: ⚪ Minimal · up to 3aa8f

This localized change makes partitioned string pieces preserve their intended encoding behavior, with the stated test suite passing; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

  • mruby/mruby#7084: Updates string operations to preserve binary-string behavior in chop!.
  • mruby/mruby#7136: Updates derived string handling to preserve binary state.
  • mruby/mruby#7137: Updates binary-state preservation for derived strings in string.c.

Suggested reviewers: matz

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes that partitioned string pieces preserve the original string's reading or encoding.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@matz
matz merged commit 91beb1b into mruby:master Aug 14, 2026
21 checks passed
@takumin
takumin deleted the binary-flag-through-partition branch August 14, 2026 05:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants