Repository navigation
Fix String case mapping returning empty string when output grows - #10122
Conversation
|
@headius this was found while looking at jruby/jcodings#72 . |
|
@headius I'd like your call on a hardening question this raised. It's not needed for the fix here. Background. The question. The loop relies on every call consuming at least one input character. After jruby/jcodings#72 (the GHSA-34m4-m9fh-gqp5 port), the single-byte encodings won't start a character unless at least 2 bytes are free ( That can't happen today: every buffer is at least 20 bytes, and this is the only caller of Options:
I'm happy to add 2 and/or 3 to this PR and jcodings#72, or leave it. |
|
I think our work crossed in the air; I just fixed and released jcodings and joni separately to address differences with CRuby related to the recent security patches. I've got another agent updating jcodings and joni and making sure our handling of these conditions matches CRuby, and then we'll reevaluate this to see what's left. |
|
Great to hear! Let me know if I should close this or if you want to reevaluate it after your done on your side. |
headius
left a comment
There was a problem hiding this comment.
Codex review: The production fix is sound: new ByteList(tgtLen) reserves capacity but starts with logical size zero, and this multi-buffer branch copies into the backing array without updating that size. Setting the final size fixes the empty result.
Change requested: move this regression from test/jruby/test_string.rb into the shared spec/ruby/core/string specs (for example, the relevant upcase/capitalize/downcase specs). These assertions cover public Ruby behavior and the expected results pass on CRuby, so this belongs in the cross-implementation suite. Keep the expansion cases that exercise the multi-buffer path; if the description claims Symbol methods too, cover those public methods or narrow the claim.
The no-progress concern raised in the discussion is not demonstrated with current buffer sizing: JRuby's sole caller adds 20 bytes of headroom. I would leave that guard out absent a reachable zero-progress case. Separately, CRuby #20006's large-output integer-overflow guard is outside this fix; JRuby's StringSupport.caseMap still uses unchecked int accumulation and merits a distinct bounded-safety follow-up.
I had my agent review this in light of the other work and you still have a good fix here. Move the specs and we'll be good. |
|
Actually... could you make this against jruby-10.0 branch please? It's such a trivial fix for a valid bug. |
StringSupport.caseMap gathers output in several buffers when the mapped
string outgrows the first one (input length + 20). The combined ByteList
was created with new ByteList(tgtLen), which only reserves capacity, and
its size was never set, so the result came back empty:
("ΐ" * 5).upcase # => "" (expected "Ϊ́" * 5)
Set the real size after copying the buffers in. This affects String and
Symbol upcase, downcase, swapcase and capitalize; add specs for each.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1cfe3cd to
2c747d1
Compare
|
Updated per review:
|
StringSupport.caseMapcollects output in several buffers when the mapped string outgrows the first one (input length + 20). When it joins them, the resultByteListis created withnew ByteList(tgtLen). That constructor only reserves capacity, and the code never set the size, so the result came back empty:This affects String and Symbol
upcase,downcase(including:fold),swapcaseandcapitalizewhose output grows by more than about 20 bytes, in UTF-8 or any other encoding. The bug has been there since 22d9d36 (2018). The fix is one line: set the real size after copying the buffers in.Specs
I added a "returns the whole result when it is much longer than self" example to each of these:
spec/ruby/core/string/{upcase,downcase,swapcase,capitalize}_spec.rb, plus adowncase(:fold)casespec/ruby/core/symbol/{upcase,downcase,swapcase,capitalize}_spec.rbResults:
casecmp_spec.rb).Related
I found this while checking JRuby against GHSA-34m4-m9fh-gqp5, the CRuby sharp s heap overflow. The jcodings side of that is in jruby/jcodings#73 (released in 1.0.66), which superseded jruby/jcodings#72.
🤖 Generated with Claude Code