Repository navigation
Conversation
Member
|
#10122 has been merged! |
StringSupport.caseMap added up the result length in an int and sized
each buffer as remaining * buffer count + 20 in int arithmetic. For a
result over 2 GB the total wrapped negative and the case conversion
failed with a raw Java exception:
("ΐ" * 400_000_000).upcase
# => Java::JavaLang::NegativeArraySizeException: -1894967296
Add up the total in a long and raise ArgumentError "string size too big"
once it exceeds the largest Java array, as CRuby does since
ruby/ruby#20006. Compute each buffer size in a long and cap it at the
largest array; caseMap stops at the end of the buffer and the loop
continues from there, so no up-front limit on the source is needed.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
zev
force-pushed
the
fix-casemap-size-overflow-10.0
branch
from
November 20, 2026 06:41
7cb6b9e to
054c51e
Compare
zev
marked this pull request as ready for review
November 20, 2026 06:44
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #10122, as suggested in its review. Rebased onto
jruby-10.0now that #10122 is merged.This ports ruby/ruby#20006 ("Fix an integer overflow in case conversion of strings").
Problem
StringSupport.caseMapadded up the result length in anint, and sized each buffer asremaining * bufferCount + 20inintarithmetic. When the result went over 2 GB, the total wrapped negative and case conversion failed with a raw Java exception:The buffer size calculation can wrap too. With 1.5 GB of input, the second buffer would be
remaining * 2 + 20. On the JVM none of this corrupts memory; it fails with the wrong exception.Fix
long. RaiseArgumentError, "string size too big", the same error CRuby raises, as soon as it exceedsHelpers.MAX_ARRAY_SIZE.longand cap it atHelpers.MAX_ARRAY_SIZE.CRuby also rejects up front any source longer than
INT_MAX - 20, because one C buffer has to hold the whole source plus headroom. JRuby doesn't need that limit. jcodings'caseMapstops at the end of the buffer and the loop continues frompp.value, so a capped buffer is safe. The only real limit is that the result has to fit in one Java array.Verification
Run with
-J-Xmx16gon the rebased branch (jcodings 1.0.67):"ΐ" * 400_000_000(800 MB → 2.4 GB)NegativeArraySizeExceptionArgumentError: string size too big"ΐ" * 750_000_000(1.5 GB → 4.5 GB)NegativeArraySizeExceptionArgumentError: string size too big"ΐ" * 300_000_000(600 MB → 1.8 GB)test_case_mapping.rb/test_case_comprehensive.rb: only the two Shift_JIS:asciitests fail. They're already intest/mri/excludes, and master fixed them in 9fba130.There's no regression spec, because reaching the limit needs several GB of heap. CRuby's change has no test for it either.
🤖 Generated with Claude Code