Skip to content

Stream validated UTF-8 folds in String#casecmp? - #9958

Draft
headius wants to merge 2 commits into
jruby:masterfrom
headius:work/unicode-casecmp-perf
Draft

headius wants to merge 2 commits into
jruby:masterfrom
headius:work/unicode-casecmp-perf

Conversation

@headius

@headius headius commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Stream validated UTF-8 case folds into bounded buffers instead of allocating folded Strings. Validate both complete inputs before returning on a mismatch; other encodings retain the existing path. Separate from #9955.

The first commit fixes missing result length in multi-buffer case mapping: ("ΐ" * 4).downcase(:fold) previously became empty. Both changes have portable Ruby specs.

Historical benchmark-ips 2.15.1 results from before the rebase, iterations/second (higher is better). Baseline: 07e498eb6a plus the standalone mapping-length fix, before master gained #9955's direct ASCII fast path. Candidate: original PR head 099c0118a8.

Input Historical correctness-fix-only baseline Original PR head Ratio
Short Unicode, equal 2,869,968 4,907,319 1.71×
Fresh short Unicode 1,520,042 1,973,721 1.30×
Unicode mismatch, 6,200 B 8,392 641,476 76.44×
Byte-identical Unicode, 6,200 B 6,950 9,087 1.31×
Changed-case Unicode, 6,400 B 7,793 8,835 1.13×
Unicode expansions 115,745 144,734 1.25×

Linux x86_64/i7-10850H, Temurin 21.0.13+9, normal JIT, 512 MB heap; three JVMs/variant, alternating order, 2 s warmup/3 s measurement per case. Rates are medians of per-process benchmark-ips reports; ratios use those displayed medians. Fresh cases include construction; no reused-string encoding-reset timings are used here.

The large mismatch gain is early exit, not full-string folding throughput. Changed-case dense results vary by fork: one pair regressed 20%. A matched benchmark-ips control on CRuby 4.1dev d8444812f4 achieved 14,637 ips versus this PR’s 8,835 ips on that workload; universal CRuby parity is not achieved.

Reproduce: WARMUP=2 TIME=3 bin/jruby -J-Xms512m -J-Xmx512m bench/bench_string_casecmp_unicode.rb LABEL SAVE_PREFIX.

Validation: 84 examples/169 expectations pass interpreted, forced-compiled and CRuby 4.0.9; 1,112,064 Unicode scalars, 200,000 sequence comparisons, 100,000 valid/malformed-byte pairs, 2,266 encoding cases and shared-slice checks pass.

Rebased onto master db5b9ef6d9 on 2026-10-30. Upstream's ASCII fast path runs first; the compact UTF-8 helper and separate mapping-length fix are unchanged. Both upstream ASCII and Unicode specs are retained. After rebase, 95 examples/199 expectations pass in interpreted and forced-compiled JRuby; each mode also passes 2,266 encoding differential cases, 2,000 valid/malformed UTF-8 comparisons against full folding, and 30 frozen shared-slice checks with verified nonzero offsets. Earlier exhaustive Unicode validation was not repeated because the folding helper and correctness fix are unchanged.

The benchmark table above records the original pre-rebase comparison against its correctness-fix-only baseline. Master now includes the ASCII optimization; these historical figures are not a new comparison against current master. No benchmarks were rerun for this rebase.

Current master includes #9955's direct ASCII fast path, which returns before UTF-8 streaming. The former fresh-ASCII comparison used the older pre-ASCII baseline and does not establish an incremental gain from this rebased PR; ASCII rows have been removed from this table. The portable runner remains available for new comparisons.

@headius
headius force-pushed the work/unicode-casecmp-perf branch from 099c011 to fc3416b Compare October 30, 2026 08:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant