Repository navigation
Add the alignment directives x! and @! to pack and unpack (Feature #22185) - #10023
Conversation
98c2893 to
85fbd15
Compare
|
@headius A question about spec coverage before I mark this ready.
Nothing in Should I propose a spec for this to ruby/spec (in |
|
@headius This is ready for review. It covers the There are two open questions for you. #10022 covers the five |
headius
left a comment
There was a problem hiding this comment.
I have reviewed this, but since there's a lot of ported code I've asked Codex to do an additional review of style and contributor recommendations. There's a few changes needed to bring this up to standards that I see immediately:
- The
@CRubyannotations should include anotewhen they are only a part of the original method, so that in future audits we can quickly correlate them with the right code. - Tests should never be added to test/mri, since those tests are copied from CRuby periodically. If there are gaps in those tests, they can be submitted upstream for ruby/ruby, but otherwise we strongly prefer that Ruby behaviors gain specs in spec/ruby (for the ruby/spec repository).
- There's a lot of features added here that probably could have been separate, smaller PRs. Large PRs are expensive to review (in time for humans and tokens for agents) and increase the chance of missing flaws.
headius
left a comment
There was a problem hiding this comment.
Please make the small traceability update in the inline comment, following the recently added CONTRIBUTING guidance.
| **/ | ||
| private static final String NATIVE_CODES = "sSiIlLjJ"; | ||
| private static final String MAPPED_CODES = "sSiIqQjJ"; | ||
| /** MRI's natstr and endstr: the types that take '_'/'!' or '<'/'>' in an x!/@! alignment. */ |
There was a problem hiding this comment.
This field explicitly ports MRI's natstr/endstr sets from pack.c. The recently merged CONTRIBUTING recommendation asks for @CRuby annotations on ported fields too. Please annotate this mapping so it is traceable during future syncs.
There was a problem hiding this comment.
Annotated as @CRuby("natstr") and @CRuby("endstr"), with a note that endstr is defined as natstr when the platform has long long. That is the case on every 64-bit platform JRuby models, so both are "sSiIlLqQjJ". I also replaced the "MRI's natstr and endstr" Javadoc with a description of what the field is for.
Any visible behavior that could reasonably be considered a "feature" of Ruby should get a spec in ruby/spec. You can commit these directly to our spec/ruby copy, which is periodically synced with the main repository. Tests should never be added to our copy of CRuby's tests in test/mri, as mentioned in my review and the recent update to the contributor documents. My agent will be adding some additional review comments based on that style review. |
85fbd15 to
1c2f40e
Compare
|
@headius Thanks for the review. I've split this up:
About test/mri: this PR didn't add or change tests there. The |
|
@headius split the pr into smaller prs. It might have lost some of your comments, but they should be addressed. |
…2185)
x!N and @!N move to the next multiple of N bytes, and x!T and @!T to the
C alignment of directive T (x!i, x!l_). x! counts from the start of the
buffer: contents in pack and from the offset: argument in unpack; @!
counts from the start of the string. pack pads with NULs; unpack raises
if the boundary is past the end of the string. Alignments follow MRI on
64-bit platforms, with l_ at 8 bytes like JRuby's native long.
Adds specs for x! counting from buffer: and offset:, the modifiers
after an alignment type, and the alignment errors, including MRI's
order for a repeated endian modifier ("x!s<>" raises RangeError before
the endian modifier is rejected).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1c2f40e to
b3b792d
Compare
Implements the
x!and@!alignment directives from Feature #22185, an item on the #9746 checklist, for both pack and unpack.This PR started with all of the Ruby 4.1 pack/unpack changes. It was split up during review, and the rest are now:
unpack1returningnil(againstjruby-10.0)offset:(Feature #21979)R/r(Feature #21785)^(Feature #21796)The
test_pack.rbsync will come in its own PR once these land. #10022 covers the excludes it needs.[1, 2].pack("C x!4 C")"\x01\x00\x00\x00\x02"ArgumentError: '33' allowed only after types sSiIlLjJ"\x01\x00\x00\x00\x02""\x01\x00\x00\x00\x02".unpack("C x!i C")[1, 2]'33'error[1, 2][1, 2].pack("C x!4 C", buffer: +"z")"z\x01\x00\x00\x00\x02"'33'error"z\x01\x00\x00\x00\x02"[1, 2].pack("C @!4 C", buffer: +"z")"z\x01\x00\x00\x02"'33'error"z\x01\x00\x00\x02"[1].pack("C x!s<")ArgumentError: endian modifier is not allowed for alignment'33'error[1].pack("C x!s<>")RangeError: Can't use both '<' and '>''33'errorChanges
All in
core/src/main/java/org/jruby/util/Pack.java:parseAlignmentportspack_alignment_size,pack_alignmentand the alignment use ofpack_modifiers.alignmentOfportspack_alignof, using fixed 64-bit alignments withl_at 8 bytes, like JRuby's native long.alignmentPadportspack_align_pad.NATINT_ALIGN_CODESis annotated as MRI'snatstr/endstr.x!/@!check runs before the_/!native-modifier handling, which used to rejectx!with the'33'error.x!counts from the start of thebuffer:contents in pack, and fromoffset:in unpack.@!counts from the start of the string.invalid alignment,alignment too big, the endian modifier error, the unknown alignment directive,x/@ outside of string, andstring size too big.RangeErrorfirst. A comment says so.Specs
spec/ruby/core/array/pack/x_spec.rbandspec/ruby/core/string/unpack/x_spec.rb, guarded byruby_version_is "4.1". They cover:x!counting frombuffer:andoffset:_/!modifiers after the alignment typex!s<>order discussed above<immediately, the two<>examples fail, so they test the order.unsigned longand would differ on 32-bit CRuby.Testing
x_spec.rb(pack and unpack): 71 examples, 0 failures, 0 errors, interpreted and with forced JIT. Before, the 6 existingx!/@!examples errored.spec/ruby/core/array/pack,spec/ruby/core/string/unpack,unpack_spec.rb,unpack1_spec.rb: 0 failures. The errors drop from 63 to 57, and there are 15 new examples.test/mri/ruby/test_pack.rb: 41 tests, 0 failures._/!/</>modifiers,buffer:, offsets, and invalid and huge alignments. I ran it with all of the split pack PRs applied, because some cases use negative offsets.Not in this PR
q!/Q!: other directives still reject them, because JRuby's native-modifier list has never includedqQ. The error also prints the modifier as'33'. The new alignment parser acceptsq!/Q!and formats its message like CRuby. I'll send the fix for the other directives as a separate PR tojruby-10.0.🤖 Generated with Claude Code