Repository navigation
Compute generated ABI record layouts in Fe - #1589
micahscopes wants to merge 11 commits into
Conversation
Rename the placeholder newsfragments to the pull request number, following the repository's <PR>.<type>.md convention.
669a95e to
3952964
Compare
Rename the placeholder newsfragments to the pull request number, following the repository's <PR>.<type>.md convention.
Rename the placeholder newsfragments to the pull request number, following the repository's <PR>.<type>.md convention.
1457f60 to
0413771
Compare
Rename the placeholder newsfragments to the pull request number, following the repository's <PR>.<type>.md convention.
Rename the placeholder newsfragments to the pull request number, following the repository's <PR>.<type>.md convention.
0413771 to
f5c46f7
Compare
Rename the placeholder newsfragments to the pull request number, following the repository's <PR>.<type>.md convention.
f5c46f7 to
aff67f8
Compare
cburgdorf
left a comment
There was a problem hiding this comment.
Generally looks good but here are a few things that should be addressed:
-
Split out unrelated commits.
- 2cf07e3 (packed record layout through raw pointer projections) has nothing to do with ABI records, and its commit message says it passes on the current compiler. Please move it to its own PR.
- I'd also move the layout recurrence fix (45904bb, structural_layout_type_embeds) to a separate PR. It is a standalone bugfix in the layout checker with its own newsfragment, and it needs a different kind
of review (does the checker still terminate on growing types) than the ABI refactor. It would also make this 75-file PR, with all its conflicts against other open PRs, easier to review and land.
-
present_generated_abi_const_diag is fragile. It suppresses duplicate diagnostics by:
- recognizing constants by name (LAYOUT/HEAD_SIZE/IS_DYNAMIC),
- comparing primary_goal.self_ty(db) for exact equality with a field type,
- collecting reported goals per module.
If the failing goal is on a nested type (e.g. Foo: AbiSize for a field Option), no field type matches and the diagnostic shows up twice. Could the constant itself say who reports its diagnostics
(extending AssocConstBodyCheckPolicy) instead of matching after the fact? If not, please at least add a uitest for a nested unsupported field type, e.g. an #[error]/msg field Option<StorageMap<…>>, so this
behavior is pinned down. -
Duplicate diagnostic in event_unsupported_field_type.snap. It now has three errors instead of two, and "StorageMap<…> doesn't implement AbiSize" appears twice (from dynamic_payload_size and from
encode_field_at). Naming the field type instead of a one-element tuple is an improvement, but the duplicate is noise. Please dedupe it, or note in the PR that it's intentional for now. -
ExpansionSourceCompatibility. Its doc comment calls it a compatibility exception ("New producers should use BodyAnalysis"). Could you open a follow-up issue so it doesn't become permanent?
Minor / design questions:
- #[event] structs implement AbiRecord over their data fields only, so Event: AbiRecord says nothing about the indexed fields. That's fine for avoiding an extra type, but the AbiRecord trait docs should
say so explicitly. - AbiRecordLayout, AbiRecord and abi_record_layout become public core::abi API. Are we happy with the names, and with tying the layout to the type rather than to the ABI (no A: Abi parameter), before this
lands?
Rename the placeholder newsfragment to the pull request number, following the repository's <PR>.<type>.md convention.
aff67f8 to
a29a4a5
Compare
|
Thanks for the careful review. Point by point:
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a29a4a50d1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| HirOrigin::Desugared(DesugaredOrigin::Msg(_)) => false, | ||
| HirOrigin::Desugared(DesugaredOrigin::Event(event)) => { | ||
| !indexed_dynamic_events.contains(event) | ||
| !indexed_dynamic_events.contains(event) && !record_fails(func) |
There was a problem hiding this comment.
Keep checking indexed topic fields when data layout fails
When an event has both a non-indexed field that fails the new ABI-record requirements and an indexed field without TopicValue (for example, two StorageMap fields), failing_records is set from the data field while indexed fields are deliberately excluded from record analysis. This filter then skips the entire generated emit body, so the indexed field's as_topic() call is never checked and its independent missing-topic diagnostic is hidden. Continue checking the topic-emission portion, or report indexed-field requirements separately, when only the data-record encoding has failed.
Useful? React with 👍 / 👎.
…ages The `#[error]` desugaring kept its own copies of the `#[event]` signature constant and of the `msg` `AbiSize` and `Encode` impl builders, so each fix had to be made twice: #1516 nested long `TOPIC0` tuples for events only, and #1592 added name chunking to both signature builders and tuple nesting to the `#[error]` one. Build both signature constants with one function, which takes the callee and the constant's name and type, and let errors use the `msg` impl builders, which now take the record type and its fields. The old `#[error]` selector builder and impl builders are deleted. An error's `SELECTOR` now names each field type as `<T as std::abi::SolCompat>::SOL_TYPE`, as `TOPIC0` does. Events and messages lower exactly as before. Add tests that an error with 8 fields reverts with the Solidity selector of its signature, and that the 7- and 8-field selectors evaluate to the `cast sig` values; nothing covered an error signature long enough to need nesting.
Whether a trait impl constant's body was checked depended on where the impl came from. Raw impls were checked. The `#[event]` and `#[error]` desugarings were checked with their body diagnostics suppressed, since an error in a field's source type would otherwise cascade into the generated body, and only the contextual CTFE diagnostic from #1522 was reported. Generated `msg` impls were skipped entirely. Record the choice on each constant instead. `AssocConstBodyCheckPolicy` defaults to `BodyAnalysis`, the ordinary check of the value against its expected type, and a generated origin alone no longer exempts a body. Message selectors keep their dedicated checker (`MsgSelectorAnalysis`), which reports their body errors before checking uniqueness. The existing event, error and message ABI constants are marked `ExpansionSourceCompatibility`, which keeps today's behavior: the body is checked, its diagnostics are suppressed, and an event or error body that type-checks still gets #1522's contextual CTFE diagnostic. A fieldless event has no field errors to cascade from, so its `TOPIC0` uses ordinary checking. No other diagnostics change.
Every `msg` variant and `#[error]` struct computed its ABI head offsets and head size in Rust, while lowering. Add `core::abi::AbiRecordLayout`, the `AbiRecord<N>` trait and the checked prefix sum `abi_record_layout`, and give each generated record an `AbiRecord<N>` impl whose `LAYOUT` is that prefix sum over its fields' head sizes. `HEAD_SIZE`, the encoder and the random-access decoder now read `LAYOUT` instead of repeating the arithmetic. `IS_DYNAMIC` stays independent of the layout, so a field's head size may depend on its record's dynamic flag. This changes the generated code of every `msg` variant and `#[error]` struct: encoders and decoders load offsets from `LAYOUT`. The new loads are constants, removed by the optimizer. The generated `LAYOUT` and `HEAD_SIZE` bodies are checked like other constants. When a field type lacks `AbiSize`, the error's generated encode and payload functions already report it, so a `HEAD_SIZE` evaluation failure in a generated `#[error]` impl is not reported again when an unsatisfied `AbiSize` goal for one of that record's field types was reported by those functions. A recursive `msg` field type is left to the analyses that own recursive and invalid types, as message field analysis already does, so it no longer adds duplicate `AbiSize` errors. Generated `msg` constants report evaluation failures in the same contextual form as `#[error]` constants, and a `msg` `HEAD_SIZE` or `LAYOUT` evaluation failure is not reported when message field analysis, or the analyses that own recursive and invalid types, already report one of that variant's fields. Message field analysis owns an invalid constant in a field type, such as a division by zero in an array length. Otherwise a field without a valid `AbiSize` would add a second error that points into `core::abi`. The generated constant names come from one place in lowering.
Rename the placeholder newsfragment to the pull request number, following the repository's <PR>.<type>.md convention.
The built-in tuple codecs in core::abi spelled out the same head-offset prefix sum by hand in every arity: `HEAD_SIZE` added up the field head sizes, the decoders kept a running `head_pos`, and the encoders and `AbiSpan` impls built `head1`, `head2`, ... chains. Each tuple arity from 1 to 16 now implements `AbiRecord<k>` with `abi_record_layout`, the checked prefix sum that generated `msg` and `#[error]` codecs already use. `HEAD_SIZE`, `decode_from`, `decode_from_prechecked_head`, `encode` and the `AbiSpan` impls read `<Self as AbiRecord<k>>::LAYOUT` instead of repeating the sum, so built-in and generated records share one definition of head layout. The encoding is unchanged. The ERC20 fixtures and five tuple and event fixtures compile to byte-identical deploy and runtime bytecode at O0, O1 and O2. Snapshots: - Three sonatina_ir snapshots of contracts that return or take a 2-tuple now show the layout constant loaded next to the folded offset (`const.ref $const_region_*`, unused) and renumbered SSA values. The offsets themselves still fold to literals, and the bytecode is the same. - Two ty_check snapshots cite `abi.fe` lines that moved down by two.
Generated `HEAD_SIZE` reads `<Self as AbiRecord<N>>::LAYOUT`, so every generated `AbiSize` impl needs a matching `AbiRecord` impl. Until now each caller (`msg` variants and `#[error]` structs) had to call `lower_abi_record_impl` before `lower_abi_size_impl`, and a new caller that forgot would emit a `HEAD_SIZE` naming a missing impl. `lower_abi_size_impl` now emits both impls. `create_head_size_assoc_const` takes the field count it actually uses instead of the field list, and the builders only used inside `msg.rs` become private. No behavior change. The two `#[error]` desugar snapshots list the `AbiRecord` impl after `ErrorVariant` instead of before it, the same order `msg` variants already use.
`emit` packed the non-indexed fields of an `#[event]` into a tuple and passed it to `std::evm::encode_abi_payload`. The tuple codecs stop at 16 elements, so an event with 17 or more data fields failed to compile with "`(u256, ...)` doesn't implement `Encode<Sol>`". Events now use the record path that `msg` variants and `#[error]` structs use. Lowering emits `impl AbiRecord<N>` for the event over its data fields, and `emit` moves the data fields into locals, allocates `LAYOUT.head_size` plus their dynamic payload sizes, and encodes them with the shared field encoder at `LAYOUT.offsets[i]`. This is the same head layout and tail placement as the tuple encoding, so the log data of events that already compiled is unchanged. The generated `emit` is `#[arithmetic(unchecked)]`, like the tuple `payload_size` and `encode` it replaces; its only arithmetic sums in-memory payload sizes. No new type or trait is introduced for the data: the `AbiRecord` impl on the event struct is the only new generated item. A separate data-record struct would have reused one more builder but added a user-visible type. Supporting changes: - `BodyBuilder` gains `bind_self_fields`, `encode_bound_fields` and `record_payload_size_expr`; `msg` payload sizes use the latter too. - A missing `AbiSize` on a data field is reported once, from the generated `emit`, as it already is for `#[error]`; the event's `LAYOUT` does not add a duplicate. - `emit` only creates a `self` receiver when indexed fields read it. Tests: - `event_many_data_fields.fe` emits a 17-field event mixing static and dynamic fields; `event_dynamic.rs` checks its topics and log data against `cast abi-encode` output. It failed to compile before this change. - The negative `event_too_many_data_fields` fixture is removed. Snapshots: - Event desugar snapshots show the new `emit` body and `AbiRecord` impl; `event_only_indexed` only gains the arithmetic attribute. - `event_unsupported_field_type` now names the field type itself, with the same three errors as `error_unsupported_field_type`. - `contract_recv_msg_event_name_collision` lists the types of the new `emit` body. - `erc20`, `event_logging` and `event_logging_via_log` IR no longer instantiate the tuple encoder for event data.
The AbiRecord docs said the trait gives the head layout of the implementing record, but an #[event] struct implements it over its data fields only, so Event: AbiRecord<N> says nothing about indexed fields. State that, and say that decode_from reads LAYOUT: tuple decode_payload walks a cursor and does not read it. The comment keeps its five lines so the abi.fe lines cited by snapshots do not move.
Generated LAYOUT, HEAD_SIZE and IS_DYNAMIC bodies were checked and then filtered by present_generated_abi_const_diag, which recognized the constants by name, matched a failing goal's self type against field types by equality, and collected reported goals per module. A goal on a nested type matched no field. A field whose own HEAD_SIZE failed to evaluate was reported again by LAYOUT and HEAD_SIZE with no location, because the generated bodies were built with no origin, and a generic field type that fails only for the record's argument had no located report at all. Each generated metadata constant now declares its owner when it is built: AssocConstBodyCheckPolicy::AbiRecordFields(kind), with AbiRecordKind naming the record (msg variant, #[error] struct, #[event] data fields). Record field analysis, generalized from msg field analysis into abi_record_fields, decides per field whether the field type has what the record's generated impls use. The constant's body is checked only when every field passes; an evaluation failure is then reported as for any constant, once, where it occurs. Generated constant bodies carry their declaration's origin, and the name matching is gone. Failing first (fe check on the PR head's compiler): - abi_record_field_head_size_fails.fe: 3 located division-by-zero errors plus 5 unlocated "failed to evaluate constant LAYOUT/HEAD_SIZE". Now the 3 located errors only. - abi_record_generic_field_head_size_fails.fe: 5 unlocated errors and no located one. Now 3 located division-by-zero errors. - abi_record_field_head_size_cycle.fe: the recursive-definition error plus 2 unlocated ones. Now the recursive-definition error only.
An #[error] or #[event] field type without ABI support was reported by the generated function bodies: payload_size through dynamic_payload_size and encode or emit through encode_field_at. Both require AbiSize, so the missing AbiSize was reported twice, and Encode<Sol> once more, each at the whole struct with a note pointing into core's abi.fe. Record field analysis now owns these fields as it owns msg variant fields: it reports one diagnostic per field, at the field's declared type, naming the missing traits, with the codes abi field analysis already uses for error and event fields. The generated function bodies of a record whose fields fail are not checked, since their failures would repeat it. A tuple field of an #[error] or #[event] struct stays with the tuple check that already rejects it. Failing first (fe check on the PR head's compiler): - error_unsupported_field_type.fe and event_unsupported_field_type.fe: three errors each at the whole struct. Now one each, at the field type. - abi_record_nested_unsupported_field.fe (Option<StorageMap<..>> in a msg, #[error] and #[event] field): one msg error plus six struct-wide errors. Now one error per field. - abi_record_field_positions.fe (alias, array, tuple, nested module, beside supported fields, an indexed field): 19 errors, 13 of them struct-wide trait-bound errors. Now 11, one per problem. Snapshots: error_unsupported_field_type and event_unsupported_field_type change from three errors to one located error each.
#[event] and #[error] lowering each spelled out the same struct item: the attribute stripped, generic parameters reported, the struct built, and the self type and name checked before any impl is generated (23 identical lines), and the same rejection of a field type that is not a path. They also accumulated their diagnostics in two structs with the same fields, EventError and ErrorDiagnostic, whose struct and field names were written but never read. Both now lower their struct with lower_abi_record_struct and check field types with check_abi_record_field_ty, in abi_field.rs next to the field diagnostic they share, and accumulate one AbiRecordDiagnostic whose kind names the record it is about. The unread names are dropped, from AbiFieldDiagnostic too. The event and error passes read the same accumulator, filtered by record kind, and the three places that located a record field's type now share abi_record_field_ty_range. No diagnostic changes: every snapshot is unchanged.
a29a4a5 to
eb4bfce
Compare
ABI head layout now has one definition, in Fe.
core::abi::abi_record_layoutis a checked prefix sum over field head sizes, and every ABI record reads its offsets and head size from it: the built-in tuple codecs, generatedmsgvariants and#[error]structs, and#[event]data.On master the tuple impls spell the prefix sum out by hand at every arity, and the generated codecs repeat it in Rust lowering. Here tuples of 1 to 16 elements,
msgvariants,#[error]structs and#[event]data implementAbiRecord<N>, andHEAD_SIZE, the encoders, the tupleAbiSpanimpls anddecode_from(with its prechecked variant) readLAYOUT. The tupledecode_payloadwalks a cursor over the fields and does not read it.What this fixes:
#[event]with more than 16 data fields.emitpacked the data fields into a tuple, and the tuple codecs stop at 16 elements, so on master the 17-field event in the new test fails with "(u256, Bytes, bool, u256, u8, u256, u256, u256, Address, DynArray<u256>, u256, u256, u256, u256, u256, u256, u256)doesn't implementEncode<Sol>" (and the same forAbiSize).emitnow encodes the data fields as one record with the same head layout and tail placement, so events that already compiled log the same bytes. A new test checks a 17-field event that mixes static and dynamic fields against acast abi-encodevector.#[event]and#[error]constants are checked with their type errors dropped. Evaluation failures are still reported, but without a location: an#[error]field type whoseHEAD_SIZEfails gives8-0089 failed to evaluate constant HEAD_SIZEwith no span, and the same field in an#[event]or amsgvariant gives nothing atfe check. Also on master, the generatedmsgHEAD_SIZEandIS_DYNAMICare not checked (selector analysis checks themsgSELECTOR). Now each generatedLAYOUT,HEAD_SIZEandIS_DYNAMICdeclares who reports its failures (AssocConstBodyCheckPolicy::AbiRecordFields). Record field analysis, whichmsgvariants already had, now covers#[error]fields and#[event]data fields too: it reports each field type that lacksAbiSizeorEncode<Sol>(and, formsg,Decode<Sol>) once, at the field. The generated constants are checked only when every field passes, and then report an evaluation failure where it occurs, with a location. So a field type whose ownHEAD_SIZEfails is reported once, at that constant, anderror_unsupported_field_typeandevent_unsupported_field_typego from three errors (#[error]) and two (#[event]) at the whole struct to one at the field. TheTOPIC0of an event with fields and the#[error]SELECTORkeep their suppression (ExpansionSourceCompatibility); see Limits.#[error]no longer keeps its own copies of the#[event]signature builder and themsgAbiSizeandEncodebuilders; its separate selector path is deleted. Mainline #1592 fixed long#[error]and#[event]signatures in both copies of the signature builder: it splits names longer than 31 bytes and nests#[error]signatures longer than 16 elements. Here both live in the one shared builder, and #1592'slong_error_event_names.fepasses unchanged.Records with many fields also compile faster. A test with a 64-field
msgvariant and a 64-field#[error]takes 31.9 to 32.2 s of CPU time on this branch and 49.8 to 50.2 s on master342048ea5(fe testCPU time, three interleaved runs each, on a shared machine).Measured on master
342048ea5against this branch rebased onto it: at O1 and O2 the deploy and runtime bytecode have the same SHA-256 onerc20.fe,erc20_minimal.fe,tuple_return_contract.fe,mode_receivers_contract.fe,event_logging.fe,event_many_fields.feandevent_dyn_array_u256.fe. At O0 the fixtures that emit events shrink, because event data no longer instantiates the tuple encoder, and the others are unchanged:erc20.feruntime goes from 11793 to 11536 bytes,event_dyn_array_u256.fefrom 4966 to 3575. The gas report of the differential deposit test, whose event has fiveBytesdata fields, is unchanged:differential_deposit/gas_report.snapis not modified here, and the test passes against it.Design notes:
#[event]data implementsAbiRecord<N>on the event struct itself, over its data fields, so no new type appears.AbiRecordtakes no ABI parameter, becauseAbiSizetakes none.emitis#[arithmetic(unchecked)], like the tupleencodeit replaces; its only arithmetic sums in-memory payload sizes.Limits:
[T; N]keeps itsN * abi_field_head_size<T>()stride, since it has no per-field sums to share.is_dynamic_event_ty); readingIS_DYNAMICthere needs trait constant evaluation in that analysis.TOPIC0of an#[event]with fields and the#[error]SELECTORare still checked with their errors dropped (ExpansionSourceCompatibility, as on master). This hides a missingSolCompaton a field type until MIR lowering; moving it to record field analysis is a follow-up issue, because it also has to take over the rules for indexed event fields. Tracked in Report all#[error]and#[event]field problems in record field analysis #1624.Buf<0>wheretype Buf<const N: usize> = [u8; { 10 / N }], is reported at the constant, but in an#[error]or#[event]it still also gives atype annotation is needederror at the whole struct (in amsgvariant it does not). Record field analysis checks invalid constants only formsgtoday; the follow-up issue covers this too. Tracked in Report all#[error]and#[event]field problems in record field analysis #1624.#[event]fails, the generatedemitis skipped, andemitis where a missingTopicValueon an indexed field is reported. That error then appears only after the data field is fixed. Master reported both at once. The follow-up issue moves the indexed-field rules into record field analysis. Tracked in Report all#[error]and#[event]field problems in record field analysis #1624.The branch is rebased onto master
1329ce2a0. Mainline also rewrote the sevensonatina_irsnapshots this PR changes (#1618 touched all of them) and moved theeffects.felines cited byevent_unsupported_field_type.snapandevent_too_many_data_fields.snap(#1601), so the snapshots were regenerated on the combined code (the second is still deleted with its fixture), and each diff against master is this PR's change alone.Overlap with open PRs: #1540 inserts lines in
abi.feabove the lines cited byabi_record_layout_overflow.snap, so whichever of #1540 and this PR lands second regenerates it. #1606's#[abi]structs callbuild_decode_head_pos_exprandcreate_head_size_assoc_const, which this PR changes to readAbiRecord, so whichever of #1606 and this PR lands second adapts#[abi]structs toAbiRecordand gives their generated constants anAbiRecordFieldsowner; besides its conflicts with master, #1606 conflicts with this PR inmsg.rs,error.rs,ty_check/mod.rsanddiagnostics.rs. #1582 and #1600 have merged, and this PR is rebased on them. #1576 and this PR merge cleanly. #1600 carried the layout recurrence fix and the packed-record test that were first part of this PR. #1427 touches the same selector andmsg/#[error]lowering, but it predates current mainline.Testing: the full CI command (
cargo nextest run --release --workspace --exclude fe-language-server --exclude fe-bench --all-features --locked --no-fail-fast) passes at this head, 3736 of 3736. The CPU times and bytecode measurements above were each repeated three times on this branch and on342048ea5, before this rebase.