Skip to content

Make Cancun frame and state reference builds reproducible - #1

Merged
sbillig merged 3 commits into
mainfrom
cancun-frame-correctness
Oct 2, 2026
Merged

sbillig merged 3 commits into
mainfrom
cancun-frame-correctness

Conversation

@sbillig

@sbillig sbillig commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Both Cancun frame and state runners now build the pinned reference with the bootstrap record's Rust toolchain and a fresh explicit Cargo target directory. This prevents missing or stale reference executables when Cargo output settings or the host default toolchain differ.

Share reference build and metadata logic, cover target redirection and toolchain forwarding with regressions, and publish Linux acceptance evidence for the rebased revision. Retained older evidence is labeled with its exact historical revisions.

@sbillig
sbillig marked this pull request as ready for review September 22, 2026 21:06
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-02T23:45:11.507187Z 80e5bb3 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a078672e4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread native/differential/run.py Resolved
Comment thread native/differential/run.py Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7f8f55013d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread native/reports/cancun-frames.md Resolved
@sbillig sbillig changed the title Correct arithmetic, PUSH and jump semantics with a Cancun frame corpus Make Cancun frame and state reference builds reproducible Oct 2, 2026
@sbillig
sbillig changed the base branch from native-acceptance to main October 2, 2026 23:38
@sbillig
sbillig force-pushed the cancun-frame-correctness branch from 1f9eaf8 to 80e5bb3 Compare October 2, 2026 23:38
@sbillig
sbillig merged commit 90a5b97 into main Oct 2, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 80e5bb35cc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".


def build_reference(out, toolchain):
target = out / 'reference-target'
reference = target / 'release/fevm-reference'

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Resolve the binary under Cargo's configured target triple

When CARGO_BUILD_TARGET or [build] target is set—even to the installed host triple—the build succeeds but Cargo writes the executable to reference-target/<triple>/release/fevm-reference, so the subsequent digest(reference) fails on this hard-coded host-layout path. cargo build --help documents --target separately as selecting the target triple, while --target-dir only selects the artifact directory; either neutralize the ambient target or obtain the executable path from Cargo's build output.

Useful? React with 👍 / 👎.

with patch.dict(os.environ, environment), patch.object(run, 'HERE', here):
if setting == 'config':
os.environ.pop('CARGO_TARGET_DIR', None)
binary, report = run.build_reference(out, RUST_VERSION)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor --toolchain in the harness regression

When a user selects an alternate installed toolchain with --toolchain but does not have the version from native/toolchain.json installed, both runners fail during their initial unittest discovery because this regression invokes build_reference twice with the module-level RUST_VERSION, before the runner can use the requested toolchain. This makes the documented diagnostic override depend on an unrelated toolchain installation; the regression should receive the selected version or avoid performing a real pinned build during runner discovery.

Useful? React with 👍 / 👎.

Comment on lines +50 to +51
Check out `7f8f550` to reproduce this historical runner and its
`native/toolchain.json`; the current branch uses a later compiler and corpus.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Publish the revision required to reproduce the evidence

The new reproduction instruction names 7f8f55013d9d5354122bd83aec0a42c3f19d9e08, but that object is absent from the reviewed repository: git cat-file/git branch --contains report no such commit, and a repo-wide search finds the hash only in the newly added reports and documentation. Consequently readers cannot check out the claimed clean source tree or verify that the recorded hashes correspond to it; include the commit or a source bundle, or base the record on a reachable revision.

Useful? React with 👍 / 👎.

Comment thread native/accept.py
commands.append(([sys.executable, HERE / 'differential/run.py', '--fe', compiler,
'--out', out / 'cancun-frames'], 'cancun-frames'))
'--out', out / 'cancun-frames',
'--toolchain', build['manifest']['rust_version']], 'cancun-frames'))

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve the bootstrap's selected rustup toolchain

When bootstrap.py is run with a valid alias such as its default stable, or with a custom named toolchain that reports the manifest's required Rust version, the compiler build succeeds but this code later substitutes the version string (for example 1.98.1) as the rustup toolchain name. Rustup does not guarantee that the equivalent version-named toolchain is separately installed, so both differential stages can fail despite receiving a valid completed bootstrap record; record and forward the actual bootstrap selector, or resolve an installed toolchain identity before invoking the runners.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant