fevm_tools conformance (native/tools) runs
the Osaka cases of an
execution-specs test fixture
release through FeVM's transact. Each case starts a fresh native
fevm_conformance process, which reads one binary-encoded case from stdin and
prints the transaction outcome, the logs hash and the complete post-state. Run
it from the repository's root:
gh release download tests@v21.0.1 -R ethereum/execution-specs -p fixtures.tar.gz
tar -xzf fixtures.tar.gz fixtures/state_tests
native/out/toolchain/bin/fe build . --ingot fevm_tools --backend native -O 2 \
--emit executable --out-dir native/out/tools
native/out/tools/fevm_tools conformance --fe native/out/toolchain/bin/fe \
--fixtures fixtures/state_tests --out native/out/conformance-run--fe builds the driver; --executable reuses an already built one instead.
--select limits the run to fixture paths containing any of the substrings
that follow it, --level picks the driver's optimization level (default O2),
--jobs sets the number of parallel cases (default: two fewer than the host's
processors) and --timeout the per-case limit in seconds (default 120). Each
output directory must be new. The runner reads one fixture file at a time, and
results.json's seconds covers reading the fixtures as well as running the
cases.
Given RUNS [TRANSACT] arguments, the driver repeats the case RUNS times
before the reported run, skipping the transaction when TRANSACT is 0; the
benchmark runner times the difference.
A case passes when the transaction's validity matches expectException, the
receipt's logs hash matches, and every account's balance, nonce, code and
non-zero storage equals the fixture's post-state. Outcomes are:
passedsignature: the transaction's signature is malformed. FeVM takes a recovered sender, so rejecting the signature is the host's part of decoding it; the case is not run.failed: a validity, logs or state mismatch, listed incases.jsonlhost_error: FeVM stopped with a host status (allocation failure, balance overflow) instead of an EVM outcomecrashandtimeout
The runner encodes the environment, pre-state and the transaction selected by
the case's data/gas/value indexes; it derives the blob base fee from
currentExcessBlobGas with the fixture's Osaka blob schedule (BPO2's update
fraction when a fixture has none) and the transaction type from its fields. As in other
state-test clients, BLOCKHASH(n) for the previous 256 blocks is the Keccak-256
hash of n's decimal digits. The driver checks the post-state directly instead
of recomputing the state root.
All 14,693 runnable cases of tests@v21.0.1 pass; 23 are signature.
The driver prints one JSON object: invalid (null, or the validity code's
name), status (the status name), host_failure, gas_used, logs_hash
and the post-state accounts. Names come from ingots/evm/src/status.fe.