bench: revmc aot default by carneiro-cw · Pull Request #2667 · cloudwalk/stratus

carneiro-cw · 2026-09-02T06:39:20Z

PR Type

Enhancement, Tests

Description

Integrate revmc JIT/AOT support by default
Add executor config flags for JIT/AOT
Wrap EVM with JitHandle for compiled execution
Bump dependencies, enable revmc feature by default
Update RPC tests with new gas/refund fields and fixtures

Diagram Walkthrough

flowchart LR
  A["ExecutorConfig"] -- "create JitHandle" --> B["JitHandle"]
  B -- "wrap EVM" --> C["ExecutorRevm<DB>"]
  C -- "used by" --> D["Evm"]
  D -- "spawned in" --> E["EvmWorkerPool threads"]

File Walkthrough

Relevant files

Configuration changes

2 files

build.rs Emit `__revmc_builtin_*` symbols when revmc feature enabled	+6/-0
config.rs `Add JIT/AOT executor command-line flags`	+20/-0

Enhancement

4 files

jit.rs `Implement revmc JIT/AOT integration and artifact store`	+310/-0
mod.rs Expose `JitHandle` and `ExecutorRevm` in mod	+7/-3
mod.rs Define `ExecutorRevm` alias for JIT-wrapped EVM	+9/-0
evm_worker_pool.rs Distribute `JitHandle` to EVM worker threads	+14/-7

Tests

4 files

server.rs Add gas/refund fields to `CallFrame` in tests	+20/-0
ExternalBlock.json Update numeric fields and add `blockTimestamp`	+20/-19
ExternalBlockWithReceipts.json Refresh fixture values and add `blockTimestamp`	+37/-36
ExternalTransaction.json Adjust transaction fields and include `blockTimestamp`	+13/-12

Dependencies

2 files

Cargo.toml Bump dependencies and enable `revmc` feature	+16/-9
Cargo.toml Update `syn` and `proc-macro2` versions	+2/-2

github-actions · 2026-09-02T06:40:48Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review Logging field name `tracing::warn!` calls log the error using `?err` but do not name it `reason`. For consistency with structured tracing guidelines and to ensure the original error is captured correctly, the error should be logged in a field called `reason`. tracing::warn!( ?err, path = ?config.executor_jit_store_path, "failed to create revmc AOT artifact store; falling back to in-memory JIT" ); runtime.aot = false; } } } match revmc::runtime::JitBackend::new(runtime) { Ok(backend) => Self { backend }, Err(err) => { tracing::warn!(?err, "failed to create revmc JIT backend; falling back to the interpreter"); Self { backend: revmc::runtime::JitBackend::disabled(), } } Loop logging parent The `tracing::error!` inside the worker method’s infinite loop logs panics without specifying `parent: None`. This can cause unbounded span children growth. Add `parent: None` to the log to avoid memory leaks in long-running loops. let _guard = kind.mark_executor_pool_busy(); if let Err(StratusError::Executor(ExecutorError::Panic { err: panic_err })) = task.execute(&mut evm) { tracing::error!(?panic_err, "executor panicked; recreating EVM"); evm = Evm::new(Arc::clone(&storage), &config, kind, jit.clone()); }

github-actions · 2026-09-02T06:41:45Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
Possible issue	Add missing gas fields The tests show new optional fields on `CallFrame` but the code that defines `CallFrame` isn't updated to include them. Add these four `Option` fields to the `CallFrame` struct so that instantiations in tests compile and deserialize correctly. src/eth/rpc/server.rs [1722-1725] -execution_gas_used: None, -state_gas_used: None, -gas_refund: None, -state_gas_refund: None, +#[derive(Serialize, Deserialize, Debug, Clone)] +pub struct CallFrame { + // ... other fields ... + pub logs: Vec<LogEntry>, + pub execution_gas_used: Option<U256>, + pub state_gas_used: Option<U256>, + pub gas_refund: Option<U256>, + pub state_gas_refund: Option<U256>, +} Suggestion importance[1-10]: 8 __ Why: The tests instantiate four new `CallFrame` fields (`execution_gas_used`, `state_gas_used`, `gas_refund`, `state_gas_refund`) which are not defined in the struct, causing compilation and deserialization failures.	Medium
General	Log parse_hash failures Failures in hex decoding or length mismatches are currently silent. Add logging for each error branch to aid in diagnosing corrupted manifests or invalid content hashes. src/eth/executor/evm/jit.rs [296-298] fn parse_hash(hex: &str) -> Option<[u8; 32]> { - const_hex::decode(hex).ok().and_then(\|bytes\| bytes.try_into().ok()) + match const_hex::decode(hex) { + Ok(bytes) => match bytes.try_into() { + Ok(arr) => Some(arr), + Err(_) => { + tracing::warn!("invalid hash length for hex '{}'", hex); + None + } + }, + Err(err) => { + tracing::warn!(?err, "failed to decode hex hash '{}'", hex); + None + } + } } Suggestion importance[1-10]: 5 __ Why: Emitting `tracing::warn!` on decode or length errors adds useful diagnostics for corrupted manifests and invalid hashes without changing core behavior.	Low

The revmc feature pulls in llvm-sys, which requires llvm-config and static LLVM libraries at build time. Extract the install into a composite action and use it in every workflow that invokes cargo.

Exempt the revm 43 family, alloy 2.4.1, and the revmc toolchain deps (inkwell, llvm-sys, libloading, wincode, pastey, oxc_index) instead of auditing them: these are benchmark-only branches. Real audits should precede any production merge. The revmc git dependencies are marked audit-as-crates-io = false since they are pinned source snapshots with no matching crates.io release.

stratus-benchmark · 2026-09-02T14:50:17Z

Benchmark:
Run ID: bench-c97d7663

Git Info:

Branch: revmc-aot-default
Commit: 2d878f6
PR: bench: revmc aot default #2667

Leader Stats:
RPS Stats: Max: 7010.00, Min: 2663.00, Avg: 3217.14, StdDev: 271.42
TPS Stats: Max: 3486.00, Min: 3.00, Avg: 3166.46, StdDev: 325.54

Follower Stats:
Imported Blocks/s: Max: 143.00, Min: 66.00, Avg: 101.33, StdDev: 31.75
Imported Transactions/s: Max: 461660.00, Min: 214223.00, Avg: 320868.00, StdDev: 103861.38

Plots:

Sample revmc's pull-based counters (compiled vs interpreted executions, compilation successes/failures, cache residency, dropped hot events, JIT memory) every 5s from a dedicated reporter thread and expose them as stratus_executor_jit_* gauges labeled by mode (jit|aot). This makes it possible to tell from /metrics whether benchmarks actually execute compiled code or silently fall back to the interpreter.

stratus-benchmark · 2026-09-02T17:05:27Z

Benchmark:
Run ID: bench-a52956c9

Git Info:

Branch: revmc-aot-default
Commit: 16e749e
PR: bench: revmc aot default #2667

Leader Stats:
RPS Stats: Max: 5814.00, Min: 2058.00, Avg: 3009.54, StdDev: 274.59
TPS Stats: Max: 3396.00, Min: 33.00, Avg: 2963.71, StdDev: 357.56

Follower Stats:
Imported Blocks/s: Max: 140.00, Min: 66.00, Avg: 101.00, StdDev: 30.34
Imported Transactions/s: Max: 423136.00, Min: 179876.00, Avg: 299335.00, StdDev: 99357.93

Plots:

revmc pins each code hash's first dispatch decision in a per-EVM lookup cache, so a contract first executed before its background compilation finished stays interpreted for that EVM's lifetime - with a single tx-pool EVM that meant benchmarks never ran compiled code. - periodically reinstall the JIT backend (every 250ms per worker) to reset the dispatch cache and adopt newly compiled programs - add --executor-jit-blocking to compile synchronously on the first execution of each contract instead, guaranteeing compiled execution from the second execution on - warn at startup when AOT is enabled but cc/ld.lld are missing, since every AOT compilation would fail and execution stays interpreted - reword executor_jit lookup metrics: they count dispatches (once per code hash per EVM), not executions

stratus-benchmark · 2026-09-02T18:40:38Z

Benchmark:
Run ID: bench-c79db679

Git Info:

Branch: revmc-aot-default
Commit: 16e749e
PR: bench: revmc aot default #2667

Leader Stats:
RPS Stats: Max: 8790.00, Min: 2569.00, Avg: 3175.47, StdDev: 371.23
TPS Stats: Max: 3425.00, Min: 544.00, Avg: 3131.40, StdDev: 213.80

Follower Stats:
Imported Blocks/s: Max: 155.00, Min: 1.00, Avg: 101.00, StdDev: 70.79
Imported Transactions/s: Max: 487675.00, Min: 3195.00, Avg: 316271.33, StdDev: 221710.89

Plots:

stratus-benchmark · 2026-09-02T20:18:36Z

Benchmark:
Run ID: bench-66cbbb08

Git Info:

Branch: revmc-aot-default
Commit: 6c50274
PR: bench: revmc aot default #2667

Leader Stats:
RPS Stats: Max: 9036.00, Min: 1789.00, Avg: 2078.95, StdDev: 432.07
TPS Stats: Max: 2182.00, Min: 838.00, Avg: 1999.49, StdDev: 108.32

Follower Stats:
Imported Blocks/s: Max: 186.00, Min: 119.00, Avg: 152.50, StdDev: 33.50
Imported Transactions/s: Max: 373674.00, Min: 236169.00, Avg: 304921.50, StdDev: 68752.50

Plots:

The conversion hardcoded code_hash as KECCAK_EMPTY while attaching the real bytecode. revm <= 42 never consumed code_hash on the execution path (the interpreter ran code bytes directly), so the bug was latent. revm 43 propagates the account's code_hash into the call frame's precomputed bytecode hash (known_bytecode), which consumers now dispatch on: - revmc JIT/AOT keyed every contract's compiled code under the empty hash, so the first contract compiled was executed for every contract (reverts, wrong results, hangs); compile logs showed only KECCAK_EMPTY (0xc5d246...) - EXTCODEHASH returned the empty hash instead of the real code hash for accounts loaded from storage Compute the real keccak via Bytecode::hash_slow (OnceLock-cached) so dispatch hashes match the executed bytecode.

stratus-benchmark · 2026-09-02T22:00:23Z

Benchmark:
Run ID: bench-afa24893

Git Info:

Branch: revmc-aot-default
Commit: 2ea301e
PR: bench: revmc aot default #2667

Leader Stats:
RPS Stats: Max: 9642.00, Min: 2040.00, Avg: 2850.63, StdDev: 447.24
TPS Stats: Max: 3220.00, Min: 496.00, Avg: 2804.20, StdDev: 249.10

Follower Stats:
Imported Blocks/s: Max: 148.00, Min: 17.00, Avg: 101.33, StdDev: 59.75
Imported Transactions/s: Max: 411793.00, Min: 46161.00, Avg: 284159.33, StdDev: 168437.84

Plots:

stratus-benchmark · 2026-09-02T22:13:13Z

Benchmark:
Run ID: bench-2865fe51

Git Info:

Branch: revmc-aot-default
Commit: 2ea301e
PR: bench: revmc aot default #2667

Leader Stats:
RPS Stats: Max: 5942.00, Min: 1492.00, Avg: 3008.54, StdDev: 359.67
TPS Stats: Max: 3385.00, Min: 154.00, Avg: 2963.06, StdDev: 410.13

Follower Stats:
Imported Blocks/s: Max: 117.00, Min: 89.00, Avg: 101.33, StdDev: 11.67
Imported Transactions/s: Max: 356137.00, Min: 243176.00, Avg: 300256.67, StdDev: 46123.95

Plots:

stratus-benchmark · 2026-09-02T22:46:59Z

Benchmark:
Run ID: bench-5156fd4b

Git Info:

Branch: revmc-aot-default
Commit: 2ea301e
PR: bench: revmc aot default #2667

Leader Stats:
RPS Stats: Max: 9986.00, Min: 2102.00, Avg: 2995.03, StdDev: 451.22
TPS Stats: Max: 3329.00, Min: 565.00, Avg: 2950.30, StdDev: 236.44

Follower Stats:
Imported Blocks/s: Max: 162.00, Min: 44.00, Avg: 101.33, StdDev: 48.23
Imported Transactions/s: Max: 471999.00, Min: 126711.00, Avg: 298963.33, StdDev: 140964.32

Plots:

carneiro-cw added 2 commits September 2, 2026 03:36

revmc

c4bdcdf

enha: enable revmc AOT by default

ad3c1b4

carneiro-cw added 4 commits September 2, 2026 09:52

ci: install LLVM 22 on build runner for revmc builds

e6e6a01

ci: install libpolly-22-dev for llvm-sys static linking

eedea1a

ci: install LLVM 22 in all workflows that build Rust

05bcdd5

The revmc feature pulls in llvm-sys, which requires llvm-config and static LLVM libraries at build time. Extract the install into a composite action and use it in every workflow that invokes cargo.

Navigation Menu

Uh oh!

bench: revmc aot default - #2667

bench: revmc aot default#2667
carneiro-cw wants to merge 9 commits into
mainfrom
revmc-aot-default

carneiro-cw commented Sep 2, 2026 •

edited by github-actions Bot

Loading

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

carneiro-cw commented Sep 2, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Type

Description

Diagram Walkthrough

File Walkthrough

Uh oh!

github-actions Bot commented Sep 2, 2026

PR Reviewer Guide 🔍

Uh oh!

github-actions Bot commented Sep 2, 2026

PR Code Suggestions ✨

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

carneiro-cw commented Sep 2, 2026 •

edited by github-actions Bot

Loading