bench: revmc jit default by carneiro-cw · Pull Request #2668 · cloudwalk/stratus

carneiro-cw · 2026-09-02T06:39:56Z

PR Type

Enhancement, Dependencies, Tests

Description

Default-enable revmc feature and upgrade dependencies
Add executor JIT/AOT CLI options and configuration
Integrate JitHandle into EVM executor and worker pool
Implement filesystem artifact store for AOT; update tests and fixtures

Diagram Walkthrough

flowchart LR
  Config["ExecutorConfig"]
  Config -- "create JIT handle" --> JitHandle["JitHandle.create()"]
  JitHandle -- "wrap EVM instance" --> ExecutorRevm["ExecutorRevm"]
  ExecutorRevm -- "spawn in worker pool" --> Workers["EVM Worker Pool"]

File Walkthrough

Relevant files

Configuration changes

2 files

build.rs `Emit revmc builtin symbols when enabled`	+6/-0
config.rs `Add JIT/AOT executor configuration options`	+20/-0

Enhancement

4 files

jit.rs `Implement revmc JIT/AOT integration backend`	+310/-0
mod.rs `Integrate JitHandle in EVM instantiation`	+7/-3
mod.rs `Define ExecutorRevm type alias for JIT wrapper`	+9/-0
evm_worker_pool.rs `Pass JitHandle to EVM worker pool threads`	+14/-7

Tests

4 files

server.rs `Add new gas fields in CallFrame tests`	+20/-0
ExternalBlock.json `Adjust ExternalBlock fixture data fields`	+20/-19
ExternalBlockWithReceipts.json `Update ExternalBlockWithReceipts fixture fields`	+37/-36
ExternalTransaction.json `Revise ExternalTransaction fixture parameters`	+13/-12

Dependencies

2 files

Cargo.toml `Upgrade dependencies and add revmc feature`	+16/-9
Cargo.toml `Update macro crate dependencies versions`	+2/-2

github-actions · 2026-09-02T06:41:33Z

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 4 🔵🔵🔵🔵⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ No major issues detected

github-actions · 2026-09-02T06:42:40Z

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Use atomic file writes Make writes atomic to avoid corrupt artifacts if the process is interrupted. Write both the `.so` and `.json` to temporary files, then rename them into place. src/eth/executor/evm/jit.rs [275-276] -std::fs::write(self.artifact_path(key), dylib_bytes)?; -std::fs::write(self.manifest_path(key), serde_json::to_vec(&dto)?)?; +let art_path = self.artifact_path(key); +let tmp_art = art_path.with_extension("so.tmp"); +std::fs::write(&tmp_art, dylib_bytes)?; +std::fs::rename(&tmp_art, &art_path)?; +let manifest_path = self.manifest_path(key); +let tmp_manifest = manifest_path.with_extension("json.tmp"); +std::fs::write(&tmp_manifest, serde_json::to_vec(&dto)?)?; +std::fs::rename(&tmp_manifest, &manifest_path)?; Suggestion importance[1-10]: 7 __ Why: Writing to temporary files and renaming ensures atomicity and avoids corrupt artifacts if the process is interrupted.	Medium
General	Require JIT when enabling AOT Enforce that AOT mode can only be enabled when JIT mode is also enabled. Add a `requires = "executor-jit"` to the CLI argument so that users cannot specify AOT without JIT. src/eth/executor/config.rs [40-41] #[cfg(feature = "revmc")] -#[arg(long = "executor-jit-aot", env = "EXECUTOR_JIT_AOT", default_value = "false")] +#[arg( + long = "executor-jit-aot", + env = "EXECUTOR_JIT_AOT", + default_value = "false", + requires = "executor-jit" +)] pub executor_jit_aot: bool, Suggestion importance[1-10]: 5 __ Why: Adding `requires = "executor-jit"` to the clap `arg` attribute prevents invalid CLI combinations and improves UX with minimal impact.	Low

The revmc feature pulls in llvm-sys, which requires llvm-config and static LLVM libraries at build time. Extract the install into a composite action and use it in every workflow that invokes cargo.

Exempt the revm 43 family, alloy 2.4.1, and the revmc toolchain deps (inkwell, llvm-sys, libloading, wincode, pastey, oxc_index) instead of auditing them: these are benchmark-only branches. Real audits should precede any production merge. The revmc git dependencies are marked audit-as-crates-io = false since they are pinned source snapshots with no matching crates.io release.

stratus-benchmark · 2026-09-02T14:38:34Z

Benchmark:
Run ID: bench-b7dfd8ec

Git Info:

Branch: revmc-jit-default
Commit: ce6382f
PR: bench: revmc jit default #2668

Leader Stats:
RPS Stats: Max: 5313.00, Min: 1679.00, Avg: 3164.31, StdDev: 272.43
TPS Stats: Max: 3501.00, Min: 83.00, Avg: 3119.44, StdDev: 373.63

Follower Stats:
Imported Blocks/s: Max: 142.00, Min: 24.00, Avg: 101.00, StdDev: 54.49
Imported Transactions/s: Max: 451514.00, Min: 55639.00, Avg: 315063.33, StdDev: 183523.20

Plots:

Sample revmc's pull-based counters (compiled vs interpreted executions, compilation successes/failures, cache residency, dropped hot events, JIT memory) every 5s from a dedicated reporter thread and expose them as stratus_executor_jit_* gauges labeled by mode (jit|aot). This makes it possible to tell from /metrics whether benchmarks actually execute compiled code or silently fall back to the interpreter.

stratus-benchmark · 2026-09-02T15:01:39Z

Benchmark:
Run ID: bench-1730df5b

Git Info:

Branch: revmc-jit-default
Commit: ce6382f
PR: bench: revmc jit default #2668

Leader Stats:
RPS Stats: Max: 7751.00, Min: 2608.00, Avg: 3239.16, StdDev: 300.97
TPS Stats: Max: 3490.00, Min: 75.00, Avg: 3194.62, StdDev: 304.50

Follower Stats:
Imported Blocks/s: Max: 151.00, Min: 57.00, Avg: 101.00, StdDev: 38.61
Imported Transactions/s: Max: 485818.00, Min: 185189.00, Avg: 322656.33, StdDev: 124068.79

Plots:

stratus-benchmark · 2026-09-02T16:53:50Z

Benchmark:
Run ID: bench-81b4e191

Git Info:

Branch: revmc-jit-default
Commit: dd88d98
PR: bench: revmc jit default #2668

Leader Stats:
RPS Stats: Max: 9821.00, Min: 2272.00, Avg: 3041.23, StdDev: 430.22
TPS Stats: Max: 3467.00, Min: 367.00, Avg: 2996.29, StdDev: 260.58

Follower Stats:
Imported Blocks/s: Max: 148.00, Min: 57.00, Avg: 101.00, StdDev: 37.21
Imported Transactions/s: Max: 445964.00, Min: 168499.00, Avg: 302625.33, StdDev: 113461.76

Plots:

revmc pins each code hash's first dispatch decision in a per-EVM lookup cache, so a contract first executed before its background compilation finished stays interpreted for that EVM's lifetime - with a single tx-pool EVM that meant benchmarks never ran compiled code. - periodically reinstall the JIT backend (every 250ms per worker) to reset the dispatch cache and adopt newly compiled programs - add --executor-jit-blocking to compile synchronously on the first execution of each contract instead, guaranteeing compiled execution from the second execution on - warn at startup when AOT is enabled but cc/ld.lld are missing, since every AOT compilation would fail and execution stays interpreted - reword executor_jit lookup metrics: they count dispatches (once per code hash per EVM), not executions

The conversion hardcoded code_hash as KECCAK_EMPTY while attaching the real bytecode. revm <= 42 never consumed code_hash on the execution path (the interpreter ran code bytes directly), so the bug was latent. revm 43 propagates the account's code_hash into the call frame's precomputed bytecode hash (known_bytecode), which consumers now dispatch on: - revmc JIT/AOT keyed every contract's compiled code under the empty hash, so the first contract compiled was executed for every contract (reverts, wrong results, hangs); compile logs showed only KECCAK_EMPTY (0xc5d246...) - EXTCODEHASH returned the empty hash instead of the real code hash for accounts loaded from storage Compute the real keccak via Bytecode::hash_slow (OnceLock-cached) so dispatch hashes match the executed bytecode.

stratus-benchmark · 2026-09-02T23:10:17Z

Benchmark:
Run ID: bench-8dcf704e

Git Info:

Branch: revmc-jit-default
Commit: bd5b246
PR: bench: revmc jit default #2668

Leader Stats:
RPS Stats: Max: 10201.00, Min: 1.00, Avg: 2692.49, StdDev: 516.09
TPS Stats: Max: 3091.00, Min: 292.00, Avg: 2657.27, StdDev: 267.88

Follower Stats:
Imported Blocks/s: Max: 175.00, Min: 9.00, Avg: 101.33, StdDev: 69.04
Imported Transactions/s: Max: 464413.00, Min: 18509.00, Avg: 269270.33, StdDev: 186239.40

Plots:

carneiro-cw added 2 commits September 2, 2026 03:36

revmc

c4bdcdf

enha: enable revmc JIT by default

04c8e2d

carneiro-cw added 4 commits September 2, 2026 09:52

ci: install LLVM 22 on build runner for revmc builds

ed96b1e

ci: install libpolly-22-dev for llvm-sys static linking

7e23a4b

ci: install LLVM 22 in all workflows that build Rust

c4a78b9

The revmc feature pulls in llvm-sys, which requires llvm-config and static LLVM libraries at build time. Extract the install into a composite action and use it in every workflow that invokes cargo.

carneiro-cw added 2 commits September 2, 2026 15:30

Navigation Menu

Uh oh!

bench: revmc jit default - #2668

bench: revmc jit default#2668
carneiro-cw wants to merge 9 commits into
mainfrom
revmc-jit-default

carneiro-cw commented Sep 2, 2026 •

edited by github-actions Bot

Loading

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

github-actions Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

carneiro-cw commented Sep 2, 2026 • edited by github-actions Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Type

Description

Diagram Walkthrough

File Walkthrough

Uh oh!

github-actions Bot commented Sep 2, 2026

PR Reviewer Guide 🔍

Uh oh!

github-actions Bot commented Sep 2, 2026

PR Code Suggestions ✨

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

stratus-benchmark Bot commented Sep 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

carneiro-cw commented Sep 2, 2026 •

edited by github-actions Bot

Loading