Skip to content

feat: preexecute transactions on followers and send access list to leader - #2664

Open
carneiro-cw wants to merge 35 commits into
mainfrom
access_list_rpc_param
Open

feat: preexecute transactions on followers and send access list to leader#2664
carneiro-cw wants to merge 35 commits into
mainfrom
access_list_rpc_param

Conversation

@carneiro-cw

@carneiro-cw carneiro-cw commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

PR Type

Enhancement


Description

  • Replace PendingBlockHeader with BlockInfo for executions

  • Add precomputed access list forwarding and cache warm-up

  • Introduce semaphore for transaction warmup in executor

  • Enhance storage caching with batch missing-keys logic


File Walkthrough

Relevant files
Enhancement
18 files
call_execution.rs
Simplify call execution input with BlockInfo                         
+4/-24   
transaction_execution.rs
Unify transaction input creation with BlockInfo                   
+6/-6     
access_list.rs
Derive AccessListOutput iterator and clone                             
+3/-1     
mod.rs
Add semaphore and support access_list param                           
+30/-29 
consensus.rs
Forward access list when sending to leader                             
+21/-3   
blockchain_client.rs
Extend RPC call to include access_list param                         
+4/-2     
fake_leader.rs
Adapt local transaction call to new signature                       
+1/-1     
supervisor.rs
Implement `forward_access_list` in consensus                         
+12/-2   
rpc_middleware.rs
Parse and trace `access_list` in eth_sendRawTransaction   
+76/-86 
cache.rs
Expose try_get and contains for cache lookups                       
+35/-4   
resolve_pending.rs
Replace transient lock guard with LatestStateReadGuard     
+13/-29 
stratus_storage.rs
Replace transient lock with `latest_state_lock` and load access list
+172/-44
rocks_permanent.rs
Add batched `read_slots` and release ordering                       
+8/-2     
rocks_state.rs
Implement multi‐get `read_slots` and simplify `read_accounts`
+16/-3   
transaction_mined.rs
Use `BlockInfo` when serializing mined transactions           
+8/-1     
mod.rs
Switch pending header to `BlockInfo`                                         
+2/-2     
transaction.rs
Batch retain missing accounts/slots in temp storage           
+34/-5   
utils.rs
Implement `Semaphore` and `Permit` for warmup                       
+57/-0   
Configuration changes
1 files
config.rs
Add `forward_access_list` CLI config option                           
+8/-2     
Formatting
1 files
mod.rs
Rename supervisor module and exports                                         
+6/-6     
Additional files
13 files
context.rs +4/-0     
mod.rs +1/-0     
mod.rs +0/-1     
server.rs +80/-22 
block_info.rs +34/-0   
mod.rs +2/-2     
pending_block.rs +3/-3     
pending_block_header.rs +0/-18   
execution_kind.rs +23/-2   
mod.rs +1/-1     
unix_time_now.rs +7/-0     
call_input.rs +12/-0   
metrics_definitions.rs +5/-8     

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-c44fd1c0

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 6020.00, Min: 2127.00, Avg: 2850.14, StdDev: 339.25
TPS Stats: Max: 3205.00, Min: 10.00, Avg: 2800.82, StdDev: 256.61

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-9b61dd1d

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 7821.00, Min: 1886.00, Avg: 2819.23, StdDev: 515.75
TPS Stats: Max: 3257.00, Min: 186.00, Avg: 2737.03, StdDev: 289.43

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-2f6ba32e

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 17617.00, Min: 2.00, Avg: 2948.41, StdDev: 1247.35
TPS Stats: Max: 3593.00, Min: 529.00, Avg: 2806.19, StdDev: 407.54

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-7a4ba645

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 8846.00, Min: 2055.00, Avg: 3120.56, StdDev: 607.09
TPS Stats: Max: 3644.00, Min: 316.00, Avg: 2999.62, StdDev: 350.62

Plots:

@carneiro-cw
carneiro-cw force-pushed the access_list_rpc_param branch from 7354462 to 9ff339a Compare September 1, 2026 06:00
@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-2a27ee61

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 24052.00, Min: 708.00, Avg: 2995.22, StdDev: 1290.88
TPS Stats: Max: 3311.00, Min: 43.00, Avg: 2805.31, StdDev: 356.04

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-9cf32730

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 8638.00, Min: 2006.00, Avg: 2915.50, StdDev: 510.90
TPS Stats: Max: 3631.00, Min: 355.00, Avg: 2875.38, StdDev: 394.91

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-1b586664

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 8576.00, Min: 2215.00, Avg: 3087.62, StdDev: 365.06
TPS Stats: Max: 3725.00, Min: 6.00, Avg: 3041.35, StdDev: 280.27

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-2c84500f

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 7548.00, Min: 2737.00, Avg: 3157.57, StdDev: 364.30
TPS Stats: Max: 3708.00, Min: 176.00, Avg: 3115.08, StdDev: 207.52

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-bd6213a3

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 10434.00, Min: 1166.00, Avg: 3528.35, StdDev: 700.02
TPS Stats: Max: 3914.00, Min: 145.00, Avg: 3360.10, StdDev: 335.55

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-84d29dd2

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 8948.00, Min: 1.00, Avg: 3427.87, StdDev: 401.85
TPS Stats: Max: 3928.00, Min: 41.00, Avg: 3395.78, StdDev: 271.75

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-af7eb494

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 9955.00, Min: 1.00, Avg: 3172.08, StdDev: 847.97
TPS Stats: Max: 3693.00, Min: 249.00, Avg: 2986.32, StdDev: 501.09

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-290944f9

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 9301.00, Min: 945.00, Avg: 3138.28, StdDev: 468.27
TPS Stats: Max: 3552.00, Min: 77.00, Avg: 3084.08, StdDev: 262.85

Plots:

@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-75977c73

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 7959.00, Min: 1581.00, Avg: 3115.32, StdDev: 351.51
TPS Stats: Max: 3391.00, Min: 16.00, Avg: 3053.58, StdDev: 287.07

Plots:

@carneiro-cw carneiro-cw changed the title Access list rpc param feat: preexecute transactions on followers and send access list to leader Sep 3, 2026
@stratus-benchmark

Copy link
Copy Markdown

Forwarding benchmark:
Run ID: bench-4c5a8571

Git Info:

Follower Stats (transactions sent to follower):
RPS Stats: Max: 8081.00, Min: 2870.00, Avg: 3384.85, StdDev: 316.01
TPS Stats: Max: 3683.00, Min: 2.00, Avg: 3334.24, StdDev: 300.41

Plots:

@carneiro-cw
carneiro-cw marked this pull request as ready for review September 3, 2026 19:25
@carneiro-cw
carneiro-cw requested a review from a team as a code owner September 3, 2026 19:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-09-03T19:31:25.701946Z e351474 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cloudwalk-review-agent cloudwalk-review-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Thanks for the substantial refactor — the direction (pre-exec on follower + forwarding access list, unified BlockInfo, and middleware-owned eth_sendRawTransaction path) is clear and valuable. I found one blocking correctness issue tied to RPC response identity handling in the new async eth_send_raw_transaction path.

Blocking

  1. eth_send_raw_transaction can return a response with Id::Null on spawn_blocking join failure, losing request/response correlation
    In src/eth/rpc/server.rs, the new middleware path captures the real request id (let id = request.id().into_owned();) and uses it for success responses, but in the error branch of .map(...) it builds MethodResponse::error(Id::Null, ...). This breaks JSON-RPC id correlation exactly in the error scenario and can confuse clients/retries in production under runtime pressure. The error response should preserve the original id rather than forcing Null.

If this is fixed, the rest of the sampled changes look consistent with the new architecture.

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 5 🔵🔵🔵🔵🔵
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Unwrap Panic

In load_slots_to_cache and load_accounts_to_cache the code uses unwrap() on read_slots/read_accounts results, causing a panic if a RocksPermanentStorage error occurs. Storage errors should be handled or use expect with context instead of panicking.

fn load_slots_to_cache(&self, slots: Vec<(Address, SlotIndex)>) {
    let existing_slots: HashMap<(Address, SlotIndex), SlotValue> = self.perm.read_slots(slots.clone()).unwrap().into_iter().collect(); //unwrap
    for (address, index) in slots {
        let value = existing_slots.get(&(address, index)).copied().unwrap_or_default();
        Slot::cache_latest_if_missing(self, (address, index), Slot { index, value });
    }
}

fn load_accounts_to_cache(&self, addresses: Vec<Address>) {
    let existing_accounts: HashMap<Address, Account> = self.perm.read_accounts(addresses.clone()).unwrap().into_iter().collect(); //unwrap
Blocking Task Naming

The eth_send_raw_transaction handler uses tokio::task::spawn_blocking directly, without a descriptive task name or span. This makes it difficult to trace and monitor blocking tasks. Use spawn_blocking_named to attach names and spans.

let future = tokio::task::spawn_blocking(move || {
    let rp = _eth_send_raw_transaction_impl(input, data, access_list, ctx, ext).into_response();
    MethodResponse::response(id, rp, usize::MAX)
})
.map(|result| match result {
    Ok(r) => r,
    Err(err) => {
        tracing::error!("Join error for blocking RPC method: {:?}", err);
        MethodResponse::error(
            Id::Null,
            StratusError::Unexpected(crate::eth::types::UnexpectedError::Unexpected(anyhow::anyhow!(err))),
        )
        .with_extensions(ext_clone)
    }
})
.boxed();
Ok((future, tracing_identifiers))
Unstructured Error Log

The join error for the blocking RPC method is logged with a formatted message ("{:?}") instead of a structured reason field, hindering error filtering and querying. Log the error as reason = ?err with a static message.

tracing::error!("Join error for blocking RPC method: {:?}", err);
MethodResponse::error(
    Id::Null,
    StratusError::Unexpected(crate::eth::types::UnexpectedError::Unexpected(anyhow::anyhow!(err))),
)
.with_extensions(ext_clone)

@github-actions

github-actions Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

PR Code Suggestions ✨

Explore these optional code suggestions:

CategorySuggestion                                                                                                                                    Impact
Possible issue
Use Latest for AccessList

Treat AccessList like a latest call rather than pending so the access-list
generation uses the correct block state. Change its mapping from Pending to Latest.

src/eth/types/execution_kind.rs [42-48]

 impl From<ExecutionKind> for BlockFilter {
     fn from(value: ExecutionKind) -> Self {
         match value {
-            ExecutionKind::Transaction | ExecutionKind::RPC(PointInTime::Pending) | ExecutionKind::AccessList => crate::eth::rpc::BlockFilter::Pending,
-            ExecutionKind::CallLatest(block_number) | ExecutionKind::CallPast(block_number) => crate::eth::rpc::BlockFilter::Number(blockNumber),
+            ExecutionKind::Transaction | ExecutionKind::RPC(PointInTime::Pending) => BlockFilter::Pending,
+            ExecutionKind::AccessList => BlockFilter::Latest,
+            ExecutionKind::CallLatest(block_number) | ExecutionKind::CallPast(block_number) => BlockFilter::Number(block_number),
             ExecutionKind::RPC(pit) => pit.into(),
         }
     }
 }
Suggestion importance[1-10]: 8

__

Why: The AccessList variant must map to BlockFilter::Latest to generate access lists against the correct block state, fixing an incorrect Pending mapping.

Medium

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

self.set_mined_block_number(block_number);

P2 Badge Publish the mined number with the latest-state update

After save_block drops the latest-state write guard, the cached header/state already represents the new block but read_mined_block_number() still returns the preceding block until this statement runs. An explicit-number eth_call arriving in that interval can therefore have its old requested number classified as Latest by read_block_info, combining historical state selection with the new block's execution header. Update the mined number before releasing the same publication lock, or base the classification solely on the guarded block info.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/eth/rpc/middleware/rpc_middleware.rs Outdated
Comment thread src/eth/storage/stratus_storage.rs Outdated
Comment thread src/eth/storage/resolve_pending.rs Outdated
Comment thread src/eth/rpc/middleware/rpc_middleware.rs Outdated
gventino-cw
gventino-cw previously approved these changes Sep 3, 2026

@cloudwalk-review-agent cloudwalk-review-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Thanks for the substantial refactor — the overall direction (custom eth_sendRawTransaction middleware path, follower pre-exec access-list forwarding, and BlockInfo unification) is good. I found one blocking correctness issue that can cause rejected clients’ transactions to still execute/forward.

Blocking

  1. eth_sendRawTransaction work starts before client rejection is enforced
    In the middleware call path, eth_send_raw_transaction(request, ...) is invoked before reject_client(...) for this method path is guaranteed, and that function immediately spawns tokio::task::spawn_blocking with the full execution/forwarding logic. Because spawning is eager and replacing the returned future does not cancel the blocking task, a blocked/unknown client can receive a rejection response while the transaction is still processed in the background. This is a behavior/security mismatch for admission control and should be fixed by ensuring rejection happens before spawning any raw-tx work (or by moving rejection inside the spawned closure before side effects).

gventino-cw
gventino-cw previously approved these changes Sep 3, 2026

@cloudwalk-review-agent cloudwalk-review-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Good progress overall on the BlockInfo unification and follower pre-exec path, but I found one blocking correctness issue in the new eth_sendRawTransaction async path.

Blocking

  1. Error responses from the spawned blocking task lose JSON-RPC request identity.

In src/eth/rpc/server.rs, eth_send_raw_transaction correctly captures the original request id (let id = request.id().into_owned();) and uses it for successful responses, but in the spawn_blocking join-error branch it builds MethodResponse::error(Id::Null, ...). This breaks request/response correlation exactly on runtime failure paths and can mis-handle client retries/dispatch; the error response should preserve the original id instead of forcing Id::Null.

gventino-cw
gventino-cw previously approved these changes Sep 3, 2026

@cloudwalk-review-agent cloudwalk-review-agent Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Good progress on the middleware split and follower pre-exec flow; the client-rejection ordering issue appears addressed. I found one blocking correctness issue still present in the new async eth_sendRawTransaction path.

Blocking

  1. eth_send_raw_transaction still returns Id::Null on spawn_blocking join failure, which breaks JSON-RPC request/response correlation on error paths. The request id is captured (let id = request.id().into_owned()), but the join-error branch builds MethodResponse::error(Id::Null, ...) instead of reusing that id; clients can no longer reliably match failures to requests under runtime pressure.

Please preserve the original id in that error response branch.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Send access list to miner as extra param when forwarding transactions

2 participants