perf: cheaper metrics and logs while transaction lock is held

[carneiro-cw]

PR Type

Enhancement

---

Description

• Replace manual timing calls with #[timed] attribute

• Introduce ExecutionMetrics and StorageMetrics with FoundAt

• Refactor StratusStorage to return (value, FoundAt)

• Extract EntityRead trait and state lock into types

---

File Walkthrough

	Relevant files
Enhancement	15 files stratus_storage.rs Refactor reads to return FoundAt and remove manual metrics +69/-324 mod.rs Apply #[timed] and use ExecutionMetrics context                    +61/-160 timed_attribute.rs Implement timed attribute macro for metrics                            +291/-0  session.rs Profile reads with Instant and record StorageMetrics          +43/-28  execution_metrics.rs Define ExecutionMetrics and StorageMetrics structs              +162/-10 metrics_types.rs Add ToMetricLabelValue trait for parameters                            +71/-61  mod.rs Add record and record_async helpers                                            +104/-0  utils.rs Introduce Semaphore and Permit types                                          +50/-63  entity.rs Extract EntityRead trait into its own module                          +90/-0    found_at.rs Add FoundAt enum for read-source tracking                                +24/-0    mod.rs Use ExecutionMetrics instead of old metrics type                  +6/-16    mod.rs Instrument read_pending_execution with #[timed]                    +8/-3      kafka.rs Wrap buffer creation and send with #[timed]                            +4/-20    fake_leader.rs Add #[timed] to import online mined block                                +2/-0      replication.rs Instrument replication importer with #[timed]                        +2/-0
Configuration changes	1 files metrics_definitions.rs Update histogram definitions for evm storage metrics          +15/-63
Refactoring	1 files state_lock.rs Move LatestStateLock into types module                                      +48/-0
Additional files	19 files lib.rs +69/-0    call_execution.rs +5/-0      mod.rs +3/-0      transaction_execution.rs +5/-0      mod.rs +3/-2      evm_worker_pool.rs +3/-3      task.rs +2/-2      execution.rs +2/-0      mod.rs +1/-3      mod.rs +3/-4      server.rs +4/-4      cache.rs +2/-2      mod.rs +2/-1      rocks_permanent.rs +3/-16    rocks_state.rs +0/-43    resolve_pending.rs +2/-2      transaction.rs +3/-7      mod.rs +5/-0      execution_kind.rs +2/-1

---

[github-actions]

PR Reviewer Guide 🔍

(Review updated until commit 6e74658)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 5 🔵🔵🔵🔵🔵

🧪 PR contains tests

🔒 No security concerns identified

⚡ Recommended focus areas for review Missing Metrics The new read_logs method is no longer wrapped with #[timed] or manual recording of elapsed time and success. As a result, storage reads for logs have lost visibility in metrics, reducing observability for performance and error tracking. pub fn read_logs(&self, filter: &LogFilter) -> Result<Vec<LogMessage>, StorageError> { self.perm.read_logs(filter) } Incomplete Instrumentation In create_buffer, per‐event timing and error metrics (inc_kafka_queue_event) were removed and only the overall method is timed. Failures in individual queue_event calls no longer surface in metrics, losing insight into queue latencies and drop rates. #[timed(kafka_create_buffer)] pub fn create_buffer<T, I>(&self, events: I, buffer_size: usize) -> Result<impl Stream<Item = Result<()>>> where T: Event, I: IntoIterator<Item = T>, { let futures: Vec<DeliveryFuture> = events.into_iter().map(|event| self.queue_event(event)).collect::<Result<Vec<_>, _>>()?; // This could fail because the queue is full (?) Ok(futures::stream::iter(futures).buffered(buffer_size).map(handle_delivery_result))

[github-actions]

PR Code Suggestions ✨

Explore these optional code suggestions:

Category	Suggestion	Impact
General	Use std Instant for timing Using tokio::time::Instant for local measurements can lead to unexpected behavior outside of a Tokio runtime. Replace it with std::time::Instant for reliable, monotonic timing. src/eth/storage/stratus_storage.rs [3] -use tokio::time::Instant; +use std::time::Instant; Suggestion importance[1-10]: 7 __ Why: The tokio::time::Instant may not behave as expected outside a Tokio context; using std::time::Instant ensures reliable, monotonic timing.	Medium
	Fix pending read metric invocation The pending‐read branch is invoking the block‐with‐changes metric instead of the transaction‐read metric, and it uses the permanent label. It should record a storage transaction read with the temporary label. src/eth/storage/stratus_storage.rs [538-541] let start = Instant::now(); let temp_tx = self.temp.read_pending_execution(tx_hash); if let Some(tx_temp) = temp_tx { - metrics::inc_storage_read_block_with_changes(start.elapsed(), label::PERM, true); + metrics::inc_storage_read_transaction(start.elapsed(), label::TEMP, true); return Ok(Some(TransactionStage::Pending(tx_temp))); } Suggestion importance[1-10]: 6 __ Why: The pending branch wrongly emits a block‐with‐changes metric with label::PERM; it should record a transaction read metric with label::TEMP to reflect the proper storage operation.	Low
	Separate timers for reads The same start timer is reused for both the pending and permanent reads, skewing the permanent‐read duration. Introduce a new timer just before the permanent read to capture its actual elapsed time. src/eth/storage/stratus_storage.rs [565-571] -let start = Instant::now(); -// ... pending read uses start ... +// after pending branch... +let perm_start = Instant::now(); let perm_tx = self .perm .read_transaction(tx_hash) .inspect_err(|err| { tracing::error!(reason = ?err, "failed to read transaction from permanent storage"); - metrics::inc_storage_read_transaction(start.elapsed(), label::PERM, false); + metrics::inc_storage_read_transaction(perm_start.elapsed(), label::PERM, false); }) - .inspect(|_| metrics::inc_storage_read_transaction(start.elapsed(), label::PERM, true))?; + .inspect(|_| metrics::inc_storage_read_transaction(perm_start.elapsed(), label::PERM, true))?; Suggestion importance[1-10]: 5 __ Why: Reusing the same start timer for both pending and permanent reads skews the latter’s timing; introducing a fresh perm_start before the permanent read yields accurate metrics.	Low

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-04T19:23:32.252371Z	6e74658	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[cloudwalk-review-agent]

Summary

Nice refactor overall — the new #[timed] attribute and per-FoundAt storage metrics significantly simplify call sites and make read-path observability richer.

I found one blocking correctness issue in transaction persistence flow introduced by the refactor.

Blocking finding

set_mined_block_number moved inside commit_changes and is now executed while latest_state_lock write guard is still held. Previously, the mined-number update happened after dropping that lock. Since reads of latest account/slot data are contractually synchronized with this lock, advancing mined block number before releasing it can expose a state where mined_number is visible as advanced while readers are still blocked from seeing corresponding latest state/cache updates, which can violate ordering assumptions in concurrent callers.

Please restore the prior ordering boundary (drop write guard before publishing mined-number advancement), or explicitly document/enforce that consumers never observe mined_number independently of this lock.

[github-actions]

Persistent review updated to latest commit 6e74658

[github-actions]

PR Code Suggestions ✨

No code suggestions found for the PR.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e746580d5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[stratus-benchmark]

Benchmark:
Run ID: bench-ed8fec22

Git Info:

• Branch: cheaper_metrics
• Commit: 6e74658
• PR: perf: cheaper metrics and logs while transaction lock is held #2675

Leader Stats:
RPS Stats: Max: 6718.00, Min: 955.00, Avg: 4153.55, StdDev: 451.82
TPS Stats: Max: 4682.00, Min: 90.00, Avg: 4106.92, StdDev: 593.98

Follower Stats:
Imported Blocks/s: Max: 125.00, Min: 61.00, Avg: 100.67, StdDev: 28.29
Imported Transactions/s: Max: 527621.00, Min: 224973.00, Avg: 413429.67, StdDev: 134252.15

Plots:

• View Leader Plot
• View Follower Plot

[stratus-benchmark]

Forwarding benchmark:
Run ID: bench-c4e1ae43

Git Info:

• Branch: cheaper_metrics
• Commit: 6e74658
• PR: perf: cheaper metrics and logs while transaction lock is held #2675

Follower Stats (transactions sent to follower):
RPS Stats: Max: 7220.00, Min: 3225.00, Avg: 4557.63, StdDev: 231.84
TPS Stats: Max: 4966.00, Min: 65.00, Avg: 4516.07, StdDev: 389.47

Plots:

• View Leader Plot

[cloudwalk-review-agent]

Summary

Great refactor direction overall (macro-based timing + FoundAt-aware storage metrics), but I found one blocking correctness issue still present in this head.

Blocking

#[timed] on async trait impl methods that return impl Future is currently timing future construction, not awaited execution, so import_online_mined_block latency is effectively near-zero and no longer represents real import work time. This regresses metric correctness in all importer workers using that annotation.

[stratus-benchmark]

Forwarding benchmark:
Run ID: bench-5ff67e45

Git Info:

• Branch: cheaper_metrics
• Commit: 988c477
• PR: perf: cheaper metrics and logs while transaction lock is held #2675

Follower Stats (transactions sent to follower):
RPS Stats: Max: 8965.00, Min: 1783.00, Avg: 4578.24, StdDev: 356.14
TPS Stats: Max: 5031.00, Min: 317.00, Avg: 4533.98, StdDev: 380.72

Plots:

• View Leader Plot

[stratus-benchmark]

Forwarding benchmark:
Run ID: bench-55304b08

Git Info:

• Branch: cheaper_metrics
• Commit: 988c477
• PR: perf: cheaper metrics and logs while transaction lock is held #2675

Follower Stats (transactions sent to follower):
RPS Stats: Max: 7869.00, Min: 3747.00, Avg: 4625.87, StdDev: 291.44
TPS Stats: Max: 5108.00, Min: 290.00, Avg: 4581.47, StdDev: 395.08

Plots:

• View Leader Plot

[stratus-benchmark]

Final benchmark:
Run ID: bench-2eb86df7

Git Info:

• Branch: main
• Commit: 695252f
• PR: perf: cheaper metrics and logs while transaction lock is held #2675

Leader Stats:
RPS Stats: Max: 9244.00, Min: 1709.00, Avg: 4211.47, StdDev: 469.11
TPS Stats: Max: 4923.00, Min: 101.00, Avg: 4167.31, StdDev: 508.13

Follower Stats:
Imported Blocks/s: Max: 115.00, Min: 20.00, Avg: 75.50, StdDev: 39.28
Imported Transactions/s: Max: 486257.00, Min: 89722.00, Avg: 314631.75, StdDev: 168570.78

Plots:

• View Leader Plot
• View Follower Plot