Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
9dae839
add evmonly executor load test harness
codchen Jun 29, 2026
15ce1e5
add prebuilt block loadtest mode
codchen Jun 30, 2026
af98434
freeze generated loadtest state after prebuild
codchen Jun 30, 2026
7c27a0a
add evmonly OCC fallback metrics
codchen Jun 30, 2026
4c643d0
add ERC20 transfer loadtest workload
codchen Jul 1, 2026
fd0b615
verify ERC20 loadtest token balances
codchen Jul 1, 2026
3fe649f
add persistent loadtest result sink
codchen Jul 1, 2026
69f2b54
clean up persistent loadtest files
codchen Jul 1, 2026
7c73081
add async persistence backpressure metrics
codchen Jul 2, 2026
98bd357
wire loadtest sinks through evmonly executor
codchen Jul 2, 2026
5d3ac20
remove synchronous loadtest file sink
codchen Jul 2, 2026
599e184
add loadtest profiling flags
codchen Jul 2, 2026
7fa0b6d
pipeline loadtest sender recovery
codchen Jul 2, 2026
1390018
add loadtest worker pinning controls
codchen Jul 2, 2026
4d23269
remove loadtest worker pinning controls
codchen Jul 2, 2026
1f29569
reuse loadtest block results
codchen Jul 2, 2026
bbe827e
fix loadtest result sink interface after rebase
codchen Jul 22, 2026
4f2763f
set loadtest blob base fee after rebase
codchen Jul 27, 2026
74905ee
mark loadtest state reader concurrent
codchen Jul 28, 2026
0d63fb4
add evmonly loadtest recipient conflicts
codchen Jul 28, 2026
1d6fe15
add snapshot revert loadtest workload
codchen Jul 28, 2026
a8d2d95
add evmonly same-sender load scenario
codchen Jul 31, 2026
e5b513a
report occ reruns in evmonly loadtest
codchen Jul 31, 2026
f27ad40
set default loadtest block gas limit
codchen Aug 3, 2026
181ab3c
fix evm-only loadtest lint issues
codchen Aug 4, 2026
e20d332
address evm-only loadtest review nits
codchen Aug 4, 2026
902daa5
address additional evm-only loadtest review feedback
codchen Aug 4, 2026
c0ebb10
remove evm-only loadtest streaming mode
codchen Aug 4, 2026
bcfba73
split evm-only loadtest harness files
codchen Aug 4, 2026
3c58557
count successful evm-only loadtest txs
codchen Aug 5, 2026
6dd1f60
reuse sei-load offline scenarios
codchen Aug 5, 2026
cf656de
address evm-only loadtest review feedback
codchen Aug 5, 2026
4bc9274
use sei-load v0.0.1
codchen Aug 6, 2026
201f2e2
fix tracing batch processor characterization
codchen Aug 6, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion app/benchmark/generator.go
Original file line number Diff line number Diff line change
Expand Up @@ -443,7 +443,7 @@ func (g *Generator) transitionToLoadPhase() {
panic("benchmark: No scenarios available for load generation")
}

g.loadGenerator = generator.NewWeightedGenerator(weightedConfigs...)
g.loadGenerator = generator.NewWeightedGenerator(nil, weightedConfigs...)
logger.Info("benchmark: Load generator initialized and ready", "scenarios", len(weightedConfigs))
}

Expand Down
2 changes: 2 additions & 0 deletions giga/evmonly/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ The `evmonly` package currently provides:
contract address, and effective gas price
- a map-backed `MemoryState` for tests and early integration
- fail-closed custom precompile placeholders
- a standalone load harness at `giga/evmonly/cmd/evmonly-loadtest` that feeds
generated transfer blocks into the executor with mock state and receipt sinks

The executor accepts config for nonce checks, gas-price checks, minimum gas
price, chain config, parse workers, OCC workers, result pooling, and the custom
Expand Down
204 changes: 204 additions & 0 deletions giga/evmonly/cmd/evmonly-loadtest/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,204 @@
# evmonly-loadtest

`evmonly-loadtest` is a standalone executable for feeding synthetic blocks to
the EVM-only executor without Cosmos SDK state, mempool, RPC, or chain
persistence.

The synthetic workload defaults to local EVM chain ID `1337`; override it with
`--chain-id` when testing another signing domain.

It currently generates pure EVM legacy transfer transactions, ERC20 transfer
transactions using `sei-load`'s compiled contract runtime, and a contract-call
workload that exercises nested StateDB
snapshot/revert behavior. By default, each generated sender account has one
nonce-0 transaction and is funded in the command's in-memory genesis state
before its block is queued. Recipients are unique by default so the transfer
workloads exercise the optimistic no-overlap case. Pass
`--recipient-conflict-rate=<0..1>` to pair that fraction of each block's
transactions onto shared recipients, or pass `--recipient=0x...` to force all
transactions to a single recipient. Pass `--same-sender` to use one sender per
block with sequential transaction nonces.

Run a bounded prebuilt test:

```bash
go run ./giga/evmonly/cmd/evmonly-loadtest --blocks=1000 --txs-per-block=1000
```

Example local saturation run:

```bash
go run ./giga/evmonly/cmd/evmonly-loadtest \
--metrics-addr= \
--report-interval=5s \
--blocks=2000 \
--txs-per-block=1000 \
--builders=16 \
--workers=1 \
--executor-workers=12 \
--gas-price-wei=0 \
--min-gas-price-wei=0 \
--queue-size=512
```

The three executor OCC scenarios have direct command-line forms:

```bash
# Conflict-free: unique senders and recipients.
go run ./giga/evmonly/cmd/evmonly-loadtest \
--blocks=400 \
--gas-price-wei=0 --min-gas-price-wei=0

# Hot recipient: unique senders all credit one account.
go run ./giga/evmonly/cmd/evmonly-loadtest \
--blocks=400 \
--gas-price-wei=0 --min-gas-price-wei=0 \
--recipient=0x00000000000000000000000000000000000000f1

# Same-sender nonce chain: one sender and sequential nonces within each block.
go run ./giga/evmonly/cmd/evmonly-loadtest \
--blocks=400 \
--gas-price-wei=0 --min-gas-price-wei=0 \
--same-sender
```

Example conflict run:

```bash
go run ./giga/evmonly/cmd/evmonly-loadtest \
--metrics-addr= \
--report-interval=5s \
--blocks=400 \
--txs-per-block=5000 \
--builders=16 \
--workers=1 \
--executor-workers=12 \
--gas-price-wei=0 \
--min-gas-price-wei=0 \
--recipient-conflict-rate=0.10
```

Example snapshot/revert contract-call run:

```bash
go run ./giga/evmonly/cmd/evmonly-loadtest \
--metrics-addr= \
--report-interval=5s \
--workload=snapshot-revert \
--blocks=400 \
--txs-per-block=5000 \
--builders=16 \
--workers=1 \
--executor-workers=12 \
--gas-price-wei=0 \
--min-gas-price-wei=0
```

The command prebuilds every bounded run before starting the prepare/recover and
executor workers, which isolates executor throughput from block generation:

```bash
go run ./giga/evmonly/cmd/evmonly-loadtest \
--metrics-addr= \
--report-interval=5s \
--blocks=400 \
--txs-per-block=5000 \
--builders=48 \
--workers=1 \
--executor-workers=24 \
--gas-price-wei=0 \
--min-gas-price-wei=0 \
--queue-size=512
```

Prebuilding requires `--blocks > 0` and stores every raw block in memory. Sender
recovery still runs in the measured phase, but it is pipelined ahead of
execution through `--prepare-workers`.

The zero gas price/min-gas settings keep the conflict-free transfer workload
focused on the optimistic no-overlap case. Non-zero fees make every transaction
update the same coinbase balance, which is a real intra-block conflict.

Useful knobs:

- `--blocks`: number of blocks to prebuild and execute. This is required and
must be greater than `0`.
- `--workers`: parallel executor workers. The default is `1`. Prepared blocks
are forwarded to workers in block-number order, but `--workers > 1` can still
finish execution out of order; this is safe for the harness because generated
state is frozen for prebuilt runs and executor changesets are not applied back
into the input state.
- `--executor-workers`: parallel OCC workers inside each executor. The default
is `min(12, GOMAXPROCS)`, following the `sei-v3` OCC worker default.
- `--prepare-workers`: parallel stateless preparation workers used for
transaction RLP decode and sender recovery. The default is `GOMAXPROCS`.
- `--parse-workers`: parallel transaction decode/sender recovery workers inside
each prepared block. The default `0` uses `1` when `--prepare-workers > 1` to
avoid multiplying block-level and intra-block parser pools, otherwise it uses
`GOMAXPROCS`.
- `--builders`: parallel block builders used to keep the input queue full. The
default is `GOMAXPROCS`.
- `--queue-size`: buffered raw and prepared blocks. The default is `64`.
- `--result-pool-size`: reusable executor result slots. The default `0`
sizes the pool for in-flight executor and async sink results; negative
disables result pooling.
- `--result-sink`: executor output sink, either `discard` or `file`. The
default is `discard`.
- `--persist-dir`: directory used by `--result-sink=file` for temporary
append-only changeset and receipt files. Files are removed on shutdown or
interrupt cleanup.
- `--persist-buffer-size`: buffered writer size for `--result-sink=file`.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] --persist-sync is missing from this knob list even though it materially changes file-sink cost (flush + fsync per record) and is exercised in main_test.go. Worth a line next to --persist-buffer-size so the durability/throughput tradeoff is discoverable.

- `--persist-queue-size`: async file-sink record queue size. The default `0`
uses `2 * --queue-size`.
Comment on lines +148 to +152

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 The 'Useful knobs' list in the loadtest README omits --persist-sync, even though it's a real flag (config.go) that materially changes the --result-sink=file durability/throughput tradeoff (adds a flush+fsync per record) and is exercised in main_test.go.

Extended reasoning...

The bug: config.go registers a real, functional flag:

fs.BoolVar(&cfg.persistSync, "persist-sync", false, "fsync persistent result files from the async sink writer")

persistSync is threaded through to newAppendRLPFile(path, bufferSize, syncOnWrite) in sinks.go, and when syncOnWrite is true, appendRLPFile.WriteRecord does an extra f.writer.Flush() + f.file.Sync() on every single record written by the async file-sink writer. This is a meaningful cost/durability knob for --result-sink=file: leaving it off buffers writes and only flushes/syncs on close, while turning it on trades throughput for per-record durability guarantees. It's exercised directly in main_test.go's TestFileResultSinkWritesRLPRecordsAndCleansUpOnCancel, which passes --persist-sync explicitly.

Where the gap is: giga/evmonly/cmd/evmonly-loadtest/README.md's "Useful knobs" list (lines ~143-148 in the PR diff) documents the sibling flags right next to where this one belongs:

- `--persist-dir`: directory used by `--result-sink=file` for temporary
  append-only changeset and receipt files. Files are removed on shutdown or
  interrupt cleanup.
- `--persist-buffer-size`: buffered writer size for `--result-sink=file`.
- `--persist-queue-size`: async file-sink record queue size. The default `0`
  uses `2 * --queue-size`.

--persist-sync never appears anywhere in this file (confirmed by grepping the README for "persist-sync" — no matches), so a user reading the docs to tune or understand --result-sink=file behavior has no way to discover that this flag exists, let alone that it changes durability/performance.

Why nothing else catches it: This is a pure documentation gap — the flag itself works correctly in code and is tested. There's no validation or generation step tying the README's knob list to the actual flag set in config.go, so a flag can be added to parseConfig without the docs staying in sync.

Step-by-step proof:

  1. A user reads giga/evmonly/cmd/evmonly-loadtest/README.md to decide how to configure --result-sink=file for a benchmark run, specifically to understand the available durability/performance knobs.
  2. They see --persist-dir, --persist-buffer-size, and --persist-queue-size documented, but no mention of --persist-sync.
  3. They conclude (incorrectly) that there is no way to request per-record fsync durability, or run a benchmark unaware that leaving --persist-sync at its default understates the cost of a "safe" persistent run — since flush+fsync only happens implicitly at shutdown, not per record.
  4. Separately, go run ./giga/evmonly/cmd/evmonly-loadtest --help (or reading config.go) would reveal the flag, but that requires already knowing to look outside the README that's supposed to be the authoritative "Useful knobs" reference.

Fix: Add a line for --persist-sync next to --persist-buffer-size/--persist-queue-size, e.g. "--persist-sync: fsync persistent result files after every record written by the async file-sink writer; trades throughput for durability. The default is false."

Severity: This is scoped to nit — it's a documentation completeness gap in a standalone dev/calibration harness (giga/evmonly/cmd/evmonly-loadtest), not app code. No behavior is incorrect; the flag itself works and is tested. This same omission was also independently flagged by reviewer seidroid in the PR timeline (both as an inline comment on the README and in a standalone PR comment), corroborating the finding.

- `--metrics-addr`: Prometheus endpoint. The default is
`127.0.0.1:9698`; set it to empty to disable HTTP metrics.
- `--report-interval`: stdout rate reporting interval. The default is `5s`.
- `--gas-price-wei`, `--min-gas-price-wei`, `--sender-balance-wei`,
`--transfer-value-wei`: transaction economics for the generated accounts.
- `--recipient-conflict-rate`: fraction of each block's transactions that are
paired onto shared recipients; `0` keeps recipients unique and `1` pairs all
possible transactions.
- `--same-sender`: use one sender per native-transfer block and assign
transaction nonces in block order.
- `--workload`: workload type, either `transfer`, `erc20-transfer`, or
`snapshot-revert`.
- `--snapshot-revert-contract`, `--snapshot-revert-helper`: generated contract
addresses used by `--workload=snapshot-revert`.

The command reports these saturation signals on stdout and at `/metrics`:

- block input throughput
- block preparation throughput
- prepared transactions per second
- block finishing throughput
- finished, successful, and failed transactions per second
- total gas consumed per second
- total OCC transaction rerun attempts
- prepared blocks queued for execution and cumulative totals
- result-sink records queued, enqueued, written, bytes written, enqueue wait,
and write time
- result-pool capacity, available slots, and overflow allocations

The default executor output path intentionally discards results through mocks:

- `generatedState` implements `evmonly.StateReader` and supplies generated
genesis balances, nonces, code, and storage.
- `discardResultSink` applies the executor `StateChangeSet` to
`discardStateWriter` and discards Ethereum receipts.

With `--result-sink=file`, the loadtest harness hands pooled
`evmonly.BlockResult` values to an async writer through the executor's
`evmonly.ResultSink` interface. The writer appends changesets to
`changesets.rlp` and receipts to `receipts.rlp` under `--persist-dir`; each
record is framed as an 8-byte big-endian block height, an 8-byte big-endian RLP
payload length, and the RLP payload. The files are temporary calibration
artifacts and are removed when the process exits normally or handles
`SIGINT`/`SIGTERM`. `sink_enqueue_wait` is the primary backpressure signal: a
non-zero value means executor workers waited for async sink queue capacity.

Native and ERC20 transaction construction and genesis seeding come from
`github.com/sei-protocol/sei-load/generator/offline`. Future reusable EVM
workloads should add an offline scenario there and a block/recipient adapter in
this command. Executor-specific workloads can remain beside
`TransferWorkload`, `ERC20TransferWorkload`, and `SnapshotRevertWorkload` and
reuse the same block producer/prepare/executor/metrics pipeline.
Loading
Loading