diff --git a/AGENTS.md b/AGENTS.md index 7d01807..446487a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -119,6 +119,11 @@ Lower-fidelity acquisition is a separate path. A venue recorder may live under `DataSource` until a named model can emit valid `MBOEvent` rows. Preserve raw evidence separately from inferred order-level events. +For crypto reconstruction changes, endpoint equality is only an integrity +check. Claims of improved fill realism need paired L2/L3 evidence or live-order +outcomes, explicit queue and latency assumptions, and held-out execution +metrics. See `docs/binance-reconstruction-study.md`. + ### Add an Instrument Spec 1. Add a plain `InstrumentSpec`. diff --git a/CHANGELOG.md b/CHANGELOG.md index b87d3df..ef2b645 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -26,6 +26,10 @@ All notable public changes to `ordersim` are documented here. - Added a streaming reconstruction study that aligns individual trades to depth intervals, validates joinable book-ticker states, preserves reconnect boundaries, and reports inferred flow and model sensitivity. +- Published the first crypto realism baseline: 99.9708% of valid captured + trades aligned, every processed L2 endpoint reconciled, every joinable + book-ticker state matched, and an open challenge to compare fill models + against paired L3 or live execution evidence. ## 0.1.3 - 2026-05-20 diff --git a/README.md b/README.md index 17770d1..7dbec36 100644 --- a/README.md +++ b/README.md @@ -19,6 +19,41 @@ interest include order-book replay, market replay, latency modeling, queue-position effects, fill simulation, execution modeling, execution-aware backtesting, and market microstructure research. +## Crypto L2, Reconstructed for MBO Replay + +Most crypto venues publish market-by-price depth, not the stable order IDs +needed for true market-by-order replay. `ordersim` takes a different path: it +combines sequence-validated L2 depth with individual trades, reconciles every +price-level endpoint, and emits a deterministic **virtual MBO** stream under +explicit queue assumptions. + +That means crypto data can use the same inspectable, queue-aware Python/C++ +execution engines as native MBO data without pretending the inferred orders +were observed at the exchange. + +The first full Binance USD-M study reconstructed: + +- **7,345,447 of 7,347,590 valid captured trades (99.9708%)**; +- **all 3,328,132 processed L2 depth endpoints**; +- **116,381 of 116,381 independently joinable top-of-book states**. + +The remaining trades occurred at snapshot or capture boundaries where the +available evidence could not place them safely. These results prove alignment +and book consistency, not knowledge of Binance's hidden FIFO queue. The +recommended conservative policy and an optimistic sensitivity policy expose +that uncertainty instead of burying it inside one fill rule. + +### The Crypto Realism Challenge + +Have a more realistic L2 execution model? Compare it with evidence. Use paired +L2/L3 data or live passive-order outcomes, disclose the queue and latency +assumptions, and report fill occurrence, filled quantity, and time-to-fill on +held-out observations. The benchmark should reward predictive execution +realism, not simply the ability to reproduce an L2 endpoint. + +See the [full reconstruction study](https://github.com/tradingexpert/ordersim/blob/main/docs/binance-reconstruction-study.md) and +join the [public validation challenge](https://github.com/tradingexpert/ordersim/issues/62). + ## What It Does - Replays order-book data and simulates order execution with explicit order diff --git a/docs/architecture.md b/docs/architecture.md index c49d177..77eac96 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -78,6 +78,12 @@ flowchart LR venue --> capture --> source --> model --> modeled --> parquet --> replay ``` +This is the crypto-enabling path: aggregated venue data reaches the same +canonical event and execution-engine boundary as native MBO, but only after a +named model makes the missing order-level structure explicit. New +reconstruction policies should compete on paired-L3 or live-fill prediction, +not on undocumented claims of realism. + Capture code may live beside connectors because it owns venue I/O and source schemas. Capture alone is not a `DataSource`: observed L2 rows must not be presented as exchange-native MBO. The reconstruction model owns that diff --git a/docs/assumptions.md b/docs/assumptions.md index 0d241cd..60d7257 100644 --- a/docs/assumptions.md +++ b/docs/assumptions.md @@ -31,6 +31,13 @@ MBP source provides aggregated size by price level, not individual queue members. Queue-position behavior inferred from MBP is necessarily a model, not an observation. +That does not make L2 useless for execution simulation. Sequence-valid depth, +individual trades, and exact endpoint reconciliation constrain the possible +order-level histories much more tightly than an unconstrained touch-fill rule. +`ordersim` represents one such history as virtual MBO so it can be audited and +replayed by the ordinary queue-aware engines. Its realism must ultimately be +judged against paired L3 or live fills, not by the label "MBO" alone. + Raw L2 capture and L2-to-L3 reconstruction are separate operations. Capture must preserve the observed price-level updates and trades. Reconstruction must name its cancellation-allocation, event-ordering, and randomness assumptions, diff --git a/docs/binance-reconstruction-study.md b/docs/binance-reconstruction-study.md index e5e1d29..015f142 100644 --- a/docs/binance-reconstruction-study.md +++ b/docs/binance-reconstruction-study.md @@ -4,6 +4,12 @@ This note records the first empirical validation of `ordersim`'s Binance L2-to-virtual-MBO path. It is evidence for a named model, not a claim that aggregated depth reveals Binance's true FIFO order queue. +The purpose is larger than one Binance connector: make aggregated crypto depth +usable by an order-level execution engine without hiding the inference needed +to cross that boundary. The model produces an auditable virtual queue, records +its assumptions, and makes competing queue interpretations directly +comparable on the same observed evidence. + ## Capture The study used locally captured Binance USD-M futures evidence from @@ -68,6 +74,11 @@ quantity. Every reconnect starts a separate snapshot-anchored segment. | Required within-window replenishment | 17,306.673 BTC | 662,705.196 ETH | | Replenishment / trade quantity | 10.90% | 14.65% | +Combined, the model aligned and represented 7,345,447 of 7,347,590 valid +captured trade messages: 99.9708% by count. The 2,143 unassigned messages fall +at snapshot, reconnect, or final-capture boundaries where assigning them would +require evidence the capture does not contain. + The book-ticker denominator includes only rows whose update ID exactly matches a processed depth endpoint. It is not the count of all captured book-ticker messages. @@ -118,3 +129,26 @@ Useful contributions include: bounds; - evidence about Binance's zero-value `@trade` messages without relying on undocumented fields as stable production contracts. + +## The Crypto Realism Challenge + +This study is a public baseline, not a declaration that the queue problem is +solved. A competing L2 fill or reconstruction model is more realistic only if +it predicts evidence that was not used merely to reconcile the book. + +A useful comparison should: + +1. consume the same sequenced L2 endpoints and individual trades; +2. disclose event-ordering, cancellation-allocation, hidden-liquidity, latency, + and market-impact assumptions; +3. preserve the observed book or report every divergence; +4. evaluate against paired native L3 data or actual passive-order outcomes; +5. report fill classification, filled-quantity error, and time-to-fill error on + held-out intervals; +6. publish enough code and aggregate results for another researcher to repeat + the comparison without redistributing restricted market data. + +Endpoint equality is an integrity requirement, not the winning metric. The +question is whether a model predicts real execution better. Results, datasets +that can legally be shared, and alternative policies belong in the +[public validation challenge](https://github.com/tradingexpert/ordersim/issues/62). diff --git a/docs/connectors.md b/docs/connectors.md index 4ee0811..3f53c51 100644 --- a/docs/connectors.md +++ b/docs/connectors.md @@ -223,6 +223,11 @@ RPI orders. When absent, `normal_quantity` is `None`, not an inferred value. ### Studying L2-to-Virtual-MBO Reconstruction +This is the central crypto execution path in `ordersim`, not merely a file +converter. It creates a deterministic, inspectable order-level representation +from the strongest public evidence available while preserving a clear line +between observed depth and inferred queue structure. + Run the evidence study before materializing modeled MBO for a new symbol or capture: @@ -266,6 +271,11 @@ segments as if the reconnect were an observed continuous MBO stream; write separate canonical files or preserve an explicit segment boundary in the research workflow. +The full-capture baseline and rules for comparing alternative fill models are +published in `docs/binance-reconstruction-study.md`. Community comparisons are +tracked in the public validation challenge rather than accepted as realism +claims without paired-L3 or live-fill evidence. + For the user-facing decision guide, see `docs/data-guide.md`. ## In-Memory Sources diff --git a/docs/data-guide.md b/docs/data-guide.md index 0ebd18e..5b7ae55 100644 --- a/docs/data-guide.md +++ b/docs/data-guide.md @@ -106,6 +106,13 @@ Binance USD-M depth is one such lower-fidelity source. The Binance capture tool records raw L2 depth, individual and aggregate trades, and integrity metadata, but its output is not accepted by `Replay` as observed MBO. +This boundary is also what enables crypto research in `ordersim`. Rather than +falling back to touch-fill rules, the reconstruction path turns L2 and +individual-trade evidence into an auditable virtual MBO stream that can run +through the ordinary queue-aware execution engines. It preserves what was +observed, labels what was inferred, and keeps multiple queue assumptions +comparable. + After a capture completes, use `BinanceCaptureSource` to stream exact typed snapshots, sequence-validated depth updates, trades, and book tickers. The main capture preserves Binance's individual `@trade` stream. Run @@ -126,7 +133,8 @@ canonical MBO has no implicit clear-book event. There is deliberately no direct `BinanceCaptureSource -> Replay` path. See `docs/connectors.md` for the capture -and validation contract. +and validation contract, and `docs/binance-reconstruction-study.md` for the +full-capture results and public realism challenge. ## Related Docs