Skip to content
View nisgemML's full-sized avatar

Block or report nisgemML

Block user

Prevent this user from interacting with your repositories and sending you notifications. Learn more about blocking users.

You must be logged in to block users.

Content in all repositories owned by your account will be closed.
Maximum 250 characters. Please don’t include any personal information such as legal names or email addresses. Markdown is supported. This note will only be visible to you.
Report abuse

Contact GitHub support about this user’s behavior. Learn more about reporting abuse.

Report abuse
nisgemML/README.md

Nishant Gemawat β€” Quantitative Systems Engineer

Low-latency trading infrastructure Β· C++20/23/26 Β· Python Β· OCaml Β· Financial Markets


System Architecture

15 repositories forming a coherent trading system β€” from raw market data through matching, execution, smart routing, and quantitative research. tick-to-trade is the integration layer: it actually wires the market-data and messaging layers below into one running, tested, measured pipeline, rather than leaving them as separate components that have only ever been benchmarked in isolation.

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                          MARKET DATA LAYER                               β”‚
β”‚                                                                          β”‚
β”‚  udp-multicast-receiver  ←  MoldUDP64 / ITCH 5.0 feed handler            β”‚
β”‚  SO_TIMESTAMPING Β· recvmmsg batch (64 datagrams/syscall) Β· 3/3 tests     β”‚
β”‚                                                                          β”‚
β”‚  fix-parser  ←  Zero-copy FIX 4.2/4.4 parser                             β”‚
β”‚  p50 112ns full parse Β· p50 60ns fast parse Β· std::span zero-copy        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚
                       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                          MESSAGING LAYER                                 β”‚
β”‚                                                                          β”‚
β”‚  mpsc-queue  ←  Lock-free MPSC Β· beats mutex/spinlock/boost::lockfree    β”‚
β”‚  by 3.7-4.3x (real hardware, median of 5 runs) Β· six-claim formal proof  β”‚
β”‚  Β· 18 TSan litmus tests Β· push_batch API                                 β”‚
β”‚                                                                          β”‚
β”‚  io-uring-queue  ←  SPSC ring + io_uring async logger                    β”‚
β”‚  ring push/pop p50=215ns Β· io_uring cuts producer-thread p50 by ~9x      β”‚
β”‚  vs synchronous write() Β· 6,322 assertions across 2 CTest suites         β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚
                       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                    INTEGRATION LAYER β€” NEW                               β”‚
β”‚                                                                          β”‚
β”‚  tick-to-trade  ←  Real two-thread pipeline: feed β†’ MpscQueue β†’          β”‚
β”‚  decision β†’ IOURingLogger, using the actual source of the three          β”‚
β”‚  repos above (vendored, not reimplemented). Differential-tested          β”‚
β”‚  against an independent reference order book, deterministic replay,      β”‚
β”‚  a real methodology mistake caught mid-benchmark and documented.         β”‚
β”‚  Milestones 1-3 of 5 complete (skeleton, correctness hardening,          β”‚
β”‚  measurement) β€” see that repo's own README for what's still open.        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚
                       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                          MATCHING ENGINE                                 β”‚
β”‚                                                                          β”‚
β”‚  options-engine  ←  p50 submit 28ns (see that repo's own PROFILING.md    β”‚
β”‚  for a noted discrepancy against its README figure, not yet reconciled)  β”‚
β”‚  SoA LOB Β· AVX2 SIMD find_level 2.0Γ— Β· pool allocator                    β”‚
β”‚                                                                          β”‚
β”‚  low-latency-trading-engine  ←  full-stack C++20 + OCaml                 β”‚
β”‚  ITCH 5.0 Β· Kyle Ξ» microstructure Β· 6/6 test suites                      β”‚
β”‚                                                                          β”‚
β”‚  hash-map  ←  Robin Hood + SSE4.2 SIMD-probe hash maps                   β”‚
β”‚  avg probe < 1.5 (Robin Hood) Β· 16-slot SIMD groups Β· 1212/1212 tests    β”‚
β”‚                                                                          β”‚
β”‚  cpp26-alloc  ←  C++26 allocator Β· Contracts P2900R6                     β”‚
β”‚  std::generator Β· std::add_sat Β· std::saturate_cast Β· 102/102 tests      β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚
                       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                          EXECUTION LAYER                                 β”‚
β”‚                                                                          β”‚
β”‚  sor  ←  Smart Order Router                                              β”‚
β”‚  BestPrice / LowestFee / ProRata Β· 4-venue fee model Β· VWAP Β· 18 tests   β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜
                       β”‚
                       β–Ό
β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                          QUANTITATIVE RESEARCH                           β”‚
β”‚                                                                          β”‚
β”‚  options-market-maker  ←  Heston Gil-Pelaez FFT Β· SSVI Β· vanna-volga     β”‚
β”‚  Sharpe 2.26 Β· 89 tests (Python + OCaml QCheck)                          β”‚
β”‚                                                                          β”‚
β”‚  avellaneda-stoikov  ←  Closed-form A-S market maker                     β”‚
β”‚  Sharpe 10.0 vs 3.6 baseline Β· 87 tests Β· multi-agent LOB simulation     β”‚
β”‚                                                                          β”‚
β”‚  lob-microstructure-calibration  ←  Kyle Ξ» Β· Roll Β· kappa MLE Β· OFI      β”‚
β”‚  HAC-robust OLS Β· Bartlett-corrected autocovariance Β· 18 tests           β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
β”‚                          FUNCTIONAL SYSTEMS                              β”‚
β”‚                                                                          β”‚
β”‚  ocaml-trading-primitives  ←  Functional LOB in OCaml                    β”‚
β”‚  Make(P:PRIORITY) functor Β· CME Rule 512.B Β· 11/11 QCheck tests          β”‚
β”‚                                                                          β”‚
β”‚  competitive-programming  ←  Trading-oriented algorithms                 β”‚
β”‚  SegTree Β· SparseTable O(1) RMQ Β· DSU+rollback Β· CHT Β· 16/16 tests       β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Repository Index

C++ Systems

Repo Signal Key numbers
tick-to-trade Real integration: feed β†’ MpscQueue β†’ decision β†’ IOURingLogger 4,850 differential comparisons vs an independent reference book, deterministic replay, Milestones 1-3/5 complete
mpsc-queue Lock-free MPSC, six-claim formal proof 3.7-4.3x faster than mutex/spinlock/boost::lockfree::queue (real hardware, median of 5 runs), 18 TSan litmus tests
options-engine AVX2 SIMD matching engine p50=28ns, 2.0Γ— SIMD speedup, PROFILING.md (see note below on an unreconciled figure)
fix-parser Zero-copy FIX 4.2/4.4 112ns full / 60ns fast, 34 tests
udp-multicast-receiver MoldUDP64/ITCH 5.0 feed handler SO_TIMESTAMPING, recvmmsg, 3 tests
hash-map Robin Hood + SSE4.2 SIMD probe avg probe <1.5, 1212 tests
io-uring-queue SPSC ring + io_uring logger ring push/pop p50=215ns, io_uring cuts producer-thread p50 ~9x vs write(), 6,322 assertions
sor Smart order router BestPrice/LowestFee/ProRata, 18 tests
cpp26-alloc C++26 Contracts allocator P2900R6, std::generator, 102 tests
low-latency-trading-engine Full-stack C++20 + OCaml Kyle Ξ» sim, ITCH 5.0, 6 test suites

Quantitative Research

Repo Signal Key numbers
options-market-maker Heston FFT, SSVI, vanna-volga Sharpe 2.26, 89 tests
avellaneda-stoikov A-S stochastic control Sharpe 10.0 vs 3.6 baseline, 87 tests
lob-microstructure-calibration Kyle Ξ», Roll, kappa MLE, OFI HAC-robust, real AAPL LOBSTER data

Functional Systems

Repo Signal Key numbers
ocaml-trading-primitives Make(P:PRIORITY) functor, CME Rule 512.B 11/11 QCheck property tests
competitive-programming SegTree, SparseTable, DSU+rollback, CHT 16/16 tests, trading use cases

Key Design Decisions

Why acq_rel is sufficient for the MPSC queue β€” no MFENCE needed The producer's release store on next and the consumer's acquire load establish the happens-before chain that makes the payload visible. seq_cst (MFENCE on x86) is unnecessary and costs real cycles per push for zero correctness benefit. The formal proof in mpsc-queue documents all six claims with an explicit x86-TSO instruction table proving the weaker ordering suffices. Real hardware confirms the payoff: 3.7-4.3x faster than mutex, spinlock, or boost::lockfree::queue on the same workload.

Why the ring buffer helps even when io_uring's own completion latency doesn't io-uring-queue's own measurements across four logging strategies found something more interesting than "io_uring is faster": decoupling the producer (matching-engine thread) from the logger via an SPSC ring cuts the producer's own p50 by ~9x vs blocking synchronous write() (267ns β†’ 30ns) β€” and that win holds identically whether the logger side uses a classic blocking writer or io_uring, because from the producer's perspective both are just "push into a ring." What io_uring adds on top is asynchronous completion on the logger side, not less contention on the producer's own hot path. SQPOLL, measured in that same environment, was ~52x worse at p50 than plain io_uring (7,988,146ns vs 154,341ns) β€” the opposite of SQPOLL's usual selling point β€” because SQPOLL's kernel polling thread needs a dedicated pinned core to deliver on its theoretical zero-syscall benefit, which that environment didn't have. That root cause is documented, not hand-waved, in the repo's own BENCHMARK_RESULTS.md, and it's a better story for an interview than a clean win would have been: it shows the difference between a technique's asymptotic promise and what it actually does on the hardware you're given.

Why tick-to-trade vendors instead of using git submodules tick-to-trade copies the exact header files it needs from mpsc-queue, io-uring-queue, and udp-multicast-receiver rather than depending on them via submodule. A portfolio reviewer gets a single cmake && cmake --build, no submodule init step, no risk of an out-of-date submodule pointer silently changing what a benchmark actually ran against. The trade-off, stated plainly: each vendored file's source of truth remains its origin repo β€” a bug fix in the queue belongs in mpsc-queue, not copy-pasted forward, and vendored copies can drift from their origin if not periodically re-synced. That trade-off is worth it for a portfolio piece meant to be cloned and built in five minutes by someone who has never seen the other 14 repos.

Why SoA beats pointer-based order books by ~25ns per match A pointer-based LOB chases pointers across cache lines during the matching sweep β€” 3 cache misses per match. SoA keeps prices[] as a hot contiguous array; at 128 levels the entire array fits in L1 cache. Measured difference: ~25ns per match, confirmed in options-engine benchmarks with committed PROFILING.md β€” though see the note in the repository table above: that repo's PROFILING.md and its README/BENCHMARK_RESULTS.md currently disagree on the exact AVX2 speedup figure (1.6x vs 2.0x) for what should be the same benchmark, a sync gap flagged for reconciliation, not yet fixed.

Why Robin Hood with backward-shift deletion Tombstone deletion accumulates probe length under heavy cancel churn β€” degrades to O(n) probe over time. Backward-shift maintains ≀1.5 average probe length indefinitely. Fibonacci hashing gives uniform distribution on sequential order IDs that modulo hashing clusters into the first N buckets.

Why recvmmsg over recvmsg for market data At 1M packets/sec, individual recvmsg() calls cost 200ms CPU/sec in syscall overhead alone. recvmmsg() with batch=64 reduces this to 3.1ms/sec β€” 64Γ— reduction. Trade-off: up to 63 Γ— inter-packet latency added; acceptable for feed handler, not for the matching engine.

Why the A-S Sharpe (10.0) is honest, not cherry-picked 1,000 paths, fresh seed per path, PnL = cash + inventory Γ— final_mid, Sharpe computed across all paths β€” not the best. Naive uses identical engine and seeds. The 2.8Γ— improvement is solely from inventory skew β€” ablated and documented in avellaneda-stoikov/SIMULATION_RESULTS.md.

Why the IC lookahead correction matters Rolling IC weights including future returns inflated the t-stat to 3.76. After fixing the weight shift, t-stat = 1.08 β€” borderline, not highly significant. This is the correct result. Reporting it honestly is the only valid approach.

Why Make(P:PRIORITY) functor in OCaml PriceTime and ProRata priority are exchange-specific rules that change without warning (CME Rule 512.B add_order_front for reduce-only replaces). The functor separates the priority policy from the matching logic β€” swap priority without touching the matching core. This is the idiom Jane Street uses in their trading systems.

A real bug caught while integrating, kept as a finding tick-to-trade's first working pipeline had no backpressure between the feed thread and the decision thread β€” the feed thread, pure CPU-bound parsing, raced arbitrarily far ahead of the decision thread, which does real work per event. Result: a 3.9ms mean latency that was almost entirely queue backlog, not processing cost. This was the identical mistake an earlier version of mpsc-queue's own tick-to-trade benchmark shipped with. Same fix both times: bound how far ahead the producer can get. Kept in both repos' git history and documentation as a finding, not silently corrected β€” the point of documenting a caught bug is that it happened, not that it was eventually fixed.


Production Readiness Scorecard

Component Status Remaining gap
MPSC queue βœ… Complete Integration into engine hot path
Options matching engine βœ… Complete Multi-symbol sharding; PROFILING.md/README figure reconciliation
FIX parser βœ… Complete Full session-layer state machine
Market data feed handler βœ… Complete Hardware timestamp correlation
Hash map βœ… Complete In-engine integration benchmarks
io_uring async logger βœ… Complete Multi-sink backends
tick-to-trade (integration) πŸ”§ In progress β€” Milestones 1-3/5 Live multicast wiring, gap/loss injection at integration level, logger-overflow policy, LIMITATIONS.md, interview-prep polish
Smart Order Router βœ… Complete Real multi-venue historical data
cpp26-alloc βœ… Complete 24h stability test under load
A-S market maker βœ… Complete Real ITCH data calibration (in progress)
Microstructure calibration βœ… Complete Live LOB data pipeline
Options market maker βœ… Complete Live vol surface feed
Risk layer (pre-trade) πŸ”§ In progress Hard latency budgets
Multi-symbol sharding πŸ”§ In progress Per-symbol CPU affinity
Kernel bypass (DPDK) πŸ“‹ Designed NIC hardware required
24h+ stability tests πŸ“‹ Planned Dedicated bare-metal required

Honest gaps: Components are individually production-quality with real benchmarks. tick-to-trade is the first attempt at wiring several of them into one measured system rather than leaving that integration as an unverified claim β€” it's mid-way through its own 5-milestone plan, not finished, and says so in its own README rather than being presented as done. Remaining work across the portfolio: full system integration beyond what tick-to-trade covers so far, long-running stability under realistic sustained load, and kernel-bypass networking (hardware-dependent). These gaps close in the first 3-6 months at a firm with appropriate infrastructure.


Background

13 years delivering software infrastructure across financial services (Morgan Stanley, State Street via TCS), payments (Worldpay), and enterprise systems. M.S. Computer Science, Texas A&M University–Commerce. Post-Graduate Certificate AI/ML, Purdue University.


All benchmark numbers reproducible β€” build instructions in each repo's README. Environment: Ubuntu 22.04/24.04, GCC 12/13/14, x86-64, plus WSL2 on real laptop hardware for mpsc-queue's cross-thread figures. Container numbers reported honestly; single-vCPU-sandbox and pending-real-hardware numbers labeled as such where applicable β€” see each repo's own BENCHMARK_RESULTS.md for the exact tier.

Pinned Loading

  1. tick-to-trade tick-to-trade Public

    Low-latency tick-to-trade pipeline integrating a lock-free MPSC queue, io_uring async logger, UDP/ITCH feed handler, and options matching engine β€” plus an inventory-aware market maker on top. Compo…

    C++

  2. mpsc-queue mpsc-queue Public

    Vyukov MPSC queue in C++20 β€” beats mutex/spinlock/boost::lockfree::queue by 3.7-4.3x on real hardware (median of 5 runs), formal memory-ordering proof, 18 TSan litmus tests, sustained-contention be…

    C++ 5

  3. options-engine options-engine Public

    C++20 options matching engine: AVX2 SIMD 2Γ— speedup, SoA LOB, pool allocator, PROFILING.md β€” p50=28ns, 6/6 tests

    C++ 2 1

  4. io-uring-queue io-uring-queue Public

    SPSCRingBuffer p50=12ns + IOURingLogger via io_uring (548ns vs write() 500–5000ns) β€” C++20 lock-free queue and async logger, 375/375 tests

    C++ 2

  5. udp-multicast-receiver udp-multicast-receiver Public

    MoldUDP64/ITCH 5.0 multicast feed handler β€” SO_TIMESTAMPING HW+SW, recvmmsg batch (64 datagrams/syscall), 3/3 tests

    C++ 1

  6. options-market-maker options-market-maker Public

    Options market maker: Heston Gil-Pelaez FFT, SSVI calibration, vanna-volga hedging β€” Sharpe 2.26, 89 tests

    Python 1