fix(executor): warn loudly when a conditional entry price is overridden by market routing (#260) - #332
Conversation
…en by market routing (#260)
…ask-vs-mid wording, sign-convention cross-reference, honest noise claim
|
Review round applied: blocker — |
…ce pipeline (#346) Version bump across all six distributions. Minor: a feature set since v0.8.1, not a patch — per-product backtest slippage (#334), the entry-override routing warning (#332), pooled promotion counting (#338), the hourly paper profile (#337), and rail-17 visibility plus the rules enable verb (#340).
…-confirm ordering Two review findings on #350's money-path arms, both test-only, plus one docstring enumeration: - The extreme-exponent fail-closed arm of `_entry_spread_gate` (the `except ArithmeticError` whose spread is uncomputable) had no test. Pinned both ways a readable-sides book can still break the arithmetic: an `Overflow` on `bid + ask` at `1E+999999999` magnitudes, and a `DivisionByZero` from a subnormal pair whose mid half-even rounds to zero while the difference survives nonzero. #332's warning swallows these (telemetry); the gate must refuse with `book_unreadable` -- each case verified to fail under a swallow-like-telemetry mutation before landing. - Every gate test ran autonomous mode, so nothing pinned that the gate sits BEFORE the confirm gate. A wide-book BUY in `mode="confirm"` with an approving confirm_fn is still refused and the approver is never consulted -- a human cannot approve around the gate. Verified to fail under a confirm-first reordering mutation. - `summarise_cycle`'s docstring enumerated two non-placement causes (rails, confirm gate); added the third honestly -- the routing-time entry-spread gate refusing a live BUY. Docstring only, no behavior change.
* feat(executor): routing-time max-spread gate for live BUY entries Fixes #350. WHAT. A live BUY whose previewed book shows `(best_ask - best_bid) / mid` at or beyond `execution.max_entry_spread_pct` (new config section, default 0.005 = 50bp) is REFUSED after the preview and before the confirm gate and placement -- so a thin book cannot be entered at a moment its spread alone makes the fill economics materially worse than the cost model assumes. The refusal is recorded in `ExecutionResult.vetoed_by` (the tokens `max_entry_spread` / `book_unreadable`, the same one-legible-token shape rail violations use) and logged at WARNING as a structured event carrying the measured spread, the threshold and the product. WHY. The issue is the rail-work agreement CONTRIBUTING requires; the operator proposed it in the Phase 10 expansion review. It is the live-path half of the spread guardrail whose sizing half (#358) caps every Tier-2 addition at a 2% target weight. Design decisions: - Post-preview placement, deliberately: `guards.check` is broker-less by design, and the book exists only in the `broker.preview_order` result -- so this is a routing-time gate BESIDE the eighteen rails, not a numbered `guards.check` rail. It consumes the SAME preview #332's `_warn_if_market_routing_overrides_entry` reads: one helper (`_preview_book`, bid/ask/mid-safe: missing keys, NaN, non-finite, non-positive all read as unreadable), two consumers. The warning keeps its exact behavior and position (its tests pass unchanged); the gate runs after it, before confirm/place. - BUY-only: exits, brackets, stop rolls and scale-outs are never gated -- the same principle that makes rail 17 halt entries, not exits. - Paper mode never runs the gate: `_paper_enter` fills synthetically without a preview, so the paper-hourly profile accrues NO evidence about it -- a reason this ships before any live resumption, not after paper expansion. - Fail-closed on an unreadable book: a live BUY whose preview carries no readable bid AND ask is refused with the distinct `book_unreadable` reason and logged loudly. "Cannot know" is a different fact from "too wide"; the real venue's preview carries both sides (`cb_client.preview_order` maps best_bid/best_ask to Decimal), so an unreadable book means a degraded response -- the moment not to spend. The spread arithmetic also refuses (rather than swallows, as #336 taught the warning to do) on extreme-exponent Overflow. - 50bp default anchored to #334's `SLIPPAGE_CAP_PCT`: the backtest never assumes more than 50bp per-leg slippage on even the thinnest book, so a spread AT the cap has consumed the model's entire worst-case cost and the taker fee rides outside it -- hence the boundary is >= (fail-closed), unlike #332's strictly-greater visibility threshold. Validated on load to (0, 0.10], ConfigError naming `execution.max_entry_spread_pct`. Test-side consequence, honestly reported: the shared test fakes' bookless default previews modelled a shape the real venue does not return, and under a fail-closed gate every such "normal successful BUY" test would refuse. The fakes (test_executor, test_agent, test_cli, test_reconcile) now carry both book sides, and the one #332 test that borrowed the default preview as its degraded/bookless shape constructs that shape explicitly -- meaning unchanged, sourcing changed. Golden config fixtures regenerated via the documented script; the defaults golden now pins the 0.005. * test(executor): pin the spread gate's fail-closed arm and gate-before-confirm ordering Two review findings on #350's money-path arms, both test-only, plus one docstring enumeration: - The extreme-exponent fail-closed arm of `_entry_spread_gate` (the `except ArithmeticError` whose spread is uncomputable) had no test. Pinned both ways a readable-sides book can still break the arithmetic: an `Overflow` on `bid + ask` at `1E+999999999` magnitudes, and a `DivisionByZero` from a subnormal pair whose mid half-even rounds to zero while the difference survives nonzero. #332's warning swallows these (telemetry); the gate must refuse with `book_unreadable` -- each case verified to fail under a swallow-like-telemetry mutation before landing. - Every gate test ran autonomous mode, so nothing pinned that the gate sits BEFORE the confirm gate. A wide-book BUY in `mode="confirm"` with an approving confirm_fn is still refused and the approver is never consulted -- a human cannot approve around the gate. Verified to fail under a confirm-first reordering mutation. - `summarise_cycle`'s docstring enumerated two non-placement causes (rails, confirm gate); added the third honestly -- the routing-time entry-spread gate refusing a live BUY. Docstring only, no behavior change.
…r, conformance green (#382) Phase A of the keel-broker-alpaca PRD (#369): `packages/keel-broker-alpaca`, an original implementation of the `keel-broker-api` port against Alpaca's publicly documented Trading + Market Data REST APIs (no `alpaca-py`, no third-party adapter code — raw REST over an injected `requests` transport, the keel-broker-robinhood convention). Zero changes under `keel/`. FR-by-FR coverage: - FR-1 Package: workspace-pinned pyproject (`keel-core==0.9.3`, `keel-broker-api==0.9.3`), `py.typed`, registered under `keel.brokers` as `alpaca`, wired into the dev group (optional venue, like robinhood), mypy strict from birth, Dependabot + the pip-audit export exclusion. - FR-2 Venue identity: `venue = "alpaca"`, USD quotes, `asset_classes = {"equity"}` (the port's keel-side vocabulary). - FR-3 Orders: all four port kinds — `market_ioc_quote` maps to Alpaca's notional market order (`notional` + `type: market` + `time_in_force: day`, as the docs require), `market_ioc_base` to fractional `qty` market, `limit_gtc`/`stop_limit_gtc` to `gtc` limit/stop-limit. Fractional sizes render positionally (`format(d, "f")`), never scientific notation. `extended_hours: false` pinned on every body. - FR-4 Preview: Alpaca has no preview endpoint, so the preview is SYNTHESIZED — the keel-broker-robinhood precedent for venues without one (Robinhood's `synthesizes_preview=True` / `supports_native_preview=False` declaration, its "no endpoint that validates the order is a quote" reasoning). The book read is `GET /v2/stocks/{sym}/quotes/latest`; `best_bid`/`best_ask` ride in `Preview.detail` for the #332 warning and the #350 spread gate, and pricing uses only the crossed side (ask for buys, bid for sells). `ap`/`bp` = 0 is the documented "no active side" and lands in `Preview.errors`, never as a price. - FR-5 Market data: `15Min`/`1Hour`/`1Day` → FIFTEEN_MINUTE/ONE_HOUR/ONE_DAY, all other granularities refused with `ValueError` (the port's sanctioned refusal). Pagination follows `next_page_token` (bounded, like the sibling transports). The data tier is a DECLARED capability: constructor-validated `iex|sip`, sent on every data request — never the venue's silent `sip` default. - FR-6 Balances/positions: USD row with `available = min(buying_power, cash)` — on a cash account (`multiplier == 1`) the gap between the two is exactly the unsettled T+1 proceeds, surfaced honestly; long positions map `qty_available` → available, `qty` → total; short rows are skipped (long-only by construction). - FR-7 Fees: commission $0; sells modelled with the regulatory pass-throughs in `fees.py` as provenance-commented constants — SEC Section 31 $22.90/$1M (Alpaca's own regulatory-fees page; SEC advisory 2026-2 moves it to $20.60/$1M as of 2026-04-04 — recorded as a re-measurement point), FINRA TAF $0.000166/share capped $8.30 (FINRA Schedule A §4(b)(7)). The model feeds `Preview.est_fee` on sells; buys are honestly zero. - FR-8 Conformance: the shared suite (`keel_broker_api.conformance.suite.BrokerConformanceTests`) runs against the adapter via `tests/conformance/test_alpaca_conformance.py`, wired the same way the coinbase/robinhood/fake suites are (fixture-backed `FakeTransport`). Green. - FR-11 Rate limits/hosts: 429 retried honoring `Retry-After` when sent, exponential backoff otherwise, bounded attempt budget, then a typed `AlpacaAPIError`. Paper/live hosts come from an endpoint enum (`TRADING_HOSTS`) — there is no URL-shaped parameter for the trading host, so a paper configuration cannot reach the live host; tested structurally. Declared capability gaps (also in the package README): - Bracket/OCO and stop-market are NOT declared: the port's `OrderSpec` has no bracket concept and no stop-market kind, and this adapter does not invent venue-side order kinds the engine cannot ask for. `MarketOnOpen`/`MarketOnClose` likewise (available at the venue, unused, unexpressible in the port). - `supports_fee_summary` is false: the Trading API publishes no fee tiers, no fees-paid total, no volume window — a fabricated `FeeSummary` would read as coverage (the #197 lesson). - FR-10 corporate actions: bars are requested split-adjusted (`transport.BAR_ADJUSTMENT = "split"` so a cached series can state its policy); announcement consumption and dividend-purification recording are Phase B. - FR-9 session awareness is wired ahead of the rails: `is_market_open()` reads the venue's `/v2/clock` (no local calendar); extended hours are off by posture. Also fixes a pre-existing red on main introduced by #380: the README trademark sentence "not affiliated with, endorsed by, or sponsored by" trips `test_no_document_claims_a_review_has_occurred`'s naive "endorsed by" substring scan (verified failing on a clean 768585d tree). Reworded to "no affiliation with / no endorsement from / no sponsorship from" — the disclaimer is unchanged in strength, the trust-scanner stays strict. Fixes #369
…renderer is gone (#524) (#569) The flip. The executor placed orders by handing hand-built Coinbase dicts to a pre-port client; it now builds `OrderSpec` values and reads `Preview`/ `PlaceResult`. **The bytes on the wire are unchanged, and that is verified rather than asserted** -- ten configurations rendered by the new path were compared against the PRE-FLIP renderers loaded out of git, and are byte-identical. ── ONE RENDERER, NOT TWO ────────────────────────────────────────────────────── `CoinbaseClient.preview_order`/`place_order` take a spec and render it through `keel_broker_coinbase.translate.to_order_configuration` -- the adapter's own function. `executor._bracket_order_configuration` is deleted, and with it the test #502 stage 1 shipped to pin the two byte-identical while both existed. That test's own words: "The test imports both; production code does not." There is one now, so there is nothing left to hold in agreement. `_order_configuration` becomes `_order_spec`: BUY is `MarketIOCByQuote`, SELL is `MarketIOCByBase`, the bracket is `BracketGTC`. #516's quantization is untouched and still runs before the spec is built, including its deliberate BUY/SELL asymmetry. ── THE TRAP I NEARLY WALKED INTO ────────────────────────────────────────────── Every `OrderSpec` carries an `initial_status` ClassVar, and using it for the order row's status is the obvious move and WRONG. The port's vocabulary is the venue's (`filled_or_rejected`, `open`); this column is keel's (`filled`, `pending`). `reconcile` sweeps for `pending`, so writing `open` would leave every resting order invisible to the sweep that exists to observe its fill -- a bracket recorded as `open` is a protective order keel would never look at again. `_initial_status` therefore stays, mapping `spec.kind` to KEEL's words. What went is the dict inspection (`next(iter(order_configuration), "")`), not the vocabulary. Caught by a test asserting `'filled'`, which is exactly what that test was for. ── SMALLER THINGS THE TYPES MADE OBVIOUS ────────────────────────────────────── `raw_response` stored the whole placement response as JSON so that `_native_order_id` could dig the id out later to cancel with. It stores `{"order_id": ...}` now, from `PlaceResult.broker_order_id`. No migration: both shapes answer the same `data.get("order_id")`, so old rows read unchanged. `_preview_book` already accepted `Preview | dict` and the Coinbase adapter already carried the book in `detail` -- the executor was written anticipating this -- so #350's spread gate and #332's override warning came through untouched. ── TESTS ────────────────────────────────────────────────────────────────────── ~13 fakes across five files moved to the port's signatures. The dict-shaped preview payloads are kept AS dicts at the call sites and converted by one helper: dozens of tests build a bespoke preview to exercise one degraded field, and rewriting each into a constructor would have been a bigger diff than the change it accompanies, with more chances to alter a case by accident. Gates: 4301 passed / 3 skipped, ruff clean, mypy clean across 347 files, one paper cycle run against real venue data. ── WHAT IS LEFT OF #524 ─────────────────────────────────────────────────────── `_build_broker` still constructs `CoinbaseClient` rather than resolving through `load_broker`. That is now a SMALL change -- the client and every adapter speak the same interface -- gated on two consumers that are not port methods: `assets discover`'s `list_products`, and `assets holdings`' `get_accounts`. Claude-Session: https://claude.ai/code/session_01T6yA5khYnJ2qzheArRToQ2 Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
What & why
Closes #260's minimum viable mitigation (the issue's own scope; full resting-order routing stays deferred).
The live executor records every rule's
Setup.entryasexpected_filland then ignores it for execution — all entries routemarket_market_ioc(_order_row/_order_configuration). For enter-at-close rules that is nearly free; forpullback_continuation, whoseentry = signal_candle.high + buffer_ticksdeliberately demands follow-through, production silently takes trades the rule meant to decline. The faithful measurement (#258) quantified it across 24 assets: median trade count 58 -> 124 (more than doubled), median gross PF 0.9219 -> 0.7736. The doubling is the count of trades the live box would take that the rule intended to decline, and the PF collapse is their quality.Fixing it means changing money-moving order routing to rescue a strategy that is independently measured dead — "Upgrading live execution to rescue a dead strategy is a bad trade" (#260). The landmine is not
pullback_continuationbut the next price-conditional rule, which would be silently mis-executed the same way. So this PR makes the override visible rather than silent — the same principle as #247 printing the fee rate:keel/execution/executor.pygainsENTRY_OVERRIDE_WARN_BP(50bp, documented below) and_warn_if_market_routing_overrides_entry, called from_run_orderright after the preview, before the confirm gate.best_askout of the preview_run_orderalready fetches — the price a market BUY actually pays, from the one book quote already in the hot path (no new broker call; a mid would understate the deviation by half the spread).CoinbaseClient.preview_ordermapsbest_bid/best_asktoDecimaltoday, and the Coinbase port adapter carries the same book inPreview.detail, so both preview shapes are read.ENTRY_OVERRIDE_WARN_BP = Decimal("50")— a VISIBILITY threshold, not a correctness one. Anchored in the repo's own cost model (1.2% taker per leg, 5bp slippage): a few bp is the microstructure drift any enter-at-close rule accumulates by routing one cycle late; tens of bp is a rule whose entry encodes a condition. 50bp is 10x the slippage assumption (noise never trips it) yet small enough that any deliberate entry condition does. Comparison is strictly greater — exactly at the line logs nothing.executor.entry_override_market_routed) carrying rule kind, product, intended entry (expected_fill), market reference and its source, signed deviation in bp, the threshold, and an explicitdetailsentence: rendered —{"level": "WARNING", "logger": "keel.execution.executor", "event": "executor.entry_override_market_routed", "rule": "pullback_continuation", "product": "BTC-USD", "expected_fill": "50300", "market_ref": "50000", "market_ref_source": "preview_best_ask", "deviation_bps": "60.00", "threshold_bps": "50.00", "detail": "the rule's conditional entry price was OVERRIDDEN -- entries are always routed as market orders (#258), so the condition this rule encoded in its entry price was bypassed and the order is going out at the venue's price instead (#260)"}Scoped to BUYs on the market configuration only: SELL intents (exits, brackets, stop rolls) carry their prices to the venue verbatim, and a future caller passing a resting
order_configurationis not on the override path. A preview with no usable book quote is silent, not fatal._order_rowand the module docstring also document the always-market decision (#258) and why resting orders are deferred (#260).Tests-first evidence
Tests written first in
tests/execution/test_executor.py::TestEntryOverrideWarningAtRouting(extending the existingTestIntentDivergenceLoghouse pattern), seen red:The end-to-end routing test (through
execute(), no private imports) failed on the assertion meant to assert — the full guard->preview->place path ran clean and no warning fired. The other 7 red on importing the then-nonexistent helper, then went green with the implementation. Cover: beyond threshold via the full routing path (rule kind + both prices + signed bp + WARNING level + the OVERRIDDEN sentence), within threshold silent (a warning that fires every order is a warning nobody reads), exactly at the threshold silent (boundary pinned from the constant), entry below market warns with negative sign, bookless/garbage preview silent and non-fatal, the portPreviewshape, SELL intents never warn, non-market configurations never warn.Gates (all must pass)
uv run ruff checkclean —All checks passed!uv run mypyclean —Success: no issues found in 237 source filesuv run pytest -qgreen —2870 passed, 1 skipped in 31.34sScope check
keel/execution/executor.py,tests/execution/test_executor.py).