From 803f44ac0b2f8f58a396f1c26558627c03d5f375 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Sun, 16 Aug 2026 19:11:15 -0400 Subject: [PATCH] docs(readme): rewrite for the newcomer, led by the compliance engine The README was an operator runbook; a stranger deciding whether to spend an afternoon here needs what is differentiated (19 rails, fails-closed attested screening, qabd as an executable check), the honest measured result stated by us first, the not-a-fatwa boundary, and a five-minute path that actually runs (#281). The operator content moved to docs/operator-runbook.md unchanged. Quickstart steps verified on a clean clone, including the AuthenticationError keel fetch raises without a CDP key -- stated rather than discovered at step four. tests/test_readme.py pins the structure so the README cannot quietly grow back into a runbook. --- README.md | 373 +++++++++++++-------------------------- docs/operator-runbook.md | 182 +++++++++++++++++++ tests/test_readme.py | 140 +++++++++++++++ 3 files changed, 449 insertions(+), 246 deletions(-) create mode 100644 tests/test_readme.py diff --git a/README.md b/README.md index 51796c7b..e1da26ca 100644 --- a/README.md +++ b/README.md @@ -1,8 +1,18 @@ # keel -An offline-first, halal (long-only, no-leverage) auto-trading agent for Coinbase. See -`docs/superpowers/specs/2026-07-15-keel-autotrade-design.md` for the full design and -`docs/superpowers/plans/2026-07-15-keel-phase1-offline-foundation.md` for the Phase 1 build plan. +**keel is an auditable Shariah-compliance engine for spot crypto trading** — deterministic +safety rails, attested asset screening that fails closed, and §65.4 *qabd* (constructive +possession) encoded as an executable check — with a reference auto-trading agent built on top +of it for Coinbase. Plenty of people have a trading bot; almost nobody has this compliance +machinery, which is the part worth reading. + +**The honest result, stated by us first:** no shipped rule family is net-positive at the +taker fee actually paid on this venue — cost is the binding constraint, and the viable +parameter/fee intersection is empty under production-faithful execution +([the experiment record](docs/experiments/2026-08-13-restated-under-a-production-faithful-engine.md)). +The point of this project is the enforcement machinery and the honest measurement of what +runs through it, not a claim of alpha. A visitor who finds that out themselves feels misled; +one who is told upfront can read it as rigour. > **keel is not a fatwa engine. It is an enforcement engine for a ruling you supply.** keel > never derives a Shariah classification from market data. You record one — with a source and an @@ -11,258 +21,129 @@ An offline-first, halal (long-only, no-leverage) auto-trading agent for Coinbase > following different schools get different answers from the same code, by design. See > `CONTRIBUTING.md` for what that means for pull requests. -## How keel works - -keel runs as a scheduled **agent loop** (`keel agent`). Each cycle, for every allowlisted -product, it: - -1. **Polls** fresh candles from Coinbase (public market data). -2. Asks each **`live` rule** to `detect()` a setup on that product's candles. -3. Sends any resulting signal through the **rails** — un-overridable safety checks in - `keel/execution/guards.py`. -4. **Previews** the order with the broker (the broker's own numbers, not an estimate). -5. Applies the **confirm / autonomy gate** (below). -6. Calls `place_order`, and **logs** before and after. - -There is deliberately **no manual "place an order" command** — every order is the output of a -rule that cleared the rails. keel never takes a discretionary trade: its judgement is -deterministic, backtested rules plus the rails, never a prediction. - -### Rules - -Orders come from four rule kinds (`keel/agent.py::RULE_REGISTRY`). A rule must be walked -`candidate → paper → live` before it can trade (`keel rules list|promote|demote|disable`): - -- **`dca`** — scheduled dip-buy accumulation: a fixed-budget market buy on a **calendar cadence** - (e.g. weekly), no stop. The only rule that fires on a schedule you control — which is why it's - the natural vehicle for a first live-order test (see `docs/go-live-runbook.md`). -- **`turtle_breakout`**, **`pullback_continuation`**, **`rsi_meanrev`** — risk-defined entries - (with a stop and target) that fire only on a **real market setup** — a breakout, a pullback, an - RSI extreme — so their timing isn't something you can summon on demand. - -### The rails (un-overridable) - -Every order — including in autonomous mode — must clear the guards in -`keel/execution/guards.py`: the **halal allowlist**, per-order and per-day spend caps, a -total-exposure cap and per-asset concentration cap, correlation-aware sizing, a minimum-move -floor, **no-martingale / no-stop-widening**, the **kill-switch** (fails closed — an unreadable -state halts trading), **total & weekly drawdown breakers**, a consecutive-loss / edge-decay -breaker, feed-staleness and quote-balance checks, and venue **subscription / withdrawal -attestations**. A rail veto names itself and the command that clears it. Nothing overrides a -rail — not even autonomy. - -### Confirm vs. autonomy - -- **confirm** (default): keel previews each order and asks `Place this order? [y/N]` at a - terminal. Run headless, it declines — nothing is placed without a human `y`. -- **autonomy on**: keel places without asking. It changes **who is asked, never what is - allowed** — every rail still runs first, and autonomy can never clear a safety halt. Prefer a - time-boxed session: `keel autonomy on --for-hours N`. To stop trading immediately, use - `keel kill`, not `keel autonomy off`. - -### How much money moves - -Four settings decide position size and how much can be spent. Three live in `config.yaml`; the -fourth does not, which is most of why they drift apart. - -- **`paper.starting_equity_usd`** — the synthetic paper account's seed. **It is a ONE-TIME seed, - applied on the FIRST paper run only** (`keel/agent.py`, the `paper_trader.get_cash() is None` - branch). Editing it afterwards does nothing at all: the seeding branch is skipped whenever - `paper_cash_usdc` is already set, so an already-seeded account keeps its balance forever. - Resizing a running paper account means clearing that persisted `paper_cash_usdc` — a key in the - `agent_state` table, with **no command that clears it** (`keel reset-hwm` does not); a fresh - database is the clean way. `0` (the default) means "seed from real mark-to-market equity - instead"; any value above `0` overrides that and seeds at exactly that amount. -- **`paper.monthly_contribution_usd`** — a recurring top-up, applied once per UTC calendar month. - It compounds, and the base is small: a contribution comparable to the seed doubles the account - monthly, and every position size below grows with it. -- **`caps.max_exposure_usd`** — has **two jobs at once**. It is the ceiling on total notional held - at any one moment (rail 4, and rail 6's concentration cap is a percentage of it), *and* it is - the **equity proxy that sizes orders** on the live path - (`keel/execution/executor.py::_build_intent`). So live `risk_pct` is a fraction of THIS number, - not of real account equity — raising the cap raises the real dollars risked per trade. Set above - actual equity it stops binding before available cash does, and the refusal comes later and less - legibly from the funding check (rail 13). In paper mode the proxy is bypassed: sizing uses the - paper account's own equity. -- **rail 14's monthly allowance** — the fee-free monthly BUY volume. It lives in the **database, - not `config.yaml`**: the `broker_subscriptions` row written by `keel subscription attest --venue - coinbase --tier `, or set directly with `keel subscription set --free-volume-usd N`. - `config.yaml` only supplies the tier catalogue and the unattested fallback - (`subscription.unsubscribed_allowance_usd`). Being in a different place from the caps is exactly - why it drifts out of step with them. - -**The interaction is the point.** Position sizing scales with equity (or, on the live path, with -the `max_exposure_usd` proxy); the rail-14 allowance is a fixed dollar figure that scales with -nothing. Let the two drift apart and *every* setup is vetoed — keel looks broken while every -component is doing exactly what it was configured to do. - -The real case: at **$11,000** paper equity with `risk_pct: 0.01`, a PAXG setup with a 3.35%-wide -stop sized to **$3,284.67** — exactly 1% of equity ($110) at risk, the correct answer. Rail 14's -allowance was **$500/month**, so it was vetoed, as was every other setup. Not a bug in either -setting; the two were simply on different scales. Reseeding the paper account at $500 sizes the -same setup at **$149.30**, which fits. - -Note the counter-intuitive mechanic behind those numbers: **a tighter stop produces a LARGER -position**, because `size = risk ÷ stop-distance` (`keel/execution/sizing.py::size`). That is how -a 1% risk becomes a **30% position** — `risk_pct` bounds what you lose if the stop holds, not what -you spend. - -### Halal by construction, and ships inert - -Long-only spot only — no leverage, shorting, or derivatives; sizing uses actual cash, so no -riba. keel ships **inert**: nothing trades until you promote a rule to `live`, attest the venue -subscription (rail 14 refuses live BUYs otherwise), fund the account, and — in confirm mode — -type `y`. See `docs/go-live-runbook.md` for the first supervised order and -`docs/operator-runbook.md` for the account-level obligations no rail can enforce. - -## Development - -```bash -uv sync # install deps (Python 3.14) -uv run pytest # run tests -uv run ruff check # lint -``` - -Copy `.env.example` to `.env` and fill in your Coinbase Developer Platform (CDP) API key/secret -for any live (network) commands. `.env` is git-ignored and never committed. Offline commands -(config loading, analysis, backtests on imported CSVs) work without it. - -Runtime settings (allowlist, target weights, risk caps, market data granularities, etc.) live in -`config.yaml` at the repo root — see `keel/config.py` for the schema and validation rules. +keel is a personal tool, not financial advice and not religious (Shariah) advice — see the +disclaimers below. -## Deploying a new version +## Try it in five minutes -Cutting a release is `docs/RELEASING.md`. Installing one into a deployment (e.g. `~/keel`) is four -commands, run **from the deployment directory** — every path below is relative to it: +Everything here is read-only and paper-side: no funds, and nothing in this path can place an +order. Verified on a clean clone. You need [uv](https://docs.astral.sh/uv/) and a **free, +read-only Coinbase Developer Platform (CDP) API key** — candle history is fetched through the +authenticated client, so `keel fetch` without a key fails with an `AuthenticationError`; say +so upfront rather than let step 4 be a surprise. ```bash -V=0.6.0 -gh release download "v$V" --repo CodeGateSoftware/keel --pattern '*.whl' --dir Release/ -uv pip install --python .venv --find-links Release \ - Release/keel_core-$V-py3-none-any.whl \ - Release/keel_broker_api-$V-py3-none-any.whl \ - Release/keel_broker_coinbase-$V-py3-none-any.whl \ - Release/keel_trader-$V-py3-none-any.whl -.venv/bin/keel versions -.venv/bin/keel status +git clone https://github.com/CodeGateSoftware/keel.git && cd keel +uv sync --all-extras --dev # Python 3.14+ (uv python install 3.14) +cp .env.example .env # put the CDP key/secret in it — market data only +uv run keel rules seed # register the rule families as candidates +uv run keel fetch # pull candle history for the default allowlist +uv run keel simulate --years 1 --skip-within-cap ``` -Set `V` to the version being deployed; nothing else changes between releases. +`keel simulate` replays the real rules deterministically over the fetched history, compares +against a DCA benchmark, and writes a GO-LIVE/TRAIN-MORE report with the gates and their +numbers. On the default rules it will very likely tell you **TRAIN MORE** and name the gates +that fail — that is the engine working, not broken; the honesty is the feature. The next +steps from there — promoting a rule through the gate (`keel rules promote`, which refuses to +promote without an overfitting check), running the paper agent (`keel agent`, paper mode is +the default), and eventually a supervised first live order — are in +[`docs/go-live-runbook.md`](docs/go-live-runbook.md). -**Every wheel is named, and that is the fix for a real bug.** Installing `keel_trader` alone -upgraded *only* `keel_trader`: its siblings were required without a version, so the `keel-core` -already on disk satisfied `keel-core` and stayed put. `~/keel` ran `keel-trader 0.5.7` against -`keel-core 0.5.5` for two releases that way. A wheel **path** is a direct requirement — that exact -file is installed whatever is already there — so naming all four is what actually moves them. -The wheels now also pin their siblings exactly (`Requires-Dist: keel-core==0.6.0`), which forces -the upgrade even for someone who installs `keel_trader` alone; the four paths are the same -guarantee stated where the operator can see it. - -**Not `Release/*.whl`.** The release ships *every* workspace wheel, two of which a deployment must -not have: `keel_broker_fake`, a dev-only fake venue that registers a `fake` entry point under -`keel.brokers`, and `keel_broker_robinhood`, an optional venue that pulls an Ed25519 stack -(`pynacl`, `cffi`) in for an adapter nothing constructs. The four named wheels are production's -whole dependency closure. `--find-links Release` still points at that directory so the pinned -siblings resolve locally rather than from PyPI, where they do not exist — which is why step 1 -downloads them all. Installing **by path** rather than by bare name is deliberate and unchanged: -`keel` on PyPI is an unrelated project, so `pip install keel` fetches a stranger's code (see -`keel/version.py`). +## How keel works -Step 3 is the check that matters, and it is `keel versions` — **not** `keel --version`, which -could not fail. `--version` reports the `keel-trader` distribution's version and nothing else, so -it printed `0.6.0` while `keel-core` sat at `0.5.5`: a verification step blind to the failure mode, -which is worse than none, because it is trusted. `keel versions` prints the same build identity, -then every keel distribution in that venv, and **exits non-zero** when they disagree: +keel runs as a scheduled **agent loop** (`keel agent`). Each cycle, for every allowlisted +product, it: polls fresh candles, asks each active **rule** to `detect()` a setup, sends any +signal through the **rails**, previews the order with the broker (the broker's own numbers), +applies the confirm/autonomy gate, then places and logs. There is deliberately no manual +"place an order" command — every order is the output of a rule that cleared the rails. + +- **Rules** (`keel/agent.py::RULE_REGISTRY`) — four families: `dca` (scheduled accumulation, + no stop) and three risk-defined entry patterns (`turtle_breakout`, + `pullback_continuation`, `rsi_meanrev`). A rule must walk `candidate → paper → live`, and + promotion clears a two-part gate: performance floors *and* an overfitting check + (PBO/CSCV). A rule that clears four floors on one in-sample parameter set is exactly what + the second gate exists to be suspicious of. +- **The rails** (`keel/execution/guards.py`) — nineteen deterministic checks no order can + skip and nothing can override, not even autonomy: the halal allowlist, per-order and + per-day spend caps, exposure and concentration caps, correlation-aware sizing, a + minimum-move floor, no-martingale/no-stop-widening, a fails-closed kill-switch, total and + weekly drawdown breakers, a consecutive-loss/edge-decay breaker, feed-staleness and + quote-balance checks, venue **subscription/withdrawal attestations** — rail 17 encodes + §65.4 *qabd*: an asset that cannot be withdrawn may not have been validly possessed, so + withdrawal capability is attested and enforced, not assumed. A rail veto names itself and + the command that clears it. +- **Screening** (`keel/compliance/screen.py`) — allowlist admission is split by what is + knowable: market facts are computed; Shariah classifications are **attested, never + inferred**, and an absent attestation is a rejection, not a default pass. +- **Confirm vs. autonomy** — by default keel previews each order and asks at the terminal; + headless, it declines. `keel autonomy on` changes **who is asked, never what is allowed**. + To stop trading, `keel kill` — the kill-switch fails closed. + +keel ships **inert**: nothing trades until you promote a rule, attest the venue subscription +(rail 14 refuses live BUYs otherwise), fund the account, and — in confirm mode — type `y`. +Long-only spot only: no leverage, no shorting, no derivatives, and sizing uses actual cash, +so no riba. Account-level obligations no rail can see (disabling USDC rewards on idle +balances, chiefly) are the operator's to verify — +[`docs/operator-runbook.md`](docs/operator-runbook.md) lists them. + +One mechanic worth knowing before any number surprises you: **a tighter stop produces a +LARGER position**, because `size = risk ÷ stop-distance`. `risk_pct` bounds what you lose if +the stop holds, not what you spend. + +## Architecture ``` -keel 0.6.0+deb8fa7e978d [release] - -keel-broker-api 0.6.0 -keel-broker-coinbase 0.6.0 -keel-core 0.6.0 -keel-trader 0.6.0 - -ok: 4 keel distributions, all at 0.6.0. +keel/ the agent and CLI +├── agent.py the loop and RULE_REGISTRY (where rules live) +├── execution/guards.py the 19 rails (where enforcement lives) +├── execution/sizing.py position sizing +├── compliance/screen.py attested allowlist admission (fails closed) +└── commands/ CLI command implementations + +packages/ +├── keel-core/ shared domain types, config, logging +├── keel-broker-api/ the broker PORT: the contract every adapter codes against +├── keel-broker-coinbase/ Coinbase Advanced Trade adapter +├── keel-broker-robinhood/ Robinhood adapter (optional venue) +└── keel-broker-fake/ deliberately divergent fake venue, dev-only + +tests/ ~2,800 tests, including the port's conformance suite ``` -A partial upgrade fails it, with the numbers: `error: PARTIAL INSTALL: 4 keel distributions at 2 -different versions (0.5.5, 0.6.0)`. So does finding `keel-broker-fake` installed — it was, at -`0.5.5`, in `~/keel`. Remove it: `uv pip uninstall --python .venv keel-broker-fake`. Nothing calls -`load_broker()` today so it is inert, but that is a property of this release, not of the package, -and no reason to leave a fake venue registered on the box that moves money. - -A build reporting `(DIRTY)` or `[checkout]` corresponds to no commit and **must not be run against -live funds**. Step 4 is a read-only snapshot — no orders, no writes — confirming the new build -opens the database and reaches the venue. - -If the deployment runs on a schedule (LaunchAgents, cron), a new build takes effect on the next -cycle with nothing to restart — each cycle is a fresh process. A **long-running** process is the -exception: a `keel tui` left open keeps the build it started with until you quit and relaunch it. - -## Paper vs. live - -A deployment such as `~/keel` runs **two of them side by side**, and they share nothing: separate -configs, separate databases, separate allowlists, separate caps, separate schedules, separate -histories. **A figure from one says nothing about the other.** Checking a paper position size -against live account equity — or a live cap against paper cash — yields a confident wrong answer, -and has already produced one. Establish which account a number came from before reasoning about it. - -| | paper | live | -| --- | --- | --- | -| config | `config.paperforward.yaml` | `config.live-sandbox.yaml` | -| database | `keel.db` (the `--db` default) | `keel-live.db` (must be passed) | -| `auto_trade.mode` | `paper` | `confirm` | -| allowlist | BTC, ETH, PAXG, SOL, XLM, LTC, ADA, LINK (8) | BTC, ETH, PAXG, ADA, XLM (5) | -| `caps.max_exposure_usd` | 5000 | 200 | -| money spent | synthetic `paper_cash_usdc` | the real broker balance | -| sizing basis | the paper account's own equity | `caps.max_exposure_usd`, as a proxy | -| rail 14 allowance | $500/month (Basic tier) | $200/month | -| `equity_state_mode` | `paper` | `live` | -| launchd job | `com.keel.paperforward` | `com.keel.live` | - -**Which one am I looking at.** On any dashboard (`keel status`, `keel insights`, `keel tui`) the -`equity_state_mode` line names the account the equity, high-water mark and drawdown figures -describe, and `paper_cash_usdc` is printed in paper mode only. On the command line it is the -`--config`/`--db` pair — and `--db` is the one that bites, because `keel.db` is its default, so a -live command that omits it silently reads the **paper** database and answers about the wrong -account. Live commands always carry both: - -```bash -keel --config config.live-sandbox.yaml --db keel-live.db status -``` - -**Placing an order is gated differently.** Paper places freely against synthetic cash — nothing is -asked and nothing real moves, which is the point. Live runs `mode: confirm`: each order is -previewed and waits for a typed `y` at a terminal, so a headless live cycle **fails closed** and -places nothing — *unless autonomy is armed*, which is exactly what makes an unattended live cycle -place. Autonomy changes who is asked, never what is allowed (see **Confirm vs. autonomy** above); -check the flag before assuming a live cycle is supervised, rather than inferring it from `confirm`. - -**Both fire hourly; both run once a day.** Each launchd job has a list of hourly triggers plus -`RunAtLoad`, and each runner is day-stamped: the first eligible trigger that finds no stamp for -today runs the cycle and writes the stamp, and every later trigger that day is a no-op. The -trigger count is **catch-up breadth, not cadence** — launchd re-runs a calendar interval missed -while asleep but *not* one that passed while the machine was off, so the extra triggers are what -stop a shutdown over the scheduled hour from losing the day outright. A cycle that **fails** leaves -no stamp, so the next hour retries it, which also covers waking with no network. The two jobs -differ only in anchor: paper fires 09:00–20:00 local and stamps the local date; live fires hourly -at :20 and gates and stamps on the **UTC** date, because a daily bar is not visible until the -00:00–01:00 UTC hourly candle has closed. On live the stamp is a correctness mechanism, not tidiness -— nothing on that path dedupes an entry, so two cycles in one UTC day means two entries off one -daily bar (`tests/test_schedule.py` pins it). - -**Sizing is a different calculation on each.** Paper sizes off its own synthetic equity, passed to -`_build_intent` as `equity_override`; the live path has no equity reading there and falls back to -`caps.max_exposure_usd` as a proxy (`keel/execution/executor.py`). The same rule, the same setup and -the same day therefore produce different quantities on the two accounts, and neither is an estimate -of the other. The settings behind those numbers are covered under **How much money moves** above. - -## Before trading live - -Read `docs/operator-runbook.md`. It lists the compliance obligations **no rail can enforce** — chiefly -that **interest/rewards on idle balances must be disabled** (Coinbase pays USDC rewards on idle balances, so riba can accrue with no order placed). Every guard in `keel/execution/guards.py` -inspects an order, so account-level obligations are invisible to all of them and are yours to verify. - -Note keel ships **inert**: rail 14 refuses live BUYs until a subscription is attested with -`keel subscription attest --venue coinbase --tier `. That one is enforced in code and needs no -checklist. +A new broker plugs in as a package under `packages/keel-broker-*`, implementing the +`keel-broker-api` port and registering itself under the `keel.brokers` entry point — no +changes to `keel/` itself. The fake venue exists precisely to keep that port honest: a second +adapter, deliberately divergent, that the conformance suite runs against. + +## Documentation map + +- [`docs/operator-runbook.md`](docs/operator-runbook.md) — operating a deployment: the + account-level compliance obligations no rail can enforce, deploying/upgrading releases, + and the paper-vs-live distinctions (two accounts that share nothing). +- [`docs/go-live-runbook.md`](docs/go-live-runbook.md) — the first supervised live order. +- [`docs/experiments/`](docs/experiments) — the experiment record, including the honest + result linked above; every document states what was measured, on what engine, with the + defect that forced a restatement. +- [`docs/RELEASING.md`](docs/RELEASING.md) — how a release is cut. + +## Asking questions, and contributing + +- **Questions, ideas, and classification discussion** → + [Discussions](https://github.com/CodeGateSoftware/keel/discussions), including the + *Compliance & classification* category for "should X be treated this way" — which is a + question, not a bug, and must not be triaged as one. +- **Contributing** → [`CONTRIBUTING.md`](CONTRIBUTING.md) — the documentation standard (this + repo's bar is unusually high, and stated, with a worked example), the gates a PR must + pass, tests-first, and scope guidance. Newcomers: look for issues labelled + `good first issue`. +- **Behaviour** → [`CODE_OF_CONDUCT.md`](CODE_OF_CONDUCT.md). +- **Anything that could make keel misbehave** → [`SECURITY.md`](SECURITY.md), privately — + a rail that can be bypassed is a security issue, not merely a bug. + +## Disclaimers + +keel is a personal tool, not financial advice and not religious (Shariah) advice. Consult a +qualified financial advisor and a knowledgeable scholar before trading. You are solely +responsible for your own trading decisions. Licensed under +[Apache-2.0](LICENSE). diff --git a/docs/operator-runbook.md b/docs/operator-runbook.md index a1fd2686..ce20d00b 100644 --- a/docs/operator-runbook.md +++ b/docs/operator-runbook.md @@ -66,3 +66,185 @@ An item belongs here only if **all** of these hold: If the third stops being true, implement the check and delete the entry. If the second stops being true, it is a rail, and it belongs in `guards.py`. + +--- + +# Part 2 — Operating a deployment + +Moved here from the README when it was rewritten for newcomers (#281): everything below is +operator knowledge — deploying, upgrading, and keeping the paper and live accounts straight — +not something a first-time reader needs. It is unchanged in substance. + +## Deploying a new version + +Cutting a release is `docs/RELEASING.md`. Installing one into a deployment (e.g. `~/keel`) is four +commands, run **from the deployment directory** — every path below is relative to it: + +```bash +V=0.6.0 +gh release download "v$V" --repo CodeGateSoftware/keel --pattern '*.whl' --dir Release/ +uv pip install --python .venv --find-links Release \ + Release/keel_core-$V-py3-none-any.whl \ + Release/keel_broker_api-$V-py3-none-any.whl \ + Release/keel_broker_coinbase-$V-py3-none-any.whl \ + Release/keel_trader-$V-py3-none-any.whl +.venv/bin/keel versions +.venv/bin/keel status +``` + +Set `V` to the version being deployed; nothing else changes between releases. + +**Every wheel is named, and that is the fix for a real bug.** Installing `keel_trader` alone +upgraded *only* `keel_trader`: its siblings were required without a version, so the `keel-core` +already on disk satisfied `keel-core` and stayed put. `~/keel` ran `keel-trader 0.5.7` against +`keel-core 0.5.5` for two releases that way. A wheel **path** is a direct requirement — that exact +file is installed whatever is already there — so naming all four is what actually moves them. +The wheels now also pin their siblings exactly (`Requires-Dist: keel-core==0.6.0`), which forces +the upgrade even for someone who installs `keel_trader` alone; the four paths are the same +guarantee stated where the operator can see it. + +**Not `Release/*.whl`.** The release ships *every* workspace wheel, two of which a deployment must +not have: `keel_broker_fake`, a dev-only fake venue that registers a `fake` entry point under +`keel.brokers`, and `keel_broker_robinhood`, an optional venue that pulls an Ed25519 stack +(`pynacl`, `cffi`) in for an adapter nothing constructs. The four named wheels are production's +whole dependency closure. `--find-links Release` still points at that directory so the pinned +siblings resolve locally rather than from PyPI, where they do not exist — which is why step 1 +downloads them all. Installing **by path** rather than by bare name is deliberate and unchanged: +`keel` on PyPI is an unrelated project, so `pip install keel` fetches a stranger's code (see +`keel/version.py`). + +Step 3 is the check that matters, and it is `keel versions` — **not** `keel --version`, which +could not fail. `--version` reports the `keel-trader` distribution's version and nothing else, so +it printed `0.6.0` while `keel-core` sat at `0.5.5`: a verification step blind to the failure mode, +which is worse than none, because it is trusted. `keel versions` prints the same build identity, +then every keel distribution in that venv, and **exits non-zero** when they disagree: + +``` +keel 0.6.0+deb8fa7e978d [release] + +keel-broker-api 0.6.0 +keel-broker-coinbase 0.6.0 +keel-core 0.6.0 +keel-trader 0.6.0 + +ok: 4 keel distributions, all at 0.6.0. +``` + +A partial upgrade fails it, with the numbers: `error: PARTIAL INSTALL: 4 keel distributions at 2 +different versions (0.5.5, 0.6.0)`. So does finding `keel-broker-fake` installed — it was, at +`0.5.5`, in `~/keel`. Remove it: `uv pip uninstall --python .venv keel-broker-fake`. Nothing calls +`load_broker()` today so it is inert, but that is a property of this release, not of the package, +and no reason to leave a fake venue registered on the box that moves money. + +A build reporting `(DIRTY)` or `[checkout]` corresponds to no commit and **must not be run against +live funds**. Step 4 is a read-only snapshot — no orders, no writes — confirming the new build +opens the database and reaches the venue. + +If the deployment runs on a schedule (LaunchAgents, cron), a new build takes effect on the next +cycle with nothing to restart — each cycle is a fresh process. A **long-running** process is the +exception: a `keel tui` left open keeps the build it started with until you quit and relaunch it. + +## Paper vs. live + +A deployment such as `~/keel` runs **two of them side by side**, and they share nothing: separate +configs, separate databases, separate allowlists, separate caps, separate schedules, separate +histories. **A figure from one says nothing about the other.** Checking a paper position size +against live account equity — or a live cap against paper cash — yields a confident wrong answer, +and has already produced one. Establish which account a number came from before reasoning about it. + +| | paper | live | +| --- | --- | --- | +| config | `config.paperforward.yaml` | `config.live-sandbox.yaml` | +| database | `keel.db` (the `--db` default) | `keel-live.db` (must be passed) | +| `auto_trade.mode` | `paper` | `confirm` | +| allowlist | BTC, ETH, PAXG, SOL, XLM, LTC, ADA, LINK (8) | BTC, ETH, PAXG, ADA, XLM (5) | +| `caps.max_exposure_usd` | 5000 | 200 | +| money spent | synthetic `paper_cash_usdc` | the real broker balance | +| sizing basis | the paper account's own equity | `caps.max_exposure_usd`, as a proxy | +| rail 14 allowance | $500/month (Basic tier) | $200/month | +| `equity_state_mode` | `paper` | `live` | +| launchd job | `com.keel.paperforward` | `com.keel.live` | + +**Which one am I looking at.** On any dashboard (`keel status`, `keel insights`, `keel tui`) the +`equity_state_mode` line names the account the equity, high-water mark and drawdown figures +describe, and `paper_cash_usdc` is printed in paper mode only. On the command line it is the +`--config`/`--db` pair — and `--db` is the one that bites, because `keel.db` is its default, so a +live command that omits it silently reads the **paper** database and answers about the wrong +account. Live commands always carry both: + +```bash +keel --config config.live-sandbox.yaml --db keel-live.db status +``` + +**Placing an order is gated differently.** Paper places freely against synthetic cash — nothing is +asked and nothing real moves, which is the point. Live runs `mode: confirm`: each order is +previewed and waits for a typed `y` at a terminal, so a headless live cycle **fails closed** and +places nothing — *unless autonomy is armed*, which is exactly what makes an unattended live cycle +place. Autonomy changes who is asked, never what is allowed; check the flag before assuming a +live cycle is supervised, rather than inferring it from `confirm`. + +**Both fire hourly; both run once a day.** Each launchd job has a list of hourly triggers plus +`RunAtLoad`, and each runner is day-stamped: the first eligible trigger that finds no stamp for +today runs the cycle and writes the stamp, and every later trigger that day is a no-op. The +trigger count is **catch-up breadth, not cadence** — launchd re-runs a calendar interval missed +while asleep but *not* one that passed while the machine was off, so the extra triggers are what +stop a shutdown over the scheduled hour from losing the day outright. A cycle that **fails** leaves +no stamp, so the next hour retries it, which also covers waking with no network. The two jobs +differ only in anchor: paper fires 09:00–20:00 local and stamps the local date; live fires hourly +at :20 and gates and stamps on the **UTC** date, because a daily bar is not visible until the +00:00–01:00 UTC hourly candle has closed. On live the stamp is a correctness mechanism, not tidiness +— nothing on that path dedupes an entry, so two cycles in one UTC day means two entries off one +daily bar (`tests/test_schedule.py` pins it). + +**Sizing is a different calculation on each.** Paper sizes off its own synthetic equity, passed to +`_build_intent` as `equity_override`; the live path has no equity reading here and falls back to +`caps.max_exposure_usd` as a proxy (`keel/execution/executor.py`). The same rule, the same setup and +the same day therefore produce different quantities on the two accounts, and neither is an estimate +of the other. The settings behind those numbers are covered next. + +## How much money moves + +Four settings decide position size and how much can be spent. Three live in `config.yaml`; the +fourth does not, which is most of why they drift apart. + +- **`paper.starting_equity_usd`** — the synthetic paper account's seed. **It is a ONE-TIME seed, + applied on the FIRST paper run only** (`keel/agent.py`, the `paper_trader.get_cash() is None` + branch). Editing it afterwards does nothing at all: the seeding branch is skipped whenever + `paper_cash_usdc` is already set, so an already-seeded account keeps its balance forever. + Resizing a running paper account means clearing that persisted `paper_cash_usdc` — a key in the + `agent_state` table, with **no command that clears it** (`keel reset-hwm` does not); a fresh + database is the clean way. `0` (the default) means "seed from real mark-to-market equity + instead"; any value above `0` overrides that and seeds at exactly that amount. +- **`paper.monthly_contribution_usd`** — a recurring top-up, applied once per UTC calendar month. + It compounds, and the base is small: a contribution comparable to the seed doubles the account + monthly, and every position size below grows with it. +- **`caps.max_exposure_usd`** — has **two jobs at once**. It is the ceiling on total notional held + at any one moment (rail 4, and rail 6's concentration cap is a percentage of it), *and* it is the + **equity proxy that sizes orders** on the live path + (`keel/execution/executor.py::_build_intent`). So live `risk_pct` is a fraction of THIS number, + not of real account equity — raising the cap raises the real dollars risked per trade. Set above + actual equity it stops binding before available cash does, and the refusal comes later and less + legibly from the funding check (rail 13). In paper mode the proxy is bypassed: sizing uses the + paper account's own equity. +- **rail 14's monthly allowance** — the fee-free monthly BUY volume. It lives in the **database, + not `config.yaml`**: the `broker_subscriptions` row written by `keel subscription attest --venue + coinbase --tier `, or set directly with `keel subscription set --free-volume-usd N`. + `config.yaml` only supplies the tier catalogue and the unattested fallback + (`subscription.unsubscribed_allowance_usd`). Being in a different place from the caps is exactly + why it drifts out of step with them. + +**The interaction is the point.** Position sizing scales with equity (or, on the live path, with +the `max_exposure_usd` proxy); the rail-14 allowance is a fixed dollar figure that scales with +nothing. Let the two drift apart and *every* setup is vetoed — keel looks broken while every +component is doing exactly what it was configured to do. + +The real case: at **$11,000** paper equity with `risk_pct: 0.01`, a PAXG setup with a 3.35%-wide +stop sized to **$3,284.67** — exactly 1% of equity ($110) at risk, the correct answer. Rail 14's +allowance was **$500/month**, so it was vetoed, as was every other setup. Not a bug in either +setting; the two were simply on different scales. Reseeding the paper account at $500 sizes the +same setup at **$149.30**, which fits. + +Note the counter-intuitive mechanic behind those numbers: **a tighter stop produces a LARGER +position**, because `size = risk ÷ stop-distance` (`keel/execution/sizing.py::size`). That is how +a 1% risk becomes a **30% position** — `risk_pct` bounds what you lose if the stop holds, not what +you spend. diff --git a/tests/test_readme.py b/tests/test_readme.py new file mode 100644 index 00000000..90b37a98 --- /dev/null +++ b/tests/test_readme.py @@ -0,0 +1,140 @@ +"""The README, rewritten for the stranger the project now wants (#281). + +The README was an operator runbook — deploy, upgrade, paper-vs-live — which is the right +document for the author's future self and the wrong one for a newcomer deciding whether to +spend an afternoon here. Phase 7 exists because that stranger is now the primary reader, and +a stranger's four questions are: what is this (led by the compliance engine, which is the +differentiated asset — plenty of people have a trading bot), is it any good (the honest +measured result, stated by us before they find it themselves), whose fiqh am I getting (the +not-a-fatwa-engine boundary), and can I try it in five minutes. + +This file pins #281's acceptance so the rewrite cannot quietly regress to a runbook: the +honest result above the fold with a live link to the experiment record, a quickstart built +from commands that actually work, an architecture sketch naming the three load-bearing +places (rails, rules, adapter port), and an ending that routes questions and contributions. +The operator content is asserted to still EXIST — moved into `docs/operator-runbook.md`, +not deleted — and to be GONE from the README, because a README that is half runbook is a +runbook again with extra scrolling. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +_ROOT = Path(__file__).resolve().parents[1] + +#: The current experiment record for the honest result — the file the README's claim must link. +_EXPERIMENT_RECORD = "docs/experiments/2026-08-13-restated-under-a-production-faithful-engine.md" + + +def _readme() -> str: + return (_ROOT / "README.md").read_text() + + +def _readme_first_screen() -> str: + """Everything before the first `##` section: what a reader sees without scrolling.""" + return _readme().split("\n## ", 1)[0] + + +def _unwrapped(text: str) -> str: + """Join markdown wrapping: drop blockquote markers, then collapse all whitespace.""" + return " ".join(re.sub(r"(?m)^\s*>\s?", "", text).split()) + + +def test_the_honest_result_is_stated_above_the_fold(): + """The measured verdict, in the first screen, linking the record that shows it. + + `docs/experiments/` records that no shipped rule family is net-positive at the taker fee + actually paid — cost is the binding constraint. A visitor who finds that themselves feels + misled; a visitor who is told upfront reads it as rigour. The claim is pinned TOGETHER + with its link, and the link is pinned to a file that exists, so the README can never + cite an experiment record that has been renamed away. + """ + first = _unwrapped(_readme_first_screen()).lower() + assert "net-positive" in first or "net positive" in first, ( + "the honest result (no shipped rule net-positive at the taker rate paid) must be " + "stated in the README's first screen" + ) + assert _EXPERIMENT_RECORD in _readme(), ( + f"the honest-result claim must link the experiment record ({_EXPERIMENT_RECORD})" + ) + assert (_ROOT / _EXPERIMENT_RECORD).is_file(), ( + "the experiment record the README cites no longer exists — update the README's link" + ) + + +def test_the_quickstart_uses_commands_that_exist(): + """A five-minute path whose steps are real: init, fetch (with the honest key caveat), simulate. + + The commands are pinned verbatim, and so is the one prerequisite that is easy to hide: + market data needs a (free, read-only) Coinbase CDP key — `keel fetch` without one dies + in an AuthenticationError, and a quickstart that omits that is a quickstart that fails + at step three. + """ + text = _unwrapped(_readme()) + # `keel rules seed`, not `keel init`: a fresh CLONE already has the tracked `config.yaml`, + # and `keel init` refuses to overwrite it -- the quickstart must work from a clone. + for command in ("uv sync --all-extras --dev", "keel rules seed", "keel fetch", "keel simulate"): + assert command in text, f"the quickstart must include the command {command!r}" + assert "CDP" in text, ( + "the quickstart must say that market data needs a free read-only CDP key — " + "verified: `keel fetch` without one raises AuthenticationError" + ) + assert "paper" in text.lower(), "the quickstart must show the paper-mode path (no funds)" + + +def test_the_architecture_sketch_names_the_three_load_bearing_places(): + """Rails, rules, and the adapter port — a newcomer's map of where things live. + + Everything else in the tree is detail; these three are the surfaces a contributor's + first PR is most likely to touch, and the sketch must name their files. + """ + text = _readme() + assert "keel/execution/guards.py" in text, "the sketch must say where the rails live" + assert "keel/agent.py" in text, "the sketch must say where the rules live" + assert "packages/keel-broker-" in text, ( + "the sketch must say where a broker adapter plugs in (the packages/keel-broker-* port)" + ) + + +def test_the_readme_ends_by_routing_questions_and_contributions(): + """Where to ask, and how to contribute — the last thing read should be the next step taken.""" + tail = _unwrapped(_readme().lower()[-2000:]) + assert "discussions" in tail, "the README must end by pointing questions at Discussions" + assert "contributing" in tail, "the README must end pointing contributions at CONTRIBUTING.md" + + +def test_operator_content_moved_not_deleted(): + """The runbook sections live on in `docs/operator-runbook.md`, verbatim enough to find. + + Deployment, upgrades, and the paper-vs-live distinctions are operator knowledge the +project is not losing — #281 moves them, it does not delete them. Both halves are pinned: +the runbook GAINS the deployment/upgrade content, and the README LOSES it (below), because +leaving it in both is how the README grows back into a runbook. + """ + runbook = (_ROOT / "docs" / "operator-runbook.md").read_text() + assert "gh release download" in runbook, ( + "the wheel-deployment procedure must live in docs/operator-runbook.md now" + ) + assert "keel versions" in runbook and "PARTIAL INSTALL" in runbook, ( + "the partial-install verification story must live in the operator runbook" + ) + assert "paper" in runbook.lower() and "live" in runbook.lower(), ( + "the paper-vs-live distinctions must live in the operator runbook" + ) + + +def test_the_readme_is_no_longer_a_runbook(): + """Operator procedure is gone from the README — newcomer-first means actually first. + + The wheel-install commands and the deployment table are the runbook's densest content; + their presence here is the regression signal that the README is drifting back." + """ + text = _readme() + assert "gh release download" not in text, ( + "deployment procedure belongs in docs/operator-runbook.md, not the README" + ) + assert "com.keel.paperforward" not in text, ( + "the launchd/deployment table belongs in docs/operator-runbook.md, not the README" + )