Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 13 additions & 1 deletion .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,13 @@ jobs:
echo "::error::artifact is not stamped as a release build"; exit 1; }
printf '%s' "$OUT" | grep -q "DIRTY" && {
echo "::error::artifact reports a dirty tree"; exit 1; } || true
# `--version` speaks for the keel-trader distribution ALONE, so it cannot see a
# sibling left behind at an older version -- the failure that had `~/keel` running
# keel-trader 0.5.7 against keel-core 0.5.5. `keel versions` checks every keel
# distribution in the venv and exits non-zero when they disagree, which also proves
# the `==` pins in the wheels actually pulled the siblings from dist/.
/tmp/verify/bin/keel versions || {
echo "::error::the installed wheels do not agree on a version"; exit 1; }

# The release ships a ready-for-live config as a downloadable asset. It must be in
# `confirm` mode: a config that trades unattended straight off a download is exactly what
Expand Down Expand Up @@ -168,7 +175,12 @@ jobs:
echo "\`keel_trader\` wheel **by path**:"
echo
echo ' pip install --find-links . ./keel_trader-${{ inputs.version }}-py3-none-any.whl'
echo " keel --version"
echo " keel versions"
echo
echo "\`keel versions\` — not \`keel --version\` — is the check: it reports **every**"
echo "keel distribution in the venv and exits non-zero if a sibling was left behind at"
echo "an older version, which \`--version\` cannot see. Upgrading an existing"
echo "deployment: see \"Deploying a new version\" in the README."
echo
echo "⚠️ **Never install by bare name.** The distribution is \`keel-trader\`; the name"
echo "\`keel\` on PyPI belongs to an unrelated project, so \`pip install keel\` fetches"
Expand Down
60 changes: 48 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,25 +129,61 @@ Cutting a release is `docs/RELEASING.md`. Installing one into a deployment (e.g.
commands, run **from the deployment directory** — every path below is relative to it:

```bash
gh release download v0.3.1 --repo CodeGateSoftware/keel --pattern '*.whl' --dir Release/
uv pip install --python .venv --find-links Release Release/keel_trader-0.3.1-py3-none-any.whl
.venv/bin/keel --version
V=0.6.0
gh release download "v$V" --repo CodeGateSoftware/keel --pattern '*.whl' --dir Release/
uv pip install --python .venv --find-links Release \
Release/keel_core-$V-py3-none-any.whl \
Release/keel_broker_api-$V-py3-none-any.whl \
Release/keel_broker_coinbase-$V-py3-none-any.whl \
Release/keel_trader-$V-py3-none-any.whl
.venv/bin/keel versions
.venv/bin/keel status
```

Substitute the version being deployed in both of the first two lines. `--find-links Release` is
what lets the single `keel_trader` wheel resolve its `keel-core` / `keel-broker-*` siblings from
that same directory — which is why step 1 downloads them all. Installing **by path** rather than
by bare name is deliberate: `keel` on PyPI is an unrelated project, so `pip install keel` fetches a
stranger's code (see `keel/version.py`).

Step 3 is the check that matters. It must report the version you just installed, bound to a
commit, from source `[release]`:
Set `V` to the version being deployed; nothing else changes between releases.

**Every wheel is named, and that is the fix for a real bug.** Installing `keel_trader` alone
upgraded *only* `keel_trader`: its siblings were required without a version, so the `keel-core`
already on disk satisfied `keel-core` and stayed put. `~/keel` ran `keel-trader 0.5.7` against
`keel-core 0.5.5` for two releases that way. A wheel **path** is a direct requirement — that exact
file is installed whatever is already there — so naming all four is what actually moves them.
The wheels now also pin their siblings exactly (`Requires-Dist: keel-core==0.6.0`), which forces
the upgrade even for someone who installs `keel_trader` alone; the four paths are the same
guarantee stated where the operator can see it.

**Not `Release/*.whl`.** The release ships *every* workspace wheel, two of which a deployment must
not have: `keel_broker_fake`, a dev-only fake venue that registers a `fake` entry point under
`keel.brokers`, and `keel_broker_robinhood`, an optional venue that pulls an Ed25519 stack
(`pynacl`, `cffi`) in for an adapter nothing constructs. The four named wheels are production's
whole dependency closure. `--find-links Release` still points at that directory so the pinned
siblings resolve locally rather than from PyPI, where they do not exist — which is why step 1
downloads them all. Installing **by path** rather than by bare name is deliberate and unchanged:
`keel` on PyPI is an unrelated project, so `pip install keel` fetches a stranger's code (see
`keel/version.py`).

Step 3 is the check that matters, and it is `keel versions` — **not** `keel --version`, which
could not fail. `--version` reports the `keel-trader` distribution's version and nothing else, so
it printed `0.6.0` while `keel-core` sat at `0.5.5`: a verification step blind to the failure mode,
which is worse than none, because it is trusted. `keel versions` prints the same build identity,
then every keel distribution in that venv, and **exits non-zero** when they disagree:

```
keel 0.3.1+deb8fa7e978d [release]
keel 0.6.0+deb8fa7e978d [release]

keel-broker-api 0.6.0
keel-broker-coinbase 0.6.0
keel-core 0.6.0
keel-trader 0.6.0

ok: 4 keel distributions, all at 0.6.0.
```

A partial upgrade fails it, with the numbers: `error: PARTIAL INSTALL: 4 keel distributions at 2
different versions (0.5.5, 0.6.0)`. So does finding `keel-broker-fake` installed — it was, at
`0.5.5`, in `~/keel`. Remove it: `uv pip uninstall --python .venv keel-broker-fake`. Nothing calls
`load_broker()` today so it is inert, but that is a property of this release, not of the package,
and no reason to leave a fake venue registered on the box that moves money.

A build reporting `(DIRTY)` or `[checkout]` corresponds to no commit and **must not be run against
live funds**. Step 4 is a read-only snapshot — no orders, no writes — confirming the new build
opens the database and reaches the venue.
Expand Down
8 changes: 7 additions & 1 deletion docs/RELEASING.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,11 @@ commit hash (`+` is the semver / PEP 440 build-metadata separator). The version
this". A build reporting `(DIRTY)` or `[checkout]` corresponds to no commit and **must not be run
against live funds**; `keel --version` warns loudly when so.

It answers for the `keel-trader` distribution **only**, though, and keel installs as six of them.
`keel versions` reports every one and exits non-zero if they disagree — that is the check a
deployment runs (README, "Deploying a new version"), and the release workflow runs it too against
the wheels it just built.

## Cutting a release

1. **Bump the version in a reviewed PR.** Edit `version` in `pyproject.toml`. The release workflow
Expand All @@ -36,7 +41,8 @@ against live funds**; `keel --version` warns loudly when so.
| asset | what it is |
|---|---|
| `keel_trader-<version>-py3-none-any.whl` | the CLI. Install **by path**, never by bare name. |
| `keel_core-*`, `keel_broker_*` wheels | workspace members `keel` depends on; download them all. |
| `keel_core-*`, `keel_broker_api-*`, `keel_broker_coinbase-*` | what `keel-trader` depends on, pinned `==` to this same version. Install all four wheels by path. |
| `keel_broker_fake-*`, `keel_broker_robinhood-*` | built by `--all-packages` and published, but **not** part of a deployment: the fake is a dev-only venue, Robinhood is optional (and drags in an Ed25519 stack). Do not install them into one. |
| `config.yaml` | the **production** config: real allowlist/caps in `auto_trade.mode: confirm`. |

`config.yaml` is `keel/templates/config.live.yaml`, committed and reviewed like any other code.
Expand Down
1 change: 1 addition & 0 deletions docs/go-live-runbook.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ exchange, correctly, and we have a record of it".
| | check | why |
|---|---|---|
| ☐ | `keel --version` reports `[release]`, no `DIRTY`, no `[checkout]` | a build that matches no commit is not reproducible; do not run it against funds |
| ☐ | `keel versions` exits **0** | `--version` speaks for the `keel-trader` distribution alone. This one checks every keel distribution in the venv and fails on a partial upgrade — new engine, old libraries — which `--version` cannot see |
| ☐ | you have a **trade-enabled** CDP key (the read-only one cannot place orders) | |
| ☐ | `.env` holds `CDP_API_KEY` / `CDP_API_SECRET`, and `.env` is git-ignored | credentials live only here — there is no vault |
| ☐ | you are at a real terminal | confirmation and every halt-releasing command fail closed off a TTY |
Expand Down
32 changes: 27 additions & 5 deletions keel/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,10 +57,11 @@
broker-touching commands (`fetch`, `agent`, `monitor`, `simulate`, `assets`) that share the
`_build_broker` seam, and the remaining top-level commands. The broker-free command groups live
in `keel/commands/*` and are registered here via `cli.add_command(...)`: `db`, `trials`,
`withdrawals`, `autonomy`, `rules`, `subscription`. The shared seams (`with_disclaimer`, the
confirmation gate, `_open_repo`/`_load_cfg`/`_build_broker`) live in `keel.commands._common` and
are re-imported here; `_is_interactive` is reached as `_common._is_interactive()` so a single
patch point in `keel.commands._common` drives every gate wherever its command is defined.
`withdrawals`, `autonomy`, `rules`, `subscription`, `versions`. The shared seams
(`with_disclaimer`, the confirmation gate, `_open_repo`/`_load_cfg`/`_build_broker`) live in
`keel.commands._common` and are re-imported here; `_is_interactive` is reached as
`_common._is_interactive()` so a single patch point in `keel.commands._common` drives every gate
wherever its command is defined.
"""

from __future__ import annotations
Expand Down Expand Up @@ -103,6 +104,7 @@
from keel.commands.subscription import subscription_group
from keel.commands.trials import trials_group
from keel.commands.tui import tui_cmd
from keel.commands.versions import versions_cmd
from keel.commands.withdrawals import withdrawals_group
from keel.compliance import purification as purification_mod
from keel.compliance import screen as screen_mod
Expand All @@ -123,7 +125,7 @@
from keel.sim import tiers as tiers_mod
from keel.strategy import promotion as promotion_mod
from keel.types import Candle, Granularity
from keel.version import build_info
from keel.version import build_info, check_install

# -- root group ---------------------------------------------------------------------------------

Expand All @@ -134,6 +136,12 @@ def _print_version(ctx: click.Context, param: object, value: bool) -> None:
Prints the working-tree state too. For a tool that can place orders, "0.1.0 (abc123, DIRTY)"
and "0.1.0 (abc123)" are materially different claims -- the first corresponds to no commit
and cannot be reproduced.

The line describes the `keel-trader` distribution ONLY, which is exactly how a deployment came
to run `keel-trader 0.5.7` against `keel-core 0.5.5` while this reported the new number. It
cannot be widened without changing what `--version` means, so instead it warns when the rest
of the install disagrees and points at `keel versions`, which reports all of them and exits
non-zero. The warning goes to stderr so the string this prints stays exactly what it was.
"""
if not value or ctx.resilient_parsing:
return
Expand All @@ -145,6 +153,12 @@ def _print_version(ctx: click.Context, param: object, value: bool) -> None:
"Do not run it against live funds.",
err=True,
)
if not check_install(source=info.source).is_consistent:
click.echo(
"warning: PARTIAL INSTALL -- this line reports the keel-trader distribution only, "
"and the other keel distributions do not agree with it. Run `keel versions`.",
err=True,
)
ctx.exit()


Expand Down Expand Up @@ -2159,6 +2173,14 @@ def simulate(
cli.add_command(insights_group)


# -- versions (the deploy check: every keel distribution, not just this one) ---------------------

# `--version` above answers for `keel-trader` alone and therefore cannot see a partial upgrade;
# this reports the whole install and exits non-zero when it disagrees with itself. Defined in
# `keel.commands.versions` and registered here.
cli.add_command(versions_cmd)


# -- kill / resume ------------------------------------------------------------------------------


Expand Down
66 changes: 66 additions & 0 deletions keel/commands/versions.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
"""`keel versions` -- the deploy check that can actually fail.

`keel --version` answers "which build is this?" for the `keel-trader` distribution and nothing
else, so it cannot see the failure it is used to rule out: installing the `keel_trader` wheel
alone leaves `keel-core` and the adapters at their old versions, and `--version` still prints the
new number. `~/keel` ran `keel-trader 0.5.7` against `keel-core 0.5.5` across two releases with
that check passing every time. A verification step blind to the failure mode is worse than none,
because it is trusted.

This command prints the same build-identity line and then every `keel-*` distribution installed
in the running interpreter's environment, and **exits non-zero** when they disagree -- so it can
be the last line of a deploy script and mean something. The rules live in
`keel.version.InstallReport.problems`, which is a pure value and unit-tested without installing
anything; this module is only the rendering and the exit code.

No config, no database, no network: nothing about it can fail for environmental reasons, which is
what makes a non-zero exit unambiguous.

**Why a CLI command and not `scripts/`.** `scripts/` is operator tooling that is not shipped in
the wheel, and a deployment is a `.venv` beside a `Release/` directory with no checkout of this
repository at all -- a script there could not be run without first fetching it. The check has to
travel inside the artifact it is checking.
"""

from __future__ import annotations

import click

from keel.version import build_info, check_install


@click.command("versions")
@click.pass_context
def versions_cmd(ctx: click.Context) -> None:
"""Verify the whole install: every keel distribution's version, not just keel-trader's."""
info = build_info()
report = check_install(source=info.source)

click.echo(info.describe())
if not info.is_reproducible:
click.echo(
"warning: this build is NOT reproducible -- it does not correspond to a commit. "
"Do not run it against live funds.",
err=True,
)

if not report.distributions:
# Nothing installed: a source checkout run via `uv run` with the workspace on the path.
# There is no install to disagree with itself, so there is nothing to fail on.
click.echo("no keel distributions installed -- nothing to compare.")
return

width = max(len(name) for name in report.distributions) + 2
click.echo("")
for name, version in sorted(report.distributions.items()):
click.echo(f"{name.ljust(width)}{version}")
click.echo("")

if not report.problems:
n = len(report.distributions)
click.echo(f"ok: {n} keel distributions, all at {report.versions[0]}.")
return

for problem in report.problems:
click.echo(f"error: {problem}", err=True)
ctx.exit(1)
Loading
Loading