Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

### Fixed

- **The access log never prints secrets** — a redaction filter on the uvicorn
`access`/`error` loggers masks sensitive query values (`token`, `signature`,
`api_key`, `nonce` — the `transport/http.py` key set, public as `redact_url`)
at all three serving commands (`serve`, `start --serve`, `dashboard`), so the
documented `?token=` script auth can no longer write the dashboard token to
journald/log files. (#232)

### Deprecated

### Removed
Expand Down
21 changes: 21 additions & 0 deletions doc/dev/03-decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,27 @@ rejected approaches as tombstones.

---

### 2026-08-02 Access-log redaction: reuse the transport scrubber, filter at logger level (PR #232) [accepted]
- **Choice**: scrub uvicorn's request log by attaching a mutating
`logging.Filter` (`AccessLogRedactionFilter`, in `application/log_setup.py`)
to the `uvicorn.access` + `uvicorn.error` **loggers** before every uvicorn
launch (`serve`, `start --serve`, `dashboard`), reusing the transport's
existing URL scrubber via a public alias (`transport/http.py:redact_url` —
same key set `token`/`signature`/`api_key`/`nonce`, same `<redacted>`
marker). Install is idempotent; the filter sanitises, never suppresses.
- **Why**: uvicorn's access logger writes the raw request target, so the
documented `?token=` script auth wrote the real dashboard token to journald
on the 2026-08-02 systemd deploy (invariant: secrets never logged; token
rotated). Logger-level placement is load-bearing: uvicorn's startup
`dictConfig` replaces the loggers' *handlers* but leaves
programmatically-attached logger *filters* in place (locked by a test), so
the scrub survives uvicorn's own logging setup at all three sites.
- **Rejected alternatives**: forking uvicorn's `LOGGING_CONFIG` dict per launch
site (three copies to keep in sync with upstream); duplicating the key set at
the interface layer (two lists to drift apart — one implementation means one
grep finds every masked value); handler-level filters (discarded with the
handler when `dictConfig` runs).

### 2026-07-14 Tick scheduler semantics pinned; timing is a health signal (PR #228) [accepted]
- **Choice**: the daemon tick job runs with explicit `coalesce=True`,
`max_instances=1`, `misfire_grace_time=interval` (cron triggers: fixed
Expand Down
8 changes: 7 additions & 1 deletion doc/dev/06-status.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# 06 — Status

_Last updated: 2026-07-10_
_Last updated: 2026-08-02_

## Where things stand

Expand Down Expand Up @@ -194,6 +194,12 @@ engine code: **real-key live enablement** (validate Kraken private endpoints +
venue-level idempotency against a real-key sandbox, then flip `live_enabled`) — the one
maintainer step in [`07-roadmap.md`](07-roadmap.md).

Ops: the dashboard daemon runs under **systemd** on the ops machine since
2026-08-02 (`deploy/trading-bot.service` installed + enabled; crash-restart and
clean SIGTERM verified; access log redacts `?token=`). Still open from
road-to-1.0 #4: an alert when the process dies, backups of the trading stores,
and the multi-week paper soak (running since 2026-07-10).


## Known gaps / deferred

Expand Down
7 changes: 4 additions & 3 deletions doc/dev/07-roadmap.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,9 +86,10 @@ order — none started yet, each is a `/pick-task` candidate:
risk-limit visibility incl. daily-loss usage); `last_error` on
`/api/strategies` (a stopped unit is indistinguishable from a crashed one).

4. [ ] **Ops readiness** (not engine code): daemon under **systemd** (restart
policy + an alert when the process dies — today it is a `nohup` from a
terminal session); a **multi-week paper soak** on the real-data books
4. [ ] **Ops readiness** (not engine code): daemon under **systemd** — **done
2026-08-02** on the ops machine (`deploy/trading-bot.service` installed +
enabled, crash-restart verified); still missing an **alert** when the
process dies; a **multi-week paper soak** on the real-data books
(running since 2026-07-10, capital 100/strategy — KPIs and equity curves as
evidence); **backup of the trading stores** (`var/dashboard/*.sqlite` — the
dccd data has its hourly rclone sync, the books have nothing).
Expand Down
100 changes: 99 additions & 1 deletion trading_bot/application/log_setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,11 @@

Secrets discipline: the formatter adds nothing beyond time / level / logger name
/ message — no request bodies, no credentials. Callers stay responsible for never
passing a credential-adjacent value into a log record.
passing a credential-adjacent value into a log record — with one exception this
module handles itself: uvicorn's *access* logger writes the request URL verbatim,
so the dashboard's documented ``?token=`` script auth would land in the journal on
every request. :func:`install_access_log_redaction` scrubs it at the source (see
:class:`AccessLogRedactionFilter`).
"""

from __future__ import annotations
Expand All @@ -40,11 +44,15 @@

# Local
from trading_bot.application.config import LoggingConfig
from trading_bot.transport.http import redact_url

__all__ = [
"configure_daemon_logging",
"install_access_log_redaction",
"AccessLogRedactionFilter",
"OWNED_HANDLER_ATTR",
"NOISY_NAMESPACES",
"ACCESS_LOG_NAMESPACES",
]

#: Attribute stamped ``True`` on every handler this module installs, so a re-call
Expand All @@ -63,6 +71,13 @@
"websockets",
)

#: The uvicorn logger namespaces whose records can carry a request URL — and so a
#: query-string secret. ``uvicorn.access`` writes one line per request (the leak
#: that motivated this); ``uvicorn.error`` carries the lifecycle/exception lines,
#: which quote the URL on a failed request. Both are scrubbed by
#: :func:`install_access_log_redaction`.
ACCESS_LOG_NAMESPACES: tuple[str, ...] = ("uvicorn.access", "uvicorn.error")

#: The shared log-line layout: ISO timestamp, padded level, logger name, message.
_LOG_FORMAT = "%(asctime)s %(levelname)-8s %(name)s — %(message)s"

Expand Down Expand Up @@ -146,3 +161,86 @@ def configure_daemon_logging(cfg: LoggingConfig) -> None:

for namespace in NOISY_NAMESPACES:
logging.getLogger(namespace).setLevel(logging.WARNING)


class AccessLogRedactionFilter(logging.Filter):
"""Scrub query-string secrets out of uvicorn's request log lines.

uvicorn's access logger formats one record per request as
``'%s - "%s %s HTTP/%s" %d'`` with the **raw request target** among its
``args`` — so the dashboard's documented ``?token=…`` script auth is written
verbatim to stderr/journald on every request. This filter rewrites the record
in place before any handler formats it, applying
:func:`~trading_bot.transport.http.redact_url` to ``record.msg`` and to every
string in ``record.args``; the value of a sensitive parameter (``token``,
``signature``, ``api_key`` / ``apiKey``, ``nonce``, case-insensitively)
becomes ``<redacted>``.

Reusing the transport's scrubber is deliberate: one key set, one marker, so a
URL is masked identically wherever it could reach a log. That function is a
no-op on a string with no query part, so the record's other args (the client
address, ``GET``, the HTTP version, the status code) pass through untouched,
as does a query carrying nothing sensitive (``?symbol=BTCUSDT``).

Never drops a record: :meth:`filter` always returns ``True``. It is a
*sanitiser*, not a gate — a suppressed access line would cost observability,
which is not the trade being made here.
"""

def filter(self, record: logging.LogRecord) -> bool:
"""Redact *record* in place; always keep it.

Parameters
----------
record : logging.LogRecord
The record about to be handled. Mutated in place — filters run once
per record before formatting, so every handler downstream (file,
stderr, journald) sees the scrubbed version.

Returns
-------
bool
Always ``True`` — the record is sanitised, never suppressed.
"""
if isinstance(record.msg, str):
record.msg = redact_url(record.msg)
# Only tuple args are positional `%s` substitutions worth scrubbing; a
# mapping-style `record.args` (`%(key)s` formatting) is left alone rather
# than rebuilt — uvicorn never uses it, and mutating an unknown mapping
# shape risks corrupting a third-party record.
if isinstance(record.args, tuple):
record.args = tuple(
redact_url(arg) if isinstance(arg, str) else arg for arg in record.args
)
return True


def install_access_log_redaction() -> None:
"""Attach one :class:`AccessLogRedactionFilter` to each uvicorn log namespace.

Call this **before** handing control to uvicorn (``uvicorn.run`` /
``Server.serve``) on every serving path, so no request line can be emitted
unscrubbed. The filters go on the *loggers* named in
:data:`ACCESS_LOG_NAMESPACES`, not on handlers, and that placement is
load-bearing: uvicorn configures logging at startup with
:func:`logging.config.dictConfig` (its ``LOGGING_CONFIG``), which replaces a
configured logger's **handlers** but leaves filters attached
programmatically to the logger itself in place. A handler-level filter would
be discarded with the handler it sat on; a logger-level one survives, and
runs once per record before any handler formats it.

Idempotent: a namespace that already carries a filter of this class is left
alone, so repeated calls in one process (three CLI serving commands, or a
test suite invoking them many times) never stack duplicate filters on the
process-global uvicorn loggers.

Returns
-------
None

"""
for namespace in ACCESS_LOG_NAMESPACES:
logger = logging.getLogger(namespace)
if any(isinstance(f, AccessLogRedactionFilter) for f in logger.filters):
continue
logger.addFilter(AccessLogRedactionFilter())
15 changes: 14 additions & 1 deletion trading_bot/interfaces/cli/main.py
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,10 @@
from trading_bot.application.config import AppConfig, BrokerConfig, StrategyConfig
from trading_bot.application.data_feed import BARS_SCHEMA, InMemoryFeed
from trading_bot.application.instrument_specs import InstrumentSpecResolver
from trading_bot.application.log_setup import configure_daemon_logging
from trading_bot.application.log_setup import (
configure_daemon_logging,
install_access_log_redaction,
)
from trading_bot.application.performance_service import PerformanceService
from trading_bot.application.run_app import run_app
from trading_bot.application.service_factory import Engine, build_engine
Expand Down Expand Up @@ -1207,6 +1210,9 @@ def serve(
f"[green]serving dashboard[/green] (read-only, mode={config.mode}) on "
f"http://{host}:{port} — use 'trading-bot dashboard' for the full control UI"
)
# Scrub uvicorn's access log first: it writes the request target verbatim, so
# a `?token=…` script-auth call would otherwise print the token in the clear.
install_access_log_redaction()
uvicorn.run(
application,
host=host,
Expand Down Expand Up @@ -1474,6 +1480,10 @@ def _schedule_info() -> dict[str, Any]:
)
if auth_token:
_console.print("[dim]control dashboard auth: token login enabled[/dim]")
# Scrub uvicorn's access log before it can emit a line: it writes the
# request target verbatim, so a `?token=…` script-auth call would
# otherwise write this daemon's auth token to the journal.
install_access_log_redaction()
server = uvicorn.Server(
uvicorn.Config(
api,
Expand Down Expand Up @@ -1856,6 +1866,9 @@ def _persist_manifest() -> None:
f"{', read-only' if read_only else ''}) on http://{host}:{port}"
" — Ctrl-C to stop"
)
# Scrub uvicorn's access log first: it writes the request target verbatim, so
# a `?token=…` script-auth call would otherwise print the token in the clear.
install_access_log_redaction()
try:
# uvicorn owns SIGINT: Ctrl-C returns from run() cleanly the first time.
uvicorn.run(
Expand Down
Loading
Loading