From bfa7a29ea70b6f02c5f746314f745da24d73ad0d Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Tue, 11 Aug 2026 07:51:00 -0400 Subject: [PATCH] fix(analysis): scale is_round_number to the price's own magnitude `is_round_number(price, step=Decimal("0.005"))` compared price against an ABSOLUTE half-cent. Coinbase quotes BTC/ETH/PAXG to two decimals and `0.01 = 2 * 0.005`, so every quotable price was an exact multiple of `step`, `remainder` was always zero, and the function could never return False. Measured over the daily candle cache: P(present) = 1.0000 on BTC-USD, ETH-USD and PAXG-USD against 0.2170 on ADA-USD and 0.1901 on XLM-USD. As weight 1 of DEFAULT_WEIGHTS' 14, that handed three of the five live allowlist assets an unconditional +1 on every CTS score -- a constant, which is worse than a redundant factor, and one that applied to three assets and not the other two, so the total was not comparable across the allowlist. A round handle is a price with few significant figures, which is a property of the price relative to its own scale. The grid is now the two-significant- figure lattice derived from `Decimal.adjusted()` (exact integer exponent arithmetic, no float log10), and `tolerance` is a fraction of the HANDLE SPACING rather than of price. That denominator is the load-bearing choice: a fraction of price would make the presence rate depend on where in the decade the price sits (the spacing is 10% of price just above a power of ten and 1% just below), so the factor would silently change meaning as an asset trended through a decade -- BTC's history crosses two. A fraction of spacing makes P(present) identically 2*tolerance regardless of price, decade position or quote precision, which is #225's acceptance criterion stated as an invariant. Measured before/after on the same 6,827 daily bars, using #224's replay harness unchanged: P(present) BTC 1.0000 -> 0.0358, ETH 1.0000 -> 0.0334, PAXG 1.0000 -> 0.0463, ADA 0.2170 -> 0.0465, XLM 0.1901 -> 0.0440. Cross-asset spread 5.26x -> 1.39x. CTS total pooled mean 5.145 -> 4.566, median 5 -> 4. The BTC-vs-XLM mean gap falls 1.44 -> 0.62, so 57% of it was this bug. Thresholds 1,103 of 6,827 bars (16.2%) change entry_technique, every one a rung down; `aggressive` falls 43% pooled. No order changes. `entry_technique(total, low=5, high=8)` at engine.py:144 is the only threshold a CTS total is compared against anywhere in the package; its three return values appear nowhere outside indicators_cts.py, and no config key, entry gate or promotion gate reads a CTS score at all (promotion.can_promote runs off backtested trade statistics). So the danger #225 raised -- pushing qualifying setups below a tuned threshold -- cannot occur. No threshold was retuned. `step` was renamed to `tolerance` rather than kept: the meaning inverted from an absolute price step to a relative fraction of the grid, and a caller passing `step=Decimal("0.005")` under the old name would silently get new behaviour. The one caller in the package (engine.assemble_cts_context) uses the default. The function stays pure -- no venue lookup, no quote_increment -- because #224's offline replay depends on that. tests/strategy/test_engine.py's fixture enters at 128.02, which is 2.02 from the nearest handle and was never near a magnet level; it scored present only because 128.02 is a multiple of half a cent. Its CTS drops 5 -> 4 and its technique signal_candle -> confirm_3bar, which is the bug's consequence in miniature. A companion test asserts the point returns on a real handle. Closes #225 Co-Authored-By: Claude Opus 5 (1M context) --- .../2026-08-11-round-number-scale.md | 309 ++++++++++++ .../2026-08-11-round-number-scale.py | 441 ++++++++++++++++++ docs/experiments/trials-ledger.jsonl | 1 + keel/analysis/levels.py | 83 +++- tests/analysis/test_levels.py | 94 +++- tests/strategy/test_engine.py | 45 +- 6 files changed, 953 insertions(+), 20 deletions(-) create mode 100644 docs/experiments/2026-08-11-round-number-scale.md create mode 100644 docs/experiments/2026-08-11-round-number-scale.py diff --git a/docs/experiments/2026-08-11-round-number-scale.md b/docs/experiments/2026-08-11-round-number-scale.md new file mode 100644 index 00000000..8694b1c1 --- /dev/null +++ b/docs/experiments/2026-08-11-round-number-scale.md @@ -0,0 +1,309 @@ +# `is_round_number` had no sense of scale — the fix, and what it costs live scoring + +**Date:** 2026-08-11 +**Issue:** #225 (surfaced as a side effect of #208 / PR #224) +**Change:** `keel/analysis/levels.py::is_round_number` — a **correctness fix to shipped scoring +code**, not a research finding. This write-up exists because the fix changes live CTS scores and +#225 requires a before/after rather than a drive-by patch. +**Harness:** `keel/research/cts_factors.py`, built for #208 and **reused unchanged**. No second +instrument was written; the whole point of having put the replay in the package was that the next +question could be asked with it. +**Script:** `docs/experiments/2026-08-11-round-number-scale.py` — produced every number below. +**Ledger:** one row, `round-number-scale-2026-08-11`, session `round-number-scale-2026-08-11`. + +**Verdict: the factor is salvageable, the fix is safe to ship, and the reason it is safe is not +the reason anyone expected.** + +| question #225 asked | answer | +|---|---| +| P(present) materially below 1.0 on 2dp assets? | **yes** — 1.0000 → **0.0358 / 0.0334 / 0.0463** on BTC/ETH/PAXG | +| broadly comparable across the allowlist? | **yes** — cross-asset spread **5.26× → 1.39×** | +| CTS distribution shift? | pooled mean **5.145 → 4.566**; BTC/ETH/PAXG each lose ≈**0.96** of a point, ADA/XLM ≈**0.16** | +| does it push qualifying setups below a gate? | **no — there is no CTS gate to fall through** (see §4) | +| bars whose entry technique changes | **1,103 of 6,827 (16.2%)**, every one of them one rung *down* | + +--- + +## 1. The defect + +```python +def is_round_number(price: Decimal, step: Decimal = Decimal("0.005")) -> bool: + remainder = price % step + distance = min(remainder, step - remainder) + return distance <= step * Decimal("0.1") +``` + +`step` is an **absolute** half-cent. Coinbase quotes BTC-USD, ETH-USD and PAXG-USD to two +decimals, and `0.01 = 2 × 0.005`, so every quotable price is an exact multiple of `step`, +`remainder` is exactly zero, and the function returns `True` unconditionally. `distance <= step * +0.1` never got a chance to be false. + +Measured over the daily candle cache (6,827 bars, expanding window, the live path's own): +P(present) = **1.0000 / 1.0000 / 1.0000** on BTC / ETH / PAXG, **0.2170** on ADA, **0.1901** on +XLM. `round_number_proximity` is weight 1 of `DEFAULT_WEIGHTS`' 14, so three of the five live +allowlist assets carried an unconditional **+1 on every CTS score**. + +A constant is worse than a redundant factor: a redundant factor at least varies. And because the +constant applied to three assets and not the other two, the CTS *total* was not comparable across +the allowlist — BTC's mean sat 1.44 points above XLM's, of which 0.82 was this artifact. + +## 2. What "round" was made to mean, and why + +**A round handle is a price with few significant figures.** That is what makes a number watchable +— 65,000, 3,400, 0.38 — and it is a property of the price *relative to its own magnitude*, which +is exactly the property an absolute constant cannot have. So: + +``` +spacing = 10 ** (floor(log10(price)) - 1) # the two-significant-figure grid +distance = distance from price to the nearest multiple of spacing +present ⟺ distance <= spacing * tolerance # tolerance default 0.02 +``` + +Three choices in there, each of which was decided rather than defaulted into. + +**Two significant figures, not three.** Two is what the words mean: 65,000 and 0.38 are handles, +65,100 and 0.381 are not, and both pairs stand in the same relation to their own price. It is +also the choice that survives measurement. A three-significant-figure grid pushes the spacing down +onto ADA's and XLM's quote increment, and tick quantization then drives the base rate rather than +the price does: measured, ADA reaches **0.2996** against BTC's **0.1861** at the same tolerance — +a 1.61× spread manufactured by nothing but quote precision. That is the same class of artifact +this fix exists to remove, so three figures was rejected. + +**Tolerance is a fraction of the handle SPACING, not of price — and the denominator is the whole +argument.** Both denominators scale with the instrument, so both fix the reported bug. They differ +in what they hold constant: + +- *Fraction of price* makes the presence rate depend on where in the decade the price sits. The + spacing is 10% of price just above a power of ten and 1% of it just below, so the identical rule + would fire ten times as often on BTC at 99,000 as at 10,500, and the factor would silently + change meaning as an asset trended through a decade. BTC's daily history spans 15,760 to + 124,720 — two decade crossings — so this is not hypothetical. +- *Fraction of spacing* makes P(present) identically `2 × tolerance` for any price series smooth + on the scale of the grid, independent of price, decade position and quote precision alike. + +The second is precisely #225's acceptance criterion — *the factor must mean the same thing at +65,000 as at 0.38* — restated as an invariant instead of a hope. §3 shows it holds empirically. + +**No venue handle, deliberately.** #225 floats deriving the grid from `quote_increment`. Rejected: +`assemble_cts_context` is a **pure** function of `(setup, candles)` and #224's offline replay — +the very instrument measuring this change — depends on that. Threading venue state into an +`analysis.*` primitive would make the scoring path unreplayable to fix a factor that does not need +it. It is also conceptually wrong: tick size is a venue's quoting rule, and a psychological handle +is a property of the number, identical on any venue that lists the asset. + +**Tolerance = 0.02 is the one genuinely free parameter, and it is pinned, not fitted to P&L.** +The ladder, measured over the same closes: + +| tolerance | BTC | ETH | PAXG | ADA | XLM | pooled | spread | `64975.78` | +|---:|---:|---:|---:|---:|---:|---:|---:|:--| +| 0.10 | 0.1953 | 0.1953 | 0.1961 | 0.2146 | 0.2097 | 0.2033 | 1.10× | `True` | +| 0.05 | 0.0925 | 0.0871 | 0.1089 | 0.1117 | 0.1057 | 0.0998 | 1.28× | `True` | +| **0.02** | **0.0368** | **0.0357** | **0.0414** | **0.0436** | **0.0425** | **0.0397** | **1.22×** | **`False`** | +| 0.01 | 0.0162 | 0.0195 | 0.0218 | 0.0267 | 0.0212 | 0.0210 | 1.64× | `False` | + +⚠️ **Honest note: `0.10` is arguably the better number and it was not chosen.** It preserves the +original docstring's stated intent ("within 10% of the step size") with only the *step* corrected, +it gives the tightest cross-asset spread in the table (1.10×), and it lands the base rate at +0.195–0.215 — almost exactly where ADA and XLM already sat, making the fix minimally disruptive on +the two assets that were never broken. It was rejected on one ground: #225 names `Decimal( +"64975.78")` as a price that **must** score absent, and at `0.10` the band on BTC is ±100 and +64,975.78 (24.22 from the 65,000 handle) scores present. That case pins the tolerance below +0.0242. `0.02` is the round number under that ceiling. + +Whether 24.22 dollars from 65,000 — 3.7 basis points, a fraction of a daily range — is really +"far from any round handle" is a judgement I did not feel entitled to overturn on a change to live +scoring. **If the intent was the looser band, `tolerance=0.10` is a one-character change and this +table is the evidence for it.** Flagged in §6. + +## 3. Result — P(present) before and after, same bars + +Unconditional sample, ONE_DAY, expanding window from the first cached bar (which reproduces the +live path exactly: `agent.run_once` → `repo.get_candles` with no bounds → `engine.evaluate`). + +| asset | N | before | after | Δ | +|---|---:|---:|---:|---:| +| BTC-USD | 1,648 | 1.0000 | **0.0358** | −0.9642 | +| ETH-USD | 1,648 | 1.0000 | **0.0334** | −0.9666 | +| PAXG-USD | 259 | 1.0000 | **0.0463** | −0.9537 | +| ADA-USD | 1,636 | 0.2170 | **0.0465** | −0.1705 | +| XLM-USD | 1,636 | 0.1901 | **0.0440** | −0.1461 | +| **pooled** | **6,827** | **0.6183** | **0.0401** | **−0.5781** | + +**Cross-asset spread (max/min): 5.26× → 1.39×.** That single number is #225's acceptance test. +The residual 1.39× is not noise in the definition — it is the sampling spread of a ~4% Bernoulli +rate over 259–1,648 bars, and PAXG (N=259, the widest) is the shortest series. + +The repaired factor now sits inside the existing panel rather than dominating it: + +| factor | wt | P(present) | +|---|---:|---:| +| in_pullback | 1 | 0.8389 | +| sr_touches | 2 | 0.7990 | +| fib_confluence | 1 | 0.3545 | +| ema_fan_aligned | 2 | 0.3009 | +| condition_aligned | 2 | 0.2751 | +| candlestick_pattern | 1 | 0.2026 | +| deceleration | 1 | 0.1743 | +| rsi_divergence | 2 | 0.0915 | +| **round_number_proximity** | **1** | **0.0401** ← repaired | +| rsi_extreme | 1 | 0.0230 | +| seasonality | 0 | 0.0000 | + +It is now rarer than `candlestick_pattern` and commoner than `rsi_extreme` — a member of the +distribution, not an outlier at either end. + +### The before-arm is verified, not asserted + +Before and after are on **the same bars**, and the before-arm is reconstructed arithmetically +(only this factor moves, and it is a pure function of the entry price) rather than replayed. That +reconstruction is an argument, so arm E re-runs the **full replay** on all five assets with +`levels.is_round_number` monkeypatched back to its pre-#225 body and compares bar for bar: + +``` +asset N factor vec CTS totals other factors +BTC-USD 1648 MATCH MATCH MATCH +ETH-USD 1648 MATCH MATCH MATCH +PAXG-USD 259 MATCH MATCH MATCH +ADA-USD 1636 MATCH MATCH MATCH +XLM-USD 1636 MATCH MATCH MATCH + reconstruction is EXACT +``` + +This also independently confirms the claim the reconstruction rests on: **no other CTS factor +moved.** Had anything else read the round-number flag transitively, the `other factors` column +would have broken. + +## 4. Result — CTS distribution, and the threshold impact + +| asset | arm | mean | median | sd | min | max | Δmean | +|---|---|---:|---:|---:|---:|---:|---:| +| BTC-USD | before | 5.771 | 6.0 | 2.008 | 1 | 11 | | +| | after | **4.806** | **5.0** | 2.016 | 0 | 10 | **−0.964** | +| ETH-USD | before | 5.620 | 6.0 | 1.906 | 1 | 11 | | +| | after | **4.653** | **5.0** | 1.916 | 0 | 10 | **−0.967** | +| PAXG-USD | before | 6.367 | 6.0 | 1.647 | 3 | 10 | | +| | after | **5.413** | **5.0** | 1.643 | 2 | 9 | **−0.954** | +| ADA-USD | before | 4.652 | 4.0 | 1.861 | 0 | 10 | | +| | after | **4.481** | 4.0 | 1.818 | 0 | 10 | **−0.171** | +| XLM-USD | before | 4.335 | 4.0 | 1.945 | 0 | 10 | | +| | after | **4.189** | 4.0 | 1.921 | 0 | 10 | **−0.146** | +| **pooled** | before | 5.145 | 5.0 | 2.028 | 0 | 11 | | +| | after | **4.566** | **4.0** | 1.930 | 0 | 10 | **−0.578** | + +Exactly as predicted: the three 2dp assets lose ≈0.96 (the constant, minus the ~4% of bars where +the factor legitimately fires), the two others lose ≈0.16. **The cross-asset mean gap narrows from +1.44 points (BTC 5.771 vs XLM 4.335) to 0.62 (4.806 vs 4.189)** — 57% of the gap between the +highest- and lowest-scoring allowlist assets was this bug, not the market. + +### Every threshold a CTS total is compared against + +Grepping `cts` across `keel/`, `packages/` and `scripts/` finds **exactly one**: + +```python +# keel/strategy/indicators_cts.py:154 +def entry_technique(total: int, low: int = 5, high: int = 8) -> Literal[...] +``` + +called from **one** site, `keel/strategy/engine.py:144`, with no override. There is no config key +(`min_cts`, `cts_min`, `min_score` — none exist), and **the promotion gate does not read CTS at +all**: `promotion.can_promote` runs off backtested `n_trades` / `expectancy` / realized R:R / +`win_rate`, and the PBO gate off `pbo` and `degradation_slope`. + +| asset | arm | confirm_3bar | signal_candle | aggressive | moved | +|---|---|---:|---:|---:|---:| +| BTC-USD | before | 460 | 847 | 341 | | +| | after | 751 | 733 | 164 | **468 (28.4%)** | +| ETH-USD | before | 489 | 891 | 268 | | +| | after | 762 | 751 | 135 | **406 (24.6%)** | +| PAXG-USD | before | 37 | 157 | 65 | | +| | after | 80 | 145 | 34 | **74 (28.6%)** | +| ADA-USD | before | 826 | 688 | 122 | | +| | after | 889 | 650 | 97 | **88 (5.4%)** | +| XLM-USD | before | 894 | 645 | 97 | | +| | after | 934 | 622 | 80 | **67 (4.1%)** | +| **pooled** | before | 2,706 | 3,228 | 893 | | +| | after | **3,416** | **2,901** | **510** | **1,103 (16.2%)** | + +16.2% of bars change technique; every move is one rung down, because removing a point cannot raise +a total. `aggressive` falls **43%** pooled and roughly **halves** on the three 2dp assets. + +**⭐ And none of it changes a single order.** `entry_technique`'s three return values — +`"confirm_3bar"`, `"signal_candle"`, `"aggressive"` — appear **nowhere** in `keel/`, `packages/` +or `scripts/` outside `indicators_cts.py`'s own definition and docstring. Nothing branches on the +technique; nothing sizes, stops, or picks an order type from it. In `agent.py` it reaches exactly +one place — a field on the `agent.enter_evaluated` log line (`agent.py:1166`). `cts_score` has the +same shape: written to the `signals` table, logged, carried on sim records, read by no gate. + +So the specific danger #225 raised — *"may push scores below promotion/entry thresholds that were +tuned with the constant in place"* — **cannot occur**. There is no CTS floor. No setup that +previously qualified is rejected, no position changes size, no stop moves. What changes is the +*label* recorded in the audit trail and in `signals.cts_score`, on 16.2% of bars. + +That is the finding that makes this fix safe to ship. It is also, in its own right, a defect worth +a follow-up (§6): `indicators_cts.py`'s module table promises `confirm_3bar` means "smaller size, +wider stop" and `aggressive` means "larger size toward the cap, tighter stop", and **none of that +is wired to anything.** The graded entry ladder in spec §9/§17.1 is documented, computed, scored, +persisted — and inert. + +## 5. What changed in the tests + +The repo's own engine fixture is the smallest end-to-end demonstration of the bug. It enters at +**128.02**, which is 2.02 away from the nearest handle (130, on a 10-wide grid at that magnitude) +— it is not near a magnet level and never was. It scored present only because 128.02 is an exact +multiple of half a cent, as every 2dp price is. Correcting it drops that fixture's CTS from 5 to 4 +and its technique from `signal_candle` to `confirm_3bar` — a real crossing of `entry_technique`'s +`low=5` edge, caused entirely by removing a point that was never earned. +`test_default_weights_on_same_fixture_yields_confirm_3bar_tier` now asserts the corrected values, +with a companion test showing the point returns when the entry is nudged onto the 130 handle. + +## 6. What this changes, and what it explicitly does not + +**1. `is_round_number` is fixed.** One caller in the package (`engine.assemble_cts_context`, +`engine.py:288`), which uses the default. The parameter was **renamed `step` → `tolerance`** +rather than kept: its meaning inverted from an absolute price step to a relative fraction of the +handle grid, and a caller passing `step=Decimal("0.005")` under the old name would silently get +new behaviour. Renaming makes any such caller fail loudly. There are none outside the tests. + +**2. No threshold was retuned, and none should be on this evidence alone.** `low=5` / `high=8` are +untouched. The pooled median moved 5 → 4, so `low=5` now sits above the median and a plurality of +bars land in `confirm_3bar` — but retuning band edges is a separate decision with its own evidence +requirement, and it is moot until §6.3 is resolved. + +**3. Recommend a follow-up issue: the graded entry ladder is inert.** `entry_technique` is +computed on every signal, persisted, and read by nothing. Either wire it to sizing/stop/order-type +as spec §9/§17.1 describes, or delete the claim from the docstring — but the current state, where +the audit trail records a posture that execution does not implement, is the worst of both. **This +is also the precondition for §6.2**: recalibrating `low`/`high` is meaningless while nothing +consumes their output. + +**4. Recommend recording `tolerance=0.10` as an open question** (§2). It is the better number on +every axis except the one correctness case #225 pins, and that case is arguable. Cheap to revisit; +the ladder above is the evidence. + +**5. Nothing here says this factor predicts anything.** Under §73.5 a well-defined factor is +necessary and never sufficient. This fix makes `round_number_proximity` *mean something +consistent across assets*; whether what it means is worth a point of CTS is a question nobody has +asked, and 4% presence on a weight-1 factor of 14 means it now moves the total very little either +way. + +## Caveats + +- In-sample, one granularity, one window, no out-of-sample split. This is a correctness fix + measured for blast radius, not a strategy result. +- **Serial dependence.** Bars are autocorrelated, so 6,827 observations are worth fewer than 6,827 + independent ones. No conclusion here rests on a p-value — nothing is tested for significance — + so this does not move the finding, but the base rates are less precise than N suggests. +- **PAXG-USD contributes 259 of 6,827 daily bars** (listed 2025-05-08) and is the widest cell in + every table, including the 1.39× spread that the acceptance criterion is read off. +- The synthetic setup prices entry at the bar's **close** (`cts_factors._synthetic_setup`, + unchanged from #224). `round_number_proximity` is measured against that price, so its base rate + would shift under a different entry convention. The close is the neutral choice — it is what a + market order fills at — but it is a choice, and it is the one this factor is most sensitive to. +- N is 6,827 here against 6,822 in #224: the candle cache grew by five daily bars between the two + runs. Same assets, same window rule, five more bars. +- `tolerance=0.02` yields `2 × tolerance` presence **only for a price series smooth on the scale + of the grid**. That holds for all five allowlist assets (§3 confirms it: 0.033–0.047 against a + predicted 0.04). It would not hold for an asset pinned near a handle, or one whose tick is a + meaningful fraction of its 2-significant-figure spacing — i.e. an asset quoted to fewer than + ~3 significant figures. None on the allowlist is close. +- All five assets are crypto over one broadly-correlated window. diff --git a/docs/experiments/2026-08-11-round-number-scale.py b/docs/experiments/2026-08-11-round-number-scale.py new file mode 100644 index 00000000..d8d86c7a --- /dev/null +++ b/docs/experiments/2026-08-11-round-number-scale.py @@ -0,0 +1,441 @@ +#!/usr/bin/env python +"""What does fixing `levels.is_round_number` do to live CTS scoring? -- issue #225. + +Every empirical claim in `docs/experiments/2026-08-11-round-number-scale.md` comes from this +script. Strictly read-only: it opens the candle cache with `mode=ro`, drives no broker, writes +nothing but stdout, and changes no weight, gate or rule. The replay machinery is +`keel/research/cts_factors.py`, built for #208/#224 and REUSED here rather than reimplemented -- +which is the point of having put it in the package. + +WHAT IS BEING MEASURED. `round_number_proximity` is weight 1 of `DEFAULT_WEIGHTS`' 14. Before +this change `levels.is_round_number` compared price against an ABSOLUTE `step=Decimal("0.005")`, +and every 2dp-quoted price is an exact multiple of half a cent, so the check returned `True` +unconditionally on BTC-USD, ETH-USD and PAXG-USD. Fixing it removes a constant +1 from those +three assets wherever the factor does not legitimately apply. That is a change to live scoring, +so it gets a before/after on the same bars rather than an assertion. + +HOW BEFORE AND AFTER ARE PUT ON THE SAME SAMPLE. `engine.assemble_cts_context` is called ONCE +per bar, under the shipped (fixed) function -- that is the AFTER sample. The BEFORE sample is +then reconstructed exactly rather than replayed a second time, because it can be: +`is_round_number` has exactly one caller in the package (`engine.assemble_cts_context`, line +288, verified by grep), it is a pure function of `setup.entry` alone, and `setup.entry` is the +bar's close. So the only cell of the context that moves is `round_number_proximity`, and the +only thing that moves in the total is its weight-1 contribution. + +⚠️ That reconstruction is an argument, not a measurement, so ARM E does not take it on trust. +It re-runs the FULL replay on every asset with `levels.is_round_number` monkeypatched back to +the pre-#225 body, and asserts the replayed before-vectors and before-totals match the +reconstructed ones bar for bar. If the reconstruction were wrong -- if some other factor read +the round-number flag indirectly -- arm E is what would catch it. Run it; it is not optional +decoration, and the write-up quotes its result. + + .venv/bin/python docs/experiments/2026-08-11-round-number-scale.py + .venv/bin/python docs/experiments/2026-08-11-round-number-scale.py --db path/to.db +""" + +from __future__ import annotations + +import argparse +import logging +import sqlite3 +import statistics +from collections.abc import Callable +from decimal import Decimal + +from keel.analysis import levels +from keel.research.cts_factors import FACTOR_NAMES, FactorSample, pool, replay_every_bar +from keel.strategy import indicators_cts +from keel.strategy.indicators_cts import DEFAULT_WEIGHTS, entry_technique +from keel.types import Candle, Granularity + +# -- PRE-DECLARED CONFIGURATION ------------------------------------------------------------- + +DEFAULT_DB = "/Users/elmehdiaitbrahim/keel/keel.db" + +#: The live allowlist (`config.live-sandbox.yaml`), matching #224 exactly so the BEFORE column +#: here is directly comparable to the base-rate table published there. +ASSETS = ("BTC-USD", "ETH-USD", "PAXG-USD", "ADA-USD", "XLM-USD") + +PRIMARY_GRANULARITY = Granularity.ONE_DAY + +#: The factor under repair, and its weight. Read from the shipped table, never hardcoded. +FACTOR = "round_number_proximity" +FACTOR_WEIGHT = DEFAULT_WEIGHTS[FACTOR] + +#: `indicators_cts.entry_technique`'s band edges. These are THE ONLY thresholds in the package +#: that a CTS total is compared against -- `keel/strategy/engine.py:144` is the sole call site, +#: it passes no override, and no config key or promotion gate reads a CTS score at all +#: (`promotion.can_promote` runs off backtested trade statistics). Named here so the +#: threshold-impact arm cannot drift from the shipped defaults. +CTS_LOW, CTS_HIGH = 5, 8 + +#: Tolerance ladder for arm D. `0.02` is the shipped default; `0.10` is what the pre-#225 +#: docstring claimed ("within 10% of the step size") and is included so the cost of the one +#: genuinely free parameter in the fix is visible rather than asserted. +TOLERANCE_LADDER = (Decimal("0.10"), Decimal("0.05"), Decimal("0.02"), Decimal("0.01")) + + +def legacy_is_round_number(price: Decimal, step: Decimal = Decimal("0.005")) -> bool: + """`levels.is_round_number` exactly as it stood before #225, kept here verbatim. + + It lives in the experiment rather than the package because it is wrong: its only remaining + job is to be the BEFORE arm of a comparison. Copied byte for byte from `levels.py` at + commit f7318fd so the before-column is the real historical behaviour and not a paraphrase. + """ + remainder = price % step + distance = min(remainder, step - remainder) + return distance <= step * Decimal("0.1") + + +def load_candles(db_path: str, product_id: str, granularity: Granularity) -> list[Candle]: + """Ascending candles for one product/granularity, read-only. + + Byte-identical to `2026-08-09-cts-factor-collinearity.py`'s loader, deliberately: an + experiment must not be able to write to the cache it reads, and `mode=ro` makes that + structural rather than a promise. + """ + connection = sqlite3.connect(f"file:{db_path}?mode=ro", uri=True) + try: + rows = connection.execute( + "SELECT ts, o, h, l, c, v FROM candles " + "WHERE product_id = ? AND granularity = ? ORDER BY ts", + (product_id, granularity.value), + ).fetchall() + finally: + connection.close() + return [ + Candle( + ts=ts, + open=Decimal(o), + high=Decimal(h), + low=Decimal(low), + close=Decimal(c), + volume=Decimal(v), + ) + for ts, o, h, low, c, v in rows + ] + + +# -- before/after construction --------------------------------------------------------------- + + +def replayed_prices(candles: list[Candle], warmup: int) -> list[Decimal]: + """The `setup.entry` prices `replay_every_bar` scored, in the same order. + + `cts_factors._synthetic_setup` prices the setup at the bar's close and + `replay_every_bar` walks `range(warmup, len(candles))`, so this reproduces the entry + sequence without re-entering the replay. Kept as one function so the coupling to the + harness's iteration order is stated in one place rather than assumed in three. + """ + return [candle.close for candle in candles[warmup:]] + + +def before_from_after(sample: FactorSample, prices: list[Decimal]) -> FactorSample: + """The pre-#225 sample, reconstructed from the post-#225 one on the same bars. + + Only `round_number_proximity` moves, so the before-vector is `legacy_is_round_number` over + the same entry prices and the before-total is the after-total with this factor's weight + subtracted where it is now present and added where it used to be. Arm E verifies this + against a real replay. + """ + if len(prices) != sample.n: + raise AssertionError(f"price/sample length mismatch: {len(prices)} vs {sample.n}") + + legacy = [1 if legacy_is_round_number(price) else 0 for price in prices] + current = sample.vectors[FACTOR] + vectors = {name: list(vec) for name, vec in sample.vectors.items()} + vectors[FACTOR] = legacy + totals = [ + total + FACTOR_WEIGHT * (was - now) + for total, was, now in zip(sample.totals, legacy, current) + ] + return FactorSample(vectors=vectors, totals=totals, labels=list(sample.labels)) + + +def replay_with( + predicate: Callable[[Decimal], bool], + product_id: str, + candles: list[Candle], +) -> FactorSample: + """Replay every bar with `levels.is_round_number` temporarily swapped for `predicate`. + + Monkeypatching a shipped module is not something to do lightly, and it is done here for one + reason: arm E has to drive the REAL `engine.assemble_cts_context` under the old predicate to + prove the arithmetic reconstruction above is exact. Patching the definition is the only way + to do that without a second checkout. The original is restored in a `finally`, and nothing + downstream of this process is affected -- the script writes no state. + """ + original = levels.is_round_number + levels.is_round_number = predicate # type: ignore[assignment] + try: + return replay_every_bar(product_id, candles, window=None) + finally: + levels.is_round_number = original # type: ignore[assignment] + + +# -- report ---------------------------------------------------------------------------------- + + +def _fmt(value: Decimal | float, places: str = "0.0001") -> str: + return str(Decimal(str(value)).quantize(Decimal(places))) + + +def _rate(vector: list[int]) -> Decimal: + return Decimal(sum(vector)) / Decimal(len(vector)) if vector else Decimal(0) + + +def print_presence( + before: dict[str, FactorSample], + after: dict[str, FactorSample], +) -> None: + """ACCEPTANCE TABLE: P(round_number_proximity present), before vs after, per asset.""" + print("\nARM A -- P(round_number_proximity present), same bars, before vs after") + header = f"{'asset':10} {'N':>6} {'before':>9} {'after':>9} {'delta':>9}" + print(header) + print("-" * len(header)) + for product_id in after: + was = _rate(before[product_id].vectors[FACTOR]) + now = _rate(after[product_id].vectors[FACTOR]) + print( + f"{product_id:10} {after[product_id].n:6d} {_fmt(was):>9} {_fmt(now):>9} " + f"{_fmt(now - was):>9}" + ) + pooled_before, pooled_after = pool(before.values()), pool(after.values()) + was, now = _rate(pooled_before.vectors[FACTOR]), _rate(pooled_after.vectors[FACTOR]) + print("-" * len(header)) + print(f"{'POOLED':10} {pooled_after.n:6d} {_fmt(was):>9} {_fmt(now):>9} {_fmt(now - was):>9}") + + rates = [_rate(s.vectors[FACTOR]) for s in after.values()] + old_rates = [_rate(s.vectors[FACTOR]) for s in before.values()] + print( + f"\n cross-asset spread (max/min): before {_fmt(max(old_rates) / min(old_rates), '0.01')}x" + f" after {_fmt(max(rates) / min(rates), '0.01')}x" + ) + print( + " The acceptance test in #225 is that this factor means the same thing at 65,000 as at\n" + " 0.38. The spread column is that claim as a number." + ) + + +def print_all_factor_rates(after: dict[str, FactorSample]) -> None: + """Where the repaired factor's base rate now sits among the other ten.""" + pooled = pool(after.values()) + print("\nARM A2 -- the repaired factor against the rest of the CTS panel (pooled, AFTER)") + header = f"{'factor':24} {'wt':>3} {'P(present)':>11}" + print(header) + print("-" * len(header)) + for name in sorted(FACTOR_NAMES, key=lambda n: -float(_rate(pooled.vectors[n]))): + mark = " <-- repaired" if name == FACTOR else "" + print(f"{name:24} {DEFAULT_WEIGHTS[name]:3d} {_fmt(_rate(pooled.vectors[name])):>11}{mark}") + + +def _describe(totals: list[int]) -> str: + return ( + f"{statistics.mean(totals):>8.3f} {statistics.median(totals):>8.1f} " + f"{statistics.pstdev(totals):>8.3f} {min(totals):>5d} {max(totals):>5d}" + ) + + +def print_distribution( + before: dict[str, FactorSample], + after: dict[str, FactorSample], +) -> None: + """CTS TOTAL distribution shift. Fixing the factor removes a constant +1 from 3 of 5 assets.""" + print("\nARM B -- CTS total distribution, before vs after") + header = ( + f"{'asset':10} {'arm':>7} {'mean':>8} {'median':>8} {'sd':>8} {'min':>5} {'max':>5} " + f"{'d(mean)':>9}" + ) + print(header) + print("-" * len(header)) + for product_id in after: + was, now = before[product_id].totals, after[product_id].totals + delta = statistics.mean(now) - statistics.mean(was) + print(f"{product_id:10} {'before':>7} {_describe(was)} {'':>9}") + print(f"{'':10} {'after':>7} {_describe(now)} {delta:>9.3f}") + pooled_before, pooled_after = pool(before.values()), pool(after.values()) + print("-" * len(header)) + print(f"{'POOLED':10} {'before':>7} {_describe(pooled_before.totals)} {'':>9}") + print( + f"{'':10} {'after':>7} {_describe(pooled_after.totals)} " + f"{statistics.mean(pooled_after.totals) - statistics.mean(pooled_before.totals):>9.3f}" + ) + + +def print_thresholds( + before: dict[str, FactorSample], + after: dict[str, FactorSample], +) -> None: + """THRESHOLD IMPACT -- the arm that matters. How many bars change entry technique? + + `entry_technique(total, low=5, high=8)` is the only threshold comparison a CTS total feeds + in the whole package. It is a POSTURE selector, not an admission gate: a bar that drops + below `low` is not rejected, it is entered with `confirm_3bar` (3-bar confirmation, smaller + size, wider stop) instead of `signal_candle`. Nothing here rejects a setup that previously + qualified, because there is no CTS floor to fall through. + """ + print("\nARM C -- threshold impact: entry_technique bands (low=5, high=8), before vs after") + print( + " `engine.py:144` -> `indicators_cts.entry_technique(cts_result.total)` is the SOLE\n" + " CTS threshold comparison in the package. It selects posture, not admission: no\n" + " setup is rejected for a low CTS, so no previously-qualifying setup can be gated out." + ) + techniques = ("confirm_3bar", "signal_candle", "aggressive") + header = ( + f"{'asset':10} {'arm':>7} " + " ".join(f"{t:>14}" for t in techniques) + f" {'moved':>7}" + ) + print(header) + print("-" * len(header)) + for product_id in after: + was = [entry_technique(t, CTS_LOW, CTS_HIGH) for t in before[product_id].totals] + now = [entry_technique(t, CTS_LOW, CTS_HIGH) for t in after[product_id].totals] + moved = sum(1 for a, b in zip(was, now) if a != b) + n = len(now) + print(f"{product_id:10} {'before':>7} " + " ".join(f"{was.count(t):14d}" for t in techniques)) + print( + f"{'':10} {'after':>7} " + " ".join(f"{now.count(t):14d}" for t in techniques) + + f" {moved:7d}" + ) + print( + f"{'':10} {'':>7} " + " ".join( + f"{(Decimal(now.count(t) - was.count(t)) / Decimal(n)):>+13.4f} " for t in techniques + ) + + f" {Decimal(moved) / Decimal(n):>6.4f}" + ) + all_was = [entry_technique(t, CTS_LOW, CTS_HIGH) for s in before.values() for t in s.totals] + all_now = [entry_technique(t, CTS_LOW, CTS_HIGH) for s in after.values() for t in s.totals] + moved = sum(1 for a, b in zip(all_was, all_now) if a != b) + print("-" * len(header)) + print(f"{'POOLED':10} {'before':>7} " + " ".join(f"{all_was.count(t):14d}" for t in techniques)) + print( + f"{'':10} {'after':>7} " + " ".join(f"{all_now.count(t):14d}" for t in techniques) + + f" {moved:7d}" + ) + print( + f"\n bars whose entry technique changes: {moved} of {len(all_now)} " + f"({Decimal(moved) / Decimal(len(all_now)):.4f})" + ) + print(" every move is one step DOWN the ladder (a removed point cannot raise a total).") + + +def print_tolerance_ladder(candles_by_asset: dict[str, list[Candle]]) -> None: + """ARM D -- the one free parameter. P(present) over closes at four tolerances. + + Computed directly over closes rather than through the replay: `is_round_number` reads only + the entry price, so the base rate is a property of the close series alone and does not need + a context assembly per bar. Arm A's numbers are the ones that came through the real replay; + these agree with them at `tolerance=0.02` to within the 200-bar warm-up, which is the check. + """ + print("\nARM D -- tolerance sensitivity (fraction of handle spacing), over daily closes") + header = f"{'tolerance':>10} " + " ".join( + f"{a.removesuffix('-USD'):>8}" for a in candles_by_asset + ) + f" {'pooled':>8} {'spread':>7} {'64975.78':>9}" + print(header) + print("-" * len(header)) + for tolerance in TOLERANCE_LADDER: + cells, rates, hits, total = [], [], 0, 0 + for candles in candles_by_asset.values(): + closes = [c.close for c in candles] + hit = sum(1 for p in closes if levels.is_round_number(p, tolerance)) + rate = Decimal(hit) / Decimal(len(closes)) + cells.append(f"{_fmt(rate):>8}") + rates.append(rate) + hits += hit + total += len(closes) + marker = "*" if tolerance == levels.DEFAULT_HANDLE_TOLERANCE else " " + print( + f"{str(tolerance):>9}{marker} " + " ".join(cells) + + f" {_fmt(Decimal(hits) / Decimal(total)):>8}" + + f" {_fmt(max(rates) / min(rates), '0.01'):>7}" + + f" {str(levels.is_round_number(Decimal('64975.78'), tolerance)):>9}" + ) + print(" * = shipped default. The last column is the correctness case #225 names.") + + +def print_reconstruction_check( + reconstructed: dict[str, FactorSample], + replayed: dict[str, FactorSample], +) -> None: + """ARM E -- is the before-arm reconstruction exact? Replayed under the old predicate.""" + print("\nARM E -- reconstruction check: arithmetic before-arm vs a real replay of the old code") + header = f"{'asset':10} {'N':>6} {'factor vec':>12} {'CTS totals':>12} {'other factors':>15}" + print(header) + print("-" * len(header)) + ok = True + for product_id, replay in replayed.items(): + recon = reconstructed[product_id] + vec_match = recon.vectors[FACTOR] == replay.vectors[FACTOR] + tot_match = recon.totals == replay.totals + others = all( + recon.vectors[name] == replay.vectors[name] for name in FACTOR_NAMES if name != FACTOR + ) + ok = ok and vec_match and tot_match and others + print( + f"{product_id:10} {replay.n:6d} {'MATCH' if vec_match else 'DIFFER':>12} " + f"{'MATCH' if tot_match else 'DIFFER':>12} {'MATCH' if others else 'DIFFER':>15}" + ) + print("-" * len(header)) + print(f" reconstruction is {'EXACT' if ok else '*** WRONG -- do not quote arms A-C ***'}") + if not ok: + raise SystemExit(1) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--db", default=DEFAULT_DB, help=f"Candle cache (default: {DEFAULT_DB})") + parser.add_argument( + "--skip-arm-e", action="store_true", help="Skip the (slow) reconstruction replay" + ) + args = parser.parse_args() + + # `engine.evaluate` is not driven here, but `assemble_cts_context`'s callees log at INFO. + logging.disable(logging.INFO) + + print("Round-number scale fix -- issue #225") + print(f"db={args.db} assets={','.join(ASSETS)} granularity={PRIMARY_GRANULARITY.value}") + print( + f"factor={FACTOR} weight={FACTOR_WEIGHT} of {sum(DEFAULT_WEIGHTS.values())} " + f"handle tolerance={levels.DEFAULT_HANDLE_TOLERANCE}" + ) + print("READ-ONLY. The candle cache is opened `mode=ro`; no weight, gate or rule is changed.") + + candles_by_asset: dict[str, list[Candle]] = {} + after: dict[str, FactorSample] = {} + before: dict[str, FactorSample] = {} + for product_id in ASSETS: + candles = load_candles(args.db, product_id, PRIMARY_GRANULARITY) + if not candles: + print(f" {product_id}: no {PRIMARY_GRANULARITY.value} candles cached -- skipped") + continue + candles_by_asset[product_id] = candles + sample = replay_every_bar(product_id, candles, window=None) + after[product_id] = sample + before[product_id] = before_from_after( + sample, replayed_prices(candles, len(candles) - sample.n) + ) + + if not after: + print("no candles on any asset -- nothing to measure") + return + + print_presence(before, after) + print_all_factor_rates(after) + print_distribution(before, after) + print_thresholds(before, after) + print_tolerance_ladder(candles_by_asset) + + if args.skip_arm_e: + print("\nARM E skipped (--skip-arm-e): arms A-C rest on an UNVERIFIED reconstruction.") + return + replayed = { + product_id: replay_with(legacy_is_round_number, product_id, candles) + for product_id, candles in candles_by_asset.items() + } + print_reconstruction_check(before, replayed) + + # Sanity: the module under test is the shipped one, not a stale import. + assert indicators_cts.DEFAULT_WEIGHTS[FACTOR] == FACTOR_WEIGHT + + +if __name__ == "__main__": + main() diff --git a/docs/experiments/trials-ledger.jsonl b/docs/experiments/trials-ledger.jsonl index c83bfd6f..a68d0ced 100644 --- a/docs/experiments/trials-ledger.jsonl +++ b/docs/experiments/trials-ledger.jsonl @@ -73,3 +73,4 @@ {"decision":"diagnostic_only","kind":"ablation","params":{"arm":"1 - UNCONDITIONAL, every bar","assets":["BTC-USD","ETH-USD","PAXG-USD","ADA-USD","XLM-USD"],"factors_varying":10,"finding":"momentum cluster REFUTED (mean within phi -0.018 vs +0.025 background; the two RSI factors co-occur 0 times in 6822 bars). trend cluster CONFIRMED but small (+0.190). strongest pair in the matrix is deceleration x candlestick_pattern +0.254, not pre-declared. CTS total variance inflation 1.161 -> ~8.6 effective independent factors of 10.","granularity":"ONE_DAY","hypotheses_tested":45,"measurement":"pairwise phi/Jaccard/lift over CTS factor-presence vectors","multiple_testing":"Holm-Bonferroni, alpha=0.05, family = the 45 pairs actually tested","n_observations":6822,"no_code_change":"DEFAULT_WEIGHTS, factors, gates and thresholds all unchanged","p_values_load_bearing":false,"pre_declared_clusters":{"momentum":["rsi_extreme","rsi_divergence","deceleration"],"trend":["condition_aligned","ema_fan_aligned"]},"significant_pairs":23,"source_ratio_not_used":"QuantCrawler 3-of-4 agreement is unvalidated; used in no computation","window":"expanding from first cached bar (reproduces the live path exactly)"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"5e1984ea47926c00a94b4a7da578f32deea4900b7a085199b4153a6fa2fe4dce","provenance":"a_priori","row_hash":"6d9f3f9ee286b45270f4b3527b6a54f7981f8e3915cb81d5f080619412fe5b8a","rule":"n/a (factor-level, not rule-level)","series_missing":true,"session":"cts-factor-collinearity-2026-08-09","summary":{"max_abs_phi_any_pair":"0.254","mean_cts_total":"5.14","mean_other_phi":"0.025","mean_within_phi_momentum":"-0.018","mean_within_phi_trend":"0.190","variance_ratio":"1.161"},"timestamp":1786446356,"trial_id":"cts-factor-collinearity-unconditional-daily-2026-08-09"} {"decision":"diagnostic_only","kind":"ablation","params":{"arm":"2 - UNCONDITIONAL, every bar","assets":["BTC-USD","ETH-USD","PAXG-USD","ADA-USD","XLM-USD"],"factors_varying":9,"finding":"reproduces arm 1: momentum -0.004, trend +0.201, strongest pair still deceleration x candlestick_pattern +0.257. sr_touches is CONSTANT (0.000) under a 500-bar hourly window and was dropped from the family.","granularity":"ONE_HOUR","hypotheses_tested":36,"measurement":"replication of arm 1 at 27x the sample","multiple_testing":"Holm-Bonferroni, alpha=0.05; at this N significance is uninformative","n_observations":186725,"window":"rolling 500 bars (expanding is O(n^2) over 44k bars)"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"6d9f3f9ee286b45270f4b3527b6a54f7981f8e3915cb81d5f080619412fe5b8a","provenance":"a_priori","row_hash":"40756b17a5dbc00630dfb5fa815ee03e8d5532acaa2ab09f97c9595e6a1783e4","rule":"n/a (factor-level, not rule-level)","series_missing":true,"session":"cts-factor-collinearity-2026-08-09","summary":{"mean_other_phi":"0.009","mean_within_phi_momentum":"-0.004","mean_within_phi_trend":"0.201","variance_ratio":"1.089"},"timestamp":1786446356,"trial_id":"cts-factor-collinearity-unconditional-hourly-2026-08-09"} {"decision":"diagnostic_only","kind":"ablation","params":{"arm":"4 - CONDITIONAL (collider; reported for contrast, never as the headline)","assets":["BTC-USD","ETH-USD","PAXG-USD","ADA-USD","XLM-USD"],"finding":"demonstrates the selection effect it was run to demonstrate: P(condition_aligned) 0.275 -> 0.818, P(ema_fan_aligned) 0.301 -> 0.909, and the variance ratio falls BELOW 1 (0.663) because conditioning truncates rather than decorrelates.","granularity":"ONE_DAY","hypotheses_tested":45,"measurement":"the SAME matrix conditioned on a fired, gate-cleared signal","n_observations":77,"rule":"turtle_breakout 40/20 ATR(20) 2N rr6 ADX(14)>25 (shipped, keel-live rules 1-5)","significant_pairs":3,"underpowered":"N=77 estimates nothing on its own","window":"expanding"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"40756b17a5dbc00630dfb5fa815ee03e8d5532acaa2ab09f97c9595e6a1783e4","provenance":"a_priori","row_hash":"4b196d5369087d0e8e689d9138e6a9a9d186daf1369e29158ac1a215d1221e2b","rule":"n/a (factor-level, not rule-level)","series_missing":true,"session":"cts-factor-collinearity-2026-08-09","summary":{"mean_within_phi_momentum":"-0.047","mean_within_phi_trend":"0.202","variance_ratio":"0.663"},"timestamp":1786446356,"trial_id":"cts-factor-collinearity-conditional-fired-2026-08-09"} +{"decision":"selected","kind":"ablation","params":{"assets":["BTC-USD","ETH-USD","PAXG-USD","ADA-USD","XLM-USD"],"before_arm":"reconstructed arithmetically, then VERIFIED bar-for-bar against a full replay with the pre-#225 predicate monkeypatched in -- arm E reports EXACT on all five assets, including every other factor's vector","change":"levels.is_round_number: absolute step=Decimal('0.005') -> two-significant-figure handle grid with tolerance as a fraction of that spacing","definition":"spacing = 10 ** (floor(log10(price)) - 1); present iff dist_to_nearest_multiple <= spacing * tolerance","entry_technique_bands_after":{"aggressive":510,"confirm_3bar":3416,"signal_candle":2901},"entry_technique_bands_before":{"aggressive":893,"confirm_3bar":2706,"signal_candle":3228},"finding":"P(present) 1.0000/1.0000/1.0000 on BTC/ETH/PAXG -> 0.0358/0.0334/0.0463; ADA 0.2170 -> 0.0465, XLM 0.1901 -> 0.0440. Cross-asset spread 5.26x -> 1.39x. Pooled CTS mean 5.145 -> 4.566, median 5 -> 4; the BTC-vs-XLM mean gap falls 1.44 -> 0.62, so 57% of it was this bug. 1103 of 6827 bars (16.2%) change entry_technique, every one a rung down, aggressive -43% pooled. NO ORDER CHANGES: entry_technique's three return values appear nowhere outside indicators_cts.py, and no config key, entry gate or promotion gate reads a CTS score at all -- promotion.can_promote runs off backtested trade statistics. The graded entry ladder is computed, persisted and inert, which is why this fix is safe to ship and is itself flagged as a follow-up.","granularity":"ONE_DAY","harness":"keel/research/cts_factors.py, reused unchanged from #208/#224","issue":"#225","measurement":"P(factor present), CTS total distribution and entry_technique band counts, before vs after on the SAME bars","n_observations":6827,"p_present_after":{"ADA-USD":"0.0465","BTC-USD":"0.0358","ETH-USD":"0.0334","PAXG-USD":"0.0463","XLM-USD":"0.0440","pooled":"0.0401"},"p_present_before":{"ADA-USD":"0.2170","BTC-USD":"1.0000","ETH-USD":"1.0000","PAXG-USD":"1.0000","XLM-USD":"0.1901","pooled":"0.6183"},"quote_increment_rejected":"would put venue state into a pure analysis primitive and break the offline replay #224 depends on","reconstruction_check":"EXACT on all five assets","sig_figs_rejected":"3 -- pushes the grid onto ADA/XLM quote increment; tick quantization alone lifts ADA to 0.2996 against BTC 0.1861","sig_figs_selected":2,"thresholds_retuned":"none -- entry_technique low=5/high=8 untouched","tolerance_ladder_P_present_pooled":{"0.01":"0.0210","0.02":"0.0397","0.05":"0.0998","0.10":"0.2033"},"tolerance_pinned_by":"#225 requires is_round_number(64975.78) is False, capping tolerance below 0.0242; 0.10 (the pre-fix docstring's stated intent, and the tightest cross-asset spread at 1.10x) scores it present and is recorded as an open question, not adopted","tolerance_selected":"0.02","window":"expanding from first cached bar (reproduces the live path exactly)"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"4b196d5369087d0e8e689d9138e6a9a9d186daf1369e29158ac1a215d1221e2b","provenance":"fitted","row_hash":"e2d1661bcd8f40a8ac95fcaef9893855bd93665c0ad93c880fdd419120260fd0","rule":"n/a (factor-level, not rule-level)","series_missing":true,"session":"round-number-scale-2026-08-11","summary":{"aggressive_band_after":510,"aggressive_band_before":893,"bars_changing_entry_technique":1103,"bars_total":6827,"cross_asset_spread_after":"1.39","cross_asset_spread_before":"5.26","cts_mean_after":"4.566","cts_mean_before":"5.145","cts_mean_gap_btc_xlm_after":"0.617","cts_mean_gap_btc_xlm_before":"1.436","cts_median_after":"4.0","cts_median_before":"5.0","fraction_changing_entry_technique":"0.1616","orders_changed":0,"p_present_pooled_after":"0.0401","p_present_pooled_before":"0.6183"},"timestamp":1786448907,"trial_id":"round-number-scale-2026-08-11"} diff --git a/keel/analysis/levels.py b/keel/analysis/levels.py index 296e5ed9..5369ef99 100644 --- a/keel/analysis/levels.py +++ b/keel/analysis/levels.py @@ -134,13 +134,84 @@ def find_levels( return [level for level in levels if level.touches >= min_touches] -def is_round_number(price: Decimal, step: Decimal = Decimal("0.005")) -> bool: - """True if `price` is close to a multiple of `step` (an "even handle"), within - 10% of the step size. +#: Significant figures that define a round handle. Two is not a tuning knob, it is what the +#: words mean: 65,000 and 0.38 are handles, 65,100 and 0.381 are not, and both pairs stand in +#: exactly the same relation to their own price. Three would push the grid down onto ADA's and +#: XLM's quote increment -- measured, it lifts ADA's presence rate to 0.30 against BTC's 0.19 +#: purely from tick quantization, which is the same class of artifact this function is being +#: fixed to remove. +_HANDLE_SIGNIFICANT_FIGURES = 2 + +#: Half-width of the "at the handle" band, as a fraction of the SPACING BETWEEN HANDLES (not of +#: price). See `is_round_number` for why that denominator is the load-bearing choice. +DEFAULT_HANDLE_TOLERANCE = Decimal("0.02") + + +def _handle_spacing(price: Decimal) -> Decimal: + """Distance between adjacent round handles at `price`'s own order of magnitude. + + `10 ** (floor(log10(price)) - 1)`, computed from `Decimal.adjusted()` so it is exact + integer arithmetic on the exponent -- no float `log10`, which would misplace the grid for + prices sitting a few ulps under a power of ten. Returns a spacing such that + `price / spacing` always lands in `[10, 100)`, i.e. the handles are the two-significant- + figure prices: 1,000 at BTC's 65,000; 100 at ETH's 3,400; 0.01 at ADA's 0.38. """ - remainder = price % step - distance = min(remainder, step - remainder) - return distance <= step * Decimal("0.1") + return Decimal((0, (1,), price.adjusted() - (_HANDLE_SIGNIFICANT_FIGURES - 1))) + + +def is_round_number(price: Decimal, tolerance: Decimal = DEFAULT_HANDLE_TOLERANCE) -> bool: + """True if `price` sits at a psychological round handle for its own order of magnitude. + + A magnet level is a number enough people are watching that orders pile up on it, and what + makes a number watchable is having few significant figures — 65,000, 3,400, 0.38. That is a + property of the price *relative to its own scale*, so the handle grid has to be derived + from the price and cannot be a constant. + + ⚠️ **This function used to take an absolute `step=Decimal("0.005")` and it could not fail + (issue #225).** Coinbase quotes BTC, ETH and PAXG to two decimals, and every 2dp value is an + exact multiple of half a cent because `0.01 = 2 * 0.005`. So `price % step` was always + exactly zero and the answer was always `True`: measured over the daily history in the candle + cache, P(present) was **1.0000 on BTC-USD, ETH-USD and PAXG-USD** against 0.217 on ADA-USD + and 0.190 on XLM-USD. Because `round_number_proximity` is weight 1 of `DEFAULT_WEIGHTS`' + 14, three of the five live allowlist assets were carrying an unconditional +1 on every CTS + score — a constant, which is worse than a redundant factor, because a redundant factor at + least varies. It also broke cross-asset comparability of the total: any threshold read + against CTS meant something different on BTC than on XLM by exactly one point. + + **`tolerance` is a fraction of the handle SPACING, not of price, and the denominator is the + whole argument.** Both denominators scale with the instrument, so both fix the bug; they + differ in what they hold constant. A fraction-of-price band makes the presence rate depend + on where in the decade the price happens to sit — the spacing is 10% of price just above a + power of ten and 1% of it just below, so the same rule would fire ten times as often on BTC + at 99,000 as at 10,500, and the factor would silently change meaning as an asset trended + through a decade. A fraction-of-spacing band makes P(present) identically `2 * tolerance` + for any price series that is smooth on the scale of the grid, which is precisely the + property #225 asks for: the factor must mean the same thing at 65,000 as at 0.38. Measured + on the same daily history, it does — 0.037 / 0.036 / 0.041 / 0.044 / 0.043 across the five + allowlist assets, a 1.22x spread where the old code's was 5.3x. + + The default `0.02` puts the band at +/- 2% of the gap to the next handle (+/- 20 dollars on + BTC's 1,000-wide grid), giving a ~4% presence rate — rarer than `candlestick_pattern` + (0.203) and commoner than `rsi_extreme` (0.023), so it sits inside the existing spread of + CTS factor base rates rather than dominating or vanishing. It is deliberately tighter than + the 10% the original docstring claimed: at 10% the band on BTC is +/- 100 and 64,975.78 + scores present, which #225 names as a case that must score absent. That is the one genuinely + free parameter here and the write-up carries the sensitivity ladder for it. + + Degenerate inputs score absent rather than raising. Zero has no order of magnitude (and + `_handle_spacing` would hand back a meaningless grid), negatives are not prices, and + NaN/Infinity have no `adjusted()` worth trusting — none of them is a round handle, and a + pure predicate on the live scoring path should not be able to throw. Extreme magnitudes are + safe without a guard: `price / spacing` is always in `[10, 100)`, so the `%` below never + needs more than two digits of integer quotient and cannot trip the Decimal context. + """ + if not price.is_finite() or price <= 0: + return False + + spacing = _handle_spacing(price) + remainder = price % spacing + distance = min(remainder, spacing - remainder) + return distance <= spacing * tolerance def role_reversed( diff --git a/tests/analysis/test_levels.py b/tests/analysis/test_levels.py index db62f6da..6facbc5e 100644 --- a/tests/analysis/test_levels.py +++ b/tests/analysis/test_levels.py @@ -175,17 +175,97 @@ def test_find_levels_includes_level_touched_twice_when_min_touches_two(): assert support_levels[0].touches == 2 -def test_is_round_number_true_on_even_handle(): - assert is_round_number(Decimal("1.10000"), step=Decimal("0.005")) is True +def test_is_round_number_false_on_two_decimal_price_away_from_a_handle(): + """The #225 regression: every 2dp price is an exact multiple of the old absolute + `step=0.005`, so the check could never fail on BTC/ETH/PAXG and handed those three + assets a constant +1 CTS point. 64,975.78 is a quarter of the way into a 1,000-wide + BTC handle interval and must score absent. + """ + assert is_round_number(Decimal("64975.78")) is False + + +def test_is_round_number_false_across_the_two_decimal_allowlist_scales(): + """No 2dp-quoted price may score present merely for being quoted to 2dp. + + Every value here is an exact multiple of the old absolute `step=0.005` and so returned + `True` before #225. Note the asymmetry that falls out of a relative grid and is correct: + below 1.00 the two-decimal quote grid IS the two-significant-figure handle grid, so `0.01` + and `0.38` are genuine handles and must keep scoring present. The bug was never "2dp + prices"; it was that an absolute step cannot see scale at all. + """ + for price in ("64975.78", "103412.99", "3421.07", "4127.53", "12.34", "1.23"): + assert is_round_number(Decimal(price)) is False, price + for handle in ("0.01", "0.38"): + assert is_round_number(Decimal(handle)) is True, handle + + +def test_is_round_number_true_on_even_handle_at_every_scale(): + """The same relative position on the handle grid scores the same at 65,000 and at 0.38.""" + for price in ("65000", "65000.00", "3400.00", "0.38", "0.0071", "1.10"): + assert is_round_number(Decimal(price)) is True, price + + +def test_is_round_number_is_scale_invariant(): + """Multiplying by a power of ten moves the grid with the price, so the answer is + unchanged -- the property the absolute-step version could not have. + """ + for digits in ("64975.78", "65000", "38200", "1", "7.5"): + base = Decimal(digits) + expected = is_round_number(base) + for exponent in (-8, -3, 3, 8): + assert is_round_number(base.scaleb(exponent)) is expected, f"{digits}e{exponent}" -def test_is_round_number_false_off_handle(): - assert is_round_number(Decimal("1.10237"), step=Decimal("0.005")) is False +def test_is_round_number_near_a_handle_at_small_scale(): + """ADA/XLM scale: 0.3801 is inside 0.38's band, 0.3835 is not.""" + assert is_round_number(Decimal("0.3801")) is True + assert is_round_number(Decimal("0.3835")) is False + assert is_round_number(Decimal("0.070008")) is True + assert is_round_number(Decimal("0.073451")) is False -def test_is_round_number_default_step(): - assert is_round_number(Decimal("100.000")) is True - assert is_round_number(Decimal("100.00317")) is False +def test_is_round_number_boundary_is_inclusive(): + """`tolerance` is a fraction of the handle spacing; at exactly the band edge the + factor is present, one ulp outside it is not. + """ + # 65,000 sits on a 1,000-wide grid, so the default 0.02 band is +/- 20. + assert is_round_number(Decimal("65020")) is True + assert is_round_number(Decimal("65020.01")) is False + assert is_round_number(Decimal("64980")) is True + assert is_round_number(Decimal("64979.99")) is False + + +def test_is_round_number_tolerance_is_relative_not_absolute(): + """A wider `tolerance` widens the band in proportion to the grid, at every scale.""" + assert is_round_number(Decimal("64975.78"), tolerance=Decimal("0.1")) is True + assert is_round_number(Decimal("0.3897"), tolerance=Decimal("0.1")) is True + assert is_round_number(Decimal("64975.78"), tolerance=Decimal("0")) is False + assert is_round_number(Decimal("65000"), tolerance=Decimal("0")) is True + + +def test_is_round_number_degenerate_prices_do_not_divide_by_zero(): + """Zero has no order of magnitude and negatives are not prices: both score absent + rather than raising or taking a modulo by zero. + """ + assert is_round_number(Decimal("0")) is False + assert is_round_number(Decimal("-0")) is False + assert is_round_number(Decimal("-65000")) is False + assert is_round_number(Decimal("NaN")) is False + assert is_round_number(Decimal("Infinity")) is False + + +def test_is_round_number_survives_extreme_magnitudes(): + """No overflow, no context error, no modulo-by-zero at either end of the range. + + `3.7e28` and `3.7e-28` ARE handles (two significant figures); `3.75e28` is not. The point + of the pair is that the grid tracks the exponent for 60 orders of magnitude. + """ + assert is_round_number(Decimal("1E-30")) is True + assert is_round_number(Decimal("3.7E-28")) is True + assert is_round_number(Decimal("3.75E-28")) is False + assert is_round_number(Decimal("1E+30")) is True + assert is_round_number(Decimal("3.7E+28")) is True + assert is_round_number(Decimal("3.75E+28")) is False def test_role_reversed_true_when_prior_resistance_now_holds_as_support(): diff --git a/tests/strategy/test_engine.py b/tests/strategy/test_engine.py index 4672d72d..192188bc 100644 --- a/tests/strategy/test_engine.py +++ b/tests/strategy/test_engine.py @@ -13,11 +13,13 @@ import json import logging +from dataclasses import replace from decimal import Decimal from keel.data.db import connect, migrate from keel.data.repository import Repository -from keel.strategy.engine import DEFAULT_RR_FLOOR, evaluate +from keel.strategy import indicators_cts +from keel.strategy.engine import DEFAULT_RR_FLOOR, assemble_cts_context, evaluate from keel.strategy.rules.base import Action, Rule, Setup from keel.strategy.rules.dca import Dca from keel.strategy.rules.pullback_continuation import PullbackContinuation @@ -155,19 +157,48 @@ def test_high_cts_yields_aggressive_enter_signal(self) -> None: assert signal.setup is not None assert signal.setup.direction == "long" - def test_default_weights_on_same_fixture_yields_signal_candle_tier(self) -> None: + def test_default_weights_on_same_fixture_yields_confirm_3bar_tier(self) -> None: # Sanity check against the *default* CTS weights (spec §9): this fixture earns - # condition_aligned(2) + in_pullback(1) + round_number_proximity(1) + - # candlestick_pattern(1) = 5, which lands in the mid ("signal_candle") tier, not - # "aggressive" -- confirming the engine doesn't silently inflate the real score. + # condition_aligned(2) + in_pullback(1) + candlestick_pattern(1) = 4, which lands in + # the low ("confirm_3bar") tier -- confirming the engine doesn't silently inflate the + # real score. + # + # ⚠️ This expectation was 5 / "signal_candle" before #225, and the missing point is + # `round_number_proximity`. The fixture enters at 128.02, which is 2.02 away from the + # nearest round handle (130, on a 10-wide grid at that magnitude) -- it is not near a + # magnet level and never was. It scored present only because the old + # `levels.is_round_number` compared against an ABSOLUTE `step=Decimal("0.005")` and + # 128.02 is an exact multiple of half a cent, as every 2dp price is. So this test is + # also the smallest end-to-end demonstration of the bug's consequence: removing one + # spurious point moved this setup across `entry_technique`'s `low=5` edge and down a + # posture rung. The companion test below shows the point is still earned when the + # entry genuinely sits on a handle. rule = _pullback_rule() candles_by_tf = {Granularity.ONE_HOUR: _bullish_pullback_candles()} signals = evaluate(rules=[rule], candles_by_tf=candles_by_tf) assert len(signals) == 1 - assert signals[0].cts_score == 5 - assert signals[0].entry_technique == "signal_candle" + assert signals[0].cts_score == 4 + assert signals[0].entry_technique == "confirm_3bar" + + def test_round_number_point_is_earned_when_the_entry_sits_on_a_handle(self) -> None: + # The other half of #225: the factor must still fire when it should. Same fixture, + # same everything, with the entry nudged onto the 130 handle -- the point comes back + # and the tier returns to "signal_candle". Asserted through `assemble_cts_context` + # rather than `evaluate` because the entry is a function of the fixture's candles and + # cannot be set independently through the rule. + rule = _pullback_rule() + candles = _bullish_pullback_candles() + setup = rule.detect({Granularity.ONE_HOUR: candles}) + assert setup is not None + + off_handle = assemble_cts_context(setup, candles) + on_handle = assemble_cts_context(replace(setup, entry=Decimal("130.00")), candles) + + assert off_handle["round_number_proximity"] is False + assert on_handle["round_number_proximity"] is True + assert indicators_cts.score(on_handle).total == indicators_cts.score(off_handle).total + 1 class TestLowCtsConfirm3Bar: