From ea609a939a3dbb38abbffb4620dbdcb0bbca9a4e Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Wed, 12 Aug 2026 13:56:08 -0400 Subject: [PATCH] =?UTF-8?q?docs(experiments):=20rsi=5Fmeanrev's=20edge=20i?= =?UTF-8?q?s=20selectivity,=20not=20alpha=20=E2=80=94=20the=20last=20live?= =?UTF-8?q?=20lead=20closes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit #252 left one open question: does rsi_meanrev's gross edge survive being made to fire more often? It does not. The rule reaches the promotion floor trivially — 21 of 24 assets at oversold=40 — and has no edge when it gets there. gross PF at oversold=20 (median n=38) : median 1.1631 gross>1 14/24 gross PF at every cell with n>=100 : median 0.8938 gross>1 11/76 net > 1.0 at n>=100, at ANY fee : 0 of 76, including maker The edge evaporates exactly when the rule becomes measurable. The 1.1631 that made this look like the best of the three rules IS what n=38 looks like. Pre-registration lives in the script docstring this time, which is the correction to the defect #252 recorded against itself. The declared statistic — a linear slope — is reported as declared (-0.0386/100 trades, 15/24 negative) AND flagged as underpowered: the phenomenon is a threshold effect at the floor, and two barely-trading assets supply most of its variance. Investigating a 34x non-monotonic trade count on UNI-USD found a simulator defect: backtest() never expires a pending setup, so a strategy whose entry is never revisited freezes silently for the rest of the series. Filed as #254, deliberately not fixed here. #252's headline results were checked against it and are clean. Closes #253. Co-Authored-By: Claude Opus 5 (1M context) --- ...-08-12-rsi-meanrev-scale-vs-selectivity.md | 241 +++++++++++++++++ ...-08-12-rsi-meanrev-scale-vs-selectivity.py | 253 ++++++++++++++++++ docs/experiments/trials-ledger.jsonl | 1 + 3 files changed, 495 insertions(+) create mode 100644 docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.md create mode 100644 docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.py diff --git a/docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.md b/docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.md new file mode 100644 index 00000000..754bb78a --- /dev/null +++ b/docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.md @@ -0,0 +1,241 @@ +# `rsi_meanrev`'s edge is selectivity, not alpha — and the search for it found a simulator defect + +**Date:** 2026-08-12 +**Issue:** #253 — closes the single open question left by #252 +**Change:** documentation only. No code, no config, no rule status, no version bump. The simulator +defect found in §5 is filed separately (#254) and deliberately **not** fixed here: a research PR +should not carry a change to the engine every prior experiment was measured on. +**Script:** `docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.py` — **pre-registration +lives in its docstring, written before the run.** That is the correction to the defect §7 of #252 +recorded against itself, where the declaration lived only in a dispatch brief. +**Ledger:** one row, `rsi-meanrev-scale-vs-selectivity-2026-08-12`. +**Deployment:** `keel 0.7.0`. Fees and slippage passed explicitly on every call. + +**Verdict: `rsi_meanrev` can be made to clear the promotion floor trivially — 21 of 24 assets — and +it has no edge when it gets there. The rule is unpromotable by construction, and the last live lead +in the shipped strategy library closes negatively.** + +| question | answer | +|---|---| +| did the primary arm reach `n≥100`? | **yes, 21/24** — the conditional proximity arm never fired | +| gross PF at defaults (median n=38) | median **1.1631**, 14/24 gross-positive | +| gross PF at every cell with `n≥100` | median **0.8938**, 11/76 gross-positive | +| net PF > 1.0 at `n≥100`, any fee | **0 of 76** — including maker | +| pre-registered slope | **−0.0386** per +100 trades (median −0.0197), 15/24 negative | +| was the declared statistic the right one? | **no** — see §4, it was underpowered for its own question | +| anything else? | a **pending setup never expires**, silently freezing a strategy for the rest of a series (§5) | + +--- + +## 1. The question #252 left open + +`2026-08-12-shipped-defaults-intersection.md` measured all three shipped signal rules at their +constructor defaults across 24 assets. `rsi_meanrev` came out of it with the best gross-edge +distribution of the three — median gross PF **1.1631** against `turtle`'s 0.9892 and `pullback`'s +0.9292 — while reaching `min_trades=100` on **zero** assets, median n=38. + +Two readings, opposite consequences, and no data separating them: + +- **(a) the edge is real** and the defaults are over-constrained, in which case relaxing them + reaches the floor with the edge intact and this is the only promotable rule in the codebase; +- **(b) the edge is an artifact of selectivity** — the rule looks good because it only fires on + rare, easy setups, and buying trades means accepting worse ones. + +#252 called this "the one live lead" and "the only route by which any rule the codebase ships +reaches its own promotion floor honestly." + +## 2. Design + +A **monotonicity test, not a search.** The declared statistic was the *slope* of gross profit +factor against n, per asset then averaged — chosen because with 120 cells a maximum is guaranteed +and a slope is not. The docstring says it outright: *"The best cell is never reported as a result."* + +**Variable axis, one only:** `oversold ∈ {20, 25, 30, 35, 40}`. + +`oversold` is the entire frequency mechanism, and that was measured rather than assumed — #248's +108-cell diagnostic found `oversold` 25→30 multiplied trade count ×2.18 and 30→35 by ×3.93, against +×1.186 for `support_proximity_pct` and ×1.185 for `level_min_touches`. + +**Held fixed:** `overbought=80`, `support_proximity_pct=0.005`, everything else at defaults. +`overbought` is held not because it is a weak lever but because it is the **wrong kind** — it +governs the exit side, so moving it changes trade *outcomes* and not merely trade *counts*, and the +slope would stop being interpretable. + +**Anchor:** `oversold=20` is the shipped default, already measured across all 24 assets by #252. +Those rows are reused rather than recomputed — identical in every other parameter and in cost +treatment — so this grid ran 4 new levels × 24 assets = **96 combinations**, 3h36m on 8 workers. + +**Conditional arm, declared before any data existed:** if fewer than 8 of 24 assets reached +`n≥100` at `oversold=40`, widen `support_proximity_pct ∈ {0.005, 0.02, 0.05}` and report it as a +**separate curve, never pooled**. Writing the trigger and the reporting rule down in advance is +what stops a widening from being invented later to rescue a disappointing primary arm. + +**It did not fire.** 21 of 24 assets reached `n≥100` at `oversold=40`. Clean single-axis answer. + +## 3. The result + +``` +gross PF at oversold=20 (median n=38) : median 1.1631 gross>1: 14/24 +gross PF at every cell with n>=100 : median 0.8938 gross>1: 11/76 +net > 1.0 at n>=100, at ANY fee : 0 of 76 (including 0.6% maker) +``` + +**The edge evaporates exactly when the rule becomes measurable.** Not gradually as trades +accumulate — as a level shift across the floor. The 1.1631 that made this rule look like the best +of the three *is* what n=38 looks like. + +The per-asset curves show the same shape almost everywhere: a high, noisy profit factor at +`oversold=20` on a few dozen trades, collapsing to ~0.9 by `oversold=25` and staying flat +thereafter. + +``` +asset 20 25 30 35 40 + n grossPF n grossPF n grossPF n grossPF n grossPF +ADA-USD 41 1.746 128 0.934 309 0.925 600 0.878 875 0.879 +FET-USD 26 1.652 94 0.993 240 0.824 496 0.815 623 0.867 +ICP-USD 22 1.566 81 0.603 235 0.783 505 0.993 684 0.957 +XLM-USD 39 1.484 124 0.904 298 0.874 571 0.876 821 0.813 +DOGE-USD 64 1.417 164 1.001 366 0.989 652 0.921 925 0.897 +PAXG-USD 12 2.877 32 1.476 70 0.913 104 0.962 145 0.908 +``` + +For completeness rather than selection — the declaration forbids reporting a best cell — **all 11 +gross-positive cells at `n≥100`**, and what each does once costs are charged: + +``` +ALGO-USD oversold=25 n= 112 gross=1.215 maker=0.678 taker=0.395 +ZEC-USD oversold=40 n= 464 gross=1.210 maker=0.645 taker=0.345 +AAVE-USD oversold=25 n= 160 gross=1.179 maker=0.645 taker=0.360 +ZEC-USD oversold=30 n= 238 gross=1.086 maker=0.581 taker=0.307 +BTC-USD oversold=40 n= 118 gross=1.072 maker=0.360 taker=0.110 +AVAX-USD oversold=25 n= 142 gross=1.045 maker=0.550 taker=0.291 +ZEC-USD oversold=35 n= 450 gross=1.031 maker=0.542 taker=0.283 +BCH-USD oversold=25 n= 160 gross=1.031 maker=0.491 taker=0.242 +AVAX-USD oversold=35 n= 583 gross=1.026 maker=0.509 taker=0.256 +CRV-USD oversold=35 n= 460 gross=1.013 maker=0.561 taker=0.315 +DOGE-USD oversold=25 n= 164 gross=1.001 maker=0.533 taker=0.293 +``` + +The best of them is 1.215 gross and 0.678 at maker. There is no cost structure reachable from here +that makes any of these viable. + +**Hypothesis (b), confirmed.** The rule is unpromotable by construction: no parameter choice +escapes buying volume with quality, because the quality was never there at volume. + +## 4. The pre-registered statistic was underpowered for its own question + +Reported as declared, because that is what pre-registration is for: + +``` +slope of gross PF per +100 trades, across 24 assets + mean -0.0386 median -0.0197 stdev 0.3487 + negative 15/24 positive 9/24 + min -1.2172 (PAXG-USD) max +1.0626 (WLD-USD) +``` + +Directionally right, and the sign is the one the prediction attached to hypothesis (b). But it is a +weak instrument for what turned out to be happening, and the write-up should say so rather than +present a marginal number as a clean one: + +1. **The relationship is not linear.** Fitting a straight line through five points whose leftmost + is n≈38 and whose rightmost is n≈800 mostly measures the leftmost point. The phenomenon is a + threshold effect at the floor, which a slope smears out. +2. **The spread is contaminated by assets that barely trade.** PAXG-USD (n 12→145) and WLD-USD + (n 1→71) supply the −1.22 and +1.06 extremes and most of the 0.35 standard deviation. Their + "slopes" are fitted through noise. + +The partition on `n≥100` in §3 is the decisive reading, and it is not a post-hoc convenience: the +floor was pre-declared as C1 in #252 and is the criterion the promotion gate actually applies. What +is fair to say against ourselves is that this partition **should have been the declared primary +statistic**, and the fact that n varies twentyfold across the grid was knowable in advance. + +Recording this because the alternative — quietly reporting the stronger analysis and omitting that +it was not the declared one — is the exact failure the pre-registration exists to prevent. + +## 5. The design's monotonicity assumption is violated, and the cause is a simulator defect + +`oversold` does not always increase firing rate. Three of 24 assets go backwards: + +``` +UNI-USD 56@20 135@25 309@30 9@35 15@40 <-- 309 -> 9 +BTC-USD 37@20 98@25 181@30 80@35 118@40 +AAVE-USD 40@20 160@25 324@30 582@35 524@40 +``` + +UNI's collapse is 34×, and it is not a rule behaviour: + +``` +oversold=30: closed=309 open=1 last exit 2026-08-04 (trades throughout) +oversold=35: closed=9 open=0 last exit 2021-11-15 (dead for 4.7 years) +``` + +No open position, so nothing is stuck in a trade. The rule stopped *detecting* for ~40,000 +consecutive bars. The cause is in `backtest()`: + +```python +if position is None and pending is None: + pending = rule.detect(candles_by_tf) + continue + +if position is None and pending is not None: + entry_touched = _touches(candle, pending.entry) + stop_touched = _touches(candle, pending.stop) + if not entry_touched: + continue # pending persists +``` + +**A pending setup has no expiry.** The only path that clears it without a fill is the stop being +touched first and `_resolve_order` confirming the stop preceded entry. If price drifts away so that +*neither* entry nor stop is touched again, `pending` stays set forever, the +`position is None and pending is None` branch never runs again, and **`rule.detect()` is never +called for the remainder of the series**. + +The strategy freezes silently. The output is indistinguishable from "the rule found no more +setups", which is precisely how it survived unnoticed: a frozen backtest looks like a selective one. + +This matters beyond the anomaly: + +- It is **rule-agnostic and study-wide**. Every backtest in #245–#252 ran on this code. +- It biases *toward* low trade counts, i.e. toward exactly the "unmeasurable" verdicts these + documents have been issuing. +- A real deployment does not behave this way: `keel`'s live path re-detects each cycle, so this is + a simulator-only divergence from production semantics — which makes it a fidelity bug, not just + a performance one. + +**The headline results of #252 were checked against it and are clean.** The three `turtle` assets +that missed the trade floor are not frozen — each trades to within days of its series end, and +their low counts are genuinely short history: + +``` +asset n last exit series end dead tail +PAXG-USD 61 2026-08-11 2026-08-12 1d (history starts 2025-05) +WLD-USD 45 2026-07-21 2026-07-23 2d (starts 2025-04) +TON-USD 29 2026-07-22 2026-07-23 1d (starts 2025-11) +ZEC-USD 250 2026-07-16 2026-07-23 7d +BTC-USD 274 2026-07-27 2026-08-12 16d +``` + +So #252 stands as written. What is **not** established is the defect's blast radius across the +lower-n cells of every prior sweep, and quantifying that means re-running them — filed as #254 +rather than guessed at here. + +## 6. What this closes, and what it leaves + +**Closed.** `rsi_meanrev` was the last rule with an untested route to the promotion floor. It +reaches the floor easily and has no edge there. Combined with #252, every signal rule the codebase +ships has now been measured at defaults across 24 assets and along its own frequency axis, and +there is no asset-rule-parameter combination that is simultaneously measurable and profitable at +any reachable fee. + +**Left open, in priority order:** + +1. **#254 — the pending-expiry defect.** A correctness issue in the instrument every one of these + conclusions was produced with. It should be fixed and the affected sweeps re-run before any + further strategy work, because until then every low trade count in this directory carries an + asterisk. +2. **The PBO/CSCV gate is deployed and unfed.** #247 wired `g4_pbo_gate` into `can_promote` where + `pbo=None` blocks. Nothing supplies it a trial matrix. That is the principled instrument for + the overfitting questions these documents keep answering by hand. +3. **Nothing else here justifies engineering investment.** In particular the maker-execution and + queue-simulation work has no target: #252 removed its only candidate, and this document removes + the rule that might have supplied another. diff --git a/docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.py b/docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.py new file mode 100644 index 00000000..b62a2723 --- /dev/null +++ b/docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.py @@ -0,0 +1,253 @@ +"""Does `rsi_meanrev`'s gross edge survive being made to fire more often? + +PRE-REGISTERED IN THIS FILE, BEFORE THE RUN. This docstring is the declaration -- unlike +`2026-08-12-shipped-defaults-intersection.py`, whose declaration lived in a dispatch brief and had +to be reconstructed afterwards. That was recorded as a defect; this is the correction. + +## The question, and why it is not an optimisation + +`2026-08-12-shipped-defaults-intersection.md` measured all three shipped rules at their +constructor defaults and found `rsi_meanrev` has the BEST gross-edge distribution of the three +(median gross PF 1.1631, 58% of assets gross-positive) while reaching the promotion floor on ZERO +of 24 assets (median n=38 against min_trades=100). It does not lose. It is not observable. + +Two explanations, with opposite consequences, and no data yet separates them: + + (a) the edge is REAL and the defaults are simply over-constrained -- in which case relaxing + them reaches n>=100 with the edge intact, and this is the only promotable rule in the + codebase; + (b) the edge is an ARTIFACT OF SELECTIVITY -- the rule looks good precisely because it only + fires on the rare, easy setups, and buying trades means accepting worse ones. + +## The design, and why it cannot cherry-pick + +This is a MONOTONICITY TEST, not a search. The reported statistic is the SLOPE of gross profit +factor against n, computed per asset across the oversold levels, then averaged across the 24 +assets. **The best cell is never reported as a result.** With 120 cells a maximum is guaranteed; +a slope is not, and a slope cannot be manufactured by trying more cells. + +PRE-REGISTERED PREDICTION, recorded so the result can contradict it: + + slope < 0 -> hypothesis (b). The rule is unpromotable BY CONSTRUCTION: no parameter choice + escapes buying volume with quality. Report and retire the line of enquiry. + slope >= 0 -> hypothesis (a). The edge is scalable and the defaults are the problem. This is + the single outcome in the whole study that points at a promotable rule, and it + would then need a fresh out-of-sample confirmation before any promotion. + +Either outcome is publishable. Neither requires a winner, which is the point. + +## Axes + +VARIABLE, one only: + oversold in {20, 25, 30, 35, 40} + +`oversold` is the entire frequency mechanism and this is measured, not assumed: the 108-cell +diagnostic in `2026-08-12-fee-curve-and-rsi-meanrev-diag.py` found oversold 25->30 multiplied +trade count x2.18 and 30->35 by x3.93, against x1.186 for `support_proximity_pct` and x1.185 for +`level_min_touches`. The other two are noise on this axis. + +FIXED, deliberately: + overbought = 80 (shipped default) + support_proximity_pct = 0.005 (shipped default) + everything else = shipped defaults + +`overbought` is held not because it is a weak lever but because it is the WRONG KIND of lever: it +governs the exit side, so moving it changes trade OUTCOMES and not merely trade COUNTS. If both +axes move, a fall in gross PF cannot be attributed to firing more rather than exiting differently, +and the slope -- the entire point of the run -- becomes uninterpretable. + +`support_proximity_pct` is held because it is a STRUCTURAL filter (distance to a level) while +`oversold` is a MOMENTUM filter. Sweeping both confounds "does firing more degrade edge" with +"does relaxing which filter degrade edge". + +## Anchor + +oversold=20 is the shipped default and was already measured across all 24 assets by the +intersection run. Those rows are REUSED as the curve's left-hand anchor rather than recomputed, so +this grid runs 4 new levels x 24 assets = 96 combinations. The anchor rows are identical in every +other parameter (all defaults) and in cost treatment (same three fees, same explicit 0.0005 +slippage pin), which is what makes them poolable with the new ones. + +## Conditional second arm, DECLARED NOW so it cannot become a post-hoc rescue + + TRIGGER: if FEWER THAN 8 of 24 assets reach n >= 100 at oversold = 40. + THEN: add support_proximity_pct in {0.005, 0.02, 0.05} as a second axis. + REPORT: as a SEPARATE curve. Never pooled with, averaged into, or compared cell-to-cell + against the primary arm -- it varies a different filter and answers a different + question. + +Declaring the trigger and the reporting rule before any data exists is what stops the widening +from being invented later to rescue a disappointing primary arm. It writes to its own output file +for the same reason. + +## Costs + +Fees 0 / 0.006 (maker) / 0.012 (taker); `slippage_pct` pinned EXPLICITLY at 0.0005. The pin makes +every figure independent of library defaults and keeps the "zero fee" column honestly labelled -- +it is zero FEE, not zero COST. + +## Compute + +`rsi_meanrev` at LOW oversold is the slowest cell in the codebase: `backtest()` calls +`rule.detect()` only while flat, so a rule that almost never fires pays full support-level +detection on nearly all 44k bars. The grid therefore gets CHEAPER as oversold rises. Expect the +oversold=25 block to dominate wall-clock. +""" + +from __future__ import annotations + +import itertools +import json +import os +import time +from concurrent.futures import ProcessPoolExecutor +from decimal import Decimal + +DB = "/Users/elmehdiaitbrahim/keel/keel.db" +SCRATCH = ( + "/private/tmp/claude-501/-Users-elmehdiaitbrahim-Development-work-CodeGate-keel/" + "28ff9a61-09d1-498b-b325-1631c0662734/scratchpad" +) +ANCHOR = f"{SCRATCH}/intersection.jsonl" # supplies the oversold=20 rows +OUT_PRIMARY = f"{SCRATCH}/rsi_scale.jsonl" +OUT_CONDITIONAL = f"{SCRATCH}/rsi_scale_proximity.jsonl" # separate file, never merged + +UNIVERSE = [ + "BTC-USD", "ETH-USD", "ADA-USD", "LINK-USD", "LTC-USD", "SOL-USD", + "XLM-USD", "PAXG-USDT", "BCH-USD", "AAVE-USD", "DOGE-USD", "DOT-USD", + "UNI-USD", "ZEC-USD", "ALGO-USD", "FET-USD", "CRV-USD", "ICP-USD", + "AVAX-USD", "NEAR-USD", "XRP-USD", "PAXG-USD", "WLD-USD", "TON-USD", +] + +FEES = ["0", "0.006", "0.012"] +SLIPPAGE = Decimal("0.0005") + +ANCHOR_OVERSOLD = 20.0 +NEW_LEVELS = [25.0, 30.0, 35.0, 40.0] +PROXIMITY_LEVELS = ["0.005", "0.02", "0.05"] + +TRIGGER_MIN_ASSETS = 8 +TRIGGER_AT_OVERSOLD = 40.0 + + +def _run(job: tuple[str, float, str]) -> list[dict]: + product, oversold, proximity = job + from keel.data.db import connect + from keel.data.repository import Repository + from keel.strategy import backtest as bt + from keel.strategy.rules.rsi_meanrev import RsiMeanReversion + from keel_core.types import Granularity + + out: list[dict] = [] + try: + candles = Repository(connect(DB)).get_candles(product, Granularity.ONE_HOUR) + except Exception as exc: + return [ + {"product": product, "oversold": oversold, "proximity": proximity, "fee": f, + "error": f"{type(exc).__name__}: {exc}"} + for f in FEES + ] + + for fee in FEES: + try: + rule = RsiMeanReversion( + product_id=product, + oversold=oversold, + support_proximity_pct=Decimal(proximity), + ) + r = bt.backtest(rule, candles, fee_pct=Decimal(fee), slippage_pct=SLIPPAGE) + out.append({ + "product": product, "oversold": oversold, "proximity": proximity, "fee": fee, + "n_trades": int(r.n_trades), + "win_rate": float(r.win_rate), + "profit_factor": float(r.profit_factor), + "expectancy": float(r.expectancy), + }) + except Exception as exc: + out.append({"product": product, "oversold": oversold, "proximity": proximity, + "fee": fee, "error": f"{type(exc).__name__}: {exc}"}) + return out + + +def _done(path: str) -> set: + if not os.path.exists(path): + return set() + seen = set() + for line in open(path): + line = line.strip() + if not line: + continue + try: + r = json.loads(line) + except json.JSONDecodeError: + continue # torn final line + seen.add((r["product"], r["oversold"], r["proximity"])) + return seen + + +def _execute(jobs: list, path: str, label: str) -> None: + jobs = [j for j in jobs if (j[0], j[1], j[2]) not in _done(path)] + print(f"[{label}] running {len(jobs)} combinations", flush=True) + if not jobs: + return + t0 = time.perf_counter() + with ProcessPoolExecutor(max_workers=8) as ex, open(path, "a") as fh: + for i, rows in enumerate(ex.map(_run, jobs, chunksize=1), 1): + for row in rows: + fh.write(json.dumps(row) + "\n") + fh.flush() + if i % 4 == 0 or i == len(jobs): + print(f"[{label}] {i}/{len(jobs)} {time.perf_counter() - t0:.0f}s", flush=True) + + +def anchor_rows() -> list[dict]: + """The oversold=20 rows from the intersection run, relabelled into this grid's schema.""" + rows = [] + for line in open(ANCHOR): + line = line.strip() + if not line: + continue + r = json.loads(line) + if r.get("rule") != "rsi" or r.get("arm") != "A" or "error" in r: + continue + rows.append({**r, "oversold": ANCHOR_OVERSOLD, "proximity": "0.005", "anchor": True}) + return rows + + +def main() -> None: + print(f"PRIMARY ARM: oversold {NEW_LEVELS} x {len(UNIVERSE)} assets " + f"(+ {ANCHOR_OVERSOLD} reused as anchor)", flush=True) + jobs = [(a, o, "0.005") for o in NEW_LEVELS for a in UNIVERSE] + _execute(jobs, OUT_PRIMARY, "primary") + + # Evaluate the PRE-DECLARED trigger. No judgement is applied here -- the condition and the + # threshold were both fixed in the docstring above before any of this ran. + rows = [json.loads(x) for x in open(OUT_PRIMARY) if x.strip()] + at_max = { + r["product"] + for r in rows + if "error" not in r and r["oversold"] == TRIGGER_AT_OVERSOLD + and r["fee"] == "0" and r["n_trades"] >= 100 + } + print(f"\nTRIGGER CHECK: {len(at_max)}/{len(UNIVERSE)} assets reach n>=100 at " + f"oversold={TRIGGER_AT_OVERSOLD} (threshold: fewer than {TRIGGER_MIN_ASSETS} fires it)", + flush=True) + + if len(at_max) < TRIGGER_MIN_ASSETS: + print("TRIGGERED -> running the pre-declared conditional proximity arm, " + "to its own file, reported as a separate curve.", flush=True) + cond = [ + (a, o, p) + for o, p in itertools.product(NEW_LEVELS, PROXIMITY_LEVELS) + if p != "0.005" # 0.005 already covered by the primary arm + for a in UNIVERSE + ] + _execute(cond, OUT_CONDITIONAL, "conditional") + else: + print("NOT triggered -- primary arm reached the frequency floor on its own.", flush=True) + + print("\ndone", flush=True) + + +if __name__ == "__main__": + main() diff --git a/docs/experiments/trials-ledger.jsonl b/docs/experiments/trials-ledger.jsonl index 25e9a701..e349b3f9 100644 --- a/docs/experiments/trials-ledger.jsonl +++ b/docs/experiments/trials-ledger.jsonl @@ -80,3 +80,4 @@ {"decision":"diagnostic_only","kind":"ablation","params":{"answer":"The inference is HALF RIGHT and the wrong half is the half that would have set the roadmap. At fee_pct=0 turtle_breakout's best-swept configuration is profitable on 4 of 4 assets (BTC 1.090, ETH 1.458, SOL 1.533, ZEC 2.713) and rsi_meanrev is not (BTC 0.775 n=255, FET 0.907 n=284; ZEC 1.094 n=341 is the sole exception and breaks even at a 0.093% fee). Turtle has a gross edge that cost destroys; rsi_meanrev has no gross edge for cost to destroy. Same symptom at 1.2%, different disease. Pooling them predicts that maker fills rescue both -- which cutting cost to ZERO does not.","changed_nothing":"documentation and two copied scripts only -- no code, no config, no parameter, no rule status, no version bump, no demotion. config.live-sandbox.yaml untouched for the third document running. The fill model is deliberately NOT fixed here.","companions":["hourly-turtle-granularity-2026-08-11","hourly-turtle-param-sweep-2026-08-11","rsi-meanrev-grid-abandoned-2026-08-12"],"diagnostic_grid":{"atr_mult":["1.5"],"fixed_rr":["2"],"level_min_touches":[2,3],"overbought":[70.0],"oversold":[25.0,30.0,35.0],"require_divergence":[false],"support_proximity_pct":["0.005","0.02","0.05"]},"diagnostic_result":"104 of 108 cells clear n>=100 and ZERO of 108 clear PF 1.0. Median n 322.5, max n 887, median PF 0.2092, best PF 0.4556, win rates 14.9-34.0%. Per-asset best PF / median n: BTC 0.0953/134, DOGE 0.3149/423, ETH 0.1444/513, FET 0.4171/304, SOL 0.2102/427.5, ZEC 0.4556/331. Spearman(n_trades, PF) = +0.034, i.e. nothing: the tightest corner has median n 127 / median PF 0.2496 and the loosest has median n 778.5 / median PF 0.2145. Loosening every gate multiplies the sample by six and leaves the profit factor where it was -- the signature of a per-trade loss rate that is structural, not a sampling artifact.","document":"docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev.md","fee_curve":"turtle_breakout, hourly, slippage 0.0005 throughout, PF by fee_pct 0/0.001/0.002/0.003/0.004/0.006/0.012: BTC (n=123) 1.090/0.961/0.854/0.764/0.688/0.564/0.333; ETH (n=121) 1.458/1.330/1.217/1.117/1.028/0.875/0.556; SOL (n=92) 1.533/1.441/1.356/1.279/1.208/1.083/0.801; ZEC (n=50) 2.713/2.525/2.355/2.202/2.063/1.822/1.303. Break-even fees were MEASURED by 18 bracketing cells, not interpolated. Brackets (last cell above 1.0 / first cell below): BTC 0.060%->1.0096 / 0.070%->0.9971 => 0.068%; ETH 0.430%->1.0025 / 0.440%->0.9944 => 0.433%; SOL 0.750%->1.0005 / 0.800%->0.9751 => 0.751%; ZEC 1.700%->1.0191 / 1.750%->0.9957 => 1.741%; rsi_meanrev ZEC 0.090%->1.0025 / 0.095%->0.9977 => 0.093%. Measuring mattered: the first SOL bracket (0.80/0.85/0.90%) was chosen from a linear interpolation predicting 0.78% and ALL THREE cells came back below 1.0, because PF is convex in the fee and a chord under a convex curve crosses late. The interpolation erred by 3bp in the direction that flatters the maker argument, on the asset it most depends on. A second round of cells was run and both rounds are counted.","fee_curve_configs":{"BTC-USD":"entry168/exit80/atr3/rr6/adx25","ETH-USD":"entry240/exit80/atr3/rr3/adx25","SOL-USD":"entry336/exit80/atr3/rr6/adx20","ZEC-USD":"entry336/exit20/atr2/rr6/adx25","note":"each is the argmax of that asset's 144-cell slice of the 864-trial sweep in hourly-turtle-param-sweep-2026-08-11 -- see selection_bias"},"fee_curve_rule":"turtle_breakout (40 of the 52 fee cells). This row's `rule` field reads rsi_meanrev because 108 of its 160 trials are the rsi_meanrev diagnostic grid. A purist would give the fee curve its own row; it is folded in here because it SELECTS NOTHING -- it re-prices six already-ledgered configurations at additional fee rates -- and a row implies a search that did not happen. The count is reported either way.","fill_model_prerequisite":"A FEE-RATE CHANGE WITHOUT A FILL-MODEL CHANGE WOULD BE A WORSE COSTING ERROR THAN THE ONE #247 JUST FIXED, because it would be deliberate. #247 (1dfe1fb) corrected an error whose whole content was that the RATE did not match the FILL MODEL: backtest fills market-style at next-bar open -- a marketable order crossing the spread, i.e. taker -- and was priced at maker in four places. Setting fee_pct=0.006 on the same simulator re-creates that error exactly, with the corrected code in the tree and the commit that corrected it in the log. Maker economics require simulating a passive queue: (1) a resting order fills only when price trades THROUGH the limit, not when it merely touches it; (2) a breakout entered as a resting limit fills on the pullback, which selects disproportionately for breakouts that FAILED -- so 1.083 and 1.822 are taker fills priced at a maker rate, an upper bound and not an estimate; (3) unfilled signals need a policy, and reducing n makes the two rescued assets LESS promotable, not more; (4) intrabar resolution needs re-deriving, since the conservative fallback was designed for market-style fills. NOT FIXED HERE.","maker_pivot":"JUSTIFIED, NARROWLY, AND BY TURTLE ALONE. At fee_pct=0.006 (round-trip friction 1.30% against taker's 2.50%): SOL 1.083 and ZEC 1.822 clear; ETH 0.875 is a near miss against a 0.433% requirement; BTC needs 0.068%, nine times below the 0.60% maker rate of the <$1k-30d-volume band this account sits in, and not a fee-tier problem. It buys rsi_meanrev NOTHING -- not less than hoped, nothing, because the rule is under water at zero cost.","question":"TWO questions, deliberately kept apart. (a) FEASIBILITY: is there any configuration under which rsi_meanrev fires enough to be evaluated at all? Primary metric n_trades, NOT profit factor, declared as such in the script docstring before the run. (b) THE DECISIVE TEST of the inference drawn from the two turtle documents -- that execution cost is the binding constraint SYSTEM-WIDE, since two structurally opposite strategies both fail at 1.2%. Set fee_pct to ZERO, which is strictly better than any execution any venue will offer and therefore bounds from above everything an execution fix could buy, and see which rule recovers.","recommended_next":"(a) Build the maker-fill model as ENGINEERING, with the four requirements above, and test it against a pre-declared configuration set; the honest null is that adverse selection eats enough of the 1.30%-vs-2.50% gap to leave SOL and ZEC short, with both already below the promotion floor. (b) Characterise the ASSET dimension -- a 31x spread in break-even is the largest measured lever in this document and can be asked with the existing harness. (c) NOT further rsi_meanrev tuning: it is negative at zero cost on 2 of 3 assets, its 108-cell response surface is flat to within 0.03 PF per asset, and firing it six times more often does not move it. A different mean-reversion entry is a new rule with a new pre-registration, not a continuation of this one.","script":"docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev-diag.py","selection_bias":"STATED AT FULL STRENGTH. Every configuration in the fee curve is the argmax of its asset's 144-cell slice of an 864-trial sweep, selected on the same data it is now re-priced on, at n = 123 (BTC) / 121 (ETH) / 92 (SOL) / 50 (ZEC). These are maxima of 144 draws and are NOT edge estimates; they must never be quoted as any asset's expected profit factor. What they legitimately support is a COMPARISON, and the bias runs AGAINST the finding: it inflates turtle's cells, and turtle is the arm that wins, while the rsi_meanrev cells are not maxima of anything -- the diagnostic grid's declared objective was n_trades. ANY MAKER RE-TEST MUST BE A FRESH PRE-DECLARED HYPOTHESIS on configurations chosen without reference to the 864-cell ranking, or the bias is laundered into a headline.","structural_finding":"Break-even varies by a factor of ~26 across four assets running the same rule on the same clock over the same window (BTC 0.068% to ZEC 1.741%), against parameter effects of a few percent in the 864-cell sweep. The ASSET is a far larger lever than tuning and nobody has characterised what drives it. Worse for the maker pivot: the two assets that clear break-even at the 0.6% maker rate are SOL (n=92) and ZEC (n=50), both BELOW min_trades=100, while the two with enough trades to promote -- ETH (n=121, 0.875) and BTC (n=123, 0.564) -- stay unprofitable. That is the companion sweep's Spearman -0.77 bind between edge and sample size reappearing on a completely different axis, which is the first evidence that it is structural rather than a property of one parameter search.","validation":"Screening result only. No walk-forward, no out-of-sample split, no CSCV/PBO and no deflated Sharpe on these configurations -- #247 wired the dormant PBO gate into promotion and none of these trials can feed it, because backtest emits aggregates and no per-trade series (series_missing). 4 of 19 assets on the fee curve, 6 of 19 in the grids, 3 of 19 in the zero-fee rsi_meanrev arm; same cached candles and same 2021-2026 cycle as everything else. slippage_pct held at 0.0005 in every cell, so 'zero fee' is zero FEE and not zero cost -- round-trip friction at fee_pct=0 is still 0.10% of notional.","wrong_diagnosis_recorded":"TWO wrong diagnoses, recorded because a document that hides its wrong turns teaches nothing about how the right answer was reached. (1) support_proximity_pct was predicted to be the binding gate on trade count and the grid was built to widen it 10x. It was NOT: over 36 matched triples, 0.005 -> 0.05 moves n by a median factor of 1.186 (range 1.071-1.426); the headline pair BTC 255 -> 290 is 1.137. (2) The CORRECTION -- that the binding gates were oversold and level_min_touches, the latter pinned at its default in the first grid and therefore never tested -- is also half wrong: over 54 matched pairs, level_min_touches 3 -> 2 moves n by a median factor of 1.185, indistinguishable from the proximity gate it was supposed to displace. The one gate that binds is oversold, alone: 25 -> 30 is x2.183 and 25 -> 35 is x3.933 over 36 matched pairs. The entire jump in median n from 17 to 322.5 is bought by oversold and require_divergence, BOTH of which were axes in the first grid; the one axis the second grid added contributed x1.19. So 'rsi_meanrev is unmeasurable' was an artifact of grid centring, not a property of the rule -- and the first grid could have answered the question without this one."},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"260702bc8597123863e20b97a74c9c5eb14a04cc2147e4d4f7f4ee19ce85182e","provenance":"a_priori","row_hash":"706422899e8fb759a4732e39aa63e79c3ebcc3371622e4f0ad9193e08ab3740e","rule":"rsi_meanrev","series_missing":true,"session":"fee-curve-and-rsi-meanrev-2026-08-12","summary":{"assets_tested_diagnostic_grid":6,"assets_tested_fee_curve":4,"best_pf":"0.4556","breakeven_fee_rsi_zec":"0.00093","breakeven_fee_turtle_btc":"0.00068","breakeven_fee_turtle_eth":"0.00433","breakeven_fee_turtle_sol":"0.00751","breakeven_fee_turtle_zec":"0.01741","breakeven_spread_across_four_assets":"25.6","cells_above_pf_1":0,"cells_tested":108,"cells_with_n_ge_100":104,"code_changed":0,"configs_changed":0,"diagnostic_grid_trials":108,"fee_curve_and_zero_fee_cells":61,"fee_curve_cells_rsi":15,"fee_curve_cells_turtle":46,"gate_effect_level_min_touches_3_to_2":"1.185","gate_effect_oversold_25_to_30":"2.183","gate_effect_oversold_25_to_35":"3.933","gate_effect_support_proximity_10x":"1.186","max_n_trades":887,"median_n_trades":"322.5","median_pf":"0.2092","min_n_trades":80,"min_pf":"0.0680","min_trades_floor":100,"n_trades_turtle_btc":123,"n_trades_turtle_eth":121,"n_trades_turtle_sol":92,"n_trades_turtle_zec":50,"n_trades_zero_fee_rsi_btc":255,"n_trades_zero_fee_rsi_fet":284,"n_trades_zero_fee_rsi_zec":341,"n_trials":169,"pf_zero_fee_rsi_btc":"0.7748","pf_zero_fee_rsi_fet":"0.9067","pf_zero_fee_rsi_zec":"1.0939","pf_zero_fee_turtle_btc":"1.0896","pf_zero_fee_turtle_eth":"1.4580","pf_zero_fee_turtle_sol":"1.5331","pf_zero_fee_turtle_zec":"2.7132","round_trip_friction_maker":"0.0130","round_trip_friction_taker":"0.0250","rsi_assets_above_pf_1_at_fee_0":1,"rsi_assets_tested_at_fee_0":3,"rules_demoted":0,"spearman_n_trades_vs_pf":"0.0342","trial_errors":0,"trials_this_line_of_work_cumulative":1481,"trials_today_both_rows":617,"turtle_assets_above_pf_1_at_fee_0":4,"turtle_assets_above_pf_1_at_maker_0006":2,"turtle_assets_above_pf_1_at_taker_0012":1,"turtle_assets_tested_at_fee_0":4,"win_rate_max":"0.3400","win_rate_min":"0.1492"},"timestamp":1786531189,"trial_id":"rsi-meanrev-diagnostic-and-fee-curve-2026-08-12"} {"decision":"diagnostic_only","kind":"ablation","params":{"arm":"A -- shipped constructor defaults, ZERO free parameters","assets":24,"construction":"each rule built with product_id= and nothing else","criteria_declared_before_run":"C1 n>=100; C2 C1 and gross PF>1; C3 C2 and PF@0.6%>1","dca_excluded":"scheduled accumulation, not a signal edge","document":"docs/experiments/2026-08-12-shipped-defaults-intersection.md","fees":["0","0.006 maker","0.012 taker"],"granularity":"ONE_HOUR","predeclaration_location":"the dispatch brief, NOT the script docstring -- weaker than the other harnesses in this directory, and recorded as such in the script and the write-up","rules":["turtle_breakout","rsi_meanrev","pullback_continuation"],"slippage_pct":"0.0005 pinned explicitly, not inherited","trials":"72 combinations x 3 fee levels = 216 backtests"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"706422899e8fb759a4732e39aa63e79c3ebcc3371622e4f0ad9193e08ab3740e","provenance":"a_priori","row_hash":"74cfa02fa95ee5d5cafe5ee6dda36ea69c3755317722233d6d2745b390243651","rule":"turtle_breakout+rsi_meanrev+pullback_continuation","series_missing":true,"session":"shipped-defaults-intersection-2026-08-12","summary":{"pullback_gross_pos":"1","pullback_maker_pos":"0","pullback_median_gross":"0.9292","pullback_n100":"4","rsi_median_gross":"1.1631","rsi_n100":"0","taker_survivors_all_rules":"0","trade_count":25,"turtle_gross_pos":"7","turtle_maker_pos":"1","turtle_median_gross":"0.9892","turtle_n100":"21"},"timestamp":1786000000,"trial_id":"shipped-defaults-intersection-arm-a-2026-08-12"} {"decision":"diagnostic_only","kind":"ablation","params":{"adx_threshold":25.0,"arm":"B -- out-of-sample transfer of the 864-trial sweep winner","atr_stop_mult":"2","consumes_no_new_selection":"the config was fixed before this arm ran; this arm spends no further multiple-testing budget, it only tests generalisation","document":"docs/experiments/2026-08-12-shipped-defaults-intersection.md","entry_lookback":336,"evaluated_on":"the 18 assets NOT in that six -- disjoint from the selection","exit_lookback":80,"selected_on":["ZEC-USD","FET-USD","SOL-USD","DOGE-USD","ETH-USD","BTC-USD"],"target_rr":"6","trials":"18 combinations x 3 fee levels = 54 backtests"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"74cfa02fa95ee5d5cafe5ee6dda36ea69c3755317722233d6d2745b390243651","provenance":"fitted","row_hash":"19ecedc7148cb12c8a33d59f5e9cbbba5145beb1ddabd7cbffa1e63dd755ba6c","rule":"turtle_breakout","series_missing":true,"session":"shipped-defaults-intersection-2026-08-12","summary":{"in_sample_mean_net_pf":"0.6335","oos_assets_over_floor":"1","oos_mean_gross_pf":"1.5317","oos_mean_net_pf":"0.6346","oos_median_n":"84","oos_median_net_pf":"0.5138","trade_count":18,"verdict_not_overfit":"1"},"timestamp":1786000001,"trial_id":"shipped-defaults-intersection-arm-b-2026-08-12"} +{"decision":"rejected","kind":"ablation","params":{"anchor":"oversold=20 reused from the #252 intersection run (identical in every other parameter and cost treatment)","assets":24,"caveat_declared_statistic":"the slope was UNDERPOWERED for its own question -- the relationship is a threshold effect at the floor, not linear, and two barely-trading assets supply most of its variance. The partition on the pre-declared n>=100 floor is the decisive reading and is reported alongside it.","conditional_arm":"declared in advance: widen support_proximity_pct if FEWER THAN 8 of 24 assets reach n>=100 at oversold=40. DID NOT FIRE (21/24 reached it).","design":"MONOTONICITY TEST, not a search -- declared statistic is the SLOPE of gross PF against n, per asset then averaged. The best cell is never reported as a result.","document":"docs/experiments/2026-08-12-rsi-meanrev-scale-vs-selectivity.md","fees":["0","0.006 maker","0.012 taker"],"granularity":"ONE_HOUR","held_fixed":{"overbought":80.0,"support_proximity_pct":"0.005","why_overbought_held":"it governs the EXIT side, so moving it changes trade outcomes and not merely trade counts, which would make the slope uninterpretable"},"instrument_defect_found":"backtest() never expires a pending setup, silently freezing a strategy for the rest of a series -- filed as #254. #252's headline results were checked and are clean.","predeclaration_location":"the script docstring, written before the run -- the correction to the defect #252 recorded against itself","slippage_pct":"0.0005 pinned explicitly","trials":"4 new levels x 24 assets = 96 combinations x 3 fees = 288 backtests","variable_axis":{"oversold":[20.0,25.0,30.0,35.0,40.0]}},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"19ecedc7148cb12c8a33d59f5e9cbbba5145beb1ddabd7cbffa1e63dd755ba6c","provenance":"a_priori","row_hash":"a16af414291b6034ccaa58ffa1577e56aa42eab1c72edb9364ac77988acd6b83","rule":"rsi_meanrev","series_missing":true,"session":"rsi-meanrev-scale-vs-selectivity-2026-08-12","summary":{"assets_negative_slope":"15","assets_reaching_floor_at_oversold_40":"21","cells_gross_positive_over_floor":"11","cells_net_positive_over_floor_any_fee":"0","cells_over_floor":"76","mean_slope_per_100_trades":"-0.0386","median_gross_pf_at_defaults":"1.1631","median_gross_pf_at_n_over_100":"0.8938","median_slope_per_100_trades":"-0.0197","trade_count":96},"timestamp":1786010000,"trial_id":"rsi-meanrev-scale-vs-selectivity-2026-08-12"}