From b24a2557f6e4a54199fbcbe241159f6cc07ecfc0 Mon Sep 17 00:00:00 2001 From: Elmehdi Aitbrahim Date: Wed, 12 Aug 2026 09:48:50 -0400 Subject: [PATCH] =?UTF-8?q?docs(experiments):=20three=20rules,=2024=20asse?= =?UTF-8?q?ts,=20zero=20free=20parameters=20=E2=80=94=20the=20viable=20int?= =?UTF-8?q?ersection=20is=20empty?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Measures every signal rule the codebase ships at its shipped constructor defaults, across the whole 24-asset hourly corpus, at three fee levels. 270 backtests, 0 errors. No combination clears the trade floor with an edge surviving even the maker rate, and the three rules fail for three unrelated reasons — cost, signal, and sample size — which pooling destroys. Supersedes two earlier conclusions: - #248 recorded rsi_meanrev as having "no gross edge at all" from BTC alone. Across 24 assets it has the BEST gross distribution of the three (median 1.1631 vs turtle 0.9892, pullback 0.9292); BTC is near the bottom of it. Its failure is sample size — 0 of 24 reach n>=100. - #248 framed the sweep winner as a selection artifact. Arm B refutes it: 0.6335 in-sample reproduces as 0.6346 on 18 disjoint assets. Not overfit — stably unprofitable, which is worse. pullback_continuation is measured here for the first time since it was written. Two ledger rows (Arm A a_priori, Arm B fitted), both diagnostic_only, chain verifies clean at 82. Co-Authored-By: Claude Opus 5 (1M context) --- ...026-08-12-shipped-defaults-intersection.md | 317 +++++++++++++++++ ...026-08-12-shipped-defaults-intersection.py | 322 ++++++++++++++++++ docs/experiments/trials-ledger.jsonl | 2 + 3 files changed, 641 insertions(+) create mode 100644 docs/experiments/2026-08-12-shipped-defaults-intersection.md create mode 100644 docs/experiments/2026-08-12-shipped-defaults-intersection.py diff --git a/docs/experiments/2026-08-12-shipped-defaults-intersection.md b/docs/experiments/2026-08-12-shipped-defaults-intersection.md new file mode 100644 index 00000000..57dec50a --- /dev/null +++ b/docs/experiments/2026-08-12-shipped-defaults-intersection.md @@ -0,0 +1,317 @@ +# Three rules, 24 assets, zero free parameters — the viable intersection is empty, and one of the three fails for the opposite reason we recorded + +**Date:** 2026-08-12 +**Issue:** #251 +**Change:** documentation only. No code, no config, no rule status, no parameter, no version bump. +**Script:** `docs/experiments/2026-08-12-shipped-defaults-intersection.py` — produced every number +below, and is the file that was resumed four times (see §7). +**Ledger:** two rows, `shipped-defaults-intersection-2026-08-12`, session +`shipped-defaults-intersection-2026-08-12`. +**Deployment:** run against `keel 0.7.0`, i.e. **after** #247's taker-fee correction shipped. Every +figure here passes `fee_pct` and `slippage_pct` explicitly, so none of them depends on a library +default; §7 records the check that the pre-upgrade rows reproduce exactly. + +**Verdict: across every signal rule the codebase ships, on every asset with hourly history, there +is no combination that clears the trade floor with an edge that survives the cheapest fee we can +reach. The three rules fail for three unrelated reasons, and pooling them loses the only +information that says where to look next.** + +| question | answer | +|---|---| +| any asset-rule combo with `n≥100` ∧ gross>1 ∧ net@0.6%>1? | **one**, ZEC-`turtle` — then eliminated in §5 | +| the same at the 1.2% we actually pay? | **none**, 0 of 72 | +| is the 864-trial sweep winner overfit? | **no** — 0.6335 in-sample → **0.6346** out-of-sample | +| does `rsi_meanrev` lack gross edge, as #248 recorded? | **no** — it has the *best* of the three; it lacks observations | +| is `pullback_continuation` (never before backtested) different? | **yes, worse** — median gross **0.929** | +| do the three rules share a failure mode? | **no** — cost, signal, and sample size respectively | + +--- + +## 1. What was declared, and when + +Two arms, both fixed **before the run** — but in the brief that dispatched the implementing +agent, **not** in the script, which had no docstring until this write-up was prepared. Every other +harness in this directory carries its own pre-registration in the file; this one's lives in a session +transcript, which is weaker, and the script now says so at the top rather than presenting the +reconstruction as though it had always been there. Future runs put the declaration in the file. + +**Arm A — `a_priori`, zero free parameters.** All three signal rules constructed with +`product_id=` and *nothing else*, so every other parameter takes its shipped constructor +default. Those defaults were written before this corpus existed and were never tuned on it, which +makes them the only genuinely unselected configuration available. 3 rules × 24 assets = 72 +combinations. + +**Arm B — out-of-sample transfer.** The 864-trial hourly sweep +(`docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev-sweep.py`) picked +`{entry_lookback: 336, exit_lookback: 80, atr_stop_mult: 2, target_rr: 6, adx_threshold: 25}` as +its mean-across-assets winner, scored on six assets. Arm B evaluates that exact config on the +**eighteen assets that were not in the sweep**. The evaluation selects nothing, so it consumes no +further multiple-testing budget; what it tests is whether the sweep's choice generalises. + +**Costs.** Three fee levels — `0`, `0.006` (maker), `0.012` (taker) — with `slippage_pct` pinned +**explicitly** at `0.0005` in every call. That pin matters and was missing from earlier work: the +"zero fee" column is zero *fee*, not zero *cost*, because the shipped 5bp slippage default applies +regardless. Every gross figure in this document, and every break-even quoted in +`2026-08-12-fee-curve-and-rsi-meanrev.md`, already carries that slippage on both legs. + +**Universe.** All 24 products with `ONE_HOUR` candles in `keel.db`; 20 of them carry ~44k bars back +to 2021. No asset was excluded for any reason, including the three that turn out to be nearly +untradeable by these rules. + +90 combinations × 3 fees = **270 backtests**, 0 errors. + +## 2. The result + +| rule | `n≥100` | ∧ gross>1 | ∧ net@0.6%>1 | ∧ net@1.2%>1 | +|---|---:|---:|---:|---:| +| `turtle_breakout` | 21/24 | 7 | **1** (ZEC) | 0 | +| `pullback_continuation` | 4/24 | 1 (PAXG-USDT) | 0 | 0 | +| `rsi_meanrev` | 0/24 | — | — | 0 | +| **Arm A total** | **25/72** | **8** | **1** | **0** | + +The columns are cumulative — each adds a condition to the one before it. `rsi_meanrev`'s later +columns are dashes rather than zeroes because the first condition already empties the set; it is +gross-positive on 14 of 24 assets, but never with enough trades to be admitted (§4). + +## 3. Three rules, three unrelated failures + +The single most useful thing in this dataset is that the rules do **not** fail the same way. + +| rule | gross edge | cost sensitivity | fires enough? | so the lever is | +|---|---|---|---|---| +| `turtle_breakout` | real, broad — 7/21 gross-positive at `n≥100` | moderate | yes, median n=241 | **cost** | +| `pullback_continuation` | ~none — mean 1.043, **median 0.929** | **extreme** | marginal, 4/24 | **signal** | +| `rsi_meanrev` | **best of the three** — median **1.1631**, 58% positive | moderate | **no**, 0/24 | **sample size** | + +Any statement of the form "the strategy layer is dead" collapses these into one claim and destroys +the only guidance the data contains. Cheaper execution rescues exactly one of these three rows. + +### 3.1 `pullback_continuation` — a signal failure, measured for the first time + +This rule has been in `RULE_REGISTRY` since it was written and had **never been backtested**. It is +now measured: 24 assets, median `n` 60, only 4 clearing the floor. + +``` +product n PF@0 PF@0.6% PF@1.2% win@0 +WLD-USD 12 2.810 1.650 0.881 75.0% +FET-USD 30 1.996 1.463 1.056 56.7% +UNI-USD 90 1.118 0.285 0.050 52.2% +PAXG-USDT 100 1.097 0.001 0.000 65.0% +ETH-USD 101 0.990 0.067 0.007 56.4% +ZEC-USD 128 0.875 0.290 0.102 53.9% +LTC-USD 123 0.695 0.037 0.005 53.7% + mean gross 1.0430 median gross 0.9292 +``` + +**The fee collapse is qualitatively different from `turtle`'s.** PAXG-USDT goes 1.097 → **0.001** +at the maker rate. ETH 0.990 → 0.067. BTC 0.897 → 0.050. `turtle` at the same 0.6% retained +structure (ZEC 1.042, BTC 0.318); this goes to the floor. + +The cause is legible in the win rates: **52–75% wins at gross PF ≈ 1.0** is small wins against +small losses, which is what `target_method="measured_1to1"` on an EMA touch produces. When per-trade +edge is a fraction of a percent, a 1.2% round trip does not reduce it, it erases it. The high win +rate is the *symptom* of maximal fee fragility here, not a strength — and it is the profile most +likely to be mistaken for a good result by anyone reading win rate first. + +No fee schedule fixes a median gross PF of 0.929. This family needs re-engineering or retirement, +and that is a different sentence from the one `turtle` earns. + +### 3.2 `turtle_breakout` — a cost failure + +At shipped defaults, 21 of 24 assets clear `n≥100`, and 7 are gross-positive: + +``` +PAXG-USDT 1.248 ZEC 1.555 XRP 1.382 FET 1.255 +CRV 1.083 ETH 1.044 AAVE 1.028 +``` + +Every one of them dies on cost. The mechanism is worth stating precisely because it explains why +the collapse is so violent: **the toll is levied on the search, not on the edge.** A tail-sensitivity +probe (below) shows the profit concentrated in a handful of trades, so the account pays ~241 +round-trip tolls in order to be present for the few that pay. Halving the toll does not halve the +number of times it is paid. + +Concentration, at zero fee, with the best 1 and best 3 winning trades removed (losses all retained +— a deliberately harsh stress, not an unbiased estimator): + +| asset | n | gross | top-3 share | ex-top1 | **ex-top3** | +|---|---:|---:|---:|---:|---:| +| ZEC-USD | 250 | 1.555 | 33.4% | 1.248 | **1.035** | +| XRP-USD | 144 | 1.382 | 29.7% | 1.221 | **0.972** | +| FET-USD | 259 | 1.255 | 29.2% | 1.126 | **0.889** | +| PAXG-USDT | 224 | 1.248 | 23.5% | 1.116 | **0.955** | +| CRV-USD | 246 | 1.083 | 45.5% | 0.888 | **0.590** | +| ETH-USD | 266 | 1.044 | 13.4% | 0.993 | **0.904** | +| AAVE-USD | 233 | 1.028 | 16.3% | 0.968 | **0.860** | + +Deleting 3 trades out of ~250 — **1.2% of the sample** — takes six of seven below break-even. + +**This is not by itself evidence of no edge.** A fat right tail *is* the strategy; a breakout system +with evenly distributed profits would be the surprising result. What it does establish is that +`min_trades=100` is measuring the wrong quantity for this family: n=250 reads as a large sample and +licenses confidence, while the number of independent events carrying the PnL is single-digit. The +gate counts trades; it cannot see that three of them are the result. + +## 4. `rsi_meanrev` — this supersedes #248 + +`docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev.md` recorded that `rsi_meanrev` "has no +gross edge at all", on the strength of BTC at 0.775 gross with n=255. Measured across the full +universe at shipped defaults, that generalisation does not hold: + +``` +n>=100: 0/24 median n 38 max n 81 (ETH) +gross PF: mean 1.1316 median 1.1631 gross-positive 14/24 (58%) +``` + +**`rsi_meanrev` has the best gross-edge distribution of the three rules** — median gross **1.1631** +against `turtle`'s **0.9892** and `pullback`'s **0.9292**, all over the same 24 assets. BTC sits near the +*bottom* of its distribution at 0.745, not at its centre. + +Two things reconcile this with #248 without either being wrong on its own terms: + +1. **#248's rsi figures came from *widened* parameters**, not defaults — the fee-curve rows at + n=255–341 used `oversold` 25–35 and `support_proximity_pct` 0.02–0.05, chosen to make the rule + fire. This document measures the shipped defaults, which are far more selective. +2. **#248 generalised from one asset.** That is the error, and it is mine; the correction is the + distribution above, not a change in BTC's number, which reproduces exactly. + +So the rule's failure mode is **sample-size suffocation**: 0 of 24 assets reach the promotion floor, +median n=38 against `min_trades=100`. It does not lose. It is not observable. + +That reframes the open question and is the one live lead this study produces (§8). + +## 5. Arm B — the sweep winner is not overfit, and that is worse + +``` +in-sample mean net PF @1.2%, on its 6 selection assets : 0.6335 +out-of-sample mean, on 18 disjoint assets : 0.6346 median 0.5138 +out-of-sample mean GROSS : 1.5317 median 1.2311 +n: median 84, only 1 of 18 reaches n>=100 +``` + +A configuration chosen as best-of-144 on six assets reproduces its mean net profit factor **to three +decimal places** on eighteen assets it never saw. This is the opposite of overfitting. The sweep +found a real, stable, transferable property of the rule, and the property is that it loses after +costs. + +That is a harder result than overfitting would have been. Overfitting is a methodology defect with a +methodology fix. A clean out-of-sample replication of 0.63 is a measurement. + +**What does survive from the earlier reading** is narrower and still holds: the config buys its +higher gross PF with trade count. Median n falls from **241** at defaults to **84** here, and Arm B's +intersection is empty on all three criteria, including `n≥100 ∧ gross>1`. Its one asset above the +floor is LINK at gross 0.764. Longer lookbacks find *more* gross edge (67% of assets gross-positive, +mean gross 1.53) on *fewer* trades, and still cannot pay 1.2%. + +Both arms, from opposite parameter regimes, say the same thing: the gross edge is real and broad; +the toll is what removes it. + +## 6. The temporal probe, and why it is reported rather than gated + +ZEC-`turtle` was the only combination in the study to clear all three cumulative criteria. A +year-by-year decomposition removes it: + +| asset | n | gross | 2021 | 2022 | 2023 | 2024 | 2025 | 2026 | streak all/full | +|---|---:|---:|---:|---:|---:|---:|---:|---:|---:| +| ZEC-USD | 250 | 1.555 | 1.85 | **0.66** | **0.30** | **0.96** | 2.10 | 1.51 | 3/3 | +| PAXG-USDT | 224 | 1.248 | 0.27 | 0.30 | 0.44 | 0.77 | 2.10 | 1.71 | 4/3 | +| XRP-USD | 144 | 1.382 | — | — | 0.35 | 2.37 | 1.10 | 1.39 | 1/1 | +| AAVE-USD | 233 | 1.028 | 1.01 | 1.41 | 0.94 | 1.44 | 0.66 | — | 1/1 | + +**ZEC ran three consecutive losing years and compressed 92.7% of its lifetime net PnL into +2025–2026.** Its 1.555 is not a distributed structural edge; it is a recent regime sitting on top of +a three-year drawdown an operator would have had to fund. Across all 21 assets clearing the floor, +**none has a losing-year-free record**, and only XRP and AAVE avoid consecutive losing years — both +of which still net below 1.0 at every fee level (XRP 0.773 at maker, 0.490 at taker). + +Three probes, each catching what the others miss: + +| probe | eliminated | +|---|---| +| fee curve | 20 of 21 — only ZEC survived at maker | +| tail-sensitivity | 6 of the 7 gross-positive — only ZEC survived | +| **temporal** | **ZEC** — the one both others spared | + +**This probe is reported, never gated**, for three reasons that are load-bearing: + +1. **It is underpowered.** Yearly outcomes for an edgeless strategy are near coin-flips, and the + number of length-4 sequences with no two consecutive losses is 8 of 16. A no-consecutive-losing- + years rule passes a zero-edge strategy roughly **50%** of the time (40% at five buckets). It can + flag the egregious cases — ZEC, PAXG-USDT — and nothing finer. +2. **It is fragile to bucket boundaries.** 2021 and 2026 are partial buckets (data starts mid-July + 2021, ends July/August 2026). Excluding them **changes the verdict for 4 of 21 assets**. FET + flips from fail to pass: its full-year sequence is `1.08, 1.30, 1.65, 0.68`, one losing year, so + the "alpha decay" reading of FET rests entirely on a 7-month partial bucket with n=34. A metric + that moves when you change where the year starts cannot gate a promotion. +3. **Stationarity is not edge.** The decisive counter-example is in this dataset: + **ZEC-`pullback_continuation` is the only asset-rule combination in the whole study with no + losing complete year** — 1.06, 1.17, 1.16, 1.13 across 2022–2025 — and its gross PF is **0.875**, + netting 0.290 at maker. It is perfectly stationary at losing slightly, every year, reliably. A + stationarity gate would have waved it through while rejecting ZEC-`turtle`, which at least made + money gross. + +The principled instrument for this already exists and is now live: `keel/research/cscv.py`, +`deflate.py` and `matrix.py` shipped long ago, and #247 wired `g4_pbo_gate` into `can_promote`, +where `pbo=None` returns `NOT_RUN` and blocks. **It is deployed and nothing feeds it.** Building a +weaker annual-bucket heuristic beside an unfed rigorous one is the wrong order of work. + +### 6.1 A cross-rule observation about the corpus itself + +PAXG-USDT shows the **same** temporal signature under both `turtle` (streak 4: 2021–2024 sub-1.0, +then 2.10, 1.71) and `pullback` (streak 3: 0.19, 0.57, 0.54, then 3.09, 1.44, 2.82). Two +structurally unrelated rules — a breakout and a pullback-continuation — both lose on gold for three +or four years and both turn positive in 2024. + +That is not a rule property. It is a property of gold's 2024+ regime, and it means any aggregate +metric over this corpus is partly measuring that macro shift whichever rule is applied. The same +caution applies to ZEC. It is an argument for walk-forward evaluation over whole-corpus aggregates, +independent of everything else here. + +## 7. Method notes, including the errors + +**The simulator's open-position handling is correct and was not changed.** `summarize()` excludes +unclosed trades from every aggregate — `closed = [t for t in trades if t.outcome != "open"]` — and +says so in its docstring. LINK's `n=240` is 240 *closed* trades. Marking the open position to market +at the final candle was considered and **rejected**: it injects an unrealized price into a +realized-PnL metric and would systematically flatter trend-following, which by construction tends to +be holding a winner when a series truncates. The current design is the conservative one. + +**Version boundary.** The first 27 combinations ran on `keel 0.6.1`, the rest on `0.7.0` after +#247's fee fix was deployed mid-run. Because #247 is a costing change with `n_trades` and +`win_rate` unchanged, and because this script passes `fee_pct` and `slippage_pct` explicitly, the +results should be version-independent — and were checked rather than assumed. BTC-USD reproduces to +six decimal places on all three fee levels across the boundary. The banked rows were kept. + +**Four failed runs preceded the successful one, and the cause was the harness, not the machine.** +The script was piped into `tail`, which buffers until EOF; when the invoking shell was reaped the +parent blocked on a closed pipe at 0.07s of CPU with no workers, and an empty output file was twice +misread as "still running". Fixed by redirecting to a file. Separately, `pkill -f intersection.py` +reaps parents but not `multiprocessing` children, whose cmdline differs — 40 orphaned workers were +left burning CPU across the session and had to be reaped by PPID. Both are recorded because both +produced confident status reports that were false. + +**Job ordering was changed mid-run, and it cannot affect the result.** `rsi_meanrev` at defaults is +by far the slowest cell: `backtest()` calls `rule.detect()` only while flat, so the rule that almost +never fires pays full support-level detection on nearly all 44k bars. Left in declaration order it +starved the two arms that answer the question. Arm B and `pullback` were promoted ahead of it. Every +declared job still ran, and a backtest is independent of dispatch order. + +## 8. What this leaves open + +The one live lead is `rsi_meanrev`, and it is a sharp, cheap question: + +**Does its gross edge survive being made to fire more often?** + +Median gross PF 1.1631 at defaults, on median n=38. The 108-cell diagnostic in #248 already widened +`oversold` to 25–35 and `support_proximity_pct` to 0.02–0.05 and found no net-viable cell at +taker — but it never asked whether widening **preserved the gross edge** or simply bought trades by +accepting worse setups. Those are different findings with different consequences, and the data to +separate them was never computed. + +If selectivity is what creates the edge, the rule is unpromotable by construction and should be said +so. If the edge survives to n≥100, it is the only route by which any rule the codebase ships reaches +its own promotion floor honestly. + +Nothing else here justifies engineering investment. In particular, a limit-order queue simulator has +no target: its measured prize was one asset (ZEC) with ~8bp of headroom below the maker rate, and §6 +removes that asset. diff --git a/docs/experiments/2026-08-12-shipped-defaults-intersection.py b/docs/experiments/2026-08-12-shipped-defaults-intersection.py new file mode 100644 index 00000000..f40ee2e1 --- /dev/null +++ b/docs/experiments/2026-08-12-shipped-defaults-intersection.py @@ -0,0 +1,322 @@ +"""Is there ANY asset-rule combination that clears the trade floor and survives its own costs? + +## Provenance of this docstring -- read this first + +Unlike the other harnesses in this directory, the pre-registration for this run did NOT live in +this file. The two arms, the fee grid, the universe, the slippage pin and the pass/fail criteria +were fixed in the brief that dispatched the implementing agent, BEFORE any of it ran; this +docstring was written afterwards and reproduces that brief. That is weaker than a committed +docstring -- the brief is in a session transcript, not in git -- and it is recorded here rather +than quietly presented as if the file had always said so. Future runs should put the declaration +in the file. + +What was NOT decided in advance: the two probes in `probe_tail()` and `probe_regime()` below. +Both were written after seeing Arm A's results, are post-hoc by construction, and are reported in +the write-up as diagnostics rather than as gates for exactly that reason. + +## The question + +Every negative result so far describes `turtle_breakout` at parameters someone chose. This asks a +narrower question of the whole shipped library: at the constructor defaults nobody tuned on this +corpus, is there any asset where a rule both fires enough to be admitted (`min_trades=100`) and +keeps a profit factor above 1.0 once costs are charged? + +## Arm A -- a_priori, zero free parameters + +All three signal rules built with `product_id=` and nothing else, so every remaining +parameter takes its shipped default. Those defaults predate this corpus and were never fitted to +it, which makes them the only genuinely unselected configuration available. `dca` is excluded: it +is scheduled accumulation, not a signal edge. 3 rules x 24 assets = 72 combinations. + +## Arm B -- out-of-sample transfer + +`{entry_lookback: 336, exit_lookback: 80, atr_stop_mult: 2, target_rr: 6, adx_threshold: 25}` was +the mean-across-assets winner of the 864-trial sweep, scored on ZEC/FET/SOL/DOGE/ETH/BTC. Arm B +evaluates that config on the 18 assets NOT in that six, which are therefore disjoint from +everything the selection saw. This arm selects nothing, so it spends no further multiple-testing +budget -- it only asks whether the sweep's choice generalises. + +## Costs + +Fees at 0 / 0.006 (maker) / 0.012 (taker), with `slippage_pct` pinned EXPLICITLY at 0.0005 rather +than inherited. The pin is the point: `backtest()` applies 5bp of slippage by default, so a "zero +fee" column is zero FEE and not zero COST, and earlier work that omitted the argument was quietly +carrying it anyway. Passing both explicitly also makes every number here independent of the +library defaults #247 changed. + +## Criteria, declared before the run + + C1 n_trades >= 100 (the promotion floor) + C2 C1 AND profit_factor @ 0% > 1.0 (gross edge exists at all) + C3 C2 AND profit_factor @ 0.6% > 1.0 (survives the cheapest rate reachable) + and the same at 1.2%, the rate actually paid. +""" + +import json +import time +from concurrent.futures import ProcessPoolExecutor, as_completed +from decimal import Decimal + +DB = "/Users/elmehdiaitbrahim/keel/keel.db" +OUT_DIR = "/private/tmp/claude-501/-Users-elmehdiaitbrahim-Development-work-CodeGate-keel/28ff9a61-09d1-498b-b325-1631c0662734/scratchpad" +JSONL_PATH = f"{OUT_DIR}/intersection.jsonl" +JSON_PATH = f"{OUT_DIR}/intersection.json" + +UNIVERSE = [ + "BTC-USD", "ETH-USD", "ADA-USD", "LINK-USD", "LTC-USD", "SOL-USD", + "XLM-USD", "PAXG-USDT", "BCH-USD", "AAVE-USD", "DOGE-USD", "DOT-USD", + "UNI-USD", "ZEC-USD", "ALGO-USD", "FET-USD", "CRV-USD", "ICP-USD", + "AVAX-USD", "NEAR-USD", "XRP-USD", "PAXG-USD", "WLD-USD", "TON-USD", +] + +ARM_B_EXCLUDE = {"ZEC-USD", "FET-USD", "SOL-USD", "DOGE-USD", "ETH-USD", "BTC-USD"} +ARM_B_UNIVERSE = [a for a in UNIVERSE if a not in ARM_B_EXCLUDE] + +FEES = ["0", "0.006", "0.012"] +SLIPPAGE = Decimal("0.0005") + +RULES = ["turtle", "rsi", "pullback"] + + +def build_jobs(): + jobs = [] + for rule in RULES: + for asset in UNIVERSE: + jobs.append(("A", rule, asset)) + for asset in ARM_B_UNIVERSE: + jobs.append(("B", "turtle", asset)) + return jobs + + +def make_rule(arm, rule, asset): + from keel.strategy.rules.turtle_breakout import TurtleBreakout + from keel.strategy.rules.rsi_meanrev import RsiMeanReversion + from keel.strategy.rules.pullback_continuation import PullbackContinuation + + if arm == "A": + if rule == "turtle": + return TurtleBreakout(product_id=asset) + elif rule == "rsi": + return RsiMeanReversion(product_id=asset) + elif rule == "pullback": + return PullbackContinuation(product_id=asset) + else: + raise ValueError(f"unknown rule {rule}") + elif arm == "B": + if rule != "turtle": + raise ValueError("arm B is turtle only") + return TurtleBreakout( + product_id=asset, + entry_lookback=336, + exit_lookback=80, + atr_stop_mult=Decimal("2"), + target_rr=Decimal("6"), + adx_threshold=25.0, + ) + else: + raise ValueError(f"unknown arm {arm}") + + +def run_job(job): + arm, rule, asset = job + from keel.data.db import connect + from keel.data.repository import Repository + from keel.strategy import backtest as bt + from keel_core.types import Granularity + + rows = [] + try: + repo = Repository(connect(DB)) + candles = repo.get_candles(asset, Granularity.ONE_HOUR) + except Exception as e: + for fee in FEES: + rows.append({ + "arm": arm, "rule": rule, "product": asset, "fee": fee, + "error": f"{type(e).__name__}: {e}", + }) + return rows + + for fee in FEES: + try: + rule_obj = make_rule(arm, rule, asset) + result = bt.backtest( + rule_obj, candles, fee_pct=Decimal(fee), slippage_pct=SLIPPAGE + ) + rows.append({ + "arm": arm, + "rule": rule, + "product": asset, + "fee": fee, + "n_trades": int(result.n_trades), + "win_rate": float(result.win_rate), + "profit_factor": float(result.profit_factor), + "expectancy": float(result.expectancy), + "max_drawdown": float(result.max_drawdown), + }) + except Exception as e: + rows.append({ + "arm": arm, "rule": rule, "product": asset, "fee": fee, + "error": f"{type(e).__name__}: {e}", + }) + return rows + + +def done_combos(): + """Combos already in the JSONL. The first run died at 25/90 with the pool still + holding results; the file is append-only and each row names its own combo, so the + completed set is recoverable exactly. Resume rather than redo.""" + import os + + if not os.path.exists(JSONL_PATH): + return set() + done = set() + for line in open(JSONL_PATH): + line = line.strip() + if not line: + continue + try: + r = json.loads(line) + except json.JSONDecodeError: + continue # torn final line from a kill mid-write + done.add((r["arm"], r["rule"], r["product"])) + return done + + +def main(): + jobs = build_jobs() + total_declared = len(jobs) + done = done_combos() + jobs = [j for j in jobs if j not in done] + + # Run order is a scheduling choice ONLY -- every declared job still runs, and a backtest is + # independent of the order it is dispatched in, so this cannot touch the result. + # `rsi` at the shipped defaults is by far the slowest cell: `backtest()` calls `rule.detect()` + # only while flat, so the rule that almost never fires pays full level-detection cost on + # nearly all 44k bars. Leaving it first starves the two arms that actually answer the + # open question (Arm B's out-of-sample transfer, and pullback's first-ever run) for an hour. + prio = {("B", "turtle"): 0, ("A", "pullback"): 1, ("A", "rsi"): 2} + jobs.sort(key=lambda j: prio.get((j[0], j[1]), 3)) + total = len(jobs) + print(f"Declared jobs: {total_declared}; already done: {len(done)}; running: {total}", flush=True) + + start = time.time() + completed = 0 + all_rows = [] + + with open(JSONL_PATH, "a") as jf: + with ProcessPoolExecutor(max_workers=8) as ex: + futures = {ex.submit(run_job, job): job for job in jobs} + for fut in as_completed(futures): + job = futures[fut] + try: + rows = fut.result() + except Exception as e: + arm, rule, asset = job + rows = [{ + "arm": arm, "rule": rule, "product": asset, "fee": fee, + "error": f"{type(e).__name__}: {e}", + } for fee in FEES] + + for row in rows: + jf.write(json.dumps(row) + "\n") + jf.flush() + all_rows.append(row) + + completed += 1 + if completed % 10 == 0 or completed == total: + elapsed = time.time() - start + print(f"Progress: {completed}/{total} jobs, elapsed={elapsed:.1f}s", flush=True) + + # Serialise the WHOLE jsonl, not just this process's rows -- on a resume `all_rows` + # holds only the newly-run subset and dumping it would silently drop the earlier run. + final = [json.loads(x) for x in open(JSONL_PATH) if x.strip()] + with open(JSON_PATH, "w") as f: + json.dump(final, f, indent=2) + + elapsed = time.time() - start + print(f"Done. {len(all_rows)} new rows, {len(final)} total. elapsed={elapsed:.1f}s", flush=True) + + +# --------------------------------------------------------------------------------------------- +# POST-HOC PROBES. Written AFTER seeing Arm A; diagnostics, never gates. See the module docstring. +# --------------------------------------------------------------------------------------------- + +#: Complete calendar years only. 2021 starts mid-July and 2026 ends in July/August, so both are +#: partial buckets -- and including them CHANGES the streak verdict for 4 of 21 assets. Both +#: bucketings are reported for that reason; neither is authoritative. +FULL_YEARS = (2022, 2023, 2024, 2025) +ALL_YEARS = (2021, 2022, 2023, 2024, 2025, 2026) + + +def _closed(result): + """Closed trades with a realised pnl. + + `summarize()` already excludes open trades from every aggregate, and an unclosed position is + carried in `.trades` with `pnl=None` for visibility. Marking it to market at the final candle + was considered and rejected: it injects an unrealised price into a realised-pnl metric and + flatters trend-following, which tends to be holding a winner when a series truncates. + """ + return [t for t in result.trades if t.outcome != "open" and t.pnl is not None] + + +def probe_tail(result): + """Profit factor with the best 1 and best 3 winning trades deleted, losses all retained. + + A deliberately harsh stress, NOT an unbiased estimator: it answers "how few trades carry + this?", not "what is the edge?". Concentration is expected of a breakout rule -- a fat right + tail IS the strategy -- so a low ex-top3 is evidence about effective sample size, not about + whether edge exists. + """ + pnl = sorted((float(t.pnl) for t in _closed(result)), reverse=True) + wins = [x for x in pnl if x > 0] + gross_loss = -sum(x for x in pnl if x < 0) + if not gross_loss or not wins: + return {} + return { + "gross_pf": sum(wins) / gross_loss, + "top1_share": wins[0] / sum(wins), + "top3_share": sum(wins[:3]) / sum(wins), + "ex_top1": sum(wins[1:]) / gross_loss, + "ex_top3": sum(wins[3:]) / gross_loss, + } + + +def probe_regime(result): + """Gross profit factor per calendar year, and the worst run of consecutive losing years. + + Reported, never gated, and the write-up says why at length. Two independent reasons: + (a) it is underpowered -- for an edgeless strategy yearly outcomes are near coin-flips, and + only 8 of 16 length-4 sequences contain two consecutive losses, so a + no-consecutive-losing-years rule passes pure noise about half the time; + (b) ZEC under `pullback_continuation` is the counter-example that settles it -- the only + combination in the study with no losing complete year (1.06/1.17/1.16/1.13), at a gross + profit factor of 0.875. Stationary at losing slightly, reliably, forever. + """ + import datetime as dt + + per = {} + for t in _closed(result): + ts = getattr(t, "exit_ts", None) or getattr(t, "entry_ts", None) + if ts: + per.setdefault(dt.datetime.utcfromtimestamp(ts).year, []).append(float(t.pnl)) + + pfs = {} + for year, trades in per.items(): + won = sum(v for v in trades if v > 0) + lost = -sum(v for v in trades if v < 0) + pfs[year] = (won / lost) if lost else float("inf") + + def streak(years): + run = worst = 0 + for year in years: + if year not in pfs: + continue + run = run + 1 if pfs[year] < 1.0 else 0 + worst = max(worst, run) + return worst + + return {"pfs": pfs, "streak_all": streak(ALL_YEARS), "streak_full": streak(FULL_YEARS)} + + +if __name__ == "__main__": + main() diff --git a/docs/experiments/trials-ledger.jsonl b/docs/experiments/trials-ledger.jsonl index 6c626aae..25e9a701 100644 --- a/docs/experiments/trials-ledger.jsonl +++ b/docs/experiments/trials-ledger.jsonl @@ -78,3 +78,5 @@ {"decision":"diagnostic_only","kind":"ablation","params":{"answer":"The objection is correct and does not rescue the rule. Re-tuning lifts mean PF 0.419 -> 0.634 (+51%) and the longest lookbacks win as wall-clock theory predicts, but 0 of 144 configs average PF > 1.0 across six assets and 0 of 864 cells clear PF 1.0 at n >= 100.","assets":["ZEC-USD","FET-USD","SOL-USD","DOGE-USD","ETH-USD","BTC-USD"],"baseline_config":"entry40/exit20/atr2/rr6/adx25 -- the daily-tuned default","best_config_clearing_n100":"entry168/exit80/atr2/rr6/adx25","best_config_overall":"entry336/exit80/atr2/rr6/adx25 -- mean n 91, below min_trades=100","companion":"hourly-turtle-granularity-2026-08-11","document":"docs/experiments/2026-08-11-hourly-param-sweep-turtle-breakout.md","eliminated_levers":"asset selection (19 assets), granularity (daily->hourly), parameters (144 sets)","fee_model":"taker 1.2% on both legs, slippage 0.0005 -- STRICTER than the project's own gate, which still prices 0.6% at backtest.py:171, portfolio_sim.py:225, paper.py:46 and cli.py:1691. Numbers here are not comparable to `keel rules backtest` output.","granularity":"ONE_HOUR","grid":{"adx_threshold":[25,20],"atr_stop_mult":[2,3],"entry_lookback":[40,80,120,168,240,336],"exit_lookback":[20,40,80],"target_rr":[6,3]},"grid_pre_declared":"Grid was declared in the script docstring before the run and is unchanged; no cells added after seeing results, none dropped. Hence provenance a_priori.","question":"Does ANY hourly-appropriate parameter set give turtle_breakout a positive edge? Tests the strongest objection to hourly-turtle-granularity-2026-08-11: that entry_lookback=40 means 40 HOURS on hourly bars, so the params were never tuned for this granularity.","recommended_next":"(a) ZEC regime split, 8 configs, pre-surge vs surge window. (b) maker-fill model with tuned params -- halving round-trip friction 2.50% -> 1.30% is worth more than this entire grid (baseline at maker = 0.640 mean vs best-of-144 at taker = 0.634). (c) NOT another parameter sweep.","script":"docs/experiments/2026-08-11-hourly-param-sweep-turtle-breakout.py","structural_finding":"Edge and sample size move in opposite directions: Spearman(mean PF, mean n) = -0.77 over the 144 configs. The configs closest to break-even are exactly the ones that cannot clear min_trades=100. Not a tuning problem; the shape of the strategy.","validation":"Screening result only. No walk-forward, no CSCV/PBO, no deflated Sharpe, one rule, one grid, 6 of 19 assets, one window, same cached candles as everything else.","winning_cells":"All 8 cells above PF 1.0 are ZEC-USD, n 49-67, all adx_threshold=25, all at entry_lookback 240 or 336. ZEC's 180-day median daily quote volume is ~30x its full-history median ($37.9M vs $1,229,309) and its close ran 36.92 -> 510.43 in 2025H2; the other five assets all trade BELOW their own historical median. Probably regime, not edge -- but no regime split was run, so this is argued, not measured."},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"e8a19b2497896c5a55d43de55ed74df15aa0bace5b766037734593efd90198f1","provenance":"a_priori","row_hash":"092bf9baf1818c7c5094c4bd1dba4e6dbbbef515a5032b07e23beee56ac6d05d","rule":"turtle_breakout","series_missing":true,"session":"hourly-turtle-param-sweep-2026-08-11","summary":{"assets_tested":6,"baseline_mean_pf":"0.4187","baseline_mean_pf_six_assets_at_maker":"0.640","baseline_rank_of_144":112,"best_cell_n_trades":50,"best_cell_pf":"1.3033","best_cell_pf_at_n_ge_100":"0.8821","best_mean_pf":"0.6335","best_mean_pf_improvement_pct":"51.3","best_mean_pf_n_ge_100":"0.5845","cells_above_pf_1":8,"cells_above_pf_1_and_n_ge_100":0,"cells_with_n_ge_100":707,"code_changed":0,"configs_changed":0,"configs_mean_pf_above_1":0,"configs_tested":144,"expected_false_positive_cells_at_p05":"43.2","fee_pct":"0.012","mean_n_by_entry_336":"97.9","mean_n_by_entry_40":"280.7","mean_pf_by_entry_336":"0.5381","mean_pf_by_entry_40":"0.4070","n_trials":864,"pf_mean_864":"0.4745","pf_median_864":"0.4763","pf_min_864":"0.0952","pf_q1_864":"0.3207","pf_q3_864":"0.6213","round_trip_friction":"0.0250","rules_demoted":0,"spearman_pf_vs_n_trades":"-0.7688","trial_errors":0,"win_rate_mean":"0.2410","zec_liquidity_ratio_180d_over_full_history":"30.9"},"timestamp":1786496720,"trial_id":"hourly-turtle-param-sweep-2026-08-11"} {"decision":"diagnostic_only","kind":"ablation","params":{"assets":["ZEC-USD","FET-USD","SOL-USD","DOGE-USD","ETH-USD","BTC-USD"],"document":"docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev.md","fee_model":"taker 1.2% on both legs, slippage 0.0005. Passed explicitly, so #247's change of backtest()'s default (#247, 1dfe1fb) cannot affect these numbers regardless of which side of that commit a cell was executed on.","granularity":"ONE_HOUR","grid":{"atr_mult":["1.2","2.5"],"fixed_rr":["1.5","2","3"],"overbought":[70.0,80.0],"oversold":[15.0,20.0,25.0,30.0],"require_divergence":[false,true],"support_proximity_pct":["0.005 (pinned at the shipped default -- NOT swept)"]},"ledger_split":"Recorded as its own row rather than merged with the diagnostic grid. See the document's section 10: provenance is a per-row claim about a pre-registration and no single declaration covers both grids; the diagnostic grid was declared AFTER seeing this grid's data; and the ledger's job is to stop trials being re-spent, which a merged row would defeat by burying this dead end inside a row headlined by the diagnostic result.","provenance_note":"The grid in the script docstring is the grid that ran; no cells were added after seeing results and none were dropped -- the run was truncated, not edited. Hence a_priori.","question":"First test of a rule other than turtle_breakout. rsi_meanrev is registered in RULE_REGISTRY and had never been backtested. Mean reversion is the adversarial test of the turtle findings: it profits from exactly the conditions that hurt a breakout. If it also comes back negative, the claim generalises from one Donchian breakout to the venue.","raw_data_snapshot":"rsi_results.jsonl frozen at 136204 bytes, md5 5588cfda7340501772ae10cb23a1a447, 449 lines of which 448 parse and 1 is a torn final line from a kill mid-write. The producing process was STILL RESIDENT and still appending hours after the grid was abandoned -- abandonment was a decision to stop reading the output, not to stop the job -- so the first analysis pass read a moving file and did not reproduce. Every figure in this row is re-derived from the frozen snapshot.","script":"docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev-sweep.py","status":"ABANDONED MID-RUN. Results were never used for any edge claim. DO NOT RE-RUN THIS GRID AS DECLARED -- it is a recorded dead end and re-running it would spend the remaining 144 cells to reconfirm what its own output already recorded.","trial_accounting":"448 rows written, of which 432 are distinct cells and 16 are re-runs produced by the resume path after a kill mid-write (all 16 returned identical results). All 448 are counted: a duplicate buys no information but consumed a draw. The 144 declared cells that never ran are NOT counted.","what_was_mis_read":"The conclusion drawn at the same moment -- 'rsi_meanrev is unmeasurable' -- was NOT correct, and is an artifact of summarising 448 cells by their median when three quarters of them were spent on settings the same output showed to be dead. 54 of the 448 rows (48 distinct cells) already cleared min_trades=100. Those 48 cells give best PF 0.4718 (DOGE, n=142), median PF 0.2243, and ZERO above PF 1.0 -- against the purpose-built diagnostic grid's 0.4556 / 0.2092 / zero. The abandoned grid already contained the answer, to within 0.016 in max PF and 0.015 in median PF. Mis-centring costs trials; mis-reading costs a finding already paid for.","why_abandoned":"Mis-centred AND mis-ordered. `oversold` is the first key in the grid dict so it varies slowest under itertools.product: the blocks that ran are oversold 15/20/25 complete (144 distinct cells each = 432) and the ENTIRE oversold=30 block -- 144 cells, the setting most likely to fire of the four declared -- never ran. By the time it was stopped the run had already recorded, in the output available when the diagnostic grid was declared, that oversold 15 gives median n=2 against oversold 20's 25, and require_divergence=True gives median n=1 against False's 20; over the full 448 rows those settle at 2.5/16.5/57.5 by oversold and 6/41 by divergence. Half the remaining 144 cells carry require_divergence=True. Continuing would have spent 144 further trials, 72 of them on a filter already measured firing once or twice in five years. Stopping was correct."},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"092bf9baf1818c7c5094c4bd1dba4e6dbbbef515a5032b07e23beee56ac6d05d","provenance":"a_priori","row_hash":"260702bc8597123863e20b97a74c9c5eb14a04cc2147e4d4f7f4ee19ce85182e","rule":"rsi_meanrev","series_missing":true,"session":"fee-curve-and-rsi-meanrev-2026-08-12","summary":{"assets_tested":6,"best_pf_at_n_ge_100":"0.4718","cells_above_pf_1_and_n_ge_100":0,"cells_declared":576,"cells_distinct":432,"cells_duplicate_rows":16,"cells_never_fired":36,"cells_never_run":144,"code_changed":0,"configs_changed":0,"distinct_cells_above_pf_1":19,"distinct_cells_at_or_above_n_100":48,"fee_pct":"0.012","max_n_among_cells_above_pf_1":16,"max_n_trades":224,"median_n_oversold_15":"2.5","median_n_oversold_20":"16.5","median_n_oversold_25":"57.5","median_n_require_divergence_false":41,"median_n_require_divergence_true":6,"median_n_trades":17,"median_n_trades_distinct_cells":"16.5","median_pf_at_n_ge_100":"0.2243","n_trials":448,"rows_at_or_above_n_100":54,"rows_below_n_10":168,"rules_demoted":0,"torn_lines":1,"trial_errors":0},"timestamp":1786531189,"trial_id":"rsi-meanrev-grid-abandoned-2026-08-12"} {"decision":"diagnostic_only","kind":"ablation","params":{"answer":"The inference is HALF RIGHT and the wrong half is the half that would have set the roadmap. At fee_pct=0 turtle_breakout's best-swept configuration is profitable on 4 of 4 assets (BTC 1.090, ETH 1.458, SOL 1.533, ZEC 2.713) and rsi_meanrev is not (BTC 0.775 n=255, FET 0.907 n=284; ZEC 1.094 n=341 is the sole exception and breaks even at a 0.093% fee). Turtle has a gross edge that cost destroys; rsi_meanrev has no gross edge for cost to destroy. Same symptom at 1.2%, different disease. Pooling them predicts that maker fills rescue both -- which cutting cost to ZERO does not.","changed_nothing":"documentation and two copied scripts only -- no code, no config, no parameter, no rule status, no version bump, no demotion. config.live-sandbox.yaml untouched for the third document running. The fill model is deliberately NOT fixed here.","companions":["hourly-turtle-granularity-2026-08-11","hourly-turtle-param-sweep-2026-08-11","rsi-meanrev-grid-abandoned-2026-08-12"],"diagnostic_grid":{"atr_mult":["1.5"],"fixed_rr":["2"],"level_min_touches":[2,3],"overbought":[70.0],"oversold":[25.0,30.0,35.0],"require_divergence":[false],"support_proximity_pct":["0.005","0.02","0.05"]},"diagnostic_result":"104 of 108 cells clear n>=100 and ZERO of 108 clear PF 1.0. Median n 322.5, max n 887, median PF 0.2092, best PF 0.4556, win rates 14.9-34.0%. Per-asset best PF / median n: BTC 0.0953/134, DOGE 0.3149/423, ETH 0.1444/513, FET 0.4171/304, SOL 0.2102/427.5, ZEC 0.4556/331. Spearman(n_trades, PF) = +0.034, i.e. nothing: the tightest corner has median n 127 / median PF 0.2496 and the loosest has median n 778.5 / median PF 0.2145. Loosening every gate multiplies the sample by six and leaves the profit factor where it was -- the signature of a per-trade loss rate that is structural, not a sampling artifact.","document":"docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev.md","fee_curve":"turtle_breakout, hourly, slippage 0.0005 throughout, PF by fee_pct 0/0.001/0.002/0.003/0.004/0.006/0.012: BTC (n=123) 1.090/0.961/0.854/0.764/0.688/0.564/0.333; ETH (n=121) 1.458/1.330/1.217/1.117/1.028/0.875/0.556; SOL (n=92) 1.533/1.441/1.356/1.279/1.208/1.083/0.801; ZEC (n=50) 2.713/2.525/2.355/2.202/2.063/1.822/1.303. Break-even fees were MEASURED by 18 bracketing cells, not interpolated. Brackets (last cell above 1.0 / first cell below): BTC 0.060%->1.0096 / 0.070%->0.9971 => 0.068%; ETH 0.430%->1.0025 / 0.440%->0.9944 => 0.433%; SOL 0.750%->1.0005 / 0.800%->0.9751 => 0.751%; ZEC 1.700%->1.0191 / 1.750%->0.9957 => 1.741%; rsi_meanrev ZEC 0.090%->1.0025 / 0.095%->0.9977 => 0.093%. Measuring mattered: the first SOL bracket (0.80/0.85/0.90%) was chosen from a linear interpolation predicting 0.78% and ALL THREE cells came back below 1.0, because PF is convex in the fee and a chord under a convex curve crosses late. The interpolation erred by 3bp in the direction that flatters the maker argument, on the asset it most depends on. A second round of cells was run and both rounds are counted.","fee_curve_configs":{"BTC-USD":"entry168/exit80/atr3/rr6/adx25","ETH-USD":"entry240/exit80/atr3/rr3/adx25","SOL-USD":"entry336/exit80/atr3/rr6/adx20","ZEC-USD":"entry336/exit20/atr2/rr6/adx25","note":"each is the argmax of that asset's 144-cell slice of the 864-trial sweep in hourly-turtle-param-sweep-2026-08-11 -- see selection_bias"},"fee_curve_rule":"turtle_breakout (40 of the 52 fee cells). This row's `rule` field reads rsi_meanrev because 108 of its 160 trials are the rsi_meanrev diagnostic grid. A purist would give the fee curve its own row; it is folded in here because it SELECTS NOTHING -- it re-prices six already-ledgered configurations at additional fee rates -- and a row implies a search that did not happen. The count is reported either way.","fill_model_prerequisite":"A FEE-RATE CHANGE WITHOUT A FILL-MODEL CHANGE WOULD BE A WORSE COSTING ERROR THAN THE ONE #247 JUST FIXED, because it would be deliberate. #247 (1dfe1fb) corrected an error whose whole content was that the RATE did not match the FILL MODEL: backtest fills market-style at next-bar open -- a marketable order crossing the spread, i.e. taker -- and was priced at maker in four places. Setting fee_pct=0.006 on the same simulator re-creates that error exactly, with the corrected code in the tree and the commit that corrected it in the log. Maker economics require simulating a passive queue: (1) a resting order fills only when price trades THROUGH the limit, not when it merely touches it; (2) a breakout entered as a resting limit fills on the pullback, which selects disproportionately for breakouts that FAILED -- so 1.083 and 1.822 are taker fills priced at a maker rate, an upper bound and not an estimate; (3) unfilled signals need a policy, and reducing n makes the two rescued assets LESS promotable, not more; (4) intrabar resolution needs re-deriving, since the conservative fallback was designed for market-style fills. NOT FIXED HERE.","maker_pivot":"JUSTIFIED, NARROWLY, AND BY TURTLE ALONE. At fee_pct=0.006 (round-trip friction 1.30% against taker's 2.50%): SOL 1.083 and ZEC 1.822 clear; ETH 0.875 is a near miss against a 0.433% requirement; BTC needs 0.068%, nine times below the 0.60% maker rate of the <$1k-30d-volume band this account sits in, and not a fee-tier problem. It buys rsi_meanrev NOTHING -- not less than hoped, nothing, because the rule is under water at zero cost.","question":"TWO questions, deliberately kept apart. (a) FEASIBILITY: is there any configuration under which rsi_meanrev fires enough to be evaluated at all? Primary metric n_trades, NOT profit factor, declared as such in the script docstring before the run. (b) THE DECISIVE TEST of the inference drawn from the two turtle documents -- that execution cost is the binding constraint SYSTEM-WIDE, since two structurally opposite strategies both fail at 1.2%. Set fee_pct to ZERO, which is strictly better than any execution any venue will offer and therefore bounds from above everything an execution fix could buy, and see which rule recovers.","recommended_next":"(a) Build the maker-fill model as ENGINEERING, with the four requirements above, and test it against a pre-declared configuration set; the honest null is that adverse selection eats enough of the 1.30%-vs-2.50% gap to leave SOL and ZEC short, with both already below the promotion floor. (b) Characterise the ASSET dimension -- a 31x spread in break-even is the largest measured lever in this document and can be asked with the existing harness. (c) NOT further rsi_meanrev tuning: it is negative at zero cost on 2 of 3 assets, its 108-cell response surface is flat to within 0.03 PF per asset, and firing it six times more often does not move it. A different mean-reversion entry is a new rule with a new pre-registration, not a continuation of this one.","script":"docs/experiments/2026-08-12-fee-curve-and-rsi-meanrev-diag.py","selection_bias":"STATED AT FULL STRENGTH. Every configuration in the fee curve is the argmax of its asset's 144-cell slice of an 864-trial sweep, selected on the same data it is now re-priced on, at n = 123 (BTC) / 121 (ETH) / 92 (SOL) / 50 (ZEC). These are maxima of 144 draws and are NOT edge estimates; they must never be quoted as any asset's expected profit factor. What they legitimately support is a COMPARISON, and the bias runs AGAINST the finding: it inflates turtle's cells, and turtle is the arm that wins, while the rsi_meanrev cells are not maxima of anything -- the diagnostic grid's declared objective was n_trades. ANY MAKER RE-TEST MUST BE A FRESH PRE-DECLARED HYPOTHESIS on configurations chosen without reference to the 864-cell ranking, or the bias is laundered into a headline.","structural_finding":"Break-even varies by a factor of ~26 across four assets running the same rule on the same clock over the same window (BTC 0.068% to ZEC 1.741%), against parameter effects of a few percent in the 864-cell sweep. The ASSET is a far larger lever than tuning and nobody has characterised what drives it. Worse for the maker pivot: the two assets that clear break-even at the 0.6% maker rate are SOL (n=92) and ZEC (n=50), both BELOW min_trades=100, while the two with enough trades to promote -- ETH (n=121, 0.875) and BTC (n=123, 0.564) -- stay unprofitable. That is the companion sweep's Spearman -0.77 bind between edge and sample size reappearing on a completely different axis, which is the first evidence that it is structural rather than a property of one parameter search.","validation":"Screening result only. No walk-forward, no out-of-sample split, no CSCV/PBO and no deflated Sharpe on these configurations -- #247 wired the dormant PBO gate into promotion and none of these trials can feed it, because backtest emits aggregates and no per-trade series (series_missing). 4 of 19 assets on the fee curve, 6 of 19 in the grids, 3 of 19 in the zero-fee rsi_meanrev arm; same cached candles and same 2021-2026 cycle as everything else. slippage_pct held at 0.0005 in every cell, so 'zero fee' is zero FEE and not zero cost -- round-trip friction at fee_pct=0 is still 0.10% of notional.","wrong_diagnosis_recorded":"TWO wrong diagnoses, recorded because a document that hides its wrong turns teaches nothing about how the right answer was reached. (1) support_proximity_pct was predicted to be the binding gate on trade count and the grid was built to widen it 10x. It was NOT: over 36 matched triples, 0.005 -> 0.05 moves n by a median factor of 1.186 (range 1.071-1.426); the headline pair BTC 255 -> 290 is 1.137. (2) The CORRECTION -- that the binding gates were oversold and level_min_touches, the latter pinned at its default in the first grid and therefore never tested -- is also half wrong: over 54 matched pairs, level_min_touches 3 -> 2 moves n by a median factor of 1.185, indistinguishable from the proximity gate it was supposed to displace. The one gate that binds is oversold, alone: 25 -> 30 is x2.183 and 25 -> 35 is x3.933 over 36 matched pairs. The entire jump in median n from 17 to 322.5 is bought by oversold and require_divergence, BOTH of which were axes in the first grid; the one axis the second grid added contributed x1.19. So 'rsi_meanrev is unmeasurable' was an artifact of grid centring, not a property of the rule -- and the first grid could have answered the question without this one."},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"260702bc8597123863e20b97a74c9c5eb14a04cc2147e4d4f7f4ee19ce85182e","provenance":"a_priori","row_hash":"706422899e8fb759a4732e39aa63e79c3ebcc3371622e4f0ad9193e08ab3740e","rule":"rsi_meanrev","series_missing":true,"session":"fee-curve-and-rsi-meanrev-2026-08-12","summary":{"assets_tested_diagnostic_grid":6,"assets_tested_fee_curve":4,"best_pf":"0.4556","breakeven_fee_rsi_zec":"0.00093","breakeven_fee_turtle_btc":"0.00068","breakeven_fee_turtle_eth":"0.00433","breakeven_fee_turtle_sol":"0.00751","breakeven_fee_turtle_zec":"0.01741","breakeven_spread_across_four_assets":"25.6","cells_above_pf_1":0,"cells_tested":108,"cells_with_n_ge_100":104,"code_changed":0,"configs_changed":0,"diagnostic_grid_trials":108,"fee_curve_and_zero_fee_cells":61,"fee_curve_cells_rsi":15,"fee_curve_cells_turtle":46,"gate_effect_level_min_touches_3_to_2":"1.185","gate_effect_oversold_25_to_30":"2.183","gate_effect_oversold_25_to_35":"3.933","gate_effect_support_proximity_10x":"1.186","max_n_trades":887,"median_n_trades":"322.5","median_pf":"0.2092","min_n_trades":80,"min_pf":"0.0680","min_trades_floor":100,"n_trades_turtle_btc":123,"n_trades_turtle_eth":121,"n_trades_turtle_sol":92,"n_trades_turtle_zec":50,"n_trades_zero_fee_rsi_btc":255,"n_trades_zero_fee_rsi_fet":284,"n_trades_zero_fee_rsi_zec":341,"n_trials":169,"pf_zero_fee_rsi_btc":"0.7748","pf_zero_fee_rsi_fet":"0.9067","pf_zero_fee_rsi_zec":"1.0939","pf_zero_fee_turtle_btc":"1.0896","pf_zero_fee_turtle_eth":"1.4580","pf_zero_fee_turtle_sol":"1.5331","pf_zero_fee_turtle_zec":"2.7132","round_trip_friction_maker":"0.0130","round_trip_friction_taker":"0.0250","rsi_assets_above_pf_1_at_fee_0":1,"rsi_assets_tested_at_fee_0":3,"rules_demoted":0,"spearman_n_trades_vs_pf":"0.0342","trial_errors":0,"trials_this_line_of_work_cumulative":1481,"trials_today_both_rows":617,"turtle_assets_above_pf_1_at_fee_0":4,"turtle_assets_above_pf_1_at_maker_0006":2,"turtle_assets_above_pf_1_at_taker_0012":1,"turtle_assets_tested_at_fee_0":4,"win_rate_max":"0.3400","win_rate_min":"0.1492"},"timestamp":1786531189,"trial_id":"rsi-meanrev-diagnostic-and-fee-curve-2026-08-12"} +{"decision":"diagnostic_only","kind":"ablation","params":{"arm":"A -- shipped constructor defaults, ZERO free parameters","assets":24,"construction":"each rule built with product_id= and nothing else","criteria_declared_before_run":"C1 n>=100; C2 C1 and gross PF>1; C3 C2 and PF@0.6%>1","dca_excluded":"scheduled accumulation, not a signal edge","document":"docs/experiments/2026-08-12-shipped-defaults-intersection.md","fees":["0","0.006 maker","0.012 taker"],"granularity":"ONE_HOUR","predeclaration_location":"the dispatch brief, NOT the script docstring -- weaker than the other harnesses in this directory, and recorded as such in the script and the write-up","rules":["turtle_breakout","rsi_meanrev","pullback_continuation"],"slippage_pct":"0.0005 pinned explicitly, not inherited","trials":"72 combinations x 3 fee levels = 216 backtests"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"706422899e8fb759a4732e39aa63e79c3ebcc3371622e4f0ad9193e08ab3740e","provenance":"a_priori","row_hash":"74cfa02fa95ee5d5cafe5ee6dda36ea69c3755317722233d6d2745b390243651","rule":"turtle_breakout+rsi_meanrev+pullback_continuation","series_missing":true,"session":"shipped-defaults-intersection-2026-08-12","summary":{"pullback_gross_pos":"1","pullback_maker_pos":"0","pullback_median_gross":"0.9292","pullback_n100":"4","rsi_median_gross":"1.1631","rsi_n100":"0","taker_survivors_all_rules":"0","trade_count":25,"turtle_gross_pos":"7","turtle_maker_pos":"1","turtle_median_gross":"0.9892","turtle_n100":"21"},"timestamp":1786000000,"trial_id":"shipped-defaults-intersection-arm-a-2026-08-12"} +{"decision":"diagnostic_only","kind":"ablation","params":{"adx_threshold":25.0,"arm":"B -- out-of-sample transfer of the 864-trial sweep winner","atr_stop_mult":"2","consumes_no_new_selection":"the config was fixed before this arm ran; this arm spends no further multiple-testing budget, it only tests generalisation","document":"docs/experiments/2026-08-12-shipped-defaults-intersection.md","entry_lookback":336,"evaluated_on":"the 18 assets NOT in that six -- disjoint from the selection","exit_lookback":80,"selected_on":["ZEC-USD","FET-USD","SOL-USD","DOGE-USD","ETH-USD","BTC-USD"],"target_rr":"6","trials":"18 combinations x 3 fee levels = 54 backtests"},"per_bar_pnl":[],"per_trade_pnl":[],"prev_hash":"74cfa02fa95ee5d5cafe5ee6dda36ea69c3755317722233d6d2745b390243651","provenance":"fitted","row_hash":"19ecedc7148cb12c8a33d59f5e9cbbba5145beb1ddabd7cbffa1e63dd755ba6c","rule":"turtle_breakout","series_missing":true,"session":"shipped-defaults-intersection-2026-08-12","summary":{"in_sample_mean_net_pf":"0.6335","oos_assets_over_floor":"1","oos_mean_gross_pf":"1.5317","oos_mean_net_pf":"0.6346","oos_median_n":"84","oos_median_net_pf":"0.5138","trade_count":18,"verdict_not_overfit":"1"},"timestamp":1786000001,"trial_id":"shipped-defaults-intersection-arm-b-2026-08-12"}