Skip to content

C1a — Lookahead and recursive bias detection #440

Description

@eaitbrahim

PRD §3 C1. Freqtrade ships lookahead-analysis and recursive-analysis; keel has no equivalent.

Why this is not covered by PBO/CSCV

They catch different failure modes. PBO/CSCV asks "did I overfit by selecting among many configs?" Lookahead bias asks "does this strategy use information it could not have had?" A strategy with a single config, never swept, can still be catastrophically lookahead-biased — and PBO would report nothing.

This is the failure that makes a backtest look brilliant and a live run lose money.

What Freqtrade does

  • lookahead-analysis — re-runs backtests on truncated data slices and diffs the signals. If a signal on bar N changes when bars after N are removed, the strategy is reading the future.
  • recursive-analysis — detects indicators whose values change depending on how much history is fed in (recursive/non-deterministic formulas), a lighter check.

Fit with keel

keel's engine already has the right seam: Rule.detect(candles_by_tf) -> Setup | None is documented as a pure decision with no side effects (keel/strategy/rules/base.py:106-157). Purity is exactly what makes truncation-diffing meaningful — call detect() on a truncated series and on the full series and compare.

Worth checking against the engine's higher-timeframe bearish-bias veto (keel/strategy/engine.py:196-213), which reads "the coarsest available higher timeframe" — a natural place for leakage if the coarse bar is not yet closed.

Scope

  • keel rules lookahead <rule-id> and a recursive-indicator variant.
  • Report which bar/indicator diverged, not just pass/fail.
  • Decide whether this becomes a promotion-gate input (see the PRD's open question 1). Recommendation: yes — a rule that fails lookahead analysis should not be promotable, which is precisely the enforcement posture no competitor has.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requestresearchMeasurement, validation or KB work (Research & validation)

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions