Repository navigation
feat: test martingales for which e-BH on running maxima rejects with probability 2Kα/(K+1), so that the bounds 4α/3 and 3α/2 are attained - #453
Conversation
…probability 2Kα/(K+1), so that the bounds 4α/3 and 3α/2 are attained For every finite type of K hypotheses and every level α ≤ (K + 1)/(2K), test martingales for one filtration, with every hypothesis true, make e-BH at level α applied to their running maxima reject with probability exactly 2Kα/(K + 1). For two and three hypotheses this attains the merged upper bounds 4α/3 and 3α/2, and for at least two hypotheses and every level 0 < α < 1 the rejection probability exceeds α. This serves the EValues roadmap, Layer 8, item 3. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0196ftMVbgjSJzPQEwdek3kM
AI review — changes requestedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
…te EVR26 Note 18 Move `measurableSet_comap_top_iff` to `MeasureTheory/MeasurableSpace/Constructions.lean` (as `MeasurableSpace.measurableSet_comap_top_iff`), `setLIntegral_sum_smul_dirac` to `MeasureTheory/Measure/Weighted.lean` (generalised to points `x k`), and `card_le_of_forall_lt_add` to the new `Data/Int/Interval.lean` (as `Int.card_le_toNat_of_forall_lt_add`), all public. Cite [EVR26, Note 18], reword the non-control bullet as an existence statement, factor the fiber sum of `ladder_climb` and the uniform weight sum, and replace bare `rfl` closes and long rewrite chains. Co-Authored-By: Claude Code <noreply@github.com>
… helpers of the e-BH lower bound Move `fiberAvg`, `sum_mul_fiberAvg`, `fiberFiltration` and `isTestMartingale_fiberAvg` to the new public module `LMLExplore/EValues/TestSupermartingale/Finite.lean`, with `fiberAvg_def`, `fiberAvg_of_sum_eq_zero` and `measurableSet_fiberFiltration_iff`. Make the general helpers public in their natural homes: `Fintype.card_filter_forall` (new `Data/Fintype/BigOperators.lean`), `LMLExplore.sum_range_add_mod` / `prod_range_add_mod` (`Algebra/BigOperators/Intervals.lean`) and `Nat.add_mod_eq_iff` (new `Data/Nat/Mod.lean`). Split `ladder_climb` into `sum_ladderWeight_climb`, `sum_ladderWeight_mul_ladderValue_climb` and `prod_card_climb`, without the `set`-defeq `rfl` and the goal-closing `ring_nf`; derive `two_mul_sum_range_sub` from `Finset.sum_range_reflect` and `Finset.sum_range_id_mul_two`; close the three-hypothesis corollary with `ENNReal.div_eq_div_iff`. Co-Authored-By: Claude Code <noreply@github.com>
There was a problem hiding this comment.
🟡 naming — request_changes claude/claude-opus-5-5
The new names describe their conclusions in Mathlib terms and sit in the right namespaces, except that fiberAvg_of_sum_eq_zero leaves its conclusion = 0 out of its name.
LMLExplore/EValues/TestSupermartingale/Finite.lean:66—fiberAvg_of_sum_eq_zeroprovesfiberAvg w f Z y = 0, but the name never sayseq_zero, so it reads as an unfolding lemma rather than a vanishing lemma. Fix: Rename tofiberAvg_eq_zero_of_sum_eq_zero, and update the module docstring and the PR description.
Reply in this thread to contest a finding; that re-runs only this rubric and posts an answer here. (To fix it, just push a commit — that re-reviews on its own. To contest again after an answer, post a NEW reply rather than editing an old one.)
claude/claude-opus-5-5 · 26s · 126k in (110.1k cache read, 15.9k cache write) / 2.5k out tokens · reviewing this diff · rubric ·
…q_zero Co-Authored-By: Claude Code <noreply@github.com>
…rvals, credit the paper project in TestSupermartingale.Finite Co-Authored-By: Claude Code <noreply@github.com>
There was a problem hiding this comment.
🟡 reuse — request_changes claude/claude-opus-5-5
The new infrastructure has no counterpart in Mathlib or LMLExplore, and the construction reuses the existing API. However, setLIntegral_sum_smul_dirac re-proves Mathlib's setLIntegral_dirac for each summand instead of calling it.
LMLExplore/MeasureTheory/Measure/Weighted.lean:191— The per-summand steprw [Measure.restrict_smul, lintegral_smul_measure, restrict_dirac]; split_ifs <;> simp [lintegral_dirac]re-proves Mathlib'sMeasureTheory.setLIntegral_dirac(restrict_dirac + lintegral_dirac). Fix: Replace the step withrw [Measure.restrict_smul, lintegral_smul_measure, setLIntegral_dirac]; split_ifs <;> simp, orsimp [Measure.restrict_smul, lintegral_smul_measure, setLIntegral_dirac, mul_ite].
Reply in this thread to contest a finding; that re-runs only this rubric and posts an answer here. (To fix it, just push a commit — that re-reviews on its own. To contest again after an answer, post a NEW reply rather than editing an old one.)
claude/claude-opus-5-5 · 44s · 396.1k in (372.2k cache read, 23.9k cache write) / 3.8k out tokens · reviewing this diff · rubric ·
There was a problem hiding this comment.
🟡 generality — request_changes claude/claude-opus-5-5
The public statements are at the natural level, and the card_eq_two/card_eq_three corollaries are derived from the general theorem; only two_mul_sum_range_sub is under-generalized, being stated for n + 1 where the formula holds for every n.
LMLExplore/Algebra/BigOperators/Intervals.lean:102—two_mul_sum_range_subis stated only forrange (n + 1)with(n + 1 - i)and(n + 1) * (n + 2). That is the successor case of Gauss' formula, which holds for everyn. Fix: State it astheorem two_mul_sum_range_sub (n : ℕ) : 2 * ∑ i ∈ range n, (n - i) = n * (n + 1)(the samesum_range_reflectproof works) and call it atn + 1inLowerBound.lean:601. Update the module docstring entry to match.
Reply in this thread to contest a finding; that re-runs only this rubric and posts an answer here. (To fix it, just push a commit — that re-reviews on its own. To contest again after an answer, post a NEW reply rather than editing an old one.)
claude/claude-opus-5-5 · 26s · 107.5k in (100.7k cache read, 6.8k cache write) / 2.3k out tokens · reviewing this diff · rubric ·
|
Closing after 3 repair attempts left review findings unresolved at head |
This pull request constructs, for every finite type
ιofKhypotheses and every levelα ≤ (K + 1) / (2 * K), test martingales for one filtration, with every hypothesis true, for which e-BH at levelαapplied to the running maxima⨆ n, M i nrejects with probability exactly2 * K * α / (K + 1). For two and three hypotheses this shows that the upper bounds4 * α / 3and3 * α / 2of #429 are attained. It adds the subject fileLMLExplore/EValues/MultipleTesting/RunningMaxima/LowerBound.lean, next to the two files of #429, and the general infrastructure it uses as public declarations in their natural homes (listed below): four new files and additions to three existing ones. It removes or renames no declaration.Main declarations
exists_isTestMartingale_measure_exists_selfConsistent_iSup_eq (ι : Type*) [Fintype ι] (hα : α ≤ (Fintype.card ι + 1) / (2 * Fintype.card ι)): there areΩ : Type, a measurable space, a probability measureP, a filtrationℱandM : ι → ℕ → Ω → ℝ≥0∞with everyM ia test martingale forℱand{P}, andP {ω | ∃ R : Finset ι, R.Nonempty ∧ ∀ i ∈ R, (Fintype.card ι : ℝ≥0∞) ≤ α * R.card * ⨆ n, M i n ω} = 2 * Fintype.card ι * α / (Fintype.card ι + 1).exists_isTestMartingale_measure_exists_selfConsistent_iSup_eq_of_card_eq_two (hι : Fintype.card ι = 2) (hα : α ≤ 3 / 4): the same with probability4 * α / 3, the bound ofmeasure_exists_selfConsistent_iSup_le_of_card_eq_two.exists_isTestMartingale_measure_exists_selfConsistent_iSup_eq_of_card_eq_three (hι : Fintype.card ι = 3) (hα : α ≤ 2 / 3): the same with probability3 * α / 2, the bound ofmeasure_exists_selfConsistent_iSup_le_of_card_eq_three.exists_isTestMartingale_lt_measure_exists_selfConsistent_iSup (hι : 2 ≤ Fintype.card ι) (hα0 : α ≠ 0) (hα1 : α < 1): the same objects withα < P {…}: e-BH on running maxima does not control the false discovery rate at its level.The rest of
LowerBound.lean(the ladder construction and its cycle lemma) is private.New public infrastructure
LMLExplore/EValues/TestSupermartingale/Finite.lean(new): Doob martingales on a finite space.ProbabilityTheory.fiberAvg w f Z y(thew-weighted average ofZover the fiberf ⁻¹' {y},0on a fiber of weight0),fiberAvg_def,fiberAvg_eq_zero_of_sum_eq_zero,sum_mul_fiberAvg,ProbabilityTheory.fiberFiltration info hinfo(the filtrationMeasurableSpace.comap (info t) ⊤for refining mapsinfo t),measurableSet_fiberFiltration_iff, andisTestMartingale_fiberAvg(the fiber averages ofZform a test martingale for∑ ω, w ω • δ_ωwhen∑ ω, w ω * Z ω ≤ 1).LMLExplore/Data/Int/Interval.lean(new):Int.card_le_toNat_of_forall_lt_add, a finite set of integers whose elements are less thandapart has at mostd.toNatelements.LMLExplore/Data/Fintype/BigOperators.lean(new):Fintype.card_filter_forall,#{f : ∀ i, α i | ∀ i, Q i (f i)} = ∏ i, #{a | Q i a}.LMLExplore/Data/Nat/Mod.lean(new):Nat.add_mod_eq_iff,(h + u) % K = x ↔ h = (x + K - u) % Kforh, u, x < K.LMLExplore/Algebra/BigOperators/Intervals.lean(existing):LMLExplore.prod_range_add_modand its additive versionLMLExplore.sum_range_add_mod,∑ x < K, f ((z + x) % K) = ∑ x < K, f x. AlsoLMLExplore.two_mul_sum_range_sub,2 * ∑ i < n + 1, (n + 1 - i) = (n + 1) * (n + 2)(Gauss' formula for the reflected indices).LMLExplore/MeasureTheory/Measure/Weighted.lean(existing):setLIntegral_sum_smul_dirac, the set lower integral against∑ k, w k • δ_{x k}with weights inℝ≥0∞.LMLExplore/MeasureTheory/MeasurableSpace/Constructions.lean(existing):MeasurableSpace.measurableSet_comap_top_iff, the measurable sets ofMeasurableSpace.comap f ⊤are the unions of fibers off.Roadmap coverage (EValues, Layer 8, item 3)
Covered: the second half of the bullet "E-BH applied to the running maxima
⨆ n, M i nof test supermartingales for a common filtration …", that is, "for everyKand everyα ≤ (K + 1) / (2 * K)someKtest martingales for a common filtration make it reject with probability2 * K * α / (K + 1), so that the two bounds are attained", and the first sentence of the bullet ("does not control the false discovery rate atα" [TGdH25]) as a corollary. The upper bounds of the bullet are merged (#429). With this pull request the bullet is complete. The rest of item 3 (closed e-BH, closed BH, closed testing, online e-BH, SeqE-Guard) and the rest of Layer 8 are left.The construction has no published source: the roadmap cites it
[EVR26, Note 18]. The proof here is a rewrite of its Lean proof in the paper project (EValuesResearch/Results/RunningMaxima.lean, commitffba78a), which the module docstrings ofLowerBound.leanandTestSupermartingale/Finite.leancredit. The same file was in the first version of #429 and was moved out when that pull request exceeded the scope limit; this is that file, adapted tomain.Notes for reviewers
Rwith(Fintype.card ι : ℝ≥0∞) ≤ α * R.card * ⨆ n, M i n ωfor everyi ∈ R. E-BH and self-consistent sets belong to Layer 8.2, which is not built yet.Ktest martingales" is any finite typeιwithK = Fintype.card ι, as in the upper bounds, rather thanFin K; the construction works onFin Kand transports alongFintype.equivFinOfCardEqwithmeasure_exists_selfConsistent_le_of_equivof feat: e-BH on the running maxima of test supermartingales rejects with probability at most 4α/3 for two hypotheses and 3α/2 for three #429 (used in both directions). "Every hypothesis true" is the null{P}for every hypothesis, the smallest null. The sample space is inType. "So that the two bounds are attained" is stated as the two corollaries…_of_card_eq_two(α ≤ 3 / 4, probability4 * α / 3) and…_of_card_eq_three(α ≤ 2 / 3, probability3 * α / 2), with the same right-hand sides as the upper bounds.K = 0andα = 0are allowed (constant test martingales1, probability0). The non-control corollary holds for every0 < α < 1, not onlyα ≤ (K + 1) / (2 * K): above that level the construction at the level(K + 1) / (2 * K)rejects with probability1, and the rejection event grows with the level.Bool × Fin K × (Fin K → Fin K): a coin that istruewith probability2 * K * α / (K + 1), a uniform cyclic shift of the positions of the processes and uniform heights of theKplayers. The player at positionlclimbs during the timesK * l, …, K * l + K - 1, which reveal its height. Each process is the Doob martingale of its terminal valueK / α, earned when the coin istrue, the players before it have not won and its height is the top one; at heighthit is then worthK / ((K - h) * α), so e-BH rejects exactly when the coin istrue. The initial expectations are at most1by a cycle lemma: among theKcyclic shifts of any heights, at mostK - jleave the firstjplayers without a win, which bounds the probability that they do not win by(K - j) / K.lintegral_sum_smul_dirac(inMeasureTheory/Measure/Weighted.lean) has real weightsENNReal.ofReal (w k)and integrates over the whole space; the construction needsℝ≥0∞weights and set integrals, hencesetLIntegral_sum_smul_dirac. A search of Mathlib, Tau Ceti, LML and Statlib found no lemma characterizing the measurable sets ofMeasurableSpace.comap f ⊤as the unions of fibers off, no counting lemma for functions with a coordinatewise condition beyondFintype.card_piFinset, no shift-invariance of sums over a period of residues and noNat.add_mod_eq_iff. The constant test martingale isisTestMartingale_one.Roadmap: EValues
Paper: RemyDegenne/evalues_research
Item: R39
🤖 Generated with Claude Code
https://claude.ai/code/session_0196ftMVbgjSJzPQEwdek3kM