Repository navigation
feat: the Bernoulli maximal inequality, counts versus pseudo-counts [DLB17, Lemma F.4] - #292
Open
RemyDegenne wants to merge 1 commit into
Conversation
…DLB17, Lemma F.4] Co-Authored-By: Claude Code <noreply@github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds the Bernoulli maximal inequality [DLB17, Lemma F.4], [EV26, Lemma 20]. If
X iisℱ (i + 1)-measurable with values in[0, 1]almost surely andP iis a version ofE[X i | ℱ i], then for everyδ > 0,ℙ(∃ n, ∑_{i < n} X i < (1/2) ∑_{i < n} P i - log (1 / δ)) ≤ δ(LMLExplore.measure_exists_sum_lt_half_mul_sum_sub_le). This is the "counts versus pseudo-counts" tooln ≥ n̄/2 - log(3SAH/δ)that ReinforcementLearning Layer 19.2 (RF-Express [MDJKLV21, Theorem 1]) lists among the tools to prove. Its instance at the visits of a step–state–action pair, at confidenceδ/(3SAH), gives the counts event of that analysis. The other Layer 19.2 tools are either in flight (KL-to-Bernstein transport and variance transfers #291, the self-bounding inequality #287) or still to port (the logarithmic sum bound). After this PR, the milestone still needs the logarithmic sum bound, the threshold β from Layer 3.4, and the RF-Express algorithm with its PAC and sample-complexity theorem onstatesEnv(Layer 7.7).The new file
LMLExplore/Probability/Martingale/BernoulliMaximal.lean(160 lines) adds:LMLExplore.condExp_exp_condExp_div_two_sub_le_one: the one-step boundE[exp (E[X | m] / 2 - X) | m] ≤ 1forXmeasurable with values in[0, 1]a.s. It is proved from the chordexp (-x) ≤ 1 - (1 - e⁻¹) xon[0, 1](Mathlib'sconvexOn_exp), the pull-out property, and1 - e⁻¹ ≥ 1/2(Mathlib'sReal.exp_neg_one_lt_half).LMLExplore.measure_exists_sum_lt_half_mul_sum_sub_le: the time-uniform inequality. It goes through LMLExplore'sLMLExplore.supermartingale_exp_sum_of_condExp_exp_le_oneand Ville's inequalityMeasureTheory.Supermartingale.measure_exists_le_le.LMLExplore's fixed-
ηFreedman inequalityLMLExplore.measure_exists_mul_sum_condExp_sq_add_log_div_le_sum_le, applied toP i - X iwithη = 1/2, gives only- 2 log (1 / δ). The sharper constant needs the dedicated one-step bound above.This PR ports
Essakine2026Tight/ForLMLExplore/Probability/Martingale/Ville.leanfrom RemyDegenne/Essakine2026Tight at commit 9c77165d56f81c1db7a9835068356bb14886a3be (Apache 2.0, author Rémy Degenne). It generalizes and re-proves that material as follows:X i ∈ {0, 1}everywhere, together withP i ∈ [0, 1]andℱ i-strong measurability ofP i. HereX i ∈ [0, 1]holds only almost surely, andP iis any version of the conditional mean. Its range and measurability are then derived rather than assumed, and the convexity chord replaces the{0, 1}identity.LMLExplorenamespace, next to Freedman's inequality, rather than inMeasureTheory.condExp_exp_sub_le_onetocondExp_exp_condExp_div_two_sub_le_one, to name its conclusion.No declaration is removed or renamed on
main.Roadmap: ReinforcementLearning
Paper: RemyDegenne/Essakine2026Tight
Item: R12
🤖 Prepared with Claude Code