Repository navigation
feat: asymptotic optimality of Thompson sampling with the Jeffreys prior, and the Bernoulli instance [KKM13, Theorem 1] - #324
Conversation
…ior, and the Bernoulli instance [KKM13, Theorem 1] Co-Authored-By: Claude Code <noreply@github.com>
There was a problem hiding this comment.
🟡 reuse — request_changes claude/claude-opus-5-5
The new Jeffreys-TS limit, consistency and Bernoulli results are new material that reuses the existing upper bound, Lai–Robbins and KL-UCB plumbing. However, the upper-bound wrapper copies the non-degeneracy and MGF-domination proof block of the KL-UCB wrapper line for line.
LMLExplore/Bandits/AsymptoticOptimality.lean:441— InexpFamilyTS_jeffreysPrior_regret_eventually_le_of_mem_image_expFamily, thehρ/hρs/hdomblock (lines 441-452) copies lines 296-307 ofklucbFam_regret_eventually_le_of_mem_image_expFamilyline for line (not_ae_eq_const_of_gap_pos×2,rw [Measure.expFamily_natParam_integral_of_mem_image ha (hν a)],(IdentDistrib.refl aemeasurable_id).mgfDominatedOn). Fix: Move this block into one shared lemma in theExpFamilysection, e.g.mgfDominatedOn_expFamily_natParam_of_mem_image (hν) (ha : ∀ c, ¬ id =ᵐ[ρ a] fun _ ↦ c) Λ : MGFDominatedOn id (ν a) id ((ρ a).expFamily id ((ρ a).natParam id (ν a)[id])) Λ. Call it from both wrappers, and inlinenot_ae_eq_const_of_gap_posat the call sites.
Reply in this thread to contest a finding; that re-runs only this rubric and posts an answer here. (To fix it, just push a commit — that re-reviews on its own. To contest again after an answer, post a NEW reply rather than editing an old one.)
claude/claude-opus-5-5 · 28s · 130.6k in (96k cache read, 34.6k cache write) / 2.6k out tokens · reviewing this diff · rubric
There was a problem hiding this comment.
🟡 proof-quality — request_changes claude/claude-opus-5-5
Proofs are short and avoid undocumented defeq tricks, but the step from the family-wide hypotheses (hM, hπ) to the hypotheses of the arm-wise upper bound is written out twice inline instead of being a lemma.
LMLExplore/Bandits/AsymptoticOptimality.lean:473—ExpFamilyTS.isConsistent_jeffreysPriorandtendsto_expFamilyTS_jeffreysPrior_regret_div_log(lines 501-507) both rebuildhμs/hμbviahM (bestArm ν) a ▸ hμ _/hM a (bestArm ν) ▸ hμ a,fun a _ ↦ hprop a, andhπsviane_top_of_le_ne_top (hπ _ _ (hμ _)) (measure_mono fun _ hθ ↦ le_trans hθ hm.le)before callingexpFamilyTS_jeffreysPrior_regret_eventually_le_of_mem_image_expFamily; the reusable fact (common means plus a finite prior below every mean give the regret bound) is never stated. Fix: Add a lemma (e.g.expFamilyTS_jeffreysPrior_regret_eventually_le_of_common_means) takinghM hprop hπ hν h hcand returning the eventualc log nbound, and use it in both theorems (thehμslocal intendsto_…is still needed foreventually_mul_log_le_integral_regret_expFamily).
Reply in this thread to contest a finding; that re-runs only this rubric and posts an answer here. (To fix it, just push a commit — that re-reviews on its own. To contest again after an answer, post a NEW reply rather than editing an old one.)
claude/claude-opus-5-5 · 20s · 47.6k in (41.9k cache read, 5.7k cache write) / 1.9k out tokens · reviewing this diff · rubric
AI review — changes requestedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
Add `mgfDominatedOn_expFamily_natParam_integral_of_mem_image` next to the exponential-family Cramér–Chernoff bounds and use it in both KL-UCB and Jeffreys-TS upper-bound wrappers; add `expFamilyTS_jeffreysPrior_regret_eventually_le_of_common_means` and use it in the consistency and limit theorems. Co-Authored-By: Claude Code <noreply@github.com>
This PR proves the limit form of [KKM13, Theorem 1] for Thompson sampling with the Jeffreys prior,
E[R_n] / log n → ∑_{a : Δ_a > 0} Δ_a / d(μ_a, μ*), together with its Bernoulli instance. This is the statement of Bandits Layer 5.3 ("Frequentist regret"): an upper bound in eventual form, matched by the Lai–Robbins lower bound of Layer 4.4. It also covers the completion check that "the asymptotic constants of Layers 3 and 5 coincide with the Lai–Robbins constants of Layer 4.4 … in particular for Gaussian and Bernoulli arms". The upper boundexpFamilyTS_jeffreysPrior_regret_eventually_le(#320) and the Gaussian limittendsto_gaussianTS_regret_div_log_gaussianBanditwere already onmain. Until now,mainhad no limit statement for Jeffreys-prior Thompson sampling on any non-Gaussian family.Roadmap: Bandits
Added to
LMLExplore/Bandits/AsymptoticOptimality.lean, next to the analogous KL-UCB statements (klucbFam_regret_eventually_le_of_mem_image_expFamily,KLUCBFam.isConsistent,tendsto_klucbFam_regret_div_log):expFamilyTS_jeffreysPrior_regret_eventually_le_of_mem_image_expFamily: the upper bound of feat: asymptotic optimality of Thompson sampling with the Jeffreys prior [KKM13, Theorem 1] #320 for bandits whose arm laws are members of the families. The domination and nondegeneracy hypotheses are discharged here, because each arm is its own member of meanμ_a.expFamilyTS_jeffreysPrior_regret_eventually_le_of_common_means: the same bound for families with a common set of means, proper Jeffreys posteriors and Jeffreys priors finite below every mean; the consistency and limit theorems below both use it.ExpFamilyTS.isConsistent_jeffreysPrior: consistency over those bandits, for families with a common set of means, Jeffreys posteriors that are proper after some observations, and Jeffreys priors that are finite on{θ | ψ'(θ) ≤ m}for every meanm.tendsto_expFamilyTS_jeffreysPrior_regret_div_logandisLittleO_expFamilyTS_jeffreysPrior_regret_sub_mul_log: [KKM13, Theorem 1], througheventually_mul_log_le_integral_regret_expFamily.tendsto_expFamilyTS_jeffreysPrior_regret_div_log_bernoulliBanditandisLittleO_…_bernoulliBandit: the Bernoulli instance, on Bernoulli bandits with means in(0, 1). It uses the Beta(1/2, 1/2) Jeffreys prior, and its constant is written(klBer (p a) (⨆ b, p b)).toReal, as intendsto_klucb_regret_div_log_bernoulliBandit.Added to
LMLExplore/Probability/Distributions/ExponentialFamily/Chernoff.lean:ProbabilityTheory.mgfDominatedOn_expFamily_natParam_integral_of_mem_image: a member of an exponential family is dominated on every set by the member of its mean. Both the existing KL-UCB wrapperklucbFam_regret_eventually_le_of_mem_image_expFamilyand the new Thompson-sampling wrapper use it instead of repeating the proof inline.Added to
LMLExplore/Probability/Distributions/ExponentialFamily/Bernoulli.lean, which the Bernoulli instance needs:isFiniteMeasure_jeffreysPrior_bernoulliMeasure_sigmoid: the Jeffreys prior has total massπ;isProbabilityMeasure_expFamilyPosterior_bernoulliMeasure_sigmoid_jeffreysPrior: the Jeffreys posteriors are probability measures for-1/2 < S < s + 1/2.No declarations are removed or renamed.
After this PR, Layer 5.3 remains open only for families whose Jeffreys prior is infinite below the optimal mean, such as exponential arms. The module docstring of
ThompsonSampling/Optimality.leanexplains why the [LS, Theorem 36.2] route fails there. The other open Bandits item, Layer 6.1, is covered by the open PR #323.🤖 Prepared with Claude Code