Repository navigation
feat: Thompson sampling for exponential families, the Jeffreys prior, and Gaussian Thompson sampling as an instance - #282
Merged
lmlexplore-automation[bot] merged 3 commits intoNov 12, 2026
Conversation
… and Gaussian Thompson sampling as an instance Co-Authored-By: Claude Code <noreply@github.com>
RemyDegenne
commented
Nov 12, 2026
Contributor
Author
AI review — approvedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
… out of the posterior proofs
Add ProbabilityTheory.cond_smul ((c • μ)[|s] = μ[|s] for c ∉ {0, ∞}) and
LMLExplore.deriv_cgf_gaussianReal_eq (the function-level Gaussian mean map),
and use them in expFamilyPosterior_smul, expFamilyPosterior_gaussianReal_volume,
iteratedDeriv_two_cgf_gaussianReal and
map_deriv_cgf_expFamilyPosterior_gaussianReal_jeffreysPrior.
Co-Authored-By: Claude Code <noreply@github.com>
RemyDegenne
commented
Nov 12, 2026
…sampling module prose Co-Authored-By: Claude Code <noreply@github.com>
Merged
via the queue into
LeanMachineLearning:main
with commit Nov 12, 2026
2374bfe
24 of 25 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR defines Thompson sampling for exponential families with priors on the natural parameter, and the Jeffreys prior. It also proves that Gaussian Thompson sampling$\pi_J(d\theta)=\sqrt{\psi''(\theta)},d\theta$ on $\Theta_\rho$ [KKM13, Section 2]. Gaussian Thompson sampling with variance proxy $\sigma^2$ and flat prior (LMLExplore's $\sigma^2$ and its Jeffreys prior, a constant multiple of Lebesgue measure; define it as that instance." The names
gaussianTS σ2is the instance with the Gaussian family of varianceσ²and its Jeffreys prior. This is the Bandits roadmap target Layer 5.1, Algorithm: "For families $(\rho_a)a$ of Layer 3 with log-partition functions $\psi_a=$cgf id (ρ a)and priors $\pi_a$, σ-finite measures on $\Theta{\rho_a}$, define Thompson sampling. […] The Jeffreys prior isgaussianTS[…]) is the instance with the Gaussian family of varianceexpFamilyTSandjeffreysPriorfollow the roadmap'sSuggested.lean.Roadmap: Bandits
LMLExplore/Probability/Distributions/ExponentialFamily/Posterior.lean(new, 180 lines) holds the exponential-family part.Measure.expFamilyPosterior ρ T ν s Sis the conjugate posterior of the natural parameter aftersobservations whose statistics sum toS. It is defined as(ν.withDensity fun θ ↦ exp (θ S - s ψ(θ)))[|Θ], withψ = cgf T ρandΘthe interior ofintegrableExpSet T ρ, so it works for any prior, improper ones included. The file provesexpFamilyPosterior_apply,isProbabilityMeasure_expFamilyPosterior(normalizing constant positive and finite),isProbabilityMeasure_expFamilyPosterior_of_ne_zero,expFamilyPosterior_eq_zero_iff, anIsZeroOrProbabilityMeasureinstance,expFamilyPosterior_smul(the posterior does not change when the prior is scaled) andmeasurable_expFamilyPosterior(measurable inS). It also definesMeasure.jeffreysPrior ρ T, with itsSigmaFiniteinstance. Mathlib'sProbabilityTheory.posterioris not reused: it is a kernel for a finite prior and is determined only up to null sets, while a sampling rule needs the posterior at every value of(s, S)and for improper priors.ExponentialFamily/Gaussian.leangets the Gaussian computations:iteratedDeriv_two_cgf_gaussianReal(ψ'' = v);jeffreysPrior_gaussianReal(the Jeffreys prior is√v • volume);expFamilyPosterior_gaussianReal_volumeandexpFamilyPosterior_gaussianReal_jeffreysPrior(the posterior is𝒩((S - s m)/(s v), 1/(s v)), by completing the square);map_deriv_cgf_expFamilyPosterior_gaussianReal_jeffreysPrior(its image under the mean map is𝒩(S/s, v/s)).Probability/Moments/Basic.leangetsmeasurable_mgfandmeasurable_cgffor s-finite measures. These are not in Mathlib at the pinned version, and the measurability of the posterior inSneeds them.LMLExplore/Bandits/ThompsonSampling/Basic.lean(new, 233 lines) has the algorithm:expFamilyTSStackLaw ρ π s S:δ_⊤ifs = 0or the posterior is0, otherwise the image of the posterior byθ ↦ ψ'(θ)inEReal. It comes with_def,_zero,_of_expFamilyPosterior_eq_zero,_of_ne_zero, a probability instance and measurability.expFamilyTSLaw ρ π n h a, with_def, measurability andexpFamilyTSLaw_history.expFamilyTS ρ π := perturbedLeaderAlgorithm …, for families and priors that are s-finite, withexpFamilyTS_def.ExpFamilyTS.lintegral_pullCount_le: [LS, Theorem 36.2] for this algorithm against any stochastic bandit, in the reward-stack form, fromPerturbedLeader.lintegral_pullCount_le.Bandits/ThompsonSampling/Gaussian.leanprovesgaussianTSStackLaw_eq_expFamilyTSStackLawandgaussianTS_eq_expFamilyTS m hσ2. The second says that forσ2 ≠ 0,gaussianTS σ2 = expFamilyTS (fun a ↦ gaussianReal (m a) σ2) (fun a ↦ (gaussianReal (m a) σ2).jeffreysPrior id), for any base meansm.gaussianTSis not redefined as the instance because the two differ atσ2 = 0. The Gaussian family of variance0is a Dirac family whose Jeffreys prior is0, so every posterior is0and every perturbation is⊤.gaussianTS 0, by contrast, samplesδ_{μ̂}, and its existing lemmas (for exampleGaussianTS.pullCount_card_eq_one) hold for everyσ2. This is the same pattern asklucbFam_gaussianReal … = ucbF σ2in Layer 3.3.To keep the generic definition upstream of its Gaussian instance, the existing Gaussian Thompson sampling file moves into a directory. Only imports and the module path change; no declaration is renamed or removed.
LMLExplore.Bandits.ThompsonSamplingLMLExplore.Bandits.ThompsonSampling.GaussianWith this PR, Layer 5.1 is done except its Bernoulli completion example (the Jeffreys prior on the mean is Beta(1/2, 1/2)). What remains for Layer 5 is 5.3's [KKM13, Theorem 1]: the asymptotic regret of
expFamilyTSwithjeffreysPriorfor bandits whose arms are members of one family. It will consumeexpFamilyTS,jeffreysPrior,expFamilyPosteriorandExpFamilyTS.lintegral_pullCount_le.Sources: [KKM13] N. Korda, E. Kaufmann and R. Munos, Thompson sampling for 1-dimensional exponential family bandits, NeurIPS 2013, Section 2 (posterior and Jeffreys prior); [LS] Lattimore and Szepesvári, Bandit Algorithms, Chapter 36.
🤖 Prepared with Claude Code