Repository navigation
feat: KL-UCB on gamma and exponential arms, and the rate function of the gamma family - #371
Conversation
…the gamma family Compute the extended divergence (Cramér rate function) of the gamma family of known shape at every level, the KL-UCB upper confidence level it determines, and deduce the asymptotic optimality of KL-UCB on bandits with gamma arms of known shapes and with exponential arms. Co-Authored-By: Claude Code <noreply@github.com>
|
Do check what the conflict actually is before resolving it mechanically. If |
…tial-rewards-worker1 # Conflicts: # LMLExplore/Probability/Distributions/ExponentialFamily/Exponential.lean # LMLExplore/Probability/Distributions/ExponentialFamily/Gamma.lean
AI review — approvedEach rubric is judged independently by multiple review agents; the PR merges only once every rubric is green — any rubric that is not green (changes requested, blocked, errored, stale, or not yet run) blocks the merge. See the rubrics.
♻️ = approved on an earlier commit, re-run before merge. Reviewing this diff at head |
…ositivity Co-Authored-By: Claude Code <noreply@github.com>
3af64fa
This PR adds the exponential-rewards completion example of KL-UCB for exponential families, together with its gamma generalization. It computes the extended divergence (the Cramér rate function) of the gamma family of known shape at every real level and the KL-UCB upper confidence level it determines. It then deduces that KL-UCB is asymptotically optimal on bandits with gamma arms of known, possibly different, shapes, and on bandits with exponential arms.
Roadmap target: Bandits, Layer 3 (KL-UCB for exponential families), the completion example "exponential rewards (
expMeasure, d(x,q)=x/q−1−log(x/q))". It is also an instance of Layer 4.5, the asymptotic optimality of KL-UCB for families with a common set of means, here(0, ∞). After this PR, the remaining Layer 3 completion examples are the Bernoulli boundary level U(0,c)=1−e^{−c} and the two-armed Bernoulli bandit.Roadmap: Bandits
New declarations (none removed or renamed):
ExponentialFamily/Gamma.lean:image_expFamily_gammaMeasure: the members are thegammaMeasure a b,b > 0.expFamilyRate_gammaMeasure: dbar(x, μ) = a (x/μ − 1 − log(x/μ)) for x > 0.expFamilyRate_gammaMeasure_of_nonpos: dbar = ∞ for x ≤ 0. This uses Tau Ceti'sae_pos_gammaMeasureandrateFunction_eq_top_of_ae_gt.le_expFamilyUpper_gammaMeasure_iff: q ≤ U(x, c) ↔ a (x/q − 1 − log(x/q)) ≤ c, for 0 < x ≤ q. This is the KL-UCB index.expFamilyUpper_gammaMeasure_of_nonpos: U(x, c) = x for x ≤ 0 and finite c.ExponentialFamily/Exponential.lean: the shape-1 corollariesimage_expFamily_expMeasure,expFamilyRate_expMeasure,expFamilyRate_expMeasure_of_nonpos,le_expFamilyUpper_expMeasure_iffandexpFamilyUpper_expMeasure_of_nonpos.Bandits/AsymptoticOptimality.lean:tendsto_klucbFam_gammaMeasure_regret_div_logandisLittleO_klucbFam_gammaMeasure_regret_sub_mul_log: for armsgammaMeasure (κ a) (r a)and KL-UCB for the families generated by anygammaMeasure (κ a) (s a), E[R_n]/log n → Σ_{Δ_a>0} Δ_a / (κ_a (μ_a/μ* − 1 − log(μ_a/μ*))), the Lai–Robbins constant.tendsto_klucbFam_expMeasure_regret_div_logandisLittleO_klucbFam_expMeasure_regret_sub_mul_log: the exponential instance.The bandit statements specialize the existing
tendsto_klucbFam_regret_div_log, the same way as its Gaussian instancetendsto_klucbFam_gaussianReal_regret_div_log. No Mathlib infrastructure is vendored.Local checks (not CI):
lake build --iofailsucceeded. The shared verification reportedscope: scoped: against da012591023b4d03a800af6fd377bc7c9b231392andlocal-axioms: checked 3 changed module(s), 61 declaration(s); 0 violation(s), and the module-system, header and environment-lint checks passed.🤖 Prepared with Claude Code