Repository navigation
Identification: mean functionals, product models, rate functions and general stopping-time tools - #10
Merged
Merged
Conversation
…general stopping-time tools Drafted at the request of the code owner, following a review of LMLExplore's generality against the literature. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Amends the Identification roadmap so that its targets have the generality of the pure-exploration literature wherever the roadmap, and the LMLExplore code built on it, are narrower. It also makes this roadmap the owner of three sequential-learning tools: the chain rule at a stopping time (Layer 2.1), the run-to-stream event transfer (Layer 4.1) and conditional domination of moment generating functions (Layer 4.2). The Bandits (#11) and ReinforcementLearning (#12) amendments cite these, together with Wald's identity (Layer 0.3) and the rate function (Layer 1.2).
Why. Several targets fix a special case that the literature does not:
expFamilyDivlives on the open interval of means, the Chernoff bound holds only at means, and the boundary GLR extension was a separate object. The Cramér–Chernoff rate function [BLM13, Section 2.2; DZ98, Section 2.2] gives all three at once. Families had a real statistic only, so the two-parameter Gaussian was handled through closed forms.IsAlgEnvSeq.klDiv_map_history_stepKernelandIsAlgEnvSeq.martingale_noiseSum, work for any environments and any events.Changes (only
LMLRoadmaps/Identification/):Environment.means/noiseSum/streamMeasure/bregDiv, and Tau Ceti'sfenchelConjugate,frameOperatorand tilted laws; the mean functional and product models in the Conventions;ℝ≥0∞-valued thresholdsβ n hdepending on the round and the history. Algorithms built onIdentTask.comap. The classical tasks takeν : Θ → Kernel α 𝓨and a mean functionalmean : Θ → α → ℝ, with the current tasks asmean = id. Two tasks are added to the list: top-$m$ with tolerance and multiplicative ε-best arm. The model of all bandits with real observations,α → ProbabilityMeasure ℝ, is defined, and its σ²-sub-Gaussian andT. The Cramér rate function is the primary object (ℝ≥0∞-valued, Tau Ceti'sfenchelConjugate), withgammaMeasure_tilted_mul); the exponential laws and the known-mean Gaussian become shape-1 and shape-½ corollaries.bregDiv, and the likelihood-ratio martingale. The Gaussian family with unknown variance is an instance. The mean parametrization stays one-dimensional.klBer). The existing alternative-cost lemmas are restated under continuity and monotonicity of per-arm divergences only.klDiv_stoppedHistMeasure_le) is its corollary, and ReinforcementLearning Layer 6.2 builds on it. 2.3. The single-answer bound holds at any model with one correct answer, in any task. The multiple-answer bound holds at every model.Task/Continuity.leanandTask/Oracle.leanfor every task. 3.4–3.5. Per-arm families with a common interval of means. 3.6. Gaussian costs in design-operator form with arm-dependent variances; the cost of a finite union of half-spaces covers best arm identification, top-$m$, thresholding and half-space identification, and Layer 9.2 is a special case [FJJR19, Theorem 1].MGFDominatedAboveand the Gaussian family; the exponential supermartingale with Ville's inequality [HRMS20]; self-normalized deviations [GC11, Theorem 10]; the KK21 bounds withSuggested.lean.baiTasktakes a mean functional. New signatures:IdentTask.comap,productBandit,kInfLower/kInfUpperwith the best-arm formula,subgaussianBaiTask,rateFunctionwith itsfenchelConjugateform and the Chernoff bound,vecExpFamilywith KL asbregDiv, the general Wald identity, the stopping-time chain rule, the event transfer,Kernel.HasCGFBound/HasCondCGFBoundwith the supermartingale, and product-model continuity.trackAndStopnow takes an estimator, an evidence function and thresholdsβ δ n h, and its asymptotic optimality is stated for any single-answer task on per-arm exponential families.Points to check.
eventually_expectedStoppingTime_glrt_le_of_isBigOalready gives everyα → ProbabilityMeasure ℝ, notKernel α ℝ, becauseIdentTaskneeds a Markov kernel for every model.stationaryEnv, which this roadmap already used, is LML'sEnvironment.banditat the pin LMLExplore uses. It is left for a separate update of all three roadmaps.Suggested.leanbuilds.Drafted with Claude Opus 5.5 at the request of the code owner.
🤖 Generated with Claude Code