Skip to content

Identification: mean functionals, product models, rate functions and general stopping-time tools - #10

Merged
RemyDegenne merged 1 commit into
mainfrom
identification/generality
Nov 5, 2026
Merged

RemyDegenne merged 1 commit into
mainfrom
identification/generality

Conversation

@RemyDegenne

@RemyDegenne RemyDegenne commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

Amends the Identification roadmap so that its targets have the generality of the pure-exploration literature wherever the roadmap, and the LMLExplore code built on it, are narrower. It also makes this roadmap the owner of three sequential-learning tools: the chain rule at a stopping time (Layer 2.1), the run-to-stream event transfer (Layer 4.1) and conditional domination of moment generating functions (Layer 4.2). The Bandits (#11) and ReinforcementLearning (#12) amendments cite these, together with Wald's identity (Layer 0.3) and the rate function (Layer 1.2).

Why. Several targets fix a special case that the literature does not:

  • The classical tasks are defined on vectors of means, although the Conventions say a model need not be one. As a result, LMLExplore's LUCB guarantee goes through an auxiliary bandit, successive and median elimination cannot quantify over all σ²-sub-Gaussian bandits [EMM06, Theorems 8 and 10; KTAS12, Theorem 1], and linear tasks are defined a second time in Layer 9.
  • Transportation costs are computed for a single exponential family, although the proofs use only the product structure and per-arm divergences. The $\mathrm K_{\inf}$ costs [AJG20; JDK23a, (1)] were out of scope.
  • expFamilyDiv lives on the open interval of means, the Chernoff bound holds only at means, and the boundary GLR extension was a separate object. The Cramér–Chernoff rate function [BLM13, Section 2.2; DZ98, Section 2.2] gives all three at once. Families had a real statistic only, so the two-parameter Gaussian was handled through closed forms.
  • Joint continuity of $D(\theta,w,\neg i)$ is a hypothesis, discharged only for best arm identification with one family, so [DK19, Theorem 4] does not reach multiple-answer tasks.
  • The change of measure at stopping times, Wald's identity for rewards and the run-to-stream transfer are stated for stationary bandits and sums of real rewards. Their proofs, like LML's IsAlgEnvSeq.klDiv_map_history_stepKernel and IsAlgEnvSeq.martingale_noiseSum, work for any environments and any events.
  • GLR thresholds depend only on the round. This excludes the thresholds of [KK21] and the pairwise thresholds of best arm identification. The δ-correct thresholds and the optimality of Track-and-Stop are asked only for the identity statistic, and LMLExplore has them only for Gaussian arms. Each sampling rule still has to redo the step from converging allocations to sample complexity.
  • Successive rejects, sequential halving, median elimination, successive elimination and RAGE are specified separately, although they share one phased-elimination structure.

Changes (only LMLRoadmaps/Identification/):

  • Scope, Conventions, layer table. The three owned tools; a dependency table with Mathlib's conditional sub-Gaussian API, LML's Environment.means/noiseSum/streamMeasure/bregDiv, and Tau Ceti's fenchelConjugate, frameOperator and tilted laws; the mean functional and product models in the Conventions; ℝ≥0∞-valued thresholds β n h depending on the round and the history. Algorithms built on $\mathrm K_{\inf}$ stay outside the roadmap, while the $\mathrm K_{\inf}$ costs and lower bounds come in.
  • 0.3. Wald's identity $\mathbb E[\sum_{t<\tau}Y_t]=\mathbb E[\sum_{t<\tau}m_t(A_t)]$ for any environment, by stopping LML's feedback martingale. The finite-arm regrouping becomes a corollary.
  • 0.5 (new). A phased-elimination constructor: an active set of answers, a pull schedule, elimination maps and an output rule. It comes with phase-wise conditional independence, the fixed-budget property and the survival bound. Successive elimination, median elimination, successive rejects, sequential halving and RAGE are its instances.
  • 1.1. IdentTask.comap. The classical tasks take ν : Θ → Kernel α 𝓨 and a mean functional mean : Θ → α → ℝ, with the current tasks as mean = id. Two tasks are added to the list: top-$m$ with tolerance and multiplicative ε-best arm. The model of all bandits with real observations, α → ProbabilityMeasure ℝ, is defined, and its σ²-sub-Gaussian and $[0,1]$ classes carry the guarantees of Layers 7 and 8.1–8.2.
  • 1.2. Means and divergence are invariant under a change of base measure to a family member, and depend only on the law of T. The Cramér rate function is the primary object (ℝ≥0∞-valued, Tau Ceti's fenchelConjugate), with $\bar d(x,\mu)$ as its value for a family member, continuous on (closure of means) × means. The Chernoff bound holds at every level, and the two-sided bound $P(s,\bar d(\hat\mu_s,\mu)\ge z)\le2e^{-z}$ follows. Gamma laws of known shape are added (Tau Ceti's gammaMeasure_tilted_mul); the exponential laws and the known-mean Gaussian become shape-1 and shape-½ corollaries.
  • 1.3. Exponential families with a statistic in a finite-dimensional inner product space: tilt composition, restriction to lines, log-likelihood ratio, KL as LML's bregDiv, and the likelihood-ratio martingale. The Gaussian family with unknown variance is an instance. The mean parametrization stays one-dimensional.
  • 1.4 (new). Product models; $\mathrm K^\pm$ and the formula $\min_{b\ne a}\inf_u(w_a\mathrm K^-_a+w_b\mathrm K^+_b)$ for best arm identification, top-$m$ and thresholding. Instances: per-arm exponential families, the two-parameter Gaussian [JDK23a, Lemma 3], and laws on $[0,1]$ (comparison with klBer). The existing alternative-cost lemmas are restated under continuity and monotonicity of per-arm divergences only.
  • 2.1. The chain rule at a stopping time for any two algorithm-environment pairs [GMS19, Section 2; KCG16, Lemma 1]. The stationary-bandit decomposition (klDiv_stoppedHistMeasure_le) is its corollary, and ReinforcementLearning Layer 6.2 builds on it. 2.3. The single-answer bound holds at any model with one correct answer, in any task. The multiple-answer bound holds at every model.
  • 3.1. Upper semicontinuity and measurability of $D$ on any topological model space, and the costs of pullbacks. 3.3. Joint continuity of $D(\theta,w,A)$ for product models with continuous, locally inf-compact per-arm divergences and any set of alternatives [FKV14; DK19, Theorem 23, Lemmas 24–27]. This hypothesis discharges the one of Task/Continuity.lean and Task/Oracle.lean for every task. 3.4–3.5. Per-arm families with a common interval of means. 3.6. Gaussian costs in design-operator form with arm-dependent variances; the cost of a finite union of half-spaces covers best arm identification, top-$m$, thresholding and half-space identification, and Layer 9.2 is a special case [FJJR19, Theorem 1].
  • 4.1. The event transfer for any event about the first $s$ observations of each arm, in any observation space; LMLExplore's sum-based transfers become the case of sums. 4.2. The conditional-domination predicate, with the bridges to Mathlib, LML, MGFDominatedAbove and the Gaussian family; the exponential supermartingale with Ville's inequality [HRMS20]; self-normalized deviations [GC11, Theorem 10]; the KK21 bounds with $\bar d$; pairwise thresholds; and GK16 Proposition 12 thresholds for any exponential families. 4.3. Plug-in GLR statistics for any estimator, history-dependent thresholds, and covers of the alternatives (pairwise stopping). 4.4. "Allocations converge on good events + lower semicontinuity ⇒ sample complexity", and the almost-sure version [GK16, Proposition 13].
  • 5.1. [GK16, Lemma 19] in event form, for Top Two. 5.2. Track-and-Stop for any single-answer task and estimator: δ-PAC and asymptotically optimal for per-arm exponential families (constant $T^*$ for every polynomial $r$), and for the two-parameter Gaussian [JDK23a]. 6.3 goes through Layers 4.4 and 5.1.
  • 7, 8.1–8.2, 9.3. These algorithms are built on the template and stated on the restricted non-parametric tasks; [KK13, Theorem 1] is added for KL-LUCB. 8.3. Cramér's theorem for every light-tailed law, and the [GJ04] exponent with rate functions. 9.1–9.2. Linear tasks are the Layer 1.1 tasks with mean functional $\theta\mapsto(\langle\varphi(a),\theta\rangle)_a$, and the characteristic time is a case of Layer 3.6.
  • Completion checks and References are updated: [AJG20], [BLM13], [BMDG05], [BN78], [Bro86], [DZ98], [FKV14], [GC11], [HRMS20], [WJ08], and [LS, Section 4.6].
  • Suggested.lean. baiTask takes a mean functional. New signatures: IdentTask.comap, productBandit, kInfLower/kInfUpper with the best-arm formula, subgaussianBaiTask, rateFunction with its fenchelConjugate form and the Chernoff bound, vecExpFamily with KL as bregDiv, the general Wald identity, the stopping-time chain rule, the event transfer, Kernel.HasCGFBound/HasCondCGFBound with the supermartingale, and product-model continuity. trackAndStop now takes an estimator, an evidence function and thresholds β δ n h, and its asymptotic optimality is stated for any single-answer task on per-arm exponential families.

Points to check.

  • 3.4. For distinct per-arm families, the uniqueness of $w^*$ rests on a strict-concavity argument of the draft, not on a published proof. It assumes that all families have the same interval of means.
  • 3.3. The per-arm hypothesis (real, continuous, locally inf-compact divergences) and the joint continuity up to the boundary of the simplex are the draft's formulation of [FKV14; DK19, Theorem 23, Lemmas 24–27]. Check them against the paper.
  • 5.2. The asymptotic constant is $T^$ for every polynomial $r$, not [GK16]'s $\alpha T^$. This comes from the reduction of 4.4; LMLExplore's eventually_expectedStoppingTime_glrt_le_of_isBigO already gives every $c>1/\kappa$.
  • 1.1. The model of all bandits is α → ProbabilityMeasure ℝ, not Kernel α ℝ, because IdentTask needs a Markov kernel for every model.
  • Not included. Top Two allocations stay specific to best arm identification. The overlapping-hypotheses lower bound of Garivier–Kaufmann 2021 is not added. AdaHedge (6.1) keeps only its worst-case bound, without the second-order and $L^*$ bounds of [dRvEGK14].
  • Cross-roadmap. Bandits Layer 1 cites Layers 1.2, 4.1 and 4.2 here, while Layer 4 here cites Bandits Layer 1. The cycle is at the level of layers only, not of items.
  • Vocabulary. stationaryEnv, which this roadmap already used, is LML's Environment.bandit at the pin LMLExplore uses. It is left for a separate update of all three roadmaps.

Suggested.lean builds.

Drafted with Claude Opus 5.5 at the request of the code owner.

🤖 Generated with Claude Code

…general stopping-time tools

Drafted at the request of the code owner, following a review of LMLExplore's generality
against the literature.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant