diff --git a/LMLRoadmaps/Refactors/README.md b/LMLRoadmaps/Refactors/README.md new file mode 100644 index 0000000..10712d5 --- /dev/null +++ b/LMLRoadmaps/Refactors/README.md @@ -0,0 +1,186 @@ +# Roadmap: Refactors toward the generality of later layers + +This roadmap restates material that LMLExplore already contains, in the generality and the place +that the later layers of the other roadmaps build on. Its targets are refactors of merged code: +each one replaces a development by a more general one, or moves it to the topic where its users +are, and keeps every result it replaces. New declarations appear only where the general statement +needs them. + +The targets of Layer 0 come before the items they unblock, which each target lists. A worker +choosing between a Layer 0 target and an item it unblocks takes the Layer 0 target first. + +## Scope and dependencies + +In scope: + +- online linear optimization: the FTRL development of `LMLExplore/OnlineLearning/FTRL.lean`; +- exponential families: `LMLExplore/Probability/Distributions/ExponentialFamily/`; +- Chernoff bounds: `LMLExplore/Probability/Moments/KLChernoff.lean` and + `LMLExplore/Probability/Distributions/ExponentialFamily/Chernoff.lean`; +- the transfer from runs of a bandit algorithm to LML's reward stream: + `LMLExplore/Bandits/SumRewards.lean`; +- Wald's identities, probability vectors on finite types, and the generic parts of + `LMLExplore/Bandits/Concentration.lean`. + +Relation to the other roadmaps. The [Bandits](../Bandits/README.md) and +[Identification](../Identification/README.md) roadmaps specify the mathematics these developments +contain, in the generality of the targets here (Bandits Layer 6 for online linear optimization, +Identification Layer 1.2 for exponential families); this roadmap restates the merged code in that +generality and moves it where its users are. It adds no target of those roadmaps. + +## Conventions + +- **A refactor keeps every result.** Each declaration it replaces is either derived from the new + general statement (when it is the form users call) or deleted with every use migrated to the + general statement. Deleted names leave no alias, wrapper or deprecated shim, as everywhere in + LMLExplore. +- **One target, one pull request**, attributed `Roadmap: Refactors`, citing the target number. + Moves and generalizations of the same development are one target; unrelated developments are + never bundled. Target 0.3 comes after target 0.2. +- **Names.** A generalized declaration keeps the name of the declaration it generalizes when the + name still describes it. +- **Losses are minimized** in online learning, as in the Bandits roadmap's Layer 6. + +## Layers and dependencies + +| Layer | Content | Prerequisites | Unblocks | +| --- | --- | --- | --- | +| 0 | Generalizations later layers build on | merged LMLExplore code | Bandits 3.2, 6.3, 6.4, 7, 10; Identification 1.2, 3.4, 4, 5, 7 | +| 1 | Consolidation | merged LMLExplore code | Identification 4, 7; Bandits 2.3, 7 | + +## Layer 0: Generalizations that later layers build on + +1. **FTRL on normed spaces, with losses in the dual.** + - State online linear optimization on a set `K` of a real normed space `E`, with losses + `y t : E →L[ℝ] ℝ`. The loss of a play `a t` is `y t (a t)`, and the regret against `u ∈ K` + after `n` rounds is `∑ t < n, y t (a t - u)`. + - Restate `IsFTRLPlay K F η y t x` with the objective `b ↦ η ∑ s < t, y s b + F b`, and the + FTRL lemma `sum_inner_sub_le_ftrl` with the objectives `b ↦ ∑ s < t, y s b + ψ t b`. + - Restate [LS, Theorem 28.5] (`ftrl_regret_le`) with the same hypotheses: `K` convex, `η > 0`, + plays in `K`, and a derivative `F' t` of the potential within `K` at each play, with LML's + `bregDiv`. + - Restate the strongly convex case for Mathlib's `StrongConvexOn K m F`, which refers to the + norm of `E`: the regret is at most + $\frac{F(u)-F(a_0)}{\eta}+\frac{\eta}{2m}\sum_{t