From 64d61944d89e3b1ba307693929f94982c6b49409 Mon Sep 17 00:00:00 2001 From: Remy Degenne Date: Sun, 15 Nov 2026 21:30:16 +0100 Subject: [PATCH 1/5] =?UTF-8?q?feat:=20Exp3=20for=20adversarial=20linear?= =?UTF-8?q?=20bandits=20with=20the=20Kiefer=E2=80=93Wolfowitz=20exploratio?= =?UTF-8?q?n,=20for=20features=20that=20need=20not=20span=20the=20space?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Code --- .../LinearBandit/Exp3/KieferWolfowitz.lean | 330 ++++++++++++++++++ 1 file changed, 330 insertions(+) create mode 100644 LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean diff --git a/LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean b/LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean new file mode 100644 index 00000000..2fb5e094 --- /dev/null +++ b/LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean @@ -0,0 +1,330 @@ +/- +Copyright (c) 2026 LMLExplore contributors. All rights reserved. +Released under Apache 2.0 license as described in the file LICENSE. +Authors: LMLExplore contributors +-/ +module + +public import LMLExplore.Analysis.InnerProductSpace.OptimalDesign +public import LMLExplore.Bandits.LinearBandit.Exp3.Algorithm + +/-! +# Exp3 for adversarial linear bandits with the Kiefer–Wolfowitz exploration + +The regret bound `2 √(3 d n log k)` of Exp3 for linear bandits [LS, Theorem 27.1] needs an +exploration distribution `π` whose design operator `V(π) = ∑ b, π b • φ b ⊗ φ b` is invertible +and whose G-value `max_b ⟪φ b, V(π)⁻¹ φ b⟫` is at most the dimension. The Kiefer–Wolfowitz +theorem provides one when the features `φ` span the space. Both hypotheses are discharged here. + +The features need not span `E`. Exp3 for linear bandits is run on the span `W` of the features, +with the features `spanFeatures φ` seen as elements of `W` and the Kiefer–Wolfowitz exploration +`kieferWolfowitzExploration φ` of `W`. The learner observes only the losses `⟪φ c, y⟫` of the arms, +which are the losses `⟪φ c, P y⟫` of the orthogonal projection `P y` of the loss vector onto `W`. +So the regret against an adaptive adversary in `E` is the regret against the projected adversary in +`W`. The bound then holds with the dimension `m` of the span of the features in place of `d`, and +`m ≤ d`, with no assumption on the features. This is [LS, Theorem 27.1] together with the +Kiefer–Wolfowitz theorem for sets that need not span the space [LS, Theorem 21.1, Exercise 21.6]. + +## Main definitions + +* `LMLExplore.Bandits.spanFeatures φ`: the features `φ a`, as elements of the span of the features. +* `LMLExplore.Bandits.kieferWolfowitzExploration φ`: a Kiefer–Wolfowitz design on the features in + their span. Its design operator is invertible on the span and its G-value is at most the + dimension of the span. +* `LMLExplore.Bandits.linearExp3KieferWolfowitz φ η γ hγ`: Exp3 for linear bandits on the span of + the features, with the Kiefer–Wolfowitz exploration. + +## Main statements + +* `inner_ringInverse_sum_smul_rankOne_spanFeatures`: the loss estimates of Exp3 for linear bandits + on the span of the features are those of [LS, Algorithm 15] in `E` with the Moore–Penrose inverse + `V⁺` in place of `V⁻¹`, and `linearExp3KieferWolfowitz_eq_linearExp3`: when the features span + `E`, the algorithm is Exp3 for linear bandits on `E` with the Kiefer–Wolfowitz exploration. +* `LinearExp3.integral_regret_le_kieferWolfowitz`: against an adaptive adversary, with the tuned + rates, the expected regret of `linearExp3KieferWolfowitz` against every arm is at most + `2 √(3 m n log k)`, where `m` is the dimension of the span of the features. +* `LinearExp3.integral_regret_le_sqrt_finrank_kieferWolfowitz`: the same expected regret is at most + `2 √(3 d n log k)`, where `d` is the dimension of `E`. +* `LinearExp3.integral_regret_detFeedback_le_kieferWolfowitz`: the first bound, for every run + against the environment of an adaptive adversary. + +## References + +* [LS] T. Lattimore and Cs. Szepesvári, *Bandit Algorithms*, Cambridge University Press, 2020, + Chapters 21 and 27. +-/ + +public section + +open MeasureTheory ProbabilityTheory Finset Learning ContinuousLinearMap InnerProductSpace Module + LMLExplore.OnlineLearning +open scoped InnerProductSpace + +namespace LMLExplore.Bandits + +variable {α 𝓞 E : Type*} [NormedAddCommGroup E] [InnerProductSpace ℝ E] + +section Exploration + +variable (φ : α → E) + +/-- The features `φ a`, as elements of the span `Submodule.span ℝ (Set.range φ)` of the features. +They span it (`span_range_spanFeatures`). -/ +def spanFeatures (a : α) : Submodule.span ℝ (Set.range φ) := + ⟨φ a, Submodule.subset_span (Set.mem_range_self a)⟩ + +@[simp] +lemma coe_spanFeatures (a : α) : (spanFeatures φ a : E) = φ a := (rfl) + +/-- The features span their span. -/ +lemma span_range_spanFeatures : Submodule.span ℝ (Set.range (spanFeatures φ)) = ⊤ := by + have : Set.range (spanFeatures φ) = (↑) ⁻¹' Set.range φ := by + ext x + simp [Subtype.ext_iff] + rw [this, Submodule.span_span_coe_preimage] + +/-- The loss of an arm against the orthogonal projection of a loss vector onto the span of the +features is its loss against the loss vector. -/ +lemma inner_spanFeatures_orthogonalProjectionOnto + [(Submodule.span ℝ (Set.range φ)).HasOrthogonalProjection] (a : α) (v : E) : + ⟪spanFeatures φ a, (Submodule.span ℝ (Set.range φ)).orthogonalProjectionOnto v⟫_ℝ = + ⟪φ a, v⟫_ℝ := by + rw [Submodule.inner_orthogonalProjectionOnto_eq_of_mem_left, coe_spanFeatures] + +variable [Fintype α] [FiniteDimensional ℝ E] [Nonempty α] + +/-- The **Kiefer–Wolfowitz exploration** of the features `φ`: weights `π` in the simplex whose +design operator `V(π) = ∑ b, π b • φ b ⊗ φ b` on the span of the features is invertible, with +`⟪φ b, V(π)⁻¹ φ b⟫ ≤ m` for every arm `b`, where `m` is the dimension of the span. They are chosen +by the Kiefer–Wolfowitz theorem `exists_mem_simplex_inner_ringInverse_le_finrank` on the span. -/ +noncomputable def kieferWolfowitzExploration : α → ℝ := + (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose + +/-- The Kiefer–Wolfowitz exploration is a probability vector. -/ +lemma kieferWolfowitzExploration_mem_simplex : kieferWolfowitzExploration φ ∈ simplex α := + (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose_spec.1 + +/-- The design operator of the Kiefer–Wolfowitz exploration is invertible on the span of the +features. -/ +lemma isUnit_kieferWolfowitzExploration : + IsUnit (∑ b, kieferWolfowitzExploration φ b • + rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) := + (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose_spec.2.1 + +/-- The G-value of the Kiefer–Wolfowitz exploration is at most the dimension of the span of the +features. -/ +lemma inner_ringInverse_kieferWolfowitzExploration_le_finrank (b : α) : + ⟪spanFeatures φ b, Ring.inverse (∑ b, kieferWolfowitzExploration φ b • + rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) (spanFeatures φ b)⟫_ℝ ≤ + finrank ℝ (Submodule.span ℝ (Set.range φ)) := + (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose_spec.2.2 b + +end Exploration + +section Estimate + +variable [Fintype α] (φ : α → E) {p : α → ℝ} + +/-- On the span of the features, the operator `V(p) = ∑ b, p b • φ b ⊗ φ b` on `E` acts as the +operator `V_W(p) = ∑ b, p b • φ b ⊗ φ b` on the span `W` of the features. -/ +lemma coe_sum_smul_rankOne_spanFeatures_apply (p : α → ℝ) + (v : Submodule.span ℝ (Set.range φ)) : + ((∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) v : E) = + (∑ b, p b • rankOne ℝ (φ b) (φ b)) v := by + simp [sum_apply, Submodule.coe_inner] + +/-- If the operator `V_W(p) = ∑ b, p b • φ b ⊗ φ b` on the span `W` of the features is +invertible, then `W` lies in the range of the operator `V(p) = ∑ b, p b • φ b ⊗ φ b` on `E`. -/ +lemma span_le_range_sum_smul_rankOne + (hu : IsUnit (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b))) : + Submodule.span ℝ (Set.range φ) ≤ + LinearMap.range ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E) := + fun v hv ↦ ⟨Ring.inverse (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) ⟨v, hv⟩, by + rw [coe_coe, ← coe_sum_smul_rankOne_spanFeatures_apply, apply_ringInverse_apply hu, + Submodule.coe_mk]⟩ + +variable [FiniteDimensional ℝ E] + +/-- **The estimates on the span are those of the Moore–Penrose inverse.** If the operator +`V_W(p) = ∑ b, p b • φ b ⊗ φ b` on the span `W` of the features is invertible, then the weighted +inner products of the features for its inverse are those for the Moore–Penrose inverse `V(p)⁺` of +the operator `V(p) = ∑ b, p b • φ b ⊗ φ b` on `E`: `⟪φ a, V_W(p)⁻¹ φ c⟫ = ⟪φ a, V(p)⁺ φ c⟫`. -/ +theorem inner_ringInverse_sum_smul_rankOne_spanFeatures + (hu : IsUnit (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b))) (a c : α) : + ⟪spanFeatures φ a, Ring.inverse (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) + (spanFeatures φ c)⟫_ℝ = + ⟪φ a, ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E).moorePenroseInverse + (φ c)⟫_ℝ := by + have hsym : ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E).IsSymmetric := + fun x y ↦ by + simp only [coe_coe, sum_apply, smul_apply, rankOne_apply, sum_inner, inner_sum, + real_inner_smul_left, real_inner_smul_right] + exact Finset.sum_congr rfl fun b _ ↦ by rw [real_inner_comm x (φ b)]; ring + -- `φ c` is the image under `V(p)` of `V_W(p)⁻¹ φ c`, which lies in the range of `V(p)` + have hc : φ c = ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E) + ↑(Ring.inverse (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) + (spanFeatures φ c)) := by + rw [coe_coe, ← coe_sum_smul_rankOne_spanFeatures_apply, apply_ringInverse_apply hu, + coe_spanFeatures] + rw [hc, hsym.moorePenroseInverse_apply_apply_of_mem_range + (span_le_range_sum_smul_rankOne φ hu (Submodule.coe_mem _)), Submodule.coe_inner, + coe_spanFeatures] + +/-- If the features span `E` and the operator `V_W(p) = ∑ b, p b • φ b ⊗ φ b` on their span is +invertible, then the loss estimates of Exp3 for linear bandits with the features in their span are +those with the features in `E`. -/ +theorem linearExp3Estimate_spanFeatures (hspan : Submodule.span ℝ (Set.range φ) = ⊤) + (hu : IsUnit (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b))) : + linearExp3Estimate (spanFeatures φ) p = linearExp3Estimate φ p := by + -- `V(p)` is onto, hence invertible, and its Moore–Penrose inverse is its inverse + have hVu : IsUnit (∑ b, p b • rankOne ℝ (φ b) (φ b)) := by + rw [isUnit_iff_isUnit_toLinearMap, LinearMap.isUnit_iff_range_eq_top, eq_top_iff, ← hspan] + exact span_le_range_sum_smul_rankOne φ hu + funext c ℓ a + rw [linearExp3Estimate_apply, linearExp3Estimate_apply, + inner_ringInverse_sum_smul_rankOne_spanFeatures φ hu, moorePenroseInverse_eq_ringInverse hVu, + coe_coe] + +end Estimate + +section Algorithm + +variable [Fintype α] [FiniteDimensional ℝ E] [Nonempty α] [MeasurableSpace α] + [MeasurableSingletonClass α] [MeasurableSpace 𝓞] (φ : α → E) (η γ : ℝ) + +/-- **Exp3 for adversarial linear bandits with the Kiefer–Wolfowitz exploration**: Exp3 for linear +bandits [LS, Algorithm 15] with learning rate `η` and exploration rate `γ ∈ [0, 1]`, run on the +span of the features `φ`, with the features `spanFeatures φ` and the exploration +`kieferWolfowitzExploration φ`. The features need not span `E`. -/ +noncomputable def linearExp3KieferWolfowitz (hγ : γ ∈ Set.Icc 0 1) : Algorithm 𝓞 α ℝ := + linearExp3 (spanFeatures φ) (kieferWolfowitzExploration φ) η γ + (kieferWolfowitzExploration_mem_simplex φ) hγ + +lemma linearExp3KieferWolfowitz_def (hγ : γ ∈ Set.Icc 0 1) : + linearExp3KieferWolfowitz (𝓞 := 𝓞) φ η γ hγ = + linearExp3 (spanFeatures φ) (kieferWolfowitzExploration φ) η γ + (kieferWolfowitzExploration_mem_simplex φ) hγ := (rfl) + +/-- When the features span `E` and the exploration rate is positive, Exp3 for linear bandits with +the Kiefer–Wolfowitz exploration is Exp3 for linear bandits [LS, Algorithm 15] with the features +`φ` in `E` and the exploration `kieferWolfowitzExploration φ`. -/ +theorem linearExp3KieferWolfowitz_eq_linearExp3 (hspan : Submodule.span ℝ (Set.range φ) = ⊤) + (hγ : γ ∈ Set.Icc 0 1) (hγ0 : 0 < γ) : + linearExp3KieferWolfowitz (𝓞 := 𝓞) φ η γ hγ = + linearExp3 φ (kieferWolfowitzExploration φ) η γ + (kieferWolfowitzExploration_mem_simplex φ) hγ := by + have hsum (n : ℕ) (h : Hist 𝓞 α ℝ n) : + linearExp3HistSum (spanFeatures φ) (kieferWolfowitzExploration φ) η γ n h = + linearExp3HistSum φ (kieferWolfowitzExploration φ) η γ n h := by + induction n with + | zero => simp + | succ n ih => + rw [linearExp3HistSum_succ, linearExp3HistSum_succ, ih, + linearExp3Estimate_spanFeatures φ hspan (isUnit_sum_smul_rankOne_linearExp3Weights + (mem_simplex_iff.1 (kieferWolfowitzExploration_mem_simplex φ)).1 + (isUnit_kieferWolfowitzExploration φ) hγ0 hγ.2 _)] + rw [linearExp3KieferWolfowitz_def, linearExp3_def, linearExp3_def] + congr 1 + funext n x + rw [hsum] + +end Algorithm + +namespace LinearExp3 + +variable [Fintype α] [FiniteDimensional ℝ E] [MeasurableSpace E] [BorelSpace E] [Nonempty α] + [MeasurableSpace α] [MeasurableSingletonClass α] [MeasurableSpace 𝓞] {φ : α → E} {η γ : ℝ} + {hγ : γ ∈ Set.Icc 0 1} + {Ω : Type*} {mΩ : MeasurableSpace Ω} {P : Measure Ω} {O : ℕ → Ω → 𝓞} {A : ℕ → Ω → α} + {Y : ℕ → Ω → ℝ} {env : Environment 𝓞 α ℝ} {y : (n : ℕ) → Hist 𝓞 α ℝ n × 𝓞 → E} {n : ℕ} + +/-- **Regret of Exp3 for adversarial linear bandits with the Kiefer–Wolfowitz exploration against +an adaptive adversary** [LS, Theorem 27.1, Exercise 21.6]. Let `m` be the dimension of the span of +the features `φ`, which need not span `E`, and `k ≥ 2` the number of arms. Let an adaptive +adversary choose at round `t` the loss vector `y t (h, o) ∈ E`, a measurable function of the +history `h` and the observation `o` of round `t` with `|⟪φ c, y t (h, o)⟫| ≤ 1` for every arm `c`, +and let the feedback `Y t` of a run of `linearExp3KieferWolfowitz φ η γ` be the loss +`⟪φ (A t), y t (h, o)⟫` of the arm played. With the learning rate `η = √(log k / (3 m n))` and the +exploration rate `γ = η m`, the expected regret after `n ≥ 1` rounds against every arm is at most +`2 √(3 m n log k)`. -/ +theorem integral_regret_le_kieferWolfowitz [IsProbabilityMeasure P] + (h : IsAlgEnvSeq O A Y (linearExp3KieferWolfowitz φ η γ hγ) env P) + (hk : 1 < Fintype.card α) (hn : 0 < n) + (hη : η = √(Real.log (Fintype.card α) / + (3 * finrank ℝ (Submodule.span ℝ (Set.range φ)) * n))) + (hγm : γ = η * finrank ℝ (Submodule.span ℝ (Set.range φ))) (hy : ∀ t < n, Measurable (y t)) + (hy1 : ∀ t < n, ∀ x c, |⟪φ c, y t x⟫_ℝ| ≤ 1) + (hY : ∀ t < n, Y t =ᵐ[P] fun ω ↦ ⟪φ (A t ω), y t (history O A Y t ω, O t ω)⟫_ℝ) (b : α) : + ∫ ω, ∑ t ∈ range n, (⟪φ (A t ω), y t (history O A Y t ω, O t ω)⟫_ℝ - + ⟪φ b, y t (history O A Y t ω, O t ω)⟫_ℝ) ∂P ≤ + 2 * √(3 * finrank ℝ (Submodule.span ℝ (Set.range φ)) * n * + Real.log (Fintype.card α)) := by + rcases subsingleton_or_nontrivial (Submodule.span ℝ (Set.range φ)) with hW | hW + · -- all the features vanish, so does the regret + have hφ (a : α) : φ a = 0 := by + simpa using congrArg Subtype.val (Subsingleton.elim (spanFeatures φ a) 0) + simp only [hφ, inner_zero_left, sub_self, sum_const_zero, integral_zero] + positivity + -- the run is a run of Exp3 for linear bandits on `W` against the projected adversary + rw [linearExp3KieferWolfowitz_def] at h + have key := integral_regret_le_two_mul_sqrt_three + (y := fun t x ↦ (Submodule.span ℝ (Set.range φ)).orthogonalProjectionOnto (y t x)) h + (isUnit_kieferWolfowitzExploration φ) + (inner_ringInverse_kieferWolfowitzExploration_le_finrank φ) hk hn hη hγm + (fun t ht ↦ (Submodule.span ℝ _).orthogonalProjectionOnto.continuous.measurable.comp + (hy t ht)) + (fun t ht x c ↦ by rw [inner_spanFeatures_orthogonalProjectionOnto]; exact hy1 t ht x c) + (fun t ht ↦ by simp only [inner_spanFeatures_orthogonalProjectionOnto]; exact hY t ht) b + simp only [inner_spanFeatures_orthogonalProjectionOnto] at key + exact key + +/-- **Regret of Exp3 for adversarial linear bandits with the Kiefer–Wolfowitz exploration against +an adaptive adversary, in terms of the dimension of the space** [LS, Theorem 27.1]. Under the +hypotheses of `LinearExp3.integral_regret_le_kieferWolfowitz`, on a space of dimension `d`, the +expected regret against every arm is at most `2 √(3 d n log k)`. -/ +theorem integral_regret_le_sqrt_finrank_kieferWolfowitz [IsProbabilityMeasure P] + (h : IsAlgEnvSeq O A Y (linearExp3KieferWolfowitz φ η γ hγ) env P) + (hk : 1 < Fintype.card α) (hn : 0 < n) + (hη : η = √(Real.log (Fintype.card α) / + (3 * finrank ℝ (Submodule.span ℝ (Set.range φ)) * n))) + (hγm : γ = η * finrank ℝ (Submodule.span ℝ (Set.range φ))) (hy : ∀ t < n, Measurable (y t)) + (hy1 : ∀ t < n, ∀ x c, |⟪φ c, y t x⟫_ℝ| ≤ 1) + (hY : ∀ t < n, Y t =ᵐ[P] fun ω ↦ ⟪φ (A t ω), y t (history O A Y t ω, O t ω)⟫_ℝ) (b : α) : + ∫ ω, ∑ t ∈ range n, (⟪φ (A t ω), y t (history O A Y t ω, O t ω)⟫_ℝ - + ⟪φ b, y t (history O A Y t ω, O t ω)⟫_ℝ) ∂P ≤ + 2 * √(3 * finrank ℝ E * n * Real.log (Fintype.card α)) := by + refine (integral_regret_le_kieferWolfowitz h hk hn hη hγm hy hy1 hY b).trans ?_ + have hm : (finrank ℝ (Submodule.span ℝ (Set.range φ)) : ℝ) ≤ finrank ℝ E := + Nat.cast_le.2 (Submodule.finrank_le _) + have hlog : 0 ≤ Real.log (Fintype.card α) := Real.log_nonneg (by exact_mod_cast hk.le) + gcongr + +/-- **Regret of Exp3 for adversarial linear bandits with the Kiefer–Wolfowitz exploration, for +every run against an adaptive adversary** [LS, Theorem 27.1, Exercise 21.6]. Let an adaptive +adversary choose at round `t` the loss vector `y t (h, o) ∈ E`, a measurable function of the +history `h` and of the observation `o` of round `t`, drawn from arbitrary observation kernels `obs`, +with `|⟪φ c, y t (h, o)⟫| ≤ 1` for every arm `c`, and let its environment +`Environment.detFeedback obs (fun t x ↦ ⟪φ x.2, y t x.1⟫) _` return the loss of the arm played. +With the rates of `LinearExp3.integral_regret_le_kieferWolfowitz`, every run of +`linearExp3KieferWolfowitz φ η γ` against it has expected regret at most `2 √(3 m n log k)` against +every arm, where `m` is the dimension of the span of the features. -/ +theorem integral_regret_detFeedback_le_kieferWolfowitz [IsProbabilityMeasure P] + {obs : (n : ℕ) → Kernel (Hist 𝓞 α ℝ n) 𝓞} [∀ n, IsMarkovKernel (obs n)] + {hf : ∀ n, Measurable fun x : (Hist 𝓞 α ℝ n × 𝓞) × α ↦ ⟪φ x.2, y n x.1⟫_ℝ} + (h : IsAlgEnvSeq O A Y (linearExp3KieferWolfowitz φ η γ hγ) + (Environment.detFeedback obs (fun n x ↦ ⟪φ x.2, y n x.1⟫_ℝ) hf) P) + (hk : 1 < Fintype.card α) (hn : 0 < n) + (hη : η = √(Real.log (Fintype.card α) / + (3 * finrank ℝ (Submodule.span ℝ (Set.range φ)) * n))) + (hγm : γ = η * finrank ℝ (Submodule.span ℝ (Set.range φ))) (hy : ∀ t < n, Measurable (y t)) + (hy1 : ∀ t < n, ∀ x c, |⟪φ c, y t x⟫_ℝ| ≤ 1) (b : α) : + ∫ ω, ∑ t ∈ range n, (⟪φ (A t ω), y t (history O A Y t ω, O t ω)⟫_ℝ - + ⟪φ b, y t (history O A Y t ω, O t ω)⟫_ℝ) ∂P ≤ + 2 * √(3 * finrank ℝ (Submodule.span ℝ (Set.range φ)) * n * + Real.log (Fintype.card α)) := + integral_regret_le_kieferWolfowitz h hk hn hη hγm hy hy1 + (fun t _ ↦ h.feedback_detFeedback_ae_eq t) b + +end LinearExp3 + +end LMLExplore.Bandits From 01c61dcef356b903a64f4cfabe2a27e1802cbf1c Mon Sep 17 00:00:00 2001 From: Remy Degenne Date: Sun, 15 Nov 2026 22:05:06 +0100 Subject: [PATCH 2/5] =?UTF-8?q?refactor:=20move=20the=20span-restricted=20?= =?UTF-8?q?Kiefer=E2=80=93Wolfowitz=20design=20to=20OptimalDesign?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Move `spanFeatures` and its lemmas, and the Kiefer–Wolfowitz design on the span of the features (now `kieferWolfowitzDesign`, with the existential `exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank`), to `Analysis/InnerProductSpace/OptimalDesign.lean`. Factor the Moore–Penrose restriction argument into the operator-level `inner_moorePenroseInverse_of_apply_coe_eq` (with `apply_coe_ringInverse_of_apply_coe_eq` and `le_range_of_apply_coe_eq`), derive `inner_moorePenroseInverse_designOperator_map_subtype` and `inner_ringInverse_sum_smul_rankOne_spanFeatures` from it, and add `isSymmetric_sum_smul_rankOne`. Rename the regret theorems after their bound. Co-Authored-By: Claude Code --- .../InnerProductSpace/OptimalDesign.lean | 160 ++++++++++++-- .../LinearBandit/Exp3/KieferWolfowitz.lean | 204 +++++------------- 2 files changed, 198 insertions(+), 166 deletions(-) diff --git a/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean b/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean index 9f761eef..9b9c26d5 100644 --- a/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean +++ b/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean @@ -67,6 +67,10 @@ point `a` for each of them gives a design with the same design operator, hence t * `LMLExplore.designOperator w`: the design operator `V(w) = ∑_a w a • a ⊗ a` of `w`. * `LMLExplore.gValue 𝒜 w`: the G-value `g(w) = sup_{a ∈ 𝒜} ⟪a, V(w)⁺ a⟫` in `ℝ≥0∞`, where a point of `𝒜` outside the range of `V(w)` contributes `∞`. +* `LMLExplore.spanFeatures φ`: the features `φ i`, as elements of the span of the features. +* `LMLExplore.kieferWolfowitzDesign φ`: a Kiefer–Wolfowitz design on finitely many arms with + features `φ`, which need not span `E`: weights in the simplex whose design operator on the span + of the features is invertible, with G-value at most the dimension of the span. ## Main statements @@ -75,7 +79,8 @@ point `a` for each of them gives a design with the same design operator, hence t * `LMLExplore.isPositive_sum_smul_rankOne`, `LMLExplore.smul_sum_smul_rankOne_le`, `LMLExplore.isUnit_sum_smul_rankOne_of_mul_le`: for a feature map `φ`, the weighted sum `∑ i ∈ s, w i • φ i ⊗ φ i` is positive for nonnegative weights (on a finite type, monotone in - the weights in the Loewner order, and invertible above an invertible one). + the weights in the Loewner order, and invertible above an invertible one), and + `LMLExplore.isSymmetric_sum_smul_rankOne`: it is symmetric for arbitrary real weights. * `LMLExplore.inner_sum_smul_rankOne_apply_self`: its quadratic form is `⟪v, (∑ i ∈ s, w i • φ i ⊗ φ i) v⟫ = ∑ i ∈ s, w i ⟪φ i, v⟫²`, and `LMLExplore.sum_smul_rankOne_apply_of_orthonormal`: for an orthonormal family `φ`, it maps `φ i` @@ -90,6 +95,8 @@ point `a` for each of them gives a design with the same design operator, hence t support of the design. * `LMLExplore.designOperator_map_subtype_apply`: on a subspace `K`, the design operator on `E` of a design on `K` restricts to its design operator on `K`. +* `LMLExplore.inner_moorePenroseInverse_of_apply_coe_eq`: if a symmetric operator `T` on `E` acts + on a subspace `K` as an invertible operator `T_K` on `K`, then `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` on `K`. * `LMLExplore.designOperator_convexCombPair`: `V(w)` is affine in `w`. * `LMLExplore.mem_convexHull_image_rankOne_iff`: the convex hull of `{a ⊗ a | a ∈ 𝒜}` is the set of design operators of the designs on `𝒜`. @@ -114,8 +121,14 @@ point `a` for each of them gives a design with the same design operator, hence t * `LMLExplore.exists_mem_simplex_inner_ringInverse_le_finrank`: **Kiefer–Wolfowitz for finitely many arms**: if the features `φ i` of finitely many arms span `E`, some weights `w` in the simplex have `⟪φ i, V⁻¹ φ i⟫ ≤ d` for every arm, with `V = ∑ i, w i • φ i ⊗ φ i` invertible. +* `LMLExplore.exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank`: **Kiefer–Wolfowitz + for finitely many arms on the span of the features** [LS, Exercise 21.6]: the same on the span `W` + of features that need not span `E`, with the dimension `m` of `W` in place of `d`. * `LMLExplore.inner_moorePenroseInverse_designOperator_map_subtype`: for a design on a subspace - `K` with invertible design operator `V_K(w)` on `K`, `⟪a, V(w)⁺ a⟫ = ⟪a, V_K(w)⁻¹ a⟫` on `K`. + `K` with invertible design operator `V_K(w)` on `K`, `⟪a, V(w)⁺ a⟫ = ⟪a, V_K(w)⁻¹ a⟫` on `K`; + `LMLExplore.inner_ringInverse_sum_smul_rankOne_spanFeatures`: the same for weights `p` on arms + with features `φ`, whose operator `V_W(p) = ∑ i, p i • φ i ⊗ φ i` on the span `W` of the features + is invertible: `⟪φ a, V_W(p)⁻¹ φ c⟫ = ⟪φ a, V(p)⁺ φ c⟫`. * `LMLExplore.exists_designOperator_eq_card_support_le`: a design with invertible design operator and `⟪a, V(w)⁻¹ a⟫ ≤ d` on its support has the design operator of a design supported in its support with at most `d (d + 1) / 2` points [LS, Theorem 21.1]. @@ -246,6 +259,14 @@ theorem isPositive_sum_smul_rankOne {ι : Type*} (s : Finset ι) (φ : ι → E) (hw : ∀ i ∈ s, 0 ≤ w i) : (∑ i ∈ s, w i • rankOne ℝ (φ i) (φ i)).IsPositive := isPositive_sum _ fun i hi ↦ (isPositive_rankOne_self (φ i)).smul_of_nonneg (hw i hi) +/-- For arbitrary real weights `w`, `∑ i ∈ s, w i • φ i ⊗ φ i` is symmetric. -/ +theorem isSymmetric_sum_smul_rankOne {ι : Type*} (s : Finset ι) (φ : ι → E) (w : ι → ℝ) : + ((∑ i ∈ s, w i • rankOne ℝ (φ i) (φ i) : E →L[ℝ] E) : E →ₗ[ℝ] E).IsSymmetric := by + rw [toLinearMap_sum] + exact LinearMap.isSymmetric_sum _ fun i _ ↦ by + rw [toLinearMap_smul] + exact (isSymmetric_rankOne_self (φ i)).smul (by simp) + /-- The quadratic form of `∑ i ∈ s, w i • φ i ⊗ φ i`: `⟪v, (∑ i ∈ s, w i • φ i ⊗ φ i) v⟫ = ∑ i ∈ s, w i * ⟪φ i, v⟫²`. -/ theorem inner_sum_smul_rankOne_apply_self {ι : Type*} (s : Finset ι) (φ : ι → E) (w : ι → ℝ) @@ -320,6 +341,67 @@ theorem isUnit_sum_smul_rankOne_of_mul_le [FiniteDimensional ℝ E] {ι : Type*} have hvp := isPositive_sum_smul_rankOne Finset.univ φ fun i _ ↦ hv i exact (hvp.smul_of_nonneg hc.le).isUnit_of_le hcu (smul_sum_smul_rankOne_le h) +section Restrict + +variable {K : Submodule ℝ E} {T : E →L[ℝ] E} {T_K : K →L[ℝ] K} + +/-- If an operator `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then +`T` maps `T_K⁻¹ v` to `v` for `v ∈ K`. -/ +theorem apply_coe_ringInverse_of_apply_coe_eq (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) + (v : K) : T ↑(Ring.inverse T_K v) = v := by + rw [hT, apply_ringInverse_apply hu] + +/-- If an operator `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then +`K` lies in the range of `T`. -/ +theorem le_range_of_apply_coe_eq (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) : + K ≤ LinearMap.range (T : E →ₗ[ℝ] E) := fun v hv ↦ + ⟨_, apply_coe_ringInverse_of_apply_coe_eq hT hu ⟨v, hv⟩⟩ + +/-- If a symmetric operator `T` on a finite-dimensional space `E` acts on a subspace `K` as an +invertible operator `T_K` on `K`, then the Moore–Penrose inverse of `T` acts on `K` as the inverse +of `T_K`, in the weak sense `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`. -/ +theorem inner_moorePenroseInverse_of_apply_coe_eq [FiniteDimensional ℝ E] + (hsym : (T : E →ₗ[ℝ] E).IsSymmetric) (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) + (a c : K) : + ⟪(a : E), (T : E →ₗ[ℝ] E).moorePenroseInverse c⟫_ℝ = ⟪a, Ring.inverse T_K c⟫_ℝ := by + have h := hsym.moorePenroseInverse_apply_apply_of_mem_range + (le_range_of_apply_coe_eq hT hu (Ring.inverse T_K c).2) + rw [coe_coe, apply_coe_ringInverse_of_apply_coe_eq hT hu] at h + rw [h, Submodule.coe_inner] + +end Restrict + +/-! ### Features in their span -/ + +section SpanFeatures + +variable {ι : Type*} (φ : ι → E) + +/-- The features `φ i`, as elements of the span `Submodule.span ℝ (Set.range φ)` of the features. +They span it (`span_range_spanFeatures`). -/ +def spanFeatures (i : ι) : Submodule.span ℝ (Set.range φ) := + ⟨φ i, Submodule.subset_span (Set.mem_range_self i)⟩ + +@[simp] +theorem coe_spanFeatures (i : ι) : (spanFeatures φ i : E) = φ i := (rfl) + +/-- The features span their span. -/ +theorem span_range_spanFeatures : Submodule.span ℝ (Set.range (spanFeatures φ)) = ⊤ := by + have : Set.range (spanFeatures φ) = (↑) ⁻¹' Set.range φ := by + ext x + simp [Subtype.ext_iff] + rw [this, Submodule.span_span_coe_preimage] + +/-- The inner product of a feature with the orthogonal projection of a vector onto the span of the +features is its inner product with the vector. -/ +theorem inner_spanFeatures_orthogonalProjectionOnto + [(Submodule.span ℝ (Set.range φ)).HasOrthogonalProjection] (i : ι) (v : E) : + ⟪spanFeatures φ i, (Submodule.span ℝ (Set.range φ)).orthogonalProjectionOnto v⟫_ℝ = + ⟪φ i, v⟫_ℝ := by + rw [Submodule.inner_orthogonalProjectionOnto_eq_of_mem_left, coe_spanFeatures] + +end SpanFeatures + /-- The design operator on `E` of a design on a subspace `K` restricts on `K` to the design operator on `K`: `V(w) v = V_K(w) v` for `v ∈ K`. -/ theorem designOperator_map_subtype_apply {K : Submodule ℝ E} (w : StdSimplex ℝ K) (v : K) : @@ -605,25 +687,73 @@ theorem exists_mem_simplex_inner_ringInverse_le_finrank {ι : Type*} [Fintype ι rw [designOperator_map] at hu hle exact ⟨v.weights, v.weights_mem_simplex, hu, fun i ↦ hle _ ⟨i, rfl⟩⟩ +/-- **The Kiefer–Wolfowitz theorem for finitely many arms on the span of the features** +[LS, Theorem 21.1, Exercise 21.6]. For features `φ i` of the elements of a nonempty finite type, +which need not span `E`, some weights `w` in the simplex have an invertible design operator +`V_W = ∑ i, w i • φ i ⊗ φ i` on the span `W` of the features, with `⟪φ i, V_W⁻¹ φ i⟫ ≤ m` for every +`i`, where `m` is the dimension of `W`. -/ +theorem exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank {ι : Type*} [Fintype ι] + [Nonempty ι] (φ : ι → E) : + ∃ w ∈ LMLExplore.simplex ι, + IsUnit (∑ i, w i • rankOne ℝ (spanFeatures φ i) (spanFeatures φ i)) ∧ + ∀ i, ⟪spanFeatures φ i, Ring.inverse (∑ i, w i • rankOne ℝ (spanFeatures φ i) + (spanFeatures φ i)) (spanFeatures φ i)⟫_ℝ ≤ finrank ℝ (Submodule.span ℝ (Set.range φ)) := + exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ) + +section KieferWolfowitzDesign + +variable {ι : Type*} [Fintype ι] [Nonempty ι] (φ : ι → E) + +/-- The **Kiefer–Wolfowitz design** of the features `φ` on finitely many arms: weights `w` in the +simplex whose design operator `V_W(w) = ∑ i, w i • φ i ⊗ φ i` on the span `W` of the features is +invertible, with `⟪φ i, V_W(w)⁻¹ φ i⟫ ≤ m` for every arm `i`, where `m` is the dimension of `W`. +They are chosen by `exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank`. -/ +noncomputable def kieferWolfowitzDesign : ι → ℝ := + (exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank φ).choose + +/-- The Kiefer–Wolfowitz design is a probability vector. -/ +theorem kieferWolfowitzDesign_mem_simplex : kieferWolfowitzDesign φ ∈ LMLExplore.simplex ι := + (exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank φ).choose_spec.1 + +/-- The design operator of the Kiefer–Wolfowitz design is invertible on the span of the +features. -/ +theorem isUnit_sum_smul_rankOne_kieferWolfowitzDesign : + IsUnit (∑ i, kieferWolfowitzDesign φ i • rankOne ℝ (spanFeatures φ i) (spanFeatures φ i)) := + (exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank φ).choose_spec.2.1 + +/-- The G-value of the Kiefer–Wolfowitz design is at most the dimension of the span of the +features. -/ +theorem inner_ringInverse_sum_smul_rankOne_kieferWolfowitzDesign_le_finrank (i : ι) : + ⟪spanFeatures φ i, Ring.inverse (∑ i, kieferWolfowitzDesign φ i • + rankOne ℝ (spanFeatures φ i) (spanFeatures φ i)) (spanFeatures φ i)⟫_ℝ ≤ + finrank ℝ (Submodule.span ℝ (Set.range φ)) := + (exists_mem_simplex_inner_ringInverse_spanFeatures_le_finrank φ).choose_spec.2.2 i + +end KieferWolfowitzDesign + /-- Let `w` be a design on a subspace `K` whose design operator `V_K(w)` on `K` is invertible. For `a ∈ K`, the weighted norm of `a` for the Moore–Penrose inverse of the design operator `V(w)` on `E` is its weighted norm for the inverse of `V_K(w)`: `⟪a, V(w)⁺ a⟫ = ⟪a, V_K(w)⁻¹ a⟫`. -/ theorem inner_moorePenroseInverse_designOperator_map_subtype {K : Submodule ℝ E} {w : StdSimplex ℝ K} (hu : IsUnit (designOperator w)) (a : K) : ⟪(a : E), (designOperator (w.map ((↑) : K → E)) : E →ₗ[ℝ] E).moorePenroseInverse a⟫_ℝ = - ⟪a, Ring.inverse (designOperator w) a⟫_ℝ := by - set V := designOperator (w.map ((↑) : K → E)) - -- `V` maps `V_K(w)⁻¹ v` to `v` for `v ∈ K`, so `K` lies in the range of `V` - have hV : ∀ v : K, V ↑(Ring.inverse (designOperator w) v) = v := fun v ↦ by - rw [designOperator_map_subtype_apply, apply_ringInverse_apply hu] - have hmem : ∀ v : K, (v : E) ∈ LinearMap.range (V : E →ₗ[ℝ] E) := fun v ↦ - ⟨_, by rw [coe_coe, hV]⟩ - calc ⟪(a : E), (V : E →ₗ[ℝ] E).moorePenroseInverse a⟫_ℝ - = ⟪(a : E), (V : E →ₗ[ℝ] E).moorePenroseInverse - ((V : E →ₗ[ℝ] E) ↑(Ring.inverse (designOperator w) a))⟫_ℝ := by rw [coe_coe, hV] - _ = ⟪a, Ring.inverse (designOperator w) a⟫_ℝ := by - rw [(isPositive_designOperator _).toLinearMap.isSymmetric - |>.moorePenroseInverse_apply_apply_of_mem_range (hmem _), Submodule.coe_inner] + ⟪a, Ring.inverse (designOperator w) a⟫_ℝ := + inner_moorePenroseInverse_of_apply_coe_eq (isPositive_designOperator _).toLinearMap.isSymmetric + (designOperator_map_subtype_apply w) hu a a + +/-- **The inverse on the span is the Moore–Penrose inverse.** If the operator +`V_W(p) = ∑ i, p i • φ i ⊗ φ i` on the span `W` of the features is invertible, then the weighted +inner products of the features for its inverse are those for the Moore–Penrose inverse `V(p)⁺` of +the operator `V(p) = ∑ i, p i • φ i ⊗ φ i` on `E`: `⟪φ a, V_W(p)⁻¹ φ c⟫ = ⟪φ a, V(p)⁺ φ c⟫`. -/ +theorem inner_ringInverse_sum_smul_rankOne_spanFeatures {ι : Type*} [Fintype ι] {φ : ι → E} + {p : ι → ℝ} (hu : IsUnit (∑ i, p i • rankOne ℝ (spanFeatures φ i) (spanFeatures φ i))) + (a c : ι) : + ⟪spanFeatures φ a, Ring.inverse (∑ i, p i • rankOne ℝ (spanFeatures φ i) (spanFeatures φ i)) + (spanFeatures φ c)⟫_ℝ = + ⟪φ a, ((∑ i, p i • rankOne ℝ (φ i) (φ i) : E →L[ℝ] E) : E →ₗ[ℝ] E).moorePenroseInverse + (φ c)⟫_ℝ := + (inner_moorePenroseInverse_of_apply_coe_eq (isSymmetric_sum_smul_rankOne _ φ p) + (fun v ↦ by simp) hu _ _).symm /-- **Support reduction of G-optimal designs** [LS, Theorem 21.1]. Let `w` be a design with invertible design operator `V(w)` on a nontrivial real inner product space of dimension `d`, with diff --git a/LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean b/LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean index 2fb5e094..b6eaa87c 100644 --- a/LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean +++ b/LMLExplore/Bandits/LinearBandit/Exp3/KieferWolfowitz.lean @@ -17,36 +17,36 @@ and whose G-value `max_b ⟪φ b, V(π)⁻¹ φ b⟫` is at most the dimension. theorem provides one when the features `φ` span the space. Both hypotheses are discharged here. The features need not span `E`. Exp3 for linear bandits is run on the span `W` of the features, -with the features `spanFeatures φ` seen as elements of `W` and the Kiefer–Wolfowitz exploration -`kieferWolfowitzExploration φ` of `W`. The learner observes only the losses `⟪φ c, y⟫` of the arms, -which are the losses `⟪φ c, P y⟫` of the orthogonal projection `P y` of the loss vector onto `W`. -So the regret against an adaptive adversary in `E` is the regret against the projected adversary in -`W`. The bound then holds with the dimension `m` of the span of the features in place of `d`, and -`m ≤ d`, with no assumption on the features. This is [LS, Theorem 27.1] together with the -Kiefer–Wolfowitz theorem for sets that need not span the space [LS, Theorem 21.1, Exercise 21.6]. +with the features `LMLExplore.spanFeatures φ` seen as elements of `W` and the Kiefer–Wolfowitz +design `LMLExplore.kieferWolfowitzDesign φ` of `W` as exploration. The learner observes only the +losses `⟪φ c, y⟫` of the arms, which are the losses `⟪φ c, P y⟫` of the orthogonal projection `P y` +of the loss vector onto `W`. So the regret against an adaptive adversary in `E` is the regret +against the projected adversary in `W`. The bound then holds with the dimension `m` of the span of +the features in place of `d`, and `m ≤ d`, with no assumption on the features. This is +[LS, Theorem 27.1] together with the Kiefer–Wolfowitz theorem for sets that need not span the space +[LS, Theorem 21.1, Exercise 21.6]. ## Main definitions -* `LMLExplore.Bandits.spanFeatures φ`: the features `φ a`, as elements of the span of the features. -* `LMLExplore.Bandits.kieferWolfowitzExploration φ`: a Kiefer–Wolfowitz design on the features in - their span. Its design operator is invertible on the span and its G-value is at most the - dimension of the span. * `LMLExplore.Bandits.linearExp3KieferWolfowitz φ η γ hγ`: Exp3 for linear bandits on the span of the features, with the Kiefer–Wolfowitz exploration. ## Main statements -* `inner_ringInverse_sum_smul_rankOne_spanFeatures`: the loss estimates of Exp3 for linear bandits - on the span of the features are those of [LS, Algorithm 15] in `E` with the Moore–Penrose inverse - `V⁺` in place of `V⁻¹`, and `linearExp3KieferWolfowitz_eq_linearExp3`: when the features span - `E`, the algorithm is Exp3 for linear bandits on `E` with the Kiefer–Wolfowitz exploration. -* `LinearExp3.integral_regret_le_kieferWolfowitz`: against an adaptive adversary, with the tuned - rates, the expected regret of `linearExp3KieferWolfowitz` against every arm is at most - `2 √(3 m n log k)`, where `m` is the dimension of the span of the features. -* `LinearExp3.integral_regret_le_sqrt_finrank_kieferWolfowitz`: the same expected regret is at most - `2 √(3 d n log k)`, where `d` is the dimension of `E`. -* `LinearExp3.integral_regret_detFeedback_le_kieferWolfowitz`: the first bound, for every run - against the environment of an adaptive adversary. +* `linearExp3Estimate_spanFeatures`: when the features span `E`, the loss estimates of Exp3 for + linear bandits with the features in their span are those with the features in `E`, and + `linearExp3KieferWolfowitz_eq_linearExp3`: then the algorithm is Exp3 for linear bandits on `E` + with the Kiefer–Wolfowitz exploration. In general, by + `LMLExplore.inner_ringInverse_sum_smul_rankOne_spanFeatures`, the loss estimates are those of + [LS, Algorithm 15] in `E` with the Moore–Penrose inverse `V⁺` in place of `V⁻¹`. +* `LinearExp3.integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three`: against an + adaptive adversary, with the tuned rates, the expected regret of `linearExp3KieferWolfowitz` + against every arm is at most `2 √(3 m n log k)`, where `m` is the dimension of the span of the + features. +* `LinearExp3.integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three_finrank`: the same + expected regret is at most `2 √(3 d n log k)`, where `d` is the dimension of `E`. +* `LinearExp3.integral_regret_detFeedback_linearExp3KieferWolfowitz_le_two_mul_sqrt_three`: the + first bound, for every run against the environment of an adaptive adversary. ## References @@ -64,111 +64,9 @@ namespace LMLExplore.Bandits variable {α 𝓞 E : Type*} [NormedAddCommGroup E] [InnerProductSpace ℝ E] -section Exploration - -variable (φ : α → E) - -/-- The features `φ a`, as elements of the span `Submodule.span ℝ (Set.range φ)` of the features. -They span it (`span_range_spanFeatures`). -/ -def spanFeatures (a : α) : Submodule.span ℝ (Set.range φ) := - ⟨φ a, Submodule.subset_span (Set.mem_range_self a)⟩ - -@[simp] -lemma coe_spanFeatures (a : α) : (spanFeatures φ a : E) = φ a := (rfl) - -/-- The features span their span. -/ -lemma span_range_spanFeatures : Submodule.span ℝ (Set.range (spanFeatures φ)) = ⊤ := by - have : Set.range (spanFeatures φ) = (↑) ⁻¹' Set.range φ := by - ext x - simp [Subtype.ext_iff] - rw [this, Submodule.span_span_coe_preimage] - -/-- The loss of an arm against the orthogonal projection of a loss vector onto the span of the -features is its loss against the loss vector. -/ -lemma inner_spanFeatures_orthogonalProjectionOnto - [(Submodule.span ℝ (Set.range φ)).HasOrthogonalProjection] (a : α) (v : E) : - ⟪spanFeatures φ a, (Submodule.span ℝ (Set.range φ)).orthogonalProjectionOnto v⟫_ℝ = - ⟪φ a, v⟫_ℝ := by - rw [Submodule.inner_orthogonalProjectionOnto_eq_of_mem_left, coe_spanFeatures] - -variable [Fintype α] [FiniteDimensional ℝ E] [Nonempty α] - -/-- The **Kiefer–Wolfowitz exploration** of the features `φ`: weights `π` in the simplex whose -design operator `V(π) = ∑ b, π b • φ b ⊗ φ b` on the span of the features is invertible, with -`⟪φ b, V(π)⁻¹ φ b⟫ ≤ m` for every arm `b`, where `m` is the dimension of the span. They are chosen -by the Kiefer–Wolfowitz theorem `exists_mem_simplex_inner_ringInverse_le_finrank` on the span. -/ -noncomputable def kieferWolfowitzExploration : α → ℝ := - (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose - -/-- The Kiefer–Wolfowitz exploration is a probability vector. -/ -lemma kieferWolfowitzExploration_mem_simplex : kieferWolfowitzExploration φ ∈ simplex α := - (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose_spec.1 - -/-- The design operator of the Kiefer–Wolfowitz exploration is invertible on the span of the -features. -/ -lemma isUnit_kieferWolfowitzExploration : - IsUnit (∑ b, kieferWolfowitzExploration φ b • - rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) := - (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose_spec.2.1 - -/-- The G-value of the Kiefer–Wolfowitz exploration is at most the dimension of the span of the -features. -/ -lemma inner_ringInverse_kieferWolfowitzExploration_le_finrank (b : α) : - ⟪spanFeatures φ b, Ring.inverse (∑ b, kieferWolfowitzExploration φ b • - rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) (spanFeatures φ b)⟫_ℝ ≤ - finrank ℝ (Submodule.span ℝ (Set.range φ)) := - (exists_mem_simplex_inner_ringInverse_le_finrank (span_range_spanFeatures φ)).choose_spec.2.2 b - -end Exploration - section Estimate -variable [Fintype α] (φ : α → E) {p : α → ℝ} - -/-- On the span of the features, the operator `V(p) = ∑ b, p b • φ b ⊗ φ b` on `E` acts as the -operator `V_W(p) = ∑ b, p b • φ b ⊗ φ b` on the span `W` of the features. -/ -lemma coe_sum_smul_rankOne_spanFeatures_apply (p : α → ℝ) - (v : Submodule.span ℝ (Set.range φ)) : - ((∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) v : E) = - (∑ b, p b • rankOne ℝ (φ b) (φ b)) v := by - simp [sum_apply, Submodule.coe_inner] - -/-- If the operator `V_W(p) = ∑ b, p b • φ b ⊗ φ b` on the span `W` of the features is -invertible, then `W` lies in the range of the operator `V(p) = ∑ b, p b • φ b ⊗ φ b` on `E`. -/ -lemma span_le_range_sum_smul_rankOne - (hu : IsUnit (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b))) : - Submodule.span ℝ (Set.range φ) ≤ - LinearMap.range ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E) := - fun v hv ↦ ⟨Ring.inverse (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) ⟨v, hv⟩, by - rw [coe_coe, ← coe_sum_smul_rankOne_spanFeatures_apply, apply_ringInverse_apply hu, - Submodule.coe_mk]⟩ - -variable [FiniteDimensional ℝ E] - -/-- **The estimates on the span are those of the Moore–Penrose inverse.** If the operator -`V_W(p) = ∑ b, p b • φ b ⊗ φ b` on the span `W` of the features is invertible, then the weighted -inner products of the features for its inverse are those for the Moore–Penrose inverse `V(p)⁺` of -the operator `V(p) = ∑ b, p b • φ b ⊗ φ b` on `E`: `⟪φ a, V_W(p)⁻¹ φ c⟫ = ⟪φ a, V(p)⁺ φ c⟫`. -/ -theorem inner_ringInverse_sum_smul_rankOne_spanFeatures - (hu : IsUnit (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b))) (a c : α) : - ⟪spanFeatures φ a, Ring.inverse (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) - (spanFeatures φ c)⟫_ℝ = - ⟪φ a, ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E).moorePenroseInverse - (φ c)⟫_ℝ := by - have hsym : ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E).IsSymmetric := - fun x y ↦ by - simp only [coe_coe, sum_apply, smul_apply, rankOne_apply, sum_inner, inner_sum, - real_inner_smul_left, real_inner_smul_right] - exact Finset.sum_congr rfl fun b _ ↦ by rw [real_inner_comm x (φ b)]; ring - -- `φ c` is the image under `V(p)` of `V_W(p)⁻¹ φ c`, which lies in the range of `V(p)` - have hc : φ c = ((∑ b, p b • rankOne ℝ (φ b) (φ b) : E →L[ℝ] E) : E →ₗ[ℝ] E) - ↑(Ring.inverse (∑ b, p b • rankOne ℝ (spanFeatures φ b) (spanFeatures φ b)) - (spanFeatures φ c)) := by - rw [coe_coe, ← coe_sum_smul_rankOne_spanFeatures_apply, apply_ringInverse_apply hu, - coe_spanFeatures] - rw [hc, hsym.moorePenroseInverse_apply_apply_of_mem_range - (span_le_range_sum_smul_rankOne φ hu (Submodule.coe_mem _)), Submodule.coe_inner, - coe_spanFeatures] +variable [Fintype α] [FiniteDimensional ℝ E] (φ : α → E) {p : α → ℝ} /-- If the features span `E` and the operator `V_W(p) = ∑ b, p b • φ b ⊗ φ b` on their span is invertible, then the loss estimates of Exp3 for linear bandits with the features in their span are @@ -179,10 +77,10 @@ theorem linearExp3Estimate_spanFeatures (hspan : Submodule.span ℝ (Set.range -- `V(p)` is onto, hence invertible, and its Moore–Penrose inverse is its inverse have hVu : IsUnit (∑ b, p b • rankOne ℝ (φ b) (φ b)) := by rw [isUnit_iff_isUnit_toLinearMap, LinearMap.isUnit_iff_range_eq_top, eq_top_iff, ← hspan] - exact span_le_range_sum_smul_rankOne φ hu + exact le_range_of_apply_coe_eq (fun v ↦ by simp) hu funext c ℓ a rw [linearExp3Estimate_apply, linearExp3Estimate_apply, - inner_ringInverse_sum_smul_rankOne_spanFeatures φ hu, moorePenroseInverse_eq_ringInverse hVu, + inner_ringInverse_sum_smul_rankOne_spanFeatures hu, moorePenroseInverse_eq_ringInverse hVu, coe_coe] end Estimate @@ -195,34 +93,34 @@ variable [Fintype α] [FiniteDimensional ℝ E] [Nonempty α] [MeasurableSpace /-- **Exp3 for adversarial linear bandits with the Kiefer–Wolfowitz exploration**: Exp3 for linear bandits [LS, Algorithm 15] with learning rate `η` and exploration rate `γ ∈ [0, 1]`, run on the span of the features `φ`, with the features `spanFeatures φ` and the exploration -`kieferWolfowitzExploration φ`. The features need not span `E`. -/ +`kieferWolfowitzDesign φ`. The features need not span `E`. -/ noncomputable def linearExp3KieferWolfowitz (hγ : γ ∈ Set.Icc 0 1) : Algorithm 𝓞 α ℝ := - linearExp3 (spanFeatures φ) (kieferWolfowitzExploration φ) η γ - (kieferWolfowitzExploration_mem_simplex φ) hγ + linearExp3 (spanFeatures φ) (kieferWolfowitzDesign φ) η γ + (kieferWolfowitzDesign_mem_simplex φ) hγ lemma linearExp3KieferWolfowitz_def (hγ : γ ∈ Set.Icc 0 1) : linearExp3KieferWolfowitz (𝓞 := 𝓞) φ η γ hγ = - linearExp3 (spanFeatures φ) (kieferWolfowitzExploration φ) η γ - (kieferWolfowitzExploration_mem_simplex φ) hγ := (rfl) + linearExp3 (spanFeatures φ) (kieferWolfowitzDesign φ) η γ + (kieferWolfowitzDesign_mem_simplex φ) hγ := (rfl) /-- When the features span `E` and the exploration rate is positive, Exp3 for linear bandits with the Kiefer–Wolfowitz exploration is Exp3 for linear bandits [LS, Algorithm 15] with the features -`φ` in `E` and the exploration `kieferWolfowitzExploration φ`. -/ +`φ` in `E` and the exploration `kieferWolfowitzDesign φ`. -/ theorem linearExp3KieferWolfowitz_eq_linearExp3 (hspan : Submodule.span ℝ (Set.range φ) = ⊤) (hγ : γ ∈ Set.Icc 0 1) (hγ0 : 0 < γ) : linearExp3KieferWolfowitz (𝓞 := 𝓞) φ η γ hγ = - linearExp3 φ (kieferWolfowitzExploration φ) η γ - (kieferWolfowitzExploration_mem_simplex φ) hγ := by + linearExp3 φ (kieferWolfowitzDesign φ) η γ + (kieferWolfowitzDesign_mem_simplex φ) hγ := by have hsum (n : ℕ) (h : Hist 𝓞 α ℝ n) : - linearExp3HistSum (spanFeatures φ) (kieferWolfowitzExploration φ) η γ n h = - linearExp3HistSum φ (kieferWolfowitzExploration φ) η γ n h := by + linearExp3HistSum (spanFeatures φ) (kieferWolfowitzDesign φ) η γ n h = + linearExp3HistSum φ (kieferWolfowitzDesign φ) η γ n h := by induction n with | zero => simp | succ n ih => rw [linearExp3HistSum_succ, linearExp3HistSum_succ, ih, linearExp3Estimate_spanFeatures φ hspan (isUnit_sum_smul_rankOne_linearExp3Weights - (mem_simplex_iff.1 (kieferWolfowitzExploration_mem_simplex φ)).1 - (isUnit_kieferWolfowitzExploration φ) hγ0 hγ.2 _)] + (mem_simplex_iff.1 (kieferWolfowitzDesign_mem_simplex φ)).1 + (isUnit_sum_smul_rankOne_kieferWolfowitzDesign φ) hγ0 hγ.2 _)] rw [linearExp3KieferWolfowitz_def, linearExp3_def, linearExp3_def] congr 1 funext n x @@ -247,7 +145,8 @@ and let the feedback `Y t` of a run of `linearExp3KieferWolfowitz φ η γ` be t `⟪φ (A t), y t (h, o)⟫` of the arm played. With the learning rate `η = √(log k / (3 m n))` and the exploration rate `γ = η m`, the expected regret after `n ≥ 1` rounds against every arm is at most `2 √(3 m n log k)`. -/ -theorem integral_regret_le_kieferWolfowitz [IsProbabilityMeasure P] +theorem integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three + [IsProbabilityMeasure P] (h : IsAlgEnvSeq O A Y (linearExp3KieferWolfowitz φ η γ hγ) env P) (hk : 1 < Fintype.card α) (hn : 0 < n) (hη : η = √(Real.log (Fintype.card α) / @@ -269,8 +168,8 @@ theorem integral_regret_le_kieferWolfowitz [IsProbabilityMeasure P] rw [linearExp3KieferWolfowitz_def] at h have key := integral_regret_le_two_mul_sqrt_three (y := fun t x ↦ (Submodule.span ℝ (Set.range φ)).orthogonalProjectionOnto (y t x)) h - (isUnit_kieferWolfowitzExploration φ) - (inner_ringInverse_kieferWolfowitzExploration_le_finrank φ) hk hn hη hγm + (isUnit_sum_smul_rankOne_kieferWolfowitzDesign φ) + (inner_ringInverse_sum_smul_rankOne_kieferWolfowitzDesign_le_finrank φ) hk hn hη hγm (fun t ht ↦ (Submodule.span ℝ _).orthogonalProjectionOnto.continuous.measurable.comp (hy t ht)) (fun t ht x c ↦ by rw [inner_spanFeatures_orthogonalProjectionOnto]; exact hy1 t ht x c) @@ -280,9 +179,10 @@ theorem integral_regret_le_kieferWolfowitz [IsProbabilityMeasure P] /-- **Regret of Exp3 for adversarial linear bandits with the Kiefer–Wolfowitz exploration against an adaptive adversary, in terms of the dimension of the space** [LS, Theorem 27.1]. Under the -hypotheses of `LinearExp3.integral_regret_le_kieferWolfowitz`, on a space of dimension `d`, the -expected regret against every arm is at most `2 √(3 d n log k)`. -/ -theorem integral_regret_le_sqrt_finrank_kieferWolfowitz [IsProbabilityMeasure P] +hypotheses of `LinearExp3.integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three`, on a +space of dimension `d`, the expected regret against every arm is at most `2 √(3 d n log k)`. -/ +theorem integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three_finrank + [IsProbabilityMeasure P] (h : IsAlgEnvSeq O A Y (linearExp3KieferWolfowitz φ η γ hγ) env P) (hk : 1 < Fintype.card α) (hn : 0 < n) (hη : η = √(Real.log (Fintype.card α) / @@ -293,7 +193,8 @@ theorem integral_regret_le_sqrt_finrank_kieferWolfowitz [IsProbabilityMeasure P] ∫ ω, ∑ t ∈ range n, (⟪φ (A t ω), y t (history O A Y t ω, O t ω)⟫_ℝ - ⟪φ b, y t (history O A Y t ω, O t ω)⟫_ℝ) ∂P ≤ 2 * √(3 * finrank ℝ E * n * Real.log (Fintype.card α)) := by - refine (integral_regret_le_kieferWolfowitz h hk hn hη hγm hy hy1 hY b).trans ?_ + refine (integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three h hk hn hη hγm hy hy1 hY + b).trans ?_ have hm : (finrank ℝ (Submodule.span ℝ (Set.range φ)) : ℝ) ≤ finrank ℝ E := Nat.cast_le.2 (Submodule.finrank_le _) have hlog : 0 ≤ Real.log (Fintype.card α) := Real.log_nonneg (by exact_mod_cast hk.le) @@ -305,10 +206,11 @@ adversary choose at round `t` the loss vector `y t (h, o) ∈ E`, a measurable f history `h` and of the observation `o` of round `t`, drawn from arbitrary observation kernels `obs`, with `|⟪φ c, y t (h, o)⟫| ≤ 1` for every arm `c`, and let its environment `Environment.detFeedback obs (fun t x ↦ ⟪φ x.2, y t x.1⟫) _` return the loss of the arm played. -With the rates of `LinearExp3.integral_regret_le_kieferWolfowitz`, every run of -`linearExp3KieferWolfowitz φ η γ` against it has expected regret at most `2 √(3 m n log k)` against -every arm, where `m` is the dimension of the span of the features. -/ -theorem integral_regret_detFeedback_le_kieferWolfowitz [IsProbabilityMeasure P] +With the rates of `LinearExp3.integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three`, +every run of `linearExp3KieferWolfowitz φ η γ` against it has expected regret at most +`2 √(3 m n log k)` against every arm, where `m` is the dimension of the span of the features. -/ +theorem integral_regret_detFeedback_linearExp3KieferWolfowitz_le_two_mul_sqrt_three + [IsProbabilityMeasure P] {obs : (n : ℕ) → Kernel (Hist 𝓞 α ℝ n) 𝓞} [∀ n, IsMarkovKernel (obs n)] {hf : ∀ n, Measurable fun x : (Hist 𝓞 α ℝ n × 𝓞) × α ↦ ⟪φ x.2, y n x.1⟫_ℝ} (h : IsAlgEnvSeq O A Y (linearExp3KieferWolfowitz φ η γ hγ) @@ -322,7 +224,7 @@ theorem integral_regret_detFeedback_le_kieferWolfowitz [IsProbabilityMeasure P] ⟪φ b, y t (history O A Y t ω, O t ω)⟫_ℝ) ∂P ≤ 2 * √(3 * finrank ℝ (Submodule.span ℝ (Set.range φ)) * n * Real.log (Fintype.card α)) := - integral_regret_le_kieferWolfowitz h hk hn hη hγm hy hy1 + integral_regret_linearExp3KieferWolfowitz_le_two_mul_sqrt_three h hk hn hη hγm hy hy1 (fun t _ ↦ h.feedback_detFeedback_ae_eq t) b end LinearExp3 From 65ebfb86beed44e64332692d64478d26f5caa7a2 Mon Sep 17 00:00:00 2001 From: Remy Degenne Date: Sun, 15 Nov 2026 22:53:25 +0100 Subject: [PATCH 3/5] refactor: move the operator restriction lemmas to MoorePenrose MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `apply_coe_ringInverse_of_apply_coe_eq`, `le_range_of_apply_coe_eq` and `inner_moorePenroseInverse_of_apply_coe_eq` are generic facts about an operator acting on a subspace and its Moore–Penrose inverse, with no design content. Move them from OptimalDesign.lean to MoorePenrose.lean, in the `ContinuousLinearMap` namespace next to `moorePenroseInverse_eq_ringInverse`, over a general `RCLike` field. Co-Authored-By: Claude Code --- .../InnerProductSpace/MoorePenrose.lean | 39 ++++++++++++++++++- .../InnerProductSpace/OptimalDesign.lean | 32 --------------- 2 files changed, 38 insertions(+), 33 deletions(-) diff --git a/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean b/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean index da3cf271..f7dd8a8b 100644 --- a/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean +++ b/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean @@ -18,6 +18,8 @@ singular, as for the Gram operator of unregularized least squares or of a design * for every Moore–Penrose pair `(A, B)`, the composition `A B` is the orthogonal projection onto the range of `A`; * the Moore–Penrose inverse of an invertible operator is its inverse; +* if an operator acts on a subspace as an invertible operator of that subspace, its Moore–Penrose + inverse acts there as the inverse of that restriction; * the Moore–Penrose inverse of a symmetric operator `V` is symmetric, has the same range as `V`, and inverts `V` on that range; that of a positive operator is positive. @@ -27,6 +29,10 @@ singular, as for the Gram operator of unregularized least squares or of a design inverse of `A`, then `A B` is the orthogonal projection onto the range of `A`. * `ContinuousLinearMap.moorePenroseInverse_eq_ringInverse`: `V⁺ = V⁻¹` for an invertible operator `V`. +* `ContinuousLinearMap.inner_moorePenroseInverse_of_apply_coe_eq`: if a symmetric operator `T` + on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then + `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`; `ContinuousLinearMap.le_range_of_apply_coe_eq`: then + `K` lies in the range of `T`. * `LinearMap.IsSymmetric.moorePenroseInverse`, `LinearMap.IsPositive.moorePenroseInverse`: the Moore–Penrose inverse of a symmetric (positive) operator is symmetric (positive). * `LinearMap.IsSymmetric.range_moorePenroseInverse`: `V⁺` and a symmetric `V` have the same @@ -138,7 +144,26 @@ end LinearMap namespace ContinuousLinearMap variable {𝕜 E : Type*} [RCLike 𝕜] [NormedAddCommGroup E] [InnerProductSpace 𝕜 E] - [FiniteDimensional 𝕜 E] + +section Restrict + +variable {K : Submodule 𝕜 E} {T : E →L[𝕜] E} {T_K : K →L[𝕜] K} + +/-- If an operator `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then +`T` maps `T_K⁻¹ v` to `v` for `v ∈ K`. -/ +theorem apply_coe_ringInverse_of_apply_coe_eq (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) + (v : K) : T ↑(Ring.inverse T_K v) = v := by + rw [hT, apply_ringInverse_apply hu] + +/-- If an operator `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then +`K` lies in the range of `T`. -/ +theorem le_range_of_apply_coe_eq (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) : + K ≤ LinearMap.range (T : E →ₗ[𝕜] E) := fun v hv ↦ + ⟨_, apply_coe_ringInverse_of_apply_coe_eq hT hu ⟨v, hv⟩⟩ + +end Restrict + +variable [FiniteDimensional 𝕜 E] /-- The Moore–Penrose inverse of an invertible operator is its inverse: `V⁺ = V⁻¹`. -/ theorem moorePenroseInverse_eq_ringInverse {V : E →L[𝕜] E} (hV : IsUnit V) : @@ -147,4 +172,16 @@ theorem moorePenroseInverse_eq_ringInverse {V : E →L[𝕜] E} (hV : IsUnit V) (.of_comp_eq_id (LinearMap.ext (ringInverse_apply_apply hV)) (LinearMap.ext (apply_ringInverse_apply hV)))).symm +/-- If a symmetric operator `T` on a finite-dimensional space `E` acts on a subspace `K` as an +invertible operator `T_K` on `K`, then the Moore–Penrose inverse of `T` acts on `K` as the inverse +of `T_K`, in the weak sense `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`. -/ +theorem inner_moorePenroseInverse_of_apply_coe_eq {K : Submodule 𝕜 E} {T : E →L[𝕜] E} + {T_K : K →L[𝕜] K} (hsym : (T : E →ₗ[𝕜] E).IsSymmetric) (hT : ∀ v : K, T v = T_K v) + (hu : IsUnit T_K) (a c : K) : + ⟪(a : E), (T : E →ₗ[𝕜] E).moorePenroseInverse c⟫_𝕜 = ⟪a, Ring.inverse T_K c⟫_𝕜 := by + have h := hsym.moorePenroseInverse_apply_apply_of_mem_range + (le_range_of_apply_coe_eq hT hu (Ring.inverse T_K c).2) + rw [coe_coe, apply_coe_ringInverse_of_apply_coe_eq hT hu] at h + rw [h, Submodule.coe_inner] + end ContinuousLinearMap diff --git a/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean b/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean index 9b9c26d5..cffb9d26 100644 --- a/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean +++ b/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean @@ -95,8 +95,6 @@ point `a` for each of them gives a design with the same design operator, hence t support of the design. * `LMLExplore.designOperator_map_subtype_apply`: on a subspace `K`, the design operator on `E` of a design on `K` restricts to its design operator on `K`. -* `LMLExplore.inner_moorePenroseInverse_of_apply_coe_eq`: if a symmetric operator `T` on `E` acts - on a subspace `K` as an invertible operator `T_K` on `K`, then `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` on `K`. * `LMLExplore.designOperator_convexCombPair`: `V(w)` is affine in `w`. * `LMLExplore.mem_convexHull_image_rankOne_iff`: the convex hull of `{a ⊗ a | a ∈ 𝒜}` is the set of design operators of the designs on `𝒜`. @@ -341,36 +339,6 @@ theorem isUnit_sum_smul_rankOne_of_mul_le [FiniteDimensional ℝ E] {ι : Type*} have hvp := isPositive_sum_smul_rankOne Finset.univ φ fun i _ ↦ hv i exact (hvp.smul_of_nonneg hc.le).isUnit_of_le hcu (smul_sum_smul_rankOne_le h) -section Restrict - -variable {K : Submodule ℝ E} {T : E →L[ℝ] E} {T_K : K →L[ℝ] K} - -/-- If an operator `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then -`T` maps `T_K⁻¹ v` to `v` for `v ∈ K`. -/ -theorem apply_coe_ringInverse_of_apply_coe_eq (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) - (v : K) : T ↑(Ring.inverse T_K v) = v := by - rw [hT, apply_ringInverse_apply hu] - -/-- If an operator `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then -`K` lies in the range of `T`. -/ -theorem le_range_of_apply_coe_eq (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) : - K ≤ LinearMap.range (T : E →ₗ[ℝ] E) := fun v hv ↦ - ⟨_, apply_coe_ringInverse_of_apply_coe_eq hT hu ⟨v, hv⟩⟩ - -/-- If a symmetric operator `T` on a finite-dimensional space `E` acts on a subspace `K` as an -invertible operator `T_K` on `K`, then the Moore–Penrose inverse of `T` acts on `K` as the inverse -of `T_K`, in the weak sense `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`. -/ -theorem inner_moorePenroseInverse_of_apply_coe_eq [FiniteDimensional ℝ E] - (hsym : (T : E →ₗ[ℝ] E).IsSymmetric) (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) - (a c : K) : - ⟪(a : E), (T : E →ₗ[ℝ] E).moorePenroseInverse c⟫_ℝ = ⟪a, Ring.inverse T_K c⟫_ℝ := by - have h := hsym.moorePenroseInverse_apply_apply_of_mem_range - (le_range_of_apply_coe_eq hT hu (Ring.inverse T_K c).2) - rw [coe_coe, apply_coe_ringInverse_of_apply_coe_eq hT hu] at h - rw [h, Submodule.coe_inner] - -end Restrict - /-! ### Features in their span -/ section SpanFeatures From c469622eb45699b76b5f5d4fd053e73050b9fe7a Mon Sep 17 00:00:00 2001 From: Remy Degenne Date: Sun, 15 Nov 2026 23:15:28 +0100 Subject: [PATCH 4/5] =?UTF-8?q?feat:=20state=20the=20Moore=E2=80=93Penrose?= =?UTF-8?q?=20restriction=20lemma=20as=20a=20vector=20identity?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add ContinuousLinearMap.moorePenroseInverse_apply_coe_of_apply_coe_eq, T⁺ c = T_K⁻¹ c for c ∈ K, and derive the inner-product form from it. Co-Authored-By: Claude Code --- .../InnerProductSpace/MoorePenrose.lean | 23 +++++++++++++------ 1 file changed, 16 insertions(+), 7 deletions(-) diff --git a/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean b/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean index f7dd8a8b..7f767d8a 100644 --- a/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean +++ b/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean @@ -29,8 +29,9 @@ singular, as for the Gram operator of unregularized least squares or of a design inverse of `A`, then `A B` is the orthogonal projection onto the range of `A`. * `ContinuousLinearMap.moorePenroseInverse_eq_ringInverse`: `V⁺ = V⁻¹` for an invertible operator `V`. -* `ContinuousLinearMap.inner_moorePenroseInverse_of_apply_coe_eq`: if a symmetric operator `T` - on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then +* `ContinuousLinearMap.moorePenroseInverse_apply_coe_of_apply_coe_eq`: if a symmetric operator + `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then `T⁺ c = T_K⁻¹ c` + for `c ∈ K`, and `ContinuousLinearMap.inner_moorePenroseInverse_of_apply_coe_eq`: `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`; `ContinuousLinearMap.le_range_of_apply_coe_eq`: then `K` lies in the range of `T`. * `LinearMap.IsSymmetric.moorePenroseInverse`, `LinearMap.IsPositive.moorePenroseInverse`: the @@ -174,14 +175,22 @@ theorem moorePenroseInverse_eq_ringInverse {V : E →L[𝕜] E} (hV : IsUnit V) /-- If a symmetric operator `T` on a finite-dimensional space `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then the Moore–Penrose inverse of `T` acts on `K` as the inverse -of `T_K`, in the weak sense `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`. -/ +of `T_K`: `T⁺ c = T_K⁻¹ c` for `c ∈ K`. -/ +theorem moorePenroseInverse_apply_coe_of_apply_coe_eq {K : Submodule 𝕜 E} {T : E →L[𝕜] E} + {T_K : K →L[𝕜] K} (hsym : (T : E →ₗ[𝕜] E).IsSymmetric) (hT : ∀ v : K, T v = T_K v) + (hu : IsUnit T_K) (c : K) : + (T : E →ₗ[𝕜] E).moorePenroseInverse c = Ring.inverse T_K c := by + have h := hsym.moorePenroseInverse_apply_apply_of_mem_range + (le_range_of_apply_coe_eq hT hu (Ring.inverse T_K c).2) + rwa [coe_coe, apply_coe_ringInverse_of_apply_coe_eq hT hu] at h + +/-- If a symmetric operator `T` on a finite-dimensional space `E` acts on a subspace `K` as an +invertible operator `T_K` on `K`, then `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`, the inner products +on the right being those of `K`. -/ theorem inner_moorePenroseInverse_of_apply_coe_eq {K : Submodule 𝕜 E} {T : E →L[𝕜] E} {T_K : K →L[𝕜] K} (hsym : (T : E →ₗ[𝕜] E).IsSymmetric) (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) (a c : K) : ⟪(a : E), (T : E →ₗ[𝕜] E).moorePenroseInverse c⟫_𝕜 = ⟪a, Ring.inverse T_K c⟫_𝕜 := by - have h := hsym.moorePenroseInverse_apply_apply_of_mem_range - (le_range_of_apply_coe_eq hT hu (Ring.inverse T_K c).2) - rw [coe_coe, apply_coe_ringInverse_of_apply_coe_eq hT hu] at h - rw [h, Submodule.coe_inner] + rw [moorePenroseInverse_apply_coe_of_apply_coe_eq hsym hT hu, Submodule.coe_inner] end ContinuousLinearMap From 1efdf6044ef85ffefbe5a7f28559730d52ddd93c Mon Sep 17 00:00:00 2001 From: Remy Degenne Date: Sun, 15 Nov 2026 23:42:49 +0100 Subject: [PATCH 5/5] =?UTF-8?q?refactor:=20put=20the=20symmetric=20Moore?= =?UTF-8?q?=E2=80=93Penrose=20restriction=20lemmas=20in=20LinearMap.IsSymm?= =?UTF-8?q?etric?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Code --- .../InnerProductSpace/MoorePenrose.lean | 20 +++++++++++++------ .../InnerProductSpace/OptimalDesign.lean | 4 ++-- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean b/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean index 7f767d8a..7c0b2ca3 100644 --- a/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean +++ b/LMLExplore/Analysis/InnerProductSpace/MoorePenrose.lean @@ -29,9 +29,9 @@ singular, as for the Gram operator of unregularized least squares or of a design inverse of `A`, then `A B` is the orthogonal projection onto the range of `A`. * `ContinuousLinearMap.moorePenroseInverse_eq_ringInverse`: `V⁺ = V⁻¹` for an invertible operator `V`. -* `ContinuousLinearMap.moorePenroseInverse_apply_coe_of_apply_coe_eq`: if a symmetric operator +* `LinearMap.IsSymmetric.moorePenroseInverse_apply_coe_of_apply_coe_eq`: if a symmetric operator `T` on `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then `T⁺ c = T_K⁻¹ c` - for `c ∈ K`, and `ContinuousLinearMap.inner_moorePenroseInverse_of_apply_coe_eq`: + for `c ∈ K`, and `LinearMap.IsSymmetric.inner_moorePenroseInverse_of_apply_coe_eq`: `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`; `ContinuousLinearMap.le_range_of_apply_coe_eq`: then `K` lies in the range of `T`. * `LinearMap.IsSymmetric.moorePenroseInverse`, `LinearMap.IsPositive.moorePenroseInverse`: the @@ -173,6 +173,13 @@ theorem moorePenroseInverse_eq_ringInverse {V : E →L[𝕜] E} (hV : IsUnit V) (.of_comp_eq_id (LinearMap.ext (ringInverse_apply_apply hV)) (LinearMap.ext (apply_ringInverse_apply hV)))).symm +end ContinuousLinearMap + +namespace LinearMap.IsSymmetric + +variable {𝕜 E : Type*} [RCLike 𝕜] [NormedAddCommGroup E] [InnerProductSpace 𝕜 E] + [FiniteDimensional 𝕜 E] + /-- If a symmetric operator `T` on a finite-dimensional space `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then the Moore–Penrose inverse of `T` acts on `K` as the inverse of `T_K`: `T⁺ c = T_K⁻¹ c` for `c ∈ K`. -/ @@ -181,8 +188,9 @@ theorem moorePenroseInverse_apply_coe_of_apply_coe_eq {K : Submodule 𝕜 E} {T (hu : IsUnit T_K) (c : K) : (T : E →ₗ[𝕜] E).moorePenroseInverse c = Ring.inverse T_K c := by have h := hsym.moorePenroseInverse_apply_apply_of_mem_range - (le_range_of_apply_coe_eq hT hu (Ring.inverse T_K c).2) - rwa [coe_coe, apply_coe_ringInverse_of_apply_coe_eq hT hu] at h + (ContinuousLinearMap.le_range_of_apply_coe_eq hT hu (Ring.inverse T_K c).2) + rwa [ContinuousLinearMap.coe_coe, + ContinuousLinearMap.apply_coe_ringInverse_of_apply_coe_eq hT hu] at h /-- If a symmetric operator `T` on a finite-dimensional space `E` acts on a subspace `K` as an invertible operator `T_K` on `K`, then `⟪a, T⁺ c⟫ = ⟪a, T_K⁻¹ c⟫` for `a c ∈ K`, the inner products @@ -191,6 +199,6 @@ theorem inner_moorePenroseInverse_of_apply_coe_eq {K : Submodule 𝕜 E} {T : E {T_K : K →L[𝕜] K} (hsym : (T : E →ₗ[𝕜] E).IsSymmetric) (hT : ∀ v : K, T v = T_K v) (hu : IsUnit T_K) (a c : K) : ⟪(a : E), (T : E →ₗ[𝕜] E).moorePenroseInverse c⟫_𝕜 = ⟪a, Ring.inverse T_K c⟫_𝕜 := by - rw [moorePenroseInverse_apply_coe_of_apply_coe_eq hsym hT hu, Submodule.coe_inner] + rw [hsym.moorePenroseInverse_apply_coe_of_apply_coe_eq hT hu, Submodule.coe_inner] -end ContinuousLinearMap +end LinearMap.IsSymmetric diff --git a/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean b/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean index cffb9d26..45e532e5 100644 --- a/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean +++ b/LMLExplore/Analysis/InnerProductSpace/OptimalDesign.lean @@ -706,7 +706,7 @@ theorem inner_moorePenroseInverse_designOperator_map_subtype {K : Submodule ℝ {w : StdSimplex ℝ K} (hu : IsUnit (designOperator w)) (a : K) : ⟪(a : E), (designOperator (w.map ((↑) : K → E)) : E →ₗ[ℝ] E).moorePenroseInverse a⟫_ℝ = ⟪a, Ring.inverse (designOperator w) a⟫_ℝ := - inner_moorePenroseInverse_of_apply_coe_eq (isPositive_designOperator _).toLinearMap.isSymmetric + (isPositive_designOperator _).toLinearMap.isSymmetric.inner_moorePenroseInverse_of_apply_coe_eq (designOperator_map_subtype_apply w) hu a a /-- **The inverse on the span is the Moore–Penrose inverse.** If the operator @@ -720,7 +720,7 @@ theorem inner_ringInverse_sum_smul_rankOne_spanFeatures {ι : Type*} [Fintype ι (spanFeatures φ c)⟫_ℝ = ⟪φ a, ((∑ i, p i • rankOne ℝ (φ i) (φ i) : E →L[ℝ] E) : E →ₗ[ℝ] E).moorePenroseInverse (φ c)⟫_ℝ := - (inner_moorePenroseInverse_of_apply_coe_eq (isSymmetric_sum_smul_rankOne _ φ p) + ((isSymmetric_sum_smul_rankOne _ φ p).inner_moorePenroseInverse_of_apply_coe_eq (fun v ↦ by simp) hu _ _).symm /-- **Support reduction of G-optimal designs** [LS, Theorem 21.1]. Let `w` be a design with