Tracking: live RL metrics suite in the statusline (gated on ruflo#2239 + RuVector#519)

[pacphi]

Tracking: live RL metrics suite in the statusline (gated on two upstream fixes)

North star: an RL metrics panel in the statusline footer, live-updated from real CLI/API use and actual learnings — not a synthetic one-shot proof. This issue collects every material needed to kick the work off the moment the two upstream blockers close, so we don't have to re-derive context.

Blocked by (do not start until both are closed/fixed)

• F3 — route Q-state encoder collapse → Route Q-state encoder discards the entire keyword block — keyword-distinct tasks collapse to one Q-state (q-learning-router.js) ruvnet/ruflo#2239
  • q-learning-router.js featureVectorToKey discards features 0–31 (the entire keyword-presence block) via a 31-bit truncating hash fold; keyword-distinct tasks collapse to one Q-state. Directly degrades a metric we want to display (|Q| undercounts true task diversity).
• F4 — SONA learn→inference loop unwired at the JS/WASM boundary → Bug: SONA learn→inference loop unwired at the JS/WASM boundary — learn_from_feedback is a no-op; MicroLoRA adapts only on multi-step varying-reward trajectories ruvnet/RuVector#519
  • WasmSonaEngine::learn_from_feedback is a no-op stub; JS SonaCoordinator.processInstantLearning is empty; single-step trajectories yield Δ=0 (REINFORCE baseline) and only up_proj ever adapts. Gates the SONA / Δ-LoRA part of the panel — must not present Δ-LoRA as "improvement" until this lands.

Upstream history (already resolved — context, not blockers)

• #2222 (closed) filed four findings; ruvnet fixed F1/F2 (route-feedback persistence via saveModel(), 3.10.6, @pacphi credited). F2b neg-reward parser → 3.10.7. Route cache + --explore false → 3.10.8. ADR-142 model-router bandit → 3.10.9 (separate subsystem). Installed/latest = 3.10.10.
• Full reconciliation + findings: docs/upstream/ruflo-self-improvement-findings.md.

---

What to build once unblocked

Revive the dropped Task 5 from the original design as a live panel. The original 📈 RL line was deferred because its only source was the synthetic improvement.json eval — "thin without live value." The fix: source the genuinely-live signals from the persisted model, keep the eval as a periodic credibility anchor, and gate the SONA/Δ-LoRA signal behind F4.

Panel	Source	Live from real CLI/API?	Gate
Route Q-learner — ε↓, δ̄ (TD error), |Q|, updateCount	.swarm/q-learning-model.json (persists since 3.10.6 F2 fix)	Yes — now	Footnote |Q| until F3 lands (shape-bucketed, undercounts)
Proof / verdict — PASS/FAIL, Δpp, CI, p, d	.claude-flow/improvement.json (from ruflo-improvement-eval)	Periodic (eval/daemon)	none
SONA patterns / trajectories	.claude-flow/neural/stats.json	Yes (already in footer)	none
Δ LoRA (adaptation signal)	transient MicroLoRA delta	No	Blocked on F4 — do not render as "improvement" until the learn path is wired

Activation checklist (when F3 + F4 close)

• Re-verify F3 fix shipped: q-learning-router.js encoder no longer truncates keyword groups (re-run the collapse probe in bin/ruflo-improvement-eval --probe-states; expect N tasks → N states).
• Re-verify F4 fix shipped: learn_from_feedback (or equivalent) actually moves weights; re-run the repro_delta_zero cargo test against the new ruvector — single-step Δ should be > 0 (or the API should expose a documented adapt+flush path).
• Implement the live route-Q panel in shell/ruflo-functions.sh (the rufloActivationSegments heredoc helper): fs-only read of .swarm/q-learning-model.json → 📈 RL ε<e>↓ · δ̄<td>↓ · |Q|<n> · upd<c>. Marker-guarded, upgrade-safe (same injector as the existing footer / ruflo-fix-statusline-version).
• Add the eval-verdict segment (PASS/FAIL + Δpp/CI/p/d from improvement.json) — render ◷ RL on FAIL or absent.
• Un-gate the SONA Δ-LoRA segment only after F4 verification passes.
• Wire a refresh path so the eval verdict stays current (e.g. ruflo daemon worker or an on-demand ruflo-improvement-eval hook). Decide cadence.
• ruflo-resync must re-inject the segment after a ruflo/aqe upgrade regenerates the statusline.
• Honesty pass: docs state which signals are live vs periodic vs gated; no fabricated "improvement."

Materials / references (everything needed to start)

• Design spec (historical, Task 5 dropped): docs/superpowers/specs/2026-05-28-self-improvement-eval-design.md — §3.1 statusline 📈 RL segment contract, R9–R11, S1–S5.
• Implementation plan (historical): docs/superpowers/plans/2026-05-28-self-improvement-eval.md — Task 5 has the full rufloActivationSegments heredoc patch, idempotency test, and live-apply steps (sourced improvement.json; adapt to source .swarm/q-learning-model.json for the live route panel).
• Proof harness: bin/ruflo-improvement-eval — --probe-states (F3 collapse check), --cli-check (F2 persistence, version-aware), --inspect-decision, --smoke, --json, --check. Writes .claude-flow/improvement.json.
• F4 reproduction test: the repro_delta_zero.rs cargo test (full source in Bug: SONA learn→inference loop unwired at the JS/WASM boundary — learn_from_feedback is a no-op; MicroLoRA adapts only on multi-step varying-reward trajectories ruvnet/RuVector#519) — re-run post-fix as the un-gate gate.
• Statusline injector: shell/ruflo-functions.sh → ruflo-fix-statusline-version (marker-guarded heredoc; never rewrite ruflo's native lines).
• Existing footer fields (claude/ruflo-reference.md "Status-line activation footer"): SONA patterns/traj, ⚡ HNSW, 🛡 aidefence, 🎓 Agentic QE, Δ LoRA (currently from ruflo-neural-train cache — revisit under F4).

Acceptance

The footer shows a live 📈 RL line whose route-Q fields (ε/δ̄/|Q|/updates) move in response to real ruflo route feedback / routing use, the eval verdict reflects the last A/B run, and the SONA adaptation signal is either un-gated (F4 fixed) or honestly absent. No metric is sourced from route stats (broken CLI path) and none is fabricated.

[pacphi]

Prior work landed (resume materials)

The completed, reusable pieces are now on main via #9 (feat/improvement-eval-harness) as new files only — no statusline/resync changes — so kickoff won't start from scratch:

• 🔬 Proof harness: bin/ruflo-improvement-eval (standalone; --probe-states is the F3 collapse check, --cli-check the F2-persistence check). Re-use as the A/B measuring stick once the panel exists.
• 📐 Panel design (the implementation to revive): docs/superpowers/plans/2026-05-28-self-improvement-eval.md Task 5 has the full rufloActivationSegments heredoc patch + idempotency test — adapt it to source the live route fields from .swarm/q-learning-model.json (not the synthetic improvement.json).
• 📄 Findings of record: docs/upstream/ruflo-self-improvement-findings.md (F4 corrected against real ruvector source).
• 🔁 bin/ruflo-patch-route-learning — retired version-gated shim (not wired into resync).

Note: the old PR #2 was closed unmerged and #9 supersedes it; the feat/self-improvement-eval branch is kept only as history. Build the panel on a fresh branch off main when ruvnet/ruflo#2239 + ruvnet/RuVector#519 close (see the activation checklist above).

[pacphi]

Reconciliation against ruflo 3.10.31 (2026-06-01) — F3 fixed, F4 still gated

Re-verified both blockers against the installed 3.10.31 source (not just changelogs) and ran the activation-checklist probes.

✅ F3 (ruvnet/ruflo#2239) — FIXED. Route-Q panel unblocked.

q-learning-router.js featureVectorToKey now uses a 32-bit FNV-1a over all 16 quantized groups (the old ((hash<<4)^q[i]) & 0x7fffffff fold that discarded the keyword block is documented in the source comment as the prior bug). ENCODER_VERSION = 2, with a v1→v2 migration guard in loadModel and encoderVersion persisted in saveModel — all carrying explicit #2239 markers.

• Probe passes: ruflo-improvement-eval --probe-states → distinct Q-states for 6 train tasks: 6 (was 1 at 3.10.9). Exit 0.
• --cli-check → persists, fixed upstream 3.10.6 (#2222).
• ⇒ |Q| is now a real task-diversity count. The live route-Q panel is meaningful and built (see below).

⚠️ F4 (ruvnet/RuVector#519) — STILL OPEN. Δ-LoRA stays gated.

@ruvector/ruvllm@2.5.5 sona.js processInstantLearning is still a no-op stub (byte-identical "In full implementation, this updates LoRA weights" comment). Matrix forward_array is consumed only in services/ruvector-training.js (manual training) + the opt-in ruvllm-* MCP tools — never in commands/route.js or any recall decision. On-disk lora-delta.json shows deltaNorm: 0.000000.

Nuance worth recording: a scalar confidence loop now does close (intelligence.js distillLearning updates pattern.confidence += loraLearningRate*reward, read back via findSimilar at recall) — but that is not the matrix Δ-LoRA. So F4's core claim holds: the trained matrix-LoRA changes no decision.

Work landed in this repo (this PR)

• 📈 RL route panel added to the rufloActivationSegments injector — ε↓ · δ̄↓ · |Q| · upd, fs-only from .swarm/q-learning-model.json (fallback .claude-flow/metrics/learning.json routing.{accuracy,decisions}), never route stats. Honesty-gated: renders only when updateCount > 0 / decisions > 0, so it stays absent until routing is actually exercised.
• Δ LoRA field REMOVED (was rendering an inert metric) — re-add only when #519 closes and the delta provably moves.
• Docs reconciled (README, ruflo-reference, BACKGROUND, design spec R16a/R16b); noted stats.json patterns/traj now persist across restarts (#2245).

Correction to the earlier resume comment

The prior comment said the harness "landed on main via #9." #9 was never merged (merged PRs: #1/#4/#5/#6/#7/#10). The harness + findings are landing on main now, via this PR — cherry-picked as new files only onto current main (the feat/improvement-eval-harness branch predated #10 and a naive merge would have reverted #10's conditional-blocks work).

Updated checklist

• F3 fix shipped — encoder no longer truncates keyword groups; --probe-states → N tasks = N states.
• F4 fix shipped — RuVector#519 still open; processInstantLearning stub, delta inert. Δ-LoRA segment remains removed/gated.
• Live route-Q panel implemented (shell/ruflo-functions.sh, marker-guarded, upgrade-safe, honesty-gated).
• Eval-verdict segment (improvement.json PASS/FAIL) — deferred (optional follow-up).
• Un-gate SONA Δ-LoRA — blocked on #519.
• ruflo-resync re-injects the segment (same injector as the existing footer).
• Honesty pass — no fabricated signals; live vs gated documented.

Net: the route-Q half is shippable now (F3 done); only the Δ-LoRA "improvement" field stays blocked on ruvnet/RuVector#519.