dashboard: verify Pareto/ownership views against real archive schema, add utopia-nadir normalization, fix optimizer-cost telemetry gap

[OsherElhadad]

Summary

Issue #684 items 9 and 10 only (dashboard Pareto/ownership verification + utopia-nadir
normalization, and the optimizer_seconds/optimizer_usd telemetry gap). Does not touch
round.py's gate logic (item 1/2), screening (5/6), merge_search.py (4), plan_round.py (3),
cost-reduction guidance (7), or hill-climb (8) — those are owned by sibling PRs #685-#690.

• Telemetry fix (item 10): optimizer_seconds/optimizer_usd were always 0 in agent-mode
  runs. Root cause: meter.py's automatic metering only works under host.py's headless driver
  (it reads a claude-code session log that only exists there) — a SKILL.md compliance gap, not a
  framework instrumentation gap. commit.py now logs an optimizer_cost_warning when real
  wall-clock time clearly passed since the run's previous decision but the new commit still
  carries zero optimizer cost (surfaced in commit.py's own warnings list and on the dashboard
  node detail, same pattern as the existing context_warning). SKILL.md's guidance was
  strengthened; the detailed rationale lives in algorithm.md's new "Optimizer cost telemetry"
  section to stay inside the repo's 5000-token skill-body budget (confirmed by
  test_agent_optimize_host.py/test_skill_authoring_lint.py, which my first draft tripped).
• Dashboard verification against real pareto data (item 9): built a synthetic multi-candidate
  reward+cost fixture with genuine dominance (two frontier candidates, one strictly dominated) and
  confirmed dashboard.reduce_run's existing node shape (val/cost_usd/per_task/
  per_task_metrics) correctly feeds the frontend's already-tested Pareto frontier logic and
  per-objective Tasks-tab deltas. Also wired through round.py's real pareto-gate output schema,
  confirmed directly against PR optimizer: native Pareto archive gate + epsilon-constraint mode in round.py #690's merged diff: objective_values/objective_stderrs/
  pareto_archive (inserted/reason/values) per candidate, read off the same gate-table lookup
  dashboard.py already used for verdict_stable — no round.py or commit.py change needed.
• Utopia/nadir normalization: optional normalized-axes toggle on the Pareto scatter
  (lib/pareto.ts's normalizePoints, MOO survey eq. 7), approximating utopia/nadir from
  best/worst observed values across the run (not solved exactly, per the paper's own cost
  caveat). Raw units stay the default view; normalized is additive.
• Per-task ownership view: new Tasks-tab panel showing which candidate currently best-scores
  each task (GEPA's per-instance ownership idea), computed client-side from each GraphNode's
  existing per_task data (lib/ownership.ts). Verified directly against PR optimizer: per-task ownership tracking for diverse parent/branch selection (GEPA-inspired) #687's merged
  compute_ownership() schema (owners/best_score/ownership_count) — the client-side
  computation matches it exactly, so this needed no new backend field and no dependency on optimizer: per-task ownership tracking for diverse parent/branch selection (GEPA-inspired) #687
  landing first.

What I found when testing against real pareto-shaped data

• The existing ParetoScatter/lib/pareto.ts frontier logic was already correct (its own unit
  tests cover dominance/ties) — the untested part was the integration: does
  dashboard.reduce_run's real node output actually feed it correctly. A new backend test
  (test_dashboard_pareto_frontier_fixture.py) proves it does, replicating the frontend's
  dominance rule in Python against reduce_run's own val/cost_usd fields.
• dashboard.py's gate_table lookup (added for verdict_stable) was the right place to add the
  new pareto-mode fields — same file already open for verdict_stable, same per-candidate lookup,
  zero round.py/commit.py changes.
• My SKILL.md addition initially pushed the skill body over its 5000-token budget, caught by
  test_agent_optimize_host.py and test_skill_authoring_lint.py — trimmed to a one-line pointer
  with the detail moved to algorithm.md.

Schema provenance

• Pareto archive schema: confirmed against PR optimizer: native Pareto archive gate + epsilon-constraint mode in round.py #690's actual merged diff (not guessed) —
  objective_values/objective_stderrs/pareto_archive: {inserted, reason, values, capacity, size_after} per candidate row, plus a top-level pareto_archive: {version, objectives, capacity, points} persisted at pareto_archive.json.
• Task ownership schema: confirmed against PR optimizer: per-task ownership tracking for diverse parent/branch selection (GEPA-inspired) #687's actual merged compute_ownership() —
  {owners, best_score, ownership_count}. My frontend lib/ownership.ts reimplements the same
  logic client-side over data already exported today (GraphNode.per_task), rather than depending
  on core/cap_evolve/task_ownership.py (which is computed on-demand for plan_round.py's branch
  planning and persists nothing to graph.jsonl/events.jsonl for the dashboard to read).

Test plan

• python -m pytest core/tests -q — 1917 passed, 1 skipped, 2 failed (both
  test_tau2_airline_eval_cost.py — pre-existing, unrelated model-pricing env failures, not
  touched by this PR).
• npx vitest run (frontend) — 28 files, 220 tests, all passing.
• npm run build — typechecks and builds cleanly; dist/ rebuilt and committed (tracked
  intentionally per dashboard/frontend/.gitignore).
• New backend tests: test_commit_optimizer_cost_compliance.py (compliance-warning logic),
  test_dashboard_pareto_frontier_fixture.py (frontier/dominated correctness against synthetic
  archive-shaped data), test_dashboard_pareto_archive_fields.py (real pareto-archive schema
  carry-through).
• New frontend tests: pareto.test.ts (normalizePoints), ownership.test.ts +
  TaskOwnership.test.tsx (ownership computation + rendering).

🤖 Generated with Claude Code

[skillberry-bot]

🏷️ Automatic Labeling

I've analyzed this pull request and added the following labels:

• algorithm - dashboard - observability - bug - enhancement - algorithm - dashboard - observability - bug - enhancement - javascript

These labels were selected based on the PR title, description, and changed files. If you believe any labels are incorrect or missing, feel free to adjust them manually.