[SushantGautam]

Summary

New workspaces had the agentic scenario set ("Agentic Audit: Explanatory Scenarios", D01–D10) but no agentic run to look at since the synthetic preloaded demo was removed in 4de31f1. This seeds a real one.

• Fixture infra/fixtures/agentic_demo_run.json: a real, executed agentic audit of the demo "Support Refund Assistant" (Acme policy KB + order lookup tool) against D01–D10, run through Studio's normal agent path with Studio tracing — 10 results with their agentic evaluations, and 213 OTLP spans exported by Open WebUI. Captured with capture_agentic_demo_fixture --run N --require-recorded. Agentic checks: 10/10 PASS; text judge: 10/10 pass.
• Neutral labels: it was recorded on a model new workspaces don't have, so the auditor/judge/agent-base-model snapshots read "Demo model (recorded)" with no endpoint, and the model id in span metadata is demo-model. Model output is untouched; recorded_run.models_relabelled records the relabel.
• Loader seed_agentic_demo (restored, recorded fixtures only): attaches the run to the workspace's demo agent and agentic set; auditor/judge rows point at the seeded OpenAI models so the run can be cloned. Doesn't need chat or a synced agent. seed_platform runs it with the demo runs; a deleted run isn't loaded again; --force replaces it.
• Capture: --relabel-models LABEL and --replace OLD=NEW (demo_fixture.relabel_models).
• Sanitizer fix: it redacted every key containing "token", so token counts (max_tokens, input_tokens, …) and tokenizer_model became [REDACTED]. Now only token fields are (token, access_token, x-client-token).
• Docs: docs/deployment.md commands table.

Testing

• infra/tests/test_seed_agentic_demo.py (8): committed fixture validates (format, recorded provenance, checksum) and carries no endpoint; loads with results, spans, agent and set; neutral snapshots; idempotent; deleted stays deleted; --force; skipped without the agent; a loaded run can't be re-exported as a recording; seed_platform loads it; run and result pages render with trace links.
• audits/agentic/tests/test_demo_fixture.py: sanitizer token cases, relabel_models.
• Full suite: 1852 passed. ruff check . clean.
• Fresh setup_local with chat off: 5 demo runs + "Demo: Agentic audit: Support Refund Assistant (D01-D10)" (10 scenarios).

Notes

• The fixture is 2.7 MB: the exporter stores each scenario's spans in the raw spans, trace evidence and judge evidence, and run detail reads those copies.
• The existing acme_agentic_real_run_v1.json (old A-series scenarios) is left as is; verify_agentic_audit still checks it.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.