Dogfood AGT with ACS and file a conformance report

[rocklambros]

Run the Microsoft Agent Governance Toolkit against an ACS Guardian and file a conformance report for every place behavior and specification disagree. Conformance evidence is what makes the benchmark credible.

[antman1p]

Dogfooded #92 with Grok Build TUI as the harness. Findings are in #107.

[avp9-nexus]

Dogfooded the host side of #92 with Claude Code 2.1.260 as the harness, no AGT and no Guardian in the loop: two hooks on one PreToolUse event. The shim's deny is enforced, but its reason is dropped when a sibling hook exits 2. Report in #109.

[carloshvp]

I opened #113 with a dogfooding report, evidence captures, and a reproduction fixture for OpenCode 1.18.30 against the ACS reference Guardian and pinned AGT policy engine.

The runs exercise OpenCode's real tool loop and ACS plugin, with a local scripted model supplying predetermined tool calls. They cover live ALLOW/DENY decisions and Guardian loss within an established session under both proceed and deny postures. File markers, host outcomes, wire responses, and failure-audit records agree with the expected behavior in those cases.

No new specification disagreement appeared in this slice. The PR does not claim full ACS-Core conformance and leaves #92 open for further dogfooding.

[AngeYobo]

Taking a slice, scoped against what's landed.

#107, #109, #111, #113 all exercise the host side. #113 covers Guardian loss
within an established session, both postures.

What's untouched is the decision/execution boundary itself:

• arguments mutated after ALLOW, before the call reaches the tool
• approved invocation replayed
• indeterminate outcome: Guardian answered, effect state unknown (distinct from
  Record OpenCode host decision-honoring evidence for #92 #113, where the decision never arrives)

Same reporting shape as #113: pinned versions, reproduction fixture, evidence
captures, untested scenarios listed explicitly. Environment pinned in the
report, not inherited from CI (#110).

@carloshvp adjacent to what you're continuing with? I'd rather extend your
fixture than fork one.

[carloshvp]

@AngeYobo Yes, extending the fixture makes sense. Your proposed decision/execution cases are adjacent to the work in #113, and I am not currently working on those cases. My current focus is the approver-authentication and timeout-attribution proposal in #51, awaiting scope acceptance.

One status clarification: #113 is still open and unmerged. The fixture is on carloshvp:agt-host-conformance-report, currently at ea1e3459c7a43d354bfd0f84391dee044b7a581f. You are welcome to build on it; please identify that dependency and the tested commit in your report so the results remain reproducible while review is pending.

The same reporting structure should work well: distinguish the approved request, the invocation that reaches the tool, and the observed effect, with an explicit unknown outcome when the evidence cannot establish completion. For replay, it would help to distinguish a repeated ACS envelope from executing an already-approved invocation again; the reference implementation already documents its missing wire replay protection.

Happy to reuse the fixture this way. Acceptance of the additional scope remains with the maintainers.