feat(eval): adversarial safety packs + pluggable evaluator interface - #63
Merged
Conversation
added 14 commits
August 3, 2026 05:54
- EvalSuites/EvalCases tables — org-scoped, RETAIN + deletion protection + PITR (release evidence posture), org and agent-target GSIs - lifecycle DRAFT→FROZEN→ARCHIVED with immutability once frozen or referenced — fast-check acceptance property (mutation → rejected, zero writes) proven to bite via guard-neutralized mutant - EvalCase expresses every dimension incl. expected governance outcomes (PERMIT/DENY/ESCALATE + finding types), tool constraints, grounding, latency/cost budgets - eval-resolver: 13 GraphQL ops behind operations dispatch with eval:author/approve/read roles + optimistic locking - replay-package import per the CIT-026 contract (schemaVersion major pin, partial guards, DRAFT-only, execution-kind input limitation documented) - deterministic seed suites for intake agent + monolithic_db template, ≥1 expected-DENY case each - CDK posture assertions added (rail baselines cannot guard new tables) with real-file mutant bites Story: CIT-101 | Epic: E10 | Task: cffa1453
- every eval case runs as a REAL execution/conversation through the governed dispatch paths — no parallel eval engine - replay packages reused unchanged as per-case artifacts (SSM-resolved bucket, eval-runs/ prefix, lifecycle rule scoped to ORG# — was bucket-wide and would have expired evidence) - idempotent runs (uuidv5 per suite+agent+key, property-tested) with SQS fan-out + DLQ under drift-guard-forced alarm coverage - forbiddenTools union into DENIED_TOOLS — 200-example property proves the real tool handler is never reached, DENY finding stamped eval_run_id is the observable artifact (block-only until record-and-block, CIT-031) - cost rows tagged evalRunId/evalContext, excluded from org rollups + budgets (pinned both ways), still queryable per run - ledger serializes evalRunId only-when-present — non-eval traffic byte-identical (property-tested) - governance.eval.run.started/completed events (catalogue 15→17) - cross-language contract byte-exact (evalRunId/evalContext/forbiddenTools ↔ eval_run_id) Story: CIT-102 | Epic: E10 | Task: bafc3ffa | Decision: d36fbbf7
Deterministic per-case scoring utilities with property-tested aggregation; eval-case-scorer and eval-run-aggregator lambdas; arbiter eval_judge module (LLM judge, mocked-proven to never write DynamoDB directly); telemetry-stack wiring, GraphQL schema and type additions; EventBridge catalog updated.
…dings - Trajectory assertions derived from run artifacts - Per-agent production sampling with redaction before LLM judge - EMF drift metrics with governance finding on threshold breach - Eval-context usage and cost tagging in the cost ledger
…y tests First-call warmup exceeded the 200ms default deadline, causing flaky failures.
…on verdict - Pure candidate-vs-baseline diff classifying each case as improved, regressed, or unstable - Unstable repeats are flagged explicitly and never averaged into a score - Per-dimension material-regression verdict using layered, configurable thresholds that reuse drift threshold semantics - No composite score is produced or persisted Persistence and GraphQL resolver slice lands separately.
- Add baseline designation per agent+suite, with re-designation support and org-scoped, role-gated mutations - Add comparison query surfacing per-case deltas and dimension-level aggregates against the designated baseline - Enforce idempotent comparison computation (deterministic comparison id) and strict cross-org isolation on all reads/writes - Emit governance.eval.baseline.designated and governance.eval.comparison.completed events for audit and future promotion-gate consumption
- Per-dimension aggregates surfaced with a material-regression verdict - Expandable per-case diff distinguishing improved/regressed/unstable/incomparable - Honest no-baseline / nothing-to-compare / incomparable / unauthorized states - Accessible icon+text status encoding rather than colour alone Raw transcript and trajectory artifact diffing is not included because the comparison GraphQL surface returns classifications and values only.
- Bounded projection with explicit truncation and cursor, never silent - Reuses the existing replay sanitisation and secret-free assertion, fail-closed when sanitisation cannot be proven - Org scoping asserted on run, case and envelope rows - Seven distinguishable honest states rather than empty defaults
… by side - Side-by-side transcript and ordered trajectory diff per case - Every per-side artifact state rendered distinguishably - Truncation shown explicitly with cursor paging - Accessible keyboard and non-colour-only encoding
- Assert row org on the by-id read paths using the caller's org claim - Add an org predicate and row assertion to the list queries - Note the same convention elsewhere in the codebase is left for a separate reviewed change
…y, DENY, delegation Add five built-in evaluation suites covering prompt injection (including tool-schema poisoning), unauthorised tool use, data-leakage canary probes, policy-bypass attempts that must produce DENY findings, and unsafe delegation beyond authority units. Packs are versioned like any suite (seed version bumped) and carry a gate-class marker so a future release gate can require them for security-sensitive agents. Scoring for the injection corpus is a pure, AWS-free path: a sanitizer weakened to an identity function fails the pack.
…s, strict score validation Add an evaluator contract (score(case, artifacts) -> dimension scores) with a registry following the agent-source registry-factory precedent. Built-in scoring is adapted to the contract without changing its logic, so custom dimensions compose alongside the canonical dimension order and aggregation. Org-registered external evaluators are invoked over the existing Lambda/HTTP integration machinery with secret-backed auth, and their responses are treated as untrusted: sanitized, then totally validated — reserved or undeclared dimension names, non-finite or out-of-range scores, missing fields, duplicates, and oversized payloads are rejected and never reach a persisted score vector.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
added 3 commits
August 6, 2026 05:08
…lper Replace require() calls in the comparison-resolver and eval-usage tests with the import style the rest of the suite uses, and remove an unused threshold type alias. Also delete an approve-permission helper that had no call sites: both approve-class mutations already enforce the permission inline with audit-before-auth logging, and passing tests prove an unauthorised caller is rejected — so the helper was dead code, not a missing check. The remaining unused rest-destructure binding is renamed rather than removed to keep the destructure shape intact.
Remove an unused pytest import from the judge tests and an unused MagicMock import from the forbidden-tools-union tests. Both were confirmed to have no remaining references, including attribute, decorator, and side-effect uses, and each file still collects and passes on its own.
The seed writer guarded its put on the seed version alone, so bumping that version overwrote a frozen or referenced suite back to draft with its reference list cleared. That bypassed the mutability guard enforced in the resolver and, by clearing the references, erased the very evidence the guard keys off — leaving the suite silently mutable again. The put now additionally requires the row to be draft and unreferenced, so a protected suite is not written at all. Brand-new suites are still created and stale draft unreferenced rows still heal. A blocked heal is no longer silent: the conditional failure is classified by reading the row, and only the blocked case emits a governance notification, at most once per suite per run — an already-current row on redeploy stays quiet. Archived rows are deliberately treated as protected, since archived is a terminal state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
feat(eval): adversarial safety packs + pluggable evaluator interface
Completes the evaluation epic's remaining two stories.
Adversarial & safety packs
Five built-in evaluation suites: prompt injection (including tool-schema poisoning), unauthorised tool use, data-leakage canary probes, policy-bypass attempts that must produce DENY findings, and unsafe delegation beyond authority units. Seed version bumped; suites carry a gate-class marker so a future release gate can require them for security-sensitive agents — the marker only, no gating machinery here.
Injection scoring runs through a pure path (injected sanitizers + the existing
scoreCase), so the "a weakened sanitizer must fail the pack" criterion is provable with no AWS calls.Pluggable evaluator interface
An
Evaluatorcontract (score(case, artifacts) → dimension scores) plus a registry, following the agent-source registry-factory precedent. Built-in scoring is adapted to the contract with its logic untouched, so custom dimensions compose alongside the unmodified canonical dimension order and aggregation. All 13 files are new — zero existing files modified, which is how the "contributes a dimension without core changes" criterion is proven.External org-registered evaluators are invoked over the existing Lambda/HTTP integration machinery with secret-backed auth, and their responses are treated as untrusted: sanitized, then totally validated. Review caught a real gap here worth calling out — an evaluator could return a well-formed score for a dimension it never declared, letting a compromised endpoint inject fabricated, operator-unapproved dimensions into persisted run reports. Now allowlisted against the evaluator's declared set at the adapter boundary.
Verification
backend
tscclean; backend jest 6041 passed / 7 skipped; frontend jest 2041 passed; arbiter pytest 1407 passed / 1 skipped. Re-run after mergingmainin, so these hold on the merge result, not just the branch head.Reviewer attention: deploy-time behaviour
Merging deploys to dev, and the first deploy at the new seed version replaces every seed-owned suite row wholesale — status forced back to
DRAFT, version and timestamps reset. User-authored suites are untouched. In dev the blast radius is small, but the underlying issue is a design flaw rather than a migration note: the seed healer's condition only compares seed versions and never checks whether a suite has been release-referenced or approved, so it bypasses the "immutable once referenced by a release" invariant that release gating is designed to depend on. Tracked separately; worth deciding whether to fix it before this pattern reaches staging or prod.Scope boundaries
@smithy/node-http-handlerteardown race. Reproduces atmain; the full suite exits 0.