Skip to content

Publish AV-EXP-001 real-repository shadow benchmark - #2

Merged
sneakocom merged 5 commits into
mainfrom
av-exp-001-shadow-benchmark
Aug 31, 2026
Merged

sneakocom merged 5 commits into
mainfrom
av-exp-001-shadow-benchmark

Conversation

@sneakocom

Copy link
Copy Markdown
Contributor

Summary

  • preserves the AV-EXP-001 preregistration before comparative outcomes
  • benchmarks FULL, native Vitest related, AV_CORE, and AV_WITH_NATIVE_EVIDENCE on pinned Zustand
  • publishes the complete full-oracle evidence, skip reasons, Visible Value receipts, and deterministic replay
  • remains SHADOW; no CI or trust gate is replaced

Result

AV selected 8/8 oracle-relevant checks with no observed miss in the frozen ten-scenario corpus. The native tests-only arm selected 6/8 and missed one relevant lint check and one relevant typecheck. This is bounded calibration evidence, not general safety.

Verification

  • npm run verify
  • node benchmark/av-exp-001/verify-results.mjs
  • 20 Visible Value receipts validated by the authoritative research validator
  • actionlint
  • gitleaks git --no-banner --redact .
  • same-host clean replay matched baseline, all scenario, summary, and analysis semantic identities

@sneakocom
sneakocom merged commit 641aee9 into main Aug 31, 2026
2 checks passed
@sneakocom
sneakocom deleted the av-exp-001-shadow-benchmark branch August 31, 2026 02:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant