Prove It is an open-source adversarial verification Agent Skill for Claude Code, OpenAI Codex, Cursor, and other AI coding agents.
English · 简体中文 · Português do Brasil · 日本語 · Español · Русский
Coding agents are excellent at finding evidence that their work succeeded.
Tests passed. Build exited 0. Healthcheck returned 200. No ERROR lines found.
Then they say done.
But a passing signal is not automatically a passing outcome. Prove It makes your coding agent actively search for evidence that its conclusion is wrong before it is allowed to trust it.
Use it to challenge bug fixes, tests, pull requests, CI results, logs, deployments, and any confident “done” claim before you trust it.
Don't try to prove it works. Try to prove it doesn't.
You need it if you let an AI coding agent:
| Your agent… | Prove It asks… |
|---|---|
| fixes bugs | Can the original bug still happen? |
| writes or updates tests | Could these tests pass while the requirement is broken? |
| reviews pull requests | Were assertions, types, or acceptance criteria weakened? |
| reads CI or logs | Is this evidence complete, current, and from the right place? |
| checks deployments | Is the intended version actually running everywhere? |
| tells you the task is done | What observable fact would make that conclusion false? |
This is for the moment between “the AI finished” and “you believe it.” You do not need to be a testing expert. Invoke one skill and give it the claim you want challenged.
Prove It is listed on skills.sh. The installer detects Claude Code, Codex, Cursor, and other supported agents:
npx skills add Pablo-aps/prove-itThen ask it directly:
# Claude Code / Cursor
/prove-it review your last implementation and try to find a case where it still fails
# OpenAI Codex
$prove-it does this CI output actually prove the issue is resolved?
The skill can also activate automatically when you ask an agent to prove, verify, validate, confirm, or double-check a claim.
Manual install
Copy SKILL.md into a prove-it skill directory for your agent:
| Agent | Project location | Invoke |
|---|---|---|
| Claude Code | .claude/skills/prove-it/SKILL.md |
/prove-it |
| OpenAI Codex | .agents/skills/prove-it/SKILL.md |
$prove-it |
| Cursor | .cursor/skills/prove-it/SKILL.md or .agents/skills/prove-it/SKILL.md |
/prove-it |
| 01 — DEFINE | 02 — BREAK |
|---|---|
| Convert “looks fixed” into a claim that can be false. Freeze the scope and acceptance criteria. | Ask what would contradict the claim. Execute the most decisive safe checks first. |
| 03 — VERIFY | 04 — VERDICT |
|---|---|
| Verify the real outcome, not a proxy such as an exit code, HTTP 200, or green healthcheck. | Return PROVEN, FAILED, NOT PROVEN, or BLOCKED with the evidence. |
CLAIM
All production replicas run abc123 and the migration is applied.
VERDICT
NOT PROVEN
WHY
The deploy job passed, but only four of six replicas are represented.
FALSIFICATION ATTEMPTS
- checked rollback events -> none found
- compared runtime image SHAs -> evidence exists for 4/6 replicas
- checked migration version -> no production schema evidence supplied
NEXT PROOF
1. read the image SHA from every replica
2. read the production migration version through an approved read-only path
| Verdict | Use it when |
|---|---|
PROVEN |
Direct evidence covers the defined scope and meaningful attacks found no contradiction. |
FAILED |
Direct evidence contradicts the claim. |
NOT PROVEN |
Available checks ran, but the evidence is indirect, incomplete, stale, or too narrow. |
BLOCKED |
A required check cannot run because access, data, credentials, tools, or testability is unavailable. |
PROVEN is deliberately scoped. Prove It does not claim mathematical proof, universal correctness, or future safety.
tests green ≠ original bug fixed
build green ≠ deployment works
deploy succeeded ≠ every replica runs the new version
healthcheck 200 ≠ workers and dependencies are healthy
HTTP 200 ≠ asynchronous operation completed
no errors in these logs ≠ no relevant error occurred
one successful request ≠ race condition fixed
agent confidence ≠ evidence
It also watches for counterfeit proof: skipped tests, weakened assertions, ignored exit codes, empty catches, hardcoded results, behavior mocked away, and timeouts increased without reproducing the timing failure.
These patterns are not automatically wrong. They become evidence against a claim when they hide, bypass, or redefine the behavior being verified.
See EXAMPLES.md for deployment, logs, async work, payment races, positive proof, and blocked verification.
Vibe verification is when an agent sees enough positive signals to feel confident and stops searching for ways the conclusion could be false.
Vibe verification: “Tests pass. Looks good.”
Prove It: “What would let these tests pass while the bug still exists?”
That shift is small, memorable, and useful across stacks because it changes the verification objective—not the test framework.
The repository includes a reproducible 12-case benchmark with:
- misleading success signals that should end in
FAILED; - incomplete evidence that should end in
NOT PROVEN; - inaccessible evidence that should end in
BLOCKED; - positive controls that should end in
PROVEN.
One directional run with Codex CLI 0.147.0 · gpt-5.6-luna · low reasoning · 2026-08-18:
| Baseline | With Prove It | |
|---|---|---|
| Correct verdict | 9/12 (75%) | 12/12 (100%) |
| Positive controls | 2/3 (67%) | 3/3 (100%) |
| False assurance | 0/12 | 0/12 |
| Attempted falsification | 12/12 | 12/12 |
Prove It changed three decisions in this run: two explicitly unavailable proof paths were correctly labeled BLOCKED, and one fully covered, scoped deployment claim was accepted as PROVEN instead of remaining indefinitely unproven.
Every case uses the same model, prompt artifact, output schema, and reasoning setting. The treatment changes one thing: it installs and explicitly invokes Prove It. Read the 24 raw outputs and full benchmark/ methodology.
This is one run per cell on cases authored while developing the skill. It is evidence that the skill changed this model's behavior on this fixture set—not an independent study, a model ranking, or a universal performance claim.
The design is also grounded in a broader finding: language models can favor confirming evidence, and counterexample-oriented prompting can reduce that bias. See Failing to Falsify, Large Language Models Cannot Self-Correct Reasoning Yet, and Anthropic's work on sycophancy in language models. These sources motivate the mechanism; they do not prove this skill works for every model or task.
Prove It is a small open-source Agent Skill that makes an AI coding agent define a falsifiable claim, search for counterevidence, verify the real outcome, and return an evidence-backed verdict.
The skill follows the open Agent Skills specification and works with Claude Code, OpenAI Codex, Cursor, and other compatible coding agents. The repository keeps the behavior in one portable SKILL.md.
Tests and review produce evidence. Prove It changes how the agent challenges and interprets that evidence. It does not replace either one.
No. It returns a scoped verdict from the checks and evidence that are actually available. It never claims formal correctness, universal safety, or certainty about unobserved behavior.
Prove It is intentionally one readable SKILL.md. It has no runtime dependencies, hooks, background process, telemetry, MCP server, or orchestration layer.
It is not:
- a testing framework or security scanner;
- a replacement for domain-specific tests, observability, or human review;
- permission to run destructive experiments against production;
- a claim that an AI can prove arbitrary software correct.
It is a behavioral guardrail: define the claim, attack it, verify the outcome, and earn the verdict.
prove-it/
├── README.md # understand it in two minutes
├── README.*.md # five localized guides
├── EXAMPLES.md # six concrete uses
├── skills/prove-it # the entire behavior
├── benchmark/ # reproducible evidence
└── LICENSE
The skill follows the open Agent Skills specification. Contributions are welcome when they make the core behavior sharper without turning the project into a framework.
AI writes the fix. Prove It tries to break it.
Created and maintained by PABLO.