🛡️ ProofGate

An evidence-first security gate for CI/CD. ProofGate runs your code, dependency, container, infrastructure, cloud, and AI-logic checks on every change, normalizes them into one format, and then does the thing nothing else does: it refuses to block a build on any finding it cannot prove. Every AI-discovered bug is verified line-by-line against the real code so it physically cannot hallucinate past the gate; every finding — scanner or AI — must survive an adversarial challenge; problems are linked across layers into attack chains; and every keep/drop decision is written to a ledger, so the tool's precision is a measured number, not a marketing claim.

Built with Python · FastAPI · GitHub Actions · Docker, plus a pluggable LLM backend. The analysis engine is pure standard library — the demo runs with no install and no API keys.

---

Why this exists (and why it's defensible)

Two things are true in security engineering right now:

1. Teams already have scanners (SAST, dependency, container, IaC, cloud) — and they drown in the output: five tools, five formats, thousands of "HIGH/MEDIUM" lines, most of them false positives nobody triages.
2. Everyone is bolting AI onto security tools — and nobody trusts the AI in the merge path, because it hallucinates: it will confidently cite a line of code that doesn't exist.

ProofGate is not another scanner and not another AI bot. It's the verification and trust layer that sits after everything else and makes the combined output safe to gate a build on. That's the defensible idea:

Treat every finding — from a scanner or an AI — as a claim to be proven, not a fact to be trusted. Only claims that survive proof are allowed to block a build.

That single principle is why ProofGate stays unique even in a market full of scanners and AI assistants:

The pain everyone has	Existing tools	ProofGate's answer
AI security tools hallucinate → can't gate merges	❌ so the AI stays advisory-only	✅ a mechanical citation check (no model) confirms every AI-cited line exists in the diff, or drops it
Scanners are noisy → alerts get ignored	tuned rules per tool	✅ an adversarial verification pass refutes what it can't defend, across all tools at once
"Port open in prod" — but which code caused it?	separate dashboards	✅ cross-layer correlation links live cloud → the IaC line → the code bug
Logic bugs (IDOR, refund abuse, is_admin) sail through	❌ scanners can't see intent	✅ an AI reviewer that must cite the line and write a working exploit
"How do I know it isn't just noise?"	🤷	✅ a trust ledger with a measured noise-reduction figure

The category is real — it's what commercial ASPM platforms (Snyk, Wiz, Apiiro) charge for. ProofGate's twist — proof-carrying findings — is the piece that makes an AI-in-the-loop gate trustworthy, which is the genuinely unsolved problem.

---

📖 In plain English (no security background needed)

Shipping software is like sending a dish out of a kitchen. Most tools are single-purpose inspectors — one tastes the recipe, one checks the ingredients, one looks at the packaging — and each hands you its own noisy report. None tells you whether the dish is actually safe to serve.

ProofGate is the head inspector: it runs all the specialists at once, rewrites their reports in one plain language, throws out any alarm that can't be defended, refuses to take the clever AI inspector's word without checking the evidence, connects the dots across the whole kitchen, ranks by real danger, and gives one answer — serve or don't serve — while recording why it kept or dropped every item.

---

🧱 The six layers it covers

Left (writing code) to right (running in the cloud):

#	Layer	Inspects	Tool	Example finding
1	CODE_LOGIC	intent & ownership bugs in a PR	AI reviewer	IDOR: any user reads any order by id
2	SAST	your source code	Bandit, Semgrep	SQL injection, eval() on user input
3	SCA	open-source dependencies	Trivy	a library version with a known CVE
4	CONTAINER	the Docker image	Trivy	image runs as root
5	IAC	your cloud blueprint	Checkov	security group open to 0.0.0.0/0
6	CLOUD	your live AWS account	Prowler	that same port, open in production

ProofGate reuses the best open-source scanners rather than reinventing them — the value is the proof, correlation, and gate layer on top.

---

🚀 Quick start — runs in 30 seconds, no keys, no tools

The engine is pure standard library and ships with realistic sample scanner outputs and a sample pull request:

# full gate across all six layers on the bundled targets
python3 -m proofgate.cli scan
#  → prints the decision + findings, writes reports/report.{html,md,json}

# or:
make demo && open reports/report.html

You'll see ~25 findings across all six layers, cross-layer attack chains (including a code bug linked to an over-permissive cloud IAM policy), a few findings refuted or floored as noise, and the build BLOCKED because critical issues exist.

pip install -r requirements.txt
make serve                     # dashboard → http://localhost:8000
docker compose up --build      # same, in Docker

Review your own code

git diff main...HEAD | python3 -m proofgate.cli scan --diff - --no-scanners
python3 -m proofgate.cli scan --fail-on CRITICAL --fail     # CI-style gate (exit 1 on block)

---

🏭 Deploying ProofGate in a real CI/CD pipeline

This is where ProofGate is meant to live. It sits after your scanners run and before a merge/deploy is allowed, as the single decision point.

 developer opens PR ─┐
                     ▼
   ┌──────────── CI runs in parallel ────────────┐
   │  Bandit  Semgrep  Trivy  Checkov  Prowler    │   ← scanners write JSON to reports/raw/
   └───────────────────┬──────────────────────────┘
                       ▼
                  ProofGate gate            ← collect → prove → correlate → score → gate
                  ├─ posts inline PR comments (code-logic findings)
                  ├─ uploads report.html / .json artifact
                  └─ exit 0 = allow merge · exit 1 = block merge

1. Where it plugs in

ProofGate consumes two inputs and needs neither at once:

• --raw-dir <dir> — a folder of <tool>.json scanner outputs (any subset; missing tools are simply skipped).
• --diff <file|-> — the pull-request diff, for the AI code reviewer.

So the pipeline pattern is: run your scanners → point ProofGate at their output → let its exit code gate the build.

2. GitHub Actions (included)

.github/workflows/proofgate.yml already implements the full pattern: independent scanner jobs (sast, sca-container, iac) each write to reports/raw/ and never fail the build on their own; a final gate job merges the artifacts and runs ProofGate as the single decision point. To adopt it in a real repo:

1. Copy proofgate/, integrations/, requirements.txt, and .github/workflows/proofgate.yml into your repository.
2. (Optional) Add an ANTHROPIC_API_KEY repository secret to enable the real LLM reviewer. Without it, ProofGate runs offline and never breaks CI.
3. Turn the gate from advisory to blocking by setting FAIL_ON_CRITICAL: "true" (or PROOFGATE_FAIL_BUILD: "true") in the workflow env, then add the gate job to your branch-protection required status checks (Settings → Branches → Branch protection).

The included gate job seeds bundled fixtures if a scanner is unavailable, so the pipeline is demonstrable even before you wire in live scanners.

3. GitLab CI (equivalent)

stages: [scan, gate]

sast:
  stage: scan
  script:
    - pip install bandit semgrep
    - mkdir -p reports/raw
    - bandit -r apps -f json -o reports/raw/bandit.json || true
    - semgrep --config auto . --json -o reports/raw/semgrep.json || true
  artifacts: { paths: [reports/raw] }

proofgate-gate:
  stage: gate
  script:
    - pip install -r requirements.txt || pip install requests
    - python -m proofgate.cli scan --raw-dir reports/raw --fail-on CRITICAL --fail
  artifacts: { when: always, paths: [reports/report.html, reports/report.json] }

4. Jenkins / any runner (generic)

stage('Security Gate') {
  steps {
    sh 'python3 -m proofgate.cli scan --raw-dir reports/raw --diff pr.diff --fail-on CRITICAL --fail'
  }
  post { always { archiveArtifacts artifacts: 'reports/report.*' } }
}

The contract is just the exit code: 0 = allow, 1 = block. Everything else (PR comments, artifacts) is optional decoration.

5. Long-lived webhook service (instead of polling CI)

Run the FastAPI app (uvicorn server.app:app) and point a GitHub webhook at POST /webhook. It verifies the HMAC signature (GITHUB_WEBHOOK_SECRET), reviews each PR as it opens, and posts inline comments. Good for orgs that want one central service instead of per-repo CI steps.

6. Rolling it out safely (the part reviewers ask about)

1. Advisory mode first. Deploy with the gate non-blocking (--fail off). It comments and reports but never blocks. Watch the ledger's noise-reduction figure for a week.
2. Tune the floors. Raise PROOFGATE_MIN_SEVERITY / PROOFGATE_MIN_CONFIDENCE / PROOFGATE_MAX_RISK until the retained set is signal, not noise.
3. Flip to blocking on CRITICAL only. --fail-on CRITICAL. Widen to HIGH once the team trusts it.
4. Split the cadence. Run the diff-scoped AI review + fast scanners on every PR (seconds), and the full cloud/live scan (Prowler) on a schedule (nightly), because live-cloud scanning needs real credentials and is slower.

7. Production concerns, addressed

Concern	How ProofGate handles it
A scanner crashes	Every parser is defensive — a bad/missing file yields no findings, not a failed gate
Secrets in CI	API keys via CI secrets/env only; the webhook verifies an HMAC signature; nothing is logged
Cost / latency	Offline mode is free and instant; the LLM runs only on the diff, not the whole repo; effort is tunable
Auditability	The ledger records every keep/drop with a reason — exportable as JSON for compliance
False positives	Layered defense: citation check → verification → confidence floor → severity floor → gate
Native GitHub UI	report.json is emitted today; SARIF export for the GitHub Security tab is the next step (see roadmap)

---

🔌 Turning on the real model

Everything above runs on a deterministic offline backend. Set one variable and the same pipeline calls a real model for detection, verification, and triage:

cp .env.example .env
export ANTHROPIC_API_KEY=...        # or PROOFGATE_LLM_API_KEY
python3 -m proofgate.cli scan       # engine mode flips "offline" → "llm"

The LLM layer is provider-pluggable: the engine talks only to an abstract LLMBackend, and the reference backend uses structured (schema-validated) outputs, adaptive thinking, and a configurable effort level. Swapping providers is one new file, not a pipeline change.

---

⚙️ How it works — the pipeline

  scanner JSON  +  PR diff
        │
  1. COLLECT     every source → one unified Finding                 sources/*.py
  2. CITE-CHECK  AI citations must exist in the diff, or drop        citations.py  ← can't hallucinate
  3. VERIFY      a panel of skeptics tries to REFUTE each finding    verify.py     ← cuts noise
  4. CORRELATE   link findings across layers into attack chains      correlate.py
  5. SCORE       0–100 real-world risk (internet-facing? admin?)     risk.py
  6. TRIAGE      plain-English explanation + concrete fix            triage.py
  7. REPORT      report.html / .md / .json                           report.py
  8. GATE        block the build if anything provable is bad         report.py:gate()
  9. LEDGER      record every keep/drop → provable precision         ledger.py

Cheap, certain filters (citation check, verification) run first, so ProofGate never wastes effort ranking a finding it's about to discard.

---

🗂️ Project structure

ProofGate/
├── proofgate/                  # ★ the engine (pure standard library)
│   ├── models.py               #   the unified Finding schema
│   ├── sources/                #   scanner adapters + the AI reviewer
│   ├── llm/                    #   pluggable backends (offline + Anthropic)
│   ├── citations.py            #   ★ mechanical anti-hallucination guard
│   ├── verify.py               #   ★ adversarial verification + gates
│   ├── correlate.py risk.py triage.py    #   chains · risk · plain-English
│   ├── ledger.py               #   ★ trust ledger (provable precision)
│   ├── report.py pipeline.py cli.py      #   report+gate · orchestration · CLI
├── server/                     # FastAPI dashboard + JSON API + GitHub webhook
├── integrations/               # GitHub REST client + Actions entrypoint
├── .github/workflows/          # the CI pipeline (scanners → one gate)
├── apps/  infra/  samples/     # INTENTIONALLY VULNERABLE scan targets
├── fixtures/                   # sample scanner outputs + sample PR (demo data)
└── tests/                      # 22 tests, fully offline

make install-dev && make test    # 22 tests, no keys required

---

🎛️ Configuration

All optional; every value has a safe default. See .env.example.

Variable	Default	Purpose
ANTHROPIC_API_KEY / PROOFGATE_LLM_API_KEY	(unset → offline)	enable the real model
PROOFGATE_MODEL	claude-opus-4-8	model id
PROOFGATE_EFFORT	high	reasoning effort
PROOFGATE_VERIFY / PROOFGATE_VERIFY_VOTES	true / 3	adversarial pass + skeptics per finding
PROOFGATE_MIN_CONFIDENCE	0.7	drop code-review findings below this
PROOFGATE_MIN_SEVERITY	MEDIUM	severity floor for reporting
PROOFGATE_FAIL_ON / PROOFGATE_MAX_RISK	HIGH / 75	what blocks the build
GITHUB_TOKEN / GITHUB_WEBHOOK_SECRET	(unset)	post PR comments / verify webhooks

---

⚠️ Honest limitations

• The AI reviewer reviews the diff, not the whole repo — a flaw that only emerges from unchanged code far away can be missed.
• Correlation joins on shared concepts, not exact resource identities, so it can occasionally over-link; ARN/tag matching is the next iteration.
• Offline mode is a deterministic stand-in for demos, not real analysis.
• It's a second pair of eyes that complements — not replaces — scanners and human review.
• The apps/, infra/, and samples/ targets are intentionally insecure teaching material. Never deploy them.

---

🧭 Roadmap

Resource-identity correlation (exact ARN/tag matching) · SARIF export → native GitHub Security tab · persistent ledger with a precision trend line · auto-generated fix PRs · a suppression/allowlist workflow with expiry for accepted risks.

---

🧰 Tech stack

Python 3.10+ · standard-library engine · FastAPI (dashboard + API) · GitHub REST API & Actions · Docker / Compose · pytest · pluggable LLM backend (reference implementation on the Anthropic Messages API: structured outputs, adaptive thinking, effort).