Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions .github/workflows/pcp-comparator.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Judge the three headline PCP statements with Comparator, at the authors' toolchain.
# The challenge and solution live in papers/pcp-full-argument/comparator/ and are copied
# into the Lean project for the run; the paper's copy of the authors' Lean is not changed
# in the repository. papers/pcp-full-argument/source-review.md records the run that is cited.
name: Judge the PCP headline statements with Comparator
on:
workflow_dispatch:
permissions:
contents: read
jobs:
comparator:
runs-on: ubuntu-24.04
timeout-minutes: 180
defaults:
run:
working-directory: papers/pcp-full-argument/lean
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1
- name: Install elan and the project's Lean toolchain
run: |
curl -sSfL https://raw.githubusercontent.com/leanprover/elan/master/elan-init.sh | sh -s -- -y --default-toolchain none
echo "$HOME/.elan/bin" >> "$GITHUB_PATH"
- name: Restore the earlier build of this project
uses: actions/cache/restore@v4
with:
path: papers/pcp-full-argument/lean/.lake
key: pcp-lake-${{ hashFiles('papers/pcp-full-argument/lean/lake-manifest.json', 'papers/pcp-full-argument/lean/lean-toolchain') }}-comparator-${{ github.run_id }}
restore-keys: pcp-lake-${{ hashFiles('papers/pcp-full-argument/lean/lake-manifest.json', 'papers/pcp-full-argument/lean/lean-toolchain') }}-
- name: Fetch Mathlib's compiled files
run: lake exe cache get
- name: Put the challenge and solution into the project for this run
run: |
mkdir -p PCP/Checked/Negative
cp ../comparator/Challenge.lean ../comparator/Solution.lean ../comparator/comparator.json PCP/Checked/
cp ../comparator/negative/Challenge.lean PCP/Checked/Negative/Challenge.lean
cp ../comparator/negative/comparator.json PCP/Checked/Negative/comparator.json
lake --version && lake comparator --help | head -8
- name: Compile the authors' proofs, the challenge and the control
run: |
set -o pipefail
lake build PCP.Checked.Solution PCP.Checked.Challenge PCP.Checked.Negative.Challenge 2>&1 | tee "$RUNNER_TEMP/build.log" | { grep -v '^✔' || true; }
- name: Install the sandbox Comparator uses
working-directory: .
run: |
sudo apt-get update -qq && sudo apt-get install -y -qq bubblewrap
sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0 || true
bwrap --version
- name: Comparator on the three statements and on the control
run: |
mkdir -p "$RUNNER_TEMP/out"
{ lake --version; cat lean-toolchain; git -C "$GITHUB_WORKSPACE" rev-parse HEAD; bwrap --version; sha256sum ../comparator/Challenge.lean ../comparator/Solution.lean ../comparator/comparator.json ../comparator/negative/Challenge.lean; } > "$RUNNER_TEMP/out/context.txt"
cp "$RUNNER_TEMP/build.log" "$RUNNER_TEMP/out/"
run() { # name, config, extra flags
echo "== $1"
local start=$(date +%s)
if timeout 100m lake comparator --config "$2" $3 > "$RUNNER_TEMP/out/$1.log" 2>&1; then echo 0 > "$RUNNER_TEMP/out/$1.exit"; else echo $? > "$RUNNER_TEMP/out/$1.exit"; fi
tail -20 "$RUNNER_TEMP/out/$1.log"; echo "exit $(cat "$RUNNER_TEMP/out/$1.exit") after $(( $(date +%s) - start ))s"
}
run control.sandboxed PCP/Checked/Negative/comparator.json ""
run headline.sandboxed PCP/Checked/comparator.json ""
if [ "$(cat "$RUNNER_TEMP/out/headline.sandboxed.exit")" != 0 ]; then
run control.unsandboxed PCP/Checked/Negative/comparator.json "--inadvisably-no-sandbox"
run headline.unsandboxed PCP/Checked/comparator.json "--inadvisably-no-sandbox"
fi
- uses: actions/upload-artifact@v4
if: always()
with:
name: pcp-comparator
path: ${{ runner.temp }}/out
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ Evidence differs by page. Some declaration data comes from Lean's report on comp
For the three PCP companions, Lean compiled the listed modules of the authors' project in a
GitHub Actions run, and every declaration's dependencies and axiom report is Lean's output from
that run. The original guide's page keeps its earlier extraction, with a statement-only
compilation of the headline theorem. Comparisons of statements with their Lean are held. The [source review](papers/pcp-full-argument/source-review.md)
compilation of the headline theorem. Comparator accepted the authors' proofs of the three headline statements in a second run.
Sentence-by-sentence comparisons of statements with their Lean have not been written. The [source review](papers/pcp-full-argument/source-review.md)
records the source pin and hashes. Neither a source link nor a correspondence check establishes
that the prose and Lean have the same meaning.

Expand Down
2 changes: 1 addition & 1 deletion papers/pcp-exposition/links.json
Original file line number Diff line number Diff line change
Expand Up @@ -613,5 +613,5 @@
"PCP.Reduction.FiniteBooleanConstraints.flagLiteral": "Literal for a prefix flag",
"PCP.Reduction.FiniteBooleanConstraints.differenceLiteral": "Literal for a pattern mismatch"
},
"fidelity_held": "The selected source has not yet been compared against elaborated Lean statements for this page."
"fidelity_held": "Sentence-by-sentence comparisons of this page's statements with their Lean have not been written yet. Comparator has judged the three headline statements; papers/pcp-full-argument/source-review.md records that run."
}
2 changes: 1 addition & 1 deletion papers/pcp-exposition/paper.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ source = "https://github.com/troyjlee/tcs-formalizations/tree/a0ce13ed0fa264f051
source_name = "tcs-formalizations at a0ce13e"
source_lines = "https://github.com/troyjlee/tcs-formalizations/blob/a0ce13ed0fa264f051c1278e039596ce65557fa4/{path}#L{line}-L{end}"
licence = "Apache-2.0"
scope = "Lean 4.35.0-rc2 compiled the listed modules of the authors' project, and each declaration's statement, dependencies and axioms shown here are Lean's own report from those compiled modules. The links from the paper's sentences to the Lean, the plain titles and the notes were made with AI and have not been reviewed by the authors."
scope = "Lean 4.35.0-rc2 compiled the listed modules of the authors' project, and each declaration's statement, dependencies and axioms shown here are Lean's own report from those compiled modules. Comparator, run in its sandbox, accepted the authors' proofs of the forward theorem, the converse and the equivalence with the reference NP class against statements copied from their source, and refused a control that changed the soundness bound. The links from the paper's sentences to the Lean, the plain titles and the notes were made with AI and have not been reviewed by the authors."

[reading]
language = "lean"
Expand Down
47 changes: 47 additions & 0 deletions papers/pcp-full-argument/comparator/Challenge.lean
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
import PCP.Complexity.NP
import PCP.Complexity.Machine
import Complexitylib.Classes.NP

/-!
# The three headline statements, with no proofs

Comparator reads this file as the challenge. It imports only the files that define the
words the statements use: the verifier, the machine model, this project's NP, and the
vendored ComplexityLib NP. It does not import any proof.

Each statement is copied from the authors' source, under the authors' own name:
`PCP/Theorem.lean` (lines 25 to 33), `PCP/ReverseConcrete.lean` and
`PCP/Complexity/ReferenceNPConverse.lean`.
-/

namespace PCP

theorem np_subset_pcp (L : Language) (hL : InNP Machine.PolyTimeRel L) :
∃ (V : Verifier) (alg : VerifierAlg) (a b q : ℕ) (p : Polynomial ℕ),
Machine.PolyTimeAlg alg ∧
V.ImplementedBy alg ∧
V.queries = q ∧
V.CanonicalRandomness a b ∧
(∀ x : List Bool, V.ReadsBelow x (p.eval x.length)) ∧
(∀ x : List Bool, L x → V.CompleteAt x) ∧
(∀ x : List Bool, ¬ L x → V.SoundAt x (1 / 2)) := by
sorry

theorem pcp_subset_np_concrete (L : Language) (V : Verifier) (alg : VerifierAlg) (a b : ℕ)
(p : Polynomial ℕ) (halg : Machine.PolyTimeAlg alg) (himpl : V.ImplementedBy alg)
(hrand : V.CanonicalRandomness a b)
(hlen : ∀ x : List Bool, V.ReadsBelow x (p.eval x.length))
(hcomplete : ∀ x : List Bool, L x → V.CompleteAt x)
(hsound : ∀ x : List Bool, ¬ L x → V.SoundAt x (1 / 2)) :
InNP Machine.PolyTimeRel L := by
sorry

end PCP

namespace PCP.ReferenceNP

theorem inNP_iff_mem {L : Complexity.Language} :
PCP.InNP Machine.PolyTimeRel L ↔ L ∈ Complexity.NP := by
sorry

end PCP.ReferenceNP
10 changes: 10 additions & 0 deletions papers/pcp-full-argument/comparator/Solution.lean
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import PCP.Theorem
import PCP.ReverseConcrete
import PCP.Complexity.ReferenceNPConverse

/-!
# The solution Comparator judges

This file adds nothing. It imports the authors' modules that prove the three statements,
so the declarations Comparator compares are the authors' own, under their own names.
-/
11 changes: 11 additions & 0 deletions papers/pcp-full-argument/comparator/comparator.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"challenge_module": "PCP.Checked.Challenge",
"solution_module": "PCP.Checked.Solution",
"theorem_names": [
"PCP.np_subset_pcp",
"PCP.pcp_subset_np_concrete",
"PCP.ReferenceNP.inNP_iff_mem"
],
"definition_names": [],
"permitted_axioms": ["propext", "Quot.sound", "Classical.choice"]
}
50 changes: 50 additions & 0 deletions papers/pcp-full-argument/comparator/negative/Challenge.lean
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
import PCP.Complexity.NP
import PCP.Complexity.Machine
import Complexitylib.Classes.NP

/-!
# A control that must be rejected

The forward theorem is restated with soundness 1/3 in place of 1/2, a statement the
authors did not prove. Comparator must refuse it.

Comparator reads this file as the challenge. It imports only the files that define the
words the statements use: the verifier, the machine model, this project's NP, and the
vendored ComplexityLib NP. It does not import any proof.

Each statement is copied from the authors' source, under the authors' own name:
`PCP/Theorem.lean` (lines 25 to 33), `PCP/ReverseConcrete.lean` and
`PCP/Complexity/ReferenceNPConverse.lean`.
-/

namespace PCP

theorem np_subset_pcp (L : Language) (hL : InNP Machine.PolyTimeRel L) :
∃ (V : Verifier) (alg : VerifierAlg) (a b q : ℕ) (p : Polynomial ℕ),
Machine.PolyTimeAlg alg ∧
V.ImplementedBy alg ∧
V.queries = q ∧
V.CanonicalRandomness a b ∧
(∀ x : List Bool, V.ReadsBelow x (p.eval x.length)) ∧
(∀ x : List Bool, L x → V.CompleteAt x) ∧
(∀ x : List Bool, ¬ L x → V.SoundAt x (1 / 3)) := by
sorry

theorem pcp_subset_np_concrete (L : Language) (V : Verifier) (alg : VerifierAlg) (a b : ℕ)
(p : Polynomial ℕ) (halg : Machine.PolyTimeAlg alg) (himpl : V.ImplementedBy alg)
(hrand : V.CanonicalRandomness a b)
(hlen : ∀ x : List Bool, V.ReadsBelow x (p.eval x.length))
(hcomplete : ∀ x : List Bool, L x → V.CompleteAt x)
(hsound : ∀ x : List Bool, ¬ L x → V.SoundAt x (1 / 2)) :
InNP Machine.PolyTimeRel L := by
sorry

end PCP

namespace PCP.ReferenceNP

theorem inNP_iff_mem {L : Complexity.Language} :
PCP.InNP Machine.PolyTimeRel L ↔ L ∈ Complexity.NP := by
sorry

end PCP.ReferenceNP
11 changes: 11 additions & 0 deletions papers/pcp-full-argument/comparator/negative/comparator.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"challenge_module": "PCP.Checked.Negative.Challenge",
"solution_module": "PCP.Checked.Solution",
"theorem_names": [
"PCP.np_subset_pcp",
"PCP.pcp_subset_np_concrete",
"PCP.ReferenceNP.inNP_iff_mem"
],
"definition_names": [],
"permitted_axioms": ["propext", "Quot.sound", "Classical.choice"]
}
49 changes: 49 additions & 0 deletions papers/pcp-full-argument/comparator/receipt.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"schema": 1,
"what": "Comparator judged the authors' proofs of three statements against a challenge file that imports only definitions.",
"run": "https://github.com/wcook04/proof-reader/actions/runs/43868208176",
"date": "2026-11-16",
"commit": "027be095d4715b0d9bbd243debf08b8d5ad6907c",
"lake": "Lake version 5.0.0-src+11acb17 (Lean version 4.35.0-rc2)",
"toolchain": "leanprover/lean4:v4.35.0-rc2",
"sandbox": "bubblewrap 0.9.0",
"mode": "standard (Lean's default kernel); sandboxed with bubblewrap; --paranoid was not completed",
"permitted_axioms": [
"propext",
"Quot.sound",
"Classical.choice"
],
"accepted": {
"config": "comparator.json",
"theorem_names": [
"PCP.np_subset_pcp",
"PCP.pcp_subset_np_concrete",
"PCP.ReferenceNP.inNP_iff_mem"
],
"exit": 0,
"output": [
"Build completed successfully (2089 jobs).",
"Build completed successfully (4042 jobs).",
"Running Lean default kernel on solution",
"Lean default kernel accepts the solution",
"Your solution is okay!"
]
},
"rejected_control": {
"config": "negative/comparator.json",
"change": "soundness 1/3 in place of 1/2 in PCP.np_subset_pcp",
"exit": 1,
"output": [
"Build completed successfully (2089 jobs).",
"Build completed successfully (4042 jobs).",
"error: Challenge and solution theorem statement do not match: 'PCP.np_subset_pcp'"
]
},
"sha256": {
"Challenge.lean": "5a3a21f36f3332098a64d62754de680e0f9607fb9d89783a6c4aef60264b5a50",
"Solution.lean": "ed29999d68b77e882636f5a4cbe9b94de01fb45c7828d4e8f2597501426633f3",
"comparator.json": "46e5d339f8aa13b28a6d5c2ba18aee716912f0a26527705bd7d1df9f75d9b75a",
"negative/Challenge.lean": "04bff4952768847ce734c606c6dd753a0705ec436a99b376a2afadcce48f41dc"
},
"limits": "Comparator establishes that the solution proves exactly the challenge's statements using only the permitted axioms. The definitions the statements use are imported from the authors' modules, not restated. It does not say that the statements mean what the papers say."
}
2 changes: 1 addition & 1 deletion papers/pcp-full-argument/links.json
Original file line number Diff line number Diff line change
Expand Up @@ -1312,5 +1312,5 @@
"PCP.Reduction.NAEThreeColor.direct": "Edges in one orientation",
"PCP.Reduction.NAEThreeColor.graph": "The graph of an NAE instance"
},
"fidelity_held": "The selected source has not yet been compared against elaborated Lean statements for this page."
"fidelity_held": "Sentence-by-sentence comparisons of this page's statements with their Lean have not been written yet. Comparator has judged the three headline statements; papers/pcp-full-argument/source-review.md records that run."
}
2 changes: 1 addition & 1 deletion papers/pcp-full-argument/paper.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ source = "https://github.com/troyjlee/tcs-formalizations/tree/a0ce13ed0fa264f051
source_name = "tcs-formalizations at a0ce13e"
source_lines = "https://github.com/troyjlee/tcs-formalizations/blob/a0ce13ed0fa264f051c1278e039596ce65557fa4/{path}#L{line}-L{end}"
licence = "Apache-2.0"
scope = "Lean 4.35.0-rc2 compiled the listed modules of the authors' project, and each declaration's statement, dependencies and axioms shown here are Lean's own report from those compiled modules. The links from the paper's sentences to the Lean, the plain titles and the notes were made with AI and have not been reviewed by the authors."
scope = "Lean 4.35.0-rc2 compiled the listed modules of the authors' project, and each declaration's statement, dependencies and axioms shown here are Lean's own report from those compiled modules. Comparator, run in its sandbox, accepted the authors' proofs of the forward theorem, the converse and the equivalence with the reference NP class against statements copied from their source, and refused a control that changed the soundness bound. The links from the paper's sentences to the Lean, the plain titles and the notes were made with AI and have not been reviewed by the authors."

[reading]
language = "lean"
Expand Down
14 changes: 14 additions & 0 deletions papers/pcp-full-argument/source-review.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,20 @@ Each companion's `generated/lean.json` is Lean's report on compiled modules. On

The original guide's page (`papers/pcp-reading-guide`) keeps its earlier extraction: 206 rows with `lean` provenance and a statement-only compilation of `PCP.np_subset_pcp`.

## Comparator on the headline statements

On 16 November 2026 a second GitHub Actions run ([run 43868208176](https://github.com/wcook04/proof-reader/actions/runs/43868208176), commit `027be095d4715b0d9bbd243debf08b8d5ad6907c`, workflow `.github/workflows/pcp-comparator.yml`) ran `lake comparator`, the judge shipped with the project's toolchain, inside its `bubblewrap` sandbox. The challenge is `comparator/Challenge.lean`: it imports `PCP.Complexity.NP`, `PCP.Complexity.Machine` and `Complexitylib.Classes.NP`, which define the words the statements use, and no proof. It states three theorems under the authors' own names, each copied from their source, with `sorry` for a proof:

| Statement | Authors' source |
| --- | --- |
| `PCP.np_subset_pcp` | `PCP/Theorem.lean`, lines 25 to 33 |
| `PCP.pcp_subset_np_concrete` | `PCP/ReverseConcrete.lean` |
| `PCP.ReferenceNP.inNP_iff_mem` | `PCP/Complexity/ReferenceNPConverse.lean` |

The solution, `comparator/Solution.lean`, only imports the authors' modules, so the declarations compared are the authors' own. Comparator reported "Lean default kernel accepts the solution" and "Your solution is okay!", with `propext`, `Quot.sound` and `Classical.choice` as the permitted axioms. A control, `comparator/negative/Challenge.lean`, restates the forward theorem with soundness 1/3 in place of 1/2; Comparator refused it with "Challenge and solution theorem statement do not match: 'PCP.np_subset_pcp'". `comparator/receipt.json` keeps the outputs and file hashes.

This was Comparator's standard mode. An earlier attempt in `--paranoid` mode, which adds further independent checkers, was stopped after 44 minutes without a verdict and is not claimed. Comparator establishes that the authors' proofs prove exactly these statements from the permitted axioms. The definitions the statements use are the authors' own, imported and not restated, and nothing here says the statements mean what the papers say.

This record says that Lean compiled these modules in that run and what it reported. It is not a review of the mathematics, and it does not establish that the papers' prose and the Lean have the same meaning. The links, plain titles and notes were made with AI and have not been reviewed by the authors; comparisons of statements with their Lean remain held.

The five guide files are byte-identical to their files in the shared source tree. Their SHA-256 hashes are:
Expand Down
2 changes: 1 addition & 1 deletion papers/pcp-specialist/links.json
Original file line number Diff line number Diff line change
Expand Up @@ -441,5 +441,5 @@
"PCP.Algebraic.ThreeColorVerifier.branchTest": "Seven concrete branch tests",
"PCP.Algebraic.ThreeColorVerifier.test": "Conjunction of the seven checks"
},
"fidelity_held": "The selected source has not yet been compared against elaborated Lean statements for this page."
"fidelity_held": "Sentence-by-sentence comparisons of this page's statements with their Lean have not been written yet. Comparator has judged the three headline statements; papers/pcp-full-argument/source-review.md records that run."
}
2 changes: 1 addition & 1 deletion papers/pcp-specialist/paper.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ source = "https://github.com/troyjlee/tcs-formalizations/tree/a0ce13ed0fa264f051
source_name = "tcs-formalizations at a0ce13e"
source_lines = "https://github.com/troyjlee/tcs-formalizations/blob/a0ce13ed0fa264f051c1278e039596ce65557fa4/{path}#L{line}-L{end}"
licence = "Apache-2.0"
scope = "Lean 4.35.0-rc2 compiled the listed modules of the authors' project, and each declaration's statement, dependencies and axioms shown here are Lean's own report from those compiled modules. The links from the paper's sentences to the Lean, the plain titles and the notes were made with AI and have not been reviewed by the authors."
scope = "Lean 4.35.0-rc2 compiled the listed modules of the authors' project, and each declaration's statement, dependencies and axioms shown here are Lean's own report from those compiled modules. Comparator, run in its sandbox, accepted the authors' proofs of the forward theorem, the converse and the equivalence with the reference NP class against statements copied from their source, and refused a control that changed the soundness bound. The links from the paper's sentences to the Lean, the plain titles and the notes were made with AI and have not been reviewed by the authors."

[reading]
language = "lean"
Expand Down