Protect and reproduce frozen research evidence

[abrahamabel]

Reproducing a completed experiment previously defaulted to overwriting its retained result, and repository checks allowed a completed claim without runnable evidence or references. This change adds a safe reproduction command that verifies frozen inputs, runs each registered experiment through --stdout, and compares every deterministic JSON field with precise mismatch paths. Optional fresh outputs must be outside the repository and cannot overwrite existing files.

The registry records the unchanged canonical engine as canonical-gng-v1 and hashes all completed experiment files. Validation now connects completed manifests to registered evidence, checks references and predecessor IDs, and catches changed or unregistered frozen files. CI runs the complete registered experiments; CODEOWNERS routes evidence and lab changes to maintainers. The roadmap records the capacity-controlled successor and later evidence gates.

Related issue: #7 (preparation only; the new experiment follows in a separate PR).

Validation

• python3 scripts/validate_repository.py — passed.
• python3 -m unittest discover -s tests -p 'test_*.py' — 43 tests passed.
• python3 scripts/reproduce_experiments.py — both complete historical experiments matched every deterministic field; only declared environment/timing/allocation fields were excluded.
• git diff --check — passed.
• Existing completed experiment files and src/atmai_gng/gng.py are byte-identical to the base revision.

Boundaries

No new algorithm is promoted. The historical reference's parameter/numerical limitations remain unchanged and require a separately evidenced successor. Exact comparisons may expose Python/platform differences; those are failures to investigate, not silently tolerated deviations. Updating a registry hash does not justify rewriting a completed claim.

Repository-owned MIT code and synthetic evidence only; no third-party implementation code or external datasets introduced. Maintainer/code-owner review remains the merge gate.

---

Note

Medium Risk
CI now executes full scientific experiments with strict byte-exact checks; platform/Python drift can fail builds, and the registry becomes the gate for any change to completed evidence or the canonical engine path.

Overview
Adds a frozen evidence pipeline so completed experiments can be re-run without overwriting retained results. Root reproducibility.json registers both completed labs, SHA-256 hashes for every frozen file, the canonical-gng-v1 engine, and per-experiment JSON paths to ignore (host/timing only).

scripts/reproduce_experiments.py verifies hashes before and after execution, runs each entrypoint with python -B … --stdout, and compares fresh JSON to retained artifacts with exact, pointer-level diffs (no float tolerance). Optional --output-dir must live outside the repo and cannot overwrite existing files.

validate_repository.py now ties complete manifests to the registry (EXP016), validates reference URLs and predecessor IDs, and surfaces registry/hash problems (REPRO001/REPRO002). CI runs the full reproduction step and bumps the job timeout to 20 minutes. CODEOWNERS covers reproducibility.json, labs/, and related docs; README, docs/reproduction.md, and docs/roadmap.md document usage and the issue #7 successor gates.

Unit tests cover reproduction safety, comparison semantics, and the new validation rules. Historical experiment bytes and src/atmai_gng/gng.py are unchanged in this PR.

^{Reviewed by Cursor Bugbot for commit c573947. Bugbot is set up for automated code reviews on this repo. Configure here.}

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.