Skip to content

docs: say what "verified" means and state the limits plainly - #253

Merged
ryoari merged 1 commit into
mainfrom
docs/what-verified-means
Oct 28, 2026
Merged

ryoari merged 1 commit into
mainfrom
docs/what-verified-means

Conversation

@ryoari

@ryoari ryoari commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Makes the docs say what has actually been shown and what it relies on.

README

  • "Getting started" is labelled a consistency check: the release matches its timestamped record, but no training is recomputed. It now also verifies the anchor's DigiCert RFC 3161 timestamp offline (openssl ts -verify, system CA store). The segment recomputation is called the actual verification.
  • Open limitations now cover:
    • toolchain trust;
    • about 110 independent random checks to catch one forged segment out of 37 with 95% probability;
    • self-reported, unsigned results;
    • timestamps prove the record's existence, not how training was done.
  • "Carries over unchanged to bigger models" now says that full checks cost as much as training, so large models rely on sampling.
  • "Verifiable unlearning" becomes verifiable exclusion: exact retraining without an article the run hadn't reached.
  • "One file" now counts the roughly 1,000 imported lines.

docs/design/simple_run.md

  • New section "What verified means" with:
    • the consistency check vs. the segment check;
    • same-architecture and toolchain trust;
    • the 1 − ((N−1)/N)^M numbers (110 checks for 95%, 169 for 99%);
    • self-reported results;
    • cost, and 284 MB per boundary (about 12 bytes per parameter).
  • The exclusion fork paragraph says what it proves and what it doesn't. The Hugging Face folder name unlearning-3i-atlas is unchanged, because it's pinned evidence.
  • Limitations note the September 2026 history rewrite and link the provenance note.

FAQ, goals, roadmap

  • The FAQ and goals use the same wording.
  • The roadmap adds signed check results with segment choice from public randomness, plus two open research questions: checking across architectures, and checking for less than the cost of training.

The roadmap line about the anchor check assumes the fix/verify-anchor-required PR is merged first.

- Getting started is labelled a consistency check and verifies the
  anchor's DigiCert timestamp; segment recomputation is the verification.
- New "What verified means" section: same-architecture and toolchain
  trust, the number of random checks needed (about 110 for 95% at 37
  segments), self-reported results, cost and per-boundary storage.
- "Unlearning" becomes "exclusion": exact retraining without an
  article the run hadn't reached.
- The one-file claim includes the imported model and data code; the
  scaling claim notes that full checks cost as much as training.
- Limitations note the September history rewrite; the roadmap adds signed
  check results, public randomness and two open research questions.
@coderabbitai

coderabbitai Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 59 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration
Commits

Reviewing files that changed from the base of the PR and between 748280a and 1b37694.

Files selected for processing: 5
  • README.md
  • docs/design/simple_run.md
  • docs/project/faq.md
  • docs/project/goals.md
  • docs/project/roadmap.md
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@ryoari
ryoari merged commit 364fb88 into main Oct 28, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant