Why? | Goals | Status | How it works | Getting started | Join us
First release: a 23M-parameter model trained on one exact pass over all of English Wikipedia, with every one of its 37 training segments recomputed bit for bit. Get it on Hugging Face.
Checkable, not just published
- Every training checkpoint is published and hash-linked
- Anyone can recompute any segment and get the same bits
- The training data rebuilds byte for byte from the raw Wikipedia dump
Cheap to check, in parallel
- One segment takes about 20 minutes on one GPU, about a dollar rented
- Segments are independent: many people, any order, at the same time
- A one-click checker on RunPod prints one line to share
Small and readable
- One file trains, verifies, forks and chats:
simple/ovl.py - Timestamped by two independent authorities
- Open data, open code, open results, GPL-3.0
Most model releases ask you to trust a list of datasets and a training description. You can't check either: training is expensive, nondeterministic, and happens on someone else's machines. That matters more as models are used for decisions, and as questions about what a model was trained on (copyright, privacy, contamination of benchmarks) become legal and scientific questions rather than marketing ones.
OpenVerifiableLLM makes the training itself the evidence. The run is cut into segments. Each segment boundary commits to the full model and optimizer state, the exact position in the data, and the previous boundary. Training is made deterministic, so recomputing a segment from its published starting checkpoint gives exactly the same bits. Anyone with the same GPU type can therefore check any part of the run without trusting the publisher, and checking all parts verifies the whole computation.
Because the record pins the exact data position, it also allows things that are normally impossible to show. For example, a model can be proven never to have trained on a given article: the run is forked, the article removed, and the removal recorded in the chain.
We are building OpenVerifiableLLM to support:
- Training runs where every step can be recomputed and compared bit for bit
- Training data traceable to its raw public source
- Exact coverage: which data was seen, in which order, exactly once
- Checks that are cheap, independent and parallel
- Records that can't be quietly rewritten
- Code simple enough to review in an afternoon
We also have explicit non-goals:
- Proving a model's answers are correct. We verify the computation, not the content
- Bit-exact results across different GPU architectures
- Large models before the method is shown on small ones
See goals for the full list and the principles behind it.
The first verified model is released. It is small (23M parameters): it writes fluent, encyclopedia-style text, often gets facts wrong, and holds a basic conversation. The method is the point, and it carries over unchanged to bigger models.
What has been shown:
- 37 of 37 segments recomputed bit for bit on a second RTX 5090. Also from public files only, on fresh machines: segments 8 and 21
- Data provenance: the training data rebuilt from the raw 2026-09-01 Wikipedia dump is byte-identical
- Robustness: a host failure, a mid-run code change and a driver change all happened during training, and every affected segment still matched
- Verifiable unlearning: a fork that provably never saw one article, with the exact effect of that article measured
- Hardware scope: RTX 5090 reproduces the bits; RTX 4090, H100 and CPU do not
Open limitations:
- Checkers need the training GPU's architecture (RTX 5090)
- All checks so far were run by the project, on separate machines. We need independent checkers: see Join us
For the full results, cost, timeline and lessons learned, see the simple run. For what's next, see the roadmap.
raw Wikipedia dump ──rebuild──▶ prepared tokens ──┐
▼
entry 0 ─▶ entry 1 ─▶ entry 2 ─▶ … ─▶ entry 37 chain.jsonl: each entry hashes the full
│ │ │ │ model + optimizer state, the data
ckpt 0 ckpt 1 ckpt 2 … ckpt 37 position and the previous entry
check segment k: load ckpt k ─▶ train to boundary k+1 ─▶ compare with entry k+1, bit for bit
- Determinism: BF16 autocast with FP32 master weights, deterministic CUDA kernels, no TF32, a fixed cuBLAS workspace, and a learning rate that is a pure function of the step
- Exact coverage: every step checks that its data continues exactly where the last one stopped, and each phase must end with every item covered once
- Timestamps: every chain entry is stamped by two RFC 3161 authorities as it is written
Details: the simple run.
Check the released model without a GPU:
pip install huggingface_hub torch numpy rfc8785==0.1.4 tokenizers==0.23.2 mwparserfromhell==0.7.2 \
"safetensors>=0.4.5,<1.0" "pynacl>=1.5,<2.0"
hf download AOSSIE/openverifiable-enwiki-20260901-20260918-r1-evidence \
--repo-type dataset --include "release/simple-run/*" --exclude "*/ckpt-*" --local-dir ovl
cd ovl/release/simple-run
sha256sum -c --ignore-missing release-hashes.txt
OVL_DEVICE=cpu python code/simple/ovl.py verify --run . --weights conversation-model.pt
# weights conversation-model.pt: MATCH chain entry 37Recompute a training segment on an RTX 5090, in one click: deploy the RunPod template, wait about an hour, and read the result line in the logs. Step by step: check a training segment.
Learn more:
- The simple run: method, results, cost, timeline, lessons
- Goals, roadmap and FAQ
- The
ovllm verifyartifact verifier and the reproducibility study - All docs
| Path | What it is |
|---|---|
simple/ |
The verifiable training loop: ovl.py (train, verify, fork, chat), the segment checker, timestamps, the data rebuild |
docs/ |
Design, project and guide docs (index) |
src/, run_experiment.py, sweep.py |
The ovllm verify artifact verifier and the reproducibility study |
src/ovl_pipeline/ |
The earlier production pipeline. Frozen; the simple run uses its data preparation |
tests/ |
Tests; tests/simple/ covers simple/ovl.py |
- Check a segment. Have access to an RTX 5090, or a dollar for an hour of one? Run the one-click check and post the result line. Every independent check makes the release more credible
- Contribute. Read CONTRIBUTING.md. New work targets
simple/onmain - Talk to us on the AOSSIE Discord or in GitHub issues
OpenVerifiableLLM is an AOSSIE project, licensed under GPL-3.0. Community standards follow AOSSIE's code of conduct and security policy.