Skip to content

feat: volunteer compute v0 (Turing-scored local training over loopback) - #17

Merged
aien-spark merged 14 commits into
mainfrom
feat/volunteer-compute-v0
Oct 16, 2026
Merged

aien-spark merged 14 commits into
mainfrom
feat/volunteer-compute-v0

Conversation

@aien-spark

Copy link
Copy Markdown
Collaborator

Evidence

cargo test --workspace --verbose (all suites, 69 tests, exit 0):

beacon (bin)            test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
beacon_client           test result: ok. 7 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
open_humanity (bin)     test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
beacon_core             test result: ok. 16 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
oh_coordinator (lib)    test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
oh_coordinator (bin)    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
loopback                test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
oh_core                 test result: ok. 33 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
oh_volunteer (lib)      test result: ok. 2 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
oh_volunteer (bin)      test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
data_stays_home         test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
doc-tests (5 crates)    test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
  • cargo clippy --workspace --all-targets: 0 warnings in oh-core, oh-coordinator, oh-volunteer. The 3 remaining warnings are in beacon-core and beacon-client, files this PR does not touch.
  • cargo fmt --check: clean for every file in this PR. Formatting differences remain in beacon-core and beacon-client on main; not touched here.
  • cargo audit: 6 vulnerabilities and 3 allowed warnings, all pre-existing. Running it on main (5db65a4) gives the identical list: RUSTSEC-2026-0258 (h2), RUSTSEC-2026-0346 and RUSTSEC-2026-0349 (rustls), RUSTSEC-2026-0099, RUSTSEC-2026-0098 and RUSTSEC-2026-0104 (rustls-webpki); warnings for bincode, rustls-pemfile and yoke-derive. New in this PR: none. The only new crates in Cargo.lock are this workspace's own oh-core, oh-coordinator and oh-volunteer.

Demo (scripts/demo-local.sh: 10 rounds, 3 volunteers, 500 steps per work unit, width 32, LICENSE as corpus, 10% sealed held-out), from docs/research/2026-10-16-v0-loopback-results.md. Final held-out bits per byte, lower is better:

Seed Start Volunteers, run 1 Volunteers, run 2 Baseline (both runs)
1 8.0065 4.6533 4.6902 4.3584
2 8.0000 4.7929 4.7204 4.3878
3 7.9989 4.8026 4.7585 4.4294
Mean 4.7496 4.7230 4.3919
Spread (sample std dev) 0.0835 0.0342 0.0357

Reviewer rerun after the review fixes: volunteers 4.7003, 4.7640, 4.7631 (mean 4.7425, spread 0.0365); baseline 4.3584, 4.3878, 4.4294 (mean 4.3919, identical to the table). Rounds applied 10 of 10 for every seed; updates accepted 16, 17 and 14 of 30.

What changed

  • oh-core: Learner trait and the toy TinyByteModel (next byte from previous byte, manual gradients, deterministic by seed); pseudo-gradient plus top-k 8-bit compression; signed WorkUnit, Update, Receipt and unsigned BaseModel (pinned by the signed base_digest); u32 length-prefixed TCP frames capped at 64 MiB before allocation; Turing scorer; append-only JSON Lines ledger with credits, per-run replay protection and (from review) rejected lines.
  • oh-coordinator binary: splits the corpus into shards and a sealed held-out tail that never goes on the wire, hands out signed work units, scores each update, keeps the plain mean of accepted updates only if the mean also lowers held-out bits (otherwise the round is discarded), writes receipts and the ledger. --baseline runs the same work in one process.
  • oh-volunteer binary: verifies the work unit signature and starting model, trains locally, sends only the signed compressed update, checks the signed receipt and prints Turings earned.
  • scripts/demo-local.sh: coordinator plus 3 volunteers on loopback against the single-process baseline, over 3 seeds.
  • Docs: docs/design/volunteer-compute-v0.md, docs/research/2026-10-16-v0-loopback-results.md, README volunteer compute section, one sentence in docs/SOVEREIGN_MANIFESTO.md.
  • README promise change. Old: "It does not train models. It does not collect user data. Nothing is harvested, and no central model learns from people." New: "The model learns only from signed updates and teaching text. Personal data never leaves the volunteer's machine. No user data is collected, and nothing is harvested." Drake approved this direction in writing on 2026-10-16: "implement the Open Humanity idea: start from the beginning, go all the way through".

Review fixes in this PR (last 3 commits):

  • Every rejection receipt now has a matching rejected line in the ledger (score stored as null when not finite, so the file always reopens).
  • New test: a harmful or zero averaged update discards the round and leaves the model unchanged.
  • The volunteer refuses a starting model whose width is zero or above 4096 before doing size arithmetic on it.

Not built in v0

  • Differential privacy.
  • Secure aggregation.
  • GPU training (CPU only).
  • A real model (the learner is a next-byte toy).
  • The DiLoCo outer optimizer (plain average of accepted updates).
  • Sybil resistance (one person can run many keys and split one update across them).
  • Protection against held-out leakage from repeated scoring.
  • Transport encryption (frames travel in the clear; v0 is for loopback only).
  • Pinned coordinator identity by default (volunteer trusts the first key it sees unless --coordinator-key is given).
  • Protection against slow or silent volunteers (one stalled volunteer holds up the round, 10 minute timeout).
  • A ledger line for dropped updates (bad signature or wrong work unit: logged only, no receipt).
  • Teaching text (a local model writing text scored the same way).
  • Open question: why the volunteer runs trail the baseline by about 0.33 bits per byte. Plain averaging and 5% compression are the suspects; neither is measured on its own yet.

Zero secrets

Zero secrets: none in this diff. Volunteer and coordinator signing keys are generated in memory and never written to disk.

🤖 Generated with Claude Code

drakestapleton and others added 14 commits October 15, 2026 19:42
Goal, principle change (approved by Drake 2026-10-16), invariants,
Turing definition, wire formats and the not-built list.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
New crate oh-core. Learner is the seam the coordinator and volunteers
use; TinyByteModel predicts the next byte from the previous one
(embedding, tanh hidden layer, softmax over 256 bytes) with hand-written
gradients and plain SGD, deterministic for a seed (SplitMix64).
Held-out bits are computed in f64.

Tests: 5 pass (bits per byte 7.999 to 0.761 on repetitive text,
held-out bits match an independent -log2 p, determinism, params round trip).

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pseudo_gradient (after - before), apply (base + delta), and
CompressedDelta: top-k by magnitude plus 8-bit linear quantization with
scale = max |kept| / 127. Zero and non-finite entries are never kept.
validate() checks deltas that arrive over the network.

Tests: 7 new, 12 pass. 10,000-entry delta at 5%: 40,008 bytes dense,
2,524 bytes compressed (15.9x).

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
WorkUnit, Update and Receipt with bincode signing bytes (sig emptied),
Ed25519 sign/verify via beacon-core, verify_from to pin the signer,
Update::verify_for to bind an update to its work unit, and BLAKE3
digests (params_digest, delta_digest, message digest). Frames are a
u32 big-endian length plus bincode payload, capped at 64 MiB and
checked before any buffer is allocated. std::io only, no async runtime.

Tests: 11 new, 23 pass.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Scorer::score returns held-out bits(base) minus bits(base + delta) and
restores the model; accept requires a finite score above epsilon.
Ledger is a JSON Lines file of seen and credit entries: per-run replay
protection on delta digests, per-node Turing totals, both rebuilt on
reopen. Credits must be finite and positive; a torn last line is refused.
Also runs rustfmt over the oh-core sources (layout only).

Tests: 8 new, 31 pass. Honest delta +454.59 Turings dense, +449.08
after 5% compression; random delta -48.74 Turings.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cargo test --workspace --verbose, exit 0:
beacon 0, beacon_client 7, open_humanity 0, beacon_core 16,
oh_core 31 passed; doc tests 0. cargo clippy -p oh-core --all-targets
reports no oh-core warnings (two existing beacon-core warnings remain).

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The volunteer needs the starting parameters and model width before it can
train; the signed WorkUnit pins them through base_digest.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
oh-coordinator splits a sealed held-out tail off the corpus, sends each
volunteer the base model and a signed work unit with its own shard, judges
every update (signature, work unit match, replay, Turing score), sends a
signed receipt, credits the ledger, and applies the plain mean of accepted
updates only if it lowers held-out bits. --baseline trains the same shards
for the same total steps in one process.

oh-volunteer verifies the work unit and base model, trains locally, and
sends back only the signed compressed update.

Tests: 2 volunteers x 2 rounds over loopback (held-out bits do not rise,
receipts verify, ledger credits match); no held-out 16-byte run in any
frame sent; replay rejected; update frame holds no shard bytes; bad or
unexpected work unit signature refused.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…opback

Builds release binaries, runs oh-coordinator with three oh-volunteer
processes for ROUNDS rounds per seed, then oh-coordinator --baseline with
the same corpus, shards and total steps, and prints mean and sample
standard deviation of final held-out bits per byte for both. Corpus is the
repository LICENSE file. Bash only.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…aseline

Observed on the Spark, 3 seeds, 10 rounds, 500 steps, dim 32, LICENSE as
corpus. Final held-out bits per byte: volunteers 4.7496 (sd 0.0835) and
4.7230 (sd 0.0342) over two runs; baseline 4.3919 (sd 0.0357), from 8.00
at the start. Proves the pipeline, not model quality.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Line 9 now carries the promise from the design doc: the model learns only
from signed updates and teaching text, and personal data never leaves the
volunteer's machine. New section covers what v0 does, what leaves the
volunteer's machine, Turing scoring, the two binaries, the demo and its
measured result, and the not-built list from the design doc. Core Crates
lists oh-core, oh-coordinator and oh-volunteer.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… receipt

Ledger gains a rejected entry (node key, score or null when not finite,
reason, receipt digest) that never changes totals. Coordinator::judge
writes it on every path that signs a rejection receipt. Tests: rejected
lines survive reopen without credit; a replayed update leaves one credit
and two rejected lines; a harmful or zero average discards the round.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
dim comes off the wire; cap it at 4096 before the parameter count
arithmetic so a hostile coordinator cannot overflow it.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Design doc not-built list adds transport encryption, default coordinator
key pinning, stalled volunteers and dropped-update records, plus an open
question on the 0.33 bits per byte gap to the baseline. README lists the
transport gap. Manifesto privacy line now states the new promise.

Co-authored-by: Drake Stapleton <drake@aien.org>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

⚖️ Sovereign Code Review & Alignment Gate

Greetings @aien-spark. I am AIEN, the automated review gate for the AIENOS project.

Your pull request #17 ("feat: volunteer compute v0 (Turing-scored local training over loopback)") has undergone automated auditing against the project standards.

✅ Doctrine Diff Audit: PASS

  • Telemetry Check: Clean (zero tracking or surveillance hooks detected).
  • Sovereign Voice & Formatting Check: Clean (zero em/en dashes, zero banned buzzwords).
  • Secrets & Credentials Check: Clean (zero plaintext secrets, hardware vault only).
  • Status: Project standards verified.

🌿 Sovereign Steward: Contributor Alignment Interview

Greetings @aien-spark. I am AIEN, the automated steward for the AIENOS project.

We welcome all contributors who share our commitment to open collaboration, global humanity, and disciplined systems craftsmanship. We build to expand computing capability, advance open-source intelligence, and share technology freely with the world.

Your pull request #17 ("feat: volunteer compute v0 (Turing-scored local training over loopback)") is ready for stewardship alignment.

Please respond directly to these four alignment inquiries:

  1. Long-Term Mission Alignment:
    This initiative is an enduring engineering commitment to open-source technology for humanity, not a vehicle for short-term hype, social media monetization, or get-rich-quick ventures. How does your contribution support global progress and the resilience of local communities?

  2. Telemetry & Integrity:
    Do you certify that this change introduces zero telemetry, zero surveillance, and zero artificial paywalls?

    "I certify that my contribution is submitted in service of human sovereignty and open collaboration. I affirm that this work maintains zero telemetry, contains no surveillance backdoors, and introduces no artificial barriers, and that I offer it under AGPL-3.0-or-later, the license that governs this project. I build so that intelligence is owned by the person who runs it, not rented."

  3. Downstream Heritage Preservation:
    Do you acknowledge and agree that any downstream branch, fork, or copy of this work remains under AGPL-3.0-or-later and retains the project's license and attribution notices, as the LICENSE file requires?

  4. Craftsmanship & Native Discipline:
    What architectural trade-offs did you make, and how did you verify that this code introduces zero regressions, zero unneeded dependencies, and complies with our pure native compiled standards?


Reply to this comment with your affirmation. Once verified against the project standards, I will grant the sovereign-interview-passed clearance. For direct sovereign coordination: Drake Stapleton (drake@aienos.com) and AIEN (aien@aienos.com)

@github-actions github-actions Bot added needs-testimony Contributor testimony required sovereign-audit-passed Constitutional diff audit passed labels Oct 16, 2026
@aien-spark
aien-spark merged commit df500d9 into main Oct 16, 2026
3 checks passed
@aien-spark
aien-spark deleted the feat/volunteer-compute-v0 branch October 16, 2026 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-testimony Contributor testimony required sovereign-audit-passed Constitutional diff audit passed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants