feat: complete the verifiable research core - #1
Open
SHi-ON wants to merge 116 commits into
Open
Conversation
The book manifest recorded the SHA-256 of RESEARCH.md as read from a Windows checkout with CRLF line endings, while git stores LF. The Research book integrity workflow therefore failed on every Linux CI run. Normalize line endings before hashing in both the build script and the test, and re-stamp the manifest with the LF hash. Page images and PDF are unchanged because the rendered content is identical. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Define the service-boundary contracts every runtime package implements against (signers, evidence writer/reader, checkpoint service, anchor publisher, run state machine, scenario engine, learner adapter, symbol gateway, nursery runtime), the hash domain separators and stream/tree naming shared by the evidence chain, zod schemas for checkpoint manifests, run manifests, anchor receipts, Merkle proofs, turn records, intervention events, and generated-analysis entries, and the evidence bundle byte-level format. Add @ald/hashing with domain-separated SHA-256, RFC 8785 canonical JSON, Ed25519 sign/verify with raw-key encodings, per-run in-memory signer registry, and a deterministic seeded PRNG. Scaffold the remaining packages (merkle, lifecycle, scenario, learners, gateway, checkpoint, anchor, verifier, orchestrator) with build wiring; vitest now aliases every packages/* directory automatically. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…erkle, lifecycle, scenario, learners, gateway Seven packages implemented in parallel against the shared contracts: - @ald/hashing: chain-walk validator with the full LEDGER §17 single-stream mutation matrix, signed-event builder bound to the stream's signer domain, canonical JSONL parser, and a file-backed per-run Ed25519 key store (ALD-008, ALD-009). - @ald/evidence: SqliteEvidenceWriter/Reader with serialized writes, atomic sender-ledger + channel-event turn commits, rejection and oracle control events, interpretation binding to the recorded delivery, turn records, intervention and generated-analysis streams, write-time fork detection with preserved conflicting artifacts, recovery, and deterministic evidence bundle export; migration v2 adds turn_records, run_signers, fork_artifacts; child-process crash-safety tests (ALD-010, ALD-011, ALD-016). - @ald/merkle: RFC 6962 ordered trees with the LEDGER §7 leaf/node domains, inclusion and consistency proofs and verifiers (ALD-012). - @ald/lifecycle: SPEC §7.2 run state machine with exhaustive transition tests, run configuration validation and defaults builder, derived-run configuration lineage (ALD-023, ALD-024, ALD-026, ALD-028). - @ald/scenario: deterministic referential scenario engine with held-out splits, opaque references, independent receiver ordering, oracle encoding, all five §9.5 interaction profiles, and the observation hygiene filter (ALD-037, ALD-038, ALD-041). - @ald/learners: learner adapter conformance harness, no-learning reference adapter, tabular REINFORCE scratch-RL adapter with agent-native ledger drafts and ledger-to-prediction function, versioned learner contracts with a banned-pattern lint (ALD-042, ALD-043, ALD-045). - @ald/gateway: Symbol Gateway with the fixed-token protocol, all six §9.6 communication-control conditions, protocol-independent rejection framework, interpretation binding, and a conformance vector suite (ALD-029, ALD-030, ALD-034, ALD-035, ALD-036). Whole-repo build, lint, and 561 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- AgentActionProposalSchema is now an explicit discriminated union so proposal.kind narrows publicArtifact for every consumer. - ledgerStreamForRole returns the narrowed LedgerStream type. - EvidenceWriter gains appendAffectEvent; EvidenceReader gains readRunSigners, matching the SQLite writer's existing surface; the gateway in-memory test double implements both. - RunConfig gains an optional evaluationTurns stage budget (SPEC §8.3). - LearnerContract carries an optional track. - viem added to @ald/anchor; contract lint wired into npm run check. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…dent verifier, Nursery orchestrator - @ald/checkpoint: EvidenceCheckpointService builds LEDGER §8 manifests from the Evidence Store with independently recomputed ordered Merkle roots, witness signatures, chained previous-checkpoint hashes, interval idempotency, inclusion/consistency proof records, bundle proof-file writer, and a background CheckpointScheduler (ALD-013, ALD-014). - @ald/anchor: ChainTransport abstraction with an in-memory fake chain and a viem-backed Base transport, BaseAnchorPublisher with idempotent submission, bounded retry, confirmation polling, single terminal receipt insert, pending-file crash safety, double-opt-in mainnet gate, anchor key file handling, and the verifier-side anchor checks (ALD-018..ALD-022). - @ald/verifier: standalone verifyBundle library and ald-verify CLI that re-derives every hash, chain link, signature, Merkle root, proof, checkpoint chain, cross-binding, configuration hash, claim statement, and anchor binding from an exported bundle alone, with the full LEDGER §17 mutation matrix as tests (ALD-015, ALD-017). - @ald/orchestrator: NurseryRuntime implementing the SPEC §8.1 turn cycle across scenario engine, learner adapters, Symbol Gateway, Evidence Writer, checkpoints, experiment records, pause/resume/abort, crash recovery, fork handling, replay digest, and sealing along the spec's unanchored path (ALD-025, ALD-026, ALD-027, partial ALD-059/ALD-071). Whole-repo build, lint, and 737 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ugh, orchestrator wiring - validateChain now honours each stream's link field (previousChannelHash for the channel transcript), removing the verifier and orchestrator workarounds and letting the pre-resume integrity check cover the channel stream. - buildRunConfig passes evaluationTurns through. - @ald/orchestrator depends on @ald/checkpoint, @ald/anchor, @ald/verifier, and the new @ald/analysis skeleton; experiment and production modules are stubbed for Phase D. - CONFIGURATION.md documents the mainnet double opt-in and key-store layout. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…or hardening - @ald/analysis: special functions (log-gamma, regularized incomplete beta, Student t, normal), descriptive statistics, one-sample t-test, TOST equivalence, Holm-Bonferroni, Wilson intervals, seeded bootstrap intervals, Cohen's h and rank-biserial effect sizes, the E03 Appendix D §D.6-§D.10 analysis, and E11 training/evaluation summaries (ALD-072 statistics scaffold). - Twin packs: baby-a/baby-b learner routes (§12.4), nursery controller, evidence, and verification routes (§12.5, §12.6), role guards and the §12.3 response envelope, a prototype-mode runtime registry, and route/role matrix and isolation tests (ALD-048..ALD-052). - Orchestrator hardening: SPEC §14.5 adapter-failure retry-then-pause, idempotent seal with retrySeal/abandonSeal, empty-message receiver path in both learner adapters so the disabled condition works with scratch-rl. Whole-repo build, lint, and 866 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ency proofs Checkpoint 0 is created before any turn, so the auxiliary turns tree is omitted from its manifest (LEDGER §8). The checkpoint service correctly emits a consistency proof from that checkpoint with fromSize 0, but the verifier rejected it as a tree not committed by both checkpoints, failing every run with two or more checkpoints. The verifier now resolves the omission to the documented empty tree; the bundle format states the rule; the fixture ships the fromSize-0 proof and a regression test pins it. Also export the orchestrator's adapter-failure and seal-recovery symbols. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…t writer, and run script createProductionRuntime wires the Nursery runtime to the real checkpoint service, proof writer, and verifier (anchoring skipped in this environment). runE03Controls runs the six §9.6 communication conditions per Appendix D seed slot with no-learning learners and feeds seed-level success into the E03 analysis; runE11NamingGame runs the tabular scratch-RL naming game with a frozen evaluation phase and reports windowed learning curves, held-out success, vocabulary utilization, symbol entropy, policy-hash constancy, and verifier results. writeQualificationReport emits summaries and a REPORT.md that carries the Prototype Mode claim boundary and the non-confirmatory qualification label verbatim. scripts/run-qualification.mjs drives both. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Anchor: no receipt row on the confirmation give-up path (pending sidecar kept so a later poll records the confirmation), unconditional per-checkpoint idempotency with in-flight coalescing, nonce-pinned viem submissions that re-broadcast identical bytes on retry, not-found classification by viem error class only, redacted transport errors that never carry the RPC URL, a 192-block safe-tag proxy with a separate confirmation-poll budget, key-file errors that never echo the key, and an exclusive fsynced 0600 sidecar write. Checkpoint: cross-package regression test verifying an exported bundle with proof files through the independent verifier, tick interval defaulting to a quarter of the time interval, and scheduler observers that cannot poison the promise chain or re-add checkpointed events. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Signature checks, hash domains, tree names, and file names are taken from the normative constants rather than trusted from run-manifest.json; every hash is rebuilt from the raw parsed file value and unknown keys are reported for manifests, configuration, checkpoints, proofs, receipts, and experiment records; bundle-relative names are contained to single segments inside the bundle; prompts/ is rebuilt into the prompt bundle hash and bound to the configuration, manifest, and every checkpoint; receipts are bound to the configured network and finality policy; experiment-record references are bound to bundle checkpoints and receipts; cross-bindings check turn agreement; the report records whether the on-chain half ran; RPC URLs are redacted; report persistence and CLI failures degrade to exit code 1 instead of losing the result. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…rom learners - hashing: canonicalJson rejects every non-JSON-native value instead of dropping data; signer objects no longer carry the raw seed. - evidence: recursive triggers so REPLACE cannot bypass append-only guards; recover() is checkpoint-aware and blocks a run whose committed prefix is shorter than, or differs from, its last checkpoint; fork-artifact stream guard fixed and constrained by migration v3. - merkle: proof verification bounds sizes and path lengths; range-root cache keyed by leaf-array identity. - lifecycle: finality policy derived from the anchor network; fixed-glyph accepts inventory and message bounds. - gateway: complexity budget on proposal inspection with a payload-too-complex rejection instead of a stack overflow. - scenario: whitespace-free prose caught by the hygiene filter. - learners: intrinsic/extrinsic configuration mismatch rejected at init; tool-only assertion checks exact artifact keys and non-enumerable fields. - types: LearnerInitContext.config is LearnerVisibleRunConfig (no randomSeed), so an adapter cannot regenerate researcher-only ground truth. Whole-repo build, lint, and 1014 tests pass. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Recovery reconstructs terminal states from evidence and is read-only for sealed, aborted, or forked runs; it refuses a signer registry that does not match the run's registered public keys. Seal idempotence rests on facts later checkpoints cannot erase, a retried abort seal keeps its abort disposition, and stage transitions are evaluated independently of pauses so a safety trigger during evaluation escalates through the §7.2 abort row instead of looping or buying an extra turn. Annotations cannot chain a checkpoint after the final one. The shuffled pre-pass stamps ledger events with their slot turn and turns a malformed envelope into a committed rejection. A nonzero verifier exit is audited with a safety trigger and deviation. The runtime no longer inserts anchor receipts itself, deferring to the publisher's single terminal row. Learners receive the configuration without randomSeed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The E03/E11 harness and scratch-RL runtime tests drive thousands of signed turns through SQLite and exceed 60 seconds on a loaded machine. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Hypothesis and first-use records are committed to the ledger before the adapter marks them, so a retried call re-emits the same event instead of losing it. Blinding nonces are position-independent hashes of the event's own identity, so a restart cannot reuse a nonce. The exported tabular policy (version 2) carries the first-use and hypothesis registries and the intrinsic predictor, restored on same-run recovery and reset for derived runs, while the checkpoint hash still stays constant through evaluation. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Viewer-only ledger filtering on the nursery ledgers route, malformed path encoding mapped into the §12.3 envelope, inbound observations passed through the hygiene filter and bound to the pack's own role with an audited rejection, Baby tool routes gated on a live run, snapshot ids validated and contained under the session directory with fixed error messages, path-safe runId validation on run creation, the 501 code removed in favour of the §12.3 set, human-view audit events on list and delta reads, and the §14.2 annotate route. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s, stricter conformance, shape-checked policy loading The scratch-RL receiver scores the delivered prefix and records the message verbatim, so control conditions that deliver a different length no longer abort the run. act() and onOutcome() replay the same draw and reward on a retried turn so seeded replay stays exact. The conformance harness rejects keys the schema would strip and scans agent-native ledger content for human language. Policy checkpoints from a different attribute space are refused. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ion state reports/qualification/20260907-2228-e9a8abe: non-confirmatory Prototype Mode run on commit e9a8abe — 30 E03 control runs (5 seeds x 6 conditions x 200 episodes) and 3 E11 naming-game runs (3000 training + 200 evaluation turns), all 33 evidence bundles independently verified. E03 non-oracle conditions sit at chance (0.238) with oracle adequacy 1.0 and separation lower bound 0.75; E11 evaluation success 0.93/0.975/0.98 against chance 0.25 with policy hashes constant through evaluation. Every run is disposition invalid by construction (unanchored) and nothing here is a research finding. README status describes the verifiable-core phase; BACKLOG §1/§2/§15 record the verified state and implementation decisions, with acceptance checkboxes set from agent-verified evidence; reports/README notes that E03 controls are trajectory-identical with a no-learning receiver under the Appendix D design. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tion binding, bundle attachments, isolation and provenance seams, affect windows, causal probes; add learner track stubs and package skeletons Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Complete and stabilize the interrupted Phase E wave across alternate carriers, affect routing, learner adapters, isolation, operations, interventions, and observation red-teaming. Add evidence-backed acceptance coverage, centralize new hash domains, and bump the working patch version to 0.1.1.
Register built-in synthetic scenarios through the quarantine pipeline and require every engine bundle to be approved before run state is created. Persist the registry in production and Nursery twin wiring, add fail-closed integration tests, and bump the working patch version to 0.1.2.
Bind held-out scenario types and fixed curriculum transitions into Nursery runs, restore active stages after restart, and ship contracts for every implemented learner track. Preserve intervention-related RunConfig overrides and record transition policy hashes in the audit stream.
Freeze ledger-derived probe schedules before evaluation, apply symbolic ablations and substitutions through the Gateway, and bind each applied probe into signed turn evidence. Rebuild schedules on recovery and independently verify descriptors, artifacts, transformations, and turn links.
Schedule one extra attempt after a failed primary episode, preserve episode budgets and carrier limits, and restore pending attempts after restart. Bind repairs to signed turn records and make the standalone verifier reject missing, duplicated, or scenario-changing repair links.
Exercise the retention job against real exported bundles and SQLite evidence. Cover eligibility, disabled and active windows, public-release and Base-mainnet guards, preserved evidence rows, and the chained retention audit log.
Require Research-Grade runs to use normalized adapters that self-report distinct container IDs. Add the locked-down two-network Docker topology and an automated smoke gate that verifies capability denial, route denial, and learner failure isolation.
Exercise all enumerated side-channel categories through real Gateway submissions, transport measurements, host capability probes, and a deliberate hidden-state correlation. Reuse the ALD-040 measurement primitives and execute the joined suite in the Mode R container gate while preserving the Mode P claim boundary.
Commit the unsigned intervention chain as an auxiliary Merkle tree so attachment bindings and governance events can be proven through witness-signed checkpoints and anchors.
Store canonical analysis artifacts atomically with intervention events, witness their prefixes, export indexed hashes, link them from Experiment Records, and verify every file, binding, anchor boundary, and record reference independently.
Use an IPC readiness handshake before sending SIGTERM so the integration test cannot race the child process signal-handler installation.
Move the existing TypeScript build ahead of executable audit scripts in local and CI check chains. This fixes the exact-checkout failure where warm dist output had masked the ordering dependency.
Bind the exact v0.1.68 comparator candidate to frozen synthetic fit/select/score fixtures, seven negative controls, clean-check evidence, and explicit remaining production chronology work. Keep B10 open and preserve the failed clean-check predecessor.
Add an eleven-binding fail-closed packet compiler and a generated readiness inventory for all 19 experiments. Keep every incomplete packet uncompiled and expose 190 unresolved experiment-specific bindings without fabricating registration hashes.
Add bounded carrier-specific distance functions and deterministic held-out nearest-prototype scoring for bitmap, canvas, and tone representations. Keep the result explicitly limited to a handcrafted diagnostic and preserve learned generalization as an open empirical question.
Bind the v0.1.71 implementation to a clean detached validation, deterministic three-carrier fixtures, and six negative controls. Keep B09 open for learned-form evidence and actual-topology side-feature qualification.
Exercise dimension, metadata, raw-media, sample-rate, compression, and container attacks through the real Gateway for all generated carriers. Keep the reusable report ineligible as study evidence until topology-specific qualification.
Bind 28 generated-carrier Gateway attacks and the current two-container Mode R timing, envelope, host, crash, and training controls to exact v0.1.73 evidence. Preserve final-topology and powered-negative-bound work as open blockers.
Bind comparator and native predictor identities in RunConfig, commit predictions after accepted delivery but before receiver action, checkpoint chronology, and source-bind post-outcome scores. Recovery and mismatched providers fail closed while eligible registered execution remains open.
Bind the exact v0.1.75 runtime candidate, focused and consolidated validation logs, provider identity and eligibility controls, and seven fail-closed cases in a machine-audited receipt. Keep B10 open for eligible registration, execution, aggregate inference, and independent review.
Reconcile notebook and manuscript status with the fail-closed campaign audit, add a machine-bound readiness check and eight-perspective critical review, anonymize working metadata, and remove a footer that overprinted the rendered paper. The review remains needs-revision because no eligible empirical data or external approvals exist.
Add a privacy-safe O01-O06 readiness ledger, bind a fresh read-only upstream enforcement observation, wire both into the consolidated and manuscript audits, and document the exact activation sequence. The ledger remains blocked at 0/6 and does not treat receipt paths as approval.
Record the exact pull-request workflow as action-required with zero jobs started, preserve O04 as unsatisfied, and repair a manuscript list-item pagination defect exposed by the new evidence text.
Capture the audited Fort and Safe provisioning, dual Base Sepolia readiness checks, and four-track files-mode qualification without promoting it to a public-chain or research result. Mark O02 provisioned-unfunded, refresh the manuscript and critical review, and bump the project to 0.1.80.
Bind simulated versus public-chain receipt class across run configuration, anchoring, preregistration, and verification. Record the scoped governance decision, make public-chain funding prerequisites non-applicable, and revise the research plan, manuscript, reports, and generated book without fabricating empirical outcomes.
Adopt the zero-spend local research profile as the experiment activation path. Bind confirmatory packets to immutable ancestral Git records and deterministic simulated commitments while preserving explicit limits on timestamps, public finality, hosted enforcement, and independent review.
Freeze all eleven E00 packet bindings, five deterministic qualification seed slots, local execution limits, dual-verifier policy, and eleven mutation cases before collecting qualification outcomes.
Supersede the outcome-free v1 packet with a v2 packet that records the amendment but excludes mutable package metadata from the execution environment binding.
Verify the immutable v2 packet from Git, drive its exact hash through the deterministic in-memory chain, and freeze the repository-native binding and three-confirmation simulated receipt before qualification outcomes.
Run five fixed 100-turn slots through both integrity implementations, exercise the eleven preregistered mutations, measure actual ledger/checkpoint counts, and require recovery-extension qualifications.
Preserve the failed E00 v2 attempt, independently verify exported inclusion proofs in Rust, and preregister E00 v3 with exact verifier source hashes and fresh qualification seeds.
Freeze the v3 repository registration and deterministic three-confirmation simulated commitment after the prior failed attempt and before any fresh-seed execution.
Freeze the E00 v4 software-qualification protocol on a fresh seed domain, require retained per-slot evidence, and keep its deterministic simulated commitment separate from outcome execution.
Bind the immutable E00 v4 packet to a deterministic in-memory chain receipt with three simulated confirmations and make only the E00 qualification executable.
Preserve the five-slot v4 failure, make the independent Rust auditor enforce the configured anchor class, and prospectively register E00 v5 on fresh seeds.
Commit the fresh E00 v5 packet binding to the deterministic local chain before any v5 outcome execution.
Audit and preserve the prospectively registered five-slot E00 qualification, classify its local bundles, and update the research report while keeping all behavioral claims pre-results.
Add a fail-closed five-slot Mode R isolation qualification path with exactly 100 recipient-visible attacks per slot, complete host-capability coverage, and detector-positive controls. Keep results outside behavioral claims until prospectively registered and executed.
Freeze the exact five-slot Mode R isolation packet, analysis source hashes, zero-spend simulated policy, seeds, attack counts, positive controls, and fail-closed stopping rules before outcome collection.
Bind the immutable E01 packet to a deterministic three-confirmation simulated pre-run receipt under the approved zero-spend profile, while preserving the boundary that no registered outcomes have yet been collected.
Record the first registered five-slot Mode R qualification, add a fail-closed receipt audit, and keep E01 in progress after internal review identifies the explicit-corpus and attempt-level evidence gap as B15.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Validation
Exact commit:
d1b4eabcde991de06eb94e6f260a62c7d4b00f8cnpm run check— 134 test files, 1,735 tests passed; secret scan passed for 578 files; zero high-severity dependency advisoriesnpm run test:mode-r— passed Docker network/process isolation, survivor behavior, twelve side-channel categories, and local-only policy updates for scratch-RL, self-supervised, and hybrid tracks9bf36f3f2e4e324c46274a0873f23e63692eb4f2fa4fc8f57ff4109a96164c997485fe6f11af29433bc51cab58009521f205840f5b4ae3a32fa7f92e8534fdf5, llama.cpp archive SHA-256819c30050d77e4305a978cacccddac8b44f21b365fe7ae67a20213f4ae4e7e92consolidated-suiteandmode-r— both passed on the exact commit; audit, runtime, JUnit, and Mode R evidence uploadedRemaining external evidence gates
The exact-head upstream workflow is awaiting maintainer approval: run 34414725835.
External registration and governance approval remain prerequisites to confirmatory collection. No experiment result is claimed.