Skip to content

JavaScript port (js/), wire-compatible with the Python store - #2

Draft
petfold wants to merge 5 commits into
mainfrom
js-port
Draft

JavaScript port (js/), wire-compatible with the Python store#2
petfold wants to merge 5 commits into
mainfrom
js-port

Conversation

@petfold

@petfold petfold commented Nov 7, 2026

Copy link
Copy Markdown
Owner

A JavaScript port of recordstore in js/, for Node 22+ and browsers, including Freedom Browser's window.swarm. Same data model and wire format: a store written by either reads in the other, and equal content gives the same root in both.

What's in it

  • Core: staging, commits, sorted iteration with staged changes, getMany, diff, three-way merge with resolvers, and commit({reconcile}) for writers sharing a pointer (MemoryPointer with compare-and-set).
  • Bytes stores: MemoryBytesStore (SHA-256 or Swarm addressing), BeeBytesStore (Bee's /bytes; every read and write checked against its reference), SwarmProviderBytesStore (Freedom's window.swarm: swarm_readData to read, chunks published with swarm_publishChunk so references match Bee's).
  • Plain ES modules with JSDoc types and no npm dependencies. keccak comes from js-sha3 0.9.3, vendored as swarmlite vendors it (js/vendor/js-sha3/PROVENANCE.md).
  • The API is asynchronous and JavaScript-shaped: get resolves to undefined for an absent key, and delete to false.

How it is checked against Python

js/tools/make_fixtures.py runs the Python code in src/ and records:

  • canonical encodings of 61 values;
  • Swarm addresses for 15 tree shapes;
  • the root after every commit of scripted and random stores (362 commits, under SHA-256 and Swarm addressing);
  • three-way merges, with their conflicts and a resolver.

The JavaScript tests replay the same operations and must reach the same bytes and roots. Locally: 37 tests pass on Node 22 and 24. Two new CI jobs run them (js) and fail when the fixtures no longer match the Python code (js-fixtures).

Live, 8 November: OntoDAG's space pack (5,134 records, written by Python) was listed in full from Swarm through the Ant node in Freedom Browser, every blob checked against its reference: 391 s cold. test/live.test.js runs this when RECORDSTORE_BEE_URL is set.

Known differences and gaps

  • Numbers: JavaScript has one number type, so a Python float such as 1.0 read and written back becomes 1 (a different blob). Large integers stay exact as BigInt. Documented in js/README.md.
  • Untested: writing through window.swarm inside Freedom.
  • Not ported yet: feed pointers (signing), proofs, history and undo, local-first storage.
  • npm: the package stays private. recordstore is free on npm, but record-store exists, and npm blocks names that differ only by punctuation; @petfold/recordstore is free.

🤖 Generated with Claude Code

petfold and others added 5 commits November 8, 2026 01:35
@petfold @claude
JavaScript port (js/): the same store, wire-compatible with Python
7938d13 
A versioned key -> record store for Node and browsers, ported from the
Python core: staging, commits, sorted iteration, diff, three-way merge
and reconciling a moved pointer, over memory, a Bee node (/bytes) or
Freedom Browser's window.swarm. Plain modules with JSDoc types and no
npm dependencies; keccak from js-sha3 0.9.3, vendored as swarmlite
vendors it.

js/tools/make_fixtures.py runs the Python code in src/ and records
canonical encodings, Swarm addresses, the root after every commit of
scripted and random stores (362 commits, both addressings) and three-way
merges; the JS tests must reach the same bytes and roots. Live: OntoDAG's
space pack (5,134 records, written by Python) listed from Swarm through
Freedom's Ant node, every blob checked against its reference.

Not ported yet: feed pointers, proofs, history, local-first. The package
stays private until it has a name on npm.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@petfold @claude
CI: the JavaScript tests on Node 22 and 24, and stale fixtures fail
7f968f5 
js runs npm test (no install: no dependencies). js-fixtures reruns
js/tools/make_fixtures.py --check against the Python code in src/, so a
change to the wire format that leaves the JS fixtures stale turns the
build red until they are regenerated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@petfold @claude
JS: writing through window.swarm, checked in real Freedom
b091c3a 
test/freedom/freedom_check.cjs runs test/freedom/provider-check.html in
Freedom built from a checkout (throwaway profile, prompts answered). On
8 November, with Swarm Desktop's Bee as Freedom's node, a store written
through window.swarm got exactly the root computed offline under Swarm
addressing and read back equal (9 chunks, 3.1 s); reads also ran through
Freedom's own Ant node. Writing through Ant itself waits for a funded
node: unfunded, it refuses with node-not-ready.

SwarmProviderBytesStore now sends its first chunk alone. Freedom asks
before each publish until "Always allow this site" is ticked, and chunks
sent in parallel from the start each waited on a prompt: five for one
commit, now one. A refused connection is asked again on the next write.
test/provider.test.js covers the store offline with a fake provider.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@petfold @claude
JS: SwarmFeedPointer, a store's latest root in a Swarm feed
8c3fdde 
Ported from the Python pointer and swarmfs's feeds: bee-js's update
format, signed with noble-secp256k1 2.3.0 (vendored from swarmlite, as
js-sha3 was). Signing is RFC 6979 with low S, as libsecp256k1, so the
fixtures pin whole signed updates: swarmfs's own FeedOps.update, run with
the clock fixed, and the JS must write the same bytes. The same defences
against light-node lookups: read-your-writes cache, monotonic index floor,
the tip probed over the update chunks, backoff that never goes back.
verifyFeedUpdate and verifyEquivocation check envelopes offline;
swarmStore(topic, {signer | owner, batch}) is Python's swarm_store.
Transports: Bee's API, and Freedom's window.swarm for reading
(swarm_readSingleOwnerChunk, re-checked here; writing waits for #647).

Live on 8 November: loopmarket's test maker book followed through its
feed (Bee, and Freedom's own Ant through window.swarm: the same root);
the OntoDAG page feed's 11 updates; a store published behind a new feed
from JS and read back from JS and from Python, both updates verified.

js-fixtures now installs the feeds extra for signing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@petfold @claude
JS: proofs, identical to Python's
d5aba4d 
store.prove(key), provePrefix(prefix), extends(base, prefixes) and
proveExtension(base, prefixes), with verifyProof, verifyPrefix and
verifyExtension: a key's record or its absence, every record under a
prefix, a later root keeping an earlier one's records. Same formats as
Python; every envelope checked before it is handed out. ProofError moves
to proofs.js, shared with the feed updates.

The fixtures require JavaScript's envelopes to equal Python's for the
same store (key, prefix and extension proofs under both addressings),
the same verdicts from Python's envelopes, the same refused extensions
(naming the same key), and every tampered proof refused.

Live: proofs about OntoDAG's space pack made through Bee (abbey present,
4 nodes; abbeyz absent; the 11 records under 'ab'), checked offline in
JS and accepted by Python's verifiers.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@petfold