Skip to content

Store backends (SQLite + PostgreSQL) and remote storage (MinIO first): first integration phase #506

Closed as not planned
2 of 2 issues completed
Closed as not planned

Description

@yifanxuaaa

Design doc: core/docs/issues/506/README.md (Store-first model, workflows, live path, layout, runtime calls).

LayerFS 2.0 areas A (Store backends) and R (remote storage), tracked on the 2.0 board. This issue covers the first integration phase of both, worked in one worktree by two sessions.

Owner direction (2026-10-29)

  • Build PostgreSQL as a second Store backend next to SQLite. Each Workspace overlay stays its own local SQLite file.
  • A PostgreSQL Store's synchronous_commit is a setting, default off, applied with SET LOCAL in each write transaction. The SQLite Store stays Disposable only.
  • A remote holds the whole SQLite Store file. Push uploads it; pull downloads it and replaces the local Store. Atomic compare-and-swap on the remote is not required.
  • The first remote provider is MinIO only. Other providers come later.

a. Persistence: SQLite and PostgreSQL behind one seam

Code: engine/crates/layerfs-persistence (and the open path in layerfs-store-service).

  1. Move the SQLite code under backend/sqlite/, grouped by role: session/, lane/, objects/, metadata/, history/, storage/, acquisition/, lifecycle/. Today 12 of 17 files in src/history/ and storage/{publication,merge_records,acquisition} call rusqlite directly. No behaviour change.
  2. backend/sqlite/snapshot.rs: one consistent copy of a live Store to a file (SQLite backup API). Push uses it. Land this early; part b depends on it.
  3. backend/postgresql/ behind a Cargo feature, with the same role folders: TLS connection, profile (synchronous_commit), schema (configurable schema name), lane (SELECT … FOR UPDATE on the head row with lock_timeout; no SERIALIZABLE, no retry), objects (pack slices as bytea with STORAGE EXTERNAL, COPY BINARY before the lane, ranged reads with substring), metadata, history, storage.
  4. postgresql/import.rs: copy a SQLite Store into PostgreSQL. Init stays SQLite only, so a PostgreSQL Store gets its data from an Init or a pulled Store.
  5. Later in this area, not this phase: the host SQLite cache that PostgreSQL compatibility for the global storage #420 found necessary (postgresql/cache/), measured against SQLite.

Prior work: #420 and its study on branch issue420-postgresql.

b. Remote storage: MinIO first

Code: a new crate engine/crates/layerfs-remote-storage, plus layerfs-store-service/src/remote/.

  1. Crate core: driver.rs (limits, upload_file, download_file, stat, delete), limits.rs (max file, max single request, piece size), pieces.rs (split a file larger than a provider allows into pieces plus a sha256 manifest), error.rs.
  2. local_folder/ for tests.
  3. s3_api/: revive the retired one-attempt SigV4 client (da0fa3a120^:core/crates/layerfs-s3) and add multipart upload (create, parts, complete, abort) and ranged download. MinIO runs plain HTTP locally, so TLS can wait for R2 and AWS.
  4. minio/: the first provider.
  5. layerfs-store-service/src/remote/: named remotes (a credential reference, never the secret), push (snapshot → size check against the remote's limits → upload), pull (download to a temporary file → sha256 check → the existing sealed-Store install).

Design adjustments (2026-10-29, after reviewing the LayerFS Connect Storage page)

These keep the engine seam, the row-lock lane and whole-file remotes. They change how projects and remotes are organized around them.

  1. One Store holds many projects; a home holds many Stores (owner, 2026-10-29). A project is a named layer_stack inside a Store; the format already scopes Branches, Layers and Commits by stack, and packs are Store-wide. Stores can be local SQLite files (one volume each), PostgreSQL schemas behind many named connections, or SQLite Stores pulled from remotes. Remotes are defined once per home and hold Store files at <remote prefix>/<store>/store.sqlite, plus piece manifests. Push, pull and clone work on a whole Store, so on all of its projects together.
  2. Push and pull state lives outside the Store. Last push and pull (Store, remote, time, sha256, size) go in a small file in the home, keyed by Store, because a pull replaces the Store file.
  3. The home lists its Stores (stores.json): each Store's kind, location (volume, or PostgreSQL connection and schema) and remote binding. Each PostgreSQL Store is one schema holding all of its projects. On PostgreSQL the lane locks the head row being published, so projects in one Store don't queue behind each other. On SQLite they share the Store's single writer.
  4. Pull refuses while anything has the project's Store open: a daemon, a mounted Workspace, or an SDK or worker handle. On macOS the worker keeps a Store open for a quiet minute after use. Download to a temporary file on the Store's filesystem, so the swap is one rename.
  5. Push stages its snapshot beside the Store, inside the Docker volume on macOS. It checks for one Store-sized copy of free space first.
  6. The worker container must reach MinIO and PostgreSQL. On macOS, Store calls run in the home's Linux worker container, so endpoints go through host.docker.internal or a shared Docker network, never the host's localhost. Credentials reach the worker as LAYERFS_REMOTE_* settings when it starts. LayerFS stores a credential reference, never the secret itself.
  7. No HTTP Store server. Connect's worker connects to PostgreSQL directly. D12/D14 (cloud sandboxes over HTTP) stay open, and the rule against a Store server stands.

c. Store-first API layer and Connect wiring

Session c owns the LayerFS side:

  • Init a project into an existing live Store (a new layer stack through the writer lane), and list and remove a Store's projects.
  • runtime.store.* and the other runtime calls in section 6 of the doc.
  • The binding, and forwarding through the worker.

The coordinating session owns Connect. The details below were written before the Store-first change; where they differ, the doc wins.

LayerFS, on this branch. Neither a nor b covers this layer.

  • layerfs-api and the layerfs-node binding:
    • Remote calls: remote.add, remote.list, remote.remove, remote.status.
    • project.push(remote) and project.pull(remote), run as background jobs like Init (start_* and get_*).
    • Open a Store by { postgresql: { url, schema, synchronous_commit } }.
    • project.import_to_postgresql.
  • @layerfs/runtime:
    • runtime.remote.*.
    • project.list and project.get_state report each project's Store location and its last push and pull.
    • The macOS and Windows worker forwards the new calls.
  • Rebuild the worker's sealed Linux build with the postgresql and minio features on.

LayerFS Connect (Ephemeral-AI-Lab/layerfs-connect, on main):

  • Host:
    • New host/storage.js holds remotes, push and pull jobs, and the PostgreSQL connection.
    • New routes in host/server.js: GET /api/storage, POST and DELETE /api/storage/remotes[/:name], POST /api/projects/:id/push and /pull, GET /api/sync-jobs/:id, POST /api/storage/postgresql, POST /api/projects/:id/import-postgresql.
    • Secrets stay in Connect: the Keychain, or an owner-only settings file. They never go into the LayerFS home.
  • Types: StorageView, RemoteInfo and SyncJob in src/shared/host-api.ts.
  • UI:
    • "Add storage" opens a MinIO form: endpoint, bucket, prefix, access key, secret.
    • The MinIO row becomes live and opens remote-detail.tsx: the bucket, which projects have a copy, last push and pull, and Push and Pull with progress.
    • The "Local only" card shows "Copied to MinIO" once a copy exists, and "Sync with..." is enabled.
    • The PostgreSQL row opens postgresql-detail.tsx: the connection, its projects, and "Import a project".
    • The PostgreSQL row says "Live, direct connection" instead of "Live over HTTP".
    • The Projects table gains Storage and Copies columns.
    • Google Drive, S3, R2 and Folder stay "Not available yet".
    • docs/ui/layerfs-storage.md is updated to match.
  • Which LayerFS code Connect runs: Connect links @layerfs/* from the connect-main worktree. To try this branch before it merges, set LAYERFS_CONNECT_API to this worktree's api/.
  • Feature detection: until the API layer lands, Connect checks whether the new runtime calls exist and keeps the current "Not available yet" state when they don't.

Ownership in the shared worktree

  • Session a owns engine/crates/layerfs-persistence/ and the open path in layerfs-store-service/src/store/.
  • Session b owns engine/crates/layerfs-remote-storage/ and layerfs-store-service/src/remote/.
  • Root Cargo.toml, Cargo.lock and layerfs-store-service/src/lib.rs: edit only your own lines, stage only your own hunks, and commit them promptly.
  • Each session commits only the files it changed. No rebase, reset, stash or branch switch.

Done when (this phase)

  • One project runs on SQLite and on PostgreSQL with identical object, root and Commit ids.
  • One Store holds two projects; a second Store and a PostgreSQL Store exist side by side.
  • A SQLite Store pushes to MinIO and is cloned into a fresh Store with identical history.
  • LayerFS Connect's Storage page lists every Store with its projects and drives push, pull, clone and import.
  • A project pushes its Store to MinIO and pulls it into a fresh location with identical history.
  • Every commit records production LOC per the repository rule. Deleted and moved tests are listed.

Related board items: layerfs-connect #11 (A), #27 (A1), #28–#31 (A2–A5), #12 (R), #37 (R2), #38 (R3), #39 (R4), #40 (R5), #67 (D1), #16 (D4). Related PR: #495 (row bundles for incremental push).

Activity

  1. yifanxuaaa commented on Oct 29, 2026

    @yifanxuaaa
    ContributorAuthor

    Updated the description with Design adjustments (home-level remotes, push and pull state outside the Store, where each project's Store is recorded, pull exclusivity, snapshot staging space, worker networking, no HTTP Store server) and a new part c. API layer and Connect wiring. Session b was sent items 1–6 directly. Progress so far: 514a79f (snapshot) and a9903e1 (SQLite code grouped under backend/sqlite).

  2. yifanxuaaa commented on Oct 29, 2026

    @yifanxuaaa
    ContributorAuthor

    Area R (remote storage, MinIO first): done for this phase — session b

    Draft PR #507 (shared with area A). Commits 5586d8f6af, b55e317c6e, 9c6ebef636. Description: core/docs/architecture/86-remote-store-files.md.

    What was built

    • engine/crates/layerfs-remote-storage (no first-party deps; only sha2, already locked): driver.rs (limits, upload_file of a file or span, download_file whole/ranged into a writer, stat, delete), limits.rs, pieces.rs, error.rs, credentials.rs; features local_folder, s3_api, minio.
    • s3_api/: the retired one-attempt SigV4 client revived, plus streamed bodies, canonical query strings, multipart (create, equal parts 5 MiB–5 GiB, ≤10,000 parts, complete; one abort on any failure) and ranged GET.
    • minio/: 50 TiB object, 5 TiB single PUT, 64 MiB default parts; LAYERFS_REMOTE_* settings or a credential reference.
    • layerfs-store-service/src/remote/: settings.rs (named remotes, credential reference only), push.rs (OpenedStore::snapshot from session a → limits check → upload with sha256), pull.rs (download to store.sqlite.pulling → sha256 → opens with the project's binding → rename over the local Store).

    Checks

    Check Where Result
    layerfs-remote-storage --test folder_pieces (5) macOS pass
    layerfs-remote-storage --test minio --include-ignored (7) against pinned MinIO macOS + Docker pass
    layerfs-store-service --test remote_settings (2) macOS pass
    layerfs-api --test remote_store: folder_round_trip + minio_round_trip Linux (Docker) pass
    Clippy -D warnings on the touched packages Linux + macOS clean

    MinIO image: cgr.dev/chainguard/minio@sha256:4692462f35d97d7e82c30371d82f057703c5d9489bcae726010594c812f2d285 (RELEASE.2026-09-22T19-25-18Z). The community repo is archived; the image is used only for tests (tests/minio_service.sh).

    The MinIO tests cover:

    • multipart above the part size (create, 3 parts, complete);
    • a refused part, aborted with no parts left;
    • a lost part reply: Uncertain, part sent once, then aborted;
    • a failed abort: reported PartsLeft and listed by pending_uploads.

    The Store round trip pushes a 12 MiB+ Store (multipart) with three Branches and four Commits, then pulls it into a fresh volume. Branches, Commit lists and file reads are identical. Also checked:

    • a Commit made after the push is not in the pulled Store;
    • an open Store is not replaced, and a closed one is;
    • another project's binding is refused.

    Decisions and deviations

    1. "No opener holds it" is decided by SQLite's -wal/-shm sidecars. On Linux they exist exactly while the Store is open, and pull only runs on Linux behind the kernel guard. There's no SQLite code in store-service. The check doesn't stop a new opener from arriving before the rename: the caller stops the project's daemons and SDK processes first.
    2. Pull follows the install rules in install.rs: kernel guard on the volume, run as the daemons' user (uid 0). installer_user became pub(crate). It doesn't call install_local itself, because that checks for an initial Branch with no Commit, and a pulled Store has history.
    3. One TCP connection per request (Connection: close) rather than keep-alive. A failure then can't leak into the next request, and an abort after an uncertain part gets a fresh connection. Socket I/O has per-operation timeouts instead of one deadline per request, so multi-GB parts work.
    4. Each body's sha256 is signed (computed before the request), so MinIO rejects changed bytes. The cost is reading each part twice.
    5. Pieces are uploaded before the manifest, which goes under the Store's own name. Stale pieces from an earlier, larger push are removed afterwards. With no compare-and-swap, a pull racing a push fails its sha256 check.
    6. Settings are a caller-placed text file (Remotes::load/save). There are no SDK methods or layerfs command for remotes yet.
    7. The MinIO service tests are #[ignore]d and run explicitly against tests/minio_service.sh. tools/worktree.py cargo runs Linux tests with --network none, so I ran the Linux MinIO round trip by starting the launcher-built test binary in a container on MinIO's Docker network. Same target volume and build image.
    8. Tooling: the crate was added to the boundary guard's layer map and to production_loc.py's engine list.

    Not in this phase

    Open item for area A

    After a9903e1286, the boundary guard reports backend/sqlite/lifecycle/file_control.rs as unsafe outside the audited path. Session a was told.

    Production LOC (R commits): 102997 → 104696 (+1699, new crate) → 105080 (+384, store-service remote/) → 105080 (0).

  3. yifanxuaaa commented on Oct 29, 2026

    @yifanxuaaa
    ContributorAuthor

    Owner direction (2026-10-29): one Store holds many projects, and a home holds many Stores (local SQLite Stores, PostgreSQL connections, remote sources). Design doc committed as 69e2909: core/docs/issues/506/README.md. Push, pull and clone now work per Store, and each PostgreSQL Store is one schema. Part c (Store-first API layer) goes to a third session. Progress so far: 514a79f snapshot, a9903e1 SQLite code grouped by role, 5586d8f remote drivers (folder, MinIO), b55e317 push/pull, 9c6ebef remote docs.

  4. yifanxuaaa commented on Oct 29, 2026

    @yifanxuaaa
    ContributorAuthor

    Area R: Store-first adjustment (6ad83a6027) — session b

    Following the parent session's design adjustment and the owner's correction: push and pull now work on a Store, with every project in it. A home can hold many Stores.

    Request Built Difference
    Remotes once per home; key <prefix>/<store>/store.sqlite HOME/remotes (Remotes::home_path); store_object(name) gives NAME/store.sqlite, plus pieces Store names are one segment of [A-Za-z0-9._-], at most 128 characters, not starting with a dot
    Push/pull state outside the Store, keyed by Store remote::state: HOME/remote-state/NAME holds remote, direction, time, sha256 and size push and pull don't take the home, so the caller saves a Transfer after each completes. This keeps them usable without a home
    Pull refuses while a daemon, mounted Workspace or SDK/worker holds the Store; temporary file on the same filesystem Refused before the download and again before the rename. The error names those openers and the macOS worker's quiet minute. The download is store.sqlite.pulling beside the Store, so the swap is one rename Equivalent, not by listing holders: every opener holds SQLite's -wal/-shm, including the macOS worker, which runs in a Linux container. Nothing stops a new opener between the check and the rename
    Push stages beside the Store; free-space check with a clear error push(opened, store, driver, name) stages store.sqlite.pushing beside the Store. PushFailure::Space names the directory, bytes needed and bytes free The amount needed is the main file + WAL plus the Store's 64 MiB write headroom (STORE_HEADROOM_BYTES), so a push can't starve the live Store. No automated test (it needs a full filesystem)
    Any endpoint URL, no localhost assumption Any host name (container names with _), IPv4 or bracketed IPv6, optional trailing /. minio_service.sh also prints the in-network endpoint The Linux MinIO round trip now reaches MinIO by container name (http://lfs506-minio:9000)

    Checks:

    • macOS: remote-storage minio (7, against MinIO), folder_pieces (5), store-service remote_settings (3).
    • Linux: remote_store folder and MinIO round trips, now with two Stores on one remote, push/pull state recorded, and an open Store refused before any download.
    • Clippy -D warnings on Linux and macOS.

    Production LOC: 105080 → 105282 (+202).

  5. yifanxuaaa commented on Oct 29, 2026

    @yifanxuaaa
    ContributorAuthor

    Area R: clone (7247d277cd) — session b

    This adds clone for the Store-first API layer (session c), following the design doc (core/docs/issues/506/README.md §3, §6).

    • remote::clone_store(driver, store_name, destination, cursor_key) pulls a Store into a new Store.
      • The destination must not exist yet. The file is hard-linked into place, so a Store that appears there in the meantime is never replaced.
      • The binding key is read from the remote copy and returned.
      • cursor_key is the new Store's own authority. This works because opening a Store checks only the binding.
    • Push records the binding as layerfs-binding metadata (hex). The binding isn't secret; the cursor key never leaves its holder. A pull now refuses a remote copy with another Store's binding before downloading anything.
    • The push precondition comes from §3's two-machine flow. Expected::{Any, Absent, Sha256}, taken from the home's last transfer, must match the remote copy; otherwise push refuses with RemoteMoved. It's checked before the snapshot and again just before the upload.
      • Not atomic. The owner requires no compare-and-swap, so a push by another machine between the check and the upload isn't detected.

    Checks: on Linux, the folder and MinIO round trips now include a clone with identical history, a refused clone over an existing Store, refused pushes after the remote moved (nothing uploaded), and a refused pull of another binding. Clippy -D warnings passes on Linux and macOS.

    Production LOC: 105282 → 105436 (+154).

  6. yifanxuaaa commented on Oct 29, 2026

    @yifanxuaaa
    ContributorAuthor

    Area A (persistence): fix record, session a

    Branch issue506-store-backends-remote, PR #507. Six commits, all on top of b's work:

    Commit What Production LOC
    514a79fdcd Snapshot of a live SQLite Store (online backup API, one step) 102831 -> 102950 (+119)
    a9903e1286 A1: SQLite code grouped by role under backend/sqlite/, no behaviour change 102950 -> 102997 (+47)
    c94749e591 A2: PostgreSQL engine behind one seam (feature postgresql) 105436 -> 107114 (+1678)
    ee7cd91365 A4: import a SQLite Store into PostgreSQL; identical-ids test 107114 -> 107544 (+430)
    f84f1d25f5 Boundary-guard fix: entry files stay declarative; guard names the moved FFI path 107544 -> 107547 (+3)
    3092604cb0 rustfmt the PostgreSQL open test (test only) 107547 -> 107547 (0)

    The gap between a9903e1286 and c94749e591 is b's remote commits. LOC is from tools/production_loc.py, first parent vs. commit tree. No tests were deleted or moved.

    Layout at f84f1d25f5

    engine/crates/layerfs-persistence/src/backend/ only, using tools/production_loc.py --files:

      backend/                                    [6815]
        admission.rs                                  118   (moved from sqlite/lane/)
        mod.rs                                         11
        records.rs                                    158
    +   seam.rs                                        35   (new: Engine, Mechanics, HistoryStatements)
    +   postgresql/                                [1560]   (new, #506 A2/A4)
    +     import.rs                                   399
    +     mechanics.rs                                 51
    +     mod.rs                                        9
    +     schema.rs                                    88
    +     history/                                   [13]
    +       mod.rs                                      1
    +       statements.rs                              12
    +     lane/                                      [56]
    +       lane.rs                                    54
    +       mod.rs                                      2
    +     metadata/                                 [169]
    +       mod.rs                                      2
    +       reads.rs                                  123
    +       writes.rs                                  44
    +     objects/                                  [208]
    +       mod.rs                                      2
    +       publish.rs                                 95
    +       read.rs                                   111
    +     session/                                  [468]
    +       connection.rs                              91
    +       mod.rs                                      5
    +       profile.rs                                 59
    +       query.rs                                   31
    +       rows.rs                                    62
    +       transaction.rs                            220
    +     storage/                                   [99]
    +       mod.rs                                      1
    +       provider.rs                                98
        sqlite/                                    [4933]   (A1: grouped by role)
    +     mechanics.rs                                 51   (new: SQLite behind the seam)
          mod.rs                                       10
          schema.rs                                   295
    +     snapshot.rs                                 100   (new, #506)
          acquisition/                             [1527]
            accounting.rs                              74
            cleanup.rs                                190
            entry_windows.rs                          236
            metadata.rs                                41
            mod.rs                                     11
            provider.rs                               246   (from src/storage/acquisition/)
            reads.rs                                  364
            root_windows.rs                            56
            statements.rs                              86
            writes.rs                                 223
          lane/                                     [124]
            lane.rs                                    42
            mod.rs                                      3
            writer_admission.rs                        79
          lifecycle/                                [311]
            file_control.rs                            30
            headroom.rs                                13
            mod.rs                                      9
            reclamation.rs                             48
            seal.rs                                    78
            seal_allocation.rs                        133
          metadata/                                 [316]
            allocation.rs                              19
            locations.rs                              129
            mod.rs                                      5
            policy.rs                                  18
            signatures.rs                              77
            values.rs                                  68
          objects/                                  [561]
            mod.rs                                      7
            publish.rs                                135
            read.rs                                    65
            selection.rs                               83
            unit_io.rs                                 70
            unit_layout.rs                             37
            units_publish.rs                           34
            units_read.rs                             130
          session/                                 [1165]
            connection.rs                             207
            mod.rs                                      8
            prepared.rs                               166
            profile.rs                                104
            query.rs                                  206
            rows.rs                                    57
            statement_work.rs                          33
            thread_work.rs                             84
            transaction.rs                            300
          storage/                                  [473]   (merge_records from src/storage/)
            merge_records.rs                          472
            mod.rs                                      1

    What is in place

    • Snapshot. Handles::snapshot, StorageProvider::snapshot and store-service OpenedStore::snapshot.
      • It copies a live Store on its own read-only connection in one backup.step(-1).
      • The new file is created exclusively. It is WAL, truncate-checkpointed and closed with no sidecar.
      • Session b's push uploads it.
    • Seam. History, Metadata/Objects bounds and publication validation stay shared. They reach an engine only through Mechanics and HistoryStatements (backend/seam.rs).
      • The shipped SQLite history SQL runs on PostgreSQL unchanged, except hint_insert_binding and statistics_totals.
    • PostgreSQL engine (postgres =0.19.14, TLS through postgres-native-tls/native-tls; all optional behind postgresql).
      • BackendSelection::Postgres(PostgresSelection { url, schema, synchronous_commit, lock_timeout_ms }).
        • synchronous_commit can be off (default), local, remote_write, on or remote_apply.
        • It is applied with SET LOCAL in the same round trip as BEGIN.
        • Debug output redacts the URL.
      • Transactions: reads are REPEATABLE READ READ ONLY; writes are READ COMMITTED. Never SERIALIZABLE, never retried.
      • Lane: a lock timeout is Busy. A lost COMMIT reply is Unknown and quarantines the session.
      • Packs: bodies go in with COPY BINARY before the lane. They are bytea STORAGE EXTERNAL, and ranged reads use substring.
      • Opening validates the schema version and binding. It never creates or migrates anything.
      • Store-service has StoreService::open_postgres. The bridge has ProviderKind::Postgres (wire 2), which an install manifest refuses.
    • Import. PostgresHandles::import(sqlite_config, selection, binding, cursor).
      • It creates the schema and copies every table and pack body (any of the three SQLite pack layouts) in one PostgreSQL transaction, over one SQLite snapshot.
      • It keeps object ids, roots, Commit ids and inode serials.
      • Init stays SQLite only.
    • Identical ids (layerfs-persistence/tests/postgresql_identical_ids.rs).
      • Setup: a Store with two Commits on a Branch is imported into PostgreSQL 17.11.
      • Image: postgres@sha256:639ab7ceb90e13123085b741fb31ef493fba25463002f6da665352e7b534b652.
      • The test publishes the same third Commit on both engines and compares object, root and Commit ids, plus reads.
      • A second test checks that a held lane makes another process's write Busy after lock_timeout.
      • The store-service test postgresql_open.rs opens the imported Store through StoreService::open_postgres.
      • The tests run only when LAYERFS_TEST_POSTGRES_URL is set; otherwise they report as skipped.

    Deviations (my calls, under the owner's full authority)

    1. The lane is per Store, not per head row. Every write locks one store_lane row FOR UPDATE. This matches SQLite (one writer per Store), so projects in one PostgreSQL Store still queue behind each other. README §2.2 asks for per-head locking.
      • Per-head locking is the next step. It needs FOR UPDATE on head reads, plus a review of name checks, the scope-allocator CAS and lineage writes. These currently rely on the single writer.
      • A held publication lane owns the row on a second connection. That thread's nested writes skip the wait.
    2. Packs are stored whole, not sliced. Each pack body is one bytea value (EXTERNAL, so TOAST slices it and substring reads only the range). Splitting into slice rows can follow if a pack goes over PostgreSQL's 1 GB value limit.
    3. Not on PostgreSQL in this phase:
      • Merge records: merges are refused, so merge gives BackendUnavailable.
      • Init and acquisition.
      • Seal, snapshot and reclamation, since there is no file.
      • The reader counters in store diagnostics are not observed.
    4. The PostgreSQL schema has its own version (1). Import rewrites schema_version, and the SQLite versions 7–12 do not apply there.
    5. No kernel guard on PostgreSQL. StoreService::open_postgres skips the Linux kernel/filesystem guard that SQLite opening keeps, because the Store is not a local file.
    6. The import opens the SQLite source writable. Handles::open_read_only cannot open a sealed or snapshot Store (see finding 1). The import does not write to it; it reads from one consistent read transaction.
    7. What stays shared. src/history/ and src/storage/publication.rs hold no SQL, so they stay shared instead of moving under backend/sqlite/. Writer admission moved up to backend/admission.rs, where both engines use it.

    Findings

    1. Handles::open_read_only cannot open a closed WAL Store whose -shm is gone (a sealed Init output or a snapshot): it fails with Uncertain. Opening read-write once fixes it. This matters for any read-only consumer of pulled or cloned files; session b's clone opens writable.
    2. PostgreSQL needs explicit casts on parameter arithmetic ($1::bigint + $2::bigint). Otherwise the shared history SQL fails with operator is not unique. The changed statements carry the casts.

    Checks (macOS, at f84f1d25f5)

    • cargo +1.85.1 fmt --all --check. At f84f1d25f5 it missed tests/postgresql_open.rs, which CI reported; 3092604cb0 formats it.
    • clippy -D warnings on persistence, bridge and store-service, with and without postgresql.
      • Store-service is clean at 740fd07615. An earlier useless_conversion report came from an in-between edit of b's that never landed.
    • Persistence suite with and without the feature: with the feature and the container, 121 tests passed and none failed.
    • Store-service and bridge tests.
    • cargo check --workspace.
    • tools/check_product_boundary.py PASS, plus its 34 self-tests.
    • Ruff format.

    Every test command ran under a 120 s wall limit. SQLite tests used the Disposable profile only.

    Gaps and next

    • The persistence and store-service suites have not run on Linux.
      • Session c's Linux run (#506 comment) built the postgresql feature on Linux, native-tls included.
      • It covered import, attach through PostgresHandles::binding, and remove_layerstack against PostgreSQL 17, in the Linux runtime image on Docker Desktop, not a bare Linux host.
    • The host SQLite cache for PostgreSQL Stores (backend/postgresql/cache/, PostgreSQL compatibility for the global storage #420) is next. Without it a Commit takes several round trips.
    • Per-head lane locking (deviation 1): accepted for phase one by the coordinating session; it stays an open item, not this phase's work.
    • Merges on PostgreSQL, and sliced packs if needed.
    • The new PostgreSQL dependencies bring a wasm-bindgen family bump in Cargo.lock, through getrandom/js-sys. It is harmless on native targets; noted for review.
  7. yifanxuaaa commented on Oct 29, 2026

    @yifanxuaaa
    ContributorAuthor

    Area A: the persistence API that session c builds on (at 740fd07615)

    Everything PostgreSQL is behind the Cargo feature postgresql (persistence, and store-service's feature that forwards to it).

    Select and open a PostgreSQL Store. A Store is one schema holding all of its projects.

    • Selection: PostgresSelection::new(url, schema).
      • .with_synchronous_commit(SynchronousCommit::{Off (default), Local, RemoteWrite, On, RemoteApply}).
      • Public field lock_timeout_ms (default 30000).
      • It goes into BackendSelection::Postgres(..).
    • PostgresHandles::open_writable(&selection, binding, cursor_key) and open_read_only(..).
      • The result is PostgresHandles { storage: Arc<PostgresStorage>, history: HistoryProvider }.
      • Opening validates; it never creates or migrates.
    • PostgresHandles::binding(&selection) -> Result<Vec<u8>, PersistenceError>: the binding key a LayerFS schema records, for attaching a schema this home did not import.
    • Store-service: StoreService::open_postgres(..), the same open path as SQLite. OpenedStore::snapshot there is SQLite only.

    Import. PostgresHandles::import(sqlite_config, &selection, binding, cursor_key) -> Result<ImportReceipt { tables, pack_bytes }, _>.

    • The target schema must not exist.
    • It copies the whole SQLite Store, every layer stack, in one PostgreSQL transaction, keeping every id.

    Remove a project. HistoryCatalog::remove_layerstack(stack) -> HistoryResult<()>, on both engines.

    • One write transaction deletes the stack's Layers, Branches and Commits, their lineage, and their logical bytes from the totals.
    • Packs, objects and the allocator scope stay; reclaiming packs is layerfs rm/compact: retire Branches and reclaim unreachable Store history #411.
    • An unknown stack is Missing(LayerStack). Other providers refuse it as Unsupported.
    • The write holds the writer for time that grows with the removed project's history. Batching can follow if that matters.

    Snapshot. Handles::snapshot(dest) and OpenedStore::snapshot(dest) -> SealedStore, SQLite only.

    None of these signatures is planned to change. As recorded in the fix record above, the PostgreSQL lane still locks per Store, not per head row.

  8. 18 remaining items

  9. added theissue type on Oct 31, 2026
  10. added this to the LayerFS 2.0 milestone on Oct 31, 2026
  11. added 4 commits that reference this issue on Oct 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

new featureA new user-facing capability

Type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions