Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
30 changes: 13 additions & 17 deletions .agents/skills/layerfs-js-sdk/references/methods/branch.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,14 +125,10 @@ reply, never out of the merge. For an incomplete summary, page

## `branch.publish`

`branch.publish(project, branch, commit?, { expected_base, expected_head, force })`
`branch.publish(project, branch, commit?)`

Makes `main` show the Branch's head, or the given Commit, by adding a Layer
to `main`. It refuses (`HeadMoved`) when `main` gained a Layer since your
Branch forked, so another agent's work is never dropped by accident.
`expected_base` (a Layer id) and `expected_head` (a Commit id) refuse unless
`main` and your Branch are still where you read them; `force: true`
overwrites `main` on purpose.
to `main`.

```bash
layerfs branch publish shop/api work
Expand All @@ -147,29 +143,29 @@ layerfs branch publish shop/api work

| `kind` | Meaning |
| --- | --- |
| `Added` | `main` now shows that Commit; `published_layer` is the new Layer; `overwrote` lists `main` Layers that a `force` publish skipped |
| `UpToDate` | That Commit is already published (`{ kind, layer }`), even if `main` moved on since |
| `NoChanges` | `main` already holds that tree (`{ kind, head_layer }`) |
| `Added` | `main` now shows that Commit; `published_layer` is the new Layer; `overwrote` lists `main` Layers that were skipped |
| `UpToDate` | `main` already shows it (`{ kind, layer }`) |
| `NoChanges` | Nothing to publish (1.1.0 shape; not seen in this pass) |

| Code | Meaning | Do |
| --- | --- | --- |
| `NoCommit` | The Branch has no Commit yet | Commit something first |
| `NoSuchBranch` | No such Branch | Check the name |
| `HeadMoved` | `main` moved since your Branch forked (or since `expected_base`), or the Branch head is not `expected_head`; `detail` has `expectedBase`/`actualBase` or `expectedHead`/`actualHead` | Merge with `main` into a new Branch (below), then publish that |
| `InvalidInput` | `commit` is not a full Commit id, or an expected id names the wrong kind | Pass a full id |
| `HeadMoved` | `main` moved during the call | Merge `main` in (below), then publish again |
| `InvalidInput` | `commit` is not a full Commit id | Pass a full id |

**Publishing replaces `main`'s view** with your Commit, so it is refused when
`main` has Layers your Commit doesn't contain. Publishing your own Branch
again at milestones is fine. If someone else published since you forked,
merge `main` in (a merge makes a new Branch), check the result, then publish
the merged Branch:
**Publishing replaces `main`'s view** with your Commit. Layers published to
`main` after your Branch's base that your Commit doesn't contain are skipped:
they stay in history and `overwrote` lists them. If someone else may have
published since you forked, merge `main` in first, check the result, then
publish the merged Branch:

```js
const P = "shop/api";
const m = await layerfs.branch.merge(P, "work-release", "work", "main", { message: "merge main before publish" });
// mount m.branch in a Sandbox and run the tests here, then:
const p = await layerfs.branch.publish(P, m.branch);
// a merge with main publishes as a fast-forward: nothing is skipped
if (p.kind === "Added" && p.overwrote.length) console.log("warning: skipped", p.overwrote);
```

## `branch.diff`
Expand Down
47 changes: 33 additions & 14 deletions .agents/skills/layerfs-js-sdk/references/methods/project.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,8 @@ project.init(project, folder, { timeout, on_progress })
project.list(store?, { page_size, page_pos })
project.get(project)
project.export(project, destination, { branch, on_progress })
project.remove(project)
project.delete(project)
project.remove_file(project, path)
```

## `project.init`
Expand Down Expand Up @@ -119,23 +120,41 @@ The project's record: its Store, the Store's kind and machine.
{"project":"shop/api","store":"shop","name":"api","kind":"sqlite","machine":"native","created":1793574061265}
```

## `project.remove`
## `project.delete`

`project.remove(project)`
`project.delete(project)`

Removes one project from its Store. Its Branches, Commits and `main` go with
it; there is no undo. The live result:
Retires one project in its Store and removes its home entry (spec §4). The
project disappears from `project.list` and `project.get`; its Layers,
Branches, Commits and stored content stay until a later `store.compact`, and
the Store stays addressable. The live result:

```json
{"stack":"3104401f11fb…","packs_kept":true,"project":"shop/api"}
{"kind":"Removed","project":"shop/api","deleted_ms":1793574061265,"home_entry_removed":true}
```

`packs_kept: true`: the project's records are gone, but its stored content
stays in the Store file. Removing a project from a PostgreSQL Store is
`Unsupported` in this build. While a Sandbox of the project is recorded it is
refused `InUse`; `detail.related` lists each as `{ kind: "sandbox", name }`
(close them first). To delete a whole Store with every project in it, use
`store.remove`.
The Store marker is the commit point. If the home entry cannot be removed the
call still succeeds with `home_entry_removed: false` and keeps a durable
intent, so the same call retried from the same home reuses `deleted_ms` and
finishes the entry. A mount of the project is refused `MountsActive`, another
user or operation `Busy`/`Unresolved`.

An agent working on a project never needs `remove`; it is for whoever owns the
home.
## `project.remove_file`

`project.remove_file(project, path)`

Hides one absolute project path from every node at or below its marker
frontier (spec §4.3). It writes a marker only; no bytes are freed. The path is
normalized (one leading `/` stripped) and must be a canonical absolute POSIX
path. The live result:

```json
{"kind":"Removed","project":"shop/api","path":"docs/old.md","frontier_serial":42,"removed_ms":1793574061265}
```

`kind` is `UpToDate` when no project node is above this path's marker
frontier. Later reads and stats of the path are refused `Removed`; a Commit
after the frontier that writes the path again is visible.

An agent working on a project never needs `delete` or `remove_file`; they are
for whoever owns the home.
10 changes: 7 additions & 3 deletions .agents/skills/layerfs-sdk-observe/references/tracing.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,12 +19,16 @@ Python, direct `layerfs_api::call`) appends one line to
and `LAYERFS_TRACE` / `LAYERFS_TRACE_STEP` do not affect it.

```json
{"v":1,"at":1793000004120,"ms":212,"client":"node","argv":["file","write","demo/app","main","/index.ts","<5321 bytes>"],"outcome":"ok","kind":"Committed","commit":"<66 hex>","tag":"dsh:s_4d2a:a91e"}
{"v":1,"at":1793000004120,"ms":212,"client":"node","argv":["file","write","demo/app","main","<8 bytes>","<5321 bytes>"],"outcome":"ok","kind":"Committed","commit":"<66 hex>","tag":"dsh:s_4d2a:a91e"}
```

It keeps 1.1.0's redaction rule (values cut to 200 characters; contents,
JSON, commands, messages, edit strings, passwords and URLs as byte counts)
and drops the daemon telemetry: no `sizes`, `scope` or `daemon`. `argv` is
JSON, commands, messages, edit strings, passwords and URLs as byte counts;
a path argument as the byte count of its normalized form, one leading `/`
stripped, so `/a` and `a` are both `<1 bytes>`; a glob or grep `pattern` by
its own bytes) and drops the daemon telemetry: no `sizes`, `scope` or
`daemon`. A refusal is logged as its code only: its message and `detail` are
never traced, so a `Removed` path stays out of the line. `argv` is
the call as CLI words (`argv[0]`/`argv[1]` are the family and call),
`outcome` is `"ok"` or the refusal code, `kind` and `commit`/`layer` come
from the result, `client` is `cli`/`node`/`python`/`rust`, and `tag` is the
Expand Down
203 changes: 203 additions & 0 deletions .github/workflows/store-ext4.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
# Store lane: the Store suites, on a filesystem that carries SQLite's locks.
#
# A Store opens only on a directory that is private, owned by the test user and
# on a local filesystem whose locks reach every opener: ext4, xfs or btrfs on
# Linux (engine/crates/layerfs-store-service/src/kernel_guard.rs). The Store
# suites keep their Stores in TMPDIR and in $CARGO_TARGET_DIR/tmp, so both must
# be on such a filesystem, and an overlay checkout or a tmpfs /tmp is refused.
# tools/ci/test_store_ext4.sh refuses any other directory before cargo starts
# and points TMPDIR, CARGO_TARGET_DIR and CARGO_HOME at the one it is given;
# this lane only provides that directory. See
# docs/architecture/engine/store/testing.md.
#
# It adds no test selection of its own: every target it runs is already in the
# push or nightly lane chosen by tools/ci_tests.py. It runs them under the
# environment contract their guard requires.
#
# Reports only, like the other lanes (AGENTS.md §4): never a required check.
# The runner's /mnt is a separate ext4 data disk while /tmp is tmpfs, so the
# prepare step asserts the filesystem: a runner image whose /mnt is not ext4
# fails there, with its own message, instead of as a guard refusal inside a
# suite.
name: store-ext4

on:
push:
branches: [main]
paths:
- "core/**"
- "engine/**"
- "control/**"
- "api/crates/**"
- "!api/crates/layerfs-node/**"
- "Cargo.*"
- ".config/**"
- "tools/ci/**"
- ".cargo/**"
- ".github/workflows/store-ext4.yml"
- "!**/*.md"
pull_request:
# off for pull requests into layerfs_v2_0_0, like the other lanes
branches-ignore: [layerfs_v2_0_0]
paths:
- "core/**"
- "engine/**"
- "control/**"
- "api/crates/**"
- "!api/crates/layerfs-node/**"
- "Cargo.*"
- ".config/**"
- "tools/ci/**"
- ".cargo/**"
- ".github/workflows/store-ext4.yml"
- "!**/*.md"
schedule:
# After nightly.yml's 04:23 UTC run; the two lanes never share runner time.
- cron: "47 4 * * *"
workflow_dispatch:

permissions:
contents: read

concurrency:
group: store-ext4-${{ github.ref }}
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always
CARGO_INCREMENTAL: "0"
# CI needs no debugger: smaller objects, much shorter links.
CARGO_PROFILE_TEST_DEBUG: "0"
# Link the test binaries with mold instead of GNU ld, as ci.yml's test job
# does. Scoped to this target, so the repository's aarch64 rustflags
# (.cargo/config.toml) stay.
CARGO_TARGET_X86_64_UNKNOWN_LINUX_GNU_RUSTFLAGS: "-C link-arg=-fuse-ld=mold"
# The harness requires this and validates it before cargo starts. /mnt is the
# runner's ext4 data disk; never /tmp, which is tmpfs there.
LAYERFS_TEST_TMPDIR: /mnt/layerfs-store-tests

jobs:
quick:
name: store ext4 (--quick)
# The push lane: one or two targets per suite, across the four suites.
if: github.event_name == 'push' || github.event_name == 'pull_request'
runs-on: ubuntu-24.04
timeout-minutes: 60
steps:
- uses: actions/checkout@v7.0.1
- run: rustup toolchain install 1.85.1 --profile minimal
- name: Install mold
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends mold
mold --version
- name: Prepare the ext4 test directory
run: |
# /mnt belongs to root on the runner image and the tests run as the
# runner, so hand the directory over before the harness probes it.
sudo mkdir -p "$LAYERFS_TEST_TMPDIR"
sudo chown "$(id -u):$(id -g)" "$LAYERFS_TEST_TMPDIR"
fs="$(findmnt -n -o FSTYPE --target "$LAYERFS_TEST_TMPDIR")"
echo "LAYERFS_TEST_TMPDIR=$LAYERFS_TEST_TMPDIR ($fs)"
df -h "$LAYERFS_TEST_TMPDIR"
case "$fs" in
ext4|xfs|btrfs) ;;
*)
echo "::error title=unsupported filesystem::$LAYERFS_TEST_TMPDIR is on $fs; the Store guard admits only ext4, xfs or btrfs"
exit 1
;;
esac
# No Swatinem/rust-cache here: the harness moves CARGO_HOME and
# CARGO_TARGET_DIR onto LAYERFS_TEST_TMPDIR, so the checkout-scoped cache
# the other lanes restore is never read. The cold build is the price of
# running the suites where their guard admits them.
- name: Run the Store suites (--quick)
run: tools/ci/test_store_ext4.sh --quick

full-postgresql:
name: store ext4 (--full, PostgreSQL 17)
# The scheduled and manual lane: every Store suite, with the PostgreSQL
# cases on. A manual dispatch runs this job only, which is the full sweep.
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: ubuntu-24.04
timeout-minutes: 120
env:
# A libpq connection string, not a URL (testing.md). A service
# container's published port is 127.0.0.1 on the runner. The server is
# scratch -- each PostgreSQL test creates and drops its own schema -- and
# lives only for this job. Setting this adds `--features postgresql` to
# every suite in the harness.
LAYERFS_TEST_POSTGRES_URL: host=127.0.0.1 port=5432 user=postgres dbname=postgres sslmode=disable
# The full-mode doctor calls amcheck's verify_heapam and bt_index_check;
# the step below creates it in this database. The physical doctor's two
# extra servers (an amcheck-free database and a physical standby) are
# not available in this job; see testing.md. Set so the diagnostic test
# asserts relation coverage is Pass instead of only printing it.
LAYERFS_TEST_EXPECT_AMCHECK: "1"
services:
postgres:
# PostgreSQL 17 is the supported version for these cases: the schema
# needs 16 or newer for STORAGE PLAIN, and the suites are qualified on
# 17.x (docs/architecture/engine/store/testing.md).
image: postgres:17
env:
POSTGRES_USER: postgres
POSTGRES_DB: postgres
# Trust authentication: a throwaway server, reachable from this job's
# steps only, and the tests create and drop their own schemas.
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
options: >-
--health-cmd "pg_isready --username=postgres --dbname=postgres"
--health-interval 5s
--health-timeout 5s
--health-retries 20
steps:
- uses: actions/checkout@v7.0.1
- run: rustup toolchain install 1.85.1 --profile minimal
- name: Install mold
run: |
sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends mold
mold --version
- name: Prepare the ext4 test directory
run: |
sudo mkdir -p "$LAYERFS_TEST_TMPDIR"
sudo chown "$(id -u):$(id -g)" "$LAYERFS_TEST_TMPDIR"
fs="$(findmnt -n -o FSTYPE --target "$LAYERFS_TEST_TMPDIR")"
echo "LAYERFS_TEST_TMPDIR=$LAYERFS_TEST_TMPDIR ($fs)"
df -h "$LAYERFS_TEST_TMPDIR"
case "$fs" in
ext4|xfs|btrfs) ;;
*)
echo "::error title=unsupported filesystem::$LAYERFS_TEST_TMPDIR is on $fs; the Store guard admits only ext4, xfs or btrfs"
exit 1
;;
esac
# The service health check gates this job on the server answering. The
# engine never installs extensions: a full-mode audit calls amcheck's
# verify_heapam and bt_index_check and reports every relation as
# Unknown when the extension is absent, while the compaction suite
# asserts a clean full audit. A fresh cluster therefore needs these
# contrib extensions before any suite connects (T-G14). The server is
# scratch, and this is the one database every suite reaches.
- name: Create the PostgreSQL extensions
run: |
set -euo pipefail
# The runner image does not ship psql; the distro client reaches the
# PostgreSQL 17 service container.
sudo apt-get update -qq
sudo apt-get install -y -qq --no-install-recommends postgresql-client
psql --version
psql "$LAYERFS_TEST_POSTGRES_URL" -X -v ON_ERROR_STOP=1 <<'SQL'
CREATE EXTENSION IF NOT EXISTS amcheck;
CREATE EXTENSION IF NOT EXISTS pgstattuple;
SQL
# Fail here, with a readable list, rather than as five Unknown
# relation checks inside a compaction suite.
psql "$LAYERFS_TEST_POSTGRES_URL" -X -A -t -v ON_ERROR_STOP=1 \
-c "SELECT extname FROM pg_catalog.pg_extension WHERE extname IN ('amcheck','pgstattuple') ORDER BY extname"
# The suites own their schemas; the harness reports the mode it took.
- name: Run the Store suites (--full)
run: tools/ci/test_store_ext4.sh --full
Loading