tracker

Shared documents, conflict-safe editing, and a task queue for coding agents — all over MCP. Postgres holds the index and coordination state; content blobs live in local files or any S3-compatible store. Tracker is a small static Go binary with a built-in UI and breezy ops for you and your agents on your personal private network.

Quickstart

Needs Docker Compose, Bash, curl, and Python 3. No Go toolchain or build is needed — it pulls a prebuilt image.

git clone https://github.com/chicagobuss/tracker && cd tracker
cp .env.example .env          # runs as-is; set PGPASSWORD before the first start
docker compose up -d --wait   # Postgres + tracker (--wait blocks until it's serving)
scripts/seed.sh               # a welcome doc + example folio, so it isn't empty
scripts/smoke.sh              # health + a full create -> lock -> write -> read round-trip

That's a working tracker on http://127.0.0.1:8770 — web UI in a browser, markdown index for agents (curl http://127.0.0.1:8770). make up does the same and runs the smoke test for you; make seed seeds it.

Use --wait: without it, compose returns as soon as the container exists, while tracker is still connecting to Postgres and running migrations — so anything you run immediately after (a smoke test, an agent, CI) can race the boot.

The Postgres image only uses PGPASSWORD while it first initializes its data volume. If you change it later, rotate the database password too; changing only .env will stop tracker from connecting.

The seeded welcome document is written for an agent to read: it explains what tracker is, the document/folio/lease/actor model, and how to behave. Point a new agent at the instance and tell it to read welcome — it can bootstrap itself from there. Delete the seed docs once you have real content.

Connect an agent

claude mcp add --transport http --scope user tracker http://127.0.0.1:8770/mcp \
  --header "X-Actor: claude-code-<host>"

Start a new Claude session and ask it to read welcome. If you enabled auth, add --header "Authorization: Bearer <token>" to the command above.

Already using port 8770? Compose will fail with address already in use. Set two lines in .env and re-run docker compose up -d:

PORT=8771
BASE_URL=http://127.0.0.1:8771

Use http://127.0.0.1:8771/mcp in the agent-registration command above.

Defaults are deliberately safe for a first run: loopback-only, blobs in ./data/blobs, auth off. Before you expose it to other machines, read Going multi-machine — it is one setting plus a token.

Common knobs, all in .env:

Want to…	Set
Use a different port (8770 taken)	PORT=8771 and BASE_URL=http://127.0.0.1:8771
Let other machines reach it	BIND_ADDR=<your LAN/Tailscale IP>
Keep blobs in S3/R2/MinIO instead of files	STORAGE_TYPE=s3 + the four required S3_* vars
Gate access behind a token	Set API_TOKENS to a token generated with openssl rand -hex 32

Identity vs. access

Two different things, easy to conflate:

• X-Actor is who did it. Required on every write, recorded as the document author, lease owner, and task claimant. It is self-asserted — tracker takes your word for it. That's the intended design on a trusted network: attribution between cooperating agents, not a security boundary.
• API_TOKENS is whether you may talk to the server at all. It is empty by default, meaning no auth — anyone who can reach the port can read and write.

So X-Actor is not a login, and running without API_TOKENS is fine on loopback or a private overlay network — but it is the only thing standing between an open port and an unauthenticated write API. Turn it on before you expose tracker anywhere you don't fully trust.

Enabling auth (optional)

Generate a token in your shell, then paste its literal output into .env (do not put the $(...) command substitution in the file):

openssl rand -hex 32
# .env
API_TOKENS=<paste-the-output-here>     # comma-separate to issue several

docker compose up -d --wait            # picks up the new .env

Every request except /healthz now needs the token, or it gets a 401:

curl http://127.0.0.1:8770/docs                                  # 401
curl -H "Authorization: Bearer <token>" http://127.0.0.1:8770/docs   # 200

Point agents at it by adding one header to the MCP registration:

claude mcp add --transport http --scope user tracker http://127.0.0.1:8770/mcp \
  --header "X-Actor: claude-code" \
  --header "Authorization: Bearer <token>"

scripts/smoke.sh and scripts/seed.sh read API_TOKENS from .env themselves, so they keep working with no extra flags. Issue a token per agent (API_TOKENS=tok-laptop,tok-ci,tok-server) if you want to be able to revoke one without rotating the rest — but note that a token only grants access; it does not yet pin which X-Actor a caller may claim (see the backlog).

Quickstart variant: blobs in S3 / R2 / MinIO

The default keeps blobs in ./data/blobs, which needs no extra infrastructure. To put them in object storage instead, set STORAGE_TYPE=s3 and the four S3_* vars before the first docker compose up. Everything else is identical — Postgres still holds the index, and it only ever stores the sha256/<hash> key, never the backend location.

Cloudflare R2 (endpoint is your account's, TLS on, no region):

# .env
STORAGE_TYPE=s3
S3_ENDPOINT=<account-id>.r2.cloudflarestorage.com
S3_ACCESS_KEY=<r2 access key id>
S3_SECRET_KEY=<r2 secret access key>
S3_BUCKET=tracker-blobs
S3_USE_SSL=true

AWS S3: S3_ENDPOINT=s3.<region>.amazonaws.com, S3_USE_SSL=true. MinIO / RustFS / anything S3-compatible: S3_ENDPOINT=host:9000, and S3_USE_SSL=false if it's plain HTTP on a private network.

docker compose up -d --wait
scripts/smoke.sh                 # writes a doc, so it proves the bucket works

tracker creates the bucket on startup if it doesn't exist and your credentials allow it; if they don't, create it first and it will just use it. All four vars are required — miss one and tracker refuses to start, naming it, rather than silently falling back to local files.

Already running on local files? Don't hand-copy anything — tracker migrate-blobs does a verified, non-destructive copy and prints the cutover step. See Switching storage backend.

Agents: MCP + skill

tracker speaks MCP natively — the server exposes a Streamable HTTP MCP endpoint at /mcp, so any agent connects with one line of config and zero local code (like Notion's remote MCP server). Add --header "Authorization: Bearer <token>" if you set API_TOKENS.

X-Actor is the agent's identity, stamped on every write. Cursor, Gemini CLI, and anything else that speaks HTTP MCP configures the same way — the tools live in the tracker binary, versioned and deployed with it, so clients can never drift out of sync.

Tools: list_docs (incl. deleted=exclude|only|include), get_doc, get_raw, create_doc, update_doc (lease + version-check + release, for you), lock_status, retag_doc (tags/metadata/title/kind without a content rewrite), soft_delete_doc / restore_doc / hard_delete_doc (hard delete requires confirm equal to the slug), list_tags, list_folios, create_folio, get_folio, get_folio_file, add_folio_file, the task-queue tools (list_tasks, get_task, enqueue_task, claim_task, complete_task), list_actors, and actor_activity.

skills/tracker/SKILL.md is the matching Claude Code skill (copy to ~/.claude/skills/tracker/, then set the base URL at the top) describing when/how to consult tracker.

The old per-machine stdio bridge (mcp/tracker_mcp.py) has been removed — /mcp replaces it, and a second copy of the tool surface is exactly the drift the native endpoint exists to prevent. It's in git history if you need it.

Design

• Leases, not advisory locks. A doc_locks row with a TTL + heartbeat answers "who is writing this right now". A crashed agent's lease auto-expires, so it can never block a doc forever.
• Two-layer write safety. A write requires (a) a live lease the caller holds (X-Lease-Token) and (b) If-Match: <version> optimistic concurrency, so a stale or lease-less write can't clobber.
• Content-addressed blobs. Bytes are stored under sha256/<hash> (immutable, deduped) in either a local directory (STORAGE_TYPE=file) or an S3 bucket (STORAGE_TYPE=s3). Either way agents fetch them from an expiring BASE_URL/blobs/... link, so a content_url is reachable from wherever tracker is and the bucket stays private.
• Task queue. tasks with FOR UPDATE SKIP LOCKED claiming — no two agents grab the same task.
• Workspaces enforced by the database. Separation is a Postgres row-level security policy, not a where clause this codebase has to remember. A request runs with app.workspace set and every table is filtered to it, so a query that forgets its predicate returns nothing rather than another workspace's rows.

Workspaces

A workspace is a hard partition of the store. Documents, tasks and actors in one are invisible from another — separate search results, separate slug namespaces, separate list_tags vocabularies. Point a throwaway experiment at its own workspace and it cannot pollute an agent's context with unrelated work.

curl -X POST localhost:8770/workspaces -H 'X-Actor: me' \
  -H 'Content-Type: application/json' -d '{"name":"greenfield-idea"}'

claude mcp add --transport http --scope user tracker-gf http://127.0.0.1:8770/mcp \
  --header "X-Actor: claude-code-$(hostname -s)" \
  --header "X-Workspace: greenfield-idea"

Names match [a-z0-9][a-z0-9_-]{0,62}. Creating one is free — a row in a registry table, no DDL — so spinning up a dozen costs nothing. An unknown name is a 404 rather than an implicit create, so a typo'd X-Workspace reports itself instead of silently presenting an empty store.

Everything that predates workspaces lives in default, and a request that names none resolves there, so existing agents keep working untouched.

Tools also take an optional workspace argument, so one session can work in another workspace without re-registering the server:

list_docs  {"q": "sequencer", "workspace": "scripture"}
create_doc {"slug": "note", "workspace": "scripture", ...}

Mind where you point a write: a create_doc aimed at the wrong workspace misfiles the doc, and nothing will warn you — the argument is honoured verbatim. It is refused outright when the token is confined, because the argument is caller-supplied and would otherwise hand back the boundary the token exists to enforce. The registry tools (list_workspaces, create_workspace) take no override; the registry is not workspace-scoped.

X-Workspace is a preference; a confined token is a boundary. The header is caller-supplied, so on its own it separates work without securing it — fine for avoiding context pollution between your own agents. To make it enforceable, bind the token: API_TOKENS=<tok-a>:personal,<tok-b>:greenfield-idea. A confined token pins its workspace and ignores the header entirely.

One caveat: /blobs/<sha256> is not workspace-scoped. Content is addressed by hash, so fetching one means already knowing its content, but revision listings do hand out hashes — so keep API_TOKENS set if that matters to you.

Going multi-machine

The default compose is loopback-only on a private bridge network, which works the same on Linux, macOS, and Windows. To let agents on other machines reach tracker, publish the port on a trusted interface and turn auth on:

# .env
BIND_ADDR=100.x.y.z                  # your LAN / Tailscale / ZeroTier IP
BASE_URL=http://100.x.y.z:8770       # what agents will use to reach it
API_TOKENS=<paste-a-generated-token> # never expose tracker without this

To bind several interfaces at once (loopback + LAN + Tailscale), use Linux host networking, where tracker binds each address in LISTEN_ADDR itself:

# .env: LISTEN_ADDR=127.0.0.1:8770,192.168.1.100:8770,10.10.10.10:8770
docker compose -f docker-compose.yml -f compose.host.yml up -d

Host networking is Linux-only — network_mode: host is a no-op on Docker Desktop, so on macOS/Windows stay with the default compose.

X-Actor is self-asserted attribution, not authenticated identity. On a trusted network that's the point. Set API_TOKENS if you need writes to be gated; bind actor→token if you need attribution to be tamper-proof (see the backlog).

Ops

make update TAG=v1.5.0                 # run a published release here (the standard deploy)
make status                            # what this host is pinned to vs what it runs
make up                                # start with whatever .env pins + smoke test
make down                              # stop; data survives in the pgdata volume
make logs                              # follow tracker logs
make smoke                             # prove a running instance round-trips a write
make deploy                            # DEV: rebuild from your working tree instead

The standard deploy is make update TAG=vX.Y.Z. It pins TRACKER_IMAGE in .env, pulls that release, restarts, smoke-tests the write path, and prints the version the binary actually reports. Rolling back is the same command with an earlier tag, so what a host runs is always one greppable line in its .env rather than whatever its working tree happened to contain.

make deploy is the development path: it builds your working tree instead of running a release, so the version it stamps is a git describe string that matches no release. Hosts that build this way set COMPOSE_FILE in .env to include compose.build.yml, whose image: tracker:local overrides TRACKER_IMAGE — make update detects that and refuses rather than pinning a release the stack would ignore.

make up pulls the published image. Contributors who want their working tree built instead use the build override (that's what make deploy does):

docker compose -f docker-compose.yml -f compose.build.yml up -d --build

The version is git describe --tags --always --dirty — logged at startup, served at /version, and recorded in each backup's manifest.json.

Running the binary directly, no container: make build && set -a && . ./.env && set +a && ./tracker (needs Go and a reachable Postgres).

Releasing

Pushing an annotated v* tag is the whole ritual. Everything else is CI.

git tag -a v1.5.0 -m "v1.5.0 — Short title

What changed and why, in as much detail as it deserves."
git push origin v1.5.0

That one push runs three jobs in order, each gated on the last:

Job	Does
test	gofmt, vet, build, go test -race against a real pgvector Postgres
image	builds and pushes ghcr.io/chicagobuss/tracker:v1.5.0 (+ sha-<short>)
release	creates the GitHub Release from the tag's annotation

Because release needs image, a Release can never point at a version that failed to publish. Re-running a workflow is safe — the job no-ops if the release already exists.

Write the notes in the tag. The annotation is the release: its first line becomes the title, everything after it the body. That keeps git tag -a the single source of truth instead of maintaining a changelog in two places.

Use -a. A lightweight tag (plain git tag v1.5.0) has no annotation, so the release falls back to notes generated from the PRs merged since the previous tag — serviceable, but blunter than what you'd write. Both v1.3.0 and the release-less v1.3.1 predate this and show the two failure modes.

latest tracks the default branch, not tags. It moves on every push to main, so it is whatever main last built — pin vX.Y.Z for anything you actually depend on.

Deploying it is one command on the host, and the same one everywhere:

make update TAG=v1.5.0

That pins TRACKER_IMAGE in .env, pulls, restarts, smoke-tests, and reports the running version. It needs no Go toolchain and no checkout of the tag — the artifact is the one CI built and tested from that commit, not a local rebuild of it. Roll back by naming an earlier tag.

Confirm what actually landed with make status or curl <host>:8770/version. The version is git describe --tags --always --dirty, so a build from a dirty or untagged tree says so rather than quietly claiming a release number — it is logged at startup, served at /version, and recorded in each backup's manifest.json.

Tests

make test          # throwaway Postgres + go test (needs a local Go toolchain)
make test-docker   # same, but Go runs in a container too (needs only Docker)

Both start a scratch pgvector container, run the suite against it, and tear it down. The lease/CAS state machine and SKIP LOCKED task claiming are tested against a real Postgres, because that's where those semantics actually live — the tests skip (rather than fail) if TEST_DATABASE_URL isn't set, so a bare go test ./... stays green without a database.

API

Method	Path	Purpose
GET	/healthz · /version · /openapi.yaml · /llms.txt	health (checks Postgres), version, spec, agent index
POST	/mcp	native MCP endpoint (Streamable HTTP, tools-only)
POST · GET	/docs	create (content seeds v1); list/search (?q=&mode=&kind=&tag=&deleted=&view=&limit=&offset=)
GET · PUT · PATCH · DELETE	/docs/{id}	read; write content (lease + If-Match); relabel; hard-delete (requires confirm = slug)
POST	/docs/{id}/soft-delete · /docs/{id}/restore	soft-delete (recoverable; optional cascade for folios) · restore
GET	/docs/{id}/raw · /docs/{id}/revisions[/{v}/raw]	content bytes; version history
POST · GET · DELETE	/docs/{id}/lock	acquire/renew (409 if held) · status · release
GET	/tags	tag vocabulary with counts
GET · POST	/folios · /folios/{slug} · /folios/{slug}/files[/{filename}[/raw]]	collections + their files
POST · GET	/tasks[/{id}] · /tasks/claim · /tasks/{id}/complete	task queue: enqueue, list (?status=), claim (TTL'd; expired claims re-claimable), complete (claimant-only)
GET	/actors · /actors/{name}/activity	entity registry + activity

The authoritative reference is openapi.yaml (served live at /openapi.yaml).

Conventions. Every response is wrapped — a single resource under its type ({"document":…}, {"folio":…}, …), lists as {"<type>s":[…],"count","total",…}, errors as {"error":{"code","message",…}} with machine codes. Lists default to a trimmed view=summary; view=table is a compact columnar {cols,rows}, view=full whole objects. Search is websearch_to_tsquery (mode=web default; mode=plain for strict AND).

Acting entity

Every mutating request must send an X-Actor: <name> header naming the entity performing it (missing → 400). That value is stamped into created_by/updated_by, the revision author, the lease owner, and task claimed_by, and upserted into the actors registry. A write must come from the entity that holds the lease (actor ≠ lease owner → 423).

Folios

A folio is a collection of related documents (modeled after GitHub gists). It's modelled tableless: the folio is itself a document with kind='folio' whose metadata holds {description, public, github_id, ...}; its files are documents tagged folio:<slug> with slug <folio-slug>/<filename>. So a folio file inherits everything (versioning, leases, attribution, search). Create one with POST /folios and add files with POST /folios/{slug}/files; import your recent gists with scripts/import_gists.py.

{id} accepts a UUID or a slug — including multi-segment folio slugs like myfolio/file.md, for reads, writes, relabels, delete, /raw, and /lock alike (an exact slug always wins over the /raw//lock//soft-delete//restore suffix). The one quirk: a file literally named raw, lock, soft-delete, or restore collides with those suffix routes — address those via /folios/{slug}/files/{filename} or the UUID.

Soft-delete vs hard-delete

• Soft-delete (POST /docs/{id}/soft-delete, MCP soft_delete_doc) sets deleted_at/deleted_by. The row and revision history stay; default search (deleted=exclude) hides it; deleted=only|include finds it; get_doc by id/slug still works; restore_doc brings it back. Prefer this.
• Hard-delete (DELETE /docs/{id}, MCP hard_delete_doc) removes the row (revisions cascade; blobs left for GC). It requires confirm equal to the document's exact slug — MCP marks confirm required in the tool schema so agents cannot call it without an explicit matching value. Folios with files need cascade=true.

Backup & restore

State lives in two places that must be captured together: Postgres (the index) and the blobs (the content). One self-contained tarball holds both — db.dump + blobs/ + manifest.json. That tarball is the portable unit; "R2 vs S3 vs a local directory" is just where you keep it.

scripts/backup.sh                 # -> ./backups/tracker-backup-<ts>.tar.gz
scripts/backup.sh --upload        # also push to R2/S3 (set BACKUP_S3_* in .env)
scripts/backup.sh --if-changed    # skip entirely if no content changed (for cron)
# hourly, only when something changed (BACKUP_KEEP prunes local tarballs, default 48):
#   7 * * * * /path/to/tracker/scripts/backup.sh --if-changed --upload >> /path/to/tracker/backups/backup.log 2>&1

scripts/restore.sh ./backups/tracker-backup-<ts>.tar.gz   # from a local file
scripts/restore.sh --from-s3 tracker-backup-<ts>.tar.gz   # pull from R2/S3 first
docker compose up -d tracker                              # then start the service

The backup dumps Postgres first, then copies blobs — and since writes are blob-first, every content_key in the dump is guaranteed to have its blob, so the tarball is always internally consistent. Restore is verified round-trip: restoring into a scratch DB+bucket reproduces the exact doc/blob counts and a tracker booted against it serves the content. scripts/s3util.py moves blobs and tarballs to any S3-compatible store (RustFS, AWS S3, Cloudflare R2).

To restore on a fresh machine: clone the repo, create .env, docker compose up -d postgres, run restore.sh, then docker compose up -d tracker.

Switching storage backend

Blobs are content-addressed and Postgres stores only the sha256/<hash> key — never the backend location — so switching between local files and S3 is just a blob copy plus a config flip. The migrate-blobs subcommand does the copy:

tracker migrate-blobs --to file --blob-dir ./data/blobs   # S3 -> local files
tracker migrate-blobs --to s3                             # local files -> S3
#   --dry-run   hash-check + count, write nothing
#   --verify    also re-read each blob from the destination

It reads every referenced blob from the current backend (STORAGE_TYPE), verifies each against its hash, and writes it to the destination. It is non-destructive (the source is left intact) and idempotent. On success it prints the cutover step — set STORAGE_TYPE (and BLOB_DIR for file) in .env and restart — so the switch is deliberate and reversible.

Status & roadmap

Running in "production" (lol) and used heavily for months. Recently landed: CI (GitHub Actions → GHCR images), the native /mcp endpoint (replacing the per-machine client script), task-queue visibility + claim expiry + claim-by-id, a real /healthz, soft/hard delete, and a test suite covering the lease/CAS and task-claim state machines.

Hardening

• Request logging / metrics — only startup/shutdown logs exist today; add access-log middleware (method, path, status, duration, actor), plus counters for writes and lease conflicts.

• Auth hardening — use constant-time bearer-token comparison; implement actor↔token binding so a token pins which X-Actor it may assert.

• Migration version tracking — every migrations/*.sql re-runs on each boot and relies on idempotency. Add transactionally applied, checksummed migration tracking before introducing a non-idempotent migration.

• Write pre-check — check lease/version before blob upload to reduce orphan blobs on rejected writes (412/423), while retaining the transactional check to prevent races.

• Orphan-blob / expired-lease GC — mark-and-sweep blobs not referenced by documents or document_revisions, and periodically remove expired doc_locks.

• HTTP/MCP tests — the store state machines are covered, but request-level REST and MCP behavior is effectively untested.

Features

• REST folio pagination — GET /folios and folio file listings hardcode limit 500 with no paging. MCP list_folios already supports limit and offset.

• Web UI tasks panel — browse the queue through the existing task endpoints, including status filtering and payload/result detail.

• pgvector semantic search — the embedding column already exists; populate it on write and add a semantic query path.

Ops

• Scheduled off-box backups — done: cron scripts/backup.sh --if-changed --upload with retention, and add automated restore verification against manifest document/blob counts.

• Public sandbox instance — a rate-limited, internet-facing demo: Caddy in front with per-IP limits and body caps; bridge networking with only the proxy exposed; a seed-restore reset every ~6h; and app-side quotas such as MAX_DOCS / MAX_TOTAL_BLOB_BYTES so one actor cannot fill the disk between resets. The native /mcp endpoint then gives visitors one-command agent onboarding.

Longer-horizon ideas—blob compression, retention thinning, and content-defined chunking—should wait until usage metrics justify their complexity.

PRs welcome but I can't promise I'll get to them!