Skip to content

A preview runs on its own copy of dev (#113) - #181

Merged
Devski merged 2 commits into
mainfrom
claude/113-preview-db
Sep 12, 2026
Merged

A preview runs on its own copy of dev (#113)#181
Devski merged 2 commits into
mainfrom
claude/113-preview-db

Conversation

@Devski

@Devski Devski commented Sep 12, 2026

Copy link
Copy Markdown
Owner

Closes #113.

A preview ran the pull request's image against dev's database, and
previews never migrate. Every pull request that added a column its pages read
therefore previewed as a server error — #112 on 09.09.2026, #170 five days
later — exactly when the preview existed to be looked at. Both times the
workaround was to apply the migration to dev by hand first, which is only safe
while migrations stay expand-only (G6) and somebody is there to reason about it.

What a preview is now

When one starts, deploy/preview-up.sh:

  1. reads DATABASE_URL from the instance's .env and derives
    platform_pr_<n> from it (query string preserved, database name swapped);
  2. drops any previous copy WITH (FORCE) and creates a fresh one;
  3. pg_dump of dev to a file inside the PostgreSQL container, then restore
    from that file with ON_ERROR_STOP=1;
  4. runs the pull request's own /app/migrate.mjs against the copy;
  5. starts the container with DATABASE_URL pointing at it.

deploy/preview-down.sh drops the copy after the container is gone, warns
rather than failing if it cannot, and now also lists the preview databases
still on the instance.

So a preview shows dev's data as of its start, plus this pull request's
schema
. Accounts created in a preview and the rows recording what was
uploaded there die with it — the last line of the script says so, and that is
the line CI puts in the job summary.

Two decisions worth stating

  • A dump and a restore, not create database … template. The issue
    suggested the template clone with a retry. PostgreSQL refuses a template
    clone while anything is connected to the source, and dev's own container
    always is — retrying would make the preview a coin toss. Read from the
    instance while writing this: dev's pool had two idle connections on
    platform_devski.
  • Dump to a file, then restore from the file — not pg_dump | psql. A
    pipeline reports only its last command's status, so a failed dump would pass
    for a successful restore of nothing, and the preview would look like a
    working one with an empty database. (Same lesson as the CI pipeline that hid
    a failure in PR preview deployments on the dev instance #31.)

What stays shared

The bucket (under pr-<n>/), the signing key, the instance, the two-preview
cap. Objects a preview uploaded are still not deleted — that needs the S3
credentials this script has no reason to hold (#34, #111). Since this change
the rows naming them go with the database, so what is left is unreferenced.

CI

deploy-config gains what the issue asked for: a preview creates its
database, migrates it, is pointed at it, and drops it on the way
out. Each of those has a silent failure mode — without the copy a preview is
back on dev's database; without the migrate step the copy carries main's
schema, which is the same bug with more steps; without DATABASE_URL the
container keeps the instance .env value and never sees the copy at all.

This is a CI configuration change (SPEC §7 "ask first") and it is the one
#113's own acceptance criteria specify.

Verification

  • This pull request's own preview is built by the new script (CI ships
    preview-up.sh from the branch), so the job log shows the copy, the
    migrate step and a healthy container.
  • pg_stat_activity shows the preview connected to platform_pr_<n>,
    not to platform_devski.
  • Dev keeps serving from its own database throughout.
  • Closing this pull request leaves no platform_pr_% database behind.

What this pull request cannot prove by itself: a preview of a pull request
that actually carries a migration. It has no migration of its own, so
migrate.mjs reports "schema is up to date" — the same code path dev runs on
every deploy, against the copy instead of dev. The next migration-carrying
pull request is the end-to-end proof, and it is the one that used to break.

🤖 Generated with Claude Code


What the two review lanes changed

Two findings were worth the whole exercise, and neither was in the code this
pull request set out to write.

The nightly copy of the databases (#168) would have stopped. It selects
every platform_% database that is not a test one — which now includes the
preview copies. Two previews open at 03:17 means an object three times the
usual size and last-success recording that; the next night without them
produces a third of it, the shrink guard refuses to upload, and because a
refusal records no success it refuses every night after that too. The
backup #168 built and drilled would have gone quiet for a reason that has
nothing to do with dev's data. The copies are excluded now, and CI pins it.

A preview could delete dev's objects. The copy carries dev's files rows,
and those rows name dev's keys; S3_PREFIX scopes only new writes. So
deleting a work inside a preview would have deleted the object out of the
shared bucket and left dev's own row pointing at nothing — permanent, silent,
and invisible to the environment that lost it. This was a regression this
change introduced: before, the row and the object went together because there
was one database. src/lib/storage.ts now refuses a delete whose key is
outside the environment's own prefix and logs the key it refused; production's
prefix is blank, so the bare bucket stays production's own. Four tests.

Also fixed, all from the reviews:

  • The old container is removed before its database is taken away. Every
    step after that can fail, and a container left serving against a database
    being replaced is the state remote-deploy.sh is built to avoid.
  • A copy with no container is dropped at the start of every run. Only
    closing a pull request removed one; an older branch's preview-down.sh
    knows nothing about copies at all, and a cancelled CI run takes its ssh with
    it. The sweep, not the teardown, is what stops copies of every dev account's
    rows accumulating on a disk that has filled once already (Dev instance keeps every image it ever pulled: the deploy must drop the old ones #119).
  • sessions and verifications are emptied in the copy. A session is
    valid for thirty days and both environments sign with the same key, so a
    copied session is a credential that signing out on dev can no longer revoke.
    You sign in to a preview now.
  • The dump is written under umask 077, and psql runs VERBOSITY=terse
    the file is every account's address and password hash in plain SQL, and psql
    prints the offending row as context when a restore fails, into a CI log
    this public repository keeps for ninety days.
  • The migrator gets only DATABASE_URL (exported, not in argv, where
    /proc/<pid>/cmdline would show the database password) and a memory cap.
    It reads one variable and was being handed the mail key, the bucket
    credentials and AUTH_SECRET.
  • The CI checks no longer pass on comment text — they match the psql flag
    rather than the SQL, and count both containers rather than either.
  • Comments this change made false are corrected, the collector's guard in
    particular: it is the one thing that keeps a 12-hourly sweep from reasoning
    about dev's objects with a copy of dev's rows, and its stated reason had
    become "previews share dev's database".

Verified on the instance

  • platform_pr_181 exists, 28 MB, 30 users and 19 works copied — and
    0 sessions, 0 verifications against dev's 46 and 1.
  • The preview's connections are to platform_pr_181 only
    (application_name platform-lite/pr-181/web); nothing of its own reaches
    platform_devski.
  • Both preview pages answer 200; dev keeps serving throughout.
  • No dump file left in the PostgreSQL container, and one copy on the instance.

Devski and others added 2 commits September 12, 2026 20:53
A preview ran the pull request's image against DEV's database, and previews
never migrate. So every pull request that added a column its pages read
previewed as a server error: #112 on 09.09.2026, #170 again five days later,
each time exactly when the preview existed to be looked at. The workaround
both times was to apply the migration to dev by hand first — safe only while
migrations stay expand-only and somebody is there to reason about it.

Now preview-up.sh copies dev into platform_pr_<n>, runs the pull request's own
migrator against the copy, and points the container at it; preview-down.sh
drops it again. A preview is therefore dev's data as of its start plus this
pull request's schema, and the accounts and uploads made inside one die with
it — said out loud in the line CI puts in the job summary.

The copy is a dump and a restore, not `create database ... template`:
PostgreSQL refuses a template clone while anything is connected to the source,
and dev's own container always is. Dump to a file rather than piping into
psql, because a pipeline reports only its last command's status — a failed
dump would have passed for a successful restore of nothing, and the preview
would have looked like a working one with an empty database.

Not "the preview migrates the shared database", which was the other candidate:
a pull request revised after its preview ran would leave a migration in dev's
journal whose hash no longer matched the file, and the next deployment of main
would fail on it.

CI gains what #113 asked for: deploy-config now pins that a preview creates
its database, migrates it, is pointed at it, and drops it on the way out.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two of them would have been expensive.

The nightly copy of the databases (#168) selects every `platform_%` database
that is not a test one — which now includes the preview copies. A night with
two previews open would copy dev's data three times over, and the night after
they merged would produce a third of that, which the shrink guard reads as a
copy gone bad: it refuses the upload, does not record a success, and so
refuses every night after it too. The backup would have stopped, silently,
for a reason that has nothing to do with dev. The copies are excluded now,
and CI pins the exclusion.

And a preview could delete dev's objects. The copy carries dev's file rows,
which name dev's keys; the bucket is shared and only new WRITES are scoped by
S3_PREFIX. So deleting a work in a preview would have deleted dev's object and
left dev's own row pointing at nothing — permanent, silent, and invisible to
the environment that lost it. lib/storage.ts now refuses a delete outside the
environment's own prefix and says which key it refused; production's prefix is
blank, so the bare bucket stays production's own.

The rest:

- The old container is removed BEFORE its database is taken from under it.
  Every step after that can fail, and what must never happen is a container
  still serving against a database being replaced.
- A copy with no container is dropped at the start of every run. Only closing
  a pull request removed one, an older branch's preview-down.sh knows nothing
  about copies at all, and a cancelled CI run takes its ssh with it — so the
  sweep, not the teardown, is what keeps copies of every dev account's rows
  from accumulating on a disk that has filled once already (#119).
- The copy is restored with `sessions` and `verifications` emptied. A session
  is valid for thirty days and both environments sign with the same key, so a
  copied session is a credential that signing out on dev can no longer revoke.
  You sign in to a preview.
- The dump is written under umask 077 and both psql runs are VERBOSITY=terse:
  the file is every account's address and password hash in plain SQL, and psql
  prints the offending ROW as context when a restore fails — into a CI log
  this public repository keeps for ninety days.
- The migrator gets only DATABASE_URL, exported rather than put in argv, and a
  memory cap. It read one variable and was handed the mail key, the bucket
  credentials and AUTH_SECRET.
- The CI checks no longer pass on comment text: they match the psql flag, not
  the SQL, and count both containers rather than either.
- Comments that this change made false are corrected — the collector's guard
  in particular, whose stated reason was that previews share dev's database.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@Devski
Devski merged commit b959471 into main Sep 12, 2026
7 checks passed
@Devski
Devski deleted the claude/113-preview-db branch September 12, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PR previews: a database cloned from dev per preview, not the shared one

1 participant