A preview runs on its own copy of dev (#113) - #181
Merged
Conversation
A preview ran the pull request's image against DEV's database, and previews never migrate. So every pull request that added a column its pages read previewed as a server error: #112 on 09.09.2026, #170 again five days later, each time exactly when the preview existed to be looked at. The workaround both times was to apply the migration to dev by hand first — safe only while migrations stay expand-only and somebody is there to reason about it. Now preview-up.sh copies dev into platform_pr_<n>, runs the pull request's own migrator against the copy, and points the container at it; preview-down.sh drops it again. A preview is therefore dev's data as of its start plus this pull request's schema, and the accounts and uploads made inside one die with it — said out loud in the line CI puts in the job summary. The copy is a dump and a restore, not `create database ... template`: PostgreSQL refuses a template clone while anything is connected to the source, and dev's own container always is. Dump to a file rather than piping into psql, because a pipeline reports only its last command's status — a failed dump would have passed for a successful restore of nothing, and the preview would have looked like a working one with an empty database. Not "the preview migrates the shared database", which was the other candidate: a pull request revised after its preview ran would leave a migration in dev's journal whose hash no longer matched the file, and the next deployment of main would fail on it. CI gains what #113 asked for: deploy-config now pins that a preview creates its database, migrates it, is pointed at it, and drops it on the way out. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two of them would have been expensive. The nightly copy of the databases (#168) selects every `platform_%` database that is not a test one — which now includes the preview copies. A night with two previews open would copy dev's data three times over, and the night after they merged would produce a third of that, which the shrink guard reads as a copy gone bad: it refuses the upload, does not record a success, and so refuses every night after it too. The backup would have stopped, silently, for a reason that has nothing to do with dev. The copies are excluded now, and CI pins the exclusion. And a preview could delete dev's objects. The copy carries dev's file rows, which name dev's keys; the bucket is shared and only new WRITES are scoped by S3_PREFIX. So deleting a work in a preview would have deleted dev's object and left dev's own row pointing at nothing — permanent, silent, and invisible to the environment that lost it. lib/storage.ts now refuses a delete outside the environment's own prefix and says which key it refused; production's prefix is blank, so the bare bucket stays production's own. The rest: - The old container is removed BEFORE its database is taken from under it. Every step after that can fail, and what must never happen is a container still serving against a database being replaced. - A copy with no container is dropped at the start of every run. Only closing a pull request removed one, an older branch's preview-down.sh knows nothing about copies at all, and a cancelled CI run takes its ssh with it — so the sweep, not the teardown, is what keeps copies of every dev account's rows from accumulating on a disk that has filled once already (#119). - The copy is restored with `sessions` and `verifications` emptied. A session is valid for thirty days and both environments sign with the same key, so a copied session is a credential that signing out on dev can no longer revoke. You sign in to a preview. - The dump is written under umask 077 and both psql runs are VERBOSITY=terse: the file is every account's address and password hash in plain SQL, and psql prints the offending ROW as context when a restore fails — into a CI log this public repository keeps for ninety days. - The migrator gets only DATABASE_URL, exported rather than put in argv, and a memory cap. It read one variable and was handed the mail key, the bucket credentials and AUTH_SECRET. - The CI checks no longer pass on comment text: they match the psql flag, not the SQL, and count both containers rather than either. - Comments that this change made false are corrected — the collector's guard in particular, whose stated reason was that previews share dev's database. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #113.
A preview ran the pull request's image against dev's database, and
previews never migrate. Every pull request that added a column its pages read
therefore previewed as a server error — #112 on 09.09.2026, #170 five days
later — exactly when the preview existed to be looked at. Both times the
workaround was to apply the migration to dev by hand first, which is only safe
while migrations stay expand-only (G6) and somebody is there to reason about it.
What a preview is now
When one starts,
deploy/preview-up.sh:DATABASE_URLfrom the instance's.envand derivesplatform_pr_<n>from it (query string preserved, database name swapped);WITH (FORCE)and creates a fresh one;pg_dumpof dev to a file inside the PostgreSQL container, then restorefrom that file with
ON_ERROR_STOP=1;/app/migrate.mjsagainst the copy;DATABASE_URLpointing at it.deploy/preview-down.shdrops the copy after the container is gone, warnsrather than failing if it cannot, and now also lists the preview databases
still on the instance.
So a preview shows dev's data as of its start, plus this pull request's
schema. Accounts created in a preview and the rows recording what was
uploaded there die with it — the last line of the script says so, and that is
the line CI puts in the job summary.
Two decisions worth stating
create database … template. The issuesuggested the template clone with a retry. PostgreSQL refuses a template
clone while anything is connected to the source, and dev's own container
always is — retrying would make the preview a coin toss. Read from the
instance while writing this: dev's pool had two idle connections on
platform_devski.pg_dump | psql. Apipeline reports only its last command's status, so a failed dump would pass
for a successful restore of nothing, and the preview would look like a
working one with an empty database. (Same lesson as the CI pipeline that hid
a failure in PR preview deployments on the dev instance #31.)
What stays shared
The bucket (under
pr-<n>/), the signing key, the instance, the two-previewcap. Objects a preview uploaded are still not deleted — that needs the S3
credentials this script has no reason to hold (#34, #111). Since this change
the rows naming them go with the database, so what is left is unreferenced.
CI
deploy-configgains what the issue asked for: a preview creates itsdatabase, migrates it, is pointed at it, and drops it on the way
out. Each of those has a silent failure mode — without the copy a preview is
back on dev's database; without the migrate step the copy carries
main'sschema, which is the same bug with more steps; without
DATABASE_URLthecontainer keeps the instance
.envvalue and never sees the copy at all.This is a CI configuration change (SPEC §7 "ask first") and it is the one
#113's own acceptance criteria specify.
Verification
preview-up.shfrom the branch), so the job log shows the copy, themigrate step and a healthy container.
pg_stat_activityshows the preview connected toplatform_pr_<n>,not to
platform_devski.platform_pr_%database behind.What this pull request cannot prove by itself: a preview of a pull request
that actually carries a migration. It has no migration of its own, so
migrate.mjsreports "schema is up to date" — the same code path dev runs onevery deploy, against the copy instead of dev. The next migration-carrying
pull request is the end-to-end proof, and it is the one that used to break.
🤖 Generated with Claude Code
What the two review lanes changed
Two findings were worth the whole exercise, and neither was in the code this
pull request set out to write.
The nightly copy of the databases (#168) would have stopped. It selects
every
platform_%database that is not a test one — which now includes thepreview copies. Two previews open at 03:17 means an object three times the
usual size and
last-successrecording that; the next night without themproduces a third of it, the shrink guard refuses to upload, and because a
refusal records no success it refuses every night after that too. The
backup #168 built and drilled would have gone quiet for a reason that has
nothing to do with dev's data. The copies are excluded now, and CI pins it.
A preview could delete dev's objects. The copy carries dev's
filesrows,and those rows name dev's keys;
S3_PREFIXscopes only new writes. Sodeleting a work inside a preview would have deleted the object out of the
shared bucket and left dev's own row pointing at nothing — permanent, silent,
and invisible to the environment that lost it. This was a regression this
change introduced: before, the row and the object went together because there
was one database.
src/lib/storage.tsnow refuses a delete whose key isoutside the environment's own prefix and logs the key it refused; production's
prefix is blank, so the bare bucket stays production's own. Four tests.
Also fixed, all from the reviews:
step after that can fail, and a container left serving against a database
being replaced is the state
remote-deploy.shis built to avoid.closing a pull request removed one; an older branch's
preview-down.shknows nothing about copies at all, and a cancelled CI run takes its ssh with
it. The sweep, not the teardown, is what stops copies of every dev account's
rows accumulating on a disk that has filled once already (Dev instance keeps every image it ever pulled: the deploy must drop the old ones #119).
sessionsandverificationsare emptied in the copy. A session isvalid for thirty days and both environments sign with the same key, so a
copied session is a credential that signing out on dev can no longer revoke.
You sign in to a preview now.
umask 077, and psql runsVERBOSITY=terse—the file is every account's address and password hash in plain SQL, and psql
prints the offending row as context when a restore fails, into a CI log
this public repository keeps for ninety days.
DATABASE_URL(exported, not in argv, where/proc/<pid>/cmdlinewould show the database password) and a memory cap.It reads one variable and was being handed the mail key, the bucket
credentials and
AUTH_SECRET.rather than the SQL, and count both containers rather than either.
particular: it is the one thing that keeps a 12-hourly sweep from reasoning
about dev's objects with a copy of dev's rows, and its stated reason had
become "previews share dev's database".
Verified on the instance
platform_pr_181exists, 28 MB, 30 users and 19 works copied — and0 sessions, 0 verifications against dev's 46 and 1.
platform_pr_181only(
application_name platform-lite/pr-181/web); nothing of its own reachesplatform_devski.