Skip to content

Schema change: put the real-PostgreSQL checks in the repo, and say what has and has not been checked - #7

Merged
swsarancodes merged 2 commits into
mainfrom
docs/schema-change-tested
Oct 25, 2026
Merged

swsarancodes merged 2 commits into
mainfrom
docs/schema-change-tested

Conversation

@swsarancodes

Copy link
Copy Markdown
Owner

What changed

  1. The real-PostgreSQL checks are now in the repository (they never were, which made every earlier claim unverifiable from the branch).
    • scripts/schema-change-pg.ts (renamed from the local schema-change-pg16.ts, since it covers 12 to 18) runs 15 sections against a real server: lock levels read from pg_locks; which type changes rewrite a table; every compiled scenario run and compared with the compiler's predicted end state; the drop and dependency rules against what PostgreSQL itself refuses; the pre-flight check and the lock retry with real sessions (including a role that cannot read other sessions); the table rename with real roles, grants and row-level security; and the probes of defaults and checks.
    • scripts/schema-change-versions.sh starts one container per version (12, 13, 14, 15, 16, 17, 18), runs the script and removes the containers.
    • scripts/lib/pg-scratch.ts and scripts/lib/plan-runner.ts are its helpers.
    • npm run test:schema-pg and npm run test:schema-pg:versions (two lines added to package.json; nothing else in it changed).
  2. The README and the page say what has and has not been checked. The Schema Change section described a page that opens a runbook and runs nothing, tested on PostgreSQL 16. It now describes the pre-flight check, running one step at a time, the lock retry and the comparison after the last step, and lists: what was checked on real servers, what was checked on small data only, what was not run against a real server, and what the model's answers are and are not checked for. The page has the same list as a collapsible note, and its "what happens when you plan" text matches the new flow.

How it was checked

  • scripts/schema-change-versions.sh 12 18 was run end to end for this PR: all 15 sections pass on PostgreSQL 12 and on 18 (earlier runs covered 13 to 17 with the same sections).
  • Both type-checks are clean; the note renders and expands on the page.

Reviewer notes

  • The README states the limits plainly: the test tables hold up to a few hundred thousand rows, so how long a validation or index build takes on a table of many gigabytes and replica lag during a large backfill are not measured; index builds on partitioned tables, replicas and logical replication, and prepared transactions beyond the empty case were not run against a real server; there is no scored evaluation of how often a model proposes the right operations.
  • The README still mentions the generated-database evaluation (npm run eval:schema); that script is still not in the repository and the README now says so.

…ostgreSQL 12 to 18

scripts/schema-change-pg.ts runs 15 sections against a real server: lock levels read from pg_locks, which type changes rewrite a table, every compiled scenario run and compared with the compiler's predicted end state, the drop and dependency rules against what PostgreSQL refuses, the pre-flight check and lock retry with real sessions, the table rename with real roles and grants, and the probes. scripts/schema-change-versions.sh starts one container per version, 12 to 18, runs it and removes the containers. npm run test:schema-pg and test:schema-pg:versions.
…nd on the page

The README described a page that opens a runbook and runs nothing, tested on PostgreSQL 16. It now describes the pre-flight check, running one step at a time, the lock retry and the comparison after the last step, and lists what was checked on real servers (PostgreSQL 12 to 18), what was checked on small data only, what was not run against a real server, and what the model's answers are and are not checked for. The page has the same list as a note.
@swsarancodes
swsarancodes merged commit a87f6bd into main Oct 25, 2026
1 check failed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant