Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions CHANGELOG
Original file line number Diff line number Diff line change
@@ -1,3 +1,59 @@
2026-06-03 v1.0.23

Features:
- Production-friendly output for the compare
command, enabled with --output-for-production
(or OUTPUT_FOR_PRODUCTION=true in the config
file). When on, the generated migration is
shaped to minimise locking on a live
database: indexes are built with CREATE INDEX
CONCURRENTLY (the UNIQUE keyword is
preserved); indexes on partitioned parents
use CREATE INDEX ... ON ONLY {parent} inside
the transaction, then CREATE INDEX
CONCURRENTLY on each partition followed by
ALTER INDEX ... ATTACH PARTITION afterwards
(falling back to a single non-concurrent
CREATE INDEX on the parent, with an
explanatory comment, when the layout cannot
be expanded safely); indexes are dropped with
DROP INDEX CONCURRENTLY (kept non-concurrent
for indexes on partitioned tables, where
concurrent drop is illegal); and new foreign
keys are added NOT VALID inside the
transaction and validated with a separate
VALIDATE CONSTRAINT afterwards. Every
statement that cannot run inside a
transaction block (CREATE/DROP INDEX
CONCURRENTLY, VALIDATE CONSTRAINT, ALTER
INDEX ... ATTACH PARTITION) is emitted in a
clearly marked Production post-commit section
after commit. Defaults to false; when off,
the output is byte-for-byte identical to
previous behaviour.
- Added the OUTPUT_FOR_PRODUCTION key to the
sample data/pgc.conf and documented the new
flag, configuration key, and behaviour in
README.md.
- Production output is now re-runnable: every
DDL form PostgreSQL supports is emitted defensively
with an idempotency guard where available, so a
partially-applied migration can be replayed without failing.
CREATE [UNLOGGED] TABLE, CREATE [UNLOGGED]
SEQUENCE, CREATE MATERIALIZED VIEW and CREATE
[UNIQUE] INDEX [CONCURRENTLY] gain IF NOT
EXISTS; CREATE VIEW becomes CREATE OR REPLACE
VIEW; ALTER TABLE ... ADD COLUMN gains IF NOT
EXISTS; and ALTER TABLE ... DROP COLUMN /
DROP CONSTRAINT gain IF EXISTS. CREATE TYPE
and ALTER TABLE ... ADD CONSTRAINT are left
as-is (PostgreSQL has no idempotency guard for
them). The rewrite is literal-, comment- and
dollar-quote-aware, so quoted text, commented
drops and function bodies are untouched, and
it only applies when
--output-for-production is on.

2026-05-26 v1.0.22

Bug fixes:
Expand Down
23 changes: 23 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,8 @@ Command line arguments can be used to execute just one function in one time.

`--grants-mode {ignore|addonly|full}` - controls how grants (privileges) are handled during comparison. `ignore` (default) skips grants entirely; `addonly` adds grants that exist in TO but not in FROM; `full` makes grants identical by adding missing and revoking extra.

`--output-for-production {true|false}` - set to `true` to generate a migration script that is convenient to run against a live production database (default `false` — output is unchanged). See [Production-friendly output](#production-friendly-output).

`--max-connections {number}` - maximum number of connections in the PostgreSQL connection pool. Default: `16`. Used by all concurrent introspection queries; table metadata is pulled schema-wide in one query per resource kind (columns, indexes, constraints, triggers, policies, partition info, definitions) so connection count mostly matters for the sibling queries (extensions, sequences, routines, views, etc.) running in parallel.

`--use-cascade` - add `CASCADE` to every `DROP` statement in the clear script. **Warning:** `CASCADE` can silently drop dependent objects that live outside the selected schema(s) (e.g., foreign keys or views in other schemas referencing the dropped objects). Use only when you are certain no cross-schema dependencies should survive. Without this flag the generated drops rely on the explicit dependency ordering and will fail cleanly if unresolved dependencies exist.
Expand All @@ -95,6 +97,24 @@ This command comparing two dumps and produce SQL script for the `FROM` database
If we add `--use-drop` argument comparer will add drop scripts for all items that non exists in target database, otherwise drop scripts will be ignored.
By default, comparer ignore drops.

### Production-friendly output

```bash
pgc --command compare --from {from_dump} --to {to_dump} --output {file} --use-single-transaction --output-for-production
```

By default the delta script favours brevity and is meant to be applied to an idle database. With `--output-for-production true` (config key `OUTPUT_FOR_PRODUCTION=true`) the comparer instead emits a script that minimises locking on a live database:

- **Indexes are built concurrently** — `CREATE INDEX` becomes `CREATE INDEX CONCURRENTLY` (the `UNIQUE` keyword is preserved), so building an index does not block writes.
- **Partitioned tables are handled correctly** — `CONCURRENTLY` is not allowed directly on a partitioned table, so for a partitioned parent the comparer emits `CREATE INDEX ... ON ONLY {parent}` (in the transaction), then `CREATE INDEX CONCURRENTLY` on each partition followed by `ALTER INDEX ... ATTACH PARTITION` (after the transaction). When the partition layout cannot be expanded safely (no known partitions, or multi-level/sub-partitioned children) it falls back to a single non-concurrent `CREATE INDEX` on the parent and explains why in a comment.
- **Indexes are dropped concurrently** — `DROP INDEX` becomes `DROP INDEX CONCURRENTLY` (kept non-concurrent for indexes on partitioned tables, where concurrent drop is illegal).
- **Foreign keys are validated separately** — a new foreign key is added `NOT VALID` inside the transaction (a fast, metadata-only operation) and a matching `VALIDATE CONSTRAINT` (the long, scan-heavy step) is emitted afterwards so it does not hold the lock for the whole migration.
- **DDL is emitted defensively so the migration is re-runnable** — every statement PostgreSQL supports a guard for is generated idempotently, so a migration that was applied partially (e.g. a post-commit step failed) can be replayed without errors. `CREATE [UNLOGGED] TABLE`, `CREATE [UNLOGGED] SEQUENCE`, `CREATE MATERIALIZED VIEW` and `CREATE [UNIQUE] INDEX [CONCURRENTLY]` gain `IF NOT EXISTS`; `CREATE VIEW` becomes `CREATE OR REPLACE VIEW`; `ALTER TABLE ... ADD COLUMN` gains `IF NOT EXISTS`; and `ALTER TABLE ... DROP COLUMN` / `DROP CONSTRAINT` gain `IF EXISTS`. `CREATE TYPE` and `ALTER TABLE ... ADD CONSTRAINT` are left unguarded because PostgreSQL has no idempotency clause for them. The rewrite is SQL-aware (it skips string literals, quoted identifiers, comments — including commented-out drops — and dollar-quoted function bodies), so only the structural keyword of each statement is touched.

Because `CREATE/DROP INDEX CONCURRENTLY`, `VALIDATE CONSTRAINT` and `ALTER INDEX ... ATTACH PARTITION` **cannot run inside a transaction block**, every such statement is moved to a clearly marked `Production post-commit` section emitted **after** the `commit;`. The rest of the migration still runs inside the single transaction when `--use-single-transaction` is set. Each post-commit statement runs in its own implicit transaction; most are safe to re-run thanks to idempotency guards, but `ALTER INDEX ... ATTACH PARTITION` has no built-in guard, so already-successful ATTACH steps may need to be skipped when replaying the section.

`--output-for-production` defaults to `false`; when off the output is byte-for-byte identical to previous behaviour.

### Generate a clear (drop-all) script for a database

```bash
Expand Down Expand Up @@ -153,8 +173,11 @@ USE_SINGLE_TRANSACTION=true
USE_COMMENTS=false
GRANTS_MODE=ignore
MAX_CONNECTIONS=16
OUTPUT_FOR_PRODUCTION=false
```

`OUTPUT_FOR_PRODUCTION` (default `false`) is the configuration-file equivalent of the `--output-for-production` flag described in [Production-friendly output](#production-friendly-output).

## Choosing `MAX_CONNECTIONS`

`MAX_CONNECTIONS` caps the connection pool the tool opens per dump. Independent introspection queries (extensions, sequences, routines, views, types, tables, etc.) are fired concurrently via `tokio::try_join!`, and the table-level data (columns, indexes, constraints, triggers, policies, partition info, definitions) is pulled with **schema-wide** queries — one query per sub-resource, independent of table count. So the connection count mostly bounds how many of the ~18 concurrent sibling queries run without queueing.
Expand Down
2 changes: 1 addition & 1 deletion app/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion app/Cargo.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[package]
name = "pgc"
version = "1.0.21"
version = "1.0.23"
edition = "2024"
license = "MIT"
authors = ["nettrash <nettrash@nettrash.me>"]
Expand Down
Loading
Loading