Skip to content

replica-identity warns when a captured Postgres table has no replica identity - #29

Merged
avison9 merged 2 commits into
mainfrom
feat/replica-identity
Nov 14, 2026
Merged

avison9 merged 2 commits into
mainfrom
feat/replica-identity

Conversation

@avison9

@avison9 avison9 commented Nov 14, 2026

Copy link
Copy Markdown
Owner

Stacked on #28. This branch includes #28's commit (glued PRIMARY KEY(id, ...)), which the rule needs to read keys correctly. Merge #28 first; this diff then shrinks to its own commit.

What changes

A new warning, replica-identity, for a captured Postgres table with no replica identity: REPLICA IDENTITY NOTHING, or DEFAULT with no primary key.

Both consequences are documented, and the finding says which one applies, based on plugin.name:

  • pgoutput. Postgres, logical-replication-publication: "Tables with a replica identity defined as NOTHING, DEFAULT without a primary key, or USING INDEX with a dropped index, cannot support UPDATE or DELETE operations when included in a publication replicating these actions." The application's own writes fail.
  • decoderbufs (Debezium's documented default). Debezium, PostgreSQL connector, Replica identity: with DEFAULT and no primary key "the connector does not emit UPDATE or DELETE events for that table". With NOTHING, its events carry no previous values.

Other behaviour:

  • The connector's override wins. replica.identity.autoset.values overrides the database for captured tables, so cdclint reads it. When the override decides, the finding points at the connector file. If two pairs disagree about one table, or a pair can't be read, the table is not judged, because the documentation doesn't say which wins.
  • It is a warning. An insert-only table loses nothing. The fix suggests -- cdclint:ignore replica-identity: append-only on the CREATE TABLE line, and I checked that this marker acknowledges the finding.
  • Not judged: USING INDEX, because whether the index still exists isn't in the files.

Reader changes, so the rule never judges a key it cannot see

  • ALTER TABLE ... ADD [CONSTRAINT x] PRIMARY KEY (...) now sets the key.
  • A table a DO block alters, or one with PRIMARY KEY USING INDEX, is marked KeyUnknown, and the rule stays quiet for it. Mattermost adds reactions' key inside a DO block. Actually running DO bodies is its own change.

Corpus first

There are six new entries. I wrote their expectations by hand, and the three that expect a finding failed before the rule existed:

  • no-primary-key
  • nothing-pgoutput
  • autoset-nothing
  • autoset-full
  • key-added-later
  • key-in-do-block (Mattermost's real 000016)

Two older entries had been cut down without the keys their sources have. They now carry them, and their expected.txt is unchanged:

  • golang-migrate-down-files: Mattermost's real teammembers primary key, and reactions' DO block.
  • flatten-bare-columns: REPLICA IDENTITY FULL. The source discussion's sample message has a whole before row on a keyless table, which only FULL produces.

Verified

  • gofmt -l . is clean; go vet, go build and go test ./... pass. There are new unit tests for the override (including the conflicting and unreadable cases) and for the reader's key handling.
  • RefuseRadar (pgoutput) at origin/main: 0 error(s), 0 warning(s), 162 info, so no new warning, which matters because its PR gate fails on warnings.
  • Mattermost v10.11.0, all 280 Postgres migration files, with a pgoutput connector capturing every table: no replica-identity finding. Every table has a key; reactions is covered by the DO-block guard.

Rejected

Judging the before image a sink reads (Flatten on the envelope reading before.<column>). That is the longer promise in the README for this rule. It needs the transform model to know which before fields a sink reads, so it is its own change. The README row now describes what this rule actually does.

After merge

The README status line ("Ten rules") changes at the release that ships this, as with earlier rules.

… a column

Postgres needs no space between a constraint keyword and its
parenthesis, and Mattermost's migrations use that form throughout:
UNIQUE(name, deleteat), PRIMARY KEY(id, channelid). The word splitter
keeps a parenthesised group glued to the word before it, so the reader
saw a column named "UNIQUE(name, deleteat)" (an info line for a column
that does not exist) and a word "KEY(id, channelid)" that hid the primary
key. The same held for CHECK( on a column and for ALTER TABLE ... ADD
UNIQUE( / ADD CHECK(, which added a column.

Keywords in the Postgres reader are now compared up to a parenthesis.
The helper stays in this reader: the shared ddl.HasPrefixFold is also
used by the ClickHouse reader, where a word like Kafka(...) is compared
whole.

Measured on all 280 of Mattermost v10.11.0's Postgres migration files:
nine made-up columns before (channels, emoji, PropertyGroups, roles,
schemes, sharedchannelremotes, teams, usergroups twice), none after,
and no real column lost. RefuseRadar unchanged at 162 info.

Not done here: Postgres folds unquoted identifiers to lower case, and
the reader keeps CREATE TABLE PropertyGroups as written. Its own change.
…identity

A table on REPLICA IDENTITY NOTHING, or on DEFAULT with no primary key,
has no replica identity. Both consequences are documented. Postgres
(logical-replication-publication): such a table "cannot support UPDATE or
DELETE operations when included in a publication replicating these
actions", which is every pgoutput connector, so the application's own
writes fail. Debezium (PostgreSQL connector, Replica identity): with
DEFAULT and no primary key "the connector does not emit UPDATE or DELETE
events for that table", and with NOTHING its events carry no previous
values. The finding says which one applies, from plugin.name (decoderbufs
when unset, the documented default).

The connector's replica.identity.autoset.values overrides the database on
the tables it captures, so it is read and wins; when it decides, the
finding points at the connector file. Two pairs that disagree about one
table, or a pair that cannot be read, leave the identity unknown and the
table unjudged: the documentation does not say which pair wins.

A warning, not an error: a table that only takes inserts loses nothing,
and the fix says how to record that with cdclint:ignore replica-identity.

To keep it from judging keys it cannot see, the Postgres reader now reads
ALTER TABLE ... ADD [CONSTRAINT x] PRIMARY KEY (a key added later), and
marks a table's key unknown when a DO block alters it or the key is
PRIMARY KEY USING INDEX. Mattermost adds reactions' key in a DO block;
running DO bodies is its own change. USING INDEX identities are not
judged: whether the index still exists is not in the files.

Two older corpus entries had been reduced without the keys their sources
have; they now carry them, and their expectations do not change.
golang-migrate-down-files gets Mattermost's real teammembers key and
reactions' DO block. flatten-bare-columns gets REPLICA IDENTITY FULL:
the discussion's sample message has a whole before row on a keyless
table, which only FULL produces.

Rejected: judging what the sink reads from the before image (Flatten on
the envelope reading before.<column>). It is the README's longer promise
for this rule, needs the transform model to say which before fields a
sink reads, and is its own change.
@avison9 avison9 self-assigned this Nov 14, 2026
@avison9
avison9 merged commit 3f25f91 into main Nov 14, 2026
2 checks passed
@avison9
avison9 deleted the feat/replica-identity branch November 14, 2026 16:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant