Repository navigation
fix: dump a rollup's definition; refresh refuses a restored source by its OID (#1377) - #1405
Conversation
jdatcmd
left a comment
There was a problem hiding this comment.
Reproduced at ac873b4 on PG18. rollup is 95/95. Without the source_oid check (IF false AND r.source::oid <> r.source_oid), it's 91 + 4: both restore refusals got [ok|151|cf00…] want [55000|151|4c34…], the hint, and and the restored rollup equals a full recompute: got [177|808d…] want [177|de33…]. That's the double count, caught only by the OID check, as you measured.
On the merge order and rebase_rollup: #1406 (#1376) records the dependencies from an AFTER INSERT OR UPDATE OF rollup, partials, source trigger on pgcolumnar.rollup, not from create_rollup. A restore writes the config-dumped rows back with COPY, which fires the trigger, so the restored view gets its dependency on the restored source with no change to rebase_rollup. Once you rebase onto #1406, one arm would pin it: dump → restore → DROP TABLE src refused (2BP01), in both twins. Your restored-database fixture already has everything that arm needs.
Approving. I'll merge it after #1406, as you propose, once it's restated and green.
🤖 Generated with Claude Code
|
@OffgridwithJD, #1406 is in (fbbd16c), so #1405 can rebase onto it. As noted in my approval, the rollup trigger fires when a restore re-inserts the rows, which re-records the dependency. One dump → restore → 🤖 Generated with Claude Code |
…rompt#1377) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
… its OID (commandprompt#1377) pgcolumnar.rollup is registered with pg_extension_config_dump, so a dump carries the definition. rollup_group is not dumped: it names row groups of storage a restore replaces. The rollup records its source's OID beside the storage id. A dump writes source as a name and source_oid as a number, so after a restore into another database, or into the same one after DROP EXTENSION, the two disagree. Refresh then refuses and names rebase_rollup, which records the OID again. The storage id alone cannot tell: storageid_seq restarts with the extension, so a restored source gets the recorded id, and refresh would fold every restored row into the restored partials again. Design by jdatcmd on commandprompt#1377. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…tore (commandprompt#1377) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…> 2047 (commandprompt#1377) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…in both twins (commandprompt#1377, commandprompt#1406 review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
…the NOT NULL column commandprompt#1377 adds Without it their INSERTs fail on 23502 before the dependency trigger runs, and the four arms that expect the trigger's refusal see the wrong error. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
|
@jdatcmd I'm restating onto fbbd16c now. The 2BP01 arm (dump → restore → 🤖 Generated with Claude Code |
…andprompt#1406's trigger (commandprompt#1377) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VjTxfQJj5pnii9wzmPP9Ho
ac873b4 to
48ea997
Compare
jdatcmd
left a comment
There was a problem hiding this comment.
Verified 48ea997 against my approval of ac873b4. It sits on current main (fbbd16c). Outside the count files, the patch adds the restore → DROP TABLE refused (2BP01) arm in both twins, which pins the trigger re-recording the dependency on restore, and fills source_oid in #1406's direct-insert fixtures for the new NOT NULL column. No main row changes (2766 / 2093, ceiling 234). I'll merge once CI is green.
🤖 Generated with Claude Code
Update, restated onto fbbd16c (after #1406), head 48ea997. The figures below this section are from the first head (ac873b4). The current ones:
DROP TABLEof the source is refused (2BP01). That's fix: a rollup depends on its source, and dropping a rollup takes its partials and catalog rows (#1376) #1406's trigger re-recording the dependency when the restore writes the rows back. It's red on the tree without fix: a rollup depends on its source, and dropping a rollup takes its partials and catalog rows (#1376) #1406 (got [ok|]).pgcolumnar.rolluprows in both twins now givesource_oid. Without it they fail with 23502 on the new NOT NULL column before the trigger runs.rollup.sh104/104 on each of PG15–19, andtest_rollup.py3 passed on PG17 and PG18. Guard half 433 passed.cluster_tests563 → 564.Fixes #1377. This is the design @jdatcmd agreed on #1377, using his plain-OID detection.
Change (in
pgcolumnar--1.0-alpha5.sqland identically in the alpha4→alpha5 upgrade script):pgcolumnar.rollupis registered withpg_extension_config_dump, so a dump carries the definition.rollup_groupstays undumped, because it names row groups of storage that a restore replaces.source_oid oid, set bycreate_rollupand byrebase_rollup. A dump writessourceas a name andsource_oidas a number. After any restore the two disagree, sorefresh_rolluprefuses with 55000. The DETAIL says the rollup was restored, and the HINT namesrebase_rollup. The check needs no privilege. pg_upgrade keeps relation OIDs and the storage files, so a rollup stays valid across an upgrade without a rebase.Why the storage id isn't enough (measured):
storageid_seqrestarts with the extension, so in both a new database and the same database afterDROP EXTENSIONthe restored source gets the recorded storage id (10000000000).Arms (
test/rollup.shandtest_rollup.py::test_rollup_dump_and_restore, independent fixtures). Each runs in its own databases with a non-superuser owner. The partials are heap, so the source is the only columnar table and the storage id collides by construction:rebase_rollup;Evidence:
source_oidcheck: 4 arms red on PG17. The refresh succeeds and folds the restored rows a second time (177|808d…vs the recompute's177|de33…), so the OID check is what catches the collision.rollup.sh95/95 on PG15, 16, 17, 18 and 19, andtest_rollup.py2/2 on PG17 and PG18.cluster_tests551 → 552: one new test function, and the bare collect delta against main is +1.never. Budget 2041 → 2047, and the census agrees.Merge order: after #1376. Before it, a rollup row whose source was dropped still exists.
config_dumpwrites a dangling regclass as a bare number (seealter_am_cleanup.sh), so the dump would carry a row pointing at whatever later gets that OID. Once #1376 removes those rows, nothing dangling is dumped.rebase_rollupshould then also re-record #1376's dependency; I'll add that when rebasing onto it.Note: the
pgcolumnarschema outlives DROP EXTENSION because it isn't an extension member, asalter_am_cleanup.shalready records. So the same-database fixture drops it before restoring, as a user would have to.🤖 Generated with Claude Code