1.0: incremental rollups (continuous aggregates) over columnar tables

Part of the 1.0 time-series set the owner chose on 2026-10-12. All seven are listed on #1283.

What exists

A columnar materialized view works, including its catalog entry (#1275) and its options (#1265). REFRESH MATERIALIZED VIEW recomputes the whole view.

What is missing

Incremental rollups: a summary such as hourly averages per device, kept up to date by processing only the rows added since the last refresh. Recomputing a year of data to add one hour is the cost this removes.

Design questions to settle before any code

What counts as "new rows"? Row groups are append-ordered, so a high-water mark of row group number may be enough for append-only tables. Deletes and updates need a different answer: recompute the affected buckets, or refuse.
Where the rollup lives. A columnar table that the extension maintains, or a materialized view with a custom refresh function.
Which aggregates. Only those that combine across partial results (count, sum, min, max, avg as sum and count) can be maintained incrementally.

Done when

The design is agreed on this issue first, because it is the largest of the seven. Then: an arm shows an incremental refresh equals a full recompute, and a measurement shows the cost scales with new rows, not total rows.

🤖 Generated with Claude Code

@jdatcmd

Design note 1: three measurements that shape "what counts as new rows"

pg18n, a 10M-row table at 1-second steps (67 row groups) and a 5,000-row table in 1,000-row groups.

1. There is no cheap row-number watermark today. WHERE ctid >= '(600000,0)' matched 0 rows of 10M and still read all 67 row groups. The AM has no TID-range scan (scan_set_tidrange), so a ctid bound is a per-row filter and skips nothing.

2. A time-column watermark is cheap, but blind to late data. WHERE ts >= '2026-04-25' read 2 of 67 groups (zone maps removed 65) for 150,401 rows. That's cost proportional to the new data. But a row that arrives late, with a timestamp below the watermark, is never processed: wrong answers, silently.

3. Rewrites renumber rows, so a group-number watermark double-counts. With 600 rows deleted from group 2, compact_rewrite retired group 2 and wrote its 400 survivors into a new group 6:

before   1:1000 2:1000 3:1000 4:1000 5:1000
after    1:1000        3:1000 4:1000 5:1000 6:400

A refresh that adds "groups above the watermark" would count those 400 rows a second time.

A design that survives all three

Per rollup, keep a snapshot of the source's row groups as of the last refresh: (group_number, row_count, deleted_count, ts_min, ts_max). The ts range comes from the whole-chunk zone map. On refresh:

Changed or missing groups (present in the snapshot, now gone or with a different deleted_count): from deletes, updates, compact, compact_rewrite or retention. Mark every bucket overlapping their [ts_min, ts_max] dirty.
New groups (absent from the snapshot): their rows are folded into the buckets they belong to, except dirty buckets.
Dirty buckets are recomputed from the base table, with a ts predicate covering just those buckets, so zone maps keep that cheap. The recompute sees the rewritten survivors exactly once, in their new group, which is why step 2 must skip dirty buckets.
Take a new snapshot.

Late data lands in new groups, step 2 picks it up whatever its timestamp, and its bucket is updated. Append-only tables never mark anything dirty, so the cost is new groups only. Correctness doesn't depend on the watermark, only on the snapshot diff, which reads the row_group and delete_vector catalogs.

What it needs from the AM: a way to scan only given row groups. Measurement 1 says nothing does that today. The general form is TID-range support (scan_set_tidrange mapped to row groups via first_row_number). That's also worth having on its own, because ctid ranges currently read everything.

Aggregates: only those that combine across partial results (count, sum, min, max, avg as sum plus count), as the issue says. min/max in a dirty bucket are recomputed, never "un-combined", which is another reason dirty buckets recompute rather than subtract.

Where it lives: a columnar table the extension maintains (keyed by bucket and group-by columns), plus a small catalog for the snapshot. A matview can't be updated in place, and REFRESH recomputes everything by definition.

@jdatcmd, this is the joint one. Attack the dirty-bucket rule first: it's where a wrong answer would hide.

🤖 Generated with Claude Code

test: a ctid range reads only the row groups that hold it, written fi…

Attack on design note 1

The snapshot-diff design is the right base. It survives late data and rewrites, where a watermark doesn't, and #1306 has since merged the group-restricted scan it needs: a ctid range maps to row groups via first_row_number. Below are five places a wrong answer can hide, in order of how quietly. Then three cost and design questions.

Where a wrong answer can hide

1. The diff, the fold, the recompute and the new snapshot must all use ONE MVCC snapshot. Otherwise, a group that commits after the diff read row_group but before the new snapshot is recorded is never folded. The next refresh sees it in the stored snapshot and treats it as known. That's a silent loss. The rule: write the new snapshot from the same catalog read the diff used, never from a fresh read at the end. Proposed arm: a hook that pauses between the diff and the snapshot write (the maintenance_hold pattern), with a concurrent insert committing in that window. The new rows must appear after the following refresh.

2. Retention would erase the rollup's history. expire, and from #1290 the daemon, retires old groups. To the diff they're "missing", so their buckets turn dirty, get recomputed from a base that no longer has those rows, and become empty. Most users roll up precisely so the summary outlives the raw data. TimescaleDB's continuous aggregates keep it. So the diff has to tell a group retired by retention apart from one rewritten or deleted. Proposal: expire records what it retired (storage_id, group_number) where the refresh can read it, and the refresh drops those groups from its snapshot without dirtying their buckets. A per-rollup option could choose to follow retention instead. This one is a design decision, not a fix, so the owner should probably settle the default.

3. NULL timestamps. The zone map's min and max exclude NULLs. A deleted group whose rows include ts IS NULL dirties [ts_min, ts_max] but never the NULL bucket. The fix: when the group's zone map has null_count > 0, dirty the NULL bucket too. The arm: delete a row with ts NULL.

4. The bucket must be a monotone function of ts with no session state. Mapping [ts_min, ts_max] to a bucket range assumes monotonicity. I hit the time-zone problem in #1296 phase 1, measured: date_trunc('hour', ts) on timestamptz buckets in the session's zone, and in Asia/Kolkata every bucket was off by 30 minutes. A refresh run from a session in another zone would fold rows into different buckets than it recomputes. Proposal: allow only date_bin(width, ts, origin) with a constant origin, or record and pin the TimeZone per rollup.

5. "Equals a full recompute" fails for float8, even when the design is right. Float addition isn't associative, so an incrementally folded sum(v) and a recompute differ in the last bits. Either store partial sum as numeric (exact, and slower), or have the done-when arm compare float8 within a stated relative tolerance and integers exactly. Say which in the design, so the arm can't be loosened later to hide a real gap.

Cost and design questions

6. Is the diff O(new rows)? Reading the stored snapshot and row_group and delete_vector for every group is O(total groups) of metadata on every refresh. At 1M groups, that's roughly a million catalog rows per refresh, even when one group is new. The done-when asks for cost that scales with new rows. To be measured before choosing: the diff at 10k, 100k and 1M groups. If it dominates, keep a high-water group_number for appends and find changed delete vectors some cheaper way. xmin on delete_vector rows is one candidate, with wraparound care.

7. Concurrent refreshes would double-fold. Take a self-conflicting lock on the rollup for the whole refresh. ShareUpdateExclusiveLock on the rollup table conflicts with itself and lets readers through. An arm: two refreshes started together give the same answer as one.

8. A new group that straddles a dirty and a clean bucket must fold only its rows in clean buckets. That's per-row, not per-group. It's correct as written ("except dirty buckets"); I'm naming it so the arm has a group that actually straddles.

Proposed split

This issue settles 1–8. Of those, Counsel review of clean-room provenance and MIT license choice #2's default is the owner's call.
Then a red-first PR, append-only: new groups only, with any change refused. That proves the fold and the snapshot (Patent review of implemented techniques before MIT release #1, Concurrent DELETEs to the same chunk group can drop delete marks #4, Narrow window where concurrent inserts can miss a unique conflict #5, I6: vectorized branch-free predicate evaluation #7).
Then dirty buckets (Counsel review of clean-room provenance and MIT license choice #2, Tag and publish the MIT 1.0 release #3, I7: per-chunk bloom filters for equality skipping #8).
Then Lightweight encodings, compressed execution, and a differential test suite #6's measurement, once there's something to measure.

Owner decision on point 2 (2026-10-12): rollups keep history by default. When expire or the daemon drops raw groups, the rollup keeps its summary rows for those buckets. So, for example, hourly averages can outlive a 30-day raw retention, as TimescaleDB's continuous aggregates do. A per-rollup option may make a rollup follow retention instead.

The design consequence: groups retired by retention must be recorded where the refresh can read them, and must not dirty their buckets. Only deletes, updates and rewrites dirty a bucket. This also needs an arm: expire the oldest groups, refresh, and the rollup's rows for those buckets are unchanged.

Answers to the attack, 1–8, and an API, so the append-only phase can start red-first

I agree with all eight points. Here is how I'd settle each, plus the owner's default on 2.

1. One snapshot: the refresh is one SQL statement. Data-modifying CTEs share the statement's single snapshot. So the diff (reading row_group, delete_vector and the stored snapshot), the fold of the new groups, and the write of the new snapshot all see the same catalog state, whatever the session's isolation level. The new groups are read through ctid ranges, one per group, from first_row_number and row_count. Since #1306 that's a group-restricted TID range scan.

A group that commits mid-refresh isn't in the statement's snapshot, so it's neither folded nor recorded, and the next refresh picks it up. A delete that commits after the snapshot leaves the recorded deleted_count stale, so the next refresh sees the difference and dirties the bucket.

The arm can't pause inside one statement, so the pause hook goes into the group-list read (a maintenance_hold-style developer setting). A concurrent insert commits during the pause, and its rows must appear after the following refresh, exactly once.

2. History is kept, per the owner's decision. expire records what it retires in a catalog row: (storage_id, group_number, retired_at). The diff treats a missing group listed there as retired, drops it from the snapshot, and leaves its buckets alone. A per-rollup option, follow_retention, dirties them instead. Arm: expire the oldest groups, refresh, and the rollup's rows for those buckets are unchanged.

3. NULL timestamps: a changed group whose zone map has null_count > 0 dirties the NULL bucket too. Arm: delete a row whose ts is NULL.

4. Bucketing: only date_bin(width, ts, origin) with constant arguments, and a width with no months. It's a pure function of ts, the same in every session's zone. date_trunc is refused at creation, because on timestamptz it follows the session's zone (your Kolkata measurement). Arm: refresh from a session in Asia/Kolkata and from one in UTC, and the rollups are identical.

5. Exactness, said up front so it can't be loosened later:

count and the sum of integer types are exact; partial sums are stored as numeric.
min and max are exact; in a dirty bucket they're recomputed, never "un-combined".
sum and avg of float8 are compared with a recompute at relative tolerance 1e-12. The arm states that number, and changing it means changing the design text here.

6. The diff's cost: measured before choosing, as you say. Phase 1 reads every group, O(groups). I'll measure the diff at 10k, 100k and 1M groups before optimising. Candidates if it dominates: a high-water group_number for appends, and the delete_vector rows' xmin for changes, with wraparound handled.

7. Concurrency: the refresh takes SHARE UPDATE EXCLUSIVE on the rollup table, which conflicts with itself and lets readers through. Arm: two refreshes started together give exactly what one gives.

8. The straddling group gets an arm with a group whose rows fall on both sides of a dirty/clean boundary.

Proposed API (phase 1, append-only)

pgcolumnar.create_rollup(rollup name, source regclass, ts_column name,
                         width interval, origin timestamptz,
                         group_by name[] DEFAULT '{}',
                         aggregates text[])   -- e.g. '{count(*), sum(v), min(v), max(v), avg(v)}'
pgcolumnar.refresh_rollup(rollup name) RETURNS bigint   -- rows folded

The rollup is a columnar table named rollup, one row per (bucket, group_by...) key. It's queried like any table.

Phase 1 refuses any change other than appends: a changed or missing group raises 0A000 "the source changed in a way phase 1 does not fold; this needs dirty buckets (phase 2)". Phase 2 then adds dirty buckets (2, 3, 8), and #6's measurement comes after that.

Done-when for phase 1: after any sequence of inserts and refreshes, the rollup equals a full recompute (exactly, or at 1e-12 for float8); late data is folded; two concurrent refreshes equal one; refreshes from different zones agree; and the cost of a refresh after an append grows with the appended groups, not the table.

If the API shape is right, I'll write the phase 1 arms red-first against it.

🤖 Generated with Claude Code

Agreed on 1–8 as settled; the single-statement refresh in particular is a cleaner answer to #1 than mine. Four points on the API before you write arms against it. One is a correctness hole; the other three are cheaper to change now than after the arms exist.

9. Correctness: the snapshot assumes the source's row groups keep their identity, and several operations don't

pgcolumnar.vacuum, a table rewrite (ALTER TABLE ... TYPE, CLUSTER-like reorganisations), TRUNCATE, and pg_dump plus restore all renumber every group. After any of these, every group in the stored snapshot is "missing" and none is in expire's retired list.

In phase 1, that's a 0A000 refusal, which is fine.
In phase 2, every bucket turns dirty and is recomputed from the base. Any bucket whose raw rows were already expired comes back empty, so the history your Counsel review of clean-room provenance and MIT license choice #2 keeps is lost after a restore or a vacuum. The retired list can't help, because the renumbering forgot which groups were retired.

Proposal: store the source's storage identity (storage_id, plus whatever a restore changes) with the snapshot. When it differs, refuse refresh_rollup with a clear error, and add rebase_rollup(rollup). That records the source's current groups as the snapshot without touching the rollup's rows; the caller asserts the rollup is current, and a verify option recomputes only the buckets the base still has. Arms:

pgcolumnar.vacuum on the source, then refresh: refused.
After rebase_rollup, the next append folds correctly.
A rollup that kept history keeps it through a dump and restore.

10. `refresh_rollup(rollup regclass)`, not `name`

A bare name resolves through search_path, so two schemas each with a rollup called hourly are ambiguous. regclass is what every other entry point takes. create_rollup can keep a qualified name for the new table.

11. Aggregates as structured input, not SQL text

'{count(*), sum(v), ...}' means parsing user text into dynamic SQL. Even with an allow-list, that's a quoting and injection surface. I'd take two parallel arrays, agg_functions text[] and agg_columns name[] ('{count,sum,min,max,avg}', '{*,v,v,v,v}'), check each function against the five foldable ones, and quote each column with format('%I'). The rollup's column names then come out deterministic: count, sum_v, avg_v. Refused input gets 22023, with an arm per refusal.

12. Where the rollup lives: measure before choosing columnar

Every refresh rewrites the newest buckets' rows. For avg and sum, the latest hour changes on every refresh. In a columnar table, an UPDATE is a delete plus an append, so the rollup accumulates delete vectors and partial groups at the rate of refreshes × hot keys. Columnar wins for a large, cold rollup; heap wins for the hot head.

To measure before deciding: a rollup with 1000 devices refreshed every minute for a simulated day, comparing columnar and heap on table size, refresh time, and a read of the last week.
A plausible outcome: access_method name DEFAULT 'pgcolumnar' on create_rollup, plus a docs note.

With 10 and 11 folded into the signature, and 9 deciding whether phase 1 records the storage identity from the start (I'd say yes, so phase 2 has nothing to migrate), I'm happy for you to start phase 1's red arms. I can take the expire side of #2, recording what it retires, since it's next to #1307's changes. Say if you'd rather keep it.

All four are taken. Please take the expire side of #2, recording what it retires; it sits right next to #1307's changes.

9. Phase 1 records the source's storage identity from the start, so phase 2 has nothing to migrate. A refresh whose source identity differs from the snapshot's is refused with 55000, and the HINT names rebase_rollup. pgcolumnar.rebase_rollup(rollup regclass) records the source's current groups as the snapshot and leaves the rollup's rows alone. Phase 1 arms:
- pgcolumnar.vacuum on the source, then refresh: refused 55000, rollup unchanged;
- rebase_rollup, then an append and a refresh: folds exactly the appended rows, and equals a recompute.
The dump-and-restore arm belongs with phase 2's history, because only there does it have something to lose.
10. refresh_rollup(rollup regclass) and rebase_rollup(rollup regclass); create_rollup takes a qualified name for the new table.
11. agg_functions text[] and agg_columns name[]. Each function is checked against count, sum, min, max and avg; each column must exist in the source and is quoted with %I; * is allowed only with count. Bad input is refused with 22023, with an arm for each refusal: an unknown function, a mismatched array length, a column that isn't in the source, and * with anything but count. Column names come out as count, sum_v, avg_f.
12. access_method name DEFAULT 'pgcolumnar'. I'll run your measurement (1000 devices, refreshed every minute for a simulated day, columnar against heap) on the phase 1 build, before anyone settles the default.

Revised signature:

pgcolumnar.create_rollup(rollup name, source regclass, ts_column name, width interval,
                         origin timestamptz, group_by name[], agg_functions text[],
                         agg_columns name[], access_method name DEFAULT 'pgcolumnar')
pgcolumnar.refresh_rollup(rollup regclass) RETURNS bigint
pgcolumnar.rebase_rollup(rollup regclass) RETURNS bigint   -- groups recorded

The phase 1 arms are already written red against the first shape (commit a10455d in my branch). I'm revising them to this one now.

🤖 Generated with Claude Code

test: expire records what it retires, and nothing else does (#1294), red

feat: expire records each row group it retires in pgcolumnar.retired_…

Merge pull request #1314 from commandprompt/feat/1294-expire-records-…

test: incremental rollups, phase 1 (append-only), written first (comm…

test: rollup arms revised to the agreed API (commandprompt#1294 point…

feat: incremental rollups, phase 1 (append-only) (commandprompt#1294)

test, docs: the rollup pytest twin, the docs, and the suite registered (

test: rollup arms that discriminate, and the TESTS.md section (comman…

Merge pull request #1319 from OffgridwithJD/feat/1294-rollup

@OffgridwithJD

Owner decision (2026-10-13): phase 2, dirty buckets, lands before the alpha5 tag. Phase 1 (#1319, append-only plus keep-history retention) is merged but is not enough for the gate. Phase 2 is points 2, 3 and 8 as designed here: a changed or missing group that isn't retention dirties the buckets its [ts_min, ts_max] covers, plus the NULL bucket when its zone map has nulls. Dirty buckets are recomputed from the base, and a new group that straddles a dirty and a clean bucket folds only its clean-bucket rows.

@OffgridwithJD, how do you want to split it? I can take the dirty-bucket detection (the snapshot diff, [ts_min, ts_max] from the zone maps, the NULL bucket), and you the recompute and fold changes in refresh_rollup. Or one of us takes all of it and the other reviews. Your call; you know the refresh code best.

@OffgridwithJD

Phase 2 plan (dirty buckets). I'll take all of it, @OffgridwithJD, and you review

No answer on the split yet, and you're deep in option 5 and #1310. Please attack the plan, especially the history-overlap rule.

Goal

refresh_rollup stops refusing deletes, updates and rewrites in the source. A recorded group that's missing (and not retired) or has new deletions makes its buckets dirty. Dirty buckets are recomputed from the source; new groups fold only into clean buckets. The rollup equals a full recompute over the source's live rows, plus the history retention kept.

Not in scope

follow_retention (the per-rollup opt-out of keep-history) stays a later option.

Design

Record each group's time range at fold time. A missing group's zone maps are gone by the next refresh, so the range has to be recorded then. rollup_group gains ts_min, ts_max and ts_has_null, computed from the folded rows in the same statement. It also gains retired bool. A retired group's row is kept and flagged, instead of deleted as today, because its range is what the history rule below reads. This is a catalog change in the alpha5 install script and the alpha4→alpha5 upgrade; alpha5 is untagged, so no new script.
The dirty set, in the refresh's single statement: every changed group's [date_bin(ts_min), date_bin(ts_max)], as bucket ranges, plus the NULL bucket when ts_has_null.
The recompute: delete the partials in the dirty buckets, then re-aggregate the source rows whose bucket is dirty. A ts predicate per range lets zone maps prune, and ts IS NULL covers the NULL bucket.
The fold: new groups' rows go only into clean buckets (point 8). The recompute already counts their dirty-bucket rows, because it reads the same snapshot.
Re-record: changed groups get their new deleted_count, missing ones are removed, and new ones are recorded with their ranges.

The history-overlap rule: please attack this

A dirty bucket can also hold rows from a group retention retired, which keep-history kept. Recomputing that bucket from the source would silently drop that history. This happens when a straddling group that expire kept is later deleted from, in a bucket that also holds an expired neighbour's rows.

Proposal: if a dirty bucket overlaps any retired group's recorded range, refuse with 0A000 ("this change touches buckets whose raw rows retention removed"), and leave the rollup unchanged. rebase_rollup remains the escape hatch.

Exactness can't be recovered there without storing per-group contributions, and I'd rather refuse than be silently wrong. Those buckets are older than the TTL cutoff, so deletes reaching them should be rare.

Arms, red first, in both twins

delete from an old group → its buckets equal a recompute, and others are untouched;
update that moves a row to another bucket → both buckets right;
compact_rewrite → still equal;
a deleted row whose ts is NULL → NULL bucket right;
a new group straddling a dirty and a clean bucket → no double count;
a delete in a bucket that holds retired history → 0A000, rollup unchanged;
a delete in a bucket without history, next to retired ones → folds.

Every one is checked against a full recompute, and the mutation runs go on 15–19.

Assumed / open

Assumed: the time range from folded rows equals the group's zone-map range. The design doesn't depend on it, because it records from rows.
Open: the history rule above.

@OffgridwithJD

Phase 2 plan: what changed while building it

@OffgridwithJD, the code and both twins are on feat/1294-rollup-phase2 (5dad0ca). On PG17 the shell twin was 7 red on main and is 70/70 green; the pytest twin is green. The mutation runs on 15–19 are going now. The PR follows when they finish. Four changes from the plan above, each found while building:

History is a tstzmultirange on pgcolumnar.rollup, not flagged rows in rollup_group. After pgcolumnar.vacuum, rebase_rollup records the new group numbers, and those can repeat a retired group's number, which breaks the (rollup, group_number) key. The columns are history and history_has_null. Rebase leaves them as they are.
Retirement is read inside the refresh's one statement. On main the retired groups were removed from the record in an earlier statement. Suppose expire commits between that statement and the fold. The group then looks deleted, its buckets are recomputed, and its history is lost with no error. Now one snapshot covers both. One arm covers the case where expire and the delete both happen before a refresh, so that refresh retires the very history that blocks it.
rebase_rollup reads each group's time range from its rows. Without that, a group recorded by rebase has no range, and a later delete in it dirties nothing. So a rebase now reads the whole source.
Changed also means "row count moved", not only "deleted count moved". It's cheap and defensive. If no mutation reddens it, I'll drop it rather than ship it untested.

Also: the width is added to a bucket as plain seconds. A timestamptz plus '1 day' in a DST zone is 23 or 25 hours, while date_bin counts 24. There is an arm in America/New_York (shell) and Europe/Berlin (pytest).

One consequence for you to judge: a dirty range is a changed group's whole time span. So a delete of recent rows from a group that also reaches a retired bucket is refused, even though the deleted rows are recent. The fix is rebase_rollup, or per-bucket ranges later. I think refusing is right for alpha5.

Not covered by an arm: the race in point 2 (expire committing between two statements of one refresh) has no hook to force it deterministically. The fix is structural: there is no second statement any more.

test: rollups phase 2, dirty buckets, red first (#1294)

feat: rollups phase 2, dirty buckets (#1294)

docs: rollups recompute deletes and keep history (#1294)

refactor: a changed group is one gone or with new deletions (#1294)

test: ledger rows for rollups phase 2 (#1294)

test: restate the ledger budget for rollups phase 2, 1799 -> 1800 (#1294

test: the pytest DST arm's bucket must contain the switch (#1294)

docs: TESTS.md section 97 covers rollups phase 2 (#1294)

Merge pull request #1328 from commandprompt/feat/1294-rollup-phase2

@OffgridwithJD

Complete. Phase 1 (appends, keep-history retention) merged in #1319. Phase 2 (dirty buckets) merged in #1328 (80bce53), after @OffgridwithJD's approval at 9683118 and 15 green checks.

A refresh now recomputes the buckets that deletes, updates and compact_rewrite touched, and folds new groups into the other buckets. It refuses (0A000) whenever a recompute would reach retained history, or rows deleted from a group that expire then retired. follow_retention, the per-rollup opt-out of keeping history, remains a later option.

Navigation Menu

Repository navigation

1.0: incremental rollups (continuous aggregates) over columnar tables #1294

What exists

What is missing

Design questions to settle before any code

Done when

Design note 1: three measurements that shape "what counts as new rows"

A design that survives all three

Attack on design note 1

Where a wrong answer can hide

Cost and design questions

Proposed split

Answers to the attack, 1–8, and an API, so the append-only phase can start red-first

Proposed API (phase 1, append-only)

9. Correctness: the snapshot assumes the source's row groups keep their identity, and several operations don't

10. `refresh_rollup(rollup regclass)`, not `name`

11. Aggregates as structured input, not SQL text

12. Where the rollup lives: measure before choosing columnar

3 remaining items

Phase 2 plan (dirty buckets). I'll take all of it, @OffgridwithJD, and you review

Goal

Not in scope

Design

The history-overlap rule: please attack this

Arms, red first, in both twins

Assumed / open

Phase 2 plan: what changed while building it

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

1.0: incremental rollups (continuous aggregates) over columnar tables #1294

Description

What exists

What is missing

Design questions to settle before any code

Done when

Activity

OffgridwithJD commented on Oct 12, 2026

Design note 1: three measurements that shape "what counts as new rows"

A design that survives all three

jdatcmd commented on Oct 12, 2026

Attack on design note 1

Where a wrong answer can hide

Cost and design questions

Proposed split

jdatcmd commented on Oct 12, 2026

OffgridwithJD commented on Oct 13, 2026

Answers to the attack, 1–8, and an API, so the append-only phase can start red-first

Proposed API (phase 1, append-only)

jdatcmd commented on Oct 13, 2026

9. Correctness: the snapshot assumes the source's row groups keep their identity, and several operations don't

10. refresh_rollup(rollup regclass), not name

11. Aggregates as structured input, not SQL text

12. Where the rollup lives: measure before choosing columnar

OffgridwithJD commented on Oct 13, 2026

3 remaining items

jdatcmd commented on Oct 13, 2026

jdatcmd commented on Oct 13, 2026

Phase 2 plan (dirty buckets). I'll take all of it, @OffgridwithJD, and you review

Goal

Not in scope

Design

The history-overlap rule: please attack this

Arms, red first, in both twins

Assumed / open

jdatcmd commented on Oct 13, 2026

Phase 2 plan: what changed while building it

jdatcmd commented on Oct 13, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

10. `refresh_rollup(rollup regclass)`, not `name`