Skip to content

Add TOP, BOTTOM and SAMPLE; fix subqueries over aggregates - #127

Merged
austin-barrington merged 2 commits into
mainfrom
fix/query-gaps
Oct 24, 2026
Merged

austin-barrington merged 2 commits into
mainfrom
fix/query-gaps

Conversation

@austin-barrington

Copy link
Copy Markdown
Member
  • top(), bottom() and sample() work. They were rejected with "unsupported aggregate function".
    • They return up to N stored points per group (GROUP BY time() bucket and tags), each at its own time, in time order. top and bottom break ties by the earliest point.
    • top/bottom take tag arguments (at most one point per combination). Other fields or tags can be selected beside them, taken from the same points.
    • LIMIT/OFFSET apply to the points returned. Combining them with another function fails with InfluxDB's error.
    • Sharded measurements: each region returns its own N per group, and the coordinator ranks the union again, then applies LIMIT/OFFSET. The top N of a union of per-region top-N lists is the global top N, also with tag arguments.
    • The SQL keeps the usual SELECT * FROM (\n...\n) layout, so region and tombstone predicates still filter the points before ranking.
  • Subqueries over aggregates work. The inner bucket was renamed to time inside its own SQL, so ClickHouse read time in its GROUP BY as the bucket (NOT_AN_AGGREGATE). An inner aggregate without buckets had no time column at all.
    • The inner query is now wrapped and exposes a real time: the bucket, a lone selector's point time, or the query's lower bound (as InfluxDB does).
  • Sharded subqueries work. The gathered inner rows exposed only __time (a string, or the client's epoch unit), so every outer query on time failed. They now carry a real time column.
  • A raw field with GROUP BY time() returns InfluxDB's "GROUP BY requires at least one aggregate function" instead of a chDB error.

Tests:

  • Compat suite (real chDB): values checked by hand against the data. The suite covers top with and without tags, extra fields, bottom per bucket and host, sample counts, the error cases, subqueries over buckets, over unbucketed aggregates, and over and under top.

  • Two-region sharded tests:

    • global top 2 and top per minute
    • bottom per host
    • LIMIT/OFFSET after ranking
    • sample
    • a subquery over buckets

    With the coordinator re-rank removed, the sharded test fails, returning each region's own top 2.

  • Unit tests for the re-rank.

  • CI-equivalent run (libchdb 26.9.0, -D warnings): passes.

Docs: the selector functions are added to the reference; the clustering deep-dive lists how they merge across regions.

- top(), bottom() and sample() were unsupported ("unsupported aggregate
  function"). They now return up to N stored points per group (GROUP BY
  time() bucket and tags), each at its own time, in time order; top and
  bottom take tag arguments (one point per combination), and other fields
  or tags may be selected beside them. Combined with another function they
  fail as in InfluxDB. On a sharded measurement each region returns its own
  N per group and the coordinator ranks the union again, then applies LIMIT
  and OFFSET.
- A subquery over an aggregate failed: the inner bucket was renamed to
  `time` inside its own SQL, so ClickHouse read `time` in its GROUP BY as the
  bucket (NOT_AN_AGGREGATE), and an inner aggregate without buckets had no
  `time` at all. The inner query is now wrapped and exposes `time`: the
  bucket, a lone selector's point time, or the query's lower bound.
- On a sharded measurement the gathered inner rows exposed only `__time`
  (a string or the client's epoch unit), so any outer query on time failed.
  They now carry a real `time` column.
- A raw field with GROUP BY time() returns InfluxDB's "GROUP BY requires at
  least one aggregate function" instead of a chDB error.
@cursor

cursor Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches core query translation, subquery wrapping, and sharded scatter-gather merge paths; behavior changes are broad but covered by compat and cluster integration tests.

Overview
Adds InfluxQL selector functions top(), bottom(), and sample(): up to N points per GROUP BY time() bucket/tag set, with optional tag arguments, LIMIT/OFFSET on returned points, and Influx-style validation (no mixing with other functions; GROUP BY time() without an aggregate errors clearly).

Query engine: New ClickHouse translation ranks points per group (hidden __bucket / __tns columns for coordinator merge). Sharded clusters: each region returns local top-N; the coordinator re-ranks the union (exact for top/bottom), then applies LIMIT/OFFSET—documented in clustering deep-dive.

Subqueries: Fixes aggregate/subquery time exposure (subquery_source, nanosecond inner results on shards) so nested GROUP BY time() and outer filters on time work instead of ClickHouse NOT_AN_AGGREGATE or missing time.

Compat and two-region integration tests cover selectors, subqueries, and multi-region behavior.

Reviewed by Cursor Bugbot for commit 3637d91. Configure here.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 3637d91. Configure here.

.await?;
return Ok(crate::domain::sharding::multi_selector::rerank(
&sel, &named, series,
));

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LIMIT applied twice on local fallback

Medium Severity

When shard_routing is set, top/bottom/sample always rerank, which applies LIMIT/OFFSET again. Region SQL strips those clauses first, but the no-space path still runs the original statement locally, so LIMIT/OFFSET is already in the SQL. A query such as top(v, 3) LIMIT 1 OFFSET 1 can return too few points or none on a sharded node that falls back to a local table.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 3637d91. Configure here.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right, thanks. Fixed in e533adf: the sharded path now gets the statement without LIMIT/OFFSET on every route (regions and the local fallback), and only the re-rank applies them. Test: top_limit_offset_on_the_local_fallback_applies_once (drops the shard space so the node falls back to its table); it returns no points with the old code.

@cursor cursor Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale comment

Left a non-blocking comment: Cursor Bugbot reported one unresolved medium-severity issue on this head (LIMIT/OFFSET can be applied twice on the local fallback path), so I am not approving. Human review is needed, and no reviewers were assigned.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

…llback

A measurement with no shard space is queried from the node's own table,
with the statement as given: its SQL applied LIMIT and OFFSET, then the
coordinator's re-rank applied them again, so `top(v, 3) LIMIT 1 OFFSET 1`
returned nothing. The sharded path now gets the statement without them, on
every route; only the re-rank applies them.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Cursor Bugbot was not running on this head after the first check poll, and the earlier LIMIT/OFFSET finding no longer applies on the current sharded path. CI completed successfully.

Open in Web View Automation 

Sent by Cursor Approval Agent: Pull Request Router and Approver

@austin-barrington
austin-barrington merged commit 31bb44d into main Oct 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant