Skip to content

Real-server tests, streaming and server-side Stop for more engines, plans for document engines - #312

Merged
broisnees merged 6 commits into
nightlyfrom
fix/hard-cases-2
Nov 20, 2026
Merged

broisnees merged 6 commits into
nightlyfrom
fix/hard-cases-2

Conversation

@broisnees

Copy link
Copy Markdown
Collaborator

The second round on the five hard cases. This time I tested against real servers (SQL Server, Oracle 23, ClickHouse, MongoDB 7, Elasticsearch 8, CouchDB 3), not only fakes.

Big results

  • Oracle streams rows as the session fetches them: 2M rows, flat memory.
  • MongoDB find and aggregate stream from the cursor past the old 5,000-document cap. Fields that first appear after the first 1,000 documents are kept in (other fields).
  • DuckDB executes in its streaming mode, so it no longer materializes the whole result first. A mid-stream error is returned as an error; the driver's own iterator would have panicked and closed the app.

Stop

  • Elasticsearch and OpenSearch: requests carry an X-Opaque-Id, and Stop cancels the tasks holding it.
  • PostHog: queries carry a client_query_id, and Stop sends DELETE for it.
  • MongoDB: streamed operations are tagged with a comment, and Stop runs killOp on them.

Plans

  • Explain draws plans for MongoDB, Elasticsearch/OpenSearch (search profile), CouchDB (_explain) and libSQL.
  • The plan view flags a MongoDB COLLSCAN or CouchDB all-documents scan that throws rows away.

Transactions

  • Live tests pass on SQL Server and Oracle for rollback, commit and an abandoned transaction.
  • A transaction's SQL Server client now leaves the cancel registry when it closes.

UI

  • Filter rows in menus and the terminal's find box use the same outline as every page search.

Startup

  • Release build: the window shows in 730-1,080 ms (median about 835 ms), down from about 1.07 s.

Checks:

  • 431 Rust tests and 1,845 vitest pass, svelte-check reports 0 errors, design lint is clean.
  • The live suites (SSH tunnels, Postgres, Oracle, ClickHouse, MongoDB, Elasticsearch, CouchDB) pass against real servers.

…erver sessions

Transactions round-trip on real SQL Server and Oracle servers (rollback,
commit, abandoned and released). ClickHouse streams 2M rows and Stop leaves
nothing in system.processes. A transaction's SQL Server client now leaves the
cancel registry when it closes instead of keeping its login there.
Oracle queries feed the grid or result store as the session fetches them
instead of collecting up to the budget; Stop breaks the call. MongoDB find
and aggregate stream from the cursor past the console's 5,000-document cap,
with fields the first 1,000 documents lacked kept in (other fields). Both
verified on real servers.
Elasticsearch and OpenSearch requests carry an X-Opaque-Id and Stop cancels
the tasks holding it. PostHog queries carry a client_query_id and Stop sends
DELETE for it. A streamed MongoDB find or aggregate is tagged with a comment
and Stop runs killOp on it. Elasticsearch and MongoDB verified on real
servers; PostHog against a fake that checks the cancel call.
Explain draws MongoDB's executed stages (aggregate stages above the
cursor), a search's profile per shard and Lucene query, CouchDB's _explain
index choice, and libSQL's SQLite plan in the plan view. The full-scan
warning knows COLLSCAN and CouchDB's all-documents scan. Verified on real
MongoDB 7, Elasticsearch 8 and CouchDB 3.
Menu and popover filter rows (chart picker, database switcher, data diff
pickers, column finder, model picker) are now an inset search-shell, the
same outline as every page search. The terminal's find box drops its drop
shadow for the outline and lights its border on focus.
Console reads execute in DuckDB's streaming mode and are read through the
row API (raw_query), so DuckDB no longer materializes the whole result
first: 2M rows now grow memory under 120 MB where DuckDB alone held 214 MB.
The driver's Arrow iterator panics on a mid-stream error; the row API
returns it, and a test covers that.
@broisnees
broisnees merged commit 39e9a92 into nightly Nov 20, 2026
1 check passed
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant