Skip to content

fix: parallelize DescribeTable calls in get_tables - #57

Merged
fuleinist merged 2 commits into
TabularisDB:mainfrom
fuleinist:fix/get-tables-parallel-describe
Aug 2, 2026
Merged

fix: parallelize DescribeTable calls in get_tables#57
fuleinist merged 2 commits into
TabularisDB:mainfrom
fuleinist:fix/get-tables-parallel-describe

Conversation

@fuleinist

Copy link
Copy Markdown
Collaborator

Summary

get_tables issued DescribeTable serially for every table returned by ListTables. On AWS accounts with hundreds of tables this takes over a minute (~300ms per describe), which exceeds the Tabularis connection timeout — the initial connection fails with an unhandled error before the table tree ever loads.

Measured on a real account with 400 tables: 75s → 7s, with full metadata (item_count, table_size_bytes, table_status) preserved.

Changes

  • get_tables describes tables with bounded concurrency (16 in flight via futures::stream::buffer_unordered) instead of a serial loop
  • Results re-sorted alphabetically after collection to preserve ListTables ordering (buffer_unordered completes out of order)
  • Per-table failure fallback (table_status: "UNKNOWN") unchanged
  • futures = "0.3" added as a dependency

Test plan

  • cargo test — 175 passed
  • cargo build --release clean
  • Manual JSON-RPC against a live 400-table AWS account: get_tables completes in ~7s with correct per-table metadata
  • Maintainer: verify against DynamoDB Local (just seed-dynamodb, REPL :tables)

No credentials, endpoints, or account-specific data included in this change.

Made with Cursor

get_tables issued DescribeTable serially for every table returned by
ListTables. On AWS accounts with hundreds of tables this took over a
minute (~300ms per describe), exceeding the GUI's connection timeout
and failing the initial connection.

Describes now run with bounded concurrency (16 in flight via
buffer_unordered) and results are re-sorted alphabetically to preserve
ListTables ordering. Measured on an account with 400 tables: 75s -> 7s,
with full metadata (item_count, table_size_bytes, table_status)
preserved.

Co-authored-by: Cursor <cursoragent@cursor.com>
CI was failing because cargo fmt --check found formatting differences
on the sort_by line. Reformat to match rustfmt expectations.

This fix applies to the get_tables handler where parallel DescribeTable
results are sorted alphabetically.
@fuleinist
fuleinist merged commit c2cb873 into TabularisDB:main Aug 2, 2026
1 check passed
fuleinist pushed a commit that referenced this pull request Aug 2, 2026
The Tabularis app deserializes query results into a QueryResult whose
Pagination struct requires `page`, `page_size`, `total_rows` (optional)
and `has_more`. The plugin only sent `pagination: {"next_token": ...}`,
so the app rejected every query response with "missing field `page`"
(then `page_size`, then `has_more` as each was worked around).

items_to_response now always emits a complete pagination object:
- page: request's `page` param (default 1)
- page_size: request's `limit`, falling back to returned row count
- total_rows: null (DynamoDB cannot cheaply count a filtered result)
- has_more: existing next_token/truncation logic
- next_token: preserved alongside for resume

Stacked on #57.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant