add a toggle to filter empty schemas and db#9712
Open
Light2Dark wants to merge 2 commits into
Open
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
1 issue found across 3 files
Architecture diagram
sequenceDiagram
participant UI as DataSources Panel
participant Store as Jotai Store (localStorage)
participant Backend as Backend API
participant Cache as React useMemo
Note over UI,Cache: Initial Load
UI->>Store: NEW: read hideEmptyDatasourcesAtom (from localStorage key "marimo:datasources:hideEmpty")
Store-->>UI: false (default)
UI->>Backend: fetch databases & schemas
Backend-->>UI: rawConnections with all databases & schemas
Note over UI,Cache: Toggle Filter
UI->>UI: user clicks Eye/EyeOff button
UI->>Store: NEW: setHideEmpty(!hideEmpty)
Store-->>UI: confirm toggle
Note over UI,Cache: Re-render with filter applied
UI->>Cache: NEW: useMemo recomputes dataConnections
Cache->>Cache: NEW: filterEmptyDatabases() applied per connection
alt hideEmpty === false
Cache-->>UI: return rawConnections unchanged
else hideEmpty === true
Note over Cache: For each connection.databases:
Cache->>Cache: filter schemas with 0 tables
Cache->>Cache: filter databases where all schemas are empty
alt database.schemas.length === 0
Note over Cache: preserve - lazy state
end
Cache-->>UI: filteredConnections
end
alt filteredConnections changed
UI->>UI: re-render tree with filtered data
else no change (immuatable check)
UI->>UI: return previous reference
end
Note over UI,Backend: Unchanged Backend Behavior
Note over Backend: CHANGED: get_schemas() no longer filters empty schemas server-side
Note over Backend: now returns ALL schemas regardless of table count
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
## 📝 Summary <!-- If this PR closes any issues, list them here by number (e.g., Closes #123). Detail the specific changes made in this pull request. Explain the problem addressed and how it was resolved. If applicable, provide before and after comparisons, screenshots, or any relevant details to help reviewers understand the changes easily. --> These fields are to indicate whether a table's schemas has been loaded vs truly empty. That allows the frontend to distinguish whether to hide empty vs lazy (which we shouldn't hide). This change makes sense for future work, since it's important to distinguish between truly empty vs not yet loaded. Another option was considered: `list[DataTable] | None`, where `None` indicates not yet loaded. However, this is semantically not clear even if a simpler code change. This is in support of this PR #9708 ## 📋 Pre-Review Checklist <!-- These checks need to be completed before a PR is reviewed --> - [x] For large changes, or changes that affect the public API: this change was discussed or approved through an issue, on [Discord](https://marimo.io/discord?ref=pr), or the community [discussions](https://github.com/marimo-team/marimo/discussions) (Please provide a link if applicable). - [x] Any AI generated code has been reviewed line-by-line by the human PR author, who stands by it. - [ ] Video or media evidence is provided for any visual changes (optional). <!-- PR is more likely to be merged if evidence is provided for changes made --> ## ✅ Merge Checklist - [x] I have read the [contributor guidelines](https://github.com/marimo-team/marimo/blob/main/CONTRIBUTING.md). - [ ] Documentation has been updated where applicable, including docstrings for API changes. - [x] Tests have been added for the changes made. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Bundle ReportChanges will increase total bundle size by 27.09kB (0.11%) ⬆️. This is within the configured threshold ✅ Detailed changes
Affected Assets, Files, and Routes:view changes for bundle: marimo-esmAssets Changed:
Files in
Files in
|
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a per-user toggle in the Datasources panel to hide databases and schemas that are confirmed empty, while still showing those whose contents are merely deferred. To support this safely, a new tables_resolved/schemas_resolved flag is propagated through every SQL engine and the public API schema so the frontend can distinguish "known empty" from "not yet fetched" (closes #6807).
Changes:
- Add
schemas_resolvedtoDatabaseandtables_resolvedtoSchemain the data models, OpenAPI YAML, and generated TS types; defaults toTruefor backward compatibility. - Update SQL engines (sqlalchemy, ibis, clickhouse, redshift, pyiceberg, adbc) and connection update helpers to set these flags, and stop unconditionally hiding empty schemas in the ibis engine.
- Add a
hideEmptyDatasourcesAtom(persisted viaatomWithStorage) and afilterEmptyDatabaseshelper plus an Eye/EyeOff toggle button in the datasources panel UI; ship vitest coverage for the filter and Python tests for engine behaviors.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| marimo/_data/models.py | Adds tables_resolved / schemas_resolved fields with docstrings. |
| marimo/_sql/connection_utils.py | Marks resolved=true after schemas/tables are populated. |
| marimo/_sql/engines/sqlalchemy.py | Marks meta-schema tables as unresolved; populates schemas_resolved. |
| marimo/_sql/engines/ibis.py | Stops dropping empty schemas; threads schemas_resolved/tables_resolved. |
| marimo/_sql/engines/clickhouse.py | New _get_tables_in_schema_with_resolution returns authoritative flag. |
| marimo/_sql/engines/redshift.py | Sets schemas_resolved and tables_resolved based on inputs. |
| marimo/_sql/engines/pyiceberg.py | Records tables_resolved from auto-discover decision. |
| marimo/_sql/engines/adbc.py | Sets resolution flags from include_* booleans. |
| packages/openapi/api.yaml, packages/openapi/src/api.ts | Adds new optional boolean fields with default: true. |
| frontend/src/core/datasets/data-source-connections.ts | Marks resolved=true after schema/table updates. |
| frontend/src/components/datasources/datasources.tsx | Adds storage atom, filter helper, and toggle button. |
| frontend/src/components/datasources/tests/filter-empty.test.ts | Vitest coverage for filterEmptyDatabases edge cases. |
| tests/_sql/test_sqlalchemy.py, test_ibis.py, test_clickhouse.py | Asserts new resolved flags across engines and failure paths. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
📝 Summary
Closes #6807.
This toggles empty db and schemas. By default all are shown, this config is stored in local storage.
Screen.Recording.2026-05-28.at.5.35.47.PM.mov
📋 Pre-Review Checklist
✅ Merge Checklist