Repository navigation
Conversation
feat(storage): initial Filebrowser UI implementation
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
…ation S3 File Browser Page Navigation (Pagination)
* #93 add modal preview for file display * #93 fix broken e2e tests * #93 add preview for parquet files, fix scrollbar visibility in preview, allow ms excel csv data for preview * #93 improve parquet row fetch handling * #93 fix lint * #93 move preview endpoint logic into separate files * #93 disable parquet preview
* refactor(ObjectTable): extract File and Folde rows into seperate components * style: format code * style: use camelCase * style: use camelCase
* feat: implement single-file download functionality and error handling for S3 storage * feat: refactor S3 download endpoint to use new API structure and improve error handling * chore: remove obsolete note about static download route segment in tech debt documentation * style: format code * fix: improve type safety in S3 error handling * fix: preview action * feat: add context menu header and improve file selection handling in FileExplorer * fix: reset selection state on context menu close and refine download action logic * style: format files * style: remove pointer events from Actions row on ContextMenu * #91 fix lint --------- Co-authored-by: F3l1x1vo <fli@ordix.de>
* #118 add pinned access * #118 add recently accessed file and locations * #118 show loading spinner for quick navigation, use daisyui tooltips, limit recent access item count * #118 show timestamp on hover, add preview action, add clickable label for recent locations * #118 refactor timestamp tooltip into own component, add timestamp tooltip to file explorer files * #131 fix download path for truncated previews, #118 add button to open context actions on pinned access * #118 refactor suggested code from #126, show pin location menu above breadcrumb menu * #118 fix timestamp string format, make timestamp display relative * #118 add month and year suffixes to last modified timestamp display * #118 fix visuals for pinned access entry width
Co-authored-by: F3l1x1vo <fli@ordix.de>
…onnection configuration (#149)
* feat: implement file upload feature with S3 integration * fix: show new filename on rename * refactor: remove mock S3 server and related E2E tests for file upload feature * refactor: reduce duplicate code * style: formatting and linting * #92 add multiple files and directory upload, improve file rename checks on upload * #144 remove trailing slash on directory name * * #94 add delete action * #94 refactor, implement delete action in hdfs provider * #94 cleaned up types, use paraglide pluralisation * * #118 add pinned access * #118 add recently accessed file and locations * #118 show loading spinner for quick navigation, use daisyui tooltips, limit recent access item count * #118 show timestamp on hover, add preview action, add clickable label for recent locations * #118 refactor timestamp tooltip into own component, add timestamp tooltip to file explorer files * #131 fix download path for truncated previews, #118 add button to open context actions on pinned access * #118 refactor suggested code from #126, show pin location menu above breadcrumb menu * #118 fix timestamp string format, make timestamp display relative * #118 add month and year suffixes to last modified timestamp display * #118 fix visuals for pinned access entry width * fix duplicate json * fix: remove duplicate invalidateAll import in FileExplorer * fix: remove s3-mock from playwright config * remove test with wrong references * fix: update eslint ignore patterns to include .github/skills directory * fix: add exclude pattern for .github/skills directory in pre-commit config --------- Co-authored-by: F3l1x1vo <fli@ordix.de>
* #130 improve various ux issues from this ticket * #130 improve various ux issues from this ticket part 2 * #130 add badge and icon to file count display * chore: remove package duplicates with npm dedupe * feat: Add support for path-style and VirtualHosted addressing in S3 connection configuration (#149) * #92 S3 File Browser: Upload (#141) * feat: implement file upload feature with S3 integration * fix: show new filename on rename * refactor: remove mock S3 server and related E2E tests for file upload feature * refactor: reduce duplicate code * style: formatting and linting * #92 add multiple files and directory upload, improve file rename checks on upload * #144 remove trailing slash on directory name * * #94 add delete action * #94 refactor, implement delete action in hdfs provider * #94 cleaned up types, use paraglide pluralisation * * #118 add pinned access * #118 add recently accessed file and locations * #118 show loading spinner for quick navigation, use daisyui tooltips, limit recent access item count * #118 show timestamp on hover, add preview action, add clickable label for recent locations * #118 refactor timestamp tooltip into own component, add timestamp tooltip to file explorer files * #131 fix download path for truncated previews, #118 add button to open context actions on pinned access * #118 refactor suggested code from #126, show pin location menu above breadcrumb menu * #118 fix timestamp string format, make timestamp display relative * #118 add month and year suffixes to last modified timestamp display * #118 fix visuals for pinned access entry width * fix duplicate json * fix: remove duplicate invalidateAll import in FileExplorer * fix: remove s3-mock from playwright config * remove test with wrong references * fix: update eslint ignore patterns to include .github/skills directory * fix: add exclude pattern for .github/skills directory in pre-commit config --------- Co-authored-by: F3l1x1vo <fli@ordix.de> * chore: remove package duplicates with npm dedupe * #130 improve various ux issues from this ticket part 3 * #130 remove rename and move actions * #130 add locale to file size display * #130 centralize file size locale formatting * #130 allow custom file size format options * chore: remove unused import * style: enhance item count badges with improved icon size and font weight * Feat/s3 file browser button on breadcrumbs (#158) * feat: add pin button to breadcrumbs for easy location management * refactor: update pinButton snippet parameters for readability * #130 add space-taking transition to pin hover and reduced motion attribute --------- Co-authored-by: F3l1x1vo <fli@ordix.de> --------- Co-authored-by: dklOrdix <dkl@ordix.de>
* feat: Trino functions in Code Completion (#78) * Add Trino functions to completion list * fix: Unclosed subqueries (#110) * correctly classify completion slots inside unclosed parenthesised subqueries * chore: Helm cleanup (#109) * add missing envs to helm chart & add probes with /healthz endpoint * update tech debt * change trino.completionDisabled to features.trino.completion.enabled * refactor: use global states * refactor: reorganise component structure * #159 Add VS Code Settings and Extensions (#160) * #159 add vs code settings and extensions * #159 update deprecated setting * #159 remove ruler * refactor: improve logging levels and clean up unused code * fix: fix bookmarks handling and improve loading state management * refactor: adjust z-index values for improved layering in BucketList component * refactor: enhance unpin menu functionality and add close button * refactor: enhance unpin context menu with improved positioning and close button * refactor: streamline unpin menu handling and improve focus management * refactor(frontend): clean up and restructure app (#84) * refactor(frontend): clean up and restructure app * fix: add lint:fix script to package.json and clean up trailing comma in +layout.svelte * docs: update AGENTS.md to include Material UI icon library usage * refactor(header): move components for header into own directory * chore: dedupe lockfile * fix(sidebar): add hover cursor style to toggle button * chore(types): centralise navigation and auth types; update header/sidebar imports Co-authored-by: Copilot <copilot@github.com> * #114 save sidebar collapse state in localStorage * refactor: merge if statements * refactor: increase timeout for new clusters Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: unused export Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: restore hr seperator Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: restore previous layout css Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: replace Iconify with unplugin-icons * refactor: update icon library usage to unplugin-icons with material-symbols --------- Co-authored-by: F3l1x1vo <fli@ordix.de> Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: update bucket icons to use gravity-ui and add new dependency * chore: move gravity-ui to dev-dependencies --------- Co-authored-by: Benedikt Labrenz <benedikt@labrenz.org> Co-authored-by: F3l1x1vo <fli@ordix.de> Co-authored-by: Lukas Krug <lukas.krug@stackable.tech>
* #135 add garage instance to GitHub workflow for e2e tests * #135 fix yaml lint * #135 fix docker command in action * #135 fix docker options in action * #135 fix yaml format * #135 refactor GitHub action for s3 deployment with code fixes * #135 fix missing entrypoint in container * #135 remove format use on s3 creation * #135 add e2e tests that require a bucket * #135 allow local starting of s3 e2e tests, fix lint * #135 fix lint * #135 move garage.toml to permanent repository * #135 create keys and bucket via curl * #135 fix lint * #135 fix lint * Filebrowser: Refactor Filebrowser by using global state (#163) * feat: Trino functions in Code Completion (#78) * Add Trino functions to completion list * fix: Unclosed subqueries (#110) * correctly classify completion slots inside unclosed parenthesised subqueries * chore: Helm cleanup (#109) * add missing envs to helm chart & add probes with /healthz endpoint * update tech debt * change trino.completionDisabled to features.trino.completion.enabled * refactor: use global states * refactor: reorganise component structure * #159 Add VS Code Settings and Extensions (#160) * #159 add vs code settings and extensions * #159 update deprecated setting * #159 remove ruler * refactor: improve logging levels and clean up unused code * fix: fix bookmarks handling and improve loading state management * refactor: adjust z-index values for improved layering in BucketList component * refactor: enhance unpin menu functionality and add close button * refactor: enhance unpin context menu with improved positioning and close button * refactor: streamline unpin menu handling and improve focus management * refactor(frontend): clean up and restructure app (#84) * refactor(frontend): clean up and restructure app * fix: add lint:fix script to package.json and clean up trailing comma in +layout.svelte * docs: update AGENTS.md to include Material UI icon library usage * refactor(header): move components for header into own directory * chore: dedupe lockfile * fix(sidebar): add hover cursor style to toggle button * chore(types): centralise navigation and auth types; update header/sidebar imports Co-authored-by: Copilot <copilot@github.com> * #114 save sidebar collapse state in localStorage * refactor: merge if statements * refactor: increase timeout for new clusters Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: unused export Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: restore hr seperator Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: restore previous layout css Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: replace Iconify with unplugin-icons * refactor: update icon library usage to unplugin-icons with material-symbols --------- Co-authored-by: F3l1x1vo <fli@ordix.de> Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * refactor: update bucket icons to use gravity-ui and add new dependency * chore: move gravity-ui to dev-dependencies --------- Co-authored-by: Benedikt Labrenz <benedikt@labrenz.org> Co-authored-by: F3l1x1vo <fli@ordix.de> Co-authored-by: Lukas Krug <lukas.krug@stackable.tech> * Implement Garage S3 deployment and configuration for E2E testing --------- Co-authored-by: dklOrdix <dkl@ordix.de> Co-authored-by: Benedikt Labrenz <benedikt@labrenz.org> Co-authored-by: Lukas Krug <lukas.krug@stackable.tech>
Renaming a pasted folder passed a destination key without a trailing slash, so children were written to "<name><child>" and the existence check matched unrelated sibling keys.
Repetitions inside groups and large bounded repetitions were not counted, so patterns like "(.*)(.*)(.*)!" passed and blocked the event loop for seconds per key. Estimate the backtracking width per alternative instead.
pollJob threw on the 404 for a missing job, so the not_found branch was unreachable and downloads waiting on a save dialog were marked as failed. Return not_found instead and keep polling for a 60 s grace period, after which the job counts as expired.
Date filters were built in local time but evaluated on the server, usually in UTC, shifting day boundaries by the user's UTC offset. The client now sends its IANA time zone and the server computes boundaries in it.
An empty name was saved as-is, and renaming to an existing name hit the unique constraint and returned a 500. Blank names now keep the current one, and a duplicate name is shown as a form error.
Exceeding 10000 results was reported with code "unknown", so the UI told the user to retry. Send a dedicated too_many_results code with its own message, log it at info level and count it as a separate outcome.
For entries needing both ZIP64 sizes and offset, zipStreamSize counted two extra-field headers while the central directory writes one, so the advertised Content-Length was 4 bytes too large and downloads failed. Derive all record lengths from the record builders instead.
Modal handles Escape in the capture phase, so pressing Escape with the search filter date picker open closed the whole search modal and aborted running searches. Leave Escape to the browser while an auto popover inside the dialog is open.
The unique constraint over the raw search parameters exceeded the btree index row limit for long queries and exclude patterns, so such searches failed to save. Store a SHA-256 search key and enforce uniqueness on it.
Changing only the access key skipped the credential check and stored the new key with the old secret. Require the matching secret key instead, and show field errors for the connection name, which were never rendered.
A non-UUID manifest id reached the uuid column comparison, which Postgres rejects, so the download and redownload routes returned 500 instead of 404.
Browser retries of a manifest download reuse the URL and its job id, which createJob rejected with 409 until the job expired. Let the same user restart a job that is no longer running; other users and running jobs still get 409.
The history list is scoped to the active connection, but clearing it deleted the user's manifests on every connection. Scope the delete to the connection sent with the request.
A stream cut off by a proxy timeout or restart ended without its complete event, and the partial results were shown as a finished search. Require the complete event and report the bucket as failed otherwise.
"date = 2026-08-17 14:30" compared to the exact millisecond and matched practically nothing. Match the whole minute, or second when given, as a date-only value matches the whole day; "<" and ">" are unchanged.
The depth field accepted values the server rejects (outside 1–20 or not whole numbers), so every bucket request failed without a hint. Show a field error and disable searching until the value is valid.
A new S3Client with its own keep-alive agent was created per request, and its idle sockets were never closed or reused. Over time this exhausted file descriptors (the e2e Garage server crashed at ~1000 connections). Share pooled agents across clients and close idle sockets after 30 s.
The test was placed after the closing of its describe block and missed the beforeEach that skips the suite when no Garage instance is configured, so it failed instead of being skipped.
Object keys are untrusted: anyone with write access to a shared bucket can create keys such as `../../.bashrc` or `/etc/profile`, which naive extractors would write outside the target directory. Name ZIP entries relative to the download prefix and drop empty, `.` and `..` segments, leading slashes, backslashes and drive letters. Files that collide after sanitising get a numeric suffix.
* #224 add tooltips to many locations * convert toast to portal * fix merge conflict * feat: Add basic search Modal (#288) * feat: Add basic search Modal * fix: Update cleanup keys for cut and paste test to include destination file * Feat/s3 file browser v2 visual improvements (#297) * feat: Add basic search Modal * feat: overhaul Storage Search Modal * Search Regex and more backend functions (#291) * #180 move filters to backend, restrict regex, move regex button, stream search results * fix lint * #180 add date and size filter to search * #180 autofocus search bar, cancel search, dead i18n keys, session shortcut hint, userside regex validation * fix arch * #180 use daisyui dropdown, show datepicker below input * add left padding to select * fix lint * make the input field as tall as the search select * test: keep search modal open when closing date picker --------- Co-authored-by: F3l1x1vo <fli@ordix.de> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> * Feat/s3 file browser v2 recent search tabs (#292) * feat: Add basic search Modal * feat: overhaul Storage Search Modal * feat: improve recent searches tab * style: use text-white for open search button * fix keybind targeting in search modal * fix merge * fix merge * fix merge --------- Co-authored-by: F3l1x1vo <fli@ordix.de> * fix merge regressions that were lost from commit 92773d6 * Multipart Download and Archived Download (#287) * #124 add multipart download, add archived download * #124 fix progress display, add download again button * #124 download again only if cached archive not expired * fix lint * fix lint * #124 restrict downloads to zip format * #124 fix cancelled jobs running checks, show compression stage notice * #128 enable download on right clicking a folder, fix worker display of multidownload * #124 use noncompressed archive directly to browser instead of multipart download to server * feat: remove duplicate code and improve download logic * feat: remove duplicate code and improve download logic * fix: changes to download history behaviour * #124 use chevrons instead of text, hide folders, improve error messages * #124 transmit final size to browser, link browser download to job * #124 ui job improvements * remove deprecated download history unit test * pull from v2 --------- Co-authored-by: dklOrdix <dkl@ordix.de> * fix: Allow running as arbitrary UID for (OpenShift) (#296) * use tabs by default, fix tooltips, rename storage routes, operations history now overflows, fix TSV preview * fix tab and bucket switching behavior * remove close button * fix lint * fix history.pushState * add nvm version * pull from v1 * fix: hide parallel search button when not on search tab * fix: hide browser downloads from operations * fix: improve download history entries * fix: report partial storage search failures * #180 add pagination to search * #180 add search term history cycling * fix lint and unit test * #180 stream search results * #289 fix e2e test * #289 fix warnings * #289 implement code review * #289 remove typebox override, add override comments * #289 fix e2e test * fix(storage): keep trailing slash on folder destination keys Renaming a pasted folder passed a destination key without a trailing slash, so children were written to "<name><child>" and the existence check matched unrelated sibling keys. * fix(storage): close regex search guard bypasses Repetitions inside groups and large bounded repetitions were not counted, so patterns like "(.*)(.*)(.*)!" passed and blocked the event loop for seconds per key. Estimate the backtracking width per alternative instead. * fix(storage): treat not-yet-started download jobs as pending pollJob threw on the 404 for a missing job, so the not_found branch was unreachable and downloads waiting on a save dialog were marked as failed. Return not_found instead and keep polling for a 60 s grace period, after which the job counts as expired. * fix(storage): evaluate search date filters in the user's time zone Date filters were built in local time but evaluated on the server, usually in UTC, shifting day boundaries by the user's UTC offset. The client now sends its IANA time zone and the server computes boundaries in it. * fix(storage): keep connection name when blank, report duplicates An empty name was saved as-is, and renaming to an existing name hit the unique constraint and returned a 500. Blank names now keep the current one, and a duplicate name is shown as a form error. * fix(storage): report the search result limit as its own error Exceeding 10000 results was reported with code "unknown", so the UI told the user to retry. Send a dedicated too_many_results code with its own message, log it at info level and count it as a separate outcome. * fix(storage): count one ZIP64 extra block in the archive size For entries needing both ZIP64 sizes and offset, zipStreamSize counted two extra-field headers while the central directory writes one, so the advertised Content-Length was 4 bytes too large and downloads failed. Derive all record lengths from the record builders instead. * fix(ui): let Escape close an open popover inside a modal first Modal handles Escape in the capture phase, so pressing Escape with the search filter date picker open closed the whole search modal and aborted running searches. Leave Escape to the browser while an auto popover inside the dialog is open. * fix(storage): key recent searches by a hash of their parameters The unique constraint over the raw search parameters exceeded the btree index row limit for long queries and exclude patterns, so such searches failed to save. Store a SHA-256 search key and enforce uniqueness on it. * fix(storage): require the secret key when changing the access key Changing only the access key skipped the credential check and stored the new key with the old secret. Require the matching secret key instead, and show field errors for the connection name, which were never rendered. * fix(storage): return 404 for manifest ids that are not UUIDs A non-UUID manifest id reached the uuid column comparison, which Postgres rejects, so the download and redownload routes returned 500 instead of 404. * fix(storage): allow retrying a failed download with the same job id Browser retries of a manifest download reuse the URL and its job id, which createJob rejected with 409 until the job expired. Let the same user restart a job that is no longer running; other users and running jobs still get 409. * fix(storage): clear download history only for the active connection The history list is scoped to the active connection, but clearing it deleted the user's manifests on every connection. Scope the delete to the connection sent with the request. * fix(storage): treat a search stream without completion as failed A stream cut off by a proxy timeout or restart ended without its complete event, and the partial results were shown as a finished search. Require the complete event and report the bucket as failed otherwise. * fix(storage): match the whole minute for "=" date filters with a time "date = 2026-08-17 14:30" compared to the exact millisecond and matched practically nothing. Match the whole minute, or second when given, as a date-only value matches the whole day; "<" and ">" are unchanged. * fix(storage): validate max folder depth before searching The depth field accepted values the server rejects (outside 1–20 or not whole numbers), so every bucket request failed without a hint. Show a field error and disable searching until the value is valid. * fix(storage): share keep-alive agents across S3 clients A new S3Client with its own keep-alive agent was created per request, and its idle sockets were never closed or reused. Over time this exhausted file descriptors (the e2e Garage server crashed at ~1000 connections). Share pooled agents across clients and close idle sockets after 30 s. * test(e2e): skip the folder rename test without Garage The test was placed after the closing of its describe block and missed the beforeEach that skips the suite when no Garage instance is configured, so it failed instead of being skipped. * fix(storage): sanitise archive entry names Object keys are untrusted: anyone with write access to a shared bucket can create keys such as `../../.bashrc` or `/etc/profile`, which naive extractors would write outside the target directory. Name ZIP entries relative to the download prefix and drop empty, `.` and `..` segments, leading slashes, backslashes and drive letters. Files that collide after sanitising get a numeric suffix. --------- Co-authored-by: dklOrdix <dkl@ordix.de> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: Benedikt Labrenz <benedikt@labrenz.org> Co-authored-by: Johannes Fleck <johannes.fleck@stackable.tech>
Conflicts were mostly from #380, which touched storage code that v1 has rewritten. Kept v1's structure and ported the #380 fixes onto it: - Type the storage connect/edit superforms messages with StorageConnectionMessage instead of the 'ok' / bare-string sentinel. - Stop trimming S3 object keys and prefixes in the storage API and in StorageObjectNameSchema; callers trim user input. - Keep v1's requireConfig (request-context.ts) instead of requireStorageConfig, and v1's server-side check that a changed access key needs a secret key. Drop main's client-side check, its duplicate e2e test and message key. - Take main's mock server ports in playwright.config.ts. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The app webServer piped stdout, so every pino log line was interleaved with the Playwright output. Server stderr is still shown.
Archive entry names are relative to the download prefix since 7d3321a (sanitise archive entry names); the download tests still expected full object keys.
…nection The browse page load read connectionStore, which the storage layout load fills as a side effect, without awaiting parent(). After a full page load the page load could run first, see no active connection and redirect a valid bucket route to the storage overview. This made storage e2e tests flaky in Firefox.
The retry worked around the browse route redirecting to the storage overview, which is fixed now. Keeping it would hide a regression.
The dialog close event fires in a later task than dialog.close(). When a modal was closed and a new one opened through the same open binding before that event arrived, the destroyed Modal's handler set open to false and closed the new dialog. This made the drag-and-drop upload e2e tests flaky: a second drop after cancelling did not reopen the dialog.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #129