Implement full data modification history and storage backup#1870
Draft
paustint wants to merge 7 commits into
Draft
Implement full data modification history and storage backup#1870paustint wants to merge 7 commits into
paustint wants to merge 7 commits into
Conversation
|
Review the following changes in direct dependencies. Learn more about Socket for GitHub.
|
- New Data History page listing all captured data modifications across orgs and features, with filters, pin/delete, payload preview/download - Data History settings section (enable/disable, retention, storage usage, clear-all) shared across web, desktop, and browser extension - Access points: header clock icon, Load Records menu, home page card, contextual links on capture surfaces, settings cross-links - Documentation page with links from the app feat: add user-selected storage locations for data history Data history can now be stored outside default browser storage: - Chrome/Edge: pick a real folder (File System Access API) — files are user-visible, backed up, survive clearing site data, and the history list can be rebuilt from the folder's manifest files - Desktop app: store history on the file system with a configurable, relocatable folder (default userData/data-history) - Existing history migrates automatically when switching locations, and entries remain readable mid-migration via per-entry backend routing - Storage location controls added to the Data History settings section feat: simplify data history limits and storage controls - Persistent storage is requested automatically; removed the manual settings control - Free plans now keep the 15 most recent entries (count-based, enforced as history is captured) instead of a storage-size cap; upgrade prompts shown on the history page and in settings - Removed the max-storage size setting; storage size remains only as an internal per-plan backstop - History can be moved to a folder directly from the Data History page feat: improve data history table usability - Org column shows the Salesforce org id beneath the org label - Download request/input/results directly from each table row - Sortable column headers (date, org, feature, objects, status, records, size) over the visible entries fix: align data history table row actions and remove size column feat: show the connected data history folder on the history page fix: skip empty data sync push requests for local-only table changes dexie-observable reports changes for every table, including local-only ones like data history — batches with no syncable-table changes are now acknowledged locally instead of POSTing an empty data array to /api/data-sync/push
- Files in a user-chosen folder (Chrome/Edge) or on disk (desktop) are now plain .csv/.json instead of gzip so they open directly in Excel and other tools on any platform; browser-managed storage stays compressed. Existing compressed entries remain readable and are re-encoded when moved between storage locations - Show 'Files are saved to' with the folder name on the history page and in settings; on desktop the full path is shown and opens in Explorer/Finder on click - Change the connected history folder from settings - Use the correct paid-plan signal for history limits (some paid plans were incorrectly limited to 15 entries)
paustint
force-pushed
the
feat/full-history-storage
branch
from
July 20, 2026 14:42
2581546 to
7150630
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
This PR introduces a device-local “Data History” system that captures inputs/requests/results for data-modifying workflows (loads, mass updates, inline edits, record modal actions), stores searchable metadata in Dexie, and persists large payloads via pluggable storage backends (OPFS / File System Access directory / Electron-native filesystem). It also adds a dedicated Data History page + navigation entry points, plus settings for retention/storage location.
Changes:
- Add core Data History types, Dexie tables, retention sweep logic, and a pluggable file-store layer (OPFS worker, directory-handle backend, native desktop backend, plus test fakes).
- Wire capture into existing flows (query inline edits, load records, multi-object loads, mass updates, record modal/create record) and add opt-out toggles per operation.
- Add UI entry points (route/nav/home/header link), docs page, and supporting shared export/builders to keep downloaded/exported shapes consistent with captured history.
Reviewed changes
Copilot reviewed 118 out of 119 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| tsconfig.base.json | Adds TS path aliases for the new feature and shared UI data-history libs. |
| pnpm-lock.yaml | Locks dependency updates including fake-indexeddb. |
| package.json | Adds fake-indexeddb dev dependency for IndexedDB-backed unit tests. |
| libs/ui/src/lib/data-table/SalesforceRecordDataTable.tsx | Adds save-capture callback to snapshot edited/prior values + results for history before in-place mutation. |
| libs/ui/src/lib/data-table/PreviewChangesModal.tsx | Refactors export builder logic into a shared module and reuses it. |
| libs/ui/src/lib/data-table/data-table-history-export.ts | New shared export/result builders for query-grid edits (edited/prior/results). |
| libs/ui/src/lib/data-table/tests/PreviewChangesModal.spec.ts | Updates tests to import shared export builders from the new module. |
| libs/ui/src/lib/data-table/tests/data-table-history-export.spec.ts | Adds unit tests for the new query-grid history export builders. |
| libs/ui/src/index.ts | Exposes the new data-table export builder utilities. |
| libs/types/src/lib/data-history-types.ts | Adds zod schemas/types for Data History items, settings, backends, and file refs. |
| libs/types/src/index.ts | Exports Data History types from the shared types package. |
| libs/shared/ui-router/src/lib/ui-router.ts | Adds the DATA_HISTORY route entry (path/docs/title/description/new-until). |
| libs/shared/ui-db/src/lib/ui-db.ts | Adds Dexie tables + schema version bump for data_history and data_history_config. |
| libs/shared/ui-db/src/lib/data-history.db.ts | New DB module for Data History catalog rows + settings/backend config persistence. |
| libs/shared/ui-db/src/lib/client-data-sync.db.ts | Filters out local-only table changes so they don’t trigger server sync POSTs. |
| libs/shared/ui-db/src/index.ts | Re-exports the new Data History DB module. |
| libs/shared/ui-data-history/vite.config.ts | Adds Vitest config for the new shared ui-data-history library. |
| libs/shared/ui-data-history/tsconfig.spec.json | Adds TS config for ui-data-history tests. |
| libs/shared/ui-data-history/tsconfig.lib.json | Adds TS config for ui-data-history library build settings. |
| libs/shared/ui-data-history/tsconfig.json | Adds TS project references for ui-data-history. |
| libs/shared/ui-data-history/src/test-setup.ts | Sets up fake-indexeddb + Blob realm compatibility for compression streams in tests. |
| libs/shared/ui-data-history/src/lib/platform.ts | Adds platform detection helpers used to select tier/backends. |
| libs/shared/ui-data-history/src/lib/file-store/worker-messages.ts | Defines typed RPC protocol for OPFS storage worker requests/responses. |
| libs/shared/ui-data-history/src/lib/file-store/path-utils.ts | Adds validated relative-path helpers and stable org folder naming. |
| libs/shared/ui-data-history/src/lib/file-store/opfs-file-store.ts | Implements OPFS backend with worker-based I/O and RPC request handling. |
| libs/shared/ui-data-history/src/lib/file-store/native-fs-file-store.ts | Implements desktop-native filesystem backend via Electron IPC. |
| libs/shared/ui-data-history/src/lib/file-store/fsa-types.ts | Adds minimal File System Access API typings + permission error type. |
| libs/shared/ui-data-history/src/lib/file-store/file-store.types.ts | Defines the pluggable file store interface + capability flags. |
| libs/shared/ui-data-history/src/lib/file-store/file-store-factory.ts | Resolves active backend from config with fallback behavior and test seams. |
| libs/shared/ui-data-history/src/lib/file-store/fake-file-store.ts | In-memory backend used for unit/contract tests (with real gzip). |
| libs/shared/ui-data-history/src/lib/data-history-state.ts | Central module state for tier limits/settings and deletion helper. |
| libs/shared/ui-data-history/src/lib/data-history-retention.ts | Adds retention + orphan reconciliation sweep with tier enforcement + locks. |
| libs/shared/ui-data-history/src/lib/data-history-manifest.ts | Adds manifest writer so on-disk folders can be re-indexed. |
| libs/shared/ui-data-history/src/lib/data-history-limits.ts | Adds tier-based limits and settings clamping defaults. |
| libs/shared/ui-data-history/src/lib/csv-utils.ts | Adds fixed-format CSV chunk serializer for streaming history writes. |
| libs/shared/ui-data-history/src/lib/tests/path-utils.spec.ts | Tests path validation/builders and org folder hashing behavior. |
| libs/shared/ui-data-history/src/lib/tests/fake-fsa-handles.ts | Adds in-memory FSA handles for DirectoryHandle backend tests. |
| libs/shared/ui-data-history/src/lib/tests/data-history-retention.spec.ts | Adds end-to-end tests for retention sweep behavior and orphan cleanup. |
| libs/shared/ui-data-history/src/lib/tests/csv-utils.spec.ts | Tests streaming CSV chunk output behavior and invariants. |
| libs/shared/ui-data-history/src/index.ts | Exports public API surface of the new shared data history library. |
| libs/shared/ui-data-history/project.json | Adds Nx project config and Vitest target for ui-data-history. |
| libs/shared/ui-data-history/eslint.config.cjs | Adds eslint config passthrough for the new library. |
| libs/shared/ui-core/src/record/ViewEditCloneRecord.tsx | Records record modal create/edit/clone actions to Data History (inline payload). |
| libs/shared/ui-core/src/mass-update-records/useDeployRecords.ts | Wires mass-update flow to create/finalize Data History entries and stream results. |
| libs/shared/ui-core/src/mass-update-records/MassUpdateRecordsDeploymentRow.tsx | Refactors result-building to shared utils to match captured history/export shape. |
| libs/shared/ui-core/src/mass-update-records/mass-update-records.utils.tsx | Adds shared builders for mass-update combined result rows + headers. |
| libs/shared/ui-core/src/mass-update-records/data-history-capture.ts | Adds thin adapters from mass-update deploy flow to history capture APIs. |
| libs/shared/ui-core/src/mass-update-records/tests/mass-update-history-builders.spec.ts | Adds unit tests for the new mass-update result/header builders. |
| libs/shared/ui-core/src/mass-update-records/tests/data-history-capture.spec.ts | Adds integration tests for mass-update → Data History wiring using fake backends. |
| libs/shared/ui-core/src/index.ts | Exports new Data History related UI components/settings from ui-core. |
| libs/shared/ui-core/src/app/HeaderNavbarItems.tsx | Adds Data History entry to header navigation menu config. |
| libs/shared/ui-core/src/app/HeaderNavbar.tsx | Adds a header icon button linking to Data History. |
| libs/shared/ui-core/src/app/DataHistoryLinks.tsx | Adds reusable header button + inline “View Data History” link components. |
| libs/shared/ui-core/src/app/AppHome/AppHome.tsx | Adds Data History card to the “Load” section on the home page. |
| libs/shared/ui-app-state/src/lib/ui-app-state.ts | Adds atom to synchronously track whether capture is enabled (seeded on init). |
| libs/shared/constants/src/lib/shared-constants.ts | Adds analytics keys and page title constant for Data History. |
| libs/icon-factory/src/lib/icon-factory.tsx | Adds new icons (standard/utility pin/pinned/asset_audit) for UI usage. |
| libs/features/update-records/src/deployment/MassUpdateRecordsDeployment.tsx | Adds per-run “skip history” option and links to Data History when enabled. |
| libs/features/query/src/QueryResults/QueryResults.tsx | Wires query inline edit save capture into recordDataHistoryAction. |
| libs/features/query/src/QueryResults/BulkUpdateFromQuery/BulkUpdateFromQueryModal.tsx | Adds “skip history” toggle in bulk-update-from-query modal when enabled. |
| libs/features/load-records/src/utils/data-history-capture.ts | Adds helper adapters for Load Records to write/append/finish/fail history entries. |
| libs/features/load-records/src/utils/tests/data-history-capture.spec.ts | Adds unit tests for load-records history capture wrappers/mappers. |
| libs/features/load-records/src/components/load-results/LoadRecordsResults.tsx | Threads an optional Data History handle through results components. |
| libs/features/load-records/src/components/load-results/LoadRecordsBatchApiResults.tsx | Streams batch results to Data History and finalizes entries with counts. |
| libs/features/load-records/src/components/load-results/load-results-utils.ts | Adds shared result-row/header builders for Load Records exports + history capture. |
| libs/features/load-records/src/components/load-results/tests/load-results-utils.spec.ts | Adds unit tests for Load Records shared result-row/header builders. |
| libs/features/load-records/src/components/load-results/tests/data-history-results-capture.spec.ts | Adds integration tests for Load Records → Data History end-to-end capture. |
| libs/features/load-records-multi-object/src/useDownloadResults.ts | Refactors exports to shared builders used by both downloads and history capture. |
| libs/features/load-records-multi-object/src/LoadRecordsMultiObject.tsx | Adds multi-object load Data History capture, per-run opt-out, and finalize step. |
| libs/features/load-records-multi-object/src/load-records-multi-object-results.ts | Adds shared request/result export builders + derived counts helper. |
| libs/features/load-records-multi-object/src/data-history-capture.ts | Adds adapters for multi-object load to write request/finalize/derive metadata. |
| libs/features/load-records-multi-object/src/tests/data-history-capture.spec.ts | Adds tests for multi-object capture helpers and history wiring. |
| libs/features/data-history/vite.config.ts | Adds Vitest config for the new feature/data-history UI library. |
| libs/features/data-history/tsconfig.spec.json | Adds test TS config for the data-history feature library. |
| libs/features/data-history/tsconfig.lib.json | Adds library TS config for the data-history feature library. |
| libs/features/data-history/tsconfig.json | Adds TS project references for the data-history feature library. |
| libs/features/data-history/src/lib/DataHistoryDetailModal.tsx | Adds detail modal with preview/download of saved payloads for an entry. |
| libs/features/data-history/src/lib/data-history-page.utils.ts | Adds labels, formatting, sorting, preview truncation, and download naming helpers. |
| libs/features/data-history/src/lib/data-history-download.ts | Adds shared download helper that reads/decompresses and triggers file save. |
| libs/features/data-history/src/lib/tests/data-history-page.utils.spec.ts | Adds unit tests for Data History page utility behaviors. |
| libs/features/data-history/src/index.ts | Exports Data History feature components/utils. |
| libs/features/data-history/project.json | Adds Nx project config and Vitest target for the new feature library. |
| libs/features/data-history/eslint.config.cjs | Adds eslint config passthrough for the new feature library. |
| libs/features/create-records/src/lib/CreateRecords.tsx | Records “create record without file” actions to Data History (inline payload). |
| libs/desktop-types/src/lib/desktop-app.types.ts | Adds Electron IPC request types + preference for native data history folder. |
| apps/jetstream/vite.config.ts | Adjusts Vite fs allow list for dev server. |
| apps/jetstream/src/app/components/settings/Settings.tsx | Adds Data History settings section to app settings page. |
| apps/jetstream/src/app/components/core/AppInitializer.tsx | Initializes Data History and seeds capture-enabled state atom at startup. |
| apps/jetstream/src/app/AppRoutes.tsx | Adds route for the new Data History page. |
| apps/jetstream-web-extension/vite.plugins.ts | Adds placeholder page entry for data-history in extension build. |
| apps/jetstream-web-extension/vite.config.ts | Adds dev server fs allow list and worker ts-paths plugin for extension. |
| apps/jetstream-web-extension/src/pages/app/App.tsx | Adds Data History route for the extension app. |
| apps/jetstream-web-extension/src/pages/additional-settings/AdditionalSettings.tsx | Adds Data History settings section to extension additional settings. |
| apps/jetstream-web-extension/src/core/AppInitializer.tsx | Initializes Data History and seeds capture-enabled state for extension. |
| apps/jetstream-desktop/src/services/ipc.service.ts | Adds IPC handlers for native data history file ops and folder configuration. |
| apps/jetstream-desktop/src/preload.ts | Exposes Data History IPC APIs to the renderer via contextBridge. |
| apps/jetstream-desktop-client/vite.config.ts | Enables worker ts-paths plugin and adjusts dev server fs allow list. |
| apps/jetstream-desktop-client/src/app/components/settings/Settings.tsx | Adds Data History settings section to desktop client settings. |
| apps/jetstream-desktop-client/src/app/components/core/AppInitializer.tsx | Initializes Data History and seeds capture-enabled state for desktop client. |
| apps/jetstream-desktop-client/src/app/AppRoutes.tsx | Adds Data History route for the desktop client. |
| apps/jetstream-canvas/vite.config.mts | Enables worker ts-paths plugin and adjusts dev server fs allow list. |
| apps/jetstream-canvas/src/app/core/AppInitializer.tsx | Initializes Data History and seeds capture-enabled state for canvas app. |
| apps/jetstream-canvas/src/app/AppRoutes.tsx | Adds Data History route for the canvas app. |
| apps/docs/sidebars.ts | Adds Data History doc page to docs sidebar. |
| apps/docs/docs/load/data-history.mdx | Adds end-user documentation for the Data History feature. |
Files not reviewed (1)
- pnpm-lock.yaml: Generated file
Comment on lines
+43
to
+48
| setLoadingKind(kind); | ||
| const file = await readDataHistoryFile(item, kind); | ||
| if (!file) { | ||
| fireToast({ type: 'warning', message: 'This data is no longer available on this device.' }); | ||
| return; | ||
| } |
Comment on lines
+186
to
+189
| failHistoryEntry( | ||
| historyHandle, | ||
| preparedDataResponse?.queryErrors?.length ? preparedDataResponse.queryErrors.join('\n') : 'Pre-processing records failed', | ||
| ); |
| setStatus(STATUSES.ERROR); | ||
| setFatalError(getErrorMessage(ex)); | ||
| onFinishRef.current({ success: 0, failure: inputFileData.length, failedRecords: [] }); | ||
| failHistoryEntry(historyHandle, getErrorMessage(ex)); |
- Detail modal now has a Summary tab plus a tab per saved payload - Input and output data render in the sortable/filterable data table (CSV parsed into columns/rows); JSON payloads show a formatted preview - Download available on the summary tab and each payload tab; the Preview button switches to that payload's tab instead of an inline dump Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| import { readDataHistoryFile } from '@jetstream/ui/data-history'; | ||
| import { FunctionComponent, useEffect, useState } from 'react'; | ||
| import { downloadDataHistoryFile, getDataHistoryReadErrorMessage } from './data-history-download'; | ||
| import { buildDataHistoryPreviewText, DataHistoryTableData, DataHistoryTableRow, parseCsvToTable } from './data-history-page.utils'; |
| import { css } from '@emotion/react'; | ||
| import { DataHistoryFileKind, DataHistoryItem, UiTabSection } from '@jetstream/types'; | ||
| import { Badge, Grid, Modal, ScopedNotification, Spinner, Tabs, TabsRef } from '@jetstream/ui'; | ||
| import { FunctionComponent, useMemo, useRef, useState } from 'react'; |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Introduce a comprehensive data history tracking system, allowing users to view and manage all modifications across various records. Enhance storage flexibility by enabling user-selected locations for data history, simplifying retention controls, and improving usability of the data history table. Add a dedicated Data History page with filtering options and export capabilities.