Skip to content

fix(api): improves the backup system to support per-database backup scheduling and retention - #48

Merged
Lftobs merged 2 commits into
devfrom
bugs-fix
Sep 13, 2026
Merged

Lftobs merged 2 commits into
devfrom
bugs-fix

Conversation

@Lftobs

@Lftobs Lftobs commented Sep 13, 2026

Copy link
Copy Markdown
Owner

Description

This pull request refactors and improves the backup system to support per-database backup scheduling and retention, simplifies configuration management, and adds better support for S3 storage. The core changes include removing the old BackupConfig in favor of per-target settings, updating the backup orchestrator and scheduler to use these new settings, and extending the S3 storage implementation. Additionally, new system-wide backup schedule and retention settings are introduced in the database schema.

Backup Configuration Refactor and Per-Target Scheduling:

  • Removed the BackupConfig interface and replaced it with per-target scheduling and retention logic, allowing each database (and the internal database) to have its own backup schedule and retention count. (apps/api/src/backup/types.ts [1] apps/api/src/api/backups/index.ts [2] [3] [4] [5] apps/api/src/backup/scheduler.ts [6] [7] [8]
  • Updated the backup scheduler to poll every 60 seconds and trigger backups according to each target's cron schedule and retention settings, using a new cron parser. (apps/api/src/backup/scheduler.ts [1] [2]
  • Added two new columns to the backup_storage_settings table: system_backup_schedule and system_backup_retention, for system-wide defaults. (apps/api/src/db/migrations/0033_add_system_backup_settings.sql [1] apps/api/src/db/migrations/meta/_journal.json [2]

Backup Orchestrator and Storage Enhancements:

  • Refactored the BackupOrchestrator to accept only storage configuration and to require explicit retention count on each backup, removing the dependency on a global config object. (apps/api/src/backup/orchestrator.ts [1] [2] [3] [4] [5]
  • Changed the BackupStorage interface and its implementations so that upload returns both the storage path and the size of the backup, improving metadata tracking. (apps/api/src/backup/storage.ts [1] apps/api/src/backup/storage/local.ts [2] apps/api/src/backup/storage/s3.ts [3]

S3 Storage Improvements:

  • Added a configurable S3 backup prefix, included in all S3 operations, and ensured that S3 keys are properly namespaced and that path style is always used. (apps/api/src/backup/types.ts [1] [2] apps/api/src/backup/storage/s3.ts [3] [4]

Other Minor Improvements:

  • Updated default values for internal database container, database name, and improved environment variable usage for better configurability. (apps/api/src/api/backups/index.ts [1] apps/api/src/backup/scheduler.ts [2]

These changes collectively make the backup system more flexible, maintainable, and robust, especially for environments with multiple databases and S3 storage.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation update
  • Refactor (no functional changes)
  • CI / Build / Tooling
  • Other (please describe):

How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce.

  • Existing tests pass (bun test in apps/api/)
  • New tests added (if applicable)
  • Manual testing performed (describe steps)

Checklist

  • My code follows the project's code style (no comments, named exports, functional components, etc.)
  • I have read the contributing guidelines
  • I have added tests that prove my fix is effective or that my feature works
  • I have updated the documentation (if applicable)
  • My changes generate no new warnings or lint errors
  • I have run bun test in apps/api/ and all tests pass
  • I have synced the VERSION file if needed (bun run sync-versions)

Summary by CodeRabbit

  • New Features

    • Added configurable system backup schedules and retention periods.
    • Added support for organizing S3 backups with configurable prefixes.
    • Backup records now display uploaded file sizes.
    • Added pagination to backup history lists.
  • Usability Improvements

    • Expanded SQL query editors with larger, resizable input areas.
    • Added helpful placeholder text for entering SQL queries.
  • Bug Fixes

    • Improved backup scheduling and restore behavior across local and S3 storage.

- Fix hardcoded containerName 'postgres' → POSTGRES_CONTAINER env var
  (actual container is dequel-postgres-1, causing all internal backups to fail)
- Fix storageType always showing 'local' in UI → now reads from StorageConfig
- Add dequel-db-backup/ prefix for S3 uploads via shared S3_BACKUP_PREFIX constant
- Extract S3_BACKUP_PREFIX to types.ts to prevent drift between scheduler and API
- Refactor orchestrator to take StorageConfig instead of BackupConfig
- Add system backup schedule/retention to backup_storage_settings table (migration 0033)
- Remove backup env vars from docker-compose (settings read from DB)
- Add pagination component for backup lists
- Upload returns { path, size } for accurate sizeBytes on completion
@Lftobs
Lftobs merged commit 1c001d6 into dev Sep 13, 2026
4 of 5 checks passed
@coderabbitai

coderabbitai Bot commented Sep 13, 2026

Copy link
Copy Markdown

Review Change StackReview Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: fdce89e8-e887-4244-bc97-a1e0aedb6b6f

📥 Commits

Reviewing files that changed from the base of the PR and between 650e1d8 and f38b1a6.

📒 Files selected for processing (19)
  • apps/api/src/api/backups/index.ts
  • apps/api/src/backup/orchestrator.ts
  • apps/api/src/backup/scheduler.ts
  • apps/api/src/backup/storage.ts
  • apps/api/src/backup/storage/local.ts
  • apps/api/src/backup/storage/s3.ts
  • apps/api/src/backup/types.ts
  • apps/api/src/db/migrations/0033_add_system_backup_settings.sql
  • apps/api/src/db/migrations/meta/_journal.json
  • apps/api/src/db/repo/settings.ts
  • apps/api/src/db/schema.ts
  • apps/api/src/index.ts
  • apps/api/src/types.ts
  • apps/web/src/components/databases/DatabaseBackupManager.tsx
  • apps/web/src/components/databases/QueryEditor.tsx
  • apps/web/src/components/databases/studio/SqlQueryView.tsx
  • apps/web/src/components/settings/BackupSettingsSection.tsx
  • apps/web/src/components/ui/pagination.tsx
  • apps/web/src/types/index.ts

Disabled knowledge base sources:

  • Linear integration is disabled

You can enable these sources in your CodeRabbit configuration.


📝 Walkthrough

Walkthrough

The backup system now loads storage, schedule, and retention settings from the database. Storage uploads report path and size. The scheduler evaluates per-target cron settings. The web interface adds system backup controls, paginates backup history, and enlarges SQL query editors.

Changes

Database-backed backup runtime

Layer / File(s) Summary
Persisted backup settings
apps/api/src/db/..., apps/api/src/types.ts, apps/web/src/types/index.ts
The backup settings schema and migration add system schedule and retention fields. Repository read and write paths persist these fields. Shared types expose them.
Storage and backup record contracts
apps/api/src/backup/types.ts, apps/api/src/backup/storage*, apps/api/src/backup/orchestrator.ts
The removed BackupConfig contract is replaced by storage configuration and per-call retention. Local and S3 uploads return paths and byte sizes. S3 uploads use a normalized prefix and path-style requests.
Scheduler and API execution
apps/api/src/backup/scheduler.ts, apps/api/src/api/backups/index.ts, apps/api/src/index.ts
The scheduler loads database settings, evaluates five-field cron expressions every 60 seconds, and applies target-specific retention. Backup API routes resolve storage separately from retention, and restore no longer loads a backup record for configuration.

Backup and query interfaces

Layer / File(s) Summary
Backup settings and history pagination
apps/web/src/components/settings/BackupSettingsSection.tsx, apps/web/src/components/databases/DatabaseBackupManager.tsx, apps/web/src/components/ui/pagination.tsx
The settings screen saves system schedule and retention values. Backup history lists display five items per page through the new Pagination component.
SQL query editor layout
apps/web/src/components/databases/QueryEditor.tsx, apps/web/src/components/databases/studio/SqlQueryView.tsx
Both SQL text areas use 12 rows and a 260px minimum height. SqlQueryView also supports vertical resizing and shows a placeholder.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant SettingsUI
  participant BackupAPI
  participant backup_storage_settings
  participant BackupOrchestrator
  participant BackupStorage
  SettingsUI->>BackupAPI: save system schedule and retention
  BackupAPI->>backup_storage_settings: persist backup settings
  BackupAPI->>BackupOrchestrator: start backup with storage and retention
  BackupOrchestrator->>BackupStorage: upload backup data
  BackupStorage-->>BackupOrchestrator: return path and size
Loading
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch bugs-fix

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant