Skip to content
dillardblomPublic

About

Encrypted, incremental backups of your whole Nextcloud (files, database, config) to S3, SFTP, WebDAV, SMB and more, with restore from the web interface.

Topics

Resources

Stars

9 stars

Watchers

1 watching

Forks

Repository files navigation

NG Backup

Encrypted, incremental backups of Nextcloud (database, configuration and data) with restore from the web interface. Works without shell access: no external programs, no extra PHP extensions beyond sodium for ng_backup itself. Runs on Nextcloud All-in-One, regular installations and hosted Nextcloud. (Choosing SMB as a storage backend is the one exception: see "Storage backend experiences" below — that requirement comes from files_external, not from ng_backup, and applies to any Nextcloud app that writes to an SMB share.)

Status: 1.0.0, stable. Disaster recovery has been tested on real installations (see "Tested disaster recoveries" below). See "Known limitations" below, PLAN.md for the design and docs/HOWTO.md for a step-by-step guide from setup to a tested restore. Server lost? See docs/DISASTER-RECOVERY.md.

License: AGPL-3.0-or-later. Developed and sponsored by Ensembia. Need help restoring a backup after a crash? Ensembia offers paid restore assistance: www.ensembia.com.

How it works

  • Backups go to a location: S3 (and S3-compatible storage), SFTP, WebDAV, FTP, Swift or a local path. NG Backup uses the connection code of Nextcloud's External storage app, but never mounts the location: it is only connected while a backup or restore runs, and it never appears in Files, for anyone.
  • Data is cut into blocks, deduplicated, compressed and encrypted (XChaCha20-Poly1305) before it leaves the server, and streamed straight to the location: no local copy, no double disk space. Unchanged files are not read again; only new data is uploaded.
  • The database is dumped through Nextcloud's own connection, inside one consistent transaction.
  • A backup runs in short steps from Nextcloud's background jobs, and every step saves where it got to. A restart of the server or the containers does not lose the run: it continues at the next background job run. Tested on a production Nextcloud AIO instance: a 1 TB full backup over SMB carried on through the nightly AIO update, which restarts every container.
  • Restores go through Nextcloud, so overwritten files keep their old content as a version and removed files go to the trash.

Requirements

  • Background jobs on system cron. NG Backup runs backups in steps from Nextcloud's background job, so a backup only moves forward when that job runs. Choosing "Cron" under Administration settings → Basic settings → Background jobs is not enough on its own: it tells Nextcloud that the system will call it, so you also need a cron line that does. Without it a backup stays at "running (db)", and so does one started with "Back up now". occ backup:run in a terminal works without cron (it runs in the foreground).

    • Regular server: crontab -u www-data -e and add */5 * * * * php -f /var/www/nextcloud/cron.php (your Nextcloud path).
    • Docker (the official nextcloud image): on the host, crontab -e as root and add */5 * * * * docker exec -u www-data <container> php -f /var/www/html/cron.php, or add a service with the same image and entrypoint: /cron.sh to your docker-compose.yml.
    • Nextcloud All-in-One already runs cron.

    After a few minutes, the Background jobs section shows that the last job ran just now. See the Nextcloud documentation on background jobs.

  • A storage location: S3 or compatible, SFTP, WebDAV, SMB (needs the PHP smbclient extension, see below), FTP or a local path.

Quick start

occ backup:key:init --label="Safe"                 # first passphrase, deletion delay (default 7 days)
occ backup:key:slot:add --label="CTO"              # optional second and third passphrase
occ backup:key:slot:add --label="Head of IT"
occ backup:key:kit --output=/var/tmp/ng_backup-kit.json
occ backup:key:confirm
occ backup:target:add offsite amazons3 -a amazons3::accesskey -o bucket=... -o key=... -o secret=...
occ backup:run

Or use Administration settings → NG Backup.

The backup key, passphrases and the recovery kit

All backups of an installation are encrypted with one master key.

  • The server keeps the master key, encrypted with the instance secret, so scheduled backups and restores work without anyone typing a passphrase. Any administrator can restore from the web interface. Consequence: whoever fully controls the server can read the backups; whoever only gets hold of the backup location cannot, as long as the passphrases are strong (see below).

  • The location holds the key too, by design (like restic and Borg in repokey mode): its config file contains the master key, encrypted separately with each passphrase (Argon2id), the same as in the recovery kit. Without a passphrase it is of no use, but whoever can read the location can try passphrases offline. Your passphrase is therefore the actual key: it must be at least 16 characters; five or more random words are a good choice.

  • The recovery kit is your own copy of that encrypted key. Still keep it off the server and off the location: an attacker who gets write access to the location can damage or delete its config file on purpose, and without the kit that makes every backup there unreadable.

  • For a restore on a new server (disaster recovery) you currently need the recovery kit and one passphrase.

  • Planned: an option to not store the key on the location at all. Then the kit is the only copy, and losing it means losing every backup; that option will come with an explicit warning.

  • Up to three passphrases ("slots") can open the key, so one person leaving or dying does not lock the company out. A suggested setup:

    Slot Kept by
    1 in the safe (sealed envelope)
    2 the CTO
    3 the head of IT

    Every holder keeps a copy of the same recovery kit and only knows their own passphrase. Using a single slot is possible, but then that one passphrase is a single point of failure.

  • When a holder leaves, give their slot a new passphrase (occ backup:key:slot:replace), download the new kit and confirm it. Note: an old kit together with the old passphrase still opens the same master key; to revoke it completely, start new locations with a new key. This is noted on the backlog and will be addressed in a future version (key rotation).

  • Downloading the kit, and every change to the slots, requires the administrator's password, is written to the audit log and is announced to all administrators.

Protection against ransomware and a compromised server

A backup is only useful if an attacker cannot destroy it. An attacker with administrator access to Nextcloud could try to remove old restore points so that only infected data remains. NG Backup makes this hard and visible, but software alone can never guarantee it: the location must refuse deletions itself.

What NG Backup does:

  • Deletion delay. Removed restore points first go to a trash on the location and are only deleted after the delay chosen at installation (default 7 days, cannot be changed afterwards). Within that time any administrator can bring them back (occ backup:trash:restore, or in the web interface).
  • Notifications to all administrators when restore points are removed, the retention is changed, append-only is turned off, a location is removed or the recovery kit is downloaded, and an entry in the audit log (admin_audit).
  • Append-only locations (occ backup:target:append-only NAME on): NG Backup never deletes anything there; old data is cleaned up by the location's own lifecycle rules.
  • Removing a location in NG Backup only removes the connection. The backups on the location are not touched and nothing keeps cleaning up in the background.

What you should do on the location (strongly recommended):

  • S3: enable Object Lock on the bucket, in compliance mode, with a default retention of for example 90 days, plus versioning. Then nobody, not even with the server's credentials, can delete or overwrite backups younger than 90 days. Combine with an append-only location in NG Backup and a lifecycle rule that removes old versions after the lock expires. With Object Lock, a deletion by NG Backup (or by an attacker through it) only adds a delete marker: the locked data stays and can be brought back from the bucket's object versions until the lock expires. Tested with an S3-compatible server; still to be verified on AWS itself, which requires a checksum on uploads to locked buckets.
  • Use credentials without delete rights for the backup location where the storage supports it.
  • SFTP / storage boxes: enable the provider's own snapshots (e.g. daily, kept 30-90 days) and give the backup user access to its own folder only, without a shell.
  • Keep at least one location off-site, on a different network than the Nextcloud server.

Restore

  • Files and folders of a user: in the web interface or occ backup:restore, into a new folder ("Restored "), merged into the original place, or replacing it.
  • Raw restore of any path (for example config/) to a local directory: occ backup:restore ... --to-directory=DIR.
  • Whole server (disaster recovery onto a fresh installation): occ backup:restore:full, see docs/DISASTER-RECOVERY.md. It runs in maintenance mode, so that nobody and no background job writes into the instance while it is being replaced.
  • One user (account, settings, files, calendars, contacts via user_migration): occ backup:user:backup and occ backup:user:restore, replacing the account or restoring it as <uid>-bak. See the how-to.

Without shell access (a hosted Nextcloud where you cannot run occ): files and folders can be restored in the web interface. Restoring a whole user or the whole server needs occ, so on such hosting it is something your provider (or whoever has shell access) runs. A restore mode that works without maintenance mode, and so from the web interface, is on the roadmap.

Tested disaster recoveries

Full restores onto a fresh server, following only the guide:

Backup Location Database Result
Production, about 1 TB, 150,000 files (Nextcloud All-in-One) Hetzner Storage Box (SFTP), Germany → Canada PostgreSQL → PostgreSQL 150,622 files restored in about 25 h (13-14 MiB/s), sample identical
138 GB simulation: 20 users, calendars, contacts, mail, shares, team folders with an ACL, versions, trash OVH S3, Warsaw → Canada MariaDB → PostgreSQL all 15,772 files and every database count identical
The same OVH S3 MariaDB → MariaDB all 15,772 files and every database count identical
The same Hetzner Storage Box (SMB), Germany → Canada MariaDB → SQLite all 15,772 files and every database count identical (27.6 h, about 1.5 MB/s)
51 GB test instance OVH S3 PostgreSQL → PostgreSQL 23,519/23,519 checksums identical

The integration tests restore across every pair of PostgreSQL, MariaDB and SQLite.

Storage backend experiences

Notes from actually running a backend, beyond what's in files_external's own documentation. Contributions welcome via PR — add what you ran into with your own provider.

  • Hetzner Storage Box, sub-account: the SFTP backend's root option must be home, not /. A sub-account's SFTP login lands in /home (confirmed via pwd() after login); writing to the literal chroot root (/) fails with a generic NET_SFTP_STATUS_FAILURE from the SFTP server, while the same write under home (or a path below it) succeeds. This affects any backend option form that lets you set a path relative to the account's root, not just ng_backup. Not yet verified whether this also applies to a Hetzner Storage Box main account (only tested with a sub-account so far) — if you've tried a main account, please say so in a PR.

  • Hetzner Storage Box over WebDAV and SMB (sub-accounts, 2026-10-07): both work end to end (backup, incremental run, restore of a folder, verify), tested from outside the Hetzner network with about 66 MiB of data. In the Hetzner Console, enable WebDAV or SMB support for the sub-account, and "External reachability" when your server is not in the Hetzner network. WebDAV: host https://uXXXXX-subN.your-storagebox.de, and the root must be a folder that already exists (or empty): WebDAV does not create missing parent folders, so a root that doesn't exist yet fails backup:target:add with Sabre\HTTP\ClientHttpException: Conflict. SMB: share = the sub-account name, port 445 must be reachable from your server (it was from a home connection here, but some providers block it), and the smbclient PECL extension is required (see the SMB entry below). Both have a noticeable fixed cost per run: an incremental run that uploads nothing took about 20 s over WebDAV and 50 s over SMB. Over WebDAV, Nextcloud gives every request at most 30 s by default, also an upload of a whole 32 MiB pack: on an upload slower than about 9 Mbit/s a backup fails with cURL error 28: Operation timed out. Raise the limit with occ config:system:set davstorage.request_timeout --value=600 --type=integer.

  • FTP, server-dependent absolute-path handling: files_external's FTP backend always builds absolute paths (e.g. MKD /repo) for the configured root, never relative ones. atmoz/sftp-style vsftpd test servers (chrooted) can reject that with a generic "create directory operation failed", even though a plain ftp_mkdir() with a relative path on the same server succeeds; pure-ftpd accepts the absolute form fine. If a working FTP server rejects ng_backup's writes, try a different FTP server implementation before assuming the backend itself is broken.

  • SMB/CIFS lists as available with just the smbclient CLI binary, but does not actually work for ng_backup without the smbclient PECL extension too. files_external falls back to a shell-wrapped smbclient process when the native extension is missing, which registers the backend in occ backup:target:backends fine, but that fallback's stream cannot seek. ng_backup reads blobs back by byte range while backing up, so even the very first (small) backup fails with Seek failed in packs/.... Install the extension (libsmbclient-dev + pecl install smbclient + docker-php-ext-enable smbclient on Debian-based images), not just the CLI tool — the backend being listed is not proof it will work here. Once the extension is present it works the same as any other backend; no ng_backup-specific config beyond host/share/user/password.

  • Backups over a long distance (high latency): on Linux, the default TCP congestion control (cubic) slows a single connection down a lot on a long line with some packet loss. From Canada to an S3 bucket in Warsaw (114 ms), one upload went from 2.4 to 7.5 MiB/s after switching the server (or the VM/container host where TCP ends) to BBR: sysctl -w net.core.default_qdisc=fq net.ipv4.tcp_congestion_control=bbr (modprobe tcp_bbr first; make it permanent in /etc/sysctl.d/). A full restore over SFTP reads the packs ahead on a second connection, with many requests in flight. A server that handles parallel reads well (not a Hetzner Storage Box, which gets slower) can be read on more connections: occ config:app:set ng_backup restore_sftp_connections --value=4 (at most 8; the server must allow that many plus one at once for the user).

  • OVH Object Storage (S3, Warsaw, 2026-10): works with the amazons3 backend (path style, region waw). A full backup of 1 TB from Germany ran at about 7 MiB/s, from Canada about 6 MiB/s with BBR; a check of a 1 TB backup took 3.5 minutes (index files are fetched 16 at a time); a restore of 138 GB from Warsaw to Canada took about 6 hours. Turn on versioning and Object Lock on the bucket (see above); OVH charges no egress.

  • Hetzner Storage Box over SFTP, full restore: about 13-14 MiB/s from Germany to Canada (122 ms) with the read-ahead of 1.0.0-rc.4; without it about 2.8 MiB/s. A Storage Box gets slower with several connections reading at once (4-6 MiB/s for 2-4 connections, it reads from disks), so one extra connection is the default. A Storage Box drops an SFTP connection that stayed quiet for a while without telling the client; NG Backup checks and replaces it.

  • SMB speed: on a local network (Samba, no delay) a backup ran at about 25 MB/s and a restore at about 28 MiB/s. Over a long line SMB waits for an answer to each request: from Canada to a Storage Box in Germany (130 ms) about 2.6 MiB/s for a backup and 1.6 MiB/s for a restore, and connecting a location with an existing backup reads its index at about one file a second. SMB is meant for a local network; use SFTP or S3 for a location far away. When pecl install smbclient fails (pecl.php.net was unreachable for hours on 2026-10-21), the extension builds from its source: libsmbclient-php on GitHub, phpize && ./configure && make && make install, then docker-php-ext-enable smbclient.

  • NFS is not a files_external backend at all. Mount the NFS export on the server yourself (e.g. via /etc/fstab or a systemd .mount unit) and point ng_backup's local backend at the mount point; ng_backup never mounts or unmounts anything on its own, so an NFS mount dropping mid-run behaves like any other local-path failure (the run resumes). See PLAN.md 3.5 for why NFS can't be driven from PHP directly.

Roadmap

What is planned, without dates. Ideas and votes are welcome in the issues; if something you need is listed here, a 👍 on its issue helps decide what comes first.

1.0.1

  • A warning on the admin page and in occ backup:status when background jobs do not run on a system cron, with how to set one up. Without cron, a backup started with "Back up now" moves on while the NG Backup page is open (#13).
  • An optional label for a backup started with "Back up now".
  • Faster full restores from S3 and SMB, reading ahead as restores over SFTP do since 1.0.0-rc.4.

1.1.0

  • Restore without the recovery kit: the backup key from the location itself, opened with one of your passphrases.
  • A schedule per location, also "every N hours".
  • Locations backed up one after the other instead of at the same time.
  • A note on a restore point that you can change afterwards.
  • An option to pause heavy background jobs (such as Memories, Recognize or the Preview Generator) while a backup runs.
  • Faster backups: reading and uploading at the same time.

Later

  • A full restore onto the existing installation, without first installing a fresh Nextcloud, in a restore mode of its own instead of maintenance mode, so that it can also run from the web interface (for hosted Nextcloud without shell access).
  • Users restoring their own files from the Files app.
  • Primary object storage and External Storage mounts (see Known limitations).
  • An option not to keep the backup key on the location (the recovery kit is then the only copy).

Known limitations

On the backlog; know them before you rely on NG Backup:

  • Only the data directory is backed up. Object storage as primary storage (objectstore in config.php) is not supported: a backup refuses to start there. Folders mounted through the External storage app are not in the data directory and are not included, without a warning. Back them up separately. On the roadmap: reading files through Nextcloud's storage layer, which covers both, with external mounts optional per mount (PLAN.md section 3.4).
  • Very large instances (well over 1 TB, millions of files): the blob index, cleanup, browsing a snapshot in the web interface and per-user export manifests are held in memory. Expect high memory use there; reports from such instances are very welcome.
  • One location, one server. Never let two Nextcloud servers write to the same location (for example a disaster recovery test server pointed at the production location): they can break each other's catalog, and the location is then refused as possibly tampered with. Test a disaster recovery against a copy of the location, or only read from it.
  • Removing a passphrase slot does not revoke access. The master key stays the same, so someone with an old recovery kit and that slot's passphrase can still decrypt the location if they can read it. Key rotation is not available yet.
  • Rollback before a disaster recovery is not detected. A fresh server has no record of the newest catalog generation, so a location rolled back to an older state looks valid there.
  • Location passwords passed with -o password=... are visible in the process list and shell history while the command runs.
  • Temporary files of a restore into a user's files end up in that user's trash.
  • Restoring a whole user or the whole server needs occ (shell access); see "Without shell access" above.
  • --install-apps depends on the app store. Nextcloud downloads the whole app list (about 38 MB) and gives up after 120 seconds; on a slow day apps are not installed. The restore then lists them: install them afterwards with occ app:install, their data is restored already.
  • A pack whose index upload failed stays on the location until removed by hand.

Commands

Command Purpose
backup:key:init, backup:key:kit, backup:key:confirm create the key, recovery kit, confirmation
backup:key:slots, backup:key:slot:add/replace/remove passphrase slots
backup:target:backends/add/edit/list/remove/append-only backup locations
backup:run, backup:status, backup:list back up, status (runs of the last 7 days; --all for all), restore points
backup:cancel <run> stop a running backup
backup:browse, backup:restore browse and restore
backup:retention, backup:prune, backup:trash, backup:trash:restore retention, cleanup, trash
backup:verify checksum audit of a restore point (--deep downloads and decrypts everything)
backup:restore:full whole-server restore onto a fresh installation (after backup:key:import)
backup:key:import import a recovery kit on a fresh installation
backup:user:backup, backup:user:restore one user via user_migration (asks before installing it)

About

Encrypted, incremental backups of your whole Nextcloud (files, database, config) to S3, SFTP, WebDAV, SMB and more, with restore from the web interface.

Topics

Resources

Stars

9 stars

Watchers

1 watching

Forks

Releases

Packages

Contributors

Languages