DA262: S3 backup (create-backup + list-backups)

[delgod]

Summary

Adds two Juju actions — create-backup and list-backups — that stream a fresh Valkey RDB snapshot from the targeted unit's valkey-cli --rdb - stdout directly to S3 via boto3 multipart upload. Any unit (leader or follower) can run the actions; locking is per-unit.

Credentials are supplied by relating the charm to the upstream s3-integrator via a new s3-credentials relation (interface s3, limit 1, optional).

Depends on

#58 (workload-exec-stream). Must be merged first — this PR uses workload.exec_stream and CliClient.build_command_prefix.

What's in the PR

src/
literals.py                # S3_RELATION_NAME, BACKUP_ID_FORMAT, BACKUP_CA_FILENAME
statuses.py                # BackupStatuses enum (3 entries)
common/exceptions.py       # ValkeyBackupError
core/base_workload.py      # tls_paths.backup_ca property
core/models.py             # s3_credentials on PeerAppModel,
                           # backup_id on PeerUnitModel,
                           # ValkeyServer.is_backup_in_progress,
                           # ValkeyCluster.s3_credentials (parsed JSON envelope)
core/cluster_state.py      # s3_relation property
managers/backup.py         # NEW – BackupManager + status protocol
events/backup.py           # NEW – BackupEvents + _exists_preventing_reason
events/base_events.py      # storage_detaching guard during backup
charm.py                   # wire BackupManager + BackupEvents;
                           # register with StatusHandler;
                           # restart_workload guard during backup
metadata.yaml                # requires.s3-credentials
actions.yaml                 # create-backup, list-backups
pyproject.toml               # boto3, mypy-boto3-s3
lib/charms/data_platform_libs/v0/s3.py   # via charmcraft fetch-lib
tests/unit/test_backup.py    # ~40 ops-scenario unit tests

Design highlights

• Streaming, no archive storage. bucket.upload_fileobj(proc.stdout, key, Config=TransferConfig(multipart_chunksize=8 MiB)) reads chunks from the pipe; no on-disk staging.
• Per-unit lock. The backup_id field in the running unit's peer-unit databag is both the lock value and the operation identifier. Two backups can run concurrently on different units (each gets a distinct S3 key); two on the same unit are rejected.
• Cleanup on failure. Partial S3 objects are deleted (bucket.Object(key).delete()) when valkey-cli exits non-zero or upload_fileobj raises.
• boto3 quirks honoured. Config(request_checksum_calculation="when_required", response_checksum_validation="when_required") (boto3#4400);
  CreateBucketConfiguration omitted for us-east-1 (aws-sdk-js#3647);
  idempotent error tokens (BucketAlreadyOwnedByYou, BucketAlreadyExists, BucketNameUnavailable).
• TLS CA chain stored on every unit (not only the leader) so any unit can use TLS to talk to a self-signed S3 endpoint (e.g. RadosGW, MicroCeph).
• Leader-switchover recovery. Re-observes leader_elected to re-trigger _on_s3_credentials_changed — covers the case where credentials_gone fired only on the old leader.

Action UX

# Any unit — leader or follower
juju run valkey/N create-backup
# → { "backup-id": "2026-05-13T10:00:00Z" }

juju run valkey/N list-backups
# → { "backups": "backup-id … | backup-status\n-…-\n2026-05-13T10:00:00Z | finished" }

Status surface:

• BACKUP_IN_PROGRESS (maintenance, unit scope) while streaming
• BACKUP_S3_PARAMETERS_MISSING (blocked, app scope) when the relation is present but the integrator hasn't supplied bucket/credentials yet
• BACKUP_FAILED (blocked, unit scope) after a failure

[Mehdi-Bendriss]

Thank you Mykola! Good work! I left some comments

[Mehdi-Bendriss]

Thank you Mykola!

[skourta]

Thank you @delgod. I left some comments/questions.

[skourta]

LGTM. The only remaining concern is if the backup procedure (streaming) takes a long time the create backup action will time out.

[reneradoi]

Thank you Mykola, and sorry for the late review. I agree with the overall concept and have only some minor comments and questions.

[reneradoi]

Hint for the failing integration tests: #76

[reneradoi]

Hey Mykola! I've tested the PR, it looks really good UX wise and the implementation is also robust against failure scenarios. I am happy to approve, there is only one issue that we need to either solve in this PR or in the next one, and it is about safeguarding the backup creation on units in parallel: I was able to produce a situation where two units create a backup at the same time, which results in the exact same name for the backup file to be uploaded to s3 storage. Both units stream their backup (according to the logs), and there is no error. But we end up with only one backup in the s3 storage (see screenshot below, sorry it is a bit chaotic 😓 ). It is not clear which unit won the race, whether the file is corrupted or not, and there is no error from the s3 client either. This might be an edge case, but it could happen for example with scheduled backups. I think we need to safeguard it. The most simple solution would be to add the unit name to the backup file, but that's just an idea.

[reneradoi]

The issue mentioned above will be treated in a separate PR and will be tracked with #77.