Repository navigation
docs: state the 1.0 scope, the backend tiers and the versioning policy - #897
LeeroyHannigan wants to merge 11 commits into
Conversation
The README gets a Scope section and a Storage backends section. The second section gives each backend a tier: PostgreSQL stable, SQLite stable on a single node, MongoDB beta and Cassandra experimental. The new docs/versioning.md says what each kind of release can change and what stays compatible within a major version. It also gives the support periods and the deprecation steps. The comment in crates/bin/src/main.rs now describes SQLite as a production backend on a single node. The dev-mode comments in main.rs and in the Cargo.toml files of crates/bin and crates/app say that dev-mode compiles only with SQLite. The upgrade manual now says that a later major release refuses CreateBackup without a backup store. Refs #662 Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
This resolves the conflict in the upgrade manual. #474 added the migration 9 paragraph, and #740 rewrote the backup store paragraph after it. The merge keeps both paragraphs as main has them. In the backup store paragraph, it applies the one change of this branch: a later major release refuses CreateBackup without a store. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
Main requires the cassandra-integration check. That job needs the Cassandra catalog checks job, which runs the checks of migrated catalogs. The Cassandra row of the README table said that neither check is required. The row now names cassandra-integration as required and drops the sentence about the catalog checks. The row also said that Cassandra has no backup store. A Cassandra deployment can configure a store, and a restore can read a backup from it. CreateBackup on Cassandra still keeps each backup in the catalog, and the row now says that. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
| @@ -26,9 +26,10 @@ mongodb-test-hooks = [ | |||
| "extenddb-app/mongodb-test-hooks", | |||
| ] | |||
| # Developer mode: plain HTTP on loopback, open authorization (SigV4 still | |||
There was a problem hiding this comment.
Here and elsewhere it might help to clarify this with "SigV4 authentication" versus just "SigV4".
There was a problem hiding this comment.
| Migration 9, `warm_throughput`, adds a nullable `warm_throughput` column to the `tables` and `indexes` catalog tables on PostgreSQL, SQLite and Cassandra. It holds the `WarmThroughput` a client sets on a table or a global secondary index, which `DescribeTable` reports; nothing acts on the value (see [Differences from DynamoDB](../differences-from-dynamodb.md)). Tables created before the migration report the defaults for their billing mode. On MongoDB it changes nothing, since the catalog collections have no validator, and the ledger records it as not applicable. Migration 9 is breaking: to go back to a release without it, restore the backup taken before the upgrade (see [Rollback Procedure](#rollback-procedure)). | ||
|
|
||
| Configure `[backup.store]` before the release that stops creating in-catalog backups. From this release CreateBackup writes to the configured store on PostgreSQL, SQLite and MongoDB, and is asynchronous there as on the service: the response is `CREATING` and the backup becomes `AVAILABLE` once the upload ends, so a client that deletes a backup it just created has to poll DescribeBackup first (the previous release returned `AVAILABLE`). A store-backed backup is listed, described, deleted and restored like any other. `RestoreTableFromBackup` of a backup in the configured store runs the restore job from the store on every backend (PostgreSQL, SQLite, MongoDB and Cassandra); in-catalog backups, those of the previous release included, restore as before. An S3 store needs `spool_path` set (the restore's download area) on PostgreSQL, MongoDB and Cassandra, and that must be a directory of its own, not the database directory; a filesystem store needs no spool. Migration 8 adds the `restore_progress` table to the PostgreSQL data database; run `extenddb migrate --yes` before starting the new binary. The progress rows of a restore in flight live in the data database, so a data database restored from a dump taken before migration 8 has none of them and no `restore_progress` table: the new binary refuses to serve it until `extenddb migrate --yes` has run again, and its reconciler then finds no progress for the restore and resumes it from its first data file into the (empty) target, or `FAILED`s it as `target_not_empty` if the target already holds items, in which case `extenddb backup remove-target --account <account> <table>` (the admin guide's `backup` section) removes the target and its record and the backup restores again. The same `target_not_empty` state is what a server stopped inside a restore's first data file on MongoDB, or inside its first batch of 10,000 items or 32 MiB on Cassandra, leaves on restart, since both record progress after the write; the command is the remedy there too. `backup remove-target` writes the catalog, so on SQLite the server is stopped for it, as for `purge`. The previous binary resumes no restore (a store-backed restore in flight at a rollback is lost with the catalog rows, as the rollback paragraph below says). Quiesce restores before a downgrade. `kind = "disabled"` is no longer needed to keep a deployment able to restore the backups it takes. Without a store CreateBackup writes the in-catalog format and logs a deprecation warning naming `[backup.store]` on each call; a later release's CreateBackup refuses without one. Cassandra writes the in-catalog format with or without a store. `kind = "disabled"` under `[backup.store]` is the opt-out that keeps a deployment on the in-catalog format on purpose and silences the warning. On file-backed SQLite an absent section is a filesystem store at `backups/` beside the database file, created `0700` at startup; that root must be owned by the extenddb user with mode `0700`, under directories owned by root or that user which others cannot write to. A default root that breaks the rule (a database file in a group-writable directory, say), or whose `extenddb-store.json` names another catalog (a catalog re-created by `destroy` and `init` under the old root), is logged at startup as a warning (`backup store: the default root ... failed its check`) and leaves the store unconfigured, not unhealthy, so the upgrade never stops a server that never configured backups and CreateBackup keeps working on it; the same root configured explicitly is a startup error or an unhealthy store. The warning applies only while the catalog has no backup rows for the default root (an upgrade, a fresh deployment, a catalog re-created by `destroy` and `init`): once a server has written rows for it, the default root is handled as a configured one, so its backups never vanish behind a warning (a failed identity check is an unhealthy store, reported on `/health`; a root that can no longer be opened is a startup error naming it). Instances that share a catalog must share one store root, an S3 bucket and prefix or a shared mount: a second instance pointed at a host-local directory under the same configured path starts with the store unhealthy (`/health` reports `backup_store` unhealthy; the admin guide's `[backup.store]` section says what the identity file and the manifest probe check). `extenddb destroy` leaves the store; `extenddb backup purge --confirm` is what deletes its backups, and a catalog re-created with `init` needs the old root's `extenddb-store.json` removed or a new root, since the file names the old catalog. | ||
| Configure `[backup.store]` before the release that stops creating in-catalog backups. From this release CreateBackup writes to the configured store on PostgreSQL, SQLite and MongoDB, and is asynchronous there as on the service: the response is `CREATING` and the backup becomes `AVAILABLE` once the upload ends, so a client that deletes a backup it just created has to poll DescribeBackup first (the previous release returned `AVAILABLE`). A store-backed backup is listed, described, deleted and restored like any other. `RestoreTableFromBackup` of a backup in the configured store runs the restore job from the store on every backend (PostgreSQL, SQLite, MongoDB and Cassandra); in-catalog backups, those of the previous release included, restore as before. An S3 store needs `spool_path` set (the restore's download area) on PostgreSQL, MongoDB and Cassandra, and that must be a directory of its own, not the database directory; a filesystem store needs no spool. Migration 8 adds the `restore_progress` table to the PostgreSQL data database; run `extenddb migrate --yes` before starting the new binary. The progress rows of a restore in flight live in the data database, so a data database restored from a dump taken before migration 8 has none of them and no `restore_progress` table: the new binary refuses to serve it until `extenddb migrate --yes` has run again, and its reconciler then finds no progress for the restore and resumes it from its first data file into the (empty) target, or `FAILED`s it as `target_not_empty` if the target already holds items, in which case `extenddb backup remove-target --account <account> <table>` (the admin guide's `backup` section) removes the target and its record and the backup restores again. The same `target_not_empty` state is what a server stopped inside a restore's first data file on MongoDB, or inside its first batch of 10,000 items or 32 MiB on Cassandra, leaves on restart, since both record progress after the write; the command is the remedy there too. `backup remove-target` writes the catalog, so on SQLite the server is stopped for it, as for `purge`. The previous binary resumes no restore (a store-backed restore in flight at a rollback is lost with the catalog rows, as the rollback paragraph below says). Quiesce restores before a downgrade. `kind = "disabled"` is no longer needed to keep a deployment able to restore the backups it takes. Without a store CreateBackup writes the in-catalog format and logs a deprecation warning naming `[backup.store]` on each call. A later major release's CreateBackup refuses without one. Cassandra writes the in-catalog format with or without a store. `kind = "disabled"` under `[backup.store]` is the opt-out that keeps a deployment on the in-catalog format on purpose and silences the warning. On file-backed SQLite an absent section is a filesystem store at `backups/` beside the database file, created `0700` at startup; that root must be owned by the extenddb user with mode `0700`, under directories owned by root or that user which others cannot write to. A default root that breaks the rule (a database file in a group-writable directory, say), or whose `extenddb-store.json` names another catalog (a catalog re-created by `destroy` and `init` under the old root), is logged at startup as a warning (`backup store: the default root ... failed its check`) and leaves the store unconfigured, not unhealthy, so the upgrade never stops a server that never configured backups and CreateBackup keeps working on it; the same root configured explicitly is a startup error or an unhealthy store. The warning applies only while the catalog has no backup rows for the default root (an upgrade, a fresh deployment, a catalog re-created by `destroy` and `init`): once a server has written rows for it, the default root is handled as a configured one, so its backups never vanish behind a warning (a failed identity check is an unhealthy store, reported on `/health`; a root that can no longer be opened is a startup error naming it). Instances that share a catalog must share one store root, an S3 bucket and prefix or a shared mount: a second instance pointed at a host-local directory under the same configured path starts with the store unhealthy (`/health` reports `backup_store` unhealthy; the admin guide's `[backup.store]` section says what the identity file and the manifest probe check). `extenddb destroy` leaves the store; `extenddb backup purge --confirm` is what deletes its backups, and a catalog re-created with `init` needs the old root's `extenddb-store.json` removed or a new root, since the file names the old catalog. |
There was a problem hiding this comment.
This paragraph could use some luv, especially if we expect humans to read it. It's a wall of text, has run-on sentences (at least one is paragraph-long in itself), contains a fair amount of implementation detail which may not help someone just trying to figure out the commands to run and what they do, and extraneous AI detail as in "a root that can no longer be opened is a startup error naming it" (just stop at "error").
There was a problem hiding this comment.
Split in 0365f0c, then corrected in 7f937f0 and 1463d68. The paragraph is now a list of how CreateBackup and RestoreTableFromBackup work in this release, followed by short paragraphs. The cut detail covers progress rows, batch sizes and the identity checks. Two statements carried over from main were wrong, and 7f937f0 corrects both:
- A restore with no recorded progress starts again from its first data file. It fails as
target_not_emptyonly when the target holds more items than the backup. The text said a stop inside the first data file on MongoDB, or the first batch on Cassandra, leavestarget_not_empty. kind = "disabled"does not silence the deprecation warning on each CreateBackup. It silences the warning about the default root on file-backed SQLite.
The paragraph on several servers now says what the README says: one server per catalog in 1.0. It no longer says that a second server's store starts unhealthy. Both servers start healthy until the catalog has backup rows for the root. The sentence about a root that can no longer be opened stops at "a startup error".
|
|
||
| ## Storage backends | ||
|
|
||
| An ExtendDB binary contains one storage backend, which the build selects. The tier of a backend says what the 1.x releases promise for it: |
There was a problem hiding this comment.
I'm reluctant to introduce tiers unless we have a process/policy for how a backend moves from one tier to another. I also thought that last week we'd converged on "a backend is production (or stable) if the ExtendDB CI is reliably green for it".
There was a problem hiding this comment.
The three tiers are the ones #662 asks for, so I kept them. 9d136d2 and 4e44a88 add the rule for moving between them, built on that definition. Both tiers for production, stable and beta, now say that the ExtendDB CI is reliably green for the backend. A backend moves up one tier in a minor release. Before that, the maintainers agree in an issue that its CI has been reliably green and that its known limits fit the higher tier. It moves down only through the deprecation steps of the versioning policy.
The dev-mode comments in crates/bin/Cargo.toml and crates/app/Cargo.toml said that dev mode still enforces SigV4. They now say SigV4 authentication. Dev mode opens authorization, and it still checks the signature of each request. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
The backup store paragraph of the upgrade manual was one block of 741 words, with several run-on sentences. It is now a list of the changes to CreateBackup and RestoreTableFromBackup. Short paragraphs follow on the S3 spool, migration 8, the remove-target cases, rollback, the default SQLite root, shared catalogs, and destroy and purge. The facts stay the same, with one detail cut. The text no longer says that the startup error for a root that cannot be opened names the root. The rollback reference now links to the Migration 6 section. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
A stable backend is one that the ExtendDB CI is reliably green for. A backend moves up one tier in a minor release, once the CI has been reliably green for it. The maintainers first agree in an issue that its known limits fit the higher tier. The release notes name the move. A backend moves down a tier only through the deprecation steps of the versioning policy. The stable tier no longer says that required CI checks test the backend, because each of the four backends now has a required check. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
The reconciler starts a restore with no recorded progress again from its first data file. It fails the restore target_not_empty only when the target holds more items than the backup. A MongoDB batch and its progress commit in one transaction. The manual said that a stop inside the first data file on MongoDB, or inside the first batch on Cassandra, leaves target_not_empty. kind = "disabled" keeps the in-catalog format, and CreateBackup still logs its deprecation warning on each call. What it silences is the warning about the default root on file-backed SQLite. The manual now says that one catalog takes one server in 1.0, as the README says. It drops the detail about progress rows and the identity checks. It gives the spool rule for every backend and says what a rollback to the Migration 6 backup loses. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
Both tiers for production now say that the ExtendDB CI is reliably green for the backend. Before a move up, the maintainers agree in an issue that the CI has been reliably green. They also agree that the known limits fit the higher tier. The move down links to the deprecation steps, which also hold the exception for a security issue. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
The manual said that a second server on the same catalog, with a host-local store root, starts with the store unhealthy. While the catalog has no backup rows for the root, each server creates its own identity file, and both start healthy. The paragraph now says only that one catalog takes one server in 1.0 and points at the admin guide. The restart rule names the restore it covers: a store-backed restore whose server stopped. The reconciler fails a dead in-catalog restore. The ancestor rule for the default root also names group, as the check does. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
The upgrade manual conflicted with #855. The resolution keeps the split and corrected backup store text of this branch. It adds the two pointers to the Backup and Restore manual that #855 added. One follows the first sentence. The other gives the recovery of a re-created catalog with `extenddb backup sync --adopt-store`. It also keeps the rule from #855 that instances that share a catalog share one store root. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
The check of the directories above a filesystem root accepts a directory that group or others can write to when it carries the sticky bit. The Backup and Restore manual already says so. A PostgreSQL data database restored from a dump taken before migration 8 has no data table for the target of a running store-backed restore. The reconciler fails that restore and removes its target. The admin guide describes remove-target in its [backup.store] section. It has no backup section. The paragraph on a re-created catalog now says that the store stays unhealthy until backup sync --adopt-store runs. It links the sections of the Backup and Restore manual. Signed-off-by: Lee Hannigan <29015211+LeeroyHannigan@users.noreply.github.com>
|
Merged main in 2ec6290. #855 rewrote the same backup store paragraph of the upgrade manual. The resolution keeps the split and the corrections of this PR, and takes the two links to the Backup and Restore manual from #855. 8493d41 also changes these points in that paragraph:
#942 lists the texts on main that repeat the statements this PR corrects. |
What
README.mdgets a Scope section after Use Cases and a Storage backends section before Quick Start. The Storage backends section defines three tiers and puts each backend in one. Its table gives the topology, upgrade path, API coverage, CI job and known limits of each backend.docs/versioning.mdsays what each kind of release can change and what stays compatible within a major version. It also gives the catalog upgrade rules, what the policy does not cover, the support periods and the deprecation steps. The README documentation table links to it.crates/bin/src/main.rsnow describessqliteas a production backend on a single node. Thedev-modecomments inmain.rs,crates/bin/Cargo.tomlandcrates/app/Cargo.tomlnow say thatdev-modecompiles only withsqlite, and that a productionsqlitebuild leavesdev-modeoff. The text of thedev-modecompile error changes to match, and its condition stays the same.docs/manuals/07-upgrade-manual.mdsplits its backup store paragraph into a list of how CreateBackup and RestoreTableFromBackup work in this release, followed by short paragraphs. The text changes in these points:target_not_emptyonly when the target holds more items than the backup. The text said that a stop inside the first data file on MongoDB, or inside the first batch on Cassandra, leavestarget_not_empty.kind = "disabled"silences only the warning about the default root on file-backed SQLite. CreateBackup still logs its deprecation warning on each call.spool_pathon every backend except file-backed SQLite, where it defaults tobackup-spool/beside the database file.inithas an unhealthy store untilextenddb backup sync --adopt-storemakes the new catalog its owner. On file-backed SQLite with the default root, the store is unconfigured instead. The paragraph links the sections of the Backup and Restore manual for this and for instances that share a catalog.Each decision in the new text has its own line here, so a reviewer can object to each one separately. The list omits text that only describes current behavior, such as the catalog statuses and their exit codes.
Scope and tiers:
Releases:
Compatibility within a major version:
extenddb.toml, theirEXTENDDB__variables and the runtime settings keep their names and meanings.extenddb managecalls, and the options and return values ofstart()in@extenddb/dev, stay compatible./healthresponse and itsstatusfield stay compatible.Storagetrait among them) or preview settings such asenable_multipart_keys./metricsresponse or the experimental backends.Support:
latestimage tag stays on the newest release.Deprecation:
This change does not edit the following text on main, which disagrees with it:
docs/manuals/07-upgrade-manual.mdsays that MongoDB deployments track their own catalog version. Its first section says that a migration ledger versions all four catalogs.docs/adr/0023-catalog-migrations.mdcounts five v1.0 migrations. The migration registry lists nine.docs/cassandra-setup.mddescribes more than one ExtendDB instance on one Cassandra cluster.docs/manuals/11-deployment-guide.mdsays that more than one ExtendDB instance can share one backend catalog.docs/getting-started.mdnames its SQLite section "SQLite backend (developer mode)" and describes SQLite as a backend for local development and CI.docs/rfcs/0002-backend-plugin-policy.mdsays that all crates follow semantic versioning. The new policy does not cover the crate APIs.README.mdsays that TLS is mandatory. The new text says that dev mode uses plain HTTP.README.mdlists the Cassandra image among the production images. The new text says that the experimental tier is not for production.Known issues:
Why
Issue #662 asks for three texts before 1.0: a scope statement, backend tiers in the README, and a versioning, compatibility, support and deprecation policy. An operator who chooses a backend reads the README first, so the scope and the tiers go there. The policy has six sections, so it gets a file of its own.
Closes #662
Testing done
cargo test --locked -p extenddb-storage-postgres --test doc_version_literalspasses. The test fails when a three-part version after the word ExtendDB inREADME.mdordocs/is not the compiled version.cargo fmt --all -- --checkpasses.Checklist
cargo test --workspace)cargo fmt --check)cargo clippy -- -W clippy::pedantic)Storagetrait, auth model, on-diskformat, or public CLI surface, an RFC has been accepted or is linked
below. Otherwise, an ADR captures the decision (link below).
ADR / RFC: n/a
By submitting this pull request, I confirm that my contribution is made under
the terms of the Apache License 2.0 and I agree to the Developer Certificate of
Origin (DCO). See CONTRIBUTING.md for details.