Problem Statement
HydraDB documents two important metadata-schema constraints:
- Existing fields cannot be renamed, removed, retyped, or have their flags changed.
PATCH /databases/{database}/metadata-schema is additive, but it does not alter or backfill existing Milvus collections for newly added dense/sparse metadata lanes.
The current documentation stops at "create a new database/schema and re-ingest." It does not give developers a complete, safe journey for deciding between an in-place addition and a versioned rebuild, replaying canonical data, verifying indexing and retrieval, cutting application traffic over, retaining a rollback path, or cleaning up safely. HydraDB also does not currently document a schema-read endpoint, full-fidelity export job, database alias, or atomic swap.
Proposed Solution
Add a production-oriented Metadata Schema Migration cookbook plus a dependency-free Node.js 20 planner.
The offline planner would:
- validate versioned current/target schema fixtures;
- normalize documented defaults and compare fields deterministically;
- classify changes as
noop, in-place-additive, or versioned-rebuild;
- treat removals, renames, type/max-length/flag changes, and new dense/sparse/analyzer lanes as rebuilds;
- emit stable Markdown or JSON with reason codes, operator steps, limitations, and inert PATCH/POST request bodies;
- provide an optional expected-strategy CI gate;
- never read credentials, call the network, mutate schemas, ingest data, cut over traffic, or delete a database.
The cookbook would cover the full operator-owned path: capture the schema contract, create a versioned target when required, re-ingest from the canonical source with stable IDs and collections, poll all returned IDs, compare manifests and retrieval, switch application configuration manually, retain the old database for rollback, and delete only through a later explicit decision.
It would also reconcile the current metadata guide's "immutable" wording with the additive PATCH behavior and add the page to the cookbook index/navigation.
Contract confirmation requested
The public v2 reference says additive updates do not alter/backfill existing Milvus collections for new dense/sparse fields. Unless maintainers advise otherwise, the planner will conservatively require a versioned database for every newly added dense/sparse/analyzer lane, even when the database has no existing sources. Please confirm that this is the correct public contract.
Alternatives Considered
- A documentation-only decision tree would be faster, but it would not give teams a deterministic, CI-gated way to review schema changes.
- A fully automated migration command would be unsafe and misleading because the API does not expose a full-fidelity source export, schema introspection, database alias/swap, or rollback primitive.
Additional Context
This is intentionally independent from #184. PR #184 measures runtime retrieval quality; this proposal fixes metadata-schema evolution, reindexing, cutover, and rollback.
This is a HydraDB documentation hackathon contribution. Per CONTRIBUTING.md, I will wait for maintainer acknowledgment before starting the implementation.
Problem Statement
HydraDB documents two important metadata-schema constraints:
PATCH /databases/{database}/metadata-schemais additive, but it does not alter or backfill existing Milvus collections for newly added dense/sparse metadata lanes.The current documentation stops at "create a new database/schema and re-ingest." It does not give developers a complete, safe journey for deciding between an in-place addition and a versioned rebuild, replaying canonical data, verifying indexing and retrieval, cutting application traffic over, retaining a rollback path, or cleaning up safely. HydraDB also does not currently document a schema-read endpoint, full-fidelity export job, database alias, or atomic swap.
Proposed Solution
Add a production-oriented Metadata Schema Migration cookbook plus a dependency-free Node.js 20 planner.
The offline planner would:
noop,in-place-additive, orversioned-rebuild;The cookbook would cover the full operator-owned path: capture the schema contract, create a versioned target when required, re-ingest from the canonical source with stable IDs and collections, poll all returned IDs, compare manifests and retrieval, switch application configuration manually, retain the old database for rollback, and delete only through a later explicit decision.
It would also reconcile the current metadata guide's "immutable" wording with the additive PATCH behavior and add the page to the cookbook index/navigation.
Contract confirmation requested
The public v2 reference says additive updates do not alter/backfill existing Milvus collections for new dense/sparse fields. Unless maintainers advise otherwise, the planner will conservatively require a versioned database for every newly added dense/sparse/analyzer lane, even when the database has no existing sources. Please confirm that this is the correct public contract.
Alternatives Considered
Additional Context
This is intentionally independent from #184. PR #184 measures runtime retrieval quality; this proposal fixes metadata-schema evolution, reindexing, cutover, and rollback.
This is a HydraDB documentation hackathon contribution. Per CONTRIBUTING.md, I will wait for maintainer acknowledgment before starting the implementation.