Skip to content

fix(metrics): keep each account's metrics apart #955

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
hitesh96db wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix(metrics): keep each account's metrics apart #955

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
e48c506
fix(metrics): Keep each account's metrics apart
Nov 11, 2026
3ec94e1
fix(metrics): Filter segments by account, name the account
Nov 11, 2026
5a44c5c
fix(metrics): Test each store filter alone, drop a redundant index
Nov 11, 2026
4d8eb8d
test(metrics): Check the account of the throttle metrics
Nov 11, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 24 additions & 23 deletions CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -375,28 +375,28 @@ SQLite, PostgreSQL, Cassandra and MongoDB migrate through this framework.

### Example: a nullable column on `tables`

Migration 10, `table_note` (10 being the next free id at the time of writing;
migrations 6, 7, 8 and 9 are `backup_catalog_index`, `stream_pending`,
`restore_progress` and `warm_throughput`), adds a nullable `note TEXT` column
to the catalog's `tables` table on SQLite and PostgreSQL.
Migration 11, `table_note` (11 being the next free id at the time of writing;
migrations 6, 7, 8, 9 and 10 are `backup_catalog_index`, `stream_pending`,
`restore_progress`, `warm_throughput` and `account_metrics`), adds a nullable
`note TEXT` column to the catalog's `tables` table on SQLite and PostgreSQL.

In `crates/storage/src/migrations/registry.rs`, at the end of the
`migration_ids!` list:

```rust
/// Adds a nullable `note` column to the `tables` catalog table.
TableNote = 10, "table_note",
TableNote = 11, "table_note",
"add a nullable note column to the tables catalog";
```

PostgreSQL has `ADD COLUMN IF NOT EXISTS`, so its body is a file,
`crates/storage-postgres/migrations/0010_table_note.sql`:
`crates/storage-postgres/migrations/0011_table_note.sql`:

```sql
-- Copyright 2026 ExtendDB contributors
-- SPDX-License-Identifier: Apache-2.0
--
-- Migration 10, table_note, PostgreSQL catalog database: a nullable note
-- Migration 11, table_note, PostgreSQL catalog database: a nullable note
-- column on tables. IF NOT EXISTS makes a second run change nothing.

ALTER TABLE tables ADD COLUMN IF NOT EXISTS note TEXT;
Expand All @@ -411,7 +411,7 @@ In `crates/storage-postgres/src/migrations/mod.rs`, the arm in `migration()`:
bodies: vec![file_body!(
StoreKind::Catalog,
"migrations",
"0010_table_note.sql"
"0011_table_note.sql"
)],
},
```
Expand All @@ -427,19 +427,20 @@ and the id in the transactional arm of `runs_in()`:
| MigrationId::StreamPending
| MigrationId::RestoreProgress
| MigrationId::WarmThroughput
| MigrationId::AccountMetrics
| MigrationId::TableNote => RunsIn::Transaction,
```

SQLite has no `ADD COLUMN IF NOT EXISTS`, and a file with a plain
`ADD COLUMN` fails when it runs a second time, so the SQLite body is code
that checks for the column first.
`crates/storage-sqlite/src/migrations/m0010_table_note.rs`:
`crates/storage-sqlite/src/migrations/m0011_table_note.rs`:

```rust
// Copyright 2026 ExtendDB contributors
// SPDX-License-Identifier: Apache-2.0

//! Migration 10, `table_note`, SQLite: a nullable `note` column on `tables`.
//! Migration 11, `table_note`, SQLite: a nullable `note` column on `tables`.
//!
//! SQLite has no `ADD COLUMN IF NOT EXISTS`, so the body reads the table's
//! columns first and adds the column only when it is missing. A second run
Expand All @@ -451,10 +452,10 @@ use extenddb_storage::migrations::{CodeBody, Done, MigrationError};
use super::SqliteStores;

/// This file, whose SHA-256 the checksum test pins.
pub(crate) const SOURCE: &str = include_str!("m0010_table_note.rs");
pub(crate) const SOURCE: &str = include_str!("m0011_table_note.rs");

/// Recorded as this migration's checksum. Changes only with a new migration.
const FINGERPRINT: &str = "0010_table_note/sqlite/1";
const FINGERPRINT: &str = "0011_table_note/sqlite/1";

pub(crate) struct TableNote;

Expand Down Expand Up @@ -493,7 +494,7 @@ beside the others:
```rust
#[rustfmt::skip]
#[allow(clippy::all, deprecated)]
mod m0010_table_note;
mod m0011_table_note;
```

and the arm in `migration()`:
Expand All @@ -504,8 +505,8 @@ and the arm in `migration()`:
convergent: true,
bodies: vec![StoreBody::code(
StoreKind::Catalog,
&m0010_table_note::TableNote,
m0010_table_note::SOURCE,
&m0011_table_note::TableNote,
m0011_table_note::SOURCE,
)],
},
```
Expand All @@ -517,8 +518,8 @@ rows of `tables`:

```rust
#[tokio::test]
async fn migration_10_adds_the_note_column_to_tables() {
let db = temp_db("m10");
async fn migration_11_adds_the_note_column_to_tables() {
let db = temp_db("m11");
load_dump(&db, DEV_0_1_14).await;
run(&db).await.expect("migrate");
let mut text = dump(
Expand All @@ -528,12 +529,12 @@ async fn migration_10_adds_the_note_column_to_tables() {
)
.await;
text.push_str(&dump(&db, "SELECT table_name, note FROM tables ORDER BY table_name").await);
golden("0010_table_note", &text);
golden("0011_table_note", &text);
}
```

`EXTENDDB_BLESS_EXPECTED=1 cargo test -p extenddb-storage-sqlite --lib migration_10`
writes `crates/storage-sqlite/tests/fixtures/golden/0010_table_note.txt`, one
`EXTENDDB_BLESS_EXPECTED=1 cargo test -p extenddb-storage-sqlite --lib migration_11`
writes `crates/storage-sqlite/tests/fixtures/golden/0011_table_note.txt`, one
tab-separated line per row under each query:

```
Expand All @@ -548,15 +549,15 @@ The PostgreSQL body is a file, so it has no golden test.
The example declares `Kind::Breaking`. A nullable column can be additive by
the rule in step 4, but declaring it additive needs the old-binary job, which
this example does not add. As a breaking migration it makes a 1.x binary
without migration 10 refuse a catalog that has it, and a rollback restores the
without migration 11 refuse a catalog that has it, and a rollback restores the
backup.

The same PR changes the code that reads and writes `tables.note` in each
backend's catalog store, and commits the `docs/schema/sqlite.txt` and
`docs/schema/postgres.txt` that the integration jobs regenerate. It changes
no per-table DDL, because `tables` is a catalog table, and adds no `RELEASED`
entry. `extenddb migrate --list` then shows migration 10, and on a catalog at
migration 9 `extenddb migrate` lists migration 10 as pending and exits 3.
entry. `extenddb migrate --list` then shows migration 11, and on a catalog at
migration 10 `extenddb migrate` lists migration 11 as pending and exits 3.

### Running the migration tests

Expand Down
Loading
Loading