Skip to content

feat(eap-items): add version column to eap_items_1 - #8424

Merged
onewland merged 2 commits into
masterfrom
feat/eap-items-version-column
Sep 2, 2026
Merged

feat(eap-items): add version column to eap_items_1#8424
onewland merged 2 commits into
masterfrom
feat/eap-items-version-column

Conversation

@onewland

@onewland onewland commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Adds a version UInt64 column to eap_items_1_local and eap_items_1_dist.

ALTER TABLE eap_items_1_local ADD COLUMN IF NOT EXISTS version UInt64 DEFAULT 0 CODEC (ZSTD(1));
ALTER TABLE eap_items_1_dist  ADD COLUMN IF NOT EXISTS version UInt64 DEFAULT toUnixTimestamp64Milli(now64(3)) CODEC (ZSTD(1));

Why

We intend to move eap_items to a ReplacingMergeTree that uses version as its version column. Before we can attach the existing parts onto the new table, the current table's schema has to line up with the target schema — so the column needs to land here first, and subsequently be copied to the downsampled tables.

This PR is schema-only. It does not change the engine, so no dedup behavior changes yet.

Why the local and dist defaults differ

This asymmetry is deliberate. Both halves were verified empirically against ClickHouse 25.3.

Local is DEFAULT 0. ADD COLUMN does not rewrite existing parts — the column is simply absent from them (confirmed via system.parts_columns), so reads synthesize it from the default expression at read time. With a now64() default that is non-deterministic; the same row returns a different value on consecutive reads:

read #1:  id=1 -> 1788300553940
read #2:  id=1 -> 1788300555980     # 2s later, same row

Under ReplacingMergeTree(version) this is actively harmful. The legacy row's version gets computed at merge time, which is always later than any real write, so the stale row wins. Inserting a genuine update against a DEFAULT now64() destination and running OPTIMIZE ... FINAL silently discarded the update in favour of the pre-existing row. Against a DEFAULT 0 destination the real update won and the untouched legacy row stayed a stable 0.

So 0 is the sentinel for "written before versioning existed", and it deterministically loses to every real write. This matches the received_at precedent in snuba/manual_jobs/create_eap_received_at_version_test.py, which uses a bare UInt64 and filters WHERE received_at != 0.

Dist is DEFAULT toUnixTimestamp64Milli(now64(3)). A Distributed table evaluates its own DEFAULT expressions on the initiator node and ships the materialized value to the shard, so for inserts routed through _dist the local default never fires. Measured, with deliberately distinguishable values:

local dist value stored in local part
DEFAULT 111 DEFAULT 999 999
DEFAULT 111 no default 0
no default DEFAULT 999 999

Note the middle row: declaring the column on _dist without a default does not fall through to the local default, it ships an explicit 0. So the dist default is what actually assigns a version to live writes, and it is required if we want inserts to carry a real timestamp before the consumer is taught to populate the column.

End-to-end, the pair gives the semantics we want:

id=1 (legacy part) -> 0              stable across re-reads
id=2 (live insert) -> 1788300664281  = 2026-09-01 22:11:04.281

Scope

Limited to _local + _dist. The downsample tiers (eap_items_1_downsample_{8,64,512}) are a follow-up, along with the corresponding materialized view rebuild.

version is intentionally not added to eap_items_1_dist_ro. It is internal bookkeeping for the ReplacingMergeTree cutover and is absent from the storage YAML, so nothing the query layer generates can reference it. Migration 0056 creates the _dist_ro tables before this one runs, so fresh and existing deployments both end up without the column there — there is no environment drift. If we ever want to read version through the read-only routing path it can be added in its own migration.

The column is likewise not added to eap_items.yaml, so it stays invisible to the query layer for now (same as client_sample_rate).

Testing

  • tests/migrations/test_runner.py::test_no_schema_differences
  • tests/migrations/test_runner.py::test_run_and_reverse_all

Adds a UInt64 `version` column defaulting to the current time in
milliseconds to eap_items_1_local and eap_items_1_dist.

This is a schema-only step toward moving eap_items to a
ReplacingMergeTree keyed on `version`. The column needs to exist on the
current table before old parts can be attached to the new table.
@onewland
onewland requested review from a team as code owners September 1, 2026 22:06
@github-actions

github-actions Bot commented Sep 1, 2026

Copy link
Copy Markdown

This PR has a migration; here is the generated SQL for ./snuba/migrations/groups.py ()

-- start migrations

-- forward migration events_analytics_platform : 0064_add_version_column
Local op: ALTER TABLE eap_items_1_local ON CLUSTER 'cluster_one_sh' ADD COLUMN IF NOT EXISTS version UInt64 DEFAULT 0 CODEC (ZSTD(1));
Distributed op: ALTER TABLE eap_items_1_dist ON CLUSTER 'cluster_one_sh' ADD COLUMN IF NOT EXISTS version UInt64 DEFAULT toUnixTimestamp64Milli(now64(3)) CODEC (ZSTD(1));
-- end forward migration events_analytics_platform : 0064_add_version_column




-- backward migration events_analytics_platform : 0064_add_version_column
Distributed op: ALTER TABLE eap_items_1_dist ON CLUSTER 'cluster_one_sh' DROP COLUMN IF EXISTS version;
Local op: ALTER TABLE eap_items_1_local ON CLUSTER 'cluster_one_sh' DROP COLUMN IF EXISTS version;
-- end backward migration events_analytics_platform : 0064_add_version_column

ADD COLUMN does not rewrite existing parts, so rows predating this
migration have no `version` on disk and the local DEFAULT is evaluated
at read/merge time. Under ReplacingMergeTree(version) a now64() default
on the local table would give those legacy rows a version of whenever
the merge ran -- always newer than a real write -- letting a stale row
silently beat a genuine update.

The local table now defaults to 0, the sentinel for rows written before
versioning existed, which deterministically loses to every real write.

Inserts go through the distributed table, which evaluates its own
DEFAULT on the initiator and ships the materialized value to the shard,
so the dist table keeps the now64() default and live writes still get a
real millisecond timestamp.
@onewland
onewland merged commit 2c99a33 into master Sep 2, 2026
108 of 136 checks passed
@onewland
onewland deleted the feat/eap-items-version-column branch September 2, 2026 15:19
onewland added a commit that referenced this pull request Sep 3, 2026
…#8428)

### Stack

1. ~#8424 — `eap_items_1` (merged)~
2. #8428 — `eap_items_1_downsample_8_local`
3. #8426 — `eap_items_1_downsample_64_local`
4. #8427 — `eap_items_1_downsample_512_local`
5. #8430 — downsample MVs carry `version`

Merge in order; each targets the one above it.

---

Adds the `version` column to the `downsample_8` tier.

```sql
ALTER TABLE eap_items_1_downsample_8_local ADD COLUMN IF NOT EXISTS version UInt64 DEFAULT 0 CODEC (ZSTD(1));
```

### Why

Part of getting the `eap_items` schemas aligned with the target
`ReplacingMergeTree` (keyed on `version`) so that existing parts can be
attached onto the new tables. This mirrors the column added to
`eap_items_1` in migration 0064.

Schema-only — no engine change, so no dedup behaviour changes yet.

### Why `DEFAULT 0`

`ADD COLUMN` does not rewrite existing parts, so rows written before
this migration have no `version` on disk and the default is evaluated at
read/merge time. That makes a non-deterministic default actively
dangerous here: under `ReplacingMergeTree(version)` a `now64()` default
gives legacy rows a version of "whenever the merge ran", which always
beats a genuine write. Verified on CH 25.3 — a real update was silently
discarded in favour of the stale row against a `DEFAULT now64()`
destination, and won as expected against a `DEFAULT 0` one.

`0` is the sentinel for "written before versioning existed" and
deterministically loses to every real write, matching the `received_at`
precedent in `snuba/manual_jobs/create_eap_received_at_version_test.py`.

### Why local only

Unlike 0064, this does not touch `eap_items_1_downsample_8_dist` (or
`_dist_ro`):

- We never query `version` — it is internal bookkeeping for the cutover
and is absent from the storage YAML, so the query layer cannot generate
SQL naming it.
- The downsample tiers are fed by materialized views that write straight
to `*_local` (`swap_downsample_materialized_views` uses
`destination_table_name={prefix}_local`,
`target=OperationTarget.LOCAL`), so the distributed table is never an
insert target.

A Distributed table declaring a subset of its local table's columns
behaves normally — verified that reads and `SELECT *` through the
narrower dist table work, and that an insert routed through it still
lets the local default fire. Only an explicit `SELECT version` through
the dist table errors, and nothing does that.

### Follow-up

The downsample MVs do not yet carry `version` through from the source,
so rows they insert will read `0` rather than the source row's version.
Fixing that needs an MV rebuild and is deliberately out of scope here.

### Testing

- `tests/migrations/test_runner.py::test_no_schema_differences`
- `tests/migrations/test_runner.py::test_run_and_reverse_all`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants