Skip to content

fix(tests): compare the two ingest paths' daily avg within tolerance - #132

Merged
olavgg merged 2 commits into
mainfrom
fix/binary-aggregate-tolerance
Sep 18, 2026
Merged

olavgg merged 2 commits into
mainfrom
fix/binary-aggregate-tolerance

Conversation

@JosteinGj

@JosteinGj JosteinGj commented Sep 17, 2026 •

Copy link
Copy Markdown
Contributor

Builds on #130 — main does not compile without it, so this branch carries that commit too. Once #130 merges, the diff here shrinks to the one commit.

test_datapoints_binary ingests the same sixty days into one series through the binary path and a twin through JSON, then requires their daily avg/min/max to match after truncating to ten decimals. It failed twice in a row on:

avg min max
binary, 2025-02-28 179.9759117537 160.0001581894 199.9998935423
JSON, 2025-02-28 179.9759117538 160.0001581894 199.9998935423

One bucket of 60, avg only, 1e-10, with both series walked to all 5,184,000 points. A dropped or duplicated point moves a daily avg by ~1e-3. This is ClickHouse summing the two series' parts in a different order, and the truncation boundary turning a difference in the last digits into an inequality.

Change

  • min / max stay exact. They select a stored value, so they still catch the two paths storing different values.
  • avg within AVG_TOLERANCE (1e-6), the bound validate_daily_avg already uses for the same reason, with the rationale written beside it.
  • truncate_10 goes with its last caller, and a bucket missing an aggregate now fails by name rather than as a NaN comparison.

Passes live.

🤖 Generated with Claude Code

JosteinGj and others added 2 commits September 17, 2026 09:41
#123 was branched before #128 gave ResponseError a content_type field
and merged after it, so main stopped compiling at the three places the
binary path builds one. All three are raised by the SDK with no server
response behind them (a 204 that fails to deserialize, a value refused
locally, a series /byids cannot find), so each carries None.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: jgjesdal <jostein@intellistream.ai>
test_datapoints_binary truncated avg/min/max to ten decimals and required
the binary and JSON series to match exactly. It failed twice in a row on
one bucket of 60 (2025-02-28): avg 179.9759117537 against ...538, min and
max identical, both series walked to all 5,184,000 points. A dropped or
duplicated point moves a daily avg by ~1e-3; a 1e-10 difference is
ClickHouse summing the two series' parts in a different order, landing
either side of the truncation boundary.

min and max select a stored value, so they stay exact and still catch
the two paths storing different values. avg is compared within
AVG_TOLERANCE, the bound validate_daily_avg already uses for the same
reason. truncate_10 goes with its last caller, and an aggregate the read
path leaves out now fails by name instead of as a NaN comparison.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Signed-off-by: jgjesdal <jostein@intellistream.ai>
@JosteinGj
JosteinGj requested a review from olavgg September 18, 2026 11:59
@olavgg
olavgg merged commit e228e20 into main Sep 18, 2026
18 checks passed
@olavgg
olavgg deleted the fix/binary-aggregate-tolerance branch September 18, 2026 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants