Skip to content

Replace bulky test snapshots with behavioral assertions - #96

Merged
adi-kumo merged 24 commits into
v2-candidatefrom
rust-fixture-cleanup
Nov 10, 2026
Merged

adi-kumo merged 24 commits into
v2-candidatefrom
rust-fixture-cleanup

Conversation

@adi-kumo

@adi-kumo adi-kumo commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Replace large recorded test outputs with direct assertions over source rows, requests and results. Failures now identify the behavior that broke, while the repository carries fewer generated recordings.

  • Remove 27.95 MB and 386,612 lines from the audited fixture inventory.
  • Keep useful protocol/API inventories, historical inputs and independent interoperability oracles.
  • Preserve Python, Rust, Java and all five SQL dialects' coverage.
  • Remove obsolete recording/compression helpers and temporary cleanup audit files; retain maintained developer guidance.

No production behavior changes.

Validation: all 78 CI jobs passed on 1ae015546, covering Rust, Python 3.10–3.14, Java 17/21, local database matrices, native packaging and runtime installation. The subsequent commit only removes four temporary documentation files; repository hygiene and reference checks pass. Fresh live warehouse/remote-model testing is not claimed.

Rebased onto v2-candidate after #75 merged (a441e59a7). The cleanup patch is preserved; only its contract-decision baseline was updated. Compatibility validation and all 27 ledger tests pass; fresh CI on the rebased head is pending. Kept in draft for review; not merged.

@coderabbitai

coderabbitai Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

@adi-kumo
adi-kumo marked this pull request as ready for review November 10, 2026 18:01
@adi-kumo
adi-kumo marked this pull request as draft November 10, 2026 18:29
Base automatically changed from release-2-0-gaps to v2-candidate November 10, 2026 19:24
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
@adi-kumo
adi-kumo force-pushed the rust-fixture-cleanup branch from e571ce7 to ac22587 Compare November 10, 2026 19:29
@adi-kumo
adi-kumo marked this pull request as ready for review November 10, 2026 20:30
@adi-kumo adi-kumo self-assigned this Nov 10, 2026
Signed-off-by: Adi Wadaskar <awadaskar@nvidia.com>
@adi-kumo
adi-kumo marked this pull request as draft November 10, 2026 20:42
@adi-kumo
adi-kumo marked this pull request as ready for review November 10, 2026 21:36
@adi-kumo
adi-kumo merged commit 868363f into v2-candidate Nov 10, 2026
158 of 159 checks passed
@adi-kumo
adi-kumo deleted the rust-fixture-cleanup branch November 10, 2026 21:54
Manushpm8 added a commit that referenced this pull request Nov 11, 2026
The rest of the coverage #96 dropped, which #99 merged without. Each of
the six open items from the #96 review was decided with Codex, and each
new assertion was proved by seeding the fault it guards against,
watching it fail, and reverting.

### Decisions
| Item | Decision | What this PR does |
| --- | --- | --- |
| Executed SQL (`l4-goldens` removed) | Restore smaller | Fixed read
budgets per journey: DuckDB (10 describe + 8 materialize, 18 logged) and
a new SQLite journey (24 fetches, 6 refreshes). One whole lookup and one
whole neighbor statement with parameters for DuckDB, SQLite and
PostgreSQL (rendered, no server). Cast, window-bound, decimal and key
projections in the four SQL family tests. No observation-wide pins. |
| Batching, query options, ranking | Restore smaller |
`contract/support/shop_requests.py` derives each call's context,
anchors, labels (window, forecast step, candidate membership), scored
entities and related rows from the case's own inputs; query-options and
ranking cases assert them. A seeded three-call fixture is compared as
three whole requests. |
| Java error messages | Restore | All 23 rows of `BindingTest` assert
the exact message, with real fields for the three parameterized
sentences. |
| PQL | Restore smaller | 11 exact `(code, span)` assertions in
`pql_corpus.rs`; fuzz proportions stay unpinned, plus an explicit
packed-date refusal. |
| Write guard | Accept | Protects only our live harness and fails loudly
there; no change. |
| Time-zone guard | Restore smaller | Also scans the 378 Python-held
DuckDB setups, `PROBES` included; zoned setups need a global pin. |

Also: PostgreSQL keys above `2**63 - 1` are now asserted as exact
strings on the wire, and CHANGELOG says so, since 1.0.0 raised
`OverflowError` there.

### Seeded faults that each new check catches
Re-reading unchanged tables; an uncached time scan; a dropped timestamp
cast, window bound, per-seed limit or wide-key `TRY_CAST`; constant or
swapped labels; a dropped related table; a short batch context; a
ranking part reusing another's labels; Java printing values without
quotes; collapsed PQL spans; a session-only time-zone pin; float64 wide
keys.

### Verified locally
Java `BindingTest` 84/0 on JDK 21; `cargo test` for the touched sdm-core
and sdm-testkit tests; the three contract files on DuckDB, SQLite and
Postgres (620 passed); `test_exact_numeric_keys.py` on Postgres;
`test_contract_cases.py`; fmt, clippy `-D warnings`, ruff and repository
hygiene clean.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Manush Murali <manushm@nvidia.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant