Skip to content

PR0: test infra — wire response-side drift through EXCLUSIONS map #158

Description

@TexasCoding

Context

Response-side spec drift (TestSpecDrift::test_additive_drift, test_required_drift, TestWsSpecDrift::test_ws_additive_drift, test_ws_required_drift, test_ws_contract_map_completeness, TestSpecDrift::test_contract_map_completeness) is warn-only. The v2.1.0 Balance.balance_dollars regression — a spec-required field absent entirely from the SDK model — slipped through five rounds of review because additive warnings don't fail CI. Promoting additive drift to a hard-fail is the highest-leverage fix.

This PR is the infrastructure prerequisite for that promotion. It wires response-side drift through the existing EXCLUSIONS mechanism in tests/_contract_support.py (already used by TestRequestParamDrift / TestRequestBodyDrift) without changing the warn → fail behavior yet. Subsequent PRs in the stack (#PR1–#PR4) backfill the real gaps; #PR5 then flips the warning sites to pytest.fail.

Goal: zero behavior change in this PR. The same 60 UserWarnings emit before and after, minus the 7 entries that move into the new allowlist.

Scope

Files

  • tests/_contract_support.py
  • tests/test_contracts.py

Changes

1. tests/_contract_support.py

  • Document in the module docstring that EXCLUSIONS now serves four drift checks: TestRequestParamDrift (query/path), TestRequestBodyDrift (request body properties), TestSpecDrift / TestWsSpecDrift (response model fields).
  • Add the 7 new entries below to EXCLUSIONS. Keep the existing ExclusionKind literal — spec_deprecated and wire_normalization already match the semantics.
# Response-side: spec-deprecated fields the SDK intentionally omits
("kalshi.models.markets.Market", "response_price_units"): Exclusion(
    reason="spec marks DEPRECATED; superseded by price_level_structure / price_ranges",
    kind="spec_deprecated",
),
("kalshi.models.orders.Order", "action"): Exclusion(
    reason="spec marks Deprecated; superseded by outcome_side / book_side",
    kind="spec_deprecated",
),
("kalshi.models.communications.CreateRFQRequest", "target_cost_centi_cents"): Exclusion(
    reason="spec marks DEPRECATED; superseded by target_cost_dollars",
    kind="spec_deprecated",
),
("kalshi.models.markets.Orderbook", "orderbook_fp"): Exclusion(
    reason="SDK reshapes the orderbook_fp nested object into yes/no level lists at the resource layer",
    kind="wire_normalization",
),
("kalshi.models.communications.CreateRFQRequest", "contracts_fp"): Exclusion(
    reason="alternative wire form of `contracts` (int); SDK accepts the int form, server accepts either",
    kind="wire_normalization",
),
("kalshi.models.orders.CreateOrderRequest", "sell_position_floor"): Exclusion(
    reason="already excluded body-side as spec-deprecated; mirror entry for response-side additive check",
    kind="spec_deprecated",
),
("kalshi.ws.models.ticker.TickerPayload", "time"): Exclusion(
    reason="spec marks deprecated; superseded by ts_ms",
    kind="spec_deprecated",
),

2. tests/test_contracts.py

  • In _classify_drift (line ~368), consult EXCLUSIONS.get((entry.sdk_model, spec_field_name)) as a second filter alongside entry.ignored_fields. Existing ignored_fields keeps working for per-entry skips; EXCLUSIONS becomes the canonical place for typed, reasoned skips.
  • Same change in the WS path (test_ws_required_drift body inline at line ~751 — extract to share with _classify_drift or duplicate the lookup; pick whichever produces the smaller diff).
  • Extend test_exclusion_map_is_current (line ~1332) with a third branch for response-side keys: walk CONTRACT_MAP + WS_CONTRACT_MAP for entries whose sdk_model matches the exclusion FQN; resolve spec_schema via the appropriate loader (_get_schema_fields or _get_ws_msg_fields) and assert name is still a property on it. Stale entries must surface here exactly like body-side stale entries do today.
  • Do not change warnings.warn(...) to pytest.fail(...) yet — that's PR5's job.

Acceptance criteria

  • uv run pytest tests/test_contracts.py -v exits 0, same as today.
  • uv run pytest tests/test_contracts.py -W error::UserWarning -v shows exactly 53 failures, not 60 (60 baseline minus the 7 newly-allowlisted entries). Confirms the EXCLUSIONS map is being consulted by the response-side checks.
  • test_exclusion_map_is_current passes with all 7 new entries; flipping any one entry's spec field name to a non-existent name in a local edit causes it to fail (manual smoke test — do not commit).
  • uv run mypy kalshi/ and uv run mypy tests/_contract_support.py tests/test_contracts.py clean.
  • uv run ruff check . clean.
  • No production code (kalshi/**) changes.

Out of scope

  • Backfilling any SDK model field (#PR1–#PR4).
  • Flipping warn → fail (#PR5).
  • Required-but-Optional drift policy (kept as warn-only; ~204 entries, separate decision).

Dependencies

None. This is the load-bearing change everyone else builds on.

Effort

~80 lines of test infrastructure. Mechanical.

Part of the response-side spec drift hardening stack.


Tracking: #157.
Blocks: PR1 / PR2 / PR3 / PR4 / PR5 (the per-model backfill stack).

Metadata

Metadata

Assignees

No one assigned

    Labels

    infraInfrastructure/toolingspec-driftUpstream OpenAPI/AsyncAPI spec changed since last synctestingTest coverage

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions