Skip to content

Nightly spec-drift: strict contract tests failing (since 2026-07-16) #475

Open
Open
Nightly spec-drift: strict contract tests failing (since 2026-07-16)#475
Labels
spec-driftUpstream OpenAPI/AsyncAPI spec changed since last sync

Description

@github-actions
github-actions
bot
opened on Jul 16, 2026

Nightly strict contract tests failed

The scheduled Spec Drift Detection run found the SDK contract suite no longer matches upstream OpenAPI/AsyncAPI.

To resolve: locally run uv run python scripts/sync_spec.py + uv run python scripts/generate.py, reconcile models/maps, and open a PR with Closes #<this issue>. This tracker auto-closes on the next green scheduled run.

To silence while a reconcile is in flight, add the spec-drift-ack label — nightly runs then stop commenting until the failing set changes.

Failing test output (last 200 lines)

tests/test_contracts.py::test_nested_body_drift_detects_phantom_field_on_nested_model PASSED [ 78%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[status] PASSED [ 78%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[enabled] PASSED [ 78%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[risk_parameters] PASSED [ 78%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[list0] PASSED [ 78%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[get0] PASSED [ 78%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[orderbook] PASSED [ 79%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[candlesticks] PASSED [ 79%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[list1] PASSED [ 79%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[list_all] PASSED [ 79%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[get1] PASSED [ 79%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[cancel] PASSED [ 79%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[list_fcm] PASSED [ 80%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[list_all_fcm] PASSED [ 80%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[list2] PASSED [ 80%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[get2] PASSED [ 80%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[delete] PASSED [ 80%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[positions] PASSED [ 80%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[fills] PASSED [ 80%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[fills_all] PASSED [ 81%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[trades] PASSED [ 81%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[trades_all] PASSED [ 81%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[balance] PASSED [ 81%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[risk] PASSED [ 81%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[notional_risk_limit] PASSED [ 81%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[fee_tiers] PASSED [ 81%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[api_limits] PASSED [ 82%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[rate_estimate] PASSED [ 82%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[historical_rates] PASSED [ 82%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_sync_params_match_spec[history] PASSED [ 82%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[status] PASSED [ 82%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[enabled] PASSED [ 82%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[risk_parameters] PASSED [ 83%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[list0] PASSED [ 83%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[get0] PASSED [ 83%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[orderbook] PASSED [ 83%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[candlesticks] PASSED [ 83%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[list1] PASSED [ 83%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[list_all] PASSED [ 83%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[get1] PASSED [ 84%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[cancel] PASSED [ 84%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[list_fcm] PASSED [ 84%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[list_all_fcm] PASSED [ 84%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[list2] PASSED [ 84%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[get2] PASSED [ 84%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[delete] PASSED [ 84%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[positions] PASSED [ 85%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[fills] PASSED [ 85%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[fills_all] PASSED [ 85%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[trades] PASSED [ 85%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[trades_all] PASSED [ 85%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[balance] PASSED [ 85%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[risk] PASSED [ 86%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[notional_risk_limit] PASSED [ 86%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[fee_tiers] PASSED [ 86%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[api_limits] PASSED [ 86%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[rate_estimate] PASSED [ 86%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[historical_rates] PASSED [ 86%]
tests/test_contracts.py::TestPerpsRequestParamDrift::test_async_params_match_spec[history] PASSED [ 86%]
tests/test_contracts.py::TestPerpsRequestBodyDrift::test_body_properties_match_spec[create0] PASSED [ 87%]
tests/test_contracts.py::TestPerpsRequestBodyDrift::test_body_properties_match_spec[decrease] PASSED [ 87%]
tests/test_contracts.py::TestPerpsRequestBodyDrift::test_body_properties_match_spec[amend] PASSED [ 87%]
tests/test_contracts.py::TestPerpsRequestBodyDrift::test_body_properties_match_spec[create1] PASSED [ 87%]
tests/test_contracts.py::TestPerpsRequestBodyDrift::test_body_properties_match_spec[update_limit] PASSED [ 87%]
tests/test_contracts.py::TestPerpsRequestBodyDrift::test_body_properties_match_spec[transfer_instance] PASSED [ 87%]
tests/test_contracts.py::TestPerpsRequestBodyDrift::test_body_properties_match_spec[transfer_subaccount] PASSED [ 87%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[ExchangeStatus] PASSED [ 88%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginEnabledResponse] PASSED [ 88%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginRiskParametersResponse] PASSED [ 88%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginMarket] FAILED [ 88%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginMarketCandlestick] PASSED [ 88%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[BidAskDistributionHistorical] PASSED [ 88%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[PriceDistributionHistorical] PASSED [ 89%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginMarketsResponse] PASSED [ 89%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginMarketCandlesticksResponse] PASSED [ 89%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[CreateMarginOrderResponse] PASSED [ 89%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginOrder] PASSED [ 89%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginOrderResponse] PASSED [ 89%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginOrdersResponse] PASSED [ 89%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[CancelMarginOrderResponse] PASSED [ 90%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[DecreaseMarginOrderResponse] PASSED [ 90%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[AmendMarginOrderResponse] PASSED [ 90%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[OrderGroup] PASSED [ 90%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetOrderGroupResponse] PASSED [ 90%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[CreateOrderGroupResponse] PASSED [ 90%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginPosition] PASSED [ 90%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginPositionsResponse] PASSED [ 91%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginFill] PASSED [ 91%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginTrade] PASSED [ 91%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginSubaccountBalance] PASSED [ 91%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginBalanceResponse] PASSED [ 91%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginRiskPosition] PASSED [ 91%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginRiskResponse] PASSED [ 92%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[NotionalRiskLimitResponse] PASSED [ 92%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginFeeTiersResponse] PASSED [ 92%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[AccountApiLimits] PASSED [ 92%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginFundingRate] PASSED [ 92%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginFundingHistoryEntry] PASSED [ 92%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginFundingRateEstimate] PASSED [ 92%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginHistoricalFundingRatesResponse] PASSED [ 93%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[GetMarginFundingHistoryResponse] PASSED [ 93%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[IntraExchangeInstanceTransferResponse] PASSED [ 93%]
tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[CreateSubaccountResponse] PASSED [ 93%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[margin_reports] PASSED [ 93%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[active_obligation] PASSED [ 93%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[active_obligations] PASSED [ 93%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[obligation_history] PASSED [ 94%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[obligation_history_all] PASSED [ 94%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[settlement_estimate] PASSED [ 94%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[settlement_estimate_by_asset_class] PASSED [ 94%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[settlement_balance] PASSED [ 94%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[guaranty_fund_balance] PASSED [ 94%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[settlement_balance_history] PASSED [ 95%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[settlement_balance_history_all] PASSED [ 95%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_sync_params_match_spec[settlement_balance_withdrawal] PASSED [ 95%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[margin_reports] PASSED [ 95%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[active_obligation] PASSED [ 95%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[active_obligations] PASSED [ 95%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[obligation_history] PASSED [ 95%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[obligation_history_all] PASSED [ 96%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[settlement_estimate] PASSED [ 96%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[settlement_estimate_by_asset_class] PASSED [ 96%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[settlement_balance] PASSED [ 96%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[guaranty_fund_balance] PASSED [ 96%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[settlement_balance_history] PASSED [ 96%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[settlement_balance_history_all] PASSED [ 96%]
tests/test_contracts.py::TestPerpsScmRequestParamDrift::test_async_params_match_spec[settlement_balance_withdrawal] PASSED [ 97%]
tests/test_contracts.py::TestPerpsScmRequestBodyDrift::test_body_properties_match_spec[withdraw_settlement_balance] PASSED [ 97%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[MarginReport] PASSED [ 97%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetMarginReportsResponse] PASSED [ 97%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[ObligationReceiveInfo] PASSED [ 97%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[SettlementDetail] PASSED [ 97%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[MaintenanceMarginDetail] PASSED [ 98%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[ObligationEntry] PASSED [ 98%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetActiveMarginObligationResponse] PASSED [ 98%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetActiveMarginObligationsResponse] PASSED [ 98%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[MarketSettlementEstimate] PASSED [ 98%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[SettlementEstimate] PASSED [ 98%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetSettlementEstimateResponse] PASSED [ 98%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[AssetClassSettlementEstimate] PASSED [ 99%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetSettlementEstimateByAssetClassResponse] PASSED [ 99%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetSettlementBalanceResponse] PASSED [ 99%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetGuarantyFundBalanceResponse] PASSED [ 99%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[WithdrawSettlementBalanceResponse] PASSED [ 99%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetSettlementBalanceWithdrawalResponse] PASSED [ 99%]
tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[SettlementBalanceHistoryEntry] PASSED [100%]

=================================== FAILURES ===================================
_____________ TestPerpsSpecDrift.test_response_drift[MarginMarket] _____________

self = <tests.test_contracts.TestPerpsSpecDrift object at 0x7f134ea5aea0>
entry = ContractEntry(sdk_model='kalshi.perps.models.markets.MarginMarket', spec_schema='MarginMarket', ignored_fields=frozenset(), notes='')

    def test_response_drift(self, entry: ContractEntry | None) -> None:
        if entry is None:
            pytest.skip("no perps REST response models registered yet")
>       _assert_perps_response_drift(entry, self.spec)

tests/test_contracts.py:2159: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

entry = ContractEntry(sdk_model='kalshi.perps.models.markets.MarginMarket', spec_schema='MarginMarket', ignored_fields=frozenset(), notes='')
spec = {'openapi': '3.0.0', 'info': {'title': 'Kalshi Trade API Manual Endpoints', 'version': '0.0.1', 'description': 'Manual...y': 'Create Order', 'description': 'Endpoint for submitting orders in a market.', 'tags': ['orders'], ...}}, ...}, ...}

    def _assert_perps_response_drift(entry: ContractEntry, spec: dict[str, Any]) -> None:
        """Response-side drift assertion for one perps contract entry."""
        spec_fields = _get_schema_fields(spec, entry.spec_schema)
        model_class = _get_sdk_model_class(entry.sdk_model)
        additive_required, additive_optional, required_issues = _classify_drift(
            entry, spec, spec_fields, model_class, exclusions=PERPS_EXCLUSIONS
        )
        _warn_additive_optional(entry.sdk_model, additive_optional)
        problems = additive_required + required_issues
        if problems:
>           pytest.fail(
                f"Perps spec drift in {entry.sdk_model}:\n"
                + "\n".join(f"  - {p}" for p in problems)
            )
E           Failed: Perps spec drift in kalshi.perps.models.markets.MarginMarket:
E             - Spec field 'schedule' has no SDK mapping

tests/test_contracts.py:2087: Failed
=============================== warnings summary ===============================
tests/test_contracts.py::TestAdditiveDriftClassification::test_optional_drift_survives_the_nightly_strict_gate
  /home/runner/work/kalshi-python-sdk/kalshi-python-sdk/tests/test_contracts.py:988: AdditiveOptionalDriftWarning: Additive-optional drift in tests.synthetic.Fake: Spec field 'opt_new' has no SDK mapping
    _warn_additive_optional("tests.synthetic.Fake", msgs)

tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[GetSettlementEstimateResponse]
  /home/runner/work/kalshi-python-sdk/kalshi-python-sdk/tests/test_contracts.py:2084: AdditiveOptionalDriftWarning: Additive-optional drift in kalshi.perps.klear.models.margin.GetSettlementEstimateResponse: Spec field 'omitted_subtrader_count' has no SDK mapping
    _warn_additive_optional(entry.sdk_model, additive_optional)

tests/test_contracts.py::TestPerpsScmSpecDrift::test_response_drift[AssetClassSettlementEstimate]
  /home/runner/work/kalshi-python-sdk/kalshi-python-sdk/tests/test_contracts.py:2084: AdditiveOptionalDriftWarning: Additive-optional drift in kalshi.perps.klear.models.margin.AssetClassSettlementEstimate: Spec field 'omitted_subtrader_count' has no SDK mapping
    _warn_additive_optional(entry.sdk_model, additive_optional)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
FAILED tests/test_contracts.py::TestPerpsSpecDrift::test_response_drift[MarginMarket] - Failed: Perps spec drift in kalshi.perps.models.markets.MarginMarket:
  - Spec field 'schedule' has no SDK mapping
============ 1 failed, 664 passed, 3 warnings in 240.16s (0:04:00) =============

Metadata

Metadata

Assignees

No one assigned

    Labels

    spec-driftUpstream OpenAPI/AsyncAPI spec changed since last sync

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions