Skip to content

test(integration): poll for query-exchange visibility after POST (orders, order_groups)#129

Merged
TexasCoding merged 2 commits into
mainfrom
fix/integration-not-found-cluster
May 17, 2026
Merged

test(integration): poll for query-exchange visibility after POST (orders, order_groups)#129
TexasCoding merged 2 commits into
mainfrom
fix/integration-not-found-cluster

Conversation

@TexasCoding

Copy link
Copy Markdown
Owner

Summary

5 integration failures in test_orders.py (× 3) and test_order_groups.py (× 2), plus 1 flapping TestOrderGroupsAsync.test_create_get_delete, all surfaced the same error:

KalshiNotFoundError: {'code': 'not_found', 'message': 'not found', 'service': 'query-exchange'}

Root cause

Demo's query-exchange replica lags writes by ~10 seconds. POST returns immediately with a valid order_id / order_group_id, but the matching GET /portfolio/orders/{id} / GET /portfolio/order_groups/{id} 404s until the replica catches up.

Probe evidence (orders):

  • createorder_id X
  • poll 1–3 (~1.5s): 404 not_found
  • ...
  • poll @ ~10s: GET works, status=resting

Same shape for order_groups (~9.5s).

Fix

Added two test helpers in tests/integration/helpers.py:

def wait_for_resource(fetch, *, timeout=15.0, interval=0.5)
async def await_resource(fetch, *, timeout=15.0, interval=0.5)

Both poll the provided fetch function, swallow KalshiNotFoundError until the timeout, and re-raise if the resource never propagates. Drop-in wrap of the GET-after-POST call sites — no SDK changes (transport retries on 404 would mask real "user typo'd the order id" bugs in user code).

Verification

Progress on #124

Closes 5 of 11. Running tally with this PR landed:

Refs #124

Test plan

  • Targeted local run: tests/integration/test_orders.py + test_order_groups.py 22/22
  • ruff check tests/integration/ clean
  • CI green

Demo's query-exchange replica lags writes by ~10 seconds. Orders and
order_groups created via POST aren't immediately readable via their
GET-by-id endpoints — the failing tests asserted GET success
immediately after CREATE and reliably hit 404 ("not_found" /
service: query-exchange).

Probe evidence (orders):
  create -> order_id X
  poll 1-3 (~1.5s): 404 not_found
  ...
  poll @ 10s: GET works, status=resting

Same pattern for order_groups (~9.5s).

- Added wait_for_resource (sync) and await_resource (async) helpers in
  tests/integration/helpers.py — bounded poll loop that swallows
  KalshiNotFoundError until timeout (default 15s, interval 0.5s) and
  re-raises if the resource never propagates.
- Wrapped GET-after-POST sites in test_orders.py
  (test_order_fill_lifecycle, test_create_get_cancel sync + async) and
  test_order_groups.py (test_create_and_get, test_update_limit,
  TestOrderGroupsAsync.test_create_get_delete).
- Dropped the now-redundant `await asyncio.sleep(0.5)` workaround +
  unused asyncio import from test_order_groups.py.

Verified locally: tests/integration/test_orders.py +
tests/integration/test_order_groups.py — 22/22 passed.

Closes 5 of 11 integration-test failures (#124).
Running tally: A (#126=4) + B (#127=1) + E (#128=1) + C (this=5) = 11/11.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented May 17, 2026

Copy link
Copy Markdown

Code Review — PR #129: poll for query-exchange visibility after POST

Overview

Solid, well-motivated fix. The root-cause explanation in the PR description is clear, the helpers are minimal and purposeful, and the call-sites wrap only what they need without touching unrelated code. The decision to keep this in test helpers (rather than adding 404-retry logic to the SDK transport) is the right call — the PR description correctly identifies that transport-level retry would mask real user bugs.


Issues

last_error in wait_for_resource is dead code

last_error: KalshiNotFoundError | None = None
while True:
    try:
        return fetch()
    except KalshiNotFoundError as exc:
        last_error = exc          # ← assigned…
        if time.monotonic() >= deadline:
            raise                 # ← …but raise re-raises `exc`, never reads last_error
        time.sleep(interval)

last_error is assigned on every swallowed exception but never read — the bare raise re-raises the active exc directly. This is ruff F841 (assigned-but-never-used). The async counterpart await_resource got this right by omitting it entirely.

Fix: drop the last_error declaration and assignment to match await_resource:

def wait_for_resource(
    fetch: Callable[[], T],
    *,
    timeout: float = 15.0,
    interval: float = 0.5,
) -> T:
    deadline = time.monotonic() + timeout
    while True:
        try:
            return fetch()
        except KalshiNotFoundError:
            if time.monotonic() >= deadline:
                raise
            time.sleep(interval)

Nits (non-blocking)

  • Docstring on await_resource just says "Async counterpart of wait_for_resource." Copying the one-liner about demo lag would make it self-contained when reading the file top-to-bottom.
  • import asyncio removal in test_order_groups.py is correct — it was only needed for the asyncio.sleep(0.5) that this PR replaces. Good cleanup.

What's Good

  • Helpers are typed correctly (Callable[[], T] / Callable[[], Awaitable[T]]) — mypy should be happy.
  • time.monotonic() for the deadline is the right choice over time.time().
  • Default 15s / 0.5s parameters match the observed ~10s lag with comfortable headroom.
  • All lambda captures are stable (no loop-variable issues).
  • The design decision to NOT retry in the SDK transport is explicitly reasoned in the PR description — that's exactly the right rationale to preserve.

Summary

One real issue (dead last_error variable that may fail ruff), one cosmetic nit. Everything else looks good.

Per bot review on PR #129:

- Removed unused last_error assignment in wait_for_resource (ruff F841;
  caused CI lint failure). bare 'raise' re-raises the active exc
  directly, so the captured ref was dead.
- Switched both helpers to PEP 695 type parameter syntax
  (def wait_for_resource[T](...)) to clear UP047. Drops the standalone
  TypeVar import.
- Expanded await_resource one-line docstring with the demo-lag context
  so the file reads top-to-bottom.

Verified: uv run ruff check . clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@TexasCoding

Copy link
Copy Markdown
Owner Author

Bot feedback addressed in latest commit:

  • 🔴 F841 dead `last_error` — dropped. The bare `raise` already re-raises the active exception; the captured ref was never read.
  • Bonus cleanup: switched both helpers to PEP 695 type-parameter syntax (`def wait_for_resourceT`) to clear the UP047 warning ruff flagged after the F841 went away. Removed the standalone `TypeVar` import.
  • 🟦 `await_resource` docstring — expanded with the demo-lag context so the file is self-documenting top-to-bottom.

Local ruff: All checks passed!

(Pre-existing conftest.py:268 mypy warning on KalshiAuth | None is unrelated to this PR — surfaces only when scanning tests/integration/; CI's mypy step targets kalshi/ per CLAUDE.md, so it won't block.)

@claude

claude Bot commented May 17, 2026

Copy link
Copy Markdown

Code Review — PR #129: Poll for query-exchange visibility after POST

Overall this is a clean, well-reasoned fix. The root cause is clearly documented, the implementation is minimal, and the PR description includes probe evidence that makes the ~10s lag concrete. A few observations below.


What the PR does

Adds two polling helpers (wait_for_resource / await_resource) in tests/integration/helpers.py that swallow KalshiNotFoundError and retry until a deadline, then wires them in at every GET-after-POST call site in test_orders.py and test_order_groups.py.


Positives

  • Correct diagnosis. Replacing the one-shot asyncio.sleep(0.5) with a bounded poll loop is the right fix — a fixed sleep is fragile in both directions (too short under load, wastes time when the replica is fast).
  • Correct use of time.monotonic() for deadline tracking — immune to wall-clock adjustments.
  • PEP 695 generic syntax (def wait_for_resource[T]) is valid; requires-python = ">=3.12" in pyproject.toml confirms this is safe.
  • Sync/async split is cleantime.sleep in the sync variant and asyncio.sleep in the async variant; no risk of blocking the event loop from the wrong helper.
  • Error re-raise preserves the original exception, so test failure messages stay informative.
  • The decision not to put this in the transport layer is explicitly called out and correct — retrying 404 at the SDK level would mask genuine user errors.

Minor issues / suggestions

1. Sequential polling in test_order_fill_lifecycle is safe but worth a comment

buy_order = wait_for_resource(lambda: sync_client.orders.get(buy_id))
sell_order = wait_for_resource(lambda: sync_client.orders.get(sell_id))

Both orders were created simultaneously, so they should become visible at roughly the same time. In practice the second call will return almost instantly after the first succeeds. Worst-case wall time is 2 × 15s = 30s, but in practice it's ~10s + ~0s. A brief comment here would make the intent clearer, similar to the comments added in the other test methods.

2. Timeout slightly tighter than necessary

timeout=15.0 is fine for a ~10s lag. One minor nit: if the demo server is temporarily slower (say 14s), the test will still fail. No action required — 15s is a reasonable margin — just flagging it in case CI shows intermittent failures down the road. If that happens, bumping to timeout=20.0 would be the fix.

3. Callable[[], Awaitable[T]] doesn't enforce coroutine-ness

fetch: Callable[[], Awaitable[T]]

This accepts any zero-arg callable returning an awaitable, which is correct for the call sites (all lambda: async_client.something.get(...)). Nothing actionable — just noting that Callable[[], Coroutine[Any, Any, T]] is more precise but not meaningfully better here.

4. import asyncio removal in test_order_groups.py

The diff correctly drops import asyncio since asyncio.sleep is no longer called directly. Clean.


Potential concerns

None blocking. The implementation is straightforward and the PR description's verification table (22/22 targeted tests passing, 5/11 tracked failures closed) gives good confidence.


Summary

Approve. The fix is correct, minimal, and well-documented. The two micro-suggestions above (sequential-poll comment, timeout margin note) are cosmetic and don't need to block merge.

@TexasCoding
TexasCoding merged commit 643b6c5 into main May 17, 2026
4 checks passed
@TexasCoding
TexasCoding deleted the fix/integration-not-found-cluster branch May 17, 2026 14:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant