Bound the post-handshake snapshot in IBBroker.connect() - #87
Merged
Conversation
`connect()` wrapped `connectAsync` in `asyncio.wait_for` and then awaited `_sync_positions()` and `_sync_orders()` with no bound at all. A Gateway can complete the API handshake and then stop answering requests, and `reqPositionsAsync()` waits on a reply that never arrives: no error, no log line, no timeout, no return. That is not hypothetical. A notebook render sat 75 minutes in this call using 7.68 seconds of CPU before it was killed, holding its connection and a scheduling slot the whole time, and a six-hour paper soak in this repo failed the same day at `controlled_reconnect` with `API connection failed: TimeoutError()` against the same Gateway. A probe on an unused client id reproduced it in 15 seconds: TCP connects, the handshake never completes. `_sync_orders()` is not actually a network wait - `openTrades()` reads cached state - but it is bounded too, so the guarantee is a property of the call and not of today's implementation of one method. The three timeouts are now named rather than written as literals at the call site. The handshake pair keeps the values it has always used, 15 and 20; the snapshot bound at 30 is the new one. Failing there reuses the existing handler, which removes the callbacks, disconnects, and records the snapshot as unavailable, so a timeout leaves the broker in the same state as any other connect failure rather than half-subscribed. The test fails without the fix and passes with it, which is the only reason to believe it: with the bound removed and the constants kept, the hang case runs into the test's own 5-second safety net and reports 5.01s; with the bound in place it ends at the snapshot timeout. The companion test covers a healthy connect so the pair can fail in both directions. Full unit suite: 1002 passed. No API surface changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AmbqG2FqRWdr6m8UEM7E6g
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
IBBroker.connect()bounded its handshake and then awaited the position and open-ordersnapshot with no timeout at all. A Gateway can complete the API handshake and then stop
answering requests;
reqPositionsAsync()waits on a reply that never arrives, and thecaller gets no error, no log line and no return.
How it showed up
Two independent failures on 2026-09-11, against the same Gateway:
was killed, holding its connection and a scheduling slot throughout. Its socket to the
Gateway was
ESTABwith bytes queued unread.paper ib soak failed during controlled_reconnect: RuntimeError, preceded byAPI connection failed: TimeoutError(). Every other qualification stage passed. Theprevious night's run had failed at 3h36m as well, so this is at least the second instance.
A probe on an unused client id reproduced it in 15 seconds: TCP connects, the API handshake
never completes. Established connections kept working, which is why the soak survived three
hours and then died the moment it deliberately reconnected.
The change
_sync_positions()and_sync_orders()are wrapped inasyncio.wait_for, and the threetimeouts are named instead of appearing as literals at the call site. The handshake pair
keeps the values it has always used (15 and 20); the 30-second snapshot bound is new.
_sync_orders()is not actually a network wait today -openTrades()reads cached state -but it is bounded too, so the guarantee belongs to
connect()rather than to the currentimplementation of one method.
A timeout there falls into the handler that already exists, which removes the callbacks,
disconnects, and records the snapshot as unavailable. So a hung Gateway now leaves the
broker in the same state as any other connect failure, rather than half-subscribed.
Verification
The test fails without the fix and passes with it, which is the only reason to believe it.
With the bound removed and the constants kept, the hang case runs into the test's own
5-second safety net and reports
connect() took 5.01s; with the bound in place it ends atthe snapshot timeout. A companion test covers a healthy connect, so the pair can fail in
both directions rather than only confirming the unhappy path.
Full unit suite: 1002 passed.
No API surface changes, so this is a patch release and nothing downstream has to move with
it. Readers running chapter 25 against a sick Gateway are the ones who benefit: today they
hang indefinitely with no message.
🤖 Generated with Claude Code
https://claude.ai/code/session_01AmbqG2FqRWdr6m8UEM7E6g