test(irq): replace real sleeps with a stepped fake clock (#7598) - #7619
Conversation
test_irq.py measured its 10ms coalescing floor in real wall clock: two assertions that a floor has NOT yet expired had only the gap between two _verdict() calls as budget, so any >=10ms scheduling stall on a loaded CI runner let the floor close and the assertion fail (observed on PR #7300, Backend Tests (3.12, 2)). An autouse fixture now installs a fake clock over the time name as kiro_crew.irq resolves it (never the stdlib module object), _settle() advances that clock instead of sleeping, and the three pre-aged state stamps derive from it. Intervals the kernel measures are now exact by construction: the clock only moves when the test moves it, so no stall can age a window between two calls. Mutation-verified: reverting the #7431 behaviour (joining entry inherits the window's age) makes test_an_entry_joining_after_a_partial_fire_serves_its_own_floor fail under the fake clock, so the converted test still guards the defect. Closes #7598
Design Review (Fable 5) — ✅ PASSDesign-level review of The seam checks out: Design-Verdict: PASS Root-cause fix at the right layer: the test owns the clock the kernel reads, zero production surface, and the guarded behavior is mutation-verified. [DESIGN-REVIEWED] 427d284 |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 4.8 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
iamwhatever
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: test-only change to test/test_irq.py -- replaces real sleep calls in the interrupt-controller tests with a stepped fake clock, so no production file is touched.
dwu96
left a comment
There was a problem hiding this comment.
Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: test-only change to test/test_irq.py — real time.sleep() calls replaced with a module-scoped fake clock monkeypatched over kiro_crew.irq.time, removing CI-timing flakiness; no production file touched.
Problem / Motivation
test/test_irq.py::test_an_entry_joining_after_a_partial_fire_serves_its_own_floorfails intermittently on loaded CI runners (observed on PR #7300,Backend Tests (3.12, 2), as a bareAssertionError: assert False; the rerun of the same commit passed). The file measures its coalescing floor in real wall clock:_COALESCE = 0.01with_settle()sleeping_COALESCE * 3. Two assertions require the opposite of settling -- that a floor has NOT yet expired -- and their entire budget is the gap between two back-to-back_verdict()calls. Any scheduling stall >= 10 ms there (4 xdist workers, ~19.8k items, memory-bounded box) lets the joining entry's own floor close, so it Reports instead of Skips.Closes #7598
Why it matters
The flake reddens whichever
Backend Testsshard carries the test plus the downstreamCoverage Gate, taxing every PR on the repo with rerun round-trips. And the test guards a recently fixed behaviour (#7431: a coalescing entry gets its own floor, not the window's age), so it must stay -- deleting/skipping/flaky-marking it would unguard that fix.What changed (motivation -> approach -> change)
Symptom: assertions about intervals staying SHORT race real scheduling. Root cause: the kernel reads
time.time()while the test controls elapsed time only by sleeping, so the clock keeps moving between the test's own statements. Change: give the test module full ownership of the clock the kernel reads._FakeClock(time()/advance()/reset()) is installed by an autouse fixture over the module attributekiro_crew.irq.time-- the name the kernel's clock reads resolve -- viamonkeypatch.setattr, never the stdlib module object, so the fake is scoped to the kernel and invisible to everything else in the process._settle()now advances the fake clock by_COALESCE * 3instead of sleeping; every existing call site keeps its exact meaning at zero wall cost.opened_at/alerted/coalesce_started_at) derive from the fake clock, and the onetime.sleep(0.05)crossing arealert_secs=0.01window (a kernel-measured interval) becomes a clock advance. No other real-time dependency exists in the file;test_future_timestamp_reads_as_stale_...uses the absolute literal2**40and is unaffected._verdict()calls. That construction -- not a reproduced flake -- is the proof of the fix (the reporter could not reproduce locally and said so; a workstation has more headroom than a loaded CI box)._FakeClock.__getattr__raises a directive error ifkiro_crew.irqever grows a clock read beyondtime.time()(e.g.monotonic), so the failure names this fixture instead of reading as harness breakage.Deliberately NOT done: no injectable-clock parameter in
src/kiro_crew/irq.py. The test-side seam is sufficient, a production seam widens the blast radius, and PR #7584 (open) is concurrently editing that file -- a production change here invites a real conflict. Also not done: raising_COALESCE(only widens the window a stall must beat, keeping the assertion probabilistic while slowing the suite).Relationship to PR #7609: opened concurrently for the same issue by a kiro-agent session under the same login, it threads a
clockcallable throughrun()insrc/kiro_crew/irq.py(+9/-2 production lines overlapping the region #7584 edits) and converts 2 of the 4 real-clock uses in the test file. This PR is the zero-production-surface alternative: noirq.pychange, no #7584 conflict, all 4 real-clock uses converted, and the mutation check below. One of the two should merge; maintainer's call.Tests
All 60 tests in
test/test_irq.pypass, now in ~5s wall (previously each_settle()burned 30 ms of real sleep). Full suite: 78786 passed; the 96 failures + 2 errors are host-environment issues byte-identical to a pristine-main control run at the same base (fail-set diff empty in both directions).Mutation check (proving the converted test still guards #7431): temporarily reverting the joining-entry behaviour in
src/kiro_crew/irq.py-- handing a joining entry the window's oldestopened_atinstead of its own -- makestest_an_entry_joining_after_a_partial_fire_serves_its_own_floorFAIL under the fake clock with exactly "a joining entry must not inherit the window's age"; restoring the file returns it to green. A determinism conversion that still catches the guarded defect is a determinism fix, not a defanged test.Pre-push review: GPT 5.6 Sol PASS (0 findings); Opus 5 PASS (0 blocking, 4 Low advisories -- 3 adopted: stale clock-read-count docstring,
__getattr__directive guard, shared-global invariant comment; 1 was explicitly no-action-needed).Manual verification
N/A -- unit coverage sufficient: the change is confined to one test module and the mutation check exercises the kernel path the tests guard.
Pattern harvest
Class: a test asserting an interval stays SHORT under a real clock is a latent flake on any loaded runner --
_settle()-style helpers make the window-CLOSED direction safe, but nothing protects the window-STILL-OPEN direction. Reusable shape: install a stepped fake clock over the clock name as the module under test resolves it (monkeypatch.setattr("pkg.mod.time", fake)), convert sleeps to advances, and mutation-check that the converted test still fails when the guarded behaviour is reverted. Candidates elsewhere would grep fortime.sleepadjacent to assertions that something has NOT happened; this PR deliberately stops attest_irq.pyper the issue scope.