fix(spec-builder): resolve the guard's live state dir per test, not at import - #1546
fix(spec-builder): resolve the guard's live state dir per test, not at import#1546chenmingwei23 wants to merge 1 commit into
Conversation
Design Review (Fable 5) — ✅ PASSAdvisory design-level review of Design-Verdict: PASS Root-cause fix at the right layer: the guard resolves the live dir per test before its own redirect, restoring the #874 invariant without touching production code. [DESIGN-REVIEWED] 4f64e5b |
GPT 5.6 Review — ✅ no blocking findingsGPT 5.6 completed its review of This comment is updated in place on each push. Review detailsNo findings. False positive or not applicable? A repository writer can comment: |
Opus 5 Review — ✅ no blocking findingsReviewed Verdict parsed from the review's SHA-scoped output markers for commit False positive or not applicable? A repository writer can comment: |
|
Superseded by #1543, which landed the same fix from the other direction: For the record, the one behavioural difference and why it does not matter here. #1543 caches the resolved dir in a module global for the process lifetime; this PR resolved it per test inside the fixture, which would additionally honour a later Thanks to whoever picked it up. |
What is the problem?
mainis red.test/test_lazy_data_home_paths.py::TestNoImportTimePathResolution::test_no_module_level_path_constants— the #874 ratchet — fails onBackend Tests (3.10, 2),(3.12, 2)and(Windows) (2), withCoverage Gatecascading (Coverage Combine is skipped when a shard fails) andPR Readinessred on top:The offending line is one module-level capture in the Spec Builder test file, landed by #518 (
af8774b8b):Why this issue matters to the user
Because the red is on
main, every open PR inherits it through its merge commit and cannot reachreadiness: passed— including PRs that touch no Python at all. It was first observed on #1530, a docs-only PR whose branch does not even contain the offending file. Two PRs (#1492, #1509) have since merged through the red, so the queue is now landing changes with a red aggregate gate, which is exactly the state the readiness signal exists to prevent.The ratchet itself is not noise.
_state_dir()reads the data home on every call, so freezing its result at import pins whicheverKIROCREW_HOMEwas active when collection imported the module. That is the #874 class of bug — a pod writes into the real data home instead of its isolated one — and this repository has already paid for it with real state pollution.How our fix solves it
Symptom: three shards plus two aggregate gates red on
main. → Root cause: the guard fixture needs the unpatched state dir, and the author got it by resolving at import, because_redirect_statemonkeypatchesroutes._STATE_DIRbefore each test body runs. Import time is the only moment that is obviously "before the patch" — but it is also the one moment the data home may not be final yet. → Change: resolve it inside the fixture, before it installs the redirect. That is still unpatched, and it is now per-test and live.src/kiro_crew/apps/builtins/spec_builder/tests/test_routes.py:_REAL_STATE_DIR._live_state_snapshot()now takes the directory as a parameter instead of reading a module global._never_touch_the_real_stateresolvesroutes._state_dir()as its first statement — before_redirect_stateruns — and passes that value to the before/after snapshots and the failure message.The guard keeps its original semantics (whole-directory names + mtimes, compared across the test) and gains one property: under a
KIROCREW_HOMEoverride it now watches the sandbox's state dir rather than a path frozen from the ambient home, so it protects the directory the test could actually pollute.No production code changed.
routes.pywas already compliant — its_STATE_DIR/_INDEX_PATH/_DELETED_PATH/_SETTINGS_PATHareNone-defaulted override hooks with per-call accessors; only the test file froze a path.What tests we did
test/test_lazy_data_home_paths.py— 16 passed, including the ratchet that was red. This is the check that unblocksmain.src/kiro_crew/apps/builtins/spec_builder/tests/— 268 passed (whole Spec Builder suite,-n auto --dist loadgroup). Combined run of both: 284 passed.test_state_guard_watches_the_whole_directory, the drift test that pins the guard's shape. It previously asserted on the identifiers this PR removes, so leaving it alone would have made it pass vacuously. It now pins the invariants that matter: the snapshot is taken against the fixture-resolved dir, the dir is resolved viaroutes._state_dir()inside the fixture, the snapshot iterates the whole directory, and no module-level_REAL_STATE_DIR =reappears.isort --check-only,flake8, andmypyon the changed file / its package: clean.Manual verification
Mutation-tested the guard, because a "fix" that quietly disarms a safety net is worse than the red it removes. Appended a throwaway test to
test_routes.pythat writesleak.jsondirectly intoconfig_dir()/workspace/spec-builder(bypassing the redirect), ran it underKIROCREW_HOME=$(mktemp -d), and confirmed the fixture still fails the run:Note the reported path is the sandbox home, which is the behaviour improvement described above. The probe was removed afterwards;
git status --porcelainshows only the one intended file.Two earlier mutation attempts did not trip the guard, and are worth recording because they look like gaps and are not: dropping the
_DELETED_PATHredirect, and dropping the_STATE_DIRredirect. Neither leaks, because_deleted_path()derives from_state_dir()and the remaining explicit*_PATHredirects still point at the tmp dir. Only a write that bypasses every hook reaches the live directory — which is what the final probe does.Any other suggestions on the work
The three-shard fan-out plus
Coverage GateplusPR Readinessmakes one line of test code look like five independent failures, and the--log-failedarchive was empty for these jobs so the real assertion was only reachable by pulling the raw job log. Surfacing the first failing test name in the readiness summary would have cut the triage on this from ~20 minutes to seconds. Filing separately rather than widening this fix.Fixes #1537