Skip to content

test: pin real config values in slot-create inheritance test (#6522) - #6533

Closed
bolichen97 wants to merge 1 commit into
mainfrom
fix/slot-create-magicmock-fixture-6522
Closed

test: pin real config values in slot-create inheritance test (#6522)#6533
bolichen97 wants to merge 1 commit into
mainfrom
fix/slot-create-magicmock-fixture-6522

Conversation

@bolichen97

Copy link
Copy Markdown
Collaborator

Closes #6522

What this fixes

test/test_dashboard_chat.py::TestFolderCRUD::test_slot_create_inherits_nearest_folder_project fails deterministically on main (since 5c925e4, #6465), redding Backend Tests shards 2+3 on 3.10/3.12/Windows for every open PR. The defect is test-only: the fixture, not the product. This PR touches exactly one file, test/test_dashboard_chat.py (+13 comment/fixture lines).

Root cause — two legs, both artifacts of a bare MagicMock config

The test stubbed KiroCrewConfig.load with a bare MagicMock (only dashboard.default_project pinned). On the agent-less POST /api/chat/slots path:

  • Leg 1 (harmless, logged, NOT the 500): mock_cfg.agents is a truthy auto-created MagicMock whose __iter__ yields nothing. resolve_agent_bindings' elif config.agents: guard passes and next(iter(config.agents)) raises StopIteration — caught by the handler's except Exception at chat_handlers.py:2019-2020 and logged (WARNING chat_handlers.py:2020 in the CI log). The loader's guard is correct; the mock's shape defeats it.
  • Leg 2 (the actual 500): with no agent in the request body the handler runs agent = cfg.default_agent or "", stamping a truthy MagicMock into the slot. The coalesced slots broadcast at suspend_slots_push exit then fails json.dumps(slots_data) (TypeError: Object of type MagicMock is not JSON serializable, state.py:8393); the TypeError escapes __exit__, aiohttp renders a text/plain 500, and the test's resp.json() raises ContentTypeError.

Why the issue's suggested fix is deliberately NOT implemented

#6522 proposes next(iter(config.agents), None) in src/kiro_crew/config/loader.py. That is not the defect: the loader already guards the genuinely-empty case (elif config.agents: / an else: branch returning safe defaults), and the StopIteration it would silence is already swallowed by the handler (Leg 1). Implementing it would leave the test red, because the 500 comes from Leg 2 — a mock-shaped default_agent reaching slot state. The divergence from the issue text is intentional, not an oversight; the corrected diagnosis was posted on the issue by triage.

The fix

Pin the two config attributes this path actually reads to real values, following the file's established narrow-mock convention (e.g. the _cc_cfg site documents its read set the same way):

  • mock_cfg.default_agent = "" — the load-bearing pin: nothing mock-shaped reaches slot state, and an empty agent keeps resolve_agent_bindings out of a test that exercises folder inheritance, not agent resolution.
  • mock_cfg.dashboard.default_project = "" (pre-existing) — belt-and-braces; the handler isinstance-guards that read.

Sibling survey (spec item): every other mock_cfg site in the file either sets a real agents dict and exercises agent-explicit paths, patches resolve_agent_bindings itself, makes load raise (so cfg is None), or patches a different module's load with a documented read set. None reaches the agent-less slot-create path with an unpinned default_agent; all pass today and are untouched per the do-not-mass-rewrite rule.

Verification

  • Named test passes locally on 3.10.20 and 3.12.13.
  • Full backend suite vs clean-main control (same host, control worktree at 5c925e4): control 87 failed / 72,599 passed / 2 errors; branch 86 failed / 72,598 passed / 2 errors. The failure-set diff is exactly this test flipping to pass (the second control-only diff line is that same failure's captured aiohttp error-log output, not a test id). Zero branch-only regressions. Remaining failures are the known host-env set (AF_UNIX path too long, missing gh binary, sandbox home).
  • Lint gates green with repo-pinned tooling: isortflake8mypy (1152 files, no issues) ✅ CI-identical black ratchet ✅.
  • git diff --stat: only test/test_dashboard_chat.py; nothing under src/kiro_crew/.

Pre-push review fleet

  • GPT 5.6 Sol: PASS, zero findings. Independently traced both legs closed; confirmed the loader normalizes non-string default_agent values so the mock shape is fixture-only (no masked product defect); assertions still discriminate (broken nearest-ancestor walking would surface root_project, the test asserts parent_project).
  • Opus 5: PASS, zero blocking. Confirmed the only AUTOSDE rule matching a test-only diff (no-test-side-effects) is moved TOWARD, not violated; verified the handler read-set claim and the sibling survey repo-wide. Two comment-wording advisories (name the load-bearing pin; date the read-set snapshot) — both adopted in this head.

Inherited main-reds (not this PR's)

Main is currently also red on Frontend Lint & Type Check and Coverage Gate (CI run 33162369859 on 5c925e4), unrelated to this change and out of scope here. Please don't attribute those lanes to this diff.

Out-of-scope follow-up filed

Hardening api_chat_slot_create so a broadcast failure during unwind cannot 500 an already-successful create is a real improvement but a failure-semantics design decision — filed separately as #6532 per the fix scope, not folded in.

test_slot_create_inherits_nearest_folder_project stubbed KiroCrewConfig.load
with a bare MagicMock, which broke api_chat_slot_create in two independent
ways on the agent-less path:

- Leg 1 (harmless, logged): mock_cfg.agents is a truthy auto-MagicMock whose
  __iter__ yields nothing, so resolve_agent_bindings' agents guard passes and
  next(iter(...)) raises StopIteration - caught by the handler's except and
  logged as 'Failed to resolve bindings for slot create'. The loader guard is
  correct; the mock's shape defeats it.
- Leg 2 (the actual 500): with no agent in the request body the handler stamps
  agent = cfg.default_agent, a truthy MagicMock, into the slot. The coalesced
  slots broadcast at suspend_slots_push exit then fails json.dumps ('Object of
  type MagicMock is not JSON serializable'); the TypeError escapes __exit__
  and aiohttp renders a text/plain 500, so the test's resp.json() raises
  ContentTypeError.

Fix is test-only: pin both config attributes this path reads to real values
(default_agent='' keeps agent resolution out of a folder-inheritance test)
and document the read set, following the file's narrow-mock convention.

The issue's suggested product fix (next(iter(...), None) in loader.py) is
deliberately NOT implemented: it would only silence the already-swallowed
Leg-1 warning and leave the test red, since the 500 comes from Leg 2.

Closes #6522
@github-actions

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of c65eb8fa40bbb8b5150ff64e432679808c5f9ca9 and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] c65eb8f

False positive or not applicable? A repository writer can comment:
/ai-review override gpt c65eb8fa40bbb8b5150ff64e432679808c5f9ca9: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed c65eb8fa40bbb8b5150ff64e432679808c5f9ca9 — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] c65eb8f

Verdict parsed from the review's SHA-scoped output markers for commit c65eb8fa40bbb8b5150ff64e432679808c5f9ca9.

False positive or not applicable? A repository writer can comment:
/ai-review override fable c65eb8fa40bbb8b5150ff64e432679808c5f9ca9: <one-sentence reason>

@github-actions

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — 🔴 BLOCK (blocking)

Design-level review of c65eb8fa40bbb8b5150ff64e432679808c5f9ca9 — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

Design-Verdict: BLOCK

Superseded PR: the identical fix already landed on main in #6517, so this diff adds only a duplicate pin and a second comment block.

Blockers

  • The problem no longer exists at the merge base. The description's premise — "fails deterministically on main … Backend Tests shards 2+3" — was verified against a control worktree "at 5c925e4", one commit before base e48ea42 (fix: repair three cross-merge breakages redding main's CI #6517), which already added mock_cfg.default_agent = "" with a comment naming the same json.dumps/500 root cause. Merging this PR therefore changes no behavior; the merged file now carries the same assignment twice (lines 10793 and 10800) plus two overlapping rationale comments for one defect, leaving future readers a fixture that looks contradictory and a history that misattributes the fix. Fix: rebase onto current main — the diff collapses to redundant comment lines — then close as superseded (or keep only the longer comment as a doc-only change if it genuinely adds value, which is a different, smaller PR).

[DESIGN-REVIEWED] c65eb8f

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Aug 28, 2026
@bolichen97

Copy link
Copy Markdown
Collaborator Author

🤖 Kiro Crew Auto-Pipeline [operator: bolichen97#bb3ad1ca]

Closing as superseded: the identical fix (mock_cfg.default_agent = "" in this test's fixture, same json.dumps/500 root cause) landed on main 25 minutes after this branch's base fetch, via #6517 (merge e48ea42, merged 12:03:11Z; this PR opened 12:05Z — a cross-operator race, not an oversight). Verified empirically: the named test passes on pristine main at e48ea42, and a test-merge of this diff would only duplicate the pin and its rationale comment, exactly as the Design Review found. The out-of-scope hardening question raised here remains tracked in #6532.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

api_chat_slot_create 500s (bare StopIteration) when config.agents is empty — main CI red since #6465

1 participant