Skip to content

test: isolate diagnostic logger spy history - #2362

Merged
jatmn merged 1 commit into
mainfrom
test/serial-diagnostic-spy-isolation
Oct 23, 2026
Merged

jatmn merged 1 commit into
mainfrom
test/serial-diagnostic-spy-isolation

Conversation

@jatmn

@jatmn jatmn commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

  • Fixes order-dependent failures in the serial single-process run bun run scripts/run-tests.ts --randomize --seed=<S> --max-concurrency=1 --feature=UNATTENDED_RETRY (also present on main). Each failing test passes alone.
  • Cause: Earlier diagnostic test files leave call history on the shared debug-logger spy. suspicious terminal columns can be flagged again after returning to normal width (src/ink/output.test.ts) and unknown openai-compatible model fallback logs one debug warning and no console errors (src/utils/context.test.ts) then counted calls from earlier tests.
  • Fix: Each victim now clears the spy's call history in its beforeEach (one line per file). The polluting files are not touched.
  • Test-only. Assertions and production code are unchanged, and no tests are skipped.

Impact

  • user-facing impact: none (test-only).
  • developer/maintainer impact: removes one source of order-dependent failures in serial runs.

Testing

  • I ran the required local preflight.
  • exact commands and results (Bun 1.4.4): on this branch, bun run typecheck, bun run lint (0 errors), the touched test files and bun run security:pr-scan -- --base origin/main --head HEAD pass. With this fix and its three sibling fixes applied together on main, bun run build, bun run check, bun run typecheck, bun run typecheck:type-tests and bun run lint pass, and these all have 0 failures: the serial command with --feature=UNATTENDED_RETRY for seeds 4242, 2027 and 101; the default serial command (--randomize --max-concurrency=1) for seeds 101, 4242 and 2027; and the eight-shard randomized runs (--shards=8) for seeds 101, 4242 and 2027.
  • focused tests: polluter followed by each victim: failing before, passing after (13 and 105 pass); with only this fix, seed 4242 passes.
  • documented skipped checks, platform limitations, or verified pre-existing failures: the background session registry ELOCKED race appeared once under heavy load and passed on rerun; it is being fixed separately.

Notes

  • provider/model path tested: n/a (test-only).
  • screenshots attached (if UI changed): n/a.
  • follow-up work or known limitations: this is one of four independent fixes for unrelated root causes in the serial bun run scripts/run-tests.ts --randomize --seed=<S> --max-concurrency=1 --feature=UNATTENDED_RETRY run. Each fixes only its own polluter/victim pair, so a full serial seed is green only once all four are merged: test/serial-model-cache-reset, test/serial-credential-mock-restore, test/serial-ultrathink-restore (sibling branches).

Summary by CodeRabbit

  • Tests
    • Updated logging-related test setup to clear previously recorded calls before assertions, helping ensure warning-count checks evaluate only the behavior under test.

@coderabbitai

coderabbitai Bot commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
📥 Commits

Reviewing files that changed from the base of the PR and between e365805 and ac438a3.

📒 Files selected for processing: 2
  • src/ink/output.test.ts
  • src/utils/context.test.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.

📜 Recent review details
⏰ Context from checks skipped due to timeout: 18
  • GitHub Check: unit-tests (8)
  • GitHub Check: unit-tests (2)
  • GitHub Check: unit-tests (3)
  • GitHub Check: unit-tests (1)
  • GitHub Check: unit-tests (5)
  • GitHub Check: unit-tests (7)
  • GitHub Check: launcher-compatibility (24.11.x)
  • GitHub Check: unit-tests (6)
  • GitHub Check: smoke-and-tests
  • GitHub Check: unit-tests (4)
  • GitHub Check: launcher-node-floor
  • GitHub Check: dedicated-tests
  • GitHub Check: launcher-compatibility (22)
  • GitHub Check: typecheck
  • GitHub Check: CodeQL (actions)
  • GitHub Check: pr-intent-scan
  • GitHub Check: CodeQL (javascript-typescript)
  • GitHub Check: Analyze (javascript-typescript)
🧰 Additional context used
📓 Path-based instructions: 3
Review tests for meaningful coverage of the changed behavior, isolation of global/env/config state, async cleanup, fake timers, provider profile leaks, and Windows-compatible assumptions.

⚙️ CodeRabbit configuration file

Files:

  • src/utils/context.test.ts
  • src/ink/output.test.ts
Apply the OpenClaude maintainer review rubric from AGENTS.md.

⚙️ CodeRabbit configuration file

Files:

  • src/utils/context.test.ts
  • src/ink/output.test.ts
Source excerpt: No hidden policy change: Verify that product, trust-model, routing-default, telemetry/network, and permission-policy changes are not hidden inside unrelated cleanup.

📄 CodeRabbit inference engine (Custom checks)

Files:

  • src/utils/context.test.ts
  • src/ink/output.test.ts
🔇 Additional comments: 2
src/ink/output.test.ts: 1

74-74: LGTM!

src/utils/context.test.ts: 1

578-578: LGTM!

📝 Walkthrough

Two test suites now clear recorded spy calls before exercising the behavior under test.

Changes: Test spy resets

Layer / File(s) Summary
Clear spy history in test setup
src/ink/output.test.ts, src/utils/context.test.ts
The output test setup clears logSpy after configuring its mock. The warning test clears logForDebugging before looking up the context.

Priority: ⬇️ Low

Estimated code review effort: 1 (Trivial) | ~5 minutes

Change: Other

Possibly related PRs

Suggested reviewers: chioarub

Merge Risk: ⚪ Minimal · up to ac438

This test-only change improves test isolation without introducing merge-blocking risk.

Pre-merge checks: Passed 7
Check name Status Explanation
Title check Passed The title "test: isolate diagnostic logger spy history" matches the changeset. Both victim test files now clear the shared debug-logger spy's call history in beforeEach, which isolates the diagnostic …
Description check Passed The description follows the repository template. It has Summary (what changed and why), Impact (user-facing and developer/maintainer impact), Testing (preflight box checked, exact commands and results…
Docstring Coverage Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Risk Surface Disclosed Passed PASS — The PR changes only test spy setup in src/ink/output.test.ts and src/utils/context.test.ts. It does not change auth, provider routing, permissions, outbound network behavior, background exe…
No Hidden Policy Change Passed The PR changes only src/ink/output.test.ts and src/utils/context.test.ts. The two added lines call mockClear() on diagnostic logger spies before assertions. No product code, trust model, routing…
Full details: Title check

Explanation

The title "test: isolate diagnostic logger spy history" matches the changeset. Both victim test files now clear the shared debug-logger spy's call history in beforeEach, which isolates the diagnostic logger spy. It is concise, scoped to the test change, and uses the conventional test: prefix. It meets the title requirements of being concise, scoped, and matching the diff.

Full details: Description check

Explanation

The description follows the repository template. It has Summary (what changed and why), Impact (user-facing and developer/maintainer impact), Testing (preflight box checked, exact commands and results, focused tests, skipped checks and pre-existing failures), and Notes (provider/model path, screenshots, follow-up work). The change is test-only, so the provider/model and screenshot fields are correctly marked n/a. The description is thorough and specific about the root cause, the fix, and the validation performed.

Full details: Docstring Coverage

Explanation

No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 2 files.

Full details: Risk Surface Disclosed

Explanation

PASS — The PR changes only test spy setup in src/ink/output.test.ts and src/utils/context.test.ts. It does not change auth, provider routing, permissions, outbound network behavior, background execution, startup/config-home behavior, skills/plugins/MCP, CI permissions, or release scripts. The risk-surface disclosure requirement is therefore not applicable.

Full details: No Hidden Policy Change

Explanation

The PR changes only src/ink/output.test.ts and src/utils/context.test.ts. The two added lines call mockClear() on diagnostic logger spies before assertions. No product code, trust model, routing defaults, telemetry/network behavior, or permission policy changes are present in the reviewed diff.

✨ Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@jatmn jatmn left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

— jatmn's PR Review agent

🟢 Thanks for the contribution. I do not see any actionable issues from my review.

Merge readiness

GitHub reports mergeable_state: blocked while mergeable: true; required checks on head ac438a31303c3e9cdc929e157fdba0968090bb35 are green (eight unit-tests shards, smoke-and-tests, typecheck, dedicated-tests, launcher jobs, pr-intent-scan, changes, and CodeQL including javascript-typescript). No merge conflict or rebase gap (behind_default_count: 0). This is one of four standalone serial-flake fixes; a full green serial seed on main still needs the sibling branches noted in the PR body.

Local validation (read-only): Bun 1.4.4 installed for CI parity (default worker Bun was 1.3.14). Serial UNATTENDED_RETRY on merge-base e365805: seed 4242 → 10431 pass / 16 fail / 31 skip (failures are launcher compile-cache and settings-notification suites, not the named output/context victims); seeds 2027 and 101 → 10474 pass / 0 fail / 3 skip each. Polluter→victim probe profilerRetention.test.ts then output.test.ts on merge-base showed intermittent extra failures in profilerRetention prompt-submit tests; on head the output victim tests (suspicious terminal columns can be flagged again after returning to normal width, etc.) pass in that chain. CI shards 2 (context.test.ts) and 7 (output.test.ts) — three local runs each, all exit 0, 0 fail; no exit 132.

Automated review: CodeRabbit (ASSERTIVE, commit ac438a3) reported no actionable comments on the two changed files. CodeQL completed successfully with no findings attributed to this diff.

@jatmn
jatmn marked this pull request as ready for review October 23, 2026 21:34
@jatmn
jatmn merged commit 81ef40e into main Oct 23, 2026
22 checks passed
@jatmn
jatmn deleted the test/serial-diagnostic-spy-isolation branch October 23, 2026 21:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant