Skip to content

test(session): remove platform and AsyncMock flakes - #6974

Merged
chenmingwei23 merged 1 commit into
mainfrom
codex/fix-session-test-determinism-windows
Aug 30, 2026
Merged

test(session): remove platform and AsyncMock flakes#6974
chenmingwei23 merged 1 commit into
mainfrom
codex/fix-session-test-determinism-windows

Conversation

@bolichen97

Copy link
Copy Markdown
Collaborator

Problem / Motivation

The session test file had three independent sources of platform or teardown noise:

  • one security test assumed the host could create symlinks, which fails on Windows without SeCreateSymbolicLinkPrivilege;
  • provider factories used bare AsyncMock objects for synchronous methods and for an async-iterator method, leaking unawaited coroutine warnings under strict warning handling;
  • one background-session test started a real SessionManager cleanup task without closing the manager.

These are test-contract defects, not reasons to retry or weaken CI.

What changed

  • Apply the repository's existing requires_symlinks capability probe to the exact symlink test. The probe actually attempts symlink creation; it does not guess from the OS name or catch arbitrary failures.
  • Give the provider doubles synchronous implementations for runtime_info, context_window_tokens, and is_process_alive.
  • Return an empty async generator from stream_command, matching the production async-iterator protocol.
  • Make the local warm-pool and replacement-provider liveness contracts explicit.
  • Close the SessionManager created by test_background_session_name.

The diff is test-only: one file, 17 added lines, one commit.

Determinism

No sleeps, retries, reruns, enlarged timeouts, catch-all skips, or warning filters were added. The symlink skip is capability-based and limited to the one test whose subject requires a symlink.

Validation

  • test/test_session.py under strict RuntimeWarning and PytestUnraisableExceptionWarning handling: 290 passed, 2 skipped;
  • exact symlink node on this Windows host: deterministic skip with the SeCreateSymbolicLinkPrivilege reason;
  • no RuntimeWarning or PytestUnraisableExceptionWarning remains;
  • isort, flake8, diff-check, and the official Black changed-file gate: pass;
  • merge-tree against main at 901ef09: clean.

The only emitted warning is pytest-xdist's own deprecation notice for the repository's looponfail option; this change does not filter it.

Open-PR overlap

All 238 open PRs were scanned. Six also touch test/test_session.py (#6853, #6600, #6595, #5395, #4550, and #3753), but their patches do not modify these marker, mock-contract, or teardown lines. Pairwise merge checks keep this test file conflict-free; their existing conflicts are in README or production session.py.

Checklist

  • One Conventional Commits commit
  • Test-only, deterministic scope
  • Strict warning validation passes
  • No behavior or assertion was relaxed
  • No secrets, credentials, or internal references

@bolichen97
bolichen97 requested a review from a team as a code owner August 30, 2026 09:00
@bolichen97
bolichen97 requested a review from iamwhatever August 30, 2026 09:00
@github-actions github-actions Bot added the readiness: checking Automated validation is still running label Aug 30, 2026
@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Opus 4.8 Review — ✅ no blocking findings

Reviewed b0283669a198b61963a9ab8d1f029509ae9aed6f — this comment is updated in place on each push.

Review details

No findings.

[OPUS-REVIEWED] b028366

Verdict parsed from the review's SHA-scoped output markers for commit b0283669a198b61963a9ab8d1f029509ae9aed6f.

False positive or not applicable? A repository writer can comment:
/ai-review override fable b0283669a198b61963a9ab8d1f029509ae9aed6f: <one-sentence reason>

@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

GPT 5.6 Review — ✅ no blocking findings

GPT 5.6 completed its review of b0283669a198b61963a9ab8d1f029509ae9aed6f and found no blocking issues.

This comment is updated in place on each push.

Review details

No findings.
[GPT-REVIEWED] b028366

False positive or not applicable? A repository writer can comment:
/ai-review override gpt b0283669a198b61963a9ab8d1f029509ae9aed6f: <one-sentence reason>

@github-actions

github-actions Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Design Review (Fable 5) — ✅ PASS

Design-level review of b0283669a198b61963a9ab8d1f029509ae9aed6f — updated in place on each push. A BLOCK verdict blocks PR readiness; PASS/CONCERNS are advisory.

This is a 17-line test-only flake fix. I've verified the claims: requires_symlinks is the repo's established capability probe (test/conftest.py:95, already used across a dozen test files via the same from conftest import pattern), the sync-lambda mock contract matches the existing convention already present in these factories (is_process_alive = lambda: True with its explanatory comment predates this PR), and the close_all() addition fixes the one test that started a real cleanup task without teardown. The fixes target root causes (AsyncMock returning truthy coroutines for sync/iterator methods; missing teardown; a genuine host-capability dependency) rather than papering over symptoms with reruns or filters, which is exactly what the repo's testing conventions demand. The description matches the diff bidirectionally.

Design-Verdict: PASS

Root-cause flake fixes using the repo's existing capability-probe and mock-contract patterns; no design-level concerns.

[DESIGN-REVIEWED] b028366

@github-actions github-actions Bot added readiness: action required A blocking check or review needs attention and removed readiness: checking Automated validation is still running labels Aug 30, 2026
@bolichen97
bolichen97 force-pushed the codex/fix-session-test-determinism-windows branch from a17fa45 to b028366 Compare August 30, 2026 21:01
@github-actions github-actions Bot added readiness: checking Automated validation is still running readiness: passed Eligible automated validation passed for the current revision and removed readiness: action required A blocking check or review needs attention readiness: checking Automated validation is still running labels Aug 30, 2026
@chenmingwei23
chenmingwei23 enabled auto-merge (squash) August 30, 2026 22:04

@chenmingwei23 chenmingwei23 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 file): remove platform and AsyncMock flakes in session tests. Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST (Semgrep + CodeQL) annotations/alerts clean, security checklist all-NO, AI reviewers green.

@chenmingwei23
chenmingwei23 merged commit ac20e30 into main Aug 30, 2026
68 checks passed
@chenmingwei23
chenmingwei23 deleted the codex/fix-session-test-determinism-windows branch August 30, 2026 22:04
@github-actions github-actions Bot removed the readiness: passed Eligible automated validation passed for the current revision label Aug 30, 2026

@iamwhatever iamwhatever left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tier 1 auto-approve: test (1 file). Criteria: no conflict, no requested changes, security path denylist clean, design-doc gate clean, SAST annotations clean, security checklist all-NO, AI reviewers green. Category: test-only flake removal in test/test_session.py -- adds missing mock provider attributes (context_window_tokens, runtime_info, stream_command), an is_process_alive stub, close_all() teardown, and a requires_symlinks marker; no production file touched.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants