Skip to content

feat: surface a possiblyStuck hint on get_test_job when no test callback fires for 30s+#32

Open
dehuaichendragonplus wants to merge 1 commit into
FunplayAI:mainfrom
dehuaichendragonplus:feat/test-runner-stuck-detection
Open

feat: surface a possiblyStuck hint on get_test_job when no test callback fires for 30s+#32
dehuaichendragonplus wants to merge 1 commit into
FunplayAI:mainfrom
dehuaichendragonplus:feat/test-runner-stuck-detection

Conversation

@dehuaichendragonplus

@dehuaichendragonplus dehuaichendragonplus commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Why this change is needed

Unity's Test Runner only supports one active run at a time engine-wide, and this package has no way to see runs started by other tools/sessions/processes against the same Editor. A concurrent caller (or a stale leftover run) can silently occupy the engine with no exception, leaving run_tests/get_test_job reporting running forever with zero diagnostic signal -- callers have no way to tell a genuinely stuck run apart from one that's simply still executing.

Change approach

  • get_test_job now flags possiblyStuck: true with a stuckHint when a run still reported as running has had no ICallbacks activity (RunStarted/TestStarted/TestFinished) for over 30 seconds.
  • Root-caused via a live investigation in a real consumer project: a GC-of-the-TestRunnerApi-instance hypothesis was tested and rejected (rooting the instance in a static field made no difference to the hang). A concurrent enter_play_mode call from a different actor was directly observed landing mid-run on two separate occasions, and the identical unmodified code/test succeeded immediately (0.62-0.72s) once the Editor returned to exclusive/idle use.
  • Deliberately does NOT reflect into Unity's private, version-fragile internal run-tracking singleton (TestJobDataHolder) to detect contention directly -- that's tightly coupled to Test Framework internals across versions. Instead this tracks a lastActivityAt timestamp already owned by this package, updated by every callback.

Benefits

  • Turns a silent, indefinite running state into an actionable diagnostic signal callers can act on (e.g. retry, or investigate engine contention) instead of polling forever with no information.
  • Avoids fragile coupling to Unity's private Test Framework internals, so the detection keeps working across Unity versions.
  • Tool description and CHANGELOG.md updated so users understand the new field before they hit it.

Test results

Scenario Result
Normal run: single trivial [Test] Completed in 0.62s with resultState: Passed, no false possiblyStuck
Simulated stuck run: manually wrote a running job with a lastActivityAt far in the past directly into SessionState get_test_job correctly returned possiblyStuck: true with the expected hint text

Compatibility testing

  • Purely additive field on the existing get_test_job response shape -- no behavior change for normal, fast-completing runs (verified above).
  • No Unity-version-specific or private-API surface touched, by design (see Change approach).
  • Tested live in a Unity 6000.3.13f1 project with the package embedded; all calls went through the live running server.

Checklist

  • I tested the package in a clean Unity 2022.3+ project (tested live in a Unity 6000.3.13f1 project with the package embedded)
  • I verified Funplay > MCP Server opens and starts correctly (all calls went through the live running server)
  • If I changed setup, update, or config flows, I verified the affected flow end-to-end (N/A)
  • I updated docs for any user-facing behavior changes (tool description + CHANGELOG updated)
  • I did not commit local junk such as .idea/ or .DS_Store
  • I updated CHANGELOG.md when the change affects users

…ack fires for 30s+

Unity's Test Runner only supports one active run at a time, engine-wide, and
this package has no way to detect a run started by another tool/session/process
against the same Editor -- a concurrent caller (or a stale leftover run) can
silently occupy the engine with no exception, leaving run_tests/get_test_job
reporting "running" forever with zero diagnostic signal.

Root-caused via a live investigation in a real consumer project: a
GC-of-the-TestRunnerApi-instance hypothesis was tested and rejected (rooting
the instance in a static field made no difference), while a concurrent
enter_play_mode call from a different actor was directly observed landing
mid-run on two separate occasions, and the identical unmodified code/test
succeeded immediately once the Editor returned to exclusive/idle use.

Rather than reflect into Unity's private, version-fragile internal run-tracking
singleton (TestJobDataHolder) to detect contention directly, this uses a
simpler, more robust signal already owned by this package: track a
lastActivityAt timestamp updated by every ICallbacks method
(RunStarted/TestStarted/TestFinished), and if a "running" job's activity
timestamp is stale by more than 30s, get_test_job now includes
possiblyStuck=true plus a stuckHint explaining the likely cause and pointing
at get_editor_state/get_console_logs for concurrent-activity signs.

Verified: normal runs complete with no false positive (0.62s single test, no
possiblyStuck flag); a simulated stale "running" job with an old
lastActivityAt correctly surfaces possiblyStuck=true with the hint.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant