Skip to content

Timeouts reach worker threads and async tasks; fix AsyncCallbackPool; revive stale unit tests - #260

Merged
b-macker merged 1 commit into
masterfrom
claude/naab-inadmissible-action-prevention-4cmn1m
Sep 28, 2026
Merged

b-macker merged 1 commit into
masterfrom
claude/naab-inadmissible-action-prevention-4cmn1m

Conversation

@b-macker

Copy link
Copy Markdown
Owner

Summary

This PR covers the remaining open items from #259: timeouts in threads other than the main one, the async-callback defects, and the out-of-date unit tests.

It also adds a regression test for F28 (the REST server exiting on a HARD block, fixed in #223 but never tested). Writing that test surfaced a new REST bug: print() output went to the server's log instead of the response.

The three items listed as "live" in CLAUDE.md, F28, F31 and F32, are all already fixed on master (#223, #218, #241). Their CLAUDE.md entries are corrected here.

Changes

  • Python on worker threads now stops at --timeout.
    • Make --timeout reach Python, http and nested blocks; run unit tests in CI #259's interrupt uses a CPython mechanism that only runs on the main thread. Make --timeout reach Python, http and nested blocks; run unit tests in CI #259 named parallel polyglot groups as the path it missed, but the tree-walker runs Python in those groups on the main thread by design. The real worker path is an async fn.
    • Measured before this PR with --timeout 3: an async fn Python busy loop on the VM ran its full 20 s, and 40 s when the loop catches every exception.
    • The fix, measured at 3 s for both:
      • Threads running Python now register themselves, each run with a sequence number.
      • A worker thread gets a timeout exception injected into it (PyThreadState_SetAsyncExc), re-sent until the run that was active when the deadline fired has left Python.
      • The exception is naab.ExecutionTimeout, which subclasses BaseException like KeyboardInterrupt does. With a normal TimeoutError, the injected exception almost always landed inside the loop's try, so except Exception kept swallowing it.
  • Timeouts are now script-wide or local.
    • Before: every timer, when it fired, set one process-wide "timed out" flag, and every arm or clear reset it. So a per-task timeout on a pool worker would stop the whole script, and a worker merely arming a timer could erase the script's own timeout after it had fired.
    • Now only the outermost timeout of a run (the CLI's, or a REST request's) is script-wide. Nested timeouts, and the new ScopedTimeout(ms, Scope::Local), set only their own thread's flag.
  • Async executors honour their timeout argument. Each pool task runs under a local timeout, so an overrun stops only that task. ShellWithTimeout now stops at 52 ms against a 50 ms budget.
  • AsyncCallbackPool deadlock and use-after-free fixed.
    • std::launch::deferred ran callbacks only when .get() was called, so a full pool deadlocked and executeRace never finished.
    • The lambdas captured this, so the pool could free a wrapper while its callback was still returning, and a timed-out detached thread kept running a member of a freed wrapper.
    • Each wrapper's state now lives in a shared_ptr held by every thread that runs it, and work starts immediately.
  • REST print() fix. The built-in print() wrote straight to std::cout in both engines, bypassing the per-request capture that io.* uses. Clients got empty output, and every request's prints ended up in the server log, mixed across concurrent requests.
  • Unit tests.
    • 14 out-of-date expectations are rewritten to the rule each now encodes, not to whatever the code happens to return: division always returns a double (DIV-001), catch is mandatory, re-registering a struct keeps the first definition (ISS-036), math.abs returns a float, the newline and |> token names, and every listed stdlib module resolves (the old test hard-coded a count of 13).
    • 580 pass. known_failures.txt goes from 30 entries to 9, with no hangs left.
  • Docs: CLAUDE.md (the timeout gotcha, the three corrected statuses, the REST print note) and docs/unit-test-findings.md (new items 2e and 2f, section 3, and the CI section).

Still a limit: Python blocked inside a C call (time.sleep, a socket read) is only interrupted when that call returns. A bare except: can still catch the timeout, the same as with KeyboardInterrupt.

Test Plan

  • tests/security/test_timeout_reach.sh, now 13 checks, all pass.
    • The new W checks cover Python in an async fn, with and without a loop that catches every exception.
    • Against a build from before this change, W-01/vm and W-02/vm fail (20 s and 40 s). Their tree-walk twins pass on both builds, so they are extra coverage, not proof.
  • New tests/api/test_rest_hard_block_survives.sh, 3 checks:
    • R-01 is the control: the request really was HARD-blocked, and the server's own config did not leak.
    • R-02: the server is still running afterwards.
    • R-03: the next request executes and returns its print output.
    • With the old _exit(3) put back temporarily, all three fail.
  • run_unit_tests.sh: 580 pass, and all 9 remaining exclusions still fail for their recorded reasons. The previously hanging or crashing pool tests passed 30 of 30 repeated runs.
  • The full suite, error-message leak check and VM/tree-walker differential run are still running locally; I'll post the results as a comment.

🤖 Generated with Claude Code

https://claude.ai/code/session_01ELUfjXZvx8kzXo1UJjrAhC


Generated by Claude Code

… revive stale unit tests

--timeout on worker threads (async fn). The Python interrupt was a CPython
pending call, which runs on the main thread only. Python inside an async fn
(on another thread, VM) ran 20.8s against --timeout 3, and 40s when the loop
caught every Exception. Threads running Python now register with a sequence
number; a worker is sent PyThreadState_SetAsyncExc, re-sent until the
execution that was running when the deadline fired leaves Python. The
exception is naab.ExecutionTimeout, a BaseException subclass: an async
exception lands at the next eval-breaker check, which inside
`try: <loop> except Exception: pass` is almost always inside the try.
(The recorded limit named the wrong path: tree-walker parallel groups run
Python on the MAIN thread, and a probe on that path passes on old builds.)

Script-wide vs local timeouts. Every timer set the process-wide flag and
every arm/clear reset it, so a per-task timeout on a pool worker would stop
the whole script and a worker arming one could erase the script's timeout
after it fired. Only the outermost timeout of a run is script-wide now;
nested and ScopedTimeout(ms, Local) set only their thread's flag.

Async executors honour `timeout`: each pool task runs under a local
ScopedTimeout (ShellWithTimeout: 52ms against 50ms).

AsyncCallbackPool: std::launch::deferred ran callbacks only on .get(), so a
full pool deadlocked and races never finished; lambdas captured `this`, so the
pool freed wrappers mid-return and a timed-out detached thread kept running a
member of a freed wrapper. State now lives in a shared_ptr held by every
thread, and work starts eagerly. Pool tests: 30/30 repeated runs.

Unit tests: 14 stale expectations rewritten to the rule each encodes
(DIV-001, mandatory catch, ISS-036 first-definition-wins, math.abs float,
newline and |> tokens, module list resolves). 580 pass; exclusions 30 -> 9.

REST: F28 had no regression test; test_rest_hard_block_survives.sh pins it
(reintroducing _exit(3) fails all three arms). Writing it found that builtin
print() wrote to std::cout in both engines, bypassing the per-request capture
stream: clients got empty output and prints landed in the server log.

CLAUDE.md: F28/F31/F32 were still marked "live"; all three were fixed
(#223, #218, #241).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ELUfjXZvx8kzXo1UJjrAhC
@github-actions

Copy link
Copy Markdown

NAAb Governance Report

Metric Count
Files checked 16
Passed 16
Failed 0

All governance checks passed!

Generated by NAAb Governance Engine v4.0

Copy link
Copy Markdown
Owner Author

Local results on 127e1d0 (Linux, Debug build):

  • Full suite: 381 of 447 pass. The 2 unexpected failures are known issues in this container: a stray /govern.json at the filesystem root breaks test_require_governance_gov007.sh and test_platform_fixes.sh. Both pass on CI.
  • The new suites all pass: test_timeout_reach.sh 13/13 and test_rest_hard_block_survives.sh 3/3.
  • Error-message leak check: 874/874 pass.
  • VM vs tree-walker differential: 56/56 match.
  • Unit tests (run_unit_tests.sh): 580 pass, and all 9 remaining exclusions still fail for their recorded reasons.

Generated by Claude Code

@b-macker
b-macker marked this pull request as ready for review September 28, 2026 08:28
@b-macker
b-macker merged commit 9267f48 into master Sep 28, 2026
23 checks passed
@b-macker
b-macker deleted the claude/naab-inadmissible-action-prevention-4cmn1m branch September 28, 2026 08:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants