Reliability hardening: interrupted-run semantics, race-clean CI, sandbox and persistence fixes

[alxxjohn]

Summary

A reliability audit of the codebase (runner/engines, network adapters, integrations, persistence, CLI, and MCP server) surfaced seven high-severity defects. This PR fixes the top six items from the audit, each verified with new regression tests. The full suite now passes go test -race -shuffle=on ./... (423 tests) via cleanr-dev check.

Fixes

1. Interrupted runs reported Passed: true (fix(runner))

A run cut short by SIGINT/SIGTERM or -timeout computed Passed from only the suites that happened to execute — a CI job killed early could exit 0, append a passing entry to trend history, and get a signed release-gate attestation with its security suites never having run.

• Report now records Interrupted and SkippedSuites; Passed is forced false on truncation.
• Engines no longer emit phantom zero-value failed cases when cancelled mid-suite (runBoundedByIndex returns the executed prefix).
• cleanr run exits 130 for interrupted runs (documented in docs/ci.md), skips trend-history persistence, and refuses to sign an attestation. The text report shows Status INTERRUPTED with the skipped suites.

2. Data race + no race detector anywhere (test)

tests/cli/snapshot_test.go swapped the global http.DefaultTransport under t.Parallel(), racing every parallel test doing HTTP — invisible because neither CI, make test, nor cleanr-dev test ever ran -race. The swap test is now sequential, the CLI-adapter helper timeout is raised for race-instrumented startup, and both CI and cleanr-dev test run -race -shuffle=on.

3. WASM plugins could hang the run forever (fix(plugins))

The wazero runtime lacked WithCloseOnContextDone(true), so guest code ignored context deadlines — a plugin with an infinite loop wedged Runner.Run permanently, defeating the sandbox. The entry timeout now interrupts guest execution and is scoped to execution only (module compilation is host-side work). A regression test proves a for {} guest is killed.

4. MCP server crash + missing HTTP timeouts (fix(mcpserver), fix(integrations))

• cleanr_generate_dataset passed a typed-nil *http.Client into the real generation path → nil-deref panic that killed the whole stdio server (no recover() anywhere). It now gets a real client, and tool dispatch is wrapped in recover() so a panicking handler returns a JSON-RPC error while the server keeps answering.
• Generic sink POSTs and trend-source fetches built clients with Timeout: 0 when timeout_ms was unset — one hung endpoint stalled a release-gate run indefinitely. All integration HTTP calls now share one constructor with a 10s default.

5. Credential-egress bypass in PostHog/Langfuse sinks (fix(integrations))

Both sinks send credentials outside applyAuth (request body / Basic auth), skipping the CredentialEgressAllowed policy — a config with project_token_env: OPENAI_API_KEY and an attacker base_url could exfiltrate a provider secret. Both now enforce the same egress policy and fail loudly.

6. Persistence and config-defaults hardening (fix(persistence), fix(config))

• New cleanr/fsatomic package (temp file + fsync + rename + dir sync) replaces two copy-pasted helpers and is adopted by the previously non-atomic writers: cleanr.yaml, the credentials-bearing profile.json, attestations, and replay artifacts. An interrupted write can no longer destroy or corrupt any of them.
• Zero-ambiguous config fields are now pointer-typed with defaulting *Value() accessors (following the MaxFailedCasesDelta precedent): require_review: false is expressible (was silently inverted to true), max_normalized_drift: 0 means zero tolerance (was replaced by 0.3), min_score: 0 survives, and trend-gate presets no longer override an explicit enabled: — exploratory + enabled: true keeps relaxed thresholds with gating active. Since accessors default at read time, SDK-built configs that bypass applyDefaults get correct thresholds too.

Behavior changes to be aware of

• New exit code 130 from cleanr run when interrupted (previously 0 or 1 depending on partial results). Pipelines treating non-zero as failure are unaffected; pipelines matching exit codes exactly should add 130.
• Interrupted runs are no longer written to trend history or attested.
• JSON/YAML reports gain optional interrupted and skipped_suites fields.
• TrendGateConfig.Enabled, ScenarioGenerationConfig.RequireReview, LLMJudgeConfig.MinScore, and six DriftConfig thresholds are now pointer-typed in the Go SDK (read via *Value() accessors). YAML/JSON configs are unaffected.
• Misconfigured PostHog/Langfuse sinks pointing provider secrets at untrusted hosts now fail to publish instead of leaking.

Test plan

• cleanr-dev check (gofiles layout, fmt, vet, go test -race -shuffle=on ./...) — 423 passed, 0 failed
• New regression tests: interrupted-run semantics, hanging-WASM-guest kill, MCP tool-panic containment, sink egress refusal, explicit-zero/false config round-trips, exploratory-preset override
• go vet ./... clean

🤖 Generated with Claude Code