[ScriptedAlchemy]

Refs #3736

T2-S6 of the Tokio to Asupersync cutover covers the daemon's project-open and serving spawns. This branch is based on master at 2de1d58, after #3981 (T2-S5).

What changed

A spawn moved to NativeTask only when its future reaches no Tokio runtime API. For the trace, each future's callees were compared against the production Tokio inventory: every tokio::spawn, JoinSet, Tokio timer, tokio::net, tokio::fs and Handle site outside test code, plus the seven remaining native_runtime::tokio_handle sites.

Site	Result	What the trace found
full_owner_mount.rs:26 HTTP router mount	Native	mount_http_application_router builds the router synchronously. The daemon client it holds only resolves the socket record. It then inserts the router under a tokio::sync::Mutex, which works on any executor.
project_open_orchestration.rs:198 automation activation	Native	The activation runs two futures. The first, reconcile_project_open_automation_effects, reaches automation-runtime, session-memory and code-index. None of these crates has a Tokio site. The second, activate_automation_scheduler_for_open_project, uses the transition gate, the scheduler retirement (whose reaper S5 made native), the configuration probe and start_automation_scheduler (native since S5). Its timers were already native.
connection_serving.rs:279 profile-identity binding	Native	The binding canonicalizes paths, loads the profile identity and opens the profile runtime through registered_profile_database. store-runtime and session-runtime have no Tokio site.
http_application.rs:922 Remote Brain TLS listener, and remote_http_transport.rs:165 connection set	Native	The listener serves only the remote protocol router. That router's operation routes already run their ports on native spawn_blocking. The enrollment route ran its port inline, so on a native worker its store-writer wait would park the worker. It now uses spawn_blocking as well (tracedecay-api remote.rs). Credential admission reads, the close middleware, hyper and the TLS IO have no Tokio site.
project_open_admission.rs:635 project-open task	Tokio	The open reaches project_server_lifecycle (see below).
invocation_executor.rs:654, 656 in-process invocation	Tokio	Work-attempt invocations reach work/attempt_operations.rs and then spawn_for_worktree, which needs tokio_handle("daemon-service.work_attempt.spawn_for_worktree"). Both timers were already native.
http_application.rs:895 loopback listener, and dashboard-api server.rs:59 connection set	Tokio	The loopback router's /remote-deletions route reaches retire_runtime_work, which spawns on Tokio and calls project_server_lifecycle (see below). serve_loopback runs each handler in its connection task, so its connection set stays with it. The set already cancels and drains gracefully on shutdown.
dashboard-api events_api.rs:647, explorer_api.rs:445	Already native	#3935 and T2-S1 moved them (6b81b0a).

Behavior notes:

• Refusals are typed. When the native runtime refuses a spawn:
  • The full mount fails the open with http application mount task could not start.
  • The activation logs automation_scheduler_activation outcome=runtime_unavailable.
  • The binding returns daemon profile identity binding could not start.
  • TLS bind returns Remote Brain TLS listener task could not start.
  • A refused TLS connection is logged and closed.
• Abort-on-drop owners are kept. HttpMountTask aborts the HTTP mount when the joined pair fails or is dropped, as the JoinSet did. RemoteBrainTlsConnections aborts its connections on drop, and DaemonHttpApplicationService::drop still aborts the listener.
• The TLS connection set is cooperative. Shutdown cancels the graceful token and waits up to shutdown_drain for every connection. It then aborts the rest and waits for them. Finished connections are reaped at each accept.
• The per-chunk TLS deadlines stay on asupersync::time::Sleep. They now run on a native worker's timer driver instead of the fallback timer, and do not use native_runtime::sleep_until.
• bind_authenticated_profile_identity_within_deadline is now pub(super), so the daemon tests can drive it.

Waiting on project_server_lifecycle.rs

This PR does not read or edit project_server_lifecycle.rs. These seams stay on Tokio until that file moves:

• ProjectOpenTasks::start_cancellable (project_open_admission.rs:635). The open calls release_one_idle_project_server_before_open and retire_failed_project_open_owner. Those call project_server_lifecycle::retire_project_servers and retire_evicted_project_owner.
• The daemon HTTP loopback listener (http_application.rs:895) and its serve_loopback connection set (dashboard-api server.rs:59). /remote-deletions reaches remote_recovery_lifecycle::retire_runtime_work, which calls retire_project_servers_now.

The in-process invocation spawns (invocation_executor.rs:654, 656) also stay on Tokio. They wait on the daemon-service work-attempt seam, not on that file.

Tests

Each off-Tokio test runs inside native_runtime::block_on, or after a Tokio setup, and asserts Handle::try_current().is_err(). Each test reads test_support::native_runtime_hazards() before and after the run and asserts that it did not change.

Red-first: each test below was run against master's production code. Each failed:

full mount:          panicked at full_owner_mount.rs:27:24: there is no reactor running, must be called from the context of a Tokio 1.x runtime
activation:          panicked at project_open_orchestration.rs:198:5: there is no reactor running ...
profile binding:     panicked at connection_serving.rs:279:19: there is no reactor running ...
TLS handler:         the remote handler must run off Tokio: HTTP/1.1 200 OK ... tokio
enrollment (api):    assertion `left == right` failed: the enrollment port must run off native runtime threads
                     left: Some(true)  right: Some(false)

These tests pass on this branch:

• full_owner_mount::tests::independent_full_mounts_join_without_a_tokio_runtime.
• tests::scheduler_config::project_open_activation_starts_the_automation_loop_without_a_tokio_runtime. The real activation futures run off Tokio, probe the configuration and register the project's automation loop.
• tests::runtime_identity::profile_identity_binding_opens_a_cold_profile_runtime_without_a_tokio_runtime. A cold profile runtime opens off Tokio, and no writer wait parks a native thread.
• remote_tls::remote_tls_connections_are_served_off_tokio.
• tracedecay-api remote::tests::enrollment_route_runs_its_port_off_native_runtime_threads. The credential fixture now also admits the grant credential.

Abort-on-drop proofs, with mutation checks. The checks emptied HttpMountTask::drop and RemoteBrainTlsConnections::drop, and these tests then failed:

• failed_owner_mount_cancels_spawned_http_mount: owner failure must cancel the spawned HTTP mount: Elapsed(()).
• cancelling_full_mount_cancels_spawned_http_mount: full mount cancellation must cancel the spawned HTTP mount: Elapsed(()).
• dropping_the_remote_tls_service_aborts_an_open_connection (new): dropping the service must abort its open connection: Elapsed(()).

One test was rewritten. scheduler_activation_drain_wins_when_discovery_is_simultaneously_ready signalled from inside discovery's first poll and then started draining. On a native worker, that poll can see the drain before it returns, so discovery won (1 failure in 1 run). The test now starts draining only after discovery returns Pending, so both arms are ready on the next poll, as the test intends. The test passed 5 of 5 runs. For a mutation check, biased; was removed from the activation's select!. The test then failed with draining must win when scheduler discovery becomes ready on the same tick.

Gates

All gates ran on 4bf25ac:

• bazel build --config=clippy passed (Build completed successfully) for each target:
  • //crates/tracedecay/...
  • //crates/tracedecay-daemon-service/...
  • //crates/tracedecay-api/...
• bazel build passed (Build completed successfully) for each format check:
  • //:rustfmt_check
  • //crates/tracedecay:rustfmt_check_2024
  • //crates/tracedecay-daemon-service:rustfmt_check_2024
  • //crates/tracedecay-api:rustfmt_check_2024
• //crates/tracedecay:unit_test: PASSED, 886 passed.
• //crates/tracedecay:daemon_suite: PASSED, 75 passed.
• //crates/tracedecay:transport_acceptance_suite: PASSED, 27 passed.
• //crates/tracedecay:dashboard_api_test: PASSED, 86 passed.
• //crates/tracedecay:mcp_suite: PASSED, 661 passed.
• //crates/tracedecay:session_suite: PASSED, 418 passed.
• //crates/tracedecay-daemon-service:unit_test: PASSED, 360 passed.
• //crates/tracedecay-api:unit_test: PASSED, 60 passed.
• //crates/tracedecay-dashboard-api:unit_test: PASSED, 193 passed and 1 ignored. The crate is unchanged.

---

[changeset-bot]

⚠️ No Changeset found

Latest commit: 4bf25ac

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[devin-ai-integration]

Devin Review found 2 potential issues.

[devin-ai-integration]

🔴 Credential reads block native TLS workers

For credentialed TLS requests, NativeTask::spawn runs admission on native workers, where the credential lookup waits synchronously for SQLite readers. Concurrent delayed reads can park native workers and stall unrelated daemon work.

Learn more

The HTTP connection task polls the router itself. Before reading the body, RemotePreBodyAdmissionV1 calls the production credential authority synchronously. credential_by_fingerprint delegates to a SQLite query; the reader client blocks in receive.recv(). A delayed reader therefore parks the native worker running the connection until the query returns. This affects enrollment and every other authenticated remote route before their operation ports reach the blocking pool.

Example: If 128 admitted TLS peers present valid credentials while the reader pool is stalled, their pre-body lookups wait synchronously on native workers. Native tasks sharing those workers cannot progress until reader replies arrive.

Recommended fix: Run credential admission on a blocking pool, or replace its synchronous SQLite reader call with an async lookup before executing the router on native workers. Cover concurrent delayed credential reads with a native-worker progress test.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

🟡 Failed TLS startup leaves false serving status

When NativeTask::spawn refuses the TLS listener, startup returns an error without clearing the previously published serving state. The remote status authority then reports a listener that never started.

Learn more

bind_remote_brain_tls_server calls publish_listener_serving() immediately after binding. This task-spawn failure returns without reaching the listener task's stopped/degraded publication. The credential authority is shared with the installed remote router and can retain Serving even though the bound socket and service were dropped.

Example: A native runtime that refuses the listener task causes bind_with_remote_tls to return Config, while a retained credential authority still reports Serving.

Recommended fix: On the spawn error path, publish listener_degraded or listener_stopped on the credential authority before returning; consider publishing Serving only after successful task admission.

---

Was this helpful? React with 👍 or 👎 to provide feedback.