diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 08d55c4e0..3bcc4d162 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -22,20 +22,21 @@ Core orchestrates protocol-defined operations. Sandbox providers, Runtime implementations, harnesses and model providers are replaceable components. User-owned machines, E2B, Docker and other environments must expose the same execution protocol. Resource management selects machines, allocates capacity and -creates, renews and reclaims Environments; Runtime prepares capabilities, executes +creates, bootstraps, renews and reclaims Environments; Runtime initializes files, +tool configuration, packages, setup and capabilities, then executes work and recovers within an Environment. Keep these contracts and lifetimes separate: closing a Session executor does not release its allocation, destroy its Environment or delete its workspace. Reclamation is an explicit resource-manager operation coordinated with active work. -The capability-unification target is one Runtime preparation path for managed -bundles and self-hosted local directories. Core resolves configuration, versions +Capability preparation uses one Runtime path for managed bundles and self-hosted +local directories. Core resolves configuration, versions and resources; Runtime uses one parser and `installed.json` snapshot to supply Skill paths and MCP declarations to adapters. Preparation must complete before execution; reconnect reuses installed contents, and new Sessions freeze a new -configuration snapshot. The protocol is platform-neutral, while the first -implementation is Linux-only. This is pending work, not a statement that current -managed and self-hosted preparation or Windows/macOS isolation is already unified. +configuration snapshot. The protocol is platform-neutral; the implementation uses the existing packaged +Linux Runtime layout for both environment origins. Windows/macOS installation and +isolation remain separate work. - Keep component boundaries explicit through shared interfaces and versioned protocols. Register implementations behind those interfaces. Adding an @@ -495,7 +496,7 @@ forwarding. The explicit daemon-executor decision supersedes the previous native executor interoperability requirement. The superseded execution route is removed; retain reusable filesystem helpers, necessary regressions and historical evidence without a compatibility layer. -The private daemon wire protocol is 0.5.0. Initial, prepared and active input use +The private daemon wire protocol is 0.7.0. Initial, prepared and active input use the same ordered MessageInput contract, replacing scalar prompts and attachments. User-message boundaries and text/image order remain intact through Core and the Runtime wire; adapters own native conversion and receipt aggregation. Text-only @@ -507,8 +508,8 @@ input list and uses blank-line separators between messages; this does not preser independent native user-message boundaries. No old wire fallback is maintained. Deploy Core and daemon together; the existing major/minor WebSocket check rejects older major/minor peers before dispatch rather than ignoring removed fields. -The independently packaged Claude bridge uses protocol 2 for ordered input; -readiness rejects packages reporting the old string-input protocol. +The independently packaged Claude bridge uses protocol 3 for a prepared Executor +and separately identified Turns; readiness rejects other protocol versions. Image-bearing messages require a qualified profile/placement before persistence and image support from the selected Runtime before native delivery. These checks apply to that operation only; ordinary text retains offline queueing. Initial, @@ -554,9 +555,11 @@ or load encrypted file bodies. Record original creation intent before resolution Template parsing, persistence and resolution must not select a harness or Provider, or depend on native tool names and private harness paths. The shared initializer uses the packaged Runtime contract for trusted commands, workspace/staging paths, -confidential input and completion receipts. Each Provider supplies that same -Runtime and carries initialization commands through RunCommand; each adapter owns -native tool configuration. A new harness or Provider must not require template +confidential input and completion receipts. Each Provider bootstraps the same +daemon; all initialization uses its authenticated `runtime_prepare` protocol. +The common runner depends on Environment and Session identity and a Runtime peer, +not a Provider, deployment type or operating system. Each adapter owns native tool +configuration. A new harness or Provider must not require template business-logic changes. Reuse qualified shared helpers even when their executable names have historical engine prefixes; renaming is not a boundary fix. Select real regressions by the changed shared, Provider and adapter boundaries, rather than @@ -596,8 +599,9 @@ never from command, package-manager or file output; unknown effects, timeouts an receipts without a status keep a generic reason. New input then gets the observed 409 `conflict_error`; expiry and pending-input settlement keep their behavior. Completed environments never reinstall initial files on reconnect or native recovery. -Provider RunCommand carries bounded stdin, not confidential argv. Only fixed trusted -initializers may run with Runtime authority. User setup and package install hooks +The typed Runtime preparation protocol carries bounded confidential input. The +daemon selects fixed trusted initializers; Core supplies no executable or host +platform field. User setup and package install hooks run in the common packaged sandbox, without daemon credentials or native history. Files, resolved Skills and inline Plugins precede system, npm/Python packages and ordered setup commands. Initialization has provisioning network access; requested network restrictions apply to native tools @@ -728,11 +732,13 @@ omission; expand inline and template initialization together in separately quali batches. Resource reads need only tenant authorization, not a live Runtime. See the [Template coverage and unresolved semantics](contracts/agents-api/environment-templates.md). -SandboxProvider has five operations: Create, GetInfo, Renew, Kill and RunCommand. -Use maintained provider SDKs and thin adapters. Hosted deployments select one deployment-wide Provider: E2B cloud, or Docker/microsandbox on administrator-owned nodes. -Provider initialization creates the sandbox and starts its daemon/harness; -RunCommand is for initialization only. Daily execution and Files use Runtime and -native or bounded local capabilities. Docker's lack of a native renewable lease +SandboxProvider owns Create, GetInfo, Renew and Kill, with placement and daemon +bootstrap at the resource-management boundary. Use maintained provider SDKs and +thin adapters. Hosted deployments select one deployment-wide Provider: E2B cloud, +or Docker/microsandbox on administrator-owned nodes. Providers do not execute Core +initialization commands. Initialization, daily execution and Files use the same +daemon through typed Runtime operations and native or bounded local capabilities. +Docker's lack of a native renewable lease does not remove service-owned hosted expiry and cleanup requirements. The official `openai_hosted` discriminator means hosting by this independent Core @@ -1095,11 +1101,12 @@ change guard. This boundary does not add cross-node Session migration, Core multi-active, autoscaling, Kubernetes or harness residency. The common -`services/agents-api/internal/sandbox` contract owns the five base operations -(Create, GetInfo, Renew, Kill, RunCommand) and the optional CheckpointProvider +`services/agents-api/internal/sandbox` contract owns the four base operations +(Create, GetInfo, Renew, Kill) and the optional CheckpointProvider capability. Core orchestration must not import an adapter or SDK. Exact compute identity, generation construction, inspection, full snapshot capture, restore, -thaw, command execution and owned artifact cleanup use that common capability. +thaw and owned artifact cleanup use that common capability. Initialization and +execution use the authenticated Runtime peer. Provider-specific names and snapshot identities are opaque to Core. Self-hosted compute and providers without checkpoint support keep their existing behavior. @@ -1169,7 +1176,7 @@ a chain of old writable disks across suspension cycles. No Kubernetes, distribut scheduler or snapshot replication belongs in this V1 profile. Queued work and live Environment file access request wake. History and published -artifact reads do not. Planned suspension uses private daemon wire 0.5.0 with an +artifact reads do not. Planned suspension uses private daemon wire 0.7.0 with an Environment and suspension token; a PID/start-time fenced local control signal wakes the parked daemon, which reauthenticates before admitting new work. A transient disconnect before confirmation retries the same armed suspension with @@ -1482,8 +1489,9 @@ the Session row/cursor in the same transaction, without borrowing subsequent activity or Turn state. Initial state is `pending`; authenticated connection observations follow the lifecycle rules below. Public creation supports `self_hosted` on the three enabled native profiles when -the daemon gateway is configured. V1 requires `/workspace` and empty/default -capability directories. Supported non-deferred functions keep their engine-specific +the daemon gateway is configured. The workspace is a clean absolute Linux path matching the bound Runtime root; +`/workspace` denotes that root through the existing logical mapping. Optional +capability directories are clean absolute host-local selections. Supported non-deferred functions keep their engine-specific validation and native callback bridge. Enrollment binds the dedicated Runtime; Session output uses the owned Environment association. Public Files reuse the exact local workspace; populated self-hosted installation metadata remains unsupported. @@ -1622,8 +1630,9 @@ rotation/revocation and digest-only storage. They grant connection authority, ne Session API access. No raw-token import or secret read-back is added. Enrollment atomically creates or recovers one dedicated device and immutable Session -binding under the Session lock. The V1 workspace is `/workspace`; nonempty -self-hosted capability directories remain unsupported. No `runtime_allocation` is +binding under the Session lock. The frozen workspace and capability directory selection come from the Session +configuration and must match the local binding. Runtime snapshots declared local +Skill or Plugin directories before creating the native executor. No `runtime_allocation` is created for user-owned compute. A retry cannot replace a device, change its bound key or adopt another native history. Gateway authentication and dispatch recheck current key authority; rotation/revocation and deletion deny further use. @@ -1698,103 +1707,130 @@ reconciles old connection observations before admitting new ones. A connection o heartbeat does not prove native readiness or process quiescence. Failed or stale observations cannot establish a current connection. -Preparation resolves the immutable local binding and holds the existing native -resource without creating model work. Start transfers that resource once; failed -start and abandoned preparation retain the established cleanup rules. Strict resume -uses the bound native history and never falls back to a new Session. - -Every executable preparation must implement `PreparedCancellation`; read-only -preparations may implement only `Prepared`. The Router rejects and closes an -executable preparation before `ready` when that contract is missing. Codex fences -future Start under the transfer lock; unused resources use preparation teardown, -while transferred resources use the existing Session cancellation path inside the -adapter. `Close` remains inert after transfer. -`CancellationOutcome` exposes observed content, Usage and verified native identity; -an unstarted resource has no measured Usage or observed resume identity. This is -best-effort cancellation and an observed snapshot, not immutable final output, -notification drain, caller-deadline compliance or remote process quiescence. -From Start admission onward, the exact `PreparedCancellation` object is the sole -release target: a late Session never receives fallback `Cancel`, and the Router -never falls back to preparation `Close` or owner-context cancellation. Successful -`Cancel` means local cleanup is complete and no more output writes can occur. A -failed or timed-out call returns without waiting for output, retains execution -ownership, and permits only a later explicit serialized retry on that same object. -Before publication, failed cleanup also retains the preparation slot. Successful -publication permanently transfers resource tracking to the Run; subsequent release -does not restore preparation ownership or make its old handle cancel that Run. -Forwarded permission and user-choice observations from that cancelled handoff do -not register actionable interactions. Codex prepared cancellation also waits for -the transferred Session's local cleanup, which can finish after output closes. -Codex cancellation has one continuing native owner: caller deadlines bound only -their wait, leaving child observation and cleanup alive for an explicit retry. -Only observed child terminal facts can settle the child; actual observation -failures remain failures. Native process-exit confirmation remains retryable. -One prepared-output consumer starts before `Prepared.Start`, so native output beyond -the 64-frame channel capacity cannot deadlock Start. It retains the first terminal -frame, drains later output, and forwards accepted frames before the observed cancellation -outcome receipt. Missing capability, failed cancellation or failed forwarding cannot -produce an applied receipt. An unused resource may supply an empty observed outcome; -the Router never fabricates one. - -The returned Session remains private until the `started` status send succeeds. The -handoff has one permanent release claim, one current native attempt and one -success-only settlement. Function results, permission decisions, user-choice -decisions and steering admitted before the claim hold the same operation barrier -through native submission, replay bookkeeping and receipt delivery. Natural -completion waits for Start publication before cancellation; an abort wakes that -same attempt and may fence a concurrent Start. The initial started-status send is -bounded by the preparation deadline, which is rechecked at publication commit. -Delayed expiry callbacks cannot cancel a successfully published handoff. The successful attempt waits for Start and the sole output -consumer, then forwards the retained terminal frame and removes the Run. Internal -paths join the current attempt; only an explicit cancel, Release, expiry, device -shutdown or later Router Shutdown may retry a failed attempt. Workspace reads require -a published, open Session but keep their independent bounded lifecycle. - -Receipt settlement waits at most ten seconds, with a separate five-second send -budget and the existing gateway settlement deadline. A timeout does not free the -resource or stop tracking late cleanup. Shutdown cancels receipt waits while owned -Start/cleanup work remains tracked. This ordering covers cancellation received while -Start is pending; ordinary post-transfer cancellation, complete native output and -remote process exit retain their separate limitations. - -The private daemon preparation controls reuse execution configuration but reject -input, RunID, Conversation, attachments and product authoring. The local profile -requires an exact Environment binding, stable state key, strict resume and completion -release. Its separate capability is registered through an execution-only factory -and preserved through the product registry wrapper and heartbeat mapping. Native -details remain inside the adapter; this private profile does not narrow upstream. - -Preparation request IDs correlate only control responses. The daemon returns a -fresh opaque handle before slow work; Start supplies that handle and the actual -RunID/prompt. Handles belong to one daemon connection. Gateway preparation -subscriptions do not register Runs. Per-handle revisions order asynchronous status -snapshots; reject responses describe control errors without inventing run events. -At most four native preparations may be preparing, ready, starting or closing. -Start admission expires five minutes after acceptance; retries do not extend it. -Failed cleanup retains the native resource and its capacity until Close succeeds; -terminal handles with retained resources cannot be pruned or started again. -At most 64 request records are retained; retired request IDs may allocate a fresh -handle, while old handles cannot consume replacements. This is not durable -exactly-once preparation or cross-connection recovery. - -Preparation and Start execute outside the receive loop and router lock, with -tracked lifetime work. Start reserves the real RunID and fixed cancellation target -before native work; cancellation in that phase uses the adapter contract across -the transfer. A late result cannot resurrect released ownership. Shutdown claims -or retries every prepared release under the lock before closing its cancellation -signal. A failed native attempt returns Shutdown without waiting on an output -consumer that may still be blocked; a later Shutdown retries the same target. -Successful publication stops the preparation deadline and uses the same prepared -output consumer and completion release; -later preparation Release cannot cancel that Run. Released/expired status makes -the handle unusable; asynchronous native cleanup still counts toward capacity and -does not promise immediate OS quiescence. Release retries retained cleanup. -Concurrent Shutdown calls join one tracked attempt within their caller deadlines; -a later call retries failed preparation cleanup and reports any remaining error. -A caller timeout does not discard ownership or repeat in-flight cleanup. These -records remain connection-local, not a persistent remote retirement fence. -The public idle-text path uses this -admission/start wiring; complete Environment lifecycle remains required work. +A Session owns one reusable Executor in its connected Runtime. A Turn owns one +input execution, its output stream and its cancellation. `agent.ExecutorFactory` +prepares the fixed configuration; `Executor.StartTurn` creates a new `agent.Turn` +without replacing healthy native resources. Normal completion settles only the +Turn. `Executor.Close` releases native resources on idle expiry, environment +shutdown or confirmed invalidation. Core does not keep a second Executor cache. +The same lifecycle applies after managed or user-managed environments connect, +and to the qualified no-environment profiles. Resource management owns machine +selection, allocation and Environment creation/reclamation. Closing an Executor +does not release the Environment allocation or delete its workspace. Environment +reclamation explicitly coordinates with Runtime execution. Connection, installed +capability snapshot, Session Executor and Turn retain separate lifetimes. + +The Runtime binds its Executor record to Session, Environment, connection and +immutable execution configuration. Resume identity and prior-Turn recovery flags +are continuity assertions, not configuration changes. A supplied native identity +must match the retained owner; exact-history recovery never starts a new root +when existing history is required. A configuration conflict is an error, not a +hot switch. Lost connections retire their owners and handles. Old timers, output +and cancellation cannot affect replacements. + +Each Turn receives a fresh wrapper, output channel and receipt state. Optional +steering, functions, permissions and user-choice interfaces belong to that fixed +Turn. Native callbacks capture the originating Turn before asynchronous work; +late events cannot be assigned to whichever Turn happens to be active. Native +processes, query/transport connections, fixed capability configuration and native +Session identity belong to the Executor. Do not reset completed `sync.Once` +values or repurpose an old Turn object. + +`StartTurn` returning nil guarantees that no native input was submitted and the +output channel was not retained. The Runtime then closes that channel. Once input +may have been submitted, return a non-nil Turn even with an error: the Turn owns +exactly-once output closure and remains tracked until settlement. Unknown input +is never replayed. A definite `executor_unavailable` Start rejection permits one +common recovery attempt only after the previous Executor has been closed and no +input was submitted. Recheck the same physical peer and current authorization. + +`Turn.Cancel` targets only that Turn and does not close a healthy Executor. +`AwaitSettlement` applies after both natural completion and cancellation. Success +means output can no longer be written and the Turn's native events, input, +functions, interactions and child work have settled. Native completion or +cancellation confirmation is independent of resource retirement: closing a +transport cannot supply missing native terminal or operation receipts. +`Reusable=true` additionally +confirms that the native owner can accept the next Turn. `Reusable=false` requires +a reason and subsequent confirmed Executor close. An error means settlement is +unconfirmed; it cannot free ownership or capacity. Caller deadlines stop waiting, +not tracked cleanup. Retry the same cleanup target serially. Failed cleanup +blocks replacement and retains its resource slot. Executor Close confirms resource +retirement independently of the Turn outcome: an immutable Turn error must not +prevent closing the native transport and releasing resources once their work and +output have stopped. + +One output consumer starts before native Start, drains the bounded 64-frame +channel, and retains the terminal observation until Start publication, Turn +settlement and admitted operation receipts finish. Natural Done never calls +Cancel. Input, function and interaction admission close before settlement, and +operations already admitted hold their barrier through native receipts and +outbound acknowledgement. Send cancellation to the fixed Turn before waiting +for that barrier: a written input may need native interruption to produce its +receipt. Join native settlement, any required confirmed Executor close, output +drain and all admitted operations before an applied acknowledgement or reuse. +A failed Close may report failure while retaining the same Run and outstanding +operations for retry. Closing a caller wait cannot manufacture an applied input receipt. Only then forward Done or an applied cancellation +receipt. Commit native continuity and release the old Run admission before +publishing Done, since the receiver may immediately start another Turn. A late +terminal-send failure belongs to the old Run; it cannot invalidate a successor +that already owns the Executor. Connection shutdown owns transport-loss cleanup. +Preserve the ten-second settlement wait and separate five-second receipt +send budget; timeout is not proof of quiescence. The observed cancellation outcome +retains native identity, Usage and output without fabricating missing evidence. +Codex native Turn interruption can leave background terminals alive. Its adapter +uses the exact thread-owned terminal list and confirmed per-terminal termination +before settling cancellation, and repeats this cleanup during Executor close. +The bulk clean acknowledgement does not prove termination. Failed cleanup keeps +the native owner available for a later close attempt. + +Private preparation controls reserve a per-Turn admission, not a new Executor. +They carry an explicit Session identity and immutable configuration without model +input or Run identity. A fresh request returns a connection-local admission handle +and the owning Executor ID. Start supplies both identities and its actual Run ID +and ordered MessageInput. Per-admission revisions order status observations; +rejections describe control errors without inventing Run events. A reused healthy +Executor returns ready without native preparation. An admission release abandons +that admission; it does not close the Session's healthy idle Executor or cancel a +later Turn. Cancellation uses the exact Run identity. + +Preparations and Start execute outside the receive loop and Router lock. Admission +expires after five minutes; retries do not extend that deadline. Bound active +preparation and execution separately from idle retained resources, and count +closing or uncertain resources until cleanup succeeds. A definite +`execution_prepare` rejection with `preparation_capacity` leaves an unclaimed +queued Turn for the existing Worker scheduler to retry, including capacity held +by cleanup. Other errors and uncertain input delivery do not authorize replay. +At most 64 admission records are retained; old handles never consume replacement +admissions. Idle expiry +is a Runtime resource policy, not Core active-Turn concurrency. Shutdown tracks and +closes active and idle Executors, retains failed close targets, and allows a later +serialized retry. Ordinary disconnection closes the failed transport and keeps +the exact Router until shutdown succeeds. A wait timeout or failed cleanup cannot +authorize reconnect; process shutdown also keeps waiting rather than silently +discarding owned native resources. These records are connection-local, not durable +input replay. + +Read-only workspace preparations remain separate bounded filesystem operations; +they cannot start model work. Workspace operations retain exact binding and +settlement rules across Turn boundaries and Executor closure. + +Successful input commits send a coalesced hint to the existing Worker scheduler. +The scheduler keeps lease, capacity, cursor fairness and per-Session ownership +checks; a hint does not admit work itself. If capacity is occupied, preserve one +rescan for completion without turning failed preparation into a busy retry loop. +HTTP readiness waits and active input delivery subscribe before reading relevant +state and wake after committed promotion or input. They recheck storage after each +hint. Polling remains the fallback for external writers, expiry and lost hints; +notifications contain no execution authority and no durable input data. + +Core readiness, Start acknowledgement and input-to-first-text logs use a single +process monotonic clock. Start acknowledgement confirms adapter ownership, not +model input consumption. +Runtime logs identify Executor creation, reuse, idle and close independently of +Turn completion. Do not call these durations model-only latency or subtract clocks +from different machines. Native process creation and same-owner successive Turns, +real provider results and same-condition timings must substantiate reuse claims. The Dispatcher prepares pending Environment input only on its exact enrolled or managed device. Preserve the same physical peer and preparation handle through @@ -2818,31 +2854,28 @@ replaced; do not carry obsolete compatibility code forward to satisfy this secti require native steering receipts. Commit terminal outcome and native Session ID together under the admission lock; unapplied messages prevent successful completion. Resolve credentials separately from the immutable non-secret snapshot. -- Internal execution requires the advertised `durable_turns` engine capability, - strict resume, and optional cancellation receipts - and `release_on_completion` support. Reject unadvertised peers before claiming; - failed strict resumes must not fall back to a new native thread. Release the native writer before forwarding - completion, so the next Turn can resume its durable native ID. For - release_on_completion Runs, close new steering admission and finish all - existing steering receipt sends before releasing the executor and forwarding - Done. Cached input identities and conflicts remain readable while completing; +- Internal execution requires advertised durable Turns, strict resume, preparation, + applied input receipts and the qualified operation capabilities. Reject + unadvertised peers before claiming; failed strict resume cannot start unrelated + history. A completed Turn closes steering admission, settles existing native + operations and receipt sends, and closes its output before the retained Executor + can accept another Turn. Normal completion does not cancel the Executor. + Cached input identities and conflicts remain readable while completing; queue/write success is not consumption. The receipt worker stays busy through its send, and router shutdown cancels its native and transport waits. - `durable_input_receipts` is required before execution binding/claiming. The - per-input `durable_receipt` opt-in requires release-on-completion and a phased - adapter. Its ten-second transport timer stops only after a complete native write; - a separate `written` acknowledgement stops the API's thirty-second delivery timer. - Neither that phase nor legacy `in_flight` advances the input cursor. Await final - native acceptance/consumption under the Run lifetime without automatic redelivery. - Receipt sends retain a separate five-second shutdown-aware context, and Done - retains a fifteen-second final settlement bound. Once cancellation is sent, its - receipt owns the terminal outcome even if an input becomes unknown first. - Calls without the opt-in retain their existing response deadlines. Existing product - requests retain their default idle-process policy. Native history still requires - the device's persisted engine files; IDs alone cannot restore deleted history. - Cancellation receipts carry the stopped engine's continuity snapshot when no - Done is emitted. Preserve separately reported usage on failure; do not add the - same counters again when Done also includes them. + The per-input `durable_receipt` opt-in requires a phased adapter. Its ten-second + transport timer stops only after a complete native write; a separate `written` + acknowledgement stops the API's thirty-second delivery timer. Neither phase + advances the input cursor. Await final native acceptance/consumption under the + Turn lifetime without automatic redelivery. Receipt sends retain a separate + five-second shutdown-aware context, and Done retains a fifteen-second final + settlement bound. Once cancellation is sent, its receipt owns the terminal + outcome even if an input becomes unknown first. Calls without the opt-in retain + their existing response deadlines. Native history still requires the device's + persisted engine files; IDs alone cannot restore deleted history. Cancellation + receipts carry the stopped Turn's confirmed continuity snapshot when no Done is + emitted. Preserve separately reported usage on failure; do not add the same + counters again when Done also includes them. Usage frames carry cumulative snapshots for the current execution, not deltas. Adapters publish observed snapshots promptly through the same ordered stream; waiting for Done unnecessarily loses known measurements if the Runtime stops. @@ -3225,14 +3258,14 @@ Owned output pipes remain readable after the leader exits. Consumers must drain stdout and stderr before calling `Wait`, which joins the cached process result and closes the readers. `Done` reports leader reaping and group cleanup signals; it is not a native execution receipt or proof of persisted history. SDK adapters -must close their query, await their native child and drain observations before -publishing completion. Process groups are lifecycle supervision, not OS isolation +must settle each Turn and drain its observations before publishing completion. +Executor close additionally closes the query and awaits the native child. Process groups are lifecycle supervision, not OS isolation or containment of descendants that deliberately leave the group. ### Harness qualification and onboarding Codex, Claude and future harnesses have equal architectural status. The common -Runtime wire protocol and Factory/Session/Prepared interfaces own lifecycle, +Runtime wire protocol and Executor/Turn interfaces own lifecycle, input receipts, cancellation, recovery and resource access; each native adapter retains its implementation and model/tool loop. A new engine supplies an adapter, a qualified profile in `services/agents-api/internal/engine`, registration and @@ -3280,9 +3313,12 @@ workspace, functions or MCP. Enabled Subagents use the separately qualified common observation path below. Native configuration disables file/shell authority and external capability discovery; the child receives a private Session home and a restricted -environment. Active-input application requires a native ACP receipt, cancellation -settles the process and output, and continuation requires the exact owned native -history. Do not infer history IDs or qualify hosted execution from this text +environment. Active-input application requires a native ACP receipt. Normal +Turns retain one ACP connection and native Session. Cancellation requires native +root and child completion evidence before reuse. Without a qualified root-state +reader, the ACP cancellation response is insufficient: stop the invalid native +process and wait for its exit before settling the Turn as non-reusable. Common +Runtime recovery then loads the exact owned native history in a new Executor. Do not infer history IDs or qualify hosted execution from this text profile. See [deployment and acceptance](services/agents-api/deploy/mcode/README.md). The MiniMax workspace profile builds one CLI from the fixed upstream source and @@ -3432,11 +3468,11 @@ qualification limits are recorded in [the coverage note](contracts/agents-api/st ### Claude SDK adapter foundation `packages/claude-sdk-adapter` privately owns the pinned official TypeScript SDK -and native message translation. The Go `claudesdk.NewFactory` uses the shared +and native message translation. The Go `claudesdk.NewExecutorFactory` uses the shared owned process runner and emits the existing daemon delta/error/Done frames. -The SDK owns the model loop. Its narrow stdio protocol carries a start request, -text deltas, function calls/results/receipts, active text input/receipts, usage -snapshots and one terminal result/error; native translation stays inside the adapter. +The SDK owns the model loop. Its narrow stdio protocol carries Executor preparation and identified Turn starts, +text deltas, function calls/results/receipts, active input/receipts, usage snapshots +and terminal result/error plus settlement; native translation stays inside the adapter. With `observe_messages`, it also emits the existing neutral `output_message` start/completion snapshots and tags deltas with the native Messages API message ID, not the SDK event UUID. Text blocks in one native message share that identity. @@ -3444,7 +3480,18 @@ The SDK's per-block assistant snapshots replace draft block text; only native `message_stop` completes the message, without replaying its text as another delta. Thinking/tool-only messages produce no text Items; interrupted messages retain their streamed partial text. No phase is inferred from the final result. -SDK/native child release and output draining precede daemon completion. +Turn-owned native work and output draining precede reuse. Executor close releases +the SDK Query and native process. The private `turn_settled` frame requires +`confirmed` independently of `reusable`: confirmed native Turn/cancellation +settlement, confirmed resource cleanup, and reuse eligibility are separate facts. +Unknown or nonempty interrupt receipts and unsettled input/function/tool work +remain unconfirmed even after successful teardown. Go rejects cancellation and +AwaitSettlement when native confirmation is missing or false, or its own receipt +ledger remains unsettled. A confirmed Turn may be non-reusable after cleanup; +that state alone does not turn a verified cancellation into an error. +Confirmed native cancellation may settle unanswered function calls after result +admission closes and callbacks drain. A submitted function result still requires +its native application receipt, including when the MCP request aborts. `claudesdk.Config.Workspace` is a private, trusted operator binding for one qualified placement. It enables native Bash/Read/Edit and declared host functions @@ -3479,50 +3526,39 @@ public preparation, shared placement quotas, command Items and Files ownership. `TestLiveClaudeWorkspaceFactory` is explicit real-provider acceptance inside a qualified placement, including effects, cancellation and same-history continuation. -The private workspace bridge also accepts `prepare` without a prompt. It freezes +The private bridge accepts `executor_prepare` without model input. It freezes validated configuration and resume identity, checks required history, and retains -one native process through the pinned SDK's `startup`/`query` API. Preparation -requires initialization and acknowledgement of the required hooks while the input -iterator remains empty. Its `prepared` receipt permits one later `start` containing -only the initial prompt; configuration replacement, premature or duplicate start -is rejected. Native Session identity and actual tool inventory are still checked -at execution initialization before `input_ready`. Existing direct execution uses -the same observation and completion path. - -Unused EOF, owner signals, invalid control input and native exit release owned -resources before a terminal event. The private `workspace_prepare` runtime feature -identifies this bridge contract only. It does not register daemon preparation, -enable public admission, or project public Environment readiness. Preparation may -write native runtime metadata outside the workspace and perform startup traffic; -it does not prove provider authentication, complete sandbox health or tenant -placement authorization. - -`claudesdk.NewPreparationFactory` privately binds that workspace bridge to -`agent.Prepared`; the existing workspace factory uses the same Prepare/Start path. -Preparation receives configuration and required resume identity without a RunID or -prompt, checks the installed `workspace_prepare` feature, and owns the process until -one successful Start transfers it. The selected configuration and environment are -fixed before returning; a later Start supplies only its actual RunID, prompt and -output channel. Early preparation failure returns without an executing Session or -fabricated completion. The non-workspace direct factory keeps its existing path. - -The preparation owner context spans the eventual Session. Start's context bounds -that operation only, and Close is inert after successful transfer. Abandoned or -failed preparation, owner cancellation and native exit release owned work. The -same output consumer and cancellation settlement follow the resource across Start; -an unstarted preparation has no measured Usage or observed native Session identity. -A cancellation deadline cannot establish cleanup completion while cleanup remains -pending. This adapter ownership seam does not register a daemon capability or -supply per-Session placement authorization, public admission or an idle Files owner. - -The optional private `agent.WorkspaceReader` on this preparation and transferred -Session requires the packaged `workspace_read` feature. It sends bounded relative +one native process and SDK Query across Turns. Preparation requires initialization +and acknowledgement of required hooks while the input iterator remains empty. +An `executor_ready` receipt permits later `turn_start` messages containing only +Turn identity and ordered input; configuration replacement and concurrent starts +are rejected. Every Turn event carries its originating `turn_id`. Native Session +identity and actual tool inventory are checked before `input_ready`. + +Each Turn ends with a result/error and `turn_settled`, independently of process +exit. The outer input iterator remains open for later Turns. `turn_cancel` invokes +the native interrupt control for that exact Turn. Unconfirmed input, native child +work or queue state invalidates the Executor and requires close before replacement. +EOF, owner signals and invalid control input close owned resources. Preparation +may write native metadata and perform startup traffic; readiness does not prove +provider authentication, complete sandbox health or placement authorization. + +`claudesdk.NewExecutorFactory` binds this bridge to `agent.Executor`. Direct-call +and read-only preparation wrappers delegate to the same implementation. Runtime +execution uses the Executor registry for both none and workspace configurations. +Its owner context spans all Turns; a Turn's caller cannot replace fixed resources. +A failed preparation returns its Executor when cleanup remains unconfirmed. +Installed runtime checks are cached by package/file identity, while capability +and request validation still run for each Executor configuration. + +The optional private `agent.WorkspaceReader` on this Executor and its delegated wrappers +requires the packaged `workspace_read` feature. It sends bounded relative paths to that same SDK Query's native `readFile` control. Only the adapter combines the path with the frozen workspace root; callers cannot replace the placement. The pinned native read handler awaits file-handle close before its successful base64 response. The adapter validates bytes and truncation, bounds each result to 1 MiB and each request to 8 KiB, and admits one read at a time. Its continuous -bridge output consumer retains read receipts during preparation and across Start. +bridge output consumer retains read receipts during preparation and across Turns. Caller cancellation detaches observation without cancelling the Run or discarding an admitted waiter; its original deadline still applies. Owner closure stops admission. Native null, malformed receipts, timeout and interrupted delivery remain @@ -3535,7 +3571,7 @@ before input and during real execution, plus effects before cancellation and rea on fresh-process history continuation. The optional private `agent.WorkspaceDirectoryLister` requires the Linux-only -`workspace_directory` bridge feature on the same preparation or transferred Session. +`workspace_directory` bridge feature on the same Executor. The pinned SDK has no directory-enumeration control; its fuzzy file suggestions are not an inventory. This narrow adapter operation therefore reads metadata inside the already qualified co-located mount/process boundary. It pins the configured workspace @@ -3552,7 +3588,7 @@ ordering, snapshots, recursion, or public pagination. Missing and permission err are returned only from distinguishable filesystem outcomes; unknown results stop the owner. Each operation closes its directory and intermediate descriptors before a successful receipt; the root descriptor remains owned until bridge release. -Caller cancellation, Start transfer and owner shutdown retain the existing workspace +Caller cancellation, Turn transitions and owner shutdown retain the existing workspace read settlement rules. This adapter gap fill alone does not enable public Claude Files; the dedicated Runtime integration supplies public placement and ownership. @@ -3758,9 +3794,10 @@ also retain all snapshots in order under `claude_sdk_results`. Main-loop `usage` is per native turn, while query-pipeline `modelUsage` and estimated `total_cost_usd` are cumulative within the query. Retain subtype/error provenance and earlier snapshots even when a later failure reports zero counters. Reuse the latest full -snapshot set in Usage and Done; never sum cumulative measurements. Each factory -invocation owns one SDK query, including cold resume, so no prior query counters -are carried forward. Missing native results do not imply zero consumption. SDK estimates stay +snapshot set in Usage and Done; never sum cumulative measurements. Each Executor owns one SDK query, including cold resume. Raw snapshots explicitly +identify per-native-turn usage versus query-cumulative model usage and cost. A new +Turn has a fresh snapshot list, but query totals may include earlier Turns; never +represent those totals as consumption by the current Turn. Missing native results do not imply zero consumption. SDK estimates stay in raw evidence, outside the billed cost field; do not select an arbitrary model or invent missing public token breakdowns. The API does not parse native counters. Precise public usage projection, unreported costs and crash/partial accounting @@ -3776,18 +3813,19 @@ confirm applied input. Typed mid-turn folds may appear only on the native result Preserve that receipt even when the result reports failure. Check pending functions after the query drains: the SDK may dispatch later-turn callbacks before the earlier result handler finishes. -Keep the iterator open until every submitted input has a consuming result, even -when an earlier result reports an empty native queue. Close admission before -releasing final receipt waiters; drain and release the SDK/native processes before -one daemon Done. Cancellation resolves unconfirmed pending receipts as unknown and ends the -owned execution. Successful private SDK Cancel waits for owned-process exit and -stdout/stderr drain, then exposes the same settled CancellationOutcome as Done. -It retains native identity verified at input readiness, partial text and observed -Usage even on cancellation/failure; requested resume identity alone is not evidence. -A caller deadline before settlement reports failure/unknown, while cleanup continues. -Settlement precedes terminal publication so router completion cleanup cannot wait -on its own Done consumer. Cancellation releases intermediate event backpressure; -the settled outcome remains readable even when connection loss prevents publication. +Keep Turn input admission open until every submitted input has a consuming result, +even when an earlier result reports an empty native queue. Close admission before +releasing final receipt waiters. The outer SDK iterator remains open across Turns. +Cancellation resolves unconfirmed pending receipts as unknown and interrupts the +exact native Turn. Successful Cancel requires confirmed Turn settlement; process +exit alone cannot establish a successful cancellation. Reuse also requires an +empty confirmed native queue and settled child work. Otherwise close the Executor. +The settled CancellationOutcome retains verified native identity, partial text +and observed Usage; a requested resume identity alone is not evidence. Caller +wait expiry reports failure/unknown while cleanup retains ownership. Output +backpressure cannot turn missing native facts into confirmed settlement. Closed +Turn output precedes successful AwaitSettlement; the settled outcome remains +readable when connection loss prevents publication. A receipt timeout after a full write preserves the process and pending identity without redelivery; a blocked write is cancelled and released. The private adapter permits one input awaiting consumption and at most 63 extra @@ -3882,6 +3920,80 @@ request on first execution; later messages and observer retries reuse it. Separa conversations get independent Sessions and environments. Edits affect future Sessions only. Keep the existing product navigation and direct empty-chat composer. +### Shared Runtime capability preparation + +Resource management retains provider placement, capacity, allocation and Environment +create/renew/reclaim operations. The authenticated Runtime connection carries +initialization, capability preparation and executor operations; it does not implicitly +allocate or destroy compute. Connection loss, executor idle close and Turn cancellation preserve +the workspace and installed snapshot. Resource reclamation coordinates with active +work through its existing owner. + +Core freezes resource versions, metadata and local source selections. Managed +initialization keeps its existing order: initial files, tool configuration, +Skill/Plugin bundle import, packages and setup commands, then directory snapshot +finalization. Every step uses the same daemon and `runtime_prepare` exchange. +Providers only place, create, bootstrap, inspect, renew and reclaim resources; they +do not execute Core initialization commands. The common runner uses only neutral +Environment/Session identity and a Runtime peer, with no Provider, deployment or OS +branch. Harness differences belong to native adapters. Package installation and +setup retain enabled provisioning network access; the requested network policy +constrains native execution after setup. +A missing authenticated Runtime connection waits before capability initialization +is claimed; an operation whose effect is unknown is not replayed. Self-hosted +capability directories remain in immutable Environment configuration and never +create a managed initialization row or allocation. + +The private `runtime_prepare`/`runtime_prepare_result` exchange carries typed +initial files, configure/system/npm/python/setup operations, inert Skill/Plugin +archives and finalization selections, with canonical Session and Environment +identities. Files and setup working directories use logical `/workspace` addresses; +setup commands are explicit typed input, while executable selection and physical +destinations remain Runtime-owned. Ordered 64 KiB chunks and SHA-256 receipts +bound each file or archive to 50 MiB, each control frame to 1 MiB and each connection +to one transfer. Initialization and finalization headers carry no file data. Begin, +chunk and commit are never retried; +rejected, failed and unknown effects remain distinct. Runtime owns transfer and +filesystem work through settlement, including disconnect or cancellation. + +The source-selection protocol accepts portable absolute Unix, Windows drive and +UNC paths without consulting Core's filesystem. Runtime interprets and authorizes +local paths. The current implementation is Linux-only, with its existing packaged +helpers and layout; the interface imposes no Linux executable or platform field. +The common Linux binding authorizes local directories, excluding aliases, Runtime +credentials, native history and initialization roots or their ancestors. One shared +parser writes `installed.json` below the operator-owned capability directory +(default `/environment/initialization/capabilities`, selected at startup through +`OAC_RUNTIME_CAPABILITY_DIRECTORY`). The root cannot overlap the workspace or +private Runtime state and is never a public input. All adapters consume the same +resolved installation root. The manifest +binds Session/Environment identities and the ordered source-selection digest. +A private completion receipt records only the installation root outside the +installed tree, so a deleted snapshot cannot be mistaken for first preparation. +It is written durably before execution, contains no capability configuration and +is never exposed as a second inventory. Filesystem locking prevents overlapping +installation. A partial, invalid or foreign +snapshot fails closed without deleting files or silently reinstalling. Directory +sources are copied only during first preparation; reconnect loads protected installed +contents and validates the selection without rereading source directories. A new +Session uses its own Environment and fresh snapshot. + +Synchronous executor admission validates the frozen descriptor only. The admitted +asynchronous preparation owner ensures capabilities are ready before invoking the +native factory, including an empty inventory. Resolved Skill paths and MCP declarations +remain Runtime-only fields; engine adapters consume this common result. Reused +Session executors retain their original configuration. Files reads retain their +separate authorization and do not require capability or native execution readiness. + +Self-hosted public input remains `workspace_directory` and optional +`capability_directories`; it does not accept managed archive fields. Its public +installation arrays describe API-managed uploads and stay empty for local directories. +MCP environment variables must come from explicit immutable Runtime tool configuration; +missing variables never fall back to daemon credentials or ambient process variables. +No project-version upgrade or historical manifest compatibility is introduced. +These implementation rules do not establish new real-deployment acceptance; +historical evidence remains limited to its recorded binaries and inputs. + ### API-key write provenance Public resource writes carry authenticated key provenance separately from the diff --git a/README.md b/README.md index e3d6d192a..c8f255455 100644 --- a/README.md +++ b/README.md @@ -54,6 +54,11 @@ Applications then set `OPENAI_BASE_URL` to `https://core.example/v1` and - **Your sandboxes.** Core-hosted Sessions run in sandboxes on your nodes (Docker or microsandbox microVMs) or on E2B. Applications can also connect their own machines as self-hosted executors. +- **One Runtime path.** The same daemon handles initial files, tool configuration, + packages, setup, Skills/Plugins and execution. Providers place and bootstrap + compute, then renew or reclaim it. The protocol is platform-neutral; the current + packaged implementation and isolation are Linux-only. Harness differences stay + in adapters. - **A console for administrators.** Web signs in with the Core key, issues Project API keys, adds nodes, sets default models and shows metrics and Session history. It never runs Agents on anyone's behalf. @@ -76,6 +81,7 @@ native differences. | [API reference](docs/api/README.md) | The `/v1`, `/core/v1` and `/api/v1` namespaces | | [Nodes and sandbox backends: operator reference](services/agents-api/HOSTED-SANDBOX-MANAGER.md) | Node protocol, manual registration, placement, reset | | [Protocol coverage](contracts/agents-api/README.md) and [harness selection](contracts/agents-api/harness-selection.md) | Supported operations and native differences | +| [Add a Harness](contracts/agents-api/harness-onboarding.md) | Required adapter interfaces, lifecycle, capabilities, registration and acceptance | | [Landing page source](site/index.html) | The public site | ### Maintainers and advanced deployments diff --git a/apps/docs/content/docs/admin-api.mdx b/apps/docs/content/docs/admin-api.mdx index ec9674f6b..1e4fe2786 100644 --- a/apps/docs/content/docs/admin-api.mdx +++ b/apps/docs/content/docs/admin-api.mdx @@ -8,7 +8,7 @@ resource inspection, Project and key management, credential issuance, audit, usa and sandbox operations. It never calls `/v1` or `/api/v1`, and has no Agent execution, copy or arbitrary asset editing operation. -Core request failures use the [Core administration error envelope](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/core-errors.md), +Core request failures use the [Core administration error envelope](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/core-errors.md), including optional typed safe details. Console sign-in endpoints retain their separate error shape described below. @@ -49,9 +49,9 @@ label, but could set any value. Never use it for authorization or as proof of origin. Web calls Core only through the typed Core clients in `packages/agents-client`: -[AdminClient](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/packages/agents-client/src/admin-client.ts) (`/core/v1`), +[AdminClient](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/packages/agents-client/src/admin-client.ts) (`/core/v1`), `SandboxAdminClient` (`/core/v1/sandbox`) and `CoreMetricsClient` -(`/core/v1/metrics`). See [the complete administrator reference](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md) +(`/core/v1/metrics`). See [the complete administrator reference](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md) for methods, fields, filters, pagination and response shapes. Routes below are relative to `/core/v1`: @@ -62,8 +62,8 @@ Routes below are relative to `/core/v1`: | Resource lists/details/deletion | `/projects/{id}/agents`, `/sessions`, `/environment-templates`, `/skills`, `/files`, `/vaults`, including the documented nested reads | | Asset ownership | `GET /projects/{id}/resource-owners` with batched resource IDs | | Key operation history | `GET /projects/{id}/write-operations` with key/resource/time filters | -| Executor credentials | `GET/POST /projects/{id}/environments/{environment_id}/executor-credentials`, `DELETE …/executor-credentials/{key_id}` ([contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-executor-credentials.md)) | -| Deployment model providers | `GET /harnesses`, `GET/PUT/DELETE /harnesses/{harness}/model-provider`; the key is write-only ([contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/model-execution.md#deployment-defaults)) | +| Executor credentials | `GET/POST /projects/{id}/environments/{environment_id}/executor-credentials`, `DELETE …/executor-credentials/{key_id}` ([contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-executor-credentials.md)) | +| Deployment model providers | `GET /harnesses`, `GET/PUT/DELETE /harnesses/{harness}/model-provider`; the key is write-only ([contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/model-execution.md#deployment-defaults)) | | Usage and health | `GET /summary`, `/sandbox/runtime-observations`, `/metrics` | | Administrator audit | `GET /audit-log`; deployment-wide entries have `project_id: null` | @@ -76,9 +76,9 @@ it is operational attribution, not per-key billing. ## Sandbox administration -The [deployment configuration contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/sandbox-deployment.md), -[nodes and sandbox backends reference](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md) -and [generated OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/core.openapi.yaml) +The [deployment configuration contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/sandbox-deployment.md), +[nodes and sandbox backends reference](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md) +and [generated OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/core.openapi.yaml) define deployment and node operations: - `GET/POST/PUT /core/v1/sandbox/deployment` and @@ -115,7 +115,7 @@ is a durable serving pin, not proof of current connectivity. Target unknown, fai or update-required state does not by itself invalidate confirmed old-generation service; consume Core's connection/provider facts separately. -Administrators may [archive an individual hosted Session](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md#administrative-session-archive) +Administrators may [archive an individual hosted Session](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md#administrative-session-archive) at the current generation without reset. History and persisted Files/Artifacts survive; unpersisted workspace is lost and the original Session cannot resume. Cancel reset stops further archives, not cleanup already requested. After zero @@ -156,7 +156,7 @@ generated schema; do not interpret every empty or failed read as an absent resou `GET /core/v1/metrics?range=1h|6h|24h|7d` returns Core process, execution queue/slots, PostgreSQL and background-job measurements. It requires the Core key, rejects arbitrary query filters and never grants -Agent execution access. See the [exact measurement contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/core-metrics.md) +Agent execution access. See the [exact measurement contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/core-metrics.md) for complete buckets, null values, units and process-local retention. Frontend implementation is maintained separately; this backend change does not modify Agent metrics or the public Agent API. @@ -165,7 +165,7 @@ Agent metrics or the public Agent API. The Core key reads a node and its host history through `GET /core/v1/sandbox/nodes/{node_id}?range=1h|6h|24h`. See the -[node host history contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/node-host-history.md) +[node host history contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/node-host-history.md) for nullable observations, freshness and aggregation. The node list is unchanged. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/api/web-management.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/api/web-management.md) diff --git a/apps/docs/content/docs/agents-and-tools.mdx b/apps/docs/content/docs/agents-and-tools.mdx index 100956bd9..888915ce6 100644 --- a/apps/docs/content/docs/agents-and-tools.mdx +++ b/apps/docs/content/docs/agents-and-tools.mdx @@ -4,7 +4,7 @@ description: "Native harness tools, shared declarations and the limits of execut --- Assessed against Core `206474a4c10901442b1faa094281bfb5559082b5` on 2026-09-22. -The contract remains [OpenAI Python 3.13.0 at `d7c41ef`](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/upstream.json), Beta +The contract remains [OpenAI Python 3.13.0 at `d7c41ef`](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/upstream.json), Beta `agents=v1`. This matrix records the bounded execution/tools milestone. It does not establish complete protocol compatibility; the remaining limits below stay open. @@ -22,20 +22,20 @@ and Environment Plugin MCP have separate inventories and qualification. | Operation | Implemented behavior and real qualification | Unsupported or unverified boundary | | --- | --- | --- | -| Initial message input, `sessions.create` | String input and ordered user-message arrays share atomic admission. Codex/Claude text and inline image execution: M1/M2; ordinary MiniMax text: M1/P1. [Input contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/message-input.md), [initial parser](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/api/session_initial_input.go). | Whitespace-only text is admitted verbatim for Codex and rejected at admission for Claude SDK and MiniMax Code ([text content](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/message-input.md#text-content)). Empty parts beside text (SES-08) and local size-limit parity need upstream evidence. Images are only qualified for inline PNG/JPEG on `none` and Core-managed Docker `openai_hosted`; MiniMax, self-hosted images and remote URLs remain gaps. | -| Prepared and active messages, `sessions.events.create` | Ordered `input_text`/`input_image` arrays retain original content and distinct public user Items. HTTP 202 confirms persistence; native receipts establish application. Initial/prepared/active Docker PNG/JPEG: M2; active PNG on `none`: M1. [Shared admission](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/api/inputs.go). | Codex flattens native messages with blank-line separators. Claude can fold or queue native turns; it does not promise Codex's same-native-turn behavior. Public durability does not prove native consumption. Claude SDK and MiniMax Code reject messages without an image or non-whitespace text at admission (`unsupported_or_invalid_configuration`); Codex delivers them unchanged. | -| Structured output, `agent.text.format` | Save/inherit/freeze `{type:json_schema,schema:...}`. Claude SDK object-root, single Agent, medium verbosity, ordinary functions returning text: S1 (`none`) and S2 (Docker, including prepared/active input and Files/Artifacts). Native final text is retained unchanged; its stream follows the official message sequence with the whole text in one `output_text.delta` ([item serialization](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/history-events-usage.md#item-serialization-2026-09-23)). [Contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/structured-output.md), [profile](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/engine/claude.go). | An explicit non-object root type is a protocol error for every harness ([validation](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/official-semantics-alignment.md#agent-configuration-validation--september-23)). Codex/MiniMax, schemas without an object root, schema numbers changed by binary64, self-hosted, Skills/Plugins, MCP, Subagent and discovery combinations reject execution. No output repair, coercion or extra model loop. Arbitrary schema dialects are unverified. | -| Function configuration, saved/inline Agents | Required name/description/schema; `defer_loading` defaults false. Protocol errors, repeated names and explicit non-object root types reject with the official fields ([validation](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/official-semantics-alignment.md#agent-configuration-validation--september-23)). Saved references resolve into an immutable Session snapshot. Codex/Claude real calls: F1/F2/M2/S1/S2. [Parser](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/api/function_configuration.go), [saved tools](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/api/saved_tools.go). | MiniMax public functions reject. Claude requires an explicit object root. Local nonblank/512-byte name and 64-definition Session bounds are compatibility gaps. Saving configuration alone does not qualify execution. | -| Function-result admission, `events.create` | Required `turn_id`, `call_id`, `success`; optional nullable `error` and `output`. Output is string or ordered text/image content. Scoped atomic batches retain field presence, original content and retry identity in storage; public result Items and events always carry `output` and `error`, null when not submitted ([item serialization](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/history-events-usage.md#item-serialization-2026-09-23)). Same result retries are accepted; changed results and results after cancellation return 409 `conflict_error`, including after terminal state. In the caller's Session, an unknown call or a call of another Turn returns 400 `invalid_request_error` without changing the pending action; missing and foreign Sessions stay 404 ([error rows](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/official-semantics-alignment.md#session-input-conflicts-and-result-targets--september-23)). F1/F2/M2 plus [controlled SDK/raw checks](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/tests/official_function_inputs.py). [Parser](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/api/function_inputs.go), [Store](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/store/function_inputs.go). | Admission is separate from application and public Item publication. Invalid or unqualified content cannot consume a pending call. Core messages omit the call and executor IDs that official messages name; hosted defaults and publication timing remain unverified. | -| Function text results and native application | Codex waits for a matching live root dynamic-tool completion; Claude waits for a matching live root native tool result. Text/error results, retry/conflict, cancellation and cold continuation: F1/F2. [Receipt contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/function-result-images.md). | Transport writes alone do not confirm application. Confirmation is not provider consumption, crash recovery or exactly-once external effects. No automatic result replay. | +| Initial message input, `sessions.create` | String input and ordered user-message arrays share atomic admission. Codex/Claude text and inline image execution: M1/M2; ordinary MiniMax text: M1/P1. [Input contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/message-input.md), [initial parser](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/api/session_initial_input.go). | Whitespace-only text is admitted verbatim for Codex and rejected at admission for Claude SDK and MiniMax Code ([text content](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/message-input.md#text-content)). Empty parts beside text (SES-08) and local size-limit parity need upstream evidence. Images are only qualified for inline PNG/JPEG on `none` and Core-managed Docker `openai_hosted`; MiniMax, self-hosted images and remote URLs remain gaps. | +| Prepared and active messages, `sessions.events.create` | Ordered `input_text`/`input_image` arrays retain original content and distinct public user Items. HTTP 202 confirms persistence; native receipts establish application. Initial/prepared/active Docker PNG/JPEG: M2; active PNG on `none`: M1. [Shared admission](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/api/inputs.go). | Codex flattens native messages with blank-line separators. Claude can fold or queue native turns; it does not promise Codex's same-native-turn behavior. Public durability does not prove native consumption. Claude SDK and MiniMax Code reject messages without an image or non-whitespace text at admission (`unsupported_or_invalid_configuration`); Codex delivers them unchanged. | +| Structured output, `agent.text.format` | Save/inherit/freeze `{type:json_schema,schema:...}`. Claude SDK object-root, single Agent, medium verbosity, ordinary functions returning text: S1 (`none`) and S2 (Docker, including prepared/active input and Files/Artifacts). Native final text is retained unchanged; its stream follows the official message sequence with the whole text in one `output_text.delta` ([item serialization](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/history-events-usage.md#item-serialization-2026-09-23)). [Contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/structured-output.md), [profile](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/engine/claude.go). | An explicit non-object root type is a protocol error for every harness ([validation](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/official-semantics-alignment.md#agent-configuration-validation--september-23)). Codex/MiniMax, schemas without an object root, schema numbers changed by binary64, self-hosted, Skills/Plugins, MCP, Subagent and discovery combinations reject execution. No output repair, coercion or extra model loop. Arbitrary schema dialects are unverified. | +| Function configuration, saved/inline Agents | Required name/description/schema; `defer_loading` defaults false. Protocol errors, repeated names and explicit non-object root types reject with the official fields ([validation](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/official-semantics-alignment.md#agent-configuration-validation--september-23)). Saved references resolve into an immutable Session snapshot. Codex/Claude real calls: F1/F2/M2/S1/S2. [Parser](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/api/function_configuration.go), [saved tools](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/api/saved_tools.go). | MiniMax public functions reject. Claude requires an explicit object root. Local nonblank/512-byte name and 64-definition Session bounds are compatibility gaps. Saving configuration alone does not qualify execution. | +| Function-result admission, `events.create` | Required `turn_id`, `call_id`, `success`; optional nullable `error` and `output`. Output is string or ordered text/image content. Scoped atomic batches retain field presence, original content and retry identity in storage; public result Items and events always carry `output` and `error`, null when not submitted ([item serialization](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/history-events-usage.md#item-serialization-2026-09-23)). Same result retries are accepted; changed results and results after cancellation return 409 `conflict_error`, including after terminal state. In the caller's Session, an unknown call or a call of another Turn returns 400 `invalid_request_error` without changing the pending action; missing and foreign Sessions stay 404 ([error rows](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/official-semantics-alignment.md#session-input-conflicts-and-result-targets--september-23)). F1/F2/M2 plus [controlled SDK/raw checks](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/tests/official_function_inputs.py). [Parser](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/api/function_inputs.go), [Store](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/store/function_inputs.go). | Admission is separate from application and public Item publication. Invalid or unqualified content cannot consume a pending call. Core messages omit the call and executor IDs that official messages name; hosted defaults and publication timing remain unverified. | +| Function text results and native application | Codex waits for a matching live root dynamic-tool completion; Claude waits for a matching live root native tool result. Text/error results, retry/conflict, cancellation and cold continuation: F1/F2. [Receipt contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/function-result-images.md). | Transport writes alone do not confirm application. Confirmation is not provider consumption, crash recovery or exactly-once external effects. No automatic result replay. | | Function image results | Successful ordered inline PNG/JPEG with text, large PNG and image-only JPEG: Codex/Claude `none` F1/F2; Docker M2. Public Items retain submitted bytes. Codex receipt regression: F2. | Claude rejects failed images and remote references before persistence; native resizing may change its image bytes. MiniMax functions and self-hosted image workflows are unqualified. Other Codex image/error/reference combinations cannot be inferred from successful-inline evidence. | -| Deferred function discovery | Type-only `tool_search` plus mixed eager/deferred functions: Claude SDK 0.3.269/native 2.1.269, Kimi K3, single Agent, medium, `none`, text results; text and PNG input D1. Native provider observations establish lazy schema loading. [Contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/tool-search.md). | A repeated `tool_search` is a protocol error. Codex/MiniMax discovery, search-only/missing-search, workspace, MCP, structured-output and Subagent combinations remain gaps. Opaque native policy changes lack a reliable pre-input deferral signal. Saved tools include `tool_search`; the pinned Session response union excludes it. Exact hosted projection is unverified. | -| Explicit disabled search/PTC | Saved and inline `web_search.mode=disabled` and `programmatic_tool_calling.enabled=false`; shared native controls on initial and cold execution. All three harnesses on `none`: P1, including native inventory/control evidence. [Contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/tool-policy.md), [parser](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/api/disabled_tools.go). | Unsupported explicit enablement rejects at Session admission; saved Agents keep every pinned search mode as resource data (TV-05), and Sessions from such Agents reject unless they replace the tools. A repeated `web_search` is a protocol error. Omission retains approved native behavior, which does not establish official default-on PTC parity. Enabled search and default/error parity remain gaps; unrelated native utilities are not implicitly removed. | -| Agent service-origin MCP | Implemented Codex/Claude HTTP `none` profiles, anonymous/static bearer, scoped Vault selection and native `mcp_call` Items. [Configuration and qualification limits](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/README.md#http-mcp-execution), [profiles](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/engine/profile.go). | This closure batch does not requalify MCP/provider combinations. MiniMax, self-hosted/hosted service-origin MCP, OAuth, nonempty inline headers/metadata and other transports/origins reject. Claude requires a static connected inventory; original MCP-envelope fidelity and continuing server health are unverified. | -| Environment Plugin MCP/native tools | Separate [Docker Plugin transport matrix](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-templates.md#environment-origin-mcp-plugins) and [V1 deployment evidence](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/user-managed-runtime-v1.md). Native tools stay within the existing colocated Runtime. | Plugin MCP is not Agent service-origin MCP or a public function action. No new optional cross-product is qualified here. Native utility inventories need not be identical. | -| Required action: `function_call` | Session GET/list and Session SSE expose persisted `{arguments,call_id,name,turn_id,type}`. Actions remain pending until native application or cancellation/terminal settlement. [Projection](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/store/function_state.go), [confirmation](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/store/function_results.go); real handling F1/F2/M2/S1/S2/D1; explicit client disconnect/query/reconnect: F3. | Function-call history is not pending-state authority. Client reconnect does not replay events or redo external application effects. | -| Required action: `environment_connection` | Offline waiting input exposes `{environment_id,type}` before Turn creation. Connect the exact Session Environment through our scoped daemon enrollment. Three-harness Docker/E2B execution: E1; explicit initial pending-input/query/connection/native completion: E2; expiry: [controlled initial-input test](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/tests/official_self_hosted_initial.py). | Idle offline Sessions without pending input request nothing. Registration alone is not connectivity or native readiness. Stock `exec-server`/Noise transport is outside the approved V1 route. Exact upstream registration/action-removal timing remains unverified. | -| Stream disconnect and execution loss | SSE is live-only. Reconnect, buffer events, retrieve Session/Turn/Items and deduplicate Item IDs. Worker recovery fails previously claimed work without replay; queued work can remain. [Recovery contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md#public-execution-admission), [connection reconciliation](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/store/environment_connection_recovery.go). | Client disconnect and daemon/Core loss are different cases. F3 qualifies client-stream loss and continued pending function handling; F2 qualifies native process loss with an unapplied saved result. Neither promises replay or recovery of unknown external effects. | +| Deferred function discovery | Type-only `tool_search` plus mixed eager/deferred functions: Claude SDK 0.3.269/native 2.1.269, Kimi K3, single Agent, medium, `none`, text results; text and PNG input D1. Native provider observations establish lazy schema loading. [Contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/tool-search.md). | A repeated `tool_search` is a protocol error. Codex/MiniMax discovery, search-only/missing-search, workspace, MCP, structured-output and Subagent combinations remain gaps. Opaque native policy changes lack a reliable pre-input deferral signal. Saved tools include `tool_search`; the pinned Session response union excludes it. Exact hosted projection is unverified. | +| Explicit disabled search/PTC | Saved and inline `web_search.mode=disabled` and `programmatic_tool_calling.enabled=false`; shared native controls on initial and cold execution. All three harnesses on `none`: P1, including native inventory/control evidence. [Contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/tool-policy.md), [parser](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/api/disabled_tools.go). | Unsupported explicit enablement rejects at Session admission; saved Agents keep every pinned search mode as resource data (TV-05), and Sessions from such Agents reject unless they replace the tools. A repeated `web_search` is a protocol error. Omission retains approved native behavior, which does not establish official default-on PTC parity. Enabled search and default/error parity remain gaps; unrelated native utilities are not implicitly removed. | +| Agent service-origin MCP | Implemented Codex/Claude HTTP `none` profiles, anonymous/static bearer, scoped Vault selection and native `mcp_call` Items. [Configuration and qualification limits](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/README.md#http-mcp-execution), [profiles](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/engine/profile.go). | This closure batch does not requalify MCP/provider combinations. MiniMax, self-hosted/hosted service-origin MCP, OAuth, nonempty inline headers/metadata and other transports/origins reject. Claude requires a static connected inventory; original MCP-envelope fidelity and continuing server health are unverified. | +| Environment Plugin MCP/native tools | Separate [Docker Plugin transport matrix](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-templates.md#environment-origin-mcp-plugins) and [V1 deployment evidence](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/user-managed-runtime-v1.md). Native tools stay within the existing colocated Runtime. | Plugin MCP is not Agent service-origin MCP or a public function action. No new optional cross-product is qualified here. Native utility inventories need not be identical. | +| Required action: `function_call` | Session GET/list and Session SSE expose persisted `{arguments,call_id,name,turn_id,type}`. Actions remain pending until native application or cancellation/terminal settlement. [Projection](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/store/function_state.go), [confirmation](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/store/function_results.go); real handling F1/F2/M2/S1/S2/D1; explicit client disconnect/query/reconnect: F3. | Function-call history is not pending-state authority. Client reconnect does not replay events or redo external application effects. | +| Required action: `environment_connection` | Offline waiting input exposes `{environment_id,type}` before Turn creation. Connect the exact Session Environment through our scoped daemon enrollment. Three-harness Docker/E2B execution: E1; explicit initial pending-input/query/connection/native completion: E2; expiry: [controlled initial-input test](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/tests/official_self_hosted_initial.py). | Idle offline Sessions without pending input request nothing. Registration alone is not connectivity or native readiness. Stock `exec-server`/Noise transport is outside the approved V1 route. Exact upstream registration/action-removal timing remains unverified. | +| Stream disconnect and execution loss | SSE is live-only. Reconnect, buffer events, retrieve Session/Turn/Items and deduplicate Item IDs. Worker recovery fails previously claimed work without replay; queued work can remain. [Recovery contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md#public-execution-admission), [connection reconciliation](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/store/environment_connection_recovery.go). | Client disconnect and daemon/Core loss are different cases. F3 qualifies client-stream loss and continued pending function handling; F2 qualifies native process loss with an unapplied saved result. Neither promises replay or recovery of unknown external effects. | ## Required-action recovery contract @@ -52,7 +52,7 @@ application already performed the function, keep and submit that saved result; do not run an external effect again merely because an action remains visible. Core's acknowledgement boundary is native application. For an environment action, connect its exact Environment using the existing enrollment authorization. -[Connection observations](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/store/environment_connections.go) +[Connection observations](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/store/environment_connections.go) fence stale generations; transport connection can clear pre-Turn activity before native preparation. Neither registration nor an action disappearing proves that a model ran successfully. Query the matching Turn and Items for its outcome. @@ -61,8 +61,8 @@ Two timing questions remain open. Repeated `requires_action` notification order and acknowledgement timing are local implementation choices, not proven upstream semantics. Also, an admitted result that is cancelled before native observation can remain internally saved without a public output Item or `item.added`. -[Item publication](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/internal/store/item_projection.go) and -[the existing coverage record](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md#public-function-configuration) preserve +[Item publication](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/internal/store/item_projection.go) and +[the existing coverage record](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md#public-function-configuration) preserve this gap. Successful retry cannot manufacture the missing observation or establish that the model consumed the result. @@ -82,12 +82,12 @@ not a claim that every historical workflow was rerun at the current baseline. | M2 | `20260922/workspace-images/validation-summary.json`; Codex `public-run-_lr5uiy_`, Claude `public-run-sb65p9e9`; candidate `2c295116d66ed9aafc808ec49a8cb6dcb5c674c2`, merge `1adb2489bc3415039440224e9a7905c53e68a557` ([#17](https://github.com/MiniMax-AI/parsar-core/pull/17)). Kimi K3, Codex 0.153.4, Claude SDK 0.3.269/native 2.1.269; Docker seven-Turn image/workspace workflow. Codex's receipt conclusion is superseded by F2. | | F1 | `20260922/function-result-images/validation-summary.json`, `validated/function-image-public-2567456666/public.json`; candidate `d3cdb225bc7628b968b76c7e1b400101894ce8cd`, merge `fd23f169e1ede1b2f1c39a1d9dcce71886e3c006` ([#16](https://github.com/MiniMax-AI/parsar-core/pull/16)). Claude SDK 0.3.269/native 2.1.269 with Kimi K3 on `none`; success PNG/JPEG, failed text, retries, cancellation, history continuation. Earlier Codex transport-only evidence does not qualify current receipts. | | F2 | `20260922/function-result-receipts/validation-summary.json`, `candidate-public-owner-sdk.log`; hosted `public-run-4fn70foy` (104.13s), `none` `function-image-public-945075091` (83.918s). Candidate `6dceb98e1c56469fcc70cd82025ce77fc870ed3d`, merge `206474a4c10901442b1faa094281bfb5559082b5` ([#20](https://github.com/MiniMax-AI/parsar-core/pull/20)); real Kimi, Codex 0.153.4. Qualifies native receipts, not completed provider consumption. | -| F3 | `20260922/execution-tools-closure/pending-actions/validation-summary.json`; Codex `codex/public-run-8gexf5a8` (65.58s), Claude `claude_sdk/public-run-5f2tr7e_` (74.33s), production source `206474a`. The same [public verifier](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/tests/official_pending_actions_native.py) checks disconnected-client pending queries, success/error/cancel, original results, target isolation, retry/conflict and no implicit call replay with real Kimi on Docker. No Core restart or native-loss injection is claimed by these runs. | +| F3 | `20260922/execution-tools-closure/pending-actions/validation-summary.json`; Codex `codex/public-run-8gexf5a8` (65.58s), Claude `claude_sdk/public-run-5f2tr7e_` (74.33s), production source `206474a`. The same [public verifier](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/tests/official_pending_actions_native.py) checks disconnected-client pending queries, success/error/cancel, original results, target isolation, retry/conflict and no implicit call replay with real Kimi on Docker. No Core restart or native-loss injection is claimed by these runs. | | S1 | `20260922/structured-output/acceptance-summary.json`, `structured-public-1905281777/public.json`; real candidate `7f533d46c472f3cdf7c16ce9471c225a2ba7eded`, merge `8716e699dbc34904499c94b6b0b03857bbebfaad` ([#11](https://github.com/MiniMax-AI/parsar-core/pull/11)). Claude/Kimi `none` schema/function execution and cold continuation. Schema-specific active steering was not separately qualified here. | | S2 | `20260922/hosted-structured-output/validation-summary.json`, `validated/public-inline.log`, `public-run-qvq3csf1` (186.09s); candidate `f6d2c3f2a1dc2ff5f177213bd2b20d78496011d0`, merge `aa85d8ecc60e0a8b7f2494b73caa81216ee197e2` ([#18](https://github.com/MiniMax-AI/parsar-core/pull/18)). Claude SDK 0.3.269/native 2.1.269, real Kimi; Docker saved/inline schema, prepared/active input, native files and four completed structured answers. | | D1 | `20260922/deferred-tools/tool-search-public-2039155387/public.json`, `public-image-isolated-tests.log` (86.71s); merge `178507ae7a63d4068e1e82bef9cd256ba398ae00` ([#13](https://github.com/MiniMax-AI/parsar-core/pull/13)). Claude SDK 0.3.269/native 2.1.269/Kimi K3 `none`; text results with initial/active PNGs, cancellation and cold continuation. `claude-native-1790052750` separately records provider schema inventories and native feasibility. | | P1 | `20260922/tool-policy/acceptance.json`, `server-evidence/tool-policy-{codex-2495445746,claude_sdk-2539346823,mcode-3747575401}/public.json`; candidate `eeb432c7495265ae3a9309e32f5b4323611c3245`, merge `4b8754a6eda6696d9b18eb8d583953ac16c6800f` ([#14](https://github.com/MiniMax-AI/parsar-core/pull/14)). Codex/Claude Kimi K3, MiniMax M2.7; four `none` Sessions per harness, native disable controls and cold continuation. | -| E1 | `20260921/self-hosted-onboarding/{source-candidate.json,source-verified.json,live,live-e2b}`; candidate `8f0cd2530d7b58cb7fb3ea124a1fc43dacecca36`. [Exact Docker/E2B run paths and limits](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/user-managed-runtime-v1.md#evidence-and-verification-boundaries): Codex Kimi K3 Responses, Claude Kimi K3 Anthropic-compatible API, MiniMax M2.7, immutable Linux amd64 Runtime builds. Native execution, restart/history, cancellation, Files/Artifacts and isolation; shared credential lifecycle evidence is not a separate live rotation run for every E2B profile. | +| E1 | `20260921/self-hosted-onboarding/{source-candidate.json,source-verified.json,live,live-e2b}`; candidate `8f0cd2530d7b58cb7fb3ea124a1fc43dacecca36`. [Exact Docker/E2B run paths and limits](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/user-managed-runtime-v1.md#evidence-and-verification-boundaries): Codex Kimi K3 Responses, Claude Kimi K3 Anthropic-compatible API, MiniMax M2.7, immutable Linux amd64 Runtime builds. Native execution, restart/history, cancellation, Files/Artifacts and isolation; shared credential lifecycle evidence is not a separate live rotation run for every E2B profile. | | E2 | `20260922/execution-tools-closure/environment-actions/result.json` contains Codex/Claude Kimi passes (13 checks each) and the retained MiniMax direct-network failure; `mcode-relay-retry/result.json` separately passes MiniMax M2.7 (13 checks) after correcting provider routing. Source `206474a`; same shared SDK/raw workflow, isolated Core/PostgreSQL and user-managed Docker. Covers initial creation SSE, client disconnect, exact pending Environment/no Turn or Items, scoped enrollment, one actual completed model Turn and creation retries preserving history. This is connection-action qualification, not another full lifecycle or E2B regression. | Controlled tests establish parser, projection, atomicity and race behavior; native @@ -121,4 +121,4 @@ remain separate limitations. They do not authorize silently relabeling an unprov principal workflow as deferred. OAuth, full Usage, release publication, new tool kinds and new execution loops are outside this closure batch. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/execution-tools.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/execution-tools.md) diff --git a/apps/docs/content/docs/api-reference/sessions.mdx b/apps/docs/content/docs/api-reference/sessions.mdx index 2aa51289f..5de4d4f81 100644 --- a/apps/docs/content/docs/api-reference/sessions.mdx +++ b/apps/docs/content/docs/api-reference/sessions.mdx @@ -51,26 +51,27 @@ _openapi: startup before the first native Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and - native OAuth login remain unsupported. The self_hosted profile - requires Codex, an absolute workspace_directory and empty - capability_directories, with optional non-deferred function tools and - HTTP MCP using service origin, optionally authenticated by the - attached Vault rules. Remote MCP and remote Bearer authentication each - require separately advertised combination support; old peers cannot - receive unsupported work. Omitted/null capability_directories use the - empty-list default; self_hosted requires configured execution plus - executor registry. Claude SDK currently requires medium verbosity and - object-root function schemas. It supports anonymous or attached - static-bearer service-origin HTTP MCP on none with boolean required - and separately advertised MCP/bearer/required runtime support. - Required servers must be connected before the first native input is - released; pending or failed startup rejects execution. The shared - Vault selection and immutable binding rules apply; unsupported native - labels/tool names reject before persistence. An attached Vault with no - matching credential may remain anonymous; missing keys or failed - credential lookup/decryption never fall back to anonymous execution. - Omitted stream defaults to false; stream and agent_id cannot be null. - Metadata may be null; non-string values and limit violations return + native OAuth login remain unsupported. The self_hosted profile uses a + qualified native harness, a clean absolute workspace_directory and + optional absolute local capability_directories prepared by Runtime, + with optional non-deferred function tools and HTTP MCP using service + origin, optionally authenticated by the attached Vault rules. Remote + MCP and remote Bearer authentication each require separately + advertised combination support; old peers cannot receive unsupported + work. Omitted/null capability_directories use the empty-list default; + self_hosted requires configured execution plus executor registry. + Claude SDK currently requires medium verbosity and object-root + function schemas. It supports anonymous or attached static-bearer + service-origin HTTP MCP on none with boolean required and separately + advertised MCP/bearer/required runtime support. Required servers must + be connected before the first native input is released; pending or + failed startup rejects execution. The shared Vault selection and + immutable binding rules apply; unsupported native labels/tool names + reject before persistence. An attached Vault with no matching + credential may remain anonymous; missing keys or failed credential + lookup/decryption never fall back to anonymous execution. Omitted + stream defaults to false; stream and agent_id cannot be null. Metadata + may be null; non-string values and limit violations return invalid_request_error with a metadata or metadata. param. The inline agent uses the Agent create configuration validation with agent.-prefixed params, reported before the input requirement and diff --git a/apps/docs/content/docs/bootstrap-projects-keys.mdx b/apps/docs/content/docs/bootstrap-projects-keys.mdx index c99088ea3..2194e6472 100644 --- a/apps/docs/content/docs/bootstrap-projects-keys.mdx +++ b/apps/docs/content/docs/bootstrap-projects-keys.mdx @@ -69,7 +69,7 @@ and display only. Keep deployment, application, node and provider credentials ou Installation creates no Project or application key. An administrator creates a Project and issues named keys on the console's **Projects and keys** page, which -uses the [administrator API](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md), or with that +uses the [administrator API](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md), or with that API directly. The Project owns one tenant and one principal; its keys share assets and permissions. Writes retain each key's provenance. All Projects and application keys live in @@ -98,4 +98,4 @@ management request points to the console's Core key or Core connection. A resour deletion conflict must remain visible; it does not authorize an execution call. Node and daemon `/api/v1` routes and native Runtime interfaces retain their own authentication. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/web/core-connection.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/web/core-connection.md) diff --git a/apps/docs/content/docs/concepts.mdx b/apps/docs/content/docs/concepts.mdx index e48f1a37f..6ff1c2e41 100644 --- a/apps/docs/content/docs/concepts.mdx +++ b/apps/docs/content/docs/concepts.mdx @@ -83,12 +83,12 @@ ownership. No request bodies, secrets or file contents enter audit records. Do not add product users, RBAC, cross-Project shared assets, administrator execution, or old private-protocol compatibility to this management model. Existing public -execution and Runtime ownership rules remain in [CONTRIBUTING.md](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/CONTRIBUTING.md). +execution and Runtime ownership rules remain in [CONTRIBUTING.md](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/CONTRIBUTING.md). Native failure classification is adapter-owned and uses finite structured native values. Optional Runtime error metadata is normalized once and retained through normal delivery and journal draining; it never replaces Core terminal authority, cancellation receipts, Usage or native identity. See -[native failure classification](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/native-error-classification.md). +[native failure classification](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/native-error-classification.md). -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/design-principles.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/design-principles.md) diff --git a/apps/docs/content/docs/configure.mdx b/apps/docs/content/docs/configure.mdx index 798ea32fe..360b9717a 100644 --- a/apps/docs/content/docs/configure.mdx +++ b/apps/docs/content/docs/configure.mdx @@ -116,7 +116,7 @@ output or in Core's settings snapshot. Model providers are not process settings; The schema is -[`deploy/install/config.schema.json`](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/deploy/install/config.schema.json); each +[`deploy/install/config.schema.json`](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/deploy/install/config.schema.json); each installation keeps a copy in `generated/config.schema.json` for editors. Core serves the non-secret settings snapshot, with the path of `config.json` and the apply command, at `GET /core/v1/installation`. @@ -145,7 +145,7 @@ Which harnesses are enabled, and the default one, are process settings Exactly one provider serves the deployment: Docker, microsandbox or E2B. The deployment specification holds the per-sandbox resources and, for Docker and microsandbox, the Runtime release. Web's setup proposes the Standard size from -[`standard-sizes.json`](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/apps/web/src/features/sandbox/standard-sizes.json) and +[`standard-sizes.json`](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/apps/web/src/features/sandbox/standard-sizes.json) and derives Small and Large from it; the installer applies the bundle's copy of the same file. The database owns the saved values. @@ -183,8 +183,8 @@ Current Runtime generation coexistence and ownership-scoped garbage collection remain supported. They do not upgrade the installed node program or convert an old installation. History stays, and existing Sessions never move between providers. See the [nodes guide](/hosted-providers#change-the-sandbox-backend-or-size), -[operator reference](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md#removal-and-reset) -and [deployment contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/sandbox-deployment.md). +[operator reference](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md#removal-and-reset) +and [deployment contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/sandbox-deployment.md). ### Node capacity @@ -211,7 +211,7 @@ Session freezes the provider it starts with, so a later change affects only new Sessions. Self-hosted Sessions never use it. A hosted or self-hosted Session with no provider is refused with 400 `model_provider_required`. See [model providers](/quickstart#model-providers) and the -[model execution contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/model-execution.md#deployment-defaults). +[model execution contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/model-execution.md#deployment-defaults). The operator file `AGENTS_API_EXECUTION_OPTIONS_FILE` is retired; see [installation version policy](/troubleshooting#installation-version-policy). @@ -231,7 +231,7 @@ A Web-only installation has only `secrets/core.key`, a copy of its Core's key. Core reads only its environment. The installer renders `generated/core.env` from `config.json`; if you run Core yourself (see -[Maintainers and advanced deployments](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/maintainers.md)), set these variables. Compose +[Maintainers and advanced deployments](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/maintainers.md)), set these variables. Compose loads the file with `env_file` and systemd with `EnvironmentFile`, so Compose must be 2.26.0 or newer. @@ -246,7 +246,7 @@ loads the file with `env_file` and systemd with `EnvironmentFile`, so Compose mu | `OAC_INSTALLATION_ID` | `state.json`; pinned to the database | | `OAC_SETTINGS_FILE` | `generated/settings.json`, the snapshot Core serves at `GET /core/v1/installation`; Core does not act on it | | `OAC_EXECUTION_CONCURRENCY`, `OAC_DEFAULT_HARNESS`, `OAC_HARNESSES`, `OAC_WRITE_AUDIT_RETENTION`, `OAC_OAUTH_TRUSTED_ORIGINS` | The matching `core.*` settings | -| `OAC_HISTORY_SETTINGS_FILE` | `generated/runtime-history.json` when `core.runtime_history` is set; see the [history contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime-history-api.md) | +| `OAC_HISTORY_SETTINGS_FILE` | `generated/runtime-history.json` when `core.runtime_history` is set; see the [history contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime-history-api.md) | | `OAC_LOG_LEVEL`, `OAC_LOG_FORMAT`, `OAC_LOG_ADD_SOURCE` | `log.*`; Web reads the same three | | `OAC_E2B_STATE_DIR`, `OAC_E2B_PROVIDER_BIN` | The E2B receipt directory and, for native Core, the bundled helper. The E2B key and template live in the database | @@ -268,4 +268,4 @@ paths must be canonical absolute paths without control characters, quotes, backs or wildcards. Keep the installation ID and the database together; Core refuses a missing installation ID when its database already has a deployment. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/configuration.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/configuration.md) diff --git a/apps/docs/content/docs/console.mdx b/apps/docs/content/docs/console.mdx index 544c13e9a..4a39b51de 100644 --- a/apps/docs/content/docs/console.mdx +++ b/apps/docs/content/docs/console.mdx @@ -27,7 +27,7 @@ management client, and it sends nothing to `/v1`. | Platform | Projects and keys, Nodes, System | Project and key lifecycle; sandbox deployment and nodes; System: the installation's public address, API base URL, ID and source commit (read-only), each harness's default model provider (write-only key) beside its read-only startup state, the sandbox configuration every Project shares: provider, sandbox size, Runtime or E2B template build, idle suspension and reset, and Core's config.json startup settings with where to change them | Missing data is shown as missing (—), never as zero. How each figure is read and -bounded is recorded in [management interface coverage](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/web/protocol-coverage.md). +bounded is recorded in [management interface coverage](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/web/protocol-coverage.md). ## Management scope @@ -60,15 +60,15 @@ only the console server sends it to Core. - [Connection and authentication](/bootstrap-projects-keys) - [Architecture and ownership](/execution-model) -- [Management interface coverage](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/web/protocol-coverage.md) -- [Frontend handoff and acceptance](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/web/roadmap.md) -- [React application](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/apps/web/README.md) +- [Management interface coverage](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/web/protocol-coverage.md) +- [Frontend handoff and acceptance](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/web/roadmap.md) +- [React application](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/apps/web/README.md) -The [administrator API contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md) defines -management routes and resource behavior. The [public API contracts](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md) +The [administrator API contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md) defines +management routes and resource behavior. The [public API contracts](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md) define the separate application interface. See the [design principles](/concepts) -for ownership and [contributor guide](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/CONTRIBUTING.md) for required checks. +for ownership and [contributor guide](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/CONTRIBUTING.md) for required checks. -OpenAgentCore Web is available under the [MIT License](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/LICENSE). +OpenAgentCore Web is available under the [MIT License](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/LICENSE). -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/web/README.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/web/README.md) diff --git a/apps/docs/content/docs/environments-and-files.mdx b/apps/docs/content/docs/environments-and-files.mdx index ff296f227..896627ce9 100644 --- a/apps/docs/content/docs/environments-and-files.mdx +++ b/apps/docs/content/docs/environments-and-files.mdx @@ -3,16 +3,19 @@ title: "Environments and files" description: "Execution placement, Environment resources and workspace ownership." --- -This assessment covers the fixed [Python SDK contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/upstream.json), with partial +This assessment covers the fixed [Python SDK contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/upstream.json), with partial coverage. Core-managed Docker runs the three qualified native harnesses. V1 user-managed `self_hosted` enrollment uses the same colocated Runtime for Codex, -Claude SDK and MiniMax at `/workspace`; see its [real deployment qualification](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/user-managed-runtime-v1.md). +Claude SDK and MiniMax. `/workspace` names the packaged workspace; a self-hosted +Session may instead select its exact canonical physical directory. The +[recorded deployment qualification](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/user-managed-runtime-v1.md) retains its historical +source, paths and tested capability scope. For hosted compute, the deployment selects E2B, Docker or microsandbox through -[Hosted Sandbox Manager](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md). +[Hosted Sandbox Manager](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md). For the separate caller-managed E2B path, the user owns allocation, renewal and -cleanup through the official SDK and [Runtime packaging](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/deploy/e2b/README.md). -[Templates](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-templates.md) remain a hosted-only resource path; -[Files](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-files.md) reuse the exact authorized local workspace. +cleanup through the official SDK and [Runtime packaging](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/deploy/e2b/README.md). +[Templates](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-templates.md) remain a hosted-only resource path; +[Files](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-files.md) reuse the exact authorized local workspace. The internal Store now owns a durable Environment association for newly created `self_hosted` and `openai_hosted` snapshots, atomically with Session creation. @@ -30,15 +33,15 @@ readiness. Rotation/revocation, Session deletion and ownership loss deny further access without promising immediate cessation of native effects. Native history remains local to the bound Runtime and cannot be replaced on retry. There is no registry/Noise relay or transient service-side harness credential. -See the [enrollment guide](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/README.md#user-managed-runtime-enrollment). +See the [enrollment guide](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/README.md#user-managed-runtime-enrollment). ## Basic public Docker-hosted profile An explicitly configured default managed Docker provider enables `type=openai_hosted` for the qualified Codex, Claude Code and MiniMax Code profiles. Each uses the same Runtime lifecycle and workspace interfaces with its own native adapter/isolation. -See the [engine profile guides](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md#public-engine-profiles) for setup and limits. -The standalone [operator configuration](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/deploy/codex/README.md#standalone-operator-configuration) +See the [engine profile guides](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md#public-engine-profiles) for setup and limits. +The standalone [operator configuration](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/deploy/codex/README.md#standalone-operator-configuration) selects the qualified immutable Runtime image; advertised capabilities alone do not enable admission. An idle or initial-text creation commits Session, Environment and retry identity before the existing leased Worker provisions its allocation. @@ -48,9 +51,14 @@ an existing allocation's Create. Omitted/null network defaults to enabled. Enabled, disabled and exact-host restricted policies use the same qualified image with adapter-selected immutable native policy. Templates and inline configuration share initial files, env, packages, ordered setup -and inline or tenant-owned referenced Skills through the hosted initializer. +and inline or tenant-owned referenced Skills through ordered managed initialization. +The same authenticated daemon handles initial files, tool configuration, packages, +setup, Skill/Plugin import and directory snapshot finalization through +`runtime_prepare`. Providers place, create, bootstrap, inspect, renew and reclaim +compute. Package installation and setup retain enabled provisioning network access; +the requested network policy constrains native execution after setup. Unsupported hostname forms and installation combinations reject explicitly; see -the [Template coverage and limits](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-templates.md). Empty/null installation +the [Template coverage and limits](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-templates.md). Empty/null installation defaults produce safe empty metadata, not a live workspace inventory. Service-origin hosted MCP remains unsupported; Environment Plugin MCP has its own qualified transport matrix. @@ -70,18 +78,19 @@ semantics; this profile does not establish complete Environment compatibility. The application creates, renews and destroys its E2B sandbox through the official SDK. It deploys the shared Runtime, then enrolls that Runtime into a `self_hosted` Session. Core neither keeps an E2B allocation nor issues Provider renew/kill calls. -The [E2B guide](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/deploy/e2b/README.md) owns packaging and +The [E2B guide](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/deploy/e2b/README.md) owns packaging and user-side lifecycle instructions. Expiry or lost workspace/history must not trigger -transparent replacement or replay. The [new enrollment qualification](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/user-managed-runtime-v1.md) +transparent replacement or replay. The [new enrollment qualification](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/user-managed-runtime-v1.md) records its own real deployment evidence. -The [prior E2B qualification](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md#e2b-v1-qualification) records a historical +The [prior E2B qualification](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md#e2b-v1-qualification) records a historical Core-managed route; it does not establish this caller-managed enrollment result. Current deployment-managed E2B is a separate hosted configuration in the manager guide. ## Initial public self-hosted profile Create a Session with `environment.type=self_hosted`, -`workspace_directory: "/workspace"` and omitted/null/empty `capability_directories`. Creation +a clean absolute `workspace_directory` and optional absolute local +`capability_directories`. `/workspace` maps to the Runtime-bound workspace. Creation accepts initial text as a string or ordered user-message array. Omitted/null input creates no Turn or connection action. Configured execution, an enabled harness and the exact local profile are validated @@ -111,7 +120,7 @@ connection alone is not readiness. Retries preserve identity and the five-minute database deadline. HTTP disconnect retains the reservation. Local expired/cancelled outcomes return 409, lost ownership 503, and deletion 404; exact hosted error status/body and pending-input crash recovery are unverified. See the -[canonical wait rules](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/CONTRIBUTING.md#environment-ownership-and-placement). +[canonical wait rules](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/CONTRIBUTING.md#environment-ownership-and-placement). Cancellation-only batches use the existing locked admission and native delivery. An idle cancellation creates no Turn, and retry identity preserves the original @@ -125,9 +134,15 @@ existing function parser and remain fixed through native preparation and continu output/error field presence and ordered content keep their existing semantics. Retries retain the original call, including during later work. New results cannot bypass pending input. Function callbacks do not populate Environment installations. -Mixed events, non-text input and nonempty capability directories remain unsupported. -The public field types are unchanged; `/workspace` is the V1 deployment limit, not -an upstream schema change. `remote_url` is the configured daemon WebSocket URL, +Mixed events and non-text input remain unsupported. Local capability directories +use the same Runtime parser and installed snapshot as managed Skills and Plugins; +preparation must finish before native execution. Reconnect reuses installed bytes, +while new Sessions capture their own sources. Directory aliases and protected Runtime +state are rejected. The current implementation uses the packaged Linux Runtime +layout; cross-platform installation and isolation are not part of this profile. +The public field types are unchanged. Enrollment must match either the `/workspace` +logical alias or the exact canonical directory bound by the Runtime; selecting an +arbitrary path does not grant access. `remote_url` is the configured daemon WebSocket URL, returned unchanged. This is our private connection contract and does not claim stock `exec-server` compatibility. Service-origin HTTP MCP is explicitly rejected on `self_hosted`; `none` MCP and hosted Template Plugin MCP retain their own scope. @@ -138,6 +153,49 @@ exercise fixed SDK/raw HTTP creation/input, actual native tools, Files/Artifacts second-Turn history, Core/Runtime restart, cancellation and credential rotation/ revocation/deletion on each declared deployment. +## Runtime capability preparation + +Core freezes resource versions, metadata and source selections. Managed initialization +keeps initial files and tool configuration before Runtime Skill/Plugin import, +then runs packages and ordered setup before Runtime snapshots local capability +directories. All steps use the common `runtime_prepare` protocol; Providers do +not execute Core initialization commands. The common runner depends on neutral +Environment/Session identity and a Runtime peer, not Provider, deployment or OS. Self-hosted input +accepts only local `capability_directories` with `workspace_directory`, not managed +Skill/Plugin archive fields or a hosted Template. Local directory discovery does not +populate the public API-managed `skills` or `plugins` installation arrays. + +Both origins use the packaged Linux Runtime's shared parser and protected +`installed.json`. Its default root is `/environment/initialization/capabilities`; +the operator may freeze another root with `OAC_RUNTIME_CAPABILITY_DIRECTORY`, kept +separate from workspace and private Runtime state. Core and public requests cannot +select this installation root. The manifest binds one +Session and Environment to the source-selection digest. The admitted asynchronous +preparation owner verifies or creates it before native execution, including for an +empty selection. Reconnect and replacement executors reuse installed contents; +source edits are observed only by a new Session with its own Environment and fresh +snapshot. A private completion record retains only the operator installation root, +so loss of a completed snapshot cannot trigger source recapture. Missing, partial or +conflicting snapshots fail without automatic repair or replay. Adapters receive only the resulting Runtime-owned Skill paths and MCP +declarations. Files access retains its separate readiness and authorization. + +Resource management owns allocation and Environment creation, renewal and reclamation. +Executor close, Turn cancellation and transport loss preserve the installed snapshot, +workspace and allocation. Reclamation is an explicit operation coordinated with active +work; a disconnected socket is not proof that native effects have stopped. + +Source selections accept portable absolute Unix, Windows drive and UNC paths; +Core does not resolve these paths on its own host. The daemon applies its local +path and access checks. Initial files and setup working directories use logical +`/workspace` paths. Harness differences remain in adapters. + +The protocol is platform-neutral, but this implementation retains the existing Linux +helpers, protected initialization storage and isolation requirements. It does not provide +a new native-host installer, another platform layout, or Windows/macOS isolation. +This implementation description does not extend historical real-deployment acceptance; +new managed and self-hosted capability coverage requires its own evidence. See the +[canonical preparation rules](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/CONTRIBUTING.md#shared-runtime-capability-preparation). + ## Contract inventory Paths below follow the SDK resource methods, before the service's `/v1` prefix. @@ -191,7 +249,7 @@ from Session and Turn status. ## Ownership and placement decision -The canonical [architecture rules](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/CONTRIBUTING.md#environment-ownership-and-placement) +The canonical [architecture rules](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/CONTRIBUTING.md#environment-ownership-and-placement) keep Environment lifecycle common while leaving process placement and native transport to adapters. Logical ownership does not require a machine per object. @@ -215,4 +273,4 @@ native history independently of disposable compute, or prove native restoration; never treat an Environment ID as a filesystem or history backup. Do not silently move an existing Session away from its bound device. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environments.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environments.md) diff --git a/apps/docs/content/docs/execution-model.mdx b/apps/docs/content/docs/execution-model.mdx index 54e8f52ed..5fe1f60d6 100644 --- a/apps/docs/content/docs/execution-model.mdx +++ b/apps/docs/content/docs/execution-model.mdx @@ -10,7 +10,7 @@ Agents API independently with their own Project keys. The management backend, `AdminClient` and the React console built on them are implemented. The [design principles](/concepts) define identity and authority. -The [administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md) defines exact +The [administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md) defines exact routes, payloads, pagination and audit records. ## Request boundaries @@ -106,6 +106,6 @@ comes from the deployment's allocations; the hosted Runtime total counts hosted observation records across projects and reported lifecycle states. These sources have different coverage and refresh independently, so the console does not infer resource retention or cleanup from their difference. -Native execution ownership remains governed by [CONTRIBUTING.md](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/CONTRIBUTING.md). +Native execution ownership remains governed by [CONTRIBUTING.md](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/CONTRIBUTING.md). -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/web/architecture.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/web/architecture.md) diff --git a/apps/docs/content/docs/hosted-providers.mdx b/apps/docs/content/docs/hosted-providers.mdx index c781c135a..53ba8ee54 100644 --- a/apps/docs/content/docs/hosted-providers.mdx +++ b/apps/docs/content/docs/hosted-providers.mdx @@ -10,7 +10,7 @@ With E2B you need no nodes. Self-hosted executors, which applications run for th own Sessions, are a different thing; see [Self-hosted executors](/self-hosted-execution). You add a node by generating a command in Web and pasting it on the host. The -[operator reference](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md) covers the +[operator reference](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md) covers the node protocol, manual registration, placement and failure handling. ## Before you add a node @@ -187,7 +187,7 @@ The installation ID is in the command (`--installation-id`) and on the **System* 1. In Web, open **Nodes** and choose **Remove node** on the node's page, or **Remove** in its list row, then **Confirm removal**. Core refuses while the node still holds sandboxes, snapshots or pending cleanup; let them finish, or - [archive their Sessions](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md#removal-and-reset) + [archive their Sessions](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md#removal-and-reset) through the Core API. Removal is permanent: the host can come back only as a new node. 2. Web then shows **Clean up the host** with the uninstall command. Run it on the host: @@ -290,7 +290,7 @@ files. Shared daemon images remain; a host administrator may remove them only af confirming no installation still needs them. Microsandbox image cleanup remains scoped to the installation's private native store and verified ownership receipts. -The [operator reference](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md#removal-and-reset) +The [operator reference](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md#removal-and-reset) describes reset and API behavior. Same-team E2B edits have a separate online Core API path and do not involve node installation. @@ -303,4 +303,4 @@ convert or delete it. Preserve the old installation, Runtime resources and histo provision a fresh node separately through **Add node**. The current installer does not adopt historical resources or remove another installation's state. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/getting-started/nodes.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/getting-started/nodes.md) diff --git a/apps/docs/content/docs/index.mdx b/apps/docs/content/docs/index.mdx index 00c7aa681..4159e280e 100644 --- a/apps/docs/content/docs/index.mdx +++ b/apps/docs/content/docs/index.mdx @@ -19,10 +19,10 @@ applications call Core with. | [API reference](/public-api) | Developers | The `/v1`, `/core/v1` and `/api/v1` namespaces and their contracts | Deeper references: the -[nodes and sandbox backends](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md) +[nodes and sandbox backends](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md) reference for operators, the -[protocol coverage and native differences](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md), and -[maintainers and advanced deployments](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/maintainers.md) for building distributions +[protocol coverage and native differences](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md), and +[maintainers and advanced deployments](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/maintainers.md) for building distributions and running Core without the installer. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/getting-started/README.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/getting-started/README.md) diff --git a/apps/docs/content/docs/install.mdx b/apps/docs/content/docs/install.mdx index e32668a5b..c52a6237f 100644 --- a/apps/docs/content/docs/install.mdx +++ b/apps/docs/content/docs/install.mdx @@ -140,7 +140,7 @@ A deployment runs its sandboxes on exactly one backend: | `none` | Nothing yet | Choose the backend on the **Nodes** page in Web | The Standard sizes come from the bundle's copy of Web's -[`standard-sizes.json`](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/apps/web/src/features/sandbox/standard-sizes.json). The +[`standard-sizes.json`](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/apps/web/src/features/sandbox/standard-sizes.json). The installer saves the choice through the same administrator API that Web uses once the services are healthy. If Core refuses it, for example because E2B rejects the key, the installer prints Core's message and exits with an error; the services keep @@ -164,7 +164,7 @@ it installs anything: ``` Prepare the template build with the -[E2B guide](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/deploy/e2b/README.md). To change the backend or +[E2B guide](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/deploy/e2b/README.md). To change the backend or the sandbox size later, use the Nodes page in Web; see [Nodes](/hosted-providers#change-the-sandbox-backend-or-size). @@ -531,4 +531,4 @@ missing images, restores a missing `oac` command, copies node files newly placed `--install-dir`. It refuses a bundle from another release; see [installation version policy](/troubleshooting#installation-version-policy). -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/getting-started/install.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/getting-started/install.md) diff --git a/apps/docs/content/docs/observability.mdx b/apps/docs/content/docs/observability.mdx index 6b12575e5..658c0b304 100644 --- a/apps/docs/content/docs/observability.mdx +++ b/apps/docs/content/docs/observability.mdx @@ -7,7 +7,7 @@ Status: Phase 2 and initial Core Web consumption implemented. The current-snapsh `packages/agents-client` projection, and generated `core.openapi.yaml` contract are implemented and consumed by the Dashboard through complete Session/observation identity joins. Durable history uses the separate optional -[Runtime history API](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime-history-api.md); lifecycle controls remain outside +[Runtime history API](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime-history-api.md); lifecycle controls remain outside this phase. These are administrator reads under `/core/v1`, authenticated by the Core key, @@ -92,7 +92,7 @@ Authorization: Bearer ... ``` This returns the same object shape as a list item's `observation`, without the -administrator list's `disk` (see the [administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md)). It never starts a Turn, creates +administrator list's `disk` (see the [administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md)). It never starts a Turn, creates an Environment, provisions compute, renews a lease, or changes lifecycle state. A valid `environment:none` Session returns `200` with status `unsupported`; the @@ -193,7 +193,7 @@ Use the existing Agents API error envelope. | 400 | `invalid_request_error` / `invalid_request_error` | List: a repeated supported query key, or an empty or invalid limit or order, with the shared Beta list messages. Unknown list query keys are ignored. | | 400 | `invalid_request_error` / `unsupported_parameter` | Single-Session retrieval with any query parameter. | | 401 | `invalid_request_error` / `invalid_admin_key` | Missing or invalid Core key. | -| 404 | `not_found_error` / `not_found_error` | Missing, malformed or foreign Session/cursor, indistinguishably, as for the [Session list cursor](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/list-query-semantics.md#list-cursor-errors--september-23-2026). | +| 404 | `not_found_error` / `not_found_error` | Missing, malformed or foreign Session/cursor, indistinguishably, as for the [Session list cursor](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/list-query-semantics.md#list-cursor-errors--september-23-2026). | | 500 | `server_error` / `internal_error` | Integrity, ownership, or invalid provider evidence. | | 503 | `server_error` / `execution_unavailable` | Required Runtime observation service is not configured, or list collection exceeded its request budget. | @@ -273,10 +273,10 @@ deletion makes the candidate incomplete and prevents publication. - Token usage: use existing Session/Turn Usage. - Billing and cost: product/backend concern. - Historical series in these routes: the optional capability uses the separate - [Runtime history API](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime-history-api.md). + [Runtime history API](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime-history-api.md). - Container logs and command output. - Provider credentials or native configuration. - Start, stop, pause, resume, restart, renew, or delete operations. - Idle classification and automatic shutdown. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime-observability-api.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime-observability-api.md) diff --git a/apps/docs/content/docs/public-api.mdx b/apps/docs/content/docs/public-api.mdx index 13597b96a..964db7ded 100644 --- a/apps/docs/content/docs/public-api.mdx +++ b/apps/docs/content/docs/public-api.mdx @@ -9,8 +9,8 @@ no credential works in another namespace. | Namespace | Caller | Credential | Contents | Reference | | --- | --- | --- | --- | --- | | `/v1` | Applications (business systems, SDKs) | Project API key | Exactly the pinned official Agents API routes. Core-only fields live only in `x_agents_core` (`harness`, `model_provider`) | [Public API](/sessions) | -| `/core/v1` | Core Web's server and operator scripts | [Core key](/troubleshooting#core-key) | Installation facts, Projects and keys, resource reads and deletion, Session archive, credential issuance, metrics, audit, sandbox deployment and nodes, deployment model providers | [Core API](#core-api), [Web API](/admin-api), [Core OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/core.openapi.yaml) | -| `/api/v1` | Nodes, Runtime daemons, self-hosted executors | Machine credentials: node enrollment tokens and executor credentials issued through `/core/v1`, node credentials registered with an enrollment token, and daemon credentials Core writes into hosted sandboxes | Machine connections only: `/api/v1/sandbox-node/*` and `/api/v1/agent-daemon/*`, including WebSockets; each credential works only on its own routes | [Node operations](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host), [executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-executor-credentials.md), [machine OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime.openapi.yaml) | +| `/core/v1` | Core Web's server and operator scripts | [Core key](/troubleshooting#core-key) | Installation facts, Projects and keys, resource reads and deletion, Session archive, credential issuance, metrics, audit, sandbox deployment and nodes, deployment model providers | [Core API](#core-api), [Web API](/admin-api), [Core OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/core.openapi.yaml) | +| `/api/v1` | Nodes, Runtime daemons, self-hosted executors | Machine credentials: node enrollment tokens and executor credentials issued through `/core/v1`, node credentials registered with an enrollment token, and daemon credentials Core writes into hosted sandboxes | Machine connections only: `/api/v1/sandbox-node/*` and `/api/v1/agent-daemon/*`, including WebSockets; each credential works only on its own routes | [Node operations](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host), [executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-executor-credentials.md), [machine OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime.openapi.yaml) | A Project API key gets 401 on `/core/v1` and `/api/v1`; the Core key gets 401 on `/v1` and `/api/v1`. Projects own assets. Multiple equally privileged keys share @@ -34,7 +34,7 @@ with the Core key on Core's loopback port, not through the public entry. Applications call every `/v1` route with a Project API key. The routes are exactly the 58 method and path pairs in -[upstream-routes.json](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/upstream-routes.json); the +[upstream-routes.json](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/upstream-routes.json); the [Public API guide](/sessions) groups them by resource. `/v1` has no Core-only route. @@ -45,18 +45,18 @@ with the Core key. | Routes | Contents | Contract | | --- | --- | --- | -| `projects`, `projects/{project_id}[/archive]`, `projects/{project_id}/keys[/{key_id}]` | Projects and their API keys | [Administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md) | -| `projects/{project_id}/{agents,environment-templates,skills,files,vaults,sessions}/**` | Resource reads and deletion, Session history, artifacts and archive | [Administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md) | -| `projects/{project_id}/sessions/{session_id}/execution-configuration` | Committed harness and model selection | [Execution configuration](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/execution-configuration.md) | -| `projects/{project_id}/sessions/{session_id}/diagnostics`, `projects/{project_id}/sessions/{session_id}/turns/{turn_id}/diagnostics` | Root failure categories and Item receipt timing | [Session diagnostics](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/session-diagnostics.md) | +| `projects`, `projects/{project_id}[/archive]`, `projects/{project_id}/keys[/{key_id}]` | Projects and their API keys | [Administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md) | +| `projects/{project_id}/{agents,environment-templates,skills,files,vaults,sessions}/**` | Resource reads and deletion, Session history, artifacts and archive | [Administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md) | +| `projects/{project_id}/sessions/{session_id}/execution-configuration` | Committed harness and model selection | [Execution configuration](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/execution-configuration.md) | +| `projects/{project_id}/sessions/{session_id}/diagnostics`, `projects/{project_id}/sessions/{session_id}/turns/{turn_id}/diagnostics` | Root failure categories and Item receipt timing | [Session diagnostics](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/session-diagnostics.md) | | `projects/{project_id}/sessions/{session_id}/runtime-observation`, `sandbox/runtime-observations` | Current Runtime observations | [Runtime observations](/observability) | -| `projects/{project_id}/sessions/{session_id}/runtime-history` | Stored Runtime history | [Runtime history](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime-history-api.md) | -| `projects/{project_id}/{resource-owners,write-operations}`, `audit-log`, `summary` | Provenance, write history, administrator audit and usage summary | [Write audit](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/write-audit.md), [administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md) | -| `projects/{project_id}/environments/{environment_id}/executor-credentials[/{key_id}]` | Executor credentials for a self-hosted Environment | [Executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-executor-credentials.md) | -| `installation` | Public URL, API base URL, source commit, the installer's process settings and what is bound to the public URL; available before any deployment | [Installation](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/installation.md) | -| `metrics` | Core's own process metrics | [Core metrics](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/core-metrics.md) | -| `sandbox/deployment[/reset]`, `sandbox/enrollment-tokens`, `sandbox/nodes[/{node_id}[/allocations]]` | Sandbox deployment, node enrollment tokens and nodes | [Sandbox deployment](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/sandbox-deployment.md), [node operations](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md), [node host history](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/node-host-history.md) | -| `harnesses`, `harnesses/{harness}/model-provider` | Supported harnesses and each harness's deployment default model provider (write-only key) | [Model execution](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/model-execution.md#deployment-defaults) | +| `projects/{project_id}/sessions/{session_id}/runtime-history` | Stored Runtime history | [Runtime history](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime-history-api.md) | +| `projects/{project_id}/{resource-owners,write-operations}`, `audit-log`, `summary` | Provenance, write history, administrator audit and usage summary | [Write audit](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/write-audit.md), [administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md) | +| `projects/{project_id}/environments/{environment_id}/executor-credentials[/{key_id}]` | Executor credentials for a self-hosted Environment | [Executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-executor-credentials.md) | +| `installation` | Public URL, API base URL, source commit, the installer's process settings and what is bound to the public URL; available before any deployment | [Installation](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/installation.md) | +| `metrics` | Core's own process metrics | [Core metrics](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/core-metrics.md) | +| `sandbox/deployment[/reset]`, `sandbox/enrollment-tokens`, `sandbox/nodes[/{node_id}[/allocations]]` | Sandbox deployment, node enrollment tokens and nodes | [Sandbox deployment](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/sandbox-deployment.md), [node operations](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md), [node host history](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/node-host-history.md) | +| `harnesses`, `harnesses/{harness}/model-provider` | Supported harnesses and each harness's deployment default model provider (write-only key) | [Model execution](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/model-execution.md#deployment-defaults) | ## Machine connection API @@ -65,42 +65,42 @@ the Core key or a Project API key. | Routes | Caller | Credential | Contract | | --- | --- | --- | --- | -| `POST sandbox-node/enroll` | Node installer | One-use enrollment token from `POST /core/v1/sandbox/enrollment-tokens` | [Node operations](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host), [machine OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime.openapi.yaml) | -| `GET sandbox-node/configuration` | Node installer and node | Enrollment token, or node credential with `X-OAC-Node-ID` | [Sandbox deployment](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/sandbox-deployment.md), [machine OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime.openapi.yaml) | -| `GET sandbox-node/identity`, WebSocket `GET sandbox-node/connect` | Node | Node credential registered at enrollment | [Node operations](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host) | -| `POST agent-daemon/enroll`, `GET agent-daemon/connection` | Self-hosted executor and its installer | Executor credential from `/core/v1/projects/{project_id}/environments/{environment_id}/executor-credentials` | [Executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-executor-credentials.md) | -| WebSocket `GET agent-daemon/ws`, `POST agent-daemon/bootstrap`, `GET agent-daemon/device-status` | Runtime daemons | Daemon credential: Core writes one into each hosted sandbox it prepares; a self-hosted executor uses its executor credential | [Runtime enrollment](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/services/agents-api/README.md#user-managed-runtime-enrollment) | +| `POST sandbox-node/enroll` | Node installer | One-use enrollment token from `POST /core/v1/sandbox/enrollment-tokens` | [Node operations](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host), [machine OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime.openapi.yaml) | +| `GET sandbox-node/configuration` | Node installer and node | Enrollment token, or node credential with `X-OAC-Node-ID` | [Sandbox deployment](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/sandbox-deployment.md), [machine OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime.openapi.yaml) | +| `GET sandbox-node/identity`, WebSocket `GET sandbox-node/connect` | Node | Node credential registered at enrollment | [Node operations](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/HOSTED-SANDBOX-MANAGER.md#register-a-host) | +| `POST agent-daemon/enroll`, `GET agent-daemon/connection` | Self-hosted executor and its installer | Executor credential from `/core/v1/projects/{project_id}/environments/{environment_id}/executor-credentials` | [Executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-executor-credentials.md) | +| WebSocket `GET agent-daemon/ws`, `POST agent-daemon/bootstrap`, `GET agent-daemon/device-status` | Runtime daemons | Daemon credential: Core writes one into each hosted sandbox it prepares; a self-hosted executor uses its executor credential | [Runtime enrollment](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/services/agents-api/README.md#user-managed-runtime-enrollment) | ## Contract sources -- [Pinned upstream baseline](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/upstream.json): OpenAI +- [Pinned upstream baseline](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/upstream.json): OpenAI Python SDK 3.13.0, exact upstream commit and `agents=v1`. -- [Pinned routes](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/upstream-routes.json) and - [fields](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/upstream-fields.json): extracted from the +- [Pinned routes](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/upstream-routes.json) and + [fields](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/upstream-fields.json): extracted from the pinned SDK by `scripts/extract-agents-api-upstream.py`. Contract tests require the public OpenAPI to have exactly these routes and to keep every other field inside `x_agents_core`. -- [Public OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/openapi.yaml): public schema snapshot; - combine it with the fixed SDK and [operation evidence](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/operation-evidence.md). -- [Core OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/core.openapi.yaml): generated `/core/v1` +- [Public OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/openapi.yaml): public schema snapshot; + combine it with the fixed SDK and [operation evidence](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/operation-evidence.md). +- [Core OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/core.openapi.yaml): generated `/core/v1` routes, all authenticated by the Core key. -- [Machine connection OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/runtime.openapi.yaml): - generated `/api/v1` sandbox node routes. The node WebSocket is described in the [generation protocol](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/node-generation-protocol.md); the private +- [Machine connection OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/runtime.openapi.yaml): + generated `/api/v1` sandbox node routes. The node WebSocket is described in the [generation protocol](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/node-generation-protocol.md); the private daemon transport is described in the Runtime credential guide. -- [Administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md): Project/key +- [Administrator contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md): Project/key lifecycle, resources, explicit hosted Session archive, summary, errors/deletion preconditions, audit and historical copy provenance. - [Web integration](/admin-api): browser, console and Core boundaries and frontend handoff. -- [Design rules](/concepts) and [contributor guide](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/CONTRIBUTING.md): +- [Design rules](/concepts) and [contributor guide](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/CONTRIBUTING.md): ownership, security and change requirements. Generated schemas do not establish complete compatibility or real execution -support. The [coverage record](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md) identifies +support. The [coverage record](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md) identifies qualified workflows, native differences and unresolved behavior. Update the relevant contract and this index when adding or moving an API surface. -Core administration failures use the [Core error envelope](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/core-errors.md), +Core administration failures use the [Core error envelope](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/core-errors.md), including typed optional safe details and distinct console proxy rejection codes. The public and machine error contracts remain unchanged. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/api/README.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/api/README.md) diff --git a/apps/docs/content/docs/quickstart.mdx b/apps/docs/content/docs/quickstart.mdx index 3df89ed80..49a88bfca 100644 --- a/apps/docs/content/docs/quickstart.mdx +++ b/apps/docs/content/docs/quickstart.mdx @@ -101,8 +101,8 @@ The provider's protocol must match the harness: | Claude Code (`claude_sdk`) | `anthropic` | The provider's exact model ID | | MiniMax Code (`mcode`) | `anthropic` | `context_window` and `max_output_tokens` | -See [harness selection](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/harness-selection.md) and -[model execution](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/model-execution.md) for the full contract. +See [harness selection](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/harness-selection.md) and +[model execution](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/model-execution.md) for the full contract. Selecting a harness doesn't make an unsupported model or operation work. ## Model providers @@ -142,8 +142,8 @@ not replay history. Don't submit the same work as a new Session when a response lost. Files, Artifacts, cancellation and their current limits are in the -[coverage ledger](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md). MCP credentials use the +[coverage ledger](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md). MCP credentials use the separate Vault API; they are neither Project API keys nor model provider keys. The [API index](/public-api) lists every route. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/getting-started/quickstart.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/getting-started/quickstart.md) diff --git a/apps/docs/content/docs/self-hosted-execution.mdx b/apps/docs/content/docs/self-hosted-execution.mdx index 3c6ae812e..a724057ff 100644 --- a/apps/docs/content/docs/self-hosted-execution.mdx +++ b/apps/docs/content/docs/self-hosted-execution.mdx @@ -98,6 +98,48 @@ Core delivers the Session's model provider only over this Environment's executor connection. The host keeps it in the Runtime's harness home, where the model's tools and the Files API can't read it but the host's owner can. +## Local capability directories + +A self-hosted Session may supply `capability_directories` alongside its +`workspace_directory`, for example: + +```python +environment = { + "type": "self_hosted", + "workspace_directory": "/workspace", + "capability_directories": ["/workspace/capabilities"], +} +``` + +The API accepts portable absolute Unix, Windows drive and UNC source paths without +checking Core's filesystem. The current Linux Runtime requires canonical local Unix +paths and applies its own access checks; this does not add Windows support. +These paths refer to directories visible inside that Runtime. Populate them before +first execution. `/workspace` maps to its bound workspace; an operator-prepared +Runtime may use its exact canonical absolute workspace instead. Directory aliases, +private credentials, native history and protected initialization trees are refused. +This does not mount arbitrary host paths into the container or add a new installer. +Self-hosted public input accepts local directories, not managed Skill/Plugin archives +or hosted Templates. + +The current implementation uses the packaged Linux helpers and protected +`installed.json`, by default under `/environment/initialization/capabilities`. +Operators may freeze another root with `OAC_RUNTIME_CAPABILITY_DIRECTORY`; it must +stay separate from the workspace and private Runtime state and is not a public API +setting. Runtime snapshots sources +before native execution and supplies the resulting Skill and MCP inventory through +the same parser as managed bundles. Reconnect reuses installed contents even after +source edits; a new Session captures its own snapshot. A missing or inconsistent +snapshot fails preparation without silently reinstalling. Local discovery does not +create entries in the public API-managed installation arrays. + +Closing an executor, cancelling a Turn or losing its connection preserves the +snapshot and workspace. The compute owner remains responsible for explicit resource +cleanup. This source-level capability behavior requires its own real acceptance; +[earlier qualification](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/user-managed-runtime-v1.md) covered +only its recorded inputs and binaries, not local directory preparation or additional +platforms. + ## Rotate or revoke | Action in Web | Effect | @@ -143,7 +185,7 @@ key_id=$(python3 -c 'import uuid; print(uuid.uuid4())'); echo "credential ID: $k Rotate with `{"key_id":"…","rotate":true}` on the same route; revoke with `DELETE …/executor-credentials/`. After an uncertain response, list the credentials with `GET` before trying again. The -[credential contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-executor-credentials.md) +[credential contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-executor-credentials.md) has every rule and error. @@ -159,4 +201,4 @@ continuation or automatic data deletion. Credential replacement for executors created by the current installer updates and restarts the same owned container. It does not authorize adopting an older executor. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/getting-started/self-hosted.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/getting-started/self-hosted.md) diff --git a/apps/docs/content/docs/sessions.mdx b/apps/docs/content/docs/sessions.mdx index 28f6e5292..9b4751d65 100644 --- a/apps/docs/content/docs/sessions.mdx +++ b/apps/docs/content/docs/sessions.mdx @@ -34,8 +34,8 @@ leaves the others valid; archiving the Project revokes all its keys. The optiona ## Public resources Paths are relative to `/v1`. Detailed fields, union types and response shapes are -in the [public OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/openapi.yaml), fixed SDK source -and linked semantic records. The [resource inventory](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/README.md#upstream-resource-inventory) +in the [public OpenAPI](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/openapi.yaml), fixed SDK source +and linked semantic records. The [resource inventory](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/README.md#upstream-resource-inventory) lists each operation and its coverage. | Resource | Route family | Semantics | @@ -44,11 +44,11 @@ lists each operation and its coverage. | Sessions and Turns | `/agents/sessions`, nested `/turns` | Admission, metadata, state and recovery queries | | Items and events | Session `/items`, `/events` | Persisted history and live SSE | | Artifacts | Session `/artifacts` | Captured output metadata, immutable content and deletion | -| Subagents | Session `/subagents`, nested `/turns` and `/items` | Scoped child execution reads; [native limits](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/subagents.md) | -| Environments and workspace Files | `/agents/environments`, nested `/files` | [Environment state](/environments-and-files) and [workspace access](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-files.md) | -| Environment Templates | `/agents/environments/templates` | [Reusable initialization](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-templates.md) | -| Source Files | `/files` | [Uploaded content resources](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/source-files.md), distinct from live workspace files | -| Skills and versions | `/skills`, nested `/versions` | [Bundles and versions](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/file-resource-semantics.md) | +| Subagents | Session `/subagents`, nested `/turns` and `/items` | Scoped child execution reads; [native limits](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/subagents.md) | +| Environments and workspace Files | `/agents/environments`, nested `/files` | [Environment state](/environments-and-files) and [workspace access](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-files.md) | +| Environment Templates | `/agents/environments/templates` | [Reusable initialization](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-templates.md) | +| Source Files | `/files` | [Uploaded content resources](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/source-files.md), distinct from live workspace files | +| Skills and versions | `/skills`, nested `/versions` | [Bundles and versions](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/file-resource-semantics.md) | | Vaults and Credentials | `/vaults`, nested `/credentials` | [Scoped write-only credential storage](/environments-and-files) | SSE is live, not a historical replay service. After a disconnect, query the Session, @@ -58,14 +58,14 @@ native execution. Do not automatically replay uncertain tool or file effects. ## Core extension fields The `/v1` route set is exactly the pinned SDK's operations, listed in -[upstream-routes.json](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/upstream-routes.json); Core adds +[upstream-routes.json](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/upstream-routes.json); Core adds no route. It adds fields only inside `x_agents_core`, because it runs several harnesses and accepts custom model access: | Field | Where | Reference | | --- | --- | --- | -| `x_agents_core.harness` | Saved Agent create, update and read; the inline Session `agent`; the Session's effective `agent` | [Harness selection](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/harness-selection.md) | -| `x_agents_core.model_provider` | Saved Agent create, update and read; Session creation. It takes `protocol`, `base_url`, optional `context_window` and `max_output_tokens`, and a write-only `api_key`; reads return `api_key_configured` instead of the key | [Model execution](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/model-execution.md) | +| `x_agents_core.harness` | Saved Agent create, update and read; the inline Session `agent`; the Session's effective `agent` | [Harness selection](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/harness-selection.md) | +| `x_agents_core.model_provider` | Saved Agent create, update and read; Session creation. It takes `protocol`, `base_url`, optional `context_window` and `max_output_tokens`, and a write-only `api_key`; reads return `api_key_configured` instead of the key | [Model execution](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/model-execution.md) | Any other member of `x_agents_core` is rejected with 400. Deployment, placement, credential issuance and operational reads are not part of `/v1`; they belong to @@ -78,7 +78,7 @@ never to `self_hosted`; `none` Sessions accept only the deployment default. `openai_hosted` and `self_hosted` Sessions must end up with one: otherwise creation returns 400 with code `model_provider_required` and param `x_agents_core.model_provider`, and nothing is written. See -[model execution](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/model-execution.md). +[model execution](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/model-execution.md). An application reads a Session's model from `agent.model` and an explicitly selected harness from `agent.x_agents_core.harness`; a Session on the deployment @@ -90,8 +90,8 @@ The deployment chooses E2B or its own Docker/microsandbox nodes. A public `self_hosted` Environment uses caller-owned compute and the same colocated Runtime. For a `self_hosted` Session, the deployment operator issues the Environment's executor credential with the Core key; see -[executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/environment-executor-credentials.md). +[executor credentials](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/environment-executor-credentials.md). Our daemon transport and the supported native harness differences are explicit; they do not imply interoperability with OpenAI's stock executor transport. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/api/public-agent-api.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/api/public-agent-api.md) diff --git a/apps/docs/content/docs/troubleshooting.mdx b/apps/docs/content/docs/troubleshooting.mdx index 5e72d6770..b1d4fe576 100644 --- a/apps/docs/content/docs/troubleshooting.mdx +++ b/apps/docs/content/docs/troubleshooting.mdx @@ -123,7 +123,7 @@ refuses. `rotate-core-key` refuses to run on a Web-only installation. ## Projects and API keys Create Projects and issue their keys in Web, on **Projects and keys**, or with the -[Core API](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md) under `/core/v1/projects`. A key's +[Core API](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md) under `/core/v1/projects`. A key's plaintext is shown once; Core stores only its digest. All keys of a Project share its assets and execution principal, and each write records the key that made it. @@ -241,9 +241,9 @@ Core records which key made each public resource write, for the console. Set `core.write_audit_retention` in `config.json` (Go duration, at least `1h`, default `2160h`) to control how long non-creation history is kept; creation ownership is kept for good. Removing keys or resources does not delete these records. See the -[query contract](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/write-audit.md); administrator mutations +[query contract](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/write-audit.md); administrator mutations have a separate -[audit log](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/contracts/agents-api/admin-api.md#monitoring-and-audit). Neither +[audit log](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/contracts/agents-api/admin-api.md#monitoring-and-audit). Neither logs bodies or secrets. -[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/cd6a32c3de31d26b73ab92d21ba2aa7b067ba226/docs/getting-started/operations.md) +[Repository source](https://github.com/MiniMax-AI/parsar-core/blob/e35d3c9eda2cdb717fd4c3627404d362cc2e4dca/docs/getting-started/operations.md) diff --git a/apps/docs/content/guide-sources.json b/apps/docs/content/guide-sources.json index 4459951fd..b5aa27c92 100644 --- a/apps/docs/content/guide-sources.json +++ b/apps/docs/content/guide-sources.json @@ -1,5 +1,5 @@ { - "revision": "cd6a32c3de31d26b73ab92d21ba2aa7b067ba226", + "revision": "e35d3c9eda2cdb717fd4c3627404d362cc2e4dca", "sources": { "docs/getting-started/README.md": "4eb3d21e073b25e2e6ffb8800e13b33032104ba44d8d93ea3e59394549dc7962", "docs/design-principles.md": "71285ff0b9a7796481b9fe42f7c4b71d0377c767157980ca6386792d9ca8951b", @@ -11,9 +11,9 @@ "docs/api/README.md": "592fc2aef474e36eb760be0dd1b2269b6766c0d984f826eaf6f47711ea139387", "contracts/agents-api/execution-tools.md": "e0b61f6c0c236c362186c5f6d0ad69a16dd8a8d9afabe1329a1fe22e5f4a71e9", "docs/api/public-agent-api.md": "18829b366bd2e49fe5beb76dc8a709bc1aa8e35d4fbfd2484a0c52b46cf75417", - "contracts/agents-api/environments.md": "89ffc903ee44d00ee87c16950202a3ca4c8345160780dcdd4ed2a71b25b1c2dc", + "contracts/agents-api/environments.md": "4cbc2efb68e7527798b3d6836593aad5af1130416c861fc6b2b638681aea328a", "docs/getting-started/nodes.md": "c1bc854c4798b9757bbb8cea4474a03442ba6f4c10444d5505733e8b94046038", - "docs/getting-started/self-hosted.md": "55d6b29b1826f5cb4aee193c4ac9011bfb59d20192938962b2572840c42e242b", + "docs/getting-started/self-hosted.md": "82f45d07962f49ebffbda9f2c4c4ac029296b31a6cfa048f9fcf7adca32e9de9", "apps/web/public/onboarding/monitor-en.webp": "29dc220cb1250c7016b7c4bf7f30e9510b07c07b814a48c3aa9303b2d76f20b2", "docs/web/README.md": "9fce295f6f6ff04acefe338679c1e54723e123dd0027a778202adfa9ab95b6c4", "docs/api/web-management.md": "dc79df33dade6139749752c9ce53d4e126354cfabccada3e7826de4dd20fae62", @@ -22,40 +22,40 @@ "apps/docs/scripts/guides.json": "1f6036e3cccaf5a71cdd9ea790be6f395acdf871c4552c490b6747957a10b174" }, "outputs": { - "content/docs/index.mdx": "0e0f91fcd4a2cdb65a85c39d81c4cc508a5750829380e2c8ad7d2955f16f1330", + "content/docs/index.mdx": "d710539d3e0b5951cbf5b980142c31feda939dda9c39a38915045efd3492ecf6", "content/docs/index.zh.mdx": "5bdd41893946f1d229eb4cf1599a97f2e38cb9e09722d2a69f62053376ac88a9", - "content/docs/concepts.mdx": "5e20f86486bf9d8b4147241ca124d3fbdf7edee59e5c6bd2ec1f82b4e2adcf10", + "content/docs/concepts.mdx": "c7eb1d9cbddeabb8b132db37b825eed24456e7729d2784b78f3cd4f88a216fae", "content/docs/concepts.zh.mdx": "c60644f395dbac5ee607cdb053249e05ec8352d28de97ef1c38d6c1e4443bbb1", - "content/docs/execution-model.mdx": "ac5bd0d655c5737b00019498be900fe2bf8a680f34e9d34ac08cbc2718ab1d3f", + "content/docs/execution-model.mdx": "cfc5d4245a7eaf693022717dc34e004a65467d1c30fe0f57de1835a569344365", "content/docs/execution-model.zh.mdx": "9976a9203c6a83d49c2c753e3a8528a369078e87aeb88850654da948ac8c17e1", - "content/docs/install.mdx": "32d58f72107f79e74f0ab291bc39a6d83908f57330f1ca5ee5c9bcc9d44f9bad", + "content/docs/install.mdx": "fbcc7f893e29e5a03a26cdcb5e51b71569a23df5d2a8d268bcba4eb35b836126", "content/docs/install.zh.mdx": "f207607f0cfceef2012a4e772fc347ef254486327584289fee1c85414dffc907", - "content/docs/configure.mdx": "523a8ed974e82634df690b3396e33e5645db55551e4f34d4e131aa411ff4d218", + "content/docs/configure.mdx": "d3c6bd0c0c5fe76725a57acc1681f66bf6eb3f642cbaf296ce40d5ea45d24d01", "content/docs/configure.zh.mdx": "1c4b149c332ade6f2eeca52319e3e7ccfaac3213db2fd084d66e2da517b6bbd6", - "content/docs/bootstrap-projects-keys.mdx": "a88aac6c38abf5237b67a54f59a68b272ecbcd7d53280bb60efb566b14a1b906", + "content/docs/bootstrap-projects-keys.mdx": "da60daed343138acaf5d961b72ca670c18171371d6e791883b170fd233461e41", "content/docs/bootstrap-projects-keys.zh.mdx": "509b4dcea4c865d6834bcaef78baa681180d48274931230f958a39e46011c88f", - "content/docs/quickstart.mdx": "23525ecf2c7d35c8920545a8490ad3f8b3ed7b28ba8eed6f0f794a8d179211c5", + "content/docs/quickstart.mdx": "0afc29fc97b166b11356343e67c4bb802b14cdc22d29d7c691683fc24f2d8a14", "content/docs/quickstart.zh.mdx": "c5f119b100b102918a45763834d0ed7fd436479fb411e16a62a408411dbcb3e1", - "content/docs/public-api.mdx": "1f1270ee51840ea1a61f5d6010a1d9b344ab13c800d6f4c14419081cb0fd3ba0", + "content/docs/public-api.mdx": "53492062e7647ebbf39f469bb0ade267769cd08eabe73d9f42e0b9870356b4d8", "content/docs/public-api.zh.mdx": "fb0efb9be2fba3f4a3dc71bf053d2c7eddffbfe010f59fa5a1ae60a3f068545b", - "content/docs/agents-and-tools.mdx": "ebd8239cb0cef20b9104abacd75cb6d6dc37f143347d7e370fa072b9569b1e05", + "content/docs/agents-and-tools.mdx": "efbfee0d551c3f362f900affde6fe2721910fabc11c85841e981d0a60655a5a5", "content/docs/agents-and-tools.zh.mdx": "cc8d4e0ef0ea894f068f3bbd143d5ee70c6346bf98f425b1d5fe07ec3931f128", - "content/docs/sessions.mdx": "d742bdb741475e68620a312ca604b13c758aa32b1f7c7fb8ecbde65465fd1ce3", + "content/docs/sessions.mdx": "57328e8fb2aa645cb8334b6e9a8eb677e4123b5505027d2d6add5c8e6d030b38", "content/docs/sessions.zh.mdx": "b1b1a7f8aacd9a902320f9692173ede24036124f85761cf02f23ef3ef2bcc4c1", - "content/docs/environments-and-files.mdx": "324b055462ad01e505d225f501bdfb7ccbe0f60f92d583b3a018c08ae2bcc9bd", + "content/docs/environments-and-files.mdx": "cd5ca9579ef981b9e7c2d37ebec89d9e9d15283964334d74efaaf118bbd5c09c", "content/docs/environments-and-files.zh.mdx": "ee64577220a798f023b330c65302c14438f25a34a78c4e36144bfdb4304a3ae5", - "content/docs/hosted-providers.mdx": "ce582156fe078421312e0ef2e09ea134b98494ab498b6c6b1ca0174c79133297", + "content/docs/hosted-providers.mdx": "de6ece49be3db64e5e5b223737673c86107e001dd540b228bb43ee2b02094a32", "content/docs/hosted-providers.zh.mdx": "d7f6115f052a09dc943f0ed516ff30a27da5847ed24d387d9d1c7e7faa006ab0", - "content/docs/self-hosted-execution.mdx": "f4f93d8d29a37959b1fdf6f1d0c2e472f72738586990ee74a42901eb4d3f688e", + "content/docs/self-hosted-execution.mdx": "b7d8c2424b02229527465b144f62059ca5c09dfe90474869ba57ce576d994e7f", "content/docs/self-hosted-execution.zh.mdx": "f11d34ce30d48039376256474bc9509a508a2f6b9906f49b5cf6e396707ae5c0", "public/images/source/apps/web/public/onboarding/monitor-en.webp": "29dc220cb1250c7016b7c4bf7f30e9510b07c07b814a48c3aa9303b2d76f20b2", - "content/docs/console.mdx": "c4d80ef14420f2206beefa0d6d85bb3e0fbd982f40123f05773d469c4cd238e4", + "content/docs/console.mdx": "40bc03f4ad3334c7b7d1e141827cd6eaf51144f34532007cfe7925c2d6cb2a82", "content/docs/console.zh.mdx": "72f953ba46dcc95933773ac0c4dffdc0d0a499be9d9a1cb9467db2659f1fa10d", - "content/docs/admin-api.mdx": "867eb84df53abf071fadef346d7f9795bfc1be6bad4a0e69475e687a5b0ae16c", + "content/docs/admin-api.mdx": "73c8f57db4e0131e3af1442a84dd61337622188f16d5c685b18b39750a1000b7", "content/docs/admin-api.zh.mdx": "07006de30df3080e54c3cd79ed0ea5337dce557aa2f11afea84edb7f92884d38", - "content/docs/observability.mdx": "60a13191cc35fb7ca0ae8ed411988e085a0cdb9ada642adc3615ef603353ca86", + "content/docs/observability.mdx": "3d9f0891a7efb0a35a5cd7ba9db97d364511ae1e70ceecbdb7f815957909bc4b", "content/docs/observability.zh.mdx": "9574b3412a5754f41ab2d3a16062ac1cad06766be481a8e6bff1b84acbde7e7e", - "content/docs/troubleshooting.mdx": "5fd41876e66537ab7631da818a0c3a3b9cbcc17ce2032da5debf5d7d297ca5ce", + "content/docs/troubleshooting.mdx": "a87de7639962d7830a2c5905d69439eb17f762f87d7cdd4df1dbedc71278e7a9", "content/docs/troubleshooting.zh.mdx": "6d8b4668e8c371120a9275daabbebc0d1ecf21a42ba32beeb92853c14b5a459a" } } diff --git a/apps/docs/openapi/public-api.yaml b/apps/docs/openapi/public-api.yaml index 29898c58f..090594dd8 100644 --- a/apps/docs/openapi/public-api.yaml +++ b/apps/docs/openapi/public-api.yaml @@ -864,7 +864,7 @@ paths: tags: - Sessions post: - description: The optional Core model_provider bundle resolves from the Session override, saved Agent defaults, then, for openai_hosted and none, the deployment default of the resolved harness; self_hosted never uses the deployment default and none accepts only it. openai_hosted and self_hosted Sessions that resolve no bundle return 400 model_provider_required with param x_agents_core.model_provider before any write. Core encrypts and freezes the resolved bundle; later Agent or deployment default edits and same-key retries cannot change it. Keys are never returned. Supports inline configuration or a tenant-owned saved agent_id with per-Session field replacements. Execution supports model/instructions, text verbosity, non-deferred function tools, adapter-qualified multi_agent with persisted Subagent reads, implicit reasoning, service tier auto and environment type none, subject to the configured engine. Codex additionally supports HTTP MCP with service origin (omitted or null on HTTP transport is saved as service), native allowed_tools and boolean required defaulting to false. Session vault_ids attach only project-owned Vaults; credential_id selects an attached static bearer or OAuth credential for the exact HTTPS URL, while null/omission selects a unique match or remains anonymous. Session reads, lists and event snapshots show that implicitly selected credential ID in a null or omitted credential_id, also after the credential is deleted; anonymous selections stay null and the stored caller intent is unchanged. After the input requirement and before any write, a credential_id without vault_ids, one outside the attached Vaults (one message for missing, foreign and unattached IDs) or one for another server_url returns 400 invalid_request_error, and several implicit matches return 409 conflict_error. Missing decryption configuration fails dispatch without anonymous fallback. Required initialization uses native startup before the first native Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and native OAuth login remain unsupported. The self_hosted profile requires Codex, an absolute workspace_directory and empty capability_directories, with optional non-deferred function tools and HTTP MCP using service origin, optionally authenticated by the attached Vault rules. Remote MCP and remote Bearer authentication each require separately advertised combination support; old peers cannot receive unsupported work. Omitted/null capability_directories use the empty-list default; self_hosted requires configured execution plus executor registry. Claude SDK currently requires medium verbosity and object-root function schemas. It supports anonymous or attached static-bearer service-origin HTTP MCP on none with boolean required and separately advertised MCP/bearer/required runtime support. Required servers must be connected before the first native input is released; pending or failed startup rejects execution. The shared Vault selection and immutable binding rules apply; unsupported native labels/tool names reject before persistence. An attached Vault with no matching credential may remain anonymous; missing keys or failed credential lookup/decryption never fall back to anonymous execution. Omitted stream defaults to false; stream and agent_id cannot be null. Metadata may be null; non-string values and limit violations return invalid_request_error with a metadata or metadata. param. The inline agent uses the Agent create configuration validation with agent.-prefixed params, reported before the input requirement and saved-Agent lookup; saved configurations with conflicting tools or schema roots reject admission with the same errors, and execution limits keep unsupported_or_invalid_configuration. Hosted network policy rejections return invalid_request_error with a null param. Initial input accepts a string or ordered user-message array. Codex and Claude SDK on none and qualified openai_hosted also accept inline PNG/JPEG image content; other image combinations and remote URLs are unsupported. None initial input atomically starts a Turn; self_hosted initial input is reserved while returning its Environment connection target, with execution deferred to native readiness and Session failure on initial timeout. Initial input is required for none and for streamed creation outside self_hosted. Omitted/null input remains valid for non-streaming hosted and self_hosted creation. With stream=true, returns live Session events starting with the committed creation snapshot and closes right after the first agent.session.idle recorded when a Turn ends or an input reservation stops being pending, or any agent.session.failed, without sending later events. A creation that admitted nothing closes after the snapshot; a settlement that records no event closes after events up to the cursor read with a settled Session projection. Required actions keep it open; disconnect does not cancel execution. The GET events stream remains live-only. New Sessions retain their authenticated creator; all creation retries require the same typed subject, including across key rotation. Saved-Agent retries and inline requests using Vault attachments or credential references retain caller intent independently of later resource changes; new hosted inline requests also freeze caller intent before deployment defaults resolve; unrelated non-hosted inline retries preserve resolved/default equivalences, and their resolved hash leaves out any deployment default. Provider keys enter retry hashes only as fingerprints keyed by the credential key. Unknown historical creators reject retries; known creators without recorded intent retain resolved-snapshot retry rules. These conflict policies are local and not verified hosted parity. A same-key stream=true retry of an existing creation returns 201 with no events and closes at once; retry with stream=false or use the GET events stream to recover. Claude SDK on none and Core-managed Docker openai_hosted supports qualified object-root json_schema output with medium verbosity, single-Agent execution and ordinary functions. Hosted execution reuses native workspace tools and Files/Artifacts; Skills, Plugins, capability directories, HTTP MCP, Subagent and tool_search combinations remain unqualified, including inherited template contents. Other non-text initial input remains unsupported. Basic Codex and Claude SDK openai_hosted creation requires an explicitly configured managed provider. The Claude workspace profile supports non-deferred function tools with text or successful inline PNG/JPEG results alongside native workspace tools; HTTP MCP remains unsupported. Idle Sessions provision automatically; initial provisioning has no caller connection action. Network defaults to enabled; disabled and restricted exact ASCII hostnames are supported. Restricted policy requires 1–100 allowed domains. Unsupported hostname forms and startup installations are rejected. Confidential env, system/npm/Python packages and ordered setup commands use the shared initialization lifecycle; requested network applies after setup. Initial inline and tenant-owned file_id files freeze encrypted bytes before provisioning, then install through the common Core lifecycle before native execution or live Files access. With a template reference, omitted/null files, env, packages and setup_commands inherit. Non-null files and command lists replace; env overlays by key; each package manager inherits on omission/null and otherwise replaces its list. Empty lists clear their selected field. Tenant-owned environment_template_id references inherit omitted/null network and allow only narrowing overrides. Inline hosted network:null retains the enabled default; updating a Template with network:null resets its saved policy to enabled. Core freezes effective configuration; template updates/deletion do not alter Session snapshots or same-intent creation retries. Inline or tenant-owned skill_reference Skills share initialization. Templates preserve default/latest/explicit selectors; Session creation freezes concrete metadata and encrypted content atomically. Skill, Plugin and capability-directory list omission/null inherit; a non-null list replaces, including empty-list clearing. Omitted/null Skill version selectors resolve the default version. Source deletion/default updates cannot change committed Session Skill contents. Deferred function discovery uses type-only tool_search and per-function defer_loading in the qualified single-agent Claude environment:none function profile, including qualified inline image messages and text results. Explicit web_search mode disabled and programmatic_tool_calling enabled false use frozen common Runtime controls. Enabled forms, including those saved on an Agent, remain unqualified and reject before any write unless the Session replaces tools. Omitted programmatic configuration preserves native behavior, a documented difference from the official default-on behavior. Other combinations remain unqualified; see the operation coverage. + description: The optional Core model_provider bundle resolves from the Session override, saved Agent defaults, then, for openai_hosted and none, the deployment default of the resolved harness; self_hosted never uses the deployment default and none accepts only it. openai_hosted and self_hosted Sessions that resolve no bundle return 400 model_provider_required with param x_agents_core.model_provider before any write. Core encrypts and freezes the resolved bundle; later Agent or deployment default edits and same-key retries cannot change it. Keys are never returned. Supports inline configuration or a tenant-owned saved agent_id with per-Session field replacements. Execution supports model/instructions, text verbosity, non-deferred function tools, adapter-qualified multi_agent with persisted Subagent reads, implicit reasoning, service tier auto and environment type none, subject to the configured engine. Codex additionally supports HTTP MCP with service origin (omitted or null on HTTP transport is saved as service), native allowed_tools and boolean required defaulting to false. Session vault_ids attach only project-owned Vaults; credential_id selects an attached static bearer or OAuth credential for the exact HTTPS URL, while null/omission selects a unique match or remains anonymous. Session reads, lists and event snapshots show that implicitly selected credential ID in a null or omitted credential_id, also after the credential is deleted; anonymous selections stay null and the stored caller intent is unchanged. After the input requirement and before any write, a credential_id without vault_ids, one outside the attached Vaults (one message for missing, foreign and unattached IDs) or one for another server_url returns 400 invalid_request_error, and several implicit matches return 409 conflict_error. Missing decryption configuration fails dispatch without anonymous fallback. Required initialization uses native startup before the first native Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and native OAuth login remain unsupported. The self_hosted profile uses a qualified native harness, a clean absolute workspace_directory and optional absolute local capability_directories prepared by Runtime, with optional non-deferred function tools and HTTP MCP using service origin, optionally authenticated by the attached Vault rules. Remote MCP and remote Bearer authentication each require separately advertised combination support; old peers cannot receive unsupported work. Omitted/null capability_directories use the empty-list default; self_hosted requires configured execution plus executor registry. Claude SDK currently requires medium verbosity and object-root function schemas. It supports anonymous or attached static-bearer service-origin HTTP MCP on none with boolean required and separately advertised MCP/bearer/required runtime support. Required servers must be connected before the first native input is released; pending or failed startup rejects execution. The shared Vault selection and immutable binding rules apply; unsupported native labels/tool names reject before persistence. An attached Vault with no matching credential may remain anonymous; missing keys or failed credential lookup/decryption never fall back to anonymous execution. Omitted stream defaults to false; stream and agent_id cannot be null. Metadata may be null; non-string values and limit violations return invalid_request_error with a metadata or metadata. param. The inline agent uses the Agent create configuration validation with agent.-prefixed params, reported before the input requirement and saved-Agent lookup; saved configurations with conflicting tools or schema roots reject admission with the same errors, and execution limits keep unsupported_or_invalid_configuration. Hosted network policy rejections return invalid_request_error with a null param. Initial input accepts a string or ordered user-message array. Codex and Claude SDK on none and qualified openai_hosted also accept inline PNG/JPEG image content; other image combinations and remote URLs are unsupported. None initial input atomically starts a Turn; self_hosted initial input is reserved while returning its Environment connection target, with execution deferred to native readiness and Session failure on initial timeout. Initial input is required for none and for streamed creation outside self_hosted. Omitted/null input remains valid for non-streaming hosted and self_hosted creation. With stream=true, returns live Session events starting with the committed creation snapshot and closes right after the first agent.session.idle recorded when a Turn ends or an input reservation stops being pending, or any agent.session.failed, without sending later events. A creation that admitted nothing closes after the snapshot; a settlement that records no event closes after events up to the cursor read with a settled Session projection. Required actions keep it open; disconnect does not cancel execution. The GET events stream remains live-only. New Sessions retain their authenticated creator; all creation retries require the same typed subject, including across key rotation. Saved-Agent retries and inline requests using Vault attachments or credential references retain caller intent independently of later resource changes; new hosted inline requests also freeze caller intent before deployment defaults resolve; unrelated non-hosted inline retries preserve resolved/default equivalences, and their resolved hash leaves out any deployment default. Provider keys enter retry hashes only as fingerprints keyed by the credential key. Unknown historical creators reject retries; known creators without recorded intent retain resolved-snapshot retry rules. These conflict policies are local and not verified hosted parity. A same-key stream=true retry of an existing creation returns 201 with no events and closes at once; retry with stream=false or use the GET events stream to recover. Claude SDK on none and Core-managed Docker openai_hosted supports qualified object-root json_schema output with medium verbosity, single-Agent execution and ordinary functions. Hosted execution reuses native workspace tools and Files/Artifacts; Skills, Plugins, capability directories, HTTP MCP, Subagent and tool_search combinations remain unqualified, including inherited template contents. Other non-text initial input remains unsupported. Basic Codex and Claude SDK openai_hosted creation requires an explicitly configured managed provider. The Claude workspace profile supports non-deferred function tools with text or successful inline PNG/JPEG results alongside native workspace tools; HTTP MCP remains unsupported. Idle Sessions provision automatically; initial provisioning has no caller connection action. Network defaults to enabled; disabled and restricted exact ASCII hostnames are supported. Restricted policy requires 1–100 allowed domains. Unsupported hostname forms and startup installations are rejected. Confidential env, system/npm/Python packages and ordered setup commands use the shared initialization lifecycle; requested network applies after setup. Initial inline and tenant-owned file_id files freeze encrypted bytes before provisioning, then install through the common Core lifecycle before native execution or live Files access. With a template reference, omitted/null files, env, packages and setup_commands inherit. Non-null files and command lists replace; env overlays by key; each package manager inherits on omission/null and otherwise replaces its list. Empty lists clear their selected field. Tenant-owned environment_template_id references inherit omitted/null network and allow only narrowing overrides. Inline hosted network:null retains the enabled default; updating a Template with network:null resets its saved policy to enabled. Core freezes effective configuration; template updates/deletion do not alter Session snapshots or same-intent creation retries. Inline or tenant-owned skill_reference Skills share initialization. Templates preserve default/latest/explicit selectors; Session creation freezes concrete metadata and encrypted content atomically. Skill, Plugin and capability-directory list omission/null inherit; a non-null list replaces, including empty-list clearing. Omitted/null Skill version selectors resolve the default version. Source deletion/default updates cannot change committed Session Skill contents. Deferred function discovery uses type-only tool_search and per-function defer_loading in the qualified single-agent Claude environment:none function profile, including qualified inline image messages and text results. Explicit web_search mode disabled and programmatic_tool_calling enabled false use frozen common Runtime controls. Enabled forms, including those saved on an Agent, remain unqualified and reject before any write unless the Session replaces tools. Omitted programmatic configuration preserves native behavior, a documented difference from the official default-on behavior. Other combinations remain unqualified; see the operation coverage. parameters: - schema: type: string diff --git a/apps/docs/openapi/sources.json b/apps/docs/openapi/sources.json index bf2a7c62a..d9d0f0b17 100644 --- a/apps/docs/openapi/sources.json +++ b/apps/docs/openapi/sources.json @@ -1,6 +1,6 @@ { "sources": { - "contracts/agents-api/openapi.yaml": "a8896e8b67d78ed0608c7cc5636077ec33f4b4895a04bd6cfb8b038afe941e01", + "contracts/agents-api/openapi.yaml": "e54687e7b3c8cba023432fc5b8ec56ee4e1312f840b8f355b5ca2ddb41917d1d", "contracts/agents-api/core.openapi.yaml": "7801fc77259893815fe9cdcb404f045b375711b9b075eab2b2ed8c5dba55e24b", "contracts/agents-api/runtime.openapi.yaml": "23852205d3b45892f2de9ecf3d52518c0dd0808962c0c512f219b565aa7ccef2" }, @@ -8,7 +8,7 @@ "content/docs/api-reference/agents.mdx": "1f7697959e9c52c9d24fbe70e116e807f49912b9a27ac638b86782e2f326af44", "content/docs/api-reference/environments.mdx": "6a9e08a67479a745983c45b9711137617c13fdf7599a64533997b00a26ac7282", "content/docs/api-reference/environment-templates.mdx": "c1daeaec159aea40b0aa5ed72b1e77ac1715471431b558d0d89e05bfe9ad8945", - "content/docs/api-reference/sessions.mdx": "cbebb6a1705b486fb228cbe0065df893227132f9b3884180a3871a3e8c78f311", + "content/docs/api-reference/sessions.mdx": "9709476af57c645f441bb287c3455ee09dd1fd2d1043558439dc6c32676a3c82", "content/docs/api-reference/artifacts.mdx": "8df8d008126b49b1df88f28a676c4b237ccffc45a470b35ec30eab2daf1c0f7f", "content/docs/api-reference/events.mdx": "83d7f56a788d073e0b19f0bc9197f39ec364375b9cc3a7ec69b79b54bc71e3eb", "content/docs/api-reference/items.mdx": "fa4d248ac5623beb3f00a4ebb5a6c4ff477865ebee351e0c543a970a7ec3654d", @@ -18,7 +18,7 @@ "content/docs/api-reference/skills.mdx": "774ea05a7a8250fe5af5164388b98dafca99d56dc2eae686e8f628acb4e7c10b", "content/docs/api-reference/vaults.mdx": "d80eade1c2f8100030866abbb0fb6b7a09c94250a859c97cdf1abce32fdfdf1e", "content/docs/api-reference/credentials.mdx": "f129a51f74a3c7021c016600518e2350023f8d29571666ac5d64b7584cd7c7d0", - "openapi/public-api.yaml": "84ae8a33717c817950d757990018edd10a4d31d5a343c8172baa099d101c5a1f", + "openapi/public-api.yaml": "17ae24606c26be371ffa0c90904fdba3deac5a744ac985034a68aa35873b9a94", "content/docs/api-reference/meta.json": "56bf1dc145ccc8adde38c5f956f1a06f18ef18b646122250d01a294c516f8f83", "content/docs/api-reference/index.mdx": "9fd6b1c54149874d2999235067c101326f9e8a2e707a70b6672c81eb3ebfe63f", "content/docs/api-reference/core/core-administration.mdx": "5fc3ada51d2190a141f13738151e4d513390cc3bd66ce6f5013994bda68569b0", diff --git a/apps/docs/scripts/generate-guides.mjs b/apps/docs/scripts/generate-guides.mjs index 5e8e157a0..f403d7a78 100644 --- a/apps/docs/scripts/generate-guides.mjs +++ b/apps/docs/scripts/generate-guides.mjs @@ -8,7 +8,7 @@ const app = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..') const repo = path.resolve(app, '../..') const guides = JSON.parse(fs.readFileSync(path.join(app, 'scripts/guides.json'), 'utf8')) const routes = new Map(guides.map(g => [g.source, g.slug === 'index' ? '/' : '/' + g.slug])) -const sourceRevision = 'cd6a32c3de31d26b73ab92d21ba2aa7b067ba226' +const sourceRevision = 'e35d3c9eda2cdb717fd4c3627404d362cc2e4dca' const sourceURL = relative => `https://github.com/MiniMax-AI/parsar-core/blob/${sourceRevision}/${relative}` const digest = text => crypto.createHash('sha256').update(text).digest('hex') function mdx(text) { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/cancellation.go b/apps/parsar-daemon/internal/agent/claudesdk/cancellation.go index b2c432aa8..972e8af08 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/cancellation.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/cancellation.go @@ -2,28 +2,51 @@ package claudesdk import ( "context" + "errors" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" ) -// Cancel confirms process exit and output drain, independently of terminal delivery. +// Cancel addresses this fixed Turn. Settlement remains observable after the caller detaches. func (s *session) Cancel(ctx context.Context) error { if ctx == nil { ctx = context.Background() } select { case <-s.settled: - return nil + return s.settlementErr default: } - s.process.Cancel() + if err := ctx.Err(); err != nil { + return err + } + if s.owner == nil { + return errors.New("claudesdk: Turn owner is unavailable") + } + s.cancelOnce.Do(func() { + close(s.cancelOutput) + s.owner.mu.Lock() + current := s.owner.active == s && !s.owner.invalid + s.owner.mu.Unlock() + if current { + stopWrite := context.AfterFunc(ctx, s.invalidate) + defer stopWrite() + if err := s.owner.write(struct { + Type string `json:"type"` + TurnID string `json:"turn_id"` + }{"turn_cancel", s.runID}); err != nil { + s.invalidate() + } + } + }) select { case <-s.settled: - return nil + return s.settlementErr case <-ctx.Done(): select { case <-s.settled: - return nil + return s.settlementErr default: return ctx.Err() } @@ -40,3 +63,5 @@ func (s *session) CancellationOutcome() proto.DonePayload { return proto.DonePayload{} } } + +var _ agent.Turn = (*session)(nil) diff --git a/apps/parsar-daemon/internal/agent/claudesdk/cancellation_test.go b/apps/parsar-daemon/internal/agent/claudesdk/cancellation_test.go index 0f94f90ad..b8e64ed3a 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/cancellation_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/cancellation_test.go @@ -3,16 +3,15 @@ package claudesdk import ( + "bufio" "bytes" "context" "encoding/json" "errors" "os" - "os/signal" "path/filepath" "reflect" "strings" - "syscall" "testing" "time" @@ -171,12 +170,16 @@ func cancellationRequest() proto.PromptRequestPayload { return proto.PromptRequestPayload{RunID: "run", Input: proto.TextInput("hello"), AgentSessionID: "native-session", AgentOptions: map[string]any{"model": "fake-model", "system_prompt": "instructions"}} } -func runCancellationHelper(request startRequest, mode string, emit func(bridgeEvent)) { +func runCancellationHelper(request startRequest, mode string, scanner *bufio.Scanner, emit func(bridgeEvent)) { if mode == "cancellation-wait" { - stopped := make(chan os.Signal, 1) - signal.Notify(stopped, syscall.SIGTERM) emit(bridgeEvent{Type: "delta", Delta: "partial"}) - <-stopped + if !scanner.Scan() { + return + } + var cancel map[string]any + if json.Unmarshal(scanner.Bytes(), &cancel) != nil || cancel["type"] != "turn_cancel" { + os.Exit(9) + } // These valid observations were in flight when cancellation started. emit(bridgeEvent{Type: "input_ready", SessionID: request.Resume}) emit(bridgeEvent{Type: "usage", ResultID: "native-result", SessionID: request.Resume, Usage: json.RawMessage(usageFixture)}) @@ -189,6 +192,8 @@ func runCancellationHelper(request startRequest, mode string, emit func(bridgeEv } _, _ = os.Stderr.WriteString(strings.Repeat("x", 2*1024*1024)) emit(bridgeEvent{Type: "delta", Delta: "drained"}) + emit(bridgeEvent{Type: "input_closed", SessionID: request.Resume}) + emit(bridgeEvent{Type: "error", Code: "cancelled"}) return } if mode != "cancellation-before-identity" { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/commands_session_test.go b/apps/parsar-daemon/internal/agent/claudesdk/commands_session_test.go index f87ed4969..ebed0792c 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/commands_session_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/commands_session_test.go @@ -3,10 +3,10 @@ package claudesdk import ( + "bufio" "context" "encoding/json" "os" - "os/signal" "path/filepath" "strings" "syscall" @@ -119,8 +119,8 @@ func TestWorkspaceCommandCancellationAndBridgeFailuresCloseOnlyPendingCalls(t *t t.Fatal(err) } } else if mode == "commands-cancel" || mode == "commands-drained" { - if err := s.Cancel(ctx); err != nil { - t.Fatal(err) + if err := s.Cancel(ctx); (err == nil) != (mode == "commands-drained") { + t.Fatalf("command cancellation confirmation: %v", err) } } } @@ -155,13 +155,7 @@ func TestWorkspaceCommandCancellationAndBridgeFailuresCloseOnlyPendingCalls(t *t } } -func runCommandsHelper(request startRequest, mode string, emit func(bridgeEvent)) { - var stopping chan os.Signal - if mode == "commands-cancel" || mode == "commands-drained" { - stopping = make(chan os.Signal, 1) - signal.Notify(stopping, syscall.SIGTERM) - defer signal.Stop(stopping) - } +func runCommandsHelper(request startRequest, mode string, scanner *bufio.Scanner, emit func(bridgeEvent)) { if mode != "commands-before-ready" { emit(bridgeEvent{Type: "input_ready", SessionID: request.Resume}) } @@ -181,7 +175,9 @@ func runCommandsHelper(request startRequest, mode string, emit func(bridgeEvent) } switch mode { case "commands-cancel", "commands-drained": - <-stopping + if !scanner.Scan() { + return + } if mode == "commands-drained" { interrupted := commandEvent("pending", "after", "incomplete", "sleep 30") interrupted.Observation.Output = json.RawMessage(`"observed while draining"`) diff --git a/apps/parsar-daemon/internal/agent/claudesdk/error_classification_test.go b/apps/parsar-daemon/internal/agent/claudesdk/error_classification_test.go index 04f76f1b4..1e7b69b27 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/error_classification_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/error_classification_test.go @@ -16,7 +16,7 @@ import ( ) func TestClassifiedBridgeFailurePreservesTerminalEvidence(t *testing.T) { - for _, mode := range []string{"valid", "wrong-session", "wrong-result", "success-usage", "cancelled", "unknown", "malformed-code", "after-terminal", "scanner-error", "process-error"} { + for _, mode := range []string{"valid", "unconfirmed", "wrong-session", "wrong-result", "success-usage", "cancelled", "unknown", "malformed-code", "after-terminal", "scanner-error", "process-error"} { t.Run(mode, func(t *testing.T) { root := t.TempDir() t.Setenv("OAC_RUNTIME_HOME", root) @@ -45,8 +45,13 @@ func TestClassifiedBridgeFailurePreservesTerminalEvidence(t *testing.T) { _ = event.DecodePayload(&done) } } + if mode == "unconfirmed" { + if _, err := s.(*session).AwaitSettlement(ctx); err == nil { + t.Fatal("native diagnostic fabricated confirmed Turn settlement") + } + } want := "" - if mode == "valid" { + if mode == "valid" || mode == "unconfirmed" { want = "authentication_error" } if errors != 1 || dones != 1 || usage != 1 || failure.Code != want || done.Metadata[proto.DoneMetaAgentSessionID] != "native-session" || done.Usage.Raw["claude_sdk_result"] == nil { @@ -78,6 +83,11 @@ func runClassifiedFailureHelper(request startRequest, mode string, encode func(b e.EngineErrorCode = json.RawMessage(`{"secret":"value"}`) } encode(e) + if mode == "unconfirmed" { + confirmed, reusable := false, false + encode(bridgeEvent{Type: "turn_settled", Confirmed: &confirmed, Reusable: &reusable, Reason: "native_execution_unavailable"}) + os.Exit(0) + } if mode == "process-error" { os.Exit(7) } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor.go b/apps/parsar-daemon/internal/agent/claudesdk/executor.go new file mode 100644 index 000000000..0abc0ca47 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor.go @@ -0,0 +1,314 @@ +package claudesdk + +import ( + "context" + "encoding/json" + "errors" + "io" + "slices" + "strings" + "sync" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +type executor struct { + mu sync.Mutex + base *session + start startRequest + active *session + ready chan error + done chan struct{} + invalid bool + nativeID string +} + +func NewExecutorFactory(config Config) agent.ExecutorFactory { + config.Env = slices.Clone(config.Env) + if config.Workspace != nil { + workspace := *config.Workspace + workspace.ProtectedDirs = slices.Clone(workspace.ProtectedDirs) + config.Workspace = &workspace + } + checked := &runtimeCheckCache{} + return func(ctx context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + if ctx == nil { + ctx = context.Background() + } + if req.RunID != "" || len(req.Input) != 0 || req.ConversationID != "" { + return nil, errors.New("claudesdk: Executor preparation cannot submit input") + } + start, env, err := prepareConfiguration(config, req) + if err != nil { + return nil, err + } + info, err := checked.check(ctx, config) + if err != nil { + return nil, err + } + if err = validateExecutorFeatures(info, start); err != nil { + return nil, err + } + start.Type = "executor_prepare" + base, err := launch(ctx, config, start, env) + if err != nil { + return nil, err + } + base.reads.supported = slices.Contains(info.Features, "workspace_read") + base.directories.supported = slices.Contains(info.Features, "workspace_directory") + e := &executor{base: base, start: start, ready: make(chan error, 1), done: make(chan struct{}), nativeID: start.Resume} + go e.read() + if err = e.write(start); err == nil { + select { + case err = <-e.ready: + case <-ctx.Done(): + err = ctx.Err() + } + } + if err != nil { + closeCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + defer cancel() + if closeErr := e.Close(closeCtx); closeErr != nil { + return e, errors.Join(err, closeErr) + } + return nil, err + } + return e, nil + } +} + +func validateExecutorFeatures(info RuntimeInfo, start startRequest) error { + if info.Protocol != 3 { + return errors.New("claudesdk: Executor bridge protocol is unavailable") + } + if start.Workspace != nil && !info.supportsWorkspacePreparation() { + return errors.New("claudesdk: workspace preparation is unavailable") + } + if start.OutputFormat != nil && (!info.SupportsStructuredOutput() || start.Workspace != nil && !info.SupportsWorkspaceStructuredOutput()) { + return errors.New("claudesdk: packaged runtime does not support workspace structured output") + } + if start.Subagents != nil && !info.SupportsSubagents() { + return errors.New("claudesdk: subagent resources are unavailable") + } + if start.Workspace != nil && start.observeFunctions && !info.supportsWorkspaceCommands() { + return errors.New("claudesdk: packaged runtime does not support workspace command observations") + } + if start.Workspace != nil && len(start.Functions) > 0 && !info.SupportsWorkspaceFunctions() { + return errors.New("claudesdk: workspace functions are unavailable") + } + if start.MCPHTTPServers != nil { + if !info.SupportsHTTPMCP() { + return errors.New("claudesdk: packaged runtime does not support HTTP MCP") + } + for _, server := range *start.MCPHTTPServers { + if server.Required && !info.SupportsHTTPMCPRequired() { + return errors.New("claudesdk: packaged runtime does not support required HTTP MCP") + } + if server.BearerTokenEnvVar != "" && !info.SupportsHTTPMCPBearer() { + return errors.New("claudesdk: packaged runtime does not support authenticated HTTP MCP") + } + } + } + return nil +} + +func (e *executor) write(value any) error { + e.base.writeMu.Lock() + defer e.base.writeMu.Unlock() + return json.NewEncoder(e.base.process.Stdin).Encode(value) +} + +func (e *executor) read() { + stderrDone := make(chan struct{}) + go func() { _, _ = io.Copy(io.Discard, e.base.process.Stderr); close(stderrDone) }() + scanner := e.base.bridgeOutput() + ready := false + readyFailure := errors.New("claudesdk: Executor readiness failed") + for scanner.Scan() { + raw := append([]byte(nil), scanner.Bytes()...) + var event bridgeEvent + if json.Unmarshal(raw, &event) != nil { + e.base.process.Cancel() + break + } + if !ready { + if event.Type != "executor_ready" || event.Protocol != 3 || event.TurnID != "" { + if event.Type == "error" { + readyFailure = bridgeFailure(event.Code) + } + e.base.process.Cancel() + break + } + ready = true + e.ready <- nil + continue + } + e.mu.Lock() + turn := e.active + valid := turn != nil && !turn.nativeEnded && event.TurnID == turn.runID + if valid && event.Type == "turn_settled" { + turn.nativeEnded = true + } + e.mu.Unlock() + if !valid { + e.base.process.Cancel() + break + } + select { + case turn.frames <- raw: + case <-e.base.process.Context().Done(): + } + } + e.base.process.Cancel() + for scanner.Scan() { + } + <-stderrDone + _ = e.base.process.Wait() + e.base.stopWorkspaceReads() + e.base.stopWorkspaceDirectories() + e.mu.Lock() + e.invalid = true + if e.active != nil { + close(e.active.frames) + } + e.mu.Unlock() + if !ready { + e.ready <- readyFailure + } + close(e.done) +} + +func (e *executor) StartTurn(ctx context.Context, run string, input proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + if ctx == nil { + ctx = context.Background() + } + if strings.TrimSpace(run) == "" || input.Validate() != nil || out == nil || ctx.Err() != nil { + return nil, errors.New("claudesdk: Turn requires live context, identity, input and output") + } + e.mu.Lock() + if e.invalid || e.active != nil || e.base.process.Context().Err() != nil { + e.mu.Unlock() + return nil, errors.New("claudesdk: Executor is unavailable") + } + s := &session{owner: e, runID: run, process: e.base.process, writeMu: e.base.writeMu, frames: make(chan []byte, 64), functions: functionState{calls: map[string]*pendingFunction{}}, settled: make(chan struct{}), outputDone: make(chan struct{}), cancelOutput: make(chan struct{})} + start := e.start + start.Resume = e.nativeID + e.active = s + e.mu.Unlock() + go s.runTurn(start, out) + // Once the write is attempted, a lost receipt has an unknown input outcome. + stopWrite := context.AfterFunc(ctx, s.invalidate) + err := e.write(struct { + Type string `json:"type"` + TurnID string `json:"turn_id"` + Input proto.MessageInput `json:"input"` + }{"turn_start", run, input}) + stopWrite() + if err != nil { + s.invalidate() + return s, errors.New("claudesdk: Turn input delivery is unknown") + } + return s, nil +} + +func (e *executor) finishTurn(s *session) { + e.mu.Lock() + if e.active != s { + e.mu.Unlock() + return + } + if native, _ := s.outcome.Metadata[proto.DoneMetaAgentSessionID].(string); native != "" { + if e.nativeID != "" && e.nativeID != native { + e.invalid = true + } else { + e.nativeID = native + } + } + if !s.turnSettlement.Reusable { + e.invalid = true + } + invalid := e.invalid + if invalid { + s.turnSettlement.Reusable = false + if s.turnSettlement.Reason == "" { + s.turnSettlement.Reason = "executor_invalidated" + } + } + e.active = nil + e.mu.Unlock() + if invalid { + e.base.process.Cancel() + <-e.done + } +} + +func (e *executor) retire() { + e.mu.Lock() + e.invalid = true + e.mu.Unlock() + e.base.process.Cancel() + <-e.done +} + +func (e *executor) Close(ctx context.Context) error { + if ctx == nil { + ctx = context.Background() + } + e.mu.Lock() + e.invalid = true + active := e.active + e.mu.Unlock() + e.base.process.Cancel() + select { + case <-e.done: + if active != nil { + select { + case <-active.outputDone: + case <-ctx.Done(): + return ctx.Err() + } + } + return nil + case <-ctx.Done(): + return ctx.Err() + } +} + +func (s *session) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + if ctx == nil { + ctx = context.Background() + } + select { + case <-s.outputDone: + return s.turnSettlement, errors.Join(s.settlementErr, s.outputErr) + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} + +var _ agent.Executor = (*executor)(nil) +var _ agent.Turn = (*session)(nil) + +// A timed-out operation retains its original Turn identity. Its callback cannot +// retire a healthy successor after that operation has otherwise settled. +func (s *session) invalidate() { + if s.owner == nil { + s.process.Cancel() + return + } + s.owner.mu.Lock() + defer s.owner.mu.Unlock() + if s.owner.active == s { + s.owner.invalid = true + s.process.Cancel() + } +} + +func (e *executor) ReadWorkspaceFile(ctx context.Context, path string, maxBytes int) (agent.WorkspaceReadResult, error) { + return e.base.ReadWorkspaceFile(ctx, path, maxBytes) +} +func (e *executor) ListWorkspaceDirectory(ctx context.Context, path string, maxEntries int) (agent.WorkspaceDirectoryResult, error) { + return e.base.ListWorkspaceDirectory(ctx, path, maxEntries) +} diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor_confirmation_test.go b/apps/parsar-daemon/internal/agent/claudesdk/executor_confirmation_test.go new file mode 100644 index 000000000..631f52793 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor_confirmation_test.go @@ -0,0 +1,71 @@ +//go:build unix + +package claudesdk + +import ( + "context" + "encoding/json" + "testing" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func TestExecutorNativeConfirmationSurvivesCleanup(t *testing.T) { + for _, mode := range []string{"unknown_cancel", "queued_cancel", "missing_confirmation", "pending_function", "pending_function_unconfirmed", "pending_input", "confirmed_closed"} { + t.Run(mode, func(t *testing.T) { + config, req := persistentConfig(t, mode) + if mode == "pending_function" || mode == "pending_function_unconfirmed" { + req.FunctionTools = []proto.FunctionTool{{Name: "lookup", Parameters: json.RawMessage(`{"type":"object"}`)}} + } + owner, err := NewExecutorFactory(config)(t.Context(), req) + if err != nil { + t.Fatal(err) + } + defer owner.Close(context.Background()) + turn, out := consumeExecutorTurn(t, owner, "cancel", "wait") + if event := <-out; event.Type != proto.TypeDelta { + t.Fatal("initial output missing") + } + if mode == "pending_function" || mode == "pending_function_unconfirmed" { + if event := <-out; event.Type != proto.TypeFunctionCall { + t.Fatal("function obligation missing") + } + } + var inputReceipt chan error + if mode == "pending_input" { + inputReceipt = make(chan error, 1) + go func() { + inputReceipt <- turn.(*session).SteerWithReceipt(t.Context(), proto.PromptSteerPayload{InputID: "input", Input: proto.TextInput("extra")}, nil) + }() + if event := <-out; event.Type != proto.TypeDelta { + t.Fatal("input write barrier missing") + } + } + cancelErr := turn.Cancel(t.Context()) + for range out { + } + settlement, settlementErr := turn.AwaitSettlement(t.Context()) + confirmed := mode == "confirmed_closed" || mode == "pending_function" + reusable := mode == "pending_function" + if (cancelErr == nil) != confirmed || (settlementErr == nil) != confirmed || settlement.Reusable != reusable { + t.Fatalf("cancel=%v settlement=%+v err=%v", cancelErr, settlement, settlementErr) + } + if inputReceipt != nil { + if err := <-inputReceipt; err == nil { + t.Fatal("missing native input receipt was confirmed") + } + } + // Resource cleanup succeeds independently. It must not erase the immutable + // native outcome used by repeat cancellation and Runtime acknowledgement. + if err := owner.Close(t.Context()); err != nil { + t.Fatal(err) + } + if (turn.Cancel(t.Context()) == nil) != confirmed { + t.Fatal("Close changed cancellation confirmation") + } + if _, err := turn.AwaitSettlement(t.Context()); (err == nil) != confirmed { + t.Fatal("Close changed Turn confirmation") + } + }) + } +} diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor_fixture_test.go b/apps/parsar-daemon/internal/agent/claudesdk/executor_fixture_test.go new file mode 100644 index 000000000..55768099f --- /dev/null +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor_fixture_test.go @@ -0,0 +1,54 @@ +//go:build unix + +package claudesdk + +import ( + "bufio" + "encoding/json" + "os" +) + +func helperTurn(scanner *bufio.Scanner, request *startRequest) (func(bridgeEvent), func()) { + _ = json.NewEncoder(os.Stdout).Encode(bridgeEvent{Type: "executor_ready", Protocol: 3}) + if !scanner.Scan() { + os.Exit(4) + } + var start struct { + Type string `json:"type"` + TurnID string `json:"turn_id"` + Input json.RawMessage `json:"input"` + } + if json.Unmarshal(scanner.Bytes(), &start) != nil || start.Type != "turn_start" || start.TurnID == "" || json.Unmarshal(start.Input, &request.Input) != nil { + os.Exit(5) + } + return helperTurnOutput(scanner, start.TurnID) +} + +func helperTurnOutput(scanner *bufio.Scanner, id string) (func(bridgeEvent), func()) { + terminal := false + reusable := true + encode := func(event bridgeEvent) { + event.TurnID = id + _ = json.NewEncoder(os.Stdout).Encode(event) + if event.Type == "result" || event.Type == "error" { + terminal = true + if event.Type == "error" && event.Code != "cancelled" { + reusable = false + } + } + } + encode(bridgeEvent{Type: "turn_started"}) + return encode, func() { + if !terminal { + return + } + reason := "" + if !reusable { + reason = "native_error" + } + confirmed := true + encode(bridgeEvent{Type: "turn_settled", Confirmed: &confirmed, Reusable: &reusable, Reason: reason}) + for scanner.Scan() { + } + } +} diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor_live_linux_test.go b/apps/parsar-daemon/internal/agent/claudesdk/executor_live_linux_test.go new file mode 100644 index 000000000..a2ac5f536 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor_live_linux_test.go @@ -0,0 +1,199 @@ +//go:build linux + +package claudesdk + +import ( + "bytes" + "context" + "encoding/json" + "fmt" + "os" + "path/filepath" + "slices" + "strconv" + "strings" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/google/uuid" +) + +// Explicit opt-in acceptance; normal test runs never contact a provider. +func TestLiveClaudeExecutorReuseAndCancel(t *testing.T) { + entry, keyPath, proof := os.Getenv("OAC_TEST_CLAUDE_EXECUTOR_ENTRYPOINT"), os.Getenv("OAC_TEST_CLAUDE_EXECUTOR_KEY_FILE"), os.Getenv("OAC_TEST_CLAUDE_EXECUTOR_PROOF_DIR") + if entry == "" || keyPath == "" || proof == "" { + t.Skip("explicit installed runtime, private key file and proof directory required") + } + if !filepath.IsAbs(proof) { + t.Fatal("absolute proof directory required") + } + if err := os.MkdirAll(proof, 0700); err != nil { + t.Fatal(err) + } + key, err := os.ReadFile(keyPath) + if err != nil { + t.Fatal("cannot read selected credential file") + } + t.Setenv("OAC_RUNTIME_HOME", proof) + config := Config{Node: os.Getenv("OAC_TEST_CLAUDE_EXECUTOR_NODE"), Entrypoint: entry, StateDir: filepath.Join(proof, "state"), Env: []string{ + "ANTHROPIC_BASE_URL=https://api.moonshot.cn/anthropic", "ANTHROPIC_AUTH_TOKEN=" + strings.TrimSpace(string(key)), "ANTHROPIC_API_KEY=", "CLAUDE_CODE_OAUTH_TOKEN=", "CLAUDE_CODE_DISABLE_EXPERIMENTAL_BETAS=1", + "ANTHROPIC_DEFAULT_SONNET_MODEL=kimi-k3", "ANTHROPIC_DEFAULT_OPUS_MODEL=kimi-k3", "ANTHROPIC_DEFAULT_HAIKU_MODEL=kimi-k3", + }} + if proxy := os.Getenv("OAC_TEST_CLAUDE_EXECUTOR_HTTP_PROXY"); proxy != "" { + config.Env = append(config.Env, "HTTP_PROXY="+proxy, "HTTPS_PROXY="+proxy, "http_proxy="+proxy, "https_proxy="+proxy) + } + ctx, cancel := context.WithTimeout(t.Context(), 4*time.Minute) + defer cancel() + readiness, err := CheckRuntime(ctx, config) + if err != nil { + t.Fatal("installed runtime readiness failed", err) + } + type turnEvidence struct { + Run string `json:"run"` + NodePID int `json:"node_pid"` + NativePIDs []int `json:"native_pids"` + FirstTextMS int64 `json:"first_text_ms"` + ElapsedMS int64 `json:"elapsed_ms"` + CancelMS int64 `json:"cancel_ms,omitempty"` + Cancelled bool `json:"cancelled"` + CancellationError string `json:"cancellation_error,omitempty"` + Settlement agent.TurnSettlement `json:"settlement"` + SettlementError string `json:"settlement_error,omitempty"` + Done proto.DonePayload `json:"done"` + Errors []string `json:"errors,omitempty"` + } + evidence := struct { + Runtime RuntimeInfo `json:"runtime"` + PrepareMS int64 `json:"prepare_ms"` + Turns []turnEvidence `json:"turns"` + Recovered bool `json:"recovered_after_cancel"` + Closed bool `json:"closed"` + }{Runtime: readiness} + persist := func() { + raw, _ := json.MarshalIndent(evidence, "", " ") + _ = os.WriteFile(filepath.Join(proof, "executor-evidence.json"), raw, 0600) + } + defer persist() + request := proto.PromptRequestPayload{StrictResume: true, DisableExecutionEnvironment: true, DisableSubagents: true, ObserveMessages: true, ExecutionControls: &proto.ExecutionControls{WebSearch: "disabled", TextVerbosity: "medium"}, AgentOptions: map[string]any{"model": "kimi-k3", "system_prompt": "Follow requested formats briefly. Remember the exact verification marker across the conversation. Use no tools."}} + factory := NewExecutorFactory(config) + prepared := time.Now() + owner, err := factory(ctx, request) + if err != nil { + t.Fatal("executor preparation failed", err) + } + evidence.PrepareMS = time.Since(prepared).Milliseconds() + defer func() { + closeCtx, stop := context.WithTimeout(context.Background(), 10*time.Second) + defer stop() + evidence.Closed = owner.Close(closeCtx) == nil + persist() + }() + nativeChildren := func(pid int) []int { + raw, _ := os.ReadFile(fmt.Sprintf("/proc/%d/task/%d/children", pid, pid)) + var ids []int + for _, value := range strings.Fields(string(raw)) { + child, _ := strconv.Atoi(value) + args, _ := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", child)) + if bytes.Contains(args, []byte("\x00--input-format\x00stream-json\x00")) && bytes.Contains(args, []byte("\x00--output-format\x00stream-json\x00")) { + ids = append(ids, child) + } + } + slices.Sort(ids) + return ids + } + run := func(id, prompt string, cancelOnText bool) turnEvidence { + t.Helper() + started := time.Now() + record := turnEvidence{Run: id, NodePID: owner.(*executor).base.process.Cmd.Process.Pid} + output := make(chan proto.Envelope, 128) + turn, err := owner.StartTurn(ctx, id, proto.TextInput(prompt), output) + if err != nil || turn == nil { + t.Fatal("Turn start failed", err) + } + cancellation := make(chan error, 1) + var cancelAt time.Time + for event := range output { + if event.ID != id { + t.Fatal("output crossed Turn identity") + } + if event.Type == proto.TypeDelta && record.FirstTextMS == 0 { + record.FirstTextMS = time.Since(started).Milliseconds() + record.NativePIDs = nativeChildren(record.NodePID) + if cancelOnText { + record.Cancelled = true + cancelAt = time.Now() + go func() { + cancelCtx, stop := context.WithTimeout(ctx, 30*time.Second) + defer stop() + cancellation <- turn.Cancel(cancelCtx) + }() + } + } + if event.Type == proto.TypeError { + var failure proto.ErrorPayload + _ = event.DecodePayload(&failure) + record.Errors = append(record.Errors, failure.Error) + } + if event.Type == proto.TypeDone { + _ = event.DecodePayload(&record.Done) + } + } + if record.Cancelled { + err := <-cancellation + record.CancelMS = time.Since(cancelAt).Milliseconds() + if err != nil { + record.CancellationError = err.Error() + } + } + record.Settlement, err = turn.AwaitSettlement(ctx) + if err != nil { + record.SettlementError = err.Error() + } + record.ElapsedMS = time.Since(started).Milliseconds() + evidence.Turns = append(evidence.Turns, record) + persist() + t.Logf("turn=%s bridge_pid=%d native_pids=%v reusable=%t first_text_ms=%d total_ms=%d", id, record.NodePID, record.NativePIDs, record.Settlement.Reusable, record.FirstTextMS, record.ElapsedMS) + return record + } + marker := "REUSE-" + uuid.NewString() + first := run("first", "Remember this marker: "+marker+". Reply with exactly the marker.", false) + if first.SettlementError != "" || !first.Settlement.Reusable || !strings.Contains(first.Done.Content, marker) { + t.Fatal("first Turn failed or was not reusable") + } + second := run("second", "What exact marker did I give you? Reply with only that marker.", false) + if second.SettlementError != "" || !second.Settlement.Reusable || !strings.Contains(second.Done.Content, marker) || first.NodePID != second.NodePID || len(first.NativePIDs) == 0 || !slices.Equal(first.NativePIDs, second.NativePIDs) { + t.Fatal("ordinary Turns did not retain native execution and history") + } + interrupted := run("cancel", "List the numbers 1 through 10000, one number per line, without stopping early.", true) + if !interrupted.Cancelled { + t.Fatal("native output ended before cancellation boundary") + } + if interrupted.SettlementError != "" || !interrupted.Settlement.Reusable { + if interrupted.Settlement.Reason == "" && interrupted.SettlementError == "" { + t.Fatal("invalid executor omitted its reason") + } + if err := owner.Close(ctx); err != nil { + t.Fatal("invalid owner cleanup was not confirmed", err) + } + native, _ := interrupted.Done.Metadata[proto.DoneMetaAgentSessionID].(string) + if native == "" { + native, _ = second.Done.Metadata[proto.DoneMetaAgentSessionID].(string) + } + request.AgentSessionID = native + request.RequireExistingNativeSession = true + owner, err = factory(ctx, request) + if err != nil { + t.Fatal("history recovery failed", err) + } + evidence.Recovered = true + } + continued := run("continued", "What exact marker did I originally give you? Reply with only the marker.", false) + if continued.SettlementError != "" || !continued.Settlement.Reusable || !strings.Contains(continued.Done.Content, marker) { + t.Fatal("history did not continue after cancellation") + } + if !evidence.Recovered && (continued.NodePID != second.NodePID || !slices.Equal(continued.NativePIDs, second.NativePIDs)) { + t.Fatal("reusable cancellation replaced native execution") + } +} diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor_test.go b/apps/parsar-daemon/internal/agent/claudesdk/executor_test.go new file mode 100644 index 000000000..39d18f03a --- /dev/null +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor_test.go @@ -0,0 +1,301 @@ +//go:build unix + +package claudesdk + +import ( + "bufio" + "context" + "encoding/json" + "os" + "path/filepath" + "strconv" + "strings" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func persistentConfig(t *testing.T, mode string) (Config, proto.PromptRequestPayload) { + t.Helper() + root := t.TempDir() + t.Setenv("OAC_RUNTIME_HOME", root) + entry := filepath.Join(root, "worker") + if err := os.WriteFile(entry, []byte("version-one"), 0600); err != nil { + t.Fatal(err) + } + return Config{Node: os.Args[0], Entrypoint: entry, StateDir: filepath.Join(root, "state"), Env: []string{"GO_CLAUDE_EXECUTOR_HELPER=1", "SDK_EXECUTOR_DIR=" + root, "SDK_EXECUTOR_MODE=" + mode, "GORACE=atexit_sleep_ms=0"}}, proto.PromptRequestPayload{DisableExecutionEnvironment: true, AgentOptions: map[string]any{"model": "fixture"}} +} + +func runPersistentExecutorHelper() { + root := os.Getenv("SDK_EXECUTOR_DIR") + if len(os.Args) > 1 && strings.HasSuffix(os.Args[1], "runtime_check.js") { + file, _ := os.OpenFile(filepath.Join(root, "probes"), os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0600) + if file != nil { + _, _ = file.WriteString("probe\n") + _ = file.Close() + } + printlnReport := json.NewEncoder(os.Stdout) + _ = printlnReport.Encode(RuntimeInfo{Type: "runtime_ready", Protocol: 3, Node: "fixture", SDK: "fixture", MCP: "fixture", Native: "fixture"}) + return + } + scanner := bufio.NewScanner(os.Stdin) + if !scanner.Scan() { + return + } + var prepare map[string]json.RawMessage + if json.Unmarshal(scanner.Bytes(), &prepare) != nil || string(prepare["type"]) != `"executor_prepare"` || prepare["input"] != nil || prepare["run_id"] != nil { + os.Exit(4) + } + _ = os.WriteFile(filepath.Join(root, "prepared"), []byte(strconv.Itoa(os.Getpid())), 0600) + encode := func(event bridgeEvent) { _ = json.NewEncoder(os.Stdout).Encode(event) } + encode(bridgeEvent{Type: "executor_ready", Protocol: 3}) + if os.Getenv("SDK_EXECUTOR_MODE") == "block" { + time.Sleep(time.Hour) + return + } + var active, old string + settle := func(cancel bool) { + encode(bridgeEvent{Type: "input_closed", TurnID: active, SessionID: "native-persistent"}) + if cancel { + encode(bridgeEvent{Type: "error", TurnID: active, Code: "cancelled"}) + } else { + encode(bridgeEvent{Type: "result", TurnID: active, SessionID: "native-persistent", Text: active}) + } + confirmed, reusable, reason := true, true, "" + switch os.Getenv("SDK_EXECUTOR_MODE") { + case "unknown_cancel", "queued_cancel", "pending_function_unconfirmed": + confirmed, reusable, reason = false, false, "cancellation_unconfirmed" + case "confirmed_closed": + reusable, reason = false, "native_closed" + } + event := bridgeEvent{Type: "turn_settled", TurnID: active, Confirmed: &confirmed, Reusable: &reusable, Reason: reason} + if os.Getenv("SDK_EXECUTOR_MODE") == "missing_confirmation" { + event.Confirmed = nil + } + encode(event) + old, active = active, "" + } + for scanner.Scan() { + var command struct { + Type string `json:"type"` + TurnID string `json:"turn_id"` + Input proto.MessageInput `json:"input"` + } + if json.Unmarshal(scanner.Bytes(), &command) != nil { + return + } + switch command.Type { + case "turn_start": + if active != "" { + os.Exit(5) + } + active = command.TurnID + if os.Getenv("SDK_EXECUTOR_MODE") == "late" && old != "" { + encode(bridgeEvent{Type: "delta", TurnID: old, Delta: "late"}) + continue + } + encode(bridgeEvent{Type: "turn_started", TurnID: active}) + encode(bridgeEvent{Type: "input_ready", TurnID: active, SessionID: "native-persistent"}) + encode(bridgeEvent{Type: "delta", TurnID: active, Delta: "partial"}) + if strings.HasPrefix(os.Getenv("SDK_EXECUTOR_MODE"), "pending_function") { + encode(bridgeEvent{Type: "function_call", TurnID: active, Call: &proto.FunctionCallPayload{CallID: "call", Name: "lookup", Arguments: json.RawMessage("{}")}}) + } + if len(command.Input) > 0 && len(command.Input[0].Content) > 0 && command.Input[0].Content[0].Text != nil && *command.Input[0].Content[0].Text == "wait" { + continue + } + settle(false) + case "steer": + // A full bridge write has happened, but no native input receipt exists. + encode(bridgeEvent{Type: "delta", TurnID: active, Delta: "steer-written"}) + case "turn_cancel": + if active == command.TurnID { + settle(true) + } + } + } +} + +func consumeExecutorTurn(t *testing.T, owner agent.Executor, id, input string) (agent.Turn, <-chan proto.Envelope) { + t.Helper() + out := make(chan proto.Envelope, 16) + turn, err := owner.StartTurn(t.Context(), id, proto.TextInput(input), out) + if err != nil || turn == nil { + t.Fatalf("StartTurn: %v", err) + } + return turn, out +} +func awaitExecutorTurn(t *testing.T, turn agent.Turn, out <-chan proto.Envelope, reusable bool) { + t.Helper() + for event := range out { + if event.Type == proto.TypeDone { + var done proto.DonePayload + _ = event.DecodePayload(&done) + if done.Metadata[proto.DoneMetaAgentSessionID] != "native-persistent" { + t.Fatal("native continuity missing", done) + } + } + } + settlement, err := turn.AwaitSettlement(t.Context()) + if err != nil || settlement.Reusable != reusable { + t.Fatalf("settlement: %+v %v", settlement, err) + } +} +func TestExecutorRetainsProcessAcrossTurnsAndCancellation(t *testing.T) { + config, req := persistentConfig(t, "") + owner, err := NewExecutorFactory(config)(t.Context(), req) + if err != nil { + t.Fatal(err) + } + defer owner.Close(context.Background()) + pid := owner.(*executor).base.process.Cmd.Process.Pid + first, out := consumeExecutorTurn(t, owner, "first", "hello") + awaitExecutorTurn(t, first, out, true) + next, out := consumeExecutorTurn(t, owner, "next", "wait") + if event := <-out; event.Type != proto.TypeDelta { + t.Fatal(event.Type) + } + if err := first.Cancel(t.Context()); err != nil { + t.Fatal(err) + } + if err := next.Cancel(t.Context()); err != nil { + t.Fatal(err) + } + awaitExecutorTurn(t, next, out, true) + third, out := consumeExecutorTurn(t, owner, "third", "hello") + awaitExecutorTurn(t, third, out, true) + if owner.(*executor).base.process.Cmd.Process.Pid != pid { + t.Fatal("native process was replaced") + } + select { + case <-owner.(*executor).base.process.Done(): + t.Fatal("native process exited between Turns") + default: + } +} +func TestExecutorLateTurnEventInvalidatesWithoutRetargeting(t *testing.T) { + config, req := persistentConfig(t, "late") + owner, err := NewExecutorFactory(config)(t.Context(), req) + if err != nil { + t.Fatal(err) + } + defer owner.Close(context.Background()) + first, out := consumeExecutorTurn(t, owner, "first", "hello") + awaitExecutorTurn(t, first, out, true) + second, out := consumeExecutorTurn(t, owner, "second", "hello") + for frame := range out { + if frame.Type == proto.TypeDelta { + t.Fatal("old Turn event reached successor") + } + } + if _, err := second.AwaitSettlement(t.Context()); err == nil { + t.Fatal("unknown native outcome marked settled") + } + next := make(chan proto.Envelope, 1) + if turn, err := owner.StartTurn(t.Context(), "third", proto.TextInput("hello"), next); turn != nil || err == nil { + t.Fatal("invalid executor accepted input") + } +} +func TestExecutorCachesReadinessUntilInstalledArtifactChanges(t *testing.T) { + config, req := persistentConfig(t, "") + factory := NewExecutorFactory(config) + for range 2 { + owner, err := factory(t.Context(), req) + if err != nil { + t.Fatal(err) + } + if err := owner.Close(t.Context()); err != nil { + t.Fatal(err) + } + } + count := func() int { + raw, _ := os.ReadFile(filepath.Join(filepath.Dir(config.Entrypoint), "probes")) + return strings.Count(string(raw), "probe\n") + } + if count() != 1 { + t.Fatal("unchanged registered runtime was reprobed") + } + time.Sleep(time.Millisecond) + if err := os.WriteFile(config.Entrypoint, []byte("version-two-changed"), 0600); err != nil { + t.Fatal(err) + } + owner, err := factory(t.Context(), req) + if err != nil { + t.Fatal(err) + } + defer owner.Close(context.Background()) + if count() != 2 { + t.Fatal("changed installation reused stale readiness") + } +} + +func TestExecutorSeparatesPreInputRejectionFromUnknownWrite(t *testing.T) { + config, req := persistentConfig(t, "block") + owner, err := NewExecutorFactory(config)(t.Context(), req) + if err != nil { + t.Fatal(err) + } + defer owner.Close(context.Background()) + rejected := make(chan proto.Envelope, 1) + cancelled, stop := context.WithCancel(t.Context()) + stop() + if turn, err := owner.StartTurn(cancelled, "not-written", proto.TextInput("hello"), rejected); turn != nil || err == nil { + t.Fatal("cancelled admission reached native input") + } + select { + case <-rejected: + t.Fatal("nil Turn consumed caller output") + default: + } + e := owner.(*executor) + if err := e.base.process.Stdin.Close(); err != nil { + t.Fatal(err) + } + out := make(chan proto.Envelope, 8) + turn, err := owner.StartTurn(t.Context(), "unknown-write", proto.TextInput("hello"), out) + if turn == nil || err == nil { + t.Fatalf("attempted write must preserve unknown Turn ownership: %T %v", turn, err) + } + for range out { + } + if _, err := turn.AwaitSettlement(t.Context()); err == nil { + t.Fatal("lost input outcome became confirmed") + } +} + +func TestExecutorCancellationDeadlineInterruptsBlockedTransport(t *testing.T) { + config, req := persistentConfig(t, "block") + owner, err := NewExecutorFactory(config)(t.Context(), req) + if err != nil { + t.Fatal(err) + } + defer owner.Close(context.Background()) + turn, out := consumeExecutorTurn(t, owner, "blocked", "hello") + e := owner.(*executor) + written := make(chan error, 1) + go func() { written <- e.write(map[string]string{"padding": strings.Repeat("x", 2*1024*1024)}) }() + deadline := time.Now().Add(time.Second) + for e.base.writeMu.TryLock() { + e.base.writeMu.Unlock() + if time.Now().After(deadline) { + t.Fatal("transport write never started") + } + time.Sleep(time.Millisecond) + } + ctx, cancel := context.WithTimeout(t.Context(), 30*time.Millisecond) + defer cancel() + returned := make(chan error, 1) + go func() { returned <- turn.Cancel(ctx) }() + select { + case err := <-returned: + if err == nil { + t.Fatal("blocked cancellation invented settlement") + } + case <-time.After(time.Second): + t.Fatal("Cancel ignored its transport deadline") + } + <-written + for range out { + } +} diff --git a/apps/parsar-daemon/internal/agent/claudesdk/executor_turn.go b/apps/parsar-daemon/internal/agent/claudesdk/executor_turn.go new file mode 100644 index 000000000..789a34a90 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/claudesdk/executor_turn.go @@ -0,0 +1,242 @@ +package claudesdk + +import ( + "encoding/json" + "fmt" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "strings" + "time" +) + +func (s *session) runTurn(start startRequest, out chan<- proto.Envelope) { + defer close(s.outputDone) + defer s.owner.finishTurn(s) + defer close(out) + defer s.stopFunctions() + defer s.stopSteering() + var failure error + outputLost := false + terminalLost := false + emit := func(kind string, payload any) { + event, err := proto.NewEnvelope(kind, s.runID, payload) + if err != nil { + return + } + select { + case out <- event: + return + default: + } + var cancelled <-chan struct{} + if kind != proto.TypeDone && kind != proto.TypeError { + cancelled = s.cancelOutput + } + timer := time.NewTimer(5 * time.Second) + defer timer.Stop() + select { + case out <- event: + case <-cancelled: + outputLost = true + case <-timer.C: + outputLost = true + if kind == proto.TypeDone || kind == proto.TypeError { + terminalLost = true + } + s.invalidate() + } + } + var content strings.Builder + var result *bridgeEvent + var classifiedFailure error + var engineCode, failedResultID string + var usage proto.Usage + var usageSession string + usageIDs := map[string]bool{} + var sequence uint64 + terminal := false + mcp := mcpState{calls: map[string]proto.ToolObservation{}} + commands := commandState{calls: map[string]proto.ToolObservation{}} + settlementReceived := false + settlementConfirmed := false + cancelled := false + reusable := false + reason := "bridge_interrupted" + for raw := range s.frames { + var event bridgeEvent + if err := json.Unmarshal(raw, &event); err != nil || event.TurnID != s.runID { + failure = fmt.Errorf("claudesdk: invalid Turn event identity") + s.invalidate() + break + } + if event.Type == "turn_started" && !terminal { + continue + } + if event.Type == "turn_settled" { + if !terminal || event.Reusable == nil || event.Confirmed == nil || (*event.Reusable && !*event.Confirmed) || (!*event.Reusable && event.Reason == "") { + failure = fmt.Errorf("claudesdk: invalid Turn settlement") + s.invalidate() + break + } + settlementReceived, settlementConfirmed, reusable, reason = true, *event.Confirmed, *event.Reusable, event.Reason + break + } + if terminal { + failure = fmt.Errorf("claudesdk: invalid SDK bridge output") + s.invalidate() + break + } + switch event.Type { + case "command_observation": + if err := commands.receive(event, start, s.inputSessionID(), emit); err != nil { + failure = err + s.invalidate() + } + case "mcp_observation": + if err := mcp.receive(event, start, emit); err != nil { + failure = err + s.invalidate() + } + case "delta": + if start.ObserveMessages && event.ItemID == "" || !start.ObserveMessages && event.ItemID != "" { + failure = fmt.Errorf("claudesdk: invalid message delta identity") + s.invalidate() + break + } + content.WriteString(event.Delta) + sequence++ + emit(proto.TypeDelta, proto.DeltaPayload{ItemID: event.ItemID, Delta: event.Delta, Sequence: sequence}) + case "output_message": + message := event.Message + if !start.ObserveMessages || message == nil || message.ID == "" || + (message.Status != "in_progress" && message.Status != "completed") || + (message.Status == "completed") != (message.Text != nil) { + failure = fmt.Errorf("claudesdk: invalid message observation") + s.invalidate() + break + } + emit(proto.TypeOutputMessage, message) + case "function_call", "function_applied": + if err := s.receiveFunction(event, start, emit); err != nil { + failure = err + s.invalidate() + } + case "input_ready", "input_closed", "input_applied", "input_rejected": + if err := s.receiveInput(event, start); err != nil { + failure = err + s.invalidate() + } + case proto.TypeSubagentIdentity, proto.TypeSubagentTurn, proto.TypeSubagentItem, proto.TypeSubagentCoordination: + if start.Subagents == nil || !json.Valid(event.Fact) { + failure = fmt.Errorf("claudesdk: unrequested native child observation") + s.invalidate() + break + } + emit(event.Type, event.Fact) + case "usage": + if event.ResultID == "" || !s.matchesInputSession(event.SessionID) || event.SessionID == "" || (start.Resume != "" && event.SessionID != start.Resume) || + (usageSession != "" && usageSession != event.SessionID) || usageIDs[event.ResultID] { + failure = fmt.Errorf("claudesdk: invalid usage identity or duplicate result") + s.invalidate() + break + } + nextUsage, err := appendNativeUsage(usage, event.Usage) + failure = err + if failure != nil { + s.invalidate() + break + } + usage = nextUsage + usageIDs[event.ResultID] = true + usageSession = event.SessionID + failedResultID = "" + var nativeResult struct { + Subtype string `json:"subtype"` + IsError bool `json:"is_error"` + } + if json.Unmarshal(event.Usage, &nativeResult) == nil && (nativeResult.Subtype == "error_during_execution" || nativeResult.Subtype == "success" && nativeResult.IsError) { + failedResultID = event.ResultID + } + emit(proto.TypeUsage, proto.UsagePayload{Usage: usage}) + case "result": + if !s.matchesInputSession(event.SessionID) || event.SessionID == "" || start.Resume != "" && event.SessionID != start.Resume || usageSession != "" && event.SessionID != usageSession || !s.functionsComplete(false) || !s.steeringComplete() || !mcp.complete() || !commands.complete() { + failure = fmt.Errorf("claudesdk: invalid native completion or unconfirmed input/result") + s.settlementErr = failure + s.invalidate() + } else { + result = &event + } + terminal = true + case "error": + cancelled = event.Code == "cancelled" + failure = bridgeFailure(event.Code) + if event.Code == "execution_failed" && event.SessionID != "" && s.matchesInputSession(event.SessionID) && + event.SessionID == usageSession && event.ResultID != "" && event.ResultID == failedResultID { + var code string + _ = json.Unmarshal(event.EngineErrorCode, &code) + engineCode, _ = proto.NormalizeEngineFailure(code, nil) + classifiedFailure = failure + } + terminal = true + default: + failure = fmt.Errorf("claudesdk: unknown SDK bridge event") + s.invalidate() + } + if failure != nil && !terminal { + break + } + } + if !settlementReceived && failure == nil { + failure = fmt.Errorf("claudesdk: Turn settlement is missing") + } + if result == nil && failure == nil { + failure = fmt.Errorf("claudesdk: SDK result is missing") + } + // Close result admission before deciding which unanswered calls cancellation settled. + s.stopFunctions() + if !s.functionsComplete(cancelled && settlementConfirmed) || !s.steeringComplete() || !mcp.complete() || !commands.complete() { + settlementConfirmed, reusable, reason = false, false, "unsettled_native_operations" + } + mcp.close(start, emit) + commands.close(start, emit) + s.stopSteering() + metadata := map[string]any{proto.DoneMetaAgentSessionType: "claude_session"} + if id := s.inputSessionID(); id != "" { + metadata[proto.DoneMetaAgentSessionID] = id + } + if failure == nil { + content.Reset() + content.WriteString(result.Text) + metadata[proto.DoneMetaAgentSessionID] = result.SessionID + } + s.outcome = proto.DonePayload{Content: content.String(), Usage: usage, Metadata: metadata} + // Publish the observed cancellation outcome before terminal delivery. + if s.settlementErr != nil || !settlementReceived || !settlementConfirmed || !reusable || outputLost || s.process.Context().Err() != nil { + s.owner.retire() + reusable = false + if reason == "" { + reason = "bridge_interrupted" + } + } + if !settlementReceived || !settlementConfirmed { + s.settlementErr = fmt.Errorf("claudesdk: native Turn settlement is unconfirmed") + } + close(s.settled) + if failure != nil { + // A valid Turn boundary replaces the former process-exit boundary. + // A native diagnostic does not itself confirm cancellation or reuse. + if failure != classifiedFailure || !settlementReceived { + engineCode = "" + } + emit(proto.TypeError, proto.ErrorPayload{Error: failure.Error(), Code: engineCode}) + } + emit(proto.TypeDone, s.outcome) + if outputLost { + s.owner.retire() + reusable, reason = false, "output_delivery_interrupted" + } + if terminalLost { + s.outputErr = fmt.Errorf("claudesdk: terminal output delivery failed") + } + s.turnSettlement = agent.TurnSettlement{Reusable: reusable, Reason: reason} +} diff --git a/apps/parsar-daemon/internal/agent/claudesdk/functions.go b/apps/parsar-daemon/internal/agent/claudesdk/functions.go index b135f8ec5..aaa56e61c 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/functions.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/functions.go @@ -115,9 +115,10 @@ func (s *session) SubmitFunctionResult(ctx context.Context, result proto.Functio return err } data, err := json.Marshal(struct { - Type string `json:"type"` + Type string `json:"type"` + TurnID string `json:"turn_id"` proto.FunctionResultPayload - }{Type: "function_result", FunctionResultPayload: result}) + }{Type: "function_result", TurnID: s.runID, FunctionResultPayload: result}) if err != nil { return err } @@ -139,7 +140,7 @@ func (s *session) SubmitFunctionResult(ctx context.Context, result proto.Functio ctx, cancel := context.WithTimeout(ctx, 10*time.Second) defer cancel() // A lost receipt has an unknown outcome; terminate this execution instead of resending. - stop := context.AfterFunc(ctx, s.process.Cancel) + stop := context.AfterFunc(ctx, s.invalidate) defer stop() s.writeMu.Lock() if err = ctx.Err(); err == nil { @@ -147,7 +148,7 @@ func (s *session) SubmitFunctionResult(ctx context.Context, result proto.Functio } s.writeMu.Unlock() if err != nil { - s.process.Cancel() + s.invalidate() return fmt.Errorf("claudesdk: function result transport failed") } select { @@ -158,11 +159,11 @@ func (s *session) SubmitFunctionResult(ctx context.Context, result proto.Functio } } -func (s *session) functionsComplete() bool { +func (s *session) functionsComplete(cancelled bool) bool { s.functions.mu.Lock() defer s.functions.mu.Unlock() for _, pending := range s.functions.calls { - if !pending.applied { + if !pending.applied && !(cancelled && pending.result == nil) { return false } } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/functions_test.go b/apps/parsar-daemon/internal/agent/claudesdk/functions_test.go index cc5c6a71a..09045a612 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/functions_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/functions_test.go @@ -75,8 +75,8 @@ func TestFunctionFactoryNativeReceipts(t *testing.T) { } case proto.TypeDelta: if mode == "functions-cancel" { - if err := running.Cancel(ctx); err != nil { - t.Fatal(err) + if err := running.Cancel(ctx); err == nil { + t.Fatal("unconfirmed function receipts became successful cancellation") } } case proto.TypeError: @@ -127,7 +127,9 @@ func runFunctionHelper(request startRequest, mode string, scanner *bufio.Scanner } if mode == "functions-cancel" { emit(bridgeEvent{Type: "delta", Delta: "waiting for confirmation"}) - time.Sleep(time.Hour) + if scanner.Scan() { + emit(bridgeEvent{Type: "error", Code: "cancelled"}) + } return } if mode == "functions-no-receipt" { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/mcp_environment_test.go b/apps/parsar-daemon/internal/agent/claudesdk/mcp_environment_test.go index 5f48890fc..e3310414b 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/mcp_environment_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/mcp_environment_test.go @@ -15,19 +15,19 @@ func TestEnvironmentMCPUsesInstalledLauncherAndSelectedCredential(t *testing.T) req := workspaceRequest() token := "selected-user-token" t.Setenv("MCP_TOKEN", "unselected-native-token") - req.LocalEnvironment = &proto.LocalEnvironment{NetworkAccess: "enabled", MCP: []proto.EnvironmentMCP{ - {PackageRoot: "plugins/local", Server: agentplugin.MCPServer{Name: "local", Type: "stdio", Command: "untrusted-package-command", Args: []string{"package-argument"}, EnvVars: []string{"MCP_TOKEN"}}}, - {PackageRoot: "plugins/remote", Server: agentplugin.MCPServer{Name: "remote", Type: "http", URL: "https://example.invalid/mcp", BearerTokenEnvVar: "MCP_TOKEN"}, BearerToken: &token}, + req.LocalEnvironment = &proto.LocalEnvironment{CapabilityRoot: "/private/runtime/capabilities", NetworkAccess: "enabled", MCP: []proto.EnvironmentMCP{ + {InstallationRoot: "/private/runtime/capabilities", WorkspaceRoot: "/private/runtime/workspace", PackageRoot: "plugins/local", Server: agentplugin.MCPServer{Name: "local", Type: "stdio", Command: "untrusted-package-command", Args: []string{"package-argument"}, EnvVars: []string{"MCP_TOKEN"}}}, + {InstallationRoot: "/private/runtime/capabilities", WorkspaceRoot: "/private/runtime/workspace", PackageRoot: "plugins/remote", Server: agentplugin.MCPServer{Name: "remote", Type: "http", URL: "https://example.invalid/mcp", BearerTokenEnvVar: "MCP_TOKEN"}, BearerToken: &token}, }} start, env, err := prepare(config, req) if err != nil { t.Fatal(err) } - if start.MCPHTTPServers != nil || len(start.Workspace.MCP) != 2 { + if start.MCPHTTPServers != nil || len(start.Workspace.MCP) != 2 || start.Workspace.CapabilityRoot != req.LocalEnvironment.CapabilityRoot { t.Fatal("environment declarations changed authority") } stdio := start.Workspace.MCP[0] - if stdio.Command != "/usr/bin/python3" || len(stdio.Args) != 6 || stdio.Args[4] != "plugins/local" || stdio.Args[5] != "local" { + if stdio.Command != "/usr/bin/python3" || len(stdio.Args) != 8 || stdio.Args[4] != "/private/runtime/capabilities" || stdio.Args[5] != "/private/runtime/workspace" || stdio.Args[6] != "plugins/local" || stdio.Args[7] != "local" { t.Fatal("stdio bypassed the shared installed entry") } raw, _ := json.Marshal(start) @@ -65,7 +65,7 @@ func TestEnvironmentMCPRejectsUnqualifiedCombinations(t *testing.T) { e.MCP[0].Server.URL = "http://example.invalid/mcp" }, } { - environment := &proto.LocalEnvironment{NetworkAccess: "enabled", MCP: []proto.EnvironmentMCP{{PackageRoot: "plugins/remote", Server: agentplugin.MCPServer{Name: "remote", Type: "http", URL: "https://example.invalid/mcp"}}}} + environment := &proto.LocalEnvironment{NetworkAccess: "enabled", MCP: []proto.EnvironmentMCP{{InstallationRoot: "/private/runtime/capabilities", WorkspaceRoot: "/private/runtime/workspace", PackageRoot: "plugins/remote", Server: agentplugin.MCPServer{Name: "remote", Type: "http", URL: "https://example.invalid/mcp"}}}} mutate(environment) if _, _, err := prepareEnvironmentMCP(environment); err == nil { t.Fatal("unsupported declaration accepted") diff --git a/apps/parsar-daemon/internal/agent/claudesdk/preparation.go b/apps/parsar-daemon/internal/agent/claudesdk/preparation.go index cdd683865..36b228698 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/preparation.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/preparation.go @@ -2,129 +2,66 @@ package claudesdk import ( "context" - "encoding/json" - "fmt" - "slices" - "strings" + "errors" "sync" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" ) +// prepared is a single admission for direct adapter callers. It uses the same +// Executor as pooled Runtime execution; workspace reads retain the owner. type prepared struct { - mu sync.Mutex - session *session - ready chan struct{} - started chan struct{} - binding *preparedStart - closed bool - failure error + mu sync.Mutex + executor *executor + session *session + binding *preparedStart + closed bool } type preparedStart struct { - runID string - prompt proto.MessageInput - out chan<- proto.Envelope + turn agent.Turn + done chan struct{} } -var _ agent.PreparedCancellation = (*prepared)(nil) - -// NewPreparationFactory retains a trusted workspace process without submitting model input. func NewPreparationFactory(config Config) agent.PreparationFactory { - return func(owner context.Context, req proto.PromptRequestPayload) (agent.Prepared, error) { - if owner == nil { - owner = context.Background() - } - if config.Workspace == nil || req.RunID != "" || len(req.Input) != 0 || req.ConversationID != "" { - return nil, fmt.Errorf("claudesdk: preparation requires workspace configuration without input or conversation") - } - snapshot := config - snapshot.Env = slices.Clone(config.Env) - workspace := *config.Workspace - workspace.ProtectedDirs = slices.Clone(workspace.ProtectedDirs) - snapshot.Workspace = &workspace - start, env, err := prepareConfiguration(snapshot, req) - if err != nil { - return nil, err - } - start.Type = "prepare" - info, err := CheckRuntime(owner, snapshot) - if err != nil || !info.supportsWorkspacePreparation() { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support workspace preparation") + factory := NewExecutorFactory(config) + return func(ctx context.Context, req proto.PromptRequestPayload) (agent.Prepared, error) { + if config.Workspace == nil { + return nil, errors.New("claudesdk: workspace preparation requires a bound workspace") } - if start.OutputFormat != nil && !info.SupportsWorkspaceStructuredOutput() { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support workspace structured output") - } - if start.Subagents != nil && !info.SupportsSubagents() { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support subagent resources") - } - if start.observeFunctions && !info.supportsWorkspaceCommands() { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support workspace command observations") - } - if len(start.Functions) > 0 && !info.SupportsWorkspaceFunctions() { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support workspace functions") - } - s, err := launch(owner, snapshot, start, env) - if err != nil { + resource, err := factory(ctx, req) + if resource == nil { return nil, err } - s.reads.supported = slices.Contains(info.Features, "workspace_read") - s.directories.supported = slices.Contains(info.Features, "workspace_directory") - p := &prepared{session: s, ready: make(chan struct{}), started: make(chan struct{})} - go s.run(owner, "", start, nil, p) - select { - case <-p.ready: - if owner.Err() == nil { - return p, nil - } - _ = p.Close() - return nil, owner.Err() - case <-s.settled: - return nil, p.failure - } + e := resource.(*executor) + return &prepared{executor: e, session: e.base}, err } } -// Start transfers ownership once; ctx bounds this operation, not the Session lifetime. -func (p *prepared) Start(ctx context.Context, runID string, prompt proto.MessageInput, out chan<- proto.Envelope) (agent.Session, error) { - if ctx == nil { - ctx = context.Background() - } +func (p *prepared) Start(ctx context.Context, run string, input proto.MessageInput, out chan<- proto.Envelope) (agent.Session, error) { p.mu.Lock() if p.closed || p.binding != nil { p.mu.Unlock() - return nil, fmt.Errorf("claudesdk: preparation is no longer available") - } - var err error - if strings.TrimSpace(runID) == "" || prompt.Validate() != nil || out == nil { - err = fmt.Errorf("claudesdk: start requires a run identity, prompt and output channel") - } else if ctx.Err() != nil { - err = ctx.Err() - } else if p.session.process.Context().Err() != nil { - err = p.session.process.Context().Err() - } else { - select { - case <-p.session.process.Done(): - err = fmt.Errorf("claudesdk: prepared process has exited") - case <-p.session.settled: - err = fmt.Errorf("claudesdk: preparation has ended") - default: - } + return nil, errors.New("claudesdk: admission is unavailable") } - if err != nil { - p.closed = true - p.mu.Unlock() - _ = p.session.Cancel(context.Background()) - return nil, err + binding := &preparedStart{done: make(chan struct{})} + p.binding = binding + p.mu.Unlock() + turn, err := p.executor.StartTurn(ctx, run, input, out) + p.mu.Lock() + binding.turn = turn + if turn == nil { + p.binding = nil } - p.binding = &preparedStart{runID: runID, prompt: prompt, out: out} - close(p.started) + close(binding.done) p.mu.Unlock() - return p.session, nil + if turn != nil { + go func() { _, _ = turn.AwaitSettlement(context.Background()); _ = p.executor.Close(context.Background()) }() + } + return turn, err } -// Close settles unused ownership and is inert after transfer. func (p *prepared) Close() error { p.mu.Lock() if p.binding != nil { @@ -133,51 +70,47 @@ func (p *prepared) Close() error { } p.closed = true p.mu.Unlock() - return p.session.Cancel(context.Background()) + return p.executor.Close(context.Background()) } -// Cancel follows the resource across transfer and waits for the existing output settlement. func (p *prepared) Cancel(ctx context.Context) error { + if ctx == nil { + ctx = context.Background() + } p.mu.Lock() p.closed = true + binding := p.binding p.mu.Unlock() - return p.session.Cancel(ctx) -} - -func (p *prepared) CancellationOutcome() proto.DonePayload { - return p.session.CancellationOutcome() -} - -func (p *prepared) awaitStart(scanner *bridgeOutput, failure error) (*preparedStart, error) { - if failure == nil { - if scanner.Scan() { - var event bridgeEvent - if json.Unmarshal(scanner.Bytes(), &event) != nil { - failure = fmt.Errorf("claudesdk: invalid SDK bridge output") - } else if event.Type == "error" { - failure = bridgeFailure(event.Code) - } else if event.Type != "prepared" { - failure = fmt.Errorf("claudesdk: SDK preparation receipt is missing") - } - } else { - failure = fmt.Errorf("claudesdk: SDK preparation receipt is missing") + if binding == nil { + return p.executor.Close(ctx) + } + select { + case <-binding.done: + default: + if err := p.executor.Close(ctx); err != nil { + return err + } + select { + case <-binding.done: + case <-ctx.Done(): + return ctx.Err() } } - if failure != nil { - p.session.process.Cancel() - return nil, failure + if binding.turn == nil { + return p.executor.Close(ctx) } - close(p.ready) - select { - case <-p.started: - case <-p.session.process.Context().Done(): - case <-p.session.process.Done(): + if err := binding.turn.Cancel(ctx); err != nil { + return err } + p.executor.retire() + return nil +} + +func (p *prepared) CancellationOutcome() proto.DonePayload { p.mu.Lock() defer p.mu.Unlock() - if p.binding != nil { - return p.binding, nil + if p.binding == nil || p.binding.turn == nil { + return proto.DonePayload{} } - p.closed = true - return nil, fmt.Errorf("claudesdk: preparation ended before start") + return p.binding.turn.CancellationOutcome() } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/preparation_fixture_test.go b/apps/parsar-daemon/internal/agent/claudesdk/preparation_fixture_test.go index 2c02bd316..a9e8e0af6 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/preparation_fixture_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/preparation_fixture_test.go @@ -50,7 +50,7 @@ func runPreparationHelper() { features = append(features, "workspace_structured_output") } } - _ = json.NewEncoder(os.Stdout).Encode(RuntimeInfo{Type: "runtime_ready", Protocol: 2, Node: "fixture", SDK: "fixture", MCP: "fixture", Native: "fixture", Features: features}) + _ = json.NewEncoder(os.Stdout).Encode(RuntimeInfo{Type: "runtime_ready", Protocol: 3, Node: "fixture", SDK: "fixture", MCP: "fixture", Native: "fixture", Features: features}) return } state := os.Getenv("CLAUDE_CONFIG_DIR") @@ -63,7 +63,7 @@ func runPreparationHelper() { _ = os.WriteFile(filepath.Join(state, "prepare.json"), raw, 0o600) var fields map[string]json.RawMessage var request startRequest - if json.Unmarshal(raw, &fields) != nil || json.Unmarshal(raw, &request) != nil || request.Type != "prepare" || fields["input"] != nil || fields["run_id"] != nil || request.Workspace == nil { + if json.Unmarshal(raw, &fields) != nil || json.Unmarshal(raw, &request) != nil || request.Type != "executor_prepare" || fields["input"] != nil || fields["run_id"] != nil || request.Workspace == nil { os.Exit(3) } emit := func(event bridgeEvent) { _ = json.NewEncoder(os.Stdout).Encode(event) } @@ -84,7 +84,7 @@ func runPreparationHelper() { time.Sleep(time.Millisecond) } } - emit(bridgeEvent{Type: "prepared"}) + emit(bridgeEvent{Type: "executor_ready", Protocol: 3}) if strings.HasPrefix(mode, "directory-") { runWorkspaceDirectoryHelper(scanner, state, mode) return @@ -99,15 +99,19 @@ func runPreparationHelper() { raw = append([]byte{}, scanner.Bytes()...) _ = os.WriteFile(filepath.Join(state, "start.json"), raw, 0o600) fields = nil - if json.Unmarshal(raw, &fields) != nil || len(fields) != 2 || string(fields["type"]) != `"start"` || string(fields["input"]) != `[{"content":[{"type":"input_text","text":"hello"}]}]` { + if json.Unmarshal(raw, &fields) != nil || len(fields) != 3 || string(fields["type"]) != `"turn_start"` || string(fields["input"]) != `[{"content":[{"type":"input_text","text":"hello"}]}]` { os.Exit(4) } + var turnID string + _ = json.Unmarshal(fields["turn_id"], &turnID) + emit, finish := helperTurnOutput(scanner, turnID) + defer finish() if mode == "cancellation" { - runCancellationHelper(request, "cancellation-wait", emit) + runCancellationHelper(request, "cancellation-wait", scanner, emit) return } if strings.HasPrefix(mode, "commands") { - runCommandsHelper(request, mode, emit) + runCommandsHelper(request, mode, scanner, emit) return } emit(bridgeEvent{Type: "input_ready", SessionID: request.Resume}) diff --git a/apps/parsar-daemon/internal/agent/claudesdk/preparation_test.go b/apps/parsar-daemon/internal/agent/claudesdk/preparation_test.go index 610c9154f..a5e1406b6 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/preparation_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/preparation_test.go @@ -78,7 +78,7 @@ func TestPreparationWaitsForReceiptAndRetainsConfiguration(t *testing.T) { if err != nil { t.Fatal(err) } - if s != p.session || s.(*session).process.Cmd.Process.Pid != pid { + if s.(*session).owner != p.executor || s.(*session).process.Cmd.Process.Pid != pid { t.Fatal("Start replaced the prepared native process") } if err := p.Close(); err != nil { @@ -184,12 +184,13 @@ func TestPreparationFailureAndUnusedRelease(t *testing.T) { if startErr == nil { t.Fatal("invalid or cancelled Start succeeded") } + err = p.Close() } if err != nil { t.Fatal(err) } select { - case <-p.session.settled: + case <-p.executor.done: case <-time.After(5 * time.Second): t.Fatal("unused process was not settled") } @@ -280,7 +281,10 @@ func TestPreparedStartRacesCloseAndCancellation(t *testing.T) { } } if err := p.Cancel(t.Context()); err != nil { - t.Fatal(err) + // A racing Close can confirm resource cleanup without proving the Turn result. + if closeErr := p.executor.Close(t.Context()); closeErr != nil { + t.Fatal(closeErr) + } } select { case <-p.session.process.Done(): diff --git a/apps/parsar-daemon/internal/agent/claudesdk/readiness.go b/apps/parsar-daemon/internal/agent/claudesdk/readiness.go index 878f3ed16..ac9a91908 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/readiness.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/readiness.go @@ -132,7 +132,7 @@ func CheckRuntime(ctx context.Context, config Config) (RuntimeInfo, error) { return RuntimeInfo{}, fmt.Errorf("claudesdk: runtime check failed") } var info RuntimeInfo - if json.Unmarshal(raw, &info) != nil || info.Type != "runtime_ready" || info.Protocol != 2 || + if json.Unmarshal(raw, &info) != nil || info.Type != "runtime_ready" || info.Protocol != 3 || info.Node == "" || info.SDK == "" || info.MCP == "" || info.Native == "" { return RuntimeInfo{}, fmt.Errorf("claudesdk: invalid runtime readiness report") } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/readiness_test.go b/apps/parsar-daemon/internal/agent/claudesdk/readiness_test.go index a4e14866c..2e4825cc0 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/readiness_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/readiness_test.go @@ -15,7 +15,7 @@ import ( "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" ) -const readyReport = `{"type":"runtime_ready","protocol":2,"node":"22.22.2","sdk":"0.3.269","mcp":"1.30.0","native":"2.1.269 (Claude Code)"}` +const readyReport = `{"type":"runtime_ready","protocol":3,"node":"22.22.2","sdk":"0.3.269","mcp":"1.30.0","native":"2.1.269 (Claude Code)"}` func TestRequiredMCPNeedsQualifiedRuntime(t *testing.T) { root := t.TempDir() @@ -128,11 +128,11 @@ func runReadinessHelper() { os.Exit(5) } } - _, _ = fmt.Fprintln(os.Stdout, strings.Replace(readyReport, `"protocol":2`, `"protocol":2,"features":["mcp_http_tools"]`, 1)) + _, _ = fmt.Fprintln(os.Stdout, strings.Replace(readyReport, `"protocol":3`, `"protocol":3,"features":["mcp_http_tools"]`, 1)) case "malformed": _, _ = fmt.Fprintln(os.Stdout, "not-json") case "wrong-protocol": - _, _ = fmt.Fprintln(os.Stdout, strings.Replace(readyReport, `"protocol":2`, `"protocol":1`, 1)) + _, _ = fmt.Fprintln(os.Stdout, strings.Replace(readyReport, `"protocol":3`, `"protocol":1`, 1)) case "missing-version": _, _ = fmt.Fprintln(os.Stdout, strings.Replace(readyReport, `"mcp":"1.30.0"`, `"mcp":""`, 1)) case "multiple": diff --git a/apps/parsar-daemon/internal/agent/claudesdk/runtime_cache.go b/apps/parsar-daemon/internal/agent/claudesdk/runtime_cache.go new file mode 100644 index 000000000..82b07f95e --- /dev/null +++ b/apps/parsar-daemon/internal/agent/claudesdk/runtime_cache.go @@ -0,0 +1,52 @@ +package claudesdk + +import ( + "context" + "fmt" + "os" + "os/exec" + "path/filepath" + "sync" +) + +type runtimeCheckCache struct { + mu sync.Mutex + stamp string + info RuntimeInfo +} + +// Registration retains the package check. Replacing an installed entrypoint, +// readiness companion, package manifest or Node binary invalidates the result. +func (c *runtimeCheckCache) check(ctx context.Context, config Config) (RuntimeInfo, error) { + node := config.Node + if node == "" { + node = "node" + } + node, err := exec.LookPath(node) + if err != nil { + return RuntimeInfo{}, err + } + root := filepath.Dir(config.Entrypoint) + stamp := "" + for _, path := range []string{node, config.Entrypoint, filepath.Join(root, "runtime_check.js"), filepath.Join(root, "../package.json"), filepath.Join(root, "../runtime-manifest.json")} { + info, err := os.Stat(path) + if err != nil { + if os.IsNotExist(err) { + stamp += path + ":missing;" + continue + } + return RuntimeInfo{}, err + } + stamp += fmt.Sprintf("%s:%d:%d;", path, info.Size(), info.ModTime().UnixNano()) + } + c.mu.Lock() + defer c.mu.Unlock() + if c.stamp == stamp && c.info.Protocol == 3 { + return c.info, nil + } + info, err := CheckRuntime(ctx, config) + if err == nil { + c.info, c.stamp = info, stamp + } + return info, err +} diff --git a/apps/parsar-daemon/internal/agent/claudesdk/session.go b/apps/parsar-daemon/internal/agent/claudesdk/session.go index fe88ea0cd..f8d7a1f82 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/session.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/session.go @@ -4,8 +4,6 @@ import ( "context" "encoding/json" "fmt" - "io" - "strings" "sync" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" @@ -14,10 +12,21 @@ import ( ) type session struct { + owner *executor + runID string + frames chan []byte + outputDone chan struct{} + turnSettlement agent.TurnSettlement + settlementErr error + outputErr error + cancelOnce sync.Once + cancelOutput chan struct{} + nativeEnded bool + reads workspaceReadState directories workspaceDirectoryState process *clirunner.Process - writeMu sync.Mutex + writeMu *sync.Mutex functions functionState steering steeringState settled chan struct{} @@ -25,284 +34,51 @@ type session struct { } func NewFactory(config Config) agent.Factory { + prepareExecutor := NewExecutorFactory(config) return func(ctx context.Context, req proto.PromptRequestPayload, out chan<- proto.Envelope) (agent.Session, error) { - if ctx == nil { - ctx = context.Background() - } - if out == nil { - return nil, fmt.Errorf("claudesdk: output channel is required") - } - if config.Workspace != nil { - runID, prompt := req.RunID, req.Input - if strings.TrimSpace(runID) == "" || prompt.Validate() != nil { - return nil, fmt.Errorf("claudesdk: run id and prompt are required") - } - req.RunID, req.Input = "", nil - prepared, err := NewPreparationFactory(config)(ctx, req) - if err != nil { - return nil, err - } - defer prepared.Close() - return prepared.Start(ctx, runID, prompt, out) - } - start, env, err := prepare(config, req) + run, input := req.RunID, req.Input + req.RunID, req.ConversationID, req.Input = "", "", nil + resource, err := prepareExecutor(ctx, req) if err != nil { return nil, err } - if start.MCPHTTPServers != nil || start.Subagents != nil { - info, err := CheckRuntime(ctx, config) - if start.Subagents != nil && (err != nil || !info.SupportsSubagents()) { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support subagent resources") - } - if start.MCPHTTPServers != nil && (err != nil || !info.SupportsHTTPMCP()) { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support HTTP MCP") - } - if start.MCPHTTPServers != nil { - for _, server := range *start.MCPHTTPServers { - if server.Required && !info.SupportsHTTPMCPRequired() { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support required HTTP MCP") - } - if server.BearerTokenEnvVar != "" && !info.SupportsHTTPMCPBearer() { - return nil, fmt.Errorf("claudesdk: packaged runtime does not support authenticated HTTP MCP") - } - } - } - } - s, err := launch(ctx, config, start, env) - if err != nil { + turn, err := resource.StartTurn(ctx, run, input, out) + if turn == nil { + _ = resource.Close(context.Background()) return nil, err } - go s.run(ctx, req.RunID, start, out, nil) - return s, nil + // Factory callers own one execution. Both entrypoints use the same + // Executor implementation; Runtime pooling uses NewExecutorFactory. + go func() { _, _ = turn.AwaitSettlement(context.Background()); _ = resource.Close(context.Background()) }() + return turn, err } } type bridgeEvent struct { - EngineErrorCode json.RawMessage `json:"engine_error_code"` - Fact json.RawMessage `json:"fact"` - InputID string `json:"input_id"` - ResultID string `json:"result_id"` - Usage json.RawMessage `json:"usage,omitempty"` - Type string `json:"type"` - Delta string `json:"delta"` - SessionID string `json:"session_id"` - Text string `json:"text"` - Code string `json:"code"` - ItemID string `json:"item_id"` - Message *proto.OutputMessagePayload `json:"message"` - Call *proto.FunctionCallPayload `json:"call"` - CallID string `json:"call_id"` - DeliveryID string `json:"delivery_id"` - ID string `json:"id"` - Stage string `json:"stage"` - Observation *proto.ToolObservation `json:"observation"` -} + EngineErrorCode json.RawMessage `json:"engine_error_code"` + TurnID string `json:"turn_id"` + Reusable *bool `json:"reusable"` + Confirmed *bool `json:"confirmed"` + Reason string `json:"reason"` + Protocol int `json:"protocol"` -func (s *session) run(ctx context.Context, runID string, start startRequest, out chan<- proto.Envelope, prepared *prepared) { - defer func() { - if out != nil { - close(out) - } - }() - defer s.stopFunctions() - defer s.stopSteering() - defer s.stopWorkspaceReads() - defer s.stopWorkspaceDirectories() - emit := func(kind string, payload any) { - event, err := proto.NewEnvelope(kind, runID, payload) - if err != nil { - return - } - // Cancellation must drain native output even if the event consumer stops. - // Terminal publication follows settlement and cannot hold up Cancel. - var stopping <-chan struct{} - if kind != proto.TypeError && kind != proto.TypeDone { - stopping = s.process.Context().Done() - } - // Preserve drained observations when the consumer can accept them immediately. - select { - case out <- event: - return - default: - } - select { - case out <- event: - case <-ctx.Done(): - case <-stopping: - } - } - stderrDone := make(chan struct{}) - go func() { _, _ = io.Copy(io.Discard, s.process.Stderr); close(stderrDone) }() - var failure error - if err := json.NewEncoder(s.process.Stdin).Encode(start); err != nil { - failure = fmt.Errorf("claudesdk: cannot submit SDK input") - s.process.Cancel() - } - scanner := s.bridgeOutput() - if prepared != nil { - binding, err := prepared.awaitStart(scanner, failure) - if binding == nil { - prepared.failure, _ = s.drain(scanner, stderrDone, err) - close(s.settled) - return - } - runID, out = binding.runID, binding.out - s.writeMu.Lock() - err = json.NewEncoder(s.process.Stdin).Encode(struct { - Type string `json:"type"` - Input proto.MessageInput `json:"input"` - }{Type: "start", Input: binding.prompt}) - s.writeMu.Unlock() - if err != nil { - failure = fmt.Errorf("claudesdk: cannot submit SDK input") - s.process.Cancel() - } - } - var content strings.Builder - var result *bridgeEvent - var classifiedFailure error - var engineCode string - var usage proto.Usage - var usageSession string - var failedResultID string - usageIDs := map[string]bool{} - var sequence uint64 - terminal := false - mcp := mcpState{calls: map[string]proto.ToolObservation{}} - commands := commandState{calls: map[string]proto.ToolObservation{}} - for scanner.Scan() { - var event bridgeEvent - if err := json.Unmarshal(scanner.Bytes(), &event); err != nil || terminal { - failure = fmt.Errorf("claudesdk: invalid SDK bridge output") - s.process.Cancel() - break - } - switch event.Type { - case "command_observation": - if err := commands.receive(event, start, s.inputSessionID(), emit); err != nil { - failure = err - s.process.Cancel() - } - case "mcp_observation": - if err := mcp.receive(event, start, emit); err != nil { - failure = err - s.process.Cancel() - } - case "delta": - if start.ObserveMessages && event.ItemID == "" || !start.ObserveMessages && event.ItemID != "" { - failure = fmt.Errorf("claudesdk: invalid message delta identity") - s.process.Cancel() - break - } - content.WriteString(event.Delta) - sequence++ - emit(proto.TypeDelta, proto.DeltaPayload{ItemID: event.ItemID, Delta: event.Delta, Sequence: sequence}) - case "output_message": - message := event.Message - if !start.ObserveMessages || message == nil || message.ID == "" || - (message.Status != "in_progress" && message.Status != "completed") || - (message.Status == "completed") != (message.Text != nil) { - failure = fmt.Errorf("claudesdk: invalid message observation") - s.process.Cancel() - break - } - emit(proto.TypeOutputMessage, message) - case "function_call", "function_applied": - if err := s.receiveFunction(event, start, emit); err != nil { - failure = err - s.process.Cancel() - } - case "input_ready", "input_closed", "input_applied", "input_rejected": - if err := s.receiveInput(event, start); err != nil { - failure = err - s.process.Cancel() - } - case proto.TypeSubagentIdentity, proto.TypeSubagentTurn, proto.TypeSubagentItem, proto.TypeSubagentCoordination: - if start.Subagents == nil || !json.Valid(event.Fact) { - failure = fmt.Errorf("claudesdk: unrequested native child observation") - s.process.Cancel() - break - } - emit(event.Type, event.Fact) - case "usage": - if event.ResultID == "" || !s.matchesInputSession(event.SessionID) || event.SessionID == "" || (start.Resume != "" && event.SessionID != start.Resume) || - (usageSession != "" && usageSession != event.SessionID) || usageIDs[event.ResultID] { - failure = fmt.Errorf("claudesdk: invalid usage identity or duplicate result") - s.process.Cancel() - break - } - nextUsage, err := appendNativeUsage(usage, event.Usage) - failure = err - if failure != nil { - s.process.Cancel() - break - } - usage = nextUsage - usageIDs[event.ResultID] = true - usageSession = event.SessionID - failedResultID = "" - var nativeResult struct { - Subtype string `json:"subtype"` - IsError bool `json:"is_error"` - } - if json.Unmarshal(event.Usage, &nativeResult) == nil && (nativeResult.Subtype == "error_during_execution" || nativeResult.Subtype == "success" && nativeResult.IsError) { - failedResultID = event.ResultID - } - emit(proto.TypeUsage, proto.UsagePayload{Usage: usage}) - case "result": - if !s.matchesInputSession(event.SessionID) || event.SessionID == "" || start.Resume != "" && event.SessionID != start.Resume || usageSession != "" && event.SessionID != usageSession || !s.functionsComplete() || !s.steeringComplete() || !mcp.complete() || !commands.complete() { - failure = fmt.Errorf("claudesdk: invalid native completion or unconfirmed input/result") - s.process.Cancel() - } else { - result = &event - } - terminal = true - case "error": - failure = bridgeFailure(event.Code) - if event.Code == "execution_failed" && event.SessionID != "" && s.matchesInputSession(event.SessionID) && - event.SessionID == usageSession && event.ResultID != "" && event.ResultID == failedResultID { - var code string - _ = json.Unmarshal(event.EngineErrorCode, &code) - engineCode, _ = proto.NormalizeEngineFailure(code, nil) - classifiedFailure = failure - } - terminal = true - default: - failure = fmt.Errorf("claudesdk: unknown SDK bridge event") - s.process.Cancel() - } - if failure != nil && !terminal { - break - } - } - var bridgeClean bool - failure, bridgeClean = s.drain(scanner, stderrDone, failure) - if result == nil && failure == nil { - failure = fmt.Errorf("claudesdk: SDK result is missing") - } - mcp.close(start, emit) - commands.close(start, emit) - s.stopFunctions() - s.stopSteering() - metadata := map[string]any{proto.DoneMetaAgentSessionType: "claude_session"} - if id := s.inputSessionID(); id != "" { - metadata[proto.DoneMetaAgentSessionID] = id - } - if failure == nil { - content.Reset() - content.WriteString(result.Text) - metadata[proto.DoneMetaAgentSessionID] = result.SessionID - } - s.outcome = proto.DonePayload{Content: content.String(), Usage: usage, Metadata: metadata} - // Router completion cleanup calls Cancel while consuming Done. Settle first. - close(s.settled) - if failure != nil { - if failure != classifiedFailure || !bridgeClean { - engineCode = "" - } - emit(proto.TypeError, proto.ErrorPayload{Error: failure.Error(), Code: engineCode}) - } - emit(proto.TypeDone, s.outcome) + Fact json.RawMessage `json:"fact"` + InputID string `json:"input_id"` + ResultID string `json:"result_id"` + Usage json.RawMessage `json:"usage,omitempty"` + Type string `json:"type"` + Delta string `json:"delta"` + SessionID string `json:"session_id"` + Text string `json:"text"` + Code string `json:"code"` + ItemID string `json:"item_id"` + Message *proto.OutputMessagePayload `json:"message"` + Call *proto.FunctionCallPayload `json:"call"` + CallID string `json:"call_id"` + DeliveryID string `json:"delivery_id"` + ID string `json:"id"` + Stage string `json:"stage"` + Observation *proto.ToolObservation `json:"observation"` } func launch(ctx context.Context, config Config, start startRequest, env []string) (*session, error) { @@ -314,7 +90,7 @@ func launch(ctx context.Context, config Config, start startRequest, env []string if err != nil { return nil, err } - return &session{process: process, functions: functionState{calls: map[string]*pendingFunction{}}, settled: make(chan struct{})}, nil + return &session{process: process, writeMu: &sync.Mutex{}, functions: functionState{calls: map[string]*pendingFunction{}}, settled: make(chan struct{})}, nil } func (s *session) drain(scanner *bridgeOutput, stderrDone <-chan struct{}, failure error) (error, bool) { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/session_test.go b/apps/parsar-daemon/internal/agent/claudesdk/session_test.go index 37d64ded3..2ced109bb 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/session_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/session_test.go @@ -101,6 +101,10 @@ func TestTextFactoryRejectsUnsupportedInput(t *testing.T) { } func TestMain(m *testing.M) { + if os.Getenv("GO_CLAUDE_EXECUTOR_HELPER") == "1" { + runPersistentExecutorHelper() + os.Exit(0) + } if os.Getenv("GO_CLAUDE_PREPARATION_HELPER") == "1" { runPreparationHelper() os.Exit(0) @@ -117,18 +121,23 @@ func TestMain(m *testing.M) { } func runSDKHelper() { + if len(os.Args) > 1 && strings.HasSuffix(os.Args[1], "runtime_check.js") { + fmt.Fprintln(os.Stdout, strings.TrimSuffix(readyReport, "}")+`,"features":["structured_output","subagents"]}`) + return + } scanner := bufio.NewScanner(os.Stdin) if !scanner.Scan() { os.Exit(2) } var request startRequest - if json.Unmarshal(scanner.Bytes(), &request) != nil || request.Type != "start" || *request.Input[0].Content[0].Text != "hello" || request.Model != "fake-model" || request.SystemPrompt != "instructions" { + if json.Unmarshal(scanner.Bytes(), &request) != nil || request.Type != "executor_prepare" || request.Model != "fake-model" || request.SystemPrompt != "instructions" { os.Exit(3) } - encode := func(event bridgeEvent) { _ = json.NewEncoder(os.Stdout).Encode(event) } + encode, finish := helperTurn(scanner, &request) + defer finish() mode := os.Getenv("SDK_HELPER_MODE") if strings.HasPrefix(mode, "cancellation-") { - runCancellationHelper(request, mode, encode) + runCancellationHelper(request, mode, scanner, encode) return } if strings.HasPrefix(mode, "steering-") { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/steering.go b/apps/parsar-daemon/internal/agent/claudesdk/steering.go index b87b16f1d..c9d2c0eae 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/steering.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/steering.go @@ -34,6 +34,11 @@ func (s *session) Steer(ctx context.Context, input proto.PromptSteerPayload) err // SteerWithReceipt separates a complete bridge write from native consumption. func (s *session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerPayload, written func()) error { + select { + case <-s.cancelOutput: + return agent.ErrSteeringInactive + default: + } if ctx == nil { ctx = context.Background() } @@ -42,9 +47,10 @@ func (s *session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP } data, err := json.Marshal(struct { Type string `json:"type"` + TurnID string `json:"turn_id"` InputID string `json:"input_id"` Input proto.MessageInput `json:"input"` - }{Type: "steer", InputID: input.InputID, Input: input.Input}) + }{Type: "steer", TurnID: s.runID, InputID: input.InputID, Input: input.Input}) if err != nil || len(data) > 1024*1024 { return fmt.Errorf("%w: input exceeds bridge limit", agent.ErrSteeringRejected) } @@ -74,7 +80,7 @@ func (s *session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP s.steering.mu.Unlock() // Cancellation must release a blocked write, but a lost receipt after a full // write preserves the process and unknown outcome without automatic redelivery. - stop := context.AfterFunc(ctx, s.process.Cancel) + stop := context.AfterFunc(ctx, s.invalidate) s.writeMu.Lock() if err = ctx.Err(); err == nil { _, err = s.process.Stdin.Write(append(data, '\n')) @@ -82,7 +88,7 @@ func (s *session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP s.writeMu.Unlock() stop() if err != nil { - s.process.Cancel() + s.invalidate() return fmt.Errorf("claudesdk: input transport failed") } if written != nil { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/steering_test.go b/apps/parsar-daemon/internal/agent/claudesdk/steering_test.go index 256856efc..7bd7f68d3 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/steering_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/steering_test.go @@ -147,10 +147,11 @@ func TestSteeringReceiptsAndLifecycle(t *testing.T) { if err := s.Steer(ctx, input); !errors.Is(err, agent.ErrSteeringInactive) { t.Fatal("completed execution accepted input", err) } - select { - case <-s.process.Done(): - default: - t.Fatal("completion preceded process release") + if _, err := s.AwaitSettlement(ctx); err != nil && (mode == "success" || mode == "phased" || mode == "timeout") { + t.Fatal(err) + } + if err := s.owner.Close(ctx); err != nil { + t.Fatal(err) } }) } @@ -186,7 +187,9 @@ func runSteeringHelper(request startRequest, mode string, scanner *bufio.Scanner os.Exit(3) } if mode == "steering-cancel" { - time.Sleep(time.Hour) + if scanner.Scan() { + emit(bridgeEvent{Type: "error", Code: "cancelled"}) + } return } if mode == "steering-timeout" { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/workspace.go b/apps/parsar-daemon/internal/agent/claudesdk/workspace.go index 3a58c38b2..42356a754 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/workspace.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/workspace.go @@ -30,6 +30,7 @@ type WorkspaceConfig struct { } type workspaceProfile struct { + CapabilityRoot string `json:"capability_root,omitempty"` MCP []environmentMCPServer `json:"mcp,omitempty"` Skills []agentcapabilities.InstalledSkill `json:"skills,omitempty"` ToolEnvironment bool `json:"tool_environment,omitempty"` @@ -67,6 +68,7 @@ func prepareWorkspace(config Config, req proto.PromptRequestPayload) (*workspace } if req.LocalEnvironment != nil { profile.Skills = req.LocalEnvironment.Skills + profile.CapabilityRoot = req.LocalEnvironment.CapabilityRoot } servers, credentials, err := prepareEnvironmentMCP(req.LocalEnvironment) if err != nil { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory.go b/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory.go index 14507ab80..4cbf89803 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory.go @@ -57,6 +57,9 @@ func (p *prepared) ListWorkspaceDirectory(ctx context.Context, path string, maxE } func (s *session) ListWorkspaceDirectory(ctx context.Context, path string, maxEntries int) (agent.WorkspaceDirectoryResult, error) { + if s.owner != nil { + return s.owner.base.ListWorkspaceDirectory(ctx, path, maxEntries) + } read, err := s.admitWorkspaceDirectory(ctx, path, maxEntries) if err != nil { return agent.WorkspaceDirectoryResult{}, err diff --git a/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory_test.go b/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory_test.go index f31003a9c..bc4822ce4 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/workspace_directory_test.go @@ -20,14 +20,25 @@ import ( ) func runWorkspaceDirectoryHelper(scanner *bufio.Scanner, state, mode string) { + turnID := "" for scanner.Scan() { var req struct { Type, ID string + TurnID string `json:"turn_id"` Path string `json:"directory"` Max int `json:"max_entries"` } _ = json.Unmarshal(scanner.Bytes(), &req) - if req.Type == "start" { + if req.Type == "turn_cancel" { + reusable := false + _ = json.NewEncoder(os.Stdout).Encode(bridgeEvent{Type: "error", TurnID: turnID, Code: "cancelled"}) + confirmed := true + _ = json.NewEncoder(os.Stdout).Encode(bridgeEvent{Type: "turn_settled", Confirmed: &confirmed, TurnID: turnID, Reusable: &reusable, Reason: "fixture_closed"}) + return + } + if req.Type == "turn_start" { + turnID = req.TurnID + _ = json.NewEncoder(os.Stdout).Encode(bridgeEvent{Type: "turn_started", TurnID: turnID}) _ = os.WriteFile(filepath.Join(state, "directory-started"), []byte("{}"), 0600) continue } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/workspace_launch_test.go b/apps/parsar-daemon/internal/agent/claudesdk/workspace_launch_test.go index 832b1dcce..5a3c4ad98 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/workspace_launch_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/workspace_launch_test.go @@ -24,12 +24,13 @@ test "$ANTHROPIC_AUTH_TOKEN" = selected-provider-fixture || exit 23 test "$TMPDIR" != "$CLAUDE_CONFIG_DIR/tmp" || exit 24 case "$1" in */runtime_check.js) - printf '%s\n' '{"type":"runtime_ready","protocol":2,"node":"fixture","sdk":"fixture","mcp":"fixture","native":"fixture","features":["workspace_tools","workspace_prepare"]}' ;; + printf '%s\n' '{"type":"runtime_ready","protocol":3,"node":"fixture","sdk":"fixture","mcp":"fixture","native":"fixture","features":["workspace_tools","workspace_prepare"]}' ;; *) IFS= read -r request - printf '%s\n' '{"type":"prepared"}' + printf '%s\n' '{"type":"executor_ready","protocol":3}' IFS= read -r request - printf '%s\n' '{"type":"result","session_id":"native","text":"completed"}' ;; + printf '%s\n' '{"type":"result","turn_id":"run","session_id":"native","text":"completed"}' + printf '%s\n' '{"type":"turn_settled","turn_id":"run","confirmed":true,"reusable":true,"reason":""}' ;; esac ` if err := os.WriteFile(config.Node, []byte(script), 0o700); err != nil { diff --git a/apps/parsar-daemon/internal/agent/claudesdk/workspace_read.go b/apps/parsar-daemon/internal/agent/claudesdk/workspace_read.go index 9ed4b64c7..5f0a4052d 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/workspace_read.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/workspace_read.go @@ -57,6 +57,9 @@ func (p *prepared) ReadWorkspaceFile(ctx context.Context, path string, maxBytes } func (s *session) ReadWorkspaceFile(ctx context.Context, path string, maxBytes int) (agent.WorkspaceReadResult, error) { + if s.owner != nil { + return s.owner.base.ReadWorkspaceFile(ctx, path, maxBytes) + } read, err := s.admitWorkspaceRead(ctx, path, maxBytes) if err != nil { return agent.WorkspaceReadResult{}, err diff --git a/apps/parsar-daemon/internal/agent/claudesdk/workspace_read_test.go b/apps/parsar-daemon/internal/agent/claudesdk/workspace_read_test.go index 61209b63b..d9bfa2221 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/workspace_read_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/workspace_read_test.go @@ -20,13 +20,24 @@ import ( ) func runWorkspaceReadHelper(scanner *bufio.Scanner, state, mode string) { + turnID := "" for scanner.Scan() { var req struct { Type, ID, Path string - Max int `json:"max_bytes"` + TurnID string `json:"turn_id"` + Max int `json:"max_bytes"` } _ = json.Unmarshal(scanner.Bytes(), &req) - if req.Type == "start" { + if req.Type == "turn_cancel" { + reusable := false + _ = json.NewEncoder(os.Stdout).Encode(bridgeEvent{Type: "error", TurnID: turnID, Code: "cancelled"}) + confirmed := true + _ = json.NewEncoder(os.Stdout).Encode(bridgeEvent{Type: "turn_settled", Confirmed: &confirmed, TurnID: turnID, Reusable: &reusable, Reason: "fixture_closed"}) + return + } + if req.Type == "turn_start" { + turnID = req.TurnID + _ = json.NewEncoder(os.Stdout).Encode(bridgeEvent{Type: "turn_started", TurnID: turnID}) _ = os.WriteFile(filepath.Join(state, "read-started"), []byte("{}"), 0600) continue } diff --git a/apps/parsar-daemon/internal/agent/claudesdk/workspace_structured_test.go b/apps/parsar-daemon/internal/agent/claudesdk/workspace_structured_test.go index a9f7a8ebc..b8ad20f00 100644 --- a/apps/parsar-daemon/internal/agent/claudesdk/workspace_structured_test.go +++ b/apps/parsar-daemon/internal/agent/claudesdk/workspace_structured_test.go @@ -52,7 +52,7 @@ func TestWorkspaceStructuredPreparationQualificationAndFrozenSchema(t *testing.T } } var started map[string]json.RawMessage - if err := json.Unmarshal(waitPreparationFile(t, filepath.Join(config.StateDir, "start.json")), &started); err != nil || len(started) != 2 || started["output_format"] != nil { + if err := json.Unmarshal(waitPreparationFile(t, filepath.Join(config.StateDir, "start.json")), &started); err != nil || len(started) != 3 || started["output_format"] != nil { t.Fatal("Start replaced the prepared configuration", err) } }) diff --git a/apps/parsar-daemon/internal/agent/codex/capability_root_test.go b/apps/parsar-daemon/internal/agent/codex/capability_root_test.go new file mode 100644 index 000000000..36201a246 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/codex/capability_root_test.go @@ -0,0 +1,13 @@ +package codex + +import ( + "testing" +) + +func TestCapabilityRootIsOneQuotedReadOnlyNativeKey(t *testing.T) { + var plan SessionPlan + configureCapabilityRoot(&plan, "managed-workspace-enabled", "/tmp/root.with.dot/quote\"inside") + if len(plan.ExtraConfig) != 1 || plan.ExtraConfig[0][0] != "permissions.\"managed-workspace-enabled\".filesystem.\"/tmp/root.with.dot/quote\\\"inside\"" || plan.ExtraConfig[0][1] != "\"read\"" { + t.Fatal("root escaped its read-only key", plan.ExtraConfig) + } +} diff --git a/apps/parsar-daemon/internal/agent/codex/executor.go b/apps/parsar-daemon/internal/agent/codex/executor.go new file mode 100644 index 000000000..2e437c20e --- /dev/null +++ b/apps/parsar-daemon/internal/agent/codex/executor.go @@ -0,0 +1,176 @@ +package codex + +import ( + "context" + "errors" + "strings" + "sync" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +// Executor owns the prepared process, fixed plan and native thread. Each StartTurn +// creates independent receipt, observation, cancellation and output ownership. +type Executor struct { + mu sync.Mutex + prepared *Prepared + active *Session + closed bool + closeMu sync.Mutex +} + +func PrepareExecutor(ctx context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + return newExecutor(ctx, req, defaultSessionConfig()) +} + +func newExecutor(ctx context.Context, req proto.PromptRequestPayload, cfg sessionConfig) (*Executor, error) { + p, err := newPreparation(ctx, req, cfg) + if err != nil { + if p != nil { + return &Executor{prepared: p}, err + } + return nil, err + } + p.mu.Lock() + p.started = true + close(p.transferred) + p.mu.Unlock() + return &Executor{prepared: p}, nil +} + +func (e *Executor) StartTurn(ctx context.Context, runID string, input proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + if out == nil || strings.TrimSpace(runID) == "" || input.Validate() != nil { + return nil, errors.New("codex: start requires a run identity, input and output") + } + if err := ctx.Err(); err != nil { + return nil, err + } + e.mu.Lock() + base := e.prepared.session + if e.closed || base.cancelCtx.Err() != nil || !base.rpc.Alive() { + e.mu.Unlock() + return nil, errors.New("codex: executor unavailable") + } + previous := e.active + if previous != nil { + select { + case <-previous.waitDone: + default: + e.mu.Unlock() + return nil, errors.New("codex: previous turn has not settled") + } + if previous.settlementErr != nil || !previous.settlement.Reusable { + e.mu.Unlock() + return nil, errors.New("codex: executor requires retirement") + } + } + functions := &functionCalls{definitions: base.functions.definitions, names: base.functions.names, pending: map[string]*pendingFunction{}} + turnCtx, cancel := context.WithCancel(base.cancelCtx) + s := &Session{executor: e, nativeHome: base.nativeHome, toolEnvironment: base.toolEnvironment, + functions: functions, observeMessages: base.observeMessages, observeTools: base.observeTools, + observeToolObservations: base.observeToolObservations, observeSubagentIdentities: base.observeSubagentIdentities, + cfg: base.cfg, rpc: base.rpc, cancelCtx: turnCtx, cancelFn: cancel, + waitDone: make(chan struct{}), outputDone: make(chan struct{}), cleanup: func() {}, + bufs: NewItemBuffers(), resolvedModel: base.resolvedModel, interactions: newPendingCodexInteractions(), runID: runID, out: out} + if previous != nil { + s.threadID = previous.currentThreadID() + s.retiredTurns = make(map[string]bool, len(previous.retiredTurns)+1) + for id := range previous.retiredTurns { + s.retiredTurns[id] = true + } + previous.steering.mu.Lock() + s.retiredTurns[previous.steering.id] = true + previous.steering.mu.Unlock() + s.resolvedModel = previous.resolvedModel + previous.usageMu.Lock() + s.usageTotal = previous.usageTotal + previous.usageMu.Unlock() + } + e.active = s + if s.observeSubagentIdentities { + s.startSubagentObservations() + } + s.registerHandlers() + e.mu.Unlock() + req := proto.PromptRequestPayload{RunID: runID, Input: input, AgentSessionID: e.prepared.resumeID, StrictResume: e.prepared.strictResume, RequireExistingNativeSession: e.prepared.requireExistingNativeSession} + // Ownership precedes any native submission. Even an uncertain start returns the + // exact Turn so its caller can await settlement without replaying the input. + err := s.startNative(ctx, e.prepared.plan, req) + if err != nil { + s.emitTerminal("codex: native start failed", true) + s.finishAfterTerminal() + } + go s.settleExecutorTurn(err) + return s, err +} + +func (e *Executor) Close(ctx context.Context) error { + e.mu.Lock() + e.closed = true + e.mu.Unlock() + done := make(chan error, 1) + go func() { + e.closeMu.Lock() + defer e.closeMu.Unlock() + base := e.prepared.session + e.mu.Lock() + running := e.active + e.mu.Unlock() + if running != nil { + select { + case <-running.waitDone: + default: + // Try native cancellation before closing its transport, but never + // let an absent terminal receipt prevent resource retirement. + cancelCtx, stop := context.WithTimeout(ctx, 10*time.Second) + _ = running.Cancel(cancelCtx) + stop() + } + } + if running != nil && base.rpc.Alive() { + // A caller deadline only stops waiting. Cleanup retains a bounded + // opportunity to release native terminals before closing their owner. + cleanupCtx, stop := context.WithTimeout(context.Background(), rpcKillTimeout) + err := running.cleanupNativeTerminals(cleanupCtx) + stop() + if err != nil { + // Preserve the native owner and exact handles for a later Close. + done <- err + return + } + } + base.cancelFn() + err := base.rpc.Close() + e.mu.Lock() + active := e.active + e.mu.Unlock() + if err == nil && active != nil { + // A Turn's permanent outcome error survives Close. Cleanup succeeds + // when its work and output have stopped, not when that outcome changes. + select { + case <-active.waitDone: + case <-ctx.Done(): + err = ctx.Err() + } + } + if err == nil { + err = base.rpc.awaitReaders(ctx) + } + if err == nil { + e.prepared.plan.Cleanup() + } + done <- err + }() + select { + case err := <-done: + return err + case <-ctx.Done(): + return ctx.Err() + } +} + +var _ agent.Executor = (*Executor)(nil) + +func NewExecutorFactory() agent.ExecutorFactory { return PrepareExecutor } diff --git a/apps/parsar-daemon/internal/agent/codex/executor_native_test.go b/apps/parsar-daemon/internal/agent/codex/executor_native_test.go new file mode 100644 index 000000000..5d4ecbc97 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/codex/executor_native_test.go @@ -0,0 +1,164 @@ +package codex + +import ( + "context" + "encoding/json" + "os" + "os/exec" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + obslog "github.com/MiniMax-AI-Dev/parsar/internal/obs/log" +) + +// This opt-in test uses the real pinned harness and an explicitly configured +// Responses provider. Credentials are read from a private file, never arguments. +func TestExecutorNativeReuse(t *testing.T) { + if os.Getenv("OAC_TEST_CODEX_EXECUTOR_NATIVE") != "1" { + t.Skip("requires an authorized native model acceptance environment") + } + binary, root := os.Getenv("OAC_TEST_CODEX_BINARY"), os.Getenv("OAC_TEST_CODEX_LIVE_ROOT") + model, endpoint := os.Getenv("OAC_TEST_CODEX_MODEL"), os.Getenv("OAC_TEST_CODEX_BASE_URL") + if binary == "" || root == "" || model == "" || endpoint == "" { + t.Fatal("missing explicit native acceptance configuration") + } + version, err := exec.Command(binary, "--version").Output() + if err != nil || strings.TrimSpace(string(version)) != "codex-cli 0.153.4" { + t.Fatal("native artifact is not Codex 0.153.4") + } + key, err := os.ReadFile(os.Getenv("OAC_TEST_CODEX_KEY_FILE")) + if err != nil || strings.TrimSpace(string(key)) == "" { + t.Fatal("private provider credential unavailable") + } + if err = os.MkdirAll(root, 0700); err != nil { + t.Fatal("cannot create isolated acceptance root") + } + isolated, err := os.MkdirTemp(root, "run-") + if err != nil { + t.Fatal("cannot create isolated acceptance workspace") + } + t.Cleanup(func() { _ = os.RemoveAll(isolated) }) + t.Setenv("OAC_RUNTIME_HOME", isolated) + for _, name := range []string{"CODEX_EXEC_SERVER_URL", "CODEX_EXEC_SERVER_NOISE_REGISTRY_URL", "CODEX_EXEC_SERVER_NOISE_ENVIRONMENT_ID", "CODEX_EXEC_SERVER_NOISE_AUTH_TOKEN"} { + t.Setenv(name, "") + } + cfg := defaultSessionConfig() + cfg.codexBinary = binary + cfg.logger = obslog.Discard() + req := proto.PromptRequestPayload{ + AgentKind: "codex", AgentStateKey: "executor-native", WorkDir: filepath.Join(isolated, "workspace"), + StrictResume: true, DisableExecutionEnvironment: true, DisableSubagents: true, ObserveMessages: true, + AgentOptions: map[string]any{"model": model, "codex_provider": map[string]any{"base_url": endpoint, "wire_api": "responses", "bearer_token": strings.TrimSpace(string(key))}}, + FunctionTools: []proto.FunctionTool{{Name: "hold", Description: "Wait until the host supplies a result.", Parameters: json.RawMessage("{\"type\":\"object\",\"properties\":{},\"additionalProperties\":false}")}}, + } + ctx, cancel := context.WithTimeout(t.Context(), 4*time.Minute) + defer cancel() + began := time.Now() + e, err := newExecutor(ctx, req, cfg) + if err != nil { + t.Fatalf("native prepare failed (%T)", err) + } + t.Cleanup(func() { + c, stop := context.WithTimeout(context.Background(), 20*time.Second) + defer stop() + if err := e.Close(c); err != nil { + t.Errorf("native owner cleanup failed (%T)", err) + } + }) + t.Logf("native_version=0.153.4 prepare_ms=%d", time.Since(began).Milliseconds()) + pid := e.prepared.session.rpc.cmd.Process.Pid + var thread string + run := func(id, prompt string, old agent.Turn, interrupt bool) (agent.Turn, string) { + t.Helper() + output := make(chan proto.Envelope, 512) + started := time.Now() + turn, err := e.StartTurn(ctx, id, proto.TextInput(prompt), output) + if err != nil { + t.Fatalf("%s start failed (%T)", id, err) + } + if old != nil { + if err := old.Cancel(ctx); err != nil { + t.Fatalf("%s stale cancellation failed", id) + } + } + done := make(chan struct{}) + called := make(chan struct{}, 1) + var text string + var terminal int + var first time.Duration + go func() { + defer close(done) + for event := range output { + if first == 0 { + first = time.Since(started) + } + if event.Type == proto.TypeFunctionCall { + select { + case called <- struct{}{}: + default: + } + } + if event.Type == proto.TypeDone { + var result proto.DonePayload + _ = event.DecodePayload(&result) + text = result.Content + terminal++ + } + } + }() + if interrupt { + select { + case <-called: + case <-done: + t.Fatal("native cancellation did not reach its function") + case <-ctx.Done(): + t.Fatal("native function admission timed out") + } + cancelStarted := time.Now() + if err := turn.Cancel(ctx); err != nil { + t.Fatalf("native interrupt failed (%T)", err) + } + t.Logf("turn=%s cancel_settled_ms=%d", id, time.Since(cancelStarted).Milliseconds()) + } + settlement, err := turn.AwaitSettlement(ctx) + if err != nil || !settlement.Reusable { + t.Fatalf("%s native settlement not reusable (error=%t reason=%s)", id, err != nil, settlement.Reason) + } + <-done + if terminal != 1 { + t.Fatalf("%s emitted %d terminals", id, terminal) + } + s := turn.(*Session) + if thread == "" { + thread = s.currentThreadID() + } + if thread == "" || s.currentThreadID() != thread || !s.rpc.Alive() || s.rpc.cmd.Process.Pid != pid { + t.Fatal("native owner/thread changed") + } + t.Logf("turn=%s first_event_ms=%d settled_ms=%d same_process=true same_thread=true", id, first.Milliseconds(), time.Since(started).Milliseconds()) + return turn, text + } + first, _ := run("cold", "Remember the exact code CEDAR-4729 for this conversation. Reply only SAVED. Do not call tools.", nil, false) + _, answer := run("warm", "What is the exact code I asked you to remember? Reply only the code. Do not call tools.", nil, false) + if strings.TrimSpace(answer) != "CEDAR-4729" { + t.Fatal("warm native Turn did not retain conversation memory") + } + cancelled, _ := run("cancel", "Call the hold function now with an empty object. Wait for its result before responding.", nil, true) + _, answer = run("followup", "What is the exact code I asked you to remember? Reply only the code. Do not call tools.", cancelled, false) + if strings.TrimSpace(answer) != "CEDAR-4729" { + t.Fatal("post-cancellation native Turn lost conversation memory") + } + if err := first.Cancel(ctx); err != nil { + t.Fatal("retired first Turn cancellation failed") + } + if err := e.Close(ctx); err != nil { + t.Fatal("native final cleanup failed") + } + if e.prepared.session.rpc.Alive() { + t.Fatal("native child remained alive after owner close") + } +} diff --git a/apps/parsar-daemon/internal/agent/codex/executor_test.go b/apps/parsar-daemon/internal/agent/codex/executor_test.go new file mode 100644 index 000000000..f6c9d56fa --- /dev/null +++ b/apps/parsar-daemon/internal/agent/codex/executor_test.go @@ -0,0 +1,262 @@ +package codex + +import ( + "context" + "errors" + "os" + "os/exec" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func executorFixture(t *testing.T, mode string) (*Executor, string) { + t.Helper() + req, cfg, root := preparationFixture(t) + t.Setenv("OAC_TEST_EXECUTOR_MODE", mode) + ownerCtx, cancelOwner := context.WithCancel(context.Background()) + t.Cleanup(cancelOwner) + e, err := newExecutor(ownerCtx, req, cfg) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + if err := e.Close(ctx); err != nil { + t.Error(err) + } + }) + return e, root +} +func awaitExecutorTurn(t *testing.T, turn agent.Turn, out <-chan proto.Envelope) agent.TurnSettlement { + t.Helper() + ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second) + defer cancel() + settlement, err := turn.AwaitSettlement(ctx) + if err != nil { + t.Fatal(err) + } + count := 0 + for frame := range out { + if frame.Type == proto.TypeDone { + count++ + } + } + if count != 1 { + t.Fatalf("terminal count %d", count) + } + return settlement +} +func TestExecutorNormalTurnsKeepProcessAndThread(t *testing.T) { + e, root := executorFixture(t, "complete") + var previous agent.Turn + for _, id := range []string{"one", "two"} { + out := make(chan proto.Envelope, 20) + turn, err := e.StartTurn(t.Context(), id, proto.TextInput("answer"), out) + if err != nil { + t.Fatal(err) + } + if !awaitExecutorTurn(t, turn, out).Reusable { + t.Fatal("healthy turn was not reusable") + } + if previous != nil { + if err := previous.Cancel(t.Context()); err != nil { + t.Fatal(err) + } + } + previous = turn + } + counts := map[string]int{} + pids := map[int]bool{} + for _, f := range preparationFrames(t, root) { + counts[f.Method]++ + pids[f.PID] = true + } + if counts["initialize"] != 1 || counts["thread/start"] != 1 || counts["turn/start"] != 2 || counts["turn/interrupt"] != 0 || len(pids) != 1 { + t.Fatal(counts, pids) + } + if !e.prepared.session.rpc.Alive() { + t.Fatal("normal completion closed executor") + } +} +func TestExecutorCancellationSettlesThenReuses(t *testing.T) { + e, root := executorFixture(t, "complete") + out := make(chan proto.Envelope, 20) + first, err := e.StartTurn(t.Context(), "cancel", proto.TextInput("hold"), out) + if err != nil { + t.Fatal(err) + } + if _, err := e.StartTurn(t.Context(), "overlap", proto.TextInput("answer"), make(chan proto.Envelope, 20)); err == nil { + t.Fatal("overlapping turn admitted") + } + ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second) + defer cancel() + if err := first.Cancel(ctx); err != nil { + t.Fatal(err) + } + if !awaitExecutorTurn(t, first, out).Reusable { + t.Fatal("cancelled healthy executor unavailable") + } + secondOut := make(chan proto.Envelope, 20) + second, err := e.StartTurn(t.Context(), "next", proto.TextInput("hold"), secondOut) + if err != nil { + t.Fatal(err) + } + if err := first.Cancel(ctx); err != nil { + t.Fatal(err) + } + // A stale handle cannot interrupt its successor. + interrupts := 0 + for _, f := range preparationFrames(t, root) { + if f.Method == "turn/interrupt" { + interrupts++ + } + } + if interrupts != 1 { + t.Fatalf("stale cancellation sent %d interrupts", interrupts) + } + if err := second.Cancel(ctx); err != nil { + t.Fatal(err) + } + if !awaitExecutorTurn(t, second, secondOut).Reusable { + t.Fatal("second cancellation did not settle") + } +} +func TestExecutorStartErrorsRetainExactOwnership(t *testing.T) { + for _, mode := range []string{"start-error", "disconnect"} { + t.Run(mode, func(t *testing.T) { + e, _ := executorFixture(t, mode) + out := make(chan proto.Envelope, 20) + ctx, cancel := context.WithCancel(t.Context()) + cancel() + turn, err := e.StartTurn(ctx, "before", proto.TextInput("answer"), out) + if turn != nil || !errors.Is(err, context.Canceled) { + t.Fatal(turn, err) + } + select { + case <-out: + t.Fatal("pre-admission output was retained or closed") + default: + } + turn, err = e.StartTurn(t.Context(), "after", proto.TextInput("answer"), out) + if turn == nil || err == nil { + t.Fatal("uncertain submission lost owner", err) + } + settlement, settleErr := turn.AwaitSettlement(t.Context()) + if settleErr == nil || settlement.Reusable { + t.Fatal("failed submission fabricated native settlement", settlement, settleErr) + } + terminalCount := 0 + for frame := range out { + if frame.Type == proto.TypeDone { + terminalCount++ + } + } + if terminalCount != 1 { + t.Fatalf("failed submission terminal count = %d", terminalCount) + } + }) + } +} + +func TestExecutorCloseRetainsPlanUntilReaped(t *testing.T) { + cmd := exec.Command(os.Args[0], "-test.run=^TestJSONRPCClientFakeCodexProcess$", "--") + cmd.Env = append(os.Environ(), "CODEX_RPC_FAKE_PROCESS=1", "GORACE=atexit_sleep_ms=0") + stdin, err := cmd.StdinPipe() + if err != nil { + t.Fatal(err) + } + if err = cmd.Start(); err != nil { + t.Fatal(err) + } + rpc := NewJSONRPCClient(JSONRPCConfig{}) + rpc.cmd, rpc.stdin, rpc.alive = cmd, stdin, true + var reap sync.Once + t.Cleanup(func() { _ = cmd.Process.Kill(); reap.Do(rpc.waitChild) }) + _, cancel := context.WithCancel(t.Context()) + var cleaned atomic.Bool + e := &Executor{prepared: &Prepared{session: &Session{rpc: rpc, cancelFn: cancel}, plan: SessionPlan{Cleanup: func() { cleaned.Store(true) }}}} + ctx, stop := context.WithTimeout(t.Context(), 20*time.Millisecond) + defer stop() + if err = e.Close(ctx); !errors.Is(err, context.DeadlineExceeded) { + t.Fatal("unreaped close", err) + } + if cleaned.Load() { + t.Fatal("plan released before cleanup settled") + } + reap.Do(rpc.waitChild) + if err = e.Close(t.Context()); err != nil { + t.Fatal(err) + } + if !cleaned.Load() { + t.Fatal("cleanup retry did not release plan") + } +} + +func TestExecutorCloseSeparatesTurnFailureFromResourceCleanup(t *testing.T) { + e, root := executorFixture(t, "interrupt-error") + out := make(chan proto.Envelope, 20) + turn, err := e.StartTurn(t.Context(), "cancel-failure", proto.TextInput("hold"), out) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(t.Context(), 5*time.Second) + defer cancel() + if err := turn.Cancel(ctx); err == nil { + t.Fatal("native interrupt failure was hidden") + } + if _, err := turn.AwaitSettlement(ctx); err == nil { + t.Fatal("failed Turn unexpectedly settled successfully") + } + if err := e.Close(ctx); err != nil { + t.Fatal("failed Turn prevented confirmed resource cleanup", err) + } + if e.prepared.session.rpc.Alive() || len(preparedCatalogs(t, root)) != 0 { + t.Fatal("Close did not release native process and plan") + } + if _, err := turn.AwaitSettlement(ctx); err == nil { + t.Fatal("resource cleanup fabricated successful Turn settlement") + } +} + +func TestExecutorCloseTerminatesAfterMissingCancellationTerminal(t *testing.T) { + e, root := executorFixture(t, "interrupt-no-terminal") + out := make(chan proto.Envelope, 20) + turn, err := e.StartTurn(t.Context(), "missing-terminal", proto.TextInput("hold"), out) + if err != nil { + t.Fatal(err) + } + cancelCtx, stopCancel := context.WithTimeout(t.Context(), 5*time.Second) + defer stopCancel() + cancelled := make(chan error, 1) + go func() { cancelled <- turn.Cancel(cancelCtx) }() + waitPreparationMethod(t, root, "turn/interrupt") + closeCtx, stopClose := context.WithTimeout(t.Context(), 20*time.Millisecond) + defer stopClose() + if err := e.Close(closeCtx); !errors.Is(err, context.DeadlineExceeded) { + t.Fatal("missing receipt did not reach the cleanup deadline", err) + } + // Retrying waits on the same owner; the expired observation must not skip + // the transport close that can actually stop an unresponsive native Turn. + retry, stopRetry := context.WithTimeout(t.Context(), 2*time.Second) + defer stopRetry() + if err := e.Close(retry); err != nil { + t.Fatal("Close never retired the native process", err) + } + if e.prepared.session.rpc.Alive() || len(preparedCatalogs(t, root)) != 0 { + t.Fatal("unresponsive native process or plan still owned after Close") + } + select { + case cancelErr := <-cancelled: + if cancelErr == nil { + t.Fatal("resource cleanup fabricated native cancellation confirmation") + } + case <-retry.Done(): + t.Fatal("original cancellation waiter was abandoned") + } +} diff --git a/apps/parsar-daemon/internal/agent/codex/executor_turn.go b/apps/parsar-daemon/internal/agent/codex/executor_turn.go new file mode 100644 index 000000000..0579e81fa --- /dev/null +++ b/apps/parsar-daemon/internal/agent/codex/executor_turn.go @@ -0,0 +1,175 @@ +package codex + +import ( + "context" + "encoding/json" + "errors" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" +) + +func (s *Session) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + select { + case <-s.waitDone: + return s.settlement, s.settlementErr + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} + +func (s *Session) settleExecutorTurn(startErr error) { + defer close(s.waitDone) + if startErr == nil { + select { + case <-s.outputDone: + case <-s.rpc.Done(): + s.emitTerminal("codex: connection closed before settlement", true) + case <-s.cancelCtx.Done(): + s.emitTerminal("codex: execution owner closed", true) + } + } + // Detach the old callbacks before waiting for their captured Turn and native + // replies. Later frames cannot acquire this Turn or redirect to a successor. + if !s.nativeSettled.Load() || startErr != nil || !s.rpc.Alive() { + s.cancelFn() + s.settlementErr = errors.Join(s.settlementErr, s.rpc.Close()) + } + if !s.nativeSettled.Load() { + s.settlementErr = errors.Join(s.settlementErr, errors.New("codex: native Turn settlement is unconfirmed")) + } + s.rpc.detachHandlers() + s.operationMu.Lock() + s.operationsClosed = true + s.operationMu.Unlock() + s.stopSteering() + s.stopFunctionCalls() + s.stopCodexInteractionTimers() + if !s.nativeSettled.Load() || startErr != nil || !s.rpc.Alive() { + s.cancelFn() + s.settlementErr = errors.Join(s.settlementErr, s.rpc.Close()) + s.settlement = agent.TurnSettlement{Reason: "native_execution_unavailable"} + } else { + s.settlement = agent.TurnSettlement{Reusable: true} + } + s.closeRunOutput() + s.operations.Wait() + if s.subagents != nil { + s.subagents.mu.Lock() + err := s.subagents.cancelResult + s.subagents.mu.Unlock() + if err != nil { + s.settlement = agent.TurnSettlement{Reason: "child_settlement_unconfirmed"} + s.settlementErr = errors.Join(s.settlementErr, err, s.rpc.Close()) + } + } + // The terminal callback has returned before detachHandlers completes. A + // concurrent cancellation owns only this Turn and finishes before reuse. + s.operationMu.Lock() + ready := s.cancelReady + s.operationMu.Unlock() + if ready != nil { + <-ready + if s.cancelErr != nil { + s.settlement.Reusable = false + s.settlement.Reason = "cancellation_unconfirmed" + s.settlementErr = errors.Join(s.settlementErr, s.cancelErr, s.rpc.Close()) + } + } + if s.cancelled.Load() && s.nativeSettled.Load() && s.cancelErr == nil && s.rpc.Alive() { + cleanupCtx, stop := context.WithTimeout(context.Background(), 10*time.Second) + err := s.cleanupNativeTerminals(cleanupCtx) + stop() + if err != nil { + s.settlement = agent.TurnSettlement{Reason: "native_terminal_cleanup_unconfirmed"} + s.settlementErr = errors.Join(s.settlementErr, err) + } + } + s.cancelFn() +} + +func (s *Session) beginOperation() bool { + if s.executor == nil { + return true + } + s.operationMu.Lock() + defer s.operationMu.Unlock() + if s.operationsClosed { + return false + } + s.operations.Add(1) + return true +} +func (s *Session) endOperation() { + if s.executor != nil { + s.operations.Done() + } +} + +func (s *Session) cancelExecutorTurn(ctx context.Context) error { + s.operationMu.Lock() + if s.operationsClosed { + s.operationMu.Unlock() + _, err := s.AwaitSettlement(ctx) + return err + } + s.cancelOnce.Do(func() { + s.cancelled.Store(true) + s.cancelReady = make(chan struct{}) + go func() { + defer close(s.cancelReady) + s.stopFunctionCalls() + turnID, active := s.stopSteering() + s.stopCodexInteractionTimers() + if !s.terminal.Load() && active { + if turnID == "" { + s.cancelErr = errors.New("codex: cancellation has no native turn identity") + s.cancelFn() + _ = s.rpc.Close() + return + } + interruptCtx, cancel := context.WithTimeout(context.Background(), 10*time.Second) + _, err := s.rpc.request(interruptCtx, "turn/interrupt", TurnInterruptParams{ThreadID: s.currentThreadID(), TurnID: turnID}, func(frame any) error { return s.rpc.writeFrameContext(interruptCtx, frame) }) + cancel() + if err != nil { + s.cancelErr = err + s.cancelFn() + _ = s.rpc.Close() + return + } + } + if s.subagents != nil { + s.cancelErr = s.cancelSubagentWork(s.cancelCtx) + } + }() + }) + s.operationMu.Unlock() + _, err := s.AwaitSettlement(ctx) + return err +} + +var _ agent.Turn = (*Session)(nil) + +// Server callbacks retain their originating Turn. MCP elicitation may be a +// standalone thread-scoped request in the native protocol; a supplied Turn ID +// must still match exactly. +func (s *Session) onServerRequest(method string, handler ServerRequestHandler) { + s.rpc.OnServerRequest(method, func(raw json.RawMessage, id any) (any, error) { + if s.executor != nil { + var scope struct { + ThreadID string `json:"threadId"` + TurnID string `json:"turnId"` + } + if json.Unmarshal(raw, &scope) != nil || !s.isRootThread(scope.ThreadID) || s.terminal.Load() || + (scope.TurnID != "" && !s.isRootTurn(scope.ThreadID, scope.TurnID)) || + (scope.TurnID == "" && method != "mcpServer/elicitation/request") { + return nil, errors.New("codex: request does not belong to the active turn") + } + } + if !s.beginOperation() { + return nil, errors.New("codex: turn has settled") + } + defer s.endOperation() + return handler(raw, id) + }) +} diff --git a/apps/parsar-daemon/internal/agent/codex/function_receipts.go b/apps/parsar-daemon/internal/agent/codex/function_receipts.go index 7a70518d4..8c594209d 100644 --- a/apps/parsar-daemon/internal/agent/codex/function_receipts.go +++ b/apps/parsar-daemon/internal/agent/codex/function_receipts.go @@ -26,6 +26,10 @@ type pendingFunction struct { } func (s *Session) SubmitFunctionResult(ctx context.Context, result proto.FunctionResultPayload) error { + if !s.beginOperation() { + return agent.ErrSteeringInactive + } + defer s.endOperation() if s.functions == nil { return agent.ErrUnknownFunctionCall } @@ -59,7 +63,11 @@ func (s *Session) SubmitFunctionResult(ctx context.Context, result proto.Functio // This operation owns both transport failure and native application. A generic // write deadline must not close the process after a native receipt has won. written := make(chan error, 1) + if !s.beginOperation() { + return agent.ErrUnknownFunctionCall + } go func() { + defer s.endOperation() if err := ctx.Err(); err != nil { written <- err return diff --git a/apps/parsar-daemon/internal/agent/codex/mcp_environment_test.go b/apps/parsar-daemon/internal/agent/codex/mcp_environment_test.go index 6ec33f262..37f368b81 100644 --- a/apps/parsar-daemon/internal/agent/codex/mcp_environment_test.go +++ b/apps/parsar-daemon/internal/agent/codex/mcp_environment_test.go @@ -13,15 +13,15 @@ import ( func TestEnvironmentMCPProjectsIsolatedStdioAndPrivateHTTPReferences(t *testing.T) { token := "user-token" local := &proto.LocalEnvironment{NetworkAccess: "enabled", MCP: []proto.EnvironmentMCP{ - {PackageRoot: "plugins/0", Server: agentplugin.MCPServer{Name: "local", Type: "stdio", Command: "must-not-be-native-command", Args: []string{"private-argument"}}}, - {PackageRoot: "plugins/1", BearerToken: &token, Server: agentplugin.MCPServer{Name: "remote", Type: "http", URL: "https://example.com/mcp", HTTPHeaders: map[string]string{"X-Key": "literal-${DO_NOT_EXPAND}"}}}, + {InstallationRoot: "/private/runtime/capabilities", WorkspaceRoot: "/private/runtime/workspace", PackageRoot: "plugins/0", Server: agentplugin.MCPServer{Name: "local", Type: "stdio", Command: "must-not-be-native-command", Args: []string{"private-argument"}}}, + {InstallationRoot: "/private/runtime/capabilities", WorkspaceRoot: "/private/runtime/workspace", PackageRoot: "plugins/1", BearerToken: &token, Server: agentplugin.MCPServer{Name: "remote", Type: "http", URL: "https://example.com/mcp", HTTPHeaders: map[string]string{"X-Key": "literal-${DO_NOT_EXPAND}"}}}, }} servers, env, err := mergeEnvironmentMCP(nil, local) if err != nil || len(servers) != 2 || len(env) != 2 { t.Fatal("environment declarations were not projected", err) } if servers["local"].Command != "/usr/bin/python3" || !servers["local"].ApproveTools || - !slices.Equal(servers["local"].Args, []string{"-I", "-S", "/usr/local/bin/oac-runtime-initialize", "stdio", "plugins/0", "local"}) { + !slices.Equal(servers["local"].Args, []string{"-I", "-S", "/usr/local/bin/oac-runtime-initialize", "stdio", "/private/runtime/capabilities", "/private/runtime/workspace", "plugins/0", "local"}) { t.Fatal("native stdio bypasses the packaged launcher") } remote := servers["remote"] diff --git a/apps/parsar-daemon/internal/agent/codex/preparation.go b/apps/parsar-daemon/internal/agent/codex/preparation.go index 3c22dcb04..2d143f97b 100644 --- a/apps/parsar-daemon/internal/agent/codex/preparation.go +++ b/apps/parsar-daemon/internal/agent/codex/preparation.go @@ -130,10 +130,7 @@ func newPreparation(parent context.Context, req proto.PromptRequestPayload, cfg } if req.DisableExecutionEnvironment { if err := verifyNoExecutionEnvironment(cancelCtx, rpc); err != nil { - cancelFn() - _ = rpc.Close() - plan.Cleanup() - return nil, err + return p.preparationFailed(err) } } if s.observeSubagentIdentities { @@ -148,18 +145,12 @@ func newPreparation(parent context.Context, req proto.PromptRequestPayload, cfg } if plan.mcpServers != nil { if err := verifyMCPConfig(cancelCtx, rpc, plan); err != nil { - cancelFn() - _ = rpc.Close() - plan.Cleanup() - return nil, err + return p.preparationFailed(err) } } if len(skillRoots) > 0 { if err := setSkillExtraRoots(cancelCtx, rpc, skillRoots); err != nil { - cancelFn() - _ = rpc.Close() - plan.Cleanup() - return nil, fmt.Errorf("codex: register skill root: %w", err) + return p.preparationFailed(fmt.Errorf("codex: register skill root: %w", err)) } } @@ -168,6 +159,8 @@ func newPreparation(parent context.Context, req proto.PromptRequestPayload, cfg } func (p *Prepared) preparationFailed(cause error) (*Prepared, error) { - _ = p.Close() + if err := p.Close(); err != nil { + return p, errors.Join(cause, err) + } return nil, cause } diff --git a/apps/parsar-daemon/internal/agent/codex/preparation_helpers_test.go b/apps/parsar-daemon/internal/agent/codex/preparation_helpers_test.go index d1b5a7267..2957a4718 100644 --- a/apps/parsar-daemon/internal/agent/codex/preparation_helpers_test.go +++ b/apps/parsar-daemon/internal/agent/codex/preparation_helpers_test.go @@ -3,6 +3,7 @@ package codex import ( "bufio" "encoding/json" + "fmt" "os" "path/filepath" "strings" @@ -139,6 +140,9 @@ func TestPreparationFakeCodexProcess(t *testing.T) { frames := json.NewEncoder(log) output := json.NewEncoder(os.Stdout) scanner := bufio.NewScanner(os.Stdin) + turnNumber := 0 + currentTurn := "" + executorMode := os.Getenv("OAC_TEST_EXECUTOR_MODE") for scanner.Scan() { var frame preparationFrame if json.Unmarshal(scanner.Bytes(), &frame) != nil { @@ -181,12 +185,70 @@ func TestPreparationFakeCodexProcess(t *testing.T) { } } result = map[string]any{"thread": map[string]string{"id": "fixture-native-thread"}, "model": "fixture-model"} + case "thread/backgroundTerminals/list": + result = map[string]any{"data": []any{}} + if executorMode == "terminal-cleanup" { + if _, err := os.Stat(os.Getenv("OAC_TEST_PREPARATION_FRAMES") + ".terminated"); os.IsNotExist(err) { + result = map[string]any{"data": []any{map[string]string{"processId": "owned-terminal"}}} + } + } + case "thread/backgroundTerminals/terminate": + var target struct { + ThreadID string `json:"threadId"` + ProcessID string `json:"processId"` + } + if json.Unmarshal(frame.Params, &target) != nil || target.ThreadID != "fixture-native-thread" || target.ProcessID != "owned-terminal" { + os.Exit(8) + } + _, allowed := os.Stat(os.Getenv("OAC_TEST_PREPARATION_FRAMES") + ".allow-cleanup") + result = map[string]any{"terminated": allowed == nil} + if allowed == nil { + _ = os.WriteFile(os.Getenv("OAC_TEST_PREPARATION_FRAMES")+".terminated", nil, 0600) + } + case "turn/interrupt": + if executorMode == "interrupt-error" { + _ = output.Encode(map[string]any{"id": frame.ID, "error": map[string]any{"code": -32603, "message": "interrupt rejected"}}) + continue + } case "turn/start": + if executorMode != "" { + turnNumber++ + currentTurn = fmt.Sprintf("fixture-turn-%d", turnNumber) + if turnNumber > 1 { + _ = output.Encode(map[string]any{"method": "turn/started", "params": map[string]any{"threadId": "fixture-native-thread", "turn": map[string]string{"id": fmt.Sprintf("fixture-turn-%d", turnNumber-1)}}}) + } + if executorMode == "disconnect" { + os.Exit(0) + } + if executorMode == "start-error" { + _ = output.Encode(map[string]any{"id": frame.ID, "error": map[string]any{"code": -32603, "message": "start rejected"}}) + continue + } + result = map[string]any{"turn": map[string]string{"id": currentTurn}} + break + } result = map[string]any{"turn": map[string]string{"id": "fixture-native-turn"}} } if output.Encode(map[string]any{"jsonrpc": "2.0", "id": frame.ID, "result": result}) != nil { os.Exit(5) } + + if executorMode != "" && (frame.Method == "turn/start" || frame.Method == "turn/interrupt") { + if frame.Method == "turn/interrupt" && executorMode == "interrupt-no-terminal" { + continue + } + if frame.Method == "turn/start" { + _ = output.Encode(map[string]any{"method": "turn/started", "params": map[string]any{"threadId": "fixture-native-thread", "turn": map[string]string{"id": currentTurn}}}) + } + if frame.Method == "turn/interrupt" || !strings.Contains(string(frame.Params), "hold") { + status := "completed" + if frame.Method == "turn/interrupt" { + status = "interrupted" + } + _ = output.Encode(map[string]any{"method": "turn/completed", "params": map[string]any{"threadId": "fixture-native-thread", "turn": map[string]string{"id": currentTurn, "status": status}}}) + } + continue + } if frame.Method == "turn/start" { _ = output.Encode(map[string]any{"jsonrpc": "2.0", "method": "turn/started", "params": map[string]any{"threadId": "fixture-native-thread", "turn": map[string]string{"id": "fixture-native-turn"}}}) if os.Getenv("OAC_TEST_PREPARATION_OBSERVE") == "1" { diff --git a/apps/parsar-daemon/internal/agent/codex/preparation_router_test.go b/apps/parsar-daemon/internal/agent/codex/preparation_router_test.go index 15bf5d432..b3410f785 100644 --- a/apps/parsar-daemon/internal/agent/codex/preparation_router_test.go +++ b/apps/parsar-daemon/internal/agent/codex/preparation_router_test.go @@ -4,9 +4,11 @@ import ( "context" "errors" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/localworkspace" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" "github.com/MiniMax-AI-Dev/parsar/internal/agentnetwork" "github.com/google/uuid" "os" + "path/filepath" "testing" "time" @@ -30,6 +32,11 @@ func TestPreparationRouterRetainsActualNativeChild(t *testing.T) { for _, start := range []bool{false, true} { t.Run(map[bool]string{false: "disconnect-before-start", true: "transfer-and-cancel"}[start], func(t *testing.T) { req, cfg, root := preparationFixture(t) + if err := os.Chmod(root, 0700); err != nil { + t.Fatal(err) + } + t.Setenv("OAC_RUNTIME_CAPABILITY_DIRECTORY", filepath.Join(t.TempDir(), "capabilities")) + t.Setenv("OAC_TEST_EXECUTOR_MODE", "complete") environment, session := uuid.NewString(), uuid.NewString() if err := os.MkdirAll(req.WorkDir, 0700); err != nil { t.Fatal(err) @@ -50,7 +57,7 @@ func TestPreparationRouterRetainsActualNativeChild(t *testing.T) { req.WorkDir = "" req.AgentStateKey = "agents-api-" + session req.DisableExecutionEnvironment = false - req.LocalEnvironment = &proto.LocalEnvironment{ID: environment, NetworkAccess: "enabled"} + req.LocalEnvironment = &proto.LocalEnvironment{ID: environment, WorkspaceDirectory: "/workspace", NetworkAccess: "enabled", CapabilitySources: &agentcapabilities.Input{}} cfg.permissionProfile = "managed-workspace" cfg.runtimeNetwork = agentnetwork.Policy{Access: "enabled"} registry := agent.NewRegistry() @@ -58,13 +65,13 @@ func TestPreparationRouterRetainsActualNativeChild(t *testing.T) { return nil, errors.New("ordinary Factory must not run") }) prepared := make(chan *Prepared, 1) - registry.RegisterPreparation("codex", false, func(ctx context.Context, req proto.PromptRequestPayload) (agent.Prepared, error) { - p, err := newPreparation(ctx, req, cfg) + registry.RegisterExecutor("codex", func(ctx context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + e, err := newExecutor(ctx, req, cfg) if err != nil { return nil, err } - prepared <- p - return p, nil + prepared <- e.prepared + return e, nil }) sender := make(preparationWireSender, 64) r, err := dispatch.New(dispatch.Config{Registry: registry, Sender: sender, LocalWorkspace: binding}) @@ -110,7 +117,7 @@ func TestPreparationRouterRetainsActualNativeChild(t *testing.T) { } } } - send(proto.TypeExecutionPrepare, "prepare-request", proto.ExecutionPreparePayload{Configuration: req}) + send(proto.TypeExecutionPrepare, "prepare-request", proto.ExecutionPreparePayload{SessionID: session, Configuration: req}) ready := await("ready") p := <-prepared assertPreparationOnly(t, root) @@ -119,7 +126,7 @@ func TestPreparationRouterRetainsActualNativeChild(t *testing.T) { t.Fatal("preparation became a Run") } if start { - input := proto.ExecutionStartPayload{Handle: ready.Handle, RunID: "actual-run", Input: proto.TextInput("actual input")} + input := proto.ExecutionStartPayload{ExecutorID: ready.ExecutorID, Handle: ready.Handle, RunID: "actual-run", Input: proto.TextInput("hold")} send(proto.TypeExecutionStart, "prepare-request", input) await("started") frames := waitPreparationMethod(t, root, "turn/start") diff --git a/apps/parsar-daemon/internal/agent/codex/prepared.go b/apps/parsar-daemon/internal/agent/codex/prepared.go index a83cf6ffe..b4e3a0b78 100644 --- a/apps/parsar-daemon/internal/agent/codex/prepared.go +++ b/apps/parsar-daemon/internal/agent/codex/prepared.go @@ -88,7 +88,9 @@ func (p *Prepared) Close() error { p.mu.Unlock() p.session.cancelFn() err := p.session.rpc.Close() - p.plan.Cleanup() + if err == nil { + p.plan.Cleanup() + } return err } diff --git a/apps/parsar-daemon/internal/agent/codex/rpc.go b/apps/parsar-daemon/internal/agent/codex/rpc.go index ec52afebb..d078c4588 100644 --- a/apps/parsar-daemon/internal/agent/codex/rpc.go +++ b/apps/parsar-daemon/internal/agent/codex/rpc.go @@ -91,11 +91,13 @@ type JSONRPCClient struct { pendingMu sync.Mutex pending map[string]*pendingRequest + handlerWork sync.WaitGroup handlersMu sync.RWMutex notifHandlers map[string][]NotificationHandler serverReqHandlers map[string]ServerRequestHandler anyNotifHandler NotificationHandler + readers sync.WaitGroup closeOnce sync.Once doneCh chan struct{} } @@ -206,8 +208,9 @@ func (c *JSONRPCClient) Start(ctx context.Context, init InitializeParams) (Initi c.alive = true c.mu.Unlock() - go c.readStdoutLoop() - go c.pumpStderr() + c.readers.Add(2) + go func() { defer c.readers.Done(); c.readStdoutLoop() }() + go func() { defer c.readers.Done(); c.pumpStderr() }() go c.waitChild() initCtx, cancel := context.WithTimeout(ctx, rpcInitTimeout) @@ -415,7 +418,13 @@ func (c *JSONRPCClient) handleNotification(method string, rawFrame []byte) { c.handlersMu.RLock() handlers := append([]NotificationHandler{}, c.notifHandlers[method]...) any := c.anyNotifHandler + if len(handlers) > 0 || any != nil { + c.handlerWork.Add(1) + } c.handlersMu.RUnlock() + if len(handlers) > 0 || any != nil { + defer c.handlerWork.Done() + } if len(handlers) == 0 && any == nil { c.cfg.Logger.Debug("codex rpc unhandled notification", "tag", c.cfg.LogTag, "method", method) return @@ -438,6 +447,9 @@ func (c *JSONRPCClient) handleServerRequest(rawFrame []byte, rawID json.RawMessa c.handlersMu.RLock() h, ok := c.serverReqHandlers[method] + if ok { + c.handlerWork.Add(1) + } c.handlersMu.RUnlock() if !ok { // Method-not-found per JSON-RPC 2.0 — codex hangs if we drop it. @@ -445,6 +457,7 @@ func (c *JSONRPCClient) handleServerRequest(rawFrame []byte, rawID json.RawMessa return } go func() { + defer c.handlerWork.Done() defer func() { if r := recover(); r != nil { _ = c.SendServerError(id, -32603, fmt.Sprintf("panic in %s handler: %v", method, r), nil) @@ -525,3 +538,25 @@ func truncate(b []byte, n int) []byte { } return b[:n] } + +// detachHandlers fences registration before joining captured callbacks, including +// asynchronous server replies. New notifications have no Turn destination. +func (c *JSONRPCClient) detachHandlers() { + c.handlersMu.Lock() + clear(c.notifHandlers) + clear(c.serverReqHandlers) + c.anyNotifHandler = nil + c.handlersMu.Unlock() + c.handlerWork.Wait() +} + +func (c *JSONRPCClient) awaitReaders(ctx context.Context) error { + done := make(chan struct{}) + go func() { c.readers.Wait(); close(done) }() + select { + case <-done: + return nil + case <-ctx.Done(): + return ctx.Err() + } +} diff --git a/apps/parsar-daemon/internal/agent/codex/server_requests.go b/apps/parsar-daemon/internal/agent/codex/server_requests.go index 69aa69d3c..656ac6ca7 100644 --- a/apps/parsar-daemon/internal/agent/codex/server_requests.go +++ b/apps/parsar-daemon/internal/agent/codex/server_requests.go @@ -195,6 +195,10 @@ func (s *Session) handleCodexUserInput(raw json.RawMessage, rpcID any) (any, err } func (s *Session) submitCodexPermission(requestID string, decision proto.PermissionDecisionPayload) error { + if !s.beginOperation() { + return agent.ErrSteeringInactive + } + defer s.endOperation() s.interactions.mu.Lock() pending, ok := s.interactions.permissions[requestID] if ok { @@ -243,6 +247,10 @@ func (s *Session) sendCodexPermissionReply(pending pendingCodexPermission, appro } func (s *Session) submitCodexUserInput(askID string, decision proto.PromptForUserChoiceDecisionPayload) error { + if !s.beginOperation() { + return agent.ErrSteeringInactive + } + defer s.endOperation() s.interactions.mu.Lock() pending, ok := s.interactions.asks[askID] if ok { @@ -311,6 +319,10 @@ func (s *Session) submitCodexUserInput(askID string, decision proto.PromptForUse } func (s *Session) expireCodexPermission(requestID string) { + if !s.beginOperation() { + return + } + defer s.endOperation() s.interactions.mu.Lock() pending, ok := s.interactions.permissions[requestID] if ok { @@ -326,6 +338,10 @@ func (s *Session) expireCodexPermission(requestID string) { } func (s *Session) expireCodexAsk(askID string) { + if !s.beginOperation() { + return + } + defer s.endOperation() s.interactions.mu.Lock() pending, ok := s.interactions.asks[askID] if ok { diff --git a/apps/parsar-daemon/internal/agent/codex/session.go b/apps/parsar-daemon/internal/agent/codex/session.go index 5c212c4e7..4d1e95ea6 100644 --- a/apps/parsar-daemon/internal/agent/codex/session.go +++ b/apps/parsar-daemon/internal/agent/codex/session.go @@ -63,6 +63,16 @@ func Factory(ctx context.Context, req proto.PromptRequestPayload, out chan<- pro // 5. turn/completed emits TypeDone + closes out. Cancel can short-cut // this by killing the child early. type Session struct { + retiredTurns map[string]bool + executor *Executor + outputDone chan struct{} + nativeSettled atomic.Bool + settlement agent.TurnSettlement + settlementErr error + terminalCleanupMu sync.Mutex + operationMu sync.Mutex + operations sync.WaitGroup + operationsClosed bool toolEnvironment bool nativeHome string subagents *subagentObservations @@ -152,14 +162,14 @@ func (s *Session) registerHandlers() { rpc.OnNotification("error", s.onErrorNotif) rpc.OnNotification("hook/completed", s.onToolEnvironmentHook) - rpc.OnServerRequest("item/commandExecution/requestApproval", s.handleCodexCommandApproval) - rpc.OnServerRequest("item/fileChange/requestApproval", s.handleCodexFileApproval) - rpc.OnServerRequest("item/permissions/requestApproval", s.handleCodexPermissionsApproval) + s.onServerRequest("item/commandExecution/requestApproval", s.handleCodexCommandApproval) + s.onServerRequest("item/fileChange/requestApproval", s.handleCodexFileApproval) + s.onServerRequest("item/permissions/requestApproval", s.handleCodexPermissionsApproval) // Older app-server releases used this unseparated method name. - rpc.OnServerRequest("item/permissionsRequestApproval", s.handleCodexPermissionsApproval) - rpc.OnServerRequest("item/tool/requestUserInput", s.handleCodexUserInput) - rpc.OnServerRequest("item/tool/call", s.handleFunctionCall) - rpc.OnServerRequest("mcpServer/elicitation/request", s.handleCodexMCPElicitation) + s.onServerRequest("item/permissionsRequestApproval", s.handleCodexPermissionsApproval) + s.onServerRequest("item/tool/requestUserInput", s.handleCodexUserInput) + s.onServerRequest("item/tool/call", s.handleFunctionCall) + s.onServerRequest("mcpServer/elicitation/request", s.handleCodexMCPElicitation) } func (s *Session) onTurnStarted(raw json.RawMessage) { @@ -193,6 +203,7 @@ func (s *Session) onTurnCompleted(raw json.RawMessage) { if json.Unmarshal(raw, &p) != nil || !s.isRootTurn(p.ThreadID, p.Turn.ID) { return } + s.nativeSettled.Store(true) s.stopSteering() usage := p.Turn.Usage @@ -396,6 +407,9 @@ func (s *Session) emitTerminal(message string, asError bool) { } func (s *Session) emitTerminalFailure(message string, asError bool, failure proto.ErrorPayload) { + if s.executor != nil { + defer s.finishAfterTerminal() + } if !s.terminal.CompareAndSwap(false, true) { return } @@ -447,6 +461,9 @@ func (s *Session) closeOut() { s.outMu.Lock() s.outClosed = true close(s.out) + if s.outputDone != nil { + close(s.outputDone) + } s.outMu.Unlock() }) } diff --git a/apps/parsar-daemon/internal/agent/codex/session_cancel.go b/apps/parsar-daemon/internal/agent/codex/session_cancel.go index 2b1fe0080..25024d5b5 100644 --- a/apps/parsar-daemon/internal/agent/codex/session_cancel.go +++ b/apps/parsar-daemon/internal/agent/codex/session_cancel.go @@ -7,6 +7,9 @@ import ( ) func (s *Session) Cancel(ctx context.Context) error { + if s.executor != nil { + return s.cancelExecutorTurn(ctx) + } s.cancelled.Store(true) s.cancelOnce.Do(func() { s.cancelReady = make(chan struct{}) diff --git a/apps/parsar-daemon/internal/agent/codex/session_plan.go b/apps/parsar-daemon/internal/agent/codex/session_plan.go index 35313f715..b0550f894 100644 --- a/apps/parsar-daemon/internal/agent/codex/session_plan.go +++ b/apps/parsar-daemon/internal/agent/codex/session_plan.go @@ -34,6 +34,9 @@ func prepareSessionPlan(ctx context.Context, req proto.PromptRequestPayload, cfg plan.Permissions = profile plan.ExtraConfig = append(plan.ExtraConfig, [2]string{"default_permissions", tomlQuoteString(profile)}) configureRestrictedShellEnvironment(&plan) + if req.LocalEnvironment != nil && req.LocalEnvironment.CapabilityRoot != "" { + configureCapabilityRoot(&plan, profile, req.LocalEnvironment.CapabilityRoot) + } // Native login-shell snapshots live outside the managed tool filesystem. plan.ExtraConfig = append(plan.ExtraConfig, [2]string{"features.shell_snapshot", "false"}) } @@ -110,3 +113,9 @@ func prepareSessionPlan(ctx context.Context, req proto.PromptRequestPayload, cfg } return plan, skillRoots, nil } + +// The Runtime supplies this resolved root after validating its binding. Quote the +// path as one TOML key; neither public options nor source directories select it. +func configureCapabilityRoot(plan *SessionPlan, profile, root string) { + plan.ExtraConfig = append(plan.ExtraConfig, [2]string{"permissions." + tomlQuoteString(profile) + ".filesystem." + tomlQuoteString(root), tomlQuoteString("read")}) +} diff --git a/apps/parsar-daemon/internal/agent/codex/session_run.go b/apps/parsar-daemon/internal/agent/codex/session_run.go index 170708625..21736ba88 100644 --- a/apps/parsar-daemon/internal/agent/codex/session_run.go +++ b/apps/parsar-daemon/internal/agent/codex/session_run.go @@ -16,15 +16,32 @@ func (s *Session) run(plan SessionPlan, req proto.PromptRequestPayload) { defer s.cleanup() defer s.closeRunOutput() - if err := s.resolveThread(req, plan); err != nil { + if err := s.startNative(s.cancelCtx, plan, req); err != nil { s.emitTerminal(err.Error(), true) return } + // Block until terminal handlers close the RPC child or cancellation arrives. + select { + case <-s.rpc.Done(): + if !s.cancelled.Load() && s.cancelCtx.Err() == nil { + s.emitTerminal("codex: connection closed before the run completed", true) + } + case <-s.cancelCtx.Done(): + _ = s.rpc.Close() + } +} + +func (s *Session) startNative(ctx context.Context, plan SessionPlan, req proto.PromptRequestPayload) error { + if s.currentThreadID() == "" { + if err := s.resolveThread(req, plan); err != nil { + return err + } + } + input, err := nativeInput(req.Input) if err != nil { - s.emitTerminal("codex: empty prompt", true) - return + return err } turnParams := TurnStartParams{ ThreadID: s.currentThreadID(), @@ -33,8 +50,7 @@ func (s *Session) run(plan SessionPlan, req proto.PromptRequestPayload) { if plan.CollaborationMode != "" { model := strings.TrimSpace(s.resolvedModel) if model == "" { - s.emitTerminal("codex: collaboration mode requires a resolved model", true) - return + return fmt.Errorf("codex: collaboration mode requires a resolved model") } var developerInstructions *string if plan.SystemPrompt != "" { @@ -48,22 +64,13 @@ func (s *Session) run(plan SessionPlan, req proto.PromptRequestPayload) { }, } } - turnCtx, turnCancel := context.WithTimeout(s.cancelCtx, 10*time.Second) + turnCtx, turnCancel := context.WithTimeout(ctx, 10*time.Second) _, ackErr := s.rpc.requestWithResult(turnCtx, "turn/start", turnParams, s.bindTurnResult) turnCancel() if ackErr != nil { s.cfg.logger.Warn("codex: turn/start ack failed", "run_id", s.runID, "err", ackErr) - s.emitTerminal(fmt.Sprintf("codex: turn/start: %v", ackErr), true) - return + return fmt.Errorf("codex: turn/start: %w", ackErr) } - // Block until terminal handlers close the RPC child or cancellation arrives. - select { - case <-s.rpc.Done(): - if !s.cancelled.Load() && s.cancelCtx.Err() == nil { - s.emitTerminal("codex: connection closed before the run completed", true) - } - case <-s.cancelCtx.Done(): - _ = s.rpc.Close() - } + return nil } diff --git a/apps/parsar-daemon/internal/agent/codex/session_steering.go b/apps/parsar-daemon/internal/agent/codex/session_steering.go index 4ad2716fe..f0d6be931 100644 --- a/apps/parsar-daemon/internal/agent/codex/session_steering.go +++ b/apps/parsar-daemon/internal/agent/codex/session_steering.go @@ -38,6 +38,10 @@ func (s *Session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP } func (s *Session) steer(ctx context.Context, input proto.PromptSteerPayload, written func()) error { + if !s.beginOperation() { + return agent.ErrSteeringInactive + } + defer s.endOperation() ctx, cancel := context.WithCancel(ctx) defer cancel() s.steering.mu.Lock() @@ -96,7 +100,7 @@ func (s *Session) steer(ctx context.Context, input proto.PromptSteerPayload, wri } func (s *Session) startSteering(threadID, turnID string) bool { - if !s.isRootThread(threadID) || turnID == "" { + if !s.isRootThread(threadID) || turnID == "" || s.retiredTurns[turnID] { return false } s.steering.mu.Lock() diff --git a/apps/parsar-daemon/internal/agent/codex/session_usage.go b/apps/parsar-daemon/internal/agent/codex/session_usage.go index 309d05b99..17e672284 100644 --- a/apps/parsar-daemon/internal/agent/codex/session_usage.go +++ b/apps/parsar-daemon/internal/agent/codex/session_usage.go @@ -30,7 +30,7 @@ func (s *Session) onUsageUpdated(raw json.RawMessage) { s.usageMu.Unlock() return } - if s.terminal.Load() { + if s.terminal.Load() || s.retiredTurns[p.TurnID] { return } var observed *TurnUsage diff --git a/apps/parsar-daemon/internal/agent/codex/terminal_cleanup.go b/apps/parsar-daemon/internal/agent/codex/terminal_cleanup.go new file mode 100644 index 000000000..49d409b2a --- /dev/null +++ b/apps/parsar-daemon/internal/agent/codex/terminal_cleanup.go @@ -0,0 +1,61 @@ +package codex + +import ( + "context" + "encoding/json" + "errors" +) + +// A native interrupted Turn can leave unified-exec terminals running. Retire +// their exact thread-owned handles; the bulk clean RPC only queues cleanup. +func (s *Session) cleanupNativeTerminals(ctx context.Context) error { + s.terminalCleanupMu.Lock() + defer s.terminalCleanupMu.Unlock() + thread := s.currentThreadID() + if thread == "" { + return nil + } + request := func(method string, params any) (json.RawMessage, error) { + return s.rpc.request(ctx, method, params, func(frame any) error { return s.rpc.writeFrameContext(ctx, frame) }) + } + seen := map[string]bool{} + for { + raw, err := request("thread/backgroundTerminals/list", map[string]any{"threadId": thread, "limit": 100}) + if err != nil { + return err + } + var page struct { + Data *[]struct { + ProcessID string `json:"processId"` + } `json:"data"` + NextCursor *string `json:"nextCursor"` + } + if json.Unmarshal(raw, &page) != nil || page.Data == nil { + return errors.New("codex: invalid native terminal list") + } + if len(*page.Data) == 0 { + if page.NextCursor != nil { + return errors.New("codex: incomplete native terminal list") + } + return nil + } + for _, terminal := range *page.Data { + if terminal.ProcessID == "" || seen[terminal.ProcessID] { + return errors.New("codex: invalid or retained native terminal identity") + } + seen[terminal.ProcessID] = true + raw, err = request("thread/backgroundTerminals/terminate", map[string]string{"threadId": thread, "processId": terminal.ProcessID}) + if err != nil { + return err + } + var receipt struct { + Terminated bool `json:"terminated"` + } + if json.Unmarshal(raw, &receipt) != nil || !receipt.Terminated { + return errors.New("codex: native terminal termination unconfirmed") + } + } + // Terminating the first page removes its handles. Read the first page + // again to confirm absence and cover any remaining native terminals. + } +} diff --git a/apps/parsar-daemon/internal/agent/codex/terminal_cleanup_test.go b/apps/parsar-daemon/internal/agent/codex/terminal_cleanup_test.go new file mode 100644 index 000000000..ace5c1a17 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/codex/terminal_cleanup_test.go @@ -0,0 +1,113 @@ +package codex + +import ( + "context" + "os" + "path/filepath" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func TestExecutorCancellationRequiresNativeTerminalCleanup(t *testing.T) { + for _, confirmed := range []bool{false, true} { + t.Run(map[bool]string{false: "retry", true: "confirmed"}[confirmed], func(t *testing.T) { + e, root := executorFixture(t, "terminal-cleanup") + allow := func() { + t.Helper() + if err := os.WriteFile(filepath.Join(root, "frames.jsonl.allow-cleanup"), nil, 0600); err != nil { + t.Fatal(err) + } + } + if confirmed { + allow() + } + out := make(chan proto.Envelope, 20) + turn, err := e.StartTurn(t.Context(), "cancel", proto.TextInput("hold"), out) + if err != nil { + t.Fatal(err) + } + ctx, stop := context.WithTimeout(t.Context(), 5*time.Second) + defer stop() + if err := turn.Cancel(ctx); (err == nil) != confirmed { + t.Fatalf("cancellation confirmed=%t: %v", confirmed, err) + } + settlement, err := turn.AwaitSettlement(ctx) + if (err == nil) != confirmed || settlement.Reusable != confirmed { + t.Fatal("cleanup changed cancellation confirmation", settlement, err) + } + for range out { + } + if !e.prepared.session.rpc.Alive() { + t.Fatal("native owner lost before cleanup/reuse") + } + frames := preparationFrames(t, root) + interrupted, terminated := false, false + for _, frame := range frames { + if frame.Method == "turn/interrupt" { + interrupted = true + } + if frame.Method == "thread/backgroundTerminals/terminate" { + if !interrupted { + t.Fatal("cleanup preceded native interrupt") + } + terminated = true + } + } + if !terminated { + t.Fatal("native interrupted receipt skipped terminal cleanup") + } + if confirmed { + nextOut := make(chan proto.Envelope, 20) + next, err := e.StartTurn(ctx, "next", proto.TextInput("answer"), nextOut) + if err != nil || !awaitExecutorTurn(t, next, nextOut).Reusable { + t.Fatal("confirmed cleanup prevented reuse", err) + } + } else { + if err := e.Close(ctx); err == nil || !e.prepared.session.rpc.Alive() { + t.Fatal("failed cleanup released the owned native process", err) + } + allow() + if err := e.Close(ctx); err != nil { + t.Fatal("cleanup retry failed", err) + } + if err := turn.Cancel(ctx); err == nil { + t.Fatal("resource cleanup fabricated native cancellation confirmation") + } + } + }) + } +} + +func TestExecutorCloseCleansNativeTerminalsAfterNormalCompletion(t *testing.T) { + e, root := executorFixture(t, "terminal-cleanup") + out := make(chan proto.Envelope, 20) + turn, err := e.StartTurn(t.Context(), "normal", proto.TextInput("answer"), out) + if err != nil { + t.Fatal(err) + } + if !awaitExecutorTurn(t, turn, out).Reusable { + t.Fatal("normal Turn did not retain its Executor") + } + for _, frame := range preparationFrames(t, root) { + if frame.Method == "thread/backgroundTerminals/terminate" { + t.Fatal("normal Turn prematurely killed its background terminal") + } + } + if err := os.WriteFile(filepath.Join(root, "frames.jsonl.allow-cleanup"), nil, 0600); err != nil { + t.Fatal(err) + } + if err := e.Close(t.Context()); err != nil { + t.Fatal(err) + } + terminated := false + for _, frame := range preparationFrames(t, root) { + if frame.Method == "thread/backgroundTerminals/terminate" { + terminated = true + } + } + if !terminated || e.prepared.session.rpc.Alive() { + t.Fatal("Executor Close left native terminals or owner alive") + } +} diff --git a/apps/parsar-daemon/internal/agent/executor.go b/apps/parsar-daemon/internal/agent/executor.go new file mode 100644 index 000000000..1e1346cf2 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/executor.go @@ -0,0 +1,61 @@ +package agent + +import ( + "context" + "fmt" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +// TurnSettlement describes native readiness after this Turn has fully drained. +// A false result must carry a reason; it requires confirmed Executor.Close. +type TurnSettlement struct { + Reusable bool + Reason string +} + +// Turn owns one output stream and never retargets cancellation to a successor. +type Turn interface { + Session + CancellationOutcome() proto.DonePayload + // Success confirms closed output and settled native input, function, + // interaction and child-work obligations. Errors cannot prove cancellation. + AwaitSettlement(context.Context) (TurnSettlement, error) +} + +// Executor retains a fixed native configuration across independently owned Turns. +type Executor interface { + // A nil Turn guarantees no input was submitted or output retained and leaves + // out with the caller. A non-nil Turn owns out, including on error; input + // may have been submitted and must never be replayed automatically. + StartTurn(context.Context, string, proto.MessageInput, chan<- proto.Envelope) (Turn, error) + // Success confirms all native work and owned output streams have stopped. + // It does not establish an otherwise unknown Turn result. An error retains + // resource ownership; another caller may await or retry Close. + Close(context.Context) error +} + +// A failed factory retains any unconfirmed cleanup in its non-nil Executor. +type ExecutorFactory func(context.Context, proto.PromptRequestPayload) (Executor, error) + +func (r *Registry) RegisterExecutor(kind string, factory ExecutorFactory) { + r.mu.Lock() + defer r.mu.Unlock() + info, exists := r.kinds[kind] + if !exists || factory == nil { + panic("agent.Registry.RegisterExecutor: registered kind and factory required") + } + r.executors[kind] = factory + info.Capabilities.Preparation = true + r.kinds[kind] = info +} + +func (r *Registry) ResolveExecutor(kind string) (ExecutorFactory, error) { + r.mu.RLock() + defer r.mu.RUnlock() + factory := r.executors[kind] + if factory == nil { + return nil, fmt.Errorf("agent: executor unavailable for %q", kind) + } + return factory, nil +} diff --git a/apps/parsar-daemon/internal/agent/mcode/connection.go b/apps/parsar-daemon/internal/agent/mcode/connection.go new file mode 100644 index 000000000..420155530 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/connection.go @@ -0,0 +1,146 @@ +package mcode + +import ( + "bufio" + "context" + "encoding/json" + "fmt" + "io" + "strconv" + "sync" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent/clirunner" +) + +// connection is the process and ACP owner. It keeps draining while no Turn owns +// output; RPC identities remain unique across every Turn on this connection. +type connection struct { + process *clirunner.Process + exited chan struct{} + exitErr error + writeMu sync.Mutex + transportMu sync.Mutex + nextID int + responses map[string]chan rpcFrame + current *Session +} + +func (c *connection) read() { + defer close(c.exited) + stderrDone := make(chan struct{}) + go func() { defer close(stderrDone); _, _ = io.Copy(io.Discard, c.process.Stderr) }() + scanner := bufio.NewScanner(c.process.Stdout) + scanner.Buffer(make([]byte, 64*1024), 16*1024*1024) + var readErr error + for scanner.Scan() { + var frame rpcFrame + if json.Unmarshal(scanner.Bytes(), &frame) != nil { + readErr = fmt.Errorf("mcode: malformed ACP response") + c.process.Cancel() + break + } + c.transportMu.Lock() + response, owner := c.responses[string(frame.ID)], c.current + c.transportMu.Unlock() + if frame.Method == "" && response != nil { + select { + case response <- frame: + default: + } + continue + } + if owner == nil { + // A late request has no Turn authority. Never hold it for a successor. + if frame.Method != "" && len(frame.ID) > 0 { + if err := c.write(rpcFrame{JSONRPC: "2.0", ID: frame.ID, Error: &rpcError{Code: -32601, Message: "No active Turn"}}); err != nil { + readErr = err + c.process.Cancel() + break + } + } + continue + } + select { + case owner.frames <- frame: + case <-owner.finished: + case <-c.process.Context().Done(): + } + } + if scanner.Err() != nil { + readErr = fmt.Errorf("mcode: ACP stream read failed") + c.process.Cancel() + } + waitErr := c.process.Wait() + <-stderrDone + if readErr != nil { + c.exitErr = readErr + } else { + c.exitErr = waitErr + } +} + +func (c *connection) write(frame rpcFrame) error { + c.writeMu.Lock() + defer c.writeMu.Unlock() + return json.NewEncoder(c.process.Stdin).Encode(frame) +} + +func (c *connection) reserveResponse() (string, chan rpcFrame) { + c.transportMu.Lock() + defer c.transportMu.Unlock() + c.nextID++ + id := strconv.Itoa(c.nextID) + reply := make(chan rpcFrame, 1) + c.responses[id] = reply + return id, reply +} + +func (c *connection) removeResponse(id string) { + c.transportMu.Lock() + delete(c.responses, id) + c.transportMu.Unlock() +} + +func (c *connection) setCurrent(s *Session) { + c.transportMu.Lock() + c.current = s + c.transportMu.Unlock() +} + +// An ordered ACP control response fences notifications left in the pipe by the +// preceding prompt. No Turn is attached while this barrier drains them. +func (c *connection) barrier(ctx context.Context, session, model string) error { + id, reply := c.reserveResponse() + defer c.removeResponse(id) + raw, _ := json.Marshal(map[string]string{"sessionId": session, "configId": "model", "value": model}) + err := c.writeContext(ctx, rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: "session/set_config_option", Params: raw}) + if err != nil { + return err + } + select { + case frame := <-reply: + if frame.Error != nil { + return fmt.Errorf("mcode: native readiness barrier failed") + } + return nil + case <-c.exited: + return fmt.Errorf("mcode: native process exited") + case <-ctx.Done(): + return ctx.Err() + } +} + +// Drain the deadline callback before releasing the caller's Turn/start guard; +// an old writer deadline must never kill a subsequent owner. +func (c *connection) writeContext(ctx context.Context, frame rpcFrame) error { + callbackDone := make(chan struct{}) + stop := context.AfterFunc(ctx, func() { + defer close(callbackDone) + c.process.Cancel() + }) + err := c.write(frame) + if !stop() { + <-callbackDone + } + return err +} diff --git a/apps/parsar-daemon/internal/agent/mcode/environment_mcp_test.go b/apps/parsar-daemon/internal/agent/mcode/environment_mcp_test.go index a47e4eb1c..3b78994a6 100644 --- a/apps/parsar-daemon/internal/agent/mcode/environment_mcp_test.go +++ b/apps/parsar-daemon/internal/agent/mcode/environment_mcp_test.go @@ -15,7 +15,7 @@ import ( ) func environmentMCPFixture() proto.EnvironmentMCP { - return proto.EnvironmentMCP{PackageRoot: "plugins/fixture", Server: agentplugin.MCPServer{ + return proto.EnvironmentMCP{InstallationRoot: "/private/runtime/capabilities", WorkspaceRoot: "/private/runtime/workspace", PackageRoot: "plugins/fixture", Server: agentplugin.MCPServer{ Name: "proof.server", Type: "stdio", Command: "never-exec-before-sandbox", Args: []string{"private-argument"}, EnvVars: []string{"USER_SELECTED"}, CWD: "resources", }} @@ -54,7 +54,7 @@ func TestEnvironmentMCPUsesFixedLauncherForNewAndLoadedSessions(t *testing.T) { } server := params.MCP[1] if server.Name != "proof.server" || server.Command != "/usr/bin/python3" || server.Env == nil || len(server.Env) != 0 || - !reflect.DeepEqual(server.Args, []string{"-I", "-S", "/usr/local/bin/oac-runtime-initialize", "stdio", "plugins/fixture", "proof.server"}) { + !reflect.DeepEqual(server.Args, []string{"-I", "-S", "/usr/local/bin/oac-runtime-initialize", "stdio", "/private/runtime/capabilities", "/private/runtime/workspace", "plugins/fixture", "proof.server"}) { t.Fatal("ACP declaration bypassed the fixed isolated launcher") } if (params.SessionID != "") != resume || strings.Contains(string(raw), "must-not") || strings.Contains(string(raw), "private-argument") { @@ -114,7 +114,7 @@ func TestEnvironmentMCPCancelSettlesPendingObservationBeforeDone(t *testing.T) { if err != nil { t.Fatal(err) } - t.Cleanup(func() { _ = session.Cancel(context.Background()) }) + t.Cleanup(func() { _ = resource.(*prepared).Cancel(context.Background()) }) select { case event := <-out: var call proto.ToolCallPayload @@ -125,7 +125,11 @@ func TestEnvironmentMCPCancelSettlesPendingObservationBeforeDone(t *testing.T) { t.Fatal("MCP start was not observed") } if err := session.Cancel(ctx); err != nil { - t.Fatal(err) + t.Fatal("stopped nonreusable MCP owner reported cancellation failure", err) + } + settlement, err := session.(*Session).AwaitSettlement(ctx) + if err != nil || settlement.Reusable { + t.Fatal("pending MCP work retained a reusable owner", settlement, err) } closed, done := false, false for event := range out { diff --git a/apps/parsar-daemon/internal/agent/mcode/execution.go b/apps/parsar-daemon/internal/agent/mcode/execution.go index b8c94bbed..6a3eb80c4 100644 --- a/apps/parsar-daemon/internal/agent/mcode/execution.go +++ b/apps/parsar-daemon/internal/agent/mcode/execution.go @@ -13,7 +13,7 @@ func SupportsExecution(version string) bool { } func validateExecutionRequest(req proto.PromptRequestPayload) error { - if !req.ReleaseOnCompletion || !req.DisableExecutionEnvironment || req.WorkDir != "" || req.AgentStateKey == "" || req.LocalEnvironment != nil || req.RequireExistingNativeSession || len(req.FunctionTools) != 0 || (req.MCPHTTPServers != nil && len(*req.MCPHTTPServers) != 0) { + if !req.DisableExecutionEnvironment || req.WorkDir != "" || req.AgentStateKey == "" || req.LocalEnvironment != nil || req.RequireExistingNativeSession || len(req.FunctionTools) != 0 || (req.MCPHTTPServers != nil && len(*req.MCPHTTPServers) != 0) { return fmt.Errorf("mcode: unsupported execution configuration") } if req.ExecutionControls == nil || req.ExecutionControls.OutputFormat != nil || req.ExecutionControls.WebSearch != "disabled" || (req.ExecutionControls.TextVerbosity != "" && req.ExecutionControls.TextVerbosity != "medium") { diff --git a/apps/parsar-daemon/internal/agent/mcode/executor.go b/apps/parsar-daemon/internal/agent/mcode/executor.go new file mode 100644 index 000000000..632bb6a40 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/executor.go @@ -0,0 +1,173 @@ +package mcode + +import ( + "context" + "errors" + "fmt" + "strings" + "sync" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +var errTurnCancelled = errors.New("mcode: Turn cancelled before submission") + +type executor struct { + mu sync.Mutex + connection *connection + req proto.PromptRequestPayload + opts launchOptions + nativeSession, model string + active *Session + starting, closed, invalid bool +} + +var _ agent.Executor = (*executor)(nil) +var _ agent.Turn = (*Session)(nil) + +// NewExecutorFactory fixes the deployment workspace once; nil selects none. +func NewExecutorFactory(config *WorkspaceConfig) agent.ExecutorFactory { + var frozen *WorkspaceConfig + if config != nil { + value := *config + value.ProtectedDirs = append([]string(nil), config.ProtectedDirs...) + value.AllowedDomains = append([]string(nil), config.AllowedDomains...) + frozen = &value + } + return func(ctx context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + if ctx == nil { + ctx = context.Background() + } + if req.RunID != "" || len(req.Input) != 0 || req.ConversationID != "" { + return nil, fmt.Errorf("mcode: Executor configuration cannot contain Turn input") + } + var err error + var opts launchOptions + binary := defaultBinary() + if frozen == nil { + opts, err = prepareOptions(ctx, req) + } else { + binary = frozen.Binary + opts, err = prepareWorkspaceOptions(ctx, *frozen, req) + } + if err != nil { + return nil, err + } + return newExecutor(ctx, req, opts, binary) + } +} + +func newExecutor(ctx context.Context, req proto.PromptRequestPayload, opts launchOptions, binary string) (*executor, error) { + bootstrap, err := launch(ctx, req, opts, binary, nil) + if err != nil { + return nil, err + } + e := &executor{connection: bootstrap.connection, req: req, opts: opts} + err = bootstrap.prepareNative() + e.connection.setCurrent(nil) + close(bootstrap.finished) + if err == nil { + err = ctx.Err() + } + if err != nil { + cleanup, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + if closeErr := e.Close(cleanup); closeErr != nil { + return e, err + } + return nil, err + } + e.nativeSession, e.model = bootstrap.sessionID, bootstrap.nativeModel + return e, nil +} + +func (e *executor) StartTurn(ctx context.Context, runID string, input proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + if ctx == nil { + ctx = context.Background() + } + if strings.TrimSpace(runID) == "" || input.Validate() != nil || out == nil || ctx.Err() != nil { + return nil, fmt.Errorf("mcode: Turn requires live context, identity, input and output") + } + text, err := input.TextOnly() + if err != nil { + return nil, err + } + e.mu.Lock() + if e.closed || e.invalid || e.starting || e.active != nil { + e.mu.Unlock() + return nil, fmt.Errorf("mcode: Executor is unavailable") + } + e.starting = true + e.mu.Unlock() + + // No model input is sent and no output channel retained until this fence. + barrier, cancel := context.WithTimeout(ctx, 60*time.Second) + err = e.connection.barrier(barrier, e.nativeSession, e.model) + cancel() + e.mu.Lock() + defer e.mu.Unlock() + e.starting = false + if err != nil || e.closed || ctx.Err() != nil { + if err != nil { + e.invalid = true + return nil, err + } + return nil, fmt.Errorf("mcode: Executor closed before input") + } + select { + case <-e.connection.exited: + e.invalid = true + return nil, fmt.Errorf("mcode: native process exited") + default: + } + req := e.req + req.RunID, req.Input = runID, proto.TextInput(text) + s := newTurnSession(e.connection.process.Context(), req, e.opts, e.connection, out) + s.executor, s.sessionID, s.nativeModel = e, e.nativeSession, e.model + s.settled, s.inputDone = make(chan struct{}), make(chan struct{}) + s.outputContext, s.outputCancel = context.WithCancel(context.Background()) + e.active = s + e.connection.setCurrent(s) + go s.runExecutorTurn() + go func() { + select { + case <-ctx.Done(): + cancellation, stop := context.WithTimeout(context.Background(), 10*time.Second) + defer stop() + _ = s.Cancel(cancellation) + case <-s.settled: + } + }() + return s, nil +} + +// Close keeps the native owner reachable until process, pipes and active Turn +// settlement all finish. Repeating it waits on those same owners. +func (e *executor) Close(ctx context.Context) error { + if ctx == nil { + ctx = context.Background() + } + e.mu.Lock() + e.closed = true + current := e.active + if current != nil { + current.outputCancel() + } + e.connection.process.Cancel() + e.mu.Unlock() + select { + case <-e.connection.exited: + case <-ctx.Done(): + return ctx.Err() + } + if current != nil { + select { + case <-current.settled: + case <-ctx.Done(): + return ctx.Err() + } + } + return nil +} diff --git a/apps/parsar-daemon/internal/agent/mcode/executor_backpressure_test.go b/apps/parsar-daemon/internal/agent/mcode/executor_backpressure_test.go new file mode 100644 index 000000000..f2f78c6df --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/executor_backpressure_test.go @@ -0,0 +1,99 @@ +package mcode + +import ( + "context" + "strings" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func TestExecutorBlockedPromptWriteCanBeCancelledOrClosed(t *testing.T) { + for _, operation := range []string{"cancel", "close"} { + t.Run(operation, func(t *testing.T) { + e, _ := executorFixture(t, "executor-backpressure", true) + out := make(chan proto.Envelope, 32) + turn, err := e.StartTurn(t.Context(), "blocked", proto.TextInput(strings.Repeat("x", 4*1024*1024)), out) + if err != nil { + t.Fatal(err) + } + deadline := time.Now().Add(3 * time.Second) + for e.connection.writeMu.TryLock() { + e.connection.writeMu.Unlock() + if time.Now().After(deadline) { + t.Fatal("prompt did not reach native pipe") + } + time.Sleep(time.Millisecond) + } + // The native fixture acknowledged readiness but does not read stdin. + // A multi-megabyte prompt therefore fills the actual child process pipe. + result := make(chan error, 1) + go func() { + ctx, cancel := context.WithTimeout(t.Context(), 200*time.Millisecond) + defer cancel() + if operation == "cancel" { + result <- turn.Cancel(ctx) + } else { + result <- e.Close(ctx) + } + }() + select { + case err := <-result: + if operation == "cancel" && err == nil { + t.Fatal("unacknowledged cancellation was reported applied") + } + case <-time.After(3 * time.Second): + // Emergency cleanup avoids leaking the intentionally blocked fixture. + e.connection.process.Cancel() + t.Fatal(operation + " could not interrupt a blocked native write") + } + ctx, cancel := context.WithTimeout(t.Context(), 3*time.Second) + defer cancel() + if err := e.Close(ctx); err != nil { + t.Fatal("Close did not retain cleanup ownership", err) + } + settlement, err := turn.AwaitSettlement(ctx) + if err == nil || settlement.Reusable { + t.Fatal("lost native prompt produced successful settlement", settlement, err) + } + for range out { + } + }) + } +} + +func TestExecutorUnknownSteeringOutcomeCannotBecomeAppliedCancellation(t *testing.T) { + e, _ := executorFixture(t, "executor-steer-unknown", true) + out := make(chan proto.Envelope, 32) + turn, err := e.StartTurn(t.Context(), "unknown-input", proto.TextInput("wait"), out) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(t.Context(), 3*time.Second) + defer cancel() + select { + case <-out: + case <-ctx.Done(): + t.Fatal("native prompt did not start") + } + session := turn.(*Session) + if err := session.Steer(ctx, proto.PromptSteerPayload{InputID: "input-unknown", Input: proto.TextInput("continue")}); err == nil { + t.Fatal("unknown native receipt was accepted") + } + if err := turn.Cancel(ctx); err == nil { + t.Fatal("unknown input became applied cancellation") + } + settlement, err := turn.AwaitSettlement(ctx) + if err == nil || settlement.Reusable { + t.Fatal("unknown input lost its settlement error", settlement, err) + } + if err := turn.Cancel(ctx); err == nil { + t.Fatal("repeated cancel lost original uncertainty") + } + if err := e.Close(ctx); err != nil { + t.Fatal("settlement failure prevented resource cleanup", err) + } + for range out { + } +} diff --git a/apps/parsar-daemon/internal/agent/mcode/executor_native_test.go b/apps/parsar-daemon/internal/agent/mcode/executor_native_test.go new file mode 100644 index 000000000..5340c3f9f --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/executor_native_test.go @@ -0,0 +1,176 @@ +package mcode + +import ( + "context" + "encoding/json" + "os" + "strings" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +// This opt-in test makes real model calls and uses an isolated native home. +// Provider options are read from a private file and never included in failures. +func TestNativeMCodeExecutorReuse(t *testing.T) { + binary, options := os.Getenv("OAC_RUNTIME_MCODE_BIN"), os.Getenv("OAC_TEST_MCODE_REAL_OPTIONS") + if binary == "" || options == "" { + t.Skip("native executable and private provider options required") + } + ctx, cancel := context.WithTimeout(t.Context(), 8*time.Minute) + defer cancel() + if version, err := CheckCLIAvailable(ctx, binary); err != nil || version != SupportedVersion { + t.Fatal("pinned native version verification failed") + } + raw, err := os.ReadFile(options) + if err != nil { + t.Fatal("private provider options unavailable") + } + req := executionRequest(t) + req.ReleaseOnCompletion = false + req.RunID, req.Input, req.ConversationID = "", nil, "" + if json.Unmarshal(raw, &req.AgentOptions) != nil { + t.Fatal("invalid private provider options") + } + value, err := NewExecutorFactory(nil)(ctx, req) + if err != nil { + t.Fatal("native Executor preparation failed") + } + e := value.(*executor) + defer func() { + cleanup, stop := context.WithTimeout(context.Background(), 15*time.Second) + defer stop() + if e.Close(cleanup) != nil { + t.Error("native owner cleanup failed") + } + }() + pid, nativeID := e.connection.process.Cmd.Process.Pid, e.nativeSession + assertOwner := func() { + t.Helper() + if e.connection.process.Cmd.Process.Pid != pid || e.nativeSession != nativeID { + t.Fatal("native owner changed") + } + select { + case <-e.connection.exited: + t.Fatal("native process exited") + default: + } + } + run := func(id, prompt string) agent.Turn { + t.Helper() + out := make(chan proto.Envelope, 256) + turn, startErr := e.StartTurn(ctx, id, proto.TextInput(prompt), out) + if startErr != nil || turn == nil { + t.Fatal("native Turn admission failed") + } + content := "" + doneCount := 0 + for event := range out { + if event.ID != id { + t.Fatal("event crossed Turn boundary") + } + switch event.Type { + case proto.TypeError: + t.Fatal("native Turn reported an error") + case proto.TypeDone: + var done proto.DonePayload + if json.Unmarshal(event.Payload, &done) != nil { + t.Fatal("invalid completion") + } + content = done.Content + doneCount++ + } + } + settlement, settleErr := turn.AwaitSettlement(ctx) + if settleErr != nil || !settlement.Reusable { + t.Fatal("normal Turn did not establish reusable settlement") + } + if doneCount != 1 || !strings.Contains(content, "ORCHID-72-BLUE") { + t.Fatal("native conversation did not retain the private test marker") + } + assertOwner() + t.Logf("Verified marker recall and retained native owner for %s", id) + return turn + } + first := run("first", "Remember the exact marker ORCHID-72-BLUE for this conversation. Reply with only that marker.") + run("second", "What exact marker did I ask you to remember? Reply with only that marker.") + out := make(chan proto.Envelope, 256) + interrupted, err := e.StartTurn(ctx, "cancelled", proto.TextInput("Produce 2000 numbered lines now. Each line must contain a different sentence about rain. Start immediately at line 1 and continue without stopping or summarizing."), out) + if err != nil || interrupted == nil { + t.Fatal("cancellation Turn admission failed") + } + streaming := false + for !streaming { + select { + case event, ok := <-out: + if !ok || event.Type == proto.TypeDone || event.Type == proto.TypeError { + t.Fatal("Turn ended before live cancellation could be tested") + } + if event.Type == proto.TypeDelta { + streaming = true + } + case <-ctx.Done(): + t.Fatal("native stream did not begin before deadline") + } + } + select { + case <-interrupted.(*Session).finished: + t.Fatal("cancellation was not in flight") + default: + } + staleCtx, staleStop := context.WithTimeout(ctx, 5*time.Second) + if first.Cancel(staleCtx) != nil { + staleStop() + t.Fatal("stale cancellation failed") + } + staleStop() + interrupted.(*Session).mu.Lock() + wasCancelled := interrupted.(*Session).cancelled + interrupted.(*Session).mu.Unlock() + if wasCancelled { + t.Fatal("stale cancellation targeted current Turn") + } + stopCtx, stop := context.WithTimeout(ctx, 30*time.Second) + cancelErr := interrupted.Cancel(stopCtx) + settlement, settleErr := interrupted.AwaitSettlement(stopCtx) + stop() + if settleErr != nil || cancelErr != nil { + t.Fatal("native cancellation did not settle", cancelErr, settleErr) + } + if !settlement.Reusable { + t.Log("Native cancellation cannot establish reusable settlement; successor must use recovery") + successor := make(chan proto.Envelope, 1) + next, nextErr := e.StartTurn(ctx, "unsafe-successor", proto.TextInput("must not execute"), successor) + if next != nil || nextErr == nil { + t.Fatal("unsettled native owner accepted a successor") + } + close(successor) + cleanup, cleanupStop := context.WithTimeout(ctx, 15*time.Second) + if e.Close(cleanup) != nil { + cleanupStop() + t.Fatal("invalid native owner did not close") + } + cleanupStop() + req.AgentSessionID = nativeID + recovered, recoverErr := NewExecutorFactory(nil)(ctx, req) + if recoverErr != nil || recovered == nil { + t.Fatal("exact native history recovery failed") + } + e = recovered.(*executor) + if e.nativeSession != nativeID || e.connection.process.Cmd.Process.Pid == pid { + t.Fatal("recovery did not replace the process while retaining native history") + } + pid = e.connection.process.Cmd.Process.Pid + run("recovery", "What exact marker did I ask you to remember at the beginning? Reply with only that marker.") + t.Log("Verified exact history continuation in a replacement native owner after nonreusable cancellation") + return + } + if cancelErr != nil { + t.Fatal("settled cancellation reported an error") + } + assertOwner() + run("continuation", "What exact marker did I ask you to remember at the beginning? Reply with only that marker.") + t.Log("Verified native process and session reuse across two normal Turns, active cancellation, and continuation") +} diff --git a/apps/parsar-daemon/internal/agent/mcode/executor_test.go b/apps/parsar-daemon/internal/agent/mcode/executor_test.go new file mode 100644 index 000000000..f0e44f72c --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/executor_test.go @@ -0,0 +1,243 @@ +package mcode + +import ( + "context" + "encoding/json" + "os" + "strings" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func executorFixture(t *testing.T, scenario string, workspace bool) (*executor, string) { + t.Helper() + config, req, record := workspaceFixture(t) + req.ReleaseOnCompletion = false + script, err := os.ReadFile(config.Binary) + if err != nil { + t.Fatal(err) + } + if err = os.WriteFile(config.Binary, []byte(strings.Replace(string(script), "HELPER=prepared", "HELPER="+scenario, 1)), 0700); err != nil { + t.Fatal(err) + } + var factory agent.ExecutorFactory + if workspace { + factory = NewExecutorFactory(&config) + } else { + req = executionRequest(t) + req.ReleaseOnCompletion = false + req.RunID, req.Input, req.ConversationID = "", nil, "" + t.Setenv("OAC_RUNTIME_MCODE_BIN", config.Binary) + factory = NewExecutorFactory(nil) + } + value, err := factory(t.Context(), req) + if err != nil { + t.Fatal(err) + } + e := value.(*executor) + t.Cleanup(func() { + cleanup, cancel := context.WithTimeout(context.Background(), 5*time.Second) + defer cancel() + if err := e.Close(cleanup); err != nil { + t.Error(err) + } + }) + return e, record +} + +func runExecutorFixtureTurn(t *testing.T, e *executor, id, text string) (*Session, []proto.Envelope) { + t.Helper() + out := make(chan proto.Envelope, 256) + turn, err := e.StartTurn(t.Context(), id, proto.TextInput(text), out) + if err != nil { + t.Fatal(err) + } + var events []proto.Envelope + for event := range out { + events = append(events, event) + } + ctx, cancel := context.WithTimeout(t.Context(), 3*time.Second) + defer cancel() + settlement, err := turn.AwaitSettlement(ctx) + if err != nil || !settlement.Reusable { + t.Fatalf("settlement = %+v, %v", settlement, err) + } + return turn.(*Session), events +} + +func TestExecutorReusesNativeOwnerWithFreshTurnsAndIdleDrain(t *testing.T) { + for _, workspace := range []bool{false, true} { + t.Run(map[bool]string{false: "none", true: "workspace"}[workspace], func(t *testing.T) { + e, record := executorFixture(t, "executor-reuse", workspace) + pid := e.connection.process.Cmd.Process.Pid + first, _ := runExecutorFixtureTurn(t, e, "first", "one") + second, events := runExecutorFixtureTurn(t, e, "second", "two") + if first == second || e.connection.process.Cmd.Process.Pid != pid { + t.Fatal("Turn reused mutable state or rebuilt the process") + } + text, tools, done := "", 0, 0 + for _, event := range events { + if event.ID != "second" { + t.Fatal("event escaped its Turn", event.ID) + } + switch event.Type { + case proto.TypeError: + t.Fatalf("execution failed: %s", event.Payload) + case proto.TypeDelta: + var delta proto.DeltaPayload + _ = json.Unmarshal(event.Payload, &delta) + text += delta.Delta + if delta.Sequence != 1 { + t.Fatal("sequence carried over from old Turn") + } + case proto.TypeToolCall: + tools++ + case proto.TypeDone: + done++ + } + } + if text != "two" || tools != 2 || done != 1 { + t.Fatalf("cross-Turn state: text=%q tools=%d done=%d", text, tools, done) + } + raw, _ := os.ReadFile(record) + if strings.Count(string(raw), "initialize\n") != 1 || strings.Count(string(raw), "session/new\n") != 1 || strings.Count(string(raw), "session/prompt\n") != 2 { + t.Fatalf("native owner was recreated: %s", raw) + } + select { + case <-e.connection.exited: + t.Fatal("completion killed native owner") + default: + } + }) + } +} + +func TestExecutorCancellationWithoutRootProofInvalidatesAndLateCancelCannotRetarget(t *testing.T) { + e, record := executorFixture(t, "executor-cancel", true) + first, _ := runExecutorFixtureTurn(t, e, "first", "one") + out := make(chan proto.Envelope, 32) + second, err := e.StartTurn(t.Context(), "second", proto.TextInput("wait"), out) + if err != nil { + t.Fatal(err) + } + select { + case event := <-out: + if event.Type != proto.TypeDelta { + t.Fatal(event.Type) + } + case <-time.After(3 * time.Second): + t.Fatal("second Turn did not start") + } + if err = first.Cancel(t.Context()); err != nil { + t.Fatal(err) + } + raw, _ := os.ReadFile(record) + if strings.Contains(string(raw), "session/cancel") { + t.Fatal("late cancellation targeted the new Turn") + } + ctx, cancel := context.WithTimeout(t.Context(), 3*time.Second) + defer cancel() + if err = second.Cancel(ctx); err != nil { + t.Fatal("stopped nonreusable owner reported cancellation failure", err) + } + settlement, err := second.AwaitSettlement(ctx) + if err != nil || settlement.Reusable { + t.Fatal(settlement, err) + } + select { + case <-e.connection.exited: + default: + t.Fatal("nonreusable settlement preceded native process exit") + } + if got := second.CancellationOutcome(); got.Content != "ready" || got.Metadata[proto.DoneMetaAgentSessionID] != "native-1" { + t.Fatal("cancel lost settled outcome", got) + } + for range out { + } + thirdOut := make(chan proto.Envelope, 1) + third, err := e.StartTurn(t.Context(), "third", proto.TextInput("three"), thirdOut) + if third != nil || err == nil { + t.Fatal("unproven native owner admitted a successor") + } + close(thirdOut) + raw, _ = os.ReadFile(record) + if strings.Count(string(raw), "session/cancel\n") != 1 || strings.Count(string(raw), "initialize\n") != 1 { + t.Fatalf("cancellation replaced owner: %s", raw) + } +} + +func TestExecutorStartFailureOwnership(t *testing.T) { + t.Run("validation", func(t *testing.T) { + e, record := executorFixture(t, "executor-reuse", true) + out := make(chan proto.Envelope, 1) + turn, err := e.StartTurn(t.Context(), "", proto.TextInput("invalid"), out) + if err == nil || turn != nil { + t.Fatal("invalid Start acquired output") + } + close(out) + raw, _ := os.ReadFile(record) + if strings.Contains(string(raw), "session/prompt") { + t.Fatal("invalid input was sent") + } + runExecutorFixtureTurn(t, e, "healthy", "valid") + }) + t.Run("before input", func(t *testing.T) { + e, record := executorFixture(t, "executor-preexit", true) + out := make(chan proto.Envelope, 1) + turn, err := e.StartTurn(t.Context(), "run", proto.TextInput("input"), out) + if err == nil || turn != nil { + t.Fatal("failed readiness retained caller output") + } + close(out) + raw, _ := os.ReadFile(record) + if strings.Contains(string(raw), "session/prompt") { + t.Fatal("pre-input failure sent input") + } + }) + t.Run("after submission", func(t *testing.T) { + e, record := executorFixture(t, "executor-exit", true) + out := make(chan proto.Envelope, 32) + turn, err := e.StartTurn(t.Context(), "run", proto.TextInput("input"), out) + if err != nil || turn == nil { + t.Fatal("submitted Turn lost ownership", err) + } + for range out { + } + ctx, cancel := context.WithTimeout(t.Context(), 3*time.Second) + defer cancel() + settlement, err := turn.AwaitSettlement(ctx) + if err == nil || settlement.Reusable { + t.Fatal("native exit lost its settlement error", settlement, err) + } + raw, _ := os.ReadFile(record) + if strings.Count(string(raw), "session/prompt\n") != 1 { + t.Fatal("uncertain input was replayed") + } + rejected := make(chan proto.Envelope) + again, err := e.StartTurn(t.Context(), "again", proto.TextInput("again"), rejected) + if again != nil || err == nil { + t.Fatal("invalid owner accepted another Turn") + } + close(rejected) + }) +} + +func TestExecutorCloseRetainsOwnerAfterDeadline(t *testing.T) { + e, _ := executorFixture(t, "executor-reuse", true) + cancelled, cancel := context.WithCancel(t.Context()) + cancel() + _ = e.Close(cancelled) + ctx, stop := context.WithTimeout(t.Context(), 5*time.Second) + defer stop() + if err := e.Close(ctx); err != nil { + t.Fatal(err) + } + select { + case <-e.connection.exited: + default: + t.Fatal("Close lost native cleanup ownership") + } +} diff --git a/apps/parsar-daemon/internal/agent/mcode/executor_turn.go b/apps/parsar-daemon/internal/agent/mcode/executor_turn.go new file mode 100644 index 000000000..6a43459f5 --- /dev/null +++ b/apps/parsar-daemon/internal/agent/mcode/executor_turn.go @@ -0,0 +1,151 @@ +package mcode + +import ( + "context" + "encoding/json" + "errors" + "fmt" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func (s *Session) runExecutorTurn() { + err := s.captureSubagentBaseline() + if err == nil { + err = s.executePrompt() + } + if errors.Is(err, errTurnCancelled) { + err = nil + } + s.mu.Lock() + s.closing, s.steeringReady = true, false + close(s.inputDone) + s.mu.Unlock() + // Written steering requests retain their original receipt owner even after + // prompt completion. No successor starts until all callers have settled. + s.operations.Wait() + if s.req.StrictResume && !s.req.DisableSubagents && s.subagentHistoryReady { + childErr := s.settleSubagents() + s.mu.Lock() + s.subagentSettlementError = childErr + s.mu.Unlock() + if childErr != nil { + err = childErr + } + } + reusable := err == nil && s.process.Context().Err() == nil + if len(s.tools) > 0 { + reusable = false + } + s.finishEnvironmentMCP() + s.mu.Lock() + // ACP cancelled can precede native root Turn settlement in 0.4.12. + // Without the qualified history reader, retain no claim of safe reuse. + if (s.cancelled && s.req.DisableSubagents) || s.inputUncertain || len(s.permissions) != 0 || len(s.questions) != 0 { + reusable = false + } + if s.inputUncertain && err == nil { + err = fmt.Errorf("mcode: native input outcome is unknown") + } + metadata := map[string]any{proto.DoneMetaAgentSessionType: "mcode", proto.DoneMetaAgentSessionID: s.sessionID} + s.outcome = proto.DonePayload{Content: s.content.String(), Metadata: metadata, SourceCompletedAtMS: s.rootCompletedAtMS} + outcome := s.outcome + s.permissions, s.questions = map[string]pendingPermission{}, map[string]pendingQuestion{} + s.mu.Unlock() + if !reusable { + // Unknown quiescence invalidates this owner. A successful settlement + // still proves its native process group and pipes have stopped. + s.process.Cancel() + <-s.exited + } + if err != nil { + s.emit(proto.TypeError, proto.ErrorPayload{Error: err.Error()}) + } + s.emit(proto.TypeDone, outcome) + close(s.out) + close(s.finished) + s.executor.mu.Lock() + s.connection.setCurrent(nil) + if !reusable { + s.executor.invalid = true + } + s.executor.active = nil + s.settlement = agent.TurnSettlement{Reusable: reusable} + s.settlementErr = err + if !reusable { + s.settlement.Reason = "native Turn did not establish reusable settlement" + } + close(s.settled) + s.executor.mu.Unlock() + s.outputCancel() +} + +func (s *Session) captureSubagentBaseline() error { + if !s.req.StrictResume || s.req.DisableSubagents { + return nil + } + snapshot, err := s.readSubagents(s.ctx) + s.subagentHistoryReady = err == nil + s.previousNativeTurns = map[string]bool{} + for _, session := range snapshot.Sessions { + if session.ID == s.sessionID { + for _, turn := range session.Turns { + s.previousNativeTurns[turn.ID] = true + } + } + } + return err +} + +func (s *Session) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + if s.settled == nil { + return agent.TurnSettlement{}, fmt.Errorf("mcode: Turn has no Executor owner") + } + select { + case <-s.settled: + return s.settlement, s.settlementErr + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} + +func (s *Session) cancelTurn(ctx context.Context) error { + e := s.executor + e.mu.Lock() + if e.active != s { + e.mu.Unlock() + _, err := s.AwaitSettlement(ctx) + return err + } + s.mu.Lock() + first := !s.cancelled && !s.closing + s.cancelled = true + if first { + s.operations.Add(1) + } + s.mu.Unlock() + // Cancellation releases event backpressure but does not cancel native owner + // context. ACP prompt completion and child/tool settlement decide reuse. + s.outputCancel() + e.mu.Unlock() + var err error + if first { + raw, _ := json.Marshal(map[string]string{"sessionId": s.sessionID}) + err = s.writeContext(ctx, rpcFrame{JSONRPC: "2.0", Method: "session/cancel", Params: raw}) + } + if first { + if err == nil && s.req.StrictResume && !s.req.DisableSubagents { + err = s.stopSubagents(ctx) + } + if err != nil { + s.markInputUncertain() + } + s.operations.Done() + } + if err != nil { + return err + } + _, err = s.AwaitSettlement(ctx) + return err +} diff --git a/apps/parsar-daemon/internal/agent/mcode/preparation.go b/apps/parsar-daemon/internal/agent/mcode/preparation.go index 97e614d82..8d9672548 100644 --- a/apps/parsar-daemon/internal/agent/mcode/preparation.go +++ b/apps/parsar-daemon/internal/agent/mcode/preparation.go @@ -3,145 +3,85 @@ package mcode import ( "context" "fmt" - "strings" "sync" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" ) +// Prepared retains its single admission API over the same native Executor. +// Runtime's Session lifecycle uses Executor directly. type prepared struct { - mu sync.Mutex - session *Session - ready chan struct{} - started chan struct{} - failure error - closed bool - binding *preparedStart -} - -type preparedStart struct { - runID string - prompt proto.MessageInput - out chan<- proto.Envelope + mu sync.Mutex + executor *executor + session *Session + started, closed bool } func NewPreparationFactory(config WorkspaceConfig) agent.PreparationFactory { + factory := NewExecutorFactory(&config) return func(ctx context.Context, req proto.PromptRequestPayload) (agent.Prepared, error) { - if ctx == nil { - ctx = context.Background() - } - if req.RunID != "" || len(req.Input) != 0 || req.ConversationID != "" { - return nil, fmt.Errorf("mcode: preparation cannot contain input or product context") - } - opts, err := prepareWorkspaceOptions(ctx, config, req) - if err != nil { - return nil, err - } - s, err := launch(ctx, req, opts, config.Binary, nil) + value, err := factory(ctx, req) if err != nil { + if value != nil { + return &prepared{executor: value.(*executor)}, err + } return nil, err } - p := &prepared{session: s, ready: make(chan struct{}), started: make(chan struct{})} - go s.run(p) - <-p.ready - if p.failure != nil { - <-s.finished - return nil, p.failure - } - if ctx.Err() != nil { - _ = p.Close() - return nil, ctx.Err() - } - return p, nil + e := value.(*executor) + return &prepared{executor: e, session: newTurnSession(ctx, req, e.opts, e.connection, nil)}, nil } } -func (p *prepared) Start(ctx context.Context, runID string, prompt proto.MessageInput, out chan<- proto.Envelope) (agent.Session, error) { - if ctx == nil { - ctx = context.Background() - } +func (p *prepared) Start(ctx context.Context, runID string, input proto.MessageInput, out chan<- proto.Envelope) (agent.Session, error) { p.mu.Lock() defer p.mu.Unlock() - if p.closed || p.binding != nil { + if p.closed || p.started { return nil, fmt.Errorf("mcode: preparation is no longer available") } - if strings.TrimSpace(runID) == "" || prompt.Validate() != nil || out == nil || ctx.Err() != nil { - return nil, fmt.Errorf("mcode: start requires a live context, identity, prompt and output") - } - select { - case <-p.session.process.Context().Done(): - return nil, fmt.Errorf("mcode: prepared process ended") - case <-p.session.exited: - return nil, fmt.Errorf("mcode: prepared process exited") - default: + turn, err := p.executor.StartTurn(ctx, runID, input, out) + if turn != nil { + p.started, p.session = true, turn.(*Session) } - p.binding = &preparedStart{runID: runID, prompt: prompt, out: out} - close(p.started) - return p.session, nil + return turn, err } func (p *prepared) Close() error { p.mu.Lock() - if p.binding != nil { - p.mu.Unlock() - return nil + started := p.started + if !started { + p.closed = true } - p.closed = true p.mu.Unlock() - return p.closeUnused(context.Background()) + if started { + return nil + } + return p.executor.Close(context.Background()) } func (p *prepared) Cancel(ctx context.Context) error { p.mu.Lock() p.closed = true - started := p.binding != nil + current, started := p.session, p.started p.mu.Unlock() - if !started { - return p.closeUnused(ctx) - } - return p.session.Cancel(ctx) -} - -// An unconsumed preparation has no model input or child work to settle. -func (p *prepared) closeUnused(ctx context.Context) error { - p.session.process.Cancel() - select { - case <-p.session.finished: - return nil - case <-ctx.Done(): - return ctx.Err() + if started { + if err := current.Cancel(ctx); err != nil { + // The single-use Prepared owns disposal as well as cancellation. + if closeErr := p.executor.Close(ctx); closeErr != nil { + return closeErr + } + return nil + } } + return p.executor.Close(ctx) } -func (p *prepared) CancellationOutcome() proto.DonePayload { return p.session.CancellationOutcome() } - -// The Session goroutine owns preparation, input submission and terminal output. -func (p *prepared) awaitStart(err error) error { - p.failure = err - close(p.ready) - if err != nil { - return err - } - for { - var ended error - select { - case <-p.started: - case <-p.session.process.Context().Done(): - ended = p.session.process.Context().Err() - case _, ok := <-p.session.frames: - if ok { - continue - } - ended = fmt.Errorf("mcode: prepared process exited") - } - p.mu.Lock() - if b := p.binding; b != nil { - p.session.req.RunID, p.session.req.Input, p.session.out = b.runID, b.prompt, b.out - } else { - p.closed = true - } - p.mu.Unlock() - return ended +func (p *prepared) CancellationOutcome() proto.DonePayload { + p.mu.Lock() + s := p.session + p.mu.Unlock() + if s == nil { + return proto.DonePayload{} } + return s.CancellationOutcome() } diff --git a/apps/parsar-daemon/internal/agent/mcode/preparation_test.go b/apps/parsar-daemon/internal/agent/mcode/preparation_test.go index 9ae612c19..65ea1a87e 100644 --- a/apps/parsar-daemon/internal/agent/mcode/preparation_test.go +++ b/apps/parsar-daemon/internal/agent/mcode/preparation_test.go @@ -88,8 +88,8 @@ func TestPreparedWorkspaceHasOneInputAndOutputOwner(t *testing.T) { done++ select { case <-p.session.exited: + t.Fatal("successful Turn disposed the reusable native owner") default: - t.Fatal("Done before native settlement") } var d proto.DonePayload _ = json.Unmarshal(e.Payload, &d) diff --git a/apps/parsar-daemon/internal/agent/mcode/session.go b/apps/parsar-daemon/internal/agent/mcode/session.go index 476ace40b..e65948f08 100644 --- a/apps/parsar-daemon/internal/agent/mcode/session.go +++ b/apps/parsar-daemon/internal/agent/mcode/session.go @@ -1,15 +1,12 @@ package mcode import ( - "bufio" "context" "encoding/json" "errors" "fmt" - "io" "net/url" "slices" - "strconv" "strings" "sync" "time" @@ -20,22 +17,30 @@ import ( ) type Session struct { - ctx context.Context - req proto.PromptRequestPayload - opts launchOptions - process *clirunner.Process + ctx context.Context + req proto.PromptRequestPayload + opts launchOptions + *connection + executor *executor + settlement agent.TurnSettlement + settlementErr error + settled chan struct{} + inputDone chan struct{} + outputCancel context.CancelFunc + operations sync.WaitGroup + closing bool + cancelled bool + inputUncertain bool out chan<- proto.Envelope frames chan rpcFrame - exited chan struct{} finished chan struct{} - writeMu sync.Mutex mu sync.Mutex sessionID string + nativeModel string + outputContext context.Context + outcome proto.DonePayload permissions map[string]pendingPermission questions map[string]pendingQuestion - exitErr error - nextID int - responses map[string]chan rpcFrame steeringReady bool steeringTurn string sequence uint64 @@ -70,7 +75,7 @@ func newSession(ctx context.Context, req proto.PromptRequestPayload, out chan<- if err != nil { return nil, err } - go s.run(nil) + go s.run() return s, nil } @@ -79,56 +84,18 @@ func launch(ctx context.Context, req proto.PromptRequestPayload, opts launchOpti if err != nil { return nil, err } - s := &Session{ctx: ctx, req: req, opts: opts, process: process, out: out, frames: make(chan rpcFrame, 32), exited: make(chan struct{}), finished: make(chan struct{}), responses: map[string]chan rpcFrame{}, permissions: map[string]pendingPermission{}, questions: map[string]pendingQuestion{}, tools: map[string]toolUpdate{}, completedTools: map[string]bool{}} - go func() { _, _ = io.Copy(io.Discard, process.Stderr) }() - go s.read() + c := &connection{process: process, exited: make(chan struct{}), responses: map[string]chan rpcFrame{}} + s := newTurnSession(ctx, req, opts, c, out) + c.current = s + go c.read() return s, nil } -func (s *Session) read() { - defer close(s.exited) - defer close(s.frames) - scanner := bufio.NewScanner(s.process.Stdout) - scanner.Buffer(make([]byte, 64*1024), 16*1024*1024) - var readErr error - for scanner.Scan() { - var frame rpcFrame - if err := json.Unmarshal(scanner.Bytes(), &frame); err != nil { - readErr = fmt.Errorf("mcode: malformed ACP response") - s.process.Cancel() - break - } - if frame.Method == "" { - s.mu.Lock() - response := s.responses[string(frame.ID)] - s.mu.Unlock() - if response != nil { - select { - case response <- frame: - default: - } - continue - } - } - select { - case s.frames <- frame: - case <-s.finished: - case <-s.process.Context().Done(): - } - } - if scanner.Err() != nil { - readErr = fmt.Errorf("mcode: ACP stream read failed") - s.process.Cancel() - } - waitErr := s.process.Wait() - if readErr != nil { - s.exitErr = readErr - } else { - s.exitErr = waitErr - } +func newTurnSession(ctx context.Context, req proto.PromptRequestPayload, opts launchOptions, c *connection, out chan<- proto.Envelope) *Session { + return &Session{ctx: ctx, req: req, opts: opts, connection: c, out: out, frames: make(chan rpcFrame, 32), finished: make(chan struct{}), permissions: map[string]pendingPermission{}, questions: map[string]pendingQuestion{}, tools: map[string]toolUpdate{}, completedTools: map[string]bool{}} } -func (s *Session) run(p *prepared) { +func (s *Session) run() { defer func() { if s.out != nil { close(s.out) @@ -136,9 +103,6 @@ func (s *Session) run(p *prepared) { }() defer close(s.finished) err := s.prepareNative() - if p != nil { - err = p.awaitStart(err) - } if err == nil { if s.req.StrictResume && !s.req.DisableSubagents { var snapshot nativeSubagentSnapshot @@ -166,7 +130,7 @@ func (s *Session) run(p *prepared) { err = observationErr } } - if p != nil || err != nil { + if err != nil { s.process.Cancel() <-s.exited } @@ -241,6 +205,7 @@ func (s *Session) prepareNative() error { if err != nil { return err } + s.nativeModel = model if err := s.call("session/set_config_option", map[string]any{"sessionId": session.SessionID, "configId": "model", "value": model}, nil, false); err != nil { return err } @@ -264,6 +229,12 @@ func (s *Session) executePrompt() error { if err != nil { return err } + s.mu.Lock() + cancelled := s.cancelled + s.mu.Unlock() + if result.StopReason == "cancelled" && cancelled { + return nil + } if result.StopReason != "end_turn" { return fmt.Errorf("mcode: prompt stopped (%s)", result.StopReason) } @@ -304,7 +275,26 @@ func (s *Session) call(method string, params any, result any, prompt bool) error if err != nil { return err } - if err := s.write(rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: method, Params: raw}); err != nil { + if prompt && s.executor != nil { + // Serialize the admission check with the wire, but release the owner + // mutex before a pipe write so cancellation and Close can stop it. + s.connection.writeMu.Lock() + s.executor.mu.Lock() + s.mu.Lock() + cancelled := s.cancelled + s.mu.Unlock() + if cancelled || s.executor.closed { + s.executor.mu.Unlock() + s.connection.writeMu.Unlock() + return errTurnCancelled + } + s.executor.mu.Unlock() + err = json.NewEncoder(s.process.Stdin).Encode(rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: method, Params: raw}) + s.connection.writeMu.Unlock() + } else { + err = s.write(rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: method, Params: raw}) + } + if err != nil { return err } ctx := s.process.Context() @@ -317,6 +307,8 @@ func (s *Session) call(method string, params any, result any, prompt bool) error select { case <-ctx.Done(): return fmt.Errorf("mcode: %s: %w", method, ctx.Err()) + case <-s.exited: + return fmt.Errorf("mcode: ACP process exited: %v", s.exitErr) case frame, ok := <-s.frames: if !ok { <-s.exited @@ -344,24 +336,30 @@ func (s *Session) call(method string, params any, result any, prompt bool) error } } -func (s *Session) write(frame rpcFrame) error { - s.writeMu.Lock() - defer s.writeMu.Unlock() - return json.NewEncoder(s.process.Stdin).Encode(frame) -} - func (s *Session) emit(kind string, payload any) { + ctx := s.ctx + if s.outputContext != nil { + ctx = s.outputContext + } env, err := proto.NewEnvelope(kind, s.req.RunID, payload) if err != nil { return } select { case s.out <- env: - case <-s.ctx.Done(): + return + default: + } + select { + case s.out <- env: + case <-ctx.Done(): } } func (s *Session) Cancel(ctx context.Context) error { + if s.executor != nil { + return s.cancelTurn(ctx) + } if s.req.StrictResume && !s.req.DisableSubagents { if err := s.cancelSubagents(ctx); err != nil { s.process.Cancel() @@ -409,18 +407,3 @@ func (s *Session) SubmitPermission(_ context.Context, id string, decision proto. delete(s.permissions, id) return nil } - -func (s *Session) reserveResponse() (string, chan rpcFrame) { - s.mu.Lock() - defer s.mu.Unlock() - s.nextID++ - id := strconv.Itoa(s.nextID) - reply := make(chan rpcFrame, 1) - s.responses[id] = reply - return id, reply -} -func (s *Session) removeResponse(id string) { - s.mu.Lock() - delete(s.responses, id) - s.mu.Unlock() -} diff --git a/apps/parsar-daemon/internal/agent/mcode/session_test.go b/apps/parsar-daemon/internal/agent/mcode/session_test.go index 2e347057a..8d0fb9537 100644 --- a/apps/parsar-daemon/internal/agent/mcode/session_test.go +++ b/apps/parsar-daemon/internal/agent/mcode/session_test.go @@ -198,6 +198,7 @@ func TestMCodeProcess(t *testing.T) { } scanner := bufio.NewScanner(os.Stdin) var promptID json.RawMessage + prompts, configurations := 0, 0 for scanner.Scan() { var frame rpcFrame if json.Unmarshal(scanner.Bytes(), &frame) != nil { @@ -248,6 +249,15 @@ func TestMCodeProcess(t *testing.T) { result = map[string]any{"configOptions": []map[string]any{{"id": "permissionMode"}}} } case "session/set_config_option": + configurations++ + if scenario == "executor-backpressure" && configurations > 1 { + send(rpcFrame{JSONRPC: "2.0", ID: frame.ID, Result: json.RawMessage("{}")}) + time.Sleep(time.Minute) + os.Exit(0) + } + if scenario == "executor-preexit" && configurations > 1 { + os.Exit(0) + } var params map[string]string _ = json.Unmarshal(frame.Params, ¶ms) if params["configId"] == "model" && params["value"] != "m:custom_provider%3Aoac:fixture:v:" { @@ -258,10 +268,33 @@ func TestMCodeProcess(t *testing.T) { Prompt []map[string]string `json:"prompt"` } _ = json.Unmarshal(frame.Params, &input) - if strict := scenario == "strict-cancel" || strings.HasPrefix(scenario, "prepared"); (strict && len(input.Prompt) != 2) || (!strict && len(input.Prompt) != 1) { + if strict := scenario == "strict-cancel" || (strings.HasPrefix(scenario, "prepared") || strings.HasPrefix(scenario, "executor")); (strict && len(input.Prompt) != 2) || (!strict && len(input.Prompt) != 1) { os.Exit(9) } + if strings.HasPrefix(scenario, "executor") { + prompts++ + promptID = frame.ID + if scenario == "executor-exit" { + os.Exit(0) + } + if input.Prompt[0]["text"] == "wait" { + update("agent_message_chunk", map[string]any{"content": map[string]string{"type": "text", "text": "ready"}}) + continue + } + update("agent_message_chunk", map[string]any{"content": map[string]string{"type": "text", "text": input.Prompt[0]["text"]}}) + update("tool_call", map[string]any{"toolCallId": "repeated-call", "name": "Read", "status": "in_progress", "rawInput": map[string]any{}}) + update("tool_call_update", map[string]any{"toolCallId": "repeated-call", "status": "completed"}) + raw, _ := json.Marshal(map[string]string{"stopReason": "end_turn"}) + send(rpcFrame{JSONRPC: "2.0", ID: frame.ID, Result: raw}) + // These frames precede the next control barrier on the wire and + // must never become the following Turn's output. + for range 100 { + update("agent_message_chunk", map[string]any{"content": map[string]string{"type": "text", "text": "OLD"}}) + } + continue + } if scenario == "prepared-mcp-cancel" { + promptID = frame.ID update("tool_call", map[string]any{"toolCallId": "native-call", "name": "mcp__proof_server__read_status", "status": "in_progress", "rawInput": map[string]any{}}) continue } @@ -294,7 +327,15 @@ func TestMCodeProcess(t *testing.T) { } update("usage_update", map[string]any{"used": 2000, "size": 64000, "cost": map[string]any{"amount": 2, "currency": "USD"}}) result = map[string]string{"stopReason": "end_turn"} + case "session/cancel": + raw, _ := json.Marshal(map[string]string{"stopReason": "cancelled"}) + send(rpcFrame{JSONRPC: "2.0", ID: promptID, Result: raw}) + continue case "mcode/session/steer": + if scenario == "executor-steer-unknown" { + send(rpcFrame{JSONRPC: "2.0", ID: frame.ID, Error: &rpcError{Code: -32000, Message: "Unknown input outcome"}}) + continue + } if scenario == "steer-lost" { os.Exit(0) } diff --git a/apps/parsar-daemon/internal/agent/mcode/steering.go b/apps/parsar-daemon/internal/agent/mcode/steering.go index 7479b2964..cb16925bc 100644 --- a/apps/parsar-daemon/internal/agent/mcode/steering.go +++ b/apps/parsar-daemon/internal/agent/mcode/steering.go @@ -18,6 +18,16 @@ func (s *Session) Steer(ctx context.Context, input proto.PromptSteerPayload) err // Native acceptance belongs to the active ACP Turn; it does not promise model consumption. func (s *Session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerPayload, written func()) error { + if s.executor != nil { + s.mu.Lock() + if s.closing || s.cancelled { + s.mu.Unlock() + return agent.ErrSteeringInactive + } + s.operations.Add(1) + s.mu.Unlock() + defer s.operations.Done() + } text, err := input.Input.TextOnly() if strings.TrimSpace(input.InputID) == "" || err != nil { return agent.ErrSteeringRejected @@ -40,10 +50,9 @@ func (s *Session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP if err := ctx.Err(); err != nil { return err } - stop := context.AfterFunc(ctx, s.process.Cancel) - err = s.write(rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: "mcode/session/steer", Params: raw}) - stop() + err = s.writeContext(ctx, rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: "mcode/session/steer", Params: raw}) if err != nil { + s.markInputUncertain() return fmt.Errorf("mcode: input transport failed") } if written != nil { @@ -53,7 +62,7 @@ func (s *Session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP var stopped error select { case frame = <-response: - case <-s.finished: + case <-s.inputSettlementDone(): stopped = fmt.Errorf("mcode: run ended with unknown input outcome") case <-s.exited: stopped = fmt.Errorf("mcode: input transport closed") @@ -65,6 +74,7 @@ func (s *Session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP select { case frame = <-response: default: + s.markInputUncertain() return stopped } } @@ -72,6 +82,7 @@ func (s *Session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP if frame.Error.Code == -32602 || frame.Error.Code == -32601 { return agent.ErrSteeringRejected } + s.markInputUncertain() return fmt.Errorf("mcode: native steering failed with unknown input outcome") } var result struct { @@ -79,23 +90,41 @@ func (s *Session) SteerWithReceipt(ctx context.Context, input proto.PromptSteerP Mode string `json:"mode"` } if json.Unmarshal(frame.Result, &result) != nil || result.TurnID == "" || (result.Mode != "steered" && result.Mode != "duplicate") { + s.markInputUncertain() return fmt.Errorf("mcode: invalid steering receipt") } s.mu.Lock() defer s.mu.Unlock() if s.steeringTurn != "" && s.steeringTurn != result.TurnID { + s.inputUncertain = true return fmt.Errorf("mcode: steering turn changed") } s.steeringTurn = result.TurnID return nil } +func (s *Session) inputSettlementDone() <-chan struct{} { + if s.inputDone != nil { + return s.inputDone + } + return s.finished +} + func (s *Session) CancellationOutcome() proto.DonePayload { s.mu.Lock() defer s.mu.Unlock() + if s.outcome.Metadata != nil { + return s.outcome + } metadata := map[string]any{proto.DoneMetaAgentSessionType: "mcode"} if s.sessionID != "" { metadata[proto.DoneMetaAgentSessionID] = s.sessionID } return proto.DonePayload{Metadata: metadata} } + +func (s *Session) markInputUncertain() { + s.mu.Lock() + s.inputUncertain = true + s.mu.Unlock() +} diff --git a/apps/parsar-daemon/internal/agent/mcode/subagent_cancel.go b/apps/parsar-daemon/internal/agent/mcode/subagent_cancel.go index 1469c1ff0..bd1b88e68 100644 --- a/apps/parsar-daemon/internal/agent/mcode/subagent_cancel.go +++ b/apps/parsar-daemon/internal/agent/mcode/subagent_cancel.go @@ -7,6 +7,21 @@ import ( ) func (s *Session) cancelSubagents(ctx context.Context) error { + if err := s.stopSubagents(ctx); err != nil { + return err + } + select { + case <-s.finished: + s.mu.Lock() + err := s.subagentSettlementError + s.mu.Unlock() + return err + case <-ctx.Done(): + return ctx.Err() + } +} + +func (s *Session) stopSubagents(ctx context.Context) error { select { case <-s.finished: return nil @@ -21,7 +36,7 @@ func (s *Session) cancelSubagents(ctx context.Context) error { id, response := s.reserveResponse() defer s.removeResponse(id) raw, _ := json.Marshal(map[string]string{"sessionId": native}) - if err := s.write(rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: "mcode/session/delegation/stop", Params: raw}); err != nil { + if err := s.writeContext(ctx, rpcFrame{JSONRPC: "2.0", ID: json.RawMessage(id), Method: "mcode/session/delegation/stop", Params: raw}); err != nil { return err } select { @@ -40,13 +55,5 @@ func (s *Session) cancelSubagents(ctx context.Context) error { case <-ctx.Done(): return ctx.Err() } - select { - case <-s.finished: - s.mu.Lock() - err := s.subagentSettlementError - s.mu.Unlock() - return err - case <-ctx.Done(): - return ctx.Err() - } + return nil } diff --git a/apps/parsar-daemon/internal/agent/mcode/subagents.go b/apps/parsar-daemon/internal/agent/mcode/subagents.go index 33fe2ed9e..645525f54 100644 --- a/apps/parsar-daemon/internal/agent/mcode/subagents.go +++ b/apps/parsar-daemon/internal/agent/mcode/subagents.go @@ -121,11 +121,9 @@ func (s *Session) settleSubagents() error { settled = false } } - if session.ID != snapshot.Root { - for _, turn := range session.Turns { - if turn.Status == "accepted" { - settled = false - } + for _, turn := range session.Turns { + if turn.Status == "accepted" { + settled = false } } } diff --git a/apps/parsar-daemon/internal/agent/mcode/subagents_test.go b/apps/parsar-daemon/internal/agent/mcode/subagents_test.go index de4b4b414..9268c5837 100644 --- a/apps/parsar-daemon/internal/agent/mcode/subagents_test.go +++ b/apps/parsar-daemon/internal/agent/mcode/subagents_test.go @@ -3,8 +3,11 @@ package mcode import ( "context" "encoding/json" + "os" + "path/filepath" "reflect" "testing" + "time" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" ) @@ -65,3 +68,35 @@ func TestSubagentSnapshotRejectsMissingParentProvenance(t *testing.T) { t.Fatal("accepted child without original spawning provenance") } } + +// The ACP cancelled response may precede the root native Turn becoming terminal. +func TestSubagentSettlementIncludesActiveRootTurn(t *testing.T) { + dir := t.TempDir() + node, bridge := filepath.Join(dir, "node"), filepath.Join(dir, "bridge.mjs") + for name, data := range map[string]string{ + node: "#!/bin/sh\ncat \"$3/snapshot.json\"\n", + bridge: "", + filepath.Join(dir, "subagent-snapshot.mjs"): "", + } { + if err := os.WriteFile(name, []byte(data), 0700); err != nil { + t.Fatal(err) + } + } + t.Setenv("OAC_RUNTIME_MCODE_NODE", node) + t.Setenv("OAC_RUNTIME_MCODE_WORKSPACE_BRIDGE", bridge) + for _, status := range []string{"accepted", "aborted"} { + t.Run(status, func(t *testing.T) { + raw := []byte(`{"version":1,"complete":true,"rootSessionId":"root","sessions":[{"id":"root","turns":[{"id":"current","status":"` + status + `"}]}]}`) + if err := os.WriteFile(filepath.Join(dir, "snapshot.json"), raw, 0600); err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(t.Context(), 100*time.Millisecond) + defer cancel() + s := &Session{ctx: ctx, sessionID: "root", opts: launchOptions{DataDir: dir}, frames: make(chan rpcFrame)} + err := s.settleSubagents() + if (err != nil) != (status == "accepted") { + t.Fatalf("root %s settlement error = %v", status, err) + } + }) + } +} diff --git a/apps/parsar-daemon/internal/agent/mcode/workspace.go b/apps/parsar-daemon/internal/agent/mcode/workspace.go index 88864e976..79ed46902 100644 --- a/apps/parsar-daemon/internal/agent/mcode/workspace.go +++ b/apps/parsar-daemon/internal/agent/mcode/workspace.go @@ -103,7 +103,7 @@ func prepareWorkspaceOptions(ctx context.Context, c WorkspaceConfig, req proto.P if err = os.WriteFile(filepath.Join(opts.DataDir, "config.yaml"), raw, 0600); err != nil { return opts, err } - profile := map[string]any{"workspace": "/workspace", "scratch": c.Scratch, "network": c.Network, "allowedDomains": (agentnetwork.Policy{Access: c.Network, AllowedDomains: c.AllowedDomains}).Hosts(), "protectedDirs": slices.Clone(c.ProtectedDirs), "skills": len(req.LocalEnvironment.Skills) > 0} + profile := map[string]any{"capabilityRoot": req.LocalEnvironment.CapabilityRoot, "workspace": "/workspace", "scratch": c.Scratch, "network": c.Network, "allowedDomains": (agentnetwork.Policy{Access: c.Network, AllowedDomains: c.AllowedDomains}).Hosts(), "protectedDirs": slices.Clone(c.ProtectedDirs), "skills": len(req.LocalEnvironment.Skills) > 0} if req.LocalEnvironment.ToolEnvironment { if err := localworkspace.VerifyToolEnvironment(req.LocalEnvironment.SystemPackages); err != nil { return opts, err diff --git a/apps/parsar-daemon/internal/agent/registry.go b/apps/parsar-daemon/internal/agent/registry.go index d9f6459f5..97d79a4c9 100644 --- a/apps/parsar-daemon/internal/agent/registry.go +++ b/apps/parsar-daemon/internal/agent/registry.go @@ -58,6 +58,7 @@ type Registry struct { mu sync.RWMutex factories map[string]Factory preparers map[string]PreparationFactory + executors map[string]ExecutorFactory kinds map[string]proto.SupportedAgentKind } @@ -65,6 +66,7 @@ func NewRegistry() *Registry { return &Registry{ factories: make(map[string]Factory), preparers: make(map[string]PreparationFactory), + executors: make(map[string]ExecutorFactory), kinds: make(map[string]proto.SupportedAgentKind), } } @@ -90,6 +92,7 @@ func (r *Registry) RegisterKind(info proto.SupportedAgentKind, f Factory) { defer r.mu.Unlock() r.factories[kind] = f delete(r.preparers, kind) + delete(r.executors, kind) info.Capabilities.Preparation = false info.Capabilities.WorkspaceReadPreparation = false r.kinds[kind] = info diff --git a/apps/parsar-daemon/internal/agent/registry_test.go b/apps/parsar-daemon/internal/agent/registry_test.go index 58c4488a4..e1ab3e8f7 100644 --- a/apps/parsar-daemon/internal/agent/registry_test.go +++ b/apps/parsar-daemon/internal/agent/registry_test.go @@ -136,3 +136,24 @@ func TestRegistrySupportedAgentKindsReportsDescriptors(t *testing.T) { t.Fatalf("opencode descriptor not preserved: %#v", got[1]) } } + +func TestRegistryExecutorRequiresExplicitRegistration(t *testing.T) { + registry := agent.NewRegistry() + registry.Register("native", stubFactory("native")) + if _, err := registry.ResolveExecutor("native"); err == nil { + t.Fatal("legacy factory implied reusable execution") + } + expected := errors.New("executor factory") + registry.RegisterExecutor("native", func(context.Context, proto.PromptRequestPayload) (agent.Executor, error) { return nil, expected }) + factory, err := registry.ResolveExecutor("native") + if err != nil { + t.Fatal(err) + } + if _, err := factory(t.Context(), proto.PromptRequestPayload{}); !errors.Is(err, expected) { + t.Fatal(err) + } + registry.Register("native", stubFactory("replacement")) + if _, err := registry.ResolveExecutor("native"); err == nil { + t.Fatal("replacing a kind retained its old executor capability") + } +} diff --git a/apps/parsar-daemon/internal/cli/agent_registration.go b/apps/parsar-daemon/internal/cli/agent_registration.go index 17126ff5a..ea93bbf30 100644 --- a/apps/parsar-daemon/internal/cli/agent_registration.go +++ b/apps/parsar-daemon/internal/cli/agent_registration.go @@ -16,6 +16,9 @@ func registerAgentKinds(registry *agent.Registry, agentCLIs agentCLIDiscovery, s registerProductAgentKind(registry, agentCLIs.ClaudeCode, withSkillUploadServer(withCapabilityDownloads(claudecode.Factory, serverURL), serverURL)) registerProductAgentKind(registry, agentCLIs.OpenCode, withSkillUploadServer(withCapabilityDownloads(opencodeagent.Factory, serverURL), serverURL)) registerProductAgentKind(registry, agentCLIs.Codex, withSkillUploadServer(withCapabilityDownloads(codex.Factory, serverURL), serverURL)) + if agentCLIs.Codex.Available { + registry.RegisterExecutor("codex", withExecutorCapabilities(codex.NewExecutorFactory(), serverURL)) + } if agentCLIs.Codex.Available && agentCLIs.Codex.Capabilities.LocalEnvironment { registry.RegisterPreparation("codex", true, func(ctx context.Context, req proto.PromptRequestPayload) (agent.Prepared, error) { prepared, err := codex.Prepare(ctx, req) @@ -32,6 +35,9 @@ func registerAgentKinds(registry *agent.Registry, agentCLIs agentCLIDiscovery, s } else { registerProductAgentKind(registry, agentCLIs.MCode, withSkillUploadServer(withCapabilityDownloads(mcode.Factory, serverURL), serverURL)) } + if agentCLIs.MCode.Available { + registry.RegisterExecutor("mcode", withExecutorCapabilities(mcode.NewExecutorFactory(agentCLIs.MCodeWorkspace), serverURL)) + } registerClaudeSDK(registry, agentCLIs.ClaudeSDK) } diff --git a/apps/parsar-daemon/internal/cli/authoring.go b/apps/parsar-daemon/internal/cli/authoring.go index b6b456906..e8fb938ce 100644 --- a/apps/parsar-daemon/internal/cli/authoring.go +++ b/apps/parsar-daemon/internal/cli/authoring.go @@ -62,6 +62,9 @@ func authoringRegistry(registry *agent.Registry, bridge *authoring.Bridge) *agen factory = withAuthoringBridge(factory, bridge) } wrapped.RegisterKind(info, factory) + if executor, err := registry.ResolveExecutor(info.Kind); err == nil { + wrapped.RegisterExecutor(info.Kind, executor) + } if prepare, err := registry.ResolvePreparation(info.Kind); err == nil { wrapped.RegisterPreparation(info.Kind, info.Capabilities.WorkspaceReadPreparation, prepare) } diff --git a/apps/parsar-daemon/internal/cli/capability_downloads.go b/apps/parsar-daemon/internal/cli/capability_downloads.go index 2f5e95819..e92fdfc0c 100644 --- a/apps/parsar-daemon/internal/cli/capability_downloads.go +++ b/apps/parsar-daemon/internal/cli/capability_downloads.go @@ -20,28 +20,7 @@ func withCapabilityDownloads(factory agent.Factory, serverURL string) agent.Fact return factory } return func(ctx context.Context, req proto.PromptRequestPayload, out chan<- proto.Envelope) (agent.Session, error) { - opts := maps.Clone(req.AgentOptions) - for _, key := range []string{"skills", "plugins"} { - items, ok := opts[key].([]any) - if !ok { - continue - } - items = slices.Clone(items) - for i, item := range items { - descriptor, ok := item.(map[string]any) - if !ok { - continue - } - raw, _ := descriptor["download_url"].(string) - if resolved := capabilityDownloadURL(raw, base); resolved != raw { - updated := maps.Clone(descriptor) - updated["download_url"] = resolved - items[i] = updated - } - } - opts[key] = items - } - req.AgentOptions = opts + req = capabilityDownloadRequest(req, base) return factory(ctx, req, out) } } @@ -65,3 +44,42 @@ func capabilityDownloadURL(raw string, base *url.URL) string { u.RawPath = "" return u.String() } + +func capabilityDownloadRequest(req proto.PromptRequestPayload, base *url.URL) proto.PromptRequestPayload { + opts := maps.Clone(req.AgentOptions) + for _, key := range []string{"skills", "plugins"} { + items, ok := opts[key].([]any) + if !ok { + continue + } + items = slices.Clone(items) + for i, item := range items { + descriptor, ok := item.(map[string]any) + if !ok { + continue + } + raw, _ := descriptor["download_url"].(string) + if resolved := capabilityDownloadURL(raw, base); resolved != raw { + updated := maps.Clone(descriptor) + updated["download_url"] = resolved + items[i] = updated + } + } + opts[key] = items + } + req.AgentOptions = opts + return req +} + +// Executor setup applies the same immutable capability snapshot as ordinary execution. +func withExecutorCapabilities(factory agent.ExecutorFactory, serverURL string) agent.ExecutorFactory { + base, err := url.Parse(serverURL) + valid := err == nil && base.Host != "" && (base.Scheme == "http" || base.Scheme == "https") + return func(ctx context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + if valid { + req = capabilityDownloadRequest(req, base) + } + req = skillUploadRequest(req, serverURL) + return factory(ctx, req) + } +} diff --git a/apps/parsar-daemon/internal/cli/capability_downloads_test.go b/apps/parsar-daemon/internal/cli/capability_downloads_test.go index 5dcaf97e8..fb63b1775 100644 --- a/apps/parsar-daemon/internal/cli/capability_downloads_test.go +++ b/apps/parsar-daemon/internal/cli/capability_downloads_test.go @@ -92,3 +92,24 @@ func TestCapabilityDownloadsInstallZIPThroughPairedServer(t *testing.T) { t.Fatal("rewrote the original request options") } } + +func TestExecutorPreparationUsesCapabilitySnapshotTransform(t *testing.T) { + originalURL := "http://localhost:8080/internal/blobs/pg:skill" + descriptor := map[string]any{"download_url": originalURL} + source := proto.PromptRequestPayload{AgentOptions: map[string]any{"skills": []any{descriptor}}} + calls := 0 + factory := withExecutorCapabilities(func(_ context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + calls++ + actual := req.AgentOptions["skills"].([]any)[0].(map[string]any)["download_url"] + if actual != "http://paired-core:8080/internal/blobs/pg:skill" { + t.Fatalf("executor capability URL = %v", actual) + } + return nil, nil + }, "http://paired-core:8080") + if _, err := factory(t.Context(), source); err != nil { + t.Fatal(err) + } + if calls != 1 || descriptor["download_url"] != originalURL { + t.Fatal("preparation changed the source snapshot or repeated setup") + } +} diff --git a/apps/parsar-daemon/internal/cli/claude_sdk.go b/apps/parsar-daemon/internal/cli/claude_sdk.go index add58c1b1..746288935 100644 --- a/apps/parsar-daemon/internal/cli/claude_sdk.go +++ b/apps/parsar-daemon/internal/cli/claude_sdk.go @@ -128,6 +128,9 @@ func registerClaudeSDK(registry *agent.Registry, discovery *claudeSDKDiscovery) } } registry.RegisterKind(discovery.Info, factory) + if discovery.Info.Available { + registry.RegisterExecutor("claude_sdk", claudesdk.NewExecutorFactory(discovery.Config)) + } if discovery.Info.Available && discovery.Info.Capabilities.LocalEnvironment { registry.RegisterPreparation("claude_sdk", true, claudesdk.NewPreparationFactory(discovery.Config)) } diff --git a/apps/parsar-daemon/internal/cli/claude_sdk_test.go b/apps/parsar-daemon/internal/cli/claude_sdk_test.go index c691d9292..ce8eaa35f 100644 --- a/apps/parsar-daemon/internal/cli/claude_sdk_test.go +++ b/apps/parsar-daemon/internal/cli/claude_sdk_test.go @@ -78,6 +78,9 @@ func TestClaudeSDKDiscoveryAndRegistration(t *testing.T) { t.Fatal(err) } info := discovery.ClaudeSDK.Info + if info.Available { + info.Capabilities.Preparation = true + } if info.Available != tc.ready { t.Fatal(info) } diff --git a/apps/parsar-daemon/internal/cli/connect.go b/apps/parsar-daemon/internal/cli/connect.go index 345ec2b48..61f0ebe4a 100644 --- a/apps/parsar-daemon/internal/cli/connect.go +++ b/apps/parsar-daemon/internal/cli/connect.go @@ -387,8 +387,8 @@ func mainLoopRemote(rc *runContext, profile string, prof auth.Profile, agentCLIs // pumpConn runs the per-connection workload: a dispatch.Router fed by // conn.Recv(), heartbeats every boot.HeartbeatInterval(), and a -// graceful router.Shutdown on exit so any in-flight subprocesses get -// SIGTERM. +// confirmed router.Shutdown before returning ownership to the reconnect loop. +// Failed cleanup keeps this exact Router alive, including after a shutdown signal. func pumpConn(parentCtx context.Context, conn *transport.Conn, registry *agent.Registry, boot *transport.BootstrapResponse, agentCLIs agentCLIDiscovery) error { local, err := localworkspace.Load() if err != nil { @@ -406,9 +406,8 @@ func pumpConn(parentCtx context.Context, conn *transport.Conn, registry *agent.R return fmt.Errorf("router init: %w", err) } defer func() { - shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second) - _ = router.Shutdown(shutdownCtx) - cancel() + _ = conn.Close() + shutdownRouterUntilConfirmed(router.Shutdown, time.Second) }() conn.StartHeartbeats(parentCtx, boot.HeartbeatInterval(), func() proto.HeartbeatPayload { diff --git a/apps/parsar-daemon/internal/cli/connect_cleanup.go b/apps/parsar-daemon/internal/cli/connect_cleanup.go new file mode 100644 index 000000000..7909d7347 --- /dev/null +++ b/apps/parsar-daemon/internal/cli/connect_cleanup.go @@ -0,0 +1,24 @@ +package cli + +import ( + "context" + "time" + + obslog "github.com/MiniMax-AI-Dev/parsar/internal/obs/log" +) + +// A wait deadline does not revoke native ownership. Keep retrying the same +// Router until it confirms cleanup; reconnect and process exit both wait here. +// Router.Shutdown serializes attempts and retains resources after a failure. +func shutdownRouterUntilConfirmed(shutdown func(context.Context) error, retryDelay time.Duration) { + for { + ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) + err := shutdown(ctx) + cancel() + if err == nil { + return + } + obslog.Bg().Warn("router cleanup unconfirmed; reconnect remains blocked", "err", err) + time.Sleep(retryDelay) + } +} diff --git a/apps/parsar-daemon/internal/cli/connect_cleanup_test.go b/apps/parsar-daemon/internal/cli/connect_cleanup_test.go new file mode 100644 index 000000000..e5da165fd --- /dev/null +++ b/apps/parsar-daemon/internal/cli/connect_cleanup_test.go @@ -0,0 +1,196 @@ +package cli + +import ( + "context" + "errors" + "net/http" + "net/http/httptest" + "os" + "strings" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/transport" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/gorilla/websocket" + "sync/atomic" + "testing" + "time" +) + +func TestReconnectWaitsForConfirmedRouterCleanup(t *testing.T) { + for _, failure := range []error{errors.New("native close failed"), context.DeadlineExceeded} { + t.Run(failure.Error(), func(t *testing.T) { + var calls atomic.Int32 + retry := make(chan struct{}) + confirm := make(chan struct{}) + returned := make(chan struct{}) + go func() { + shutdownRouterUntilConfirmed(func(ctx context.Context) error { + if ctx.Err() != nil { + t.Error("cleanup inherited a cancelled connection context") + } + if _, ok := ctx.Deadline(); !ok { + t.Error("cleanup wait has no deadline") + } + if calls.Add(1) == 1 { + return failure + } + close(retry) + <-confirm + return nil + }, time.Millisecond) + close(returned) + }() + select { + case <-retry: + case <-time.After(time.Second): + t.Fatal("same cleanup was not retried") + } + select { + case <-returned: + t.Fatal("reconnect released unconfirmed ownership") + default: + } + close(confirm) + select { + case <-returned: + case <-time.After(time.Second): + t.Fatal("confirmed cleanup did not release reconnect") + } + if calls.Load() != 2 { + t.Fatalf("cleanup attempts=%d", calls.Load()) + } + }) + } +} + +type cleanupExecutor struct { + closes atomic.Int32 + retry chan struct{} + confirm chan struct{} +} + +func (e *cleanupExecutor) StartTurn(context.Context, string, proto.MessageInput, chan<- proto.Envelope) (agent.Turn, error) { + return nil, errors.New("unexpected Turn") +} +func (e *cleanupExecutor) Close(context.Context) error { + if e.closes.Add(1) == 1 { + return errors.New("native cleanup temporarily unavailable") + } + close(e.retry) + <-e.confirm + return nil +} + +func TestDisconnectedPumpRetainsExactExecutorUntilCleanup(t *testing.T) { + for _, suspend := range []bool{false, true} { + name := "ordinary" + if suspend { + name = "suspension-enabled" + } + t.Run(name, func(t *testing.T) { testDisconnectedPumpCleanup(t, suspend) }) + } +} + +func testDisconnectedPumpCleanup(t *testing.T, suspend bool) { + t.Setenv("OAC_RUNTIME_WORKSPACE", "") + peers := make(chan *websocket.Conn, 1) + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + peer, err := (&websocket.Upgrader{}).Upgrade(w, r, nil) + if err == nil { + peers <- peer + } + })) + defer server.Close() + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + dial := func(ctx context.Context) (*transport.Conn, error) { + return transport.Dial(ctx, transport.DialOptions{ + WSURL: "ws" + strings.TrimPrefix(server.URL, "http"), DeviceID: "device", + Credential: "credential", DaemonVersion: proto.Version, + }) + } + owner := &cleanupExecutor{retry: make(chan struct{}), confirm: make(chan struct{})} + registry := agent.NewRegistry() + registry.RegisterKind(proto.SupportedAgentKind{Kind: "cleanup", Available: true, Capabilities: proto.AgentKindCapabilities{EnvironmentNone: true}}, + func(context.Context, proto.PromptRequestPayload, chan<- proto.Envelope) (agent.Session, error) { + return nil, errors.New("unexpected legacy factory") + }) + var factories atomic.Int32 + registry.RegisterExecutor("cleanup", func(context.Context, proto.PromptRequestPayload) (agent.Executor, error) { + factories.Add(1) + return owner, nil + }) + finished := make(chan error, 1) + go func() { + boot := &transport.BootstrapResponse{HeartbeatSeconds: 60} + if suspend { + control := &suspendControl{signal: make(chan os.Signal, 1)} + finished <- runSuspendLoop(ctx, dial, registry, boot, agentCLIDiscovery{}, control) + return + } + conn, err := dial(ctx) + if err != nil { + finished <- err + return + } + defer conn.Close() + finished <- pumpConn(ctx, conn, registry, boot, agentCLIDiscovery{}) + }() + var peer *websocket.Conn + select { + case peer = <-peers: + case <-time.After(3 * time.Second): + t.Fatal("initial connection missing") + } + defer peer.Close() + env, err := proto.NewEnvelope(proto.TypeExecutionPrepare, "prepare", proto.ExecutionPreparePayload{SessionID: "cleanup", + Configuration: proto.PromptRequestPayload{AgentKind: "cleanup", AgentStateKey: "agents-api-cleanup", StrictResume: true, DisableExecutionEnvironment: true}}) + if err != nil { + t.Fatal(err) + } + if err := peer.WriteJSON(env); err != nil { + t.Fatal(err) + } + _ = peer.SetReadDeadline(time.Now().Add(3 * time.Second)) + for { + var result proto.Envelope + if err := peer.ReadJSON(&result); err != nil { + t.Fatal(err) + } + if result.Type != proto.TypePreparationStatus { + continue + } + var status proto.PreparationStatusPayload + if err := result.DecodePayload(&status); err != nil { + t.Fatal(err) + } + if status.State == "ready" { + break + } + if status.State == "failed" { + t.Fatalf("preparation failed: %+v", status) + } + } + _ = peer.Close() + select { + case <-owner.retry: + case <-time.After(3 * time.Second): + t.Fatal("original executor cleanup was not retried") + } + select { + case err := <-finished: + t.Fatalf("pump discarded native ownership: %v", err) + default: + } + cancel() + close(owner.confirm) + select { + case <-finished: + case <-time.After(3 * time.Second): + t.Fatal("pump did not return after confirmed cleanup") + } + if factories.Load() != 1 || owner.closes.Load() != 2 { + t.Fatalf("factory=%d close=%d", factories.Load(), owner.closes.Load()) + } +} diff --git a/apps/parsar-daemon/internal/cli/connect_environment.go b/apps/parsar-daemon/internal/cli/connect_environment.go index e02c1c551..bfdb5a263 100644 --- a/apps/parsar-daemon/internal/cli/connect_environment.go +++ b/apps/parsar-daemon/internal/cli/connect_environment.go @@ -18,6 +18,7 @@ import ( "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/auth" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/daemonize" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/transport" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" "github.com/google/uuid" ) @@ -113,7 +114,7 @@ func enrollEnvironment(ctx context.Context, client *http.Client, base, environme return out, fmt.Errorf("connect: Environment enrollment rejected (HTTP %d)", resp.StatusCode) } raw, err := io.ReadAll(io.LimitReader(resp.Body, 16*1024+1)) - if err != nil || len(raw) > 16*1024 || decodeEnvironmentJSON(raw, &out) != nil || !environmentUUID(out.DeviceID) || !environmentUUID(out.SessionID) || out.EnvironmentID != environment || out.WorkspaceDirectory != "/workspace" { + if err != nil || len(raw) > 16*1024 || decodeEnvironmentJSON(raw, &out) != nil || !environmentUUID(out.DeviceID) || !environmentUUID(out.SessionID) || out.EnvironmentID != environment || out.WorkspaceDirectory == "/" || agentcapabilities.ValidateLocalDirectories([]string{out.WorkspaceDirectory}) != nil { return environmentEnrollment{}, errors.New("connect: invalid Environment enrollment response") } return out, nil diff --git a/apps/parsar-daemon/internal/cli/connect_environment_binding.go b/apps/parsar-daemon/internal/cli/connect_environment_binding.go index df3d31833..3eb887058 100644 --- a/apps/parsar-daemon/internal/cli/connect_environment_binding.go +++ b/apps/parsar-daemon/internal/cli/connect_environment_binding.go @@ -11,13 +11,15 @@ import ( "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/localworkspace" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/paths" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" ) // This receipt contains identity only; executor credentials remain in their file. type environmentBinding struct { - RemoteURL string `json:"remote_url"` - Enrollment environmentEnrollment `json:"enrollment"` - LocalWorkspace string `json:"local_workspace"` + RemoteURL string `json:"remote_url"` + Enrollment environmentEnrollment `json:"enrollment"` + LocalWorkspace string `json:"local_workspace"` + CapabilityDirectory string `json:"capability_directory"` } // Check persisted ownership before transmitting the executor credential. @@ -70,8 +72,25 @@ func bindEnvironmentRuntime(remote string, bound environmentEnrollment, credenti return errors.New("connect: Runtime state directory unavailable") } workspace := os.Getenv("OAC_RUNTIME_WORKSPACE") - if workspace != "/environment/workspace" { - return errors.New("connect: packaged /workspace Runtime required") + if workspace == "/" || agentcapabilities.ValidateLocalDirectories([]string{workspace}) != nil { + return errors.New("connect: clean absolute Runtime workspace required") + } + canonical, err := filepath.EvalSymlinks(workspace) + info, statErr := os.Stat(workspace) + if err != nil || canonical != workspace || statErr != nil || !info.IsDir() { + return errors.New("connect: existing canonical Runtime workspace directory required") + } + if bound.WorkspaceDirectory != "/workspace" && bound.WorkspaceDirectory != workspace { + return errors.New("connect: enrollment does not match the Runtime workspace") + } + initialization := localworkspace.InitializationDirectory + if resolved, resolveErr := filepath.EvalSymlinks(initialization); resolveErr == nil { + initialization = resolved + } else if !errors.Is(resolveErr, os.ErrNotExist) { + return errors.New("connect: protected initialization directory unavailable") + } + if environmentPathsOverlap(workspace, initialization) { + return errors.New("connect: Runtime workspace overlaps protected initialization") } for key, value := range map[string]string{ "OAC_RUNTIME_ENVIRONMENT_ID": bound.EnvironmentID, @@ -95,8 +114,8 @@ func bindEnvironmentRuntime(remote string, bound environmentEnrollment, credenti return errors.New("connect: private Runtime path unavailable") } for _, public := range []string{"/workspace", workspace} { - if relative, e := filepath.Rel(public, resolved); e == nil && (relative == "." || filepath.IsLocal(relative)) { - return errors.New("connect: private Runtime state cannot be inside the workspace") + if environmentPathsOverlap(public, resolved) { + return errors.New("connect: private Runtime state cannot overlap the workspace") } } } @@ -109,7 +128,14 @@ func bindEnvironmentRuntime(remote string, bound environmentEnrollment, credenti if err != nil || !filepath.IsLocal(relative) || relative == "." { return errors.New("connect: executor credential must be inside the protected daemon directory") } - want := environmentBinding{RemoteURL: remote, Enrollment: bound, LocalWorkspace: workspace} + capabilityDirectory := os.Getenv("OAC_RUNTIME_CAPABILITY_DIRECTORY") + if capabilityDirectory == "" { + capabilityDirectory = localworkspace.CapabilityDirectory + } + if _, err := localworkspace.NewWithCapabilityDirectory(bound.EnvironmentID, bound.SessionID, workspace, os.Getenv("OAC_RUNTIME_DIRECTORY_HELPER"), capabilityDirectory); err != nil { + return errors.New("connect: local Runtime layout unavailable") + } + want := environmentBinding{RemoteURL: remote, Enrollment: bound, LocalWorkspace: workspace, CapabilityDirectory: capabilityDirectory} if err = saveEnvironmentBinding(root, want); err != nil { return err } @@ -125,6 +151,17 @@ func bindEnvironmentRuntime(remote string, bound environmentEnrollment, credenti return nil } +// Neither selecting private state nor selecting its parent grants workspace access. +func environmentPathsOverlap(first, second string) bool { + for _, pair := range [][2]string{{first, second}, {second, first}} { + relative, err := filepath.Rel(pair[0], pair[1]) + if err == nil && (relative == "." || filepath.IsLocal(relative)) { + return true + } + } + return false +} + func saveEnvironmentBinding(root string, want environmentBinding) error { // All three native sandboxes protect this existing daemon state directory. dir := filepath.Join(root, "daemon") diff --git a/apps/parsar-daemon/internal/cli/connect_environment_binding_test.go b/apps/parsar-daemon/internal/cli/connect_environment_binding_test.go new file mode 100644 index 000000000..b9223181e --- /dev/null +++ b/apps/parsar-daemon/internal/cli/connect_environment_binding_test.go @@ -0,0 +1,136 @@ +package cli + +import ( + "os" + "path/filepath" + "testing" + + "github.com/google/uuid" +) + +func environmentRuntimeFixture(t *testing.T) (root, workspace, credential string, bound environmentEnrollment) { + t.Helper() + base := t.TempDir() + root, workspace = filepath.Join(base, "private"), filepath.Join(base, "workspace") + for _, dir := range []string{filepath.Join(root, "daemon"), workspace} { + if err := os.MkdirAll(dir, 0700); err != nil { + t.Fatal(err) + } + } + credential = filepath.Join(root, "daemon", "executor.json") + if err := os.WriteFile(credential, []byte("private-test-credential"), 0600); err != nil { + t.Fatal(err) + } + helper := filepath.Join(base, "directory-helper") + if err := os.WriteFile(helper, []byte("#!/bin/sh\nexit 0\n"), 0700); err != nil { + t.Fatal(err) + } + t.Setenv("OAC_RUNTIME_HOME", root) + t.Setenv("OAC_RUNTIME_WORKSPACE", workspace) + t.Setenv("OAC_RUNTIME_DIRECTORY_HELPER", helper) + for _, name := range []string{"OAC_RUNTIME_ENVIRONMENT_ID", "OAC_RUNTIME_SESSION_ID", "OAC_RUNTIME_NETWORK_ACCESS", "OAC_RUNTIME_ALLOWED_DOMAINS", "OAC_RUNTIME_WRITE_HELPER", "OAC_RUNTIME_EXPORT_HELPER", "OAC_RUNTIME_STAGING"} { + t.Setenv(name, "") + } + return root, workspace, credential, environmentEnrollment{uuid.NewString(), uuid.NewString(), uuid.NewString(), "/workspace"} +} + +func TestBindEnvironmentRuntimeUsesCanonicalOperatorWorkspace(t *testing.T) { + for _, physical := range []bool{false, true} { + t.Run(map[bool]string{false: "logical", true: "physical"}[physical], func(t *testing.T) { + root, workspace, credential, bound := environmentRuntimeFixture(t) + if physical { + bound.WorkspaceDirectory = workspace + } + if err := bindEnvironmentRuntime("wss://core/api/v1/agent-daemon/ws", bound, credential); err != nil { + t.Fatal(err) + } + raw, err := readEnvironmentPrivateFile(filepath.Join(root, "daemon", "environment.json")) + if err != nil { + t.Fatal(err) + } + var receipt environmentBinding + if decodeEnvironmentJSON(raw, &receipt) != nil || receipt.LocalWorkspace != workspace || receipt.Enrollment != bound { + t.Fatal("workspace identity changed", receipt) + } + if err := bindEnvironmentRuntime(receipt.RemoteURL, bound, credential); err != nil { + t.Fatal("same binding refused", err) + } + replacement := filepath.Join(filepath.Dir(workspace), "replacement") + if err := os.Mkdir(replacement, 0700); err != nil { + t.Fatal(err) + } + t.Setenv("OAC_RUNTIME_WORKSPACE", replacement) + if err := bindEnvironmentRuntime(receipt.RemoteURL, bound, credential); err == nil { + t.Fatal("existing workspace identity replaced") + } + after, _ := os.ReadFile(filepath.Join(root, "daemon", "environment.json")) + if string(after) != string(raw) { + t.Fatal("failed rebind changed identity") + } + contents, _ := os.ReadFile(credential) + if string(contents) != "private-test-credential" { + t.Fatal("credential changed") + } + }) + } +} + +func TestBindEnvironmentRuntimeRejectsUnsafeOrMismatchedWorkspace(t *testing.T) { + for _, kind := range []string{"relative", "unclean", "missing", "file", "symlink", "parent symlink", "workspace contains private", "workspace inside private", "foreign enrollment", "credential outside private", "identity conflict"} { + t.Run(kind, func(t *testing.T) { + root, workspace, credential, bound := environmentRuntimeFixture(t) + selected := workspace + switch kind { + case "relative": + selected = "workspace" + case "unclean": + selected = workspace + "/../workspace" + case "missing": + selected = filepath.Join(filepath.Dir(workspace), "missing") + case "file": + selected = filepath.Join(filepath.Dir(workspace), "regular") + if err := os.WriteFile(selected, nil, 0600); err != nil { + t.Fatal(err) + } + case "symlink": + selected = filepath.Join(filepath.Dir(workspace), "linked") + if err := os.Symlink(workspace, selected); err != nil { + t.Fatal(err) + } + case "parent symlink": + child := filepath.Join(workspace, "child") + if err := os.Mkdir(child, 0700); err != nil { + t.Fatal(err) + } + alias := filepath.Join(filepath.Dir(workspace), "linked-parent") + if err := os.Symlink(workspace, alias); err != nil { + t.Fatal(err) + } + selected = filepath.Join(alias, "child") + case "workspace contains private": + selected = filepath.Dir(root) + case "workspace inside private": + selected = filepath.Join(root, "nested-workspace") + if err := os.Mkdir(selected, 0700); err != nil { + t.Fatal(err) + } + case "foreign enrollment": + bound.WorkspaceDirectory = filepath.Join(filepath.Dir(workspace), "different") + case "credential outside private": + credential = filepath.Join(filepath.Dir(root), "credential") + if err := os.WriteFile(credential, []byte("private-test-credential"), 0600); err != nil { + t.Fatal(err) + } + case "identity conflict": + t.Setenv("OAC_RUNTIME_ENVIRONMENT_ID", uuid.NewString()) + } + t.Setenv("OAC_RUNTIME_WORKSPACE", selected) + if err := bindEnvironmentRuntime("wss://core/api/v1/agent-daemon/ws", bound, credential); err == nil { + t.Fatal("unsafe binding accepted") + } + if _, err := os.Lstat(filepath.Join(root, "daemon", "environment.json")); !os.IsNotExist(err) { + t.Fatal("invalid binding published identity", err) + } + }) + } +} diff --git a/apps/parsar-daemon/internal/cli/connect_environment_test.go b/apps/parsar-daemon/internal/cli/connect_environment_test.go index dd03b66dd..7cefe7261 100644 --- a/apps/parsar-daemon/internal/cli/connect_environment_test.go +++ b/apps/parsar-daemon/internal/cli/connect_environment_test.go @@ -154,7 +154,7 @@ func TestEnvironmentBindingPreservesIdentityAndHistory(t *testing.T) { if err := os.Chmod(root, 0700); err != nil { t.Fatal(err) } - want := environmentBinding{"wss://core/api/v1/agent-daemon/ws", environmentEnrollment{uuid.NewString(), uuid.NewString(), uuid.NewString(), "/workspace"}, "/environment/workspace"} + want := environmentBinding{"wss://core/api/v1/agent-daemon/ws", environmentEnrollment{uuid.NewString(), uuid.NewString(), uuid.NewString(), "/workspace"}, "/environment/workspace", "/environment/initialization/capabilities"} if err := saveEnvironmentBinding(root, want); err != nil { t.Fatal(err) } @@ -191,7 +191,7 @@ func TestEnvironmentBindingAllowsPackagedFilesAndRejectsUnsafeReceipt(t *testing if err := os.Chmod(root, 0700); err != nil { t.Fatal(err) } - want := environmentBinding{"wss://core/api/v1/agent-daemon/ws", environmentEnrollment{uuid.NewString(), uuid.NewString(), uuid.NewString(), "/workspace"}, "/environment/workspace"} + want := environmentBinding{"wss://core/api/v1/agent-daemon/ws", environmentEnrollment{uuid.NewString(), uuid.NewString(), uuid.NewString(), "/workspace"}, "/environment/workspace", "/environment/initialization/capabilities"} if err := os.WriteFile(filepath.Join(root, "installed-bundle"), []byte("bundle"), 0600); err != nil { t.Fatal(err) } @@ -213,7 +213,7 @@ func TestEnvironmentTargetCheckedBeforeCredentialTransmission(t *testing.T) { t.Fatal(err) } t.Setenv("OAC_RUNTIME_HOME", root) - want := environmentBinding{"wss://core/api/v1/agent-daemon/ws", environmentEnrollment{uuid.NewString(), uuid.NewString(), uuid.NewString(), "/workspace"}, "/environment/workspace"} + want := environmentBinding{"wss://core/api/v1/agent-daemon/ws", environmentEnrollment{uuid.NewString(), uuid.NewString(), uuid.NewString(), "/workspace"}, "/environment/workspace", "/environment/initialization/capabilities"} if err := saveEnvironmentBinding(root, want); err != nil { t.Fatal(err) } @@ -231,7 +231,7 @@ func TestEnvironmentTargetCheckedBeforeCredentialTransmission(t *testing.T) { func TestEnvironmentEnrollmentRejectsWrongIdentityAndWorkspace(t *testing.T) { environment := uuid.NewString() good := environmentEnrollment{uuid.NewString(), uuid.NewString(), environment, "/workspace"} - for _, mutate := range []func(*environmentEnrollment){func(b *environmentEnrollment) { b.EnvironmentID = uuid.NewString() }, func(b *environmentEnrollment) { b.DeviceID = "" }, func(b *environmentEnrollment) { b.SessionID = "invalid" }, func(b *environmentEnrollment) { b.WorkspaceDirectory = "/different" }} { + for _, mutate := range []func(*environmentEnrollment){func(b *environmentEnrollment) { b.EnvironmentID = uuid.NewString() }, func(b *environmentEnrollment) { b.DeviceID = "" }, func(b *environmentEnrollment) { b.SessionID = "invalid" }, func(b *environmentEnrollment) { b.WorkspaceDirectory = "relative" }} { bad := good mutate(&bad) server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _ = json.NewEncoder(w).Encode(bad) })) @@ -253,3 +253,14 @@ func TestEnvironmentConnectRejectsPairing(t *testing.T) { t.Fatal("pairing accepted or leaked") } } + +func TestEnvironmentEnrollmentAcceptsPhysicalWorkspaceSelection(t *testing.T) { + environment := uuid.NewString() + want := environmentEnrollment{uuid.NewString(), uuid.NewString(), environment, "/srv/runtime/workspace"} + server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { _ = json.NewEncoder(w).Encode(want) })) + defer server.Close() + actual, err := enrollEnvironment(t.Context(), environmentClient(), server.URL, environment, "secret") + if err != nil || actual != want { + t.Fatal("canonical workspace selection refused", actual, err) + } +} diff --git a/apps/parsar-daemon/internal/cli/connect_suspend.go b/apps/parsar-daemon/internal/cli/connect_suspend.go index 38a391252..c471a12f0 100644 --- a/apps/parsar-daemon/internal/cli/connect_suspend.go +++ b/apps/parsar-daemon/internal/cli/connect_suspend.go @@ -55,10 +55,8 @@ func newSuspendedRouter(conn *transport.Conn, registry *agent.Registry) (*suspen return &suspendedRouter{router: router, bridge: bridge, sender: sender, registry: wrapped, local: local}, nil } -func (s *suspendedRouter) shutdown() error { - ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second) - defer cancel() - return s.router.Shutdown(ctx) +func (s *suspendedRouter) shutdown() { + shutdownRouterUntilConfirmed(s.router.Shutdown, time.Second) } // runSuspendLoop uses the ordinary connection authentication and dispatch chain. @@ -78,10 +76,7 @@ func runSuspendLoop(ctx context.Context, dial transport.DialFn, registry *agent. return err } err = serveSuspendLifecycle(ctx, conn, dial, state, boot, discovery, control) - closeErr := state.shutdown() - if closeErr != nil { - return errors.Join(err, closeErr) - } + state.shutdown() if ctx.Err() != nil { return nil } diff --git a/apps/parsar-daemon/internal/cli/retired_configuration_test.go b/apps/parsar-daemon/internal/cli/retired_configuration_test.go index 740c8640a..8ba18fdc7 100644 --- a/apps/parsar-daemon/internal/cli/retired_configuration_test.go +++ b/apps/parsar-daemon/internal/cli/retired_configuration_test.go @@ -11,7 +11,7 @@ func TestRetiredRuntimeSettingsRejectBeforeEverySubcommand(t *testing.T) { t.Setenv("PARSAR_UNKNOWN_RUNTIME_SETTING", "private-canary") t.Setenv("PARSAR_CODEX_BIN", "") t.Setenv("PARSAR_MCP_BEARER_EXAMPLE", "private-bearer") - for _, command := range []string{"version", "--help", "connect", "runtime-capabilities"} { + for _, command := range []string{"version", "--help", "connect", "runtime-mcp-exec"} { stdout, stderr, err := runArgv(t, command) if err == nil || stdout != "" || stderr != "" { t.Fatalf("%s ran before retirement validation: %q %q %v", command, stdout, stderr, err) diff --git a/apps/parsar-daemon/internal/cli/root.go b/apps/parsar-daemon/internal/cli/root.go index 85d0ac474..46f72f553 100644 --- a/apps/parsar-daemon/internal/cli/root.go +++ b/apps/parsar-daemon/internal/cli/root.go @@ -34,7 +34,6 @@ func defaultRunContext() *runContext { // likely flow connect → status → stop / logs → logout. var commands = []command{ {name: "resume", summary: "Wake one planned hosted suspension", run: runResume}, - {name: "runtime-capabilities", summary: "Install frozen capabilities in the packaged Runtime", run: runRuntimeCapabilities}, {name: "runtime-mcp-exec", summary: "Execute installed MCP inside the packaged Runtime sandbox", run: runRuntimeMCP}, {name: "placement", summary: "Enroll or retire an explicitly managed local execution placement", run: runPlacement}, {name: "connect", summary: "Pair, open the reverse WebSocket, and start serving prompts", run: runConnect}, diff --git a/apps/parsar-daemon/internal/cli/root_test.go b/apps/parsar-daemon/internal/cli/root_test.go index 0bb8d576b..77f1be614 100644 --- a/apps/parsar-daemon/internal/cli/root_test.go +++ b/apps/parsar-daemon/internal/cli/root_test.go @@ -60,16 +60,15 @@ func TestSubcommandsAreRegistered(t *testing.T) { // Guards against dropping a subcommand off the commands slice — // the public CLI surface is the shipped contract. want := map[string]bool{ - "resume": false, - "runtime-capabilities": false, - "runtime-mcp-exec": false, - "placement": false, - "connect": false, - "status": false, - "stop": false, - "logs": false, - "logout": false, - "version": false, + "resume": false, + "runtime-mcp-exec": false, + "placement": false, + "connect": false, + "status": false, + "stop": false, + "logs": false, + "logout": false, + "version": false, } for _, c := range commands { if _, ok := want[c.name]; !ok { @@ -84,3 +83,14 @@ func TestSubcommandsAreRegistered(t *testing.T) { } } } + +func TestRemovedCapabilityHelperIsNotACommand(t *testing.T) { + help, _, err := runArgv(t, "--help") + if err != nil || strings.Contains(help, "runtime-capabilities") { + t.Fatal("removed helper remains advertised", err) + } + _, _, err = runArgv(t, "runtime-capabilities") + if err == nil || !strings.Contains(err.Error(), "unknown subcommand") { + t.Fatal("removed helper still dispatches", err) + } +} diff --git a/apps/parsar-daemon/internal/cli/runtime_capabilities.go b/apps/parsar-daemon/internal/cli/runtime_capabilities.go deleted file mode 100644 index ac17058ba..000000000 --- a/apps/parsar-daemon/internal/cli/runtime_capabilities.go +++ /dev/null @@ -1,56 +0,0 @@ -package cli - -import ( - "encoding/json" - "fmt" - "io" - "os" - - "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" -) - -// This packaged helper is invoked by the existing Core initializer. It has no -// daemon connection, credentials, scheduling or retry behavior. -func runRuntimeCapabilities(ctx *runContext, args []string) error { - if len(args) != 0 { - return agentcapabilities.ErrInvalid - } - var request agentcapabilities.Operation - decoder := json.NewDecoder(io.LimitReader(os.Stdin, 32<<20)) - decoder.DisallowUnknownFields() - if decoder.Decode(&request) != nil || request.Version != 1 { - return agentcapabilities.ErrInvalid - } - var extra any - if decoder.Decode(&extra) != io.EOF { - return agentcapabilities.ErrInvalid - } - root, err := os.OpenRoot("/environment") - if err != nil { - return agentcapabilities.ErrInvalid - } - defer root.Close() - installed, err := root.OpenRoot("initialization/capabilities") - if err != nil { - return agentcapabilities.ErrInvalid - } - defer installed.Close() - switch request.Action { - case "plugin": - err = agentcapabilities.InstallPlugin(installed, request.Slot, request.Archive, request.Plugin) - case "finalize": - workspace, openErr := root.OpenRoot("workspace") - if openErr != nil { - return agentcapabilities.ErrInvalid - } - defer workspace.Close() - err = agentcapabilities.Finalize(workspace, installed, request.Sources) - default: - err = agentcapabilities.ErrInvalid - } - if err != nil { - return agentcapabilities.ErrInvalid - } - _, err = fmt.Fprintln(ctx.stdout, `{"version":1,"outcome":"completed"}`) - return err -} diff --git a/apps/parsar-daemon/internal/cli/runtime_mcp.go b/apps/parsar-daemon/internal/cli/runtime_mcp.go index b8f6e7483..fcd030536 100644 --- a/apps/parsar-daemon/internal/cli/runtime_mcp.go +++ b/apps/parsar-daemon/internal/cli/runtime_mcp.go @@ -23,10 +23,10 @@ type mcpInvocation struct { // runRuntimeMCP executes only inside the packaged initializer's sandbox. It // never consumes MCP stdin, opens a daemon connection or owns a child process. func runRuntimeMCP(_ *runContext, args []string) error { - if len(args) != 2 { + if len(args) != 3 || args[0] == "/" || agentcapabilities.ValidateLocalDirectories([]string{args[0]}) != nil { return errRuntimeMCP } - root, err := os.OpenRoot(agentcapabilities.Directory) + root, err := os.OpenRoot(args[0]) if err != nil { return errRuntimeMCP } @@ -35,18 +35,21 @@ func runRuntimeMCP(_ *runContext, args []string) error { if err != nil { return errRuntimeMCP } - values, err := localworkspace.ReadToolEnvironment() + values, err := localworkspace.ReadOptionalToolEnvironment() if err != nil { return errRuntimeMCP } - invocation, err := resolveMCPInvocation(manifest, args[0], args[1], values) + invocation, err := resolveMCPInvocation(manifest, args[0], args[1], args[2], values) if err != nil { return errRuntimeMCP } return execRuntimeMCP(invocation) } -func resolveMCPInvocation(manifest agentcapabilities.Manifest, pkg, name string, values map[string]string) (mcpInvocation, error) { +func resolveMCPInvocation(manifest agentcapabilities.Manifest, installationRoot, pkg, name string, values map[string]string) (mcpInvocation, error) { + if installationRoot == "/" || agentcapabilities.ValidateLocalDirectories([]string{installationRoot}) != nil { + return mcpInvocation{}, errRuntimeMCP + } for _, installed := range manifest.MCP { server := installed.Server if installed.PackageRoot != pkg || server.Name != name { @@ -72,7 +75,7 @@ func resolveMCPInvocation(manifest agentcapabilities.Manifest, pkg, name string, } cwd := server.CWD if !filepath.IsAbs(cwd) { - cwd = filepath.Join(agentcapabilities.Directory, installed.PackageRoot, cwd) + cwd = filepath.Join(installationRoot, installed.PackageRoot, cwd) } result := mcpInvocation{command: server.Command, args: append([]string{server.Command}, server.Args...), cwd: cwd} for key, value := range env { diff --git a/apps/parsar-daemon/internal/cli/runtime_mcp_test.go b/apps/parsar-daemon/internal/cli/runtime_mcp_test.go index 39b43ec54..960b517a5 100644 --- a/apps/parsar-daemon/internal/cli/runtime_mcp_test.go +++ b/apps/parsar-daemon/internal/cli/runtime_mcp_test.go @@ -14,16 +14,16 @@ func TestRuntimeMCPSelectsOnlyInstalledUserEnvironment(t *testing.T) { }}}} t.Setenv("SELECTED", "private-native-value") t.Setenv("NATIVE_ONLY", "private-native-value") - got, err := resolveMCPInvocation(manifest, "plugins/0", "local", map[string]string{"SELECTED": "user-value", "UNDECLARED": "not-injected"}) - if err != nil || got.command != "python3" || got.cwd != agentcapabilities.Directory+"/plugins/0/server" || + got, err := resolveMCPInvocation(manifest, "/private/runtime/capabilities", "plugins/0", "local", map[string]string{"SELECTED": "user-value", "UNDECLARED": "not-injected"}) + if err != nil || got.command != "python3" || got.cwd != "/private/runtime/capabilities/plugins/0/server" || !slices.Equal(got.args, []string{"python3", "proof.py", "${LITERAL}"}) || !slices.Contains(got.env, "SELECTED=user-value") || slices.Contains(got.env, "UNDECLARED=not-injected") || slices.Contains(got.env, "NATIVE_ONLY=private-native-value") { t.Fatalf("incorrect isolated invocation: %+v %v", got, err) } - if _, err := resolveMCPInvocation(manifest, "plugins/0", "local", map[string]string{}); err == nil { + if _, err := resolveMCPInvocation(manifest, "/private/runtime/capabilities", "plugins/0", "local", map[string]string{}); err == nil { t.Fatal("missing user value fell back to native environment") } - if _, err := resolveMCPInvocation(manifest, "plugins/1", "local", map[string]string{"SELECTED": "x"}); err == nil { + if _, err := resolveMCPInvocation(manifest, "/private/runtime/capabilities", "plugins/1", "local", map[string]string{"SELECTED": "x"}); err == nil { t.Fatal("undeclared package selected") } } @@ -32,12 +32,20 @@ func TestRuntimeMCPKeepsAbsoluteCWDAndRejectsHTTP(t *testing.T) { manifest := agentcapabilities.Manifest{MCP: []agentcapabilities.InstalledMCP{{PackageRoot: "plugins/0", Server: agentplugin.MCPServer{ Name: "local", Type: "stdio", Command: "python3", CWD: "/workspace", }}}} - got, err := resolveMCPInvocation(manifest, "plugins/0", "local", nil) + got, err := resolveMCPInvocation(manifest, "/private/runtime/capabilities", "plugins/0", "local", nil) if err != nil || got.cwd != "/workspace" { t.Fatalf("absolute cwd changed: %+v %v", got, err) } manifest.MCP[0].Server.Type = "http" - if _, err := resolveMCPInvocation(manifest, "plugins/0", "local", nil); err == nil { + if _, err := resolveMCPInvocation(manifest, "/private/runtime/capabilities", "plugins/0", "local", nil); err == nil { t.Fatal("HTTP configuration treated as a process") } } + +func TestRuntimeMCPRejectsUncleanInstallationRoot(t *testing.T) { + for _, root := range []string{"", "/", "relative", "/tmp/../private", "/tmp/root/", "/tmp/line\n"} { + if _, err := resolveMCPInvocation(agentcapabilities.Manifest{}, root, "plugins/0", "local", nil); err == nil { + t.Fatal("invalid root accepted", root) + } + } +} diff --git a/apps/parsar-daemon/internal/cli/skill_upload.go b/apps/parsar-daemon/internal/cli/skill_upload.go index 935b35cd3..c77b0fefe 100644 --- a/apps/parsar-daemon/internal/cli/skill_upload.go +++ b/apps/parsar-daemon/internal/cli/skill_upload.go @@ -12,16 +12,7 @@ import ( func withSkillUploadServer(factory agent.Factory, serverURL string) agent.Factory { return func(ctx context.Context, req proto.PromptRequestPayload, out chan<- proto.Envelope) (agent.Session, error) { - env, _ := req.AgentOptions["env"].(map[string]any) - if token, _ := env["PARSAR_CAPABILITY_UPLOAD_TOKEN"].(string); token != "" { - env = maps.Clone(env) - env["PARSAR_SERVER_URL"] = serverURL - if executable, err := os.Executable(); err == nil { - addCompanionCLIPath(env, filepath.Dir(executable)) - } - req.AgentOptions = maps.Clone(req.AgentOptions) - req.AgentOptions["env"] = env - } + req = skillUploadRequest(req, serverURL) return factory(ctx, req, out) } } @@ -37,3 +28,17 @@ func addCompanionCLIPath(env map[string]any, dir string) { } env["PATH"] = dir + string(os.PathListSeparator) + path } + +func skillUploadRequest(req proto.PromptRequestPayload, serverURL string) proto.PromptRequestPayload { + env, _ := req.AgentOptions["env"].(map[string]any) + if token, _ := env["PARSAR_CAPABILITY_UPLOAD_TOKEN"].(string); token != "" { + env = maps.Clone(env) + env["PARSAR_SERVER_URL"] = serverURL + if executable, err := os.Executable(); err == nil { + addCompanionCLIPath(env, filepath.Dir(executable)) + } + req.AgentOptions = maps.Clone(req.AgentOptions) + req.AgentOptions["env"] = env + } + return req +} diff --git a/apps/parsar-daemon/internal/dispatch/executor.go b/apps/parsar-daemon/internal/dispatch/executor.go new file mode 100644 index 000000000..afe959a66 --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/executor.go @@ -0,0 +1,376 @@ +package dispatch + +import ( + "context" + "crypto/sha256" + "encoding/json" + "errors" + "strings" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/google/uuid" +) + +const executorIdleCapacity = 16 + +type executorState struct { + id, sessionID, environmentID, stateKey string + fingerprint [32]byte + native agent.Executor + nativeID string + ctx context.Context + cancel context.CancelFunc + admission *preparationState + run *sessionState + preparing bool + invalid bool + closeDone chan struct{} + closeReason string + closeErr error + timer *time.Timer + idleLease uint64 +} + +func executorFingerprint(req proto.PromptRequestPayload) ([32]byte, error) { + req.RunID, req.Input = "", nil + req.AgentSessionID = "" + req.RequireExistingNativeSession = false + req.ReleaseOnCompletion = false + data, err := json.Marshal(req) + return sha256.Sum256(data), err +} + +func (r *Router) handleExecutorPrepare(ctx context.Context, env proto.Envelope, input proto.ExecutionPreparePayload) error { + req := input.Configuration + if strings.TrimSpace(input.SessionID) == "" || req.RunID != "" || len(req.Input) != 0 || req.ConversationID != "" || req.WorkspaceAuthoring || req.AgentStateKey != "agents-api-"+input.SessionID || !req.StrictResume { + return r.rejectPreparation(env, "invalid_configuration") + } + caps := r.availableCapabilities(req.AgentKind) + factory, err := r.registry.ResolveExecutor(req.AgentKind) + if err != nil || !caps.Preparation { + return r.rejectPreparation(env, "unsupported_preparation") + } + req, err = r.localWorkspace.Configure(req) + if err != nil { + return r.rejectPreparation(env, "invalid_configuration") + } + if validateExecutionEnvironment(req, caps) != nil || len(req.FunctionTools) > 0 && !caps.FunctionTools { + return r.rejectPreparation(env, "unsupported_configuration") + } + fingerprint, err := executorFingerprint(req) + if err != nil { + return r.rejectPreparation(env, "invalid_configuration") + } + encoded, err := json.Marshal(input) + if err != nil { + return r.rejectPreparation(env, "invalid_configuration") + } + requestFingerprint := sha256.Sum256(encoded) + r.mu.Lock() + if r.closed || r.suspension != nil { + r.mu.Unlock() + return ErrRouterClosed + } + if r.workspaceWrite != nil || r.workspaceExport != nil || r.runtimePreparation != nil { + r.mu.Unlock() + return r.rejectPreparation(env, "resource_unavailable") + } + r.prunePreparationsLocked() + if old := r.preparationRequests[env.ID]; old != nil { + matches, status := old.fingerprint == requestFingerprint, old.status + r.mu.Unlock() + if !matches { + return r.rejectPreparation(env, "request_conflict") + } + r.publishPreparation(old, status) + return nil + } + if len(r.preparations) >= preparationRecords { + r.mu.Unlock() + return r.rejectPreparation(env, "preparation_capacity") + } + active := 0 + for _, candidate := range r.executors { + if candidate.sessionID != input.SessionID && candidate.stateKey == req.AgentStateKey { + r.mu.Unlock() + return r.rejectPreparation(env, "session_binding_conflict") + } + } + for _, owner := range r.executors { + if owner.preparing || owner.admission != nil || owner.run != nil || owner.invalid { + active++ + } + } + if active >= preparationCapacity { + r.mu.Unlock() + return r.rejectPreparation(env, "preparation_capacity") + } + owner := r.executors[input.SessionID] + reused := owner != nil + if owner != nil { + code := "" + switch { + case owner.environmentID != req.EnvironmentID() || owner.fingerprint != fingerprint: + code = "configuration_conflict" + case owner.invalid: + code = "executor_cleanup_unconfirmed" + case owner.preparing || owner.admission != nil || owner.run != nil: + code = "resource_unavailable" + case req.AgentSessionID != owner.nativeID: + code = "native_session_conflict" + case req.RequireExistingNativeSession && owner.nativeID == "": + code = "history_unavailable" + } + if code != "" { + r.mu.Unlock() + return r.rejectPreparation(env, code) + } + if owner.timer != nil { + owner.timer.Stop() + } + owner.idleLease++ + } else { + if len(r.executors) >= executorIdleCapacity+preparationCapacity { + r.mu.Unlock() + return r.rejectPreparation(env, "executor_capacity") + } + ownerCtx, cancel := context.WithCancel(context.WithoutCancel(ctx)) + owner = &executorState{id: uuid.NewString(), sessionID: input.SessionID, environmentID: req.EnvironmentID(), stateKey: req.AgentStateKey, fingerprint: fingerprint, ctx: ownerCtx, cancel: cancel, preparing: true, nativeID: req.AgentSessionID} + r.executors[input.SessionID] = owner + r.log.Info("executor owner_created", "executor_id", owner.id, "session_id", owner.sessionID) + } + operation, cancel := context.WithCancel(context.WithoutCancel(ctx)) + p := &preparationState{requestID: env.ID, trace: env.Trace, fingerprint: requestFingerprint, ctx: operation, cancel: cancel, stateKey: req.AgentStateKey, environmentID: req.EnvironmentID(), executor: owner, owns: true, busy: !reused, deadline: time.Now().Add(r.preparationTimeout)} + state := "preparing" + if reused { + state = "ready" + } + p.status = proto.PreparationStatusPayload{Handle: uuid.NewString(), ExecutorID: owner.id, Revision: 1, State: state, Reused: reused, ExpiresAt: p.deadline.UnixMilli()} + owner.admission = p + r.preparations[p.status.Handle], r.preparationRequests[env.ID] = p, p + p.timer = time.AfterFunc(r.preparationTimeout, func() { r.releasePreparation(p, "expired", "", true, true) }) + if !reused { + r.shutdownWG.Add(1) + } + status := p.status + r.mu.Unlock() + if reused { + r.publishPreparation(p, status) + } else { + go r.prepareExecutor(p, req, factory) + } + return nil +} + +func (r *Router) prepareExecutor(p *preparationState, req proto.PromptRequestPayload, factory agent.ExecutorFactory) { + defer r.shutdownWG.Done() + owner := p.executor + started := time.Now() + r.mu.Lock() + initial := p.status + r.mu.Unlock() + if !r.sendPreparation(p.requestID, p.trace, initial) { + r.abandonExecutorAdmission(p, "failed", "status_delivery_failed", false) + } + var native agent.Executor + var err error + if owner.ctx.Err() == nil { + req, err = r.localWorkspace.Prepare(owner.ctx, req) + if err == nil && owner.ctx.Err() == nil { + native, err = factory(owner.ctx, req) + } + } else { + err = owner.ctx.Err() + } + r.mu.Lock() + owner.native, owner.preparing = native, false + r.log.Info("executor native_prepare", "executor_id", owner.id, "session_id", owner.sessionID, "duration_ms", time.Since(started).Milliseconds(), "success", err == nil && native != nil) + ready := native != nil && err == nil && !owner.invalid && !r.closed && r.suspension == nil && p.status.State == "preparing" && p.ctx.Err() == nil + p.busy = false + if ready { + p.status.State, p.status.Revision = "ready", p.status.Revision+1 + } else { + owner.invalid = true + owner.closeReason = "preparation_failed" + if p.status.State == "preparing" { + p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "preparation_failed", p.status.Revision+1 + } + p.timer.Stop() + } + status := p.status + r.mu.Unlock() + if !ready { + closeErr := r.closeExecutor(owner) + r.mu.Lock() + p.owns, p.closeErr = closeErr != nil, closeErr + if closeErr != nil { + p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "executor_cleanup_unconfirmed", p.status.Revision+1 + } + status = p.status + r.mu.Unlock() + } + if !r.sendPreparation(p.requestID, p.trace, status) && ready { + r.abandonExecutorAdmission(p, "failed", "status_delivery_failed", false) + } +} + +func (r *Router) abandonExecutorAdmission(p *preparationState, state, code string, publish bool) { + r.mu.Lock() + owner := p.executor + if p.handoff != nil || p.status.State == "started" { + status := p.status + r.mu.Unlock() + if publish && state == "released" { + r.publishPreparation(p, status) + } + return + } + if p.status.State == "preparing" || p.status.State == "ready" { + p.status.State, p.status.ErrorCode, p.status.Revision = state, code, p.status.Revision+1 + p.timer.Stop() + p.cancel() + p.owns = false + if owner.admission == p { + owner.admission = nil + } + if owner.preparing { + owner.invalid = true + owner.cancel() + } else if !owner.invalid { + r.scheduleExecutorIdleLocked(owner) + } + } + status := p.status + r.mu.Unlock() + if publish { + r.publishPreparation(p, status) + } +} + +func (r *Router) scheduleExecutorIdleLocked(owner *executorState) { + if owner.invalid || owner.preparing || owner.run != nil || owner.admission != nil { + return + } + if owner.timer != nil { + owner.timer.Stop() + } + idle := 0 + for _, candidate := range r.executors { + if candidate != owner && candidate.run == nil && candidate.admission == nil && !candidate.preparing { + idle++ + } + } + if idle >= executorIdleCapacity { + owner.invalid = true + owner.closeReason = "idle_capacity" + r.shutdownWG.Add(1) + go func() { defer r.shutdownWG.Done(); _ = r.closeExecutor(owner) }() + return + } + owner.idleLease++ + lease := owner.idleLease + r.log.Info("executor owner_idle", "executor_id", owner.id, "session_id", owner.sessionID) + owner.timer = time.AfterFunc(r.idleTimeout, func() { + r.mu.Lock() + if r.executors[owner.sessionID] != owner || owner.idleLease != lease || owner.run != nil || owner.admission != nil || owner.invalid { + r.mu.Unlock() + return + } + owner.invalid = true + owner.closeReason = "idle_expired" + r.shutdownWG.Add(1) + r.mu.Unlock() + defer r.shutdownWG.Done() + _ = r.closeExecutor(owner) + }) +} + +// Close failures keep the exact owner; a later call retries only settled failures. +func (r *Router) closeExecutor(owner *executorState) error { + r.mu.Lock() + owner.invalid = true + if owner.closeReason == "" { + owner.closeReason = "invalidated" + } + if owner.timer != nil { + owner.timer.Stop() + } + if owner.preparing { + owner.cancel() + r.mu.Unlock() + return errors.New("executor preparation is still settling") + } + if done := owner.closeDone; done != nil { + select { + case <-done: + if owner.closeErr == nil { + r.mu.Unlock() + return nil + } + default: + r.mu.Unlock() + <-done + r.mu.Lock() + err := owner.closeErr + r.mu.Unlock() + return err + } + } + done := make(chan struct{}) + owner.closeDone = done + native := owner.native + r.mu.Unlock() + var err error + if native != nil { + ctx, cancel := context.WithTimeout(context.Background(), preparedCancelTimeout) + err = native.Close(ctx) + cancel() + } + r.mu.Lock() + owner.closeErr = err + if err == nil { + owner.cancel() + } + r.log.Info("executor owner_closed", "executor_id", owner.id, "session_id", owner.sessionID, "confirmed", err == nil, "reason", owner.closeReason) + if err == nil && owner.run == nil && r.executors[owner.sessionID] == owner { + delete(r.executors, owner.sessionID) + } + close(done) + r.mu.Unlock() + return err +} + +func (r *Router) closeIdleExecutorsLocked() []*executorState { + var owners []*executorState + for _, owner := range r.executors { + owner.invalid = true + owner.closeReason = "shutdown" + if owner.preparing { + owner.cancel() + } + if owner.timer != nil { + owner.timer.Stop() + } + if owner.admission != nil && owner.run == nil { + p := owner.admission + p.cancel() + p.timer.Stop() + p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "connection_closed", p.status.Revision+1 + p.owns = false + owner.admission = nil + } + if !owner.preparing && owner.run == nil { + r.shutdownWG.Add(1) + owners = append(owners, owner) + } + } + return owners +} + +func (r *Router) closeIdleExecutors(owners []*executorState) { + for _, owner := range owners { + go func() { defer r.shutdownWG.Done(); _ = r.closeExecutor(owner) }() + } +} diff --git a/apps/parsar-daemon/internal/dispatch/executor_cancel_receipt_test.go b/apps/parsar-daemon/internal/dispatch/executor_cancel_receipt_test.go new file mode 100644 index 000000000..8ea3b6ee2 --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/executor_cancel_receipt_test.go @@ -0,0 +1,249 @@ +package dispatch_test + +import ( + "context" + "errors" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/dispatch" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +type receiptCancelSender struct { + *recSender + entered, release chan struct{} + once sync.Once +} + +func (s *receiptCancelSender) Send(ctx context.Context, env proto.Envelope) error { + if env.Type == proto.TypePromptSteerAck { + var ack proto.PromptSteerAckPayload + _ = env.DecodePayload(&ack) + if ack.InputID == "pending" && !ack.Written { + s.once.Do(func() { close(s.entered) }) + select { + case <-s.release: + case <-ctx.Done(): + return ctx.Err() + } + } + } + return s.recSender.Send(ctx, env) +} + +type receiptCancelExecutor struct { + turn chan *receiptCancelTurn + current *receiptCancelTurn + cancelFails, closeFailsFirst bool + closeEntered, closeRelease chan struct{} + closes atomic.Int32 + closeOnce sync.Once +} + +func (e *receiptCancelExecutor) StartTurn(_ context.Context, id string, _ proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + turn := &receiptCancelTurn{id: id, out: out, owner: e, written: make(chan struct{}), cancelled: make(chan struct{}), closed: make(chan struct{}), settled: make(chan struct{}), awaiting: make(chan struct{})} + e.current = turn + e.turn <- turn + return turn, nil +} +func (e *receiptCancelExecutor) Close(context.Context) error { + attempt := e.closes.Add(1) + if e.closeFailsFirst && attempt == 1 { + return errors.New("native close unconfirmed") + } + e.closeOnce.Do(func() { close(e.closeEntered) }) + <-e.closeRelease + if e.current != nil { + e.current.closeOnce.Do(func() { close(e.current.closed) }) + e.current.finish() + } + return nil +} + +type receiptCancelTurn struct { + id string + out chan<- proto.Envelope + owner *receiptCancelExecutor + written, cancelled, closed, settled, awaiting chan struct{} + cancelOnce, closeOnce, finishOnce, terminalOnce, awaitOnce sync.Once + cancels atomic.Int32 +} + +func (t *receiptCancelTurn) terminal() { + t.terminalOnce.Do(func() { event, _ := proto.NewEnvelope(proto.TypeDone, t.id, t.CancellationOutcome()); t.out <- event }) +} +func (t *receiptCancelTurn) finish() { + t.finishOnce.Do(func() { t.terminal(); close(t.out); close(t.settled) }) +} +func (t *receiptCancelTurn) Cancel(context.Context) error { + t.cancels.Add(1) + t.cancelOnce.Do(func() { close(t.cancelled) }) + if t.owner.cancelFails { + return errors.New("native cancellation unconfirmed") + } + t.finish() + return nil +} +func (t *receiptCancelTurn) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + t.awaitOnce.Do(func() { close(t.awaiting) }) + select { + case <-t.settled: + return agent.TurnSettlement{Reusable: !t.owner.cancelFails}, nil + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} +func (t *receiptCancelTurn) CancellationOutcome() proto.DonePayload { + return proto.DonePayload{Content: "observed", Metadata: map[string]any{proto.DoneMetaAgentSessionID: "native-session"}} +} +func (t *receiptCancelTurn) Steer(ctx context.Context, input proto.PromptSteerPayload) error { + return t.SteerWithReceipt(ctx, input, nil) +} +func (t *receiptCancelTurn) SteerWithReceipt(ctx context.Context, _ proto.PromptSteerPayload, written func()) error { + written() + close(t.written) + native := t.cancelled + if t.owner.cancelFails { + native = t.closed + } + select { + case <-native: + case <-ctx.Done(): + return ctx.Err() + } + return errors.New("native input consumption is unknown") +} + +func TestExecutorCancellationReachesNativeBeforeDurableReceiptJoin(t *testing.T) { + for _, mode := range []string{"cancel", "done_then_cancel", "cancel_failure", "close_failure_retry"} { + t.Run(mode, func(t *testing.T) { + sender := &receiptCancelSender{recSender: &recSender{}, entered: make(chan struct{}), release: make(chan struct{})} + owner := &receiptCancelExecutor{turn: make(chan *receiptCancelTurn, 2), cancelFails: mode == "cancel_failure" || mode == "close_failure_retry", closeFailsFirst: mode == "close_failure_retry", closeEntered: make(chan struct{}), closeRelease: make(chan struct{})} + reg := agent.NewRegistry() + reg.RegisterKind(proto.SupportedAgentKind{Kind: "reusable", Available: true, Capabilities: proto.AgentKindCapabilities{EnvironmentNone: true}}, func(context.Context, proto.PromptRequestPayload, chan<- proto.Envelope) (agent.Session, error) { + return nil, errors.New("legacy factory forbidden") + }) + reg.RegisterExecutor("reusable", func(context.Context, proto.PromptRequestPayload) (agent.Executor, error) { return owner, nil }) + r, err := dispatch.New(dispatch.Config{Registry: reg, Sender: sender, IdleTimeout: time.Hour}) + if err != nil { + t.Fatal(err) + } + var receiptOnce, closeOnce sync.Once + releaseReceipt := func() { receiptOnce.Do(func() { close(sender.release) }) } + releaseClose := func() { closeOnce.Do(func() { close(owner.closeRelease) }) } + t.Cleanup(func() { + releaseReceipt() + releaseClose() + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + if err := r.Shutdown(ctx); err != nil { + t.Error(err) + } + }) + wait := func(ch <-chan struct{}, reason string) { + t.Helper() + select { + case <-ch: + case <-time.After(time.Second): + t.Fatal(reason) + } + } + p := executorAdmission(t, r, sender.recSender, "first", executorRequest()) + startExecutorTurn(t, r, sender.recSender, "first", "run", p) + turn := <-owner.turn + input := proto.PromptSteerPayload{InputID: "pending", Input: proto.TextInput("second input"), DurableReceipt: true} + if err := r.Handle(t.Context(), mustEnv(t, proto.TypePromptSteer, "run", input)); err != nil { + t.Fatal(err) + } + wait(turn.written, "input did not reach native write") + if mode == "done_then_cancel" { + turn.terminal() + wait(turn.awaiting, "natural Done did not begin settlement") + } + cancel := func(id string) { + t.Helper() + if err := r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "run", proto.PromptCancelPayload{DeliveryID: id})); err != nil { + t.Fatal(err) + } + } + cancel("cancel") + wait(turn.cancelled, "receipt barrier prevented native cancellation") + acknowledgements := 0 + if mode == "close_failure_retry" { + waitFor(t, func() bool { return len(cancellationAcks(sender.recSender)) == 1 }, "failed Close acknowledgement") + ack := cancellationAcks(sender.recSender)[0] + if ack.Applied || ack.Outcome != nil || r.ActiveRuns() != 1 { + t.Fatal("failed Close released ownership or fabricated cancellation") + } + acknowledgements = 1 + cancel("retry") + } + if owner.cancelFails { + wait(owner.closeEntered, "failed cancellation did not reach resource cleanup") + if r.ActiveRuns() != 1 || len(cancellationAcks(sender.recSender)) != acknowledgements { + t.Fatal("cleanup was acknowledged before resource close") + } + releaseClose() + } + wait(sender.entered, "native cancellation did not unblock the written input") + if r.ActiveRuns() != 1 || len(cancellationAcks(sender.recSender)) != acknowledgements { + t.Fatal("cancellation passed an uncommitted input receipt") + } + for _, env := range sender.snapshot() { + if env.Type == proto.TypeDone { + t.Fatal("Done preceded the admitted input receipt") + } + } + releaseReceipt() + waitFor(t, func() bool { + return len(cancellationAcks(sender.recSender)) == acknowledgements+1 && r.ActiveRuns() == 0 && hasFrame(sender.recSender, proto.TypeDone, "run") + }, "joined cancellation") + frames := sender.snapshot() + inputAt, doneAt, cancelAt := -1, -1, -1 + for i, env := range frames { + switch env.Type { + case proto.TypePromptSteerAck: + var ack proto.PromptSteerAckPayload + _ = env.DecodePayload(&ack) + if !ack.Written { + if ack.Accepted || ack.ErrorCode != "outcome_unknown" { + t.Fatalf("unknown input fabricated receipt: %+v", ack) + } + inputAt = i + } + case proto.TypeDone: + doneAt = i + case proto.TypeInteractionDecisionAck: + cancelAt = i + } + } + if inputAt < 0 || doneAt < inputAt || cancelAt < inputAt || !owner.cancelFails && cancelAt < doneAt { + t.Fatalf("receipt/Done/cancel order=%d/%d/%d", inputAt, doneAt, cancelAt) + } + ack := cancellationAcks(sender.recSender)[acknowledgements] + if ack.Applied == owner.cancelFails || (ack.Outcome != nil) == owner.cancelFails { + t.Fatalf("cancellation evidence changed during cleanup: %+v", ack) + } + if !owner.cancelFails { + req := executorRequest() + req.Configuration.AgentSessionID = "native-session" + req.Configuration.RequireExistingNativeSession = true + next := executorAdmission(t, r, sender.recSender, "next", req) + if next.ExecutorID != p.ExecutorID || !next.Reused { + t.Fatal("healthy cancellation replaced owner") + } + startExecutorTurn(t, r, sender.recSender, "next", "next-run", next) + successor := <-owner.turn + cancel("late-old-run") + if successor.cancels.Load() != 0 || owner.closes.Load() != 0 { + t.Fatal("old cancellation reached successor") + } + successor.finish() + } + }) + } +} diff --git a/apps/parsar-daemon/internal/dispatch/executor_handoff_test.go b/apps/parsar-daemon/internal/dispatch/executor_handoff_test.go new file mode 100644 index 000000000..730698ef6 --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/executor_handoff_test.go @@ -0,0 +1,228 @@ +package dispatch + +import ( + "context" + "errors" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +type terminalHandoffSender struct { + frames chan proto.Envelope + visible chan struct{} + release chan struct{} + failure error +} + +func (s *terminalHandoffSender) Send(ctx context.Context, env proto.Envelope) error { + select { + case s.frames <- env: + case <-ctx.Done(): + return ctx.Err() + } + if env.Type == proto.TypeDone && env.ID == "first" { + close(s.visible) + select { + case <-s.release: + return s.failure + case <-ctx.Done(): + return ctx.Err() + } + } + return nil +} + +type terminalHandoffExecutor struct { + turns chan *terminalHandoffTurn + closes atomic.Int32 +} + +func (e *terminalHandoffExecutor) Close(context.Context) error { e.closes.Add(1); return nil } +func (e *terminalHandoffExecutor) StartTurn(_ context.Context, id string, _ proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + turn := &terminalHandoffTurn{id: id, out: out, settled: make(chan struct{})} + e.turns <- turn + return turn, nil +} + +type terminalHandoffTurn struct { + id string + out chan<- proto.Envelope + settled chan struct{} + once sync.Once +} + +func (t *terminalHandoffTurn) finish() { + t.once.Do(func() { + env, _ := proto.NewEnvelope(proto.TypeDone, t.id, t.CancellationOutcome()) + t.out <- env + close(t.out) + close(t.settled) + }) +} +func (t *terminalHandoffTurn) Cancel(context.Context) error { t.finish(); return nil } +func (t *terminalHandoffTurn) CancellationOutcome() proto.DonePayload { + return proto.DonePayload{Content: t.id, Metadata: map[string]any{proto.DoneMetaAgentSessionID: "native"}} +} +func (t *terminalHandoffTurn) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + select { + case <-t.settled: + return agent.TurnSettlement{Reusable: true}, nil + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} + +func TestPreparedDonePublishesAfterExecutorHandoff(t *testing.T) { + for _, fail := range []bool{false, true} { + name := "delivered" + if fail { + name = "late_delivery_error" + } + t.Run(name, func(t *testing.T) { + sender := &terminalHandoffSender{frames: make(chan proto.Envelope, 32), visible: make(chan struct{}), release: make(chan struct{})} + if fail { + sender.failure = errors.New("old terminal delivery failed") + } + owner := &terminalHandoffExecutor{turns: make(chan *terminalHandoffTurn, 3)} + registry := agent.NewRegistry() + registry.RegisterKind(proto.SupportedAgentKind{Kind: "handoff", Available: true, Capabilities: proto.AgentKindCapabilities{EnvironmentNone: true}}, + func(context.Context, proto.PromptRequestPayload, chan<- proto.Envelope) (agent.Session, error) { + return nil, errors.New("legacy path forbidden") + }) + var creates atomic.Int32 + registry.RegisterExecutor("handoff", func(context.Context, proto.PromptRequestPayload) (agent.Executor, error) { + creates.Add(1) + return owner, nil + }) + r, err := New(Config{Registry: registry, Sender: sender, IdleTimeout: time.Hour}) + if err != nil { + t.Fatal(err) + } + var releaseOnce sync.Once + unblock := func() { releaseOnce.Do(func() { close(sender.release) }) } + t.Cleanup(func() { + unblock() + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + if err := r.Shutdown(ctx); err != nil { + t.Error(err) + } + }) + handle := func(kind, id string, payload any) { + t.Helper() + env, err := proto.NewEnvelope(kind, id, payload) + if err != nil { + t.Fatal(err) + } + if err = r.Handle(t.Context(), env); err != nil { + t.Fatal(err) + } + } + status := func(id, state string) proto.PreparationStatusPayload { + t.Helper() + deadline := time.After(time.Second) + for { + select { + case env := <-sender.frames: + if env.Type != proto.TypePreparationStatus || env.ID != id { + continue + } + var value proto.PreparationStatusPayload + if err := env.DecodePayload(&value); err != nil { + t.Fatal(err) + } + if value.State == state { + return value + } + if value.State == "rejected" || value.State == "failed" { + t.Fatalf("next admission failed: %+v", value) + } + case <-deadline: + t.Fatal("preparation status missing") + } + } + } + request := proto.ExecutionPreparePayload{SessionID: "session", Configuration: proto.PromptRequestPayload{AgentKind: "handoff", AgentStateKey: "agents-api-session", StrictResume: true, DisableExecutionEnvironment: true}} + admit := func(id string) proto.PreparationStatusPayload { + t.Helper() + handle(proto.TypeExecutionPrepare, id, request) + return status(id, "ready") + } + start := func(id string, p proto.PreparationStatusPayload) *terminalHandoffTurn { + t.Helper() + handle(proto.TypeExecutionStart, id, proto.ExecutionStartPayload{Handle: p.Handle, ExecutorID: p.ExecutorID, RunID: id, Input: proto.TextInput(id)}) + status(id, "started") + select { + case turn := <-owner.turns: + return turn + case <-time.After(time.Second): + t.Fatal("native Turn missing") + return nil + } + } + first := admit("first") + old := start("first", first) + r.mu.Lock() + handoff := r.sessions["first"].preparedHandoff + r.mu.Unlock() + old.finish() + select { + case <-sender.visible: + case <-time.After(time.Second): + t.Fatal("Done not visible") + } + // Simulate Core admitting its successor after seeing Done, while the old + // Send is still blocked. Exact native continuity must already be committed. + request.Configuration.AgentSessionID = "native" + request.Configuration.RequireExistingNativeSession = true + second := admit("second") + if second.ExecutorID != first.ExecutorID || !second.Reused || creates.Load() != 1 { + t.Fatal("settled owner was replaced") + } + next := start("second", second) + unblock() + r.mu.Lock() + settled := handoff.release.settled + r.mu.Unlock() + select { + case <-settled: + case <-time.After(time.Second): + t.Fatal("old publication did not finish") + } + r.mu.Lock() + current := r.executors["session"] + valid := current != nil && !current.invalid && current.run == r.sessions["second"] && current.admission == r.preparations[second.Handle] + deliveryErr := handoff.outputErr + r.mu.Unlock() + if !valid || r.ActiveRuns() != 1 || owner.closes.Load() != 0 { + t.Fatal("old terminal cleanup damaged its successor") + } + if (deliveryErr != nil) != fail { + t.Fatalf("delivery error=%v", deliveryErr) + } + next.finish() + // The second terminal does not block: observing it must likewise permit an + // immediate third admission without losing or recreating the retained owner. + for { + select { + case env := <-sender.frames: + if env.Type == proto.TypeDone && env.ID == "second" { + goto completed + } + case <-time.After(time.Second): + t.Fatal("second Done missing") + } + } + completed: + third := admit("third") + if third.ExecutorID != first.ExecutorID || !third.Reused || creates.Load() != 1 { + t.Fatal("successor failed to retain owner") + } + }) + } +} diff --git a/apps/parsar-daemon/internal/dispatch/executor_test.go b/apps/parsar-daemon/internal/dispatch/executor_test.go new file mode 100644 index 000000000..db02f9ff8 --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/executor_test.go @@ -0,0 +1,312 @@ +package dispatch_test + +import ( + "context" + "errors" + "fmt" + "sync" + "sync/atomic" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/dispatch" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +type reusableExecutor struct { + starts chan *reusableTurn + closes atomic.Int32 + mu sync.Mutex + closeErr error + startErr error +} + +func (e *reusableExecutor) Close(context.Context) error { + e.closes.Add(1) + e.mu.Lock() + defer e.mu.Unlock() + return e.closeErr +} +func (e *reusableExecutor) StartTurn(_ context.Context, id string, _ proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + if e.startErr != nil { + return nil, e.startErr + } + t := &reusableTurn{id: id, out: out, settled: make(chan struct{})} + e.starts <- t + return t, nil +} + +type reusableTurn struct { + id string + out chan<- proto.Envelope + settled chan struct{} + once sync.Once + cancels atomic.Int32 +} + +func (t *reusableTurn) finish() { + t.once.Do(func() { + frame, _ := proto.NewEnvelope(proto.TypeDone, t.id, t.CancellationOutcome()) + t.out <- frame + close(t.out) + close(t.settled) + }) +} +func (t *reusableTurn) Cancel(context.Context) error { t.cancels.Add(1); t.finish(); return nil } +func (t *reusableTurn) CancellationOutcome() proto.DonePayload { + return proto.DonePayload{Content: t.id, Metadata: map[string]any{proto.DoneMetaAgentSessionID: "native-session"}} +} +func (t *reusableTurn) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + select { + case <-t.settled: + return agent.TurnSettlement{Reusable: true}, nil + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} + +func executorRequest() proto.ExecutionPreparePayload { + return proto.ExecutionPreparePayload{SessionID: "session", Configuration: proto.PromptRequestPayload{AgentKind: "reusable", AgentStateKey: "agents-api-session", StrictResume: true, DisableExecutionEnvironment: true}} +} +func executorRouter(t *testing.T, owner *reusableExecutor, idle time.Duration) (*dispatch.Router, *recSender, *atomic.Int32) { + t.Helper() + calls := &atomic.Int32{} + reg := agent.NewRegistry() + reg.RegisterKind(proto.SupportedAgentKind{Kind: "reusable", Available: true, Capabilities: proto.AgentKindCapabilities{EnvironmentNone: true}}, func(context.Context, proto.PromptRequestPayload, chan<- proto.Envelope) (agent.Session, error) { + return nil, errors.New("ordinary factory is forbidden") + }) + reg.RegisterExecutor("reusable", func(context.Context, proto.PromptRequestPayload) (agent.Executor, error) { + calls.Add(1) + return owner, nil + }) + sender := &recSender{} + router, err := dispatch.New(dispatch.Config{Registry: reg, Sender: sender, IdleTimeout: idle}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { + owner.mu.Lock() + owner.closeErr = nil + owner.mu.Unlock() + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + if err := router.Shutdown(ctx); err != nil { + t.Error(err) + } + }) + return router, sender, calls +} +func executorAdmission(t *testing.T, r *dispatch.Router, s *recSender, key string, req proto.ExecutionPreparePayload) proto.PreparationStatusPayload { + t.Helper() + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, key, req)); err != nil { + t.Fatal(err) + } + return waitPreparationStatus(t, s, key, "ready", "") +} +func startExecutorTurn(t *testing.T, r *dispatch.Router, s *recSender, key, id string, p proto.PreparationStatusPayload) { + t.Helper() + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, key, proto.ExecutionStartPayload{Handle: p.Handle, ExecutorID: p.ExecutorID, RunID: id, Input: proto.TextInput("input")})); err != nil { + t.Fatal(err) + } + waitPreparationStatus(t, s, key, "started", "") +} +func TestExecutorRetainsOwnerAcrossIndependentTurns(t *testing.T) { + owner := &reusableExecutor{starts: make(chan *reusableTurn, 2)} + r, s, calls := executorRouter(t, owner, time.Minute) + first := executorAdmission(t, r, s, "first", executorRequest()) + startExecutorTurn(t, r, s, "first", "one", first) + turn := <-owner.starts + turn.finish() + waitFor(t, func() bool { return r.ActiveRuns() == 0 }, "first Turn settled") + if owner.closes.Load() != 0 || turn.cancels.Load() != 0 { + t.Fatal("normal completion tore down or cancelled the executor") + } + _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionRelease, "first", proto.ExecutionReleasePayload{Handle: first.Handle})) + req := executorRequest() + req.Configuration.AgentSessionID = "native-session" + req.Configuration.RequireExistingNativeSession = true + second := executorAdmission(t, r, s, "second", req) + if second.Handle == first.Handle || second.ExecutorID != first.ExecutorID || !second.Reused || calls.Load() != 1 { + t.Fatalf("reuse identities: first=%+v second=%+v calls=%d", first, second, calls.Load()) + } + startExecutorTurn(t, r, s, "second", "two", second) + next := <-owner.starts + _ = turn.Cancel(t.Context()) + if next.cancels.Load() != 0 { + t.Fatal("old Turn cancellation reached its successor") + } + next.finish() + waitFor(t, func() bool { return r.ActiveRuns() == 0 }, "second Turn settled") + if owner.closes.Load() != 0 { + t.Fatal("executor closed between Turns") + } +} +func TestExecutorCancelSettlesTurnWithoutClosingOwner(t *testing.T) { + owner := &reusableExecutor{starts: make(chan *reusableTurn, 1)} + r, s, _ := executorRouter(t, owner, time.Minute) + p := executorAdmission(t, r, s, "first", executorRequest()) + startExecutorTurn(t, r, s, "first", "run", p) + turn := <-owner.starts + if err := r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "run", proto.PromptCancelPayload{DeliveryID: "cancel"})); err != nil { + t.Fatal(err) + } + waitFor(t, func() bool { return len(cancellationAcks(s)) == 1 }, "cancellation receipt") + ack := cancellationAcks(s)[0] + if !ack.Applied || ack.Outcome == nil || turn.cancels.Load() != 1 || owner.closes.Load() != 0 { + t.Fatalf("cancel=%+v closes=%d", ack, owner.closes.Load()) + } +} +func TestExecutorRejectsConfigurationAndNativeIdentityChanges(t *testing.T) { + owner := &reusableExecutor{starts: make(chan *reusableTurn, 1)} + r, s, calls := executorRouter(t, owner, time.Minute) + p := executorAdmission(t, r, s, "first", executorRequest()) + startExecutorTurn(t, r, s, "first", "one", p) + (<-owner.starts).finish() + waitFor(t, func() bool { return r.ActiveRuns() == 0 }, "Turn settlement") + for _, kind := range []string{"configuration", "native"} { + req := executorRequest() + if kind == "configuration" { + req.Configuration.AgentOptions = map[string]any{"model": "changed"} + } else { + req.Configuration.AgentSessionID = "other-native" + } + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, kind, req)); err == nil { + t.Fatalf("accepted changed %s", kind) + } + } + if calls.Load() != 1 || owner.closes.Load() != 0 { + t.Fatal("conflict replaced executor") + } +} +func TestExecutorIdleExpiryClosesRetainedOwner(t *testing.T) { + owner := &reusableExecutor{starts: make(chan *reusableTurn, 1)} + r, s, _ := executorRouter(t, owner, 30*time.Millisecond) + p := executorAdmission(t, r, s, "first", executorRequest()) + startExecutorTurn(t, r, s, "first", "one", p) + (<-owner.starts).finish() + waitFor(t, func() bool { return owner.closes.Load() == 1 }, "idle executor close") +} +func TestExecutorPreInputFailureConfirmsCloseBeforeRetrySignal(t *testing.T) { + for _, unconfirmed := range []bool{false, true} { + t.Run(map[bool]string{false: "closed", true: "unconfirmed"}[unconfirmed], func(t *testing.T) { + owner := &reusableExecutor{starts: make(chan *reusableTurn, 1), startErr: errors.New("native exited")} + if unconfirmed { + owner.closeErr = errors.New("cleanup unknown") + } + r, s, _ := executorRouter(t, owner, time.Minute) + p := executorAdmission(t, r, s, "first", executorRequest()) + _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "first", proto.ExecutionStartPayload{Handle: p.Handle, ExecutorID: p.ExecutorID, RunID: "run", Input: proto.TextInput("input")})) + status := waitPreparationStatus(t, s, "first", "rejected", "") + want := "executor_unavailable" + if unconfirmed { + want = "executor_cleanup_unconfirmed" + } + if status.ErrorCode != want || owner.closes.Load() == 0 { + t.Fatalf("status=%+v closes=%d", status, owner.closes.Load()) + } + if unconfirmed { + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "retry", executorRequest())); err == nil { + t.Fatal("replaced unconfirmed owner") + } + } + }) + } +} + +func poolRouter(t *testing.T, factory agent.ExecutorFactory) (*dispatch.Router, *recSender) { + t.Helper() + registry := agent.NewRegistry() + registry.RegisterKind(proto.SupportedAgentKind{Kind: "reusable", Available: true, Capabilities: proto.AgentKindCapabilities{EnvironmentNone: true}}, func(context.Context, proto.PromptRequestPayload, chan<- proto.Envelope) (agent.Session, error) { + return nil, errors.New("unexpected legacy factory") + }) + registry.RegisterExecutor("reusable", factory) + sender := &recSender{} + router, err := dispatch.New(dispatch.Config{Registry: registry, Sender: sender, IdleTimeout: time.Minute}) + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { + ctx, cancel := context.WithTimeout(context.Background(), time.Second) + defer cancel() + if err := router.Shutdown(ctx); err != nil { + t.Error(err) + } + }) + return router, sender +} +func poolRequest(id string) proto.ExecutionPreparePayload { + req := executorRequest() + req.SessionID = id + req.Configuration.AgentStateKey = "agents-api-" + id + return req +} + +func TestExecutorIdleAndActiveCapacitiesAreIndependent(t *testing.T) { + var mu sync.Mutex + var owners []*reusableExecutor + r, s := poolRouter(t, func(context.Context, proto.PromptRequestPayload) (agent.Executor, error) { + e := &reusableExecutor{} + mu.Lock() + owners = append(owners, e) + mu.Unlock() + return e, nil + }) + for i := 0; i < 17; i++ { + id := fmt.Sprint("idle-", i) + ready := executorAdmission(t, r, s, id, poolRequest(id)) + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionRelease, id, proto.ExecutionReleasePayload{Handle: ready.Handle})); err != nil { + t.Fatal(err) + } + } + waitFor(t, func() bool { + mu.Lock() + defer mu.Unlock() + var closed int32 + for _, e := range owners { + closed += e.closes.Load() + } + return closed == 1 + }, "idle capacity eviction") + for i := 0; i < 4; i++ { + id := fmt.Sprint("active-", i) + executorAdmission(t, r, s, id, poolRequest(id)) + } + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "overflow", poolRequest("overflow"))); err == nil { + t.Fatal("active capacity exceeded") + } +} + +func TestExecutorRejectsSessionStateScopeMismatch(t *testing.T) { + e := &reusableExecutor{} + r, s, calls := executorRouter(t, e, time.Minute) + executorAdmission(t, r, s, "one", executorRequest()) + req := executorRequest() + req.SessionID = "another" + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "scope", req)); err == nil { + t.Fatal("two sessions shared native state") + } + if calls.Load() != 1 { + t.Fatal("mismatched session invoked native preparation") + } +} + +func TestExecutorRejectsOutputFromAnotherTurn(t *testing.T) { + e := &reusableExecutor{starts: make(chan *reusableTurn, 1)} + r, s, _ := executorRouter(t, e, time.Minute) + p := executorAdmission(t, r, s, "one", executorRequest()) + startExecutorTurn(t, r, s, "one", "current", p) + turn := <-e.starts + wrong, _ := proto.NewEnvelope(proto.TypeDelta, "old-turn", proto.DeltaPayload{Delta: "late"}) + turn.out <- wrong + turn.finish() + waitFor(t, func() bool { return r.ActiveRuns() == 0 }, "invalid native event cleanup") + if e.closes.Load() != 1 { + t.Fatal("invalid event left executor reusable") + } + for _, frame := range s.snapshot() { + if frame.ID == "old-turn" { + t.Fatal("old Turn output escaped") + } + } +} diff --git a/apps/parsar-daemon/internal/dispatch/functions_native_test.go b/apps/parsar-daemon/internal/dispatch/functions_native_test.go index b47d0aec9..9502539ac 100644 --- a/apps/parsar-daemon/internal/dispatch/functions_native_test.go +++ b/apps/parsar-daemon/internal/dispatch/functions_native_test.go @@ -40,6 +40,9 @@ func TestNativeFunctionBridge(t *testing.T) { if err != nil { t.Fatal(err) } + if err := os.Chmod(home, 0o700); err != nil { + t.Fatal(err) + } t.Setenv("OAC_RUNTIME_HOME", home) var count atomic.Int32 model := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { diff --git a/apps/parsar-daemon/internal/dispatch/local_directory_test.go b/apps/parsar-daemon/internal/dispatch/local_directory_test.go index 0515a59d6..56605ca58 100644 --- a/apps/parsar-daemon/internal/dispatch/local_directory_test.go +++ b/apps/parsar-daemon/internal/dispatch/local_directory_test.go @@ -57,7 +57,7 @@ func TestLocalDirectoryPreparationNeedsNoHarnessAndRejectsOtherOwners(t *testing if got := waitWorkspaceRead(t, sender, "foreign"); got.Outcome != "rejected" { t.Fatal("foreign directory accepted", got) } - if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "idle", proto.ExecutionStartPayload{Handle: ready.Handle, RunID: "forbidden", Input: proto.TextInput("work")})); err == nil { + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "idle", proto.ExecutionStartPayload{Handle: ready.Handle, ExecutorID: ready.ExecutorID, RunID: "forbidden", Input: proto.TextInput("work")})); err == nil { t.Fatal("read preparation admitted execution") } bad := request diff --git a/apps/parsar-daemon/internal/dispatch/mcp_http_test.go b/apps/parsar-daemon/internal/dispatch/mcp_http_test.go index 127a7f96d..13bb7e76e 100644 --- a/apps/parsar-daemon/internal/dispatch/mcp_http_test.go +++ b/apps/parsar-daemon/internal/dispatch/mcp_http_test.go @@ -102,7 +102,7 @@ func TestLocalMCPRejectsBeforePreparationFactory(t *testing.T) { t.Error("ordinary factory called") return nil, errors.New("unexpected") }) - h.reg.RegisterPreparation("prepared", false, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { + h.reg.RegisterExecutor("prepared", func(context.Context, proto.PromptRequestPayload) (agent.Executor, error) { entered <- struct{}{} return nil, errors.New("controlled stop") }) diff --git a/apps/parsar-daemon/internal/dispatch/preparation.go b/apps/parsar-daemon/internal/dispatch/preparation.go index acfe25be6..dd218cd88 100644 --- a/apps/parsar-daemon/internal/dispatch/preparation.go +++ b/apps/parsar-daemon/internal/dispatch/preparation.go @@ -19,6 +19,7 @@ const preparationRecords = 64 // All mutable fields are protected by Router.mu. owns includes resources whose // cancellation is underway; a slow close cannot bypass the capacity bound. type preparationState struct { + executor *executorState requestID string trace string fingerprint [32]byte @@ -44,6 +45,9 @@ func (r *Router) handleExecutionPrepare(ctx context.Context, env proto.Envelope) return r.rejectPreparation(env, "invalid_request") } req := input.Configuration + if !req.WorkspaceReadOnly { + return r.handleExecutorPrepare(ctx, env, input) + } caps := r.availableCapabilities(req.AgentKind) prepare, err := r.registry.ResolvePreparation(req.AgentKind) if err != nil || !caps.Preparation { @@ -74,7 +78,7 @@ func (r *Router) handleExecutionPrepare(ctx context.Context, env proto.Envelope) r.mu.Unlock() return ErrRouterClosed } - if (r.workspaceWrite != nil || r.workspaceExport != nil) && !req.WorkspaceReadOnly { + if r.runtimePreparation != nil || ((r.workspaceWrite != nil || r.workspaceExport != nil) && !req.WorkspaceReadOnly) { r.mu.Unlock() return r.rejectPreparation(env, "resource_unavailable") } @@ -179,6 +183,11 @@ func (r *Router) releasePreparation(p *preparationState, state, code string, pub r.mu.Unlock() return } + if p.executor != nil { + r.mu.Unlock() + r.abandonExecutorAdmission(p, state, code, publish) + return + } if p.handoff != nil { if active := r.sessions[p.status.RunID]; active != nil && active.preparedHandoff == p.handoff { if state == "expired" && p.handoff.published { diff --git a/apps/parsar-daemon/internal/dispatch/preparation_cancel_test.go b/apps/parsar-daemon/internal/dispatch/preparation_cancel_test.go index d5fd98ec6..ecba30e74 100644 --- a/apps/parsar-daemon/internal/dispatch/preparation_cancel_test.go +++ b/apps/parsar-daemon/internal/dispatch/preparation_cancel_test.go @@ -3,7 +3,6 @@ package dispatch_test import ( "context" "errors" - "fmt" "reflect" "sync" "sync/atomic" @@ -49,7 +48,7 @@ func startCancellationPreparation(t *testing.T, r *dispatch.Router, sender *recS t.Fatal(err) } ready := waitPreparationStatus(t, sender, "request", "ready", "") - if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, RunID: "run", Input: proto.TextInput("input")})); err != nil { + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, ExecutorID: ready.ExecutorID, RunID: "run", Input: proto.TextInput("input")})); err != nil { t.Fatal(err) } return ready @@ -167,7 +166,7 @@ func TestPreparedCancellationWaitsForOutputAndCleanup(t *testing.T) { } func TestPreparedCancellationBeforeTransferPreservesUnknownOutcome(t *testing.T) { - for _, boundary := range []string{"start_failure", "release", "expiry", "shutdown"} { + for _, boundary := range []string{"start_failure", "release", "shutdown"} { t.Run(boundary, func(t *testing.T) { sender := &recSender{} entered, cancelled, allowReturn := make(chan struct{}), make(chan struct{}), make(chan struct{}) @@ -195,7 +194,7 @@ func TestPreparedCancellationBeforeTransferPreservesUnknownOutcome(t *testing.T) switch boundary { case "release": _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionRelease, "request", proto.ExecutionReleasePayload{Handle: ready.Handle})) - waitPreparationStatus(t, sender, "request", "released", "") + // An admission release does not terminate a handed-off Turn. case "expiry": waitPreparationStatus(t, sender, "request", "expired", "") case "shutdown": @@ -248,6 +247,9 @@ func TestPreparedCancellationFailuresRemainConservative(t *testing.T) { startCancellationPreparation(t, r, sender.recSender) <-entered _ = r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "run", proto.PromptCancelPayload{DeliveryID: "cancel"})) + if failure == "cancel_failed" { + waitFor(t, func() bool { return p.calls.Load() == 1 }, "failed pre-transfer cleanup") + } close(allowReturn) waitFor(t, func() bool { return len(cancellationAcks(sender.recSender)) == 1 }, "failed cancellation") ack := cancellationAcks(sender.recSender)[0] @@ -255,17 +257,21 @@ func TestPreparedCancellationFailuresRemainConservative(t *testing.T) { t.Fatalf("uncertain cancellation accepted: %+v", ack) } if failure == "cancel_failed" { - if r.ActiveRuns() != 1 || session.cancels() != 0 { - t.Fatal("failed attempt released ownership or used a fallback target") + waitFor(t, func() bool { p.mu.Lock(); defer p.mu.Unlock(); return p.session != nil }, "late Turn returned") + if r.ActiveRuns() != 1 { + t.Fatal("unconfirmed cleanup lost owner") } _ = r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "run", proto.PromptCancelPayload{DeliveryID: "retry"})) - waitFor(t, func() bool { return len(cancellationAcks(sender.recSender)) == 2 && r.ActiveRuns() == 0 }, "same-target cancellation retry") - if retry := cancellationAcks(sender.recSender)[1]; !retry.Applied || retry.ErrorCode != "" || session.cancels() != 1 || p.calls.Load() != 2 { - t.Fatalf("same target was not retried exactly once: ack=%+v calls=%d session=%d", retry, p.calls.Load(), session.cancels()) + waitFor(t, func() bool { return len(cancellationAcks(sender.recSender)) == 2 }, "cleanup retry receipt") + if !cancellationAcks(sender.recSender)[1].Applied { + t.Fatal("confirmed retry was rejected") } - } else if r.ActiveRuns() != 0 || session.cancels() != 1 { - t.Fatal("successful cleanup did not settle output failure") } + waitFor(t, func() bool { return r.ActiveRuns() == 0 }, "confirmed resource cleanup") + if session.cancels() != 1 { + t.Fatal("confirmed cleanup did not stop the native Turn") + } + }) } } @@ -310,11 +316,7 @@ func TestPreparedCancellationTimeoutKeepsCapacityUntilStartReturns(t *testing.T) }) startCancellationPreparation(t, r, sender) <-entered - for i := 0; i < 3; i++ { - id := fmt.Sprint(i) - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, id, preparationRequest())) - waitPreparationStatus(t, sender, id, "ready", "") - } + _ = r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "run", proto.PromptCancelPayload{DeliveryID: "cancel"})) deadline := time.Now().Add(12 * time.Second) for len(cancellationAcks(sender)) == 0 && time.Now().Before(deadline) { diff --git a/apps/parsar-daemon/internal/dispatch/preparation_cleanup.go b/apps/parsar-daemon/internal/dispatch/preparation_cleanup.go index 2c3cd176b..2a51c0e8c 100644 --- a/apps/parsar-daemon/internal/dispatch/preparation_cleanup.go +++ b/apps/parsar-daemon/internal/dispatch/preparation_cleanup.go @@ -32,6 +32,9 @@ func (r *Router) closePendingPreparationsLocked() []*preparationState { var closeNow []*preparationState for _, p := range r.preparations { p.timer.Stop() + if p.executor != nil { + continue + } if !p.owns { continue } diff --git a/apps/parsar-daemon/internal/dispatch/preparation_cleanup_test.go b/apps/parsar-daemon/internal/dispatch/preparation_cleanup_test.go index 7c08293d6..a36355bd5 100644 --- a/apps/parsar-daemon/internal/dispatch/preparation_cleanup_test.go +++ b/apps/parsar-daemon/internal/dispatch/preparation_cleanup_test.go @@ -55,7 +55,7 @@ func TestPreparedCancellationDoesNotAcknowledgeFailedCleanup(t *testing.T) { if owned, _ := r.PreparationOwnershipForTest(ready.Handle); !owned { t.Fatal("cancel receipt discarded unsettled preparation") } - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionRelease, "request", proto.ExecutionReleasePayload{Handle: ready.Handle})) + _ = r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "run", proto.PromptCancelPayload{DeliveryID: "cancel-retry"})) waitFor(t, func() bool { owned, _ := r.PreparationOwnershipForTest(ready.Handle); return !owned }, "retained cancellation cleanup") } @@ -69,7 +69,7 @@ func TestShutdownRetriesFailedPreparedCancellationOnSameTarget(t *testing.T) { return session, nil } p.cancel = func(ctx context.Context) error { - if p.calls.Load() == 1 { + if p.calls.Load() <= 2 { return want } return session.Cancel(ctx) @@ -80,7 +80,7 @@ func TestShutdownRetriesFailedPreparedCancellationOnSameTarget(t *testing.T) { if err := r.Shutdown(t.Context()); !errors.Is(err, want) { t.Fatalf("first shutdown lost native failure: %v", err) } - if r.ActiveRuns() != 1 || p.calls.Load() != 1 || session.cancels() != 0 { + if r.ActiveRuns() != 1 || p.calls.Load() != 2 || session.cancels() != 0 { t.Fatal("failed shutdown released ownership or changed release target") } select { @@ -91,78 +91,11 @@ func TestShutdownRetriesFailedPreparedCancellationOnSameTarget(t *testing.T) { if err := r.Shutdown(t.Context()); err != nil { t.Fatal(err) } - if r.ActiveRuns() != 0 || p.calls.Load() != 2 || session.cancels() != 1 { + if r.ActiveRuns() != 0 || p.calls.Load() != 3 || session.cancels() != 1 { t.Fatalf("shutdown did not retry the same target once: active=%d target=%d session=%d", r.ActiveRuns(), p.calls.Load(), session.cancels()) } } -func TestPreparationCloseFailureRetainsCapacityAndRetries(t *testing.T) { - sender := &recSender{} - entered, release := make(chan struct{}), make(chan struct{}) - var once sync.Once - unblock := func() { once.Do(func() { close(release) }) } - p := &retryablePreparation{close: func(call int32) error { - if call == 1 { - return errors.New("cleanup incomplete") - } - if call == 2 { - close(entered) - <-release - } - return nil - }} - var factories atomic.Int32 - r := preparationRouter(t, sender, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { - if factories.Add(1) == 1 { - return p, nil - } - return &controlledPreparation{closed: make(chan struct{})}, nil - }) - t.Cleanup(unblock) - var handle string - for i := range 4 { - id := fmt.Sprint(i) - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, id, preparationRequest())) - ready := waitPreparationStatus(t, sender, id, "ready", "") - if i == 0 { - handle = ready.Handle - } - } - request := mustEnv(t, proto.TypeExecutionRelease, "0", proto.ExecutionReleasePayload{Handle: handle}) - _ = r.Handle(t.Context(), request) - waitFor(t, func() bool { _, busy := r.PreparationOwnershipForTest(handle); return p.calls.Load() == 1 && !busy }, "failed cleanup return") - if owned, _ := r.PreparationOwnershipForTest(handle); !owned { - t.Fatal("failed cleanup discarded ownership") - } - replacement := mustEnv(t, proto.TypeExecutionPrepare, "replacement", preparationRequest()) - if err := r.Handle(t.Context(), replacement); err == nil { - t.Fatal("failed cleanup released capacity") - } - start := proto.ExecutionStartPayload{Handle: handle, RunID: "run", Input: proto.TextInput("do not execute")} - if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "0", start)); err == nil { - t.Fatal("failed cleanup allowed Start") - } - _ = r.Handle(t.Context(), request) - select { - case <-entered: - case <-time.After(time.Second): - t.Fatal("release did not retry retained resource") - } - _ = r.Handle(t.Context(), request) - if p.calls.Load() != 2 { - t.Fatal("release started overlapping cleanup") - } - unblock() - waitFor(t, func() bool { owned, _ := r.PreparationOwnershipForTest(handle); return !owned }, "successful cleanup") - if err := r.Handle(t.Context(), replacement); err != nil { - t.Fatalf("settled cleanup retained capacity: %v", err) - } - waitPreparationStatus(t, sender, "replacement", "ready", "") - if p.starts.Load() != 0 { - t.Fatal("cleanup restarted native execution") - } -} - func TestShutdownRetriesFailedPreparationCleanup(t *testing.T) { want := errors.New("native cleanup incomplete") p := &retryablePreparation{close: func(call int32) error { @@ -174,18 +107,16 @@ func TestShutdownRetriesFailedPreparationCleanup(t *testing.T) { sender := &recSender{} r := preparationRouter(t, sender, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { return p, nil }) _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "request", preparationRequest())) - handle := waitPreparationStatus(t, sender, "request", "ready", "").Handle + waitPreparationStatus(t, sender, "request", "ready", "") if err := r.Shutdown(t.Context()); !errors.Is(err, want) { t.Fatalf("shutdown lost cleanup error: %v", err) } - if owned, _ := r.PreparationOwnershipForTest(handle); !owned { - t.Fatal("shutdown discarded unsettled resource") - } + if err := r.Shutdown(t.Context()); err != nil { t.Fatal(err) } - if owned, _ := r.PreparationOwnershipForTest(handle); owned || p.calls.Load() != 2 { - t.Fatalf("retry did not settle original resource: owned=%t calls=%d", owned, p.calls.Load()) + if p.calls.Load() != 2 { + t.Fatalf("retry did not settle original resource: calls=%d", p.calls.Load()) } } @@ -239,7 +170,7 @@ func TestPublishedPreparedRunRetainsRetryAfterHandleRetirement(t *testing.T) { return session, nil } p.cancel = func(ctx context.Context) error { - if p.calls.Load() == 1 { + if p.calls.Load() <= 2 { return errors.New("cleanup incomplete") } return session.Cancel(ctx) @@ -260,7 +191,7 @@ func TestPublishedPreparedRunRetainsRetryAfterHandleRetirement(t *testing.T) { } // A later admission retires the expired preparation record. The Run still // owns the exact cancellation target independently of that old handle. - if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "replacement", preparationRequest())); err != nil { + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "replacement", readOnlyPreparationRequest())); err != nil { t.Fatal(err) } waitPreparationStatus(t, sender, "replacement", "ready", "") @@ -278,7 +209,7 @@ func TestPublishedPreparedRunRetainsRetryAfterHandleRetirement(t *testing.T) { } } } - if p.calls.Load() != 2 || session.cancels() != 1 || r.ActiveRuns() != 0 { + if p.calls.Load() != 3 || session.cancels() != 1 || r.ActiveRuns() != 0 { t.Fatal("Run cancellation did not retry and settle the same target") } select { @@ -287,3 +218,9 @@ func TestPublishedPreparedRunRetainsRetryAfterHandleRetirement(t *testing.T) { default: } } + +func readOnlyPreparationRequest() proto.ExecutionPreparePayload { + req := preparationRequest() + req.Configuration.WorkspaceReadOnly = true + return req +} diff --git a/apps/parsar-daemon/internal/dispatch/preparation_executor_fixture_test.go b/apps/parsar-daemon/internal/dispatch/preparation_executor_fixture_test.go new file mode 100644 index 000000000..c82ac841e --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/preparation_executor_fixture_test.go @@ -0,0 +1,133 @@ +package dispatch_test + +import ( + "context" + "errors" + "sync" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +// Old fault-injection fixtures model disposable executors, not reusable native implementations. +func preparationExecutorFixture(factory agent.PreparationFactory) agent.ExecutorFactory { + return func(ctx context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + prepared, err := factory(ctx, req) + if prepared == nil { + return nil, err + } + owner := &preparationExecutor{prepared: prepared} + if _, ok := prepared.(agent.PreparedCancellation); !ok { + return owner, errors.New("fixture has no cancellation target") + } + return owner, err + } +} + +type preparationExecutor struct { + prepared agent.Prepared + mu sync.Mutex + cancelled bool +} + +func (e *preparationExecutor) Close(ctx context.Context) error { + e.mu.Lock() + defer e.mu.Unlock() + if e.cancelled { + return nil + } + var err error + if resource, ok := e.prepared.(*retryablePreparation); ok { + err = resource.Close() + } else if cancel, ok := e.prepared.(agent.PreparedCancellation); ok { + err = cancel.Cancel(ctx) + } else { + err = e.prepared.Close() + } + e.cancelled = err == nil + return err +} + +func (e *preparationExecutor) StartTurn(ctx context.Context, id string, input proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + upstream := make(chan proto.Envelope, 64) + terminal := make(chan struct{}) + var once sync.Once + go func() { + defer close(out) + defer once.Do(func() { close(terminal) }) + for event := range upstream { + if event.Type == proto.TypeDone { + once.Do(func() { close(terminal) }) + } + out <- event + } + }() + session, err := e.prepared.Start(ctx, id, input, upstream) + if session == nil { + close(upstream) + // This fixture's old Start may have produced output before rejecting; retain its Turn. + return &preparationTurn{owner: e, settled: terminal}, err + } + return &preparationTurn{Session: session, owner: e, settled: terminal}, err +} + +type preparationTurn struct { + agent.Session + owner *preparationExecutor + settled <-chan struct{} +} + +func (t *preparationTurn) Cancel(ctx context.Context) error { return t.owner.Close(ctx) } +func (t *preparationTurn) CancellationOutcome() proto.DonePayload { + if target, ok := t.owner.prepared.(agent.PreparedCancellation); ok { + return target.CancellationOutcome() + } + return proto.DonePayload{} +} +func (t *preparationTurn) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + select { + case <-t.settled: + return agent.TurnSettlement{Reason: "disposable fault fixture"}, t.owner.Close(ctx) + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} +func (t *preparationTurn) SubmitFunctionResult(ctx context.Context, p proto.FunctionResultPayload) error { + if target, ok := t.Session.(agent.FunctionResultSubmitter); ok { + return target.SubmitFunctionResult(ctx, p) + } + return agent.ErrUnknownFunctionCall +} +func (t *preparationTurn) Steer(ctx context.Context, p proto.PromptSteerPayload) error { + if target, ok := t.Session.(agent.Steerer); ok { + return target.Steer(ctx, p) + } + return agent.ErrSteeringInactive +} +func (t *preparationTurn) SteerWithReceipt(ctx context.Context, p proto.PromptSteerPayload, write func()) error { + if target, ok := t.Session.(agent.DurableSteerer); ok { + return target.SteerWithReceipt(ctx, p, write) + } + if target, ok := t.Session.(agent.Steerer); ok { + return target.Steer(ctx, p) + } + return agent.ErrSteeringInactive +} +func (t *preparationTurn) SubmitPermission(ctx context.Context, id string, p proto.PermissionDecisionPayload) error { + if target, ok := t.Session.(agent.PermissionResponder); ok { + return target.SubmitPermission(ctx, id, p) + } + return agent.ErrUnknownPermission +} +func (t *preparationTurn) SubmitPromptForUserChoice(ctx context.Context, id string, p proto.PromptForUserChoiceDecisionPayload) error { + if target, ok := t.Session.(agent.UserChoiceResponder); ok { + return target.SubmitPromptForUserChoice(ctx, id, p) + } + return agent.ErrUnknownAsk +} +func (t *preparationTurn) ReadWorkspaceFile(ctx context.Context, path string, limit int) (agent.WorkspaceReadResult, error) { + if target, ok := t.Session.(agent.WorkspaceReader); ok { + return target.ReadWorkspaceFile(ctx, path, limit) + } + return agent.WorkspaceReadResult{}, agent.ErrWorkspaceReadUnsupported +} diff --git a/apps/parsar-daemon/internal/dispatch/preparation_start.go b/apps/parsar-daemon/internal/dispatch/preparation_start.go index 0607d407f..74d267cac 100644 --- a/apps/parsar-daemon/internal/dispatch/preparation_start.go +++ b/apps/parsar-daemon/internal/dispatch/preparation_start.go @@ -14,17 +14,17 @@ import ( func (r *Router) handleExecutionStart(_ context.Context, env proto.Envelope) error { var input proto.ExecutionStartPayload - if env.DecodePayload(&input) != nil || input.Handle == "" || strings.TrimSpace(input.RunID) == "" || input.Input.Validate() != nil { + if env.DecodePayload(&input) != nil || input.Handle == "" || input.ExecutorID == "" || strings.TrimSpace(input.RunID) == "" || input.Input.Validate() != nil { return r.rejectPreparation(env, "invalid_start") } encoded, _ := json.Marshal(input) fingerprint := sha256.Sum256(encoded) r.mu.Lock() - if r.closed { + if r.closed || r.suspension != nil { r.mu.Unlock() return ErrRouterClosed } - if r.workspaceExport != nil { + if r.workspaceWrite != nil || r.workspaceExport != nil || r.runtimePreparation != nil { r.mu.Unlock() return r.rejectPreparation(env, "resource_unavailable") } @@ -37,23 +37,25 @@ func (r *Router) handleExecutionStart(_ context.Context, env proto.Envelope) err r.mu.Unlock() return r.rejectPreparation(env, "read_only_preparation") } + owner := p.executor + if owner == nil || owner.id != input.ExecutorID { + r.mu.Unlock() + return r.rejectPreparation(env, "unknown_executor") + } if p.status.State == "starting" || p.status.State == "started" { matches, status := p.startFingerprint == fingerprint, p.status - if matches { - if state := r.sessions[input.RunID]; state != nil && state.preparedHandoff != nil && - (state.session == nil || state.preparedHandoff.release != nil) { - r.mu.Unlock() - return nil - } - } + pending := p.handoff != nil && !p.handoff.published r.mu.Unlock() if !matches { return r.rejectPreparation(env, "start_conflict") } + if pending { + return nil + } r.publishPreparation(p, status) return nil } - if p.status.State != "ready" || p.ctx.Err() != nil { + if p.status.State != "ready" || p.ctx.Err() != nil || owner.admission != p || owner.run != nil || owner.invalid || owner.native == nil { r.mu.Unlock() return r.rejectPreparation(env, "preparation_not_ready") } @@ -66,24 +68,13 @@ func (r *Router) handleExecutionStart(_ context.Context, env proto.Envelope) err r.mu.Unlock() return r.rejectPreparation(env, "run_conflict") } - target, ok := p.prepared.(agent.PreparedCancellation) - if !ok { - r.mu.Unlock() - return r.rejectPreparation(env, "preparation_not_ready") - } p.status.State, p.status.RunID, p.status.Revision = "starting", input.RunID, p.status.Revision+1 p.startFingerprint, p.busy = fingerprint, true - state := &sessionState{ - runID: input.RunID, stateKey: p.stateKey, environmentID: p.environmentID, - out: make(chan proto.Envelope, 64), ctx: p.ctx, ctxCancel: p.cancel, - pendingIDs: make(map[string]struct{}), pendingAsks: make(map[string]struct{}), - traceparent: env.Trace, releaseOnCompletion: true, - } - state.preparedHandoff = newPreparedHandoff(p, target) - p.handoff = state.preparedHandoff + state := &sessionState{runID: input.RunID, stateKey: p.stateKey, environmentID: p.environmentID, out: make(chan proto.Envelope, 64), ctx: p.ctx, ctxCancel: p.cancel, pendingIDs: make(map[string]struct{}), pendingAsks: make(map[string]struct{}), traceparent: env.Trace} + state.preparedHandoff = newPreparedHandoff(p, owner.native) + p.handoff, owner.run = state.preparedHandoff, state r.sessions[input.RunID] = state status := p.status - // Track the Start owner and output consumer before either can publish. r.shutdownWG.Add(2) r.mu.Unlock() go r.startPreparedExecution(p, state, input, status) @@ -95,100 +86,74 @@ func (r *Router) startPreparedExecution(p *preparationState, state *sessionState handoff := state.preparedHandoff go r.forwardPreparedOutput(state) <-handoff.outputReady - - if !r.sendPreparation(p.requestID, p.trace, starting) { - r.mu.Lock() - handoff.outputErr = errors.Join(handoff.outputErr, errPreparedStatusDelivery) - if p.status.State == "starting" { - p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "status_delivery_failed", p.status.Revision+1 - } - p.timer.Stop() - r.claimPreparedReleaseLocked(state, true, "", false) - r.mu.Unlock() - close(state.out) - close(handoff.startDone) - return - } - - // A release admitted before this point must not cause native work to start. + delivered := r.sendPreparation(p.requestID, p.trace, starting) r.mu.Lock() - blocked := handoff.release != nil && handoff.release.aborted() + blocked := !delivered || handoff.release != nil && handoff.release.aborted() || p.ctx.Err() != nil r.mu.Unlock() - var session agent.Session + var turn agent.Turn var startErr error if blocked { startErr = context.Canceled } else { - session, startErr = handoff.target.Start(p.ctx, input.RunID, input.Input, state.out) + turn, startErr = handoff.target.StartTurn(p.ctx, input.RunID, input.Input, state.out) } - r.mu.Lock() - release := handoff.release - notAborted := release == nil || !release.aborted() - started := startErr == nil && session != nil && p.status.State == "starting" && p.ctx.Err() == nil && notAborted - if started { + handoff.turn = turn + handoff.startErr = startErr + p.timer.Stop() + p.busy = false + if turn != nil { p.status.State, p.status.ErrorCode, p.status.Revision = "started", "", p.status.Revision+1 } else { - p.timer.Stop() - if p.status.State == "starting" { - p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "start_failed", p.status.Revision+1 - } - r.claimPreparedReleaseLocked(state, true, "prepared execution could not start", false) + p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "executor_unavailable", p.status.Revision+1 } status := p.status - // Keep release behind successful or failed status publication. This also - // orders an early native Done after the started status. - if started { - handoff.operations.RLock() - } r.mu.Unlock() - - // A nil Session leaves output ownership with the Router. A non-nil Session - // owns the close even when Start also returned an error. - if session == nil { + if turn == nil { close(state.out) - } - deadline := time.Now().Add(5 * time.Second) - if started && p.deadline.Before(deadline) { - deadline = p.deadline - } - delivered := r.sendPreparationUntil(p.requestID, p.trace, status, deadline) - - r.mu.Lock() - if started && !time.Now().Before(p.deadline) { - if p.status.State == "started" { - p.status.State, p.status.ErrorCode, p.status.Revision = "expired", "", p.status.Revision+1 + code := "executor_unavailable" + closeErr := r.closeExecutor(p.executor) + if closeErr != nil { + code = "executor_cleanup_unconfirmed" + } + if blocked { + code = "start_cancelled" + } + r.mu.Lock() + p.status.ErrorCode = code + release, _ := r.claimPreparedReleaseLocked(state, true, "", false) + close(handoff.startDone) + r.mu.Unlock() + if closeErr == nil { + <-release.settled + } + status = proto.PreparationStatusPayload{Handle: p.status.Handle, ExecutorID: p.executor.id, State: "rejected", ErrorCode: code, Operation: proto.TypeExecutionStart} + if delivered { + r.sendPreparation(p.requestID, p.trace, status) } - r.claimPreparedReleaseLocked(state, true, "", false) + return } - if started && handoff.release != nil && handoff.release.aborted() && p.status.State == "started" { - p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "start_failed", p.status.Revision+1 + r.mu.Lock() + aborted := handoff.release != nil && handoff.release.aborted() + r.mu.Unlock() + if aborted { + status.State, status.ErrorCode = "failed", "start_cancelled" } - if !delivered { + delivered = delivered && !aborted && r.sendPreparationUntil(p.requestID, p.trace, status, p.deadline) + r.mu.Lock() + if !delivered && !aborted { handoff.outputErr = errors.Join(handoff.outputErr, errPreparedStatusDelivery) - if started && p.status.State == "started" { - p.status.State, p.status.ErrorCode, p.status.Revision = "failed", "status_delivery_failed", p.status.Revision+1 - } - r.claimPreparedReleaseLocked(state, true, "", false) - } else if started && p.status.State == "started" && - (handoff.release == nil || !handoff.release.aborted()) { - handoff.published = true - p.timer.Stop() } + handoff.published = turn != nil && delivered if handoff.published { - // Publication transfers resource tracking even when natural completion - // has already closed input admission. - if handoff.release == nil { - state.session = session - } - p.prepared = nil p.owns = false - p.busy = false - p.handoff = nil + } + if handoff.published && handoff.release == nil { + state.session = turn + } + if turn == nil || startErr != nil || !delivered { + r.claimPreparedReleaseLocked(state, true, "prepared execution could not start", false) } close(handoff.startDone) r.mu.Unlock() - if started { - handoff.operations.RUnlock() - } } diff --git a/apps/parsar-daemon/internal/dispatch/preparation_test.go b/apps/parsar-daemon/internal/dispatch/preparation_test.go index 4dee25e40..2304ca1a2 100644 --- a/apps/parsar-daemon/internal/dispatch/preparation_test.go +++ b/apps/parsar-daemon/internal/dispatch/preparation_test.go @@ -3,7 +3,6 @@ package dispatch_test import ( "context" "errors" - "fmt" "os" "path/filepath" "sync" @@ -14,6 +13,7 @@ import ( "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/dispatch" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/localworkspace" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" ) @@ -32,7 +32,9 @@ func (p *controlledPreparation) Close() error { if p.closeHook != nil { p.closeHook() } - close(p.closed) + if p.closed != nil { + close(p.closed) + } }) return nil } @@ -76,16 +78,22 @@ func preparationWorkspace(t *testing.T) *localworkspace.Binding { if err := os.WriteFile(helper, []byte("#!/bin/sh\nprintf '%s' '{\"version\":1,\"directory\":{\"entries\":[{\"name\":\"file\",\"kind\":\"file\",\"size_bytes\":3}],\"truncated\":true}}'\n"), 0o700); err != nil { t.Fatal(err) } + runtimeHome := t.TempDir() + if err := os.Chmod(runtimeHome, 0o700); err != nil { + t.Fatal(err) + } for name, value := range map[string]string{ - "OAC_RUNTIME_ENVIRONMENT_ID": preparationEnvironmentID, - "OAC_RUNTIME_SESSION_ID": preparationSessionID, - "OAC_RUNTIME_WORKSPACE": t.TempDir(), - "OAC_RUNTIME_DIRECTORY_HELPER": helper, - "OAC_RUNTIME_NETWORK_ACCESS": "enabled", - "OAC_RUNTIME_ALLOWED_DOMAINS": "", - "OAC_RUNTIME_WRITE_HELPER": "", - "OAC_RUNTIME_EXPORT_HELPER": "", - "OAC_RUNTIME_STAGING": "", + "OAC_RUNTIME_ENVIRONMENT_ID": preparationEnvironmentID, + "OAC_RUNTIME_SESSION_ID": preparationSessionID, + "OAC_RUNTIME_WORKSPACE": t.TempDir(), + "OAC_RUNTIME_CAPABILITY_DIRECTORY": t.TempDir(), + "OAC_RUNTIME_HOME": runtimeHome, + "OAC_RUNTIME_DIRECTORY_HELPER": helper, + "OAC_RUNTIME_NETWORK_ACCESS": "enabled", + "OAC_RUNTIME_ALLOWED_DOMAINS": "", + "OAC_RUNTIME_WRITE_HELPER": "", + "OAC_RUNTIME_EXPORT_HELPER": "", + "OAC_RUNTIME_STAGING": "", } { t.Setenv(name, value) } @@ -111,7 +119,7 @@ func localPreparationHarness(t *testing.T) *harness { } func preparationRequest() proto.ExecutionPreparePayload { - return proto.ExecutionPreparePayload{Configuration: proto.PromptRequestPayload{AgentKind: "prepared", AgentStateKey: "agents-api-" + preparationSessionID, StrictResume: true, ReleaseOnCompletion: true, LocalEnvironment: &proto.LocalEnvironment{ID: preparationEnvironmentID, NetworkAccess: "enabled"}}} + return proto.ExecutionPreparePayload{SessionID: preparationSessionID, Configuration: proto.PromptRequestPayload{AgentKind: "prepared", AgentStateKey: "agents-api-" + preparationSessionID, StrictResume: true, ReleaseOnCompletion: true, LocalEnvironment: &proto.LocalEnvironment{ID: preparationEnvironmentID, NetworkAccess: "enabled", WorkspaceDirectory: "/workspace", CapabilitySources: &agentcapabilities.Input{}}}} } func preparationRouter(t *testing.T, sender dispatch.Sender, timeout time.Duration, factory agent.PreparationFactory) *dispatch.Router { @@ -121,6 +129,7 @@ func preparationRouter(t *testing.T, sender dispatch.Sender, timeout time.Durati return nil, errors.New("ordinary Factory must not be used for preparation") }) reg.RegisterPreparation("prepared", true, factory) + reg.RegisterExecutor("prepared", preparationExecutorFixture(factory)) r, err := dispatch.New(dispatch.Config{Registry: reg, Sender: sender, PreparationTimeout: timeout, LocalWorkspace: preparationWorkspace(t)}) if err != nil { t.Fatal(err) @@ -225,7 +234,7 @@ func TestPreparationSingleTransferAndReleaseDoesNotCancelRun(t *testing.T) { if err := r.Handle(t.Context(), prepare); err != nil { t.Fatal(err) } - start := mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, RunID: "real-run", Input: proto.TextInput("actual input")}) + start := mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, ExecutorID: ready.ExecutorID, RunID: "real-run", Input: proto.TextInput("actual input")}) if err := r.Handle(t.Context(), start); err != nil { t.Fatal(err) } @@ -276,7 +285,7 @@ func TestPreparationCancelDuringStartClosesLateSession(t *testing.T) { r := preparationRouter(t, sender, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { return p, nil }) _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "request", preparationRequest())) ready := waitPreparationStatus(t, sender, "request", "ready", "") - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, RunID: "real-run", Input: proto.TextInput("input")})) + _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, ExecutorID: ready.ExecutorID, RunID: "real-run", Input: proto.TextInput("input")})) <-entered if err := r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "real-run", proto.PromptCancelPayload{DeliveryID: "cancel"})); err != nil { t.Fatal(err) @@ -307,37 +316,6 @@ func TestPreparationCancelDuringStartClosesLateSession(t *testing.T) { } } -func TestPreparationCapacityAndConnectionOwnership(t *testing.T) { - sender := &recSender{} - var count atomic.Int32 - r := preparationRouter(t, sender, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { - count.Add(1) - return &controlledPreparation{closed: make(chan struct{})}, nil - }) - var handles []string - for i := 0; i < 4; i++ { - id := fmt.Sprint(i) - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, id, preparationRequest())) - handles = append(handles, waitPreparationStatus(t, sender, id, "ready", "").Handle) - } - if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "overflow", preparationRequest())); err == nil { - t.Fatal("capacity unbounded") - } - if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "0", preparationRequest())); err != nil { - t.Fatal(err) - } - if count.Load() != 4 { - t.Fatal("retry recreated native resource") - } - other := preparationRouter(t, &recSender{}, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { - t.Error("unexpected new native resource") - return nil, errors.New("unexpected") - }) - if err := other.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "0", proto.ExecutionStartPayload{Handle: handles[0], RunID: "run", Input: proto.TextInput("input")})); err == nil { - t.Fatal("another connection consumed handle") - } -} - func TestPreparationExpiryAndOldHandleCannotStartReplacement(t *testing.T) { sender := &recSender{} created := make(chan *controlledPreparation, 2) @@ -350,13 +328,18 @@ func TestPreparationExpiryAndOldHandleCannotStartReplacement(t *testing.T) { _ = r.Handle(t.Context(), env) old := waitPreparationStatus(t, sender, "request", "ready", "") waitPreparationStatus(t, sender, "request", "expired", "") - waitPreparationClosed(t, <-created) + owner := <-created + select { + case <-owner.closed: + t.Fatal("admission expiry closed a healthy executor") + default: + } _ = r.Handle(t.Context(), env) - next := waitPreparationStatus(t, sender, "request", "preparing", old.Handle) + next := waitPreparationStatus(t, sender, "request", "ready", old.Handle) if next.Handle == old.Handle || next.ExpiresAt <= old.ExpiresAt { t.Fatal("replacement reused expired identity") } - if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: old.Handle, RunID: "late", Input: proto.TextInput("late")})); err == nil { + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: old.Handle, ExecutorID: old.ExecutorID, RunID: "late", Input: proto.TextInput("late")})); err == nil { t.Fatal("old handle started replacement") } } @@ -371,61 +354,25 @@ func (s failReadySender) Send(ctx context.Context, env proto.Envelope) error { return s.recSender.Send(ctx, env) } -func TestPreparationFailedReadyDeliveryClosesResource(t *testing.T) { +func TestPreparationFailedReadyDeliveryAbandonsAdmission(t *testing.T) { + created := make(chan struct{}) p := &controlledPreparation{closed: make(chan struct{})} - r := preparationRouter(t, failReadySender{&recSender{}}, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { return p, nil }) + r := preparationRouter(t, failReadySender{&recSender{}}, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { + close(created) + return p, nil + }) _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "request", preparationRequest())) + <-created + waitFor(t, func() bool { return r.ActiveRuns() == 0 }, "abandoned admission") + if err := r.Shutdown(t.Context()); err != nil { + t.Fatal(err) + } waitPreparationClosed(t, p) if r.ActiveRuns() != 0 { t.Fatal("failed preparation became a run") } } -func TestPreparationCapacityIncludesClosingResources(t *testing.T) { - sender := &recSender{} - entered, unblock := make(chan struct{}), make(chan struct{}) - var releaseOnce sync.Once - release := func() { releaseOnce.Do(func() { close(unblock) }) } - defer release() - var count atomic.Int32 - r := preparationRouter(t, sender, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { - p := &controlledPreparation{closed: make(chan struct{})} - if count.Add(1) == 1 { - p.closeHook = func() { close(entered); <-unblock } - } - return p, nil - }) - var first proto.PreparationStatusPayload - for i := 0; i < 4; i++ { - id := fmt.Sprint(i) - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, id, preparationRequest())) - ready := waitPreparationStatus(t, sender, id, "ready", "") - if i == 0 { - first = ready - } - } - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionRelease, "0", proto.ExecutionReleasePayload{Handle: first.Handle})) - select { - case <-entered: - case <-time.After(time.Second): - t.Fatal("cleanup did not start") - } - request := mustEnv(t, proto.TypeExecutionPrepare, "replacement", preparationRequest()) - if err := r.Handle(t.Context(), request); err == nil { - t.Fatal("closing resource returned capacity early") - } - release() - deadline := time.Now().Add(time.Second) - for time.Now().Before(deadline) { - if err := r.Handle(t.Context(), request); err == nil { - waitPreparationStatus(t, sender, "replacement", "ready", "") - return - } - time.Sleep(time.Millisecond) - } - t.Fatal("completed cleanup did not release capacity") -} - func TestPreparationRejectsInputAndProductConfiguration(t *testing.T) { for name, change := range map[string]func(*proto.PromptRequestPayload){ "run": func(p *proto.PromptRequestPayload) { p.RunID = "run" }, @@ -437,7 +384,6 @@ func TestPreparationRejectsInputAndProductConfiguration(t *testing.T) { }, "missing environment": func(p *proto.PromptRequestPayload) { p.LocalEnvironment = nil }, "resume": func(p *proto.PromptRequestPayload) { p.StrictResume = false }, - "release": func(p *proto.PromptRequestPayload) { p.ReleaseOnCompletion = false }, } { t.Run(name, func(t *testing.T) { r := preparationRouter(t, &recSender{}, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { diff --git a/apps/parsar-daemon/internal/dispatch/prepared_handoff.go b/apps/parsar-daemon/internal/dispatch/prepared_handoff.go index b5416df44..89c3d1862 100644 --- a/apps/parsar-daemon/internal/dispatch/prepared_handoff.go +++ b/apps/parsar-daemon/internal/dispatch/prepared_handoff.go @@ -40,19 +40,21 @@ func (release *preparedRelease) aborted() bool { } } -// preparedHandoff owns the exact PreparedCancellation from Start admission -// through native cleanup and output settlement. It never falls back to the -// returned Session or Prepared.Close. +// preparedHandoff owns one Turn admission and settlement on a retained Executor. type preparedHandoff struct { preparation *preparationState - target agent.PreparedCancellation + target agent.Executor + turn agent.Turn + startErr error + protocolErr error startDone chan struct{} outputReady chan struct{} outputDone chan struct{} // Every admitted native mutation holds a read lock through its receipt. - // A release attempt takes the write lock before native cancellation. + // Release joins these receipts after native cancellation/settlement, since + // an admitted receipt may itself require cancellation to finish. operations sync.RWMutex published bool // started status committed; protected by Router.mu @@ -62,7 +64,7 @@ type preparedHandoff struct { shutdownInterrupted bool } -func newPreparedHandoff(p *preparationState, target agent.PreparedCancellation) *preparedHandoff { +func newPreparedHandoff(p *preparationState, target agent.Executor) *preparedHandoff { return &preparedHandoff{ preparation: p, target: target, @@ -116,6 +118,9 @@ func (r *Router) claimPreparedReleaseLocked(state *sessionState, abort bool, fai } if abort && !release.aborted() { close(release.abort) + if handoff.turn == nil { + handoff.preparation.cancel() + } } if release.succeeded { return release, release.attempt @@ -142,64 +147,144 @@ func (r *Router) claimPreparedReleaseLocked(state *sessionState, abort bool, fai func (r *Router) runPreparedRelease(state *sessionState, handoff *preparedHandoff, release *preparedRelease, attempt *preparedReleaseAttempt) { defer r.shutdownWG.Done() - - // Natural completion must publish started before it can release the native - // resource. Abort is allowed to fence a Start that is still in progress. select { case <-handoff.startDone: case <-release.abort: + select { + case <-handoff.startDone: + default: + if err := r.closeExecutor(handoff.preparation.executor); err != nil { + r.mu.Lock() + attempt.err = err + handoff.preparation.busy = false + handoff.preparation.closeErr = err + close(attempt.done) + r.mu.Unlock() + return + } + } + <-handoff.startDone } - handoff.operations.Lock() - ctx, cancel := context.WithTimeout(context.Background(), preparedCancelTimeout) - nativeErr := handoff.target.Cancel(ctx) - cancel() + // Admission is closed, but native cancellation must be able to release an + // already-written input or tool operation that is waiting for its receipt. + turn := handoff.turn + var settlement agent.TurnSettlement + var nativeErr error var outcome *proto.DonePayload + if turn == nil { + outcome = &proto.DonePayload{} + } + if turn != nil { + ctx, cancel := context.WithTimeout(context.Background(), preparedCancelTimeout) + cancelDone := make(chan error, 1) + settled := make(chan struct{}) + go func() { + select { + case <-release.abort: + err := turn.Cancel(ctx) + if err != nil { + cancel() + } + cancelDone <- err + case <-settled: + cancelDone <- nil + } + }() + settlement, nativeErr = turn.AwaitSettlement(ctx) + close(settled) + nativeErr = errors.Join(nativeErr, <-cancelDone) + cancel() + if nativeErr == nil { + value := turn.CancellationOutcome() + outcome = &value + } + } if nativeErr == nil { - observed := handoff.target.CancellationOutcome() - outcome = &observed + <-handoff.outputDone + } + r.mu.Lock() + owner := handoff.preparation.executor + if turn != nil && handoff.terminal == nil && !release.aborted() { + handoff.protocolErr = errors.New("executor output ended without a terminal result") + } + if handoff.protocolErr != nil { + release.failure = handoff.protocolErr.Error() + } + invalidate := nativeErr != nil || !settlement.Reusable || handoff.startErr != nil || handoff.outputErr != nil || handoff.protocolErr != nil || r.closed || owner.invalid + if invalidate { + owner.invalid = true + } + r.mu.Unlock() + var closeErr error + if invalidate { + closeErr = r.closeExecutor(owner) + // A failed Turn settlement is not repaired into a cancellation receipt by Close. + nativeErr = errors.Join(nativeErr, closeErr) + } + if closeErr == nil { + // Native settlement (or confirmed resource close after failure) comes + // first. Keep every admitted operation and its outbound receipt owned + // until it returns; neither cancellation acknowledgement nor reuse can + // pass this join. Failed Close retains the same owner for a later retry. + handoff.operations.Lock() + handoff.operations.Unlock() + <-handoff.outputDone } - handoff.operations.Unlock() - r.mu.Lock() attempt.err = nativeErr - if nativeErr != nil { + if closeErr != nil { handoff.preparation.busy = false handoff.preparation.closeErr = nativeErr close(attempt.done) r.mu.Unlock() return } - release.succeeded = true - release.outcome = outcome + if nativeErr != nil { + release.failure = "executor Turn settlement failed" + } + release.succeeded, release.outcome = true, outcome close(attempt.done) r.mu.Unlock() - - // Cancel success promises local cleanup and no further output writes. Wait - // for Start and the one output consumer before publishing terminal state. - <-handoff.startDone - <-handoff.outputDone - r.mu.Lock() - outputErr := handoff.outputErr - terminal := handoff.terminal - closed := r.closed + outputErr, terminal, closed := handoff.outputErr, handoff.terminal, r.closed + r.mu.Unlock() + // Done can become visible before Sender.Send returns. Commit the settled + // owner and retire this Run before publication so an immediate successor + // observes both the new native identity and an available Executor. + r.cleanupSession(state) + r.mu.Lock() + p := handoff.preparation + p.busy, p.owns, p.handoff = false, false, nil + p.cancel() + if owner.run == state { + owner.run = nil + } + if owner.invalid && owner.closeDone != nil && owner.closeErr == nil && r.executors[owner.sessionID] == owner { + delete(r.executors, owner.sessionID) + } + if owner.admission == p { + owner.admission = nil + } + if outcome != nil { + if id, ok := outcome.Metadata[proto.DoneMetaAgentSessionID].(string); ok && id != "" { + owner.nativeID = id + } + } + if !owner.invalid { + r.scheduleExecutorIdleLocked(owner) + } r.mu.Unlock() var terminalErr error - if outputErr == nil && !closed { - terminalErr = r.forwardPreparedTerminal(state, release.failure, terminal, release.failure != "" && terminal == nil) + if outputErr == nil && !closed && turn != nil { + terminalErr = r.forwardPreparedTerminal(state, release.failure, terminal, terminal == nil) } - r.cleanupSession(state) - r.mu.Lock() + // Delivery failure belongs to this Run, not to a successor that may have + // already acquired the settled owner. Connection shutdown owns transport + // failure cleanup; terminal publication does not change native settlement. handoff.outputErr = errors.Join(handoff.outputErr, terminalErr) - p := handoff.preparation - p.busy = false p.closeErr = terminalErr - p.prepared = nil - p.owns = false - p.handoff = nil - p.cancel() close(release.settled) r.mu.Unlock() } @@ -230,7 +315,9 @@ func (r *Router) forwardPreparedOutput(state *sessionState) { return } r.mu.Lock() - if handoff.terminal != nil { + if env.ID != state.runID || handoff.terminal != nil { + handoff.protocolErr = errors.New("executor output crossed the Turn boundary") + r.claimPreparedReleaseLocked(state, true, handoff.protocolErr.Error(), false) r.mu.Unlock() continue } diff --git a/apps/parsar-daemon/internal/dispatch/prepared_handoff_mutation_test.go b/apps/parsar-daemon/internal/dispatch/prepared_handoff_mutation_test.go index fa0717b4f..fce4c8640 100644 --- a/apps/parsar-daemon/internal/dispatch/prepared_handoff_mutation_test.go +++ b/apps/parsar-daemon/internal/dispatch/prepared_handoff_mutation_test.go @@ -15,16 +15,12 @@ type preparedMutationSession struct { *fakeSession cancelEntered chan struct{} cancelOnce sync.Once - beforeCancel func() functions atomic.Int32 steers atomic.Int32 reads atomic.Int32 } func (s *preparedMutationSession) Cancel(ctx context.Context) error { - if s.beforeCancel != nil { - s.beforeCancel() - } s.cancelOnce.Do(func() { close(s.cancelEntered) }) return s.fakeSession.Cancel(ctx) } @@ -162,8 +158,11 @@ func TestPreparedHandoffReleaseWaitsForMutationReceipt(t *testing.T) { } select { case <-session.cancelEntered: - t.Fatal("native release overtook an admitted receipt") - case <-time.After(50 * time.Millisecond): + case <-time.After(2 * time.Second): + t.Fatal("native cleanup was blocked by the outbound receipt") + } + if r.ActiveRuns() != 1 || hasFrame(sender.recSender, proto.TypeDone, "run") { + t.Fatal("Run ownership or Done overtook the admitted receipt") } close(sender.release) select { @@ -179,7 +178,7 @@ func TestPreparedHandoffReleaseWaitsForMutationReceipt(t *testing.T) { case <-time.After(2 * time.Second): t.Fatal("release did not follow the completed receipt") } - waitFor(t, func() bool { return r.ActiveRuns() == 0 }, "mutation release cleanup") + waitFor(t, func() bool { return r.ActiveRuns() == 0 && hasFrame(sender.recSender, proto.TypeDone, "run") }, "mutation release cleanup") if session.cancels() != 1 { t.Fatalf("Session release calls = %d, want 1", session.cancels()) } @@ -239,7 +238,6 @@ func TestPreparedHandoffRouterShutdownWaitsForReceiptAttempt(t *testing.T) { t.Run(operation, func(t *testing.T) { sender := &blockingPreparedReceiptSender{recSender: &recSender{}, deliveryID: operation + "-delivery", inputID: operation + "-input", entered: make(chan struct{}), release: make(chan struct{}), exited: make(chan struct{})} defer close(sender.release) - var cancelBeforeReceipt atomic.Bool session := &preparedMutationSession{fakeSession: &fakeSession{closeOutOnCancel: true}, cancelEntered: make(chan struct{})} p := &controlledPreparation{closed: make(chan struct{})} p.start = func(_ context.Context, _ string, _ proto.MessageInput, out chan<- proto.Envelope) (agent.Session, error) { @@ -250,13 +248,6 @@ func TestPreparedHandoffRouterShutdownWaitsForReceiptAttempt(t *testing.T) { startCancellationPreparation(t, r, sender.recSender) waitPreparationStatus(t, sender.recSender, "request", "started", "") - session.beforeCancel = func() { - select { - case <-sender.exited: - default: - cancelBeforeReceipt.Store(true) - } - } var mutation proto.Envelope switch operation { case "function": @@ -283,8 +274,13 @@ func TestPreparedHandoffRouterShutdownWaitsForReceiptAttempt(t *testing.T) { if err := r.Shutdown(ctx); err != nil { t.Fatal(err) } - if cancelBeforeReceipt.Load() || session.cancels() != 1 || r.ActiveRuns() != 0 { - t.Fatalf("shutdown crossed receipt barrier: early=%t cancels=%d active=%d", cancelBeforeReceipt.Load(), session.cancels(), r.ActiveRuns()) + select { + case <-sender.exited: + default: + t.Fatal("shutdown returned before the receipt attempt ended") + } + if session.cancels() != 1 || r.ActiveRuns() != 0 { + t.Fatalf("shutdown did not join native cleanup: cancels=%d active=%d", session.cancels(), r.ActiveRuns()) } }) } diff --git a/apps/parsar-daemon/internal/dispatch/prepared_handoff_test.go b/apps/parsar-daemon/internal/dispatch/prepared_handoff_test.go index d12c8246e..db420f8dc 100644 --- a/apps/parsar-daemon/internal/dispatch/prepared_handoff_test.go +++ b/apps/parsar-daemon/internal/dispatch/prepared_handoff_test.go @@ -208,7 +208,7 @@ func TestPreparedHandoffDuplicateStartDoesNotReexecuteDuringPublication(t *testi if session.functions.Load() != 0 || session.steers.Load() != 0 || session.reads.Load() != 0 || len(session.submissions()) != 0 || askCalls != 0 { t.Fatal("private Session accepted work before started publication") } - duplicate := mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, RunID: "run", Input: proto.TextInput("input")}) + duplicate := mustEnv(t, proto.TypeExecutionStart, "request", proto.ExecutionStartPayload{Handle: ready.Handle, ExecutorID: ready.ExecutorID, RunID: "run", Input: proto.TextInput("input")}) if err := r.Handle(t.Context(), duplicate); err != nil { t.Fatal(err) } @@ -315,7 +315,7 @@ func TestPreparedHandoffEarlyDoneDetachesPublishedPreparation(t *testing.T) { return session, nil } p.cancel = func(ctx context.Context) error { - if p.calls.Load() == 1 { + if p.calls.Load() <= 2 { return errors.New("cleanup incomplete") } return session.Cancel(ctx) @@ -327,7 +327,7 @@ func TestPreparedHandoffEarlyDoneDetachesPublishedPreparation(t *testing.T) { waitPreparationStatus(t, sender, "request", "started", "") waitFor(t, func() bool { _, busy := r.PreparationOwnershipForTest(ready.Handle) - return p.calls.Load() == 1 && !busy + return p.calls.Load() == 2 && !busy }, "failed natural cleanup") if owned, _ := r.PreparationOwnershipForTest(ready.Handle); owned { t.Fatal("published Run retained preparation capacity after early Done") @@ -345,14 +345,14 @@ func TestPreparedHandoffEarlyDoneDetachesPublishedPreparation(t *testing.T) { } return count == 2 }, "published preparation release response") - if p.calls.Load() != 1 { + if p.calls.Load() != 2 { t.Fatal("old preparation handle retried native cancellation") } if err := r.Handle(t.Context(), mustEnv(t, proto.TypePromptCancel, "run", proto.PromptCancelPayload{DeliveryID: "retry"})); err != nil { t.Fatal(err) } waitFor(t, func() bool { return len(cancellationAcks(sender)) == 1 && r.ActiveRuns() == 0 }, "Run-owned retry settlement") - if p.calls.Load() != 2 || session.cancels() != 1 || !cancellationAcks(sender)[0].Applied { + if p.calls.Load() != 3 || session.cancels() != 1 || !cancellationAcks(sender)[0].Applied { t.Fatal("explicit Run cancellation did not retry the retained native target") } } diff --git a/apps/parsar-daemon/internal/dispatch/prompt.go b/apps/parsar-daemon/internal/dispatch/prompt.go index f92d972ca..e0b4c07d4 100644 --- a/apps/parsar-daemon/internal/dispatch/prompt.go +++ b/apps/parsar-daemon/internal/dispatch/prompt.go @@ -5,6 +5,7 @@ import ( "errors" "fmt" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" obslog "github.com/MiniMax-AI-Dev/parsar/internal/obs/log" ) @@ -74,7 +75,7 @@ func (r *Router) handlePromptRequest(callerCtx context.Context, env proto.Envelo r.log.ErrorContext(callerCtx, "handlePromptRequest: router closed", "run_id", runID) return ErrRouterClosed } - if r.workspaceWrite != nil || r.workspaceExport != nil { + if r.runtimePreparation != nil || r.workspaceWrite != nil || r.workspaceExport != nil { r.mu.Unlock() err := errors.New("local workspace has an unsettled write") r.emitTerminalError(callerCtx, runID, err.Error()) @@ -111,7 +112,11 @@ func (r *Router) handlePromptRequest(callerCtx context.Context, env proto.Envelo r.mu.Unlock() r.log.InfoContext(callerCtx, "handlePromptRequest: calling factory", "run_id", runID) - sess, err := factory(sessionCtx, req, out) + req, err = r.localWorkspace.Prepare(sessionCtx, req) + var sess agent.Session + if err == nil { + sess, err = factory(sessionCtx, req, out) + } if err != nil { r.log.ErrorContext(callerCtx, "handlePromptRequest: factory call failed", "run_id", runID, "agent_kind", req.AgentKind, "err", err) // Roll back the registration, cancel ctx, surface error+done diff --git a/apps/parsar-daemon/internal/dispatch/router.go b/apps/parsar-daemon/internal/dispatch/router.go index 61c97f360..0a8a4ca9d 100644 --- a/apps/parsar-daemon/internal/dispatch/router.go +++ b/apps/parsar-daemon/internal/dispatch/router.go @@ -47,9 +47,11 @@ type Router struct { shutdownWG dispatchWork // waits for all pump goroutines idleTimeout time.Duration closed bool + executors map[string]*executorState preparations map[string]*preparationState preparationRequests map[string]*preparationState preparationTimeout time.Duration + runtimePreparation *runtimePreparationTransfer workspaceWrite *workspaceUpload workspaceExport *workspaceExport workspaceReads map[string]struct{} @@ -135,6 +137,7 @@ func New(cfg Config) (*Router, error) { applied: make(map[string]appliedInteractionDecision), shutdownCh: make(chan struct{}), idleTimeout: idleTimeout, + executors: make(map[string]*executorState), preparations: make(map[string]*preparationState), preparationRequests: make(map[string]*preparationState), preparationTimeout: preparationTimeout, @@ -165,6 +168,8 @@ func (r *Router) Handle(ctx context.Context, env proto.Envelope) error { ctx = adoptEnvelopeTrace(ctx, env) switch env.Type { + case proto.TypeRuntimePrepare: + return r.handleRuntimePrepare(ctx, env) case proto.TypeWorkspaceExport: return r.handleWorkspaceExport(ctx, env) case proto.TypeWorkspaceWrite: diff --git a/apps/parsar-daemon/internal/dispatch/runtime_preparation.go b/apps/parsar-daemon/internal/dispatch/runtime_preparation.go new file mode 100644 index 000000000..9975e9a41 --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/runtime_preparation.go @@ -0,0 +1,197 @@ +package dispatch + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/localworkspace" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/google/uuid" +) + +const runtimePreparationTimeout = 120 * time.Second + +// Router.mu protects one connection-local transfer. Partial installation data +// belongs to the bound Environment and is never removed by transfer cleanup. +type runtimePreparationTransfer struct { + envelope proto.Envelope + request proto.RuntimePreparePayload + data []byte + ready chan struct{} + cancel context.CancelFunc + finished bool + apply bool + uncertain bool +} + +func (r *Router) handleRuntimePrepare(ctx context.Context, env proto.Envelope) error { + id, err := uuid.Parse(env.ID) + if err != nil || id == uuid.Nil || id.String() != env.ID { + return errors.New("dispatch: invalid Runtime preparation identity") + } + var request proto.RuntimePreparePayload + if len(env.Payload) > proto.RuntimePrepareMaxFrameBytes || env.DecodePayload(&request) != nil || !proto.ValidRuntimePrepareRequest(request) { + r.mu.Lock() + pending := r.runtimePreparation != nil && r.runtimePreparation.envelope.ID == env.ID + if pending && !r.runtimePreparation.finished { + r.finishRuntimePreparationTransferLocked(r.runtimePreparation, false) + } + r.mu.Unlock() + if pending { + // A late malformed frame cannot report rejection of an earlier commit. + return errors.New("dispatch: malformed pending Runtime preparation frame") + } + return r.sendRuntimePrepareResult(ctx, env.ID, rejectedRuntimePreparation("invalid_request")) + } + r.mu.Lock() + if r.closed || r.suspension != nil { + r.mu.Unlock() + return ErrRouterClosed + } + if request.Step == "begin" { + if r.runtimePreparation != nil { + duplicate := r.runtimePreparation.envelope.ID == env.ID + r.mu.Unlock() + if duplicate { + return errors.New("dispatch: Runtime preparation already admitted") + } + return r.sendRuntimePrepareResult(ctx, env.ID, rejectedRuntimePreparation("runtime_preparation_capacity")) + } + if r.localWorkspace == nil || !r.localWorkspace.Matches(request.EnvironmentID, request.SessionID) || r.runtimePreparationResourcesBusyLocked() { + r.mu.Unlock() + return r.sendRuntimePrepareResult(ctx, env.ID, rejectedRuntimePreparation("resource_unavailable")) + } + owner, cancel := context.WithTimeout(context.WithoutCancel(ctx), runtimePreparationTimeout) + u := &runtimePreparationTransfer{ + envelope: env, request: request, data: make([]byte, 0, request.SizeBytes), + ready: make(chan struct{}), cancel: cancel, + } + r.runtimePreparation = u + r.shutdownWG.Add(1) + r.mu.Unlock() + go r.runRuntimePreparationTransfer(owner, u, r.localWorkspace.ApplyRuntimePreparation) + if err := r.sendRuntimePrepareResult(ctx, env.ID, proto.RuntimePrepareResultPayload{Outcome: "ready"}); err != nil { + cancel() + return err + } + return nil + } + u := r.runtimePreparation + if u == nil || u.envelope.ID != env.ID { + r.mu.Unlock() + return r.sendRuntimePrepareResult(ctx, env.ID, rejectedRuntimePreparation("resource_unavailable")) + } + if u.finished { + r.mu.Unlock() + return errors.New("dispatch: Runtime preparation body already closed") + } + if request.Step == "chunk" && request.Offset == len(u.data) && len(request.Data) <= u.request.SizeBytes-len(u.data) { + u.data = append(u.data, request.Data...) + offset := len(u.data) + r.mu.Unlock() + if err := r.sendRuntimePrepareResult(ctx, env.ID, proto.RuntimePrepareResultPayload{Outcome: "received", Offset: offset}); err != nil { + u.cancel() + return err + } + return nil + } + apply := false + if request.Step == "commit" && len(u.data) == u.request.SizeBytes { + if u.request.Action == "finalize" || u.request.Action == "initialize" { + apply = true + } else { + digest := sha256.Sum256(u.data) + apply = hex.EncodeToString(digest[:]) == u.request.SHA256 + } + } + r.finishRuntimePreparationTransferLocked(u, apply) + r.mu.Unlock() + return nil +} + +func (r *Router) runtimePreparationResourcesBusyLocked() bool { + if r.workspaceWrite != nil || r.workspaceExport != nil || len(r.workspaceReads) != 0 || len(r.sessions) != 0 || len(r.idle) != 0 || len(r.executors) != 0 { + return true + } + for _, p := range r.preparations { + if p.owns || p.busy { + return true + } + } + return false +} + +func (r *Router) finishRuntimePreparationTransferLocked(u *runtimePreparationTransfer, apply bool) { + u.apply, u.finished = apply, true + close(u.ready) +} + +// apply must return only after its local mutations stop. Cancellation requests +// shutdown, but cannot release ownership while that call is still running. +func (r *Router) runRuntimePreparationTransfer(ctx context.Context, u *runtimePreparationTransfer, apply func(context.Context, proto.RuntimePreparePayload, []byte) error) { + defer r.shutdownWG.Done() + defer u.cancel() + select { + case <-u.ready: + case <-r.shutdownCh: + case <-ctx.Done(): + } + r.mu.Lock() + admitted := u.apply && !r.closed && ctx.Err() == nil + u.finished = true + data := u.data + u.data = nil + r.mu.Unlock() + result := rejectedRuntimePreparation("invalid_request") + if admitted { + result = runtimePreparationResult(apply(ctx, u.request, data), u.request.SizeBytes) + } + // Release the potentially large body before waiting on transport delivery. + data = nil + r.mu.Lock() + u.uncertain = result.Outcome == "unknown" + if !u.uncertain && r.runtimePreparation == u { + r.runtimePreparation = nil + } + r.mu.Unlock() + // The result has a separate send budget, independent of an installation timeout. + _ = r.sendRuntimePrepareResult(context.WithoutCancel(ctx), u.envelope.ID, result) +} + +func rejectedRuntimePreparation(code string) proto.RuntimePrepareResultPayload { + return proto.RuntimePrepareResultPayload{Outcome: "rejected", ErrorCode: code} +} + +func runtimePreparationResult(err error, size int) proto.RuntimePrepareResultPayload { + if err == nil { + return proto.RuntimePrepareResultPayload{Outcome: "completed", SizeBytes: size} + } + var initialization *localworkspace.InitializationFailure + if !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) && errors.As(err, &initialization) { + code := 0 + if initialization.ExitCode != nil { + code = *initialization.ExitCode + } + if (initialization.ExitCode == nil || code > 0) && code <= 255 { + return proto.RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: "runtime_preparation_failed", ExitCode: code} + } + } + if !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) && errors.Is(err, agentcapabilities.ErrInvalid) { + return proto.RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: "runtime_preparation_failed"} + } + return proto.RuntimePrepareResultPayload{Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"} +} + +func (r *Router) sendRuntimePrepareResult(ctx context.Context, id string, result proto.RuntimePrepareResultPayload) error { + ctx, cancel := context.WithTimeout(ctx, 5*time.Second) + defer cancel() + env, err := proto.NewEnvelope(proto.TypeRuntimePrepareResult, id, result) + if err != nil { + return err + } + return r.sender.Send(ctx, env) +} diff --git a/apps/parsar-daemon/internal/dispatch/runtime_preparation_execution_test.go b/apps/parsar-daemon/internal/dispatch/runtime_preparation_execution_test.go new file mode 100644 index 000000000..eb7d59f7f --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/runtime_preparation_execution_test.go @@ -0,0 +1,56 @@ +package dispatch_test + +import ( + "context" + "errors" + "os" + "path/filepath" + "sync/atomic" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +// Directory selection is syntactically valid and passes frozen configuration +// admission. Runtime preparation must fail before creating a native Executor. +func TestRuntimePreparationUnavailablePreventsNativeExecutor(t *testing.T) { + for _, mode := range []string{"missing-directory", "invalid-skill"} { + t.Run(mode, func(t *testing.T) { + source := filepath.Join(t.TempDir(), "capability") + if mode == "invalid-skill" { + if err := os.Mkdir(source, 0700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(source, "SKILL.md"), []byte("not a portable Skill"), 0600); err != nil { + t.Fatal(err) + } + } + sender := &recSender{} + var calls atomic.Int32 + r := preparationRouter(t, sender, time.Minute, func(context.Context, proto.PromptRequestPayload) (agent.Prepared, error) { + calls.Add(1) + return nil, errors.New("native factory must not be reached") + }) + installation := os.Getenv("OAC_RUNTIME_CAPABILITY_DIRECTORY") + if info, err := os.Stat(installation); err != nil || !info.IsDir() { + t.Fatalf("capability fixture is unavailable: %v", err) + } + request := preparationRequest() + request.Configuration.LocalEnvironment.Capabilities = true + request.Configuration.LocalEnvironment.CapabilitySources = &agentcapabilities.Input{Directories: []string{source}} + if err := r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionPrepare, "missing-capability", request)); err != nil { + t.Fatalf("valid frozen selection rejected before preparation: %v", err) + } + status := waitPreparationStatus(t, sender, "missing-capability", "failed", "") + if _, err := os.Stat(filepath.Join(installation, agentcapabilities.ManifestName)); !os.IsNotExist(err) { + t.Fatal("invalid source published an installed snapshot") + } + if status.ErrorCode != "preparation_failed" || calls.Load() != 0 || r.ActiveRuns() != 0 { + t.Fatalf("failed capability preparation reached native execution: status=%+v calls=%d", status, calls.Load()) + } + }) + } +} diff --git a/apps/parsar-daemon/internal/dispatch/runtime_preparation_test.go b/apps/parsar-daemon/internal/dispatch/runtime_preparation_test.go new file mode 100644 index 000000000..dd58170e6 --- /dev/null +++ b/apps/parsar-daemon/internal/dispatch/runtime_preparation_test.go @@ -0,0 +1,355 @@ +package dispatch + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "os" + "path/filepath" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/localworkspace" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" + "github.com/google/uuid" +) + +type capabilitiesTestSender struct{ frames chan proto.Envelope } + +func (s *capabilitiesTestSender) Send(ctx context.Context, env proto.Envelope) error { + select { + case s.frames <- env: + return nil + case <-ctx.Done(): + return ctx.Err() + } +} + +func capabilitiesTestRouter(t *testing.T) (*Router, *capabilitiesTestSender, string, string) { + t.Helper() + environment, session := uuid.NewString(), uuid.NewString() + helper := filepath.Join(t.TempDir(), "helper") + if err := os.WriteFile(helper, []byte("#!/bin/sh\nexit 0\n"), 0700); err != nil { + t.Fatal(err) + } + binding, err := localworkspace.New(environment, session, t.TempDir(), helper) + if err != nil { + t.Fatal(err) + } + sender := &capabilitiesTestSender{frames: make(chan proto.Envelope, 64)} + router, err := New(Config{Registry: agent.NewRegistry(), Sender: sender, LocalWorkspace: binding}) + if err != nil { + t.Fatal(err) + } + return router, sender, environment, session +} + +func capabilityEnvelope(t *testing.T, id string, request proto.RuntimePreparePayload) proto.Envelope { + t.Helper() + env, err := proto.NewEnvelope(proto.TypeRuntimePrepare, id, request) + if err != nil { + t.Fatal(err) + } + return env +} + +func capabilityBegin(environment, session string, body []byte) proto.RuntimePreparePayload { + digest := sha256.Sum256(body) + return proto.RuntimePreparePayload{ + Step: "begin", Action: "skill", EnvironmentID: environment, SessionID: session, + Skill: &agentskill.Metadata{Type: "inline", Name: "proof", Description: "A proof."}, + SizeBytes: len(body), SHA256: hex.EncodeToString(digest[:]), + } +} + +func capabilitiesReceipt(t *testing.T, sender *capabilitiesTestSender, id, outcome string) proto.RuntimePrepareResultPayload { + t.Helper() + select { + case env := <-sender.frames: + var result proto.RuntimePrepareResultPayload + if env.Type != proto.TypeRuntimePrepareResult || env.ID != id || env.DecodePayload(&result) != nil || result.Outcome != outcome { + t.Fatalf("unexpected capability receipt: id=%s type=%s result=%+v", env.ID, env.Type, result) + } + return result + case <-time.After(3 * time.Second): + t.Fatal("missing capability receipt") + } + return proto.RuntimePrepareResultPayload{} +} + +func shutdownCapabilitiesRouter(t *testing.T, router *Router) { + t.Helper() + ctx, cancel := context.WithTimeout(context.Background(), 3*time.Second) + defer cancel() + if err := router.Shutdown(ctx); err != nil { + t.Fatal(err) + } +} + +func TestRuntimePreparationTransferValidatesCompleteBodyBeforeMutation(t *testing.T) { + for _, mode := range []string{"offset", "digest", "short", "oversized-chunk"} { + t.Run(mode, func(t *testing.T) { + r, sender, environment, session := capabilitiesTestRouter(t) + defer shutdownCapabilitiesRouter(t, r) + body := bytes.Repeat([]byte("a"), proto.RuntimePrepareChunkBytes+1) + request := capabilityBegin(environment, session, body) + if mode == "digest" { + request.SHA256 = stringsOfZeroDigest() + } + id := uuid.NewString() + if err := r.Handle(t.Context(), capabilityEnvelope(t, id, request)); err != nil { + t.Fatal(err) + } + capabilitiesReceipt(t, sender, id, "ready") + r.mu.Lock() + owner := r.runtimePreparation + r.mu.Unlock() + duplicate := uuid.NewString() + if err := r.Handle(t.Context(), capabilityEnvelope(t, duplicate, request)); err != nil { + t.Fatal(err) + } + if got := capabilitiesReceipt(t, sender, duplicate, "rejected"); got.ErrorCode != "runtime_preparation_capacity" { + t.Fatal(got) + } + switch mode { + case "offset": + _ = r.Handle(t.Context(), capabilityEnvelope(t, id, proto.RuntimePreparePayload{Step: "chunk", Offset: 1, Data: []byte("a")})) + case "oversized-chunk": + _ = r.Handle(t.Context(), capabilityEnvelope(t, id, proto.RuntimePreparePayload{Step: "chunk", Data: body})) + default: + if err := r.Handle(t.Context(), capabilityEnvelope(t, id, proto.RuntimePreparePayload{Step: "chunk", Data: body[:proto.RuntimePrepareChunkBytes]})); err != nil { + t.Fatal(err) + } + if got := capabilitiesReceipt(t, sender, id, "received"); got.Offset != proto.RuntimePrepareChunkBytes { + t.Fatal(got) + } + if mode == "digest" { + _ = r.Handle(t.Context(), capabilityEnvelope(t, id, proto.RuntimePreparePayload{Step: "chunk", Offset: proto.RuntimePrepareChunkBytes, Data: body[proto.RuntimePrepareChunkBytes:]})) + capabilitiesReceipt(t, sender, id, "received") + } + _ = r.Handle(t.Context(), capabilityEnvelope(t, id, proto.RuntimePreparePayload{Step: "commit"})) + } + capabilitiesReceipt(t, sender, id, "rejected") + r.mu.Lock() + defer r.mu.Unlock() + if owner.apply || owner.data != nil || r.runtimePreparation != nil { + t.Fatal("invalid body retained or admitted a mutation") + } + }) + } +} + +func stringsOfZeroDigest() string { return hex.EncodeToString(make([]byte, sha256.Size)) } + +func TestRuntimePreparationBeginRequiresExactBindingAndBounds(t *testing.T) { + r, sender, environment, session := capabilitiesTestRouter(t) + defer shutdownCapabilitiesRouter(t, r) + for _, mode := range []string{"environment", "session", "size", "nil-binding"} { + request := capabilityBegin(environment, session, []byte("abc")) + switch mode { + case "environment": + request.EnvironmentID = uuid.NewString() + case "session": + request.SessionID = uuid.NewString() + case "size": + request.SizeBytes = proto.RuntimePrepareMaxBytes + 1 + case "nil-binding": + r.localWorkspace = nil + } + id := uuid.NewString() + if err := r.Handle(t.Context(), capabilityEnvelope(t, id, request)); err != nil { + t.Fatal(err) + } + capabilitiesReceipt(t, sender, id, "rejected") + if r.runtimePreparation != nil { + t.Fatal("invalid scope allocated a transfer") + } + } +} + +func TestRuntimePreparationPreparationExcludesOwnedResources(t *testing.T) { + for _, mode := range []string{"write", "export", "read", "run", "idle", "executor", "preparation"} { + t.Run(mode, func(t *testing.T) { + r, sender, environment, session := capabilitiesTestRouter(t) + switch mode { + case "write": + r.workspaceWrite = &workspaceUpload{} + case "export": + r.workspaceExport = &workspaceExport{} + case "read": + r.workspaceReads = map[string]struct{}{"read": {}} + case "run": + r.sessions["run"] = &sessionState{} + case "idle": + r.idle["state"] = map[*sessionState]struct{}{} + case "executor": + r.executors[session] = &executorState{} + case "preparation": + r.preparations["p"] = &preparationState{owns: true} + } + id := uuid.NewString() + if err := r.Handle(t.Context(), capabilityEnvelope(t, id, capabilityBegin(environment, session, []byte("abc")))); err != nil { + t.Fatal(err) + } + if got := capabilitiesReceipt(t, sender, id, "rejected"); got.ErrorCode != "resource_unavailable" { + t.Fatal(got) + } + if r.runtimePreparation != nil { + t.Fatal("busy Runtime admitted capability preparation") + } + r.workspaceWrite = nil + r.workspaceExport = nil + r.workspaceReads = nil + clear(r.sessions) + clear(r.idle) + clear(r.executors) + clear(r.preparations) + shutdownCapabilitiesRouter(t, r) + }) + } +} + +func TestRuntimePreparationUploadBlocksWorkspaceWriteAndSuspension(t *testing.T) { + r, sender, environment, session := capabilitiesTestRouter(t) + id := uuid.NewString() + if err := r.Handle(t.Context(), capabilityEnvelope(t, id, capabilityBegin(environment, session, []byte("abc")))); err != nil { + t.Fatal(err) + } + capabilitiesReceipt(t, sender, id, "ready") + if err := r.Quiesce(t.Context(), proto.EnvironmentSuspendPayload{EnvironmentID: environment, SuspendID: uuid.NewString()}); !errors.Is(err, ErrRouterBusy) { + t.Fatal(err) + } + digest := sha256.Sum256([]byte("abc")) + write, err := proto.NewEnvelope(proto.TypeWorkspaceWrite, uuid.NewString(), proto.WorkspaceWritePayload{Step: "begin", EnvironmentID: environment, SessionID: session, Path: "proof", SizeBytes: 3, SHA256: hex.EncodeToString(digest[:])}) + if err != nil { + t.Fatal(err) + } + if err := r.Handle(t.Context(), write); err != nil { + t.Fatal(err) + } + select { + case env := <-sender.frames: + var result proto.WorkspaceWriteResultPayload + if env.Type != proto.TypeWorkspaceWriteResult || env.DecodePayload(&result) != nil || result.Outcome != "rejected" || result.ErrorCode != "resource_unavailable" { + t.Fatal("workspace mutation bypassed capability transfer") + } + case <-time.After(time.Second): + t.Fatal("missing write rejection") + } + r.mu.Lock() + owner := r.runtimePreparation + r.mu.Unlock() + shutdownCapabilitiesRouter(t, r) + if owner.data != nil || r.runtimePreparation != nil { + t.Fatal("disconnect retained uncommitted body") + } +} + +func TestRuntimePreparationCancellationKeepsOwnershipUntilApplyStops(t *testing.T) { + r, sender, environment, session := capabilitiesTestRouter(t) + ctx, cancel := context.WithCancel(context.Background()) + id := uuid.NewString() + request := proto.RuntimePreparePayload{Step: "begin", Action: "finalize", EnvironmentID: environment, SessionID: session, Sources: &agentcapabilities.Input{}} + owner := &runtimePreparationTransfer{envelope: capabilityEnvelope(t, id, request), request: request, ready: make(chan struct{}), cancel: cancel, finished: true, apply: true} + close(owner.ready) + r.runtimePreparation = owner + r.shutdownWG.Add(1) + started, interrupted, release := make(chan struct{}), make(chan struct{}), make(chan struct{}) + retained := filepath.Join(t.TempDir(), "installed.json") + go r.runRuntimePreparationTransfer(ctx, owner, func(ctx context.Context, got proto.RuntimePreparePayload, data []byte) error { + if got.Action != "finalize" || len(data) != 0 { + return agentcapabilities.ErrInvalid + } + close(started) + <-ctx.Done() + close(interrupted) + <-release + return os.WriteFile(retained, []byte("retained"), 0400) + }) + <-started + wait, stop := context.WithTimeout(context.Background(), 20*time.Millisecond) + err := r.Shutdown(wait) + stop() + if !errors.Is(err, context.DeadlineExceeded) { + t.Fatalf("shutdown discarded running mutation: %v", err) + } + <-interrupted + r.mu.Lock() + owned := r.runtimePreparation == owner + r.mu.Unlock() + if !owned { + t.Fatal("cancel released unsettled capability ownership") + } + close(release) + shutdownCapabilitiesRouter(t, r) + capabilitiesReceipt(t, sender, id, "completed") + if _, err := os.Stat(retained); err != nil { + t.Fatal("shutdown deleted installation result") + } + if r.runtimePreparation != nil { + t.Fatal("confirmed completion retained capacity") + } +} + +func TestRuntimePreparationInitializationReceipts(t *testing.T) { + for _, code := range []int{-1, 0, 1, 255, 256} { + got := runtimePreparationResult(&localworkspace.InitializationFailure{ExitCode: &code}, 0) + if code > 0 && code <= 255 { + if got.Outcome != "failed" || got.ExitCode != code { + t.Fatalf("lost confirmed exit code: %+v", got) + } + } else if got.Outcome != "unknown" { + t.Fatalf("accepted invalid failure receipt: %+v", got) + } + } + if got := runtimePreparationResult(&localworkspace.InitializationFailure{}, 0); got.Outcome != "failed" || got.ExitCode != 0 { + t.Fatalf("lost confirmed generic failure: %+v", got) + } + if got := runtimePreparationResult(errors.Join(&localworkspace.InitializationFailure{}, context.Canceled), 0); got.Outcome != "unknown" { + t.Fatalf("cancellation reported confirmed: %+v", got) + } +} + +func TestRuntimePreparationResultCategoriesAndUnknownOwnership(t *testing.T) { + for _, tc := range []struct { + err error + outcome, code string + }{ + {nil, "completed", ""}, + {agentcapabilities.ErrInvalid, "failed", "runtime_preparation_failed"}, + {context.DeadlineExceeded, "unknown", "runtime_preparation_unconfirmed"}, + {errors.Join(agentcapabilities.ErrInvalid, context.Canceled), "unknown", "runtime_preparation_unconfirmed"}, + {errors.New("private native diagnostic"), "unknown", "runtime_preparation_unconfirmed"}, + } { + got := runtimePreparationResult(tc.err, 3) + if got.Outcome != tc.outcome || got.ErrorCode != tc.code || !proto.ValidRuntimePrepareResult(got, "completed", 0, 3) { + t.Fatalf("unsafe result: %+v", got) + } + } + r, sender, environment, session := capabilitiesTestRouter(t) + ctx, cancel := context.WithCancel(context.Background()) + id := uuid.NewString() + request := capabilityBegin(environment, session, []byte("abc")) + owner := &runtimePreparationTransfer{envelope: capabilityEnvelope(t, id, request), request: request, data: []byte("abc"), ready: make(chan struct{}), cancel: cancel, finished: true, apply: true} + close(owner.ready) + r.runtimePreparation = owner + r.shutdownWG.Add(1) + go r.runRuntimePreparationTransfer(ctx, owner, func(context.Context, proto.RuntimePreparePayload, []byte) error { return context.DeadlineExceeded }) + capabilitiesReceipt(t, sender, id, "unknown") + r.mu.Lock() + owned := r.runtimePreparation == owner && owner.uncertain && owner.data == nil + r.mu.Unlock() + if !owned { + t.Fatal("unknown mutation released its ownership") + } + wait, stop := context.WithTimeout(context.Background(), time.Second) + defer stop() + if err := r.Shutdown(wait); err == nil { + t.Fatal("shutdown claimed uncertain mutation settled") + } +} diff --git a/apps/parsar-daemon/internal/dispatch/shutdown.go b/apps/parsar-daemon/internal/dispatch/shutdown.go index a59808ead..15d7f7061 100644 --- a/apps/parsar-daemon/internal/dispatch/shutdown.go +++ b/apps/parsar-daemon/internal/dispatch/shutdown.go @@ -43,6 +43,9 @@ func (r *Router) Shutdown(ctx context.Context) error { } if first { r.closed = true + if r.runtimePreparation != nil { + r.runtimePreparation.cancel() + } for _, states := range r.idle { for state := range states { state.retain = false @@ -57,12 +60,14 @@ func (r *Router) Shutdown(ctx context.Context) error { close(r.shutdownCh) } preparations := r.closePendingPreparationsLocked() + executors := r.closeIdleExecutorsLocked() attempt := &shutdownAttempt{done: make(chan struct{})} r.shutdownAttempt = attempt // Keep the WaitGroup non-zero until all cancellation dispatch is complete. r.shutdownWG.Add(1) r.mu.Unlock() + r.closeIdleExecutors(executors) for _, p := range preparations { go func() { defer r.shutdownWG.Done(); r.closePreparationResource(p) }() } @@ -98,6 +103,9 @@ func (r *Router) runShutdownAttempt(attempt *shutdownAttempt, victims []sessionC r.shutdownWG.Wait() r.mu.Lock() + if r.runtimePreparation != nil && r.runtimePreparation.uncertain { + attempt.err = errors.Join(attempt.err, errors.New("dispatch: capability preparation remains uncertain")) + } if r.workspaceWrite != nil && r.workspaceWrite.uncertain { attempt.err = errors.Join(attempt.err, errors.New("dispatch: local workspace write remains uncertain")) } @@ -110,6 +118,9 @@ func (r *Router) runShutdownAttempt(attempt *shutdownAttempt, victims []sessionC attempt.err = errors.Join(attempt.err, fmt.Errorf("dispatch: preparation %s: %w", p.status.Handle, cause)) } } + for _, owner := range r.executors { + attempt.err = errors.Join(attempt.err, fmt.Errorf("dispatch: executor %s cleanup unconfirmed: %w", owner.id, owner.closeErr)) + } close(attempt.done) r.mu.Unlock() } @@ -139,6 +150,9 @@ func (r *Router) handleDeviceShutdown(ctx context.Context, env proto.Envelope) e r.mu.Unlock() return ErrRouterClosed } + if r.runtimePreparation != nil { + r.runtimePreparation.cancel() + } victims := make([]sessionCancellation, 0, len(r.sessions)) for _, state := range r.sessions { state.retain = false @@ -155,7 +169,9 @@ func (r *Router) handleDeviceShutdown(ctx context.Context, env proto.Envelope) e } r.idle = make(map[string]map[*sessionState]struct{}) preparations := r.closePendingPreparationsLocked() + executors := r.closeIdleExecutorsLocked() r.mu.Unlock() + r.closeIdleExecutors(executors) for _, p := range preparations { go func() { defer r.shutdownWG.Done(); r.closePreparationResource(p) }() } diff --git a/apps/parsar-daemon/internal/dispatch/steering.go b/apps/parsar-daemon/internal/dispatch/steering.go index 00265cb91..c6636fa1b 100644 --- a/apps/parsar-daemon/internal/dispatch/steering.go +++ b/apps/parsar-daemon/internal/dispatch/steering.go @@ -98,8 +98,8 @@ func (r *Router) queueSteering(ctx context.Context, env proto.Envelope, input pr return &ack } if input.DurableReceipt { - if _, ok := session.(agent.DurableSteerer); !ok || !state.releaseOnCompletion { - ack.ErrorCode, ack.Error = "unsupported", "Durable input receipts require a supported release-on-completion run." + if _, ok := session.(agent.DurableSteerer); !ok || (!state.releaseOnCompletion && state.preparedHandoff == nil) { + ack.ErrorCode, ack.Error = "unsupported", "Durable input receipts require a supported Turn settlement contract." return &ack } } diff --git a/apps/parsar-daemon/internal/dispatch/suspend.go b/apps/parsar-daemon/internal/dispatch/suspend.go index 924fe237e..e52df4858 100644 --- a/apps/parsar-daemon/internal/dispatch/suspend.go +++ b/apps/parsar-daemon/internal/dispatch/suspend.go @@ -31,7 +31,7 @@ func (r *Router) Quiesce(ctx context.Context, request proto.EnvironmentSuspendPa r.mu.Unlock() return ErrRouterQuiesced } - if len(r.sessions) != 0 || len(r.workspaceReads) != 0 || r.workspaceWrite != nil || r.workspaceExport != nil || len(r.permIndex) != 0 || len(r.askIndex) != 0 { + if r.runtimePreparation != nil || len(r.sessions) != 0 || len(r.workspaceReads) != 0 || r.workspaceWrite != nil || r.workspaceExport != nil || len(r.permIndex) != 0 || len(r.askIndex) != 0 { r.mu.Unlock() return ErrRouterBusy } @@ -49,6 +49,12 @@ func (r *Router) Quiesce(ctx context.Context, request proto.EnvironmentSuspendPa } } } + for _, owner := range r.executors { + if owner.invalid || owner.preparing || owner.run != nil || owner.admission != nil || owner.environmentID != request.EnvironmentID { + r.mu.Unlock() + return ErrRouterBusy + } + } r.suspension = &request for _, p := range r.preparations { if p.timer != nil { @@ -63,6 +69,12 @@ func (r *Router) Quiesce(ctx context.Context, request proto.EnvironmentSuspendPa } } } + for _, owner := range r.executors { + owner.idleLease++ + if owner.timer != nil { + owner.timer.Stop() + } + } r.mu.Unlock() err := r.shutdownWG.waitContext(ctx) r.mu.Lock() @@ -89,6 +101,9 @@ func (r *Router) Resume(request proto.EnvironmentSuspendPayload, sender Sender) } r.sender = sender r.suspension = nil + for _, owner := range r.executors { + r.scheduleExecutorIdleLocked(owner) + } for _, states := range r.idle { for state := range states { r.scheduleIdleLocked(state) diff --git a/apps/parsar-daemon/internal/dispatch/workspace_directory_test.go b/apps/parsar-daemon/internal/dispatch/workspace_directory_test.go index 49835c073..28a14c02a 100644 --- a/apps/parsar-daemon/internal/dispatch/workspace_directory_test.go +++ b/apps/parsar-daemon/internal/dispatch/workspace_directory_test.go @@ -33,7 +33,7 @@ func TestWorkspaceDirectoryRetainsEnvironmentAndTransferredOwner(t *testing.T) { t.Fatal(got) } if phase == "idle" { - _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "prepare", proto.ExecutionStartPayload{Handle: ready.Handle, RunID: "run", Input: proto.TextInput("start")})) + _ = r.Handle(t.Context(), mustEnv(t, proto.TypeExecutionStart, "prepare", proto.ExecutionStartPayload{Handle: ready.Handle, ExecutorID: ready.ExecutorID, RunID: "run", Input: proto.TextInput("start")})) waitPreparationStatus(t, sender, "prepare", "started", "") _ = r.Handle(t.Context(), mustEnv(t, proto.TypeWorkspaceRead, "stale", request)) if got := waitWorkspaceRead(t, sender, "stale"); got.Outcome != "rejected" { diff --git a/apps/parsar-daemon/internal/dispatch/workspace_read.go b/apps/parsar-daemon/internal/dispatch/workspace_read.go index 77aeb8b67..0932387d2 100644 --- a/apps/parsar-daemon/internal/dispatch/workspace_read.go +++ b/apps/parsar-daemon/internal/dispatch/workspace_read.go @@ -68,6 +68,9 @@ func (r *Router) workspaceResourceLocked(request proto.WorkspaceReadPayload) (an return nil, "resource_unavailable" } resource = p.prepared + if p.executor != nil { + resource = p.executor.native + } } else { s := r.sessions[request.RunID] if s == nil || s.environmentID != request.EnvironmentID || s.session == nil || diff --git a/apps/parsar-daemon/internal/dispatch/workspace_write.go b/apps/parsar-daemon/internal/dispatch/workspace_write.go index 751cdb807..7226ef20d 100644 --- a/apps/parsar-daemon/internal/dispatch/workspace_write.go +++ b/apps/parsar-daemon/internal/dispatch/workspace_write.go @@ -46,7 +46,7 @@ func (r *Router) handleWorkspaceWrite(ctx context.Context, env proto.Envelope) e return ErrRouterClosed } if request.Step == "begin" { - if r.workspaceExport != nil { + if r.workspaceExport != nil || r.runtimePreparation != nil { r.mu.Unlock() return r.sendWorkspaceWrite(ctx, env.ID, rejectedWorkspaceWrite("resource_unavailable")) } @@ -58,10 +58,16 @@ func (r *Router) handleWorkspaceWrite(ctx context.Context, env proto.Envelope) e } return r.sendWorkspaceWrite(ctx, env.ID, rejectedWorkspaceWrite("write_capacity")) } - if !r.localWorkspace.AcceptsFileWrite(request.EnvironmentID, request.SessionID) || len(r.sessions) != 0 || len(r.idle) != 0 { + if !r.localWorkspace.AcceptsFileWrite(request.EnvironmentID, request.SessionID) || len(r.sessions) != 0 || len(r.idle) != 0 || len(r.workspaceReads) != 0 { r.mu.Unlock() return r.sendWorkspaceWrite(ctx, env.ID, rejectedWorkspaceWrite("resource_unavailable")) } + for _, owner := range r.executors { + if owner.preparing || owner.admission != nil || owner.run != nil || owner.invalid { + r.mu.Unlock() + return r.sendWorkspaceWrite(ctx, env.ID, rejectedWorkspaceWrite("resource_unavailable")) + } + } for _, p := range r.preparations { if p.owns { r.mu.Unlock() diff --git a/apps/parsar-daemon/internal/localworkspace/binding.go b/apps/parsar-daemon/internal/localworkspace/binding.go index 928db7fdb..c533216d4 100644 --- a/apps/parsar-daemon/internal/localworkspace/binding.go +++ b/apps/parsar-daemon/internal/localworkspace/binding.go @@ -5,6 +5,10 @@ import ( "os" "path/filepath" "strings" + "sync" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/paths" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/MiniMax-AI-Dev/parsar/internal/agentnetwork" @@ -21,6 +25,8 @@ type Binding struct { helper string exportHelper string writer *fileWriter + capabilityMu sync.Mutex + capabilityRoot string } func New(environment, session, workspace, helper string) (*Binding, error) { @@ -43,7 +49,30 @@ func New(environment, session, workspace, helper string) (*Binding, error) { if err != nil || !program.Mode().IsRegular() || program.Mode().Perm()&0111 == 0 || strings.HasPrefix(helper, workspace+string(filepath.Separator)) { return nil, errors.New("local workspace helper must be executable outside the workspace") } - return &Binding{environment: environment, stateKey: "agents-api-" + session, workspace: workspace, helper: helper}, nil + return &Binding{environment: environment, stateKey: "agents-api-" + session, workspace: workspace, helper: helper, capabilityRoot: CapabilityDirectory}, nil +} + +// NewWithCapabilityDirectory freezes an operator-owned installation layout. +// Core and capability transfer messages cannot choose this destination. +func NewWithCapabilityDirectory(environment, session, workspace, helper, directory string) (*Binding, error) { + b, err := New(environment, session, workspace, helper) + if err != nil { + return nil, err + } + if agentcapabilities.ValidateLocalDirectories([]string{directory}) != nil || directory == "/" || containsPath(workspace, directory) || containsPath(directory, workspace) || containsPath(PackageDirectory, directory) || containsPath(directory, PackageDirectory) { + return nil, agentcapabilities.ErrInvalid + } + private, err := paths.Root() + if err != nil || !filepath.IsAbs(private) || containsPath(private, directory) || containsPath(directory, private) { + return nil, agentcapabilities.ErrInvalid + } + for _, name := range []string{workspace, private, directory} { + if !canonicalExistingParent(name) { + return nil, agentcapabilities.ErrInvalid + } + } + b.capabilityRoot = directory + return b, nil } func Load() (*Binding, error) { @@ -55,10 +84,14 @@ func Load() (*Binding, error) { network := policy.Access writeHelper, staging := os.Getenv("OAC_RUNTIME_WRITE_HELPER"), os.Getenv("OAC_RUNTIME_STAGING") exportHelper := os.Getenv("OAC_RUNTIME_EXPORT_HELPER") - if strings.Join(values, "") == "" && writeHelper == "" && staging == "" && network == "" && exportHelper == "" { + capabilityDirectory := os.Getenv("OAC_RUNTIME_CAPABILITY_DIRECTORY") + if strings.Join(values, "") == "" && writeHelper == "" && staging == "" && network == "" && exportHelper == "" && capabilityDirectory == "" { return nil, nil } - b, err := New(values[0], values[1], values[2], values[3]) + if capabilityDirectory == "" { + capabilityDirectory = CapabilityDirectory + } + b, err := NewWithCapabilityDirectory(values[0], values[1], values[2], values[3], capabilityDirectory) if err != nil { return nil, err } @@ -90,7 +123,7 @@ func (b *Binding) Configure(r proto.PromptRequestPayload) (proto.PromptRequestPa } if b == nil || r.LocalEnvironment == nil || r.LocalEnvironment.ID != b.environment || r.AgentStateKey != b.stateKey || r.DisableExecutionEnvironment || r.WorkDir != "" || - r.ConversationID != "" || r.WorkspaceAuthoring || !r.StrictResume || !r.ReleaseOnCompletion { + r.ConversationID != "" || r.WorkspaceAuthoring || !r.StrictResume { return r, errors.New("request does not match the dedicated local Environment") } if !r.WorkspaceReadOnly || r.LocalEnvironment.NetworkAccess != "" || len(r.LocalEnvironment.AllowedDomains) > 0 { @@ -100,39 +133,43 @@ func (b *Binding) Configure(r proto.PromptRequestPayload) (proto.PromptRequestPa } } if !r.WorkspaceReadOnly { - if r.LocalEnvironment.SystemPackages && !r.LocalEnvironment.ToolEnvironment { - return r, errors.New("system packages require initialized tool configuration") + local := *r.LocalEnvironment + if local.WorkspaceDirectory != "/workspace" && local.WorkspaceDirectory != b.workspace { + return r, errors.New("request does not match the local workspace selection") } - if r.LocalEnvironment.ToolEnvironment { - if err := VerifyToolEnvironment(r.LocalEnvironment.SystemPackages); err != nil { - return r, err - } + if local.CapabilitySources == nil || agentcapabilities.ValidateInput(*local.CapabilitySources) != nil { + return r, agentcapabilities.ErrInvalid } - local := *r.LocalEnvironment - local.Skills = nil - local.MCP = nil - if local.Capabilities { - manifest, err := LoadCapabilities() - if err != nil { - return r, err - } - local.Skills = manifest.Skills - if len(manifest.MCP) != 0 { - if b.NetworkPolicy().Access != "enabled" { - return r, errors.New("environment MCP requires qualified enabled-network execution") - } - values, err := ReadToolEnvironment() - if err != nil { - return r, err - } - local.MCP, err = resolveEnvironmentMCP(manifest.MCP, values) - if err != nil { - return r, err - } - } + sources := *local.CapabilitySources + present := len(sources.Skills)+len(sources.Plugins)+len(sources.Directories) > 0 + if present != local.Capabilities || local.SystemPackages && !local.ToolEnvironment { + return r, agentcapabilities.ErrInvalid } + local.Skills, local.MCP, local.CapabilityRoot = nil, nil, "" r.LocalEnvironment = &local r.WorkDir = b.workspace } return r, nil } + +func (b *Binding) Matches(environment, session string) bool { + return b != nil && b.environment == environment && b.stateKey == "agents-api-"+session +} + +// Inspect existing ancestors before creating an operator-owned path. This prevents +// MkdirAll from creating directories through an alias before its final validation. +func canonicalExistingParent(name string) bool { + for { + if _, err := os.Lstat(name); err == nil { + actual, err := filepath.EvalSymlinks(name) + return err == nil && actual == name + } else if !errors.Is(err, os.ErrNotExist) { + return false + } + parent := filepath.Dir(name) + if parent == name { + return false + } + name = parent + } +} diff --git a/apps/parsar-daemon/internal/localworkspace/binding_test.go b/apps/parsar-daemon/internal/localworkspace/binding_test.go index 98973300b..4e0da0b2d 100644 --- a/apps/parsar-daemon/internal/localworkspace/binding_test.go +++ b/apps/parsar-daemon/internal/localworkspace/binding_test.go @@ -8,12 +8,18 @@ import ( "testing" "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/google/uuid" ) func testBinding(t *testing.T) (*Binding, proto.PromptRequestPayload) { t.Helper() + private := t.TempDir() + if err := os.Chmod(private, 0700); err != nil { + t.Fatal(err) + } + t.Setenv("OAC_RUNTIME_HOME", private) root := t.TempDir() helper := filepath.Join(t.TempDir(), "helper") if err := os.WriteFile(helper, []byte("#!/bin/sh\nexit 0\n"), 0o700); err != nil { @@ -25,7 +31,8 @@ func testBinding(t *testing.T) (*Binding, proto.PromptRequestPayload) { t.Fatal(err) } b.networkAccess = "disabled" - return b, proto.PromptRequestPayload{LocalEnvironment: &proto.LocalEnvironment{ID: environment, NetworkAccess: "disabled"}, AgentStateKey: "agents-api-" + session, StrictResume: true, ReleaseOnCompletion: true} + b.capabilityRoot = t.TempDir() + return b, proto.PromptRequestPayload{LocalEnvironment: &proto.LocalEnvironment{ID: environment, NetworkAccess: "disabled", WorkspaceDirectory: "/workspace", CapabilitySources: &agentcapabilities.Input{}}, AgentStateKey: "agents-api-" + session, StrictResume: true, ReleaseOnCompletion: true} } func TestBindingRejectsScopeAndPathOverrides(t *testing.T) { @@ -107,3 +114,32 @@ func TestDirectoryRejectsMalformedOrIncompleteResponses(t *testing.T) { } } } + +func TestBindingAllowsRetainedExecutor(t *testing.T) { + b, req := testBinding(t) + req.ReleaseOnCompletion = false + if _, err := b.Configure(req); err != nil { + t.Fatal(err) + } +} + +func TestCapabilityLayoutExcludesWritableAndPrivateRoots(t *testing.T) { + b, _ := testBinding(t) + private := os.Getenv("OAC_RUNTIME_HOME") + for _, directory := range []string{b.workspace, filepath.Join(b.workspace, "caps"), private, filepath.Join(private, "caps"), PackageDirectory, filepath.Join(PackageDirectory, "caps"), "/"} { + if _, err := NewWithCapabilityDirectory(b.environment, b.capabilityIdentity().SessionID, b.workspace, b.helper, directory); err == nil { + t.Fatal("unsafe installation layout accepted") + } + } + root := t.TempDir() + alias := filepath.Join(root, "alias") + if err := os.Symlink(t.TempDir(), alias); err != nil { + t.Fatal(err) + } + if _, err := NewWithCapabilityDirectory(b.environment, b.capabilityIdentity().SessionID, b.workspace, b.helper, filepath.Join(alias, "new")); err == nil { + t.Fatal("installation parent alias accepted") + } + if _, err := NewWithCapabilityDirectory(b.environment, b.capabilityIdentity().SessionID, b.workspace, b.helper, t.TempDir()); err != nil { + t.Fatal(err) + } +} diff --git a/apps/parsar-daemon/internal/localworkspace/capabilities.go b/apps/parsar-daemon/internal/localworkspace/capabilities.go new file mode 100644 index 000000000..e762b177f --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/capabilities.go @@ -0,0 +1,225 @@ +package localworkspace + +import ( + "context" + "errors" + "os" + "path/filepath" + "strings" + "syscall" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/paths" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +// Prepare runs under the admitted executor's lifetime, before native startup. +// Reconnection validates installed contents without reopening mutable sources. +func (b *Binding) Prepare(ctx context.Context, r proto.PromptRequestPayload) (proto.PromptRequestPayload, error) { + if b == nil && r.LocalEnvironment == nil || r.WorkspaceReadOnly { + return r, nil + } + if b == nil || r.LocalEnvironment == nil || r.LocalEnvironment.CapabilitySources == nil || + r.LocalEnvironment.ID != b.environment || r.AgentStateKey != b.stateKey || r.WorkDir != b.workspace { + return r, agentcapabilities.ErrInvalid + } + b.capabilityMu.Lock() + defer b.capabilityMu.Unlock() + marker, err := b.openSnapshotMarker() + if err != nil { + return r, err + } + defer marker.close() + root, unlock, err := b.openCapabilities(ctx, marker.completed) + if err != nil { + return r, err + } + defer unlock() + input := *r.LocalEnvironment.CapabilitySources + identity := b.capabilityIdentity() + manifest, err := b.loadCapabilitySnapshot(root, input, identity, marker.completed) + if err != nil || marker.complete() != nil { + return r, agentcapabilities.ErrInvalid + } + if err = ctx.Err(); err != nil { + return r, err + } + local := *r.LocalEnvironment + local.Skills, local.MCP, local.CapabilityRoot = manifest.Skills, nil, b.capabilityRoot + for i := range local.Skills { + local.Skills[i].InstallationRoot = b.capabilityRoot + } + if local.ToolEnvironment { + if err = VerifyToolEnvironment(local.SystemPackages); err != nil { + return r, err + } + } + if len(manifest.MCP) != 0 { + if b.NetworkPolicy().Access != "enabled" { + return r, agentcapabilities.ErrInvalid + } + values, readErr := ReadOptionalToolEnvironment() + if readErr != nil { + return r, readErr + } + local.MCP, err = resolveEnvironmentMCP(manifest.MCP, values) + for i := range local.MCP { + local.MCP[i].InstallationRoot = b.capabilityRoot + local.MCP[i].WorkspaceRoot = b.workspace + } + if err != nil { + return r, err + } + } + r.LocalEnvironment = &local + return r, nil +} + +// ApplyRuntimePreparation settles every filesystem operation before returning. A +// cancelled caller never leaves an unowned installation goroutine behind. +func (b *Binding) ApplyRuntimePreparation(ctx context.Context, input proto.RuntimePreparePayload, data []byte) error { + if b == nil || !b.Matches(input.EnvironmentID, input.SessionID) || !proto.ValidRuntimePrepareRequest(input) || input.Step != "begin" { + return agentcapabilities.ErrInvalid + } + b.capabilityMu.Lock() + defer b.capabilityMu.Unlock() + marker, err := b.openSnapshotMarker() + if err != nil { + return err + } + defer marker.close() + if marker.completed { + return agentcapabilities.ErrInvalid + } + if err := ctx.Err(); err != nil { + return err + } + switch input.Action { + case "file": + if len(data) != input.SizeBytes { + return agentcapabilities.ErrInvalid + } + return b.installInitialFile(ctx, *input.File, data) + case "initialize": + if len(data) != 0 { + return agentcapabilities.ErrInvalid + } + return b.initializeRuntime(ctx, *input.Initialization) + } + root, unlock, err := b.openCapabilities(ctx, false) + if err != nil { + return err + } + defer unlock() + switch input.Action { + case "skill": + err = agentcapabilities.InstallSkill(root, data, *input.Skill) + case "plugin": + err = agentcapabilities.InstallPlugin(root, input.Slot, data, *input.Plugin) + case "finalize": + _, err = b.loadCapabilitySnapshot(root, *input.Sources, b.capabilityIdentity(), false) + if err == nil { + err = marker.complete() + } + default: + err = agentcapabilities.ErrInvalid + } + if err != nil { + return agentcapabilities.ErrInvalid + } + return ctx.Err() +} + +func (b *Binding) loadCapabilitySnapshot(root *os.Root, input agentcapabilities.Input, identity agentcapabilities.Identity, completed bool) (agentcapabilities.Manifest, error) { + if _, err := root.Lstat(agentcapabilities.ManifestName); errors.Is(err, os.ErrNotExist) { + if completed || agentcapabilities.Finalize(root, input, identity, b.resolveCapabilityDirectory) != nil { + return agentcapabilities.Manifest{}, agentcapabilities.ErrInvalid + } + } else if err != nil { + return agentcapabilities.Manifest{}, agentcapabilities.ErrInvalid + } + manifest, err := agentcapabilities.Load(root) + if err != nil || agentcapabilities.ValidateSelection(manifest, input, identity) != nil { + return agentcapabilities.Manifest{}, agentcapabilities.ErrInvalid + } + return manifest, nil +} + +func (b *Binding) capabilityIdentity() agentcapabilities.Identity { + return agentcapabilities.Identity{EnvironmentID: b.environment, SessionID: strings.TrimPrefix(b.stateKey, "agents-api-")} +} + +// The current Linux Runtime layout is shared by user-owned and managed hosts. +// Deployment paths never come from a capability transport request. +func (b *Binding) openCapabilities(ctx context.Context, completed bool) (*os.Root, func(), error) { + if err := ctx.Err(); err != nil { + return nil, nil, err + } + if b.capabilityRoot == "" || !filepath.IsAbs(b.capabilityRoot) || !canonicalExistingParent(b.capabilityRoot) { + return nil, nil, agentcapabilities.ErrInvalid + } + if !completed { + if err := os.MkdirAll(b.capabilityRoot, 0700); err != nil { + return nil, nil, agentcapabilities.ErrInvalid + } + } + actual, err := filepath.EvalSymlinks(b.capabilityRoot) + if err != nil || actual != b.capabilityRoot { + return nil, nil, agentcapabilities.ErrInvalid + } + root, err := os.OpenRoot(b.capabilityRoot) + if err != nil { + return nil, nil, agentcapabilities.ErrInvalid + } + dir, err := root.Open(".") + if err != nil { + root.Close() + return nil, nil, agentcapabilities.ErrInvalid + } + if err = syscall.Flock(int(dir.Fd()), syscall.LOCK_EX|syscall.LOCK_NB); err != nil { + dir.Close() + root.Close() + return nil, nil, agentcapabilities.ErrInvalid + } + return root, func() { _ = syscall.Flock(int(dir.Fd()), syscall.LOCK_UN); dir.Close(); root.Close() }, nil +} + +func containsPath(parent, child string) bool { + relative, err := filepath.Rel(parent, child) + return err == nil && (relative == "." || filepath.IsLocal(relative)) +} + +func (b *Binding) resolveCapabilityDirectory(source string) (*os.Root, error) { + if agentcapabilities.ValidateLocalDirectories([]string{source}) != nil { + return nil, agentcapabilities.ErrInvalid + } + if source == "/workspace" || strings.HasPrefix(source, "/workspace/") { + source = filepath.Join(b.workspace, strings.TrimPrefix(source, "/workspace")) + } + actual, err := filepath.EvalSymlinks(source) + if err != nil || actual != source { + return nil, agentcapabilities.ErrInvalid + } + private, err := paths.Root() + if err != nil || !filepath.IsAbs(private) { + return nil, agentcapabilities.ErrInvalid + } + // Protect both descendants and ancestors: selecting a parent must not copy + // credentials, native history, installed capabilities or initialization env. + for _, protected := range []string{private, b.capabilityRoot, InitializationDirectory} { + canonical, e := filepath.EvalSymlinks(protected) + if e == nil { + protected = canonical + } else if !errors.Is(e, os.ErrNotExist) { + return nil, agentcapabilities.ErrInvalid + } + if containsPath(source, protected) || containsPath(protected, source) { + return nil, agentcapabilities.ErrInvalid + } + } + root, err := os.OpenRoot(source) + if err != nil { + return nil, agentcapabilities.ErrInvalid + } + return root, nil +} diff --git a/apps/parsar-daemon/internal/localworkspace/capabilities_test.go b/apps/parsar-daemon/internal/localworkspace/capabilities_test.go index be69fcb28..2713ae93e 100644 --- a/apps/parsar-daemon/internal/localworkspace/capabilities_test.go +++ b/apps/parsar-daemon/internal/localworkspace/capabilities_test.go @@ -11,7 +11,8 @@ import ( func TestCapabilityPathsStayRuntimeOwnedAndDoNotGateReads(t *testing.T) { binding, request := testBinding(t) - request.LocalEnvironment.Skills = []agentcapabilities.InstalledSkill{{RelativeRoot: "caller/private", PackageRoot: "caller"}} + request.LocalEnvironment.CapabilityRoot = "/caller/installation" + request.LocalEnvironment.Skills = []agentcapabilities.InstalledSkill{{RelativeRoot: "caller/private", PackageRoot: "caller", InstallationRoot: "/caller/installation"}} secret := "private-mcp-marker" request.LocalEnvironment.MCP = []proto.EnvironmentMCP{{PackageRoot: "caller/private", BearerToken: &secret}} raw, err := json.Marshal(request.LocalEnvironment) @@ -19,7 +20,7 @@ func TestCapabilityPathsStayRuntimeOwnedAndDoNotGateReads(t *testing.T) { t.Fatal("Runtime paths crossed the public daemon descriptor", err) } configured, err := binding.Configure(request) - if err != nil || len(configured.LocalEnvironment.Skills) != 0 || len(configured.LocalEnvironment.MCP) != 0 { + if err != nil || len(configured.LocalEnvironment.Skills) != 0 || len(configured.LocalEnvironment.MCP) != 0 || configured.LocalEnvironment.CapabilityRoot != "" { t.Fatal("execution accepted caller-supplied Skill paths", err) } request.LocalEnvironment.Capabilities = true diff --git a/apps/parsar-daemon/internal/localworkspace/capability_preparation_test.go b/apps/parsar-daemon/internal/localworkspace/capability_preparation_test.go new file mode 100644 index 000000000..656d5ddd5 --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/capability_preparation_test.go @@ -0,0 +1,147 @@ +package localworkspace + +import ( + "context" + "os" + "path/filepath" + "testing" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/google/uuid" +) + +func writeSourceSkill(t *testing.T, directory, text string) { + t.Helper() + if err := os.MkdirAll(directory, 0700); err != nil { + t.Fatal(err) + } + if err := os.WriteFile(filepath.Join(directory, "SKILL.md"), []byte("---\nname: local-proof\ndescription: Local preparation proof\n---\n"+text), 0600); err != nil { + t.Fatal(err) + } +} + +func TestPreparationFreezesLocalContentsAcrossReconnect(t *testing.T) { + b, req := testBinding(t) + source := t.TempDir() + writeSourceSkill(t, source, "first") + req.LocalEnvironment.Capabilities = true + req.LocalEnvironment.CapabilitySources = &agentcapabilities.Input{Directories: []string{source}} + configured, err := b.Configure(req) + if err != nil { + t.Fatal(err) + } + if _, err = os.Stat(filepath.Join(b.capabilityRoot, agentcapabilities.ManifestName)); !os.IsNotExist(err) { + t.Fatal("binding installed before asynchronous admission") + } + first, err := b.Prepare(t.Context(), configured) + if err != nil || len(first.LocalEnvironment.Skills) != 1 { + t.Fatalf("first preparation: %v", err) + } + writeSourceSkill(t, source, "second") + reconnect, err := New(b.environment, b.capabilityIdentity().SessionID, b.workspace, b.helper) + if err != nil { + t.Fatal(err) + } + reconnect.networkAccess = b.networkAccess + reconnect.capabilityRoot = b.capabilityRoot + again, err := reconnect.Prepare(t.Context(), configured) + if err != nil || len(again.LocalEnvironment.Skills) != 1 { + t.Fatalf("reconnection: %v", err) + } + frozen, err := os.ReadFile(filepath.Join(b.capabilityRoot, again.LocalEnvironment.Skills[0].RelativeRoot, "SKILL.md")) + if err != nil || string(frozen[len(frozen)-5:]) != "first" { + t.Fatal("reconnection recaptured source", err) + } + next, nextReq := testBinding(t) + nextReq.LocalEnvironment.Capabilities = true + nextReq.LocalEnvironment.CapabilitySources = req.LocalEnvironment.CapabilitySources + nextConfigured, err := next.Configure(nextReq) + if err != nil { + t.Fatal(err) + } + if _, err = next.Prepare(t.Context(), nextConfigured); err != nil { + t.Fatal(err) + } + fresh, err := os.ReadFile(filepath.Join(next.capabilityRoot, "directories/0/SKILL.md")) + if err != nil || string(fresh[len(fresh)-6:]) != "second" { + t.Fatal("new Session did not capture new source", err) + } + // Reusing a snapshot under another identity or selection cannot start native work. + reconnect.stateKey = "agents-api-" + uuid.NewString() + changed := configured + changed.AgentStateKey = reconnect.stateKey + if _, err = reconnect.Prepare(t.Context(), changed); err == nil { + t.Fatal("foreign snapshot accepted") + } + changed = configured + local := *configured.LocalEnvironment + local.CapabilitySources = &agentcapabilities.Input{} + changed.LocalEnvironment = &local + if _, err = b.Prepare(t.Context(), changed); err == nil { + t.Fatal("changed selection accepted") + } +} + +func TestPreparationRejectsPrivateSourcesAndLeavesFailuresInert(t *testing.T) { + b, req := testBinding(t) + private := t.TempDir() + t.Setenv("OAC_RUNTIME_HOME", private) + writeSourceSkill(t, private, "private") + for _, source := range []string{private, filepath.Dir(private), b.capabilityRoot, InitializationDirectory} { + if root, err := b.resolveCapabilityDirectory(source); err == nil { + root.Close() + t.Fatal("protected source accepted") + } + } + source := filepath.Join(b.workspace, "selected") + writeSourceSkill(t, source, "valid") + link := filepath.Join(b.workspace, "link") + if err := os.Symlink(source, link); err != nil { + t.Fatal(err) + } + if root, err := b.resolveCapabilityDirectory(link); err == nil { + root.Close() + t.Fatal("source alias accepted") + } + root, err := b.resolveCapabilityDirectory("/workspace/selected") + if err != nil { + t.Fatal("logical workspace source rejected", err) + } + root.Close() + req.LocalEnvironment.Capabilities = true + req.LocalEnvironment.CapabilitySources = &agentcapabilities.Input{Directories: []string{source, t.TempDir()}} + configured, err := b.Configure(req) + if err != nil { + t.Fatal(err) + } + if _, err = b.Prepare(t.Context(), configured); err == nil { + t.Fatal("invalid second source accepted") + } + if _, err = os.Stat(filepath.Join(b.capabilityRoot, agentcapabilities.ManifestName)); !os.IsNotExist(err) { + t.Fatal("partial snapshot ready") + } + if _, err = b.Prepare(t.Context(), configured); err == nil { + t.Fatal("partial snapshot silently replayed") + } +} + +func TestPreparationEmptySelectionAndCancellation(t *testing.T) { + b, req := testBinding(t) + configured, err := b.Configure(req) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithCancel(t.Context()) + cancel() + if _, err = b.Prepare(ctx, configured); err == nil { + t.Fatal("cancelled preparation started") + } + if _, err = b.Prepare(t.Context(), configured); err != nil { + t.Fatal(err) + } + read := proto.PromptRequestPayload{WorkspaceReadOnly: true} + if _, err = b.Prepare(t.Context(), read); err != nil { + t.Fatal("Files required capability installation", err) + } +} diff --git a/apps/parsar-daemon/internal/localworkspace/initialization.go b/apps/parsar-daemon/internal/localworkspace/initialization.go index 11c8a69c2..b1f87f0de 100644 --- a/apps/parsar-daemon/internal/localworkspace/initialization.go +++ b/apps/parsar-daemon/internal/localworkspace/initialization.go @@ -65,3 +65,12 @@ func VerifyToolEnvironment(systemPackages bool) error { } return nil } + +// ReadOptionalToolEnvironment permits a directory-only Runtime without user +// variables. Declared MCP variables still require the immutable explicit file. +func ReadOptionalToolEnvironment() (map[string]string, error) { + if _, err := os.Lstat(ToolEnvironmentJSON); errors.Is(err, os.ErrNotExist) { + return map[string]string{}, nil + } + return ReadToolEnvironment() +} diff --git a/apps/parsar-daemon/internal/localworkspace/mcp.go b/apps/parsar-daemon/internal/localworkspace/mcp.go index 312ee5e4d..b196fd857 100644 --- a/apps/parsar-daemon/internal/localworkspace/mcp.go +++ b/apps/parsar-daemon/internal/localworkspace/mcp.go @@ -13,7 +13,7 @@ const MCPInitializer = "/usr/local/bin/oac-runtime-initialize" // MCPStdioCommand contains only installed identities. The server's executable, // arguments and selected user variables are resolved after entering isolation. func MCPStdioCommand(server proto.EnvironmentMCP) (string, []string) { - return "/usr/bin/python3", []string{"-I", "-S", MCPInitializer, "stdio", server.PackageRoot, server.Server.Name} + return "/usr/bin/python3", []string{"-I", "-S", MCPInitializer, "stdio", server.InstallationRoot, server.WorkspaceRoot, server.PackageRoot, server.Server.Name} } func resolveEnvironmentMCP(installed []agentcapabilities.InstalledMCP, values map[string]string) ([]proto.EnvironmentMCP, error) { diff --git a/apps/parsar-daemon/internal/localworkspace/mcp_test.go b/apps/parsar-daemon/internal/localworkspace/mcp_test.go index a7d85cef4..36834c4de 100644 --- a/apps/parsar-daemon/internal/localworkspace/mcp_test.go +++ b/apps/parsar-daemon/internal/localworkspace/mcp_test.go @@ -28,11 +28,11 @@ func TestEnvironmentMCPCredentialsNeverFallBackToNativeEnv(t *testing.T) { } func TestMCPStdioLauncherContainsOnlyInstalledIdentity(t *testing.T) { - server := proto.EnvironmentMCP{PackageRoot: "plugins/0", Server: agentplugin.MCPServer{ + server := proto.EnvironmentMCP{InstallationRoot: "/private/runtime/capabilities", WorkspaceRoot: "/private/runtime/workspace", PackageRoot: "plugins/0", Server: agentplugin.MCPServer{ Name: "package_tool", Type: "stdio", Command: "untrusted-command", Args: []string{"private-argument"}, }} command, args := MCPStdioCommand(server) - if command != "/usr/bin/python3" || !slices.Equal(args, []string{"-I", "-S", MCPInitializer, "stdio", "plugins/0", "package_tool"}) { + if command != "/usr/bin/python3" || !slices.Equal(args, []string{"-I", "-S", MCPInitializer, "stdio", "/private/runtime/capabilities", "/private/runtime/workspace", "plugins/0", "package_tool"}) { t.Fatal("native configuration included untrusted process configuration") } } diff --git a/apps/parsar-daemon/internal/localworkspace/runtime_initialization.go b/apps/parsar-daemon/internal/localworkspace/runtime_initialization.go new file mode 100644 index 000000000..c8c639eb1 --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/runtime_initialization.go @@ -0,0 +1,11 @@ +package localworkspace + +import "errors" + +// InitializationFailure contains only a confirmed step's safe exit status. +type InitializationFailure struct{ ExitCode *int } + +func (*InitializationFailure) Error() string { return "Runtime initialization failed" } + +// ErrInitializationUnconfirmed means execution or its receipt was not confirmed. +var ErrInitializationUnconfirmed = errors.New("Runtime initialization unconfirmed") diff --git a/apps/parsar-daemon/internal/localworkspace/runtime_initialization_linux.go b/apps/parsar-daemon/internal/localworkspace/runtime_initialization_linux.go new file mode 100644 index 000000000..34603e821 --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/runtime_initialization_linux.go @@ -0,0 +1,174 @@ +//go:build linux + +package localworkspace + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/json" + "errors" + "io" + "strconv" + "strings" + "time" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent" + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/agent/clirunner" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func (b *Binding) initializeRuntime(ctx context.Context, input proto.RuntimeInitialization) error { + payload := struct { + Version int `json:"version"` + proto.RuntimeInitialization + }{Version: 1, RuntimeInitialization: input} + raw, err := json.Marshal(payload) + if err != nil || len(raw) > proto.RuntimePrepareMaxFrameBytes { + return agentcapabilities.ErrInvalid + } + result, err := runInitializationProcess(ctx, "/usr/bin/python3", []string{"-I", "-S", MCPInitializer, "initialize", b.workspace, b.capabilityRoot}, bytes.NewReader(raw)) + if err != nil { + return err + } + return decodeInitialization(result) +} + +func (b *Binding) installInitialFile(ctx context.Context, input proto.RuntimeInitialFile, data []byte) (err error) { + if b.writer == nil || !strings.HasPrefix(input.Path, "/workspace/") || len(data) > proto.RuntimePrepareMaxBytes { + return agentcapabilities.ErrInvalid + } + w := b.writer + if !w.mu.TryLock() { + return agentcapabilities.ErrInvalid + } + defer w.mu.Unlock() + if w.uncertain { + return ErrInitializationUnconfirmed + } + defer func() { w.uncertain = errors.Is(err, ErrInitializationUnconfirmed) }() + digest := sha256.Sum256(data) + result, err := runInitializationProcess(ctx, "/usr/bin/python3", []string{"-I", "-S", "-c", initialFileInstaller, + b.workspace, strings.TrimPrefix(input.Path, "/workspace/"), strconv.Itoa(len(data)), w.staging, w.helper}, + io.MultiReader(bytes.NewReader(data), bytes.NewReader(digest[:]))) + if err != nil { + return err + } + if result.exit != 0 || len(result.stderr) != 0 { + return ErrInitializationUnconfirmed + } + _, err = decodeWrite(result.stdout, len(data)) + if errors.Is(err, agent.ErrWorkspaceWriteRejected) { + return &InitializationFailure{} + } + if err != nil { + return ErrInitializationUnconfirmed + } + return nil +} + +// This only creates fd-anchored parents, then delegates replacement to the +// existing atomic writer. Deployment paths come exclusively from Binding. +const initialFileInstaller = `import os, sys +workspace, path, size, staging, helper = sys.argv[1:] +parts = path.split('/') +if any(not p or p in ('.', '..') or any(c in p for c in ('\\', '\x00', '\r', '\n')) for p in parts): + raise SystemExit(2) +flags = os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC +fd = os.open('/', flags) +for component in workspace.lstrip('/').split('/'): + child = os.open(component, flags, dir_fd=fd) + os.close(fd) + fd = child +for component in parts[:-1]: + try: + child = os.open(component, flags, dir_fd=fd) + except FileNotFoundError: + os.mkdir(component, mode=0o700, dir_fd=fd) + child = os.open(component, flags, dir_fd=fd) + os.close(fd) + fd = child +os.close(fd) +os.execv(helper, [helper, workspace, path, size, staging]) +` + +type initializationResult struct { + stdout, stderr []byte + exit int +} + +// runInitializationProcess uses the existing process-group owner and joins all +// readers/writers before returning. No child output becomes a public error. +func runInitializationProcess(ctx context.Context, binary string, args []string, input io.Reader) (initializationResult, error) { + result := initializationResult{exit: -1} + operation, cancel := context.WithTimeout(ctx, 30*time.Minute) + defer cancel() + if operation.Err() != nil { + return result, ErrInitializationUnconfirmed + } + process, err := clirunner.Start(clirunner.StartOptions{Parent: operation, Binary: binary, Args: args, + Dir: "/", Env: []string{"PATH=/usr/bin:/bin", "LANG=C.UTF-8"}, NeedStdin: true, + OwnProcessGroup: true, KillTimeout: 250 * time.Millisecond}) + if err != nil { + return result, ErrInitializationUnconfirmed + } + defer process.Cancel() + inputDone := make(chan error, 1) + go func() { + _, err := io.Copy(process.Stdin, input) + closeErr := process.Stdin.Close() + if err == nil { + err = closeErr + } + inputDone <- err + }() + type output struct { + data []byte + err error + } + read := func(stream io.Reader, done chan<- output) { + raw, err := io.ReadAll(io.LimitReader(stream, 1025)) + if err != nil || len(raw) > 1024 { + process.Cancel() + } + _, _ = io.Copy(io.Discard, stream) + done <- output{raw, err} + } + stdout, stderr := make(chan output, 1), make(chan output, 1) + go read(process.Stdout, stdout) + go read(process.Stderr, stderr) + out, diagnostic := <-stdout, <-stderr + inputErr := <-inputDone + _ = process.Wait() + result.stdout, result.stderr = out.data, diagnostic.data + result.exit = process.Cmd.ProcessState.ExitCode() + if operation.Err() != nil || inputErr != nil || out.err != nil || diagnostic.err != nil || + len(out.data) > 1024 || len(diagnostic.data) > 1024 || result.exit < 0 { + return result, ErrInitializationUnconfirmed + } + return result, nil +} + +func decodeInitialization(result initializationResult) error { + var receipt struct { + Version int `json:"version"` + Outcome string `json:"outcome"` + ExitCode *int `json:"exit_code"` + } + decoder := json.NewDecoder(bytes.NewReader(result.stdout)) + decoder.DisallowUnknownFields() + if len(result.stderr) != 0 || decoder.Decode(&receipt) != nil || decoder.Decode(new(any)) != io.EOF || receipt.Version != 1 { + return ErrInitializationUnconfirmed + } + if result.exit == 0 && receipt.Outcome == "completed" && receipt.ExitCode == nil { + return nil + } + if result.exit == 1 && receipt.Outcome == "failed" { + if receipt.ExitCode != nil && (*receipt.ExitCode < 1 || *receipt.ExitCode > 255) { + return ErrInitializationUnconfirmed + } + return &InitializationFailure{ExitCode: receipt.ExitCode} + } + return ErrInitializationUnconfirmed +} diff --git a/apps/parsar-daemon/internal/localworkspace/runtime_initialization_linux_test.go b/apps/parsar-daemon/internal/localworkspace/runtime_initialization_linux_test.go new file mode 100644 index 000000000..c318ae0f8 --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/runtime_initialization_linux_test.go @@ -0,0 +1,194 @@ +//go:build linux + +package localworkspace + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "os" + "path/filepath" + "strings" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func TestRuntimeInitializationSafeReceipts(t *testing.T) { + for _, tc := range []struct { + raw string + exit int + failed bool + code int + }{ + {`{"version":1,"outcome":"completed"}`, 0, false, 0}, + {`{"version":1,"outcome":"failed","exit_code":3}`, 1, true, 3}, + {`{"version":1,"outcome":"failed"}`, 1, true, 0}, + } { + err := decodeInitialization(initializationResult{stdout: []byte(tc.raw), exit: tc.exit}) + var failure *InitializationFailure + if tc.failed { + if !errors.As(err, &failure) || tc.code != 0 && (failure.ExitCode == nil || *failure.ExitCode != tc.code) { + t.Fatal("failed receipt", err) + } + } else if err != nil { + t.Fatal(err) + } + } + for _, raw := range []string{ + `{"version":1,"outcome":"failed","exit_code":0}`, + `{"version":1,"outcome":"failed","exit_code":256}`, + `{"version":1,"outcome":"failed","exit_code":3,"stderr":"secret"}`, + `{"version":1,"outcome":"completed","exit_code":3}`, + `{"version":1,"outcome":"failed"} {}`, + } { + if !errors.Is(decodeInitialization(initializationResult{stdout: []byte(raw), exit: 1}), ErrInitializationUnconfirmed) { + t.Fatal("unsafe receipt") + } + } +} + +func TestRuntimeInitializationProcessDrainsAndCancels(t *testing.T) { + t.Setenv("RUNTIME_TEST_PRIVATE", "do-not-inherit") + result, err := runInitializationProcess(t.Context(), "/bin/sh", []string{"-c", `test -z "$RUNTIME_TEST_PRIVATE" || exit 4; cat >/dev/null; printf '{"version":1,"outcome":"completed"}'`}, strings.NewReader("{}")) + if err != nil || decodeInitialization(result) != nil { + t.Fatal("clean process", err) + } + _, err = runInitializationProcess(t.Context(), "/bin/sh", []string{"-c", "head -c 2048 /dev/zero; sleep 5"}, strings.NewReader("")) + if !errors.Is(err, ErrInitializationUnconfirmed) { + t.Fatal("unbounded output accepted", err) + } + directory := t.TempDir() + ctx, cancel := context.WithCancel(t.Context()) + done := make(chan error, 1) + go func() { + _, err := runInitializationProcess(ctx, "/bin/sh", []string{"-c", `(trap '' TERM; sleep 1; touch "$1/late") & touch "$1/started"; wait`, "fixture", directory}, strings.NewReader("{}")) + done <- err + }() + deadline := time.Now().Add(3 * time.Second) + for { + if _, err := os.Stat(filepath.Join(directory, "started")); err == nil { + break + } + if time.Now().After(deadline) { + cancel() + <-done + t.Fatal("child did not start") + } + time.Sleep(5 * time.Millisecond) + } + cancel() + if err := <-done; !errors.Is(err, ErrInitializationUnconfirmed) { + t.Fatal("cancellation confirmed", err) + } + time.Sleep(1100 * time.Millisecond) + if _, err := os.Stat(filepath.Join(directory, "late")); !os.IsNotExist(err) { + t.Fatal("child survived return", err) + } +} + +func TestRuntimeInitializationPreservesSetupNetworkPhase(t *testing.T) { + for _, policy := range []string{"disabled", "restricted"} { + t.Run(policy, func(t *testing.T) { + b, _ := testBinding(t) + b.networkAccess = policy + t.Setenv("OAC_RUNTIME_NETWORK_ACCESS", policy) + // Initialization's typed network remains independent of native policy. + script := "import json, os, sys; request=json.load(sys.stdin); assert request['network']=='enabled'; assert 'OAC_RUNTIME_NETWORK_ACCESS' not in os.environ; print('{\"version\":1,\"outcome\":\"completed\"}')" + result, err := runInitializationProcess(t.Context(), "/usr/bin/python3", []string{"-I", "-S", "-c", script}, + strings.NewReader(`{"version":1,"action":"setup","network":"enabled","command":"true","cwd":"/workspace"}`)) + if err != nil || decodeInitialization(result) != nil || b.NetworkPolicy().Access != policy { + t.Fatal("initialization changed native network policy", err) + } + marker, err := b.openSnapshotMarker() + if err != nil { + t.Fatal(err) + } + if err = marker.complete(); err != nil { + marker.close() + t.Fatal(err) + } + marker.close() + input := proto.RuntimePreparePayload{Step: "begin", EnvironmentID: b.environment, SessionID: b.capabilityIdentity().SessionID, + Action: "initialize", Initialization: &proto.RuntimeInitialization{Action: "setup", Network: "enabled", Command: "true", CWD: "/workspace"}} + if err = b.ApplyRuntimePreparation(t.Context(), input, nil); !errors.Is(err, agentcapabilities.ErrInvalid) { + t.Fatal("initialization reopened after native admission", err) + } + }) + } +} + +func TestRuntimeInitialFileUsesBoundAtomicWriterAndAnchoredParents(t *testing.T) { + b := writableBinding(t, `[ "$#" = 4 ] || exit 7 +[ -z "$RUNTIME_TEST_PRIVATE" ] || exit 8 +cat >/dev/null +printf '{"version":1,"outcome":"completed","size_bytes":3}' +`) + t.Setenv("RUNTIME_TEST_PRIVATE", "secret") + if err := b.installInitialFile(t.Context(), proto.RuntimeInitialFile{Path: "/workspace/nested/child/file"}, []byte("abc")); err != nil { + t.Fatal(err) + } + info, err := os.Stat(filepath.Join(b.workspace, "nested/child")) + if err != nil || !info.IsDir() || info.Mode().Perm() != 0700 { + t.Fatal("parents", err) + } + outside := t.TempDir() + if err := os.Symlink(outside, filepath.Join(b.workspace, "escape")); err != nil { + t.Fatal(err) + } + if err := b.installInitialFile(t.Context(), proto.RuntimeInitialFile{Path: "/workspace/escape/new/file"}, []byte("abc")); err == nil { + t.Fatal("alias accepted") + } + if _, err := os.Stat(filepath.Join(outside, "new")); !os.IsNotExist(err) { + t.Fatal("escaped workspace") + } +} + +func TestRuntimePreparationRejectsFilesAfterFinalization(t *testing.T) { + b, req := testBinding(t) + configured, err := b.Configure(req) + if err != nil { + t.Fatal(err) + } + if _, err = b.Prepare(t.Context(), configured); err != nil { + t.Fatal(err) + } + digest := sha256.Sum256(nil) + input := proto.RuntimePreparePayload{Step: "begin", EnvironmentID: b.environment, SessionID: b.capabilityIdentity().SessionID, + Action: "file", File: &proto.RuntimeInitialFile{Path: "/workspace/file"}, SHA256: hex.EncodeToString(digest[:])} + if err = b.ApplyRuntimePreparation(t.Context(), input, nil); !errors.Is(err, agentcapabilities.ErrInvalid) { + t.Fatal("finalized Runtime accepted file", err) + } + input.Action = "initialize" + input.File = nil + input.SHA256 = "" + input.Initialization = &proto.RuntimeInitialization{Action: "configure", Env: map[string]string{}} + if err = b.ApplyRuntimePreparation(t.Context(), input, nil); !errors.Is(err, agentcapabilities.ErrInvalid) { + t.Fatal("finalized Runtime accepted initialize", err) + } +} + +func TestRuntimeInitialFileNativeAtomicReplacement(t *testing.T) { + helper := os.Getenv("OAC_TEST_LOCAL_WRITE_HELPER") + if helper == "" { + t.Skip("requires built native installer") + } + b := writableBinding(t, "exit 1\n") + if err := b.bindWriter(helper, b.writer.staging); err != nil { + t.Fatal(err) + } + path := "/workspace/n1/n2/file" + for _, body := range [][]byte{nil, []byte("first"), bytes.Repeat([]byte("x"), 1<<20), []byte("replacement")} { + if err := b.installInitialFile(t.Context(), proto.RuntimeInitialFile{Path: path}, body); err != nil { + t.Fatal(err) + } + actual, err := os.ReadFile(filepath.Join(b.workspace, "n1/n2/file")) + if err != nil || !bytes.Equal(actual, body) { + t.Fatal("atomic replacement", err) + } + } +} diff --git a/apps/parsar-daemon/internal/localworkspace/runtime_initialization_other.go b/apps/parsar-daemon/internal/localworkspace/runtime_initialization_other.go new file mode 100644 index 000000000..16ba0a627 --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/runtime_initialization_other.go @@ -0,0 +1,16 @@ +//go:build !linux + +package localworkspace + +import ( + "context" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +func (b *Binding) initializeRuntime(context.Context, proto.RuntimeInitialization) error { + return agentcapabilities.ErrInvalid +} +func (b *Binding) installInitialFile(context.Context, proto.RuntimeInitialFile, []byte) error { + return agentcapabilities.ErrInvalid +} diff --git a/apps/parsar-daemon/internal/localworkspace/skills.go b/apps/parsar-daemon/internal/localworkspace/skills.go index cefa50acd..5a9a90ca5 100644 --- a/apps/parsar-daemon/internal/localworkspace/skills.go +++ b/apps/parsar-daemon/internal/localworkspace/skills.go @@ -1,7 +1,6 @@ package localworkspace import ( - "os" "path/filepath" "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" @@ -9,20 +8,6 @@ import ( const CapabilityDirectory = agentcapabilities.Directory -// LoadCapabilities consumes the packaged installation after binding authorization. -func LoadCapabilities() (agentcapabilities.Manifest, error) { - actual, err := filepath.EvalSymlinks(CapabilityDirectory) - if err != nil || actual != CapabilityDirectory { - return agentcapabilities.Manifest{}, agentcapabilities.ErrInvalid - } - root, err := os.OpenRoot(CapabilityDirectory) - if err != nil { - return agentcapabilities.Manifest{}, agentcapabilities.ErrInvalid - } - defer root.Close() - return agentcapabilities.Load(root) -} - func SkillPath(skill agentcapabilities.InstalledSkill) string { - return filepath.Join(CapabilityDirectory, skill.RelativeRoot) + return filepath.Join(skill.InstallationRoot, skill.RelativeRoot) } diff --git a/apps/parsar-daemon/internal/localworkspace/skills_test.go b/apps/parsar-daemon/internal/localworkspace/skills_test.go new file mode 100644 index 000000000..a4038ea1c --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/skills_test.go @@ -0,0 +1,15 @@ +package localworkspace + +import ( + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "path/filepath" + "testing" +) + +func TestSkillPathUsesRuntimeInstallationRoot(t *testing.T) { + root := t.TempDir() + skill := agentcapabilities.InstalledSkill{InstallationRoot: root, RelativeRoot: "plugins/0/skills/proof"} + if got := SkillPath(skill); got != filepath.Join(root, "plugins/0/skills/proof") { + t.Fatal("Runtime root lost", got) + } +} diff --git a/apps/parsar-daemon/internal/localworkspace/snapshot_marker.go b/apps/parsar-daemon/internal/localworkspace/snapshot_marker.go new file mode 100644 index 000000000..2aaab5e8f --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/snapshot_marker.go @@ -0,0 +1,177 @@ +package localworkspace + +import ( + "encoding/json" + "errors" + "io" + "os" + "path/filepath" + "syscall" + + "github.com/MiniMax-AI-Dev/parsar/apps/parsar-daemon/internal/paths" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/google/uuid" +) + +// snapshotMarker remembers completion outside the installed tree. It contains +// only the operator root, never capability configuration or a second inventory. +type snapshotMarker struct { + directory *os.Root + lock *os.File + name string + root string + completed bool +} + +func (b *Binding) openSnapshotMarker() (*snapshotMarker, error) { + identity := b.capabilityIdentity() + for _, id := range []string{identity.EnvironmentID, identity.SessionID} { + parsed, err := uuid.Parse(id) + if err != nil || parsed == uuid.Nil || parsed.String() != id { + return nil, agentcapabilities.ErrInvalid + } + } + private, err := paths.Root() + if err != nil || agentcapabilities.ValidateLocalDirectories([]string{private}) != nil || private == "/" || + !canonicalExistingParent(private) { + return nil, agentcapabilities.ErrInvalid + } + canonical, err := filepath.EvalSymlinks(private) + if err != nil || canonical != private { + return nil, agentcapabilities.ErrInvalid + } + current, err := os.OpenRoot(private) + if err != nil { + return nil, agentcapabilities.ErrInvalid + } + for _, child := range []string{"daemon", "capability-installations"} { + if !privateSnapshotDirectory(current) { + current.Close() + return nil, agentcapabilities.ErrInvalid + } + err = current.Mkdir(child, 0700) + if err != nil && !errors.Is(err, os.ErrExist) { + current.Close() + return nil, agentcapabilities.ErrInvalid + } + created := err == nil + info, err := current.Lstat(child) + if err != nil || !info.IsDir() || info.Mode()&os.ModeSymlink != 0 { + current.Close() + return nil, agentcapabilities.ErrInvalid + } + if created { + parent, openErr := current.Open(".") + if openErr != nil { + current.Close() + return nil, agentcapabilities.ErrInvalid + } + syncErr := parent.Sync() + parent.Close() + if syncErr != nil { + current.Close() + return nil, agentcapabilities.ErrInvalid + } + } + next, err := current.OpenRoot(child) + current.Close() + if err != nil { + return nil, agentcapabilities.ErrInvalid + } + current = next + } + if !privateSnapshotDirectory(current) { + current.Close() + return nil, agentcapabilities.ErrInvalid + } + lock, err := current.Open(".") + if err != nil { + current.Close() + return nil, agentcapabilities.ErrInvalid + } + if syscall.Flock(int(lock.Fd()), syscall.LOCK_EX|syscall.LOCK_NB) != nil { + lock.Close() + current.Close() + return nil, agentcapabilities.ErrInvalid + } + marker := &snapshotMarker{directory: current, lock: lock, name: identity.EnvironmentID + "-" + identity.SessionID + ".json", root: b.capabilityRoot} + completed, err := marker.read() + if err != nil { + marker.close() + return nil, agentcapabilities.ErrInvalid + } + marker.completed = completed + return marker, nil +} + +func privateSnapshotDirectory(root *os.Root) bool { + file, err := root.Open(".") + if err != nil { + return false + } + defer file.Close() + info, err := file.Stat() + if err != nil || !info.IsDir() || info.Mode().Perm()&0077 != 0 { + return false + } + stat, ok := info.Sys().(*syscall.Stat_t) + return ok && stat.Uid == uint32(os.Getuid()) +} + +func (m *snapshotMarker) read() (bool, error) { + file, err := m.directory.OpenFile(m.name, os.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK, 0) + if errors.Is(err, os.ErrNotExist) { + return false, nil + } + if err != nil { + return false, err + } + defer file.Close() + info, err := file.Stat() + if err != nil { + return false, err + } + stat, ok := info.Sys().(*syscall.Stat_t) + if !ok || !info.Mode().IsRegular() || info.Mode().Perm() != 0600 || stat.Uid != uint32(os.Getuid()) || stat.Nlink != 1 || info.Size() > 8192 { + return false, agentcapabilities.ErrInvalid + } + // Exact bytes also reject duplicate members, trailing data and extra fields. + expected, _ := json.Marshal(struct { + Root string `json:"capability_root"` + }{m.root}) + actual, err := io.ReadAll(io.LimitReader(file, 8193)) + if err != nil || string(actual) != string(expected)+"\n" { + return false, agentcapabilities.ErrInvalid + } + return true, nil +} + +func (m *snapshotMarker) complete() error { + if m.completed { + return nil + } + data, err := json.Marshal(struct { + Root string `json:"capability_root"` + }{m.root}) + if err != nil { + return agentcapabilities.ErrInvalid + } + file, err := m.directory.OpenFile(m.name, os.O_WRONLY|os.O_CREATE|os.O_EXCL|syscall.O_NOFOLLOW, 0600) + if err != nil { + return agentcapabilities.ErrInvalid + } + _, writeErr := file.Write(append(data, '\n')) + syncErr := file.Sync() + closeErr := file.Close() + if writeErr != nil || syncErr != nil || closeErr != nil || m.lock.Sync() != nil { + return agentcapabilities.ErrInvalid + } + m.completed = true + return nil +} + +func (m *snapshotMarker) close() { + _ = syscall.Flock(int(m.lock.Fd()), syscall.LOCK_UN) + m.lock.Close() + m.directory.Close() +} diff --git a/apps/parsar-daemon/internal/localworkspace/snapshot_marker_test.go b/apps/parsar-daemon/internal/localworkspace/snapshot_marker_test.go new file mode 100644 index 000000000..ca0b6696c --- /dev/null +++ b/apps/parsar-daemon/internal/localworkspace/snapshot_marker_test.go @@ -0,0 +1,250 @@ +package localworkspace + +import ( + "context" + "encoding/json" + "os" + "path/filepath" + "testing" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" +) + +func markerBinding(t *testing.T) (*Binding, proto.PromptRequestPayload) { + t.Helper() + b, request := testBinding(t) + // This host's inherited ACL gives TempDir group permissions unless cleared. + if err := os.Chmod(os.Getenv("OAC_RUNTIME_HOME"), 0700); err != nil { + t.Fatal(err) + } + configured, err := b.Configure(request) + if err != nil { + t.Fatal(err) + } + return b, configured +} +func markerPath(b *Binding) string { + identity := b.capabilityIdentity() + return filepath.Join(os.Getenv("OAC_RUNTIME_HOME"), "daemon", "capability-installations", identity.EnvironmentID+"-"+identity.SessionID+".json") +} +func TestSnapshotMarkerPrivateRoundTrip(t *testing.T) { + b, _ := markerBinding(t) + m, err := b.openSnapshotMarker() + if err != nil { + t.Fatal(err) + } + if m.completed { + t.Fatal("fresh marker complete") + } + if err := m.complete(); err != nil { + t.Fatal(err) + } + m.close() + info, err := os.Stat(markerPath(b)) + if err != nil || info.Mode().Perm() != 0600 { + t.Fatal("marker not private", err) + } + raw, err := os.ReadFile(markerPath(b)) + if err != nil { + t.Fatal(err) + } + var fields map[string]string + if json.Unmarshal(raw, &fields) != nil || len(fields) != 1 || fields["capability_root"] != b.capabilityRoot { + t.Fatal("marker contains more than installation root") + } + m, err = b.openSnapshotMarker() + if err != nil { + t.Fatal(err) + } + defer m.close() + if !m.completed { + t.Fatal("completion not retained") + } + if err := m.complete(); err != nil { + t.Fatal("completed marker rewritten", err) + } +} + +func TestCompletedSnapshotLossNeverRecapturesSources(t *testing.T) { + for _, populated := range []bool{false, true} { + for _, removeRoot := range []bool{false, true} { + name := map[bool]string{false: "empty", true: "directory"}[populated] + "/" + map[bool]string{false: "manifest", true: "root"}[removeRoot] + t.Run(name, func(t *testing.T) { + b, request := markerBinding(t) + source := t.TempDir() + if populated { + writeSourceSkill(t, source, "first") + request.LocalEnvironment.Capabilities = true + request.LocalEnvironment.CapabilitySources = &agentcapabilities.Input{Directories: []string{source}} + } + if _, err := b.Prepare(t.Context(), request); err != nil { + t.Fatal(err) + } + marker, err := os.ReadFile(markerPath(b)) + if err != nil { + t.Fatal(err) + } + if removeRoot { + if err := os.RemoveAll(b.capabilityRoot); err != nil { + t.Fatal(err) + } + } else if err := os.Remove(filepath.Join(b.capabilityRoot, agentcapabilities.ManifestName)); err != nil { + t.Fatal(err) + } + if populated { + writeSourceSkill(t, source, "changed") + } + if _, err := b.Prepare(t.Context(), request); err == nil { + t.Fatal("lost snapshot recreated") + } + if _, err := os.Stat(filepath.Join(b.capabilityRoot, agentcapabilities.ManifestName)); !os.IsNotExist(err) { + t.Fatal("manifest recreated", err) + } + if removeRoot { + if _, err := os.Stat(b.capabilityRoot); !os.IsNotExist(err) { + t.Fatal("installation root recreated", err) + } + } + after, _ := os.ReadFile(markerPath(b)) + if string(after) != string(marker) { + t.Fatal("completion evidence changed") + } + }) + } + } +} + +func TestSnapshotMarkerBackfillsOnlyVerifiedManifest(t *testing.T) { + b, request := markerBinding(t) + source := t.TempDir() + writeSourceSkill(t, source, "frozen") + input := agentcapabilities.Input{Directories: []string{source}} + root, err := os.OpenRoot(b.capabilityRoot) + if err != nil { + t.Fatal(err) + } + if err := agentcapabilities.Finalize(root, input, b.capabilityIdentity(), b.resolveCapabilityDirectory); err != nil { + t.Fatal(err) + } + root.Close() + if err := os.RemoveAll(source); err != nil { + t.Fatal(err) + } + request.LocalEnvironment.Capabilities = true + request.LocalEnvironment.CapabilitySources = &input + if _, err := b.Prepare(t.Context(), request); err != nil { + t.Fatal("valid manifest was not recovered without sources", err) + } + if _, err := os.Stat(markerPath(b)); err != nil { + t.Fatal("verified completion not recorded", err) + } + previousRoot := b.capabilityRoot + b.capabilityRoot = t.TempDir() + if _, err := b.Prepare(t.Context(), request); err == nil { + t.Fatal("operator root changed after completion") + } + if _, err := os.Stat(filepath.Join(b.capabilityRoot, agentcapabilities.ManifestName)); !os.IsNotExist(err) { + t.Fatal("replacement root installed", err) + } + b.capabilityRoot = previousRoot +} + +func TestSnapshotMarkerRefusesDamagedOrAliasedPrivateState(t *testing.T) { + for _, kind := range []string{"corrupt", "foreign root", "public file", "symlink file", "hardlink file", "symlink directory", "symlink private root"} { + t.Run(kind, func(t *testing.T) { + b, request := markerBinding(t) + if _, err := b.Prepare(t.Context(), request); err != nil { + t.Fatal(err) + } + name := markerPath(b) + switch kind { + case "corrupt": + if err := os.WriteFile(name, []byte("{"), 0600); err != nil { + t.Fatal(err) + } + case "foreign root": + if err := os.WriteFile(name, []byte("{\"capability_root\":\"/another\"}\n"), 0600); err != nil { + t.Fatal(err) + } + case "public file": + if err := os.Chmod(name, 0644); err != nil { + t.Fatal(err) + } + case "symlink file": + saved := filepath.Join(t.TempDir(), "saved") + if err := os.Rename(name, saved); err != nil { + t.Fatal(err) + } + if err := os.Symlink(saved, name); err != nil { + t.Fatal(err) + } + case "hardlink file": + if err := os.Link(name, filepath.Join(t.TempDir(), "linked")); err != nil { + t.Fatal(err) + } + case "symlink directory": + saved := filepath.Join(filepath.Dir(filepath.Dir(name)), "moved") + if err := os.Rename(filepath.Dir(name), saved); err != nil { + t.Fatal(err) + } + if err := os.Symlink(saved, filepath.Dir(name)); err != nil { + t.Fatal(err) + } + case "symlink private root": + alias := filepath.Join(t.TempDir(), "private") + if err := os.Symlink(os.Getenv("OAC_RUNTIME_HOME"), alias); err != nil { + t.Fatal(err) + } + t.Setenv("OAC_RUNTIME_HOME", alias) + } + before, _ := os.ReadFile(name) + if _, err := b.Prepare(t.Context(), request); err == nil { + t.Fatal("invalid completion evidence accepted") + } + after, _ := os.ReadFile(name) + if string(before) != string(after) { + t.Fatal("invalid evidence overwritten") + } + }) + } +} + +func TestCapabilityFinalizeRecordsCompletionAndRejectsLaterImports(t *testing.T) { + b, _ := markerBinding(t) + identity := b.capabilityIdentity() + finalize := proto.RuntimePreparePayload{Step: "begin", EnvironmentID: identity.EnvironmentID, SessionID: identity.SessionID, Action: "finalize", Sources: &agentcapabilities.Input{}} + if err := b.ApplyRuntimePreparation(t.Context(), finalize, nil); err != nil { + t.Fatal(err) + } + if _, err := os.Stat(markerPath(b)); err != nil { + t.Fatal("finalize omitted marker", err) + } + // Even a fresh empty root cannot make an already completed identity writable. + if err := os.RemoveAll(b.capabilityRoot); err != nil { + t.Fatal(err) + } + skill := proto.RuntimePreparePayload{Step: "begin", EnvironmentID: identity.EnvironmentID, SessionID: identity.SessionID, Action: "skill", Skill: &agentskill.Metadata{Type: "inline", Name: "example", Description: "Example"}, SizeBytes: 1, SHA256: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"} + if err := b.ApplyRuntimePreparation(t.Context(), skill, []byte("x")); err == nil { + t.Fatal("completed identity accepted import") + } + if err := b.ApplyRuntimePreparation(t.Context(), finalize, nil); err == nil { + t.Fatal("completed identity finalized again") + } + if _, err := os.Stat(b.capabilityRoot); !os.IsNotExist(err) { + t.Fatal("rejected operation recreated root", err) + } +} + +func TestSnapshotMarkerCancellationDoesNotPublishCompletion(t *testing.T) { + b, request := markerBinding(t) + ctx, cancel := context.WithCancel(t.Context()) + cancel() + if _, err := b.Prepare(ctx, request); err == nil { + t.Fatal("cancelled preparation accepted") + } + if _, err := os.Stat(markerPath(b)); !os.IsNotExist(err) { + t.Fatal("cancelled preparation published marker", err) + } +} diff --git a/apps/parsar-daemon/testdata/onboarding/main.go b/apps/parsar-daemon/testdata/onboarding/main.go index 67a29f92b..679f98338 100644 --- a/apps/parsar-daemon/testdata/onboarding/main.go +++ b/apps/parsar-daemon/testdata/onboarding/main.go @@ -28,34 +28,81 @@ func (s *sender) Send(_ context.Context, e proto.Envelope) error { return s.encoder.Encode(e) } -type harness struct{ history map[string]string } +type harness struct { + mu sync.Mutex + history map[string]string +} -func (h *harness) start(_ context.Context, req proto.PromptRequestPayload, out chan<- proto.Envelope) (agent.Session, error) { - if _, err := req.Input.TextOnly(); err != nil { - return nil, err +func (h *harness) prepare(_ context.Context, req proto.PromptRequestPayload) (agent.Executor, error) { + if req.RunID != "" || len(req.Input) != 0 || req.ConversationID != "" { + return nil, errors.New("preparation submitted fixture input") } - if !req.StrictResume || !req.ReleaseOnCompletion || !req.DisableExecutionEnvironment || !req.DisableSubagents || len(req.FunctionTools) > 0 || req.MCPHTTPServers != nil { + if !req.StrictResume || !req.DisableExecutionEnvironment || !req.DisableSubagents || len(req.FunctionTools) > 0 || req.MCPHTTPServers != nil { return nil, errors.New("unsupported fixture operation") } + h.mu.Lock() + defer h.mu.Unlock() previous := h.history[req.AgentStateKey] - if req.AgentSessionID != previous || req.RequireExistingNativeSession { + if req.AgentSessionID != previous || (req.RequireExistingNativeSession && previous == "") { return nil, errors.New("native history mismatch") } - id := previous - if id == "" { - id = "fixture-" + req.AgentStateKey - h.history[req.AgentStateKey] = id + if previous == "" { + previous = "fixture-" + req.AgentStateKey + h.history[req.AgentStateKey] = previous + } + return &executor{native: previous}, nil +} + +type executor struct { + mu sync.Mutex + native string + active *session + closed bool +} + +func (e *executor) StartTurn(ctx context.Context, run string, input proto.MessageInput, out chan<- proto.Envelope) (agent.Turn, error) { + if ctx.Err() != nil || run == "" || out == nil { + return nil, errors.New("invalid fixture Start") + } + if _, err := input.TextOnly(); err != nil { + return nil, err + } + e.mu.Lock() + defer e.mu.Unlock() + if e.closed || e.active != nil { + return nil, errors.New("fixture Executor unavailable") + } + s := &session{out: out, run: run, native: e.native, settled: make(chan struct{})} + s.release = func() { + e.mu.Lock() + if e.active == s { + e.active = nil + } + e.mu.Unlock() } - s := &session{out: out, run: req.RunID, native: id} + e.active = s s.emit(proto.TypeDelta, proto.DeltaPayload{Delta: "ready", Sequence: 1}) return s, nil } +func (e *executor) Close(ctx context.Context) error { + e.mu.Lock() + e.closed = true + current := e.active + e.mu.Unlock() + if current != nil { + return current.Cancel(ctx) + } + return nil +} + type session struct { mu sync.Mutex out chan<- proto.Envelope run, native string closed bool + settled chan struct{} + release func() } func (s *session) emit(kind string, payload any) { @@ -68,6 +115,8 @@ func (s *session) Cancel(context.Context) error { if !s.closed { s.closed = true close(s.out) + s.release() + close(s.settled) } return nil } @@ -92,15 +141,29 @@ func (s *session) SteerWithReceipt(_ context.Context, p proto.PromptSteerPayload s.emit(proto.TypeDone, proto.DonePayload{Content: "ready" + text, Metadata: map[string]any{proto.DoneMetaAgentSessionID: s.native}}) s.closed = true close(s.out) + s.release() + close(s.settled) return nil } +func (s *session) AwaitSettlement(ctx context.Context) (agent.TurnSettlement, error) { + select { + case <-s.settled: + return agent.TurnSettlement{Reusable: true}, nil + case <-ctx.Done(): + return agent.TurnSettlement{}, ctx.Err() + } +} + func run() error { registry := agent.NewRegistry() h := &harness{history: map[string]string{}} registry.RegisterKind(proto.SupportedAgentKind{Kind: "fixture_harness", Available: true, Capabilities: proto.AgentKindCapabilities{ Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, ExecutionControls: true, ToolObservations: true, SubagentControl: true, EnvironmentNone: true, - }}, h.start) + }}, func(context.Context, proto.PromptRequestPayload, chan<- proto.Envelope) (agent.Session, error) { + return nil, errors.New("fixture execution requires an Executor") + }) + registry.RegisterExecutor("fixture_harness", h.prepare) sink := &sender{encoder: json.NewEncoder(os.Stdout)} router, err := dispatch.New(dispatch.Config{Registry: registry, Sender: sink, Log: slog.New(slog.NewTextHandler(io.Discard, nil))}) if err != nil { diff --git a/contracts/agents-api/environment-files.md b/contracts/agents-api/environment-files.md index 0ca4916f4..671ee4d02 100644 --- a/contracts/agents-api/environment-files.md +++ b/contracts/agents-api/environment-files.md @@ -279,10 +279,12 @@ Decisions: - The shared `oac-codex-write` helper takes an explicit per-call mode. Only the daemon's local workspace writer, which serves only Files.create, passes - `create`. Core's initial Session file installer and the Skill installer keep the - four-argument replace mode, which older Runtime images also understand. Plugins - use the separate capability installer, and completed recovery, including cold - resume, never reinstalls files. Their semantics are unchanged. + `create`. Core's initial Session file installer keeps the four-argument replace + mode. At this historical acceptance, the Skill installer also used that helper; + current Skills and Plugins use the + [shared Runtime capability preparation](environments.md#runtime-capability-preparation). + Completed recovery, including cold resume, never reinstalls files. The Files.create + semantics recorded here remain unchanged. - Parents are created only after the complete body is verified, so incomplete or corrupt input creates nothing. Each new directory is reopened without following links. The install itself never replaces: `renameat2(RENAME_NOREPLACE)`, or diff --git a/contracts/agents-api/environment-templates.md b/contracts/agents-api/environment-templates.md index 10f284c8d..f548fc5f5 100644 --- a/contracts/agents-api/environment-templates.md +++ b/contracts/agents-api/environment-templates.md @@ -73,8 +73,9 @@ Session, omission and null inherit, while a supplied list replaces the entire fi set, including `[]` clearing it. Paths are not merged between the two sources. The effective list retains the existing validation and tenant-owned source checks. -Core initializes both paths with the same trusted file installer through Provider -RunCommand. Daemon authentication remains available, but native preparation and live +Core initializes both paths through the authenticated daemon's typed `runtime_prepare` +file operation. The Runtime resolves the logical workspace path and owns the trusted +file installer. Daemon authentication remains available, but native preparation and live Files wait for all writes. Each file gets a two-minute transfer budget; the batch has a thirty-minute local budget and shares maintenance scans with other allocations. These are local operational limits, not verified upstream timing. Initial input @@ -126,14 +127,14 @@ in its creation transaction. Later source deletion, default changes or template updates cannot change that Session or its committed creation retry. A supplied Session Skill list replaces the template list; omission/null inherit. Template responses include `version: null` for an unresolved default selector; resolved -Session references retain a concrete version string. See [resource selector -qualification](resource-selector-semantics.md) and [null selection evidence](template-null-selection.md). +Session references retain a concrete version string. See +[resource selector qualification](resource-selector-semantics.md) and [null selection evidence](template-null-selection.md). References return type/skill_id/version/name/description in Session metadata, while template responses retain unresolved selectors. Confidential bundle content never appears in these metadata responses. The common Runtime installation path receives frozen files and descriptive metadata, without source or template IDs. -Inline Skill ZIPs use the same initializer: +Inline Skill ZIPs use the shared Runtime capability installer: ```python import base64 @@ -163,10 +164,15 @@ resource-bound template and Session snapshots. Updates replace supplied `skills` omission preserves and null/[] clears. Existing Sessions retain their frozen content after template update/deletion. -The shared initializer installs Skills under -`/environment/initialization/capabilities/skills/` before setup and native execution. -Setup and native tools can read that tree but cannot write it; completed recovery -never reinstalls it. The public descriptor requests capability installation; the common daemon resolves protected metadata and paths for native preparation. +Core transfers frozen Skill archives over the authenticated Runtime connection. +Runtime's shared parser installs them under its operator-bound capability root +(default `/environment/initialization/capabilities/skills/` for each Skill) +before setup and native execution; +The same daemon handles initial files, configuration, packages and setup; Providers +only bootstrap it and manage compute resources. Setup and native tools can read +that tree but cannot write it. Completed recovery loads the protected installation +without reinstalling. The common Runtime resolves installed metadata and paths before +invoking the native executor factory. Codex registers native extra roots, Claude creates its own explicit Skill plugin envelope, and MiniMax points its native user-global catalog at the shared root. MiniMax retains disabled unrestricted built-in tools and uses its existing @@ -192,18 +198,23 @@ Referencing Session Plugin lists inherit on omission/null and replace when a non-null list is supplied, including empty-list clearing. Template resource updates continue to accept null/empty clearing. -`capability_directories` currently accepts clean absolute paths within `/workspace`. -Initial files and setup can populate them. The shared initializer snapshots these -directories after setup, then publishes one protected installed manifest. Recovery -uses those installed bytes even if the source directory changes or is removed. -This timing and workspace restriction are local implementation choices, not claims -about unspecified upstream behavior. A supplied Session directory list replaces +Hosted Template and inline `capability_directories` accept clean absolute paths +within `/workspace`; self-hosted local selections have their +[separate public input](environments.md#runtime-capability-preparation). +Initial files and setup can populate hosted directories. Runtime snapshots them after +setup and writes the shared protected `installed.json`, bound to the Session, +Environment and source-selection digest. Recovery loads those installed bytes even +if a source directory changes or is removed; a new Session captures its own snapshot. +Partial or conflicting installation fails rather than silently recapturing sources. +This timing and the hosted path restriction are local implementation choices, not +claims about unspecified upstream behavior. A supplied Session directory list replaces the template list; omitted/null lists inherit. Missing, overlapping duplicate Skill names, unsupported manifests and nonregular files reject initialization without publishing completion. Directory-discovered Skills do not become fabricated inline entries in public `skills` or `plugins` metadata. -The common daemon resolves the installed manifest before executable preparation; +The common Runtime resolves the installed manifest during admitted asynchronous +preparation, before creating a native executor; read-only Files operations retain their existing minimal requirements. Native adapters consume Runtime-owned Skill and package roots. Codex uses explicit roots, MiniMax uses its native registry and Claude uses controlled envelopes with explicit @@ -218,8 +229,8 @@ combined installed capabilities are limited to 50 Skills and 50 MiB. Limits are implementation bounds. Native shell preprocessing, dependency activation and nested discovery retain the existing adapter restrictions. -Real standalone Docker acceptance on 2026-09-21 passed with the fixed official -SDK and raw HTTP against current Core/daemon/adapters on qualified native images: +Historical standalone Docker acceptance on 2026-09-21 passed with the fixed official +SDK and raw HTTP against the then-current Core/daemon/adapters on qualified native images: Codex/Kimi (258.08s), Claude/Kimi (187.81s) and MiniMax Code/MiniMax (250.23s). Each exercised two Plugin Skills with package-relative resources and one directory Skill generated during setup, public Files/Artifacts and tenant rejection, retained @@ -237,7 +248,8 @@ fixture is `services/agents-api/tests/official_environment_plugins.py`; operator runners reuse existing standalone acceptance and private model configuration. A user-authorized reused-context GPT-6 Astra high independent review of all 60 changed files found no material actionable findings. Those historical results do -not qualify Plugin MCP. Portable root `plugin.json` applicability and the unconfirmed +not qualify Plugin MCP or the later shared Runtime transport and self-hosted directory +preparation. Portable root `plugin.json` applicability and the unconfirmed semantics above remain gaps; these results do not establish complete Environment Templates or protocol compatibility. @@ -387,30 +399,37 @@ Templates workflow, not all upstream semantics, transports or Provider combinati ## Packaged Runtime initialization contract Template handlers and stores resolve public configuration without choosing a -harness, native path or compute backend. The common initialization lifecycle uses -the following existing Linux Runtime packaging requirements through Provider -`RunCommand`; these are private deployment requirements, not public Template fields. - -- `/workspace` is the public workspace. `/environment/workspace` names the same - storage for trusted initialization; `/environment/staging` is private staging. -- `/usr/bin/python3 -I -S` runs the trusted, fd-anchored initial-file installer. - It invokes the existing `/usr/local/bin/oac-codex-write` atomic writer in - its four-argument replace mode; only public Files.create uses the create mode. - That executable is a shared filesystem helper packaged for every harness; its - historical name does not select Codex or invoke native Codex tools. -- Confidential content travels on bounded stdin. Successful initialization needs - the writer's versioned completion receipt and confirmed process exit. Unknown - effects use the existing allocation cleanup path rather than replay. -- Provider implementations preserve argv, stdin, exit status and allocation - ownership. They do not interpret public templates. Runtime adapters own native - configuration; initialization must not consume a harness's private history, - model credentials or native tool protocol. - -New hosted harnesses reuse these helpers and paths; new Providers deploy the same -Runtime contract. Neither addition should change template validation, storage or -resolution. Extend this contract only for an accepted initialization requirement. +harness, native path or compute backend. The common runner depends on neutral +Environment/Session identity and an authenticated Runtime peer. It sends initial +files, configure/system/npm/python/setup operations, Skills, Plugins and finalization +through `runtime_prepare`, then uses the same daemon for execution. Providers own +placement, creation, daemon bootstrap, inspection, renewal and reclamation; they do +not run Core initialization commands. Harness-specific behavior stays in adapters. + +The protocol accepts logical `/workspace` file and working-directory addresses and +portable absolute source selections. Physical paths, executable selection and local +access checks belong to Runtime. The current implementation uses these existing +Linux packaging requirements; they are not public Template fields or requirements +for future Runtime implementations on other platforms: + +- `/workspace` is the logical workspace. The packaged Linux default binds it to + `/environment/workspace`; `/environment/staging` is private staging. +- The daemon invokes the fixed trusted, fd-anchored initial-file installer and the + shared atomic writer. These helpers do not select a harness or invoke native + model tools. +- Confidential content travels in bounded protocol frames and private helper stdin. + The daemon verifies native completion before sending a typed Runtime receipt. + Unknown effects use the existing allocation cleanup path rather than replay. +- Runtime adapters own native configuration. Initialization cannot access a + harness's private history, model credentials or native tool protocol. + +New Providers bootstrap the same Runtime contract. New harnesses reuse common +preparation, with native differences confined to their adapters. Neither addition +changes template validation, storage or resolution. This interface separation does +not claim a non-Linux installer, layout or isolation implementation. The trusted `/usr/local/bin/oac-runtime-initialize` receives a bounded -version-1 JSON operation on stdin. It configures read-only tool env under +version-1 JSON operation on stdin from the Linux daemon. This private helper format +is not the Core-to-Runtime protocol. It configures read-only tool env under `/environment/initialization`, installs packages under `/environment/packages`, and runs ordered commands through distro bubblewrap. The fixed mount/process map excludes daemon credentials, native history and staging. User values are applied @@ -424,7 +443,9 @@ Runtime receives a `tool_environment` execution flag, without template identity provider information. Adapters validate the common files and apply them in their native tool sandbox: Claude uses its native Bash hook, Codex its managed Bash hook, and MiniMax its isolated native-tool worker. Native transports remain unchanged. -Files reads do not require initialized tool configuration. Docker setup requires +Files reads do not require initialized tool configuration. Packages and setup keep +enabled provisioning network access; requested Session network restrictions apply +to native execution after setup. Docker setup requires the existing nested-sandbox deployment profile for every harness; E2B supplies the same Runtime layout and kernel isolation. @@ -435,6 +456,10 @@ exist; this is not an atomic hook-failure prevention guarantee. ## Initialization failure — September 23 +The September 23 observations below are historical acceptance evidence. Current +initialization uses the shared daemon protocol described above; these observations +do not qualify that new transport against a real deployment. + When a hosted Environment fails to provision, Core now reports it the way the official service does (evidence and rows H1–H8 in [official semantics](official-semantics-alignment.md#hosted-initialization-failure--september-23)). @@ -452,19 +477,19 @@ The reason names only the failed step and its exit status: | Setup command `i` | `Failed to provision environment: script "setup_commands[i]" failed with exit code N` (observed) | | Python packages | `... script "Python package installation" failed with exit code N` (observed label; the official reason appends raw pip output, Core never does) | | npm or system packages | `... script "npm package installation"` / `"System package installation"` `failed with exit code N` (unverified) | -| Initial file or Skill with a confirmed failed write | `Failed to provision environment: initial file installation failed` / `Skill installation failed` (unverified) | +| Initial file write or Runtime Skill preparation with a confirmed failure | `Failed to provision environment: initial file installation failed` / `Skill installation failed` (unverified) | | Anything else | `Failed to provision environment: initialization did not complete` | "Anything else" covers timeouts, the thirty-minute budget, unknown effects, -missing or malformed receipts, receipts without `exit_code` from Runtime images -built before this change, Plugin and capability installation, bootstrap -rejection and Core restart during initialization. Core treats a step as -confirmed failed only when the process exits 1 with empty stderr and stdout -decodes as a version-1 receipt whose `outcome` is `failed`. The decoder is -deliberately lenient so older images keep working: other receipt fields are -ignored and never read. The only value ever taken from the receipt is -`exit_code`, used only when it is an integer from 1 to 255. The Store composes the -reason from a fixed label and integers, so commands, env values, package names, +missing or malformed receipts, Plugin installation and directory finalization, +bootstrap rejection and Core restart during initialization. All initialization +operations return typed Runtime `rejected`, `failed` or `unknown` outcomes. The +Linux daemon confirms the trusted helper's process exit and private receipt before +reporting completion or failure. Only `failed` can carry a bounded `exit_code`; +For setup and package steps, Core projects a confirmed nonzero status, never +process output; zero or missing status retains the generic reason. Only a confirmed +Skill preparation failure gets the Skill label. Plugin/finalization failures and +uncertain effects retain the generic reason. The Store composes the reason from a fixed label and integers, so commands, env values, package names, paths and any process output never reach the reason, events, logs or responses. The failed step is not retried and later steps do not run. diff --git a/contracts/agents-api/environments.md b/contracts/agents-api/environments.md index 45b6d8755..f0bf8ea7d 100644 --- a/contracts/agents-api/environments.md +++ b/contracts/agents-api/environments.md @@ -3,7 +3,10 @@ This assessment covers the fixed [Python SDK contract](upstream.json), with partial coverage. Core-managed Docker runs the three qualified native harnesses. V1 user-managed `self_hosted` enrollment uses the same colocated Runtime for Codex, -Claude SDK and MiniMax at `/workspace`; see its [real deployment qualification](user-managed-runtime-v1.md). +Claude SDK and MiniMax. `/workspace` names the packaged workspace; a self-hosted +Session may instead select its exact canonical physical directory. The +[recorded deployment qualification](user-managed-runtime-v1.md) retains its historical +source, paths and tested capability scope. For hosted compute, the deployment selects E2B, Docker or microsandbox through [Hosted Sandbox Manager](../../services/agents-api/HOSTED-SANDBOX-MANAGER.md). For the separate caller-managed E2B path, the user owns allocation, renewal and @@ -45,7 +48,12 @@ an existing allocation's Create. Omitted/null network defaults to enabled. Enabled, disabled and exact-host restricted policies use the same qualified image with adapter-selected immutable native policy. Templates and inline configuration share initial files, env, packages, ordered setup -and inline or tenant-owned referenced Skills through the hosted initializer. +and inline or tenant-owned referenced Skills through ordered managed initialization. +The same authenticated daemon handles initial files, tool configuration, packages, +setup, Skill/Plugin import and directory snapshot finalization through +`runtime_prepare`. Providers place, create, bootstrap, inspect, renew and reclaim +compute. Package installation and setup retain enabled provisioning network access; +the requested network policy constrains native execution after setup. Unsupported hostname forms and installation combinations reject explicitly; see the [Template coverage and limits](environment-templates.md). Empty/null installation defaults produce safe empty metadata, not a live workspace inventory. Service-origin @@ -78,7 +86,8 @@ Current deployment-managed E2B is a separate hosted configuration in the manager ## Initial public self-hosted profile Create a Session with `environment.type=self_hosted`, -`workspace_directory: "/workspace"` and omitted/null/empty `capability_directories`. Creation +a clean absolute `workspace_directory` and optional absolute local +`capability_directories`. `/workspace` maps to the Runtime-bound workspace. Creation accepts initial text as a string or ordered user-message array. Omitted/null input creates no Turn or connection action. Configured execution, an enabled harness and the exact local profile are validated @@ -122,9 +131,15 @@ existing function parser and remain fixed through native preparation and continu output/error field presence and ordered content keep their existing semantics. Retries retain the original call, including during later work. New results cannot bypass pending input. Function callbacks do not populate Environment installations. -Mixed events, non-text input and nonempty capability directories remain unsupported. -The public field types are unchanged; `/workspace` is the V1 deployment limit, not -an upstream schema change. `remote_url` is the configured daemon WebSocket URL, +Mixed events and non-text input remain unsupported. Local capability directories +use the same Runtime parser and installed snapshot as managed Skills and Plugins; +preparation must finish before native execution. Reconnect reuses installed bytes, +while new Sessions capture their own sources. Directory aliases and protected Runtime +state are rejected. The current implementation uses the packaged Linux Runtime +layout; cross-platform installation and isolation are not part of this profile. +The public field types are unchanged. Enrollment must match either the `/workspace` +logical alias or the exact canonical directory bound by the Runtime; selecting an +arbitrary path does not grant access. `remote_url` is the configured daemon WebSocket URL, returned unchanged. This is our private connection contract and does not claim stock `exec-server` compatibility. Service-origin HTTP MCP is explicitly rejected on `self_hosted`; `none` MCP and hosted Template Plugin MCP retain their own scope. @@ -135,6 +150,49 @@ exercise fixed SDK/raw HTTP creation/input, actual native tools, Files/Artifacts second-Turn history, Core/Runtime restart, cancellation and credential rotation/ revocation/deletion on each declared deployment. +## Runtime capability preparation + +Core freezes resource versions, metadata and source selections. Managed initialization +keeps initial files and tool configuration before Runtime Skill/Plugin import, +then runs packages and ordered setup before Runtime snapshots local capability +directories. All steps use the common `runtime_prepare` protocol; Providers do +not execute Core initialization commands. The common runner depends on neutral +Environment/Session identity and a Runtime peer, not Provider, deployment or OS. Self-hosted input +accepts only local `capability_directories` with `workspace_directory`, not managed +Skill/Plugin archive fields or a hosted Template. Local directory discovery does not +populate the public API-managed `skills` or `plugins` installation arrays. + +Both origins use the packaged Linux Runtime's shared parser and protected +`installed.json`. Its default root is `/environment/initialization/capabilities`; +the operator may freeze another root with `OAC_RUNTIME_CAPABILITY_DIRECTORY`, kept +separate from workspace and private Runtime state. Core and public requests cannot +select this installation root. The manifest binds one +Session and Environment to the source-selection digest. The admitted asynchronous +preparation owner verifies or creates it before native execution, including for an +empty selection. Reconnect and replacement executors reuse installed contents; +source edits are observed only by a new Session with its own Environment and fresh +snapshot. A private completion record retains only the operator installation root, +so loss of a completed snapshot cannot trigger source recapture. Missing, partial or +conflicting snapshots fail without automatic repair or replay. Adapters receive only the resulting Runtime-owned Skill paths and MCP +declarations. Files access retains its separate readiness and authorization. + +Resource management owns allocation and Environment creation, renewal and reclamation. +Executor close, Turn cancellation and transport loss preserve the installed snapshot, +workspace and allocation. Reclamation is an explicit operation coordinated with active +work; a disconnected socket is not proof that native effects have stopped. + +Source selections accept portable absolute Unix, Windows drive and UNC paths; +Core does not resolve these paths on its own host. The daemon applies its local +path and access checks. Initial files and setup working directories use logical +`/workspace` paths. Harness differences remain in adapters. + +The protocol is platform-neutral, but this implementation retains the existing Linux +helpers, protected initialization storage and isolation requirements. It does not provide +a new native-host installer, another platform layout, or Windows/macOS isolation. +This implementation description does not extend historical real-deployment acceptance; +new managed and self-hosted capability coverage requires its own evidence. See the +[canonical preparation rules](../../CONTRIBUTING.md#shared-runtime-capability-preparation). + ## Contract inventory Paths below follow the SDK resource methods, before the service's `/v1` prefix. diff --git a/contracts/agents-api/harness-onboarding.md b/contracts/agents-api/harness-onboarding.md index ce98e8e66..c6e7418c3 100644 --- a/contracts/agents-api/harness-onboarding.md +++ b/contracts/agents-api/harness-onboarding.md @@ -1,64 +1,142 @@ # Add a native harness to OpenAgentCore -This reference is for adapter developers using the shared contract on main after -PR #701 (2026-09-19). Start with a working native SDK or machine-readable protocol. -The goal is to register an engine without changing public handlers, storage or the -scheduler. Codex, Claude and additional harnesses have equal architectural status; -each engine qualifies its own supported operations. +Implement one adapter for a native SDK or machine-readable protocol. Core and +Runtime use the common contract; adding a harness should not add engine-name +branches to public handlers, storage, scheduling or environment providers. -The [harness contract](harnesses.md) is the semantic baseline. The +The [harness contract](harnesses.md) defines observable behavior. The [contributor guide](../../CONTRIBUTING.md#harness-qualification-and-onboarding) -owns architecture and delivery rules. The pinned external Agents API in -[upstream.json](upstream.json) remains separate from this private adapter contract. - -## Ownership and implementation locations - -| Component | Responsibility | Existing location | +owns architecture and required checks. The pinned public Agents API in +[upstream.json](upstream.json) is separate from the internal adapter protocol. + +## Ownership + +```text +Core: Session, Turn, immutable configuration, durable events + | + common Runtime protocol + | +Runtime: Executor preparation, reuse, idle expiry, recovery + | + Harness adapter package + | + native SDK, process or connection +``` + +| Component | Responsibility | Location | | --- | --- | --- | -| Core | Public protocol, authentication, resource ownership, durable state and scheduling | `services/agents-api` | -| Runtime | Authenticated connection, dispatch, input receipts and preparation ownership | `apps/parsar-daemon/internal/dispatch` | -| Adapter | Native configuration, process/SDK calls, event translation and native restrictions | `apps/parsar-daemon/internal/agent/` | -| Harness | Model/tool loop and native history | Pinned upstream SDK or executable | -| Service profile | Pure validation of qualified placements and option values | `services/agents-api/internal/engine` | -| Runtime registration | Factory and verified capabilities available in this installation | `apps/parsar-daemon/internal/cli` | - -A service profile authorizes supported combinations. A Runtime advertisement says -what that installation can execute. Neither replaces public schema validation, -tenant authorization or the other boundary. Native limits belong in adapters and -profiles, not engine-name branches in Core. - -## Implement the existing interfaces - -Use `internal/agentdaemon/proto` for requests, events, receipts and errors. Do not -introduce a parallel wire protocol or another model/tool loop. - -| Interface | When required | Observable obligation | +| Core | Public API, authority, durable state, scheduling and configuration snapshots | `services/agents-api` | +| Runtime | Authenticated connection, shared capability preparation and common Executor/Turn lifecycle | `apps/parsar-daemon/internal/dispatch` | +| Adapter | Native configuration, resources, API calls, event translation and restrictions | `apps/parsar-daemon/internal/agent/` | +| Harness | Native model/tool loop and history | Pinned SDK or executable | +| Service profile | Pure validation of qualified operations and placements | `services/agents-api/internal/engine` | +| Registration | Installed factories and verified capability declarations | `apps/parsar-daemon/internal/cli` | + +An Environment supplies execution resources. Managed Docker, E2B and user-managed +machines differ in provisioning and connection; their connected Runtime uses this +same contract. Operating-system support belongs in the implementation and its +qualification. The current implementation is Linux; a platform-neutral interface +alone does not qualify another platform. Resource management owns machine selection, allocation and Environment creation +and reclamation. Executor close does not release that allocation or delete the +workspace. Environment reclamation is an explicit resource-management action that +coordinates with execution. Runtime connection, installed capability snapshot, +Session Executor and Turn each have their own lifetime. Native factories receive +capabilities only after the common Runtime has loaded its bound installed snapshot; +see [capability preparation](environments.md#runtime-capability-preparation). +Model providers supply +model communication configuration, not Turn scheduling or native process ownership. + +## Required adapter interfaces + +The canonical signatures and ownership comments are in +[`agent/executor.go`](../../apps/parsar-daemon/internal/agent/executor.go): + +```go +type ExecutorFactory func(context.Context, proto.PromptRequestPayload) (Executor, error) + +type Executor interface { + StartTurn(context.Context, string, proto.MessageInput, chan<- proto.Envelope) (Turn, error) + Close(context.Context) error +} + +type Turn interface { + Cancel(context.Context) error + CancellationOutcome() proto.DonePayload + AwaitSettlement(context.Context) (TurnSettlement, error) +} + +type TurnSettlement struct { + Reusable bool + Reason string +} +``` + +The factory prepares a fixed configuration without sending model input. Executor +owns the native process or connection, native Session, capability configuration and +cleanup resources. If preparation fails while cleanup remains unconfirmed, return +that non-nil Executor together with the error so Runtime retains its cleanup owner. +`StartTurn` creates a fresh Turn wrapper and event stream; +normal completion retains the Executor. Never reset a completed Turn object or +restart the process merely to implement the next Turn. + +A nil Turn from `StartTurn` guarantees no input was submitted and no output writer +was retained. Runtime closes the output channel in that case. Once input may have +been submitted, return a non-nil Turn even with an error: that Turn owns exactly +one output-channel close, and uncertain input must not be replayed. + +`Cancel` addresses that exact Turn. `AwaitSettlement` confirms that native events, +inputs, function results, interactions, child work and output writes have settled. +Only then may `Reusable` be true. False requires a reason and confirmed Executor +close before replacement. An error means settlement is unconfirmed. A caller +cancelling its wait cannot discard resource ownership or redirect a late callback +to the next Turn. Close must retain its exact cleanup target after failure so the +Runtime can retry serially. Successful resource close alone does not prove that a +cancelled input was applied or that its original outcome is known. + +The Runtime owns admission handles, idle deadlines and resource limits. Adapters +own native translation. For example, a Codex adapter retains its app-server and +thread, a Claude adapter retains one streaming Query, and a MiniMax adapter retains +its ACP connection and native Session. Core sees the same Executor and Turn +semantics in each case. + +## Events, inputs and optional capabilities + +Use [`internal/agentdaemon/proto`](../../internal/agentdaemon/proto) for neutral +requests, events and receipts. Each Turn emits only its own events with its Run ID, +in order, and one terminal outcome. Native IDs and usage must be observed rather +than invented. Capture the originating Turn before asynchronous native callbacks; +never attribute a late result to whichever Turn is currently active. + +Initial input and steering use ordered `proto.MessageInput`. Preserve user-message +and content order. Text-only adapters reject images through `TextOnly()` instead +of dropping them. A successful transport write is distinct from confirmed native +application. Resume only the exact history bound to the Session; missing or +ambiguous required history fails before new model input. + +| Interface or contract | When required | Obligation | | --- | --- | --- | -| `agent.Factory` and `agent.Session` | Every harness | Start a run; cancel idempotently; own output closure and native process lifetime | -| `agent.DurableSteerer` | Current public text execution contract | Distinguish a complete native write from confirmed application; return rejection/inactive/not-ready accurately | -| `agent.PreparationFactory` and `agent.Prepared` | Placements requiring preparation | Prepare without consuming model input; transfer ownership once in Start; retain failed cleanup ownership | -| `agent.PreparedCancellation` | Executable preparations | Cancel the same resource across Start; settle only after native effects and output writes stop | -| `agent.PermissionResponder`, `agent.UserChoiceResponder` | Only when emitting those interactions | Route exact interaction identities and preserve application receipts | -| Other optional interfaces | Only for declared operations | Implement the existing operation semantics and validate native support | - -Read the source comments in `registry.go`, `steering.go`, `preparation.go` and -`interactions.go` before implementing. Base `Session.Cancel` is a cancellation -signal; its return alone is not proof that all effects stopped. Use the existing -router's settlement path and adapter outcome interfaces. Do not give both the -adapter and router ownership of closing the same output channel. - -Preserve event order and native call identities. Emit one run terminal outcome; -never manufacture applied input, usage counters or successful cleanup. Resume only -the native history bound to the execution Session. Missing or ambiguous history -fails before new model input. A disconnected observer does not authorize replay. +| `agent.DurableSteerer` | Current public text execution | Distinguish write and application receipts; preserve retry identity | +| `agent.FunctionResultSubmitter` | Public function tools | Match call/result identity and acknowledge native application | +| `agent.PermissionResponder`, `agent.UserChoiceResponder` | When emitting these interactions | Route exact identities and settle receipts | +| `agent.WorkspaceReader`, `agent.WorkspaceDirectoryLister` | Qualified workspace operations | Use the fixed authorized workspace and retain accepted operations through close | +| Neutral message, image, MCP, structured-output and Subagent observations | Only when qualified and advertised | Preserve the operation-specific contract and reject unsupported combinations | + +Optional features need not match another harness. The service profile qualifies +public combinations; the Runtime advertises this installation's available support. +Neither replaces schema validation or tenant authorization. Declaring a capability +without implementing its semantics is an error. Workspace reads may use separate +read-only preparations; those do not start model work or provide another execution +lifecycle. ## Register a supported operation set 1. Pin the upstream source/package version and document the native entry point. 2. Implement the adapter using its SDK or native protocol. Reuse shared process, credential/configuration and local workspace helpers where applicable. -3. Register its `SupportedAgentKind` and factory with `RegisterKind`; register - preparation afterward with `RegisterPreparation` when supported. Runtime +3. Register its `SupportedAgentKind` and direct-call factory with `RegisterKind`, + then register the shared `ExecutorFactory` with `RegisterExecutor`. A direct-call + factory should delegate to the same Executor implementation. Register separate + read-only preparations only when required by qualified workspace operations. Runtime advertisements must describe behavior verified for that installation. 4. Add a profile to the existing static catalog, or supply an immutable catalog through service composition. Custom composition supplies the same @@ -90,7 +168,7 @@ confirmed native output through the existing Message contract. Register public qualification separately from the Runtime capability; see the [structured-output boundary](structured-output.md). No Core engine-name branch is required. -Initial requests, `Prepared.Start` and steering consume the same ordered +Initial requests, `Executor.StartTurn` and steering consume the same ordered `proto.MessageInput`. Text-only adapters use `TextOnly()` to reject images without discarding content. Image adapters translate each part natively and acknowledge an active batch only after all its messages are applied. Register @@ -109,14 +187,18 @@ Before implementation, record the operation set, expected results, exclusions an stopping conditions in the board. A batch ends when its declared operations pass; it does not expand to match another harness's feature list. -- Adapter tests: native failures, ordered events, input write/application receipts, - cancellation settlement, strict continuation and unknown-outcome handling. +- Adapter tests: two ordinary Turns share one native process/connection and history; + cancellation followed by another Turn; stale cancellation and late events; native + exit, cleanup failure, input write/application receipts and unknown outcomes. + Verify fresh per-Turn usage, function, input and child-observation state. - Shared integration: public admission, actual Worker/device selection, gateway, daemon registration/dispatch and durable terminal projection. See `TestThirdHarnessPublicOnboarding` for a synthetic example, not native evidence. - Real acceptance: pinned official Python SDK and raw HTTP against our Agents API, real provider API, native harness and independent execution database. Verify - initial execution, follow-up input, cancellation and restart/continuation. + initial execution, warm follow-up, cancellation and restart/continuation. + Record native owner identity and same-condition cold/warm timing; mock results + cannot establish native reuse or performance gains. For hosted qualification also verify Files/Artifacts, workspace identity, credential protection and foreign-history rejection. - Regression: existing qualified engines keep working. Run targeted tests during @@ -132,31 +214,24 @@ Feishu. Failed or synthetic runs cannot be counted as real acceptance. Merge the bounded PR after required checks/review, update its board child and reassess the full board. Acceptance of one engine is not complete public protocol compatibility. -## MiniMax Code application - -The upstream project is [MiniMax-AI/minimax-code](https://github.com/MiniMax-AI/minimax-code). -The repository already contains an ACP stdio adapter under `agent/mcode`, including -native session creation/loading, events and human interactions. Its existing -product integration is not Agents API qualification. The current service catalog -registers Codex, Claude Code and MiniMax Code. Text qualification in #702 and -workspace qualification in #703 are separate recorded milestones. - -The implementation reuses that adapter with an explicit native 0.4.12 opt-in for -`environment:none` text execution. It adds native active-input receipts and -cancellation settlement, a pure service profile and verified registration. No -public handler, store schema or scheduler engine branch is needed. Existing -product behavior and accepted Codex/Claude features remain protected. - -See [the MiniMax Code deployment guide](../../services/agents-api/deploy/mcode/README.md) -for setup, real-provider acceptance and limits. Hosted workspace, Files/Artifacts, -public functions and MCP are not qualified by the text profile. Native differences -remain separately tracked work; they do not require feature equality for onboarding. - -The requested MiniMax Code workspace Runtime is qualified separately in -[MCODE-WORKSPACE-V1-001](mcode-workspace-v1.md). Its explicit scope includes -workspace execution and shared Files/Artifacts, cancellation/recovery and independent -Docker deployment. Text-only qualification is an intermediate milestone for that -scope, not completion of the requested Runtime integration. +## Minimal runnable example and native references + +[`testdata/onboarding/main.go`](../../apps/parsar-daemon/testdata/onboarding/main.go) +is a runnable synthetic adapter. It demonstrates registration, a Session-owned +Executor, fresh Turns, durable steering, cancellation and history binding through +the actual gateway and Router. Run `TestThirdHarnessPublicOnboarding` against a +dedicated test database to exercise the complete integration path. The fixture is +not shipped and is not evidence for a real SDK or sandbox. + +Use the native adapters as implementation references after choosing a native API: + +- [Codex](../../apps/parsar-daemon/internal/agent/codex/executor.go): app-server transport. +- [Claude](../../apps/parsar-daemon/internal/agent/claudesdk/executor.go): Go ownership and a [TypeScript SDK bridge](../../packages/claude-sdk-adapter). +- [MiniMax](../../apps/parsar-daemon/internal/agent/mcode): ACP and native workspace companion. + +Start with the mandatory text lifecycle, then qualify optional operations using +[harness coverage](harnesses.md) and each operation's contract. Do not copy an +adapter's native limitations into the shared Core protocol. ## Optional Subagent observations diff --git a/contracts/agents-api/harnesses.md b/contracts/agents-api/harnesses.md index 0aaeda64a..5c8976f84 100644 --- a/contracts/agents-api/harnesses.md +++ b/contracts/agents-api/harnesses.md @@ -13,15 +13,15 @@ For implementation steps and interface obligations, see ## Integration surface -1. Implement the existing daemon `agent.Factory`/`Session` and, for prepared - environments, `PreparationFactory`/`Prepared` interfaces. `Session` requires - cancellation; adapters that emit permission or user-choice requests additionally - implement `PermissionResponder` or `UserChoiceResponder`. Other optional - interfaces, such as function results, follow their declared operations. Reuse - `internal/agentdaemon/proto` requests, neutral events, input receipts and errors. -2. Register the factory, preparation factory and verified Runtime capabilities in - the daemon registry. Keep native translation inside the adapter. Dedicated local - environments reuse shared Files/write/export helpers and binding checks. +1. Implement `agent.ExecutorFactory`, `Executor` and `Turn`. Prepare fixed native + resources once, start fresh Turns on that owner, cancel an exact Turn, confirm + settlement and close the Executor. Implement optional interfaces only for + declared operations. Reuse `internal/agentdaemon/proto` requests, neutral events, + input receipts and errors. +2. Register the Executor factory and verified Runtime capabilities in the daemon + registry. Keep native translation in the adapter. Managed and user-managed + environments use the same execution lifecycle after connection; dedicated local + environments retain shared Files/write/export helpers and binding checks. 3. Add a pure qualified profile to `services/agents-api/internal/engine` and its static catalog (or supply an immutable catalog at service composition). Declare supported placements, public configuration/result limits @@ -49,9 +49,10 @@ not a different contract or authority level. The documented extension remains se ## Shared behavioral obligations -- Preparation holds resources without consuming input; start transfers ownership - once. Unused preparation releases through `Close`; cancellation remains valid - across the transfer and reports settlement only after native effects stop. +- Preparation holds resources without consuming input. Normal Turn completion + retains a settled healthy Executor. Cancellation targets one Turn; idle expiry, + environment shutdown and invalidation close the Executor. Failed cleanup retains + ownership and capacity, and never authorizes replay of uncertain input. - Confirm accepted/applied inputs separately. Preserve ordered public Items and events, stable call identity and one terminal outcome. Never replay uncertain work merely because a connection closed. diff --git a/contracts/agents-api/message-input.md b/contracts/agents-api/message-input.md index 165584d93..7d7f70f1f 100644 --- a/contracts/agents-api/message-input.md +++ b/contracts/agents-api/message-input.md @@ -74,7 +74,7 @@ before. ## Runtime boundary -Private wire 0.5.0 uses `MessageInput` for initial requests, prepared start and +Private wire 0.7.0 uses `MessageInput` for initial requests, prepared start and active steering. Each message contains ordered `InputContent` parts, also reused by function-result content. Core does not download, transcode or repair media. The separate Claude bridge reports protocol 2; daemon readiness rejects protocol 1. diff --git a/contracts/agents-api/openapi.yaml b/contracts/agents-api/openapi.yaml index 00bc1685d..c24705ceb 100644 --- a/contracts/agents-api/openapi.yaml +++ b/contracts/agents-api/openapi.yaml @@ -2966,84 +2966,84 @@ paths: Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and native OAuth login remain unsupported. The self_hosted profile - requires Codex, an absolute workspace_directory and empty capability_directories, - with optional non-deferred function tools and HTTP MCP using service origin, - optionally authenticated by the attached Vault rules. Remote MCP and remote - Bearer authentication each require separately advertised combination support; - old peers cannot receive unsupported work. Omitted/null capability_directories - use the empty-list default; self_hosted requires configured execution plus - executor registry. Claude SDK currently requires medium verbosity and object-root - function schemas. It supports anonymous or attached static-bearer service-origin - HTTP MCP on none with boolean required and separately advertised MCP/bearer/required - runtime support. Required servers must be connected before the first native - input is released; pending or failed startup rejects execution. The shared - Vault selection and immutable binding rules apply; unsupported native labels/tool - names reject before persistence. An attached Vault with no matching credential - may remain anonymous; missing keys or failed credential lookup/decryption - never fall back to anonymous execution. Omitted stream defaults to false; - stream and agent_id cannot be null. Metadata may be null; non-string values - and limit violations return invalid_request_error with a metadata or metadata. - param. The inline agent uses the Agent create configuration validation with - agent.-prefixed params, reported before the input requirement and saved-Agent - lookup; saved configurations with conflicting tools or schema roots reject - admission with the same errors, and execution limits keep unsupported_or_invalid_configuration. - Hosted network policy rejections return invalid_request_error with a null - param. Initial input accepts a string or ordered user-message array. Codex - and Claude SDK on none and qualified openai_hosted also accept inline PNG/JPEG - image content; other image combinations and remote URLs are unsupported. None - initial input atomically starts a Turn; self_hosted initial input is reserved - while returning its Environment connection target, with execution deferred - to native readiness and Session failure on initial timeout. Initial input - is required for none and for streamed creation outside self_hosted. Omitted/null - input remains valid for non-streaming hosted and self_hosted creation. With - stream=true, returns live Session events starting with the committed creation - snapshot and closes right after the first agent.session.idle recorded when - a Turn ends or an input reservation stops being pending, or any agent.session.failed, - without sending later events. A creation that admitted nothing closes after - the snapshot; a settlement that records no event closes after events up to - the cursor read with a settled Session projection. Required actions keep it - open; disconnect does not cancel execution. The GET events stream remains - live-only. New Sessions retain their authenticated creator; all creation retries - require the same typed subject, including across key rotation. Saved-Agent - retries and inline requests using Vault attachments or credential references - retain caller intent independently of later resource changes; new hosted inline - requests also freeze caller intent before deployment defaults resolve; unrelated - non-hosted inline retries preserve resolved/default equivalences, and their - resolved hash leaves out any deployment default. Provider keys enter retry - hashes only as fingerprints keyed by the credential key. Unknown historical - creators reject retries; known creators without recorded intent retain resolved-snapshot - retry rules. These conflict policies are local and not verified hosted parity. - A same-key stream=true retry of an existing creation returns 201 with no events - and closes at once; retry with stream=false or use the GET events stream to - recover. Claude SDK on none and Core-managed Docker openai_hosted supports - qualified object-root json_schema output with medium verbosity, single-Agent - execution and ordinary functions. Hosted execution reuses native workspace - tools and Files/Artifacts; Skills, Plugins, capability directories, HTTP MCP, - Subagent and tool_search combinations remain unqualified, including inherited - template contents. Other non-text initial input remains unsupported. Basic - Codex and Claude SDK openai_hosted creation requires an explicitly configured - managed provider. The Claude workspace profile supports non-deferred function - tools with text or successful inline PNG/JPEG results alongside native workspace - tools; HTTP MCP remains unsupported. Idle Sessions provision automatically; - initial provisioning has no caller connection action. Network defaults to - enabled; disabled and restricted exact ASCII hostnames are supported. Restricted - policy requires 1–100 allowed domains. Unsupported hostname forms and startup - installations are rejected. Confidential env, system/npm/Python packages and - ordered setup commands use the shared initialization lifecycle; requested - network applies after setup. Initial inline and tenant-owned file_id files - freeze encrypted bytes before provisioning, then install through the common - Core lifecycle before native execution or live Files access. With a template - reference, omitted/null files, env, packages and setup_commands inherit. Non-null - files and command lists replace; env overlays by key; each package manager - inherits on omission/null and otherwise replaces its list. Empty lists clear - their selected field. Tenant-owned environment_template_id references inherit - omitted/null network and allow only narrowing overrides. Inline hosted network:null - retains the enabled default; updating a Template with network:null resets - its saved policy to enabled. Core freezes effective configuration; template - updates/deletion do not alter Session snapshots or same-intent creation retries. - Inline or tenant-owned skill_reference Skills share initialization. Templates - preserve default/latest/explicit selectors; Session creation freezes concrete - metadata and encrypted content atomically. Skill, Plugin and capability-directory + uses a qualified native harness, a clean absolute workspace_directory and + optional absolute local capability_directories prepared by Runtime, with optional + non-deferred function tools and HTTP MCP using service origin, optionally + authenticated by the attached Vault rules. Remote MCP and remote Bearer authentication + each require separately advertised combination support; old peers cannot receive + unsupported work. Omitted/null capability_directories use the empty-list default; + self_hosted requires configured execution plus executor registry. Claude SDK + currently requires medium verbosity and object-root function schemas. It supports + anonymous or attached static-bearer service-origin HTTP MCP on none with boolean + required and separately advertised MCP/bearer/required runtime support. Required + servers must be connected before the first native input is released; pending + or failed startup rejects execution. The shared Vault selection and immutable + binding rules apply; unsupported native labels/tool names reject before persistence. + An attached Vault with no matching credential may remain anonymous; missing + keys or failed credential lookup/decryption never fall back to anonymous execution. + Omitted stream defaults to false; stream and agent_id cannot be null. Metadata + may be null; non-string values and limit violations return invalid_request_error + with a metadata or metadata. param. The inline agent uses the Agent create + configuration validation with agent.-prefixed params, reported before the + input requirement and saved-Agent lookup; saved configurations with conflicting + tools or schema roots reject admission with the same errors, and execution + limits keep unsupported_or_invalid_configuration. Hosted network policy rejections + return invalid_request_error with a null param. Initial input accepts a string + or ordered user-message array. Codex and Claude SDK on none and qualified + openai_hosted also accept inline PNG/JPEG image content; other image combinations + and remote URLs are unsupported. None initial input atomically starts a Turn; + self_hosted initial input is reserved while returning its Environment connection + target, with execution deferred to native readiness and Session failure on + initial timeout. Initial input is required for none and for streamed creation + outside self_hosted. Omitted/null input remains valid for non-streaming hosted + and self_hosted creation. With stream=true, returns live Session events starting + with the committed creation snapshot and closes right after the first agent.session.idle + recorded when a Turn ends or an input reservation stops being pending, or + any agent.session.failed, without sending later events. A creation that admitted + nothing closes after the snapshot; a settlement that records no event closes + after events up to the cursor read with a settled Session projection. Required + actions keep it open; disconnect does not cancel execution. The GET events + stream remains live-only. New Sessions retain their authenticated creator; + all creation retries require the same typed subject, including across key + rotation. Saved-Agent retries and inline requests using Vault attachments + or credential references retain caller intent independently of later resource + changes; new hosted inline requests also freeze caller intent before deployment + defaults resolve; unrelated non-hosted inline retries preserve resolved/default + equivalences, and their resolved hash leaves out any deployment default. Provider + keys enter retry hashes only as fingerprints keyed by the credential key. + Unknown historical creators reject retries; known creators without recorded + intent retain resolved-snapshot retry rules. These conflict policies are local + and not verified hosted parity. A same-key stream=true retry of an existing + creation returns 201 with no events and closes at once; retry with stream=false + or use the GET events stream to recover. Claude SDK on none and Core-managed + Docker openai_hosted supports qualified object-root json_schema output with + medium verbosity, single-Agent execution and ordinary functions. Hosted execution + reuses native workspace tools and Files/Artifacts; Skills, Plugins, capability + directories, HTTP MCP, Subagent and tool_search combinations remain unqualified, + including inherited template contents. Other non-text initial input remains + unsupported. Basic Codex and Claude SDK openai_hosted creation requires an + explicitly configured managed provider. The Claude workspace profile supports + non-deferred function tools with text or successful inline PNG/JPEG results + alongside native workspace tools; HTTP MCP remains unsupported. Idle Sessions + provision automatically; initial provisioning has no caller connection action. + Network defaults to enabled; disabled and restricted exact ASCII hostnames + are supported. Restricted policy requires 1–100 allowed domains. Unsupported + hostname forms and startup installations are rejected. Confidential env, system/npm/Python + packages and ordered setup commands use the shared initialization lifecycle; + requested network applies after setup. Initial inline and tenant-owned file_id + files freeze encrypted bytes before provisioning, then install through the + common Core lifecycle before native execution or live Files access. With a + template reference, omitted/null files, env, packages and setup_commands inherit. + Non-null files and command lists replace; env overlays by key; each package + manager inherits on omission/null and otherwise replaces its list. Empty lists + clear their selected field. Tenant-owned environment_template_id references + inherit omitted/null network and allow only narrowing overrides. Inline hosted + network:null retains the enabled default; updating a Template with network:null + resets its saved policy to enabled. Core freezes effective configuration; + template updates/deletion do not alter Session snapshots or same-intent creation + retries. Inline or tenant-owned skill_reference Skills share initialization. + Templates preserve default/latest/explicit selectors; Session creation freezes + concrete metadata and encrypted content atomically. Skill, Plugin and capability-directory list omission/null inherit; a non-null list replaces, including empty-list clearing. Omitted/null Skill version selectors resolve the default version. Source deletion/default updates cannot change committed Session Skill contents. diff --git a/contracts/agents-api/user-managed-runtime-v1.md b/contracts/agents-api/user-managed-runtime-v1.md index 41e037fe0..52335471c 100644 --- a/contracts/agents-api/user-managed-runtime-v1.md +++ b/contracts/agents-api/user-managed-runtime-v1.md @@ -15,8 +15,9 @@ file operations continue through Core and daemon, not E2B commands or files. ## Accepted scope -Recorded on 2026-09-22 against source candidate `8f0cd2530d7b58cb7fb3ea124a1fc43dacecca36`. -Documentation-only follow-up commits do not change the accepted binaries. +Historical qualification recorded on 2026-09-22 against source candidate `8f0cd2530d7b58cb7fb3ea124a1fc43dacecca36`. +These records qualify only those binaries and tested inputs. Later implementation +changes, including shared Runtime capability preparation, require separate acceptance. | Deployment | Codex | Claude Code | MiniMax Code | | --- | --- | --- | --- | @@ -97,8 +98,12 @@ or a saved Agent, frozen and delivered over the same daemon transport; see [model execution](model-execution.md). Deployment default model providers never reach self-hosted executors. -The public self-hosted profile accepts `/workspace` and empty capability -directories. Service-origin HTTP MCP on self-hosted remains explicitly rejected; +The historical runs above used `/workspace` and empty capability directories. +Current self-hosted input accepts a canonical absolute workspace and optional local +capability directories, prepared through the +[shared Runtime snapshot](environments.md#runtime-capability-preparation). That added +scope is not qualified by these earlier runs. Service-origin HTTP MCP on self-hosted +remains explicitly rejected; none-environment HTTP MCP and separately qualified hosted Plugin MCP keep their own scope. Environment Templates remain hosted-only. Unspecified upstream defaults, errors, lifecycle edge cases and broader resource semantics remain in the protocol diff --git a/docs/getting-started/self-hosted.md b/docs/getting-started/self-hosted.md index 004985864..4425ca8bd 100644 --- a/docs/getting-started/self-hosted.md +++ b/docs/getting-started/self-hosted.md @@ -95,6 +95,48 @@ Core delivers the Session's model provider only over this Environment's executor connection. The host keeps it in the Runtime's harness home, where the model's tools and the Files API can't read it but the host's owner can. +## Local capability directories + +A self-hosted Session may supply `capability_directories` alongside its +`workspace_directory`, for example: + +```python +environment = { + "type": "self_hosted", + "workspace_directory": "/workspace", + "capability_directories": ["/workspace/capabilities"], +} +``` + +The API accepts portable absolute Unix, Windows drive and UNC source paths without +checking Core's filesystem. The current Linux Runtime requires canonical local Unix +paths and applies its own access checks; this does not add Windows support. +These paths refer to directories visible inside that Runtime. Populate them before +first execution. `/workspace` maps to its bound workspace; an operator-prepared +Runtime may use its exact canonical absolute workspace instead. Directory aliases, +private credentials, native history and protected initialization trees are refused. +This does not mount arbitrary host paths into the container or add a new installer. +Self-hosted public input accepts local directories, not managed Skill/Plugin archives +or hosted Templates. + +The current implementation uses the packaged Linux helpers and protected +`installed.json`, by default under `/environment/initialization/capabilities`. +Operators may freeze another root with `OAC_RUNTIME_CAPABILITY_DIRECTORY`; it must +stay separate from the workspace and private Runtime state and is not a public API +setting. Runtime snapshots sources +before native execution and supplies the resulting Skill and MCP inventory through +the same parser as managed bundles. Reconnect reuses installed contents even after +source edits; a new Session captures its own snapshot. A missing or inconsistent +snapshot fails preparation without silently reinstalling. Local discovery does not +create entries in the public API-managed installation arrays. + +Closing an executor, cancelling a Turn or losing its connection preserves the +snapshot and workspace. The compute owner remains responsible for explicit resource +cleanup. This source-level capability behavior requires its own real acceptance; +[earlier qualification](../../contracts/agents-api/user-managed-runtime-v1.md) covered +only its recorded inputs and binaries, not local directory preparation or additional +platforms. + ## Rotate or revoke | Action in Web | Effect | diff --git a/internal/agentcapabilities/install.go b/internal/agentcapabilities/install.go index b5badc74a..ce8c24c80 100644 --- a/internal/agentcapabilities/install.go +++ b/internal/agentcapabilities/install.go @@ -12,8 +12,19 @@ import ( "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" ) +func InstallSkill(root *os.Root, archive []byte, metadata agentskill.Metadata) error { + if unfinished(root) != nil { + return ErrInvalid + } + files, err := agentskill.Read(archive, metadata) + if err != nil { + return ErrInvalid + } + return writeTree(root, "skills/"+metadata.Name, files) +} + func InstallPlugin(root *os.Root, slot int, archive []byte, metadata agentplugin.Metadata) error { - if slot < 0 || slot >= 50 { + if slot < 0 || slot >= 50 || unfinished(root) != nil { return ErrInvalid } bundle, err := agentplugin.Read(archive, metadata) @@ -23,16 +34,20 @@ func InstallPlugin(root *os.Root, slot int, archive []byte, metadata agentplugin return writeTree(root, "plugins/"+strconv.Itoa(slot), bundle.Files) } -// Finalize runs once after setup under the existing allocation initializer. +// Finalize captures declared directories once after setup and binds the snapshot. // It emits an installation artifact, not another lifecycle or public catalog. -func Finalize(workspace, installed *os.Root, input Input) error { - if len(input.Skills) > MaxSkills || len(input.Plugins) > 50 || ValidateDirectories(input.Directories) != nil { +func Finalize(installed *os.Root, input Input, identity Identity, resolve DirectoryResolver) error { + digest, err := selectionHash(input) + if err != nil || validateIdentity(identity) != nil || (len(input.Directories) > 0 && resolve == nil) { return ErrInvalid } if _, err := installed.Lstat(ManifestName); !os.IsNotExist(err) { return ErrInvalid } - manifest := Manifest{Version: 1, Skills: []InstalledSkill{}} + if validateStaged(installed, input) != nil { + return ErrInvalid + } + manifest := Manifest{Version: 1, Identity: identity, SelectionSHA256: digest, Skills: []InstalledSkill{}} total := 0 count := func(files []agentbundle.File) error { for _, file := range files { @@ -69,12 +84,18 @@ func Finalize(workspace, installed *os.Root, input Input) error { } } for slot, source := range input.Directories { - name := strings.TrimPrefix(source, "/workspace") - name = strings.TrimPrefix(name, "/") - if name == "" { - name = "." + directory, err := resolve(source) + if err != nil || directory == nil { + if directory != nil { + directory.Close() + } + return ErrInvalid + } + files, err := ReadTree(directory, ".", false) + closeErr := directory.Close() + if closeErr != nil { + return ErrInvalid } - files, err := ReadTree(workspace, name, false) if err != nil || count(files) != nil { return ErrInvalid } @@ -84,7 +105,7 @@ func Finalize(workspace, installed *os.Root, input Input) error { } } body, err := json.Marshal(manifest) - if err != nil || writeFile(installed, ManifestName+".tmp", body, 0400) != nil || installed.Rename(ManifestName+".tmp", ManifestName) != nil { + if err != nil || len(body) > 256<<10 || writeFile(installed, ManifestName+".tmp", body, 0400) != nil || installed.Rename(ManifestName+".tmp", ManifestName) != nil { return ErrInvalid } return syncDirectory(installed, ".") @@ -186,3 +207,55 @@ func Load(root *os.Root) (Manifest, error) { } return manifest, nil } + +// Only selected, fully imported bundles may precede finalization. A leftover +// directory capture or temporary manifest is an incomplete installation, not a +// reason to silently recapture mutable sources. +func validateStaged(root *os.Root, input Input) error { + expected := map[string]map[string]bool{"skills": {}, "plugins": {}} + for _, skill := range input.Skills { + expected["skills"][skill.Name] = true + } + for slot := range input.Plugins { + expected["plugins"][strconv.Itoa(slot)] = true + } + file, err := root.Open(".") + if err != nil { + return ErrInvalid + } + entries, err := file.ReadDir(-1) + closeErr := file.Close() + if err != nil || closeErr != nil { + return ErrInvalid + } + for _, entry := range entries { + selected, ok := expected[entry.Name()] + if !ok || !entry.IsDir() { + return ErrInvalid + } + directory, err := root.Open(entry.Name()) + if err != nil { + return ErrInvalid + } + children, err := directory.ReadDir(-1) + closeErr := directory.Close() + if err != nil || closeErr != nil || len(children) != len(selected) { + return ErrInvalid + } + for _, child := range children { + if !child.IsDir() || !selected[child.Name()] { + return ErrInvalid + } + } + } + return nil +} + +func unfinished(root *os.Root) error { + for _, name := range []string{ManifestName, ManifestName + ".tmp"} { + if _, err := root.Lstat(name); !os.IsNotExist(err) { + return ErrInvalid + } + } + return nil +} diff --git a/internal/agentcapabilities/install_test.go b/internal/agentcapabilities/install_test.go index 3e510f519..2407e100f 100644 --- a/internal/agentcapabilities/install_test.go +++ b/internal/agentcapabilities/install_test.go @@ -27,7 +27,7 @@ func TestInstalledCapabilitiesPreserveSourcesWithoutRescanning(t *testing.T) { writeWorkspace(t, workspace, "generated/nested/SKILL.md", skillManifest("generated")) writeWorkspace(t, workspace, "generated/shared/proof.txt", []byte("original-resource")) input := Input{Skills: []agentskill.Metadata{inline}, Plugins: []agentplugin.Metadata{plugin}, Directories: []string{"/workspace/generated"}} - if err := Finalize(workspace, installed, input); err != nil { + if err := Finalize(installed, input, testIdentity, workspaceResolver(workspace)); err != nil { t.Fatal(err) } before, err := Load(installed) @@ -51,7 +51,7 @@ func TestInstalledCapabilitiesPreserveSourcesWithoutRescanning(t *testing.T) { t.Fatalf("snapshot changed or writable: %s", name) } } - if Finalize(workspace, installed, input) == nil { + if Finalize(installed, input, testIdentity, workspaceResolver(workspace)) == nil { t.Fatal("completed initialization was replayed") } } @@ -61,7 +61,7 @@ func TestDirectoryDiscoveryDoesNotActivateChildPluginMCP(t *testing.T) { writeWorkspace(t, workspace, "parent/child/.codex-plugin/plugin.json", []byte(`{"name":"remote","description":"Remote","skills":"./skills","mcpServers":"./.mcp.json"}`)) writeWorkspace(t, workspace, "parent/child/.mcp.json", []byte(`{"mcpServers":{"remote":{"type":"http","url":"https://example.com"}}}`)) writeWorkspace(t, workspace, "parent/child/skills/proof/SKILL.md", skillManifest("proof")) - if err := Finalize(workspace, installed, Input{Directories: []string{"/workspace/parent"}}); err != nil { + if err := Finalize(installed, Input{Directories: []string{"/workspace/parent"}}, testIdentity, workspaceResolver(workspace)); err != nil { t.Fatal(err) } manifest, err := Load(installed) @@ -72,7 +72,7 @@ func TestDirectoryDiscoveryDoesNotActivateChildPluginMCP(t *testing.T) { t.Fatal("parent directory activated child Plugin MCP") } exact := openTestRoot(t) - if err := Finalize(workspace, exact, Input{Directories: []string{"/workspace/parent/child"}}); err != nil { + if err := Finalize(exact, Input{Directories: []string{"/workspace/parent/child"}}, testIdentity, workspaceResolver(workspace)); err != nil { t.Fatal(err) } manifest, err = Load(exact) @@ -102,7 +102,7 @@ func TestInvalidDirectorySnapshotsNeverPublish(t *testing.T) { case "file": input.Directories = []string{"/workspace/first/SKILL.md"} } - if Finalize(workspace, installed, input) == nil { + if Finalize(installed, input, testIdentity, workspaceResolver(workspace)) == nil { t.Fatal("invalid capability source accepted") } if _, err := installed.Lstat(ManifestName); !os.IsNotExist(err) { diff --git a/internal/agentcapabilities/manifest.go b/internal/agentcapabilities/manifest.go index 3ecda2e7e..e499388ce 100644 --- a/internal/agentcapabilities/manifest.go +++ b/internal/agentcapabilities/manifest.go @@ -2,11 +2,17 @@ package agentcapabilities import ( + "crypto/sha256" + "encoding/hex" "encoding/json" "errors" "io/fs" + "os" "path" "strings" + "unicode/utf8" + + "github.com/google/uuid" "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" @@ -21,16 +27,30 @@ var ErrInvalid = errors.New("capability installation unavailable or unsupported" // InstalledSkill contains Runtime-owned paths, never public resource metadata. type InstalledSkill struct { - Metadata agentskill.Metadata `json:"metadata"` - RelativeRoot string `json:"relative_root"` - PackageRoot string `json:"package_root"` + // InstallationRoot is resolved by Runtime and never persisted or accepted over wire. + InstallationRoot string `json:"-"` + Metadata agentskill.Metadata `json:"metadata"` + RelativeRoot string `json:"relative_root"` + PackageRoot string `json:"package_root"` +} + +// Identity binds an installation to one immutable Environment and Session. +type Identity struct { + EnvironmentID string `json:"environment_id"` + SessionID string `json:"session_id"` } +// DirectoryResolver opens a declared source after local authorization and path +// checks. Finalize owns and closes each returned root; callers retain no handle. +type DirectoryResolver func(string) (*os.Root, error) + type Manifest struct { - Version int `json:"version"` - Skills []InstalledSkill `json:"skills"` - Plugins []string `json:"plugins,omitempty"` - MCP []InstalledMCP `json:"-"` + Identity Identity `json:"identity"` + SelectionSHA256 string `json:"selection_sha256"` + Version int `json:"version"` + Skills []InstalledSkill `json:"skills"` + Plugins []string `json:"plugins,omitempty"` + MCP []InstalledMCP `json:"-"` } // InstalledMCP is resolved from a frozen package at load time. The manifest @@ -47,13 +67,29 @@ type Input struct { Directories []string `json:"directories,omitempty"` } +// ValidateDirectories preserves the managed public workspace-path contract. func ValidateDirectories(directories []string) error { + if ValidateLocalDirectories(directories) != nil { + return ErrInvalid + } + for _, directory := range directories { + if directory != "/workspace" && !strings.HasPrefix(directory, "/workspace/") { + return ErrInvalid + } + } + return nil +} + +// ValidateLocalDirectories checks source spelling, not local access authority. +// This Linux implementation requires canonical Unix paths. The Runtime resolver +// owns directory access and protected-root exclusions. +func ValidateLocalDirectories(directories []string) error { if len(directories) > 50 { return ErrInvalid } seen := map[string]bool{} for _, directory := range directories { - if (directory != "/workspace" && !strings.HasPrefix(directory, "/workspace/")) || + if !path.IsAbs(directory) || len(directory) > 4096 || !utf8.ValidString(directory) || path.Clean(directory) != directory || strings.ContainsAny(directory, "\\\x00\r\n") || seen[directory] { return ErrInvalid } @@ -81,7 +117,7 @@ func (m *Manifest) add(metadata agentskill.Metadata, root, pkg string) error { func decodeManifest(body []byte) (Manifest, error) { var result Manifest - if len(body) > 256<<10 || json.Unmarshal(body, &result) != nil || result.Version != 1 || len(result.Skills) > MaxSkills { + if len(body) > 256<<10 || json.Unmarshal(body, &result) != nil || result.Version != 1 || validateIdentity(result.Identity) != nil || !validSelectionHash(result.SelectionSHA256) || len(result.Skills) > MaxSkills { return Manifest{}, ErrInvalid } checked := Manifest{Version: 1} @@ -110,3 +146,64 @@ func (m *Manifest) addMCPPackage(root string) error { m.Plugins = append(m.Plugins, root) return nil } + +func validateIdentity(identity Identity) error { + for _, value := range []string{identity.EnvironmentID, identity.SessionID} { + id, err := uuid.Parse(value) + if err != nil || id == uuid.Nil || id.String() != value { + return ErrInvalid + } + } + return nil +} + +func validSelectionHash(value string) bool { + digest, err := hex.DecodeString(value) + return err == nil && len(digest) == sha256.Size && hex.EncodeToString(digest) == value +} + +// ValidateInput bounds source selections without reading archives or directories. +// Portable manifest grammar is checked by the shared Skill and Plugin parsers. +func ValidateInput(input Input) error { + if len(input.Skills) > MaxSkills || len(input.Plugins) > 50 || ValidateSourceDirectories(input.Directories) != nil { + return ErrInvalid + } + validMetadata := func(kind, name, description string) bool { + return kind == "inline" && name != "" && description != "" && utf8.ValidString(name) && utf8.ValidString(description) + } + for _, skill := range input.Skills { + if !validMetadata(skill.Type, skill.Name, skill.Description) { + return ErrInvalid + } + } + for _, plugin := range input.Plugins { + if !validMetadata(plugin.Type, plugin.Name, plugin.Description) { + return ErrInvalid + } + } + return nil +} + +func selectionHash(input Input) (string, error) { + if ValidateInput(input) != nil { + return "", ErrInvalid + } + // omitempty gives nil and empty selections the same canonical representation; + // list order remains significant because it selects installed package slots. + body, err := json.Marshal(input) + if err != nil { + return "", ErrInvalid + } + digest := sha256.Sum256(body) + return hex.EncodeToString(digest[:]), nil +} + +// ValidateSelection checks reconnect configuration without revisiting sources. +// Use it only with a manifest returned by Load. +func ValidateSelection(manifest Manifest, input Input, identity Identity) error { + digest, err := selectionHash(input) + if err != nil || validateIdentity(identity) != nil || manifest.Version != 1 || manifest.Identity != identity || manifest.SelectionSHA256 != digest { + return ErrInvalid + } + return nil +} diff --git a/internal/agentcapabilities/mcp_test.go b/internal/agentcapabilities/mcp_test.go index 65f80dfe2..0b110117c 100644 --- a/internal/agentcapabilities/mcp_test.go +++ b/internal/agentcapabilities/mcp_test.go @@ -12,7 +12,7 @@ func TestMCPOnlyInstalledPackageSurvivesSourceDeletion(t *testing.T) { writeWorkspace(t, workspace, "plugin/.codex-plugin/plugin.json", []byte(`{"name":"native-tools","description":"Package tools.","mcpServers":"./.mcp.json"}`)) writeWorkspace(t, workspace, "plugin/.mcp.json", []byte(`{"mcpServers":{"proof":{"command":"python3","args":["proof.py"],"env_vars":["PLUGIN_TOKEN"]}}}`)) writeWorkspace(t, workspace, "plugin/proof.py", []byte("# Preserved server resource.\n")) - if err := Finalize(workspace, installed, Input{Directories: []string{"/workspace/plugin"}}); err != nil { + if err := Finalize(installed, Input{Directories: []string{"/workspace/plugin"}}, testIdentity, workspaceResolver(workspace)); err != nil { t.Fatal(err) } before, err := Load(installed) @@ -40,7 +40,7 @@ func TestMCPOnlyInstalledPackageSurvivesSourceDeletion(t *testing.T) { func TestInstalledMCPManifestRejectsInvalidPackageRoots(t *testing.T) { for _, roots := range [][]string{{"../private"}, {"/private"}, {"plugins/0", "plugins/0"}} { - body, err := json.Marshal(Manifest{Version: 1, Plugins: roots}) + body, err := json.Marshal(Manifest{Version: 1, Identity: testIdentity, SelectionSHA256: strings.Repeat("0", 64), Plugins: roots}) if err != nil { t.Fatal(err) } diff --git a/internal/agentcapabilities/operation.go b/internal/agentcapabilities/operation.go deleted file mode 100644 index 739ad1087..000000000 --- a/internal/agentcapabilities/operation.go +++ /dev/null @@ -1,13 +0,0 @@ -package agentcapabilities - -import "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" - -// Operation is confidential stdin for the packaged helper, not a public API. -type Operation struct { - Version int `json:"version"` - Action string `json:"action"` - Slot int `json:"slot,omitempty"` - Archive []byte `json:"archive,omitempty"` - Plugin agentplugin.Metadata `json:"plugin,omitempty"` - Sources Input `json:"sources,omitempty"` -} diff --git a/internal/agentcapabilities/selection_test.go b/internal/agentcapabilities/selection_test.go new file mode 100644 index 000000000..5c5dfbd47 --- /dev/null +++ b/internal/agentcapabilities/selection_test.go @@ -0,0 +1,338 @@ +package agentcapabilities + +import ( + "archive/zip" + "bytes" + "encoding/json" + "errors" + "os" + "reflect" + "strings" + "testing" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentbundle" + "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" + "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" +) + +var testIdentity = Identity{ + EnvironmentID: "a1000000-0000-4000-8000-000000000001", + SessionID: "a1000000-0000-4000-8000-000000000002", +} + +// The fixture maps declared managed paths; Finalize knows nothing about it. +func workspaceResolver(workspace *os.Root) DirectoryResolver { + return func(source string) (*os.Root, error) { + if ValidateDirectories([]string{source}) != nil { + return nil, ErrInvalid + } + name := strings.TrimPrefix(strings.TrimPrefix(source, "/workspace"), "/") + if name == "" { + name = "." + } + info, err := workspace.Lstat(name) + if err != nil || !info.IsDir() { + return nil, ErrInvalid + } + return workspace.OpenRoot(name) + } +} + +func testArchive(t *testing.T, files map[string][]byte) []byte { + t.Helper() + var buffer bytes.Buffer + writer := zip.NewWriter(&buffer) + for name, body := range files { + file, err := writer.Create("package/" + name) + if err != nil { + t.Fatal(err) + } + if _, err := file.Write(body); err != nil { + t.Fatal(err) + } + } + if err := writer.Close(); err != nil { + t.Fatal(err) + } + return buffer.Bytes() +} + +func TestBundleAndLocalDirectoryInventoriesAgree(t *testing.T) { + for _, plugin := range []bool{false, true} { + name := "skill" + if plugin { + name = "plugin" + } + t.Run(name, func(t *testing.T) { + bundled, local, source := openTestRoot(t), openTestRoot(t), openTestRoot(t) + files := map[string][]byte{"SKILL.md": skillManifest("proof"), "resource.txt": []byte("preserved")} + skill := agentskill.Metadata{Type: "inline", Name: "proof", Description: "Check proof."} + input := Input{Skills: []agentskill.Metadata{skill}} + if plugin { + files = map[string][]byte{ + ".codex-plugin/plugin.json": []byte(`{"name":"package","description":"Package proof.","skills":"./skills","mcpServers":"./.mcp.json"}`), + ".mcp.json": []byte(`{"mcpServers":{"proof":{"command":"python3","args":["proof.py"]}}}`), + "skills/proof/SKILL.md": skillManifest("proof"), + "proof.py": []byte("# Preserved server.\n"), + } + metadata := agentplugin.Metadata{Type: "inline", Name: "package", Description: "Package proof."} + input = Input{Plugins: []agentplugin.Metadata{metadata}} + if err := InstallPlugin(bundled, 0, testArchive(t, files), metadata); err != nil { + t.Fatal(err) + } + } else if err := InstallSkill(bundled, testArchive(t, files), skill); err != nil { + t.Fatal(err) + } + for path, content := range files { + writeWorkspace(t, source, path, content) + } + localInput := Input{Directories: []string{"/caller/capabilities"}} + resolve := func(path string) (*os.Root, error) { + if path != "/caller/capabilities" { + t.Fatalf("unexpected source: %s", path) + } + return source.OpenRoot(".") + } + if err := Finalize(bundled, input, testIdentity, nil); err != nil { + t.Fatal(err) + } + if err := Finalize(local, localInput, testIdentity, resolve); err != nil { + t.Fatal(err) + } + left, err := Load(bundled) + if err != nil { + t.Fatal(err) + } + right, err := Load(local) + if err != nil { + t.Fatal(err) + } + if len(left.Skills) != 1 || len(right.Skills) != 1 || left.Skills[0].Metadata != right.Skills[0].Metadata { + t.Fatal("Skill inventories diverged") + } + if len(left.MCP) != len(right.MCP) { + t.Fatal("MCP inventories diverged") + } + for i := range left.MCP { + if !reflect.DeepEqual(left.MCP[i].Server, right.MCP[i].Server) { + t.Fatal("MCP declarations diverged") + } + } + if ValidateSelection(left, input, testIdentity) != nil || ValidateSelection(right, localInput, testIdentity) != nil { + t.Fatal("matching selection rejected") + } + }) + } +} + +func TestSnapshotIdentityAndSourceLifetime(t *testing.T) { + source, first, second := openTestRoot(t), openTestRoot(t), openTestRoot(t) + writeWorkspace(t, source, "SKILL.md", skillManifest("proof")) + writeWorkspace(t, source, "resource.txt", []byte("first")) + input := Input{Directories: []string{"/caller/source"}} + resolve := func(string) (*os.Root, error) { return source.OpenRoot(".") } + if err := Finalize(first, input, testIdentity, resolve); err != nil { + t.Fatal(err) + } + writeWorkspace(t, source, "resource.txt", []byte("second")) + before, err := Load(first) + if err != nil { + t.Fatal(err) + } + if content, err := first.ReadFile("directories/0/resource.txt"); err != nil || string(content) != "first" { + t.Fatal("old snapshot followed source edit") + } + next := testIdentity + next.SessionID = "a1000000-0000-4000-8000-000000000003" + for _, scenario := range []string{"session", "environment", "selection"} { + identity, selection := testIdentity, input + switch scenario { + case "session": + identity = next + case "environment": + identity.EnvironmentID = next.SessionID + case "selection": + selection.Directories = []string{"/caller/other"} + } + if !errors.Is(ValidateSelection(before, selection, identity), ErrInvalid) { + t.Fatalf("%s mismatch accepted", scenario) + } + } + if err := Finalize(second, input, next, resolve); err != nil { + t.Fatal(err) + } + if content, err := second.ReadFile("directories/0/resource.txt"); err != nil || string(content) != "second" { + t.Fatal("new Session did not capture new source bytes") + } +} + +func TestEmptySnapshotAndRequiredBinding(t *testing.T) { + installed := openTestRoot(t) + if err := Finalize(installed, Input{}, testIdentity, nil); err != nil { + t.Fatal(err) + } + manifest, err := Load(installed) + if err != nil || len(manifest.Skills) != 0 || len(manifest.MCP) != 0 { + t.Fatalf("empty snapshot: %+v %v", manifest, err) + } + if ValidateSelection(manifest, Input{Skills: []agentskill.Metadata{}, Plugins: []agentplugin.Metadata{}, Directories: []string{}}, testIdentity) != nil { + t.Fatal("nil/empty selections differ") + } + for _, scenario := range []string{"identity", "hash", "hash-uppercase", "old-manifest"} { + t.Run(scenario, func(t *testing.T) { + invalid := manifest + switch scenario { + case "identity": + invalid.Identity = Identity{} + case "hash": + invalid.SelectionSHA256 = "bad" + case "hash-uppercase": + invalid.SelectionSHA256 = strings.ToUpper(invalid.SelectionSHA256) + case "old-manifest": + invalid.Identity, invalid.SelectionSHA256 = Identity{}, "" + } + root := openTestRoot(t) + data, err := json.Marshal(invalid) + if err != nil { + t.Fatal(err) + } + if err := root.WriteFile(ManifestName, data, 0400); err != nil { + t.Fatal(err) + } + if _, err := Load(root); !errors.Is(err, ErrInvalid) { + t.Fatal("invalid binding loaded") + } + }) + } + skill := agentskill.Metadata{Type: "inline", Name: "proof", Description: "Check proof."} + if InstallSkill(installed, testArchive(t, map[string][]byte{"SKILL.md": skillManifest("proof")}), skill) == nil { + t.Fatal("completed snapshot accepted a late import") + } +} + +func TestPartialAndOversizedSnapshotsFailClosed(t *testing.T) { + for _, scenario := range []string{"temporary-manifest", "partial-directory", "unexpected-bundle", "missing-bundle", "oversized"} { + t.Run(scenario, func(t *testing.T) { + source, installed := openTestRoot(t), openTestRoot(t) + writeWorkspace(t, source, "SKILL.md", skillManifest("proof")) + input := Input{Directories: []string{"/caller/source"}} + switch scenario { + case "temporary-manifest": + writeWorkspace(t, installed, ManifestName+".tmp", []byte("partial")) + case "partial-directory": + writeWorkspace(t, installed, "directories/0/partial", []byte("partial")) + case "unexpected-bundle": + writeWorkspace(t, installed, "skills/other/SKILL.md", skillManifest("other")) + case "missing-bundle": + input.Skills = []agentskill.Metadata{{Type: "inline", Name: "missing", Description: "Check proof."}} + case "oversized": + file, err := source.OpenFile("large", os.O_CREATE|os.O_WRONLY, 0600) + if err != nil { + t.Fatal(err) + } + if err := file.Truncate(agentbundle.MaxExpandedBytes + 1); err != nil { + t.Fatal(err) + } + if err := file.Close(); err != nil { + t.Fatal(err) + } + } + calls := 0 + resolve := func(string) (*os.Root, error) { calls++; return source.OpenRoot(".") } + if err := Finalize(installed, input, testIdentity, resolve); !errors.Is(err, ErrInvalid) { + t.Fatal("invalid snapshot finalized") + } + if _, err := Load(installed); !errors.Is(err, ErrInvalid) { + t.Fatal("partial installation loaded") + } + if _, err := installed.Lstat(ManifestName); !os.IsNotExist(err) { + t.Fatal("failure published installed.json") + } + if scenario == "partial-directory" && calls != 0 { + t.Fatal("partial installation recaptured sources") + } + }) + } +} + +func TestSourceValidationKeepsManagedBoundary(t *testing.T) { + if ValidateLocalDirectories([]string{"/home/caller/skills"}) != nil || ValidateDirectories([]string{"/home/caller/skills"}) == nil { + t.Fatal("local spelling and managed access policy were conflated") + } + for _, sources := range [][]string{{"relative"}, {"/one/../two"}, {"/one", "/one"}, {"/one\\two"}, {"/one\n"}} { + if ValidateLocalDirectories(sources) == nil { + t.Fatalf("invalid source accepted: %q", sources) + } + } +} + +func TestInstallerRejectsSourceRootSymlinksAndInvalidSkills(t *testing.T) { + source, installed := openTestRoot(t), openTestRoot(t) + writeWorkspace(t, source, "real/SKILL.md", skillManifest("proof")) + if err := source.Symlink("real", "alias"); err != nil { + t.Fatal(err) + } + if Finalize(installed, Input{Directories: []string{"/workspace/alias"}}, testIdentity, workspaceResolver(source)) == nil { + t.Fatal("source alias accepted") + } + metadata := agentskill.Metadata{Type: "inline", Name: "proof", Description: "Check proof."} + if InstallSkill(installed, testArchive(t, map[string][]byte{"SKILL.md": skillManifest("different")}), metadata) == nil { + t.Fatal("mismatched Skill metadata installed") + } + if _, err := installed.Lstat("skills"); !os.IsNotExist(err) { + t.Fatal("invalid Skill wrote installation data") + } +} + +func TestInstallRejectsAliasedParent(t *testing.T) { + root := openTestRoot(t) + if err := root.Mkdir("other", 0700); err != nil { + t.Fatal(err) + } + if err := root.Symlink("other", "skills"); err != nil { + t.Fatal(err) + } + metadata := agentskill.Metadata{Type: "inline", Name: "proof", Description: "Check proof."} + if InstallSkill(root, testArchive(t, map[string][]byte{"SKILL.md": skillManifest("proof")}), metadata) == nil { + t.Fatal("aliased installation parent accepted") + } + if _, err := root.Lstat("other/proof"); !os.IsNotExist(err) { + t.Fatal("installer wrote through aliased parent") + } +} + +func TestFailedDirectoryCaptureCannotBeReplayed(t *testing.T) { + source, installed := openTestRoot(t), openTestRoot(t) + writeWorkspace(t, source, "first/SKILL.md", skillManifest("first")) + input := Input{Directories: []string{"/workspace/first", "/workspace/second"}} + if Finalize(installed, input, testIdentity, workspaceResolver(source)) == nil { + t.Fatal("missing second source accepted") + } + writeWorkspace(t, source, "first/SKILL.md", skillManifest("changed")) + writeWorkspace(t, source, "second/SKILL.md", skillManifest("second")) + if Finalize(installed, input, testIdentity, workspaceResolver(source)) == nil { + t.Fatal("partial capture was silently retried") + } + body, err := installed.ReadFile("directories/0/SKILL.md") + if err != nil || !bytes.Equal(body, skillManifest("first")) { + t.Fatal("partial capture was replaced") + } +} + +func TestSelectionDigestKeepsSourceOrderAndMetadata(t *testing.T) { + input := Input{Directories: []string{"/one", "/two"}, Skills: []agentskill.Metadata{{Type: "inline", Name: "proof", Description: "Check proof."}}} + digest, err := selectionHash(input) + if err != nil { + t.Fatal(err) + } + manifest := Manifest{Version: 1, Identity: testIdentity, SelectionSHA256: digest} + swapped := input + swapped.Directories = []string{"/two", "/one"} + changed := input + changed.Skills = []agentskill.Metadata{{Type: "inline", Name: "proof", Description: "Changed."}} + for _, selection := range []Input{swapped, changed} { + if ValidateSelection(manifest, selection, testIdentity) == nil { + t.Fatal("changed selection accepted") + } + } +} diff --git a/internal/agentcapabilities/source_directories.go b/internal/agentcapabilities/source_directories.go new file mode 100644 index 000000000..9c8adac61 --- /dev/null +++ b/internal/agentcapabilities/source_directories.go @@ -0,0 +1,50 @@ +package agentcapabilities + +import ( + "path" + "strings" + "unicode/utf8" +) + +// ValidateSourceDirectories checks portable absolute source addresses without +// interpreting them on the Core host. Runtime authorizes and opens local paths. +func ValidateSourceDirectories(directories []string) error { + if len(directories) > 50 { + return ErrInvalid + } + seen := make(map[string]bool, len(directories)) + for _, directory := range directories { + if len(directory) == 0 || len(directory) > 4096 || !utf8.ValidString(directory) || seen[directory] || !portableAbsoluteDirectory(directory) { + return ErrInvalid + } + for _, b := range []byte(directory) { + if b < 0x20 || b == 0x7f { + return ErrInvalid + } + } + seen[directory] = true + } + return nil +} + +func portableAbsoluteDirectory(value string) bool { + // Drive letters and UNC shares are wire syntax, not a Windows implementation. + if len(value) >= 3 && ((value[0] >= 'A' && value[0] <= 'Z') || (value[0] >= 'a' && value[0] <= 'z')) && value[1] == ':' && (value[2] == '/' || value[2] == '\\') { + rest := strings.ReplaceAll(value[2:], "\\", "/") + return path.Clean(rest) == rest && !strings.ContainsAny(rest, ":*?\"<>|") + } + if strings.HasPrefix(value, "\\\\") || strings.HasPrefix(value, "//") { + rest := strings.ReplaceAll(value[2:], "\\", "/") + parts := strings.Split(rest, "/") + if len(parts) < 2 || strings.ContainsAny(rest, ":*?\"<>|") { + return false + } + for _, part := range parts { + if part == "" || part == "." || part == ".." { + return false + } + } + return true + } + return strings.HasPrefix(value, "/") && !strings.Contains(value, "\\") && path.Clean(value) == value +} diff --git a/internal/agentcapabilities/source_directories_test.go b/internal/agentcapabilities/source_directories_test.go new file mode 100644 index 000000000..2d1908177 --- /dev/null +++ b/internal/agentcapabilities/source_directories_test.go @@ -0,0 +1,25 @@ +package agentcapabilities + +import ( + "strings" + "testing" +) + +func TestPortableSourceDirectories(t *testing.T) { + for _, value := range []string{"/", "/home/operator/skills", `C:\`, `C:\Users\operator\skills`, "D:/skills", `\\host\share`, `\\host\share\skills`, "//host/share/skills"} { + if ValidateSourceDirectories([]string{value}) != nil { + t.Fatal("valid source refused", value) + } + } + for _, value := range []string{"", "relative", "C:relative", `\rooted`, `\\host`, `\\?\C:\skills`, `/a/../b`, `C:\a\..\b`, "/a/", "/a\tb", "/a\x7fb", string([]byte{'/', 0xff}), strings.Repeat("a", 4097)} { + if ValidateSourceDirectories([]string{value}) == nil { + t.Fatal("invalid source accepted", value) + } + } + if ValidateSourceDirectories([]string{"/a", "/a"}) == nil || ValidateSourceDirectories(make([]string, 51)) == nil { + t.Fatal("source bounds not enforced") + } + if ValidateDirectories([]string{`C:\skills`}) == nil || ValidateLocalDirectories([]string{`C:\skills`}) == nil { + t.Fatal("portable source weakened local/managed authorization") + } +} diff --git a/internal/agentcapabilities/tree.go b/internal/agentcapabilities/tree.go index a1709f4a8..61ace6284 100644 --- a/internal/agentcapabilities/tree.go +++ b/internal/agentcapabilities/tree.go @@ -72,7 +72,7 @@ func ReadTree(root *os.Root, name string, immutable bool) ([]agentbundle.File, e // writeTree only creates a fresh owned destination. Uncertain writes are never // retried here; the existing Core initialization owner decides cleanup. func writeTree(root *os.Root, name string, files []agentbundle.File) error { - if !validRelative(name) || root.MkdirAll(path.Dir(name), 0700) != nil || root.Mkdir(name, 0700) != nil { + if !validRelative(name) || makeDirectories(root, path.Dir(name)) != nil || root.Mkdir(name, 0700) != nil { return ErrInvalid } directory, err := root.OpenRoot(name) @@ -81,7 +81,7 @@ func writeTree(root *os.Root, name string, files []agentbundle.File) error { } defer directory.Close() for _, file := range files { - if !validRelative(file.Path) || directory.MkdirAll(path.Dir(file.Path), 0700) != nil { + if !validRelative(file.Path) || makeDirectories(directory, path.Dir(file.Path)) != nil { return ErrInvalid } mode := fs.FileMode(0400) @@ -135,3 +135,23 @@ func syncDirectory(root *os.Root, name string) error { } return nil } + +// Create only real directory parents. os.Root prevents escapes, while these +// checks also reject aliases that remain inside the protected installation. +func makeDirectories(root *os.Root, name string) error { + if name == "." { + return nil + } + current := "" + for _, component := range strings.Split(name, "/") { + current = path.Join(current, component) + if err := root.Mkdir(current, 0700); err != nil && !os.IsExist(err) { + return ErrInvalid + } + info, err := root.Lstat(current) + if err != nil || !info.IsDir() { + return ErrInvalid + } + } + return nil +} diff --git a/internal/agentdaemon/gateway/cancellation_test.go b/internal/agentdaemon/gateway/cancellation_test.go index 1f0caa264..9b8c5aa61 100644 --- a/internal/agentdaemon/gateway/cancellation_test.go +++ b/internal/agentdaemon/gateway/cancellation_test.go @@ -58,7 +58,7 @@ func TestArchivedReceiptTracksDeliveryBeyondDoneAndRejectsNewWork(t *testing.T) if _, err := peer.TrackExecutionDelivery("new-run"); err == nil { t.Fatal("new delivery admitted while draining") } - for _, kind := range []string{proto.TypePromptRequest, proto.TypePromptSteer, proto.TypeExecutionPrepare, proto.TypeWorkspaceWrite, proto.TypeWorkspaceRead} { + for _, kind := range []string{proto.TypePromptRequest, proto.TypePromptSteer, proto.TypeExecutionPrepare, proto.TypeRuntimePrepare, proto.TypeWorkspaceWrite, proto.TypeWorkspaceRead} { env, _ := proto.NewEnvelope(kind, "run", nil) if err := peer.Send(t.Context(), env); err == nil { t.Fatal("drain permitted new operation", kind) diff --git a/internal/agentdaemon/gateway/chunk_exchange.go b/internal/agentdaemon/gateway/chunk_exchange.go new file mode 100644 index 000000000..b7bb2058e --- /dev/null +++ b/internal/agentdaemon/gateway/chunk_exchange.go @@ -0,0 +1,26 @@ +package gateway + +import ( + "context" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" +) + +// exchangeChunkFrame sends once and waits for the owning transfer's next receipt. +// Callers validate their own frame and receipt; no transport outcome is retried. +func (s *Session) exchangeChunkFrame(ctx context.Context, env proto.Envelope, replies <-chan proto.Envelope) (proto.Envelope, error) { + if err := s.Send(ctx, env); err != nil { + return proto.Envelope{}, err + } + select { + case reply, ok := <-replies: + if !ok { + return proto.Envelope{}, ErrSessionClosed + } + return reply, nil + case <-ctx.Done(): + return proto.Envelope{}, ctx.Err() + case <-s.closed: + return proto.Envelope{}, ErrSessionClosed + } +} diff --git a/internal/agentdaemon/gateway/runtime_prepare.go b/internal/agentdaemon/gateway/runtime_prepare.go new file mode 100644 index 000000000..0a5076387 --- /dev/null +++ b/internal/agentdaemon/gateway/runtime_prepare.go @@ -0,0 +1,111 @@ +package gateway + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/google/uuid" +) + +// PrepareRuntime sends one typed preparation operation once. An +// interrupted or unconfirmed transfer reports unknown; callers must not replay it. +func (s *Session) PrepareRuntime(ctx context.Context, id string, request proto.RuntimePreparePayload, data []byte) (proto.RuntimePrepareResultPayload, error) { + unknown := proto.RuntimePrepareResultPayload{Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"} + parsed, err := uuid.Parse(id) + if err != nil || parsed == uuid.Nil || parsed.String() != id || len(data) > proto.RuntimePrepareMaxBytes || + (request.Step != "" && request.Step != "begin") { + return unknown, errors.New("agentdaemon gateway: invalid Runtime preparation") + } + request.Step = "begin" + if request.Action == "finalize" || request.Action == "initialize" { + if len(data) != 0 { + return unknown, errors.New("agentdaemon gateway: invalid Runtime finalization") + } + } else { + digest := sha256.Sum256(data) + expected := hex.EncodeToString(digest[:]) + if (request.SizeBytes != 0 && request.SizeBytes != len(data)) || (request.SHA256 != "" && request.SHA256 != expected) { + return unknown, errors.New("agentdaemon gateway: Runtime archive mismatch") + } + request.SizeBytes, request.SHA256 = len(data), expected + } + if !proto.ValidRuntimePrepareRequest(request) { + return unknown, errors.New("agentdaemon gateway: invalid Runtime preparation") + } + s.capabilitiesMu.Lock() + if s.IsClosed() { + s.capabilitiesMu.Unlock() + return unknown, ErrSessionClosed + } + if len(s.capabilities) != 0 { + s.capabilitiesMu.Unlock() + return unknown, errors.New("agentdaemon gateway: Runtime preparation capacity") + } + replies := make(chan proto.Envelope, 1) + s.capabilities = map[string]chan proto.Envelope{id: replies} + s.capabilitiesMu.Unlock() + defer func() { s.capabilitiesMu.Lock(); delete(s.capabilities, id); s.capabilitiesMu.Unlock() }() + ctx, cancel := context.WithTimeout(ctx, 195*time.Second) + defer cancel() + exchange := func(payload proto.RuntimePreparePayload, outcome string, offset int) (proto.RuntimePrepareResultPayload, error) { + env, err := proto.NewEnvelope(proto.TypeRuntimePrepare, id, payload) + if err != nil { + return unknown, errors.New("agentdaemon gateway: invalid Runtime frame") + } + encoded, err := json.Marshal(env) + if err != nil || len(encoded) > proto.RuntimePrepareMaxFrameBytes { + return unknown, errors.New("agentdaemon gateway: invalid Runtime frame") + } + reply, err := s.exchangeChunkFrame(ctx, env, replies) + if err != nil { + return unknown, err + } + var result proto.RuntimePrepareResultPayload + if len(reply.Payload) > proto.RuntimePrepareMaxFrameBytes || reply.DecodePayload(&result) != nil || + !proto.ValidRuntimePrepareResult(result, outcome, offset, len(data)) { + return unknown, errors.New("agentdaemon gateway: invalid Runtime receipt") + } + if result.Outcome == "unknown" { + return result, errors.New("agentdaemon gateway: Runtime preparation unconfirmed") + } + return result, nil + } + result, err := exchange(request, "ready", 0) + if err != nil || result.Outcome != "ready" { + return result, err + } + for offset := 0; offset < len(data); { + end := min(offset+proto.RuntimePrepareChunkBytes, len(data)) + result, err = exchange(proto.RuntimePreparePayload{Step: "chunk", Offset: offset, Data: data[offset:end]}, "received", end) + if err != nil || result.Outcome != "received" { + return result, err + } + offset = end + } + return exchange(proto.RuntimePreparePayload{Step: "commit"}, "completed", 0) +} + +func (s *Session) dispatchCapabilities(env proto.Envelope) { + s.capabilitiesMu.Lock() + defer s.capabilitiesMu.Unlock() + if replies := s.capabilities[env.ID]; replies != nil { + select { + case replies <- env: + default: + } + } +} + +func (s *Session) closeCapabilities() { + s.capabilitiesMu.Lock() + defer s.capabilitiesMu.Unlock() + for id, replies := range s.capabilities { + close(replies) + delete(s.capabilities, id) + } +} diff --git a/internal/agentdaemon/gateway/runtime_prepare_test.go b/internal/agentdaemon/gateway/runtime_prepare_test.go new file mode 100644 index 000000000..223d226a3 --- /dev/null +++ b/internal/agentdaemon/gateway/runtime_prepare_test.go @@ -0,0 +1,329 @@ +package gateway + +import ( + "bytes" + "context" + "crypto/sha256" + "encoding/hex" + "encoding/json" + "errors" + "fmt" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" + "github.com/google/uuid" +) + +func skillPreparation() proto.RuntimePreparePayload { + return proto.RuntimePreparePayload{EnvironmentID: uuid.NewString(), SessionID: uuid.NewString(), Action: "skill", + Skill: &agentskill.Metadata{Type: "inline", Name: "example", Description: "Example"}} +} + +type capabilityOutcome struct { + result proto.RuntimePrepareResultPayload + err error +} + +func beginCapabilities(s *Session, ctx context.Context, id string, request proto.RuntimePreparePayload, data []byte) <-chan capabilityOutcome { + done := make(chan capabilityOutcome, 1) + go func() { r, err := s.PrepareRuntime(ctx, id, request, data); done <- capabilityOutcome{r, err} }() + return done +} +func nextCapabilityFrame(t *testing.T, s *Session) proto.Envelope { + t.Helper() + select { + case env := <-s.sendCh: + return env + case <-time.After(3 * time.Second): + t.Fatal("transfer stopped unexpectedly") + return proto.Envelope{} + } +} +func finishCapabilities(t *testing.T, done <-chan capabilityOutcome) capabilityOutcome { + t.Helper() + select { + case result := <-done: + return result + case <-time.After(3 * time.Second): + t.Fatal("transfer did not settle") + return capabilityOutcome{} + } +} +func replyCapabilities(s *Session, id string, result proto.RuntimePrepareResultPayload) { + reply, _ := proto.NewEnvelope(proto.TypeRuntimePrepareResult, id, result) + s.dispatch(reply) +} +func noCapabilityFrame(t *testing.T, s *Session) { + t.Helper() + select { + case env := <-s.sendCh: + t.Fatal("unexpected extra frame", env.Type) + default: + } +} + +func TestCapabilitiesTransfersMoreThanFrameLimitAndCorrelates(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + data := bytes.Repeat([]byte{0, 255, 3}, 2<<20) + id := uuid.NewString() + request := skillPreparation() + done := beginCapabilities(s, t.Context(), id, request, data) + var received []byte + for { + env := nextCapabilityFrame(t, s) + encoded, err := json.Marshal(env) + var p proto.RuntimePreparePayload + if err != nil || len(encoded) > proto.RuntimePrepareMaxFrameBytes || env.ID != id || env.Type != proto.TypeRuntimePrepare || env.DecodePayload(&p) != nil || !proto.ValidRuntimePrepareRequest(p) { + t.Fatal("invalid frame") + } + result := proto.RuntimePrepareResultPayload{} + switch p.Step { + case "begin": + digest := sha256.Sum256(data) + if p.EnvironmentID != request.EnvironmentID || p.SessionID != request.SessionID || p.Skill == nil || *p.Skill != *request.Skill || p.SizeBytes != len(data) || p.SHA256 != hex.EncodeToString(digest[:]) { + t.Fatal("identity, metadata or digest changed") + } + result.Outcome = "ready" + case "chunk": + if p.Offset != len(received) { + t.Fatal("noncontiguous chunks") + } + received = append(received, p.Data...) + result.Outcome, result.Offset = "received", len(received) + case "commit": + if !bytes.Equal(received, data) { + t.Fatal("archive changed") + } + result.Outcome, result.SizeBytes = "completed", len(data) + } + replyCapabilities(s, uuid.NewString(), result) + replyCapabilities(s, id, result) + if p.Step == "commit" { + break + } + } + result := finishCapabilities(t, done) + if result.err != nil || result.result.Outcome != "completed" || result.result.SizeBytes != len(data) { + t.Fatal(result) + } + noCapabilityFrame(t, s) +} + +func TestCapabilitiesFinalizeTransfersNoArchive(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + request := proto.RuntimePreparePayload{EnvironmentID: uuid.NewString(), SessionID: uuid.NewString(), Action: "finalize", Sources: &agentcapabilities.Input{}} + id := uuid.NewString() + done := beginCapabilities(s, t.Context(), id, request, nil) + env := nextCapabilityFrame(t, s) + var p proto.RuntimePreparePayload + if env.DecodePayload(&p) != nil || p.Step != "begin" || p.Sources == nil || p.SizeBytes != 0 || p.SHA256 != "" { + t.Fatal("invalid finalization", p) + } + replyCapabilities(s, id, proto.RuntimePrepareResultPayload{Outcome: "ready"}) + env = nextCapabilityFrame(t, s) + if env.DecodePayload(&p) != nil || p.Step != "commit" { + t.Fatal("finalization sent archive") + } + replyCapabilities(s, id, proto.RuntimePrepareResultPayload{Outcome: "completed"}) + if result := finishCapabilities(t, done); result.err != nil || result.result.Outcome != "completed" { + t.Fatal(result) + } +} + +func TestCapabilitiesRefusesConflictingArchiveBeforeSending(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + for _, mutate := range []func(*proto.RuntimePreparePayload){ + func(p *proto.RuntimePreparePayload) { p.SHA256 = "incorrect" }, + func(p *proto.RuntimePreparePayload) { p.SizeBytes = 999 }, + func(p *proto.RuntimePreparePayload) { p.Sources = &agentcapabilities.Input{} }, + } { + request := skillPreparation() + mutate(&request) + if result, err := s.PrepareRuntime(t.Context(), uuid.NewString(), request, []byte("data")); err == nil || result.Outcome != "unknown" { + t.Fatal("conflict admitted", result, err) + } + noCapabilityFrame(t, s) + } +} + +func TestCapabilitiesStopsAtTerminalOrMalformedReceipt(t *testing.T) { + for name, receipt := range map[string]proto.RuntimePrepareResultPayload{ + "rejected": {Outcome: "rejected", ErrorCode: "runtime_preparation_rejected"}, + "failed": {Outcome: "failed", ErrorCode: "runtime_preparation_failed"}, + "unknown": {Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"}, + "premature completed": {Outcome: "completed", SizeBytes: 4}, + "wrong offset": {Outcome: "ready", Offset: 1}, + "unsafe code": {Outcome: "rejected", ErrorCode: "private detail"}, + } { + t.Run(name, func(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + id := uuid.NewString() + done := beginCapabilities(s, t.Context(), id, skillPreparation(), []byte("data")) + nextCapabilityFrame(t, s) + replyCapabilities(s, id, receipt) + result := finishCapabilities(t, done) + if name == "rejected" || name == "failed" { + if result.err != nil || result.result != receipt { + t.Fatal(result) + } + } else if result.err == nil || result.result.Outcome != "unknown" { + t.Fatal("uncertainty lost", result) + } + noCapabilityFrame(t, s) + }) + } +} + +func TestCapabilitiesRejectsWrongChunkReceipt(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + id := uuid.NewString() + done := beginCapabilities(s, t.Context(), id, skillPreparation(), []byte("data")) + nextCapabilityFrame(t, s) + replyCapabilities(s, id, proto.RuntimePrepareResultPayload{Outcome: "ready"}) + nextCapabilityFrame(t, s) + replyCapabilities(s, id, proto.RuntimePrepareResultPayload{Outcome: "received", Offset: 3}) + result := finishCapabilities(t, done) + if result.err == nil || result.result.Outcome != "unknown" { + t.Fatal(result) + } + noCapabilityFrame(t, s) +} + +func TestCapabilitiesConnectionOwnershipAndUnknownInterruption(t *testing.T) { + for _, closeConnection := range []bool{false, true} { + t.Run(map[bool]string{false: "deadline", true: "disconnect"}[closeConnection], func(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + ctx, cancel := context.WithTimeout(t.Context(), 150*time.Millisecond) + defer cancel() + done := beginCapabilities(s, ctx, uuid.NewString(), skillPreparation(), []byte("data")) + nextCapabilityFrame(t, s) + if _, err := s.PrepareRuntime(t.Context(), uuid.NewString(), skillPreparation(), []byte("second")); err == nil { + t.Fatal("concurrent transfer admitted") + } + if closeConnection { + s.Close("lost connection") + } + result := finishCapabilities(t, done) + if result.result.Outcome != "unknown" || result.result.ErrorCode != "runtime_preparation_unconfirmed" { + t.Fatal("interruption claimed rejection", result) + } + expected := error(context.DeadlineExceeded) + if closeConnection { + expected = ErrSessionClosed + } + if !errors.Is(result.err, expected) { + t.Fatal(result.err) + } + noCapabilityFrame(t, s) + s.capabilitiesMu.Lock() + remaining := len(s.capabilities) + s.capabilitiesMu.Unlock() + if remaining != 0 { + t.Fatal("transfer ownership retained") + } + }) + } +} + +func TestRuntimeInitialFileChunking(t *testing.T) { + for _, size := range []int{0, (4 << 20) + 1} { + t.Run(fmt.Sprint(size), func(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + request := proto.RuntimePreparePayload{EnvironmentID: uuid.NewString(), SessionID: uuid.NewString(), Action: "file", File: &proto.RuntimeInitialFile{Path: "/workspace/project/file"}} + data := bytes.Repeat([]byte("z"), size) + id := uuid.NewString() + done := beginCapabilities(s, t.Context(), id, request, data) + var received []byte + for { + env := nextCapabilityFrame(t, s) + var p proto.RuntimePreparePayload + if env.DecodePayload(&p) != nil || !proto.ValidRuntimePrepareRequest(p) { + t.Fatal("invalid file frame") + } + result := proto.RuntimePrepareResultPayload{} + switch p.Step { + case "begin": + sum := sha256.Sum256(data) + if p.File == nil || p.File.Path != request.File.Path || p.SizeBytes != size || p.SHA256 != hex.EncodeToString(sum[:]) { + t.Fatal("file header changed") + } + result.Outcome = "ready" + case "chunk": + if p.Offset != len(received) { + t.Fatal("wrong chunk offset") + } + received = append(received, p.Data...) + result.Outcome, result.Offset = "received", len(received) + case "commit": + if !bytes.Equal(data, received) { + t.Fatal("file bytes changed") + } + result.Outcome, result.SizeBytes = "completed", size + } + replyCapabilities(s, id, result) + if p.Step == "commit" { + break + } + } + got := finishCapabilities(t, done) + if got.err != nil || got.result.Outcome != "completed" { + t.Fatal(got) + } + noCapabilityFrame(t, s) + }) + } +} + +func TestRuntimeInitializationNoDataAndExitReceipt(t *testing.T) { + for _, exit := range []int{0, 42, 256} { + t.Run(fmt.Sprint(exit), func(t *testing.T) { + s := NewSession(newFakeConn(), "device", "tenant", "test", nil, nil) + defer s.Close("test") + request := proto.RuntimePreparePayload{EnvironmentID: uuid.NewString(), SessionID: uuid.NewString(), Action: "initialize", Initialization: &proto.RuntimeInitialization{Action: "setup", Network: "enabled", Command: "echo test"}} + if _, err := s.PrepareRuntime(t.Context(), uuid.NewString(), request, []byte("forbidden")); err == nil { + t.Fatal("initialization body accepted") + } + noCapabilityFrame(t, s) + id := uuid.NewString() + done := beginCapabilities(s, t.Context(), id, request, nil) + env := nextCapabilityFrame(t, s) + var p proto.RuntimePreparePayload + if env.DecodePayload(&p) != nil { + t.Fatal("invalid initialization frame") + } + if p.Step != "begin" || p.Initialization == nil || p.Initialization.Command != request.Initialization.Command || p.SizeBytes != 0 || p.SHA256 != "" { + t.Fatal("initialization changed") + } + replyCapabilities(s, id, proto.RuntimePrepareResultPayload{Outcome: "ready"}) + env = nextCapabilityFrame(t, s) + p = proto.RuntimePreparePayload{} + if env.DecodePayload(&p) != nil || p.Step != "commit" { + t.Fatal("initialization sent data") + } + result := proto.RuntimePrepareResultPayload{Outcome: "completed"} + if exit != 0 { + result = proto.RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: "runtime_preparation_failed", ExitCode: exit} + } + replyCapabilities(s, id, result) + got := finishCapabilities(t, done) + if exit == 256 { + if got.err == nil || got.result.Outcome != "unknown" { + t.Fatal("unsafe exit accepted", got) + } + } else if got.err != nil || got.result != result { + t.Fatal(got) + } + noCapabilityFrame(t, s) + }) + } +} diff --git a/internal/agentdaemon/gateway/session.go b/internal/agentdaemon/gateway/session.go index 7e79ddfdb..edd779e1c 100644 --- a/internal/agentdaemon/gateway/session.go +++ b/internal/agentdaemon/gateway/session.go @@ -107,6 +107,8 @@ type Session struct { subs map[string]*Subscription preparationMu sync.Mutex preparations map[string]*preparationSubscription + capabilitiesMu sync.Mutex + capabilities map[string]chan proto.Envelope workspaceWriteMu sync.Mutex workspaceWrites map[string]chan proto.Envelope suspendMu sync.Mutex @@ -273,6 +275,7 @@ func (s *Session) Close(reason string) { s.closeWorkspaceReads() s.closeSuspendReplies() s.closeWorkspaceWrites() + s.closeCapabilities() s.closeWorkspaceExports() s.markOfflineOnClose() s.releaseOwnerLease() @@ -605,6 +608,9 @@ func (s *Session) dispatch(env proto.Envelope) { case proto.TypeWorkspaceExportResult: s.dispatchWorkspaceExport(env) return + case proto.TypeRuntimePrepareResult: + s.dispatchCapabilities(env) + return case proto.TypeWorkspaceWriteResult: s.dispatchWorkspaceWrite(env) case proto.TypeWorkspaceReadResult: diff --git a/internal/agentdaemon/gateway/workspace_write.go b/internal/agentdaemon/gateway/workspace_write.go index d614acfb3..b2ee6799f 100644 --- a/internal/agentdaemon/gateway/workspace_write.go +++ b/internal/agentdaemon/gateway/workspace_write.go @@ -44,24 +44,15 @@ func (s *Session) WriteWorkspaceFile(ctx context.Context, id string, request pro if err != nil || len(env.Payload) > proto.WorkspaceWriteMaxFrameBytes { return empty, errors.New("agentdaemon gateway: invalid write frame") } - if err := s.Send(ctx, env); err != nil { + reply, err := s.exchangeChunkFrame(ctx, env, replies) + if err != nil { return empty, err } - select { - case env, ok := <-replies: - if !ok { - return empty, ErrSessionClosed - } - var result proto.WorkspaceWriteResultPayload - if env.DecodePayload(&result) != nil || !validWorkspaceWriteResult(result, outcome, offset, len(data)) { - return empty, errors.New("agentdaemon gateway: invalid write receipt") - } - return result, nil - case <-ctx.Done(): - return empty, ctx.Err() - case <-s.closed: - return empty, ErrSessionClosed + var result proto.WorkspaceWriteResultPayload + if reply.DecodePayload(&result) != nil || !validWorkspaceWriteResult(result, outcome, offset, len(data)) { + return empty, errors.New("agentdaemon gateway: invalid write receipt") } + return result, nil } result, err := exchange(request, "ready", 0) if err != nil || result.Outcome != "ready" { diff --git a/internal/agentdaemon/proto/envelope_test.go b/internal/agentdaemon/proto/envelope_test.go index 8aace4d66..c7388daca 100644 --- a/internal/agentdaemon/proto/envelope_test.go +++ b/internal/agentdaemon/proto/envelope_test.go @@ -88,8 +88,8 @@ func TestVersionCompatible(t *testing.T) { ok bool }{ {Version, true}, // exact match - {"0.5.99", true}, // patch drift OK - {"0.4.99", false}, // minor drift NOT OK + {"0.7.99", true}, // patch drift OK + {"0.6.99", false}, // minor drift NOT OK {"1.0.0", false}, // major drift NOT OK {"", false}, // missing {"garbage", false}, // unparseable diff --git a/internal/agentdaemon/proto/environment.go b/internal/agentdaemon/proto/environment.go index 2ad6c4910..74772af6a 100644 --- a/internal/agentdaemon/proto/environment.go +++ b/internal/agentdaemon/proto/environment.go @@ -5,10 +5,15 @@ import ( "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" ) -// LocalEnvironment references a deployment-bound workspace; it never supplies a path. +// LocalEnvironment names a frozen workspace selection. Runtime must verify it +// against the bound local root before resolving capabilities or native execution. type LocalEnvironment struct { - ID string `json:"id"` - // Capabilities requests the completed, protected Runtime installation. + CapabilityRoot string `json:"-"` + ID string `json:"id"` + WorkspaceDirectory string `json:"workspace_directory"` + CapabilitySources *agentcapabilities.Input `json:"capability_sources"` + // Capabilities is derived from the frozen selection for engine qualification; + // Runtime still ensures and loads the protected installation before execution. Capabilities bool `json:"capabilities,omitempty"` // Skills is resolved by the bound daemon; wire input cannot supply paths. Skills []agentcapabilities.InstalledSkill `json:"-"` @@ -26,9 +31,11 @@ type LocalEnvironment struct { // EnvironmentMCP is transient Runtime configuration. Do not log it: HTTP headers // and the selected user bearer may be confidential. It is not agent.tools MCP. type EnvironmentMCP struct { - PackageRoot string - Server agentplugin.MCPServer - BearerToken *string + InstallationRoot string + WorkspaceRoot string + PackageRoot string + Server agentplugin.MCPServer + BearerToken *string } func (r PromptRequestPayload) EnvironmentID() string { diff --git a/internal/agentdaemon/proto/preparation.go b/internal/agentdaemon/proto/preparation.go index f26c9eb7e..b03ba0905 100644 --- a/internal/agentdaemon/proto/preparation.go +++ b/internal/agentdaemon/proto/preparation.go @@ -10,16 +10,18 @@ const ( ) // ExecutionPreparePayload reuses execution configuration without accepting input -// or product authoring. The initial private profile requires a bound local environment, -// stable state key, strict resume and completion release. +// or product authoring. SessionID identifies the immutable configuration owner; +// each request reserves a separate Turn admission on its Runtime Executor. type ExecutionPreparePayload struct { + SessionID string `json:"session_id"` Configuration PromptRequestPayload `json:"configuration"` } type ExecutionStartPayload struct { - Handle string `json:"handle"` - RunID string `json:"run_id"` - Input MessageInput `json:"input"` + ExecutorID string `json:"executor_id"` + Handle string `json:"handle"` + RunID string `json:"run_id"` + Input MessageInput `json:"input"` } type ExecutionReleasePayload struct { @@ -31,6 +33,10 @@ type ExecutionReleasePayload struct { // order. ExpiresAt is Unix milliseconds. Repeated requests do not extend it. Retired request IDs // may allocate a fresh handle, but an old handle can never start its replacement. type PreparationStatusPayload struct { + // ExecutorID identifies the native resource owner, independently of this admission handle. + ExecutorID string `json:"executor_id,omitempty"` + // Reused is true when readiness reused an existing settled Executor. + Reused bool `json:"reused,omitempty"` Handle string `json:"handle,omitempty"` Revision uint64 `json:"revision"` State string `json:"state"` diff --git a/internal/agentdaemon/proto/runtime_prepare.go b/internal/agentdaemon/proto/runtime_prepare.go new file mode 100644 index 000000000..921f75efc --- /dev/null +++ b/internal/agentdaemon/proto/runtime_prepare.go @@ -0,0 +1,214 @@ +package proto + +import ( + "encoding/hex" + "encoding/json" + "path" + "strings" + "unicode/utf8" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" + "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" + "github.com/google/uuid" +) + +const ( + TypeRuntimePrepare = "runtime_prepare" + TypeRuntimePrepareResult = "runtime_prepare_result" + RuntimePrepareMaxBytes = 50 << 20 + RuntimePrepareChunkBytes = WorkspaceWriteChunkBytes + RuntimePrepareMaxFrameBytes = 1 << 20 +) + +// RuntimeInitialFile addresses a file within the logical workspace. +type RuntimeInitialFile struct { + Path string `json:"path"` +} + +type RuntimeInitialization struct { + Action string `json:"action"` + Env map[string]string `json:"env,omitempty"` + Network string `json:"network,omitempty"` + Packages []string `json:"packages,omitempty"` + Command string `json:"command,omitempty"` + CWD string `json:"cwd,omitempty"` +} + +// RuntimePreparePayload carries typed initialization or capability preparation. +// Runtime resolves logical paths and owns installation destinations. Envelope.ID +// identifies one connection-local transfer. +type RuntimePreparePayload struct { + Step string `json:"step"` + EnvironmentID string `json:"environment_id,omitempty"` + SessionID string `json:"session_id,omitempty"` + Action string `json:"action,omitempty"` + Slot int `json:"slot,omitempty"` + Skill *agentskill.Metadata `json:"skill,omitempty"` + Plugin *agentplugin.Metadata `json:"plugin,omitempty"` + File *RuntimeInitialFile `json:"file,omitempty"` + Initialization *RuntimeInitialization `json:"initialization,omitempty"` + Sources *agentcapabilities.Input `json:"sources,omitempty"` + SizeBytes int `json:"size_bytes,omitempty"` + SHA256 string `json:"sha256,omitempty"` + Offset int `json:"offset,omitempty"` + Data []byte `json:"data,omitempty"` +} + +type RuntimePrepareResultPayload struct { + ExitCode int `json:"exit_code,omitempty"` + Outcome string `json:"outcome"` + Offset int `json:"offset,omitempty"` + SizeBytes int `json:"size_bytes,omitempty"` + ErrorCode string `json:"error_code,omitempty"` +} + +func ValidRuntimePrepareRequest(p RuntimePreparePayload) bool { + if p.Step == "begin" { + for _, id := range []string{p.EnvironmentID, p.SessionID} { + value, err := uuid.Parse(id) + if err != nil || value == uuid.Nil || value.String() != id { + return false + } + } + if p.Offset != 0 || len(p.Data) != 0 { + return false + } + metadata := 0 + for _, present := range []bool{p.Skill != nil, p.Plugin != nil, p.Sources != nil, p.File != nil, p.Initialization != nil} { + if present { + metadata++ + } + } + if metadata != 1 { + return false + } + switch p.Action { + case "skill": + if p.Skill == nil || p.Plugin != nil || p.Sources != nil || p.Slot != 0 || + agentcapabilities.ValidateInput(agentcapabilities.Input{Skills: []agentskill.Metadata{*p.Skill}}) != nil { + return false + } + case "plugin": + if p.Plugin == nil || p.Skill != nil || p.Sources != nil || p.Slot < 0 || p.Slot >= 50 || + agentcapabilities.ValidateInput(agentcapabilities.Input{Plugins: []agentplugin.Metadata{*p.Plugin}}) != nil { + return false + } + case "file": + if p.File == nil || p.Slot != 0 || !validWorkspacePath(p.File.Path, false) { + return false + } + case "initialize": + if p.Initialization == nil || p.Slot != 0 || p.SizeBytes != 0 || p.SHA256 != "" || !validRuntimeInitialization(*p.Initialization) { + return false + } + case "finalize": + if p.Skill != nil || p.Plugin != nil || p.Sources == nil || p.Slot != 0 || + p.SizeBytes != 0 || p.SHA256 != "" || agentcapabilities.ValidateInput(*p.Sources) != nil { + return false + } + default: + return false + } + if p.Action != "finalize" && p.Action != "initialize" { + digest, err := hex.DecodeString(p.SHA256) + if err != nil || len(digest) != 32 || strings.ToLower(p.SHA256) != p.SHA256 || + p.SizeBytes < 0 || (p.SizeBytes == 0 && p.Action != "file") || p.SizeBytes > RuntimePrepareMaxBytes { + return false + } + } + encoded, err := json.Marshal(p) + return err == nil && len(encoded) <= RuntimePrepareMaxFrameBytes + } + if p.EnvironmentID != "" || p.SessionID != "" || p.Action != "" || p.Slot != 0 || + p.Skill != nil || p.Plugin != nil || p.Sources != nil || p.File != nil || p.Initialization != nil || p.SizeBytes != 0 || p.SHA256 != "" { + return false + } + switch p.Step { + case "chunk": + return p.Offset >= 0 && p.Offset <= RuntimePrepareMaxBytes && len(p.Data) > 0 && + len(p.Data) <= RuntimePrepareChunkBytes && len(p.Data) <= RuntimePrepareMaxBytes-p.Offset + case "commit": + return p.Offset == 0 && len(p.Data) == 0 + default: + return false + } +} + +// ValidRuntimePrepareResult accepts only the expected success receipt or a finite +// terminal category. An unknown effect can never be represented as rejection. +func ValidRuntimePrepareResult(r RuntimePrepareResultPayload, expected string, offset, size int) bool { + if r.ExitCode < 0 || r.ExitCode > 255 || (r.Outcome != "failed" && r.ExitCode != 0) { + return false + } + switch r.Outcome { + case "rejected": + if r.Offset != 0 || r.SizeBytes != 0 { + return false + } + switch r.ErrorCode { + case "invalid_request", "resource_unavailable", "runtime_preparation_capacity", "runtime_preparation_unsupported", "runtime_preparation_rejected": + return true + default: + return false + } + case "failed": + return r.Offset == 0 && r.SizeBytes == 0 && r.ErrorCode == "runtime_preparation_failed" + case "unknown": + return r.Offset == 0 && r.SizeBytes == 0 && r.ErrorCode == "runtime_preparation_unconfirmed" + } + if r.Outcome != expected || r.Offset != offset || r.ErrorCode != "" { + return false + } + switch expected { + case "completed": + return r.SizeBytes == size + case "ready", "received": + return r.SizeBytes == 0 + default: + return false + } +} + +// Workspace paths are logical protocol addresses, resolved by the Runtime. +func validWorkspacePath(value string, allowRoot bool) bool { + if len(value) > 4096 || !utf8.ValidString(value) || path.Clean(value) != value || strings.ContainsAny(value, "\\\x00\r\n") { + return false + } + return (allowRoot && value == "/workspace") || strings.HasPrefix(value, "/workspace/") +} + +func validRuntimeInitialization(p RuntimeInitialization) bool { + if p.Network != "" && p.Network != "enabled" && p.Network != "disabled" { + return false + } + if p.CWD != "" && !validWorkspacePath(p.CWD, true) { + return false + } + switch p.Action { + case "configure": + if p.Packages != nil || p.Command != "" || p.CWD != "" || len(p.Env) > 1000 { + return false + } + for key, value := range p.Env { + if key == "" || !utf8.ValidString(key) || !utf8.ValidString(value) || strings.ContainsAny(key, "=\x00\r\n") || strings.ContainsRune(value, 0) { + return false + } + } + return true + case "system", "npm", "python": + if p.Env != nil || p.Command != "" || p.CWD != "" || len(p.Packages) == 0 || len(p.Packages) > 1000 || p.Network == "" { + return false + } + for _, value := range p.Packages { + if value == "" || !utf8.ValidString(value) || strings.HasPrefix(value, "-") || strings.ContainsAny(value, "\x00\r\n") { + return false + } + } + return true + case "setup": + return p.Env == nil && p.Packages == nil && p.Network != "" && p.Command != "" && utf8.ValidString(p.Command) && !strings.ContainsRune(p.Command, 0) + default: + return false + } +} diff --git a/internal/agentdaemon/proto/runtime_prepare_test.go b/internal/agentdaemon/proto/runtime_prepare_test.go new file mode 100644 index 000000000..588520ca8 --- /dev/null +++ b/internal/agentdaemon/proto/runtime_prepare_test.go @@ -0,0 +1,275 @@ +package proto + +import ( + "encoding/json" + "reflect" + "strings" + "testing" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" + "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" + "github.com/google/uuid" +) + +func capabilityBegin() RuntimePreparePayload { + return RuntimePreparePayload{Step: "begin", EnvironmentID: uuid.NewString(), SessionID: uuid.NewString(), + Action: "skill", Skill: &agentskill.Metadata{Type: "inline", Name: "example", Description: "Example"}, SizeBytes: 10, SHA256: strings.Repeat("a", 64)} +} + +func TestCapabilitiesRequestValidation(t *testing.T) { + valid := capabilityBegin() + if !ValidRuntimePrepareRequest(valid) { + t.Fatal("valid skill refused") + } + for name, mutate := range map[string]func(*RuntimePreparePayload){ + "noncanonical identity": func(p *RuntimePreparePayload) { p.EnvironmentID = "AAAAAAAA-AAAA-4AAA-8AAA-AAAAAAAAAAAA" }, + "zero identity": func(p *RuntimePreparePayload) { p.SessionID = uuid.Nil.String() }, + "begin data": func(p *RuntimePreparePayload) { p.Data = []byte("x") }, + "begin offset": func(p *RuntimePreparePayload) { p.Offset = 1 }, + "mixed metadata": func(p *RuntimePreparePayload) { + p.Plugin = &agentplugin.Metadata{Type: "inline", Name: "example", Description: "Example"} + }, + "mixed sources": func(p *RuntimePreparePayload) { p.Sources = &agentcapabilities.Input{} }, + "missing metadata": func(p *RuntimePreparePayload) { p.Skill = nil }, + "wrong metadata type": func(p *RuntimePreparePayload) { + p.Skill = &agentskill.Metadata{Type: "reference", Name: "example", Description: "Example"} + }, + "missing description": func(p *RuntimePreparePayload) { p.Skill = &agentskill.Metadata{Type: "inline", Name: "example"} }, + "oversized header": func(p *RuntimePreparePayload) { + p.Skill = &agentskill.Metadata{Type: "inline", Name: "example", Description: strings.Repeat("x", RuntimePrepareMaxFrameBytes)} + }, + "skill slot": func(p *RuntimePreparePayload) { p.Slot = 1 }, + "no digest": func(p *RuntimePreparePayload) { p.SHA256 = "" }, + "uppercase digest": func(p *RuntimePreparePayload) { p.SHA256 = strings.ToUpper(p.SHA256) }, + "short digest": func(p *RuntimePreparePayload) { p.SHA256 = "aa" }, + "no archive": func(p *RuntimePreparePayload) { p.SizeBytes = 0 }, + "oversized archive": func(p *RuntimePreparePayload) { p.SizeBytes = RuntimePrepareMaxBytes + 1 }, + } { + t.Run(name, func(t *testing.T) { + p := valid + mutate(&p) + if ValidRuntimePrepareRequest(p) { + t.Fatal("invalid begin accepted") + } + }) + } + plugin := valid + plugin.Action, plugin.Skill, plugin.Plugin, plugin.Slot = "plugin", nil, &agentplugin.Metadata{Type: "inline", Name: "example", Description: "Example"}, 49 + if !ValidRuntimePrepareRequest(plugin) { + t.Fatal("valid plugin refused") + } + plugin.Slot = 50 + if ValidRuntimePrepareRequest(plugin) { + t.Fatal("out of range slot accepted") + } + for _, p := range []RuntimePreparePayload{ + {Step: "chunk", Offset: 0, Data: []byte("x")}, + {Step: "chunk", Offset: RuntimePrepareMaxBytes - RuntimePrepareChunkBytes, Data: make([]byte, RuntimePrepareChunkBytes)}, + {Step: "commit"}, + } { + if !ValidRuntimePrepareRequest(p) { + t.Fatal("valid continuation refused", p.Step) + } + } + for _, p := range []RuntimePreparePayload{ + {Step: "chunk", Data: []byte("x"), Action: "skill"}, + {Step: "chunk", Data: []byte("x"), EnvironmentID: valid.EnvironmentID}, + {Step: "chunk", Data: []byte("x"), Skill: valid.Skill}, + {Step: "chunk", Data: []byte("x"), Offset: -1}, + {Step: "chunk", Data: []byte("x"), Offset: RuntimePrepareMaxBytes}, + {Step: "chunk", Data: make([]byte, RuntimePrepareChunkBytes+1)}, + {Step: "chunk"}, + {Step: "commit", Offset: 1}, + {Step: "commit", Data: []byte("x")}, + {Step: "commit", Sources: &agentcapabilities.Input{}}, + } { + if ValidRuntimePrepareRequest(p) { + t.Fatal("invalid continuation accepted", p.Step) + } + } +} + +func TestCapabilitiesFinalizeRequiresExplicitBoundedSelection(t *testing.T) { + p := RuntimePreparePayload{Step: "begin", EnvironmentID: uuid.NewString(), SessionID: uuid.NewString(), Action: "finalize", Sources: &agentcapabilities.Input{}} + if !ValidRuntimePrepareRequest(p) { + t.Fatal("explicit empty finalization refused") + } + for _, mutate := range []func(*RuntimePreparePayload){ + func(p *RuntimePreparePayload) { p.Sources = nil }, + func(p *RuntimePreparePayload) { p.SHA256 = strings.Repeat("a", 64) }, + func(p *RuntimePreparePayload) { p.SizeBytes = 1 }, + func(p *RuntimePreparePayload) { + p.Sources = &agentcapabilities.Input{Directories: make([]string, 51)} + }, + func(p *RuntimePreparePayload) { + p.Sources = &agentcapabilities.Input{Directories: []string{"/workspace/../secret"}} + }, + func(p *RuntimePreparePayload) { + p.Sources = &agentcapabilities.Input{Directories: []string{"/workspace/a", "/workspace/a"}} + }, + } { + bad := p + mutate(&bad) + if ValidRuntimePrepareRequest(bad) { + t.Fatal("invalid finalization accepted") + } + } +} + +func TestCapabilitiesResultCannotMisrepresentUncertainty(t *testing.T) { + for _, r := range []RuntimePrepareResultPayload{ + {Outcome: "rejected", ErrorCode: "runtime_preparation_unconfirmed"}, + {Outcome: "unknown", ErrorCode: "runtime_preparation_rejected"}, + {Outcome: "failed", ErrorCode: "private detail"}, + {Outcome: "rejected", ErrorCode: "invalid_request", Offset: 1}, + {Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed", SizeBytes: 10}, + {Outcome: "completed", SizeBytes: 10}, + {Outcome: "ready", Offset: 1}, + {Outcome: "ready", SizeBytes: 10}, + {Outcome: "ready", ErrorCode: "invalid_request"}, + } { + if ValidRuntimePrepareResult(r, "ready", 0, 10) { + t.Fatal("unsafe receipt accepted", r) + } + } + if ValidRuntimePrepareResult(RuntimePrepareResultPayload{Outcome: "received", Offset: 1}, "received", 2, 10) { + t.Fatal("wrong offset accepted") + } + if ValidRuntimePrepareResult(RuntimePrepareResultPayload{Outcome: "completed", SizeBytes: 9}, "completed", 0, 10) { + t.Fatal("wrong size accepted") + } + for _, r := range []RuntimePrepareResultPayload{ + {Outcome: "rejected", ErrorCode: "invalid_request"}, + {Outcome: "failed", ErrorCode: "runtime_preparation_failed"}, + {Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"}, + {Outcome: "completed", SizeBytes: 10}, + } { + if !ValidRuntimePrepareResult(r, "completed", 0, 10) { + t.Fatal("safe receipt refused", r) + } + } +} + +func TestLocalEnvironmentCarriesSelectionButNeverInstalledRoots(t *testing.T) { + input := &agentcapabilities.Input{Directories: []string{"/workspace/capabilities"}} + original := LocalEnvironment{ID: uuid.NewString(), WorkspaceDirectory: "/workspace", CapabilitySources: input, Capabilities: true, + Skills: []agentcapabilities.InstalledSkill{{RelativeRoot: "private-installed-root"}}, + MCP: []EnvironmentMCP{{PackageRoot: "private-mcp-root"}}} + raw, err := json.Marshal(original) + if err != nil { + t.Fatal(err) + } + if strings.Contains(string(raw), "private-") { + t.Fatal("installed roots leaked to wire") + } + var decoded LocalEnvironment + if err := json.Unmarshal(raw, &decoded); err != nil { + t.Fatal(err) + } + if decoded.ID != original.ID || decoded.WorkspaceDirectory != "/workspace" || !reflect.DeepEqual(decoded.CapabilitySources, input) || !decoded.Capabilities || len(decoded.Skills) != 0 || len(decoded.MCP) != 0 { + t.Fatal("selection round trip changed", decoded) + } +} + +func TestRuntimePreparationInitialActions(t *testing.T) { + base := RuntimePreparePayload{Step: "begin", EnvironmentID: uuid.NewString(), SessionID: uuid.NewString()} + for _, initialization := range []RuntimeInitialization{ + {Action: "configure", Env: map[string]string{"EXAMPLE": "value"}}, + {Action: "system", Network: "enabled", Packages: []string{"git"}}, + {Action: "npm", Network: "disabled", Packages: []string{"typescript"}}, + {Action: "python", Network: "enabled", Packages: []string{"requests"}}, + {Action: "setup", Network: "enabled", Command: "echo done"}, + {Action: "setup", Network: "disabled", Command: "echo done", CWD: "/workspace/project"}, + } { + p := base + p.Action, p.Initialization = "initialize", &initialization + if !ValidRuntimePrepareRequest(p) { + t.Fatalf("valid initialization refused: %s", initialization.Action) + } + for _, mutate := range []func(*RuntimePreparePayload){ + func(p *RuntimePreparePayload) { p.File = &RuntimeInitialFile{Path: "/workspace/a"} }, + func(p *RuntimePreparePayload) { p.SizeBytes = 1 }, + func(p *RuntimePreparePayload) { p.SHA256 = strings.Repeat("a", 64) }, + func(p *RuntimePreparePayload) { p.Data = []byte("x") }, + } { + bad := p + mutate(&bad) + if ValidRuntimePrepareRequest(bad) { + t.Fatal("mixed initialization accepted") + } + } + } + for _, initialization := range []RuntimeInitialization{ + {Action: "exec", Command: "echo done"}, + {Action: "configure", Packages: []string{"git"}}, + {Action: "configure", Env: map[string]string{"A=B": "value"}}, + {Action: "system", Network: "enabled"}, + {Action: "npm", Network: "invalid", Packages: []string{"a"}}, + {Action: "python", Network: "enabled", Packages: []string{"--help"}}, + {Action: "setup", Network: "enabled", Command: "x", CWD: "/environment"}, + {Action: "setup", Network: "enabled", Command: "x", CWD: "/workspace/../private"}, + {Action: "setup", Network: "enabled", Command: "x", Env: map[string]string{}}, + {Action: "setup", Network: "enabled", Command: "x", Packages: []string{}}, + } { + p := base + p.Action, p.Initialization = "initialize", &initialization + if ValidRuntimePrepareRequest(p) { + t.Fatalf("invalid initialization accepted: %+v", initialization) + } + } + p := base + p.Action, p.File, p.SHA256 = "file", &RuntimeInitialFile{Path: "/workspace/empty"}, strings.Repeat("a", 64) + if !ValidRuntimePrepareRequest(p) { + t.Fatal("empty initial file refused") + } + for _, name := range []string{"/workspace", "/workspace/../secret", "/workspace/a/", "/environment/file", `C:\workspace\file`, "/workspace/a\\b"} { + bad := p + bad.File = &RuntimeInitialFile{Path: name} + if ValidRuntimePrepareRequest(bad) { + t.Fatal("invalid logical file accepted", name) + } + } + for _, p := range []RuntimePreparePayload{{Step: "commit", File: &RuntimeInitialFile{Path: "/workspace/a"}}, {Step: "chunk", Data: []byte("a"), Initialization: &RuntimeInitialization{Action: "configure"}}} { + if ValidRuntimePrepareRequest(p) { + t.Fatal("continuation metadata accepted") + } + } +} + +func TestRuntimePreparationExitCodes(t *testing.T) { + for _, code := range []int{0, 1, 255} { + if !ValidRuntimePrepareResult(RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: "runtime_preparation_failed", ExitCode: code}, "ready", 0, 0) { + t.Fatal("valid exit refused", code) + } + } + for _, outcome := range []string{"ready", "received", "completed", "rejected", "unknown", "failed"} { + for _, code := range []int{-1, 256, 1} { + if outcome == "failed" && code == 1 { + continue + } + r := RuntimePrepareResultPayload{Outcome: outcome, ExitCode: code} + switch outcome { + case "rejected": + r.ErrorCode = "invalid_request" + case "unknown": + r.ErrorCode = "runtime_preparation_unconfirmed" + case "failed": + r.ErrorCode = "runtime_preparation_failed" + } + if ValidRuntimePrepareResult(r, outcome, 0, 0) { + t.Fatal("invalid exit combination", r) + } + } + } +} + +func TestRuntimePreparationPortableSources(t *testing.T) { + p := RuntimePreparePayload{Step: "begin", EnvironmentID: uuid.NewString(), SessionID: uuid.NewString(), Action: "finalize", Sources: &agentcapabilities.Input{Directories: []string{`C:\Users\operator\skills`, `\\host\share\skills`}}} + if !ValidRuntimePrepareRequest(p) { + t.Fatal("portable sources rejected by wire") + } + if agentcapabilities.ValidateLocalDirectories(p.Sources.Directories) == nil { + t.Fatal("Windows sources accepted by Linux resolver") + } +} diff --git a/internal/agentdaemon/proto/version.go b/internal/agentdaemon/proto/version.go index a030854b8..8644ad91e 100644 --- a/internal/agentdaemon/proto/version.go +++ b/internal/agentdaemon/proto/version.go @@ -5,7 +5,7 @@ package proto // upgrade query (`version=`) and in the bootstrap HTTP // response; mismatches fail closed at WS upgrade. const ( - Version = "0.5.0" + Version = "0.7.0" ) // VersionCompatible returns true when clientVersion's "X.Y" prefix diff --git a/packages/claude-sdk-adapter/src/adapter.ts b/packages/claude-sdk-adapter/src/adapter.ts index 6be20e8e9..029eaca17 100644 --- a/packages/claude-sdk-adapter/src/adapter.ts +++ b/packages/claude-sdk-adapter/src/adapter.ts @@ -1,4 +1,5 @@ import { NativeFailure } from "./native_failure.js"; +import { ExecutorTurns, type ExecutorEvent } from "./executor_protocol.js"; import { StructuredOutput } from "./structured_output.js"; import { toolSearchEnvironment } from "./tool_search.js"; import { Subagents } from "./subagents.js"; @@ -7,7 +8,7 @@ import { WorkspaceDirectories, type WorkspaceDirectoryEvent } from "./workspace_ import { getSessionInfo, query, startup, type McpServerConfig, type Options, type WarmQuery } from "@anthropic-ai/claude-agent-sdk"; import { WorkspaceReads, type WorkspaceReadEvent } from "./workspace_reads.js"; import { Inputs, type InputEvent } from "./inputs.js"; -import { resultUsage, type NativeUsage } from "./usage.js"; +import { executorResultUsage, resultUsage, type NativeUsage } from "./usage.js"; import { spawnNative } from "./native.js"; import { MessageObserver, type MessageEvent } from "./messages.js"; import { createFunctionServer } from "./functions.js"; @@ -16,11 +17,12 @@ import { MCPProfile } from "./mcp.js"; import { MCPObserver, type MCPEvent } from "./mcp_observer.js"; import { CommandObserver, type CommandEvent } from "./command_observer.js"; import { WorkspaceProfile } from "./workspace.js"; -import { immediateInput, type Prepare, type Start } from "./request.js"; +import { immediateInput, type Prepare, type Start, type ExecutorPrepare } from "./request.js"; import { recoverSession } from "./recovery.js"; export { parseStart, type Start } from "./request.js"; export type Event = + | ExecutorEvent | Fact | WorkspaceDirectoryEvent | WorkspaceReadEvent @@ -35,7 +37,9 @@ export type Event = | { type: "result"; session_id: string; text: string } | { type: "error"; code: "invalid_request" | "history_unavailable" | "execution_failed" | "cancelled"; engine_error_code?: string; session_id?: string; result_id?: string }; -export async function execute(request: Start | Prepare, emit: (event: Event) => Promise, abort: AbortController, functions = new FunctionBridge(emit), inputs = new Inputs(immediateInput(request)), reads = new WorkspaceReads(emit, abort), directories = new WorkspaceDirectories(emit, abort)): Promise { +export async function execute(request: Start | Prepare | ExecutorPrepare, emit: (event: Event) => Promise, abort: AbortController, functions = new FunctionBridge(emit), inputs = new Inputs(immediateInput(request)), reads = new WorkspaceReads(emit, abort), directories = new WorkspaceDirectories(emit, abort), turns?: ExecutorTurns): Promise { + const ownerEmit=emit; + if(turns) emit=event=>turns.emit(event); const nativeFailure = new NativeFailure(); const nativeResultFailure = new Error("unsuccessful native result"); let classifiedFailure: { engine_error_code: string; session_id: string; result_id: string } | undefined; @@ -46,7 +50,7 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => const profile = declarations === undefined ? undefined : new MCPProfile(declarations, names); const subagents = request.subagents ? new Subagents(request.cwd, request.subagents.max_concurrent, request.resume) : undefined; const workspace = request.workspace === undefined ? undefined : new WorkspaceProfile(request.cwd, request.workspace, names, profile, subagents, !!request.output_format); - const commands = workspace ? new CommandObserver() : undefined; + let commands = workspace ? new CommandObserver() : undefined; if (request.type === "prepare" && !workspace) throw new Error("invalid_request"); if (workspace && "mcp_http_servers" in request) throw new Error("invalid_request"); if (request.require_history && !request.resume) { @@ -63,8 +67,8 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => } subagents?.expectSession(request.resume); const mcpServers: Record = Object.create(null); - if (definitions.length) mcpServers.functions = createFunctionServer(definitions, functions.invoke); - const mcp = profile ? new MCPObserver(profile.identities) : undefined; + if (definitions.length) mcpServers.functions = createFunctionServer(definitions, (call,signal) => { const target=functions; return turns ? turns.track(()=>target.invoke(call,signal)) : target.invoke(call,signal); }); + let mcp = profile ? new MCPObserver(profile.identities) : undefined; if (profile) Object.assign(mcpServers, profile.servers); const children: Promise[] = []; let result: Extract | undefined; @@ -72,12 +76,12 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => const resultIDs = new Set(); let failed = false; let cancellationFactsFailed = false; - const structured = request.output_format ? new StructuredOutput() : undefined; - const messages = request.observe_messages ? new MessageObserver() : undefined; + let structured = request.output_format ? new StructuredOutput() : undefined; + let messages = request.observe_messages ? new MessageObserver() : undefined; let stream: ReturnType | undefined; let warm: WarmQuery | undefined; let nativeAlive = false; - const closeInputs = () => inputs.close(); + const closeInputs = () => { inputs.close(); turns?.close(); }; abort.signal.addEventListener("abort", closeInputs, { once: true }); try { if (abort.signal.aborted) throw new Error("cancelled"); @@ -115,20 +119,46 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => return child; }, }; - if (request.type === "prepare") { + if(turns) { + for(const matchers of Object.values(options.hooks ?? {})) for(const matcher of matchers ?? []) { + matcher.hooks=matcher.hooks.map(hook=>(...args)=>turns.track(()=>hook(...args))); + } + const canUseTool=options.canUseTool; + if(canUseTool) options.canUseTool=(...args)=>turns.track(()=>canUseTool(...args)); + } + if (request.type === "prepare" || request.type === "executor_prepare") { warm = await startup({ options, initializeTimeoutMs: 15000 }); - stream = warm.query(inputs); + stream = warm.query(turns ?? inputs); const initialized = await stream.initializationResult(); - if (initialized.hooks_applied !== true || children.length !== 1 || !nativeAlive || abort.signal.aborted) { + const hooksRequested = Object.values(options.hooks ?? {}).some(matchers => matchers?.some(matcher => matcher.hooks.length > 0)); + if ((hooksRequested && initialized.hooks_applied !== true) || children.length !== 1 || !nativeAlive || abort.signal.aborted) { throw new Error("preparation unavailable"); } - reads.bind(stream, request.cwd); - if (process.platform === "linux") await directories.bind(request.cwd); - await emit({ type: "prepared" }); + if(workspace) { + reads.bind(stream, request.cwd); + if (process.platform === "linux") await directories.bind(request.cwd); + } + if (request.mcp_http_servers?.some(server=>server.required)) profile?.verifyRequired(await stream.mcpServerStatus()); + if(turns) { + turns.configure(stream,(input,output)=>{ + inputs=new Inputs(input); + functions=new FunctionBridge(output); + messages=request.observe_messages ? new MessageObserver() : undefined; + structured=request.output_format ? new StructuredOutput() : undefined; + commands=workspace ? new CommandObserver() : undefined; + mcp=profile ? new MCPObserver(profile.identities) : undefined; + subagents?.beginTurn(); + return inputs; + },async value=>{ + if(value.type==="steer") for(const event of inputs.submit(value)) await emit(event); + else functions.submit(JSON.stringify(value)); + }); + await ownerEmit({type:"executor_ready",protocol:3}); + } else await emit({ type: "prepared" }); } else if (profile) { if (inputs.hasInput) throw new Error("MCP input released before initialization"); warm = await startup({ options, initializeTimeoutMs: 15000 }); - stream = warm.query(inputs); + stream = warm.query(turns ?? inputs); const initialized = await stream.initializationResult(); if (initialized.hooks_applied !== true || children.length !== 1) throw new Error("MCP initialization unavailable"); if (request.mcp_http_servers?.some(server => server.required)) profile.verifyRequired(await stream.mcpServerStatus()); @@ -136,10 +166,14 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => inputs.release(request.input); } else stream = query({ prompt: inputs, options }); for await (const message of stream) { + // Native command queue telemetry can follow a result. It is not model + // output or a consumed-input receipt, and has no authority over a Turn. + if (isCommandLifecycle(message)) continue; + if(turns && !turns.id && !(message.type==="system" && message.subtype==="init")) throw new Error("unbound native event"); const errorCode = nativeFailure.observe(message, nativeID, inputs.pendingInputIDs(message)); subagents?.consume(message); structured?.consume(message, nativeID); - await functions.consume(message, nativeID); + await functions.consume(message, nativeID, turns?.cancelled); if (mcp) for (const event of mcp.consume(message, nativeID)) await emit(event); if (commands) for (const event of commands.consume(message, nativeID, inputs.hasInput)) await emit(event); if (messages) for (const event of messages.consume(message)) await emit(event); @@ -153,25 +187,37 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => message.mcp_servers.some(server => server.name !== "functions" || server.status !== "connected")) { throw new Error("unexpected native configuration"); } - for (const event of inputs.start(nativeID)) await emit(event); + turns?.identify(nativeID); + if(!turns || turns.id) for (const event of inputs.start(nativeID)) await emit(event); } else if (!messages && message.type === "stream_event" && message.parent_tool_use_id === null && message.event.type === "content_block_delta" && message.event.delta.type === "text_delta") { await emit({ type: "delta", delta: message.event.delta.text }); } else if (message.type === "result") { if (!message.uuid || resultIDs.has(message.uuid) || !nativeID || message.session_id !== nativeID) throw new Error("invalid native result identity"); resultIDs.add(message.uuid); - await emit({ type: "usage", session_id: nativeID, result_id: message.uuid, usage: resultUsage(message) }); + await emit({ type: "usage", session_id: nativeID, result_id: message.uuid, usage: turns ? executorResultUsage(message) : resultUsage(message) }); for (const event of inputs.consume(message)) await emit(event); - if (message.subtype !== "success" || message.is_error) { + if ((message.subtype !== "success" || message.is_error) && !turns?.cancelled) { if (errorCode) classifiedFailure = { engine_error_code: errorCode, session_id: nativeID, result_id: message.uuid }; throw nativeResultFailure; } - if (structured) await emit(structured.complete(message)); - result = { type: "result", session_id: nativeID, text: message.result }; + if (structured && !turns?.cancelled) await emit(structured.complete(message)); + result = { type: "result", session_id: nativeID, text: message.subtype === "success" ? message.result : "" }; + if(turns?.cancelled && !inputs.complete) throw new Error("unconfirmed cancelled inputs"); + } + if (message.type !== "result" && (!turns || turns.id)) for (const event of inputs.consume(message)) await emit(event); + if(turns && result && inputs.complete && (!subagents || subagents.complete)) { + if (turns.cancelled) functions.cancelUnanswered(); + if(!await turns.quiescent() || abort.signal.aborted) throw new Error("unconfirmed interrupt"); + functions.assertComplete(); mcp?.assertComplete(); commands?.assertComplete(); + if(subagents) for(const event of await subagents.facts()) await emit(event); + await emit(turns.cancelled ? {type:"error",code:"cancelled"} : result); + functions.close(); + await turns.settled(true,true,""); + result=undefined; } - if (message.type !== "result") for (const event of inputs.consume(message)) await emit(event); } - if (subagents) for (const event of await subagents.facts()) await emit(event); + if (subagents && (!turns || turns.id)) for (const event of await subagents.facts()) await emit(event); functions.assertComplete(); mcp?.assertComplete(); commands?.assertComplete(); @@ -181,6 +227,7 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => } finally { profile?.close(); inputs.close(); + turns?.close(); functions.close(); try { await directories.close(); } catch { failed = true; classifiedFailure = undefined; } await reads.close(); @@ -189,16 +236,29 @@ export async function execute(request: Start | Prepare, emit: (event: Event) => abort.signal.removeEventListener("abort", closeInputs); const exits = await Promise.all(children); if (!exits.length) classifiedFailure = undefined; + await turns?.drain(); if (!exits.length || exits.some(code => code !== 0)) failed = true; - if (subagents && abort.signal.aborted && exits.length === 1) { + if (subagents && (!turns || turns.id) && abort.signal.aborted && exits.length === 1) { try { for (const event of await subagents.facts(Date.now())) await emit(event); } catch { cancellationFactsFailed = true; } } if (mcp) for (const event of mcp.close()) await emit(event); if (commands) for (const event of commands.close()) await emit(event); } + if(turns) { + if(turns.id) { + await emit({type:"error",code: turns.cancelled && !cancellationFactsFailed ? "cancelled" : "execution_failed", ...(!turns.cancelled && !cancellationFactsFailed ? classifiedFailure : undefined)}); + await turns.settled(false, false, turns.cancelled ? "cancellation_unconfirmed" : "native_execution_unavailable"); + } else if(failed && !abort.signal.aborted) await ownerEmit({type:"error",code:"execution_failed"}); + return; + } if (cancellationFactsFailed) await emit({ type: "error", code: "execution_failed" }); else if (abort.signal.aborted) await emit({ type: "error", code: "cancelled" }); else if (failed || !result || !inputs.complete) await emit({ type: "error", code: "execution_failed", ...classifiedFailure }); else await emit(result); } + +function isCommandLifecycle(message: unknown): boolean { + return typeof message === "object" && message !== null && + "type" in message && message.type === "command_lifecycle"; +} diff --git a/packages/claude-sdk-adapter/src/executor_protocol.ts b/packages/claude-sdk-adapter/src/executor_protocol.ts new file mode 100644 index 000000000..b8e715adf --- /dev/null +++ b/packages/claude-sdk-adapter/src/executor_protocol.ts @@ -0,0 +1,129 @@ +import type { Query, SDKUserMessage } from "@anthropic-ai/claude-agent-sdk"; +import { Inputs } from "./inputs.js"; +import { parseMessageInput, type MessageInput } from "./message_input.js"; + +export type ExecutorEvent = + | { type: "executor_ready"; protocol: 3 } + | { type: "turn_started"; turn_id: string } + | { type: "turn_settled"; turn_id: string; confirmed: boolean; reusable: boolean; reason: string }; +type WireEvent = { type: string; [key: string]: unknown }; +type Turn = { id: string; inputs: Inputs; cancelled: boolean; interrupt?: Promise; callbacks: Set> }; + +// The SDK sees one iterator for the Executor lifetime. Each yielded batch belongs +// to a fresh receipt ledger; finishing a Turn never closes this outer iterator. +export class ExecutorTurns implements AsyncIterable { + private active?: Turn; + private readonly used = new Set(); + private readonly queue: Inputs[] = []; + private wake?: () => void; + private ended = false; + private stream?: Query; + private begin?: (input: MessageInput, emit: (event: WireEvent) => Promise) => Inputs; + private submitInput?: (value: Record) => Promise; + private nativeID = ""; + constructor(private readonly output: (event: WireEvent) => Promise, private readonly abort: AbortController) {} + + configure(stream: Query, begin: NonNullable, submit: (value: Record)=>Promise): void { this.stream = stream; this.begin = begin; this.submitInput=submit; } + async submit(value: Record): Promise { + const payload=this.assertCurrent(value); + if(!this.submitInput || (payload.type!=="steer" && payload.type!=="function_result")) throw new Error("invalid turn input"); + await this.track(()=>this.submitInput!(payload)); + } + identify(id: string): void { + if (!id || this.nativeID && this.nativeID !== id) throw new Error("invalid native identity"); + this.nativeID = id; + } + get id(): string | undefined { return this.active?.id; } + get cancelled(): boolean { return this.active?.cancelled ?? false; } + get inputs(): Inputs { if (!this.active) throw new Error("no active turn"); return this.active.inputs; } + + async start(value: Record): Promise { + const id = value.turn_id; + if (this.ended || this.active || !this.stream || !this.begin || typeof id !== "string" || !id.trim() || id.length > 256 || + this.used.has(id) || Object.keys(value).some(key => !["type", "turn_id", "input"].includes(key))) throw new Error("invalid turn start"); + const input = parseMessageInput(value.input); + const output = async (event: WireEvent) => { + if (this.active?.id !== id) throw new Error("late turn event"); + await this.output({ ...event, turn_id: id }); + }; + const inputs = this.begin(input, output); + this.active = { id, inputs, cancelled: false, callbacks: new Set() }; + this.used.add(id); + await this.output({ type: "turn_started", turn_id: id }); + if (this.nativeID) for (const event of inputs.start(this.nativeID)) await output(event); + this.queue.push(inputs); + this.wake?.(); + } + + async cancel(value: Record): Promise { + if (typeof value.turn_id !== "string" || Object.keys(value).some(key => !["type", "turn_id"].includes(key))) throw new Error("invalid cancellation"); + const turn = this.active; + // A retired identity has no authority over its successor. + if (!turn || value.turn_id !== turn.id) { + if (this.used.has(value.turn_id)) return; + throw new Error("unknown turn"); + } + if (turn.interrupt) return; + turn.cancelled = true; + const discarded=turn.inputs.cancelQueued(); + turn.interrupt = this.stream!.interrupt().then(receipt => { + const confirmed = !discarded && receipt !== undefined && Array.isArray(receipt.still_queued) && receipt.still_queued.length === 0; + if (!confirmed) this.abort.abort(); + return confirmed; + }, () => { this.abort.abort(); return false; }); + await turn.interrupt; + } + + assertCurrent(value: Record): Record { + if (!this.active || this.active.cancelled || value.turn_id !== this.active.id) throw new Error("inactive turn input"); + const { turn_id: _id, ...payload } = value; + return payload; + } + async emit(event: WireEvent): Promise { + if (!this.active) throw new Error("unbound native event"); + await this.output({ ...event, turn_id: this.active.id }); + } + track(operation: () => Promise): Promise { + const turn = this.active; + if (!turn) return Promise.reject(new Error("unbound native callback")); + const promise = Promise.resolve().then(operation); + turn.callbacks.add(promise); + void promise.then(() => turn.callbacks.delete(promise), () => turn.callbacks.delete(promise)); + return promise; + } + async quiescent(): Promise { + const turn = this.active; + if (!turn) return false; + if (this.abort.signal.aborted) return false; + let stop: () => void = () => {}; + const aborted = new Promise(resolve => { + stop = () => resolve(false); + this.abort.signal.addEventListener("abort", stop, { once: true }); + }); + try { + return await Promise.race([ + this.drain().then(async () => !turn.cancelled || await turn.interrupt === true), + aborted, + ]); + } finally { this.abort.signal.removeEventListener("abort", stop); } + } + async drain(): Promise { + const turn = this.active; + while (turn?.callbacks.size) await Promise.allSettled([...turn.callbacks]); + } + async settled(confirmed: boolean, reusable: boolean, reason: string): Promise { + const turn = this.active; + if (!turn) return; + turn.inputs.close(); + this.active = undefined; + await this.output({ type: "turn_settled", turn_id: turn.id, confirmed, reusable, reason }); + } + close(): void { this.ended = true; this.active?.inputs.close(); this.wake?.(); } + async *[Symbol.asyncIterator](): AsyncIterator { + while (!this.ended) { + const inputs = this.queue.shift(); + if (inputs) yield* inputs; + else await new Promise(resolve => { this.wake = resolve; }); + } + } +} diff --git a/packages/claude-sdk-adapter/src/function_bridge.ts b/packages/claude-sdk-adapter/src/function_bridge.ts index d849288c6..777d9ea15 100644 --- a/packages/claude-sdk-adapter/src/function_bridge.ts +++ b/packages/claude-sdk-adapter/src/function_bridge.ts @@ -19,6 +19,7 @@ type Pending = { reject: (error: Error) => void; cleanup: () => void; result?: FunctionResult; + aborted?: boolean; }; export class FunctionBridge { @@ -32,7 +33,9 @@ export class FunctionBridge { this.seen.add(call.id); const waiting = new Promise((resolve, reject) => { const stop = () => { - this.pending.delete(call.id); + const pending = this.pending.get(call.id); + if (pending?.result) pending.aborted = true; + else this.pending.delete(call.id); reject(new Error("Native function call aborted.")); }; signal.addEventListener("abort", stop, { once: true }); @@ -69,15 +72,19 @@ export class FunctionBridge { }), isError: !result.success }); } - async consume(message: SDKMessage, sessionID: string): Promise { + async consume(message: SDKMessage, sessionID: string, cancelled = false): Promise { if (message.type !== "user" || message.parent_tool_use_id !== null || message.session_id !== sessionID || message.isSynthetic || ("isReplay" in message && message.isReplay) || !Array.isArray(message.message.content)) return; for (const block of message.message.content) { if (block.type !== "tool_result") continue; const pending = this.pending.get(block.tool_use_id); - if (!pending) continue; + if (!pending || pending.aborted) continue; const result = pending.result; - if (!result) throw new Error("Native response preceded host result."); + if (!result) { + if (!cancelled || !block.is_error) throw new Error("Native response preceded host result."); + this.cancelUnanswered(block.tool_use_id); + continue; + } // Native error results join MCP text blocks; retain the original parts in the host. const matches = result.success ? matchesNativeContent(result.content, block.content) : block.content === result.content.map(part => part.type === "input_text" ? part.text : "").join("\n"); @@ -90,6 +97,17 @@ export class FunctionBridge { } } + // Only a native cancellation result authorizes abandoning unanswered calls. + // Submitted results retain their receipt obligation, including after abort. + cancelUnanswered(callID?: string): void { + for (const [id, pending] of this.pending) { + if (pending.result || (callID !== undefined && id !== callID)) continue; + pending.cleanup(); + this.pending.delete(id); + pending.reject(new Error("Native function call cancelled.")); + } + } + assertComplete(): void { if (this.pending.size) throw new Error("Native function results are unconfirmed."); } diff --git a/packages/claude-sdk-adapter/src/inputs.ts b/packages/claude-sdk-adapter/src/inputs.ts index 1e77ffcd1..f2effbffe 100644 --- a/packages/claude-sdk-adapter/src/inputs.ts +++ b/packages/claude-sdk-adapter/src/inputs.ts @@ -103,6 +103,11 @@ export class Inputs implements AsyncIterable { get hasInput(): boolean { return this.submitted.size > 0; } + cancelQueued(): boolean { + const discarded = this.queue.length > 0; + this.queue.length = 0; this.close(); return discarded; + } + close(): void { this.ended = true; this.wake?.(); } async *[Symbol.asyncIterator](): AsyncIterator { diff --git a/packages/claude-sdk-adapter/src/main.ts b/packages/claude-sdk-adapter/src/main.ts index 496c9ab0a..86b12aa09 100644 --- a/packages/claude-sdk-adapter/src/main.ts +++ b/packages/claude-sdk-adapter/src/main.ts @@ -1,3 +1,4 @@ +import { ExecutorTurns } from "./executor_protocol.js"; import { WorkspaceDirectories } from "./workspace_directories.js"; import { WorkspaceReads } from "./workspace_reads.js"; import { Inputs } from "./inputs.js"; @@ -12,7 +13,7 @@ const stop = () => abort.abort(); process.once("SIGTERM", stop); process.once("SIGINT", stop); lines.once("close", stop); -const emit = (event: Event): Promise => new Promise((resolve, reject) => { +const emit = (event: Event | {type:string;[key:string]:unknown}): Promise => new Promise((resolve, reject) => { process.stdout.write(JSON.stringify(event) + "\n", error => error ? reject(error) : resolve()); }); try { @@ -30,6 +31,7 @@ try { const reads = new WorkspaceReads(output, abort); const directories = new WorkspaceDirectories(output, abort); const prompts = new Inputs(immediateInput(request)); + const turns=request.type==="executor_prepare" ? new ExecutorTurns(emit,abort) : undefined; const incoming = (async () => { try { for await (const line of { [Symbol.asyncIterator]: () => input }) { @@ -39,6 +41,12 @@ try { reads.submit(value as Record); } else if (value && typeof value === "object" && "type" in value && value.type === "workspace_directory") { directories.submit(value as Record); + } else if(turns) { + if(!value || typeof value!=="object" || Array.isArray(value)) throw new Error("invalid_request"); + const control=value as Record; + if(control.type==="turn_start") await turns.start(control); + else if(control.type==="turn_cancel") await turns.cancel(control); + else await turns.submit(control); } else if (request.type === "prepare" && phase !== "running") { if (phase !== "prepared" || abort.signal.aborted) throw new Error("invalid_request"); prompts.release(preparedInput(value)); @@ -50,7 +58,7 @@ try { } catch { invalid = request.type === "prepare"; abort.abort(); } })(); - try { await execute(request, output, abort, functions, prompts, reads, directories); } + try { await execute(request, output, abort, functions, prompts, reads, directories, turns); } catch { await output({ type: "error", code: "execution_failed" }); } finally { prompts.close(); diff --git a/packages/claude-sdk-adapter/src/mcp_environment.ts b/packages/claude-sdk-adapter/src/mcp_environment.ts index 65f70d56e..b60623d3d 100644 --- a/packages/claude-sdk-adapter/src/mcp_environment.ts +++ b/packages/claude-sdk-adapter/src/mcp_environment.ts @@ -1,3 +1,4 @@ +import { posix } from "node:path"; import { parseHTTPServers, type HTTPServer } from "./mcp.js"; export type StdioServer = { @@ -20,12 +21,16 @@ export function parseEnvironmentMCP(value: unknown): EnvironmentMCPServer[] | un if (Object.keys(server).some(key => !["server_label", "command", "args", "allowed_tools"].includes(key)) || typeof server.server_label !== "string" || !/^[a-zA-Z0-9_-]+$/.test(server.server_label) || server.server_label === "functions" || server.allowed_tools !== null || server.command !== "/usr/bin/python3" || - !Array.isArray(server.args) || server.args.length !== 6 || + !Array.isArray(server.args) || server.args.length !== 8 || server.args[0] !== "-I" || server.args[1] !== "-S" || server.args[2] !== "/usr/local/bin/oac-runtime-initialize" || server.args[3] !== "stdio" || - typeof server.args[4] !== "string" || !server.args[4] || server.args[4].startsWith("/") || - server.args[4].split("/").some((part: string) => !part || part === "." || part === "..") || - /[\x00-\x1f\x7f\\]/.test(server.args[4]) || server.args[5] !== server.server_label) throw new Error("invalid_request"); + typeof server.args[4] !== "string" || !posix.isAbsolute(server.args[4]) || server.args[4].endsWith("/") || posix.normalize(server.args[4]) !== server.args[4] || + /[\x00-\x1f\x7f\\]/.test(server.args[4]) || + typeof server.args[5] !== "string" || !posix.isAbsolute(server.args[5]) || server.args[5].endsWith("/") || posix.normalize(server.args[5]) !== server.args[5] || + /[\x00-\x1f\x7f\\]/.test(server.args[5]) || + typeof server.args[6] !== "string" || !server.args[6] || server.args[6].startsWith("/") || + server.args[6].split("/").some((part: string) => !part || part === "." || part === "..") || + /[\x00-\x1f\x7f\\]/.test(server.args[6]) || server.args[7] !== server.server_label) throw new Error("invalid_request"); } else { parseHTTPServers([server]); if (server.allowed_tools !== null || server.required !== undefined) throw new Error("invalid_request"); diff --git a/packages/claude-sdk-adapter/src/request.ts b/packages/claude-sdk-adapter/src/request.ts index dbf353008..ca9636ca0 100644 --- a/packages/claude-sdk-adapter/src/request.ts +++ b/packages/claude-sdk-adapter/src/request.ts @@ -20,20 +20,21 @@ export type Start = { mcp_http_servers?: HTTPServer[]; workspace?: Workspace; }; +export type ExecutorPrepare = Omit & { type: "executor_prepare" }; export type Prepare = Omit & { type: "prepare"; workspace: Workspace }; // MCP startup confirms its hooks before the native input iterator yields. -export function immediateInput(request: Start | Prepare): MessageInput | undefined { +export function immediateInput(request: Start | Prepare | ExecutorPrepare): MessageInput | undefined { return request.type === "start" && request.mcp_http_servers === undefined && !request.workspace?.mcp?.length ? request.input : undefined; } -export function parseRequest(line: string): Start | Prepare { +export function parseRequest(line: string): Start | Prepare | ExecutorPrepare { const value: unknown = JSON.parse(line); if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("invalid_request"); const request = value as Record; const allowed = new Set(["type", "input", "model", "system_prompt", "cwd", "resume", "require_history", "observe_messages", "output_format", "subagents", "functions", "tool_search", "mcp_http_servers", "workspace"]); if (Object.keys(request).some(key => !allowed.has(key)) || - (request.type !== "start" && request.type !== "prepare") || + (request.type !== "start" && request.type !== "prepare" && request.type !== "executor_prepare") || (request.type === "start" ? !Array.isArray(request.input) : "input" in request) || typeof request.model !== "string" || !request.model.trim() || typeof request.system_prompt !== "string" || @@ -68,7 +69,7 @@ export function parseRequest(line: string): Start | Prepare { if (request.require_history && !workspace) throw new Error("invalid_request"); if ((workspace && "mcp_http_servers" in request) || (request.type === "prepare" && !workspace)) throw new Error("invalid_request"); - return request as Start | Prepare; + return request as Start | Prepare | ExecutorPrepare; } export function parseStart(line: string): Start { diff --git a/packages/claude-sdk-adapter/src/runtime_check.ts b/packages/claude-sdk-adapter/src/runtime_check.ts index e221c67fd..dc6caec94 100644 --- a/packages/claude-sdk-adapter/src/runtime_check.ts +++ b/packages/claude-sdk-adapter/src/runtime_check.ts @@ -45,7 +45,7 @@ try { assert.equal(smoke.error, undefined, "bridge_unavailable"); assert.equal(smoke.status, 0, "bridge_unavailable"); assert.deepEqual(JSON.parse(smoke.stdout), { type: "error", code: "invalid_request" }); - process.stdout.write(JSON.stringify({ type: "runtime_ready", protocol: 2, features: [...(process.platform === "linux" ? ["workspace_directory", "local_runtime_v1", "workspace_functions", "workspace_structured_output"] : []), "message_images", "function_result_images", "tool_search", "structured_output", "subagent_resources", "mcp_http_tools", "mcp_http_bearer_auth", "mcp_http_required", "workspace_tools", "workspace_prepare", "workspace_read", "workspace_command_observations"], node: process.versions.node, sdk: sdk.version, mcp: mcp.version, native: nativeVersion }) + "\n"); + process.stdout.write(JSON.stringify({ type: "runtime_ready", protocol: 3, features: ["executor_reuse", ...(process.platform === "linux" ? ["workspace_directory", "local_runtime_v1", "workspace_functions", "workspace_structured_output"] : []), "message_images", "function_result_images", "tool_search", "structured_output", "subagent_resources", "mcp_http_tools", "mcp_http_bearer_auth", "mcp_http_required", "workspace_tools", "workspace_prepare", "workspace_read", "workspace_command_observations"], node: process.versions.node, sdk: sdk.version, mcp: mcp.version, native: nativeVersion }) + "\n"); } catch { // Native diagnostics can include operator environment; never forward them. process.stdout.write(JSON.stringify({ type: "runtime_unavailable" }) + "\n"); diff --git a/packages/claude-sdk-adapter/src/subagents.ts b/packages/claude-sdk-adapter/src/subagents.ts index 0441d0f1c..add5e689d 100644 --- a/packages/claude-sdk-adapter/src/subagents.ts +++ b/packages/claude-sdk-adapter/src/subagents.ts @@ -14,6 +14,11 @@ export class Subagents { constructor(private readonly cwd: string, private readonly limit: number, private resume: string | undefined) { if (!Number.isSafeInteger(limit) || limit < 1) throw new Error("invalid_request"); } + get complete(): boolean { return this.pending.size === 0 && this.running.size === 0; } + beginTurn(): void { + if (!this.complete) throw new Error("native child work is unsettled"); + this.observedCalls.clear(); this.admitted.clear(); + } expectSession(id: string | undefined): void { this.resume = id; } permitsActor(id?: string): boolean { return id === undefined || this.known.has(id); } isCoordination(name: string): boolean { return ["Agent", "Task", "SendMessage"].includes(name); } diff --git a/packages/claude-sdk-adapter/src/usage.ts b/packages/claude-sdk-adapter/src/usage.ts index e7ad54077..03baf1b57 100644 --- a/packages/claude-sdk-adapter/src/usage.ts +++ b/packages/claude-sdk-adapter/src/usage.ts @@ -7,3 +7,14 @@ export function resultUsage(message: SDKResultMessage): NativeUsage { return structuredClone({ usage: message.usage, modelUsage: message.modelUsage, total_cost_usd: message.total_cost_usd, subtype: message.subtype, is_error: message.is_error }); } + +// Streaming-input results mix two scopes. Keep that distinction alongside the +// untouched values so a Turn consumer cannot sum query totals across Turns. +export function executorResultUsage(message: SDKResultMessage): NativeUsage & { + scopes: { usage: "native_turn_main_loop"; modelUsage: "query_cumulative"; total_cost_usd: "query_cumulative_estimate" }; +} { + return { ...resultUsage(message), scopes: { + usage: "native_turn_main_loop", modelUsage: "query_cumulative", + total_cost_usd: "query_cumulative_estimate", + } }; +} diff --git a/packages/claude-sdk-adapter/src/workspace.ts b/packages/claude-sdk-adapter/src/workspace.ts index 361ed5ec8..7b525d9d2 100644 --- a/packages/claude-sdk-adapter/src/workspace.ts +++ b/packages/claude-sdk-adapter/src/workspace.ts @@ -15,6 +15,7 @@ export type Workspace = { dependency_path: string; env_names: string[]; skills?: WorkspaceSkill[]; + capability_root?: string; mcp?: EnvironmentMCPServer[]; tool_environment?: boolean; system_packages?: boolean; @@ -49,7 +50,7 @@ export function parseWorkspace(value: unknown, cwd: string): Workspace | undefin if (value === undefined) return undefined; if (!value || typeof value !== "object" || Array.isArray(value)) throw new Error("invalid_request"); const config = value as Record; - if (Object.keys(config).some(key => !["home", "state", "scratch", "protected_dirs", "dependency_path", "env_names", "network_access", "allowed_domains", "tool_environment", "system_packages", "skills", "mcp"].includes(key)) || + if (Object.keys(config).some(key => !["home", "state", "scratch", "protected_dirs", "dependency_path", "env_names", "network_access", "allowed_domains", "tool_environment", "system_packages", "skills", "mcp", "capability_root"].includes(key)) || (config.tool_environment !== undefined && typeof config.tool_environment !== "boolean") || (config.system_packages !== undefined && typeof config.system_packages !== "boolean") || (config.system_packages === true && config.tool_environment !== true) || @@ -59,6 +60,12 @@ export function parseWorkspace(value: unknown, cwd: string): Workspace | undefin config.env_names.some(name => typeof name !== "string" || !environmentNames.has(name)) || new Set(config.env_names).size !== config.env_names.length) throw new Error("invalid_request"); const mcp = parseEnvironmentMCP(config.mcp); + if (config.capability_root !== undefined) { + const capabilityRoot = directory(config.capability_root, true); + if ([cwd, config.home, config.state, config.scratch].some(root => + typeof root === "string" && (contains(root, capabilityRoot) || contains(capabilityRoot, root)))) throw new Error("invalid_request"); + } + if ((Array.isArray(config.skills) && config.skills.length || mcp?.length) && !config.capability_root) throw new Error("invalid_request"); if (mcp?.length && config.network_access !== "enabled") throw new Error("invalid_request"); const domains = config.allowed_domains ?? []; const hostname = /^(?=.{1,253}$)[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?(?:\.[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?)*$/i; @@ -105,7 +112,7 @@ export class WorkspaceProfile { env.CLAUDE_CODE_SHELL_PREFIX = "/usr/local/bin/oac-claude-shell-prefix"; env.OAC_RUNTIME_TOOL_SCRATCH = config.scratch; } - const skills = workspaceSkills(config.skills ?? []); + const skills = workspaceSkills(config.skills ?? [], config.capability_root ?? ""); this.skillNames = skills?.names ?? []; const skillTools = skills ? ["Skill"] : []; const protectedRoots = [config.home, config.state, ...config.protected_dirs]; @@ -126,7 +133,7 @@ export class WorkspaceProfile { enabled: true, failIfUnavailable: true, autoAllowBashIfSandboxed: false, allowUnsandboxedCommands: false, excludedCommands: [], enableWeakerNestedSandbox: false, enableWeakerNetworkIsolation: false, filesystem: { disabled: false, allowWrite: [cwd, config.scratch, ...(config.tool_environment ? ["/environment/packages"] : [])], denyRead: protectedRoots, - denyWrite: [...protectedRoots, ...(skills || config.mcp?.length ? ["/environment/initialization/capabilities"] : []), + denyWrite: [...protectedRoots, ...(config.capability_root ? [config.capability_root] : []), ...(config.system_packages ? ["/environment/packages/system"] : [])], allowRead: [] }, credentials: { envVars: [...new Set([...credentialNames, ...config.env_names, ...(mcp?.credentialReferences() ?? [])])].map(name => ({ name, mode: "deny" })), diff --git a/packages/claude-sdk-adapter/src/workspace_skills.ts b/packages/claude-sdk-adapter/src/workspace_skills.ts index 3d03c5a48..e15f69909 100644 --- a/packages/claude-sdk-adapter/src/workspace_skills.ts +++ b/packages/claude-sdk-adapter/src/workspace_skills.ts @@ -6,7 +6,6 @@ export type WorkspaceSkill = { relative_root: string; package_root: string; }; -export const capabilityRoot = "/environment/initialization/capabilities"; const relativePath = (value: unknown): value is string => typeof value === "string" && value !== "." && value.length > 0 && !value.startsWith("/") && !/[\\\x00-\x1f\x7f]/.test(value) && posix.normalize(value) === value && !value.split("/").includes(".."); @@ -53,7 +52,7 @@ function projectPackage(source: string, destination: string): void { // Only the generated envelope is activated. Original plugin control files stay // beneath content; exact Skill roots, not public plugin manifests, drive loading. -export function workspaceSkills(skills: readonly WorkspaceSkill[]): { paths: string[]; names: string[] } | undefined { +export function workspaceSkills(skills: readonly WorkspaceSkill[], capabilityRoot: string): { paths: string[]; names: string[] } | undefined { if (!skills.length) return undefined; const packages = new Map(); for (const skill of skills) { diff --git a/packages/claude-sdk-adapter/tests/executor.test.mjs b/packages/claude-sdk-adapter/tests/executor.test.mjs new file mode 100644 index 000000000..d30b7505d --- /dev/null +++ b/packages/claude-sdk-adapter/tests/executor.test.mjs @@ -0,0 +1,195 @@ +import assert from "node:assert/strict"; +import { spawn } from "node:child_process"; +import test from "node:test"; + +const fixture = ` +import assert from "node:assert/strict"; +import { registerHooks } from "node:module"; +import { Client } from "@modelcontextprotocol/sdk/client/index.js"; +import { InMemoryTransport } from "@modelcontextprotocol/sdk/inMemory.js"; +const sdk = "export async function startup(args){return globalThis.startupFixture(args)} export function query(){throw new Error('one warm query required')} export async function getSessionInfo(){return {sessionId:'native'}}"; +registerHooks({resolve(specifier,context,next){if(specifier==='@anthropic-ai/claude-agent-sdk') return {url:'data:text/javascript,'+encodeURIComponent(sdk),shortCircuit:true};return next(specifier,context)}}); +globalThis.startupFixture=async({options})=>{ + const child=options.spawnClaudeCodeProcess({command:process.execPath,args:['-e',"process.stdin.resume();process.stdin.on('end',()=>process.exit(0))"],env:options.env,signal:options.abortController.signal}); + process.send({type:'native',pid:child.pid}); + const exited=new Promise(resolve=>child.once('close',()=>{process.send({type:'native_closed'});resolve()})); + let client; + if(options.mcpServers.functions){ + client=new Client({name:'fixture',version:'1'}); + const [a,b]=InMemoryTransport.createLinkedPair(); + await Promise.all([client.connect(a),options.mcpServers.functions.instance.connect(b)]); + } + let queried=false,interrupt,number=0; + const close=()=>{child.stdin.end();interrupt?.()}; + options.abortController.signal.addEventListener('abort',close); + return {close,query(prompt){assert.equal(queried,false);queried=true;process.send({type:'query'});return { + close,async initializationResult(){return process.argv[1]==="no-hook-report" ? {} : process.argv[1]==="false-hook-report" ? {hooks_applied:false} : {hooks_applied:true}}, + async interrupt(){process.send({type:'interrupt'});interrupt?.();if(process.argv[1]==='rejected')throw new Error('interrupt failed');return ['unknown','pending-function-unknown'].includes(process.argv[1]) ? undefined : {still_queued:process.argv[1]==='queued' ? ['not-consumed'] : []}}, + async *[Symbol.asyncIterator](){ + for await(const user of prompt){ + number++; + const text=user.message.content[0].text; + process.send({type:'input',text}); + yield {type:'system',subtype:'init',session_id:'native',tools:client ? ['mcp__functions__lookup'] : [],mcp_servers:client ? [{name:'functions',status:'connected'}] : []}; + if(client){ + if(text==='pending-function'){ + void client.callTool({name:'lookup',arguments:{text},_meta:{'claudecode/toolUseId':'pending-call'}}).catch(()=>{}); + await Promise.race([new Promise(resolve=>{interrupt=resolve}),exited]); + if(process.argv[1]==='pending-function-result') yield {type:'user',session_id:'native',parent_tool_use_id:null,message:{content:[{type:'tool_result',tool_use_id:'pending-call',content:'Interrupted',is_error:true}]}}; + } + if(text==='features'){ + const called=await client.callTool({name:'lookup',arguments:{text},_meta:{'claudecode/toolUseId':'same-native-call'}}); + yield {type:'user',session_id:'native',parent_tool_use_id:null,message:{content:[{type:'tool_result',tool_use_id:'same-native-call',content:called.content,is_error:called.isError}]}}; + } + for(const event of [ + {type:'message_start',message:{id:'same-native-message'}}, + {type:'content_block_start',index:0,content_block:{type:'text',text}}, + {type:'content_block_stop',index:0},{type:'message_stop'} + ])yield {type:'stream_event',parent_tool_use_id:null,session_id:'native',event}; + } + if(process.argv[1]==='classified'){ + yield {type:'assistant',uuid:'assistant-error',session_id:'native',user_message_uuids:[user.uuid],parent_tool_use_id:null,error:'authentication_failed',message:{content:[]}}; + yield {type:'result',uuid:'error-result',session_id:'native',user_message_uuids:[user.uuid],subtype:'error_during_execution',is_error:true,usage:{input_tokens:1,output_tokens:1},modelUsage:{},total_cost_usd:0}; + continue; + } + if(text==='hold') await Promise.race([new Promise(resolve=>{interrupt=resolve}),exited]); + if(options.abortController.signal.aborted)return; + yield {type:'result',uuid:'result-'+number,session_id:'native',user_message_uuids:[user.uuid],subtype:'success',is_error:false,result:'answer-'+number,usage:{input_tokens:1,output_tokens:1},modelUsage:{fixture:{inputTokens:number,outputTokens:number,costUSD:number/100}},total_cost_usd:number/100}; + interrupt=undefined; + yield {type:'command_lifecycle',uuid:user.uuid,state:'completed'}; + } + } + }}}; +}; +await import(${JSON.stringify(new URL("../dist/main.js", import.meta.url).href)}); +process.disconnect(); + +`; + +async function launch(t,mode="normal") { + const child=spawn(process.execPath,["--input-type=module","-e",fixture,mode],{stdio:["pipe","pipe","pipe","ipc"]}); + const events=[],observations=[];let buffer="",stderr=""; + child.stdout.setEncoding("utf8");child.stderr.setEncoding("utf8"); + child.stdout.on("data",data=>{buffer+=data;while(buffer.includes("\n")){const end=buffer.indexOf("\n");events.push(JSON.parse(buffer.slice(0,end)));buffer=buffer.slice(end+1)}}); + child.stderr.on("data",data=>{stderr+=data});child.on("message",event=>observations.push(event)); + const closed=new Promise(resolve=>child.once("close",(code,signal)=>resolve({code,signal}))); + t.after(async()=>{child.kill("SIGKILL");await closed}); + const wait=async predicate=>{const until=Date.now()+5000;while(!predicate()){assert.ok(Date.now()setTimeout(resolve,5))}}; + const send=value=>child.stdin.write(JSON.stringify(value)+"\n"); + const start=(id,text)=>send({type:"turn_start",turn_id:id,input:[{content:[{type:"input_text",text}]}]}); + send({type:"executor_prepare",cwd:"/tmp",model:"fixture",system_prompt:"", + ...(mode==="features" || mode.startsWith("pending-function") ? {observe_messages:true,functions:[{name:"lookup",description:"lookup",parameters:{type:"object",properties:{text:{type:"string"}}}}]} : {})}); + await wait(()=>events.some(event=>event.type==="executor_ready")); + assert.equal(observations.filter(event=>event.type==="input").length,0); + return {child,events,observations,closed,wait,send,start}; +} + +test("executor retains one native process and one Query over two settled Turns",{timeout:10000},async t=>{ + const {child,events,observations,closed,wait,start}=await launch(t); + for(const id of ["first","second"]){start(id,"answer");await wait(()=>events.some(event=>event.type==="turn_settled"&&event.turn_id===id));assert.equal(events.find(event=>event.type==="turn_settled"&&event.turn_id===id).confirmed,true);assert.equal(events.find(event=>event.type==="turn_settled"&&event.turn_id===id).reusable,true)} + assert.equal(observations.filter(event=>event.type==="native").length,1); + assert.equal(observations.filter(event=>event.type==="query").length,1); + for(const event of events.filter(event=>event.type!=="executor_ready")) assert.ok(["first","second"].includes(event.turn_id)); + assert.equal(events.filter(event=>event.type==="input_ready").length,2); + const usage=events.filter(event=>event.type==="usage"); + assert.deepEqual(usage.map(event=>[event.turn_id,event.usage.usage.input_tokens]),[["first",1],["second",1]]); + assert.deepEqual(usage.map(event=>event.usage.modelUsage.fixture.inputTokens),[1,2]); + assert.deepEqual(usage.map(event=>event.usage.total_cost_usd),[0.01,0.02]); + for(const event of usage) assert.deepEqual(event.usage.scopes,{usage:"native_turn_main_loop",modelUsage:"query_cumulative",total_cost_usd:"query_cumulative_estimate"}); + child.stdin.end();assert.deepEqual(await closed,{code:0,signal:null}); + assert.equal(observations.filter(event=>event.type==="native_closed").length,1); +}); + +test("public interrupt settles cancellation and stale cancellation cannot stop the next Turn",{timeout:10000},async t=>{ + const {child,events,observations,closed,wait,send,start}=await launch(t); + start("first","hold");await wait(()=>events.some(event=>event.type==="input_ready")); + send({type:"turn_cancel",turn_id:"first"}); + await wait(()=>events.some(event=>event.type==="turn_settled")); + assert.equal(events.find(event=>event.type==="turn_settled").confirmed,true); + assert.equal(events.find(event=>event.type==="turn_settled").reusable,true); + assert.ok(events.some(event=>event.type==="error"&&event.code==="cancelled"&&event.turn_id==="first")); + start("second","hold");await wait(()=>events.some(event=>event.type==="input_ready"&&event.turn_id==="second")); + send({type:"turn_cancel",turn_id:"first"}); + await new Promise(resolve=>setTimeout(resolve,20)); + assert.equal(observations.filter(event=>event.type==="interrupt").length,1); + send({type:"turn_cancel",turn_id:"second"}); + await wait(()=>events.some(event=>event.type==="turn_settled"&&event.turn_id==="second")); + child.stdin.end();assert.deepEqual(await closed,{code:0,signal:null}); +}); + +for(const mode of ["queued","unknown","rejected"])test(`interrupt ${mode} receipt retires native ownership before non-reusable settlement`,{timeout:10000},async t=>{ + const {events,observations,closed,wait,send,start}=await launch(t,mode); + start("first","hold");await wait(()=>events.some(event=>event.type==="input_ready")); + send({type:"turn_cancel",turn_id:"first"}); + await wait(()=>events.some(event=>event.type==="turn_settled")); + const settlement=events.find(event=>event.type==="turn_settled"); + assert.equal(settlement.confirmed,false); + assert.equal(settlement.reusable,false); + assert.equal(settlement.reason,"cancellation_unconfirmed"); + assert.equal(observations.filter(event=>event.type==="native_closed").length,1); + assert.deepEqual(await closed,{code:0,signal:null}); +}); + +test("reused Turns retain functions, native receipts, steering and message boundaries", {timeout:10000}, async t => { + const {child,events,closed,wait,send,start}=await launch(t,"features"); + for(const id of ["first","second"]){ + start(id,"features"); + await wait(()=>events.some(event=>event.type==="function_call"&&event.turn_id===id)); + send({type:"steer",turn_id:id,input_id:"same-input-id",input:[{content:[{type:"input_text",text:"extra"}]}]}); + send({type:"function_result",turn_id:id,call_id:"same-native-call",delivery_id:"same-delivery-id",success:true,content:[{type:"input_text",text:"function result"}]}); + await wait(()=>events.some(event=>event.type==="turn_settled"&&event.turn_id===id)); + const own=events.filter(event=>event.turn_id===id); + assert.equal(own.find(event=>event.type==="turn_settled").confirmed,true); + assert.equal(own.find(event=>event.type==="turn_settled").reusable,true); + assert.equal(own.filter(event=>event.type==="function_applied").length,1); + assert.equal(own.find(event=>event.type==="input_applied").input_id,"same-input-id"); + assert.deepEqual(own.filter(event=>event.type==="output_message"&&event.message.status==="completed").map(event=>event.message.text),["features","extra"]); + assert.equal(own.filter(event=>event.type==="usage").length,2); + } + child.stdin.end();assert.deepEqual(await closed,{code:0,signal:null}); +}); + +for(const mode of ["no-hook-report","false-hook-report"])test("none Executor without requested hooks accepts "+mode,{timeout:10000},async t=>{ + const {child,events,closed,wait,start}=await launch(t,mode); + start("first","answer"); + await wait(()=>events.some(event=>event.type==="turn_settled")); + assert.equal(events.find(event=>event.type==="turn_settled").confirmed,true); + assert.equal(events.find(event=>event.type==="turn_settled").reusable,true); + child.stdin.end();assert.deepEqual(await closed,{code:0,signal:null}); +}); + + +test("executor retains a classified native result without confirming failed settlement",{timeout:10000},async t=>{ + const {events,wait,start}=await launch(t,"classified"); + start("failed-turn","answer"); + await wait(()=>events.some(event=>event.type==="turn_settled")); + const failure=events.find(event=>event.type==="error"); + assert.equal(failure.turn_id,"failed-turn"); + assert.equal(failure.engine_error_code,"authentication_error"); + assert.equal(failure.session_id,"native"); + assert.equal(failure.result_id,"error-result"); + const settled=events.find(event=>event.type==="turn_settled"); + assert.equal(settled.confirmed,false); + assert.equal(settled.reusable,false); +}); + +for (const mode of ["pending-function-result", "pending-function-terminal", "pending-function-unknown"]) test(`unanswered function cancellation ${mode} retains native confirmation requirements`, {timeout:10000}, async t => { + const {child,events,closed,wait,send,start}=await launch(t,mode); + start("first","pending-function"); + await wait(()=>events.some(event=>event.type==="function_call")); + send({type:"turn_cancel",turn_id:"first"}); + await wait(()=>events.some(event=>event.type==="turn_settled")); + const settled=events.find(event=>event.type==="turn_settled"); + const confirmed=mode!=="pending-function-unknown"; + assert.equal(settled.confirmed,confirmed); + assert.equal(settled.reusable,confirmed); + assert.ok(events.some(event=>event.type==="error"&&event.code==="cancelled")); + assert.equal(events.filter(event=>event.type==="function_applied").length,0); + if(confirmed){ + start("second","answer"); + await wait(()=>events.some(event=>event.type==="turn_settled"&&event.turn_id==="second")); + assert.equal(events.find(event=>event.type==="turn_settled"&&event.turn_id==="second").confirmed,true); + child.stdin.end(); + } + assert.deepEqual(await closed,{code:0,signal:null}); +}); diff --git a/packages/claude-sdk-adapter/tests/function_bridge.test.mjs b/packages/claude-sdk-adapter/tests/function_bridge.test.mjs index fac8458d9..bd78b0e27 100644 --- a/packages/claude-sdk-adapter/tests/function_bridge.test.mjs +++ b/packages/claude-sdk-adapter/tests/function_bridge.test.mjs @@ -62,7 +62,10 @@ test("an aborted submitted result never becomes a native application receipt", { bridge.submit(JSON.stringify(result("a", false))); await waiting; abort.abort(); - await bridge.consume(native(result("a", false)), "native"); + bridge.cancelUnanswered(); + assert.throws(() => bridge.assertComplete(), /unconfirmed/); + await bridge.consume(native(result("a", false)), "native", true); + assert.throws(() => bridge.assertComplete(), /unconfirmed/); assert.deepEqual(events.map(event => event.type), ["function_call"]); }); @@ -112,3 +115,28 @@ test("native error-image rejection leaves the pending call available for a suppo await bridge.consume(native(result("error", false)), "native"); bridge.assertComplete(); }); + +for (const terminal of ["tool_result", "result"]) test(`native cancellation ${terminal} settles unanswered callbacks without application`, async () => { + const events = []; + const bridge = new FunctionBridge(async event => { events.push(event); }); + const waiting = bridge.invoke(call("a"), new AbortController().signal); + const rejected = assert.rejects(waiting, /cancelled/); + if (terminal === "tool_result") { + const cancellation = native(result("a", false), "Interrupted"); + await assert.rejects(bridge.consume(cancellation, "native"), /preceded/); + await bridge.consume(cancellation, "native", true); + } else bridge.cancelUnanswered(); + await rejected; + bridge.assertComplete(); + assert.deepEqual(events.map(event => event.type), ["function_call"]); +}); + +test("native cancellation preserves a submitted result's missing receipt", async () => { + const bridge = new FunctionBridge(async () => {}); + const waiting = bridge.invoke(call("a"), new AbortController().signal); + bridge.submit(JSON.stringify(result("a"))); + await waiting; + bridge.cancelUnanswered(); + assert.throws(() => bridge.assertComplete(), /unconfirmed/); + bridge.close(); +}); diff --git a/packages/claude-sdk-adapter/tests/mcp_workspace.test.mjs b/packages/claude-sdk-adapter/tests/mcp_workspace.test.mjs index 38a06fdb8..d037130c4 100644 --- a/packages/claude-sdk-adapter/tests/mcp_workspace.test.mjs +++ b/packages/claude-sdk-adapter/tests/mcp_workspace.test.mjs @@ -9,18 +9,18 @@ import { immediateInput, parseStart } from "../dist/request.js"; import { WorkspaceProfile } from "../dist/workspace.js"; const stdio = { server_label: "installed", command: "/usr/bin/python3", allowed_tools: null, - args: ["-I", "-S", "/usr/local/bin/oac-runtime-initialize", "stdio", "plugins/installed", "installed"] }; + args: ["-I", "-S", "/usr/local/bin/oac-runtime-initialize", "stdio", "/private/runtime/capabilities", "/private/runtime/workspace", "plugins/installed", "installed"] }; const native = "mcp__installed__echo_v1"; const statuses = [{ name: "installed", status: "connected", tools: [{ name: "echo.v1" }] }]; const baseline = ["Bash", "Read", "Edit"]; function fixture(t, declarations = [stdio]) { const root = realpathSync(mkdtempSync(join(tmpdir(), "oac-mcp-workspace-"))); - const dirs = Object.fromEntries(["work", "home", "state", "scratch", "secrets", "deps"].map(name => { + const dirs = Object.fromEntries(["work", "home", "state", "scratch", "secrets", "deps", "capabilities"].map(name => { const path = join(root, name); mkdirSync(path); return [name, path]; })); const config = { home: dirs.home, state: dirs.state, scratch: dirs.scratch, protected_dirs: [dirs.secrets], - dependency_path: dirs.deps, env_names: ["ANTHROPIC_AUTH_TOKEN"], network_access: "enabled", mcp: declarations }; + capability_root: dirs.capabilities, dependency_path: dirs.deps, env_names: ["ANTHROPIC_AUTH_TOKEN"], network_access: "enabled", mcp: declarations }; const previous = process.env; process.env = { HOME: dirs.home, CLAUDE_CONFIG_DIR: dirs.state, ANTHROPIC_AUTH_TOKEN: "model-secret" }; t.after(() => { process.env = previous; rmSync(root, { recursive: true, force: true }); }); @@ -37,9 +37,15 @@ test("installed MCP private projection cannot launch arbitrary unsandboxed comma assert.deepEqual(parseEnvironmentMCP([stdio]), [stdio]); for (const value of [[stdio, stdio], [{ ...stdio, command: "/bin/sh" }], [{ ...stdio, env: { TOKEN: "secret" } }], [{ ...stdio, args: ["-c", "untrusted"] }], [{ ...stdio, allowed_tools: ["*"] }], - [{ ...stdio, server_url: "https://example.invalid" }], [{ ...stdio, args: [...stdio.args.slice(0, 4), "../escape", "installed"] }]]) { + [{ ...stdio, server_url: "https://example.invalid" }], [{ ...stdio, args: [...stdio.args.slice(0, 6), "../escape", "installed"] }]]) { assert.throws(() => parseEnvironmentMCP(value), /invalid_request/); } + for (const index of [4, 5]) { + for (const invalid of ["/", "relative", "/tmp/../escape", "/tmp/root/", "/tmp/line\n"]) { + const args = [...stdio.args]; args[index] = invalid; + assert.throws(() => parseEnvironmentMCP([{ ...stdio, args }]), /invalid_request/); + } + } assert.throws(() => parseStart(JSON.stringify({ ...request, workspace: { ...request.workspace, network_access: "disabled" } })), /invalid_request/); }); @@ -66,7 +72,7 @@ test("combined inventory admits exact MCP identities without granting local file } assert.deepEqual(workspace.options.allowedTools, ["mcp__installed__*"]); assert.deepEqual(workspace.options.tools, baseline); - assert.ok(workspace.options.sandbox.filesystem.denyWrite.includes("/environment/initialization/capabilities")); + assert.ok(workspace.options.sandbox.filesystem.denyWrite.includes(dirs.capabilities)); assert.equal(workspace.options.sandbox.allowUnsandboxedCommands, false); mcp.close(); assert.equal((await workspace.canUseTool(native, {}, { signal })).behavior, "deny"); diff --git a/packages/claude-sdk-adapter/tests/usage.test.mjs b/packages/claude-sdk-adapter/tests/usage.test.mjs index 39e65a059..e1f4365cc 100644 --- a/packages/claude-sdk-adapter/tests/usage.test.mjs +++ b/packages/claude-sdk-adapter/tests/usage.test.mjs @@ -1,6 +1,6 @@ import assert from "node:assert/strict"; import test from "node:test"; -import { resultUsage } from "../dist/usage.js"; +import { executorResultUsage, resultUsage } from "../dist/usage.js"; test("preserve separate SDK scopes and price provenance without recalculating", () => { const first = { type: "result", subtype: "success", is_error: false, @@ -25,3 +25,17 @@ test("reported error usage survives and missing fields are not manufactured", () const missing = JSON.parse(JSON.stringify(resultUsage({ subtype: "error_during_execution", is_error: true }))); assert.deepEqual(missing, { subtype: "error_during_execution", is_error: true }); }); + +test("executor preserves cumulative counter resets without inventing per-Turn accounting", () => { + const values = [8, 15, 0, 2].map(total => executorResultUsage({ + subtype: "success", is_error: false, usage: { input_tokens: 2 }, + modelUsage: { primary: { inputTokens: total, costUSD: total / 100 } }, + total_cost_usd: total / 100, + })); + assert.deepEqual(values.map(value => value.modelUsage.primary.inputTokens), [8, 15, 0, 2]); + assert.deepEqual(values.map(value => value.usage.input_tokens), [2, 2, 2, 2]); + for (const value of values) { + assert.equal(value.scopes.modelUsage, "query_cumulative"); + assert.equal(value.scopes.usage, "native_turn_main_loop"); + } +}); diff --git a/packages/claude-sdk-adapter/tests/workspace_skills.test.mjs b/packages/claude-sdk-adapter/tests/workspace_skills.test.mjs index 1b8296aa9..4f32cd444 100644 --- a/packages/claude-sdk-adapter/tests/workspace_skills.test.mjs +++ b/packages/claude-sdk-adapter/tests/workspace_skills.test.mjs @@ -15,3 +15,22 @@ test('Runtime Skill paths remain inside the declared package; public metadata is ]) assert.throws(() => parseSkills([invalid]), /invalid_request/); assert.throws(() => parseSkills([skill, skill]), /invalid_request/); }); + +test('native Skill envelope stays under the resolved Runtime root', async t => { + const { mkdtempSync, mkdirSync, writeFileSync, rmSync, statSync } = await import('node:fs'); + const { tmpdir } = await import('node:os'); + const { join } = await import('node:path'); + const { workspaceSkills } = await import('../dist/workspace_skills.js'); + const root = mkdtempSync(join(tmpdir(), 'oac-capability-root-')); + t.after(() => rmSync(root, { recursive: true, force: true })); + const source = join(root, 'plugins/0/custom/proof'); + mkdirSync(source, { recursive: true }); + writeFileSync(join(source, 'SKILL.md'), '---\nname: proof\ndescription: A proof.\n---\nUse this Skill.'); + const skill = { metadata: { type: 'inline', name: 'proof', description: 'A proof.' }, + relative_root: 'plugins/0/custom/proof', package_root: 'plugins/0' }; + const result = workspaceSkills([skill], root); + assert.deepEqual(result.paths, [join(root, 'native/claude/environment-skills-0')]); + const projected = join(result.paths[0], 'content/custom/proof/SKILL.md'); + assert.equal(statSync(projected).ino, statSync(join(source, 'SKILL.md')).ino); + assert.deepEqual(workspaceSkills([skill], root), result); +}); diff --git a/packages/mcode-harness/launch.mjs b/packages/mcode-harness/launch.mjs index ae3389f0f..81fb52b44 100644 --- a/packages/mcode-harness/launch.mjs +++ b/packages/mcode-harness/launch.mjs @@ -2,7 +2,7 @@ import { SandboxManager } from './dist/sandbox.mjs'; import { spawn } from 'node:child_process'; import { readFileSync, mkdirSync } from 'node:fs'; import { isIP } from 'node:net'; -import { dirname, join } from 'node:path'; +import { dirname, join, isAbsolute, normalize } from 'node:path'; import { fileURLToPath } from 'node:url'; const here = dirname(fileURLToPath(import.meta.url)); const profile = JSON.parse(readFileSync(process.argv[2], 'utf8')); @@ -19,6 +19,8 @@ if (profile.systemPackages) { if (!profile.toolEnvironment) throw new Error('System packages require initialized tool configuration'); baseEnv.OAC_RUNTIME_TOOL_SCRATCH=profile.scratch; } +if (profile.capabilityRoot && (typeof profile.capabilityRoot !== 'string' || !isAbsolute(profile.capabilityRoot) || normalize(profile.capabilityRoot) !== profile.capabilityRoot || profile.capabilityRoot === '/' || /[\\\x00-\x1f\x7f]/.test(profile.capabilityRoot))) throw new Error('Invalid capability root'); +if (profile.skills && !profile.capabilityRoot) throw new Error('Missing capability root'); let child; let cancelled=false; const cancel=()=>{cancelled=true;child?.kill('SIGKILL');}; @@ -27,7 +29,7 @@ try { mkdirSync(profile.scratch,{recursive:true}); await SandboxManager.initialize({ network:{allowedDomains:domains,deniedDomains:profile.network==='disabled'?['*']:[],allowAll:profile.network==='enabled',strictAllowlist:true,allowAllUnixSockets:false,allowLocalBinding:false}, - filesystem:{denyRead:profile.protectedDirs,allowWrite:[profile.workspace,profile.scratch,...(profile.toolEnvironment ? ['/environment/packages'] : [])],denyWrite:[...(profile.skills ? ["/environment/initialization/capabilities"] : []),...(profile.systemPackages ? ['/environment/packages/system'] : [])]}, + filesystem:{denyRead:profile.protectedDirs,allowRead:profile.capabilityRoot?[profile.capabilityRoot]:[],allowWrite:[profile.workspace,profile.scratch,...(profile.toolEnvironment ? ['/environment/packages'] : [])],denyWrite:[...(profile.capabilityRoot ? [profile.capabilityRoot] : []),...(profile.systemPackages ? ['/environment/packages/system'] : [])]}, seccomp:{applyPath:join(here,'dist/vendor/seccomp/x64/apply-seccomp')}, },undefined,false); const quote=s=>"'"+s.replaceAll("'","'\\''")+"'"; diff --git a/packages/mcode-harness/snapshot.test.mjs b/packages/mcode-harness/snapshot.test.mjs new file mode 100644 index 000000000..e5463f13d --- /dev/null +++ b/packages/mcode-harness/snapshot.test.mjs @@ -0,0 +1,74 @@ +import assert from 'node:assert/strict'; +import test from 'node:test'; +import { copyFile, mkdir, mkdtemp, readFile, rm, stat, symlink, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { tmpdir } from 'node:os'; +import { spawnSync } from 'node:child_process'; + +// This opt-in test requires an isolated Linux Runtime containing the built +// sandbox/worker artifact. It runs only native tools; no model is contacted. +const artifact = process.env.OAC_TEST_MCODE_NATIVE_ARTIFACT; +const isolated = process.env.OAC_TEST_MCODE_SNAPSHOT_ISOLATED === '1'; +test('capability read exceptions preserve denied ancestors and immutable snapshots', { + skip: !artifact || !isolated, timeout: 60_000, +}, async t => { + const fixture = await mkdtemp(join(tmpdir(), 'oac-mcode-snapshot-')); + t.after(() => rm(fixture, { recursive: true, force: true })); + await copyFile(new URL('./launch.mjs', import.meta.url), join(fixture, 'launch.mjs')); + await symlink(join(artifact, 'dist'), join(fixture, 'dist')); + const quote = value => "'" + value.replaceAll("'", "'\\''") + "'"; + for (const [name, ancestor, root] of [ + ['standard', '/environment', '/environment/initialization/capabilities'], + ['custom', join(fixture, 'private-layout'), join(fixture, 'private-layout', 'frozen', 'skills')], + ]) { + await t.test(name, async t => { + const scratch = join(fixture, name + '-scratch'); + const secret = join(ancestor, 'private', 'snapshot-test-secret'); + await mkdir(root, { recursive: true }); + await mkdir(join(ancestor, 'private'), { recursive: true }); + await mkdir(scratch); + const script = join(root, 'proof.sh'), attachment = join(root, 'attachment.txt'); + await writeFile(script, 'printf SNAPSHOT_SCRIPT_READABLE', { mode: 0o700 }); + await writeFile(attachment, 'SNAPSHOT_ATTACHMENT_READABLE', { mode: 0o600 }); + await writeFile(secret, 'PRIVATE_SNAPSHOT_CANARY', { mode: 0o600 }); + const profile = join(fixture, name + '.json'); + const config = { workspace: '/workspace', scratch, network: 'disabled', + allowedDomains: [], protectedDirs: [ancestor], capabilityRoot: root, skills: true }; + await writeFile(profile, JSON.stringify(config)); + const executor = { execute(tool, input) { + const child = spawnSync(process.execPath, [join(fixture, 'launch.mjs'), profile, '/workspace'], { + input: JSON.stringify({ tool, input }), encoding: 'utf8', timeout: 15_000, + env: { PATH: '/usr/local/bin:/usr/bin:/bin', HOME: '/tmp', LANG: 'C.UTF-8' }, + }); + assert.ok(child.stdout, child.stderr || String(child.error)); + return JSON.parse(child.stdout); + } }; + const read = await executor.execute('bash', { + command: 'sh ' + quote(script) + '; cat ' + quote(attachment), + }); + assert.notEqual(read.isError, true, read.text); + assert.ok(read.text.includes('SNAPSHOT_SCRIPT_READABLE')); + assert.ok(read.text.includes('SNAPSHOT_ATTACHMENT_READABLE')); + const protectedResult = await executor.execute('bash', { + command: [ + 'if printf changed >> ' + quote(attachment) + '; then exit 41; fi', + 'if chmod 0777 ' + quote(script) + '; then exit 42; fi', + 'if touch ' + quote(join(root, 'new-sibling')) + '; then exit 43; fi', + 'if cat ' + quote(secret) + '; then exit 44; fi', + 'printf SNAPSHOT_PROTECTION_OK', + ].join('; '), + }); + assert.notEqual(protectedResult.isError, true); + assert.ok(protectedResult.text.includes('SNAPSHOT_PROTECTION_OK')); + assert.ok(!protectedResult.text.includes('PRIVATE_SNAPSHOT_CANARY')); + assert.equal(await readFile(attachment, 'utf8'), 'SNAPSHOT_ATTACHMENT_READABLE'); + assert.equal((await stat(script)).mode & 0o777, 0o700); + await assert.rejects(stat(join(root, 'new-sibling')), { code: 'ENOENT' }); + // Negative control proves that the ancestor deny really masks this root. + await writeFile(profile, JSON.stringify({ ...config, capabilityRoot: undefined, skills: false })); + const hidden = await executor.execute('bash', { command: 'cat ' + quote(attachment) }); + assert.equal(hidden.isError, true); + assert.ok(!hidden.text.includes('SNAPSHOT_ATTACHMENT_READABLE')); + }); + } +}); diff --git a/scripts/check-claude-sdk-runtime.mjs b/scripts/check-claude-sdk-runtime.mjs index 156f79983..960bb36de 100644 --- a/scripts/check-claude-sdk-runtime.mjs +++ b/scripts/check-claude-sdk-runtime.mjs @@ -22,8 +22,8 @@ const probe = spawnSync(process.execPath, [join(root, "dist/runtime_check.js"), assert.equal(probe.status, 0, "Exported runtime is unavailable"); const report = JSON.parse(probe.stdout); assert.equal(report.type, "runtime_ready"); -assert.equal(report.protocol, 2); -assert.deepEqual(report.features, [...(process.platform === "linux" ? ["workspace_directory", "local_runtime_v1", "workspace_functions", "workspace_structured_output"] : []), "message_images", "function_result_images", "tool_search", "structured_output", "subagent_resources", "mcp_http_tools", "mcp_http_bearer_auth", "mcp_http_required", "workspace_tools", "workspace_prepare", "workspace_read", "workspace_command_observations"]); +assert.equal(report.protocol, 3); +assert.deepEqual(report.features, ["executor_reuse", ...(process.platform === "linux" ? ["workspace_directory", "local_runtime_v1", "workspace_functions", "workspace_structured_output"] : []), "message_images", "function_result_images", "tool_search", "structured_output", "subagent_resources", "mcp_http_tools", "mcp_http_bearer_auth", "mcp_http_required", "workspace_tools", "workspace_prepare", "workspace_read", "workspace_command_observations"]); assert.equal(report.sdk, source.dependencies["@anthropic-ai/claude-agent-sdk"]); assert.equal(report.mcp, source.dependencies["@modelcontextprotocol/sdk"]); console.log(`Verified exported SDK ${report.sdk}, MCP ${report.mcp}, ${report.native}`); diff --git a/scripts/name-allowlist.json b/scripts/name-allowlist.json index a8c53ae8f..37c3f3d0d 100644 --- a/scripts/name-allowlist.json +++ b/scripts/name-allowlist.json @@ -823,5 +823,35 @@ "path": "apps/docs/lib/legacy-redirects.ts", "regex": "\"agents-api(?:-sessions|-errors)?\"", "reason": "Retired documentation URL keys redirect to the current public API guide; these are not current commands or API namespaces." + }, + { + "path": "apps/parsar-daemon/internal/cli/connect_cleanup_test.go", + "regex": "\"agents-api-cleanup\"", + "reason": "Executor ownership validates the current Core and local-workspace AgentStateKey namespace; this change does not introduce an alternate identity or compatibility path." + }, + { + "path": "apps/parsar-daemon/internal/dispatch/executor.go", + "regex": "\"agents-api-(?:session)?\"", + "reason": "Executor ownership validates the current Core and local-workspace AgentStateKey namespace; this change does not introduce an alternate identity or compatibility path." + }, + { + "path": "apps/parsar-daemon/internal/dispatch/executor_handoff_test.go", + "regex": "\"agents-api-(?:session)?\"", + "reason": "Executor ownership validates the current Core and local-workspace AgentStateKey namespace; this change does not introduce an alternate identity or compatibility path." + }, + { + "path": "apps/parsar-daemon/internal/dispatch/executor_test.go", + "regex": "\"agents-api-(?:session)?\"", + "reason": "Executor ownership validates the current Core and local-workspace AgentStateKey namespace; this change does not introduce an alternate identity or compatibility path." + }, + { + "path": "apps/parsar-daemon/internal/localworkspace/capabilities.go", + "regex": "\"agents-api-\"", + "reason": "AgentStateKey retains the existing daemon/native-session resume identity; capability snapshots bind to the Session UUID carried in that identity." + }, + { + "path": "apps/parsar-daemon/internal/localworkspace/capability_preparation_test.go", + "regex": "\"agents-api-\"", + "reason": "AgentStateKey retains the existing daemon/native-session resume identity; capability snapshots bind to the Session UUID carried in that identity." } ] diff --git a/services/agents-api/deploy/runtime/initialize.py b/services/agents-api/deploy/runtime/initialize.py index c3e3990f7..901501885 100644 --- a/services/agents-api/deploy/runtime/initialize.py +++ b/services/agents-api/deploy/runtime/initialize.py @@ -3,9 +3,7 @@ Core owns sequencing and the completion ledger. This helper executes one bounded operation; it never retries, schedules, selects a harness or interprets templates. """ -import base64 import ctypes -import hashlib import json import os from pathlib import Path @@ -21,8 +19,6 @@ CONFIG = ROOT / 'initialization' PACKAGES = ROOT / 'packages' ENV_FILE = CONFIG / 'tool-env.sh' -CAPABILITIES = CONFIG / 'capabilities' -SKILLS = CAPABILITIES / 'skills' MAX_INPUT = 32 * 1024 * 1024 BASE_ENV = {'PATH': '/usr/local/bin:/usr/bin:/bin', 'HOME': '/tmp', 'LANG': 'C.UTF-8'} DIRECTORY = os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC @@ -33,78 +29,20 @@ def open_directory(parent, name): return os.open(name, DIRECTORY, dir_fd=parent) -def roots(): +def roots(workspace=None): root = os.open('/', DIRECTORY) try: environment = open_directory(root, 'environment') finally: os.close(root) try: - for name in ('workspace', 'packages', 'initialization'): + for name in (('workspace', 'packages', 'initialization') if workspace is None else ('packages', 'initialization')): child = open_directory(environment, name) os.close(child) finally: os.close(environment) - - -def install_skill(request): - name, files = request['name'], request['files'] - if not isinstance(name, str) or not re.fullmatch('[a-z0-9]+(?:[-_][a-z0-9]+)*', name) or len(name) > 64 or not isinstance(files, list) or not 0 < len(files) <= 1000: - raise ValueError('invalid skill') - directory = os.open(SKILLS, DIRECTORY) - try: - os.mkdir(name, mode=0o700, dir_fd=directory) - skill = open_directory(directory, name) - finally: - os.close(directory) - total = 0 - try: - for file in files: - relative = file['path'] - if not isinstance(relative, str) or len(relative) > 4096 or any(c in relative for c in ('\\', '\x00', '\r', '\n')): - raise ValueError('invalid skill path') - parts = relative.split('/') - if any(not part or part in ('.', '..') for part in parts): - raise ValueError('invalid skill path') - body = base64.b64decode(file['data'], validate=True) - total += len(body) - if total > 20 * 1024 * 1024: - raise ValueError('skill too large') - parent = os.dup(skill) - try: - for part in parts[:-1]: - try: - os.mkdir(part, mode=0o700, dir_fd=parent) - except FileExistsError: - pass - child = open_directory(parent, part) - os.close(parent) - parent = child - # A fresh installation cannot replace an existing member. - try: - os.stat(parts[-1], dir_fd=parent, follow_symlinks=False) - except FileNotFoundError: - pass - else: - raise ValueError('duplicate skill member') - result = subprocess.run(['/usr/local/bin/oac-codex-write', str(SKILLS / name), relative, - str(len(body)), str(ROOT / 'staging')], - input=body + hashlib.sha256(body).digest(), env=BASE_ENV, - capture_output=True, check=True) - receipt = json.loads(result.stdout) - if result.stderr or receipt != {'version': 1, 'outcome': 'completed', 'size_bytes': len(body)}: - raise ValueError('skill write unconfirmed') - fd = os.open(parts[-1], os.O_RDONLY | os.O_NOFOLLOW, dir_fd=parent) - try: - os.fchmod(fd, 0o500 if file.get('executable', False) else 0o400) - os.fsync(fd) - finally: - os.close(fd) - finally: - os.close(parent) - os.fsync(skill) - finally: - os.close(skill) + if workspace is not None: + os.close(os.open(workspace, DIRECTORY)) def configure(env): @@ -123,16 +61,6 @@ def configure(env): } materialized = json.dumps(values, ensure_ascii=True) script = ''.join('export ' + name + '=' + shlex.quote(value) + '\n' for name, value in sorted(values.items())) - environment = os.open(CONFIG, DIRECTORY) - try: - os.mkdir('capabilities', mode=0o700, dir_fd=environment) - capabilities = open_directory(environment, 'capabilities') - try: - os.mkdir('skills', mode=0o700, dir_fd=capabilities) - finally: - os.close(capabilities) - finally: - os.close(environment) directory = os.open(CONFIG, DIRECTORY) try: fd = os.open('tool-env.sh', os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o400, dir_fd=directory) @@ -150,14 +78,14 @@ def configure(env): os.close(directory) -def sandbox(network, cwd): +def sandbox(network, cwd, workspace=None): if network not in ('enabled', 'disabled') or not isinstance(cwd, str) or not cwd.startswith('/') or '\x00' in cwd: raise ValueError('invalid execution configuration') if (CONFIG / 'system-root.json').exists(): tools = runpy.run_path('/usr/local/bin/oac-tool-root') if not tools['installed'](): raise ValueError('system tools unavailable') - return tools['initialization_sandbox'](cwd, network) + return tools['initialization_sandbox'](cwd, network, workspace=workspace or str(ROOT / 'workspace')) # One packaging contract for every Provider/harness. No native state, daemon # credential, staging payload or parent process is visible in this mount map. args = ['/usr/bin/bwrap', '--unshare-user', '--unshare-pid', '--unshare-ipc', '--unshare-uts', @@ -168,8 +96,10 @@ def sandbox(network, cwd): '--ro-bind', '/etc/resolv.conf', '/etc/resolv.conf', '--ro-bind', '/etc/hosts', '/etc/hosts', '--ro-bind', '/etc/ssl', '/etc/ssl', '--proc', '/proc', '--dev', '/dev', '--tmpfs', '/tmp', '--dir', '/home', - '--dir', '/environment', '--bind', str(ROOT / 'workspace'), '/workspace', + '--dir', '/environment', '--bind', workspace or str(ROOT / 'workspace'), '/workspace', '--bind', str(PACKAGES), str(PACKAGES), '--ro-bind', str(CONFIG), str(CONFIG)] + if workspace is not None and workspace != '/workspace': + args += ['--bind', workspace, workspace] if network == 'disabled': args += ['--unshare-net'] for key, value in BASE_ENV.items(): @@ -177,18 +107,16 @@ def sandbox(network, cwd): return args + ['--chdir', cwd, '--'] -def run(request): +def run(request, workspace=None, capability_root=None): action = request['action'] if action == 'configure': - configure(request['env']) - return - if action == 'skill': - install_skill(request) + configure(request.get('env', {})) return if action == 'system': - runpy.run_path('/usr/local/bin/oac-tool-root')['install'](request['packages']) + runpy.run_path('/usr/local/bin/oac-tool-root')['install'](request['packages'], workspace=workspace or str(ROOT / 'workspace')) return - args = sandbox(request['network'], request.get('cwd', '/workspace')) + args = sandbox(request['network'], request.get('cwd', '/workspace'), workspace=workspace) + expose_capabilities(args, capability_root) if action == 'setup': command = request['command'] if not isinstance(command, str) or not command or '\x00' in command: @@ -283,24 +211,52 @@ def bind_parent(): os.close(parent_fd) -def stdio(package, server): +def expose_capabilities(args, capability_root): + """Expose only the Runtime-owned capability tree, including custom layouts.""" + if capability_root is None: + return + path = Path(capability_root) + if (not capability_root.startswith('/') or capability_root == '/' + or os.path.normpath(capability_root) != capability_root + or any(ord(c) < 32 or ord(c) == 127 or c == '\\' for c in capability_root) + or path.resolve() != path): + raise ValueError('invalid Runtime capability directory') + # Configuration may precede the first bundle import or empty finalization. + if path.exists(): + if not path.is_dir(): + raise ValueError('invalid Runtime capability directory') + args[-1:-1] = ['--ro-bind', capability_root, capability_root] + + +def stdio(installation_root, workspace, package, server): """Preserve the native MCP descriptors while entering the existing sandbox.""" - roots() - args = sandbox('enabled', '/workspace') + for value in (installation_root, workspace): + if (not isinstance(value, str) or not value.startswith('/') or value == '/' + or os.path.normpath(value) != value or any(ord(c) < 32 or ord(c) == 127 or c == '\\' for c in value)): + raise ValueError('invalid Runtime directory') + path = Path(value) + if path.resolve(strict=True) != path or not path.is_dir(): + raise ValueError('invalid Runtime directory') + roots(workspace) + args = sandbox('enabled', '/workspace', workspace=workspace) helper = '/tmp/oac-mcp-exec' # System-package roots predate daemon installation. Mount only the fixed # static helper, never native configuration, credentials or Runtime state. + expose_capabilities(args, installation_root) args[-1:-1] = ['--ro-bind', '/usr/local/bin/oac-daemon', helper] - args += [helper, 'runtime-mcp-exec', package, server] + args += [helper, 'runtime-mcp-exec', installation_root, package, server] return stdio_lifetime(args) def main(): - if len(sys.argv) != 1: + workspace, capability_root = None, None + if len(sys.argv) == 4 and sys.argv[1] == 'initialize': + workspace, capability_root = sys.argv[2:] + elif len(sys.argv) != 1: try: - if len(sys.argv) != 4 or sys.argv[1] != 'stdio': + if len(sys.argv) != 6 or sys.argv[1] != 'stdio': raise ValueError('invalid stdio invocation') - return stdio(sys.argv[2], sys.argv[3]) + return stdio(sys.argv[2], sys.argv[3], sys.argv[4], sys.argv[5]) except Exception: print('Environment MCP unavailable', file=sys.stderr) return 1 @@ -311,8 +267,18 @@ def main(): request = json.loads(raw) if not isinstance(request, dict) or request.get('version') != 1: raise ValueError('invalid version') - roots() - run(request) + if workspace is not None: + path = Path(workspace) + if (not workspace.startswith('/') or workspace == '/' or os.path.normpath(workspace) != workspace + or any(ord(c) < 32 or ord(c) == 127 or c == '\\' for c in workspace) + or path.resolve(strict=True) != path or not path.is_dir()): + raise ValueError('invalid Runtime workspace') + if workspace is None: + roots() + run(request) + else: + roots(workspace) + run(request, workspace=workspace, capability_root=capability_root) except Exception as error: print(failed_receipt(error)) return 1 diff --git a/services/agents-api/deploy/runtime/initialize_receipt_test.py b/services/agents-api/deploy/runtime/initialize_receipt_test.py index 8b92f2ab0..2a6b51f3d 100644 --- a/services/agents-api/deploy/runtime/initialize_receipt_test.py +++ b/services/agents-api/deploy/runtime/initialize_receipt_test.py @@ -10,6 +10,7 @@ import json from pathlib import Path import subprocess +import tempfile import unittest from unittest import mock @@ -42,6 +43,119 @@ def assert_receipt(self, failure, expected): self.assertNotIn(CANARY, stdout) self.assertNotIn('echo', stdout) + def test_configure_does_not_create_or_change_runtime_capabilities(self): + with tempfile.TemporaryDirectory() as directory: + config = Path(directory) + capabilities = config / 'capabilities' + capabilities.mkdir() + manifest = capabilities / 'installed.json' + manifest.write_bytes(b'fixed-runtime-fixture') + with mock.patch.object(initialize, 'CONFIG', config): + initialize.configure({'VALUE': 'configured'}) + self.assertEqual(manifest.read_bytes(), b'fixed-runtime-fixture') + self.assertFalse((capabilities / 'skills').exists()) + self.assertIn('configured', (config / 'tool-env.sh').read_text()) + with self.assertRaises(FileExistsError): + initialize.configure({'VALUE': 'changed'}) + + def test_skill_is_not_a_provider_initializer_action(self): + with mock.patch.object(initialize, 'sandbox', return_value=[]), \ + mock.patch.object(initialize.subprocess, 'run') as run: + with self.assertRaisesRegex(ValueError, 'invalid action'): + initialize.run({'action': 'skill', 'network': 'enabled'}) + run.assert_not_called() + + def test_stdio_forwards_and_mounts_resolved_runtime_directories(self): + with tempfile.TemporaryDirectory() as directory: + base = Path(directory).resolve() + root, workspace = base / 'capabilities', base / 'actual-work' + root.mkdir() + workspace.mkdir() + root, workspace = str(root), str(workspace) + with mock.patch.object(initialize, 'roots') as roots, \ + mock.patch.object(initialize, 'sandbox', return_value=['bwrap', '--']) as sandbox, \ + mock.patch.object(initialize, 'stdio_lifetime', return_value=0) as launch: + with mock.patch.object(initialize.sys, 'argv', ['initialize', 'stdio', root, workspace, 'plugins/0', 'server']): + self.assertEqual(initialize.main(), 0) + roots.assert_called_once_with(workspace) + sandbox.assert_called_once_with('enabled', '/workspace', workspace=workspace) + argv = launch.call_args.args[0] + self.assertEqual(argv[-5:], ['/tmp/oac-mcp-exec', 'runtime-mcp-exec', root, 'plugins/0', 'server']) + self.assertIn(['--ro-bind', root, root], [argv[i:i+3] for i in range(len(argv)-2)]) + alias = base / 'alias' + alias.symlink_to(workspace, target_is_directory=True) + for invalid in ('/', 'relative', '/tmp/../secret', workspace + '/', workspace + '\n', str(alias)): + for values in ((invalid, workspace), (root, invalid)): + with self.assertRaises(ValueError): + initialize.stdio(*values, 'plugins/0', 'server') + + def test_stdio_mounts_workspace_at_native_alias_and_physical_path(self): + with tempfile.TemporaryDirectory() as directory: + config = Path(directory) + workspace = str(config / 'actual-work') + with mock.patch.object(initialize, 'CONFIG', config): + args = initialize.sandbox('enabled', '/workspace', workspace=workspace) + mounts = [args[i:i+3] for i in range(len(args)-2)] + self.assertIn(['--bind', workspace, '/workspace'], mounts) + self.assertIn(['--bind', workspace, workspace], mounts) + self.assertNotIn(['--bind', '/environment/workspace', '/workspace'], mounts) + self.assertIn(['--ro-bind', str(config), str(config)], mounts) + default = initialize.sandbox('enabled', '/workspace') + self.assertIn(['--bind', '/environment/workspace', '/workspace'], + [default[i:i+3] for i in range(len(default)-2)]) + tools = initialize.runpy.run_path(str(Path(__file__).with_name('tool-root.py'))) + (config / 'system-root.json').write_text('{"version":1}') + tools['installed'] = lambda: True + with mock.patch.object(initialize.runpy, 'run_path', return_value=tools): + args = initialize.sandbox('enabled', '/workspace', workspace=workspace) + mounts = [args[i:i+3] for i in range(len(args)-2)] + self.assertIn(['--bind', workspace, '/workspace'], mounts) + self.assertIn(['--bind', workspace, workspace], mounts) + self.assertIn(['--ro-bind', '/environment/initialization', '/environment/initialization'], mounts) + self.assertIn(['--chdir', '/workspace'], [args[i:i+2] for i in range(len(args)-1)]) + + def test_private_initialization_uses_bound_workspace_and_empty_environment(self): + with tempfile.TemporaryDirectory() as directory: + workspace = str(Path(directory).resolve()) + stdin = mock.Mock() + stdin.buffer = io.BytesIO(b'{"version":1,"action":"configure"}') + with mock.patch.object(initialize.sys, 'argv', ['initialize', 'initialize', workspace, workspace + '/caps']), \ + mock.patch.object(initialize.sys, 'stdin', stdin), \ + mock.patch.object(initialize, 'roots') as roots, \ + mock.patch.object(initialize, 'run') as run, \ + contextlib.redirect_stdout(io.StringIO()) as output: + self.assertEqual(initialize.main(), 0) + roots.assert_called_once_with(workspace) + run.assert_called_once_with({'version': 1, 'action': 'configure'}, workspace=workspace, capability_root=workspace + '/caps') + self.assertEqual(json.loads(output.getvalue()), {'version': 1, 'outcome': 'completed'}) + with mock.patch.object(initialize, 'configure') as configure: + initialize.run({'action': 'configure'}, workspace=workspace) + configure.assert_called_once_with({}) + + def test_setup_mounts_custom_capabilities_read_only(self): + with tempfile.TemporaryDirectory() as directory: + base = Path(directory).resolve() + capabilities = base / 'capabilities' + capabilities.mkdir() + for system in (False, True): + with self.subTest(system=system), \ + mock.patch.object(initialize, 'sandbox', return_value=['system-root' if system else 'bwrap', '--']), \ + mock.patch.object(initialize.subprocess, 'run') as execute: + initialize.run({'action': 'setup', 'network': 'enabled', 'command': 'true'}, + workspace=str(base / 'workspace'), capability_root=str(capabilities)) + args = execute.call_args.args[0] + self.assertIn(['--ro-bind', str(capabilities), str(capabilities)], + [args[i:i+3] for i in range(len(args)-2)]) + self.assertLess(args.index(str(capabilities)), args.index('--')) + alias = base / 'alias' + alias.symlink_to(capabilities, target_is_directory=True) + for invalid in ('relative', '/', str(alias), str(base / 'capabilities' / '..')): + with self.assertRaises(ValueError): + initialize.expose_capabilities(['bwrap', '--'], invalid) + args = ['bwrap', '--'] + initialize.expose_capabilities(args, str(base / 'not-installed')) + self.assertEqual(args, ['bwrap', '--']) + def test_sandboxed_step_reports_only_its_exit_status(self): for status in (1, 3, 100, 255): failure = subprocess.CalledProcessError(status, SANDBOXED, output=CANARY.encode(), stderr=CANARY.encode()) diff --git a/services/agents-api/deploy/runtime/initialize_test.py b/services/agents-api/deploy/runtime/initialize_test.py index a10c4cc32..722d9fb7f 100644 --- a/services/agents-api/deploy/runtime/initialize_test.py +++ b/services/agents-api/deploy/runtime/initialize_test.py @@ -4,7 +4,6 @@ support the same nested isolation as its deployed Provider. No model is mocked; these checks exercise initialization only, not public native-model acceptance. """ -import base64 import json import os from pathlib import Path @@ -41,15 +40,13 @@ def main(): env.update(http_proxy=os.environ['OAC_TEST_PACKAGE_PROXY'], https_proxy=os.environ['OAC_TEST_PACKAGE_PROXY']) invoke('configure', env=env) - skill = [{'path': 'SKILL.md', 'data': base64.b64encode(b'---\nname: proof\ndescription: A proof.\n---\nRead check.sh.').decode()}, - {'path': 'scripts/check.sh', 'data': base64.b64encode(b'#!/bin/sh\nprintf skill-proof').decode(), 'executable': True}, - {'path': 'data.bin', 'data': base64.b64encode(bytes(range(256))).decode()}] - invoke('skill', name='proof', files=skill) - assert Path('/environment/initialization/capabilities/skills/proof/data.bin').read_bytes() == bytes(range(256)) - assert Path('/environment/initialization/capabilities/skills/proof/scripts/check.sh').stat().st_mode & 0o777 == 0o500 - invoke('skill', succeeds=False, name='proof', files=skill) - invoke('skill', succeeds=False, name='invalid', files=[{'path': '../../private/credential', 'data': 'YmFk'}]) - assert Path('/environment/private/credential').read_text() == CANARY + # A fixed Runtime-installed fixture tests setup access, not archive parsing. + skill = Path('/environment/initialization/capabilities/skills/proof') + (skill / 'scripts').mkdir(parents=True) + (skill / 'SKILL.md').write_text('---\nname: proof\ndescription: A proof.\n---\nRead check.sh.') + (skill / 'SKILL.md').chmod(0o400) + (skill / 'scripts/check.sh').write_text('#!/bin/sh\nprintf skill-proof') + (skill / 'scripts/check.sh').chmod(0o500) invoke('setup', command='/environment/initialization/capabilities/skills/proof/scripts/check.sh > /workspace/skill-result') assert Path('/environment/workspace/skill-result').read_text() == 'skill-proof' if '--system' in sys.argv: diff --git a/services/agents-api/deploy/runtime/tool-root.py b/services/agents-api/deploy/runtime/tool-root.py index f2693ccd9..85cd60cb3 100644 --- a/services/agents-api/deploy/runtime/tool-root.py +++ b/services/agents-api/deploy/runtime/tool-root.py @@ -38,6 +38,8 @@ def sandbox(cwd, *, writable=False, network='enabled', workspace='/workspace', s '--bind', workspace, '/environment/workspace', '--bind', '/environment/packages', '/environment/packages', '--ro-bind', str(ROOT), str(ROOT), '--ro-bind', str(CONFIG), str(CONFIG)] + if workspace not in ('/workspace', '/environment/workspace'): + args += ['--bind', workspace, workspace] if network == 'disabled': args += ['--unshare-net'] elif network != 'enabled': @@ -47,15 +49,15 @@ def sandbox(cwd, *, writable=False, network='enabled', workspace='/workspace', s return args + ['--chdir', cwd, '--'] -def initialization_sandbox(cwd, network='enabled', *, writable=False): - args = sandbox(cwd, writable=writable, network=network, workspace='/environment/workspace') +def initialization_sandbox(cwd, network='enabled', *, writable=False, workspace='/environment/workspace'): + args = sandbox(cwd, writable=writable, network=network, workspace=workspace) args[1:1] = ['--new-session', '--clearenv'] for key, value in {**BASE_ENV, 'DEBIAN_FRONTEND': 'noninteractive'}.items(): args[-1:-1] = ['--setenv', key, value] return args -def install(packages): +def install(packages, *, workspace='/environment/workspace'): if not isinstance(packages, list) or not packages or any( not isinstance(p, str) or not p or p.startswith('-') or '\x00' in p for p in packages ): @@ -69,7 +71,7 @@ def install(packages): ROOT.mkdir(mode=0o700) subprocess.run(['/usr/bin/tar', '--no-same-owner', '--no-same-permissions', '-xzf', str(SEED), '-C', str(ROOT)], check=True, env=BASE_ENV, stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - args = initialization_sandbox('/workspace', writable=True) + args = initialization_sandbox('/workspace', writable=True, workspace=workspace) # Namespace root maps only to the unprivileged Runtime UID; _apt is unmapped. apt = ['/usr/bin/apt-get', '-o', 'APT::Sandbox::User=root', '-o', 'Acquire::Retries=0'] for command in (apt + ['update'], apt + ['install', '-y', '--no-install-recommends', '--', *packages]): diff --git a/services/agents-api/internal/api/environment_request.go b/services/agents-api/internal/api/environment_request.go index faee62171..9a79b645f 100644 --- a/services/agents-api/internal/api/environment_request.go +++ b/services/agents-api/internal/api/environment_request.go @@ -2,10 +2,9 @@ package api import ( "encoding/json" - "path" - "strings" v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" ) @@ -21,7 +20,7 @@ func decodeSessionEnvironment(raw json.RawMessage) (*v1.Environment, error) { return decodeHostedEnvironment(raw) case "self_hosted": fields = append(fields, "workspace_directory", "capability_directories") - if !path.IsAbs(environment.WorkspaceDirectory) || strings.ContainsRune(environment.WorkspaceDirectory, 0) { + if agentcapabilities.ValidateSourceDirectories([]string{environment.WorkspaceDirectory}) != nil || agentcapabilities.ValidateSourceDirectories(environment.CapabilityDirectories) != nil { return nil, store.ErrInvalidInput } default: diff --git a/services/agents-api/internal/api/environment_request_test.go b/services/agents-api/internal/api/environment_request_test.go new file mode 100644 index 000000000..330579d63 --- /dev/null +++ b/services/agents-api/internal/api/environment_request_test.go @@ -0,0 +1,39 @@ +package api + +import ( + "encoding/json" + "testing" +) + +func TestSelfHostedPathsDoNotCreateManagedInitialization(t *testing.T) { + raw := json.RawMessage(`{"type":"self_hosted","workspace_directory":"/home/user/work","capability_directories":["/opt/skills","/home/user/plugins"]}`) + request, err := (decodedSessionRequest{Environment: raw}).validated() + if err != nil { + t.Fatal(err) + } + if !request.initialization.Empty() || request.Environment.WorkspaceDirectory != "/home/user/work" || len(request.Environment.CapabilityDirectories) != 2 { + t.Fatal("local selections became managed initialization", request.initialization, request.Environment) + } + for _, raw := range []string{ + `{"type":"self_hosted","workspace_directory":"/a/../b"}`, + `{"type":"self_hosted","workspace_directory":"/work/"}`, + `{"type":"self_hosted","workspace_directory":"/work","capability_directories":["relative"]}`, + `{"type":"self_hosted","workspace_directory":"/work","capability_directories":["/a","/a"]}`, + `{"type":"self_hosted","workspace_directory":"/work","plugins":[]}`, + `{"type":"self_hosted","workspace_directory":"/work","skills":[]}`, + } { + if _, err := (decodedSessionRequest{Environment: json.RawMessage(raw)}).validated(); err == nil { + t.Fatal("invalid local input accepted", raw) + } + } +} + +func TestSelfHostedSourcePathsArePlatformNeutral(t *testing.T) { + for _, directory := range []string{`C:\work`, `D:/skills`, `\\server\share\project`, `/Users/user/work`} { + raw, _ := json.Marshal(map[string]any{"type": "self_hosted", "workspace_directory": directory, "capability_directories": []string{directory}}) + request, err := (decodedSessionRequest{Environment: raw}).validated() + if err != nil || request.Environment.WorkspaceDirectory != directory || !request.initialization.Empty() { + t.Fatal("Core interpreted a Runtime source path", directory, err) + } + } +} diff --git a/services/agents-api/internal/api/environments.go b/services/agents-api/internal/api/environments.go index 475043b83..e80c918bf 100644 --- a/services/agents-api/internal/api/environments.go +++ b/services/agents-api/internal/api/environments.go @@ -36,7 +36,7 @@ func (h *Handler) getEnvironment(w http.ResponseWriter, r *http.Request) { func environmentResponse(environment store.Environment) (v1.EnvironmentInfo, error) { configuration, err := storedEnvironment(environment.Configuration) - if err != nil || (configuration.Type != "self_hosted" && configuration.Type != "openai_hosted") || (configuration.Type == "self_hosted" && len(configuration.CapabilityDirectories) != 0) || environment.ID == "" { + if err != nil || (configuration.Type != "self_hosted" && configuration.Type != "openai_hosted") || environment.ID == "" { return v1.EnvironmentInfo{}, errors.New("unsupported stored environment metadata configuration") } switch environment.Status { diff --git a/services/agents-api/internal/api/environments_test.go b/services/agents-api/internal/api/environments_test.go index 94df3e386..22801539b 100644 --- a/services/agents-api/internal/api/environments_test.go +++ b/services/agents-api/internal/api/environments_test.go @@ -74,7 +74,7 @@ func TestEnvironmentResourceExactProjectionWithoutExecution(t *testing.T) { } }) } - for _, capabilities := range []string{"", `,"capability_directories":null`, `,"capability_directories":[]`} { + for _, capabilities := range []string{"", `,"capability_directories":null`, `,"capability_directories":[]`, `,"capability_directories":["/skills"]`} { _, f := environmentResourceHandler(t) f.environment.Configuration = json.RawMessage(`{"type":"self_hosted","workspace_directory":"/workspace"` + capabilities + `}`) value, err := environmentResponse(f.environment) @@ -86,15 +86,15 @@ func TestEnvironmentResourceExactProjectionWithoutExecution(t *testing.T) { func TestEnvironmentResourceRejectsUnknownInventoryAndInvalidState(t *testing.T) { for name, configuration := range map[string]string{ - "missing": `{}`, - "none": `{"type":"none"}`, - "hosted": `{"type":"openai_hosted","env":{"SECRET":"private-canary"}}`, - "capabilities": `{"type":"self_hosted","workspace_directory":"/workspace","capability_directories":["/skills"]}`, - "files": `{"type":"self_hosted","workspace_directory":"/workspace","files":[{"data":"private-canary"}]}`, - "plugins": `{"type":"self_hosted","workspace_directory":"/workspace","plugins":[]}`, - "skills": `{"type":"self_hosted","workspace_directory":"/workspace","skills":[]}`, - "wrong type": `{"type":"self_hosted","workspace_directory":"/workspace","capability_directories":false}`, - "invalid": `{"type":"self_hosted","workspace_directory":`, + "missing": `{}`, + "none": `{"type":"none"}`, + "hosted": `{"type":"openai_hosted","env":{"SECRET":"private-canary"}}`, + "unclean directories": `{"type":"self_hosted","workspace_directory":"/workspace","capability_directories":["/skills/../private"]}`, + "files": `{"type":"self_hosted","workspace_directory":"/workspace","files":[{"data":"private-canary"}]}`, + "plugins": `{"type":"self_hosted","workspace_directory":"/workspace","plugins":[]}`, + "skills": `{"type":"self_hosted","workspace_directory":"/workspace","skills":[]}`, + "wrong type": `{"type":"self_hosted","workspace_directory":"/workspace","capability_directories":false}`, + "invalid": `{"type":"self_hosted","workspace_directory":`, } { t.Run(name, func(t *testing.T) { h, f := environmentResourceHandler(t) diff --git a/services/agents-api/internal/api/handler.go b/services/agents-api/internal/api/handler.go index 318132d41..2a7112fe1 100644 --- a/services/agents-api/internal/api/handler.go +++ b/services/agents-api/internal/api/handler.go @@ -159,7 +159,7 @@ func (h *Handler) routes() *chi.Mux { // createSession atomically reserves or admits initial text with the Session. // @Summary Create an execution Session -// @Description The optional Core model_provider bundle resolves from the Session override, saved Agent defaults, then, for openai_hosted and none, the deployment default of the resolved harness; self_hosted never uses the deployment default and none accepts only it. openai_hosted and self_hosted Sessions that resolve no bundle return 400 model_provider_required with param x_agents_core.model_provider before any write. Core encrypts and freezes the resolved bundle; later Agent or deployment default edits and same-key retries cannot change it. Keys are never returned. Supports inline configuration or a tenant-owned saved agent_id with per-Session field replacements. Execution supports model/instructions, text verbosity, non-deferred function tools, adapter-qualified multi_agent with persisted Subagent reads, implicit reasoning, service tier auto and environment type none, subject to the configured engine. Codex additionally supports HTTP MCP with service origin (omitted or null on HTTP transport is saved as service), native allowed_tools and boolean required defaulting to false. Session vault_ids attach only project-owned Vaults; credential_id selects an attached static bearer or OAuth credential for the exact HTTPS URL, while null/omission selects a unique match or remains anonymous. Session reads, lists and event snapshots show that implicitly selected credential ID in a null or omitted credential_id, also after the credential is deleted; anonymous selections stay null and the stored caller intent is unchanged. After the input requirement and before any write, a credential_id without vault_ids, one outside the attached Vaults (one message for missing, foreign and unattached IDs) or one for another server_url returns 400 invalid_request_error, and several implicit matches return 409 conflict_error. Missing decryption configuration fails dispatch without anonymous fallback. Required initialization uses native startup before the first native Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and native OAuth login remain unsupported. The self_hosted profile requires Codex, an absolute workspace_directory and empty capability_directories, with optional non-deferred function tools and HTTP MCP using service origin, optionally authenticated by the attached Vault rules. Remote MCP and remote Bearer authentication each require separately advertised combination support; old peers cannot receive unsupported work. Omitted/null capability_directories use the empty-list default; self_hosted requires configured execution plus executor registry. Claude SDK currently requires medium verbosity and object-root function schemas. It supports anonymous or attached static-bearer service-origin HTTP MCP on none with boolean required and separately advertised MCP/bearer/required runtime support. Required servers must be connected before the first native input is released; pending or failed startup rejects execution. The shared Vault selection and immutable binding rules apply; unsupported native labels/tool names reject before persistence. An attached Vault with no matching credential may remain anonymous; missing keys or failed credential lookup/decryption never fall back to anonymous execution. Omitted stream defaults to false; stream and agent_id cannot be null. Metadata may be null; non-string values and limit violations return invalid_request_error with a metadata or metadata. param. The inline agent uses the Agent create configuration validation with agent.-prefixed params, reported before the input requirement and saved-Agent lookup; saved configurations with conflicting tools or schema roots reject admission with the same errors, and execution limits keep unsupported_or_invalid_configuration. Hosted network policy rejections return invalid_request_error with a null param. Initial input accepts a string or ordered user-message array. Codex and Claude SDK on none and qualified openai_hosted also accept inline PNG/JPEG image content; other image combinations and remote URLs are unsupported. None initial input atomically starts a Turn; self_hosted initial input is reserved while returning its Environment connection target, with execution deferred to native readiness and Session failure on initial timeout. Initial input is required for none and for streamed creation outside self_hosted. Omitted/null input remains valid for non-streaming hosted and self_hosted creation. With stream=true, returns live Session events starting with the committed creation snapshot and closes right after the first agent.session.idle recorded when a Turn ends or an input reservation stops being pending, or any agent.session.failed, without sending later events. A creation that admitted nothing closes after the snapshot; a settlement that records no event closes after events up to the cursor read with a settled Session projection. Required actions keep it open; disconnect does not cancel execution. The GET events stream remains live-only. New Sessions retain their authenticated creator; all creation retries require the same typed subject, including across key rotation. Saved-Agent retries and inline requests using Vault attachments or credential references retain caller intent independently of later resource changes; new hosted inline requests also freeze caller intent before deployment defaults resolve; unrelated non-hosted inline retries preserve resolved/default equivalences, and their resolved hash leaves out any deployment default. Provider keys enter retry hashes only as fingerprints keyed by the credential key. Unknown historical creators reject retries; known creators without recorded intent retain resolved-snapshot retry rules. These conflict policies are local and not verified hosted parity. A same-key stream=true retry of an existing creation returns 201 with no events and closes at once; retry with stream=false or use the GET events stream to recover. Claude SDK on none and Core-managed Docker openai_hosted supports qualified object-root json_schema output with medium verbosity, single-Agent execution and ordinary functions. Hosted execution reuses native workspace tools and Files/Artifacts; Skills, Plugins, capability directories, HTTP MCP, Subagent and tool_search combinations remain unqualified, including inherited template contents. Other non-text initial input remains unsupported. Basic Codex and Claude SDK openai_hosted creation requires an explicitly configured managed provider. The Claude workspace profile supports non-deferred function tools with text or successful inline PNG/JPEG results alongside native workspace tools; HTTP MCP remains unsupported. Idle Sessions provision automatically; initial provisioning has no caller connection action. Network defaults to enabled; disabled and restricted exact ASCII hostnames are supported. Restricted policy requires 1–100 allowed domains. Unsupported hostname forms and startup installations are rejected. Confidential env, system/npm/Python packages and ordered setup commands use the shared initialization lifecycle; requested network applies after setup. Initial inline and tenant-owned file_id files freeze encrypted bytes before provisioning, then install through the common Core lifecycle before native execution or live Files access. With a template reference, omitted/null files, env, packages and setup_commands inherit. Non-null files and command lists replace; env overlays by key; each package manager inherits on omission/null and otherwise replaces its list. Empty lists clear their selected field. Tenant-owned environment_template_id references inherit omitted/null network and allow only narrowing overrides. Inline hosted network:null retains the enabled default; updating a Template with network:null resets its saved policy to enabled. Core freezes effective configuration; template updates/deletion do not alter Session snapshots or same-intent creation retries. Inline or tenant-owned skill_reference Skills share initialization. Templates preserve default/latest/explicit selectors; Session creation freezes concrete metadata and encrypted content atomically. Skill, Plugin and capability-directory list omission/null inherit; a non-null list replaces, including empty-list clearing. Omitted/null Skill version selectors resolve the default version. Source deletion/default updates cannot change committed Session Skill contents. Deferred function discovery uses type-only tool_search and per-function defer_loading in the qualified single-agent Claude environment:none function profile, including qualified inline image messages and text results. Explicit web_search mode disabled and programmatic_tool_calling enabled false use frozen common Runtime controls. Enabled forms, including those saved on an Agent, remain unqualified and reject before any write unless the Session replaces tools. Omitted programmatic configuration preserves native behavior, a documented difference from the official default-on behavior. Other combinations remain unqualified; see the operation coverage. +// @Description The optional Core model_provider bundle resolves from the Session override, saved Agent defaults, then, for openai_hosted and none, the deployment default of the resolved harness; self_hosted never uses the deployment default and none accepts only it. openai_hosted and self_hosted Sessions that resolve no bundle return 400 model_provider_required with param x_agents_core.model_provider before any write. Core encrypts and freezes the resolved bundle; later Agent or deployment default edits and same-key retries cannot change it. Keys are never returned. Supports inline configuration or a tenant-owned saved agent_id with per-Session field replacements. Execution supports model/instructions, text verbosity, non-deferred function tools, adapter-qualified multi_agent with persisted Subagent reads, implicit reasoning, service tier auto and environment type none, subject to the configured engine. Codex additionally supports HTTP MCP with service origin (omitted or null on HTTP transport is saved as service), native allowed_tools and boolean required defaulting to false. Session vault_ids attach only project-owned Vaults; credential_id selects an attached static bearer or OAuth credential for the exact HTTPS URL, while null/omission selects a unique match or remains anonymous. Session reads, lists and event snapshots show that implicitly selected credential ID in a null or omitted credential_id, also after the credential is deleted; anonymous selections stay null and the stored caller intent is unchanged. After the input requirement and before any write, a credential_id without vault_ids, one outside the attached Vaults (one message for missing, foreign and unattached IDs) or one for another server_url returns 400 invalid_request_error, and several implicit matches return 409 conflict_error. Missing decryption configuration fails dispatch without anonymous fallback. Required initialization uses native startup before the first native Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and native OAuth login remain unsupported. The self_hosted profile uses a qualified native harness, a clean absolute workspace_directory and optional absolute local capability_directories prepared by Runtime, with optional non-deferred function tools and HTTP MCP using service origin, optionally authenticated by the attached Vault rules. Remote MCP and remote Bearer authentication each require separately advertised combination support; old peers cannot receive unsupported work. Omitted/null capability_directories use the empty-list default; self_hosted requires configured execution plus executor registry. Claude SDK currently requires medium verbosity and object-root function schemas. It supports anonymous or attached static-bearer service-origin HTTP MCP on none with boolean required and separately advertised MCP/bearer/required runtime support. Required servers must be connected before the first native input is released; pending or failed startup rejects execution. The shared Vault selection and immutable binding rules apply; unsupported native labels/tool names reject before persistence. An attached Vault with no matching credential may remain anonymous; missing keys or failed credential lookup/decryption never fall back to anonymous execution. Omitted stream defaults to false; stream and agent_id cannot be null. Metadata may be null; non-string values and limit violations return invalid_request_error with a metadata or metadata. param. The inline agent uses the Agent create configuration validation with agent.-prefixed params, reported before the input requirement and saved-Agent lookup; saved configurations with conflicting tools or schema roots reject admission with the same errors, and execution limits keep unsupported_or_invalid_configuration. Hosted network policy rejections return invalid_request_error with a null param. Initial input accepts a string or ordered user-message array. Codex and Claude SDK on none and qualified openai_hosted also accept inline PNG/JPEG image content; other image combinations and remote URLs are unsupported. None initial input atomically starts a Turn; self_hosted initial input is reserved while returning its Environment connection target, with execution deferred to native readiness and Session failure on initial timeout. Initial input is required for none and for streamed creation outside self_hosted. Omitted/null input remains valid for non-streaming hosted and self_hosted creation. With stream=true, returns live Session events starting with the committed creation snapshot and closes right after the first agent.session.idle recorded when a Turn ends or an input reservation stops being pending, or any agent.session.failed, without sending later events. A creation that admitted nothing closes after the snapshot; a settlement that records no event closes after events up to the cursor read with a settled Session projection. Required actions keep it open; disconnect does not cancel execution. The GET events stream remains live-only. New Sessions retain their authenticated creator; all creation retries require the same typed subject, including across key rotation. Saved-Agent retries and inline requests using Vault attachments or credential references retain caller intent independently of later resource changes; new hosted inline requests also freeze caller intent before deployment defaults resolve; unrelated non-hosted inline retries preserve resolved/default equivalences, and their resolved hash leaves out any deployment default. Provider keys enter retry hashes only as fingerprints keyed by the credential key. Unknown historical creators reject retries; known creators without recorded intent retain resolved-snapshot retry rules. These conflict policies are local and not verified hosted parity. A same-key stream=true retry of an existing creation returns 201 with no events and closes at once; retry with stream=false or use the GET events stream to recover. Claude SDK on none and Core-managed Docker openai_hosted supports qualified object-root json_schema output with medium verbosity, single-Agent execution and ordinary functions. Hosted execution reuses native workspace tools and Files/Artifacts; Skills, Plugins, capability directories, HTTP MCP, Subagent and tool_search combinations remain unqualified, including inherited template contents. Other non-text initial input remains unsupported. Basic Codex and Claude SDK openai_hosted creation requires an explicitly configured managed provider. The Claude workspace profile supports non-deferred function tools with text or successful inline PNG/JPEG results alongside native workspace tools; HTTP MCP remains unsupported. Idle Sessions provision automatically; initial provisioning has no caller connection action. Network defaults to enabled; disabled and restricted exact ASCII hostnames are supported. Restricted policy requires 1–100 allowed domains. Unsupported hostname forms and startup installations are rejected. Confidential env, system/npm/Python packages and ordered setup commands use the shared initialization lifecycle; requested network applies after setup. Initial inline and tenant-owned file_id files freeze encrypted bytes before provisioning, then install through the common Core lifecycle before native execution or live Files access. With a template reference, omitted/null files, env, packages and setup_commands inherit. Non-null files and command lists replace; env overlays by key; each package manager inherits on omission/null and otherwise replaces its list. Empty lists clear their selected field. Tenant-owned environment_template_id references inherit omitted/null network and allow only narrowing overrides. Inline hosted network:null retains the enabled default; updating a Template with network:null resets its saved policy to enabled. Core freezes effective configuration; template updates/deletion do not alter Session snapshots or same-intent creation retries. Inline or tenant-owned skill_reference Skills share initialization. Templates preserve default/latest/explicit selectors; Session creation freezes concrete metadata and encrypted content atomically. Skill, Plugin and capability-directory list omission/null inherit; a non-null list replaces, including empty-list clearing. Omitted/null Skill version selectors resolve the default version. Source deletion/default updates cannot change committed Session Skill contents. Deferred function discovery uses type-only tool_search and per-function defer_loading in the qualified single-agent Claude environment:none function profile, including qualified inline image messages and text results. Explicit web_search mode disabled and programmatic_tool_calling enabled false use frozen common Runtime controls. Enabled forms, including those saved on an Agent, remain unqualified and reject before any write unless the Session replaces tools. Omitted programmatic configuration preserves native behavior, a documented difference from the official default-on behavior. Other combinations remain unqualified; see the operation coverage. // @Tags Sessions // @Accept json // @Produce json,text/event-stream diff --git a/services/agents-api/internal/api/session_request.go b/services/agents-api/internal/api/session_request.go index 40db4f215..edaac5777 100644 --- a/services/agents-api/internal/api/session_request.go +++ b/services/agents-api/internal/api/session_request.go @@ -61,9 +61,15 @@ func (request decodedSessionRequest) validated() (sessionRequest, error) { return input, store.ErrInvalidInput } var err error - input.initialization, err = decodeEnvironmentSetup(environmentFields) - if err != nil { - return input, err + var environmentType string + _ = json.Unmarshal(environmentFields["type"], &environmentType) + // Self-hosted paths are frozen in Environment.Configuration. They do not + // create managed initialization state or require a managed allocation. + if environmentType != "self_hosted" { + input.initialization, err = decodeEnvironmentSetup(environmentFields) + if err != nil { + return input, err + } } input.initialFiles, err = decodeInitialFiles(environmentFields["files"]) if err != nil { diff --git a/services/agents-api/internal/execution/delivery.go b/services/agents-api/internal/execution/delivery.go index 8c31ec0c4..d2f2f1872 100644 --- a/services/agents-api/internal/execution/delivery.go +++ b/services/agents-api/internal/execution/delivery.go @@ -3,6 +3,7 @@ package execution import ( "context" "encoding/json" + "errors" "strconv" "time" @@ -50,6 +51,8 @@ func abort(peer *gateway.Session, runID string) { } func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, peer *gateway.Session, request proto.PromptRequestPayload, first int64, prepared *preparedStart) (result Result, status string) { + changed, unsubscribeChanges := d.notifications.subscribe(tenantID, sessionID) + defer unsubscribeChanges() status = store.TurnFailed result.AppliedThrough = first subscription, err := peer.SubscribeDurable(request.RunID) @@ -80,6 +83,9 @@ func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, pe } }() var preparationEvents <-chan proto.Envelope + var executorRetried, nativeObserved bool + inputStarted := time.Now() + firstTextObserved := false if prepared != nil { preparationEvents = prepared.sub.Events err = prepared.start(ctx, request) @@ -92,6 +98,8 @@ func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, pe } ticker := time.NewTicker(250 * time.Millisecond) defer ticker.Stop() + workReady := make(chan struct{}, 1) + workReady <- struct{}{} flushTicker := time.NewTicker(100 * time.Millisecond) defer flushTicker.Stop() var pending *pendingInput @@ -118,6 +126,31 @@ func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, pe } started, err := prepared.started(env, request.RunID) if err != nil { + var rejection *preparationRejection + if errors.As(err, &rejection) && rejection.operation == proto.TypeExecutionStart && rejection.code == "executor_unavailable" && !executorRetried && !nativeObserved && cancelReply == nil { + // This receipt guarantees no native input was submitted and + // the previous owner was closed. Unknown delivery is never retried. + executorRetried = true + prepared.close() + currentPeer, peerErr := d.authorizedPeer(ctx, peer.DeviceID) + if peerErr != nil || currentPeer != peer { + result.ErrorCode = "device_disconnected" + return + } + replacement, prepareErr := d.prepareTurnExecutor(ctx, peer, tenantID, sessionID, request.RunID, request, store.TurnInProgress) + if prepareErr != nil { + result.ErrorCode = "executor_recovery_failed" + return + } + prepared = replacement + defer prepared.close() + preparationEvents = prepared.sub.Events + if prepared.start(ctx, request) != nil { + result.ErrorCode = "delivery_unknown" + return + } + continue + } // Cancellation owns the receipt even when it interrupts native Start. if cancelReply != nil { preparationEvents = nil @@ -165,6 +198,11 @@ func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, pe result.ErrorCode = "device_disconnected" return } + nativeObserved = true + if !firstTextObserved && hasText(env) { + firstTextObserved = true + recordFirstText(ctx, sessionID, request.RunID, inputStarted) + } writeErr := journal.observe(ctx, env) if result.mergeObservation(env) != nil { result.ErrorCode = "invalid_executor_result" @@ -223,6 +261,16 @@ func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, pe return } case <-ticker.C: + select { + case workReady <- struct{}{}: + default: + } + case <-changed: + select { + case workReady <- struct{}{}: + default: + } + case <-workReady: if !cancelSent.IsZero() { if time.Since(cancelSent) > 15*time.Second { result.ErrorCode = "cancel_unconfirmed" @@ -233,6 +281,9 @@ func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, pe turn, err := d.Store.GetTurn(ctx, tenantID, sessionID, request.RunID) if err != nil { result.ErrorCode = "execution_state_unavailable" + if ctx.Err() != nil { + result.ErrorCode = "execution_interrupted" + } return } if turn.Status != store.TurnInProgress && turn.Status != store.TurnWaiting { @@ -255,6 +306,9 @@ func (d *Dispatcher) deliver(ctx context.Context, tenantID, sessionID string, pe inputs, err := d.Store.ListTurnInputs(ctx, tenantID, sessionID, request.RunID, result.AppliedThrough, 1) if err != nil { result.ErrorCode = "execution_state_unavailable" + if ctx.Err() != nil { + result.ErrorCode = "execution_interrupted" + } return } if len(inputs) == 0 { diff --git a/services/agents-api/internal/execution/directory_preparation.go b/services/agents-api/internal/execution/directory_preparation.go index 6d4aca014..a5c64a2f9 100644 --- a/services/agents-api/internal/execution/directory_preparation.go +++ b/services/agents-api/internal/execution/directory_preparation.go @@ -42,7 +42,7 @@ func (d *Dispatcher) withPreparedWorkspace(owner context.Context, peer *gateway. } }() prepare, stop := context.WithTimeout(owner, 10*time.Second) - err = send(prepare, peer, proto.TypeExecutionPrepare, prepared.requestID, proto.ExecutionPreparePayload{Configuration: req}) + err = send(prepare, peer, proto.TypeExecutionPrepare, prepared.requestID, proto.ExecutionPreparePayload{SessionID: session.ID, Configuration: req}) if err == nil { err = prepared.awaitDirectoryReady(prepare) } diff --git a/services/agents-api/internal/execution/dispatcher.go b/services/agents-api/internal/execution/dispatcher.go index 83541f628..3cab39fc1 100644 --- a/services/agents-api/internal/execution/dispatcher.go +++ b/services/agents-api/internal/execution/dispatcher.go @@ -29,6 +29,7 @@ type DaemonConfig struct { } type Dispatcher struct { + notifications *executionNotifications Policy Store *store.Store Registry *gateway.Registry @@ -91,17 +92,22 @@ func (d *Dispatcher) Run(ctx context.Context, tenantID, sessionID, turnID string if err != nil { return store.Turn{}, err } + req.WorkDir, req.DisableExecutionEnvironment = workDir, noEnvironment + prepared, err := d.prepareTurnExecutor(ctx, peer, tenantID, sessionID, turnID, req, store.TurnQueued) + if err != nil { + return store.Turn{}, err + } + defer prepared.close() if _, err := d.Store.TransitionTurn(ctx, tenantID, sessionID, turnID, store.TurnTransition{ExpectedStatus: store.TurnQueued, Status: store.TurnInProgress}); err != nil { return store.Turn{}, err } req.ConversationID, req.RunID, req.Input = sessionID, turnID, text - req.WorkDir, req.DisableExecutionEnvironment = workDir, noEnvironment release, err := peer.TrackExecutionDelivery(req.RunID) if err != nil { return d.finishRun(tenantID, sessionID, turnID, snapshot.Agent.Model, Result{ErrorCode: "delivery_unknown", AppliedThrough: through}, store.TurnFailed) } defer release() - result, status := d.deliver(ctx, tenantID, sessionID, peer, req, through, nil) + result, status := d.deliver(ctx, tenantID, sessionID, peer, req, through, prepared) return d.finishRun(tenantID, sessionID, turnID, snapshot.Agent.Model, result, status) } diff --git a/services/agents-api/internal/execution/environment_admission.go b/services/agents-api/internal/execution/environment_admission.go index b915e37af..da0da4a1c 100644 --- a/services/agents-api/internal/execution/environment_admission.go +++ b/services/agents-api/internal/execution/environment_admission.go @@ -85,7 +85,7 @@ func (w *Worker) submitEnvironmentInputs(ctx context.Context, session store.Sess } if (kind == "cancel" || kind == "tool_result") && !slices.ContainsFunc(inputs, func(input store.Input) bool { return input.Kind != kind }) { // Neither kind creates a Turn. The Session lock preserves target and retry identity. - return w.admission.SubmitInputs(ctx, session.TenantID, session.ID, key, inputs) + return w.admitInputs(ctx, session.TenantID, session.ID, key, inputs) } // Messages start work. A Session from before deployment defaults moved into // Core may have no frozen provider; reject it here instead of queueing work @@ -94,12 +94,15 @@ func (w *Worker) submitEnvironmentInputs(ctx context.Context, session store.Sess if json.Unmarshal(session.Configuration, &snapshot) != nil || !snapshot.ModelProviderConfigured { return nil, store.ErrModelProviderRequired } + changed, unsubscribe := w.dispatcher.notifications.subscribe(session.TenantID, session.ID) + defer unsubscribe() reserve, cancel := context.WithTimeout(ctx, 5*time.Second) reservation, err := w.admission.ReserveEnvironmentInput(reserve, session.TenantID, session.ID, key, inputs) cancel() if err != nil { return nil, err } + w.wakeScheduler() if reservation.State == store.EnvironmentInputPending && !reservation.IsInitial { w.hintRuntimeWake(ctx, session) } @@ -120,6 +123,7 @@ func (w *Worker) submitEnvironmentInputs(ctx context.Context, session store.Sess case <-ctx.Done(): return nil, ctx.Err() case <-ticker.C: + case <-changed: } if err := w.checkAdmissionOwnership(ctx); err != nil { return nil, err diff --git a/services/agents-api/internal/execution/environment_capabilities_test.go b/services/agents-api/internal/execution/environment_capabilities_test.go new file mode 100644 index 000000000..722494c8e --- /dev/null +++ b/services/agents-api/internal/execution/environment_capabilities_test.go @@ -0,0 +1,48 @@ +package execution + +import ( + "encoding/json" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" + "slices" + "testing" +) + +func TestSelfHostedCapabilitySourcesAreFrozenAndStrict(t *testing.T) { + session := store.Session{ID: "session", TenantID: "tenant"} + environment := store.Environment{ID: "environment", SessionID: session.ID, TenantID: session.TenantID, + Configuration: []byte(`{"type":"self_hosted","workspace_directory":"/home/user/project","capability_directories":["/home/user/skills","/opt/plugins"]}`)} + var request proto.PromptRequestPayload + if err := (&Dispatcher{}).configurePreparedEnvironment(session, environment, store.ExecutionDevice{EnvironmentID: environment.ID}, &request); err != nil { + t.Fatal(err) + } + local := request.LocalEnvironment + if local.WorkspaceDirectory != "/home/user/project" || !local.Capabilities || local.ToolEnvironment || len(local.Skills) != 0 || + local.CapabilitySources == nil || !slices.Equal(local.CapabilitySources.Directories, []string{"/home/user/skills", "/opt/plugins"}) { + t.Fatal("frozen source selections lost", local) + } + for _, configuration := range []string{ + `{"type":"self_hosted","workspace_directory":"/a/../b"}`, + `{"type":"self_hosted","workspace_directory":"relative"}`, + `{"type":"self_hosted","workspace_directory":"/work","capability_directories":["/a","/a"]}`, + `{"type":"self_hosted","workspace_directory":"/work","skills":[]}`, + `{"type":"self_hosted","workspace_directory":"/work","initialization":true}`, + } { + if LocalWorkspaceConfiguration([]byte(configuration)) { + t.Fatal("invalid self-hosted shape accepted", configuration) + } + } +} + +func TestSelfHostedPreparedPathsArePlatformNeutral(t *testing.T) { + for _, directory := range []string{`C:\work`, `D:/skills`, `\\server\share\project`, `/Users/user/work`} { + raw, _ := json.Marshal(map[string]any{"type": "self_hosted", "workspace_directory": directory, "capability_directories": []string{directory}}) + session := store.Session{ID: "session", TenantID: "tenant"} + environment := store.Environment{ID: "environment", SessionID: session.ID, TenantID: session.TenantID, Configuration: raw} + var request proto.PromptRequestPayload + err := (&Dispatcher{}).configurePreparedEnvironment(session, environment, store.ExecutionDevice{EnvironmentID: environment.ID}, &request) + if err != nil || request.LocalEnvironment.WorkspaceDirectory != directory || request.LocalEnvironment.CapabilitySources.Directories[0] != directory { + t.Fatal("Core interpreted a Runtime source path", directory, err) + } + } +} diff --git a/services/agents-api/internal/execution/environment_placement.go b/services/agents-api/internal/execution/environment_placement.go index bbd3f1121..ddb9b619b 100644 --- a/services/agents-api/internal/execution/environment_placement.go +++ b/services/agents-api/internal/execution/environment_placement.go @@ -38,11 +38,19 @@ func parseEnvironmentPlacement(configuration json.RawMessage) (environmentPlacem } switch placement.Type { case "self_hosted": - if placement.WorkspaceDirectory == "/workspace" && len(placement.CapabilityDirectories) == 0 { + var local struct { + Type string `json:"type"` + WorkspaceDirectory string `json:"workspace_directory"` + CapabilityDirectories []string `json:"capability_directories"` + } + decoder := json.NewDecoder(bytes.NewReader(configuration)) + decoder.DisallowUnknownFields() + if decoder.Decode(&local) == nil && validSelfHostedPlacement(placement) { placement.NetworkAccess = "enabled" return placement, nil } case "openai_hosted": + placement.WorkspaceDirectory = "/workspace" // Stored policy is shared by preparation and provider bootstrap. var local struct { Plugins []agentplugin.Metadata `json:"plugins,omitempty"` @@ -90,14 +98,26 @@ func (d *Dispatcher) configurePreparedEnvironment(session store.Session, environ if placement.SystemPackages && !placement.ToolEnvironment { return store.ErrInvalidInput } - req.LocalEnvironment = &proto.LocalEnvironment{ID: environment.ID, Capabilities: len(placement.Plugins)+len(placement.CapabilityDirectories) > 0, ToolEnvironment: placement.ToolEnvironment, SystemPackages: placement.SystemPackages} + sources := &agentcapabilities.Input{Plugins: append([]agentplugin.Metadata(nil), placement.Plugins...), Directories: append([]string(nil), placement.CapabilityDirectories...)} for _, metadata := range placement.Skills { if store.ValidateInstalledSkillMetadata(metadata) != nil { return store.ErrInvalidInput } - req.LocalEnvironment.Capabilities = true + sources.Skills = append(sources.Skills, (store.EnvironmentSkill{Metadata: metadata}).InstallationMetadata()) + } + req.LocalEnvironment = &proto.LocalEnvironment{ + ID: environment.ID, WorkspaceDirectory: placement.WorkspaceDirectory, + CapabilitySources: sources, + Capabilities: len(sources.Skills)+len(sources.Plugins)+len(sources.Directories) > 0, + ToolEnvironment: placement.ToolEnvironment, SystemPackages: placement.SystemPackages, } req.LocalEnvironment.NetworkAccess = placement.NetworkAccess req.LocalEnvironment.AllowedDomains = append([]string(nil), placement.AllowedDomains...) return nil } + +func validSelfHostedPlacement(placement environmentPlacement) bool { + return agentcapabilities.ValidateSourceDirectories([]string{placement.WorkspaceDirectory}) == nil && + agentcapabilities.ValidateSourceDirectories(placement.CapabilityDirectories) == nil && + !placement.ToolEnvironment && len(placement.Skills)+len(placement.Plugins) == 0 +} diff --git a/services/agents-api/internal/execution/environment_placement_test.go b/services/agents-api/internal/execution/environment_placement_test.go index 922962670..647e57538 100644 --- a/services/agents-api/internal/execution/environment_placement_test.go +++ b/services/agents-api/internal/execution/environment_placement_test.go @@ -119,11 +119,11 @@ func TestSelfHostedUsesSameLocalBinding(t *testing.T) { for _, binding := range []string{"", "foreign", environment.ID} { var request proto.PromptRequestPayload err := (&Dispatcher{}).configurePreparedEnvironment(session, environment, store.ExecutionDevice{EnvironmentID: binding}, &request) - valid := binding == environment.ID && workspace == "/workspace" + valid := binding == environment.ID if (err == nil) != valid { t.Fatal(workspace, binding, err) } - if valid && (request.LocalEnvironment == nil || request.LocalEnvironment.ID != environment.ID || request.LocalEnvironment.NetworkAccess != "enabled") { + if valid && (request.LocalEnvironment == nil || request.LocalEnvironment.ID != environment.ID || request.LocalEnvironment.NetworkAccess != "enabled" || request.LocalEnvironment.WorkspaceDirectory != workspace || request.LocalEnvironment.CapabilitySources == nil) { t.Fatal("self-hosted placement did not retain common local contract", request.LocalEnvironment) } } diff --git a/services/agents-api/internal/execution/executor_observation.go b/services/agents-api/internal/execution/executor_observation.go new file mode 100644 index 000000000..12eec8293 --- /dev/null +++ b/services/agents-api/internal/execution/executor_observation.go @@ -0,0 +1,41 @@ +package execution + +import ( + "context" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + obslog "github.com/MiniMax-AI-Dev/parsar/internal/obs/log" +) + +// These durations use one Core process monotonic clock. They describe control +// readiness and input-to-first-text latency, not model-only or native tool time. +func recordExecutorReadiness(p *preparedStart, status proto.PreparationStatusPayload) { + obslog.Bg().Info("executor ready", "request_id", p.requestID, + "executor_id", status.ExecutorID, "reused", status.Reused, + "control_ready_ms", time.Since(p.createdAt).Milliseconds()) +} + +// The started receipt confirms adapter ownership, not model consumption. +func recordExecutorStart(p *preparedStart, turn string) { + obslog.Bg().Info("executor start acknowledged", "turn_id", turn, + "request_id", p.requestID, "executor_id", p.executorID, + "start_control_ms", time.Since(p.startSentAt).Milliseconds()) +} + +func recordFirstText(ctx context.Context, session, turn string, submitted time.Time) { + obslog.Bg().InfoContext(ctx, "executor first text", "session_id", session, + "turn_id", turn, "input_to_first_text_ms", time.Since(submitted).Milliseconds()) +} + +func hasText(env proto.Envelope) bool { + switch env.Type { + case proto.TypeDelta: + var text proto.DeltaPayload + return env.DecodePayload(&text) == nil && text.Delta != "" + case proto.TypeOutputMessage: + var text proto.OutputMessagePayload + return env.DecodePayload(&text) == nil && text.Text != nil && *text.Text != "" + } + return false +} diff --git a/services/agents-api/internal/execution/executor_preparation.go b/services/agents-api/internal/execution/executor_preparation.go new file mode 100644 index 000000000..11078598b --- /dev/null +++ b/services/agents-api/internal/execution/executor_preparation.go @@ -0,0 +1,63 @@ +package execution + +import ( + "context" + "errors" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/gateway" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" +) + +// prepareTurnExecutor reserves one admission on the Runtime-owned Executor. +// Core does not cache native ownership. A replacement requires the Runtime to +// confirm cleanup and recover the exact Session history before returning ready. +func (d *Dispatcher) prepareTurnExecutor(ctx context.Context, peer *gateway.Session, tenant, session, turn string, request proto.PromptRequestPayload, expectedStatus string) (*preparedStart, error) { + prepared, err := newPreparedStart(peer) + if err != nil { + return nil, err + } + request.RunID, request.ConversationID, request.Input = "", "", nil + if err = send(ctx, peer, proto.TypeExecutionPrepare, prepared.requestID, proto.ExecutionPreparePayload{SessionID: session, Configuration: request}); err == nil { + err = d.awaitTurnExecutor(ctx, tenant, session, turn, expectedStatus, prepared) + } + if err != nil { + prepared.close() + return nil, err + } + return prepared, nil +} + +func (d *Dispatcher) awaitTurnExecutor(ctx context.Context, tenant, session, turn, expectedStatus string, prepared *preparedStart) error { + tick := time.NewTicker(250 * time.Millisecond) + defer tick.Stop() + for { + select { + case <-ctx.Done(): + return ctx.Err() + case <-tick.C: + current, err := d.Store.GetTurn(ctx, tenant, session, turn) + if err != nil { + return err + } + if current.Status != expectedStatus || !current.CancelRequestedAt.IsZero() { + return store.ErrTurnConflict + } + case env, ok := <-prepared.sub.Events: + if !ok { + return errors.New("executor preparation control stream closed") + } + status, err := prepared.observation(env) + if err != nil { + return err + } + if status.State == "ready" && status.RunID == "" && status.ExecutorID != "" { + return nil + } + if status.State != "preparing" { + return errors.New("unexpected executor state before Turn start") + } + } + } +} diff --git a/services/agents-api/internal/execution/notifications.go b/services/agents-api/internal/execution/notifications.go new file mode 100644 index 000000000..830f36859 --- /dev/null +++ b/services/agents-api/internal/execution/notifications.go @@ -0,0 +1,50 @@ +package execution + +import "sync" + +// executionNotifications carries process-local hints after durable commits. +// Every consumer rechecks storage and retains polling for external changes. +// Subscriptions exist only while a caller is waiting or a Turn is active. +type executionNotifications struct { + mu sync.Mutex + waiters map[string]map[chan struct{}]struct{} +} + +func (n *executionNotifications) subscribe(tenant, session string) (<-chan struct{}, func()) { + if n == nil { + return nil, func() {} + } + key := tenant + ":" + session + ch := make(chan struct{}, 1) + n.mu.Lock() + if n.waiters == nil { + n.waiters = make(map[string]map[chan struct{}]struct{}) + } + if n.waiters[key] == nil { + n.waiters[key] = make(map[chan struct{}]struct{}) + } + n.waiters[key][ch] = struct{}{} + n.mu.Unlock() + return ch, func() { + n.mu.Lock() + defer n.mu.Unlock() + delete(n.waiters[key], ch) + if len(n.waiters[key]) == 0 { + delete(n.waiters, key) + } + } +} + +func (n *executionNotifications) notify(tenant, session string) { + if n == nil { + return + } + n.mu.Lock() + defer n.mu.Unlock() + for ch := range n.waiters[tenant+":"+session] { + select { + case ch <- struct{}{}: + default: + } + } +} diff --git a/services/agents-api/internal/execution/notifications_test.go b/services/agents-api/internal/execution/notifications_test.go new file mode 100644 index 000000000..b39588fc9 --- /dev/null +++ b/services/agents-api/internal/execution/notifications_test.go @@ -0,0 +1,50 @@ +package execution + +import ( + "sync" + "testing" +) + +func TestExecutionNotificationsKeepCommitHintsAndIsolateSessions(t *testing.T) { + notifications := &executionNotifications{} + one, leaveOne := notifications.subscribe("tenant", "one") + other, leaveOther := notifications.subscribe("other-tenant", "one") + defer leaveOther() + var writers sync.WaitGroup + for range 20 { + writers.Go(func() { notifications.notify("tenant", "one") }) + } + writers.Wait() + select { + case <-one: + default: + t.Fatal("committed change was lost") + } + select { + case <-other: + t.Fatal("hint crossed tenant ownership") + default: + } + select { + case <-one: + t.Fatal("hints did not coalesce") + default: + } + notifications.notify("tenant", "one") + select { + case <-one: + default: + t.Fatal("next commit was lost") + } + leaveOne() + notifications.notify("tenant", "one") + select { + case <-one: + t.Fatal("detached observer received a hint") + default: + } + leaveOther() + if len(notifications.waiters) != 0 { + t.Fatal("subscriptions leaked") + } +} diff --git a/services/agents-api/internal/execution/preparation.go b/services/agents-api/internal/execution/preparation.go index 8f1706d3d..f33db9a38 100644 --- a/services/agents-api/internal/execution/preparation.go +++ b/services/agents-api/internal/execution/preparation.go @@ -11,11 +11,23 @@ import ( "github.com/google/uuid" ) +type preparationRejection struct { + code string + operation string +} + +func (e *preparationRejection) Error() string { return "preparation control rejected: " + e.code } + type preparedStart struct { - peer *gateway.Session - requestID string - handle string - sub *gateway.Subscription + createdAt time.Time + startSentAt time.Time + startObserved bool + readyObserved bool + peer *gateway.Session + requestID string + handle string + executorID string + sub *gateway.Subscription } func newPreparedStart(peer *gateway.Session) (*preparedStart, error) { @@ -24,7 +36,7 @@ func newPreparedStart(peer *gateway.Session) (*preparedStart, error) { if err != nil { return nil, err } - return &preparedStart{peer: peer, requestID: id, sub: sub}, nil + return &preparedStart{peer: peer, requestID: id, sub: sub, createdAt: time.Now()}, nil } func (p *preparedStart) close() { @@ -45,7 +57,14 @@ func (p *preparedStart) controlStatus(env proto.Envelope) (proto.PreparationStat if status.Handle == "" || status.Revision == 0 || (p.handle != "" && p.handle != status.Handle) { return status, errors.New("preparation identity changed") } - p.handle = status.Handle + if p.executorID != "" && status.ExecutorID != p.executorID { + return status, errors.New("executor identity changed") + } + p.handle, p.executorID = status.Handle, status.ExecutorID + if status.State == "ready" && status.ExecutorID != "" && !p.readyObserved { + p.readyObserved = true + recordExecutorReadiness(p, status) + } return status, nil } @@ -55,7 +74,7 @@ func (p *preparedStart) observation(env proto.Envelope) (proto.PreparationStatus return status, err } if status.State == "rejected" { - return status, errors.New("preparation control rejected") + return status, &preparationRejection{code: status.ErrorCode, operation: status.Operation} } switch status.State { case "preparing", "ready", "starting", "started": @@ -85,7 +104,7 @@ func (d *Dispatcher) awaitPreparation(ctx context.Context, tenant, session strin if err != nil { return pending, err } - if status.State == "ready" && status.RunID == "" { + if status.State == "ready" && status.RunID == "" && status.ExecutorID != "" { return pending, nil } if status.State != "preparing" { @@ -96,7 +115,8 @@ func (d *Dispatcher) awaitPreparation(ctx context.Context, tenant, session strin } func (p *preparedStart) start(ctx context.Context, request proto.PromptRequestPayload) error { - return send(ctx, p.peer, proto.TypeExecutionStart, p.requestID, proto.ExecutionStartPayload{Handle: p.handle, RunID: request.RunID, Input: request.Input}) + p.startSentAt = time.Now() + return send(ctx, p.peer, proto.TypeExecutionStart, p.requestID, proto.ExecutionStartPayload{Handle: p.handle, ExecutorID: p.executorID, RunID: request.RunID, Input: request.Input}) } func (p *preparedStart) started(env proto.Envelope, runID string) (bool, error) { @@ -107,5 +127,9 @@ func (p *preparedStart) started(env proto.Envelope, runID string) (bool, error) if status.RunID != runID || (status.State != "starting" && status.State != "started") { return false, errors.New("unexpected preparation state after admission") } + if status.State == "started" && !p.startObserved { + p.startObserved = true + recordExecutorStart(p, runID) + } return status.State == "started", nil } diff --git a/services/agents-api/internal/execution/prepared_dispatch.go b/services/agents-api/internal/execution/prepared_dispatch.go index f13d3f48d..c03aa2bb0 100644 --- a/services/agents-api/internal/execution/prepared_dispatch.go +++ b/services/agents-api/internal/execution/prepared_dispatch.go @@ -16,7 +16,7 @@ type EnvironmentRun struct { Turn store.Turn } -// RunEnvironmentInput owns a private preparation through its first admitted Run. +// RunEnvironmentInput reserves a Turn on the Session-owned Runtime Executor. func (d *Dispatcher) RunEnvironmentInput(ctx context.Context, tenantID, sessionID, reservationID string) (run EnvironmentRun, err error) { if err = d.Store.CheckExecutionOwnership(ctx); err != nil { return run, err @@ -81,7 +81,7 @@ func (d *Dispatcher) RunEnvironmentInput(ctx context.Context, tenantID, sessionI return run, err } defer prepared.close() - if err = send(owner, peer, proto.TypeExecutionPrepare, prepared.requestID, proto.ExecutionPreparePayload{Configuration: req}); err != nil { + if err = send(owner, peer, proto.TypeExecutionPrepare, prepared.requestID, proto.ExecutionPreparePayload{SessionID: sessionID, Configuration: req}); err != nil { return run, err } run.Reservation, err = d.awaitPreparation(owner, tenantID, sessionID, run.Reservation, prepared) @@ -96,6 +96,9 @@ func (d *Dispatcher) RunEnvironmentInput(ctx context.Context, tenantID, sessionI // A rejected claim leaves the reservation pending for a later attempt. return run, err } + if err == nil { + d.notifications.notify(tenantID, sessionID) + } run.Reservation = promoted if err != nil || run.Reservation.State != store.EnvironmentInputAdmitted { return run, err diff --git a/services/agents-api/internal/execution/request.go b/services/agents-api/internal/execution/request.go index 8d398e622..000435f17 100644 --- a/services/agents-api/internal/execution/request.go +++ b/services/agents-api/internal/execution/request.go @@ -52,7 +52,7 @@ func (d *Dispatcher) executionRequest(ctx context.Context, session store.Session } request := proto.PromptRequestPayload{AgentKind: session.Engine, FunctionTools: tools.Functions, ToolSearch: tools.Search, AgentOptions: options, ExecutionControls: controls, AgentStateKey: "agents-api-" + session.ID, - AgentSessionID: bound.NativeSessionID, ReleaseOnCompletion: true, StrictResume: true, + AgentSessionID: bound.NativeSessionID, StrictResume: true, RequireExistingNativeSession: recoverNativeSession, ObserveMessages: caps.MessageItems, ObserveToolObservations: true, ObserveSubagentIdentities: snapshot.Agent.MultiAgent.Enabled, diff --git a/services/agents-api/internal/execution/runtime_capabilities_test.go b/services/agents-api/internal/execution/runtime_capabilities_test.go new file mode 100644 index 000000000..4e6fb9030 --- /dev/null +++ b/services/agents-api/internal/execution/runtime_capabilities_test.go @@ -0,0 +1,85 @@ +package execution + +import ( + "bytes" + "context" + "errors" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/gateway" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" + "github.com/google/uuid" + "testing" +) + +type capabilityFixture struct { + request proto.RuntimePreparePayload + body []byte + requestID string + outcome string + err error +} + +func (f *capabilityFixture) PrepareRuntime(_ context.Context, id string, request proto.RuntimePreparePayload, body []byte) (proto.RuntimePrepareResultPayload, error) { + f.requestID, f.request, f.body = id, request, append([]byte(nil), body...) + return proto.RuntimePrepareResultPayload{Outcome: f.outcome, ErrorCode: "safe_failure"}, f.err +} +func TestRuntimeCapabilitiesPreserveRawBundlesAndSetupOrdering(t *testing.T) { + archive := []byte("opaque archive bytes must be expanded only by Runtime") + setup := store.EnvironmentSetup{ + Skills: []store.EnvironmentSkill{{Metadata: store.EnvironmentSkillMetadata{Type: "skill_reference", SkillID: "private-reference", Version: "1", Name: "example", Description: "Safe description"}, Archive: archive}}, + Plugins: []store.EnvironmentPlugin{{Metadata: agentplugin.Metadata{Type: "inline", Name: "plugin", Description: "Safe plugin"}, Archive: archive}}, + Commands: []store.SetupCommand{{Command: "prepare directory"}}, + CapabilityDirectories: []string{"/workspace/generated"}, + } + operations := setupOperations(setup) + if len(operations) != 5 || operations[0].Request.Initialization.Action != "configure" || operations[1].Request.Action != "skill" || operations[2].Request.Action != "plugin" || operations[3].Request.Initialization.Action != "setup" || operations[4].Request.Action != "finalize" { + t.Fatal("bundle-before-setup or directory-after-setup ordering changed", operations) + } + owner := agentcapabilities.Identity{EnvironmentID: uuid.NewString(), SessionID: uuid.NewString()} + for _, op := range operations { + peer := &capabilityFixture{outcome: "completed"} + if err := runRuntimeSetup(t.Context(), peer, owner, op); err != nil { + t.Fatal(err) + } + if _, err := uuid.Parse(peer.requestID); err != nil { + t.Fatal("request ID", err) + } + if peer.request.EnvironmentID != owner.EnvironmentID || peer.request.SessionID != owner.SessionID { + t.Fatal("allocation identity lost") + } + if (op.Request.Action == "skill" || op.Request.Action == "plugin") && !bytes.Equal(peer.body, archive) { + t.Fatal("Core expanded or changed archive") + } + if op.Request.Action == "skill" && (peer.request.Skill.Type != "inline" || peer.request.Skill.Name != "example") { + t.Fatal("unsafe or unresolved metadata") + } + if op.Request.Action == "finalize" && (len(peer.body) != 0 || peer.request.Sources == nil || len(peer.request.Sources.Skills) != 1 || len(peer.request.Sources.Plugins) != 1 || peer.request.Sources.Directories[0] != "/workspace/generated") { + t.Fatal("finalization selections changed") + } + + } +} +func TestRuntimeCapabilitiesConfirmedAndUnknownFailures(t *testing.T) { + for _, outcome := range []string{"failed", "rejected", "unknown", "unexpected"} { + peer := &capabilityFixture{outcome: outcome} + err := runRuntimeSetup(t.Context(), peer, agentcapabilities.Identity{}, runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: "finalize"}}) + var confirmed *runtimeStepFailure + if err == nil || errors.As(err, &confirmed) != (outcome == "failed" || outcome == "rejected") { + t.Fatal(outcome, err) + } + } + peer := &capabilityFixture{outcome: "completed", err: errors.New(setupCanary)} + err := runRuntimeSetup(t.Context(), peer, agentcapabilities.Identity{}, runtimeSetupOperation{Request: proto.RuntimePreparePayload{}}) + if err == nil || bytes.Contains([]byte(err.Error()), []byte(setupCanary)) { + t.Fatal("transport error leaked or succeeded", err) + } +} +func TestRuntimeInitializationWaitsForMissingSocket(t *testing.T) { + lifecycle := runtimeLifecycle{registry: gateway.NewRegistry()} + peer, err := lifecycle.initializationPeer(t.Context(), store.RuntimeAllocation{DeviceID: uuid.NewString()}) + if err != nil || peer != nil || lifecycle.initializing != nil { + t.Fatal("missing socket became initialization failure", peer, err) + } +} diff --git a/services/agents-api/internal/execution/runtime_initialization.go b/services/agents-api/internal/execution/runtime_initialization.go index b04fdec28..cea3198ef 100644 --- a/services/agents-api/internal/execution/runtime_initialization.go +++ b/services/agents-api/internal/execution/runtime_initialization.go @@ -2,13 +2,12 @@ package execution import ( "context" - "crypto/sha256" "encoding/json" "errors" - "strconv" - "strings" "time" + "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/gateway" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" ) @@ -57,6 +56,11 @@ func (r *runtimeLifecycle) observeInitialization(ctx context.Context, owner stor _, err = r.store.RequestRuntimeCleanup(ctx, owner) return err } + // No initialization is claimed before the authenticated Runtime is available. + peer, err := r.initializationPeer(ctx, owner) + if err != nil || peer == nil { + return err + } claimed, err := r.store.ClaimRuntimeInitialization(ctx, owner) if err != nil { return err @@ -97,18 +101,23 @@ func (r *runtimeLifecycle) advanceInitialization(ctx context.Context) error { r.initializing = nil return err } + peer, err := r.initializationPeer(operation, owner) + if err == nil && peer == nil { + return nil + } + identity := agentcapabilities.Identity{EnvironmentID: owner.EnvironmentID, SessionID: owner.SessionID} step := store.ProvisioningFailure{Step: store.ProvisioningInitialFile} - if active.next < active.files { + if err == nil && active.next < active.files { var file store.InitialFileMetadata var body []byte file, body, err = r.store.ReadInitialEnvironmentFile(operation, owner.TenantID, owner.SessionID, active.next) if err == nil { - err = installInitialFile(operation, r.config.Provider, runtimeReference(owner), file, body) + err = installInitialFile(operation, peer, identity, file, body) } - } else { + } else if err == nil { setup := active.operations[active.next-active.files] step = setup.provisioningFailure(0) - err = runRuntimeSetup(operation, r.config.Provider, runtimeReference(owner), setup) + err = runRuntimeSetup(operation, peer, identity, setup) } if err != nil { // Clearing the in-memory owner makes the next observation request cleanup, @@ -136,54 +145,15 @@ func (r *runtimeLifecycle) advanceInitialization(ctx context.Context) error { return nil } -func installInitialFile(ctx context.Context, provider sandbox.Provider, reference sandbox.Reference, file store.InitialFileMetadata, body []byte) error { - if provider == nil || file.SizeBytes == nil || *file.SizeBytes != int64(len(body)) || len(body) > store.MaxInitialFileBytes || !strings.HasPrefix(file.Path, "/workspace/") { - return sandbox.ErrInvalid - } - digest := sha256.Sum256(body) - input := make([]byte, 0, len(body)+len(digest)) - input = append(input, body...) - input = append(input, digest[:]...) - result, err := provider.RunCommand(ctx, reference, sandbox.Command{Directory: "/", Args: []string{"/usr/bin/python3", "-I", "-S", "-c", initialFileInstaller, strings.TrimPrefix(file.Path, "/workspace/"), strconv.Itoa(len(body))}, Stdin: input}) - if err != nil { - return err - } - var receipt struct { - Version int `json:"version"` - Outcome string `json:"outcome"` - SizeBytes *int64 `json:"size_bytes"` +// Missing authority/socket before sending consumes no initialization operation. +// Once PrepareRuntime is called, unknown effects use the existing cleanup. +func (r *runtimeLifecycle) initializationPeer(ctx context.Context, owner store.RuntimeAllocation) (*gateway.Session, error) { + if r.registry == nil { + return nil, nil } - valid := result.Stderr == "" && json.Unmarshal([]byte(result.Stdout), &receipt) == nil && receipt.Version == 1 - if valid && result.ExitCode == 0 && receipt.Outcome == "completed" && receipt.SizeBytes != nil && *receipt.SizeBytes == int64(len(body)) { - return nil + peer, err := authorizedRuntimePeer(ctx, r.store, r.registry, owner.DeviceID) + if errors.Is(err, store.ErrNotFound) || errors.Is(err, gateway.ErrSessionClosed) || errors.Is(err, gateway.ErrDeviceNotRegistered) { + return nil, nil } - if valid && receipt.Outcome == "failed" { - // The writer exits 0 with a failed receipt when it committed nothing; - // "unknown" and every other result stay generic. - return &runtimeStepFailure{} - } - return errors.New("initial environment file installation unconfirmed") + return peer, err } - -// Isolated Python creates only fd-anchored workspace parents, then replaces itself with the existing atomic writer. -const initialFileInstaller = `import os, sys -parts = sys.argv[1].split('/') -if any(not p or p in ('.', '..') or any(c in p for c in ('\\', '\x00', '\r', '\n')) for p in parts): - raise SystemExit(2) -flags = os.O_RDONLY | os.O_DIRECTORY | os.O_NOFOLLOW | os.O_CLOEXEC -fd = os.open('/', flags) -for component in ('environment', 'workspace'): - child = os.open(component, flags, dir_fd=fd) - os.close(fd) - fd = child -for component in parts[:-1]: - try: - os.mkdir(component, mode=0o700, dir_fd=fd) - except FileExistsError: - pass - child = os.open(component, flags, dir_fd=fd) - os.close(fd) - fd = child -os.close(fd) -os.execv('/usr/local/bin/oac-codex-write', ['oac-codex-write', '/environment/workspace', sys.argv[1], sys.argv[2], '/environment/staging']) -` diff --git a/services/agents-api/internal/execution/runtime_setup.go b/services/agents-api/internal/execution/runtime_setup.go index 584fb9156..4225f3280 100644 --- a/services/agents-api/internal/execution/runtime_setup.go +++ b/services/agents-api/internal/execution/runtime_setup.go @@ -2,46 +2,34 @@ package execution import ( "context" - "encoding/json" "errors" - "github.com/MiniMax-AI-Dev/parsar/internal/agentbundle" "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" - "github.com/MiniMax-AI-Dev/parsar/internal/agentskill" - "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" + "github.com/google/uuid" ) -// runtimeSetupOperation is the packaged initializer's confidential stdin contract. -// Public templates and native harness configuration never cross this boundary. +// runtimeSetupOperation carries one frozen, typed Runtime operation. Runtime owns +// host paths, executable selection and native installation receipts. type runtimeSetupOperation struct { - Capabilities *agentcapabilities.Operation `json:"-"` - Skill *store.EnvironmentSkill `json:"-"` - Name string `json:"name,omitempty"` - Files []agentbundle.File `json:"files,omitempty"` - Version int `json:"version"` - Action string `json:"action"` - Network string `json:"network,omitempty"` - Env map[string]string `json:"env"` - Packages []string `json:"packages,omitempty"` - Command string `json:"command,omitempty"` - CWD string `json:"cwd,omitempty"` - // Index is the setup command position, used only for the public failure label. - Index int `json:"-"` + Request proto.RuntimePreparePayload + Data []byte + Index int } -// runtimeStepFailure is a confirmed failed initialization receipt. It holds only -// the Runtime-reported exit status (0 when absent), never command or output. type runtimeStepFailure struct{ exitCode int } func (*runtimeStepFailure) Error() string { return "environment initialization operation failed" } -// provisioningFailure labels a confirmed failed setup operation for the Store, -// which composes the public reason. Other actions keep the generic reason. func (operation runtimeSetupOperation) provisioningFailure(exitCode int) store.ProvisioningFailure { - switch operation.Action { + action := operation.Request.Action + if operation.Request.Initialization != nil { + action = operation.Request.Initialization.Action + } + switch action { case "setup", "python", "npm", "system", "skill": - return store.ProvisioningFailure{Step: operation.Action, Index: operation.Index, ExitCode: exitCode} + return store.ProvisioningFailure{Step: action, Index: operation.Index, ExitCode: exitCode} } return store.ProvisioningFailure{} } @@ -50,88 +38,68 @@ func setupOperations(setup store.EnvironmentSetup) []runtimeSetupOperation { if setup.Empty() { return nil } + initialize := func(input proto.RuntimeInitialization) runtimeSetupOperation { + return runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: "initialize", Initialization: &input}} + } env := setup.Env if env == nil { env = map[string]string{} } - result := []runtimeSetupOperation{{Version: 1, Action: "configure", Env: env}} - for i := range setup.Skills { - result = append(result, runtimeSetupOperation{Version: 1, Action: "skill", Skill: &setup.Skills[i]}) + result := []runtimeSetupOperation{initialize(proto.RuntimeInitialization{Action: "configure", Env: env})} + for _, skill := range setup.Skills { + metadata := skill.InstallationMetadata() + result = append(result, runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: "skill", Skill: &metadata}, Data: skill.Archive}) } for i, plugin := range setup.Plugins { - result = append(result, runtimeSetupOperation{Capabilities: &agentcapabilities.Operation{Version: 1, Action: "plugin", Slot: i, Archive: plugin.Archive, Plugin: plugin.Metadata}}) - } - // The public network policy applies after setup completes. Provisioning uses - // the isolated initializer's network; adapters enforce the runtime policy. - const network = "enabled" - if len(setup.Packages.System) > 0 { - result = append(result, runtimeSetupOperation{Version: 1, Action: "system", Network: network, Packages: setup.Packages.System}) - } - if len(setup.Packages.NPM) > 0 { - result = append(result, runtimeSetupOperation{Version: 1, Action: "npm", Network: network, Packages: setup.Packages.NPM}) - } - if len(setup.Packages.Python) > 0 { - result = append(result, runtimeSetupOperation{Version: 1, Action: "python", Network: network, Packages: setup.Packages.Python}) + result = append(result, runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: "plugin", Slot: i, Plugin: &plugin.Metadata}, Data: plugin.Archive}) + } + // Provisioning network policy is distinct from the policy enforced for Turns. + for _, packages := range []struct { + action string + values []string + }{{"system", setup.Packages.System}, {"npm", setup.Packages.NPM}, {"python", setup.Packages.Python}} { + if len(packages.values) > 0 { + result = append(result, initialize(proto.RuntimeInitialization{Action: packages.action, Network: "enabled", Packages: packages.values})) + } } for i, command := range setup.Commands { - cwd := command.CWD - if cwd == "" { - cwd = "/workspace" - } - result = append(result, runtimeSetupOperation{Version: 1, Action: "setup", Network: network, Command: command.Command, CWD: cwd, Index: i}) + operation := initialize(proto.RuntimeInitialization{Action: "setup", Network: "enabled", Command: command.Command, CWD: command.CWD}) + operation.Index = i + result = append(result, operation) } if len(setup.Skills)+len(setup.Plugins)+len(setup.CapabilityDirectories) > 0 { sources := agentcapabilities.Input{Plugins: setup.PluginMetadata(), Directories: setup.CapabilityDirectories} for _, skill := range setup.Skills { sources.Skills = append(sources.Skills, skill.InstallationMetadata()) } - result = append(result, runtimeSetupOperation{Capabilities: &agentcapabilities.Operation{Version: 1, Action: "finalize", Sources: sources}}) + result = append(result, runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: "finalize", Sources: &sources}}) } return result } -func runRuntimeSetup(ctx context.Context, provider sandbox.Provider, reference sandbox.Reference, operation runtimeSetupOperation) error { - if provider == nil { - return sandbox.ErrInvalid - } - if operation.Skill != nil { - files, err := agentskill.Read(operation.Skill.Archive, operation.Skill.InstallationMetadata()) - if err != nil { - return err - } - operation.Name, operation.Files = operation.Skill.Metadata.Name, files - } - var payload any = operation - args := []string{"/usr/bin/python3", "-I", "-S", "/usr/local/bin/oac-runtime-initialize"} - if operation.Capabilities != nil { - payload = operation.Capabilities - args = []string{"/usr/local/bin/oac-daemon", "runtime-capabilities"} - } - input, err := json.Marshal(payload) - if err != nil { - return err - } - result, err := provider.RunCommand(ctx, reference, sandbox.Command{Directory: "/", Args: args, Stdin: input}) - if err != nil { - return err - } - var receipt struct { - Version int `json:"version"` - Outcome string `json:"outcome"` - ExitCode *int `json:"exit_code"` - } - valid := result.Stderr == "" && json.Unmarshal([]byte(result.Stdout), &receipt) == nil && receipt.Version == 1 - if valid && result.ExitCode == 0 && receipt.Outcome == "completed" { +type runtimePreparer interface { + PrepareRuntime(context.Context, string, proto.RuntimePreparePayload, []byte) (proto.RuntimePrepareResultPayload, error) +} + +func runRuntimeSetup(ctx context.Context, peer runtimePreparer, identity agentcapabilities.Identity, operation runtimeSetupOperation) error { + if peer == nil { + return errors.New("environment initialization request unavailable") + } + request := operation.Request + request.EnvironmentID, request.SessionID = identity.EnvironmentID, identity.SessionID + result, err := peer.PrepareRuntime(ctx, uuid.NewString(), request, operation.Data) + if err == nil && result.Outcome == "completed" { return nil } - // The initializer confirms a failed step with status 1 and, for a sandboxed - // step, that step's exit status only. Images without exit_code stay generic. - if valid && result.ExitCode == 1 && receipt.Outcome == "failed" && operation.Capabilities == nil { - failure := &runtimeStepFailure{} - if receipt.ExitCode != nil && *receipt.ExitCode > 0 && *receipt.ExitCode < 256 { - failure.exitCode = *receipt.ExitCode - } - return failure + if err == nil && (result.Outcome == "rejected" || result.Outcome == "failed") { + return &runtimeStepFailure{exitCode: result.ExitCode} } return errors.New("environment initialization operation unconfirmed") } + +func installInitialFile(ctx context.Context, peer runtimePreparer, identity agentcapabilities.Identity, file store.InitialFileMetadata, body []byte) error { + if file.SizeBytes == nil || *file.SizeBytes != int64(len(body)) || len(body) > store.MaxInitialFileBytes { + return errors.New("environment initialization request unavailable") + } + return runRuntimeSetup(ctx, peer, identity, runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: "file", File: &proto.RuntimeInitialFile{Path: file.Path}}, Data: body}) +} diff --git a/services/agents-api/internal/execution/runtime_setup_test.go b/services/agents-api/internal/execution/runtime_setup_test.go index b64d0a0aa..9a8a44bb2 100644 --- a/services/agents-api/internal/execution/runtime_setup_test.go +++ b/services/agents-api/internal/execution/runtime_setup_test.go @@ -7,129 +7,79 @@ import ( "testing" "github.com/MiniMax-AI-Dev/parsar/internal/agentcapabilities" - "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" ) const setupCanary = "CANARY-runtime-setup-9b3e" -// receiptProvider answers every initialization command with one fixed result. -type receiptProvider struct { - sandbox.Provider - result sandbox.CommandResult - err error +type receiptRuntime struct { + result proto.RuntimePrepareResultPayload + err error + request proto.RuntimePreparePayload + data []byte } -func (p receiptProvider) RunCommand(context.Context, sandbox.Reference, sandbox.Command) (sandbox.CommandResult, error) { +func (p *receiptRuntime) PrepareRuntime(_ context.Context, _ string, request proto.RuntimePreparePayload, data []byte) (proto.RuntimePrepareResultPayload, error) { + p.request, p.data = request, data return p.result, p.err } - -// Only a process status 1, empty stderr and a version-1 failed receipt confirm a -// failed step, and only an integer exit_code from 1 to 255 is taken from it. -// Other receipt fields are ignored, never read; everything else stays generic. func TestRuntimeSetupReceiptOutcomes(t *testing.T) { - setup := runtimeSetupOperation{Version: 1, Action: "setup", Network: "enabled", Command: "echo " + setupCanary, CWD: "/workspace", Index: 2} - plugin := runtimeSetupOperation{Capabilities: &agentcapabilities.Operation{Version: 1, Action: "plugin"}} - unconfirmed := -1 for _, test := range []struct { - name string - operation runtimeSetupOperation - result sandbox.CommandResult - err error - exitCode int // -1 unconfirmed, 0 confirmed without a status + name, outcome string + code int + err error + confirmed bool }{ - {"completed", setup, sandbox.CommandResult{Stdout: `{"version":1,"outcome":"completed"}`}, nil, -2}, - {"exit status", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3}`}, nil, 3}, - {"highest exit status", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":255}` + "\n"}, nil, 255}, - {"old receipt", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed"}`}, nil, 0}, - {"zero exit status", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":0}`}, nil, 0}, - {"exit status above 255", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":256}`}, nil, 0}, - {"negative exit status", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":-9}`}, nil, 0}, - {"null exit status", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":null}`}, nil, 0}, - // Lenient for older and newer images: other fields are never read. - {"ignored output field", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3,"output":"` + setupCanary + `"}`}, nil, 3}, - {"duplicate key keeps the last", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3,"exit_code":4}`}, nil, 4}, - {"string exit status", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":"3"}`}, nil, unconfirmed}, - {"fractional exit status", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3.5}`}, nil, unconfirmed}, - {"stderr", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3}`, Stderr: setupCanary}, nil, unconfirmed}, - {"process status 2", setup, sandbox.CommandResult{ExitCode: 2, Stdout: `{"version":1,"outcome":"failed","exit_code":3}`}, nil, unconfirmed}, - {"process status 0", setup, sandbox.CommandResult{Stdout: `{"version":1,"outcome":"failed","exit_code":3}`}, nil, unconfirmed}, - {"completed with status 1", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"completed"}`}, nil, unconfirmed}, - {"version 2", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":2,"outcome":"failed","exit_code":3}`}, nil, unconfirmed}, - {"raw output", setup, sandbox.CommandResult{ExitCode: 3, Stdout: setupCanary}, nil, unconfirmed}, - {"trailing output", setup, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3}` + setupCanary}, nil, unconfirmed}, - {"provider error", setup, sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed, unconfirmed}, - {"plugin step", plugin, sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3}`}, nil, unconfirmed}, + {"completed", "completed", 0, nil, false}, {"failed", "failed", 3, nil, true}, {"failed without status", "failed", 0, nil, true}, {"maximum status", "failed", 255, nil, true}, {"rejected", "rejected", 0, nil, true}, {"unknown", "unknown", 0, nil, false}, {"transport", "completed", 0, errors.New(setupCanary), false}, } { t.Run(test.name, func(t *testing.T) { - err := runRuntimeSetup(t.Context(), receiptProvider{result: test.result, err: test.err}, sandbox.Reference{}, test.operation) - var failed *runtimeStepFailure - switch { - case test.exitCode == -2: + peer := &receiptRuntime{result: proto.RuntimePrepareResultPayload{Outcome: test.outcome, ExitCode: test.code}, err: test.err} + err := runRuntimeSetup(t.Context(), peer, agentcapabilities.Identity{}, runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: "initialize", Initialization: &proto.RuntimeInitialization{Action: "setup", Command: setupCanary}}}) + if test.outcome == "completed" && test.err == nil { if err != nil { t.Fatal(err) } - case test.exitCode == unconfirmed: - if err == nil || errors.As(err, &failed) { - t.Fatal("unconfirmed result became a confirmed failure", err) - } - default: - if !errors.As(err, &failed) || failed.exitCode != test.exitCode { - t.Fatal("confirmed failure", err, failed) - } + return } - if err != nil && strings.Contains(err.Error(), setupCanary) { - t.Fatal("receipt or output reached the error", err) + var failed *runtimeStepFailure + if err == nil || errors.As(err, &failed) != test.confirmed || strings.Contains(err.Error(), setupCanary) { + t.Fatal("unsafe outcome", err) + } + if failed != nil && failed.exitCode != test.code { + t.Fatal("exit status lost") } }) } } - func TestRuntimeSetupFailureLabels(t *testing.T) { - for operation, want := range map[string]store.ProvisioningFailure{ - "setup": {Step: store.ProvisioningSetupCommand, Index: 2, ExitCode: 3}, - "python": {Step: store.ProvisioningPythonPackages, Index: 2, ExitCode: 3}, - "npm": {Step: store.ProvisioningNPMPackages, Index: 2, ExitCode: 3}, - "system": {Step: store.ProvisioningSystemPackages, Index: 2, ExitCode: 3}, - "skill": {Step: store.ProvisioningSkill, Index: 2, ExitCode: 3}, - "configure": {}, - "": {}, - } { - if got := (runtimeSetupOperation{Action: operation, Index: 2}).provisioningFailure(3); got != want { - t.Errorf("%q: %+v", operation, got) + for action, want := range map[string]store.ProvisioningFailure{"setup": {Step: store.ProvisioningSetupCommand, Index: 2, ExitCode: 3}, "python": {Step: store.ProvisioningPythonPackages, Index: 2, ExitCode: 3}, "npm": {Step: store.ProvisioningNPMPackages, Index: 2, ExitCode: 3}, "system": {Step: store.ProvisioningSystemPackages, Index: 2, ExitCode: 3}, "skill": {Step: store.ProvisioningSkill, Index: 2, ExitCode: 3}, "configure": {}, "": {}} { + op := runtimeSetupOperation{Request: proto.RuntimePreparePayload{Action: action}, Index: 2} + if action != "skill" { + op.Request = proto.RuntimePreparePayload{Action: "initialize", Initialization: &proto.RuntimeInitialization{Action: action}} + } + if got := op.provisioningFailure(3); got != want { + t.Fatal(action, got) } } - // setupOperations numbers setup commands from zero. operations := setupOperations(store.EnvironmentSetup{Commands: []store.SetupCommand{{Command: "a"}, {Command: "b"}}}) - if len(operations) != 3 || operations[1].Index != 0 || operations[2].Index != 1 || operations[2].Action != "setup" { - t.Fatal("setup command positions", operations) + if len(operations) != 3 || operations[1].Index != 0 || operations[2].Index != 1 || operations[2].Request.Initialization.CWD != "" { + t.Fatal("command index or Runtime default changed") } } - -// The file writer exits 0 with a failed receipt when it committed nothing; an -// unknown outcome, stderr or a wrong size stays unconfirmed. -func TestInitialFileReceiptOutcomes(t *testing.T) { +func TestInitialFileUsesTypedRuntimeBytes(t *testing.T) { body := []byte(setupCanary) - file := store.InitialFileMetadata{Path: "/workspace/a"} size := int64(len(body)) - file.SizeBytes = &size - for _, test := range []struct { - name string - result sandbox.CommandResult - confirmed bool - }{ - {"failed", sandbox.CommandResult{Stdout: `{"version":1,"outcome":"failed","error":"write_failed"}`}, true}, - {"unknown", sandbox.CommandResult{Stdout: `{"version":1,"outcome":"unknown","error":"write_failed"}`}, false}, - {"stderr", sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed"}`, Stderr: setupCanary}, false}, - {"wrong size", sandbox.CommandResult{Stdout: `{"version":1,"outcome":"completed","size_bytes":1}`}, false}, - {"raw output", sandbox.CommandResult{ExitCode: 2, Stdout: setupCanary}, false}, - } { - t.Run(test.name, func(t *testing.T) { - err := installInitialFile(t.Context(), receiptProvider{result: test.result}, sandbox.Reference{}, file, body) - var failed *runtimeStepFailure - if err == nil || errors.As(err, &failed) != test.confirmed || strings.Contains(err.Error(), setupCanary) { - t.Fatal(err) - } - }) + owner := agentcapabilities.Identity{EnvironmentID: "environment", SessionID: "session"} + peer := &receiptRuntime{result: proto.RuntimePrepareResultPayload{Outcome: "completed"}} + if err := installInitialFile(t.Context(), peer, owner, store.InitialFileMetadata{Path: "/workspace/a", SizeBytes: &size}, body); err != nil { + t.Fatal(err) + } + if peer.request.Action != "file" || peer.request.File.Path != "/workspace/a" || peer.request.EnvironmentID != owner.EnvironmentID || peer.request.SessionID != owner.SessionID || string(peer.data) != setupCanary { + t.Fatal("file transport changed") + } + size++ + if err := installInitialFile(t.Context(), peer, owner, store.InitialFileMetadata{SizeBytes: &size}, body); err == nil { + t.Fatal("mismatched source size accepted") } } diff --git a/services/agents-api/internal/execution/support.go b/services/agents-api/internal/execution/support.go index c00dbc5cf..9982f8f81 100644 --- a/services/agents-api/internal/execution/support.go +++ b/services/agents-api/internal/execution/support.go @@ -26,7 +26,7 @@ func (p Policy) ValidateSessionConfiguration(engine string, configuration json.R return err } if snapshot.Environment != nil && snapshot.Environment.Type == "self_hosted" { - if snapshot.Daemon != nil || strings.TrimSpace(snapshot.Agent.Model) == "" || snapshot.Environment.WorkspaceDirectory != "/workspace" || len(snapshot.Environment.CapabilityDirectories) != 0 { + if snapshot.Daemon != nil || strings.TrimSpace(snapshot.Agent.Model) == "" || !validSelfHostedPlacement(environmentPlacement{WorkspaceDirectory: snapshot.Environment.WorkspaceDirectory, CapabilityDirectories: snapshot.Environment.CapabilityDirectories}) { return store.ErrInvalidInput } } @@ -86,6 +86,9 @@ func (p Policy) engineCapabilities(peer *gateway.Session, engine string, snapsho if !known || !found || !info.Available || !caps.Streaming || !caps.Steering || !caps.DurableTurns || !caps.DurableInputReceipts { return fail("device must advertise streaming, steering and durable turns for this engine") } + if !caps.Preparation { + return fail("device must advertise executor preparation") + } if !caps.ExecutionControls { return fail("device must advertise execution_controls") } diff --git a/services/agents-api/internal/execution/worker.go b/services/agents-api/internal/execution/worker.go index 832ab8229..57d3fabfa 100644 --- a/services/agents-api/internal/execution/worker.go +++ b/services/agents-api/internal/execution/worker.go @@ -7,6 +7,7 @@ import ( "sync" "time" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/MiniMax-AI-Dev/parsar/internal/obs/log" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" ) @@ -22,6 +23,7 @@ type Worker struct { lease *store.ExecutionLease directoryReads chan directoryReadRequest fileWrites chan fileWriteRequest + scheduleWake chan struct{} stopped chan struct{} stopOnce sync.Once runtimes *runtimeManager @@ -38,7 +40,8 @@ func StartWorker(ctx context.Context, dispatcher *Dispatcher) (*Worker, error) { } owned := *dispatcher owned.Store = lease.Store() - worker := &Worker{concurrency: dispatcher.MaxConcurrentExecutions, dispatcher: &owned, admission: dispatcher.Store, lease: lease, directoryReads: make(chan directoryReadRequest), fileWrites: make(chan fileWriteRequest), stopped: make(chan struct{}), enrolledConnections: make(map[string]*runtimeConnection)} + owned.notifications = &executionNotifications{} + worker := &Worker{concurrency: dispatcher.MaxConcurrentExecutions, dispatcher: &owned, admission: dispatcher.Store, lease: lease, directoryReads: make(chan directoryReadRequest), fileWrites: make(chan fileWriteRequest), stopped: make(chan struct{}), scheduleWake: make(chan struct{}, 1), enrolledConnections: make(map[string]*runtimeConnection)} worker.runtimes, err = newRuntimeManager(owned.Store, owned.Registry, owned.ManagedRuntimes) if err != nil { _ = lease.Close(context.Background()) @@ -105,7 +108,7 @@ func (w *Worker) SubmitInputs(ctx context.Context, tenant, session, key string, if !w.dispatcher.canAdmitInputs(value.Engine, value.Configuration) { return nil, store.ErrInvalidInput } - return w.admission.SubmitInputs(ctx, tenant, session, key, inputs) + return w.admitInputs(ctx, tenant, session, key, inputs) } // CreateSession validates execution support before reserving or admitting initial work. @@ -113,7 +116,11 @@ func (w *Worker) CreateSession(ctx context.Context, tenant string, input store.C if err := w.validateCreation(ctx, input); err != nil { return store.Session{}, err } - return w.admission.CreateSession(ctx, tenant, input) + session, err := w.admission.CreateSession(ctx, tenant, input) + if err == nil && len(input.InitialInputs) > 0 { + w.wakeScheduler() + } + return session, err } // CreateSessionStream applies the same execution admission before creating a stream. @@ -121,7 +128,11 @@ func (w *Worker) CreateSessionStream(ctx context.Context, tenant string, input s if err := w.validateCreation(ctx, input); err != nil { return store.SessionCreation{}, err } - return w.admission.CreateSessionStream(ctx, tenant, input) + creation, err := w.admission.CreateSessionStream(ctx, tenant, input) + if err == nil && len(input.InitialInputs) > 0 { + w.wakeScheduler() + } + return creation, err } // Run retains queued work across restarts, but never replays an uncertain claim. @@ -174,7 +185,12 @@ func (w *Worker) Run(ctx context.Context) (runErr error) { ticker := time.NewTicker(250 * time.Millisecond) defer ticker.Stop() schedule := workerSchedule{} + // A hint can encounter occupied slots or a preceding execution finishing + // on the same Session. Allow one immediate retry when a slot is released; + // failed preparations still wait for polling instead of spinning. + rescanOnCompletion := false for { + maintenance := false select { case <-ctx.Done(): return ctx.Err() @@ -192,10 +208,15 @@ func (w *Worker) Run(ctx context.Context) (runErr error) { defer running.Done() writesCompleted <- writeCompletion{request: request, result: w.runFileWrite(ctx, request)} }() + continue case write := <-writesCompleted: delete(active, write.request.environment.SessionID) w.observeSlots(len(active)) write.request.result <- write.result + if !rescanOnCompletion { + continue + } + rescanOnCompletion = false case request := <-w.directoryReads: if request.ctx.Err() != nil || reads == w.executionConcurrency() || (!active[request.environment.SessionID] && len(active) == w.executionConcurrency()) { request.reply(directoryReadResult{err: ErrExecutionUnavailable}) @@ -217,6 +238,7 @@ func (w *Worker) Run(ctx context.Context) (runErr error) { } readsCompleted <- readCompletion{id: id, request: request, result: result} }() + continue case read := <-readsCompleted: reads-- if read.id != "" { @@ -224,20 +246,33 @@ func (w *Worker) Run(ctx context.Context) (runErr error) { w.observeSlots(len(active)) } read.request.reply(read.result) + if read.id == "" || !rescanOnCompletion { + continue + } + rescanOnCompletion = false case result := <-completed: delete(active, result.id) w.observeSlots(len(active)) if result.err != nil { return result.err } - case <-ticker.C: - check, stop := context.WithTimeout(ctx, 5*time.Second) - err := w.CheckOwnership(check) - stop() - if err != nil { - w.observeSchedulerPoll(0, err) - return err + if !rescanOnCompletion { + continue } + rescanOnCompletion = false + case <-w.scheduleWake: + rescanOnCompletion = true + case <-ticker.C: + maintenance = true + } + check, stop := context.WithTimeout(ctx, 5*time.Second) + err := w.CheckOwnership(check) + stop() + if err != nil { + w.observeSchedulerPoll(0, err) + return err + } + if maintenance { if _, err := w.dispatcher.Store.ExpireEnvironmentInputs(ctx); err != nil { w.observeSchedulerPoll(0, err) return err @@ -246,35 +281,40 @@ func (w *Worker) Run(ctx context.Context) (runErr error) { w.observeSchedulerPoll(0, err) return err } - if len(active) == w.executionConcurrency() { - w.observeSchedulerPoll(0, nil) - continue - } - devices := w.dispatcher.Registry.Devices() - if len(devices) == 0 { - w.observeSchedulerPoll(0, nil) - continue - } - work, err := schedule.selectWork(ctx, w, devices, active) - w.observeSlots(len(active)) - if err != nil { - w.observeSchedulerPoll(0, err) - return err - } - w.observeSchedulerPoll(len(work), nil) - for _, item := range work { - running.Add(1) - go func() { - defer running.Done() - var err error - if item.reservationID != "" { - err = w.runEnvironmentInput(ctx, item) - } else { - err = w.runClaim(ctx, item.ExecutionWork) - } - completed <- completion{id: item.SessionID, err: err} - }() - } + } + if len(active) == w.executionConcurrency() { + w.observeSchedulerPoll(0, nil) + continue + } + devices := w.dispatcher.Registry.Devices() + if len(devices) == 0 { + w.observeSchedulerPoll(0, nil) + continue + } + if !maintenance { + schedule.nextEnvironmentScan = time.Time{} + } + work, err := schedule.selectWork(ctx, w, devices, active) + w.observeSlots(len(active)) + if err != nil { + w.observeSchedulerPoll(0, err) + return err + } + w.observeSchedulerPoll(len(work), nil) + rescanOnCompletion = rescanOnCompletion && (len(active) > len(work) || len(active) == w.executionConcurrency()) + for _, item := range work { + running.Add(1) + go func() { + defer running.Done() + var err error + if item.reservationID != "" { + err = w.runEnvironmentInput(ctx, item) + } else { + err = w.runClaim(ctx, item.ExecutionWork) + } + w.dispatcher.notifications.notify(item.TenantID, item.SessionID) + completed <- completion{id: item.SessionID, err: err} + }() } } } @@ -304,6 +344,8 @@ func (w *Worker) runClaim(ctx context.Context, item store.ExecutionWork) error { if err == nil || errors.Is(err, store.ErrTurnConflict) { return nil } + var rejection *preparationRejection + capacityRejected := errors.As(err, &rejection) && rejection.operation == proto.TypeExecutionPrepare && rejection.code == "preparation_capacity" outcome := json.RawMessage(`{"error_code":"execution_unavailable"}`) if errors.Is(err, store.ErrModelProviderRequired) { outcome = json.RawMessage(`{"error_code":"model_provider_required"}`) @@ -314,6 +356,11 @@ func (w *Worker) runClaim(ctx context.Context, item store.ExecutionWork) error { if err != nil { return err } + if turn.Status == store.TurnQueued && capacityRejected { + // No input was sent. Leave durable work for the existing scheduler tick; + // active and cleanup-held Runtime capacity have the same rejection. + return nil + } if turn.Status == store.TurnCompleted || turn.Status == store.TurnFailed || turn.Status == store.TurnCancelled { return nil } diff --git a/services/agents-api/internal/execution/worker_schedule.go b/services/agents-api/internal/execution/worker_schedule.go index 3e5321483..fbb932320 100644 --- a/services/agents-api/internal/execution/worker_schedule.go +++ b/services/agents-api/internal/execution/worker_schedule.go @@ -25,8 +25,12 @@ func (s *workerSchedule) selectWork(ctx context.Context, w *Worker, devices []st if err != nil { return nil, err } - if len(turns) == 0 { + if len(turns) == 0 && s.turnCursor != "" { s.turnCursor = "" + turns, err = w.dispatcher.Store.ListExecutionWork(ctx, "", []string{store.TurnQueued}, devices) + if err != nil { + return nil, err + } } var environments []store.EnvironmentInputWork if !time.Now().Before(s.nextEnvironmentScan) { diff --git a/services/agents-api/internal/execution/worker_wakeup.go b/services/agents-api/internal/execution/worker_wakeup.go new file mode 100644 index 000000000..fa5539189 --- /dev/null +++ b/services/agents-api/internal/execution/worker_wakeup.go @@ -0,0 +1,25 @@ +package execution + +import ( + "context" + + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" +) + +// wakeScheduler only hints at committed work. The existing loop retains lease, +// capacity and Session ownership; polling recovers absent or coalesced hints. +func (w *Worker) wakeScheduler() { + select { + case w.scheduleWake <- struct{}{}: + default: + } +} + +func (w *Worker) admitInputs(ctx context.Context, tenant, session, key string, inputs []store.Input) ([]store.InputReceipt, error) { + receipts, err := w.admission.SubmitInputs(ctx, tenant, session, key, inputs) + if err == nil { + w.wakeScheduler() + w.dispatcher.notifications.notify(tenant, session) + } + return receipts, err +} diff --git a/services/agents-api/internal/execution/worker_wakeup_test.go b/services/agents-api/internal/execution/worker_wakeup_test.go new file mode 100644 index 000000000..8a0a48f15 --- /dev/null +++ b/services/agents-api/internal/execution/worker_wakeup_test.go @@ -0,0 +1,35 @@ +package execution + +import ( + "sync" + "testing" +) + +func TestSchedulerWakeCoalescesConcurrentAdmissionsAndKeepsNextHint(t *testing.T) { + worker := &Worker{scheduleWake: make(chan struct{}, 1)} + var callers sync.WaitGroup + for range 100 { + callers.Go(func() { + for range 100 { + worker.wakeScheduler() + } + }) + } + callers.Wait() + if got := len(worker.scheduleWake); got != 1 { + t.Fatalf("queued wakeups = %d, want one", got) + } + <-worker.scheduleWake + // A commit while a previous scan is running needs a subsequent scan. + worker.wakeScheduler() + select { + case <-worker.scheduleWake: + default: + t.Fatal("admission during a scan lost its wakeup") + } + select { + case <-worker.scheduleWake: + t.Fatal("coalesced submissions caused an extra scan") + default: + } +} diff --git a/services/agents-api/internal/store/claude_execution_test.go b/services/agents-api/internal/store/claude_execution_test.go index f6eb75005..deca0c94c 100644 --- a/services/agents-api/internal/store/claude_execution_test.go +++ b/services/agents-api/internal/store/claude_execution_test.go @@ -30,7 +30,7 @@ func claudeSession(t *testing.T, h *dispatchHarness, configuration string, prebo func claudeHeartbeat(t *testing.T, h *dispatchHarness, ready bool) { t.Helper() - caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: ready, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, FunctionTools: true, ToolObservations: true} + caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: ready, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, FunctionTools: true, ToolObservations: true, Preparation: true} h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "claude_sdk", Available: true, Capabilities: caps}}}) deadline := time.Now().Add(3 * time.Second) for { @@ -85,7 +85,7 @@ func TestClaudeWorkerSelectsStoredEngineAndRestrictiveCapabilities(t *testing.T) queued() // Durable application receipts are required for this engine too. claudeHeartbeat(t, h, true) var prompt proto.PromptRequestPayload - if h.read(proto.TypePromptRequest).DecodePayload(&prompt) != nil { + if h.read(testExecutionRequest).DecodePayload(&prompt) != nil { t.Fatal("invalid prompt") } if prompt.AgentKind != "claude_sdk" || len(prompt.FunctionTools) != 1 || !prompt.DisableExecutionEnvironment || !prompt.DisableSubagents || prompt.ExecutionControls == nil || *prompt.ExecutionControls != (proto.ExecutionControls{WebSearch: "disabled", TextVerbosity: "medium"}) { diff --git a/services/agents-api/internal/store/claude_mcp_test.go b/services/agents-api/internal/store/claude_mcp_test.go index 87a7bbc7d..c09fed51c 100644 --- a/services/agents-api/internal/store/claude_mcp_test.go +++ b/services/agents-api/internal/store/claude_mcp_test.go @@ -28,7 +28,7 @@ func TestClaudeMCPWaitsForCapableRuntime(t *testing.T) { } claudeSession(t, h, configuration, prebound) // Base MCP support does not imply authentication or required initialization. - caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, ToolObservations: true, MCPHTTPTools: authenticated || required} + caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, ToolObservations: true, MCPHTTPTools: authenticated || required, Preparation: true} h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "claude_sdk", Available: true, Capabilities: caps}}}) awaitDaemonRemoteCondition(t, t.Context(), 3*time.Second, "Claude MCP heartbeat", func() bool { peer, _ := h.registry.LookupDevice(h.device.ID) @@ -71,7 +71,7 @@ func TestClaudeMCPWaitsForCapableRuntime(t *testing.T) { caps.MCPHTTPRequired = required h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "claude_sdk", Available: true, Capabilities: caps}}}) var prompt proto.PromptRequestPayload - if h.read(proto.TypePromptRequest).DecodePayload(&prompt) != nil || prompt.AgentKind != "claude_sdk" || prompt.MCPHTTPServers == nil || len(*prompt.MCPHTTPServers) != 1 { + if h.read(testExecutionRequest).DecodePayload(&prompt) != nil || prompt.AgentKind != "claude_sdk" || prompt.MCPHTTPServers == nil || len(*prompt.MCPHTTPServers) != 1 { t.Fatal("missing typed MCP dispatch") } server := (*prompt.MCPHTTPServers)[0] @@ -104,7 +104,7 @@ func TestClaudeMCPUnsupportedSnapshotRejectedBeforeClaim(t *testing.T) { configuration = strings.Replace(configuration, `"tickets"`, `"functions"`, 1) } claudeSession(t, h, configuration, true) - caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, ToolObservations: true, MCPHTTPTools: true, MCPHTTPRequired: true, MCPHTTPBearerAuth: true} + caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, ToolObservations: true, MCPHTTPTools: true, MCPHTTPRequired: true, MCPHTTPBearerAuth: true, Preparation: true} if profile == "missing required capability" { caps.MCPHTTPRequired = false } diff --git a/services/agents-api/internal/store/command_output_test.go b/services/agents-api/internal/store/command_output_test.go index ff5d27964..34b7889bb 100644 --- a/services/agents-api/internal/store/command_output_test.go +++ b/services/agents-api/internal/store/command_output_test.go @@ -131,7 +131,7 @@ func TestExecutionJournalsCommandOutputBeforeCancellation(t *testing.T) { ctx := context.Background() input := h.message("command", "run a command") result := h.run(ctx, input.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(input.TurnID, proto.TypeToolCall, proto.ToolCallPayload{ID: "cmd", Stage: "before", Observation: &proto.ToolObservation{Kind: "command", Command: "wait", Status: "in_progress"}}) h.write(input.TurnID, proto.TypeCommandOutput, proto.CommandOutputPayload{ID: "cmd", Delta: "partial"}) if _, err := h.s.RequestCancel(ctx, h.tenant, h.session.ID, "cancel"); err != nil { diff --git a/services/agents-api/internal/store/dispatch_test.go b/services/agents-api/internal/store/dispatch_test.go index ed371e923..e68ff6213 100644 --- a/services/agents-api/internal/store/dispatch_test.go +++ b/services/agents-api/internal/store/dispatch_test.go @@ -22,6 +22,8 @@ import ( ) type dispatchHarness struct { + writeMu sync.Mutex + admissions map[string]fixtureAdmission t *testing.T s *store.Store d *execution.Dispatcher @@ -92,7 +94,7 @@ func newDispatchHarnessForSession(t *testing.T, configuration []byte, local bool t.Fatal("device connection failed") } t.Cleanup(func() { h.conn.Close() }) - h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, Resume: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, SubagentObservations: true, ToolObservations: true, NativeSessionRecovery: true}}}}) + h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, Resume: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, SubagentObservations: true, ToolObservations: true, NativeSessionRecovery: true, Preparation: true}}}}) deadline := time.Now().Add(3 * time.Second) for { peer, e := h.registry.LookupDevice(h.device.ID) @@ -122,6 +124,12 @@ func (h *dispatchHarness) message(key, text string) store.InputReceipt { func (h *dispatchHarness) write(run, kind string, payload any) { h.t.Helper() + h.writeMu.Lock() + defer h.writeMu.Unlock() + if status, ok := payload.(proto.PreparationStatusPayload); ok && status.Handle != "" && status.ExecutorID == "" { + status.ExecutorID = "executor-" + status.Handle + payload = status + } env, err := proto.NewEnvelope(kind, run, payload) if err != nil { h.t.Fatal(err) @@ -139,6 +147,13 @@ func (h *dispatchHarness) read(kind string) proto.Envelope { if err := h.conn.ReadJSON(&env); err != nil { h.t.Fatal(err) } + if kind == testExecutionRequest { + var keep bool + env, keep = h.executionFrame(env) + if !keep { + continue + } + } if env.Type == kind { return env } @@ -175,7 +190,7 @@ func TestExecutionDispatchSteeringAndNativeContinuity(t *testing.T) { ctx := context.Background() first := h.message("first", "Initial input") result := h.run(ctx, first.TurnID) - request := h.read(proto.TypePromptRequest) + request := h.read(testExecutionRequest) var prompt proto.PromptRequestPayload _ = request.DecodePayload(&prompt) if inputTextForTest(t, prompt.Input) != "Initial input" || prompt.ConversationID != h.session.ID || prompt.AgentOptions["model"] != "test-model" || prompt.AgentOptions["system_prompt"] != "Keep this instruction." { @@ -217,7 +232,7 @@ func TestExecutionDispatchSteeringAndNativeContinuity(t *testing.T) { } next := h.message("third", "Continue the session") result = h.run(ctx, next.TurnID) - request = h.read(proto.TypePromptRequest) + request = h.read(testExecutionRequest) _ = request.DecodePayload(&prompt) if prompt.AgentSessionID != "native-thread-1" || prompt.AgentStateKey != "agents-api-"+h.session.ID { t.Fatal("native continuity lost") @@ -232,7 +247,7 @@ func TestExecutionCancellationRequiresReceiptAndSurvivesContextEnd(t *testing.T) h := newDispatchHarness(t) first := h.message("first", "Run") result := h.run(context.Background(), first.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) _, err := h.s.RequestCancel(context.Background(), h.tenant, h.session.ID, "cancel") if err != nil { t.Fatal(err) @@ -255,7 +270,7 @@ func TestExecutionCancellationRequiresReceiptAndSurvivesContextEnd(t *testing.T) h.finished(result, store.TurnCancelled) next := h.message("next", "Continue after cancellation") result = h.run(context.Background(), next.TurnID) - env = h.read(proto.TypePromptRequest) + env = h.read(testExecutionRequest) var prompt proto.PromptRequestPayload _ = env.DecodePayload(&prompt) if prompt.AgentSessionID != "cancelled-native" { @@ -269,7 +284,7 @@ func TestExecutionCancellationRequiresReceiptAndSurvivesContextEnd(t *testing.T) first := h.message("first", "Run") ctx, cancel := context.WithCancel(context.Background()) result := h.run(ctx, first.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) cancel() done := h.finished(result, store.TurnFailed) var outcome execution.Result @@ -285,7 +300,7 @@ func TestExecutionFailureDoesNotBecomeSuccessOrReplay(t *testing.T) { h := newDispatchHarness(t) first := h.message("first", "Run") result := h.run(context.Background(), first.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) switch kind { case "disconnect": h.conn.Close() diff --git a/services/agents-api/internal/store/environment_expiry_dispatch_test.go b/services/agents-api/internal/store/environment_expiry_dispatch_test.go index 17cc9c360..fc2cfc462 100644 --- a/services/agents-api/internal/store/environment_expiry_dispatch_test.go +++ b/services/agents-api/internal/store/environment_expiry_dispatch_test.go @@ -11,7 +11,7 @@ import ( func enableEnvironmentExpiryDispatch(h *dispatchHarness) { h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{ Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, EnvironmentNone: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, - }}}}) + Preparation: true}}}}) } func TestWorkerEnvironmentExpiryAtFullExecutionCapacity(t *testing.T) { @@ -26,7 +26,7 @@ func TestWorkerEnvironmentExpiryAtFullExecutionCapacity(t *testing.T) { if _, err := worker.SubmitInputs(t.Context(), h.tenant, session.ID, key, []store.Input{{Kind: "message", Payload: json.RawMessage(`{"text":"remain active"}`)}}); err != nil { t.Fatal(err) } - requests = append(requests, h.read(proto.TypePromptRequest)) + requests = append(requests, h.read(testExecutionRequest)) sessions = append(sessions, session) } tenant, due := newEnvironmentExpiryReservation(t, h.s) @@ -70,7 +70,7 @@ func TestWorkerEnvironmentExpirySkipsBusySessionAndAllowsDispatch(t *testing.T) t.Fatal(err) } waitEnvironmentExpiry(t, h.s, otherTenant, other) - request := h.read(proto.TypePromptRequest) + request := h.read(testExecutionRequest) if request.ID != receipt[0].TurnID { t.Fatal("unrelated dispatch mismatch", request.ID) } diff --git a/services/agents-api/internal/store/environment_worker_helpers_test.go b/services/agents-api/internal/store/environment_worker_helpers_test.go index 3687d4637..168df067a 100644 --- a/services/agents-api/internal/store/environment_worker_helpers_test.go +++ b/services/agents-api/internal/store/environment_worker_helpers_test.go @@ -63,6 +63,11 @@ func workerFrames(t *testing.T, runtimes ...*dispatchHarness) <-chan proto.Envel if h.conn.ReadJSON(&env) != nil { return } + var keep bool + env, keep = h.executionFrame(env) + if !keep { + continue + } select { case frames <- env: case <-t.Context().Done(): diff --git a/services/agents-api/internal/store/environment_worker_scan_test.go b/services/agents-api/internal/store/environment_worker_scan_test.go index 7627e29e0..bca895242 100644 --- a/services/agents-api/internal/store/environment_worker_scan_test.go +++ b/services/agents-api/internal/store/environment_worker_scan_test.go @@ -24,7 +24,7 @@ func TestWorkerEnvironmentRetriesNewlyReadyAtNextScan(t *testing.T) { _, stop := startEnvironmentExpiryWorker(t, h.d) // Dispatch starts only after selectWork has examined the pending input on // the same pass, while its exact Runtime is still incapable of preparation. - barrier := nextWorkerFrame(t, frames, proto.TypePromptRequest) + barrier := nextWorkerFrame(t, frames, testExecutionRequest) if barrier.ID != receipt.TurnID { t.Fatal("unexpected scan barrier") } @@ -70,7 +70,7 @@ func TestWorkerEnvironmentPaginationReachesReadyTail(t *testing.T) { receipt := h.message("barrier", "ordinary work") scanned := time.Now() _, stop := startEnvironmentExpiryWorker(t, h.d) - barrier := nextWorkerFrame(t, frames, proto.TypePromptRequest) + barrier := nextWorkerFrame(t, frames, testExecutionRequest) if barrier.ID != receipt.TurnID { t.Fatal("unexpected page barrier") } diff --git a/services/agents-api/internal/store/environment_worker_test.go b/services/agents-api/internal/store/environment_worker_test.go index 64615d1b4..313cf2cb6 100644 --- a/services/agents-api/internal/store/environment_worker_test.go +++ b/services/agents-api/internal/store/environment_worker_test.go @@ -38,7 +38,7 @@ func TestWorkerEnvironmentSharesCapacityThroughClaimAndCleanup(t *testing.T) { select { case frame := <-frames: switch frame.Type { - case proto.TypePromptRequest: + case testExecutionRequest: normal = append(normal, frame) case proto.TypeExecutionPrepare: preparing = append(preparing, frame) @@ -96,7 +96,7 @@ func TestWorkerEnvironmentSharesCapacityThroughClaimAndCleanup(t *testing.T) { t.Fatal("duplicate preparation release") } release = frame - case proto.TypePromptRequest: + case testExecutionRequest: if resumed.ID != "" { t.Fatal("cleanup freed more than one capacity slot") } @@ -140,7 +140,7 @@ func TestWorkerEnvironmentRetriesPendingWithoutExtendingDeadline(t *testing.T) { nextWorkerFrame(t, frames, proto.TypeExecutionRelease) h.session = publicSession(t, h, "unrelated") receipt := h.message("ordinary", "make progress after preparation failure") - request := nextWorkerFrame(t, frames, proto.TypePromptRequest) + request := nextWorkerFrame(t, frames, testExecutionRequest) if request.ID != receipt.TurnID { t.Fatal("preparation failure blocked ordinary work") } diff --git a/services/agents-api/internal/store/execution_events_test.go b/services/agents-api/internal/store/execution_events_test.go index ba1404bb0..31bef15ff 100644 --- a/services/agents-api/internal/store/execution_events_test.go +++ b/services/agents-api/internal/store/execution_events_test.go @@ -17,7 +17,7 @@ func TestExecutionPersistsLiveAndCancelledPartialOutput(t *testing.T) { ctx := context.Background() input := h.message("start", "Stream then cancel") result := h.run(ctx, input.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(input.TurnID, proto.TypeDelta, proto.DeltaPayload{Delta: "已输出", Sequence: 1}) h.write(input.TurnID, proto.TypeToolCall, proto.ToolCallPayload{ID: "tool-1", Name: "Bash", Stage: "before", Observation: &proto.ToolObservation{Kind: "command", Command: "pwd", Status: "in_progress"}}) h.write(input.TurnID, proto.TypeToolCall, proto.ToolCallPayload{ID: "tool-1", Name: "Bash", Stage: "after", Observation: &proto.ToolObservation{Kind: "command", Command: "pwd", Status: "completed"}}) @@ -79,7 +79,7 @@ func TestExecutionDoesNotCompleteAfterEventPersistenceFailure(t *testing.T) { ctx := context.Background() input := h.message("start", "Output beyond storage budget") result := h.run(ctx, input.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) _, pool := store.NewTestStore(t) defer pool.Close() if _, err := pool.Exec(ctx, "UPDATE turns SET event_bytes=33554432 WHERE id=$1", input.TurnID); err != nil { diff --git a/services/agents-api/internal/store/execution_messages_test.go b/services/agents-api/internal/store/execution_messages_test.go index c5d622d49..cbda2d87e 100644 --- a/services/agents-api/internal/store/execution_messages_test.go +++ b/services/agents-api/internal/store/execution_messages_test.go @@ -16,14 +16,14 @@ func TestExecutionNegotiatesAndPersistsMessageObservations(t *testing.T) { first := h.message("legacy", "legacy observation policy") result := h.run(ctx, first.TurnID) var request proto.PromptRequestPayload - env := h.read(proto.TypePromptRequest) + env := h.read(testExecutionRequest) _ = env.DecodePayload(&request) if request.ObserveMessages { t.Fatal("unadvertised observation capability requested") } h.write(first.TurnID, proto.TypeDone, proto.DonePayload{}) h.finished(result, store.TurnCompleted) - h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, Resume: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, MessageItems: true, NativeSessionRecovery: true}}}}) + h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, Resume: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, MessageItems: true, NativeSessionRecovery: true, Preparation: true}}}}) deadline := time.Now().Add(3 * time.Second) for { peer, err := h.registry.LookupDevice(h.device.ID) @@ -41,7 +41,7 @@ func TestExecutionNegotiatesAndPersistsMessageObservations(t *testing.T) { } input := h.message("observed", "stream separate messages") result = h.run(ctx, input.TurnID) - env = h.read(proto.TypePromptRequest) + env = h.read(testExecutionRequest) _ = env.DecodePayload(&request) if !request.ObserveMessages { t.Fatal("advertised capability was not requested") diff --git a/services/agents-api/internal/store/execution_tools_test.go b/services/agents-api/internal/store/execution_tools_test.go index 607fb324a..347aec57b 100644 --- a/services/agents-api/internal/store/execution_tools_test.go +++ b/services/agents-api/internal/store/execution_tools_test.go @@ -16,7 +16,7 @@ func TestExecutionNegotiatesAndPersistsToolObservations(t *testing.T) { ctx := context.Background() input := h.message("observed", "run tools") result := h.run(ctx, input.TurnID) - env := h.read(proto.TypePromptRequest) + env := h.read(testExecutionRequest) var request proto.PromptRequestPayload _ = env.DecodePayload(&request) if !request.ObserveToolObservations || request.ObserveTools || request.ObserveMessages { diff --git a/services/agents-api/internal/store/executor_fixture_test.go b/services/agents-api/internal/store/executor_fixture_test.go new file mode 100644 index 000000000..ef8712387 --- /dev/null +++ b/services/agents-api/internal/store/executor_fixture_test.go @@ -0,0 +1,63 @@ +package store_test + +import ( + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/google/uuid" +) + +// testExecutionRequest is a test projection of configuration plus a confirmed +// Start. It is never sent over the Runtime connection. Tests which exercise +// preparation failures consume the actual control frames directly. +const testExecutionRequest = "test_execution_request" + +type fixtureAdmission struct { + prepare proto.ExecutionPreparePayload + handle string + executor string +} + +func (h *dispatchHarness) executionFrame(env proto.Envelope) (proto.Envelope, bool) { + if h.admissions == nil { + h.admissions = make(map[string]fixtureAdmission) + } + switch env.Type { + case proto.TypeExecutionPrepare: + var prepare proto.ExecutionPreparePayload + if env.DecodePayload(&prepare) != nil { + h.t.Fatal("invalid execution preparation") + } + if prepare.Configuration.LocalEnvironment != nil || prepare.Configuration.WorkspaceReadOnly { + return env, true + } + if prepare.SessionID == "" || prepare.Configuration.RunID != "" || len(prepare.Configuration.Input) != 0 { + h.t.Fatal("preparation changed Session identity or submitted input early") + } + admission := fixtureAdmission{prepare: prepare, handle: uuid.NewString(), executor: "executor-" + prepare.SessionID} + h.admissions[env.ID] = admission + h.write(env.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: admission.handle, ExecutorID: admission.executor, Revision: 1, State: "ready"}) + return proto.Envelope{}, false + case proto.TypeExecutionStart: + admission, ok := h.admissions[env.ID] + if !ok { + return env, true + } + var start proto.ExecutionStartPayload + if env.DecodePayload(&start) != nil || start.Handle != admission.handle || start.ExecutorID != admission.executor || start.RunID == "" || start.Input.Validate() != nil { + h.t.Fatal("Start changed admission, Executor or input identity") + } + h.write(env.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: admission.handle, ExecutorID: admission.executor, Revision: 2, State: "started", RunID: start.RunID}) + request := admission.prepare.Configuration + request.RunID, request.ConversationID, request.Input = start.RunID, admission.prepare.SessionID, start.Input + projected, err := proto.NewEnvelope(testExecutionRequest, start.RunID, request) + if err != nil { + h.t.Fatal(err) + } + return projected, true + case proto.TypeExecutionRelease: + if _, ok := h.admissions[env.ID]; ok { + delete(h.admissions, env.ID) + return proto.Envelope{}, false + } + } + return env, true +} diff --git a/services/agents-api/internal/store/executor_recovery_test.go b/services/agents-api/internal/store/executor_recovery_test.go new file mode 100644 index 000000000..eba3602cb --- /dev/null +++ b/services/agents-api/internal/store/executor_recovery_test.go @@ -0,0 +1,58 @@ +package store_test + +import ( + "testing" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" + "github.com/google/uuid" +) + +func readyExecutorAttempt(t *testing.T, h *dispatchHarness) (proto.Envelope, proto.ExecutionStartPayload) { + t.Helper() + prepare := h.read(proto.TypeExecutionPrepare) + var configuration proto.ExecutionPreparePayload + if prepare.DecodePayload(&configuration) != nil || configuration.SessionID != h.session.ID || len(configuration.Configuration.Input) != 0 { + t.Fatal("invalid Executor preparation identity") + } + handle, executor := uuid.NewString(), uuid.NewString() + h.write(prepare.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: handle, ExecutorID: executor, Revision: 1, State: "ready"}) + frame := h.read(proto.TypeExecutionStart) + var start proto.ExecutionStartPayload + if frame.ID != prepare.ID || frame.DecodePayload(&start) != nil || start.Handle != handle || start.ExecutorID != executor { + t.Fatal("Start lost its admission or Executor identity") + } + return frame, start +} + +func TestExecutorRecoveryRetriesOnlyConfirmedUnsubmittedInput(t *testing.T) { + for _, code := range []string{"executor_unavailable", "executor_cleanup_unconfirmed", "start_failed"} { + t.Run(code, func(t *testing.T) { + h := newDispatchHarness(t) + receipt := h.message("input", "execute once") + result := h.run(t.Context(), receipt.TurnID) + first, start := readyExecutorAttempt(t, h) + h.write(first.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{State: "rejected", Operation: proto.TypeExecutionStart, ErrorCode: code}) + if code != "executor_unavailable" { + h.finished(result, store.TurnFailed) + return + } + next, replacement := readyExecutorAttempt(t, h) + if next.ID == first.ID || replacement.ExecutorID == start.ExecutorID || replacement.RunID != start.RunID || inputTextForTest(t, replacement.Input) != "execute once" { + t.Fatal("recovery changed input or reused retired control ownership") + } + h.write(next.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: replacement.Handle, ExecutorID: replacement.ExecutorID, Revision: 2, State: "started", RunID: replacement.RunID}) + h.write(replacement.RunID, proto.TypeDone, proto.DonePayload{Content: "once"}) + h.finished(result, store.TurnCompleted) + }) + } +} + +func TestExecutorReadinessRejectsChangedOwner(t *testing.T) { + h := newDispatchHarness(t) + receipt := h.message("input", "execute once") + result := h.run(t.Context(), receipt.TurnID) + frame, start := readyExecutorAttempt(t, h) + h.write(frame.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: start.Handle, ExecutorID: uuid.NewString(), Revision: 2, State: "started", RunID: start.RunID}) + h.finished(result, store.TurnFailed) +} diff --git a/services/agents-api/internal/store/function_execution_test.go b/services/agents-api/internal/store/function_execution_test.go index 19416593d..0e138b18c 100644 --- a/services/agents-api/internal/store/function_execution_test.go +++ b/services/agents-api/internal/store/function_execution_test.go @@ -26,7 +26,7 @@ func newFunctionHarness(t *testing.T) *dispatchHarness { if err := h.s.BindSessionDevice(t.Context(), h.tenant, h.session.ID, h.device.ID); err != nil { t.Fatal(err) } - h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, Resume: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, EnvironmentNone: true, FunctionTools: true, FunctionResultImages: true}}}}) + h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, Resume: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, EnvironmentNone: true, FunctionTools: true, FunctionResultImages: true, Preparation: true}}}}) deadline := time.Now().Add(time.Second) for { peer, _ := h.registry.LookupDevice(h.device.ID) @@ -64,7 +64,7 @@ func TestExecutionFunctionsWaitForEveryApplicationReceipt(t *testing.T) { input := h.message("start", "Run functions") result := h.run(t.Context(), input.TurnID) var prompt proto.PromptRequestPayload - _ = h.read(proto.TypePromptRequest).DecodePayload(&prompt) + _ = h.read(testExecutionRequest).DecodePayload(&prompt) if len(prompt.FunctionTools) != 1 || prompt.FunctionTools[0].Name != "lookup_ticket" { t.Fatal(prompt.FunctionTools) } @@ -107,7 +107,7 @@ func TestExecutionFunctionsWaitForEveryApplicationReceipt(t *testing.T) { functionState(t, h, 0) next := h.message("next", "Resume") result = h.run(t.Context(), next.TurnID) - _ = h.read(proto.TypePromptRequest).DecodePayload(&prompt) + _ = h.read(testExecutionRequest).DecodePayload(&prompt) if prompt.AgentSessionID != "native-functions" || len(prompt.FunctionTools) != 1 { t.Fatal(prompt) } @@ -121,7 +121,7 @@ func TestExecutionFunctionsCancellationAndUnconfirmedResults(t *testing.T) { h := newFunctionHarness(t) input := h.message("start", "Run") result := h.run(t.Context(), input.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(input.TurnID, proto.TypeFunctionCall, proto.FunctionCallPayload{CallID: "a", Name: "lookup_ticket", Arguments: json.RawMessage(`{}`)}) state := functionState(t, h, 1) id := state.RequiredActions[0].CallID @@ -178,7 +178,7 @@ func TestExecutionFunctionsRejectUndeclaredCallsAndPrematureDone(t *testing.T) { ctx, cancel := context.WithTimeout(t.Context(), 10*time.Second) defer cancel() result := h.run(ctx, input.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(input.TurnID, proto.TypeFunctionCall, proto.FunctionCallPayload{CallID: "a", Name: name, Arguments: json.RawMessage(`{}`)}) h.write(input.TurnID, proto.TypeDone, proto.DonePayload{}) h.finished(result, store.TurnFailed) diff --git a/services/agents-api/internal/store/function_input_execution_test.go b/services/agents-api/internal/store/function_input_execution_test.go index 99e763047..a932a988c 100644 --- a/services/agents-api/internal/store/function_input_execution_test.go +++ b/services/agents-api/internal/store/function_input_execution_test.go @@ -14,7 +14,7 @@ func TestExecutionFunctionInputBatchStillSteersMessages(t *testing.T) { h := newFunctionHarness(t) input := h.message("start", "Run") running := h.run(t.Context(), input.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(input.TurnID, proto.TypeFunctionCall, proto.FunctionCallPayload{CallID: "a", Name: "lookup_ticket", Arguments: json.RawMessage(`{}`)}) state := functionState(t, h, 1) raw, _ := json.Marshal(store.FunctionResultInput{TurnID: input.TurnID, CallID: state.RequiredActions[0].CallID, Result: json.RawMessage(`{"success":true,"output":"answer"}`)}) diff --git a/services/agents-api/internal/store/function_worker_test.go b/services/agents-api/internal/store/function_worker_test.go index 3fed06915..f3965da77 100644 --- a/services/agents-api/internal/store/function_worker_test.go +++ b/services/agents-api/internal/store/function_worker_test.go @@ -16,7 +16,7 @@ import ( ) func TestWorkerWaitsForToolCapabilities(t *testing.T) { - for _, missing := range []string{"durable_input_receipts", "execution_controls", "function_tools", "tool_observations", "mcp_http_tools", "mcp_http_bearer_auth", "mcp_http_required"} { + for _, missing := range []string{"preparation", "durable_input_receipts", "execution_controls", "function_tools", "tool_observations", "mcp_http_tools", "mcp_http_bearer_auth", "mcp_http_required"} { for _, prebound := range []bool{false, true} { t.Run(missing+"/"+map[bool]string{false: "select", true: "bound"}[prebound], func(t *testing.T) { h := newFunctionHarness(t) @@ -44,7 +44,7 @@ func TestWorkerWaitsForToolCapabilities(t *testing.T) { t.Fatal(err) } } - caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: missing != "durable_input_receipts", EnvironmentNone: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: missing != "execution_controls", SubagentControl: true, ToolObservations: missing != "tool_observations", MCPHTTPTools: missing != "mcp_http_tools", MCPHTTPRequired: missing != "mcp_http_required", MCPHTTPBearerAuth: missing != "mcp_http_bearer_auth", FunctionTools: missing != "function_tools" && !isMCP} + caps := proto.AgentKindCapabilities{Preparation: missing != "preparation", Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: missing != "durable_input_receipts", EnvironmentNone: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: missing != "execution_controls", SubagentControl: true, ToolObservations: missing != "tool_observations", MCPHTTPTools: missing != "mcp_http_tools", MCPHTTPRequired: missing != "mcp_http_required", MCPHTTPBearerAuth: missing != "mcp_http_bearer_auth", FunctionTools: missing != "function_tools" && !isMCP} heartbeat := func() { h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: caps}}}) } @@ -53,7 +53,7 @@ func TestWorkerWaitsForToolCapabilities(t *testing.T) { for { peer, _ := h.registry.LookupDevice(h.device.ID) info, _, _ := peer.AgentKindStatus("codex") - if info.Capabilities.ExecutionControls == caps.ExecutionControls && info.Capabilities.FunctionTools == caps.FunctionTools && info.Capabilities.ToolObservations == caps.ToolObservations && info.Capabilities.MCPHTTPTools == caps.MCPHTTPTools && info.Capabilities.MCPHTTPBearerAuth == caps.MCPHTTPBearerAuth && info.Capabilities.MCPHTTPRequired == caps.MCPHTTPRequired { + if info.Capabilities.Preparation == caps.Preparation && info.Capabilities.DurableInputReceipts == caps.DurableInputReceipts && info.Capabilities.ExecutionControls == caps.ExecutionControls && info.Capabilities.FunctionTools == caps.FunctionTools && info.Capabilities.ToolObservations == caps.ToolObservations && info.Capabilities.MCPHTTPTools == caps.MCPHTTPTools && info.Capabilities.MCPHTTPBearerAuth == caps.MCPHTTPBearerAuth && info.Capabilities.MCPHTTPRequired == caps.MCPHTTPRequired { break } if time.Now().After(deadline) { @@ -88,10 +88,10 @@ func TestWorkerWaitsForToolCapabilities(t *testing.T) { t.Fatal("bound an incapable device", err) } } - caps.DurableInputReceipts, caps.ExecutionControls, caps.ToolObservations, caps.MCPHTTPTools = true, true, true, true + caps.Preparation, caps.DurableInputReceipts, caps.ExecutionControls, caps.ToolObservations, caps.MCPHTTPTools = true, true, true, true, true caps.MCPHTTPBearerAuth, caps.FunctionTools, caps.MCPHTTPRequired = true, !isMCP, true heartbeat() - request := h.read(proto.TypePromptRequest) + request := h.read(testExecutionRequest) var prompt proto.PromptRequestPayload if request.DecodePayload(&prompt) != nil { t.Fatal("invalid prompt") diff --git a/services/agents-api/internal/store/harness_onboarding_test.go b/services/agents-api/internal/store/harness_onboarding_test.go index 9b723d412..c3655ddde 100644 --- a/services/agents-api/internal/store/harness_onboarding_test.go +++ b/services/agents-api/internal/store/harness_onboarding_test.go @@ -221,19 +221,42 @@ func startOnboardingPeer(t *testing.T, h *dispatchHarness) (<-chan proto.PromptR }() go func() { enc := json.NewEncoder(in) + preparations := make(map[string]proto.ExecutionPreparePayload) for { var e proto.Envelope if err := h.conn.ReadJSON(&e); err != nil { down <- err return } - if e.Type == proto.TypePromptRequest { - var p proto.PromptRequestPayload + if e.Type == proto.TypeExecutionPrepare { + var p proto.ExecutionPreparePayload if err := e.DecodePayload(&p); err != nil { down <- err return } - started <- p + if p.SessionID == "" || p.Configuration.RunID != "" || len(p.Configuration.Input) != 0 { + down <- fmt.Errorf("fixture preparation submitted input or lost Session identity") + return + } + preparations[e.ID] = p + } + if e.Type == proto.TypeExecutionStart { + var start proto.ExecutionStartPayload + if err := e.DecodePayload(&start); err != nil { + down <- err + return + } + p, ok := preparations[e.ID] + if !ok || start.ExecutorID == "" || start.Handle == "" { + down <- fmt.Errorf("fixture Start lacks prepared Executor ownership") + return + } + request := p.Configuration + request.RunID, request.ConversationID, request.Input = start.RunID, p.SessionID, start.Input + started <- request + } + if e.Type == proto.TypeExecutionRelease { + delete(preparations, e.ID) } if err := enc.Encode(e); err != nil { down <- err diff --git a/services/agents-api/internal/store/hosted_initialization_failure_public_test.go b/services/agents-api/internal/store/hosted_initialization_failure_public_test.go index 7f422425e..d2566bacd 100644 --- a/services/agents-api/internal/store/hosted_initialization_failure_public_test.go +++ b/services/agents-api/internal/store/hosted_initialization_failure_public_test.go @@ -6,13 +6,11 @@ import ( "context" "encoding/json" "errors" - "fmt" "io" "log/slog" "net/http" "net/http/httptest" "reflect" - "strconv" "strings" "sync" "testing" @@ -20,6 +18,8 @@ import ( v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/device" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/gateway" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/api" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/credentialcrypto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" @@ -29,10 +29,10 @@ import ( const hostedFailureCanary = "CANARY-hosted-init-7c21" -// leakyReceipt is a failed receipt that also carries canary output in fields -// Core must never read, as a leaking or newer Runtime could send. -func leakyReceipt(fields string) sandbox.CommandResult { - return sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed",` + fields + `"output":"` + hostedFailureCanary + `","stderr":"` + hostedFailureCanary + `"}`} +// Failed receipts carry only a finite code and a bounded status. Public +// projections must never include the confidential operation's canary text. +func failedInitialization(exitCode int) proto.RuntimePrepareResultPayload { + return proto.RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: "runtime_preparation_failed", ExitCode: exitCode} } func hostedFailureSkill(t *testing.T) store.EnvironmentSkill { @@ -53,45 +53,49 @@ func hostedFailureSkill(t *testing.T) store.EnvironmentSkill { } // hostedFailureProvider fails one initialization step with a controlled result. -// Every failure it reports is produced next to canary output, which the -// initializer discards and Core must never publish. +// The confidential inputs contain canaries that Core must never publish. type hostedFailureProvider struct { lifecycleProvider + initializationPeer fail string // runtime-initialize action, or "file" for the initial file writer skip int // matching steps that succeed before the failure - result sandbox.CommandResult + result proto.RuntimePrepareResultPayload err error steps []string } -func (p *hostedFailureProvider) RunCommand(_ context.Context, _ sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - action := "file" - if c.Args[len(c.Args)-1] == "/usr/local/bin/oac-runtime-initialize" { - var operation struct { - Action string `json:"action"` - } - if json.Unmarshal(c.Stdin, &operation) != nil || operation.Action == "" { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - action = operation.Action +func (p *hostedFailureProvider) setRuntimeGateway(t *testing.T, endpoint string, registry *gateway.Registry) { + p.initializationPeer.setRuntimeGateway(t, endpoint, registry) + p.apply = p.prepare +} +func (p *hostedFailureProvider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Info, error) { + info, err := p.lifecycleProvider.Create(ctx, b) + if err == nil { + err = p.connect(b) + } + return info, err +} +func (p *hostedFailureProvider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { + return p.initializationPeer.RunCommand(ctx, r, c) +} +func (p *hostedFailureProvider) prepare(request proto.RuntimePreparePayload, _ []byte) proto.RuntimePrepareResultPayload { + action := request.Action + if request.Initialization != nil { + action = request.Initialization.Action } p.mu.Lock() + defer p.mu.Unlock() p.steps = append(p.steps, action) - p.mu.Unlock() if action == p.fail { if p.skip == 0 { - return p.result, p.err + if p.err != nil { + return proto.RuntimePrepareResultPayload{Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"} + } + return p.result } p.skip-- } - if action == "file" { - size, err := strconv.Atoi(c.Args[len(c.Args)-1]) - if err != nil { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - return sandbox.CommandResult{Stdout: fmt.Sprintf(`{"version":1,"outcome":"completed","size_bytes":%d}`, size)}, nil - } - return sandbox.CommandResult{Stdout: `{"version":1,"outcome":"completed"}`}, nil + return completedInitialization(request, nil) } func hostedFailureStore(t *testing.T) *store.Store { @@ -140,7 +144,7 @@ func TestHostedInitializationFailureRecordsSafeSessionFailure(t *testing.T) { type failure struct { fail string skip int - result sandbox.CommandResult + result proto.RuntimePrepareResultPayload err error } for _, test := range []struct { @@ -151,28 +155,28 @@ func TestHostedInitializationFailureRecordsSafeSessionFailure(t *testing.T) { steps []string }{ {"setup exit status", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Commands: commands[1:]}}, - failure{fail: "setup", result: leakyReceipt(`"exit_code":3,`)}, + failure{fail: "setup", result: failedInitialization(3)}, `Failed to provision environment: script "setup_commands[0]" failed with exit code 3`, []string{"configure", "setup"}}, {"later setup command", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Commands: commands}}, - failure{fail: "setup", skip: 1, result: sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3}` + "\n"}}, + failure{fail: "setup", skip: 1, result: failedInitialization(3)}, `Failed to provision environment: script "setup_commands[1]" failed with exit code 3`, []string{"configure", "setup", "setup"}}, {"python package", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Packages: v1.EnvironmentPackages{Python: []string{"oac-nonexistent-zz"}}, Commands: commands[2:]}}, - failure{fail: "python", result: leakyReceipt(`"exit_code":1,`)}, + failure{fail: "python", result: failedInitialization(1)}, `Failed to provision environment: script "Python package installation" failed with exit code 1`, []string{"configure", "python"}}, - {"old image without exit status", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Commands: commands[1:]}}, - failure{fail: "setup", result: leakyReceipt("")}, + {"failure without exit status", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Commands: commands[1:]}}, + failure{fail: "setup", result: failedInitialization(0)}, "Failed to provision environment: initialization did not complete", []string{"configure", "setup"}}, {"unknown effect", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Commands: commands[1:]}}, failure{fail: "setup", err: sandbox.ErrCommandUnconfirmed}, "Failed to provision environment: initialization did not complete", []string{"configure", "setup"}}, - {"output instead of a receipt", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Commands: commands[1:]}}, - failure{fail: "setup", result: sandbox.CommandResult{ExitCode: 3, Stdout: hostedFailureCanary, Stderr: hostedFailureCanary}}, + {"invalid failure code", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Commands: commands[1:]}}, + failure{fail: "setup", result: proto.RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: hostedFailureCanary}}, "Failed to provision environment: initialization did not complete", []string{"configure", "setup"}}, {"initial file", store.CreateSessionInput{InitialFiles: []store.InitialFile{{Type: "inline", Path: "/workspace/a", Data: []byte(hostedFailureCanary)}}}, - failure{fail: "file", result: sandbox.CommandResult{Stdout: `{"version":1,"outcome":"failed","error":"write_failed","detail":"` + hostedFailureCanary + `"}`}}, + failure{fail: "file", result: failedInitialization(0)}, "Failed to provision environment: initial file installation failed", []string{"file"}}, {"Skill", store.CreateSessionInput{Initialization: store.EnvironmentSetup{Skills: []store.EnvironmentSkill{hostedFailureSkill(t)}, Commands: commands[2:]}}, - failure{fail: "skill", result: leakyReceipt("")}, + failure{fail: "skill", result: failedInitialization(0)}, "Failed to provision environment: Skill installation failed", []string{"configure", "skill"}}, } { t.Run(test.name, func(t *testing.T) { @@ -182,7 +186,7 @@ func TestHostedInitializationFailureRecordsSafeSessionFailure(t *testing.T) { p := &hostedFailureProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, fail: test.p.fail, skip: test.p.skip, result: test.p.result, err: test.p.err} failHostedInitialization(t, s, tenant, environment, p) - if !reflect.DeepEqual(p.steps, test.steps) || p.kills != 1 { + if !reflect.DeepEqual(p.steps, test.steps) || p.kills != 1 || p.commandCalls.Load() != 0 { t.Fatal("failed initialization continued or was not reclaimed", p.steps, p.kills) } @@ -243,7 +247,7 @@ func TestHostedInitializationFailureSettlesPendingInitialInput(t *testing.T) { InitialInputs: []store.Input{{Kind: "message", Payload: json.RawMessage(`{"text":"initial"}`)}}, }) p := &hostedFailureProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, fail: "setup", - result: sandbox.CommandResult{ExitCode: 1, Stdout: `{"version":1,"outcome":"failed","exit_code":3}`}} + result: failedInitialization(3)} failHostedInitialization(t, s, tenant, environment, p) read, err := s.GetSession(t.Context(), tenant, session.ID) if err != nil || read.PendingInput || read.EnvironmentInputActivity == nil || read.EnvironmentInputActivity.Status != "failed" || @@ -277,8 +281,8 @@ func TestHostedInitializationFailurePublicHTTP(t *testing.T) { Metadata: map[string]string{"case": "setup-exit3"}, }) key := uuid.NewString() - // The failed receipt carries canary output in fields Core must never read. - p := &hostedFailureProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, fail: "setup", result: leakyReceipt(`"exit_code":3,`)} + // A failed typed Runtime receipt exposes only a safe status. + p := &hostedFailureProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, fail: "setup", result: failedInitialization(3)} logs := &lockedBuffer{} previous := slog.Default() slog.SetDefault(slog.New(slog.NewTextHandler(logs, &slog.HandlerOptions{Level: slog.LevelDebug}))) diff --git a/services/agents-api/internal/store/public_execution_test.go b/services/agents-api/internal/store/public_execution_test.go index 5f7adea85..6d9b2c28b 100644 --- a/services/agents-api/internal/store/public_execution_test.go +++ b/services/agents-api/internal/store/public_execution_test.go @@ -23,7 +23,7 @@ func publicSession(t *testing.T, h *dispatchHarness, key string) store.Session { func TestExecutionWorkerAdmissionBindingAndRecovery(t *testing.T) { h := newDispatchHarness(t) - h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, EnvironmentNone: true}}}}) + h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, EnvironmentNone: true, Preparation: true}}}}) h.session = publicSession(t, h, "public") ctx, cancel := context.WithCancel(context.Background()) defer cancel() @@ -58,7 +58,7 @@ func TestExecutionWorkerAdmissionBindingAndRecovery(t *testing.T) { if err != nil || !retry[0].Replayed || retry[0].TurnID != receipts[0].TurnID { t.Fatal(retry, err) } - request := h.read(proto.TypePromptRequest) + request := h.read(testExecutionRequest) var prompt proto.PromptRequestPayload if err := request.DecodePayload(&prompt); err != nil { t.Fatal(err) @@ -84,7 +84,7 @@ func TestExecutionWorkerAdmissionBindingAndRecovery(t *testing.T) { if err != nil { t.Fatal(err) } - request = h.read(proto.TypePromptRequest) + request = h.read(testExecutionRequest) _ = request.DecodePayload(&prompt) if prompt.AgentSessionID != "worker-native" { t.Fatal(prompt) diff --git a/services/agents-api/internal/store/runtime_capabilities_pending_test.go b/services/agents-api/internal/store/runtime_capabilities_pending_test.go new file mode 100644 index 000000000..6c7265fbd --- /dev/null +++ b/services/agents-api/internal/store/runtime_capabilities_pending_test.go @@ -0,0 +1,82 @@ +package store_test + +import ( + "bytes" + "encoding/json" + "errors" + "testing" + + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/credentialcrypto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" + "github.com/google/uuid" +) + +func TestManagedCapabilitiesWaitBeforeInitializationClaim(t *testing.T) { + _, pool := store.NewManagedTestStore(t) + cipher, err := credentialcrypto.New(bytes.Repeat([]byte{9}, 32)) + if err != nil { + t.Fatal(err) + } + s := store.NewWithCredentialCipher(pool, cipher) + tenant := uuid.NewString() + session, err := s.CreateSession(t.Context(), tenant, store.CreateSessionInput{ + Creator: store.FixtureCreator(), Engine: "codex", IdempotencyKey: uuid.NewString(), + Configuration: json.RawMessage(`{"environment":{"type":"openai_hosted"}}`), + Initialization: store.EnvironmentSetup{CapabilityDirectories: []string{"/workspace/generated"}}, + }) + if err != nil { + t.Fatal(err) + } + env, err := s.GetSessionEnvironment(t.Context(), tenant, session.ID) + if err != nil { + t.Fatal(err) + } + provider := &initializingProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, initializationPeer: initializationPeer{deferred: true}} + key := uuid.NewString() + worker, _ := managedWorker(t, s, key, provider) + owner, err := worker.ProvisionEnvironment(t.Context(), tenant, env.ID, key) + if err != nil || owner.Initialization != "pending" { + t.Fatal(owner, err) + } + for range 4 { + if err := worker.ReconcileManagedRuntimes(t.Context()); err != nil { + t.Fatal(err) + } + } + owner, err = s.GetRuntimeAllocation(t.Context(), tenant, env.ID) + if err != nil || owner.Initialization != "pending" || owner.State != "running" || provider.writes.Load() != 0 || provider.kills != 0 { + t.Fatal("missing socket consumed initialization or requested cleanup", owner, err, provider.writes.Load(), provider.kills) + } + if _, err := s.GetSessionExecutionBinding(t.Context(), tenant, session.ID); !errors.Is(err, store.ErrNotFound) { + t.Fatal("ordinary readiness gate bypassed", err) + } +} + +func TestSelfHostedCapabilityConfigurationNeedsNoManagedSetup(t *testing.T) { + s, pool := store.NewManagedTestStore(t) + tenant := uuid.NewString() + configuration := json.RawMessage(`{"environment":{"type":"self_hosted","workspace_directory":"/home/user/project","capability_directories":["/opt/skills"]}}`) + session, err := s.CreateSession(t.Context(), tenant, store.CreateSessionInput{Creator: store.FixtureCreator(), Engine: "codex", IdempotencyKey: uuid.NewString(), Configuration: configuration}) + if err != nil { + t.Fatal(err) + } + environment, err := s.GetSessionEnvironment(t.Context(), tenant, session.ID) + if err != nil { + t.Fatal(err) + } + var frozen struct { + Workspace string `json:"workspace_directory"` + Directories []string `json:"capability_directories"` + } + if json.Unmarshal(environment.Configuration, &frozen) != nil || frozen.Workspace != "/home/user/project" || len(frozen.Directories) != 1 || frozen.Directories[0] != "/opt/skills" { + t.Fatal("frozen paths lost") + } + var setupCount int + if err := pool.QueryRow(t.Context(), "SELECT count(*) FROM environment_setups WHERE session_id=$1", session.ID).Scan(&setupCount); err != nil || setupCount != 0 { + t.Fatal("self-hosted initialization row created", setupCount, err) + } + if _, err := s.GetRuntimeAllocation(t.Context(), tenant, environment.ID); !errors.Is(err, store.ErrNotFound) { + t.Fatal("self-hosted managed allocation created", err) + } +} diff --git a/services/agents-api/internal/store/runtime_compute_lifecycle_test.go b/services/agents-api/internal/store/runtime_compute_lifecycle_test.go index 15bf70eb1..a5deba40b 100644 --- a/services/agents-api/internal/store/runtime_compute_lifecycle_test.go +++ b/services/agents-api/internal/store/runtime_compute_lifecycle_test.go @@ -26,6 +26,7 @@ import ( // The controlled provider records external effects independently of DB phases. // Lost replies retain those effects so recovery must use observation, not replay. type fakeCheckpointProvider struct { + preparation *initializationPeer lifecycleProvider computes map[string]sandbox.ComputeState snapshots map[string]sandbox.SnapshotIdentity @@ -191,11 +192,23 @@ func (p *fakeCheckpointProvider) connect(ctx context.Context, b sandbox.Bootstra } go func() { defer conn.Close() + transfer := initializationTransfer{peer: p.preparation} for { var env proto.Envelope if conn.ReadJSON(&env) != nil { return } + if env.Type == proto.TypeRuntimePrepare && p.preparation != nil { + reply, err := transfer.receive(env) + if err != nil { + p.preparation.t.Error(err) + return + } + if conn.WriteJSON(reply) != nil { + return + } + continue + } if env.Type == proto.TypeDeviceShutdown { return } diff --git a/services/agents-api/internal/store/runtime_initialization_peer_test.go b/services/agents-api/internal/store/runtime_initialization_peer_test.go new file mode 100644 index 000000000..cdb9f5368 --- /dev/null +++ b/services/agents-api/internal/store/runtime_initialization_peer_test.go @@ -0,0 +1,119 @@ +package store_test + +import ( + "context" + "crypto/sha256" + "encoding/hex" + "errors" + "net/http" + "sync/atomic" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/gateway" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" + "github.com/gorilla/websocket" +) + +// initializationPeer exercises the real authenticated gateway and chunk receipts. +// The provider fixture bootstraps its socket; all initialization runs on that peer. +type initializationPeer struct { + t *testing.T + endpoint string + registry *gateway.Registry + apply func(proto.RuntimePreparePayload, []byte) proto.RuntimePrepareResultPayload + writes atomic.Int32 + commandCalls atomic.Int32 + deferred bool + bootstrap sandbox.Bootstrap +} + +func (p *initializationPeer) setRuntimeGateway(t *testing.T, endpoint string, registry *gateway.Registry) { + p.t, p.endpoint, p.registry = t, endpoint, registry +} +func (p *initializationPeer) connect(b sandbox.Bootstrap) error { + p.bootstrap = b + if p.deferred { + return nil + } + c, _, err := websocket.DefaultDialer.Dial(p.endpoint+"?device_id="+b.DeviceID+"&version="+proto.Version, http.Header{"Authorization": {"Bearer " + b.Credential}}) + if err != nil { + return err + } + p.t.Cleanup(func() { _ = c.Close() }) + go func() { + transfer := initializationTransfer{peer: p} + for { + var env proto.Envelope + if c.ReadJSON(&env) != nil { + return + } + if env.Type != proto.TypeRuntimePrepare { + continue + } + reply, err := transfer.receive(env) + if err != nil { + p.t.Error(err) + return + } + if c.WriteJSON(reply) != nil { + return + } + } + }() + end := time.Now().Add(time.Second) + for time.Now().Before(end) { + if _, err := p.registry.LookupDevice(b.DeviceID); err == nil { + return nil + } + time.Sleep(time.Millisecond) + } + return context.DeadlineExceeded +} +func (p *initializationPeer) RunCommand(context.Context, sandbox.Reference, sandbox.Command) (sandbox.CommandResult, error) { + p.commandCalls.Add(1) + return sandbox.CommandResult{}, sandbox.ErrInvalid +} +func completedInitialization(proto.RuntimePreparePayload, []byte) proto.RuntimePrepareResultPayload { + return proto.RuntimePrepareResultPayload{Outcome: "completed"} +} + +// Each authenticated socket owns its own bounded preparation transfer. +type initializationTransfer struct { + peer *initializationPeer + request proto.RuntimePreparePayload + data []byte +} + +func (x *initializationTransfer) receive(env proto.Envelope) (proto.Envelope, error) { + var frame proto.RuntimePreparePayload + if env.DecodePayload(&frame) != nil || !proto.ValidRuntimePrepareRequest(frame) { + return proto.Envelope{}, errors.New("invalid Runtime frame") + } + result := proto.RuntimePrepareResultPayload{} + switch frame.Step { + case "begin": + x.request, x.data = frame, nil + result.Outcome = "ready" + case "chunk": + if frame.Offset != len(x.data) { + return proto.Envelope{}, errors.New("unordered initialization bytes") + } + x.data = append(x.data, frame.Data...) + result.Outcome, result.Offset = "received", len(x.data) + case "commit": + if x.request.Action == "file" || x.request.Action == "skill" || x.request.Action == "plugin" { + sum := sha256.Sum256(x.data) + if x.request.SizeBytes != len(x.data) || x.request.SHA256 != hex.EncodeToString(sum[:]) { + return proto.Envelope{}, errors.New("initialization digest changed") + } + } + result = x.peer.apply(x.request, x.data) + x.peer.writes.Add(1) + if result.Outcome == "completed" { + result.SizeBytes = len(x.data) + } + } + return proto.NewEnvelope(proto.TypeRuntimePrepareResult, env.ID, result) +} diff --git a/services/agents-api/internal/store/runtime_initialization_test.go b/services/agents-api/internal/store/runtime_initialization_test.go index 52a41f8a1..d58d36d33 100644 --- a/services/agents-api/internal/store/runtime_initialization_test.go +++ b/services/agents-api/internal/store/runtime_initialization_test.go @@ -1,17 +1,18 @@ package store_test import ( + "archive/zip" "bytes" "context" - "crypto/sha256" "encoding/json" "errors" - "fmt" - "strconv" + "github.com/MiniMax-AI-Dev/parsar/internal/agentplugin" + "reflect" "strings" "testing" v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/credentialcrypto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" @@ -20,38 +21,18 @@ import ( type initializingProvider struct { lifecycleProvider - writes int - fail bool - check func() + initializationPeer } -func (p *initializingProvider) RunCommand(_ context.Context, _ sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - p.writes++ - if p.check != nil { - p.check() +func (p *initializingProvider) Create(ctx context.Context, b sandbox.Bootstrap) (sandbox.Info, error) { + info, err := p.lifecycleProvider.Create(ctx, b) + if err == nil { + err = p.connect(b) } - if p.fail { - return sandbox.CommandResult{}, sandbox.ErrCommandUnconfirmed - } - if c.Args[len(c.Args)-1] == "/usr/local/bin/oac-runtime-initialize" { - var operation struct { - Version int `json:"version"` - Action string `json:"action"` - } - if json.Unmarshal(c.Stdin, &operation) != nil || operation.Version != 1 || operation.Action == "" { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - return sandbox.CommandResult{Stdout: `{"version":1,"outcome":"completed"}`}, nil - } - size, err := strconv.Atoi(c.Args[len(c.Args)-1]) - if err != nil || len(c.Stdin) != size+32 { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - digest := sha256.Sum256(c.Stdin[:size]) - if !bytes.Equal(digest[:], c.Stdin[size:]) { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - return sandbox.CommandResult{Stdout: fmt.Sprintf(`{"version":1,"outcome":"completed","size_bytes":%d}`, size)}, nil + return info, err +} +func (p *initializingProvider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { + return p.initializationPeer.RunCommand(ctx, r, c) } func TestManagedInitialFilesGateFairnessCompletionAndRestart(t *testing.T) { @@ -81,7 +62,7 @@ func TestManagedInitialFilesGateFairnessCompletionAndRestart(t *testing.T) { if err != nil { t.Fatal(err) } - p := &initializingProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, fail: mode == "uncertain"} + p := &initializingProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, initializationPeer: initializationPeer{deferred: true}} key := uuid.NewString() w, stop := managedWorker(t, s, key, p) owner, err := w.ProvisionEnvironment(t.Context(), tenant, env.ID, key) @@ -92,8 +73,11 @@ func TestManagedInitialFilesGateFairnessCompletionAndRestart(t *testing.T) { if err != nil || !ok || credential.ID != owner.DeviceID { t.Fatal("pending initialization blocks daemon authentication") } + targetCredential := p.credential lastStepGets := 0 - p.check = func() { + p.apply = func(_ proto.RuntimePreparePayload, _ []byte) proto.RuntimePrepareResultPayload { + p.mu.Lock() + defer p.mu.Unlock() if p.gets-lastStepGets < 33 { t.Fatal("initialization advanced before a full allocation scan", p.gets-lastStepGets) } @@ -104,6 +88,10 @@ func TestManagedInitialFilesGateFairnessCompletionAndRestart(t *testing.T) { if _, err := s.GetSessionExecutionBinding(t.Context(), tenant, session.ID); !errors.Is(err, store.ErrNotFound) { t.Fatal("premature native preparation", err) } + if mode == "uncertain" { + return proto.RuntimePrepareResultPayload{Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"} + } + return completedInitialization(proto.RuntimePreparePayload{}, nil) } // A full page of other allocations is serviced between initialization steps. for range 32 { @@ -112,13 +100,27 @@ func TestManagedInitialFilesGateFairnessCompletionAndRestart(t *testing.T) { t.Fatal(err) } } - for n := 0; p.writes == 0 && n < 100; n++ { + // A missing socket must leave every kind of initialization unclaimed. + for range 3 { if err := w.ReconcileManagedRuntimes(t.Context()); err != nil { t.Fatal(err) } } - if p.writes != 1 { - t.Fatal("initialization did not perform one bounded file step", p.writes) + pending, err := s.GetRuntimeAllocation(t.Context(), tenant, env.ID) + if err != nil || pending.Initialization != "pending" || p.writes.Load() != 0 { + t.Fatal("missing peer claimed initialization", pending, err) + } + p.deferred = false + if err := p.connect(sandbox.Bootstrap{DeviceID: owner.DeviceID, Credential: targetCredential}); err != nil { + t.Fatal(err) + } + for n := 0; int(p.writes.Load()) == 0 && n < 100; n++ { + if err := w.ReconcileManagedRuntimes(t.Context()); err != nil { + t.Fatal(err) + } + } + if int(p.writes.Load()) != 1 { + t.Fatal("initialization did not perform one bounded file step", int(p.writes.Load())) } afterFirst := p.gets if mode == "restart" { @@ -126,14 +128,14 @@ func TestManagedInitialFilesGateFairnessCompletionAndRestart(t *testing.T) { w, _ = managedWorker(t, s, key, p) } if mode == "complete" { - for n := 0; p.writes < expectedSteps && n < 100; n++ { + for n := 0; int(p.writes.Load()) < expectedSteps && n < 100; n++ { if err := w.ReconcileManagedRuntimes(t.Context()); err != nil { t.Fatal(err) } } got, err := s.GetRuntimeAllocation(t.Context(), tenant, env.ID) - if err != nil || got.Initialization != "complete" || p.writes != expectedSteps || p.gets <= afterFirst { - t.Fatal("completion or maintenance", got, err, p.writes) + if err != nil || got.Initialization != "complete" || int(p.writes.Load()) != expectedSteps || p.gets <= afterFirst { + t.Fatal("completion or maintenance", got, err, int(p.writes.Load())) } if _, err := s.GetSessionExecutionBinding(t.Context(), tenant, session.ID); err != nil { t.Fatal("ready execution still blocked", err) @@ -145,19 +147,87 @@ func TestManagedInitialFilesGateFairnessCompletionAndRestart(t *testing.T) { t.Fatal(err) } } - if p.writes != expectedSteps { + if int(p.writes.Load()) != expectedSteps { t.Fatal("completed initialization replayed") } } else { reconcileManagedState(t, w, s, tenant, env.ID, "released") - if p.writes != 1 || p.kills != 1 { - t.Fatal("uncertain initialization replayed or released twice", p.writes, p.kills) + if int(p.writes.Load()) != 1 || p.kills != 1 { + t.Fatal("uncertain initialization replayed or released twice", int(p.writes.Load()), p.kills) } failed, err := s.GetEnvironment(t.Context(), tenant, env.ID) if err != nil || failed.Status != "failed" { t.Fatal("failed initialization exposed", failed, err) } } + if p.commandCalls.Load() != 0 { + t.Fatal("initialization invoked Provider.RunCommand") + } }) } } + +func TestManagedRuntimePreparationAllOperationsUsePeer(t *testing.T) { + s := hostedFailureStore(t) + var archive bytes.Buffer + writer := zip.NewWriter(&archive) + for path, body := range map[string]string{"proof/.codex-plugin/plugin.json": `{"name":"plugin","description":"A plugin.","skills":"./skills"}`, "proof/skills/example/SKILL.md": "---\nname: plugin-proof\ndescription: A plugin Skill.\n---\nProof."} { + file, err := writer.Create(path) + if err != nil { + t.Fatal(err) + } + if _, err := file.Write([]byte(body)); err != nil { + t.Fatal(err) + } + } + if err := writer.Close(); err != nil { + t.Fatal(err) + } + tenant := uuid.NewString() + fileBody := bytes.Repeat([]byte("bounded bytes"), 12000) + session, environment := hostedFailureSession(t, s, tenant, store.CreateSessionInput{ + InitialFiles: []store.InitialFile{{Type: "inline", Path: "/workspace/first", Data: fileBody}}, + Initialization: store.EnvironmentSetup{Skills: []store.EnvironmentSkill{hostedFailureSkill(t)}, Plugins: []store.EnvironmentPlugin{{Metadata: agentplugin.Metadata{Type: "inline", Name: "plugin", Description: "A plugin."}, Archive: archive.Bytes()}}, Packages: v1.EnvironmentPackages{System: []string{"jq"}, NPM: []string{"is-number@7.0.0"}, Python: []string{"packaging==24.2"}}, Commands: []store.SetupCommand{{Command: "read installed bundles and create directory"}}, CapabilityDirectories: []string{"/workspace/generated"}}, + }) + provider := &initializingProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}} + var actions []string + provider.apply = func(request proto.RuntimePreparePayload, data []byte) proto.RuntimePrepareResultPayload { + if request.SessionID != session.ID || request.EnvironmentID != environment.ID { + t.Error("Runtime identity changed") + } + action := request.Action + if request.Initialization != nil { + action = request.Initialization.Action + } + if action == "file" && !bytes.Equal(data, fileBody) { + t.Error("initial bytes changed") + } + actions = append(actions, action) + return completedInitialization(request, data) + } + key := uuid.NewString() + worker, _ := managedWorker(t, s, key, provider) + if _, err := worker.ProvisionEnvironment(t.Context(), tenant, environment.ID, key); err != nil { + t.Fatal(err) + } + for range 30 { + if err := worker.ReconcileManagedRuntimes(t.Context()); err != nil { + t.Fatal(err) + } + allocation, err := s.GetRuntimeAllocation(t.Context(), tenant, environment.ID) + if err != nil { + t.Fatal(err) + } + if allocation.Initialization == "complete" { + break + } + } + expected := []string{"file", "configure", "skill", "plugin", "system", "npm", "python", "setup", "finalize"} + if !reflect.DeepEqual(actions, expected) || provider.commandCalls.Load() != 0 { + t.Fatal("typed ordering or provider isolation", actions, provider.commandCalls.Load()) + } + allocation, err := s.GetRuntimeAllocation(t.Context(), tenant, environment.ID) + if err != nil || allocation.Initialization != "complete" { + t.Fatal("initialization incomplete", allocation, err) + } +} diff --git a/services/agents-api/internal/store/runtime_lifecycle_test.go b/services/agents-api/internal/store/runtime_lifecycle_test.go index 58da0521d..37d00da64 100644 --- a/services/agents-api/internal/store/runtime_lifecycle_test.go +++ b/services/agents-api/internal/store/runtime_lifecycle_test.go @@ -4,6 +4,9 @@ import ( "context" "encoding/json" "errors" + "net/http" + "net/http/httptest" + "strings" "sync" "testing" @@ -75,7 +78,16 @@ func managedWorker(t *testing.T, s *store.Store, key string, p sandbox.Provider) func managedWorkerMode(t *testing.T, s *store.Store, key string, p sandbox.Provider, maintenance bool) (*execution.Worker, func()) { t.Helper() - w, err := execution.StartWorker(t.Context(), &execution.Dispatcher{Store: s, Registry: gateway.NewRegistry(), ManagedRuntimes: &execution.RuntimeProvider{CoreURL: "http://core.invalid/api/v1", InstallationID: key, BackendFingerprint: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Provider: p, AdmissionPaused: maintenance}}) + registry := gateway.NewRegistry() + if peer, ok := p.(interface { + setRuntimeGateway(*testing.T, string, *gateway.Registry) + }); ok { + handler := gateway.NewHandler(gateway.HandlerConfig{Authenticator: gateway.NewAuthenticator(s), Registry: registry}) + server := httptest.NewServer(http.HandlerFunc(handler.WS)) + t.Cleanup(server.Close) + peer.setRuntimeGateway(t, "ws"+strings.TrimPrefix(server.URL, "http"), registry) + } + w, err := execution.StartWorker(t.Context(), &execution.Dispatcher{Store: s, Registry: registry, ManagedRuntimes: &execution.RuntimeProvider{CoreURL: "http://core.invalid/api/v1", InstallationID: key, BackendFingerprint: "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", Provider: p, AdmissionPaused: maintenance}}) if err != nil { t.Fatal(err) } diff --git a/services/agents-api/internal/store/runtime_node_lifecycle_fixture_test.go b/services/agents-api/internal/store/runtime_node_lifecycle_fixture_test.go index 638d2013d..3208ee543 100644 --- a/services/agents-api/internal/store/runtime_node_lifecycle_fixture_test.go +++ b/services/agents-api/internal/store/runtime_node_lifecycle_fixture_test.go @@ -5,10 +5,8 @@ import ( "context" "encoding/json" "errors" - "fmt" "net/http" "net/http/httptest" - "strconv" "strings" "sync" "sync/atomic" @@ -17,6 +15,7 @@ import ( "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/device" "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/gateway" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/credentialcrypto" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/execution" "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/sandbox" @@ -70,28 +69,21 @@ func (p *nodeIsolationProvider) GetCompute(ctx context.Context, r sandbox.Refere return p.fakeCheckpointProvider.GetCompute(ctx, r, c) } func (p *nodeIsolationProvider) RunCommand(ctx context.Context, r sandbox.Reference, c sandbox.Command) (sandbox.CommandResult, error) { - if err := p.block(ctx, r, "initialize"); err != nil { - return sandbox.CommandResult{}, err - } - size, err := strconv.Atoi(c.Args[len(c.Args)-1]) - if err != nil || len(c.Stdin) != size+32 { - return sandbox.CommandResult{}, sandbox.ErrInvalid - } - p.writes.Add(1) - return sandbox.CommandResult{Stdout: fmt.Sprintf("{\"version\":1,\"outcome\":\"completed\",\"size_bytes\":%d}", size)}, nil + return p.preparation.RunCommand(ctx, r, c) } type nodeIsolationFixture struct { - t *testing.T - store *store.Store - pool *pgxpool.Pool - worker *execution.Worker - provider *nodeIsolationProvider - key, nodeA, nodeB string - epoch uint64 - runCancel context.CancelFunc - runDone chan error - stopOnce sync.Once + t *testing.T + store *store.Store + pool *pgxpool.Pool + worker *execution.Worker + provider *nodeIsolationProvider + key, nodeA, nodeB string + epoch uint64 + initializationCancel context.CancelFunc + runCancel context.CancelFunc + runDone chan error + stopOnce sync.Once } func newNodeIsolationFixture(t *testing.T, mode string) *nodeIsolationFixture { @@ -105,6 +97,19 @@ func newNodeIsolationFixture(t *testing.T, mode string) *nodeIsolationFixture { registry := gateway.NewRegistry() cp := &fakeCheckpointProvider{lifecycleProvider: lifecycleProvider{resources: map[string]sandbox.Info{}}, computes: map[string]sandbox.ComputeState{}, snapshots: map[string]sandbox.SnapshotIdentity{}, bootstraps: map[string]sandbox.Bootstrap{}, peers: map[string]*websocket.Conn{}, registry: registry} p := &nodeIsolationProvider{fakeCheckpointProvider: cp, blocked: map[string]bool{}, mode: mode, entered: make(chan struct{})} + preparationContext, cancelPreparation := context.WithCancel(t.Context()) + t.Cleanup(cancelPreparation) + cp.preparation = &initializationPeer{t: t, apply: func(request proto.RuntimePreparePayload, data []byte) proto.RuntimePrepareResultPayload { + if err := p.block(preparationContext, sandbox.Reference{EnvironmentID: request.EnvironmentID}, "initialize"); err != nil { + return proto.RuntimePrepareResultPayload{Outcome: "unknown", ErrorCode: "runtime_preparation_unconfirmed"} + } + if request.Action != "file" || request.File.Path != "/workspace/seed" || string(data) != "retained" { + t.Error("unexpected node initialization request") + return proto.RuntimePrepareResultPayload{Outcome: "failed", ErrorCode: "runtime_preparation_failed"} + } + p.writes.Add(1) + return completedInitialization(request, data) + }} handler := gateway.NewHandler(gateway.HandlerConfig{Authenticator: gateway.NewAuthenticator(s), Registry: registry}) server := httptest.NewServer(http.HandlerFunc(handler.WS)) cp.endpoint = "ws" + strings.TrimPrefix(server.URL, "http") @@ -116,7 +121,7 @@ func newNodeIsolationFixture(t *testing.T, mode string) *nodeIsolationFixture { cp.mu.Unlock() server.Close() }) - f := &nodeIsolationFixture{t: t, store: s, pool: pool, provider: p, key: uuid.NewString(), nodeA: uuid.NewString(), nodeB: uuid.NewString()} + f := &nodeIsolationFixture{initializationCancel: cancelPreparation, t: t, store: s, pool: pool, provider: p, key: uuid.NewString(), nodeA: uuid.NewString(), nodeB: uuid.NewString()} // Keep restored compute awake throughout the isolation assertions. // The suspension setup explicitly dates its activity two minutes in the past. policy := &execution.RuntimeSuspensionPolicy{IdleTimeout: time.Minute, Retention: time.Hour, MaxActive: 100, MaxRetained: 100} @@ -252,6 +257,7 @@ func (f *nodeIsolationFixture) run() { } func (f *nodeIsolationFixture) stop() { f.stopOnce.Do(func() { + f.initializationCancel() if f.runDone == nil { ctx, cancel := context.WithCancel(context.Background()) cancel() diff --git a/services/agents-api/internal/store/runtime_node_lifecycle_test.go b/services/agents-api/internal/store/runtime_node_lifecycle_test.go index 385d1d76d..3ba6bdfa6 100644 --- a/services/agents-api/internal/store/runtime_node_lifecycle_test.go +++ b/services/agents-api/internal/store/runtime_node_lifecycle_test.go @@ -45,7 +45,7 @@ func TestManagedNodesIsolateBlockedProviderAndInitialization(t *testing.T) { select { case <-f.provider.entered: case <-time.After(8 * time.Second): - t.Fatal("node A did not enter blocked provider") + t.Fatalf("node A did not enter blocked %s operation", mode) } online, err := f.store.RuntimeNodeAvailable(t.Context(), f.nodeA) if err != nil || !online { @@ -53,7 +53,7 @@ func TestManagedNodesIsolateBlockedProviderAndInitialization(t *testing.T) { } // Same-node direct callers and the scan all consume the same one-shot - // allocation receipt. A's in-flight provider operation remains blocked. + // allocation receipt. A's provider or Runtime operation remains blocked. tenant, _, env := f.session(f.nodeB, true) f.provider.mu.Lock() before := f.provider.creates @@ -120,6 +120,9 @@ func TestManagedNodesIsolateBlockedProviderAndInitialization(t *testing.T) { f.provider.mu.Lock() restores := f.provider.restores f.provider.mu.Unlock() + if f.provider.preparation.commandCalls.Load() != 0 { + t.Fatal("initialization invoked Provider.RunCommand") + } if restores != 1 || f.provider.promptFrames.Load() != 0 { t.Fatal("restore replayed or lifecycle sent model work") } diff --git a/services/agents-api/internal/store/session_deletion_execution_test.go b/services/agents-api/internal/store/session_deletion_execution_test.go index 11ef5efa6..d33489c8a 100644 --- a/services/agents-api/internal/store/session_deletion_execution_test.go +++ b/services/agents-api/internal/store/session_deletion_execution_test.go @@ -31,7 +31,7 @@ func TestDeletedSessionWaitingTurnSettlesWithoutStoppingWorker(t *testing.T) { } }() input := h.message("start", "Run") - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(input.TurnID, proto.TypeFunctionCall, proto.FunctionCallPayload{CallID: "pending", Name: "lookup_ticket", Arguments: json.RawMessage(`{}`)}) state := functionState(t, h, 1) if err := h.s.DeleteSession(ctx, h.tenant, h.session.ID); !errors.Is(err, store.ErrSessionNotIdle) { @@ -63,7 +63,7 @@ func TestDeletedSessionWaitingTurnSettlesWithoutStoppingWorker(t *testing.T) { } h.session = publicSession(t, h, "unrelated") next := h.message("next", "Unrelated work") - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(next.TurnID, proto.TypeDone, proto.DonePayload{Content: "unaffected"}) waitTurn(t, h, next.TurnID, store.TurnCompleted) } @@ -120,7 +120,7 @@ func TestWaitingSessionCancelsThenDeletesThroughWorker(t *testing.T) { } }() input := h.message("start", "Run") - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(input.TurnID, proto.TypeFunctionCall, proto.FunctionCallPayload{CallID: "pending", Name: "lookup_ticket", Arguments: json.RawMessage(`{}`)}) state := functionState(t, h, 1) if err := h.s.DeleteSession(ctx, h.tenant, h.session.ID); !errors.Is(err, store.ErrSessionNotIdle) { @@ -165,7 +165,7 @@ func TestWaitingSessionCancelsThenDeletesThroughWorker(t *testing.T) { } h.session = publicSession(t, h, "unrelated") next := h.message("next", "Unrelated work") - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) h.write(next.TurnID, proto.TypeDone, proto.DonePayload{Content: "unaffected"}) waitTurn(t, h, next.TurnID, store.TurnCompleted) } diff --git a/services/agents-api/internal/store/steering_receipts_test.go b/services/agents-api/internal/store/steering_receipts_test.go index 9b330c3d0..df620a347 100644 --- a/services/agents-api/internal/store/steering_receipts_test.go +++ b/services/agents-api/internal/store/steering_receipts_test.go @@ -19,7 +19,7 @@ func TestExecutionDurableInputReceiptLifetime(t *testing.T) { defer cancel() first := h.message("first", "initial") result := h.run(ctx, first.TurnID) - h.read(proto.TypePromptRequest) + h.read(testExecutionRequest) extra := h.message("extra", "additional") var input proto.PromptSteerPayload if err := h.read(proto.TypePromptSteer).DecodePayload(&input); err != nil || !input.DurableReceipt { diff --git a/services/agents-api/internal/store/structured_output_dispatch_test.go b/services/agents-api/internal/store/structured_output_dispatch_test.go index 91f2c1109..78d26936e 100644 --- a/services/agents-api/internal/store/structured_output_dispatch_test.go +++ b/services/agents-api/internal/store/structured_output_dispatch_test.go @@ -28,7 +28,7 @@ func TestStructuredOutputDispatchRechecksOperationQualification(t *testing.T) { if err = h.s.BindSessionDevice(t.Context(), h.tenant, h.session.ID, h.device.ID); err != nil { t.Fatal(err) } - caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, ToolObservations: true, StructuredOutput: true, MessageItems: true} + caps := proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, ExecutionControls: true, EnvironmentNone: true, SubagentControl: true, ToolObservations: true, StructuredOutput: true, MessageItems: true, Preparation: true} h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "fixture_harness", Available: true, Capabilities: caps}}}) peer, _ := h.registry.LookupDevice(h.device.ID) for deadline := time.Now().Add(3 * time.Second); ; { diff --git a/services/agents-api/internal/store/subagent_dispatch_test.go b/services/agents-api/internal/store/subagent_dispatch_test.go index a032b0b3a..146b8510b 100644 --- a/services/agents-api/internal/store/subagent_dispatch_test.go +++ b/services/agents-api/internal/store/subagent_dispatch_test.go @@ -36,7 +36,7 @@ func TestSubagentIdentityUsesLeasedDispatchJournal(t *testing.T) { input := h.message("first", "root message") running := h.run(ctx, input.TurnID) var request proto.PromptRequestPayload - if err = h.read(proto.TypePromptRequest).DecodePayload(&request); err != nil || request.ObserveSubagentIdentities != enabled || request.MaxConcurrentSubagents == nil || *request.MaxConcurrentSubagents != 3 { + if err = h.read(testExecutionRequest).DecodePayload(&request); err != nil || request.ObserveSubagentIdentities != enabled || request.MaxConcurrentSubagents == nil || *request.MaxConcurrentSubagents != 3 { t.Fatal("private observation policy not carried", err) } identity := proto.SubagentIdentityPayload{NativeID: "child", ParentNativeID: "root", NativeCreatedAt: 100, ParentTurnID: "native-turn", SourceItemID: "spawn-item"} diff --git a/services/agents-api/internal/store/worker_capacity_test.go b/services/agents-api/internal/store/worker_capacity_test.go new file mode 100644 index 000000000..ccb884df2 --- /dev/null +++ b/services/agents-api/internal/store/worker_capacity_test.go @@ -0,0 +1,189 @@ +package store_test + +import ( + "fmt" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" +) + +// Consume the actual controls so capacity rejection is exercised before any +// input, without the ordinary fixture's automatic preparation acknowledgement. +func capacityWorkerFrames(t *testing.T, h *dispatchHarness) <-chan proto.Envelope { + t.Helper() + frames := make(chan proto.Envelope, 64) + go func() { + defer close(frames) + for { + var frame proto.Envelope + if h.conn.ReadJSON(&frame) != nil { + return + } + select { + case frames <- frame: + case <-t.Context().Done(): + return + } + } + }() + return frames +} + +func TestWorkerDefersPreparationCapacityUntilCleanupReleasesSlot(t *testing.T) { + h := newDispatchHarness(t) + enableWorkerEnvironment(t, h) + h.d.MaxConcurrentExecutions = 5 + sessions := make(map[string]string) + for i := range 5 { + h.session = publicSession(t, h, fmt.Sprintf("capacity-%d", i)) + receipt := h.message("work", "once") + sessions[h.session.ID] = receipt.TurnID + } + frames := capacityWorkerFrames(t, h) + _, stop := startEnvironmentExpiryWorker(t, h.d) + defer stop() + admissions := make(map[string]string) + started := make(map[string]int) + var blocked, first string + capacityReleased := false + rejections := 0 + process := func(frame proto.Envelope) { + switch frame.Type { + case proto.TypeExecutionPrepare: + var prepare proto.ExecutionPreparePayload + if frame.DecodePayload(&prepare) != nil || sessions[prepare.SessionID] == "" || len(prepare.Configuration.Input) != 0 || prepare.Configuration.RunID != "" { + t.Fatal("invalid input-free preparation", prepare) + } + if blocked == "" && len(admissions) == 4 { + blocked = prepare.SessionID + } + if prepare.SessionID == blocked && !capacityReleased { + rejections++ + h.write(frame.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{State: "rejected", Operation: proto.TypeExecutionPrepare, ErrorCode: "preparation_capacity"}) + return + } + admissions[frame.ID] = prepare.SessionID + h.write(frame.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: frame.ID, ExecutorID: "executor-" + prepare.SessionID, Revision: 1, State: "ready"}) + case proto.TypeExecutionStart: + var start proto.ExecutionStartPayload + session := admissions[frame.ID] + if frame.DecodePayload(&start) != nil || session == "" || start.RunID != sessions[session] || inputTextForTest(t, start.Input) != "once" { + t.Fatal("Start changed the original input or Turn", start) + } + started[session]++ + if started[session] != 1 || (session == blocked && !capacityReleased) { + t.Fatal("capacity rejection replayed or prematurely started input") + } + if first == "" { + first = session + } + h.write(frame.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: start.Handle, ExecutorID: start.ExecutorID, Revision: 2, State: "started", RunID: start.RunID}) + case proto.TypeExecutionRelease: + default: + t.Fatal("unexpected worker frame", frame.Type) + } + } + next := func() { + t.Helper() + select { + case frame, ok := <-frames: + if !ok { + t.Fatal("Runtime connection closed") + } + process(frame) + case <-time.After(5 * time.Second): + t.Fatal("worker did not send preparation or Start") + } + } + for len(started) != 4 || rejections == 0 { + next() + } + assertQueued := func() { + t.Helper() + turn, err := h.s.GetTurn(t.Context(), h.tenant, blocked, sessions[blocked]) + if err != nil || turn.Status != store.TurnQueued { + t.Fatal("capacity rejection failed queued work", turn, err) + } + } + // Observe multiple scheduler ticks with four active native owners. The + // bounded attempt count detects a completion-triggered busy retry loop. + observeCapacity := func() { + t.Helper() + before := rejections + timer := time.NewTimer(800 * time.Millisecond) + defer timer.Stop() + for { + select { + case frame := <-frames: + process(frame) + if rejections-before > 6 { + t.Fatal("capacity rejection caused a busy retry loop") + } + case <-timer.C: + if rejections == before { + t.Fatal("queued work was not retried by scheduler") + } + assertQueued() + return + } + } + } + observeCapacity() + // After completion, Runtime can retire the idle Executor. Its cleanup + // still owns a slot and reports the same capacity rejection. + h.write(sessions[first], proto.TypeDone, proto.DonePayload{Content: "done"}) + awaitDaemonRemoteCondition(t, t.Context(), 5*time.Second, "first completed Turn", func() bool { + turn, err := h.s.GetTurn(t.Context(), h.tenant, first, sessions[first]) + return err == nil && turn.Status == store.TurnCompleted + }) + observeCapacity() + capacityReleased = true + for started[blocked] == 0 { + next() + } + for session, turn := range sessions { + if session != first { + h.write(turn, proto.TypeDone, proto.DonePayload{Content: "done"}) + } + } + awaitDaemonRemoteCondition(t, t.Context(), 5*time.Second, "all five Turns completed", func() bool { + for session, id := range sessions { + turn, err := h.s.GetTurn(t.Context(), h.tenant, session, id) + if err != nil || turn.Status != store.TurnCompleted { + return false + } + } + return true + }) + if len(started) != 5 || started[blocked] != 1 { + t.Fatal("queued Turn did not start exactly once", started) + } +} + +func TestWorkerDoesNotDeferOtherPreparationOrStartRejections(t *testing.T) { + for _, test := range []struct{ name, operation, code string }{ + {"invalid_configuration", proto.TypeExecutionPrepare, "invalid_configuration"}, + {"resource_unavailable", proto.TypeExecutionPrepare, "resource_unavailable"}, + {"unknown_operation", "", "preparation_capacity"}, + {"start_capacity", proto.TypeExecutionStart, "preparation_capacity"}, + } { + t.Run(test.name, func(t *testing.T) { + h := newDispatchHarness(t) + enableWorkerEnvironment(t, h) + h.session = publicSession(t, h, test.name) + receipt := h.message("work", "once") + frames := capacityWorkerFrames(t, h) + _, stop := startEnvironmentExpiryWorker(t, h.d) + defer stop() + prepare := nextWorkerFrame(t, frames, proto.TypeExecutionPrepare) + if test.operation == proto.TypeExecutionStart { + h.write(prepare.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: prepare.ID, Revision: 1, State: "ready"}) + nextWorkerFrame(t, frames, proto.TypeExecutionStart) + } + h.write(prepare.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{State: "rejected", Operation: test.operation, ErrorCode: test.code}) + waitTurn(t, h, receipt.TurnID, store.TurnFailed) + }) + } +} diff --git a/services/agents-api/internal/store/worker_input_race_test.go b/services/agents-api/internal/store/worker_input_race_test.go index 8e87c1277..d983e037b 100644 --- a/services/agents-api/internal/store/worker_input_race_test.go +++ b/services/agents-api/internal/store/worker_input_race_test.go @@ -100,7 +100,7 @@ func TestWorkerInputReadSkipsConcurrentlyCancelledCandidate(t *testing.T) { t.Fatal("a stale candidate terminated the global Worker", err) case <-time.After(650 * time.Millisecond): } - frame := h.read(proto.TypePromptRequest) + frame := h.read(testExecutionRequest) if frame.ID != healthy.TurnID { t.Fatal("cancelled candidate reached the Runtime", frame.ID) } diff --git a/services/agents-api/internal/store/worker_lease_loss_test.go b/services/agents-api/internal/store/worker_lease_loss_test.go index d54918a11..f1dbf42cc 100644 --- a/services/agents-api/internal/store/worker_lease_loss_test.go +++ b/services/agents-api/internal/store/worker_lease_loss_test.go @@ -14,7 +14,7 @@ import ( func TestWorkerLeaseLossLeavesUncertainWorkForSuccessor(t *testing.T) { h := newDispatchHarness(t) _, pool := store.NewTestStore(t) - h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, EnvironmentNone: true}}}}) + h.write("", proto.TypeHeartbeat, proto.HeartbeatPayload{SupportedAgentKinds: []proto.SupportedAgentKind{{Kind: "codex", Available: true, Capabilities: proto.AgentKindCapabilities{Streaming: true, Steering: true, DurableTurns: true, DurableInputReceipts: true, WebSearchControl: true, TextVerbosity: true, ExecutionControls: true, SubagentControl: true, ToolObservations: true, EnvironmentNone: true, Preparation: true}}}}) h.session = publicSession(t, h, "active") queued := publicSession(t, h, "queued") worker, err := execution.StartWorker(t.Context(), h.d) @@ -37,7 +37,7 @@ func TestWorkerLeaseLossLeavesUncertainWorkForSuccessor(t *testing.T) { if err != nil { t.Fatal(err) } - request := h.read(proto.TypePromptRequest) + request := h.read(testExecutionRequest) if request.ID != receipts[0].TurnID { t.Fatal("unexpected dispatch", request.ID) } diff --git a/services/agents-api/internal/store/worker_wakeup_test.go b/services/agents-api/internal/store/worker_wakeup_test.go new file mode 100644 index 000000000..d53516f3d --- /dev/null +++ b/services/agents-api/internal/store/worker_wakeup_test.go @@ -0,0 +1,141 @@ +package store_test + +import ( + "context" + "encoding/json" + "strings" + "sync/atomic" + "testing" + "time" + + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/proto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/execution" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" + "github.com/jackc/pgx/v5" + "github.com/jackc/pgx/v5/pgxpool" +) + +type schedulerQueryOrder struct { + armed atomic.Bool + first chan string +} + +func (trace *schedulerQueryOrder) TraceQueryStart(ctx context.Context, _ *pgx.Conn, data pgx.TraceQueryStartData) context.Context { + for _, name := range []string{"ListExecutionWork", "ListDueEnvironmentInputs"} { + if strings.HasPrefix(data.SQL, "-- name: "+name+" ") && trace.armed.CompareAndSwap(true, false) { + trace.first <- name + } + } + return ctx +} +func (*schedulerQueryOrder) TraceQueryEnd(context.Context, *pgx.Conn, pgx.TraceQueryEndData) {} + +func TestWorkerSchedulerCommittedAdmissionWakesBeforeMaintenance(t *testing.T) { + for _, operation := range []string{"submit", "create", "stream", "rejected"} { + t.Run(operation, func(t *testing.T) { + h := newDispatchHarness(t) + enableWorkerEnvironment(t, h) + h.session = publicSession(t, h, "wakeup") + _, pool := store.NewTestStore(t) + trace := &schedulerQueryOrder{first: make(chan string, 1)} + config := pool.Config() + config.ConnConfig.Tracer = trace + instrumented, err := pgxpool.NewWithConfig(t.Context(), config) + if err != nil { + t.Fatal(err) + } + defer instrumented.Close() + h.d.Store = store.NewWithCredentialCipher(instrumented, store.FixtureCipher()) + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + worker, err := execution.StartWorker(ctx, h.d) + if err != nil { + t.Fatal(err) + } + done := make(chan error, 1) + started := false + defer func() { + cancel() + if !started { + go func() { done <- worker.Run(ctx) }() + } + select { + case <-done: + case <-time.After(10 * time.Second): + t.Error("worker did not stop") + } + }() + input := []store.Input{{Kind: "message", Payload: json.RawMessage(`{"text":"wake"}`)}} + switch operation { + case "submit": + _, err = worker.SubmitInputs(ctx, h.tenant, h.session.ID, "wake", input) + case "create", "stream": + creation := store.CreateSessionInput{Creator: store.FixtureCreator(), Engine: "codex", IdempotencyKey: "wake", Configuration: h.session.Configuration, InitialInputs: input} + if operation == "create" { + _, err = worker.CreateSession(ctx, h.tenant, creation) + } else { + _, err = worker.CreateSessionStream(ctx, h.tenant, creation) + } + case "rejected": + h.message("wake", "different") + _, err = worker.SubmitInputs(ctx, h.tenant, h.session.ID, "wake", input) + if err == nil { + t.Fatal("changed retry was accepted") + } + err = nil + } + if err != nil { + t.Fatal(err) + } + trace.armed.Store(true) + started = true + go func() { done <- worker.Run(ctx) }() + // Maintenance always starts with expiry. A committed admission must + // select work first, without depending on a short timing threshold. + select { + case first := <-trace.first: + want := "ListExecutionWork" + if operation == "rejected" { + want = "ListDueEnvironmentInputs" + } + if first != want { + t.Fatalf("first scheduler query = %s, want %s", first, want) + } + case <-time.After(5 * time.Second): + t.Fatal("worker did not schedule") + } + }) + } +} + +func TestWorkerSchedulerHintBypassesEnvironmentScanThrottle(t *testing.T) { + h := newDispatchHarnessForSession(t, []byte(`{"agent":{"model":"test-model"},"environment":{"type":"self_hosted","workspace_directory":"/workspace"}}`), false) + enableWorkerEnvironment(t, h) + frames := workerFrames(t, h) + worker, stop := startEnvironmentExpiryWorker(t, h.d) + defer stop() + awaitDaemonRemoteCondition(t, t.Context(), 5*time.Second, "initial empty scheduler scan", func() bool { + return worker.MetricsSnapshot().Scheduler.LastRunAt != nil + }) + ctx, cancel := context.WithCancel(t.Context()) + defer cancel() + admitted := make(chan error, 1) + started := time.Now() + go func() { + _, err := worker.SubmitInputs(ctx, h.tenant, h.session.ID, "wake", []store.Input{{Kind: "message", Payload: json.RawMessage(`{"text":"wake"}`)}}) + admitted <- err + }() + prepare := nextWorkerFrame(t, frames, proto.TypeExecutionPrepare) + if elapsed := time.Since(started); elapsed >= 750*time.Millisecond { + t.Fatalf("committed input waited for the environment scan interval: %s", elapsed) + } + handle := acknowledgePreparation(h, prepare.ID) + h.write(prepare.ID, proto.TypePreparationStatus, proto.PreparationStatusPayload{Handle: handle, Revision: 2, State: "failed"}) + nextWorkerFrame(t, frames, proto.TypeExecutionRelease) + cancel() + select { + case <-admitted: + case <-time.After(5 * time.Second): + t.Fatal("detached admission waiter did not stop") + } +} diff --git a/services/agents-api/tests/official_hosted_functions_native.py b/services/agents-api/tests/official_hosted_functions_native.py index 9f57c183f..a573d89ee 100644 --- a/services/agents-api/tests/official_hosted_functions_native.py +++ b/services/agents-api/tests/official_hosted_functions_native.py @@ -83,7 +83,7 @@ def failure(arguments): raise RuntimeError(private_error) _, result = invoke("Call lookup exactly once with key failure. If it fails, do not retry. Then reply with the exact string returned by lookup in our first turn, using conversation history and no file tools.", "function-failure", failure) - assert result["error"] == "Tool handler failed." and "output" not in result + assert result["error"] == "Tool handler failed." and "output" in result and result["output"] is None messages = [i for i in items() if i["type"] == "message" and i["role"] == "assistant"] assert marker in "\n".join(p.get("text", "") for p in messages[-1]["content"]) assert len(calls) == 2 diff --git a/services/agents-api/tests/official_user_runtime.py b/services/agents-api/tests/official_user_runtime.py index c006fce52..9101634b0 100644 --- a/services/agents-api/tests/official_user_runtime.py +++ b/services/agents-api/tests/official_user_runtime.py @@ -43,7 +43,9 @@ def expired(signum, frame): signal.signal(signal.SIGALRM, previous) -def run_acceptance(client, foreign, http, session, runtime, evidence_path, secret_markers=()): +def run_acceptance(client, foreign, http, session, runtime, evidence_path, secret_markers=(), + expected_environment_type="self_hosted", + protected_credential_path="/home/runtime/.oac/daemon/executor-key.json", turn_observer=None): """Return a safe report after real execution; raise on any failed assertion.""" assert threading.current_thread() is threading.main_thread(), "Run acceptance on the main thread" pin = json.loads((Path(__file__).resolve().parents[3] / "contracts/agents-api/upstream.json").read_text()) @@ -71,7 +73,7 @@ def run_acceptance(client, foreign, http, session, runtime, evidence_path, secre outputs = {"/workspace/outputs/a.bin": bytes(range(256)), "/workspace/outputs/b.txt": ("native-user-runtime-" + nonce + "\n").encode()} artifacts = {} - private_paths = ["/home/runtime/.oac/daemon/executor-key.json", + private_paths = [protected_credential_path, "/home/runtime/.oac/daemon/.user-runtime-isolation-canary", "/environment/staging/.user-runtime-isolation-canary"] @@ -149,6 +151,7 @@ def snapshot_items(name): def run_turn(text, number, cancellation=False): observed, stop, requested = [], threading.Event(), threading.Event() + submitted_at, first_text_at, terminal_at = None, None, None cancel_key = nonce + "-cancel" def cancel_running(): @@ -165,6 +168,7 @@ def cancel_running(): raise AssertionError("Cancellation ended before observing native effects") def consume(stream, journal, cancelling=None): + nonlocal first_text_at, terminal_at for event in stream: value = event.to_dict() journal.write(json.dumps(redact(value)) + "\n") @@ -172,6 +176,10 @@ def consume(stream, journal, cancelling=None): safe(value) observed.append(value) kind = value["type"] + if kind == "agent.session.turn.output_text.delta" and first_text_at is None and value.get("delta"): + first_text_at = time.monotonic() + if kind in ("agent.session.turn.completed", "agent.session.turn.cancelled", "agent.session.turn.failed"): + terminal_at = time.monotonic() assert kind not in ("error", "agent.session.failed", "agent.session.turn.failed"), "Native execution failed" if cancelling is not None and cancelling.done(): cancelling.result() @@ -194,6 +202,7 @@ def consume(stream, journal, cancelling=None): with _prompt_deadline(), path.open("w") as journal: path.chmod(0o600) with sessions.events.stream(sid, timeout=240) as stream: + submitted_at = time.monotonic() sessions.events.create(sid, events=[{"type": "agent.session.input.message", "input": [ {"role": "user", "content": [{"type": "input_text", "text": text}]}]}], idempotency_key=nonce + "-turn-" + str(number)) if cancellation: @@ -216,7 +225,11 @@ def consume(stream, journal, cancelling=None): assert current, "Native Turn has no public Items" if not cancellation: assert all(item.get("status") not in ("failed", "incomplete", "in_progress") for item in current), "Completed Turn contains failed native work" - report["turns"].append({"id": turn.id, "status": turn.status, "event_count": len(observed)}) + report["turns"].append({"id": turn.id, "status": turn.status, "event_count": len(observed), + "input_to_first_text_ms": round((first_text_at - submitted_at) * 1000, 3) if first_text_at is not None else None, + "input_to_terminal_ms": round((terminal_at - submitted_at) * 1000, 3) if terminal_at is not None else None}) + if turn_observer is not None: + report["turns"][-1]["runtime_observation"] = safe(turn_observer(number, turn.id)) return turn, current except BaseException: # No model request is retried; an already started tool is cancelled for cleanup. @@ -236,7 +249,10 @@ def check_artifacts(): safe([item.to_dict() for item in values]) try: - assert session.environment.type == "self_hosted" and session.environment.workspace_directory == "/workspace" + assert expected_environment_type in ("self_hosted", "openai_hosted") + assert session.environment.type == expected_environment_type + if expected_environment_type == "self_hosted": + assert session.environment.workspace_directory == "/workspace" assert client.beta.agents.environments.retrieve(eid).status == "connected", "Runtime is not connected" assert not list(sessions.turns.list(sid)) and not list(sessions.items.list(sid)), "Use an unused Session" assert not list(sessions.artifacts.list(sid)), "Use a Session without existing Artifacts" @@ -299,6 +315,12 @@ def isolation_proof(phase): pass else: raise AssertionError("Foreign tenant accessed SDK Items") + warm, items = run_turn("Return the entire exact conversation-only remember- token from our previous turn, including its prefix. " + "Do not use tools, read files, or repeat any earlier command.", 2) + assert memory in answer(items), "Warm continuation lost native conversation history" + assert runtime.read(prefix + "-published") == b"published\n", "Warm continuation repeated side effects" + check_artifacts() + report["checks"].append("ordinary_warm_turn_preserves_history_and_effect_counts") committed = snapshot_items("before-restart") report["checks"].append("native_outputs_files_artifacts_items_and_tenant_isolation") runtime.restart() @@ -307,7 +329,7 @@ def isolation_proof(phase): assert snapshot_items("after-restart") == committed, "Restart changed committed Items" second, items = run_turn("Run exactly `python3 " + prefix + "-recover.py` once with your native shell tool; do not edit or retry it. " "Then return the entire exact conversation-only remember- token from our previous turn, including its prefix. " - "Do not look for the token in files or rerun any earlier command.", 2) + "Do not look for the token in files or rerun any earlier command.", 3) assert any(item.get("type") == "command_execution" and item.get("status") == "completed" and prefix + "-recover.py" in item.get("command", "") for item in items), "Missing recovered native isolation command" isolation_proof("recovered") assert memory in answer(items), "Cold continuation lost native conversation history" @@ -316,18 +338,18 @@ def isolation_proof(phase): check_artifacts() report["checks"].append("runtime_and_core_restart_preserve_history_and_outputs") third, items = run_turn("Use your native shell tool to run exactly `python3 " + prefix + "-hold.py` once and wait in the foreground. " - "It intentionally runs until cancelled. Do not background, delegate, retry, edit the script, or restart the command.", 3, cancellation=True) + "It intentionally runs until cancelled. Do not background, delegate, retry, edit the script, or restart the command.", 4, cancellation=True) commands = [item for item in items if item.get("type") == "command_execution" and prefix + "-hold.py" in item.get("command", "")] assert len(commands) == 1 and commands[0].get("status") in ("failed", "incomplete"), "Cancelled command lost its native outcome" stopped = runtime.read(ticks) time.sleep(2) assert runtime.read(ticks) == stopped and runtime.read(starts) == b"started\n", "Effects continued after cancellation settled" sessions.events.create(sid, events=[{"type": "agent.session.input.cancel"}], idempotency_key=nonce + "-cancel") - assert len(list(sessions.turns.list(sid))) == 3, "Duplicate cancellation created work" + assert len(list(sessions.turns.list(sid))) == 4, "Duplicate cancellation created work" check_artifacts() fourth, items = run_turn("Continue after the cancelled command. Never restart it or rerun the publish script. " "Run exactly `python3 " + prefix + "-resume.py` once with your native shell tool, without retries. " - "Then return the entire original conversation-only remember- token, including its prefix.", 4) + "Then return the entire original conversation-only remember- token, including its prefix.", 5) assert any(item.get("type") == "command_execution" and item.get("status") == "completed" and prefix + "-resume.py" in item.get("command", "") for item in items), "Missing resumed native command observation" assert memory in answer(items) and runtime.read(prefix + "-resumed") == b"resumed\n", "Post-cancel continuation failed" assert runtime.read(starts) == b"started\n" and runtime.read(ticks) == stopped, "Continuation repeated cancelled effects"