diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index b0ffb31be..79d8cc75e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -311,6 +311,13 @@ creation, freeze the effective ordinary hosted configuration and reuse inline initialization. Do not pass template IDs into Provider or Runtime. Omitted network inherits; overrides may only narrow policy. Preserve unresolved caller intent for creation retries and recover committed results before reading mutable templates. +For template-reference Session initialization, omitted/null env, files, commands +and packages inherit. Overlay non-null env keys; replace non-null files and command +lists, including empty lists. Select each package manager independently: omitted/null +inherits, while a supplied list replaces that manager. Revalidate the effective +configuration through the existing validators and freeze it through the same +transaction as inline initialization. Keep caller intent separate from resolved +configuration; do not add a second installer or pass merge rules to adapters. Updates and deletion cannot rewrite existing Session snapshots. Initial files use one Core-owned installer for template and inline configurations. Keep confidential bytes encrypted under the execution-service key and resource-bound AEAD, separately diff --git a/contracts/agents-api/README.md b/contracts/agents-api/README.md index 788906e4a..65680aa69 100644 --- a/contracts/agents-api/README.md +++ b/contracts/agents-api/README.md @@ -174,7 +174,7 @@ user-managed enrollment remain outside this qualification. | Area | Missing or unverified scope | | --- | --- | | Subagents / multi_agent | Six reads and same-child recovery have three-harness Docker evidence; optional native operations, live child progress, full lifecycle/interactions and tool combinations remain explicit gaps | -| Environment Templates | Unsupported restricted hostname forms, unqualified installation overrides/null network and exact hosted errors remain gaps. CRUD/list, files, env/setup/system/npm/Python, inline/referenced Skills, Plugins, workspace capability directories and Session references have recorded coverage. Environment Plugin MCP transport and placement limits are [listed separately](environment-templates.md#environment-origin-mcp-plugins) | +| Environment Templates | Unsupported restricted hostname forms, null network/list selection and exact hosted errors remain gaps. Template-reference env/files/commands/packages composition follows [qualified field rules](environment-templates.md#template-and-inline-configuration-composition). CRUD/list, files, env/setup/system/npm/Python, inline/referenced Skills, Plugins, workspace capability directories and Session references have recorded coverage. Environment Plugin MCP transport and placement limits are [listed separately](environment-templates.md#environment-origin-mcp-plugins) | | Input and configuration | Non-text initial input, broader content/configuration unions and reasoning/verbosity combinations; [structured output](structured-output.md) has qualified Claude function profiles on none and Core-managed Docker openai_hosted, with other combinations remaining gaps | | Tools and interactions | [Deferred discovery qualification](tool-search.md), other tool types, effective tool-set enforcement and result/cancel publication ordering; MiniMax public functions and service-origin MCP remain unsupported | | Vault and Credentials | Archive semantics, in-flight token withdrawal and exact hosted selection/error behavior; static/OAuth CRUD, replacement and scoped dispatch-time refresh are implemented (see credential guide for qualification) | diff --git a/contracts/agents-api/environment-templates.md b/contracts/agents-api/environment-templates.md index 1e05b8179..4d8439cf0 100644 --- a/contracts/agents-api/environment-templates.md +++ b/contracts/agents-api/environment-templates.md @@ -67,10 +67,10 @@ sizes for both variants. Initialization keeps file data out of ordinary configur resource responses, lifecycle events and command arguments. Templates keep references; each Session authorizes and freezes its own encrypted source bytes. Later source deletion cannot change them. -Template `files` omission preserves on update; null/[] clears. Referenced Sessions -inherit files. Supplying `files` together with `environment_template_id`, including -null/[], explicitly rejects while replacement/merge/null semantics remain unconfirmed. -Use a complete standalone inline configuration when a different file set is needed. +Template `files` omission preserves on update; null/[] clears. In a referencing +Session, omission and null inherit, while a supplied list replaces the entire file +set, including `[]` clearing it. Paths are not merged between the two sources. +The effective list retains the existing validation and tenant-owned source checks. Core initializes both paths with the same trusted file installer through Provider RunCommand. Daemon authentication remains available, but native preparation and live @@ -495,8 +495,8 @@ precede setup commands, nonzero setup prevents start, and runtime-reserved env n must reject. The shared initialization batch implements those fields with encrypted snapshots and the existing readiness gate. Public reads show packages but omit env/commands. Template updates replace each supplied field; omission preserves it and null clears -it. Referenced Sessions inherit the snapshot; explicit env/packages/setup overrides -with a template ID reject while override semantics remain unconfirmed. +it. Referencing Sessions use the composition rules below; these differ from +Template.update replacement rules. Files and resolved Skills are installed first, followed by system, npm/Python packages and ordered commands; the default cwd is `/workspace`. One command or package operation has the existing @@ -520,6 +520,100 @@ errors, no-op timestamps, concurrent pagination and referenced Session null-netw override semantics remain unverified. The last case explicitly rejects in this batch rather than guessing inheritance. This batch is not full protocol compatibility. +## Template and inline configuration composition + +The pinned Session description applies the template before inline configuration. +Owned official API probes on 2026-09-23 establish the following narrower behavior: + +| Session field | Omitted or null | Non-null inline value | +|---|---|---| +| `env` | Inherit template keys | Overlay by key; inline value wins, `{}` preserves all keys | +| `setup_commands` | Inherit template sequence | Replace the sequence; `[]` clears it | +| `files` | Inherit template file set | Replace the complete set; `[]` clears it | +| `packages` | Inherit all managers | Resolve Python/npm/system independently; `{}` inherits all | +| Individual package manager | Inherit its template list | Replace that list; `[]` clears it | + +Core performs this composition once, before the existing encrypted Session snapshot +transaction. Env values, command bodies and inline bytes remain absent from public +metadata. Effective package/file metadata reflects the selected inputs. Caller +intent still distinguishes omission, null and explicit fields for the local creation +retry policy; composition does not rewrite it. Later template/source changes do not +rewrite committed initialization. Existing per-source and effective-size validation, +network narrowing, Skill/Plugin selection and source authorization remain in place. +No harness or Provider participates in the merge. + +The official evidence used SDK 3.13.0, upstream `d7c41ef`, and `agents=v1`: +86 authenticated calls, nine owned Sessions, four lifetime Templates (including two +cleaned setup-script assertion failures), and six completed real `gpt-6-astra` Turns. +Four exit-zero command outputs confirmed env values and command replacement/order +for omitted, populated, empty and null inputs. Two further outputs confirmed full +file replacement and null inheritance, and an initialized empty listing confirmed +`files=[]`. The omitted-files case reached the bounded readiness limit, so its +inheritance proof is metadata-only. Package composition is also metadata evidence; +these official probes do not independently establish installed package versions. +All owned resources have successful public DELETE receipts; physical upstream +sandbox destruction was not independently observed. + +Private evidence: `~/.parsar/remediation/20260923/template-inline-composition/`, +subdirectories `official-env-setup` and `official-files-packages`. Reports retain +fixed-source snapshots, raw status/body/request IDs, command-output proofs, +accounting, cleanup and credential scans. This covers the observed fixtures rather +than all possible combinations. Null network and null Skill/Plugin/directory list +selection remain outside this batch. No new protocol version is introduced. + +### Core checks for composition (2026-09-23) + +`TestTemplateCompositionOfficialClientPostgres` uses the fixed strict SDK and raw +HTTP against real Core handlers and PostgreSQL. Six accepted cases and seven +rejected creation keys cover effective metadata, confidential frozen bytes and +ordered commands, tenant/source isolation, combined setup-size rejection without +partial records, and same-intent retries after template and source deletion. +A second Store/handler verifies reopened persistence; it is not an OS process +restart and does not run a model. API tests additionally cover raw caller intent, +non-mutating composition and retained field validation. + +The integrated server gate ran `make -o check-web check` at `5589df0`; this includes +real PostgreSQL tests, byte-for-byte sqlc generation, builds, native bridge checks +and Rust tests/format/Clippy. A fresh `make check-web` ran locally against the same +unchanged production diff: 287 client, 583 Web and 76 browser tests passed. Together +they cover every required `make check` target. `make openapi` regenerated the +Session description. The optional MiniMax packaged-native scratch/large-output +probe was skipped because its optional profile variables were unset; this batch +does not add MiniMax, Claude or E2B native qualification. + +### Real Codex Docker composition acceptance (2026-09-23) + +The exact `8019ac5` production build ran independently with Core, PostgreSQL and +Docker Runtime against the real Kimi API. Three completed native model Turns prove: + +- A populated-inline Session observed env key precedence, whole-file replacement, + ordered replacement commands, Python `packaging==26.0`, and inherited npm/system + tools. Its prompt named only the read operation, not the expected canary values. +- A separate inheritance Session observed null env/files/commands inheritance, + retained npm/system tools and absence of the explicitly cleared Python installation + directory. After template mutation/deletion and an actual Core process restart, + identical-key creation recovered its existing identity/configuration. A second + native Turn returned the exact same values and append trace: initialization did + not run again. The populated Session itself was not resumed in this run. + +Five owned Core Sessions were created over the acceptance attempts. The first two +failed before any model Turn because the reused private Skill-only runner omitted +Docker `nested_sandbox: true`, already required by the documented initialization +profile. Correcting that operator setting resolved the proc-mount failure without +production changes. In the corrected pair, the populated Session completed its +native command, but the private runner then required an optional assistant `phase` +and unwrapped JSON. The original message instead contained matching fenced JSON +without phase. An offline check preserved and verified the original native output +and message bytes; no model call was repeated. One new inheritance Session completed +the remaining two Turns. These failed assertions and operator diagnostics are +retained alongside successful evidence, not counted as extra successful runs. + +Private evidence is under the same batch's `live/` directory and records source, +binary/image hashes, operator-setting changes, exact native output and cleanup. +This is Codex/Docker qualification for the described composition paths, not renewed +qualification of other harnesses or Providers, every package manager combination, +or complete Template/Agents API semantics. + ## Verification ### Restricted-network Docker acceptance (2026-09-21) diff --git a/contracts/agents-api/openapi.yaml b/contracts/agents-api/openapi.yaml index 008de4ff4..89e7fb9da 100644 --- a/contracts/agents-api/openapi.yaml +++ b/contracts/agents-api/openapi.yaml @@ -2879,13 +2879,15 @@ paths: initialization lifecycle; requested network applies after setup. Initial inline and tenant-owned file_id files freeze encrypted bytes before provisioning, then install through the common Core lifecycle before native execution or - live Files access. Referenced files/env/packages/setup overrides are rejected - pending semantic verification. Tenant-owned environment_template_id references - inherit omitted network and allow only narrowing overrides. Referenced network:null - is explicitly unsupported pending semantic verification. Core freezes effective - configuration; template updates/deletion do not alter Session snapshots or - same-intent creation retries. Inline or tenant-owned skill_reference Skills - share initialization. Templates preserve default/latest/explicit selectors; + live Files access. With a template reference, omitted/null files, env, packages + and setup_commands inherit. Non-null files and command lists replace; env + overlays by key; each package manager inherits on omission/null and otherwise + replaces its list. Empty lists clear their selected field. Tenant-owned environment_template_id + references inherit omitted network and allow only narrowing overrides. Referenced + network:null is explicitly unsupported pending semantic verification. Core + freezes effective configuration; template updates/deletion do not alter Session + snapshots or same-intent creation retries. Inline or tenant-owned skill_reference + Skills share initialization. Templates preserve default/latest/explicit selectors; Session creation freezes concrete metadata and encrypted content atomically. Skill-list omission inherits and a supplied list replaces; null overrides and null version selectors remain unqualified and reject. Source deletion/default diff --git a/services/agents-api/internal/api/environment_templates_test.go b/services/agents-api/internal/api/environment_templates_test.go index e2b35f6cc..cf6f3eb3d 100644 --- a/services/agents-api/internal/api/environment_templates_test.go +++ b/services/agents-api/internal/api/environment_templates_test.go @@ -88,7 +88,7 @@ func TestTemplateResolutionAndCreationIntent(t *testing.T) { if err := h.resolveTemplateEnvironment(t.Context(), "tenant-a", &narrower); err != nil { t.Fatal(err) } - for _, raw := range []string{`{"type":"openai_hosted","environment_template_id":null}`, `{"type":"none","environment_template_id":"saved"}`, `{"type":"openai_hosted","environment_template_id":"saved","network":null}`, `{"type":"openai_hosted","environment_template_id":"saved","env":{"KEY":"secret"}}`} { + for _, raw := range []string{`{"type":"openai_hosted","environment_template_id":null}`, `{"type":"none","environment_template_id":"saved"}`, `{"type":"openai_hosted","environment_template_id":"saved","network":null}`} { if _, _, _, err := decodeTemplateEnvironment(json.RawMessage(raw)); err == nil { t.Fatal("invalid reference accepted", raw) } diff --git a/services/agents-api/internal/api/handler.go b/services/agents-api/internal/api/handler.go index d61f3e349..fb6202008 100644 --- a/services/agents-api/internal/api/handler.go +++ b/services/agents-api/internal/api/handler.go @@ -124,7 +124,7 @@ func NewHandler(s ResourceStore, auth *Authenticator, engine string, options ... // createSession atomically reserves or admits initial text with the Session. // @Summary Create an execution Session -// @Description Supports inline configuration or a tenant-owned saved agent_id with per-Session field replacements. Execution supports model/instructions, text verbosity, non-deferred function tools, adapter-qualified multi_agent with persisted Subagent reads, implicit reasoning, service tier auto and environment type none, subject to the configured engine. Codex additionally supports HTTP MCP with explicit service origin, native allowed_tools and boolean required defaulting to false. Session vault_ids attach only project-owned Vaults; credential_id selects an attached static bearer credential for the exact HTTPS URL, while null/omission selects a unique match or remains anonymous. Ambiguous selection rejects creation. Frozen private selections never populate an omitted public credential_id; missing decryption configuration fails dispatch without anonymous fallback. Required initialization uses native startup before the first native Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and OAuth remain unsupported. The self_hosted profile requires Codex, an absolute workspace_directory and empty capability_directories, with optional non-deferred function tools and HTTP MCP using explicit service origin, optionally authenticated by the attached Vault rules. Remote MCP and remote Bearer authentication each require separately advertised combination support; old peers cannot receive unsupported work. Omitted/null capability_directories use the empty-list default; self_hosted requires configured execution plus executor registry. Claude SDK currently requires medium verbosity and object-root function schemas. It supports anonymous or attached static-bearer service-origin HTTP MCP on none with boolean required and separately advertised MCP/bearer/required runtime support. Required servers must be connected before the first native input is released; pending or failed startup rejects execution. The shared Vault selection and immutable binding rules apply; unsupported native labels/tool names reject before persistence. An attached Vault with no matching credential may remain anonymous; missing keys or failed credential lookup/decryption never fall back to anonymous execution. Omitted stream defaults to false; stream and agent_id cannot be null. Metadata may be null, but its values must be strings. Initial input accepts a string or ordered user-message array. Codex and Claude SDK on none and qualified openai_hosted also accept inline PNG/JPEG image content; other image combinations and remote URLs are unsupported. None initial input atomically starts a Turn; self_hosted initial input is reserved while returning its Environment connection target, with execution deferred to native readiness and Session failure on initial timeout. Initial input is required for none and for streamed creation outside self_hosted. Omitted/null input remains valid for non-streaming hosted and self_hosted creation. With stream=true, returns live Session events starting at creation; disconnect does not cancel execution. New Sessions retain their authenticated creator; all creation retries require the same typed subject, including across key rotation. Saved-Agent retries and inline requests using Vault attachments or credential references retain caller intent independently of later resource changes; unrelated inline retries preserve resolved/default equivalences. Unknown historical creators reject retries; known creators without recorded intent retain resolved-snapshot retry rules. These conflict policies are local and not verified hosted parity. Creation retries observe future events without replay; retry with stream=false to retrieve the Session. Claude SDK on none and Core-managed Docker openai_hosted supports qualified object-root json_schema output with medium verbosity, single-Agent execution and ordinary functions. Hosted execution reuses native workspace tools and Files/Artifacts; Skills, Plugins, capability directories, HTTP MCP, Subagent and tool_search combinations remain unqualified, including inherited template contents. Other non-text initial input remains unsupported. Basic Codex and Claude SDK openai_hosted creation requires an explicitly configured managed provider. The Claude workspace profile supports non-deferred function tools with text or successful inline PNG/JPEG results alongside native workspace tools; HTTP MCP remains unsupported. Idle Sessions provision automatically; initial provisioning has no caller connection action. Network defaults to enabled; disabled and restricted exact ASCII hostnames are supported. Restricted policy requires 1–100 allowed domains. Unsupported hostname forms and startup installations are rejected. Confidential env, system/npm/Python packages and ordered setup commands use the shared initialization lifecycle; requested network applies after setup. Initial inline and tenant-owned file_id files freeze encrypted bytes before provisioning, then install through the common Core lifecycle before native execution or live Files access. Referenced files/env/packages/setup overrides are rejected pending semantic verification. Tenant-owned environment_template_id references inherit omitted network and allow only narrowing overrides. Referenced network:null is explicitly unsupported pending semantic verification. Core freezes effective configuration; template updates/deletion do not alter Session snapshots or same-intent creation retries. Inline or tenant-owned skill_reference Skills share initialization. Templates preserve default/latest/explicit selectors; Session creation freezes concrete metadata and encrypted content atomically. Skill-list omission inherits and a supplied list replaces; null overrides and null version selectors remain unqualified and reject. Source deletion/default updates cannot change committed Session Skill contents. Deferred function discovery uses type-only tool_search and per-function defer_loading in the qualified single-agent Claude environment:none function profile, including qualified inline image messages and text results. Explicit web_search mode disabled and programmatic_tool_calling enabled false use frozen common Runtime controls. Enabled forms remain unqualified. Omitted programmatic configuration preserves native behavior, a documented difference from the official default-on behavior. Other combinations remain unqualified; see the operation coverage. +// @Description Supports inline configuration or a tenant-owned saved agent_id with per-Session field replacements. Execution supports model/instructions, text verbosity, non-deferred function tools, adapter-qualified multi_agent with persisted Subagent reads, implicit reasoning, service tier auto and environment type none, subject to the configured engine. Codex additionally supports HTTP MCP with explicit service origin, native allowed_tools and boolean required defaulting to false. Session vault_ids attach only project-owned Vaults; credential_id selects an attached static bearer credential for the exact HTTPS URL, while null/omission selects a unique match or remains anonymous. Ambiguous selection rejects creation. Frozen private selections never populate an omitted public credential_id; missing decryption configuration fails dispatch without anonymous fallback. Required initialization uses native startup before the first native Turn, including cold resume, and requires a separately advertised capability; exact hosted creation timing and error parity remain unverified. Other MCP origins and OAuth remain unsupported. The self_hosted profile requires Codex, an absolute workspace_directory and empty capability_directories, with optional non-deferred function tools and HTTP MCP using explicit service origin, optionally authenticated by the attached Vault rules. Remote MCP and remote Bearer authentication each require separately advertised combination support; old peers cannot receive unsupported work. Omitted/null capability_directories use the empty-list default; self_hosted requires configured execution plus executor registry. Claude SDK currently requires medium verbosity and object-root function schemas. It supports anonymous or attached static-bearer service-origin HTTP MCP on none with boolean required and separately advertised MCP/bearer/required runtime support. Required servers must be connected before the first native input is released; pending or failed startup rejects execution. The shared Vault selection and immutable binding rules apply; unsupported native labels/tool names reject before persistence. An attached Vault with no matching credential may remain anonymous; missing keys or failed credential lookup/decryption never fall back to anonymous execution. Omitted stream defaults to false; stream and agent_id cannot be null. Metadata may be null, but its values must be strings. Initial input accepts a string or ordered user-message array. Codex and Claude SDK on none and qualified openai_hosted also accept inline PNG/JPEG image content; other image combinations and remote URLs are unsupported. None initial input atomically starts a Turn; self_hosted initial input is reserved while returning its Environment connection target, with execution deferred to native readiness and Session failure on initial timeout. Initial input is required for none and for streamed creation outside self_hosted. Omitted/null input remains valid for non-streaming hosted and self_hosted creation. With stream=true, returns live Session events starting at creation; disconnect does not cancel execution. New Sessions retain their authenticated creator; all creation retries require the same typed subject, including across key rotation. Saved-Agent retries and inline requests using Vault attachments or credential references retain caller intent independently of later resource changes; unrelated inline retries preserve resolved/default equivalences. Unknown historical creators reject retries; known creators without recorded intent retain resolved-snapshot retry rules. These conflict policies are local and not verified hosted parity. Creation retries observe future events without replay; retry with stream=false to retrieve the Session. Claude SDK on none and Core-managed Docker openai_hosted supports qualified object-root json_schema output with medium verbosity, single-Agent execution and ordinary functions. Hosted execution reuses native workspace tools and Files/Artifacts; Skills, Plugins, capability directories, HTTP MCP, Subagent and tool_search combinations remain unqualified, including inherited template contents. Other non-text initial input remains unsupported. Basic Codex and Claude SDK openai_hosted creation requires an explicitly configured managed provider. The Claude workspace profile supports non-deferred function tools with text or successful inline PNG/JPEG results alongside native workspace tools; HTTP MCP remains unsupported. Idle Sessions provision automatically; initial provisioning has no caller connection action. Network defaults to enabled; disabled and restricted exact ASCII hostnames are supported. Restricted policy requires 1–100 allowed domains. Unsupported hostname forms and startup installations are rejected. Confidential env, system/npm/Python packages and ordered setup commands use the shared initialization lifecycle; requested network applies after setup. Initial inline and tenant-owned file_id files freeze encrypted bytes before provisioning, then install through the common Core lifecycle before native execution or live Files access. With a template reference, omitted/null files, env, packages and setup_commands inherit. Non-null files and command lists replace; env overlays by key; each package manager inherits on omission/null and otherwise replaces its list. Empty lists clear their selected field. Tenant-owned environment_template_id references inherit omitted network and allow only narrowing overrides. Referenced network:null is explicitly unsupported pending semantic verification. Core freezes effective configuration; template updates/deletion do not alter Session snapshots or same-intent creation retries. Inline or tenant-owned skill_reference Skills share initialization. Templates preserve default/latest/explicit selectors; Session creation freezes concrete metadata and encrypted content atomically. Skill-list omission inherits and a supplied list replaces; null overrides and null version selectors remain unqualified and reject. Source deletion/default updates cannot change committed Session Skill contents. Deferred function discovery uses type-only tool_search and per-function defer_loading in the qualified single-agent Claude environment:none function profile, including qualified inline image messages and text results. Explicit web_search mode disabled and programmatic_tool_calling enabled false use frozen common Runtime controls. Enabled forms remain unqualified. Omitted programmatic configuration preserves native behavior, a documented difference from the official default-on behavior. Other combinations remain unqualified; see the operation coverage. // @Tags Sessions // @Accept json // @Produce json,text/event-stream diff --git a/services/agents-api/internal/api/initial_files_test.go b/services/agents-api/internal/api/initial_files_test.go index 88c353dd4..aa25c75e1 100644 --- a/services/agents-api/internal/api/initial_files_test.go +++ b/services/agents-api/internal/api/initial_files_test.go @@ -36,8 +36,8 @@ func TestInitialFilesDecodeAndConfidentialMetadata(t *testing.T) { } } for _, value := range []string{"null", "[]", `[{"type":"inline","path":"/workspace/a","data":""}]`} { - if _, _, _, err := decodeTemplateEnvironment(json.RawMessage(`{"type":"openai_hosted","environment_template_id":"template","files":` + value + `}`)); err == nil { - t.Fatal("unconfirmed template file override accepted") + if _, _, _, err := decodeTemplateEnvironment(json.RawMessage(`{"type":"openai_hosted","environment_template_id":"template","files":` + value + `}`)); err != nil { + t.Fatal("supported template file override rejected", err) } } } diff --git a/services/agents-api/internal/api/session_template.go b/services/agents-api/internal/api/session_template.go index bf819b6c9..816ac5867 100644 --- a/services/agents-api/internal/api/session_template.go +++ b/services/agents-api/internal/api/session_template.go @@ -4,6 +4,8 @@ import ( "bytes" "context" "encoding/json" + "maps" + "slices" v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" "github.com/MiniMax-AI-Dev/parsar/internal/agentnetwork" @@ -35,11 +37,6 @@ func decodeTemplateEnvironment(raw json.RawMessage) (*v1.Environment, string, js return nil, "", nil, store.ErrInvalidInput } } - for _, name := range []string{"files", "env", "setup_commands", "packages"} { - if _, supplied := fields[name]; supplied { - return nil, "", nil, store.ErrInvalidInput - } - } delete(fields, "environment_template_id") inline, err := json.Marshal(fields) if err != nil { @@ -80,17 +77,60 @@ func (h *Handler) resolveTemplateEnvironment(ctx context.Context, tenant string, if _, supplied := fields["capability_directories"]; !supplied { directories = template.Initialization.CapabilityDirectories } - input.initialization = template.Initialization - input.initialization.Skills = skills - input.initialization.Plugins = plugins - input.initialization.CapabilityDirectories = directories + setup := template.Initialization + setup.Env = maps.Clone(setup.Env) + if len(input.initialization.Env) > 0 { + if setup.Env == nil { + setup.Env = make(map[string]string) + } + maps.Copy(setup.Env, input.initialization.Env) + } + if templateFieldOverride(fields, "setup_commands") { + setup.Commands = input.initialization.Commands + } + setup.Commands = slices.Clone(setup.Commands) + if templateFieldOverride(fields, "files") { + files = input.initialFiles + } + files = slices.Clone(files) + var managers map[string]json.RawMessage + if templateFieldOverride(fields, "packages") { + if json.Unmarshal(fields["packages"], &managers) != nil { + return store.ErrInvalidInput + } + } + if templateFieldOverride(managers, "npm") { + setup.Packages.NPM = input.initialization.Packages.NPM + } + if templateFieldOverride(managers, "python") { + setup.Packages.Python = input.initialization.Packages.Python + } + if templateFieldOverride(managers, "system") { + setup.Packages.System = input.initialization.Packages.System + } + setup.Packages.NPM = slices.Clone(setup.Packages.NPM) + setup.Packages.Python = slices.Clone(setup.Packages.Python) + setup.Packages.System = slices.Clone(setup.Packages.System) + setup.Skills, setup.Plugins, setup.CapabilityDirectories = skills, plugins, directories + if err := setup.Validate(); err != nil { + return err + } + if err := store.ValidateInitialFiles(files); err != nil { + return err + } + input.initialization = setup input.Environment.Plugins = pluginResponse(input.initialization.PluginMetadata()) input.Environment.CapabilityDirectories = append([]string{}, directories...) input.Environment.Skills = skillResponse(input.initialization.SkillMetadata()) - packages := template.Initialization.PackageMetadata() + packages := setup.PackageMetadata() input.Environment.Packages = &packages input.initialFiles = files input.Environment.Files = initialFileResponse(files) // Runtime sees only the effective ordinary hosted configuration. return nil } + +func templateFieldOverride(fields map[string]json.RawMessage, name string) bool { + value, supplied := fields[name] + return supplied && !bytes.Equal(bytes.TrimSpace(value), []byte("null")) +} diff --git a/services/agents-api/internal/api/session_template_composition_test.go b/services/agents-api/internal/api/session_template_composition_test.go new file mode 100644 index 000000000..04e079832 --- /dev/null +++ b/services/agents-api/internal/api/session_template_composition_test.go @@ -0,0 +1,205 @@ +package api + +import ( + "bytes" + "context" + "encoding/json" + "reflect" + "strings" + "testing" + + v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" +) + +type compositionTemplateStore struct { + ResourceStore + template store.EnvironmentTemplate + files []store.InitialFile +} + +func (s *compositionTemplateStore) ResolveEnvironmentTemplate(context.Context, string, string) (store.EnvironmentTemplate, []store.InitialFile, error) { + return s.template, s.files, nil +} + +func compositionRequest(t *testing.T, fields string) sessionRequest { + t.Helper() + var request decodedSessionRequest + body := `{"agent":{"model":"test"},"environment":{"type":"openai_hosted","environment_template_id":"saved"` + fields + `}}` + if err := json.Unmarshal([]byte(body), &request); err != nil { + t.Fatal(err) + } + input, err := request.validated() + if err != nil { + t.Fatal(err) + } + return input +} + +func compositionFixture() *compositionTemplateStore { + return &compositionTemplateStore{ + template: store.EnvironmentTemplate{NetworkAccess: "enabled", Initialization: store.EnvironmentSetup{ + Env: map[string]string{"TEMPLATE": "private-template-env", "SHARED": "private-old-value"}, + Commands: []store.SetupCommand{{Command: "printf private-template-command"}}, + Packages: v1.EnvironmentPackages{NPM: []string{"semver@7.7.2"}, Python: []string{"packaging==25.0"}, System: []string{"jq"}}, + }}, + files: []store.InitialFile{{Type: "inline", Path: "/workspace/template", Data: []byte("private-template-bytes")}}, + } +} + +func TestTemplateInlineCompositionRules(t *testing.T) { + for _, test := range []struct { + name, fields string + change func(*store.EnvironmentSetup, *[]store.InitialFile) + }{ + {name: "omitted"}, + {name: "null", fields: `,"env":null,"setup_commands":null,"files":null,"packages":null`}, + {name: "empty objects", fields: `,"env":{},"packages":{}`}, + {name: "manager null", fields: `,"packages":{"npm":null,"python": null ,"system":null}`}, + {name: "clear lists", fields: `,"setup_commands":[],"files":[],"packages":{"npm":[],"python":[],"system":[]}`, change: func(s *store.EnvironmentSetup, f *[]store.InitialFile) { + s.Commands, *f = nil, nil + s.Packages = v1.EnvironmentPackages{} + }}, + {name: "populated", fields: `,"env":{"SHARED":"private-inline-env","INLINE":"private-new-value"},"setup_commands":[{"command":"printf private-first"},{"command":"printf private-second","cwd":"/workspace"}],"files":[{"type":"inline","path":"/workspace/inline","data":"cHJpdmF0ZS1pbmxpbmUtYnl0ZXM="}],"packages":{"python":["idna==3.10"],"npm":[]}`, change: func(s *store.EnvironmentSetup, f *[]store.InitialFile) { + s.Env["SHARED"], s.Env["INLINE"] = "private-inline-env", "private-new-value" + s.Commands = []store.SetupCommand{{Command: "printf private-first"}, {Command: "printf private-second", CWD: "/workspace"}} + *f = []store.InitialFile{{Type: "inline", Path: "/workspace/inline", Data: []byte("private-inline-bytes")}} + s.Packages.NPM, s.Packages.Python = nil, []string{"idna==3.10"} + }}, + {name: "mixed managers", fields: `,"packages":{"npm":null,"system":[],"python":["idna==3.10"]}`, change: func(s *store.EnvironmentSetup, _ *[]store.InitialFile) { + s.Packages.Python, s.Packages.System = []string{"idna==3.10"}, nil + }}, + {name: "overlapping path replaces whole list", fields: `,"files":[{"type":"inline","path":"/workspace/template","data":"bmV3"}]`, change: func(_ *store.EnvironmentSetup, f *[]store.InitialFile) { + *f = []store.InitialFile{{Type: "inline", Path: "/workspace/template", Data: []byte("new")}} + }}, + } { + t.Run(test.name, func(t *testing.T) { + lookup, expected := compositionFixture(), compositionFixture() + wantSetup, wantFiles := expected.template.Initialization, expected.files + if test.change != nil { + test.change(&wantSetup, &wantFiles) + } + input := compositionRequest(t, test.fields) + intent, err := sessionCreationRequest(input, nil) + if err != nil { + t.Fatal(err) + } + beforeTemplate, _ := json.Marshal(lookup.template) + beforeFiles, _ := json.Marshal(lookup.files) + h := Handler{store: lookup} + if err := h.resolveTemplateEnvironment(t.Context(), "tenant", &input); err != nil { + t.Fatal(err) + } + // JSON omitempty normalizes nil/empty private slices without erasing public lists. + actual, _ := json.Marshal(input.initialization) + want, _ := json.Marshal(wantSetup) + if !bytes.Equal(actual, want) || !reflect.DeepEqual(initialFileResponse(input.initialFiles), initialFileResponse(wantFiles)) { + t.Fatalf("wrong composed initialization: %s; files=%+v", actual, input.initialFiles) + } + for i := range wantFiles { + if !bytes.Equal(input.initialFiles[i].Data, wantFiles[i].Data) { + t.Fatal("composed file bytes changed") + } + } + if !reflect.DeepEqual(*input.Environment.Packages, wantSetup.PackageMetadata()) || !reflect.DeepEqual(input.Environment.Files, initialFileResponse(wantFiles)) { + t.Fatal("public metadata differs from effective initialization") + } + public, _ := json.Marshal(input.Environment) + for _, private := range []string{"private-", `"env"`, `"setup_commands"`, `"data"`, "environment_template_id"} { + if bytes.Contains(public, []byte(private)) { + t.Fatalf("confidential input in public metadata: %s", public) + } + } + afterIntent, _ := sessionCreationRequest(input, nil) + afterTemplate, _ := json.Marshal(lookup.template) + afterFiles, _ := json.Marshal(lookup.files) + if !bytes.Equal(intent, afterIntent) || !bytes.Equal(beforeTemplate, afterTemplate) || !bytes.Equal(beforeFiles, afterFiles) { + t.Fatal("composition changed caller intent or template") + } + }) + } +} + +func TestTemplateCompositionDoesNotAliasChangedInputs(t *testing.T) { + for _, fields := range []string{"", `,"env":{"INLINE":"value"},"setup_commands":[{"command":"true"}],"files":[{"type":"inline","path":"/workspace/inline","data":""}],"packages":{"npm":["idna"]}`} { + lookup := compositionFixture() + input := compositionRequest(t, fields) + originalSetup, originalFiles := input.initialization, input.initialFiles + beforeInline, _ := json.Marshal(originalSetup) + beforeInlineFiles, _ := json.Marshal(originalFiles) + beforeTemplate, _ := json.Marshal(lookup.template) + beforeFiles, _ := json.Marshal(lookup.files) + h := Handler{store: lookup} + if err := h.resolveTemplateEnvironment(t.Context(), "tenant", &input); err != nil { + t.Fatal(err) + } + input.initialization.Env["SHARED"] = "changed" + input.initialization.Commands[0].Command = "changed" + input.initialization.Packages.NPM[0] = "changed" + input.initialFiles[0].Path = "/workspace/changed" + afterInline, _ := json.Marshal(originalSetup) + afterInlineFiles, _ := json.Marshal(originalFiles) + afterTemplate, _ := json.Marshal(lookup.template) + afterFiles, _ := json.Marshal(lookup.files) + if !bytes.Equal(beforeInline, afterInline) || !bytes.Equal(beforeInlineFiles, afterInlineFiles) || !bytes.Equal(beforeTemplate, afterTemplate) || !bytes.Equal(beforeFiles, afterFiles) { + t.Fatal("composed maps/slices alias caller or template") + } + } +} + +func TestTemplateCompositionRevalidatesCombinedSetupLimit(t *testing.T) { + for _, field := range []string{"env", "packages"} { + lookup := compositionFixture() + lookup.template.Initialization.Env = map[string]string{"TEMPLATE": strings.Repeat("a", 300<<10)} + var override any = map[string]any{"INLINE": strings.Repeat("b", 300<<10)} + if field == "packages" { + override = map[string]any{"python": []string{strings.Repeat("b", 300<<10)}} + } + raw, _ := json.Marshal(override) + input := compositionRequest(t, `,"`+field+`":`+string(raw)) + if lookup.template.Initialization.Validate() != nil || input.initialization.Validate() != nil { + t.Fatal("each side must be valid independently") + } + h := Handler{store: lookup} + if err := h.resolveTemplateEnvironment(t.Context(), "tenant", &input); err == nil { + t.Fatalf("combined setup limit bypassed for %s", field) + } + } +} + +func TestTemplateInlineCompositionRetainsFieldValidation(t *testing.T) { + for _, fields := range []string{ + `,"env":{"PATH":"private-canary"}`, `,"env":{"PARSAR_HOME":"private-canary"}`, `,"env":{"KEY":null}`, + `,"setup_commands":[{"command":"true","cwd":"relative"}]`, `,"packages":{"python":[null]}`, `,"packages":{"npm":["--unsafe"]}`, + `,"files":[{"type":"inline","path":"/workspace/../private","data":""}]`, + `,"files":[{"type":"inline","path":"/workspace/a","data":""},{"type":"inline","path":"/workspace/a","data":""}]`, + } { + var request decodedSessionRequest + body := `{"agent":{"model":"test"},"environment":{"type":"openai_hosted","environment_template_id":"saved"` + fields + `}}` + if err := json.Unmarshal([]byte(body), &request); err != nil { + t.Fatal(err) + } + if _, err := request.validated(); err == nil { + t.Fatalf("invalid inline field accepted: %s", fields) + } + } +} + +func TestTemplateFilesReplacementDoesNotCombineCounts(t *testing.T) { + lookup := compositionFixture() + files := make([]store.InitialFile, 30) + for i := range files { + files[i] = store.InitialFile{Type: "inline", Path: "/workspace/" + strings.Repeat("a", i+1)} + } + lookup.files = files + wire := make([]map[string]string, len(files)) + for i, file := range files { + wire[i] = map[string]string{"type": "inline", "path": file.Path, "data": ""} + } + raw, _ := json.Marshal(wire) + input := compositionRequest(t, `,"files":`+string(raw)) + h := Handler{store: lookup} + if err := h.resolveTemplateEnvironment(t.Context(), "tenant", &input); err != nil || len(input.initialFiles) != 30 { + t.Fatalf("file lists were combined: count=%d err=%v", len(input.initialFiles), err) + } +} diff --git a/services/agents-api/internal/store/template_composition_public_test.go b/services/agents-api/internal/store/template_composition_public_test.go new file mode 100644 index 000000000..ec7354430 --- /dev/null +++ b/services/agents-api/internal/store/template_composition_public_test.go @@ -0,0 +1,166 @@ +package store_test + +import ( + "bytes" + "context" + "encoding/json" + "errors" + "net/http/httptest" + "os" + "os/exec" + "reflect" + "testing" + "time" + + v1 "github.com/MiniMax-AI-Dev/parsar/contracts/agents-api/v1" + "github.com/MiniMax-AI-Dev/parsar/internal/agentdaemon/device" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/api" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/credentialcrypto" + "github.com/MiniMax-AI-Dev/parsar/services/agents-api/internal/store" + "github.com/google/uuid" +) + +func TestTemplateCompositionOfficialClientPostgres(t *testing.T) { + python := os.Getenv("PARSAR_OFFICIAL_SDK_PYTHON") + if python == "" { + t.Skip("pinned official Python SDK required") + } + _, pool := store.NewTestStore(t) + cipher, err := credentialcrypto.New(bytes.Repeat([]byte{84}, 32)) + if err != nil { + t.Fatal(err) + } + s := store.NewWithCredentialCipher(pool, cipher) + reopenedStore := store.NewWithCredentialCipher(pool, cipher) + tenant, foreignTenant, token, foreign := uuid.NewString(), uuid.NewString(), uuid.NewString(), uuid.NewString() + auth, err := api.NewAuthenticator([]api.APIKey{ + {OrganizationID: "test-org", ProjectID: uuid.NewString(), SubjectKind: "service_account", SubjectID: "composition-owner", TokenSHA256: device.HashCredential(token), TenantID: tenant}, + {OrganizationID: "test-org", ProjectID: uuid.NewString(), SubjectKind: "service_account", SubjectID: "composition-foreign", TokenSHA256: device.HashCredential(foreign), TenantID: foreignTenant}, + }) + if err != nil { + t.Fatal(err) + } + serve := func(current *store.Store) *httptest.Server { + t.Helper() + // Hosted admission and freezing use the real Store; no Runtime or model runs. + h, err := api.NewHandler(current, auth, "codex", api.WithHostedEnvironments(), api.WithExecution(current), api.WithSourceFiles(current)) + if err != nil { + t.Fatal(err) + } + server := httptest.NewServer(h) + t.Cleanup(server.Close) + return server + } + server, reopened := serve(s), serve(reopenedStore) + marker := "private-composition-" + uuid.NewString() + settings, err := json.Marshal(map[string]string{"base": server.URL, "recovered": reopened.URL, "token": token, "foreign": foreign, "canary": marker}) + if err != nil { + t.Fatal(err) + } + ctx, cancel := context.WithTimeout(t.Context(), time.Minute) + defer cancel() + command := exec.CommandContext(ctx, python, "../../tests/official_template_composition.py") + command.Stdin = bytes.NewReader(settings) + output, err := command.CombinedOutput() + if err != nil { + t.Fatalf("template composition official client: %v %s", err, output) + } + var receipt struct { + Sessions map[string]string `json:"sessions"` + RejectedKeys []string `json:"rejected_keys"` + } + if err := json.Unmarshal(output, &receipt); err != nil { + t.Fatalf("invalid acceptance receipt: %v %s", err, output) + } + if len(receipt.Sessions) != 6 || len(receipt.RejectedKeys) != 7 { + t.Fatalf("incomplete acceptance receipt: %s", output) + } + t.Cleanup(func() { + for _, id := range receipt.Sessions { + if err := s.DeleteSession(context.Background(), tenant, id); err != nil { + t.Error(err) + } + } + }) + // A second handler/Store exercises reopened persistence, not an OS process restart. + for label, id := range receipt.Sessions { + env := map[string]string{"TEMPLATE_ONLY": marker + "-template-env", "SHARED": marker + "-template-shared"} + commands := []store.SetupCommand{{Command: "printf " + marker + "-template-command", CWD: "/workspace"}} + packages := v1.EnvironmentPackages{Python: []string{"packaging==25.0"}, NPM: []string{"semver@7.7.2"}, System: []string{"jq"}} + paths := []string{"/workspace/template-only.txt", "/workspace/overlap.txt"} + contents := []string{marker + "-template-file", marker + "-source"} + switch label { + case "populated": + env["SHARED"], env["INLINE_ONLY"] = marker+"-inline-shared", marker+"-inline-env" + commands = []store.SetupCommand{{Command: "printf " + marker + "-inline-one"}, {Command: "printf " + marker + "-inline-two", CWD: "/workspace"}} + packages.Python, packages.NPM = []string{"idna==3.10"}, []string{} + paths = []string{"/workspace/overlap.txt", "/workspace/selected-source.txt"} + contents = []string{marker + "-inline-file", marker + "-source"} + case "empty": + commands = nil + paths, contents = nil, nil + case "excluded-source": + paths, contents = nil, nil + case "omitted", "null", "nested-null": + default: + t.Fatalf("unknown case %q", label) + } + for _, current := range []*store.Store{s, reopenedStore} { + setup, err := current.ReadEnvironmentSetup(t.Context(), tenant, id) + if err != nil || !reflect.DeepEqual(setup.Env, env) || !reflect.DeepEqual(setup.PackageMetadata(), packages) || len(setup.Commands) != len(commands) { + t.Fatalf("%s durable setup differs: %v", label, err) + } + for i, want := range commands { + if setup.Commands[i] != want { + t.Fatalf("%s command order differs", label) + } + } + if _, err := current.ReadEnvironmentSetup(t.Context(), foreignTenant, id); !errors.Is(err, store.ErrNotFound) { + t.Fatalf("%s foreign setup read: %v", label, err) + } + for position, want := range contents { + metadata, body, err := current.ReadInitialEnvironmentFile(t.Context(), tenant, id, position) + if err != nil || string(body) != want || metadata.Path != paths[position] || metadata.SizeBytes == nil || *metadata.SizeBytes != int64(len(want)) { + t.Fatalf("%s frozen file %d differs: %v", label, position, err) + } + if _, _, err := current.ReadInitialEnvironmentFile(t.Context(), foreignTenant, id, position); err == nil { + t.Fatalf("%s foreign file read succeeded", label) + } + var encrypted []byte + if err := pool.QueryRow(t.Context(), "SELECT contents FROM initial_environment_files WHERE id=$1", metadata.ID).Scan(&encrypted); err != nil || bytes.Contains(encrypted, []byte(marker)) { + t.Fatalf("%s plaintext file storage: %v", label, err) + } + } + } + var encrypted, configuration []byte + if err := pool.QueryRow(t.Context(), "SELECT e.contents,s.configuration FROM environment_setups e JOIN sessions s ON s.id=e.session_id WHERE s.id=$1", id).Scan(&encrypted, &configuration); err != nil || bytes.Contains(encrypted, []byte(marker)) || bytes.Contains(configuration, []byte(marker)) { + t.Fatalf("%s plaintext setup storage: %v", label, err) + } + } + // Fresh tenant counts include deleted rows too, so rejected transactional creates + // cannot hide partial Sessions, Environments or snapshots behind soft deletion. + checks := []struct { + query string + want int + }{ + {"SELECT count(*) FROM sessions WHERE tenant_id=$1", 6}, + {"SELECT count(*) FROM environments e JOIN sessions s ON s.id=e.session_id WHERE s.tenant_id=$1", 6}, + {"SELECT count(*) FROM environment_setups e JOIN sessions s ON s.id=e.session_id WHERE s.tenant_id=$1", 6}, + {"SELECT count(*) FROM initial_environment_files f JOIN sessions s ON s.id=f.session_id WHERE s.tenant_id=$1", 8}, + {"SELECT count(*) FROM turns t JOIN sessions s ON s.id=t.session_id WHERE s.tenant_id=$1", 0}, + {"SELECT count(*) FROM environment_input_reservations e JOIN sessions s ON s.id=e.session_id WHERE s.tenant_id=$1", 0}, + } + for _, check := range checks { + var count int + if err := pool.QueryRow(t.Context(), check.query, tenant).Scan(&count); err != nil || count != check.want { + t.Fatalf("admission residue: got %d want %d: %v", count, check.want, err) + } + } + for _, owner := range []string{tenant, foreignTenant} { + var rejected int + if err := pool.QueryRow(t.Context(), "SELECT count(*) FROM sessions WHERE tenant_id=$1 AND idempotency_key=ANY($2::text[])", owner, receipt.RejectedKeys).Scan(&rejected); err != nil || rejected != 0 { + t.Fatalf("rejected creation persisted: %d %v", rejected, err) + } + } + t.Log("six composition cases passed fixed SDK/raw HTTP, reopened encrypted snapshots, isolation, rejection rollback and frozen retries; no native/model execution") +} diff --git a/services/agents-api/tests/official_template_composition.py b/services/agents-api/tests/official_template_composition.py new file mode 100644 index 000000000..ec04e88ef --- /dev/null +++ b/services/agents-api/tests/official_template_composition.py @@ -0,0 +1,187 @@ +"""Public template composition on Core HTTP/PostgreSQL; no native/model execution.""" + +import base64 +import copy +import importlib.metadata +import json +import secrets +import sys +from contextlib import ExitStack +from pathlib import Path + +import httpx2 +from openai import DefaultHttpxClient, OpenAI + + +def main(): + settings = json.load(sys.stdin) + pin = json.loads((Path(__file__).resolve().parents[3] / "contracts/agents-api/upstream.json").read_text()) + distribution = importlib.metadata.distribution("openai") + assert distribution.version == pin["sdk_version"] + assert json.loads(distribution.read_text("direct_url.json"))["vcs_info"]["commit_id"] == pin["commit"] + marker = settings["canary"] + packages = {"python": ["packaging==25.0"], "npm": ["semver@7.7.2"], "system": ["jq"]} + populated_packages = {"python": ["idna==3.10"], "npm": [], "system": ["jq"]} + sessions, rejected_keys, templates, files = {}, [], [], [] + succeeded = False + + def inline(path, suffix): + return {"type": "inline", "path": path, "data": base64.b64encode((marker + suffix).encode()).decode()} + + with ExitStack() as stack: + raw = stack.enter_context(httpx2.Client(trust_env=False, timeout=20)) + + def sdk(base, token): + return stack.enter_context(OpenAI( + base_url=base + "/v1", api_key=token, max_retries=0, + _strict_response_validation=True, http_client=DefaultHttpxClient(trust_env=False), + )) + + owner = sdk(settings["base"], settings["token"]) + foreign = sdk(settings["base"], settings["foreign"]) + reopened = sdk(settings["recovered"], settings["token"]) + api = owner.beta.agents.environments.templates + headers = {"Authorization": "Bearer " + settings["token"], "OpenAI-Beta": "agents=v1"} + + def safe(response): + assert all(value not in response.text for value in (marker, settings["token"], settings["foreign"])), "private response content" + assert "\"data\": \"" not in response.text, "inline source data in resource response" + + def request(body, *, key=None, token=None, base=None): + response = raw.post((base or settings["base"]) + "/v1/agents/sessions", json=body, + headers={**headers, "Authorization": "Bearer " + (token or settings["token"]), + "Idempotency-Key": key or secrets.token_hex(20)}) + safe(response) + return response + + def rejected(body, status, *, token=None): + key = secrets.token_hex(20) + rejected_keys.append(key) + response = request(body, key=key, token=token) + assert response.status_code == status, (status, response.status_code, response.text) + assert response.json()["error"]["param"] is None + return response.json() + + def projection(client, session_id, expected_files, expected_packages): + response = client.beta.agents.sessions.with_raw_response.retrieve(session_id) + assert response.status_code == 200 + safe(response.http_response) + body = response.http_response.json() + parsed = response.parse() + environment = body["environment"] + assert environment["packages"] == expected_packages + assert parsed.environment.packages.to_dict() == expected_packages + assert "env" not in environment and "setup_commands" not in environment + actual = environment["files"] + assert len(actual) == len(expected_files) + for file, expected in zip(actual, expected_files): + assert file["id"] and all(file[k] == value for k, value in expected.items()), file + assert set(file) == set(expected) | {"id"}, file + env = client.beta.agents.environments.with_raw_response.retrieve(environment["id"]) + assert env.status_code == 200 + safe(env.http_response) + assert env.http_response.json()["files"] == actual + assert [file.to_dict() for file in env.parse().files] == actual + assert list(client.beta.agents.sessions.turns.list(session_id)) == [] + return body + + try: + source = owner.files.create(file=("owned.txt", (marker + "-source").encode()), purpose="user_data") + files.append((owner, source.id)) + other = foreign.files.create(file=("foreign.txt", (marker + "-foreign").encode()), purpose="user_data") + files.append((foreign, other.id)) + template_files = [inline("/workspace/template-only.txt", "-template-file"), + {"type": "file_id", "path": "/workspace/overlap.txt", "file_id": source.id}] + populated_files = [inline("/workspace/overlap.txt", "-inline-file"), + {"type": "file_id", "path": "/workspace/selected-source.txt", "file_id": source.id}] + template_body = { + "name": "composition", "network": {"access": "disabled"}, "files": template_files, + "env": {"TEMPLATE_ONLY": marker + "-template-env", "SHARED": marker + "-template-shared"}, + "setup_commands": [{"command": "printf " + marker + "-template-command", "cwd": "/workspace"}], + "packages": packages, + } + created = api.with_raw_response.create(**template_body) + template_id = created.http_response.json()["id"] + templates.append(template_id) + assert created.status_code == 201 + safe(created.http_response) + template_snapshot = created.parse().to_dict() + base = {"agent": {"model": "composition-fixture"}, + "environment": {"type": "openai_hosted", "environment_template_id": template_id}} + default_files = [ + {"type": "inline", "path": "/workspace/template-only.txt", "size_bytes": len(marker + "-template-file")}, + {"type": "file_id", "path": "/workspace/overlap.txt", "file_id": source.id, "size_bytes": len(marker + "-source")}, + ] + selected_files = [ + {"type": "inline", "path": "/workspace/overlap.txt", "size_bytes": len(marker + "-inline-file")}, + {"type": "file_id", "path": "/workspace/selected-source.txt", "file_id": source.id, "size_bytes": len(marker + "-source")}, + ] + rows = [ + ("omitted", {}, default_files, packages), + ("populated", {"env": {"SHARED": marker + "-inline-shared", "INLINE_ONLY": marker + "-inline-env"}, + "setup_commands": [{"command": "printf " + marker + "-inline-one"}, + {"command": "printf " + marker + "-inline-two", "cwd": "/workspace"}], + "files": populated_files, "packages": {"python": ["idna==3.10"], "npm": []}}, selected_files, populated_packages), + ("empty", {"env": {}, "setup_commands": [], "files": [], "packages": {}}, [], packages), + ("null", {"env": None, "setup_commands": None, "files": None, "packages": None}, default_files, packages), + ("nested-null", {"packages": {"python": None, "npm": None, "system": None}}, default_files, packages), + ] + for label, overrides, expected_files, expected_packages in rows: + body = {**base, "environment": {**base["environment"], **overrides}} + key = secrets.token_hex(20) + response = request(body, key=key) + if response.status_code == 201: + sessions[label] = response.json()["id"] + assert response.status_code == 201, (label, response.status_code, response.text) + projected = projection(owner, sessions[label], expected_files, expected_packages) + assert response.json() == projected + if label == "populated": + retry_body, retry_key, frozen = body, key, projected + assert api.retrieve(template_id).to_dict() == template_snapshot + + denied = rejected(base, 404, token=settings["foreign"]) + missing = copy.deepcopy(base) + missing["environment"]["environment_template_id"] = "00000000-0000-0000-0000-000000000001" + assert rejected(missing, 404, token=settings["foreign"]) == denied + foreign_files = [{"type": "file_id", "path": "/workspace/foreign.txt", "file_id": other.id}] + rejected({**base, "environment": {**base["environment"], "files": foreign_files}}, 404) + rejected({**base, "environment": {**base["environment"], "env": {"PATH": marker}}}, 400) + rejected({**base, "environment": {**base["environment"], "files": [inline("/workspace/../escape", "-bad")]}}, 400) + # Referenced source ownership is checked only for the effective file list. + api.update(template_id, files=foreign_files) + rejected(base, 404) + response = request({**base, "environment": {**base["environment"], "files": []}}) + if response.status_code == 201: + sessions["excluded-source"] = response.json()["id"] + assert response.status_code == 201, response.text + projection(owner, sessions["excluded-source"], [], packages) + # Individually valid maps must not bypass the combined setup-size limit. + api.update(template_id, env={"LARGE_TEMPLATE": "t" * (300 * 1024)}) + rejected({**base, "environment": {**base["environment"], "files": [], + "env": {"LARGE_INLINE": "i" * (300 * 1024)}}}, 400) + api.delete(template_id) + templates.remove(template_id) + owner.files.delete(source.id) + files.remove((owner, source.id)) + replay = request(retry_body, key=retry_key, base=settings["recovered"]) + assert replay.status_code == 201 and replay.json() == frozen, replay.text + assert projection(reopened, sessions["populated"], selected_files, populated_packages) == frozen + changed = copy.deepcopy(retry_body) + changed["environment"]["env"]["SHARED"] = marker + "-changed" + assert request(changed, key=retry_key, base=settings["recovered"]).status_code == 409 + succeeded = True + print(json.dumps({"sessions": sessions, "rejected_keys": rejected_keys, + "result": "passed", "postgres": True, "native_model_execution": False})) + finally: + for template_id in templates: + api.delete(template_id) + for client, file_id in files: + client.files.delete(file_id) + if not succeeded: + for session_id in sessions.values(): + owner.beta.agents.sessions.delete(session_id) + # Successful Sessions stay alive for the Go owner's private snapshot assertions. + + +if __name__ == "__main__": + main()