Skip to content

Persist hosted sessions and approvals in the Foundry state store #164

Description

@shibayan

What happens

When ResponsesHostServer runs inside a Foundry container, the default session store and approval storage are files under the sandbox's home directory (packages/foundry/src/hosting/handler.ts:55-68):

  • FileSystemAgentSessionStore writes ~/.sessions/<user>/<key>.json (session-store.ts:68-73)
  • FileApprovalStorage writes under ~/.function_approvals (approval-storage.ts:48-61)

A sandbox home is preserved for the lifetime of that one sandbox and is visible only from it. A conversation is not bound to a sandbox: a client may continue a conversation or a previous_response_id from a request that lands on another sandbox, or after the original one was recycled. In both cases the snapshot is missing and the turn fails with a 500 (handler.ts:346-360) whose message asks the caller to "reuse the response's agent_session_id so the request is routed to the same persistent Foundry sandbox". Conversation continuity therefore depends on a sandbox affinity that the Responses protocol does not require and that a client has no reason to know about.

Both reference implementations keep this state in the platform's durable state store, not on the sandbox:

  • .NET Microsoft.Agents.AI.Foundry.Hosting: the default AgentSessionStore is FoundryAgentSessionStore (ServiceCollectionExtensions.cs:80,136,452), which writes through the AgentServer SDK's FoundryStateStore. Its doc comment: "In Foundry hosting it writes to the platform's durable state store, so a session survives container replacement and is visible to every instance of the agent. Outside Foundry hosting it uses the SDK's local state-store fallback under ~/.agentserver/state_stores." Pending tool approvals live in the session state bag (ToolApprovalIdMap.cs), so they inherit the same durability. A FileSystemAgentSessionStore exists in that package but is not the default.
  • Python agent_framework_foundry_hosting: ResponsesHostServer defaults to AgentSessionStoreProvider() and FunctionApprovalStoreProvider() (_responses.py:424-431), which return FoundryAgentSessionStore and FoundryFunctionApprovalStore "in all environments" (_state_store.py:216-311). Both are backed by FoundryStateStore.get_or_create(<scope>, user_isolation=True). The AgentServer SDK falls back to a local JSON file per store when the hosting environment variable is absent (azure-ai-agentserver-core, storage/_state.py).

The comments on defaultSessionStore and defaultApprovalStorage attribute the filesystem choice to Python ("FoundrySessionStore when is_hosted, in-memory otherwise"; "Python uses FileBasedFunctionApprovalStorage when hosted"). Neither class exists in the Python package. The comments describe a design Python does not have.

Why it matters

  • A conversation resumed on another instance, or after a sandbox recycle, fails instead of continuing. With either reference implementation it continues.
  • An mcp_approval_response for an approval raised on a different sandbox cannot be resolved. handler.ts:62-65 describes exactly this failure and names the filesystem as the protection against it, when it is the cause.
  • The response resource already lives in the Foundry storage service (FoundryResponseStore), so one turn's data is split today between a durable store and one sandbox's disk.

Required implementation

  • Make the hosted default for sessions and approvals a store backed by the Foundry state store: the storage service's state_stores/<name> and state_stores/<name>/items/<key> resources that the AgentServer SDKs use (azure-ai-agentserver-core, storage/_state.py:208-213). Reuse the existing FoundryStorageClient transport where it fits.
  • Partition per agent, user and conversation as the .NET store does, and keep the user-isolation guarantee the current stores document.
  • Keep a local fallback outside a container under the AGENTSERVER_STATE_ROOT convention already used for responses, so a node main.ts restart keeps working.
  • Keep FileSystemAgentSessionStore and FileApprovalStorage as explicit opt-ins or remove them; either way they stop being the hosted default.
  • Correct the comments and TSDoc that attribute the current design to Python.
  • Revisit the "no snapshot for previous_response_id" 500 once state is durable: a missing snapshot then means an id the store never saw.

Acceptance criteria

  • A test drives two turns of one conversation through two handler instances that share no filesystem and asserts the second turn continues, failing before the change with the current 500.
  • The same for an approval raised on one instance and answered on another.
  • The defaultSessionStore(true) / defaultApprovalStorage(true) expectations in hosting.test.ts:874-875 are updated to the new defaults.
  • A local run persists sessions and approvals across a process restart under AGENTSERVER_STATE_ROOT.
  • Every comment that cites Python or .NET behaviour here matches the sources named above.
  • CHANGELOG.md records the change of default; pnpm check passes.

Explicitly out of scope

The response store, which already uses the storage service, and workflow checkpoints, which this implementation does not have.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    foundryUsage: [Issues, PRs], Target: Microsoft Foundry integrationshostingUsage: [Issues, PRs], Target: agent hosting protocols and runtime

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions