fix(azure-foundry): stateless getAgentStatus for multi-replica deployments - #1446
Open
shaileshpadave wants to merge 5 commits into
Open
fix(azure-foundry): stateless getAgentStatus for multi-replica deployments#1446shaileshpadave wants to merge 5 commits into
shaileshpadave wants to merge 5 commits into
Conversation
…eployments
AzureFoundryAgentClient stored per-run state (endpoint, runId, token provider)
in an in-memory ConcurrentHashMap keyed by threadId. In multi-replica server
deployments the 5-second status-poll callback could arrive on a different pod
than the one that ran startAgent, causing "No execution found" failures on every
execution after the first.
Fix: startAgent now returns a compound executionId that base64-encodes the
non-sensitive run context {threadId, runId, endpoint, assistantId, apiVersion}.
getAgentStatus decodes this to reconstruct the Azure API call on any pod and
re-authenticates using the original task credentialRef, which is available in
the ConductorAgentRequest passed by the updated ConductorAgentDelegate.
Interface change: ConductorAgentClient.getAgentStatus(String) gains a second
ConductorAgentRequest parameter. All five implementations and both test fakes
are updated. ServiceConductorAgentClient ignores the new parameter (AgentService
has its own persistent state). BedrockAgentClient signature is updated; its
in-memory model is unchanged pending a follow-up — the stateless fix for Bedrock
requires a similar compound-executionId approach.
The respondContexts map is retained for respond() and cancelAgent(), which
receive no credentialRef and therefore cannot re-authenticate on a different
replica; single-turn agents (the common case) no longer use it.
…RunStore SPI Following the same pattern as ServiceConductorAgentClient -> AgentService, run context (threadId, runId, endpoint, credentialRef, scope) is now persisted in an injected AzureAgentRunStore rather than encoded in the executionId. The default store is in-memory (InMemoryAzureAgentRunStore, @ConditionalOnMissingBean); HA deployments can substitute a Redis or DB-backed implementation, mirroring how orkes-conductor overrides SkillMetadataDAO and SkillPackageStore. This also removes the previous same-pod limitation on respond() and cancelAgent(): by storing credentialRef in the run context, any replica can re-authenticate and handle multi-turn and cancellation requests without in-process state.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
AzureFoundryAgentClientstored per-run state (endpoint, runId, token provider) in an in-memoryConcurrentHashMapkeyed by threadId. In multi-replica server deployments the 5-second status-poll callback arrives on a different pod than the one that ranstartAgent, causing "No execution found" failures on every execution.Confirmed on a 3-pod cluster: pod A runs
startAgent, pod B/C handles the callback → empty map → task FAILED.Fix
startAgentnow returns a compound executionId that base64-encodes the non-sensitive run context{threadId, runId, endpoint, assistantId, apiVersion}.getAgentStatusdecodes this to reconstruct the Azure API call on any pod and re-authenticates using the original taskcredentialRef.Interface change:
ConductorAgentClient.getAgentStatus(String)gains aConductorAgentRequestparameter so implementations can access credentials without in-process state. All five implementations and both test fakes are updated.The
respondContextsmap is retained only forrespond()/cancelAgent()(multi-turn WAITING), which receive nocredentialRef. Single-turn agents (the common case) no longer depend on in-memory state.Test plan
./gradlew :conductor-ai:compileJava :conductor-ai:compileTestJava :conductor-agentspan:compileJavapasses./gradlew :conductor-ai:testpasses./gradlew :conductor-agentspan:spotlessCheckpasses