agent host: overlap provider session startup with the turn-start checkpoint (experiment)

[Vijay Upadya (vijayupadya)]

Follow-up to #339016

TL;DR

• What: on a session's first turn, the provider's SDK session is now created while the turn-start checkpoint is captured, instead of after it. Dispatch still waits for both, so checkpoint behavior is unchanged.
• Gate: experiment-controlled setting chat.agentHost.experimental.overlapProviderPreparation, default off, for an ExP rollout. A trigger event marks exposure in both arms.
• Measured (MSBench A/B, same build, setting off vs on): time from turn start until the prompt is handed to the runtime drops by 160–240 ms (−20 to −27%) on Windows and 60–180 ms on Linux.
• Risk: low. Providers opt in; a failed preparation falls back to the normal send path; checkpoint ordering is unchanged.

Why

On a first turn, the agent host waits for the turn-start checkpoint (git status and a tree capture), then dispatches. Only then does the Copilot provider create the SDK session (client, customization snapshot, config, create/resume). On Windows each costs 0.3–1 s, and they ran back to back. They don't depend on each other: the provider can't touch the workspace before the prompt is sent, so session creation can run alongside the checkpoint.

Changes

• IAgentChats.prepareTurn(chat, turnId, workingDirectories, context): optional provider hook to set up session state before the send. It must not send a prompt or modify the working tree. Providers that don't implement it behave exactly as before.
• AgentSideEffects: when the setting is on, starts prepareTurn right after the turn's model and agent selection is applied. It runs alongside attachments, contributions and the checkpoint capture, and the dispatch waits for both.
  • A preparation failure is only logged; sendMessage then prepares as usual, so errors surface exactly as they do without the overlap.
  • A turn cancelled before dispatch takes the existing checkpoint-discard path.
• Copilot provider:
  • prepareTurn creates the session for a not-yet-started chat, or resumes the default or peer chat's session, if the chat has no live session. It runs on the same per-chat queue as sendMessage and retries a failed plugin sync first, as the send path does.
  • It records which turn it prepared the session for. That turn's sendMessage skips the snapshot-based configuration re-check: plugin-sync retry, customization snapshot, getRestartReason, disabled root MCP servers. The session was just created with that state. Explicit restart flags and additional-directory changes are still honored. Keying on the turn ID means a cancelled turn's leftover marker can never affect a later turn.
• Setting: chat.agentHost.experimental.overlapProviderPreparation (application scope, experiment: { mode: 'auto' }), forwarded to the agent host as the root key overlapProviderPreparation. It's a rollout gate, not an enterprise control, so it has no policy entry.
• Telemetry:
  • New sendStageProviderPreparationMs in agentHost.turnCompleted, the OTel turn_timing span and OTEL.md: the time dispatch still waits for preparation after the overlap, not the preparation's total cost.
  • Provider stage marks made during preparation happen before dispatch, so they don't appear in providerStage*Ms.
  • The experiment trigger (logSettingExperimentTrigger) fires in both arms once a turn reaches a provider that implements prepareTurn. It waits until the workbench has forwarded the assignment context, like AgentHostGitStateService.

Results

MSBench say_hello (Linux) and say_hello_windows (Windows Server 2022) containers. Both arms use the same build (Insiders 0d30d69 with #339016 and this change) and differ only in the setting. 4 runs per cell. Time from turn start until session.send() returns, p50:

Variant	Off	On	Change
Windows, no repo	611 ms	449 ms	−162 ms (−27%)
Windows, large repo (VS Code, 19.6K files)	1,180 ms	940 ms	−240 ms (−20%)
Linux, no repo	580 ms	398 ms	−182 ms (−31%)
Linux, large repo	710 ms	650 ms	−60 ms (−8%)

• Every "on" run reported sendStageProviderPreparationMs, so the treatment applied in all of them.
• Turn start → first model call moves the same way (−192 ms Windows no repo, −243 ms Linux no repo). End-to-end time to first progress also drops on Windows (−0.75 to −0.94 s p50), but model latency varies too much to separate at 4 runs per cell.
• Dispatch happens later in the "on" arm (+64 to +170 ms) because preparation moved before it, while the post-dispatch provider stages (create, finalize, persist, snapshot) disappear. Evaluate the experiment on time to first progress, not timeToProviderDispatch.
• Without the redundant-refresh skip, the overlap gained nothing. An earlier A/B, before that change, showed no improvement. sendMessage re-checked the configuration of the session prepareTurn had just created, which cancelled out the overlap.
• Windows contention: with a large repo, the checkpoint slows down when it runs alongside session creation (its remaining wait drops only 796 → 482 ms). The remaining checkpoint cost is the next thing to tackle.

[Copilot]

Copilot review overview

🟡 Changes recommended

Configuration changes during preparation can be missed, and cancelled prepared chats leak marker entries.

Review effort: Balanced
Findings: 2

Open (2)

• Clean up cancelled-turn markers during chat disposal · New
• Revalidate configuration before dispatching prepared turns · New

What changed in this PR

Overlaps Copilot provider session preparation with checkpoint capture to reduce first-turn latency behind an experiment gate.

Changes:

• Adds provider preparation hooks and Copilot session materialization.
• Adds experiment configuration, triggering, and tests.
• Adds provider-preparation timing to telemetry and OTel.

File	Description
common/​agent.ts	Defines the preparation hook.
common/​agentHostSchema.ts	Adds host configuration schema.
common/​agentHostStarter.config.contribution.ts	Registers the experiment setting.
common/​agentService.ts	Defines the setting ID.
common/​agentHostTelemetry.ts	Adds the preparation timing stage.
common/​otel/​agentHostTiming.ts	Exposes timing through OTel.
node/​agentSideEffects.ts	Coordinates preparation and checkpoint capture.
node/​copilot/​copilotAgent.ts	Implements Copilot preparation and refresh skipping.
node/​agentHostTelemetryReporter.ts	Reports the new timing measurement.
OTEL.md	Documents the timing attribute.
test/​node/​agentSideEffects.test.ts	Tests overlap and experiment behavior.
test/​node/​copilotAgent.test.ts	Tests materialization and refresh skipping.

---

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[github-actions]

Screenshot Changes

Base: 4fbe620f Current: 3d960b47

Changed (1)

chat/aiCustomizations/aiCustomizationManagementEditor/DiscoverPluginsLoadingMore/Light

Before	After

1 insignificant change(s) omitted (≤20 px, Δ≤2). See CI logs for details.