From 4771098e0d421706f4ccb3fff19c5a71ac58b090 Mon Sep 17 00:00:00 2001 From: "Ross A. Wollman" Date: Thu, 17 Sep 2026 22:00:08 -0700 Subject: [PATCH 1/6] Checkpoint restart-based enterprise OTel recovery Preserve the event-driven, single-attempt recovery implementation and OTel settings-block replacement before evaluating an in-process alternative. Refs #336102. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../docs/monitoring/agent_monitoring.md | 35 +- .../extension/vscode-node/services.ts | 35 +- .../otel/common/otelStaleConfigMonitor.ts | 140 ++++++++ .../test/otelStaleConfigMonitor.spec.ts | 331 ++++++++++++++++++ .../otel/vscode-node/otelConfigResolver.ts | 28 ++ .../extension/otel/vscode-node/otelContrib.ts | 87 +++-- .../src/platform/otel/common/otelConfig.ts | 2 +- .../otel/common/otelConfigResolution.ts | 129 +++++++ .../common/test/otelConfigResolution.spec.ts | 220 ++++++++++++ .../otel/common/test/otelTestSettings.ts | 21 ++ 10 files changed, 962 insertions(+), 66 deletions(-) create mode 100644 extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts create mode 100644 extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts create mode 100644 extensions/copilot/src/extension/otel/vscode-node/otelConfigResolver.ts create mode 100644 extensions/copilot/src/platform/otel/common/otelConfigResolution.ts create mode 100644 extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts create mode 100644 extensions/copilot/src/platform/otel/common/test/otelTestSettings.ts diff --git a/extensions/copilot/docs/monitoring/agent_monitoring.md b/extensions/copilot/docs/monitoring/agent_monitoring.md index cd020a2f9c22cf..fbb323fea14d37 100644 --- a/extensions/copilot/docs/monitoring/agent_monitoring.md +++ b/extensions/copilot/docs/monitoring/agent_monitoring.md @@ -38,7 +38,7 @@ Open **Settings** (`Ctrl+,`) and add: } ``` -> **Note:** You can also use environment variables instead of VS Code settings (see [Configuration](#configuration)). Precedence is **enterprise policy > environment variables > settings**. +> **Note:** You can also use environment variables instead of VS Code settings (see [Configuration](#configuration)). Applied policy is included in settings values, but environment variables can still override those values in this extension. See the [activation limitations](#activation). ### 3. Generate Telemetry @@ -82,7 +82,11 @@ Open **Settings** (`Ctrl+,`) and search for `copilot otel`: ### Environment Variables -Environment variables take precedence over VS Code settings, and **enterprise managed settings (policy) take precedence over both** — admins can centrally mandate any `github.copilot.chat.otel.*` value. +Environment variables retain their existing precedence. When enterprise OTel configuration is +recognized through the application-scoped policy defaults, the entire Copilot OTel settings +block comes from those policy values and schema defaults. Personal `settings.json` values are +not used to fill omitted fields: headers and resource attributes default to empty maps, not +the user's maps. Other VS Code settings are unaffected. | Variable | Default | Description | |---|---|---| @@ -102,9 +106,34 @@ Environment variables take precedence over VS Code settings, and **enterprise ma ### Activation +When late enterprise OTel settings turn on external export after Copilot's telemetry service +started without it, Copilot can restart the extension hosts for that window to recover. It warns +before requesting the restart and confirms it afterward. This also interrupts other extensions +in the window. If the restart is unavailable, vetoed, or fails to apply the settings, +a warning offers **Reload Window** instead. User changes and policy withdrawal remain +opt-in reloads. Exporter behavior and environment-variable precedence are unchanged. +It uses changes to the application-scoped, policy-backed configuration defaults as a recovery +signal, without a new API. Normal personal settings changes do not change those defaults. +Automatic recovery additionally requires policy-enabled OTLP export targeting the collector in +those defaults. Disabled and DB-only pipelines, unrelated partial policies, and configurations +still redirected by environment variables to a different collector or file do not qualify. +Policy edits indistinguishable from schema defaults cannot be identified as new policy and retain +the opt-in reload behavior. Conflicting environment variables can still prevent recovery, and this +does not enforce precedence over environment variables or guarantee telemetry produced before +the restart. The default-value signal cannot distinguish a policy consisting entirely of +schema-default values from no policy. + +There is no periodic polling or restart loop. A startup check and configuration events trigger +checks, coalesced by a 500 ms debounce. At most one automatic off-to-on recovery is attempted per +workspace and editor session, identified by `vscode.env.sessionId`. The attempt remains recorded +even after success or failure. Later policy updates only offer a reload, deduplicated while stale. +If policy first arrives after a later sign-in, that single recovery can happen then rather than +immediately at launch. A one-off 15-second grace period allows a requested restart to finish +before a still-running host shows the reload fallback. + OTel is **off by default** with zero overhead. It activates when: -- enterprise policy enables it (managed `telemetry.enabled` or a managed endpoint), or +- an applied enterprise policy makes `github.copilot.chat.otel.enabled` true, or - `COPILOT_OTEL_ENABLED=true`, or - `OTEL_EXPORTER_OTLP_ENDPOINT` is set, or - `github.copilot.chat.otel.enabled` is `true`, or diff --git a/extensions/copilot/src/extension/extension/vscode-node/services.ts b/extensions/copilot/src/extension/extension/vscode-node/services.ts index 9271573284f753..63950742c8406d 100644 --- a/extensions/copilot/src/extension/extension/vscode-node/services.ts +++ b/extensions/copilot/src/extension/extension/vscode-node/services.ts @@ -57,7 +57,7 @@ import { IFetcherService } from '../../../platform/networking/common/fetcherServ import { IToolDeferralService } from '../../../platform/networking/common/toolDeferralService'; import { ChatWebSocketManager, IChatWebSocketManager } from '../../../platform/networking/node/chatWebSocketManager'; import { FetcherService } from '../../../platform/networking/vscode-node/fetcherServiceImpl'; -import { resolveOTelConfig } from '../../../platform/otel/common/otelConfig'; +import { IOTelConfigResolver } from '../../../platform/otel/common/otelConfigResolution'; import { IOTelService } from '../../../platform/otel/common/otelService'; import { InMemoryOTelService } from '../../../platform/otel/node/inMemoryOTelService'; import { IOTelSqliteStore, OTelSqliteStore } from '../../../platform/otel/node/sqlite/otelSqliteStore'; @@ -118,6 +118,7 @@ import { ILinkifyService, LinkifyService } from '../../linkify/common/linkifySer import { DebugCommandToConfigConverter, IDebugCommandToConfigConverter } from '../../onboardDebug/node/commandToConfigConverter'; import { DebuggableCommandIdentifier, IDebuggableCommandIdentifier } from '../../onboardDebug/node/debuggableCommandIdentifier'; import { ILanguageToolsProvider, LanguageToolsProvider } from '../../onboardDebug/node/languageToolsProvider'; +import { VSCodeOTelConfigResolver } from '../../otel/vscode-node/otelConfigResolver'; import { IPowerService } from '../../power/common/powerService'; import { PowerService } from '../../power/vscode-node/powerService'; import { ChatMLFetcherImpl } from '../../prompt/node/chatMLFetcher'; @@ -293,34 +294,10 @@ export function registerServices(builder: IInstantiationServiceBuilder, extensio const otelSqliteStore = new OTelSqliteStore(otelDbPath); builder.define(IOTelSqliteStore, otelSqliteStore); - // OTel service — resolve config from env + settings, create appropriate impl - const otelSettings = workspace.getConfiguration('github.copilot.chat.otel'); - const policyValue = (key: string): T | undefined => (otelSettings.inspect(key) as { policyValue?: T } | undefined)?.policyValue; - const otelConfig = resolveOTelConfig({ - env: process.env, - settingEnabled: otelSettings.get('enabled'), - settingExporterType: otelSettings.get<'otlp-grpc' | 'otlp-http' | 'console' | 'file'>('exporterType'), - settingOtlpEndpoint: otelSettings.get('otlpEndpoint'), - settingCaptureContent: otelSettings.get('captureContent'), - settingMaxAttributeSizeChars: otelSettings.get('maxAttributeSizeChars'), - settingOutfile: otelSettings.get('outfile') || undefined, - settingDbSpanExporter: otelSettings.get('dbSpanExporter.enabled'), - settingProtocol: otelSettings.get('protocol') || undefined, - policyEnabled: policyValue('enabled'), - policyExporterType: policyValue<'otlp-grpc' | 'otlp-http' | 'console' | 'file'>('exporterType'), - policyOtlpEndpoint: policyValue('otlpEndpoint'), - policyCaptureContent: policyValue('captureContent'), - policyOutfile: policyValue('outfile'), - policyProtocol: policyValue('protocol'), - settingServiceName: otelSettings.get('serviceName') || undefined, - policyServiceName: policyValue('serviceName'), - settingResourceAttributes: otelSettings.get>('resourceAttributes'), - policyResourceAttributes: policyValue>('resourceAttributes'), - settingHeaders: otelSettings.get>('headers'), - policyHeaders: policyValue>('headers'), - extensionVersion: extensionContext.extension.packageJSON.version ?? '0.0.0', - sessionId: env.sessionId, - }); + // Keep the exact resolution the service uses so late policy can be detected. + const otelConfigResolver = new VSCodeOTelConfigResolver(process.env, extensionContext.extension.packageJSON.version ?? '0.0.0', env.sessionId); + builder.define(IOTelConfigResolver, otelConfigResolver); + const otelConfig = otelConfigResolver.activeResolution.config; if (otelConfig.enabled) { // Dynamic import to avoid loading OTel SDK when disabled const { NodeOTelService } = require('../../../platform/otel/node/otelServiceImpl') as typeof import('../../../platform/otel/node/otelServiceImpl'); diff --git a/extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts b/extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts new file mode 100644 index 00000000000000..1e9240d6a93505 --- /dev/null +++ b/extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts @@ -0,0 +1,140 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + * Licensed under the MIT License. See License.txt in the project root for license information. + *--------------------------------------------------------------------------------------------*/ + +import { ILogService } from '../../../platform/log/common/logService'; +import { parseOtlpEndpoint } from '../../../platform/otel/common/otelConfig'; +import { classifyOTelConfigDrift, describeOTelConfigDrift, IOTelConfigResolver, IResolvedOTelConfig, OTelConfigDrift } from '../../../platform/otel/common/otelConfigResolution'; +import { StringSHA1 } from '../../../util/vs/base/common/hash'; + +export interface IOTelPolicyRestartRecord { + readonly sessionId: string; + readonly fingerprint: string; + readonly acknowledged: boolean; +} + +export interface IOTelStaleConfigHost { + getRestartRecord(): IOTelPolicyRestartRecord | undefined; + setRestartRecord(record: IOTelPolicyRestartRecord | undefined): Promise; + restartExtensionHost(): Promise; + warnPolicyNotApplied(): void; + promptReload(current: IResolvedOTelConfig): void; + notifyPolicyRestarted(): void; +} + +/** Compares against service construction, not the later contribution's initial settings. */ +export class OTelStaleConfigMonitor { + private _handledFingerprint: string | undefined; + private _pendingCheck: Promise = Promise.resolve(OTelConfigDrift.None); + private _policyNoticeShown = false; + + constructor( + private readonly _resolver: IOTelConfigResolver, + private readonly _host: IOTelStaleConfigHost, + private readonly _logService: ILogService, + ) { } + + check(): Promise { + const check = this._pendingCheck.then(() => this._check()); + // Return failures to the caller, but leave the queue usable for the next event. + this._pendingCheck = check.catch(() => OTelConfigDrift.None); + return check; + } + + private async _check(): Promise { + const active = this._resolver.activeResolution; + const current = this._resolver.resolve(); + const drift = classifyOTelConfigDrift(active, current); + if (drift === OTelConfigDrift.None) { + this._handledFingerprint = undefined; + this._policyNoticeShown = false; + const record = this._host.getRestartRecord(); + if (record?.sessionId === active.config.sessionId && record.fingerprint === fingerprintOf(active) && !record.acknowledged) { + try { + // Retain the session budget after success; future policy updates must not + // cause another automatic restart in this editor session. + await this._host.setRestartRecord({ ...record, acknowledged: true }); + this._host.notifyPolicyRestarted(); + } catch (error) { + this._logService.warn(`[OTel] Failed to acknowledge the telemetry policy restart: ${error}`); + } + } + return drift; + } + + const fingerprint = fingerprintOf(current); + if (this._handledFingerprint === fingerprint) { + return drift; + } + if (drift === OTelConfigDrift.User || drift === OTelConfigDrift.Withdrawal) { + this._handledFingerprint = fingerprint; + this._host.promptReload(current); + return drift; + } + if (active.config.enabledExplicitly || !isPolicyEnabledOtlp(current)) { + this._handledFingerprint = fingerprint; + if (!this._policyNoticeShown) { + this._policyNoticeShown = true; + this._host.promptReload(current); + } + return drift; + } + + const changed = describeOTelConfigDrift(active.config, current.config).join(', '); + const record = this._host.getRestartRecord(); + if (record?.sessionId === active.config.sessionId) { + this._handledFingerprint = fingerprint; + this._logService.warn(`[OTel] Automatic telemetry recovery was already attempted in this editor session (${changed}). Not restarting again.`); + this._warnPolicyNotApplied(); + return drift; + } + + try { + await this._host.setRestartRecord({ sessionId: active.config.sessionId, fingerprint, acknowledged: false }); + } catch (error) { + this._logService.warn(`[OTel] Cannot store the telemetry policy restart guard: ${error}`); + this._warnPolicyNotApplied(); + return drift; + } + + this._handledFingerprint = fingerprint; + this._logService.warn(`[OTel] Enterprise telemetry policy changed after OTel was initialized (${changed}). Restarting the extension host to apply it.`); + try { + await this._host.restartExtensionHost(); + // A slow restart could still succeed, so keep its guard. + this._logService.warn('[OTel] The extension host was not restarted. Enterprise telemetry policy is not applied until the window is reloaded.'); + } catch (error) { + this._logService.warn(`[OTel] Failed to restart the extension host: ${error}`); + } + this._warnPolicyNotApplied(); + return drift; + } + + private _warnPolicyNotApplied(): void { + if (!this._policyNoticeShown) { + this._policyNoticeShown = true; + this._host.warnPolicyNotApplied(); + } + } +} + +function isPolicyEnabledOtlp(resolution: IResolvedOTelConfig): boolean { + const { config, defaultValues } = resolution; + if (defaultValues.enabled !== true || !config.enabled || !config.enabledExplicitly + || (config.exporterType !== 'otlp-http' && config.exporterType !== 'otlp-grpc') + || typeof defaultValues.otlpEndpoint !== 'string') { + return false; + } + const endpoint = parseOtlpEndpoint(defaultValues.otlpEndpoint, config.exporterType === 'otlp-grpc' ? 'grpc' : 'http'); + return endpoint !== undefined && /^https?:\/\//.test(endpoint) && config.otlpEndpoint === endpoint; +} + +function fingerprintOf(resolution: IResolvedOTelConfig): string { + const sha = new StringSHA1(); + sha.update(JSON.stringify([resolution.config, resolution.defaultValues], (_key, value) => + value && typeof value === 'object' && !Array.isArray(value) + ? Object.fromEntries(Object.entries(value).sort(([a], [b]) => a < b ? -1 : a > b ? 1 : 0)) + : value)); + return sha.digest(); +} diff --git a/extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts b/extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts new file mode 100644 index 00000000000000..fc5e2ce1199610 --- /dev/null +++ b/extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts @@ -0,0 +1,331 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + * Licensed under the MIT License. See License.txt in the project root for license information. + *--------------------------------------------------------------------------------------------*/ + +import { beforeEach, describe, expect, it } from 'vitest'; +import { IOTelConfigResolver, IResolvedOTelConfig, OTelConfigDrift, resolveOTelConfigFromSettings } from '../../../../platform/otel/common/otelConfigResolution'; +import { TestOTelSettings } from '../../../../platform/otel/common/test/otelTestSettings'; +import { TestLogService } from '../../../../platform/testing/common/testLogService'; +import { IOTelPolicyRestartRecord, IOTelStaleConfigHost, OTelStaleConfigMonitor } from '../otelStaleConfigMonitor'; + +class TestResolver implements IOTelConfigResolver { + declare readonly _serviceBrand: undefined; + readonly activeResolution: IResolvedOTelConfig; + constructor(private readonly _settings: TestOTelSettings, private readonly _env: Record = {}, private readonly _sessionId = 'session') { + this.activeResolution = this.resolve(); + } + resolve(): IResolvedOTelConfig { + return resolveOTelConfigFromSettings(this._settings, this._env, '1.0.0', this._sessionId); + } +} + +/** The record outlives the extension host, as workspaceState does. */ +class TestHost implements IOTelStaleConfigHost { + record: IOTelPolicyRestartRecord | undefined; + recordAtRestart: IOTelPolicyRestartRecord | undefined; + restarts = 0; + warnings = 0; + prompts = 0; + notifications = 0; + restartError: Error | undefined; + storageError: Error | undefined; + restartCompleted: (() => void) | undefined; + + getRestartRecord() { return this.record; } + async setRestartRecord(record: IOTelPolicyRestartRecord | undefined) { + if (this.storageError) { + throw this.storageError; + } + this.record = record; + } + async restartExtensionHost(): Promise { + this.restarts++; + this.recordAtRestart = this.record; + if (this.restartError) { + throw this.restartError; + } + // Simulates the host remaining alive after the command and grace period. + await new Promise(resolve => { this.restartCompleted = resolve; }); + } + warnPolicyNotApplied() { this.warnings++; } + promptReload() { this.prompts++; } + notifyPolicyRestarted() { this.notifications++; } +} + +class RecordingLogService extends TestLogService { + readonly warnings: string[] = []; + override warn(message: string): void { this.warnings.push(message); } +} + +const managedPolicy = { enabled: true, otlpEndpoint: 'https://collector.example.com', headers: { authorization: 'secret' } }; + +describe('OTelStaleConfigMonitor', () => { + let settings: TestOTelSettings; + let host: TestHost; + let log: RecordingLogService; + const newHost = () => new OTelStaleConfigMonitor(new TestResolver(settings), host, log); + + /** Drain the promise queue as far as the simulated, non-returning restart. */ + async function startRestart(monitor: OTelStaleConfigMonitor) { + const pending = monitor.check(); + await Promise.resolve(); + await Promise.resolve(); + expect(host.recordAtRestart).toBeDefined(); + return { pending }; + } + + beforeEach(() => { + settings = new TestOTelSettings(); + host = new TestHost(); + log = new RecordingLogService(); + }); + + it('restarts for policy that lands before the contribution can register its watcher', async () => { + const resolver = new TestResolver(settings); + expect(resolver.activeResolution.config.enabled).toBe(false); + settings.policy = managedPolicy; + const monitor = new OTelStaleConfigMonitor(resolver, host, log); + await startRestart(monitor); + expect(host.restarts).toBe(1); + expect(host.warnings + host.prompts + host.notifications).toBe(0); + expect(log.warnings.join('\n')).toContain('headers'); + expect(log.warnings.join('\n')).not.toContain('secret'); + expect(log.warnings.join('\n')).not.toContain('collector.example'); + }); + + it('acknowledges a successful restart exactly once and retains the session budget', async () => { + const first = newHost(); + settings.policy = managedPolicy; + await startRestart(first); + const second = newHost(); + expect(await second.check()).toBe(OTelConfigDrift.None); + await second.check(); + await newHost().check(); + expect(host.record).toMatchObject({ sessionId: 'session', acknowledged: true }); + expect(host.notifications).toBe(1); + expect(host.restarts).toBe(1); + expect(host.warnings + host.prompts).toBe(0); + }); + + it('does not repeat the acknowledgement for concurrent checks', async () => { + const first = newHost(); + settings.policy = managedPolicy; + await startRestart(first); + const second = newHost(); + await Promise.all([second.check(), second.check(), second.check()]); + expect(host.notifications).toBe(1); + }); + + it('does not acknowledge a different target or forget its pending guard', async () => { + const first = newHost(); + settings.policy = managedPolicy; + await startRestart(first); + settings.policy = { ...managedPolicy, serviceName: 'different' }; + await newHost().check(); + expect(host.notifications).toBe(0); + expect(host.record).toBeDefined(); + }); + + it('does not loop when the restarted host loses the race again', async () => { + const first = newHost(); + settings.policy = managedPolicy; + await startRestart(first); + settings.policy = {}; + const second = newHost(); + await second.check(); + expect(host.record).toBeDefined(); + expect(host.notifications).toBe(0); + settings.policy = managedPolicy; + expect(await second.check()).toBe(OTelConfigDrift.Policy); + expect(host.restarts).toBe(1); + expect(host.warnings).toBe(1); + // A different policy still cannot spend a second automatic attempt in this session. + settings.policy = { ...managedPolicy, otlpEndpoint: 'https://different.example' }; + await second.check(); + expect(host.restarts).toBe(1); + expect(host.warnings).toBe(1); + + settings.policy = {}; + const third = new OTelStaleConfigMonitor(new TestResolver(settings, {}, 'new-editor-session'), host, log); + settings.policy = managedPolicy; + await startRestart(third); + expect(host.restarts).toBe(2); + }); + + it('canonicalizes object key order when checking the restart guard', async () => { + const first = newHost(); + settings.policy = { ...managedPolicy, headers: { a: '1', b: '2' } }; + await startRestart(first); + settings.policy = {}; + const second = newHost(); + settings.policy = { ...managedPolicy, headers: { b: '2', a: '1' } }; + await second.check(); + expect(host.restarts).toBe(1); + expect(host.record?.fingerprint).toMatch(/^[a-f0-9]{40}$/); + }); + + it('waits for the restart grace, then warns without clearing the guard', async () => { + const monitor = newHost(); + settings.policy = managedPolicy; + const { pending } = await startRestart(monitor); + expect(host.warnings).toBe(0); + host.restartCompleted!(); + expect(await pending).toBe(OTelConfigDrift.Policy); + await monitor.check(); + expect(host.warnings).toBe(1); + expect(host.restarts).toBe(1); + expect(host.record).toBeDefined(); + }); + + it('retains the session budget and warns when the restart command throws', async () => { + const monitor = newHost(); + settings.policy = managedPolicy; + host.restartError = new Error('command unavailable'); + await monitor.check(); + expect(host.record?.sessionId).toBe('session'); + expect(host.warnings).toBe(1); + settings.policy = { ...managedPolicy, headers: { authorization: 'changed' } }; + await monitor.check(); + expect(host.restarts).toBe(1); + expect(host.warnings).toBe(1); + }); + + it('retries storage on the next event and never restarts without a stored guard', async () => { + const monitor = newHost(); + settings.policy = managedPolicy; + host.storageError = new Error('storage unavailable'); + await monitor.check(); + expect(host.restarts).toBe(0); + expect(host.warnings).toBe(1); + host.storageError = undefined; + await startRestart(monitor); + expect(host.restarts).toBe(1); + }); + + it('retries consuming the success record if storage fails, without duplicate information', async () => { + const first = newHost(); + settings.policy = managedPolicy; + await startRestart(first); + const second = newHost(); + host.storageError = new Error('storage unavailable'); + await second.check(); + expect(host.notifications).toBe(0); + expect(host.record).toBeDefined(); + host.storageError = undefined; + await second.check(); + await second.check(); + expect(host.notifications).toBe(1); + expect(host.record?.acknowledged).toBe(true); + }); + + it('prompts only once for later policy updates after successful recovery', async () => { + const first = newHost(); + settings.policy = managedPolicy; + await startRestart(first); + const second = newHost(); + await second.check(); + settings.policy = { ...managedPolicy, headers: { authorization: 'rotation-1' } }; + await second.check(); + settings.policy = { ...managedPolicy, headers: { authorization: 'rotation-2' } }; + await second.check(); + expect(host.restarts).toBe(1); + expect(host.prompts).toBe(1); + }); + + it('does not auto-restart for mid-session policy changes when policy was present at startup', async () => { + settings.policy = managedPolicy; + const monitor = newHost(); + settings.policy = { ...managedPolicy, otlpEndpoint: 'https://new.example' }; + await monitor.check(); + expect(host.restarts).toBe(0); + expect(host.prompts).toBe(1); + }); + + it('only prompts when policy is withdrawn', async () => { + settings.policy = managedPolicy; + const monitor = newHost(); + settings.policy = {}; + expect(await monitor.check()).toBe(OTelConfigDrift.Withdrawal); + expect(host.restarts).toBe(0); + expect(host.prompts).toBe(1); + }); + + it('only prompts for user changes, once per target', async () => { + const monitor = newHost(); + settings.user = { enabled: true }; + expect(await monitor.check()).toBe(OTelConfigDrift.User); + await monitor.check(); + expect(host.restarts).toBe(0); + expect(host.prompts).toBe(1); + }); + + it('does not mistake a personal collector setting for policy', async () => { + const monitor = newHost(); + settings.user = { enabled: true, otlpEndpoint: 'https://personal.example' }; + expect(await monitor.check()).toBe(OTelConfigDrift.User); + expect(host.restarts).toBe(0); + expect(host.prompts).toBe(1); + }); + + it('does not restart pointlessly when env precedence leaves the resolved config unchanged', async () => { + const resolver = new TestResolver(settings, { COPILOT_OTEL_ENABLED: 'false' }); + const monitor = new OTelStaleConfigMonitor(resolver, host, log); + settings.policy = managedPolicy; + expect(await monitor.check()).toBe(OTelConfigDrift.None); + expect(host.restarts + host.notifications).toBe(0); + }); + + it.each([ + { + name: 'a DB-only pipeline and a managed service name', + user: { 'dbSpanExporter.enabled': true }, + policy: { serviceName: 'managed' }, + env: {}, + }, + { + name: 'user-enabled export with only managed headers', + user: { enabled: true }, + policy: { headers: { managed: '1' } }, + env: {}, + }, + { + name: 'a personal endpoint that overrides the managed collector', + user: {}, + policy: managedPolicy, + env: { OTEL_EXPORTER_OTLP_ENDPOINT: 'https://personal.example' }, + }, + { + name: 'a personal file exporter that overrides OTLP', + user: {}, + policy: managedPolicy, + env: { COPILOT_OTEL_FILE_EXPORTER_PATH: '/tmp/personal-otel.jsonl' }, + }, + { + name: 'a non-HTTP collector URL', + user: {}, + policy: { ...managedPolicy, otlpEndpoint: 'file:///tmp/not-a-collector' }, + env: {}, + }, + ])('does not automatically restart for $name', async ({ user, policy, env }) => { + settings.user = user; + const resolver = new TestResolver(settings, env); + const monitor = new OTelStaleConfigMonitor(resolver, host, log); + settings.policy = policy; + await monitor.check(); + expect(host.restarts).toBe(0); + expect(host.notifications).toBe(0); + }); + + it('recovers explicit policy enablement even when the collector equals the schema default', async () => { + const monitor = newHost(); + settings.policy = { enabled: true, otlpEndpoint: 'http://localhost:4318' }; + await startRestart(monitor); + expect(host.restarts).toBe(1); + }); + + it('does nothing on a normal, unchanged startup', async () => { + expect(await newHost().check()).toBe(OTelConfigDrift.None); + expect(host.restarts + host.prompts + host.warnings + host.notifications).toBe(0); + }); +}); diff --git a/extensions/copilot/src/extension/otel/vscode-node/otelConfigResolver.ts b/extensions/copilot/src/extension/otel/vscode-node/otelConfigResolver.ts new file mode 100644 index 00000000000000..ad46f687f27c1a --- /dev/null +++ b/extensions/copilot/src/extension/otel/vscode-node/otelConfigResolver.ts @@ -0,0 +1,28 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + * Licensed under the MIT License. See License.txt in the project root for license information. + *--------------------------------------------------------------------------------------------*/ + +import { workspace } from 'vscode'; +import { IOTelConfigResolver, IResolvedOTelConfig, resolveOTelConfigFromSettings, snapshotOTelEnv } from '../../../platform/otel/common/otelConfigResolution'; + +export const OTEL_SETTINGS_SECTION = 'github.copilot.chat.otel'; + +export class VSCodeOTelConfigResolver implements IOTelConfigResolver { + declare readonly _serviceBrand: undefined; + private readonly _env: Record; + readonly activeResolution: IResolvedOTelConfig; + + constructor( + env: Record, + private readonly _extensionVersion: string, + private readonly _sessionId: string, + ) { + this._env = snapshotOTelEnv(env); + this.activeResolution = this.resolve(); + } + + resolve(): IResolvedOTelConfig { + return resolveOTelConfigFromSettings(workspace.getConfiguration(OTEL_SETTINGS_SECTION), this._env, this._extensionVersion, this._sessionId); + } +} diff --git a/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts b/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts index 2b42094f6deb25..3c2a70517d5492 100644 --- a/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts +++ b/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts @@ -5,20 +5,24 @@ import * as os from 'os'; import * as vscode from 'vscode'; -import { ConfigKey } from '../../../platform/configuration/common/configurationService'; import { IVSCodeExtensionContext } from '../../../platform/extContext/common/extensionContext'; import { ILogService } from '../../../platform/log/common/logService'; import { DEFAULT_OTLP_ENDPOINT } from '../../../platform/otel/common/otelConfig'; +import { IOTelConfigResolver } from '../../../platform/otel/common/otelConfigResolution'; import { IOTelService } from '../../../platform/otel/common/otelService'; import { IOTelSqliteStore, type OTelSqliteStore } from '../../../platform/otel/node/sqlite/otelSqliteStore'; import { ITelemetryService } from '../../../platform/telemetry/common/telemetry'; +import { RunOnceScheduler, timeout } from '../../../util/vs/base/common/async'; import { Disposable } from '../../../util/vs/base/common/lifecycle'; import type { IExtensionContribution } from '../../common/contributions'; +import { IOTelPolicyRestartRecord, OTelStaleConfigMonitor } from '../common/otelStaleConfigMonitor'; +import { OTEL_SETTINGS_SECTION } from './otelConfigResolver'; const OPEN_OTEL_SETTINGS_COMMAND = 'github.copilot.chat.otel.openSettings'; const STATUS_ACTIVE_COMMAND = 'github.copilot.chat.otel.statusActive'; const OTEL_ENABLED_EXPLICITLY_CONTEXT_KEY = 'github.copilot.otel.enabledExplicitly'; const CHAT_STATUS_ITEM_ID = 'copilot.otelStatus'; +const POLICY_RESTART_RECORD_KEY = 'github.copilot.otel.latePolicyRestart'; const DOCS_URL = 'https://code.visualstudio.com/docs/agents/guides/monitoring-agents'; /** @@ -34,6 +38,7 @@ export class OTelContrib extends Disposable implements IExtensionContribution { @ILogService private readonly _logService: ILogService, @ITelemetryService private readonly _telemetryService: ITelemetryService, @IVSCodeExtensionContext private readonly _extensionContext: IVSCodeExtensionContext, + @IOTelConfigResolver private readonly _otelConfigResolver: IOTelConfigResolver, ) { super(); if (this._otelService.config.enabled) { @@ -57,8 +62,7 @@ export class OTelContrib extends Disposable implements IExtensionContribution { this._logService.info('[OTel] Flush complete'); })); - // Prompt for reload when OTel settings change — these are read once at - // activation and the OTel SDK cannot be reconfigured at runtime. + // Recover policy arriving after service construction; user changes remain opt-in. this._watchForReloadRequiredChanges(); // Export the agent-traces.db file. @@ -114,41 +118,58 @@ export class OTelContrib extends Disposable implements IExtensionContribution { } private _watchForReloadRequiredChanges(): void { - const reloadSettings = [ - ConfigKey.Advanced.OTelEnabled, - ConfigKey.Advanced.OTelExporterType, - ConfigKey.Advanced.OTelOtlpEndpoint, - ConfigKey.Advanced.OTelCaptureContent, - ConfigKey.Advanced.OTelOutfile, - ConfigKey.Advanced.OTelDbSpanExporter, - ]; - - // Snapshot initial values to avoid prompting when the setting hasn't actually changed - const initialValues = new Map(reloadSettings.map(s => [s.fullyQualifiedId, vscode.workspace.getConfiguration().get(s.fullyQualifiedId)])); - - this._register(vscode.workspace.onDidChangeConfiguration(async e => { - const currentConfig = vscode.workspace.getConfiguration(); - const changedSettings = reloadSettings.filter(s => - e.affectsConfiguration(s.fullyQualifiedId) && - currentConfig.get(s.fullyQualifiedId) !== initialValues.get(s.fullyQualifiedId) - ); - if (changedSettings.length === 0) { - return; + const state = this._extensionContext.workspaceState; + const monitor = new OTelStaleConfigMonitor(this._otelConfigResolver, { + getRestartRecord: () => state.get(POLICY_RESTART_RECORD_KEY), + setRestartRecord: async record => state.update(POLICY_RESTART_RECORD_KEY, record), + restartExtensionHost: async () => { + void vscode.window.showWarningMessage(vscode.l10n.t("VS Code needs to restart extensions in this window to apply your organization's Copilot telemetry settings. Active sessions may ask you to confirm.")).then(undefined, + error => this._logService.error(error, '[OTel] Failed to show the telemetry policy restart warning')); + await vscode.commands.executeCommand('workbench.action.restartExtensionHost'); + // Successful restart destroys this host. This one-off grace period is only + // for deciding when a still-running host should show the reload fallback. + await timeout(15_000); + }, + warnPolicyNotApplied: () => { + void this._promptReload(vscode.l10n.t("Your organization's Copilot telemetry policy could not be applied automatically. Reload the window to apply it."), true); + }, + promptReload: current => { + const endpoint = current.config.otlpEndpoint; + const endpointChanged = current.config.enabled && endpoint !== this._otelConfigResolver.activeResolution.config.otlpEndpoint; + void this._promptReload(endpointChanged + ? vscode.l10n.t("Copilot OTel endpoint will change to {0} after reload.", String(endpoint)) + : vscode.l10n.t("Copilot OTel settings changed - a reload is required for the change to take effect."), false); + }, + notifyPolicyRestarted: () => { + this._logService.info('[OTel] Extensions were restarted to apply enterprise telemetry policy.'); + void vscode.window.showInformationMessage(vscode.l10n.t("Extensions were restarted to apply your organization's Copilot telemetry policy.")).then(undefined, + error => this._logService.error(error, '[OTel] Failed to show the telemetry policy restart confirmation')); + }, + }, this._logService); + // One startup check and configuration-event checks; no polling. + const scheduler = this._register(new RunOnceScheduler(() => { + monitor.check().catch(error => this._logService.error(error, '[OTel] Failed to check for stale telemetry configuration')); + }, 500)); + this._register(vscode.workspace.onDidChangeConfiguration(e => { + if (e.affectsConfiguration(OTEL_SETTINGS_SECTION)) { + scheduler.schedule(); } - const endpointSetting = ConfigKey.Advanced.OTelOtlpEndpoint; - const endpointChanged = changedSettings.some(s => s.fullyQualifiedId === endpointSetting.fullyQualifiedId); + })); + scheduler.schedule(); + } + + private async _promptReload(message: string, warning: boolean): Promise { + try { const reloadWindowLabel = vscode.l10n.t("Reload Window"); - const message = endpointChanged - ? vscode.l10n.t("Copilot OTel endpoint will change to {0} after reload.", String(currentConfig.get(endpointSetting.fullyQualifiedId))) - : vscode.l10n.t("Copilot OTel settings changed - a reload is required for the change to take effect."); - const selection = await vscode.window.showInformationMessage( - message, - reloadWindowLabel, - ); + const selection = warning + ? await vscode.window.showWarningMessage(message, reloadWindowLabel) + : await vscode.window.showInformationMessage(message, reloadWindowLabel); if (selection === reloadWindowLabel) { await vscode.commands.executeCommand('workbench.action.reloadWindow'); } - })); + } catch (error) { + this._logService.error(error, '[OTel] Failed to prompt for a window reload'); + } } /** diff --git a/extensions/copilot/src/platform/otel/common/otelConfig.ts b/extensions/copilot/src/platform/otel/common/otelConfig.ts index cc1a62c108618a..c71675650a8ad6 100644 --- a/extensions/copilot/src/platform/otel/common/otelConfig.ts +++ b/extensions/copilot/src/platform/otel/common/otelConfig.ts @@ -70,7 +70,7 @@ function parseResourceAttributes(raw: string | undefined): Record; +const settingKeys = Object.keys(OTEL_SETTING_DEFAULTS) as OTelSettingKey[]; +const policySettingKeys = [ + 'enabled', 'exporterType', 'protocol', 'otlpEndpoint', 'captureContent', + 'serviceName', 'resourceAttributes', 'headers', 'outfile', +] as const; + +export interface IOTelSettingsReader { + get(key: string): T | undefined; + inspect(key: string): { defaultValue?: T } | undefined; +} + +export interface IResolvedOTelConfig { + readonly config: OTelConfig; + readonly defaultValues: OTelDefaultValues; +} + +export const IOTelConfigResolver = createServiceIdentifier('IOTelConfigResolver'); + +export interface IOTelConfigResolver { + readonly _serviceBrand: undefined; + readonly activeResolution: IResolvedOTelConfig; + resolve(): IResolvedOTelConfig; +} + +/** Excludes later process.env mutations used to configure the embedded runtime. */ +export function snapshotOTelEnv(env: Record): Record { + const keys = [ + 'COPILOT_OTEL_ENABLED', 'COPILOT_OTEL_ENDPOINT', 'COPILOT_OTEL_PROTOCOL', + 'COPILOT_OTEL_FILE_EXPORTER_PATH', 'COPILOT_OTEL_CAPTURE_CONTENT', + 'COPILOT_OTEL_MAX_ATTRIBUTE_SIZE_CHARS', 'COPILOT_OTEL_LOG_LEVEL', + 'COPILOT_OTEL_HTTP_INSTRUMENTATION', 'OTEL_EXPORTER_OTLP_ENDPOINT', + 'OTEL_EXPORTER_OTLP_PROTOCOL', 'OTEL_EXPORTER_OTLP_HEADERS', + 'OTEL_SERVICE_NAME', 'OTEL_RESOURCE_ATTRIBUTES', + ]; + return Object.fromEntries(keys.filter(key => env[key] !== undefined).map(key => [key, env[key]])); +} + +export function resolveOTelConfigFromSettings( + settings: IOTelSettingsReader, + env: Record, + extensionVersion: string, + sessionId: string, +): IResolvedOTelConfig { + const defaultValues: OTelDefaultValues = { ...OTEL_SETTING_DEFAULTS }; + for (const key of settingKeys) { + defaultValues[key] = deepClone(settings.inspect(key)?.defaultValue); + } + // For these application-scoped settings, inspect().defaultValue contains policy + // when present, otherwise the schema default. Once policy is recognizable, use + // that entire OTel block instead of filling missing fields from personal settings. + const hasEnterpriseSettings = policySettingKeys.some(key => + defaultValues[key] !== undefined && !equals(defaultValues[key], OTEL_SETTING_DEFAULTS[key])); + const read = (key: OTelSettingKey): T | undefined => hasEnterpriseSettings + ? (defaultValues[key] ?? OTEL_SETTING_DEFAULTS[key]) as T + : settings.get(key); + + const config = resolveOTelConfig({ + env, + settingEnabled: read('enabled'), + settingExporterType: read('exporterType'), + settingOtlpEndpoint: read('otlpEndpoint'), + settingCaptureContent: read('captureContent'), + settingMaxAttributeSizeChars: read('maxAttributeSizeChars'), + settingOutfile: read('outfile') || undefined, + settingDbSpanExporter: read('dbSpanExporter.enabled'), + settingProtocol: read('protocol') || undefined, + settingServiceName: read('serviceName') || undefined, + settingResourceAttributes: read>('resourceAttributes'), + settingHeaders: read>('headers'), + extensionVersion, + sessionId, + }); + return { config, defaultValues }; +} + +export const enum OTelConfigDrift { + None = 'none', + User = 'user', + Policy = 'policy', + Withdrawal = 'withdrawal', +} + +export function classifyOTelConfigDrift(active: IResolvedOTelConfig, current: IResolvedOTelConfig): OTelConfigDrift { + if (equals(active.config, current.config)) { + return OTelConfigDrift.None; + } + // Applied policy is folded into defaultValue. These policy-backed settings are + // application-scoped, so extensions cannot contribute default overrides for them. + const changedDefaults = policySettingKeys.filter(key => !equals(active.defaultValues[key], current.defaultValues[key])); + if (changedDefaults.length === 0) { + return OTelConfigDrift.User; + } + // Only returning to schema defaults is recognizable as withdrawal without policy provenance. + return changedDefaults.every(key => current.defaultValues[key] === undefined || equals(current.defaultValues[key], OTEL_SETTING_DEFAULTS[key])) + ? OTelConfigDrift.Withdrawal + : OTelConfigDrift.Policy; +} + +/** Field names only: values may contain credentials. */ +export function describeOTelConfigDrift(active: OTelConfig, current: OTelConfig): string[] { + const keys = Object.keys(active) as (keyof OTelConfig)[]; + return keys.filter(key => !equals(active[key], current[key])).sort(); +} diff --git a/extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts b/extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts new file mode 100644 index 00000000000000..008c23625df663 --- /dev/null +++ b/extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts @@ -0,0 +1,220 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + * Licensed under the MIT License. See License.txt in the project root for license information. + *--------------------------------------------------------------------------------------------*/ + +import { readFileSync } from 'node:fs'; +import { describe, expect, it } from 'vitest'; +import { classifyOTelConfigDrift, describeOTelConfigDrift, OTEL_SETTING_DEFAULTS, OTelConfigDrift, resolveOTelConfigFromSettings, snapshotOTelEnv } from '../otelConfigResolution'; +import { TestOTelSettings } from './otelTestSettings'; + +const manifest = JSON.parse(readFileSync(new URL('../../../../../package.json', import.meta.url), 'utf8')); +const sections = manifest.contributes.configuration; +const properties = Object.assign({}, ...(Array.isArray(sections) ? sections : [sections]).map(section => section.properties)); +const prefix = 'github.copilot.chat.otel.'; + +function resolve(settings: TestOTelSettings, env: Record = {}) { + return resolveOTelConfigFromSettings(settings, env, '1.0.0', 'session'); +} + +describe('OTel config resolution', () => { + it('snapshots every OTel schema default, including settings outside the old six-key watcher', () => { + const otelProperties = Object.entries(properties).filter(([key]) => key.startsWith(prefix)); + const defaults = Object.fromEntries(otelProperties + .map(([key, schema]) => [key.slice(prefix.length), (schema as { default: unknown }).default])); + expect(otelProperties.every(([, schema]) => (schema as { scope?: string }).scope === 'application')).toBe(true); + expect(OTEL_SETTING_DEFAULTS).toEqual(defaults); + expect(resolve(new TestOTelSettings()).defaultValues).toEqual(defaults); + }); + + it('preserves existing effective-setting resolution and env precedence', () => { + const settings = new TestOTelSettings(); + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example', headers: { managed: '1' } }; + expect(resolve(settings, { COPILOT_OTEL_ENABLED: 'false' }).config.enabled).toBe(false); + expect(resolve(settings, { + OTEL_EXPORTER_OTLP_ENDPOINT: 'https://env.example', + OTEL_EXPORTER_OTLP_HEADERS: 'env=2', + }).config).toMatchObject({ + enabledVia: 'setting', + otlpEndpoint: 'https://env.example/', + headers: { managed: '1', env: '2' }, + }); + }); + + it('replaces the whole personal OTel settings block with enterprise values and defaults', () => { + const settings = new TestOTelSettings(); + settings.user = { + enabled: false, + exporterType: 'file', + protocol: 'grpc', + otlpEndpoint: 'https://personal.example', + captureContent: true, + serviceName: 'personal-service', + resourceAttributes: { personal: 'attribute' }, + headers: { personal: 'header' }, + maxAttributeSizeChars: 10, + outfile: '/tmp/personal-otel.jsonl', + 'dbSpanExporter.enabled': true, + }; + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example' }; + expect(resolve(settings).config).toMatchObject({ + enabled: true, + exporterType: 'otlp-http', + otlpProtocol: 'http/json', + otlpEndpoint: 'https://managed.example/', + captureContent: false, + serviceName: 'copilot-chat', + resourceAttributes: {}, + headers: {}, + maxAttributeSizeChars: 0, + fileExporterPath: undefined, + dbSpanExporter: false, + }); + }); + + it('uses enterprise maps verbatim without personal header or attribute keys', () => { + const settings = new TestOTelSettings(); + settings.user = { headers: { personal: 'header' }, resourceAttributes: { personal: 'attribute' } }; + settings.policy = { + enabled: true, + otlpEndpoint: 'https://managed.example', + headers: { organization: 'header' }, + resourceAttributes: { organization: 'attribute' }, + }; + expect(resolve(settings).config).toMatchObject({ + headers: { organization: 'header' }, + resourceAttributes: { organization: 'attribute' }, + }); + }); + + it('keeps personal OTel settings when no enterprise block is recognized', () => { + const settings = new TestOTelSettings(); + settings.user = { + enabled: true, + otlpEndpoint: 'https://personal.example', + captureContent: true, + serviceName: 'personal-service', + headers: { personal: 'header' }, + resourceAttributes: { personal: 'attribute' }, + maxAttributeSizeChars: 10, + 'dbSpanExporter.enabled': true, + }; + expect(resolve(settings).config).toMatchObject({ + enabled: true, + otlpEndpoint: 'https://personal.example/', + captureContent: true, + serviceName: 'personal-service', + headers: { personal: 'header' }, + resourceAttributes: { personal: 'attribute' }, + maxAttributeSizeChars: 10, + dbSpanExporter: true, + }); + }); + + it('ignores subsequent personal edits while enterprise OTel applies', () => { + const settings = new TestOTelSettings(); + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example' }; + const active = resolve(settings); + settings.user = { + exporterType: 'file', + outfile: '/tmp/personal-otel.jsonl', + headers: { personal: 'header' }, + resourceAttributes: { personal: 'attribute' }, + 'dbSpanExporter.enabled': true, + }; + expect(resolve(settings).config).toEqual(active.config); + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.None); + }); + + it('returns to personal preferences after the enterprise block is withdrawn', () => { + const settings = new TestOTelSettings(); + settings.user = { enabled: true, otlpEndpoint: 'https://personal.example', headers: { personal: 'header' } }; + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example' }; + const active = resolve(settings); + settings.policy = {}; + expect(resolve(settings).config).toMatchObject({ + otlpEndpoint: 'https://personal.example/', + headers: { personal: 'header' }, + }); + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.Withdrawal); + }); + + it('detects policy in the activation blind spot', () => { + const settings = new TestOTelSettings(); + const active = resolve(settings); + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example' }; + expect(active.config.enabled).toBe(false); + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.Policy); + }); + + it.each([ + ['enabled', false], + ['exporterType', 'console'], + ['protocol', 'http/protobuf'], + ['otlpEndpoint', 'https://changed.example'], + ['captureContent', true], + ['serviceName', 'changed-service'], + ['resourceAttributes', { team: 'test' }], + ['headers', { authorization: 'secret-must-not-be-logged' }], + ['maxAttributeSizeChars', 50], + ['outfile', '/tmp/test-otel.jsonl'], + ['dbSpanExporter.enabled', true], + ] as const)('detects user and default drift for %s', (key, value) => { + const settings = new TestOTelSettings(); + settings.user = { enabled: true }; + const active = resolve(settings); + settings.user[key] = value; + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.User); + settings.policy[key] = value; + // False equals the enabled default; non-policy-backed defaults are not policy signals. + const hasPolicy = properties[`${prefix}${key}`].policyReference !== undefined; + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(key === 'enabled' || !hasPolicy ? OTelConfigDrift.User : OTelConfigDrift.Policy); + }); + + it('recognizes complete and partial withdrawal, but not a replacement policy', () => { + const settings = new TestOTelSettings(); + settings.policy = { enabled: true, serviceName: 'managed' }; + const active = resolve(settings); + settings.policy = { enabled: true }; + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.Withdrawal); + settings.policy = {}; + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.Withdrawal); + settings.policy = { enabled: true, otlpEndpoint: 'https://new.example' }; + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.Policy); + }); + + it('deep-snapshots defaults and describes changed fields without their values', () => { + const settings = new TestOTelSettings(); + const headers = { authorization: 'old' }; + settings.policy = { enabled: true, headers }; + const active = resolve(settings); + headers.authorization = 'secret'; + const current = resolve(settings); + expect(active.defaultValues.headers).toEqual({ authorization: 'old' }); + expect(classifyOTelConfigDrift(active, current)).toBe(OTelConfigDrift.Policy); + expect(describeOTelConfigDrift(active.config, current.config)).toEqual(['headers']); + }); + + it('does nothing when no effective configuration changed', () => { + const settings = new TestOTelSettings(); + settings.policy = { enabled: true }; + const active = resolve(settings); + settings.user.enabled = false; + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.None); + settings.policy.headers = {}; + expect(classifyOTelConfigDrift(active, resolve(settings))).toBe(OTelConfigDrift.None); + }); + + it('isolates resolution from later process.env rewrites', () => { + const processEnv: Record = { PATH: 'bin', OTEL_SERVICE_NAME: 'original' }; + const env = snapshotOTelEnv(processEnv); + const settings = new TestOTelSettings(); + settings.user = { 'dbSpanExporter.enabled': true }; + const active = resolve(settings, env); + processEnv.COPILOT_OTEL_FILE_EXPORTER_PATH = 'null-device'; + processEnv.OTEL_SERVICE_NAME = 'rewritten'; + expect(env).toEqual({ OTEL_SERVICE_NAME: 'original' }); + expect(classifyOTelConfigDrift(active, resolve(settings, env))).toBe(OTelConfigDrift.None); + expect(classifyOTelConfigDrift(active, resolve(settings, processEnv))).toBe(OTelConfigDrift.User); + }); +}); diff --git a/extensions/copilot/src/platform/otel/common/test/otelTestSettings.ts b/extensions/copilot/src/platform/otel/common/test/otelTestSettings.ts new file mode 100644 index 00000000000000..d5be1d4e44598d --- /dev/null +++ b/extensions/copilot/src/platform/otel/common/test/otelTestSettings.ts @@ -0,0 +1,21 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + * Licensed under the MIT License. See License.txt in the project root for license information. + *--------------------------------------------------------------------------------------------*/ + +import { IOTelSettingsReader, OTEL_SETTING_DEFAULTS } from '../otelConfigResolution'; + +/** Mirrors the stable extension API: policy is folded into get() and inspect().defaultValue. */ +export class TestOTelSettings implements IOTelSettingsReader { + user: Record = {}; + policy: Record = {}; + private readonly _defaults: Record = OTEL_SETTING_DEFAULTS; + + get(key: string): T | undefined { + return (this.policy[key] ?? this.user[key] ?? this._defaults[key]) as T | undefined; + } + + inspect(key: string): { defaultValue?: T } { + return { defaultValue: (this.policy[key] ?? this._defaults[key]) as T | undefined }; + } +} From 5aeae1f614e4a200eb909e03e6639d84d5efcabd Mon Sep 17 00:00:00 2001 From: "Ross A. Wollman" Date: Thu, 17 Sep 2026 23:03:34 -0700 Subject: [PATCH 2/6] Restrict OTel recovery to newly arriving enterprise policy Track recognized managed settings independently of export enablement so later updates to a disabled startup policy only offer reload. Add regression coverage and align OTel Settings descriptions with the retained environment-variable precedence. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../docs/monitoring/agent_monitoring.md | 2 ++ extensions/copilot/package.json | 18 +++++++-------- .../otel/common/otelStaleConfigMonitor.ts | 2 +- .../test/otelStaleConfigMonitor.spec.ts | 23 +++++++++++++++++++ .../otel/common/otelConfigResolution.ts | 4 +++- .../common/test/otelConfigResolution.spec.ts | 18 +++++++++++++++ 6 files changed, 56 insertions(+), 11 deletions(-) diff --git a/extensions/copilot/docs/monitoring/agent_monitoring.md b/extensions/copilot/docs/monitoring/agent_monitoring.md index fbb323fea14d37..4a4d9f70a8df4b 100644 --- a/extensions/copilot/docs/monitoring/agent_monitoring.md +++ b/extensions/copilot/docs/monitoring/agent_monitoring.md @@ -114,6 +114,8 @@ a warning offers **Reload Window** instead. User changes and policy withdrawal r opt-in reloads. Exporter behavior and environment-variable precedence are unchanged. It uses changes to the application-scoped, policy-backed configuration defaults as a recovery signal, without a new API. Normal personal settings changes do not change those defaults. +If a recognizable enterprise OTel block was already present at initialization, later changes +only offer a reload, including enabling a previously disabled managed configuration. Automatic recovery additionally requires policy-enabled OTLP export targeting the collector in those defaults. Disabled and DB-only pipelines, unrelated partial policies, and configurations still redirected by environment variables to a different collector or file do not qualify. diff --git a/extensions/copilot/package.json b/extensions/copilot/package.json index a7cf8495e9b389..a136afbad15055 100644 --- a/extensions/copilot/package.json +++ b/extensions/copilot/package.json @@ -5308,7 +5308,7 @@ "policyReference": { "name": "CopilotOtelEnabled" }, - "markdownDescription": "Enable OpenTelemetry trace/metric/log emission for Copilot Chat operations. Precedence: enterprise policy > env var `COPILOT_OTEL_ENABLED` > user setting. Requires window reload.", + "markdownDescription": "Enable OpenTelemetry trace/metric/log emission for Copilot Chat operations. Recognized enterprise OTel settings replace personal OTel settings, but env var `COPILOT_OTEL_ENABLED` can still override enablement. Requires window reload.", "tags": [ "advanced" ] @@ -5326,7 +5326,7 @@ "policyReference": { "name": "CopilotOtelProtocol" }, - "markdownDescription": "OTel exporter type for Copilot Chat telemetry. Configurable in user settings or managed by enterprise policy (policy takes precedence). Requires window reload.", + "markdownDescription": "OTel exporter type for Copilot Chat telemetry. Enterprise-managed OTel settings replace personal OTel settings. A configured output file, including env var `COPILOT_OTEL_FILE_EXPORTER_PATH`, can still select file export. Requires window reload.", "tags": [ "advanced" ] @@ -5344,7 +5344,7 @@ "policyReference": { "name": "CopilotOtelOtlpProtocol" }, - "markdownDescription": "OTLP wire protocol for Copilot Chat OTel data, mirroring `OTEL_EXPORTER_OTLP_PROTOCOL`. `http/protobuf` selects the protobuf-over-HTTP exporter; the default (empty) uses `http/json`. Precedence: enterprise policy > env var > user setting. Requires window reload.", + "markdownDescription": "OTLP wire protocol for Copilot Chat OTel data, mirroring `OTEL_EXPORTER_OTLP_PROTOCOL`. `http/protobuf` selects the protobuf-over-HTTP exporter; the default (empty) uses `http/json`. Env vars `OTEL_EXPORTER_OTLP_PROTOCOL`, then `COPILOT_OTEL_PROTOCOL`, take precedence over enterprise-managed or personal settings. Requires window reload.", "tags": [ "advanced" ] @@ -5356,7 +5356,7 @@ "policyReference": { "name": "CopilotOtelEndpoint" }, - "markdownDescription": "OTLP collector endpoint URL for Copilot Chat OTel data. Precedence: enterprise policy > env var `OTEL_EXPORTER_OTLP_ENDPOINT` > user setting. Requires window reload.", + "markdownDescription": "OTLP collector endpoint URL for Copilot Chat OTel data. Env vars `COPILOT_OTEL_ENDPOINT`, then `OTEL_EXPORTER_OTLP_ENDPOINT`, take precedence over enterprise-managed or personal settings. Requires window reload.", "tags": [ "advanced" ] @@ -5368,7 +5368,7 @@ "policyReference": { "name": "CopilotOtelCaptureContent" }, - "markdownDescription": "Capture input/output messages, system instructions, and tool definitions in OTel telemetry. **Contains potentially sensitive data.** Precedence: enterprise policy > env var `COPILOT_OTEL_CAPTURE_CONTENT` > user setting. Requires window reload.", + "markdownDescription": "Capture input/output messages, system instructions, and tool definitions in OTel telemetry. **Contains potentially sensitive data.** Env var `COPILOT_OTEL_CAPTURE_CONTENT` takes precedence over enterprise-managed or personal settings. Requires window reload.", "tags": [ "advanced" ] @@ -5380,7 +5380,7 @@ "policyReference": { "name": "CopilotOtelServiceName" }, - "markdownDescription": "OTel `service.name` resource attribute for Copilot Chat OTel data. Configurable in user settings only. Env var `OTEL_SERVICE_NAME` takes precedence over the setting; enterprise policy takes precedence over both. Requires window reload.", + "markdownDescription": "OTel `service.name` resource attribute for Copilot Chat OTel data. Env var `OTEL_SERVICE_NAME` takes precedence over enterprise-managed or personal settings. Requires window reload.", "tags": [ "advanced" ] @@ -5395,7 +5395,7 @@ "policyReference": { "name": "CopilotOtelResourceAttributes" }, - "markdownDescription": "Additional OTel resource attributes for Copilot Chat OTel data, as a `{ \"key\": \"value\" }` map. Configurable in user settings only. Merged per-key with `OTEL_RESOURCE_ATTRIBUTES` env (env wins over the setting); enterprise policy wins over both. Requires window reload.", + "markdownDescription": "Additional OTel resource attributes for Copilot Chat OTel data, as a `{ \"key\": \"value\" }` map. When enterprise OTel settings apply, personal OTel settings are ignored. Merged per-key with env var `OTEL_RESOURCE_ATTRIBUTES` (env values take precedence, including over enterprise-managed values). Requires window reload.", "tags": [ "advanced" ] @@ -5410,7 +5410,7 @@ "policyReference": { "name": "CopilotOtelHeaders" }, - "markdownDescription": "Extra OTLP exporter headers (e.g. auth tokens) for Copilot Chat OTel data, as a `{ \"key\": \"value\" }` map. Applied directly to the OTLP exporter, not via environment variables. Configurable in user settings only. Merged per-key with `OTEL_EXPORTER_OTLP_HEADERS` env (env wins over the setting); enterprise policy wins over both. **Contains potentially sensitive credentials.** Requires window reload.", + "markdownDescription": "Extra OTLP exporter headers (e.g. auth tokens) for Copilot Chat OTel data, as a `{ \"key\": \"value\" }` map. Applied directly to the OTLP exporter, not via environment variables. When enterprise OTel settings apply, personal OTel settings are ignored. Merged per-key with env var `OTEL_EXPORTER_OTLP_HEADERS` (env values take precedence, including over enterprise-managed values). **Contains potentially sensitive credentials.** Requires window reload.", "tags": [ "advanced" ] @@ -5432,7 +5432,7 @@ "policyReference": { "name": "CopilotOtelOutfile" }, - "markdownDescription": "File path for file-based OTel exporter output (JSON-lines). When set, overrides exporter type to `file`. Configurable in user settings or managed by enterprise policy (policy takes precedence). Requires window reload.", + "markdownDescription": "File path for file-based OTel exporter output (JSON-lines). When set, overrides exporter type to `file`. Enterprise-managed OTel settings replace personal OTel settings, but env var `COPILOT_OTEL_FILE_EXPORTER_PATH` can still override the output path. Requires window reload.", "tags": [ "advanced" ] diff --git a/extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts b/extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts index 1e9240d6a93505..14e7cf11b096d2 100644 --- a/extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts +++ b/extensions/copilot/src/extension/otel/common/otelStaleConfigMonitor.ts @@ -72,7 +72,7 @@ export class OTelStaleConfigMonitor { this._host.promptReload(current); return drift; } - if (active.config.enabledExplicitly || !isPolicyEnabledOtlp(current)) { + if (active.hasEnterpriseSettings || active.config.enabledExplicitly || !isPolicyEnabledOtlp(current)) { this._handledFingerprint = fingerprint; if (!this._policyNoticeShown) { this._policyNoticeShown = true; diff --git a/extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts b/extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts index fc5e2ce1199610..b2d27b5b9cab5f 100644 --- a/extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts +++ b/extensions/copilot/src/extension/otel/common/test/otelStaleConfigMonitor.spec.ts @@ -242,6 +242,29 @@ describe('OTelStaleConfigMonitor', () => { expect(host.prompts).toBe(1); }); + it.each([ + { otlpEndpoint: 'https://managed.example' }, + { serviceName: 'managed-service' }, + { headers: { managed: '1' } }, + ])('only prompts when a disabled managed block present at startup is later enabled: %j', async policy => { + settings.policy = { ...policy, enabled: false }; + const resolver = new TestResolver(settings); + const monitor = new OTelStaleConfigMonitor(resolver, host, log); + host.restartError = new Error('Unexpected automatic restart'); + expect(resolver.activeResolution.config.enabled).toBe(false); + settings.policy = { ...policy, enabled: true }; + expect(await monitor.check()).toBe(OTelConfigDrift.Policy); + await monitor.check(); + settings.policy = { ...settings.policy, captureContent: true }; + await monitor.check(); + expect({ + restarts: host.restarts, + prompts: host.prompts, + warnings: host.warnings, + restartRecord: host.record, + }).toEqual({ restarts: 0, prompts: 1, warnings: 0, restartRecord: undefined }); + }); + it('only prompts when policy is withdrawn', async () => { settings.policy = managedPolicy; const monitor = newHost(); diff --git a/extensions/copilot/src/platform/otel/common/otelConfigResolution.ts b/extensions/copilot/src/platform/otel/common/otelConfigResolution.ts index 9f2c37af19ffaf..be4ad547bc3a76 100644 --- a/extensions/copilot/src/platform/otel/common/otelConfigResolution.ts +++ b/extensions/copilot/src/platform/otel/common/otelConfigResolution.ts @@ -38,6 +38,8 @@ export interface IOTelSettingsReader { export interface IResolvedOTelConfig { readonly config: OTelConfig; readonly defaultValues: OTelDefaultValues; + /** Recognizable policy-backed defaults, independent of whether export is enabled. */ + readonly hasEnterpriseSettings: boolean; } export const IOTelConfigResolver = createServiceIdentifier('IOTelConfigResolver'); @@ -96,7 +98,7 @@ export function resolveOTelConfigFromSettings( extensionVersion, sessionId, }); - return { config, defaultValues }; + return { config, defaultValues, hasEnterpriseSettings }; } export const enum OTelConfigDrift { diff --git a/extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts b/extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts index 008c23625df663..ef63a6539f4799 100644 --- a/extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts +++ b/extensions/copilot/src/platform/otel/common/test/otelConfigResolution.spec.ts @@ -25,6 +25,23 @@ describe('OTel config resolution', () => { expect(otelProperties.every(([, schema]) => (schema as { scope?: string }).scope === 'application')).toBe(true); expect(OTEL_SETTING_DEFAULTS).toEqual(defaults); expect(resolve(new TestOTelSettings()).defaultValues).toEqual(defaults); + expect(resolve(new TestOTelSettings()).hasEnterpriseSettings).toBe(false); + }); + + it('recognizes an enterprise block independently of export enablement', () => { + const settings = new TestOTelSettings(); + settings.policy = { enabled: false, serviceName: 'managed-service' }; + const active = resolve(settings); + expect(active).toMatchObject({ hasEnterpriseSettings: true, config: { enabled: false } }); + settings.policy = {}; + expect(resolve(settings).hasEnterpriseSettings).toBe(false); + expect(active.hasEnterpriseSettings).toBe(true); + }); + + it('does not claim policy provenance for a block consisting entirely of schema defaults', () => { + const settings = new TestOTelSettings(); + settings.policy = { enabled: false, otlpEndpoint: OTEL_SETTING_DEFAULTS.otlpEndpoint }; + expect(resolve(settings).hasEnterpriseSettings).toBe(false); }); it('preserves existing effective-setting resolution and env precedence', () => { @@ -109,6 +126,7 @@ describe('OTel config resolution', () => { maxAttributeSizeChars: 10, dbSpanExporter: true, }); + expect(resolve(settings).hasEnterpriseSettings).toBe(false); }); it('ignores subsequent personal edits while enterprise OTel applies', () => { From 9643b45706f8a0c58d3e19318a4ef0dc1928a974 Mon Sep 17 00:00:00 2001 From: "Ross A. Wollman" Date: Thu, 17 Sep 2026 23:16:13 -0700 Subject: [PATCH 3/6] Approve reviewed PR state Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> From 5dce72ab2da0eebd63466a21f8259044c37ff0ac Mon Sep 17 00:00:00 2001 From: "Ross A. Wollman" Date: Thu, 17 Sep 2026 23:56:12 -0700 Subject: [PATCH 4/6] Clear enterprise OTel restart notice with its extension host Use a lifecycle-bound progress notification instead of a persistent warning so successful restart cannot leave contradictory notices. Preserve the existing restart guard and fallback grace, and cover command failure, fallback, and unmanaged settings in contribution tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../docs/monitoring/agent_monitoring.md | 6 +- .../extension/otel/vscode-node/otelContrib.ts | 11 +- .../otel/vscode-node/test/otelContrib.spec.ts | 121 ++++++++++++++++++ 3 files changed, 132 insertions(+), 6 deletions(-) create mode 100644 extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts diff --git a/extensions/copilot/docs/monitoring/agent_monitoring.md b/extensions/copilot/docs/monitoring/agent_monitoring.md index 4a4d9f70a8df4b..cdbf2291bdcce7 100644 --- a/extensions/copilot/docs/monitoring/agent_monitoring.md +++ b/extensions/copilot/docs/monitoring/agent_monitoring.md @@ -107,8 +107,10 @@ the user's maps. Other VS Code settings are unaffected. ### Activation When late enterprise OTel settings turn on external export after Copilot's telemetry service -started without it, Copilot can restart the extension hosts for that window to recover. It warns -before requesting the restart and confirms it afterward. This also interrupts other extensions +started without it, Copilot can restart the extension hosts for that window to recover. It shows +a progress notification before requesting the restart and confirms it afterward. The progress +notification clears automatically when the host restarts or the attempt ends, rather than +leaving a stale restart warning. This also interrupts other extensions in the window. If the restart is unavailable, vetoed, or fails to apply the settings, a warning offers **Reload Window** instead. User changes and policy withdrawal remain opt-in reloads. Exporter behavior and environment-variable precedence are unchanged. diff --git a/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts b/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts index 3c2a70517d5492..d470f7c306568a 100644 --- a/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts +++ b/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts @@ -122,14 +122,17 @@ export class OTelContrib extends Disposable implements IExtensionContribution { const monitor = new OTelStaleConfigMonitor(this._otelConfigResolver, { getRestartRecord: () => state.get(POLICY_RESTART_RECORD_KEY), setRestartRecord: async record => state.update(POLICY_RESTART_RECORD_KEY, record), - restartExtensionHost: async () => { - void vscode.window.showWarningMessage(vscode.l10n.t("VS Code needs to restart extensions in this window to apply your organization's Copilot telemetry settings. Active sessions may ask you to confirm.")).then(undefined, - error => this._logService.error(error, '[OTel] Failed to show the telemetry policy restart warning')); + // Unlike ordinary messages, progress notifications close when their host is disposed. + restartExtensionHost: async () => vscode.window.withProgress({ + location: vscode.ProgressLocation.Notification, + title: vscode.l10n.t("Restarting extensions in this window to apply your organization's Copilot telemetry settings. Active sessions may ask you to confirm."), + cancellable: false, + }, async () => { await vscode.commands.executeCommand('workbench.action.restartExtensionHost'); // Successful restart destroys this host. This one-off grace period is only // for deciding when a still-running host should show the reload fallback. await timeout(15_000); - }, + }), warnPolicyNotApplied: () => { void this._promptReload(vscode.l10n.t("Your organization's Copilot telemetry policy could not be applied automatically. Reload the window to apply it."), true); }, diff --git a/extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts b/extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts new file mode 100644 index 00000000000000..8cf17fe16645f5 --- /dev/null +++ b/extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts @@ -0,0 +1,121 @@ +/*--------------------------------------------------------------------------------------------- + * Copyright (c) Microsoft Corporation. All rights reserved. + * Licensed under the MIT License. See License.txt in the project root for license information. + *--------------------------------------------------------------------------------------------*/ + +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { ProgressLocation, type GlobalEnvironmentVariableCollection, type ProgressOptions } from 'vscode'; +import { IVSCodeExtensionContext } from '../../../../platform/extContext/common/extensionContext'; +import { NoopOTelService } from '../../../../platform/otel/common/noopOtelService'; +import { IOTelConfigResolver, resolveOTelConfigFromSettings } from '../../../../platform/otel/common/otelConfigResolution'; +import { TestOTelSettings } from '../../../../platform/otel/common/test/otelTestSettings'; +import { OTelSqliteStore } from '../../../../platform/otel/node/sqlite/otelSqliteStore'; +import { NullTelemetryService } from '../../../../platform/telemetry/common/nullTelemetryService'; +import { MockExtensionContext } from '../../../../platform/test/node/extensionContext'; +import { TestLogService } from '../../../../platform/testing/common/testLogService'; +import { mock } from '../../../../util/common/test/simpleMock'; +import { OTelContrib } from '../otelContrib'; + +const ui = vi.hoisted(() => ({ + withProgress: vi.fn(), + showWarningMessage: vi.fn(), + showInformationMessage: vi.fn(), + executeCommand: vi.fn(), +})); + +vi.mock('vscode', async importOriginal => ({ + ...await importOriginal(), + ProgressLocation: { Notification: 15 }, + commands: { + registerCommand: () => ({ dispose() { } }), + executeCommand: ui.executeCommand, + }, + workspace: { onDidChangeConfiguration: () => ({ dispose() { } }) }, + window: { + withProgress: ui.withProgress, + showWarningMessage: ui.showWarningMessage, + showInformationMessage: ui.showInformationMessage, + }, +})); + +class TestExtensionContext extends mock() { + override readonly workspaceState = new MockExtensionContext().workspaceState; + override readonly environmentVariableCollection = new class extends mock() { + override delete(): void { } + }(); +} + +describe('OTelContrib restart notification', () => { + let settings: TestOTelSettings; + let contribution: OTelContrib; + let events: string[]; + + beforeEach(() => { + vi.useFakeTimers(); + vi.clearAllMocks(); + events = []; + ui.withProgress.mockImplementation(async (_options: ProgressOptions, task: () => Promise) => { + events.push('progress opened'); + try { + return await task(); + } finally { + events.push('progress completed'); + } + }); + ui.showWarningMessage.mockImplementation(async () => { events.push('reload warning'); }); + ui.showInformationMessage.mockResolvedValue(undefined); + ui.executeCommand.mockImplementation(async (command: string) => { + if (command === 'workbench.action.restartExtensionHost') { + events.push('restart requested'); + } + }); + settings = new TestOTelSettings(); + const resolve = () => resolveOTelConfigFromSettings(settings, {}, '1.0.0', 'session'); + const resolver: IOTelConfigResolver = { _serviceBrand: undefined, activeResolution: resolve(), resolve }; + contribution = new OTelContrib( + new NoopOTelService(resolver.activeResolution.config), + new OTelSqliteStore('/unused-otel-test.db'), + new TestLogService(), + new NullTelemetryService(), + new TestExtensionContext(), + resolver, + ); + }); + + afterEach(async () => { + contribution.dispose(); + await vi.runAllTimersAsync(); + vi.useRealTimers(); + }); + + it('shows lifecycle-bound progress instead of a persistent warning and ends it before fallback', async () => { + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example' }; + await vi.advanceTimersByTimeAsync(500); + expect(ui.withProgress).toHaveBeenCalledWith(expect.objectContaining({ + location: ProgressLocation.Notification, + title: expect.stringContaining('Restarting extensions'), + cancellable: false, + }), expect.any(Function)); + expect(events).toEqual(['progress opened', 'restart requested']); + expect(ui.showWarningMessage).not.toHaveBeenCalled(); + await vi.advanceTimersByTimeAsync(15_000); + expect(events).toEqual(['progress opened', 'restart requested', 'progress completed', 'reload warning']); + expect(ui.showWarningMessage).toHaveBeenCalledWith(expect.stringContaining('could not be applied automatically'), 'Reload Window'); + }); + + it('ends progress when the restart command fails, then offers a manual reload', async () => { + ui.executeCommand.mockRejectedValue(new Error('Restart unavailable')); + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example' }; + await vi.advanceTimersByTimeAsync(500); + expect(events).toEqual(['progress opened', 'progress completed', 'reload warning']); + expect(ui.showWarningMessage).toHaveBeenCalledTimes(1); + }); + + it('does not show automatic restart progress for personal settings changes', async () => { + settings.user = { enabled: true, otlpEndpoint: 'https://personal.example' }; + await vi.advanceTimersByTimeAsync(500); + expect(ui.withProgress).not.toHaveBeenCalled(); + expect(ui.showWarningMessage).not.toHaveBeenCalled(); + expect(ui.showInformationMessage).toHaveBeenCalledWith(expect.stringContaining('after reload'), 'Reload Window'); + }); +}); From 9ff9a507646cb73861c2ed9b5adcdb9988c72852 Mon Sep 17 00:00:00 2001 From: "Ross A. Wollman" Date: Fri, 18 Sep 2026 00:12:54 -0700 Subject: [PATCH 5/6] Log successful enterprise OTel recovery without another toast Remove the redundant success notification while retaining the persisted restart acknowledgement, output log, monitoring indicator, and manual reload warnings. Add regression coverage for silent success and update the monitoring guide. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .../docs/monitoring/agent_monitoring.md | 8 +-- .../extension/otel/vscode-node/otelContrib.ts | 2 - .../otel/vscode-node/test/otelContrib.spec.ts | 54 +++++++++++++++---- 3 files changed, 48 insertions(+), 16 deletions(-) diff --git a/extensions/copilot/docs/monitoring/agent_monitoring.md b/extensions/copilot/docs/monitoring/agent_monitoring.md index cdbf2291bdcce7..01ebf68235a16a 100644 --- a/extensions/copilot/docs/monitoring/agent_monitoring.md +++ b/extensions/copilot/docs/monitoring/agent_monitoring.md @@ -108,10 +108,10 @@ the user's maps. Other VS Code settings are unaffected. When late enterprise OTel settings turn on external export after Copilot's telemetry service started without it, Copilot can restart the extension hosts for that window to recover. It shows -a progress notification before requesting the restart and confirms it afterward. The progress -notification clears automatically when the host restarts or the attempt ends, rather than -leaving a stale restart warning. This also interrupts other extensions -in the window. If the restart is unavailable, vetoed, or fails to apply the settings, +a progress notification before requesting the restart; that notice clears automatically when +the host restarts or the attempt ends. Successful recovery is logged without another toast. +Restarting also interrupts other extensions in the window. If the restart is unavailable, +vetoed, or fails to apply the settings, a warning offers **Reload Window** instead. User changes and policy withdrawal remain opt-in reloads. Exporter behavior and environment-variable precedence are unchanged. It uses changes to the application-scoped, policy-backed configuration defaults as a recovery diff --git a/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts b/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts index d470f7c306568a..6a3ccc725339b6 100644 --- a/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts +++ b/extensions/copilot/src/extension/otel/vscode-node/otelContrib.ts @@ -145,8 +145,6 @@ export class OTelContrib extends Disposable implements IExtensionContribution { }, notifyPolicyRestarted: () => { this._logService.info('[OTel] Extensions were restarted to apply enterprise telemetry policy.'); - void vscode.window.showInformationMessage(vscode.l10n.t("Extensions were restarted to apply your organization's Copilot telemetry policy.")).then(undefined, - error => this._logService.error(error, '[OTel] Failed to show the telemetry policy restart confirmation')); }, }, this._logService); // One startup check and configuration-event checks; no polling. diff --git a/extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts b/extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts index 8cf17fe16645f5..bfce85df9d4cf6 100644 --- a/extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts +++ b/extensions/copilot/src/extension/otel/vscode-node/test/otelContrib.spec.ts @@ -35,6 +35,7 @@ vi.mock('vscode', async importOriginal => ({ withProgress: ui.withProgress, showWarningMessage: ui.showWarningMessage, showInformationMessage: ui.showInformationMessage, + createChatStatusItem: () => ({ show() { }, dispose() { } }), }, })); @@ -42,13 +43,34 @@ class TestExtensionContext extends mock() { override readonly workspaceState = new MockExtensionContext().workspaceState; override readonly environmentVariableCollection = new class extends mock() { override delete(): void { } + override replace(): void { } }(); } +class RecordingLogService extends TestLogService { + readonly messages: string[] = []; + override info(message: string): void { this.messages.push(message); } +} + describe('OTelContrib restart notification', () => { let settings: TestOTelSettings; let contribution: OTelContrib; let events: string[]; + let context: TestExtensionContext; + let log: RecordingLogService; + + function createContribution(): OTelContrib { + const resolve = () => resolveOTelConfigFromSettings(settings, {}, '1.0.0', 'session'); + const resolver: IOTelConfigResolver = { _serviceBrand: undefined, activeResolution: resolve(), resolve }; + return new OTelContrib( + new NoopOTelService(resolver.activeResolution.config), + new OTelSqliteStore('/unused-otel-test.db'), + log, + new NullTelemetryService(), + context, + resolver, + ); + } beforeEach(() => { vi.useFakeTimers(); @@ -70,16 +92,9 @@ describe('OTelContrib restart notification', () => { } }); settings = new TestOTelSettings(); - const resolve = () => resolveOTelConfigFromSettings(settings, {}, '1.0.0', 'session'); - const resolver: IOTelConfigResolver = { _serviceBrand: undefined, activeResolution: resolve(), resolve }; - contribution = new OTelContrib( - new NoopOTelService(resolver.activeResolution.config), - new OTelSqliteStore('/unused-otel-test.db'), - new TestLogService(), - new NullTelemetryService(), - new TestExtensionContext(), - resolver, - ); + context = new TestExtensionContext(); + log = new RecordingLogService(); + contribution = createContribution(); }); afterEach(async () => { @@ -111,6 +126,25 @@ describe('OTelContrib restart notification', () => { expect(ui.showWarningMessage).toHaveBeenCalledTimes(1); }); + it('acknowledges successful recovery and logs it once without a success toast', async () => { + settings.policy = { enabled: true, otlpEndpoint: 'https://managed.example' }; + await vi.advanceTimersByTimeAsync(500); + contribution.dispose(); + // Simulate the old host ending while its restart task is still pending. + vi.clearAllTimers(); + vi.clearAllMocks(); + contribution = createContribution(); + await vi.advanceTimersByTimeAsync(500); + expect(context.workspaceState.get('github.copilot.otel.latePolicyRestart')).toMatchObject({ acknowledged: true }); + contribution.dispose(); + contribution = createContribution(); + await vi.advanceTimersByTimeAsync(500); + expect(log.messages.filter(message => message === '[OTel] Extensions were restarted to apply enterprise telemetry policy.')).toHaveLength(1); + expect(ui.withProgress).not.toHaveBeenCalled(); + expect(ui.showInformationMessage).not.toHaveBeenCalled(); + expect(ui.showWarningMessage).not.toHaveBeenCalled(); + }); + it('does not show automatic restart progress for personal settings changes', async () => { settings.user = { enabled: true, otlpEndpoint: 'https://personal.example' }; await vi.advanceTimersByTimeAsync(500); From 6c9fe4db7dffce440a8c2c37eaacdf1781c5b720 Mon Sep 17 00:00:00 2001 From: "Ross A. Wollman" Date: Fri, 18 Sep 2026 08:06:40 -0700 Subject: [PATCH 6/6] Approve reviewed PR state Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>