Test: Late enterprise OTel policy recovery in Copilot Local

[rwoll]

Refs: #336701, #336102

• anyOS Joaquín Ruales (@jruales)

Complexity: 3

Authors: Ross Wollman (@rwoll)

Assigned to: Joaquín Ruales (@jruales)

---

This TPI ensures that enterprise-managed VS Code instances do not drop OTel from Chat when enterprise policy is updated on disk. Copilot Chat should restart its OTel service so the new policy takes effect.

This does not affect Agent Host. Ensure Local is selected in the harness picker while testing.

Set up

• Close other VS Code windows.
• Open user settings.json and ensure you comment out any OTel related settings if you personally have them. (Most people don't.)
• Add "chat.editor.localAgent.enabled": true to settings.json since this bug is about the Local harness and not Agent Host.

Test

1. Outside of vscode, start the mock OTel Collector server locally via node ./collector.js (see script in Appendix). When VS Code triggers requests to it, you will see something like:

   POST /v1/logs: 4100 bytes (application/json)
   POST /v1/metrics: 9374 bytes (application/json)
   POST /v1/traces: 273817 bytes (application/json)
   

2. Open VS Code Editor, create a new conversation and select Local for the harness. Send hello and wait for Copilot to respond normally. You should not yet see any requests from the server logs.

3. Write the following Policy file to your OS's respective policy store:

   {
      "telemetry": {
      "enabled": true,
      "endpoint": "http://localhost:4318",
      "protocol": "http/json"
      }
   }

   • Windows: %ProgramFiles%\GitHubCopilot\managed-settings.json
   • Linux:  /etc/github-copilot/managed-settings.json
   • macOS:  /Library/Application\ Support/GitHubCopilot/managed-settings.json

4. Observe the Extension Host briefly restart in VS Code automatically.

5. Send another message to Copilot Chat (ensuring you are still on Local harness).

6. Observe the mock OTel server log logging requests like POST /v1/logs: 4100 bytes (application/json).

Cleanup

Remove the managed settings json file based on your OS.

Appendix

collector.js

 // collector.js
 require('node:http').createServer((req, res) => {
   let bytes = 0;
   req.on('data', chunk => bytes += chunk.length);
   req.on('end', () => {
     const type = req.headers['content-type'] || 'application/x-protobuf';
     console.log(`${req.method} ${req.url}: ${bytes} bytes (${type})`);
     res.writeHead(200, { 'content-type': type });
     res.end(type.includes('json') ? '{}' : undefined);
   });
 }).listen(4318, () => console.log('OTLP collector: http://localhost:4318'));

[saitanallensantiago26-beep]

Refs: #336701, #336102

• macOS

Complexity: 3

Authors: Ross Wollman (@rwoll)

Assigned to: Joaquín Ruales (@jruales)

---

This TPI ensures that enterprise-managed VS Code instances do not drop OTel from Chat when enterprise policy is updated on disk. Copilot Chat should restart its OTel service so the new policy takes effect.

This does not affect Agent Host. Ensure Local is selected in the harness picker while testing.

Set up

• Close other VS Code windows. The helper changes machine-wide policy; do not remove existing organizational policy.
• Start without managed OTel policy or OTel environment overrides. Keep github.copilot.chat.otel.enabled and github.copilot.chat.otel.dbSpanExporter.enabled off.
• Enable chat.editor.localAgent.enabled in Settings and sign in. Ensure Local, not Copilot, is selected in the harness picker.

Test

1. Send Reply with OK and wait for completion.

2. Download otel-restart-test.cjs and run in an external terminal:

   sudo node otel-restart-test.cjs

   Press Enter and return to Insiders. Policy is applied after 8 seconds.

3. Look for a brief progress notice and one automatic extension-host restart. Do not manually reload.

4. Ensure Local is still selected and send Reply with OK. Within about 10 seconds, the terminal should show POST /v1/traces with a nonzero span count. There should be no repeated restarts.

Cleanup

Wait 120 seconds or press Ctrl+C. Confirm the helper removed its test policy, then fully quit Insiders and restore test-only settings changes.