screenshot can emit a model-unprocessable image (400 'Could not process image') that crashes the whole session

[duongau]

Summary

During a long-running session driving the MCP server (VS Code + GitHub Copilot, model claude-opus-4.8), a screenshot result eventually produced an image the model backend could not ingest. The Copilot API rejected the request with 400 invalid_request_error: "Could not process image", which aborted the in-flight turn and shut the whole session down. A single bad image took down the entire session, mid-task.

This was a remote/RDP session taking many region + full-monitor screenshots over a couple of hours.

What happened

• Repeated screenshot calls across a long session (UI smoke test, ~2.5 hours).
• One screenshot payload was unprocessable by the model provider.
• The turn errored out and the session terminated (session.shutdown).

Error (from the host runtime)

CAPIError: 400 {"type":"error","error":{"type":"invalid_request_error",
  "message":"Could not process image"}}
  request_id: req_011CcYG6A3S8GEuJnYyNDca3
  statusCode: 400
  model: claude-opus-4.8

(IDs included for provider-side correlation only — no payload content is sensitive.)

Suspected root cause

The screenshot tool can emit an image the vision endpoint rejects. Most likely one of:

1. Oversized payload — full-monitor base64 PNGs on a high-res display can exceed provider image limits (dimensions and/or encoded byte size). The accumulation of many large images in one context makes this more likely over a long session.
2. Corrupt/partial capture — on RDP, BitBlt screen capture intermittently returns degraded/partial frames (the tool already surfaces Windows graphics function failed: BitBlt in other cases). A partially-rendered capture can still encode to a PNG that the model then refuses.

Suggested fixes (tool side)

• Validate & normalize every screenshot before returning it: cap longest edge and encoded byte size to known-good provider limits, re-encode to a clean baseline PNG, and downscale automatically when over the limit (the tool already supports max_edge/scaling — apply a safe default ceiling).
• Detect bad captures: if a BitBlt/capture is partial or fails the integrity check, retry (or focus_window + retry) rather than returning a questionable image.
• Fail soft: return a structured tool error ({ok:false, error:"image too large / capture failed"}) instead of a payload that can hard-fail the model request and kill the session. Dropping/flagging one image is far better than ending the session.

Environment

• copilot-computer-use 0.1.0 (pipx), CCU_CONFIRM_MODE=confirm, CCU_DEFAULT_MONITOR=primary
• Windows, remote/RDP session, high-DPI (full-monitor shots report scale_applied: 0.75, image 1920 vs actual 2560)
• VS Code 1.126.0, Node v24.15.0, model claude-opus-4.8

Impact

A single unprocessable screenshot ends the entire session and any in-progress work tracking, with no graceful recovery. For long automation/QA runs this is a hard stop.