Repository navigation
Live capture for remote viewers, JPEG on macOS, cached display list - #13
Merged
Merged
Conversation
MT Code's Computer View polled `screenshot` up to 30 times a second: every
frame enumerated displays, set up a capture, encoded it, and the viewer then
compared bytes to learn nothing had moved. Measured ~100 ms a frame on both
macOS and Windows, and the macOS helper only produced PNG (~400-600 KB).
- `screenshot { display, live: true, after?, wait_ms? }` keeps a capture
stream open (ScreenCaptureKit on macOS, Desktop Duplication on Windows) and
answers with the newest frame plus a `live: {"seq","changed"}` line. Given
`after`, it waits until the screen changes past that frame, so an idle
viewer costs nothing. Viewer-only and absent from the tool schema, like
`cursor: true`. Linux and unsupported displays answer with an ordinary
screenshot (no live line), which the viewer treats as "keep polling".
- The stream stops itself 5 s after the last request, so the macOS
screen-recording indicator is only up while someone is watching. Stopping
it from the menu bar is honoured for the life of the process.
- macOS `screenshot`/`zoom` honour `format` and `quality` like Windows/Linux
(both now in the Swift schema; the parity allowance is gone).
- macOS keeps the display list between captures (dropped on screen changes),
so a display screenshot no longer walks every window on screen first.
Measured: macOS live frame ~2 ms (one-shot PNG 120 -> 90 ms, JPEG ~105 KB
vs 576 KB PNG); Windows live frame <1 ms when nothing changed (Blade).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
MT Code's Computer View polled
screenshotup to 30 times a second. Each frame enumerated displays, set up a capture and encoded it, and then the viewer compared bytes to find out whether anything had moved. That measured ~100 ms a frame on macOS and Windows, and the macOS helper only produced PNG (~400–600 KB a frame).What changes
screenshot { display, live: true, after?, wait_ms? }keeps a capture stream open: ScreenCaptureKit on macOS, Desktop Duplication on Windows. It answers with the newest frame plus alive: {"seq","changed"}line. Withafter, it waits until the screen changes past that frame. Likecursor: true, this is viewer-only and not in the tool schema. Linux and unsupported displays (rotated or HDR desktops, remote sessions) answer with an ordinary screenshot and no live line, so a viewer keeps polling.screenshot/zoomnow honourformatandqualitylike Windows/Linux. Both are now in the Swift schema, and the parity allowance is gone.Measured
Not tested: lock screen and UAC (Desktop Duplication cannot read the secure desktop; the code returns a clear error and reopens when it closes). Multi-monitor Windows (Blade has one display).
🤖 Generated with Claude Code