feat(screen-capture): native, AWT-free screen and window capture

[kdroidFilter]

Closes #637.

Summary

• New runtime module nucleus.screen-capture: native, AWT-free screen and window capture.
  • Images are in physical pixels: a 4K display at 200 % captures as 3840×2160, not the 1920×1080 java.awt.Robot returns.
  • No AWT and no Compose dependency.
  • GraalVM native-image ready: no reflection, and the JNI metadata ships with the module.
• Backends:
  • Windows: GDI BitBlt (CAPTUREBLT) in a per-monitor-v2 thread DPI context. Windows are captured with PrintWindow(PW_RENDERFULLCONTENT), which includes covered parts and DirectX/ANGLE content.
  • macOS: ScreenCaptureKit on 14+ (SCScreenshotManager). Before 14 it uses CGDisplayCreateImageForRect / CGWindowListCreateImage through dlsym, because both are obsoleted in the 15 SDK. ScreenCaptureKit is weak-linked. The code waits on a semaphore with a timeout and never pumps the main run loop.
  • Linux X11: XGetImage (any TrueColor mask, 16-bit, and 8-bit PseudoColor), RandR 1.5 monitors, the XFixes cursor, and XComposite for covered windows. Every library is dlopened and X errors are trapped per call, so a bad XID or a dying X server never takes the JVM down.
  • Linux Wayland: org.freedesktop.portal.Screenshot over a private D-Bus connection. It captures the whole desktop, which is then cropped. The portal's file is decoded with gdk-pixbuf and then deleted.
• The PrintWindow trap is handled by design:
  • PrintWindow sends WM_PRINT and waits forever on a thread that does not pump messages. That covers a busy or hung window, and the app's own UI thread when it is blocked waiting for the capture. We verified that it blocks even when IsHungAppWindow is already TRUE.
  • It therefore runs on a worker thread with a 2 s timeout. On timeout the capture falls back to the visible part of the window read from the screen.
  • The abandoned worker frees itself when the target window finally answers.
  • When the caller is the window's own thread, it prints inline.
• examples/screen-capture-demo: an interactive demo (list displays, capture a display, a region or this window, preview, save PNG). It is also the E2E fixture (SCREEN_CAPTURE_DEMO_SELFTEST=1).
• CI: build steps and verify lists added to build-natives.yaml, pre-merge.yaml and publish-maven.yaml. The Linux apt line now names libxrandr-dev libxfixes-dev libxcomposite-dev explicitly; only their headers are needed, nothing is linked.

Documentation

Installation

dependencies {
    implementation("dev.nucleusframework:nucleus.screen-capture:<version>")
}

Capturing a display

import dev.nucleusframework.screencapture.*

// Every call blocks until the platform answers: call it off the UI thread.
val image: ScreenImage = withContext(Dispatchers.IO) {
    val display = ScreenCapture.primaryDisplay() ?: error("no display")
    ScreenCapture.captureDisplay(display, includeCursor = true)
}
File("shot.png").writeBytes(image.toPng())

ScreenCapture.displays() lists the connected displays, primary first. Each CaptureDisplay has:

Property	Meaning
id	Stable for the session: the GDI device name (\\.\DISPLAY1) on Windows, the CGDirectDisplayID on macOS, the RandR output name on X11, portal on Wayland
name	The monitor's name (its EDID friendly name on Windows)
bounds	The display's rectangle in the platform's desktop coordinates: physical px on Windows and X11, points of the global display space on macOS, null on Wayland
widthPx / heightPx	Size of a full capture, in physical pixels (0 on Wayland until the first capture)
scaleFactor	The DPI scale (dpi / 96) on Windows, px per point on macOS, 1 on X11 and Wayland
isPrimary	Exactly one display is primary

Capturing a region

The region is in the display's own pixels, relative to its top-left corner: the coordinates of the image a full capture returns. It means the same thing on every platform.

The region is clipped to the display. A region that does not intersect the display throws CaptureFailure.InvalidRegion.

val part = ScreenCapture.captureDisplay(display, region = CaptureRegion(x = 100, y = 50, width = 800, height = 600))

Capturing a window

if (ScreenCapture.isWindowCaptureSupported) {
    val shot = ScreenCapture.captureWindow(windowId)
}

What windowId is on each platform:

Platform	windowId	From Tao
Windows	HWND	TaoWindow.nativeHandle
macOS	CGWindowID (NSWindow.windowNumber)	not exposed yet
X11	XID	TaoWindow.x11WindowId

• Covered parts: they are included on Windows and macOS, and on X11 when the window is composited. Otherwise only the window's on-screen part is captured.
• Errors: a minimized, hidden or unknown window throws CaptureFailure.WindowNotFound.
• Wayland: window capture is not available there. isWindowCaptureSupported is false, and captureWindow throws CaptureFailure.Unsupported.

Using the pixels

ScreenImage holds opaque 0xAARRGGBB pixels, row-major:

image.width; image.height; image.scaleFactor
image.pixelAt(x, y)          // 0xAARRGGBB
image.toArgbArray()          // a copy, in BufferedImage.TYPE_INT_ARGB layout
image.crop(CaptureRegion(...))
image.toPng()                // RGB PNG, pure Kotlin encoder (no ImageIO)

// Compose: straight into Skia, no conversion
val bitmap = org.jetbrains.skia.Image.makeRaster(
    ImageInfo(image.width, image.height, ColorType.BGRA_8888, ColorAlphaType.OPAQUE),
    image.toBgraBytes(), image.width * 4,
).toComposeImageBitmap()

Permissions

when (ScreenCapture.permissionStatus()) {
    CapturePermission.Granted, CapturePermission.NotRequired -> capture()
    CapturePermission.NotDetermined -> ScreenCapture.requestPermission() // macOS system prompt
    CapturePermission.Denied -> showHowToEnableScreenRecording()
}

• macOS: the app needs the Screen Recording permission (TCC).
  • requestPermission() shows the system prompt once per app. A grant takes effect after the app restarts.
  • The public API cannot tell "never asked" from "denied", so a missing grant reads as NotDetermined.
  • A capture without the grant throws CaptureFailure.PermissionDenied.
  • macOS 15+ periodically asks the user to confirm again; that is system behaviour.
• Windows and X11: NotRequired.
• Wayland: the compositor may show its own permission dialog on the first capture. A refusal throws PermissionDenied, and dismissing the dialog throws Cancelled.

Backends

ScreenCapture.backend is one of Gdi, ScreenCaptureKit, CoreGraphics, X11, XdgDesktopPortal or Unavailable.

On Linux, the portal is used on a Wayland session (XDG_SESSION_TYPE=wayland, or WAYLAND_DISPLAY set). XWayland's root window does not contain native Wayland windows, so X11 capture would miss them. To force a backend, pass -Dnucleus.screencapture.linuxBackend=x11 or =portal.

Errors

Every failure is a ScreenCaptureException. Its failure is one of:

• Unsupported
• PermissionDenied
• Cancelled
• DisplayNotFound (for example, the monitor was unplugged)
• WindowNotFound
• InvalidRegion
• Timeout
• Failed

The message includes the platform's diagnostic (a Win32 error, an NSError, or a D-Bus error).

Threading

Every function is safe to call from several threads at once. Every function blocks, so call them from Dispatchers.IO.

Capturing from the UI thread works, including capturing the app's own window, which is then printed inline. Avoid it anyway: the portal and ScreenCaptureKit can take a while to answer.

Test plan

• :screen-capture:check on Windows: unit and live tests, detekt, ktlint and apiCheck.

• Windows E2E, scripts/screen-capture-windows-e2e.ps1 -Torture 1500: 39/39 checks, run twice.

  • The test pattern is a 1-px checkerboard, a 0–255 ramp and a colour grid.

  • Pixel-exact captures of it:

    • a display;
    • regions, at every ±3 px shift;
    • the window;
    • the window while it is covered;
    • the window from its own UI thread;
    • the window while its UI thread is blocked on the capture (returns in 2.0 s);
    • the window after it is restored.

  • A minimized window throws WindowNotFound.

  • A genuinely hung foreign window is captured in about 10 ms.

  • The cursor is drawn at its hotspot.

  • Display ids, bounds, scale and primary flag match TaoMonitors.

  • A 2560×1080 capture takes about 22 ms.

  • Torture: 9,516 captures from 11 threads, with random, extreme and Int.MIN/Int.MAX regions, garbage HWNDs and concurrent window captures. Resources before and after:

    	Before	After
    GDI objects	24	24
    USER objects	48	48
    Handles	833	824

• Linux E2E, scripts/screen-capture-linux-e2e.sh (WSL, Xvfb): 14/14 scenarios.

  • Pixel-exact against xwd at depths 24, 16 and 8, at 4K, and with RandR monitors.
  • Cursor, window capture with and without XComposite, and destroyed or garbage XIDs.
  • Torture from 8 threads, with file descriptors and RSS flat.
  • X server killed in the middle of a capture: the JVM survives.
  • Fake portal:
    • success, cancel, error and timeout;
    • bogus, missing or garbage files;
    • percent-encoded paths and symlinks;
    • a portal that ignores handle_token;
    • a response that arrives before the method reply;
    • concurrent calls.
  • ASan and UBSan are clean on the X11 path.

• Real Kotlin API on Linux (Xvfb, depth 24 with monitors and depth 16):

  • ScreenCaptureLiveTest + ScreenImageTest: 12/12.
  • A driver using the public API checked both backends, X11 and portal (portal forced and auto-detected). It also checked the mapping to Cancelled, Unsupported and PermissionDenied, and that the portal refuses window capture.

• macOS: never compiled locally.

  • This PR's build-natives job is the first build of NucleusScreenCapture.m, with -Werror=unguarded-availability-new.
  • After that it needs a manual run on a Mac with Screen Recording granted: ./gradlew :screen-capture:test and the demo.

• Windows HiDPI, and multiple monitors with mixed DPI: the test machine has a single display at 100 %.

• A real Wayland portal (GNOME / KDE): only tested against the fake portal.

• Linux aarch64 build (CI).