feat(computer-use): native desktop automation for macOS

AGENTS.md

@@ -90,6 +90,31 @@ infrastructure that survives the upstream sync. Full map:
   so the agent cannot read/write its own ceiling — the single mechanism that makes
   the ceiling un-disableable. When editing `security.py`'s sensitive-path or
   bash-command matchers, keep these covered (incl. write/extract verbs).
+- **Computer use is deliberately NOT governed.** It is ONE operator opt-in on the
+  keystone `computer_use.json` (fenced by `security._SENSITIVE_HOME_DIRS`, so the
+  agent can neither read nor flip it). There are **no** `computer_use.*` scopes, no
+  `capabilities.computer_use*` rows, no approval ordinal and no pointer permit — an
+  earlier revision had all eight and they were removed by product decision. Do NOT
+  reintroduce them without that decision being reversed; `gate.py` is audit-only and
+  always permits.
+  What still refuses, all enforced **in band** on the `tools._dispatch` path (never
+  at the `hooks` PreToolUse gate, which is fail-OPEN and can be skipped by a
+  pre-authorized tool): the keystone enable, **KiroCrew's own window**
+  (`policy.check_app` — driving our Settings UI would route around the keystone),
+  secure/password fields, the sensitive-text scan, and credential redaction. Keep
+  these on the dispatch path. Secure-field redaction is an **always-on floor with no
+  policy key** — never make it governable.
+  **Element-targeted, non-pointer input is the DEFAULT.** `click_method: "global"` is
+  the only path that warps the operator's REAL pointer; it needs no separate opt-in,
+  but the model MUST NAME it — `auto` must NEVER resolve onto it (load-bearing
+  invariant with its own test; it is now the only thing between an ordinary click and
+  the user's cursor). Every such gesture is SEL-audited under its own `tool_kind`.
+  `sky_click` is deliberately not ported (private SkyLight API).
+  Flipping the enable RESTARTS chat sessions (`_reset_all_sessions`), because
+  kiro-cli caches `tools/list` per session and ACP has no `tools/list_changed`.
+  Keep prose in sync: `config/prompt.md` and `builtin_skills/computer-use/SKILL.md`
+  ship to users and must not describe refusals that no longer exist. See
+  `docs/system-specs/modules/computer-use.md` + `.../governance.md`.
 - Keep the `ACP_BACKEND_CLAUDE` seam and `platform/` extension points intact; don't
   add public registration glue, and keep the stubs stubbed.
 
@@ -371,6 +396,7 @@ Jane Doe (janedoe), John Smith (jsmith)
 | `cli_server.py` | CLI gateway/server commands (split from cli.py) |
 | `cli_setup.py` | CLI setup wizard (split from cli.py) |
 | `dashboard/chat_runner.py` | Chat execution logic (split from `dashboard/chat.py`) |
+| `computer_use/` | Native desktop GUI automation (macOS today; Windows/Linux refuse). `ComputerUseBackend` ABC + swap registry, the keystone primary enable, the audit-only `gate.py`, the in-band refusals in `policy.py`, and the ctypes driver — all in-gateway native work confined to `macos_ffi.py`. The **one documented exception**: `overlay_proc.py` (the Cursor Motion AppKit child) has its own ctypes surface because AppKit needs a main-thread run loop and the gateway's main thread is the asyncio loop, so it MUST be out of process. See `docs/system-specs/modules/computer-use.md`. |
 | `platform/` | **Composed Platform Providers (CPP) seam + Governance model** — see the dedicated section below. |
 
 ### Platform layer: Composed Platform Providers (CPP) + Governance
@@ -496,10 +522,11 @@ KiroCrew exposes capabilities to the LLM via two mechanisms:
 1. **MCP tools** (native): kiro-cli calls them directly with structured JSON params — **preferred for all LLM-facing operations**
    - `kirocrew-cron` MCP server: `cron_list`, `cron_add`, `cron_update`, `cron_remove`, `cron_remove_all`, `cron_pause`, `cron_resume`, `cron_trigger`
    - `kirocrew-core` MCP server: `spawn_run`, `spawn_list`, `spawn_status`, `learn_add`, `learn_list`, `learn_remove`, `task_run`, `wait`, `register_hook`, `send_message`, `send_notification`, `local_knowledge_search`
+   - `kirocrew-computer` MCP server (10 tools): `computer_list_apps`, `computer_get_state`, `computer_click`, `computer_drag`, `computer_type_text`, `computer_press_key`, `computer_set_value`, `computer_scroll`, `computer_perform_action`, `computer_end_turn` — native desktop GUI automation. Default-OFF behind a keystone primary enable (`~/.kiro/crew/computer_use.json`, NOT `config.json`); the stdio process is a thin shim and the authoritative fail-closed gate runs in the gateway. `computer_click` takes **either** `element_index` **or** `x`+`y` (never both) plus optional `click_count` (1-3), `mouse_button` (`left`/`right`/`middle`) and `click_method` (`auto`/`accessibility`/`app_post`/`global`); `computer_drag` is coordinate-only. Both keyboard tools (`computer_type_text`, `computer_press_key`) REQUIRE `element_index`: an unnamed target has no role/subrole, so the always-on secure-field (password) refusal could not inspect it. See `docs/system-specs/modules/computer-use.md`.
    - `playwright` MCP server (`@playwright/mcp`): `browser_navigate`, `browser_click`, `browser_snapshot`, `browser_take_screenshot`, `browser_fill_form`, `browser_type`, `browser_press_key`, `browser_evaluate`, `browser_hover`, `browser_drag`, `browser_select_option`, `browser_tabs`, `browser_close`, `browser_wait_for`, `browser_resize`
    - `slack-mcp` (mcpServers): Slack integration
    - Configured in `agents/defaults.json` → `mcpServers` → installed to `kirocrew.json`
-   - `kirocrew-cron` and `kirocrew-core` are managed MCP servers in `agent.py:_MANAGED_MCP_SERVERS` — auto-registered, refreshed preserving user customizations
+   - `kirocrew-cron`, `kirocrew-core` and `kirocrew-computer` are managed MCP servers in `agent.py:_MANAGED_MCP_SERVERS` — auto-registered, refreshed preserving user customizations. `kirocrew-computer` is deliberately added to `tools` but **NOT** `allowedTools`, and its managed spec carries **no `autoApprove` key** (an autoApproved MCP tool never reaches `hooks.on_tool_call`)
    - MCP discovery (`mcp_discovery.py`): on-demand only — users trigger from dashboard "Discover & Sync" button
 
 2. **Skills** (`skills/*/SKILL.md`): on-demand knowledge files for specialized workflows
@@ -557,6 +584,17 @@ should always use the MCP tool equivalents.
 | — | `artifact_post_comment` | kirocrew-core |
 | — | `artifact_mark_review` | kirocrew-core |
 | — | `artifact_delete_comment` | kirocrew-core |
+| `kirocrew computer apps` | `computer_list_apps` | kirocrew-computer |
+| — | `computer_get_state` | kirocrew-computer |
+| — | `computer_click` | kirocrew-computer |
+| — | `computer_drag` | kirocrew-computer |
+| — | `computer_type_text` | kirocrew-computer |
+| — | `computer_press_key` | kirocrew-computer |
+| — | `computer_set_value` | kirocrew-computer |
+| — | `computer_scroll` | kirocrew-computer |
+| — | `computer_perform_action` | kirocrew-computer |
+| — | `computer_end_turn` | kirocrew-computer |
+| `kirocrew computer call <tool>` / `call --calls '[…]'` | — (deliberately none) | — |
 | — | `browser_navigate` | playwright |
 | — | `browser_click` | playwright |
 | — | `browser_snapshot` | playwright |
@@ -568,6 +606,7 @@ should always use the MCP tool equivalents.
 
 - **Handler keywords**: only for instant user-typed commands with no LLM round-trip (e.g. `cron list`, `spawn list`)
 - **Do NOT** add regex to match NL variants — the LLM handles NL interpretation
+- **`kirocrew computer call` is the one deliberate "no MCP twin" row.** It is not a capability — it is a human debug/repro harness that runs the ten existing `computer_*` tools through the same gated chokepoint (optionally a JSON array of them in ONE process, so `element_index` values stay resolvable). The MCP-first rule exists so the model gets a structured tool instead of shelling out, and the model already has all ten. A tool that runs other tools would let a model launder one per-call gate decision into many — so do NOT add `computer_call`.
 
 #### Project-Level Configuration
 

CHANGELOG.md

@@ -11,6 +11,9 @@ All notable changes to KiroCrew are documented in this file.
 - **Agent templates can map skills** — Agent Capabilities → Agent Templates gains a **Skills** section: pick which skills an agent template loads, add and remove them inline, and see the count on the agent list. Previously there was no way to do this at all — the skill count shown next to every agent read zero because it only recognised an internal `builder-mcp --skill-name-filter` convention, `PATCH /api/agents/detail/{name}` accepted nothing but `model`, and at runtime the choice was all-or-nothing: the `kirocrew` agent received the entire catalog while every custom agent received none. A mapping is stored the kiro-cli-native way, as `skill://` entries in the agent's `resources`, so kiro-cli loads the SKILL.md files itself when the agent starts, and KiroCrew's own injected skills block narrows to the same set on the Claude Code backend (where agent `resources` are not read). Agents with no mapping keep exactly their old behavior. `file://` steering globs are never touched, and hand-authored `skill://` entries the editor cannot express — wildcards, or paths outside the known skill roots — are shown read-only and preserved across edits. See `docs/agents.md`.
 
 - **Official Docker image** — The gateway now ships as a multi-arch container at `ghcr.io/kirodotdev/kirocrew` (`stable` / `insider` / `nightly` channel tags plus immutable version tags, linux/amd64 + linux/arm64), built from the exact same wheel pip users install and carrying SLSA build provenance. Registry access remains private for now and requires GHCR authentication with package access. One `docker run` with a single volume gives an always-on headless gateway — dashboard, Slack/Discord/Telegram/WeCom/Webex bots, crons — with kiro-cli preinstalled; `docker exec` in for the one-time `kiro-cli login` and to mint dashboard links. A new `KIROCREW_BIND` env override (validated, fail-narrow) lets the gateway bind beyond loopback inside the container's network namespace so published ports actually work; token auth, CSRF, and Host validation are unchanged and every request still needs a dashboard token. Orchestrator liveness/readiness probes (`/api/health`, `/api/live`, `/api/ready`) are now reachable when addressed by container/pod IP: they bypass Host validation, and in exchange the build-identity fields are additionally gated on a served Host header — a DNS-rebound loopback request learns only `{"ok": true}`. See `docs/DOCKER.md`.
+- **Computer use — let the agent drive your desktop apps** — KiroCrew can now read and operate native applications through the accessibility layer, not just the browser: list what's on screen, get one window as a structured, numbered outline of its buttons, fields, rows and menus, then act on an element by number — press it, type into it, set its value, scroll it, or run one of its own named actions. This reaches the work that lives outside a browser tab: pull a figure out of a spreadsheet into a report, walk a desktop-only internal tool, refile a batch of rows in a native app, or read an error dialog and tell you what it says. The structured outline is the primary channel, so a turn is a few thousand tokens rather than a screenshot dump; an optional compressed screenshot is written to disk and only its path is handed over, to be opened if the outline isn't enough. **Your pointer stays where you left it by default** — actions go to the target application directly, so your cursor, your focused window and your keyboard are left alone (see the entry below for the opt-in exception and for coordinate clicking and dragging). **Off by default and macOS-only in this release** (Windows and Linux say so clearly rather than half-working): turn it on in Settings → Computer Use, which also shows the macOS Accessibility and Screen Recording permission state. It is one switch, and it is the only one: once you turn it on, the agent drives your desktop the way you would. That switch lives in a file the agent can neither read nor write, so a prompt-injected agent cannot enable it — but past that point there is no per-app allow-list and no per-action approval prompt. What still refuses: **password fields are never read and a window holding one is never photographed**, text that looks like a credential is never typed, and **KiroCrew's own dashboard is refused outright** (driving our own Settings would route around the switch that holds the enable — including the dashboard open in a browser tab). Everything else, a terminal included, is reachable. That is a deliberate choice for a single-user machine where you are trusted with your own desktop, and the trade is written up in full in `docs/system-specs/modules/computer-use.md`. Every call is written to the audit log, allowed or refused — with the feature ungoverned, that record is the accountability.
+
+- **Computer use reaches the UI that has no buttons — coordinates, dragging, and an optional visible cursor** — Some things on a screen simply are not controls: a drawing canvas, a map, a timeline, a chart, a slider, a custom-drawn panel in an old internal tool. The agent can now click a *point* in an app (`x`, `y`) and drag between two points, so it can sweep a range, reorder a list, stroke a canvas or drag a slider — plus right-click and middle-click, and double- and triple-click for select-word and select-line. Addressing a control by its number from the outline is still the default and still what you should prefer: it is checked against the window moving underneath it, while a coordinate lands on whatever happens to be at that spot when it arrives. **Your mouse pointer still does not move.** A coordinate click is delivered straight to the target application, so it works on a background window without stealing your cursor, your focus or your keyboard. For the rare UI that only responds to a physically real click — a Dock item, a menu-bar extra — the agent can take your actual pointer — but **only if it asks for that path by name**. The automatic choice never resolves onto it, so your cursor cannot be warped as a side effect of an ordinary click, and every such gesture is logged under its own kind so "did the agent take my mouse?" is one filter in the audit log. There is also a third path for the opposite problem: clicking a window that is **behind** another one, without raising it and without touching your pointer, for renderers that ignore a delivered click because they check with the window server which window is in front. Two new ways to *watch* what is happening, both purely for you and neither granting the agent anything: a floating **live view** panel mirrors the screenshots the agent takes (never a fresh capture of its own, and never a window holding a password field), and an optional **Cursor Motion** overlay draws a moving cursor on your real desktop along the path a click is about to take, so you can see where the agent is aiming before it acts. The overlay is deliberately invisible to screenshots — including the agent's own — so it can never end up in the pixels the agent is reading. For debugging and bug reports there is also a new `kirocrew computer call` command that runs one tool, or a whole sequence in a single pass, through exactly the same permission checks the agent goes through, so a problem can be reproduced from a terminal without a model in the loop.
 
 - **WeCom channel settings panel** — Settings gains a WeCom tab so the WeCom (企业微信) channel can be set up and modified from the dashboard instead of hand-editing `.env` and `config.json`: paste the Bot ID and Secret from the WeCom admin console (stored in `.env`, masked previews, independent replace/clear per credential), manage the userid allow-list (display names in `config.json` are preserved), and optionally flip **Allow all organization members** — an explicit opt-in that lets everyone in your WeCom org tenant DM the bot without listing each userid (an empty allow-list still denies everyone; messages without a userid are always dropped). The status badge reports whether the channel actually started this session, with the failure reason when it didn't.
 

NOTICE

@@ -12,3 +12,11 @@ You may obtain a copy of the License at
 
 Portions of this software incorporate third-party components; see
 THIRD-PARTY-NOTICES for their respective copyright notices and licenses.
+
+The macOS computer-use feature's optional background-window click path
+(src/kiro_crew/computer_use/macos_skylight.py) uses undocumented macOS
+window-server interfaces. The symbol declarations and the event-field
+recipe are derived from prior MIT-licensed open-source projects that
+reverse-engineered that path. No third-party code is copied; the
+implementation is independent, and the module is isolated so the
+dependency is auditable in one place.