WebOperator Agent Protocol

This is the preferred protocol for Hermes and other local agents.

Agent
  -> framed JSON Unix socket
  -> WebOperator Native Messaging host
  -> Chrome extension
  -> active browser tab

The Chrome extension starts the Native Messaging host with chrome.runtime.connectNative("com.weboperator.bridge"). Agents do not talk to Chrome directly. They connect to the bridge socket.

Transport

Default socket:

/tmp/weboperator-bridge.sock

Override:

WEBOPERATOR_AGENT_SOCKET=/path/to/socket

Messages use the Chrome Native Messaging frame format:

4-byte little-endian unsigned JSON byte length
UTF-8 JSON payload

Auth

Every agent request must include WEBOPERATOR_API_TOKEN when the bridge starts:

{
  "token": "<WEBOPERATOR_API_TOKEN>"
}

Events do not require a token after the socket is connected. If no token is configured, requests are rejected unless WEBOPERATOR_ALLOW_UNAUTHENTICATED_BRIDGE=1 is set for development.

Request

{
  "id": "req-1",
  "type": "browser.snapshot",
  "payload": {},
  "timeoutMs": 30000,
  "token": "optional-if-auth-enabled"
}

Fields:

id: caller-generated request id
type: command name
payload: command payload object
timeoutMs: optional request timeout
token: required only when WEBOPERATOR_API_TOKEN is set

Response

{
  "id": "req-1",
  "result": {}
}

Error:

{
  "id": "req-1",
  "error": "WebOperator extension is not connected to the bridge"
}

Events

The bridge pushes task events to connected socket clients:

{
  "kind": "event",
  "event": {
    "kind": "task:update",
    "task": {}
  }
}

Common event kinds:

task:update
task:step
task:error
skills:detected

Large screenshots, page snapshots, prompts, and thinking text are omitted from live step events. Use tasks.get for stored task state and HTTP GET /v1/tasks/:id/trace if a full compatibility trace is needed.

Command Types

Bridge:

bridge.health

Browser:

browser.snapshot
browser.screenshot
browser.navigate
browser.click
browser.type
browser.press
browser.scroll
browser.extract

Tasks:

tasks.list
tasks.get
tasks.start
tasks.stop
tasks.pause
tasks.resume
tasks.confirm
tasks.wait

Payloads

bridge.health:

{}

browser.navigate:

{
  "url": "https://example.com",
  "tabId": 123
}

browser.click:

{
  "ref": "@e12",
  "reason": "open details",
  "tabId": 123
}

browser.type:

{
  "ref": "@e4",
  "text": "hello",
  "mode": "replace",
  "submit": "false",
  "tabId": 123
}

tasks.start:

{
  "goal": "Extract the visible invoice total",
  "startUrl": "https://example.com",
  "tabId": 123,
  "autoConfirm": true,
  "attachments": [
    { "id": "cv", "name": "cv.pdf", "path": "/absolute/path/cv.pdf", "mimeType": "application/pdf" }
  ],
  "timeoutMs": 60000
}

Attachments are scoped to that task and are exposed to the browser agent only by ID. The internal upload_attachment action resolves the path inside the extension and uploads it to a visible file input. The side panel's paperclip button produces the same records: a picked file is copied into Downloads/weboperator-attachments/ first, because CDP can only fill a file input from an on-disk path. Keep autoConfirm false for application flows; final Apply/Submit/Send controls always require confirmation.

tasks.get, tasks.stop, tasks.pause, tasks.resume, tasks.wait:

{
  "id": "task-id"
}

tasks.confirm:

{
  "id": "task-id",
  "allow": true
}

Example

WEBOPERATOR_API_TOKEN=dev-token \
  node weboperator-bridge/agent-client.js '{"type":"bridge.health"}'

WEBOPERATOR_API_TOKEN=dev-token \
  node weboperator-bridge/agent-client.js '{"type":"tasks.start","payload":{"goal":"Read the current page title","autoConfirm":true}}'

FilesExpand file tree

agent-protocol.md

Latest commit

History