Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
e0b86a4
feat: detect local SLAB installation
beubax Aug 18, 2026
7457b2a
feat: install signed SLAB releases on macOS
beubax Aug 18, 2026
48b2276
fix: harden SLAB macOS installation
beubax Aug 18, 2026
a2bd915
feat: offer SLAB during local setup
beubax Aug 18, 2026
156c312
fix: show SLAB guidance without a shell
beubax Aug 18, 2026
88c6624
feat: connect to the SLAB bridge
beubax Aug 18, 2026
d06615f
fix: wait for the SLAB bridge
beubax Aug 18, 2026
dc74852
refactor: rename local runtime boundary to SLAB
beubax Aug 18, 2026
78222f4
test: preserve browser run exclusion after SLAB rename
beubax Aug 18, 2026
c3f9a83
refactor: finish local runtime SLAB rename
beubax Aug 18, 2026
032ae7d
refactor: finish SLAB internal boundary rename
beubax Aug 18, 2026
f32aca9
feat: attach local sessions to SLAB
beubax Aug 18, 2026
daca922
fix: release SLAB attachments safely
beubax Aug 18, 2026
da3beac
fix: await SLAB parking cleanup
beubax Aug 18, 2026
310d9c4
feat: use SLAB as the local browser provider
beubax Aug 18, 2026
87fbd10
fix: preserve SLAB provider status compatibility
beubax Aug 18, 2026
159a1d1
feat: require SLAB for local browser commands
beubax Aug 18, 2026
6d0d0e9
fix: enforce SLAB before browser commands
beubax Aug 18, 2026
d7bb911
test: lock SLAB local and hosted parity
beubax Aug 18, 2026
1ad9a6f
fix: remove stale Cloak gate surfaces
beubax Aug 18, 2026
7d95e95
test: stub SLAB install in execution metadata test
beubax Aug 18, 2026
01c2743
fix: update SLAB parity surfaces
beubax Aug 18, 2026
a119239
fix: clarify legacy benchmark browser boundary
beubax Aug 18, 2026
8818e2c
fix webcmd SLAB bridge integration
beubax Aug 19, 2026
dce286c
fix SLAB bridge endpoint compatibility
beubax Aug 19, 2026
0530a8f
fix SLAB app running detection
beubax Aug 19, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -203,8 +203,8 @@ jobs:
- name: Run smoke tests
run: npx vitest run --project smoke --reporter=verbose

windows-cloak-smoke:
name: Windows real Cloak smoke
windows-slab-smoke:
name: Windows SLAB smoke
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
runs-on: windows-latest
timeout-minutes: 30
Expand All @@ -229,8 +229,8 @@ jobs:
shell: pwsh
run: ./scripts/collect-ci-diagnostics.ps1 -SelfTest

- name: Run real Cloak lifecycle smoke
run: npx vitest run --project e2e tests/e2e/cloak-runtime.test.ts tests/e2e/cloak-session-concurrency.test.ts
- name: Run SLAB lifecycle smoke
run: npx vitest run --project e2e tests/e2e/slab-runtime.test.ts tests/e2e/slab-session-concurrency.test.ts

- name: Collect sanitized diagnostics
if: failure()
Expand All @@ -242,5 +242,5 @@ jobs:
- uses: actions/upload-artifact@v4
if: failure()
with:
name: windows-cloak-diagnostics
path: artifacts/windows-cloak/**
name: windows-slab-diagnostics
path: artifacts/windows-slab/**
4 changes: 2 additions & 2 deletions .github/workflows/e2e-headed.yml
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ jobs:
run: |
xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" \
npx vitest run --project e2e-fixed-port --project e2e \
tests/e2e/browser-tabs.test.ts tests/e2e/cloak-runtime.test.ts \
tests/e2e/browser-tabs.test.ts tests/e2e/slab-runtime.test.ts \
--reporter=verbose

- name: Run browser e2e tests (macOS)
Expand All @@ -68,5 +68,5 @@ jobs:
WEBCMD_E2E: '0'
run: |
npx vitest run --project e2e-fixed-port --project e2e \
tests/e2e/browser-tabs.test.ts tests/e2e/cloak-runtime.test.ts \
tests/e2e/browser-tabs.test.ts tests/e2e/slab-runtime.test.ts \
--reporter=verbose
2 changes: 1 addition & 1 deletion PRIVACY.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# webcmd Privacy

The webcmd-managed CloakBrowser runtime communicates only with the local Webcmd daemon on `localhost:9777`.
The webcmd-managed SLAB browser runtime communicates only with the local Webcmd daemon on `localhost:9777`.

The runtime can access browser pages and cookies because browser automation requires those permissions. Webcmd does not send browser data to AgentR. Commands run locally, and command output is printed to the local CLI process.

Expand Down
6 changes: 3 additions & 3 deletions TESTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,12 @@ npx vitest run --project unit src/convention-audit.test.ts src/runtime-copy.test
npm run test:plugin -- --reporter=verbose
```

## Cloak Runtime Smoke
## SLAB Runtime Smoke

Run:

```bash
npx vitest run --project e2e tests/e2e/cloak-runtime.test.ts
npx vitest run --project e2e tests/e2e/slab-runtime.test.ts
```

The first run may download the CloakBrowser Chromium binary. Browser-backed tests no longer require a Chrome extension.
The live run requires an installed SLAB browser. Browser-backed tests no longer require a Chrome extension.
21 changes: 13 additions & 8 deletions benchmarks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,9 +85,14 @@ webcmd ███████████████████

Run a controlled, sequential browser-tool benchmark. Keep task data and local evidence private.

Legacy boundary: this private benchmark harness intentionally keeps its pinned
CloakBrowser dependency so historical AXI/agent-browser/dev-browser/libretto
runs remain comparable. It is not the webcmd local runtime and is not part of
the SLAB browser distribution path.

## Workflow

1. Confirm `uv`, the selected controller CLI, selected browser tool, and judge authentication are available (`GOOGLE_API_KEY` for `google`, `OPENAI_API_KEY` for `openai`, or a ChatGPT-authenticated `codex login` for `codex`). AXI, agent-browser, and dev-browser runs also require CloakBrowser.
1. Confirm `uv`, the selected controller CLI, selected browser tool, and judge authentication are available (`GOOGLE_API_KEY` for `google`, `OPENAI_API_KEY` for `openai`, or a ChatGPT-authenticated `codex login` for `codex`). AXI, agent-browser, and dev-browser runs also require the legacy benchmark CloakBrowser package.
2. Ask the user to choose a controller, model, benchmark, and task selection if any is missing.
3. Start with one task unless the user explicitly requests a larger or full run.
4. Run `scripts/run_eval.py` with the explicit choices.
Expand Down Expand Up @@ -166,7 +171,7 @@ Its `estimated_api_cost_usd` is an API-equivalent comparison metric, not a
subscription charge; the manifest records `billing_mode` as
`chatgpt_subscription`.

For AXI with one dedicated CloakBrowser process and profile per task:
For AXI with one dedicated legacy benchmark CloakBrowser process and profile per task:

```bash
uv run python benchmarks/scripts/run_eval.py \
Expand All @@ -178,7 +183,7 @@ uv run python benchmarks/scripts/run_eval.py \
--tools chrome-devtools-axi
```

For agent-browser with one dedicated CloakBrowser process and profile per task:
For agent-browser with one dedicated legacy benchmark CloakBrowser process and profile per task:

```bash
uv run python benchmarks/scripts/run_eval.py \
Expand All @@ -190,7 +195,7 @@ uv run python benchmarks/scripts/run_eval.py \
--tools agent-browser
```

For dev-browser with one dedicated CloakBrowser process and profile per task:
For dev-browser with one dedicated legacy benchmark CloakBrowser process and profile per task:

```bash
npm install -g dev-browser
Expand All @@ -212,10 +217,10 @@ its `bash` and `read` tools.
`dev-browser install` is required because it installs the daemon's Playwright
and QuickJS dependencies. It may also download dev-browser's Chromium, but the
benchmark does not use that browser: the task-private shim always connects
dev-browser to the task's dedicated CloakBrowser CDP endpoint.
dev-browser to the task's dedicated legacy benchmark CloakBrowser CDP endpoint.

For Libretto Browser Tools with Codex or Pi and one dedicated CloakBrowser per
task:
For Libretto Browser Tools with Codex or Pi and one dedicated legacy benchmark
CloakBrowser per task:

```bash
npm ci
Expand All @@ -234,7 +239,7 @@ Pi, it registers the equivalent native Pi custom tools directly; use
`--controller pi --model openai-codex/gpt-5.6-sol`. Both expose only `browser_open`,
`browser_exec`, `browser_snapshot`, `browser_status`, and `browser_close`.
`browser_connect` is disabled so the agent cannot leave the task's dedicated
CloakBrowser.
legacy benchmark CloakBrowser.

Use `--stealth-view official` only with `Stealth_Bench_V1`. Never add a parallel flag or publish `results/`.

Expand Down
19 changes: 10 additions & 9 deletions benchmarks/scripts/axi_runtime.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,14 @@


def find_cloak_package() -> Path:
"""Locate the private benchmark harness's legacy CloakBrowser package."""
override = os.environ.get("BROWSER_BENCH_CLOAK_PACKAGE")
if override:
candidates = [Path(override).expanduser()]
else:
npm = shutil.which("npm")
if not npm:
raise RuntimeError("npm is required to locate CloakBrowser")
raise RuntimeError("npm is required to locate the legacy benchmark CloakBrowser")
result = subprocess.run(
[npm, "root", "-g"],
text=True,
Expand All @@ -47,7 +48,7 @@ def find_cloak_package() -> Path:
if package.get("name") == "cloakbrowser":
return candidate.resolve()
raise RuntimeError(
"CloakBrowser was not found. Install it globally, keep Webcmd installed, "
"Legacy benchmark CloakBrowser was not found. Install it globally, keep Webcmd installed, "
"or set BROWSER_BENCH_CLOAK_PACKAGE to its package directory."
)

Expand Down Expand Up @@ -82,13 +83,13 @@ async def _launch_options(profile_dir: Path, env: dict[str, str]) -> dict:
except asyncio.TimeoutError:
process.kill()
await process.wait()
raise RuntimeError("timed out resolving CloakBrowser launch options")
raise RuntimeError("timed out resolving legacy benchmark CloakBrowser launch options")
if process.returncode:
raise RuntimeError(f"CloakBrowser launch configuration failed: {stderr.decode(errors='replace').strip()}")
raise RuntimeError(f"Legacy benchmark CloakBrowser launch configuration failed: {stderr.decode(errors='replace').strip()}")
try:
return json.loads(stdout)
except json.JSONDecodeError as error:
raise RuntimeError("CloakBrowser returned invalid launch configuration") from error
raise RuntimeError("Legacy benchmark CloakBrowser returned invalid launch configuration") from error


async def _wait_for_devtools_port(profile_dir: Path) -> int:
Expand All @@ -102,14 +103,14 @@ async def _wait_for_devtools_port(profile_dir: Path) -> int:
except (FileNotFoundError, IndexError, ValueError):
pass
await asyncio.sleep(0.05)
raise RuntimeError("timed out waiting for CloakBrowser's CDP endpoint")
raise RuntimeError("timed out waiting for legacy benchmark CloakBrowser's CDP endpoint")


async def _launch_cloak(profile_dir: Path, env: dict[str, str]) -> int:
options = await _launch_options(profile_dir, env)
executable = Path(str(options.get("executablePath") or ""))
if not executable.is_file():
raise RuntimeError(f"CloakBrowser executable is missing: {executable}")
raise RuntimeError(f"Legacy benchmark CloakBrowser executable is missing: {executable}")
arguments = [
*map(str, options.get("args") or []),
"--password-store=basic",
Expand All @@ -123,7 +124,7 @@ async def _launch_cloak(profile_dir: Path, env: dict[str, str]) -> int:
marker = "/Contents/MacOS/"
executable_text = str(executable)
if marker not in executable_text:
raise RuntimeError("CloakBrowser executable is not inside a macOS app bundle")
raise RuntimeError("Legacy benchmark CloakBrowser executable is not inside a macOS app bundle")
app_path = executable_text.split(marker, 1)[0]
command = ["/usr/bin/open", "-g", "-n", app_path, "--args", *arguments]
else:
Expand All @@ -137,7 +138,7 @@ async def _launch_cloak(profile_dir: Path, env: dict[str, str]) -> int:
)
_, stderr = await process.communicate()
if process.returncode:
raise RuntimeError(f"CloakBrowser failed to launch: {stderr.decode(errors='replace').strip()}")
raise RuntimeError(f"Legacy benchmark CloakBrowser failed to launch: {stderr.decode(errors='replace').strip()}")
return await _wait_for_devtools_port(profile_dir)


Expand Down
10 changes: 5 additions & 5 deletions benchmarks/scripts/run_controller.py
Original file line number Diff line number Diff line change
Expand Up @@ -242,28 +242,28 @@ def _build_prompt(tool: Tool, session: str, shots_dir: Path, task: str) -> str:
- Every shell command must begin with exactly `npx -y chrome-devtools-axi`; use no substitutions, redirections, pipes, or other executables.
- Do not use Web search, browser MCPs, Playwright, Puppeteer, curl, wget, raw HTTP, or any non-AXI automation tool.
- Use the `$chrome-devtools-axi` skill for AXI usage guidance.
- The AXI session and its dedicated CloakBrowser connection are already configured in the environment; do not start another browser."""
- The AXI session and its dedicated legacy benchmark CloakBrowser connection are already configured in the environment; do not start another browser."""
elif tool == "agent-browser":
tool_rules = """- Use only `agent-browser` for browser interaction.
- Use one `agent-browser` command per shell invocation; use no substitutions, redirections, pipes, or other executables.
- Do not use `batch`, `close`, connection/profile flags, or multiple agent-browser commands in one shell invocation.
- Never pass a URL to `agent-browser read`; use `agent-browser open URL` and then `agent-browser read` so all page traffic stays inside CloakBrowser.
- Never pass a URL to `agent-browser read`; use `agent-browser open URL` and then `agent-browser read` so all page traffic stays inside the legacy benchmark CloakBrowser.
- Do not use Web search, browser MCPs, Playwright, Puppeteer, curl, wget, raw HTTP, or any non-agent-browser automation tool.
- Use the `$agent-browser` skill for agent-browser usage guidance.
- The agent-browser session and its dedicated CloakBrowser connection are already configured in the environment; do not start, connect, configure, or close another browser."""
- The agent-browser session and its dedicated legacy benchmark CloakBrowser connection are already configured in the environment; do not start, connect, configure, or close another browser."""
elif tool == "dev-browser":
tool_rules = f"""- Use only `dev-browser` for browser interaction.
- Invoke `dev-browser` with one quoted heredoc per shell command. The heredoc body must contain only the sandboxed JavaScript described by the `$dev-browser` skill.
- You may add only `--timeout SECONDS`; do not use `--browser`, `--connect`, `--headless`, `run`, `stop`, or any other subcommand or connection option.
- Use no substitutions, pipes, extra redirections, or other executables.
- Do not use Web search, browser MCPs, external Playwright, Puppeteer, curl, wget, raw HTTP, or any non-dev-browser automation tool.
- Use the `$dev-browser` skill for dev-browser usage guidance.
- The dev-browser command is already pinned to this task's dedicated CloakBrowser connection; do not start, connect, configure, or close another browser.
- The dev-browser command is already pinned to this task's dedicated legacy benchmark CloakBrowser connection; do not start, connect, configure, or close another browser.
- Save screenshots with `await saveScreenshot(await page.screenshot(), "{session}-step_001.png")`, then `{session}-step_002.png`, and so on."""
elif tool == "libretto":
tool_rules = """- Use only the Libretto MCP tools `browser_open`, `browser_exec`, `browser_snapshot`, `browser_status`, and `browser_close` for browser interaction.
- Do not use shell commands, Web Search, other MCP servers or tools, Playwright outside `browser_exec`, Puppeteer, curl, wget, or raw HTTP.
- The Libretto provider is already pinned to this task's dedicated CloakBrowser; open it with `browser_open` and do not configure another browser.
- The Libretto provider is already pinned to this task's dedicated legacy benchmark CloakBrowser; open it with `browser_open` and do not configure another browser.
- Reuse the session ID returned by `browser_open` and close it with `browser_close` when the task is complete."""
else:
tool_rules = f"""- Use only `webcmd` raw-browser commands for task execution. Do not use Web Search, browser MCPs, external Playwright, Puppeteer, curl, wget, raw HTTP, adapters, fetch commands, plugins, or any non-Webcmd automation tool.
Expand Down
6 changes: 3 additions & 3 deletions benchmarks/tests/test_controller.py
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ def test_agent_browser_prompt_uses_installed_skill_and_dedicated_cloak(tmp_path)

assert "`$agent-browser` skill" in prompt
assert "only `agent-browser`" in prompt
assert "dedicated CloakBrowser" in prompt
assert "dedicated legacy benchmark CloakBrowser" in prompt
assert "Do not use `batch`" in prompt
assert "one `agent-browser` command per shell invocation" in prompt
assert "Never pass a URL to `agent-browser read`" in prompt
Expand All @@ -374,7 +374,7 @@ def test_dev_browser_prompt_uses_installed_skill_quoted_heredoc_and_task_screens
assert "`$dev-browser` skill" in prompt
assert "only `dev-browser`" in prompt
assert "quoted heredoc" in prompt
assert "dedicated CloakBrowser" in prompt
assert "dedicated legacy benchmark CloakBrowser" in prompt
assert "saveScreenshot" in prompt
assert "session-1-step_001.png" in prompt
assert "Webcmd" not in prompt
Expand All @@ -395,7 +395,7 @@ def test_libretto_prompt_uses_only_native_tools_on_dedicated_cloak(tmp_path):
):
assert f"`{tool}`" in prompt
assert "browser_connect" not in prompt
assert "dedicated CloakBrowser" in prompt
assert "dedicated legacy benchmark CloakBrowser" in prompt
assert "screenshot: true" in prompt
assert "shell commands" in prompt
assert "$libretto" not in prompt
Expand Down
3 changes: 0 additions & 3 deletions bun.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion docs/cli-reference.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ webcmd --profile work session close \
session_7d8f2c10-4a11-4f3e-9c22-1b6de0a91f45
```

Local browser commands use Cloak. Hosted browser commands use Webcmd Cloud and Browser Use; `web fetch` still runs locally.
Local browser commands use SLAB. Hosted browser commands use Webcmd Cloud and Browser Use; `web fetch` still runs locally.

## Browser Programs

Expand Down
Loading
Loading