From 18a4f4d95b7aff22fa5746d10d6632c4a4d25864 Mon Sep 17 00:00:00 2001 From: Ankit Ranjan Date: Thu, 20 Aug 2026 12:20:33 +0530 Subject: [PATCH 1/4] skills: steer raw 403s through web fetch --- skill-src/smart-search/SKILL.src.md | 4 +++- skill-src/webcmd-usage/SKILL.src.md | 3 +++ skills/smart-search/SKILL.md | 4 +++- skills/webcmd-usage/SKILL.md | 3 +++ 4 files changed, 12 insertions(+), 2 deletions(-) diff --git a/skill-src/smart-search/SKILL.src.md b/skill-src/smart-search/SKILL.src.md index cec36919..4d93aa63 100644 --- a/skill-src/smart-search/SKILL.src.md +++ b/skill-src/smart-search/SKILL.src.md @@ -1,6 +1,6 @@ --- name: smart-search -description: Use when a request needs search, research, source discovery, direct URL fetch, evidence fetching, or search-capable Webcmd adapter discovery. +description: Use when a request needs search, research, source discovery, direct URL fetch, a blocked/403/Cloudflare retry, evidence fetching, or search-capable Webcmd adapter discovery. --- # Smart Search @@ -43,6 +43,8 @@ webcmd web fetch --url Try fetch once. Only `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER` permits browser fallback; otherwise report the returned failure rather than retrying the URL. +If you already fetched the URL outside Webcmd and got non-2xx, 403, blocked, or Cloudflare, still run `webcmd web fetch --url ` once. That command does the TLS-impersonating retry. Do not jump to `browser run` from the raw status line. + For browser fallback, create one Session, navigate the failed URL, inspect it, reuse that Session for allowed fallbacks, then close it. Local browser commands use Cloak; hosted browser commands use Webcmd Cloud and Browser Use. `web fetch` remains local in both modes. ```bash diff --git a/skill-src/webcmd-usage/SKILL.src.md b/skill-src/webcmd-usage/SKILL.src.md index 2654e3c7..29bfbd56 100644 --- a/skill-src/webcmd-usage/SKILL.src.md +++ b/skill-src/webcmd-usage/SKILL.src.md @@ -10,6 +10,8 @@ Webcmd turns websites, Electron desktop apps, and external CLIs into a uniform ` For any search, research, source discovery, direct URL fetch, evidence gathering, or “look this up” request, load `smart-search` before choosing commands. `smart-search` owns fetch-first search, browser-fetch escalation, last-resort search adapters, and source-summary rules. +If a direct fetch (`webfetch`, `curl`, or any non-webcmd HTTP client) returns non-2xx, 403, blocked, or a Cloudflare challenge, the next command is `webcmd web fetch --url `. Do not open a browser until that command returns `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER`. + ## CLI Preflight Before the first Webcmd command in a session, run: @@ -274,6 +276,7 @@ Do not invoke these removed commands: - Do not emit a large unfiltered registry into a bounded output or infer absence from a truncation warning; filter at the source and narrow until the result is complete. - Do not assume every adapter needs a browser; check `strategy`. - Do not silently fall back from a failing adapter to hand-rolled `fetch`; use `--trace retain-on-failure` first. +- Do not treat a raw 403 or challenge from a direct fetch as a browser gate; run `webcmd web fetch --url ` first. diff --git a/skill-src/webcmd-usage/SKILL.src.md b/skill-src/webcmd-usage/SKILL.src.md index 29bfbd56..223911f8 100644 --- a/skill-src/webcmd-usage/SKILL.src.md +++ b/skill-src/webcmd-usage/SKILL.src.md @@ -285,4 +285,10 @@ Author-only. Stripped by litprompt, so it costs the running agent nothing. Append one dated line whenever a correction lands, or whenever an approach is tried and rejected. Record what was tried and why it failed, not just what won. + +- 2026-08-20: A prior generic-client 403 is not authority to escalate to the + browser; browser use remains gated on Webcmd returning `FETCH_BLOCKED` or + `FETCH_REQUIRES_BROWSER`. +- 2026-08-20: Review rejected saying `web fetch` always performs a + TLS-impersonating retry; it may do so only when it detects a challenge. --> From 2b2e9cf4b19d2ea984f2d7de72ac314274e10ea2 Mon Sep 17 00:00:00 2001 From: Ankit Ranjan Date: Thu, 20 Aug 2026 19:09:59 +0530 Subject: [PATCH 4/4] skills: make web fetch the first fetch path --- skill-src/smart-search/SKILL.src.md | 8 +++----- skill-src/webcmd-usage/SKILL.src.md | 2 +- skills/smart-search/SKILL.md | 8 +++----- skills/webcmd-usage/SKILL.md | 2 +- src/skills.test.ts | 2 ++ 5 files changed, 10 insertions(+), 12 deletions(-) diff --git a/skill-src/smart-search/SKILL.src.md b/skill-src/smart-search/SKILL.src.md index fcc84d06..f8cbcaf4 100644 --- a/skill-src/smart-search/SKILL.src.md +++ b/skill-src/smart-search/SKILL.src.md @@ -1,6 +1,6 @@ --- name: smart-search -description: Use when a request needs search, research, source discovery, direct URL fetch, a blocked/403/Cloudflare retry, evidence fetching, or search-capable Webcmd adapter discovery. +description: Use when a request needs search, research, source discovery, direct URL fetch, the first-choice Webcmd fetch path, evidence fetching, or search-capable Webcmd adapter discovery. --- # Smart Search @@ -35,15 +35,13 @@ Prefer primary sources, official docs, and direct content over search snippets. ## Direct URL -For a supplied HTTP(S) URL, fetch it: +For a supplied HTTP(S) URL, use the first-choice Webcmd fetch path: ```bash webcmd web fetch --url ``` -Try fetch once. Only `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER` permits browser fallback; otherwise report the returned failure rather than retrying the URL. - -If you already fetched the URL outside Webcmd and got non-2xx, 403, blocked, or Cloudflare, still run `webcmd web fetch --url ` once. It may retry with TLS impersonation for detected challenges. Do not jump to `browser run` from the raw status line. +Run `webcmd web fetch` before browser work or non-Webcmd HTTP clients. Only `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER` permits browser fallback; otherwise report the returned failure rather than retrying the URL. If a URL was already fetched outside Webcmd and got non-2xx, 403, blocked, or Cloudflare, that does not change the order: run `webcmd web fetch --url ` once before any browser escalation. For browser fallback, create one Session, navigate the failed URL, inspect it, reuse that Session for allowed fallbacks, then close it. Local browser commands use Cloak; hosted browser commands use Webcmd Cloud and Browser Use. `web fetch` remains local in both modes. diff --git a/skill-src/webcmd-usage/SKILL.src.md b/skill-src/webcmd-usage/SKILL.src.md index 9f2c1923..d30ba0ac 100644 --- a/skill-src/webcmd-usage/SKILL.src.md +++ b/skill-src/webcmd-usage/SKILL.src.md @@ -10,7 +10,7 @@ Webcmd turns websites, Electron desktop apps, and external CLIs into a uniform ` For any search, research, source discovery, direct URL fetch, evidence gathering, or “look this up” request, load `smart-search` before choosing commands. `smart-search` owns fetch-first search, browser-fetch escalation, last-resort search adapters, and source-summary rules. -If a direct fetch (`webfetch`, `curl`, or any non-webcmd HTTP client) returns non-2xx, 403, blocked, or a Cloudflare challenge, the next command is `webcmd web fetch --url `. Do not open a browser until that command returns `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER`. +For a known URL, the first-choice Webcmd fetch path is `webcmd web fetch --url `. Use it before browser work or non-Webcmd HTTP clients. Do not open a browser until that command returns `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER`. ## CLI Preflight diff --git a/skills/smart-search/SKILL.md b/skills/smart-search/SKILL.md index 11387cf1..a54351bd 100644 --- a/skills/smart-search/SKILL.md +++ b/skills/smart-search/SKILL.md @@ -1,6 +1,6 @@ --- name: smart-search -description: Use when a request needs search, research, source discovery, direct URL fetch, a blocked/403/Cloudflare retry, evidence fetching, or search-capable Webcmd adapter discovery. +description: Use when a request needs search, research, source discovery, direct URL fetch, the first-choice Webcmd fetch path, evidence fetching, or search-capable Webcmd adapter discovery. --- # Smart Search @@ -35,15 +35,13 @@ Prefer primary sources, official docs, and direct content over search snippets. ## Direct URL -For a supplied HTTP(S) URL, fetch it: +For a supplied HTTP(S) URL, use the first-choice Webcmd fetch path: ```bash webcmd web fetch --url ``` -Try fetch once. Only `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER` permits browser fallback; otherwise report the returned failure rather than retrying the URL. - -If you already fetched the URL outside Webcmd and got non-2xx, 403, blocked, or Cloudflare, still run `webcmd web fetch --url ` once. It may retry with TLS impersonation for detected challenges. Do not jump to `browser run` from the raw status line. +Run `webcmd web fetch` before browser work or non-Webcmd HTTP clients. Only `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER` permits browser fallback; otherwise report the returned failure rather than retrying the URL. If a URL was already fetched outside Webcmd and got non-2xx, 403, blocked, or Cloudflare, that does not change the order: run `webcmd web fetch --url ` once before any browser escalation. For browser fallback, create one Session, navigate the failed URL, inspect it, reuse that Session for allowed fallbacks, then close it. Local browser commands use Cloak; hosted browser commands use Webcmd Cloud and Browser Use. `web fetch` remains local in both modes. diff --git a/skills/webcmd-usage/SKILL.md b/skills/webcmd-usage/SKILL.md index d2c92055..326d0dad 100644 --- a/skills/webcmd-usage/SKILL.md +++ b/skills/webcmd-usage/SKILL.md @@ -10,7 +10,7 @@ Webcmd turns websites, Electron desktop apps, and external CLIs into a uniform ` For any search, research, source discovery, direct URL fetch, evidence gathering, or “look this up” request, load `smart-search` before choosing commands. `smart-search` owns fetch-first search, browser-fetch escalation, last-resort search adapters, and source-summary rules. -If a direct fetch (`webfetch`, `curl`, or any non-webcmd HTTP client) returns non-2xx, 403, blocked, or a Cloudflare challenge, the next command is `webcmd web fetch --url `. Do not open a browser until that command returns `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER`. +For a known URL, the first-choice Webcmd fetch path is `webcmd web fetch --url `. Use it before browser work or non-Webcmd HTTP clients. Do not open a browser until that command returns `FETCH_BLOCKED` or `FETCH_REQUIRES_BROWSER`. ## CLI Preflight diff --git a/src/skills.test.ts b/src/skills.test.ts index 08d3b165..b4f04d7e 100644 --- a/src/skills.test.ts +++ b/src/skills.test.ts @@ -65,6 +65,7 @@ describe('webcmd skills content', () => { expect(skill).toContain('webcmd plugin search'); expect(skill).toContain('webcmd plugin install'); expect(skill).toContain('webcmd web fetch --url'); + expect(skill).toContain('first-choice Webcmd fetch path'); expect(skill).toContain('FETCH_BLOCKED'); expect(skill).toContain('FETCH_REQUIRES_BROWSER'); for (const guide of skills) { @@ -280,6 +281,7 @@ describe('webcmd skills content', () => { const autofix = bundledSkill('webcmd-autofix'); expect(usage).toContain('webcmd session create -f json'); + expect(usage).toContain('first-choice Webcmd fetch path'); expect(usage).toContain('webcmd profile create work'); expect(usage).toContain('webcmd --session session_abc browser'); expect(usage).toContain('SESSION_BUSY');