Replace old product taxonomy with canonical primitives (Fetch / Extract / Batch / Browser Sessions) - #3
Merged
Merged
Conversation
Update descriptive/label/help/comment text across the CLI, registry, skills, templates, workflows, and docs to use the four canonical primitives: - "Universal Scraper API" -> Fetch / Extract / "Fetch and Extract" (umbrella) - "Scraping Browser" -> Browser Sessions - "Residential Proxies" (standalone) -> folded into the primitives - "Batch Scraper API" -> "Batch" in prose/labels, "Batch API" in error strings Text-only: no command names, flags, endpoints, or hosts changed. Typecheck passes. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsXfB6dR3rRX64K3NwyuoD
Open-beta capabilities are usable; AUTH010 on extract=auto retries once with autoparse so any domain still works.
Add src/core/browser-api.ts — a thin REST client for the managed Browser Sessions API (https://mcp.zenrows.com/browser/sessions/*, Authorization: Bearer), modeled on batch-api.ts (separate host, header auth, injectable fetch). Covers session create/verb/close, base64 capture decode (screenshot/PDF), the CDP connect-URL builder, and normalizeSelectValue (the backend matches <option> by CSS selector, so a bare value becomes option[value="…"]). Unit-tested. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…mitive Browser Sessions is a GA Zenrows product (formerly Scraping Browser), so treat it like fetch/extract rather than an experimental gate: capability status is now "available" and policy.allow_browser defaults to true (opt out with `zenrows policy set allow_browser false`). The gate becomes a plain opt-out — new POLICY_BROWSER_DISABLED error (replacing the experimental code for browser) with a copy-paste re-enable command. Update policy + capability tests. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Rewrite the browser command as subcommand dispatch: open/close, the full set of session verbs (navigate, click, type, fill, select, get-text/html, screenshot, pdf, wait*, cookies, local-storage, tabs, …), `run <script.json>` (one process, auto-closes in a finally; steps use action or verb), and `connect` (prints the CDP wss URL for bring-your-own Playwright). Binary-safe screenshot/PDF via --out. Tests: dispatch/validation, run sequencing + always-close, verb discriminator. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…kill Document the Browser Sessions primitive: GA product (formerly Scraping Browser), escalation-only (prefer fetch/extract — they cost less), billed by bandwidth + session time with a 15-minute auto-terminate cap, on by default with a policy.allow_browser opt-out. Updates README, docs/capabilities.md, the registry skill entry, and the interact-browser + master skills. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
AnderRV
marked this pull request as ready for review
August 1, 2026 12:05
Wire extract=auto as default with Autoparse fallback.
Resolve capability-matrix conflicts: take base's extract=beta (extract=auto open beta) and batch open-beta/usable; keep browser 'available' (GA) with the new REST-session entry; merge the status legend. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NsXfB6dR3rRX64K3NwyuoD
…ation Browser sessions bypassed the domain allow/deny policy that fetch/extract enforce. Add assertDomainAllowed on every navigation target — `open <url>`, the `navigate`/`new-tab` verbs, and `navigate`/`new_tab` steps in `run` — so a blocked domain is rejected (POLICY_BLOCKED_DOMAIN) before any session is created. Test proves the block fires before any network call. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… caps note Finish the label cleanup the earlier pass missed in agent-facing files: registry skills (extract available→beta, interact-browser experimental→available), the master zenrows skill routing table/tree, and compliance-policy (browser is on-by-default, opt-out). Clarify that policy numeric caps (credits/pages/ concurrency) are advisory budgets, not hard-enforced. Document extract --outputs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…vailable `plugin install`/`update` filtered skills to status=available, which dropped the now-beta extract and batch-jobs skills from the payload — even though beta is a usable status. Filter by assetRunnable (all required capabilities usable) so core skills like extract ship with the plugin again. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…T-1514)
extract=auto/autoparse could return an empty result (null / {} / [] / non-JSON)
and the CLI reported it as a normal success — an agent gets nothing but thinks
it worked. Detect empty structured results (new outcome.empty), print a visible
warning suggesting --js-render / --css, add `empty` to --json output, and make
--validate fail (EXTRACT_VALIDATION_FAILED) on an empty result, not just on
non-JSON. Verified live on example.com; real extractions are unaffected.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Batch was the odd primitive out: `create` skipped the capability gate, used
`requireApiKey` (no auto-signup), never enforced the domain policy or the
numeric run caps, and wrote no run artifact — unlike fetch/extract.
Batch is the one primitive that fans out into many requests, so the per-run
caps (max_pages_per_run / max_credits_per_run) actually bind here (a single
fetch/extract is one request; browser bills separately). Enforce them, plus
the domain allow/deny policy, all as local pre-flight — before any network call.
- add `POLICY_LIMIT_EXCEEDED` + `assertWithinLimits({pages?,credits?}, policy)`
- `batch create`: `assertUsable("batch")`, `assertDomainAllowed` per task URL,
`assertWithinLimits` (task count + estimated credits), `ensureApiKey`
honoring `--no-signup`, and a `writeRun` artifact on success/error
- add `assertUsable("batch")` gate parity to status/results/cancel/wait/retry
- tests: policy caps (pure) + batch create pre-flight blocks (no-network)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ow polish
Additive CLI-consistency pass (the safe half of the coherence review;
the exit-code contract change is deferred to its own PR):
- every success --json path now leads with a flat `ok` field, matching the
established shape (fetch/browser/batch already did). Errors already emit
`{ok:false, error}` via printError, so agents can branch on `.ok` uniformly.
Touched: account/config/policy/usage/mcp/plugin/trace/browser(info)/batch
(estimate) + asset explain/validate. `ok` reflects validity where meaningful
(estimate, asset validate). Raw-dump paths left as-is by intent (trace
export, eval stored results, browser human-readable fallback).
- batch `--output`: fail loudly (INVALID_USAGE) on an unknown format instead of
silently dropping it — matching fetch's normalizeOutput (no silent drops).
- batch `--follow`: clearer name for "poll until done"; `--wait` kept as a
back-compat alias (fetch/extract use `--wait <ms>` for a different meaning).
- tests: --output loud-fail (no-network) + estimate --json envelope (ok true/false).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The top-level catch mapped CAPABILITY_UNAVAILABLE to exit 2, but the capability-gated commands (fetch/extract/batch/browser) all catch internally and return 1, so that branch never fired — the CLI already exits 1 on every error in practice, and no docs promised otherwise. Make the contract honest: any error exits 1. Machine consumers read the precise `error.code` from --json output to tell a denial from a failure, which is finer- grained than a 1-vs-2 exit code would ever be. Behavior-preserving. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
feat(cli): consistent --json success envelope + batch --output/--follow polish
Extract empty result
Make batch a first-class, governed primitive
Dead code: nothing in src/ or tests/ references it (assertUsable is the sole CAPABILITY_UNAVAILABLE thrower). Drop the exported helper. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
State the contract plainly (any error exits 1; read error.code from --json for the kind) instead of explaining why the removed CAPABILITY_UNAVAILABLE→2 branch was dead — that archaeology belongs in the commit that removed it, not inline. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Tidy dead code
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The CLI's README, help text, capability matrix, skills, templates, workflows, and code comments described the product with the old taxonomy — "Universal Scraper API", "Scraping Browser", and "Residential Proxies". This PR rewrites all of that descriptive text to the new canonical taxonomy: the four primitives Fetch, Extract, Batch, and Browser Sessions.
Before → After