Skip to content

docs + fix: v1.4.0 documentation overhaul, repaired install paths, CI-verified claims - #2

Merged
celikgo merged 4 commits into
mainfrom
docs/skills-and-docs-overhaul
Aug 19, 2026
Merged

docs + fix: v1.4.0 documentation overhaul, repaired install paths, CI-verified claims#2
celikgo merged 4 commits into
mainfrom
docs/skills-and-docs-overhaul

Conversation

@celikgo

@celikgo celikgo commented Aug 18, 2026

Copy link
Copy Markdown
Owner

The doc set had drifted from the shipped v1.4.0 surface. Three separate tool-reference sections claimed 51 tools while actually listing ~43, Sprint 18's authenticated-session feature had zero skill coverage, and the test/skill/binary counts were stale across four files.

Scope note. The original commit was docs and npm keywords only. Three later commits
(see Install paths repaired below) added source, test, workflow and tooling changes, so
that no longer describes this PR.

Skills: 17 → 20

New

Skill Job
testing-web-authenticated-sessions The storageState capture/replay workflow — Mode A capture (headed, with a manual MFA pause), Mode B replay via storage_state_path / --storage-state / the scenario storageState field. Plus login-wall detection mid-run, expiry and re-capture, and secret hygiene.
running-web-ci-suites Suite files as a real CI gate — --workers, --shard, --tag, the exit-code contract, JUnit wiring, artifact upload, and a complete GitHub Actions example.
troubleshooting-webmobai-setup A webmobai-doctor-driven decision tree for browser / engine / Lighthouse / AI-key / storageState / Gatekeeper failures.

Updated — all 17 existing skills had drifted claims corrected against source, and gained auth handoffs, the versioned-baseline tools, the opt-in AI tools, and the v1.4.0 assert_url + scenario-runner semantics changes.

.claude/skills/README.md was rebuilt: 20-skill index, updated decision tree, new Preflight / Authenticated sessions / AI layer conventions, and a tool table whose category counts sum to exactly 51.

Docs

  • USER_MANUAL.md §7 rebuilt as a per-tool table — all 51 tools, one row each, with a column for whether a launched browser is required (+538 lines).
  • New: docs/AUTHENTICATION.md (426), docs/CI.md (477), docs/README.md (doc index).
  • docs/SCENARIO_FORMAT.md: storageState, saveStorageState, pauseForManual.
  • FEATURES.md: Sprint 18 sections, the 20-skill list, corrected limitations.
  • ROADMAP.md: Sprint 18 entry + What's next.
  • CONTRIBUTING.md: skill authoring, and a "keeping docs honest" checklist of every place each count is asserted.
  • README.md / mcp-server/README.md: accurate counts, docs index, auth example.

Counts corrected throughout

51 tools across 16 tool files · 7 binaries · 214 tests across 26 files (was 200 / 23) · 20 skills (FEATURES.md said 8).

Verification run against the final tree

  • All 20 skill frontmatter names match their directory name.
  • All 51 tools appear exactly once, no duplicates or omissions, in each of the three tool references; every category count sums to 51.
  • 0 dangling relative links across 32 files.
  • Every JSON and YAML code block parses.

One product finding (not fixed here)

pauseForManual never actually pauses under either shipped CLI. scenario-cli.ts:84 and suite/runner.ts:105 both hardcode headless: true, and scenario/runner.ts:143-151 makes the step a warn-and-continue no-op when headless. A scenario doing pauseForManualsaveStorageState therefore writes an unauthenticated session file and records the run as green. The step only works for programmatic BrowserManager + runScenario callers.

The new auth skill documents this as a trap to avoid rather than a recommended path, but the real fix is either a --headed flag on those CLIs or making the step fail loudly in headless. Worth a follow-up issue.

🤖 Generated with Claude Code


Install paths repaired, and every published number made CI-verified

Added after the original doc overhaul. Two of the four documented install paths were dead, and
the numbers the README advertises were unenforced prose.

What the investigation actually found

The npm publish job already existed. release.yml has carried a publish-npm job since the
v1.4.0 hardening. It ran on the v1.4.0 tag and failed in 26s with
E404 Not Found - PUT https://registry.npmjs.org/webmobai-mcp. A 404 on a PUT is npm's
signature for a token that cannot authenticate, so NPM_TOKEN is invalid, expired, or
read-only. This was a credential problem, not a packaging one.

webmobai-mcp --help would have hung even once published. src/index.ts had no argv
handling at all — it connected the stdio MCP transport and blocked forever on stdin. The command
the README teaches for verifying an install could never have worked.

All three headline numbers were already correct. 51 tools, 7 binaries, 214 tests each
recounted true, so none were "fixed". They are now verified rather than trusted.

Changes

Install paths

  • index.ts gains --help / --version, plus 5 subprocess tests in which a regression hangs
    and the timeout is the assertion. Also drops a hardcoded v1.2.0 startup banner that had
    drifted from the real 1.4.0, by reading the version from package.json.
  • release.yml publishes with --provenance, guards that the tag matches package.json, and
    smoke-tests the built entrypoint before publishing.
  • server.json + an mcpName in package.json, staged for the official MCP Registry. mcpName
    has to be in the published tarball — the registry reads it from there to verify ownership, and
    npm versions are immutable — so it lands before the first publish, not after.

Links

  • github.com/modelcontextprotocol/sdk 404s; corrected to typescript-sdk.
  • Placeholder domains in the skill docs (foo.com, bar.com, x.com) are real registrable
    domains a link checker will chase. Replaced with RFC 2606 reserved names.
  • New docs-links workflow (lychee), modeled on the salman repo's. It found 3 real dead links
    on its first run and now checks 250 with zero dead.

Numbers

  • scripts/check-doc-claims.mjs recounts tools, tool files, binaries, tests, test files, skills
    and the eval rates from the tree, checking 29 claims across 3 files. Verified to fail on
    injected drift — it caught this PR's own test additions, which is why the README now reads
    219 tests / 27 files.

Designed for agents

  • docs/DESIGNED_FOR_AGENTS.md — why a selector miss returns a prior fingerprint, ranked
    candidates and page-state triage, with a transcript captured from the running code rather than
    illustrated.

  • An 18-case eval measures it, running in CI and gating at an 80% floor:

    Top-1 recovery: 86.7%  (13/15)
    Top-3 recovery: 93.3%  (14/15)
    Correct abstention: 66.7%  (2/3)
    Overall: 83.3%  (15/18)
    

    The corpus deliberately includes cases the tool fails, and the doc names all three — a
    suite containing only passes measures the author's taste in examples, not the algorithm. The
    eval runs against dist/, not src/: an early 0% reading was esbuild's keepNames helper
    leaking a __name reference into page.evaluate, a property of the loader rather than the
    product. The shipped build is sound.

Hygiene

  • SECURITY.md — what storageState files actually contain (live session tokens), that they are
    never transmitted, and the disclosure path.
  • CODE_OF_CONDUCT.md (Contributor Covenant 2.1).
  • Social preview: a split frame — the request in plain English beside the real browser run with
    the assertion passing. HTML source kept in-repo so the image is reproducible. It carries no
    counts, because a number baked into a PNG is the one claim check-doc-claims cannot reach.
  • Release matrix extended to Linux and Windows, on ubuntu-22.04 rather than latest so the
    AppImage does not silently raise the glibc floor.

Two CI fixes that were not planned

The link checker failed on its first real run, twice, and both causes were worth fixing rather
than merging past:

  1. A circular failure. It flagged the blob/main links in mcp-server/README.md pointing at
    docs/AUTHENTICATION.md and docs/CI.md — files added on this branch, so they 404 until it
    merges, and the check blocks the merge that fixes it. Those links cannot become relative
    (that README renders on npmjs.com, where ../docs/CI.md resolves to nothing), so this repo's
    own blob URLs are remapped onto the checked-out working copy. That tests the invariant worth
    testing — the file this link points at exists in this repository — before and after merge.
  2. A flake. [ERROR] https://tauri.app/ | Connection failed, on a link that is fine;
    re-running passed unchanged. lychee defaults to 128 concurrent requests, which produces
    connection resets indistinguishable from dead links. Concurrency capped at 16, harder
    retries, real user agent. A 404 still fails the build.

Still open after this merge

  • The npm publish itself — blocked on npm credentials on the publishing machine. Nothing has
    been published.
  • The homepage field — points at the npm package once it exists.
  • The social preview upload — GitHub has no API for it; docs/assets/README.md has the path.
  • Registry submissions — staged in docs/MCP_DIRECTORY_SUBMISSIONS.md and deliberately not
    sent.
  • The Linux and Windows release jobs are unproven. The matrix, the webkit2gtk-4.1
    dependencies and the shell: bash fix are written from the documented requirements, but
    release.yml only fires on v*, so the first real proof is the next tag.

celikgo and others added 4 commits August 18, 2026 20:42
The doc set had drifted badly from the shipped surface: Sprint 18's
authenticated-session feature had zero skill coverage, three tool-reference
sections claimed 51 tools while listing ~43, and several counts (tests,
skills, binaries) were stale across four files.

Skills — 17 -> 20:
- NEW testing-web-authenticated-sessions: the storageState capture/replay
  workflow (Mode A capture with a manual MFA pause, Mode B replay via
  storage_state_path / --storage-state / scenario storageState), login-wall
  detection mid-run, expiry + re-capture, and secret hygiene.
- NEW running-web-ci-suites: suite files as a CI gate — --workers, --shard,
  --tag, exit-code contract, JUnit wiring, artifact upload, GH Actions YAML.
- NEW troubleshooting-webmobai-setup: webmobai-doctor-driven decision tree
  for browser/engine/Lighthouse/AI-key/storageState/Gatekeeper failures.
- All 17 existing skills: corrected drifted claims against source, added
  auth handoffs, the versioned-baseline tools, the opt-in AI tools, the
  v1.4.0 assert_url and scenario-runner semantics changes.
- skills/README.md rebuilt: 20-skill index, updated decision tree, new
  Preflight / Authenticated sessions / AI layer conventions, and a tool
  table whose category counts sum to exactly 51.

Docs:
- USER_MANUAL section 7 rebuilt as a per-tool table: all 51 tools, one row
  each, with a column for whether a launched browser is required.
- NEW docs/AUTHENTICATION.md, docs/CI.md, docs/README.md (doc index).
- SCENARIO_FORMAT: storageState, saveStorageState, pauseForManual.
- FEATURES: Sprint 18 sections, 20-skill list, corrected limitations.
- ROADMAP: Sprint 18 entry + What's next.
- CONTRIBUTING: skill authoring, "keeping docs honest" count checklist.
- README / mcp-server README: accurate counts, docs index, auth example.

Corrected throughout: 51 tools (16 files), 7 binaries, 214 tests across 26
files, 20 skills. Every relative link resolves; every JSON/YAML block parses.

Docs and npm keywords only — no source, build, or test changes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ified

Two of the four documented install paths were dead, and the numbers the
README advertises were unenforced prose. This fixes the first and makes the
second structurally impossible to get wrong again.

Findings that changed the shape of the work:

* The npm publish job already existed. `release.yml` has carried a
  `publish-npm` job since the v1.4.0 hardening; it ran on the tag and failed
  in 26s with `E404 Not Found - PUT https://registry.npmjs.org/webmobai-mcp`.
  A 404 on a PUT is npm's signature for a token that cannot authenticate, so
  NPM_TOKEN is invalid/expired/read-only. This was a credential problem, not
  a packaging one. The publish itself is done from a logged-in machine.

* `webmobai-mcp --help` would have hung even once published. src/index.ts had
  no argv handling at all: it connected the stdio MCP transport and blocked
  forever on stdin. The command the README teaches for verifying an install
  could never have worked. Adds --help/--version, and drops a hardcoded
  "v1.2.0" startup banner that had drifted from the real 1.4.0 by reading the
  version from package.json instead.

* The three headline numbers were already correct. 51 tools, 7 binaries and
  214 tests all recounted true, so none of them were "fixed" — they are now
  verified instead of trusted.

Install paths:
* index.ts gains --help/--version, with 5 subprocess tests in which a
  regression hangs and the timeout is the assertion.
* release.yml publishes with --provenance, guards that the tag matches
  package.json, and smoke-tests the built entrypoint before publishing.
* server.json plus an `mcpName` in package.json, staged for the official MCP
  Registry. mcpName must be in the published tarball — the registry reads it
  from there to verify ownership, and npm versions are immutable — so it
  lands before the first publish, not after.

Links:
* github.com/modelcontextprotocol/sdk 404s; corrected to typescript-sdk.
* Placeholder domains across the skill docs (foo.com, bar.com, x.com) are
  real registrable domains that a link checker will chase. Replaced with RFC
  2606 reserved names so the new workflow needs no arbitrary exceptions.
* New docs-links workflow (lychee), modeled on the salman repo's. Every
  exclusion is a URL that cannot resolve by design, and says why in-file.

Numbers:
* scripts/check-doc-claims.mjs recounts tools, tool files, binaries, tests,
  test files, skills and the eval rates from the tree and checks 29 claims
  across 3 files. Verified to fail on injected drift. It caught the test
  additions in this very commit, which is why the README now reads 219/27.

Designed for agents:
* docs/DESIGNED_FOR_AGENTS.md explains why a selector miss returns a prior
  fingerprint, ranked candidates and page-state triage, with a transcript
  captured from the running code rather than illustrated.
* An 18-case eval measures it: 86.7% top-1 recovery, 93.3% top-3, 83.3%
  overall. It runs in CI and gates at an 80% floor. The corpus deliberately
  includes cases the tool fails, and the doc names all three — a suite that
  only contains passes measures the author's taste in examples.
* The eval runs against dist/, not src/. An early 0% reading was esbuild's
  keepNames helper leaking a `__name` reference into page.evaluate — a
  property of the loader, not the product. The shipped build is sound.

Hygiene:
* SECURITY.md, covering what storageState files actually contain (live
  session tokens), that they are never transmitted, and the disclosure path.
* CODE_OF_CONDUCT.md (Contributor Covenant 2.1).
* Social preview: a split frame — the request in plain English beside the
  real browser run with the assertion passing. HTML source kept in-repo so
  the image is reproducible; it carries no counts, because a number baked
  into a PNG is the one claim check-doc-claims cannot reach.
* Release matrix extended to Linux and Windows. Ubuntu 22.04 rather than
  latest, so the AppImage does not silently raise the glibc floor.

Still open: the npm publish itself, the homepage field (points at the npm
package once it exists), uploading the social preview (GitHub has no API for
it), and the registry submissions, which are staged in
docs/MCP_DIRECTORY_SUBMISSIONS.md and deliberately not sent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ixfhPuvZGgvevYHsJRp2V
The first run of this job did its job and failed: 250 links checked, 3
errors, all three `blob/main` URLs in mcp-server/README.md pointing at
docs/AUTHENTICATION.md and docs/CI.md.

Those links are correct. The files exist — on this branch. They are not at
`blob/main` until the branch merges, which makes the failure circular: the
check blocks the pull request that would make the check pass. Waiting it out
would mean merging with a red check and trusting that it goes green, which is
the habit this workflow exists to break.

The links cannot simply become relative. mcp-server/README.md is the npm
package page, rendered on npmjs.com, where `../docs/CI.md` resolves to
nothing — absolute URLs are right for the reader.

So remap this repository's own blob URLs onto the checked-out working copy.
That tests the invariant actually worth testing — the file this link points
at exists in this repository — which holds both before and after the merge,
and still fails loudly when a link names a file that genuinely is not there.
Links to any other repository are still fetched over the network.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ixfhPuvZGgvevYHsJRp2V
The first green run was preceded by a red one whose only error was
`[ERROR] https://tauri.app/ | Connection failed` — a link that is fine.
Re-running passed with nothing changed.

A gate that fails for reasons unrelated to what it guards teaches everyone to
re-run it without reading it, and a check nobody reads costs time and buys
nothing. The main cause here is concurrency: lychee fires up to 128 requests
at once by default, and aiming that at a couple of dozen hosts produces
connection resets and rate-limit refusals that are indistinguishable from
dead links.

Caps concurrency at 16 (250 links still finish in seconds), retries harder
before declaring a link dead, and sends a real identifiable user agent so
CDNs that refuse unknown agents give an honest answer instead of a refusal.

A 404 still fails the build. Only the noise is gone.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011ixfhPuvZGgvevYHsJRp2V
@celikgo celikgo changed the title docs: comprehensive skills + documentation overhaul for v1.4.0 docs + fix: v1.4.0 documentation overhaul, repaired install paths, CI-verified claims Aug 19, 2026
@celikgo
celikgo merged commit 189332c into main Aug 19, 2026
5 checks passed
@celikgo
celikgo deleted the docs/skills-and-docs-overhaul branch August 19, 2026 23:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant