Skip to content

feat(coding-agent): clean webfetch HTML with reader mode#64

Merged
code-yeongyu merged 1 commit into
mainfrom
code-yeongyu/webfetch-reader-mode-sync
Jun 22, 2026
Merged

feat(coding-agent): clean webfetch HTML with reader mode#64
code-yeongyu merged 1 commit into
mainfrom
code-yeongyu/webfetch-reader-mode-sync

Conversation

@code-yeongyu

@code-yeongyu code-yeongyu commented Jun 22, 2026

Copy link
Copy Markdown
Owner

Summary

  • Sync the merged pi-webfetch reader-mode implementation into senpi's vendored webfetch extension.
  • Convert HTML markdown/text responses through @mozilla/readability + jsdom before fallback conversion, so article content is returned without page chrome.
  • Add a regression test for markdown and text output, and refresh package lock/shrinkwrap metadata for the new dependencies.

External upstream work: code-yeongyu/pi-webfetch#1

QA

  • npx tsx ../../node_modules/vitest/dist/cli.js --run test/suite/webfetch-reader-mode.test.ts
    • Evidence: local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-webfetch-reader-test-6.txt
  • npm run check
    • Evidence: local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-check-4.txt
  • node .agents/skills/senpi-qa/scripts/lib/common.mjs --self-check
    • Evidence: local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-qa-common-self-check.txt
  • node .agents/skills/senpi-qa/scripts/mock-loop.mjs --with-tool
    • Evidence: local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-qa-mock-loop-with-tool.txt
  • Real CLI webfetch QA with isolated fake model + local article server + --permission webfetch=allow.
    • Evidence: local-ignore/qa-evidence/20260622-webfetch-reader-mode/senpi-webfetch-cli-qa-3.txt
    • Observed: actual tool result contains ## Readable Fixture Article, Alpha fixture paragraph, and Beta fixture paragraph; excludes title/header/nav/aside/footer/script chrome.
  • Pre-commit hook passed: npm run check, node scripts/verify-package-managers.mjs for npm/bun/pnpm install+build, and browser smoke.

Review

  • Code review status: clear/approved after replacing the initial incomplete CLI QA artifact.
  • Audit note: npm audit --omit=dev still reports one high undici advisory under pre-existing @earendil-works/gondolin/node_modules/undici; the new jsdom subtree resolves undici@7.28.0.

Summary by cubic

Run HTML through reader mode before conversion so webfetch returns the main article (not page chrome) when requesting markdown or text. This improves readability and consistency across sites.

  • New Features

    • Use @mozilla/readability with jsdom to extract article content before markdown/text conversion; include the title if missing and avoid duplicates.
    • Pass the fetched URL into the converter for correct base URL handling, and add tests covering markdown and text outputs.
  • Dependencies

    • Add @mozilla/readability, jsdom, and @types/jsdom.
    • Update package-lock.json and npm-shrinkwrap.json.

Written for commit feedcb8. Summary will update on new commits.

Review in cubic

@code-yeongyu code-yeongyu merged commit 254da6b into main Jun 22, 2026
3 checks passed
@code-yeongyu code-yeongyu deleted the code-yeongyu/webfetch-reader-mode-sync branch June 22, 2026 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant