Skip to content

Weekly research scan [2026-10-09] - #179

Merged
jgoldfoot merged 2 commits into
mainfrom
research-scan-2026-10-09
Oct 9, 2026
Merged

jgoldfoot merged 2 commits into
mainfrom
research-scan-2026-10-09

Conversation

@jgoldfoot

Copy link
Copy Markdown
Owner

Automated weekly research scan. Draft for human review. Do not merge without checking the Verification section.

Window covered: 2026-09-27 → 2026-10-09. The previous scan PR (#175) is still open as a draft. This branch is cut from main and does not include #175's changes. Its edits were placed outside #175's hunks so the two can merge in either order (see "Merge order" below).

Findings

Category Claim Primary source
[STALE] Whitepaper §9 and the README quick-start tell sites to announce MCP servers with <link rel="alternate" type="application/mcp+json">. No MCP specification defines that element or media type. On 6 Oct 2026, SEP-2127 "MCP Server Cards" reached Final (Extensions Track) and standardized discovery a different way: a Server Card (application/mcp-server-card+json), listed in an AI Catalog at /.well-known/ai-catalog.json. The SEP considered DNS-based and header-based discovery and rejected both. HTML <link> discovery is not mentioned. SEP-2127 · PR #2127 · discovery.md
[CITATION] WebMCP's declarative API was removed from the spec on 8 Oct 2026. This covers the toolname / tooldescription / toolautosubmit attributes on <form>. The spec now covers only imperative registerTool(). The declarative work continues only in an explainer, where schema synthesis and response handling are still TBD. PR #338 · declarative-api-explainer.md · spec
[CITATION] Meta Ray-Ban Display was added to WebMCP's implementation-status.md on 29 Sept 2026 with "WebMCP for Web Apps is coming soon". It is off by default. implementation-status.md (commit for PR #325)
[NEW] "AX is the New AEO" is a field study of 37,927 agent journeys over 1,056 real businesses, using four harnesses with matched groups. Agent-ready businesses were clearly recommended 20% vs 11% (1.9×). Site-built answers got 48.3% vs 34.3% of asked facts right (p<0.0001). It is a correlational study, run by a vendor, and uses the vendor's proprietary readiness score. arXiv:2609.34951 (v3)

Proposed edits (all in this diff)

  • whitepaper §9, after the WebMCP Discovery paragraph: new paragraph "Standardized MCP Server Discovery (Server Cards)". It says the <link> is the framework's own convention, describes SEP-2127, and recommends a Server Card plus an AI Catalog entry as the standard mechanism. The in-page <link> is kept as an optional, non-standard hint.
  • whitepaper §5.1 evidence list, after BenchJack: new "AX is the New AEO (2026)" bullet with three explicit cautions: it is correlational, it uses a proprietary composite score, and the authors are the score's vendor. It is framed as directional evidence for Layers 1 and 3, not a BiModal test.
  • whitepaper References → Agent Protocols: new items 6 (SEP-2127) and 7 (WebMCP October changes, supplementing item 2).
  • README quick-start code comment above the <link> example: points to SEP-2127 as the standard mechanism. The <link> line itself is unchanged.
  • README References: three new bullets after "MCP Roadmap".

Items that most need a human look

  1. The <link rel="alternate" type="application/mcp+json"> recommendation (framework claim). It now has an official competitor. I did not remove or rewrite it. I added text that labels it non-standard and points to Server Cards as the standard route. Whether to keep it at all, demote it further, or replace the README example with an ai-catalog.json snippet is a framework decision. The same <link> pattern likely appears on bimodal.design (see the website issue).
  2. Declarative WebMCP removal. The framework never cited declarative WebMCP, so nothing it says is now wrong. Still, the declarative form was the most natural bridge between Layer 2 (semantic forms) and Layer 5 (tools). Its removal means WebMCP is now JS-only. The explainer still exists and Chromium is still prototyping it, so this is a narrowing, not a kill. You may want to say so explicitly in the Layer 5 section.
  3. "AX is the New AEO" is vendor research. All four authors are at ora (ora.ai), and "agent-ready" is ora's own ranker score. I kept it because it is the largest real-world evidence on the framework's core question, and because it reports matching, multiple harnesses and its own limitations. If you'd rather not cite vendor studies, drop the bullet; nothing else depends on it.

Verification

  • SEP-2127: read the SEP page itself. It shows Status Final, Type Extensions Track, created 2026-01-21, and lists the authors. The well-known path and both media types were read from the normative docs/discovery.md in experimental-ext-server-card. The 6 Oct merge date comes from the repo's closed-PR listing (GitHub API), which shows merged_at 2026-10-06. Refutation pass: an earlier search summary claimed the path was /.well-known/mcp/server-cards.json. That is wrong. The SEP defines only /.well-known/ai-catalog.json (catalog) and reserves <streamable-http-url>/server-card (card). Only the primary-source values are used. I also checked whether discovery.md defines any HTML <link> mechanism. It does not.
  • WebMCP PR #338: the PR page shows it was merged by domfarolino on 8 Oct, after approvals from anssiko, markafoltz and domfarolino. Its description says "The spec now focuses on the imperative registerTool() API. Declarative WebMCP is still being explored in the Declarative API Explainer." The live spec (header "9 October 2026") has no Declarative section. The attribute names and TBD status were read from declarative-api-explainer.md.
  • Meta Ray-Ban Display: both quoted sentences are verbatim from implementation-status.md at main. The 29 Sept date and the #325 reference come from the file's commit history (GitHub API).
  • arXiv:2609.34951: metadata from the arXiv API (v1 published 2026-09-28T11:41Z, updated 2026-10-06, cs.AI/cs.IR). The authors' affiliations, the instrument definition and the limitations were read from the v3 HTML. The quoted figures are verbatim from the body: "agent-ready businesses get one 20% of the time against 11%, or 1.9×", "Site-built answers get 48.3% of asked facts right against 34.3% (p<0.0001)", and "An intervention study ... would settle it." Skeptic pass: the p-value attaches to the accuracy figure, not to the 1.9×, so it is placed accordingly.
  • Checked, no change: MCP spec releases (still 2026-07-28); A2A (still v1.0.1, 28 May); WAI-ARIA (editorial and chore commits only since 27 Sept); WCAG 3.0 (still the 10 Sept WD, which is in Weekly research scan [2026-09-27] #175); schema.org (v30.1 on 15 Sept, no release notes); OpenAPI 3.2.1 (10 Sept, outside the window, and the framework cites 3.x generically). WebMCP #339 (UnknownError → specific DOMException types) and #335 (typos) are editorial and were not proposed.

Cut during the skeptic pass

  • "78% vs 56%" first-party share (arXiv:2609.34951). The abstract says 56% but the body (§5.3) says 58% for the same comparison. Because of that internal inconsistency, neither figure is cited.
  • OSWorld-Verified "October 2026" rankings (Qwen3.8 Max 86.1%, etc.) came from secondary aggregators (benchlm.ai, codesota) and were not checked against the maintainers' results file. Not used. Weekly research scan [2026-09-27] #175 already carries the file-verified figures.
  • WebMirage (arXiv:2610.09240): 91.9% attack success on vision-grounded web agents. It is relevant to vision-only fragility, but it makes no comparison with DOM or accessibility-tree agents, so it does not bear on a framework claim.
  • HybridCUA (arXiv:2609.38008): 53.6% on OSWorld with GUI+CLI orchestration. This is an agent-training result, the same category as CUA-Universe, with no interface claim.
  • Auditing WebArena-Lite (arXiv:2610.01491): human review recovered 5.45–8.49 pp that automatic evaluators missed. It supports the existing "directional" framing but was not needed. Noted for a possible future benchmark-reliability passage.
  • SEP-2127 "voted Accept on 8/13/2026": this came from a summarizer, and I could not confirm it from a primary page. Not used.

Merge order

This branch is independent of #175. Its edits sit outside #175's hunks: whitepaper §5.1 after BenchJack (#175 inserts after CUA-Universe), §9 after the WebMCP Discovery paragraph, new reference items 6–7, and README references after "MCP Roadmap". Either order should merge cleanly. Once both are in, consider folding reference item 7 into item 2 and adding AX as a numbered research reference (#29 after #175's 26–28).

Settled conventions preserved

  • Benchmark numbers stay directional projections. The AX study is explicitly framed as "not a test of BiModal Design".
  • data-agent-* stays supplementary. Untouched.
  • npm, not pnpm. DRIFT.md is not modified: the <link> vs. Server Card question is flagged above for decision rather than logged as drift.

Checks

  • npm test: 155 passed, 7 suites.
  • npm run lint / format:check were not run, because they are outside this routine's allowed command list. Prose was hand-wrapped at 80 columns to match the files, but a Prettier pass may still want to re-wrap.
  • Only README.md and docs/whitepaper.md touched. No changes to package.json, package-lock.json, or .github/workflows.

🤖 Generated with Claude Code

jgoldfoot and others added 2 commits October 9, 2026 09:07
MCP Server Cards (SEP-2127, Final 6 Oct) as the standard Layer 5 discovery mechanism; WebMCP declarative API removed from spec (PR #338) and Meta Ray-Ban Display added to implementation status (PR #325); AX is the New AEO (arXiv:2609.34951) as caveated directional evidence for Layers 1 and 3.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jgoldfoot
jgoldfoot marked this pull request as ready for review October 9, 2026 16:13
@jgoldfoot
jgoldfoot merged commit 21e0440 into main Oct 9, 2026
16 checks passed
@jgoldfoot
jgoldfoot deleted the research-scan-2026-10-09 branch October 9, 2026 16:13
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-09T16:15:35.099484Z 8642907 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8642907f41

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread README.md
Comment on lines +299 to +301
Standard mechanism (SEP-2127): publish an MCP Server Card and list it in
/.well-known/ai-catalog.json. The <link> below is a non-standard in-page
hint for agents already reading the DOM. -->

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Update all MCP discovery guidance together

A repo-wide search shows that this new disclaimer conflicts with the primary implementation guidance: docs/implementation-guide.md:537-552 still calls the same application/mcp+json link “standardized,” and examples/agentic-browser-web-mcp.md:12-14 and examples/mcp-discovery-and-safety.md:10-12 still claim agents can use it for discovery. DRIFT.md:12-31 also remains stale and says the README and whitepaper have not yet been corrected. Consequently, readers following the guide or examples will still implement the undefined mechanism rather than Server Cards; update those artifacts and the living drift record as part of this correction.

AGENTS.md reference: AGENTS.md:L253-L261

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant