Skip to content

crawl-text: static About section + portfolio backlink on the demo page - #40

Merged
alloevil merged 2 commits into
masterfrom
crawl-text
Sep 8, 2026
Merged

crawl-text: static About section + portfolio backlink on the demo page#40
alloevil merged 2 commits into
masterfrom
crawl-text

Conversation

@alloevil

@alloevil alloevil commented Sep 8, 2026

Copy link
Copy Markdown
Owner

Why

frontend/dist is a Vite SPA build, so the raw HTML a non-JS fetcher receives was 93 chars of text — everything visible came from assets/index-*.js. Google executes JS sometimes, with delay; LLM crawlers and answer-engine fetchers generally do not. For retrieval the landing page was effectively blank. Separately, no page on this site linked back to the portfolio hub.

Generated vs hand-maintained

The publish root is generated: .github/workflows/pages.yml runs cd frontend && npm ci && VITE_DEMO=1 npm run build and uploads frontend/dist, which is gitignored. So the source of truth is frontend/index.html (+ frontend/src/index.css for styling) and that is what is edited here, not the build output. frontend/public/ (robots.txt, sitemap.xml, llms.txt, llms-full.txt, claims.json) is untouched and still copied into dist verbatim.

What was added

A static <section id="about"> in frontend/index.html, placed after <div id="root"></div> (index.html line 41) — deliberately outside the mount container, because ReactDOM.createRoot(document.getElementById('root')) in src/main.tsx replaces that container's children on mount, so anything inside would be destroyed. Verified in the live DOM: root.contains(about) === false, about.parentElement === BODY.

Content is a genuine "what is this" section for a first-time visitor, worded from README.md and frontend/public/llms-full.txt so the two do not contradict: what AgentXRay is (local-first reader, not a tracing SDK), the seven registered log-format adapters with their default directories (OpenClaw, Codex, Claude Code, OMP, DeepSeek Harness, Gemini CLI, Hermes) plus a pointer to the PLATFORMS registry, the per-turn tool-call / token / cost ledger and the inference-vs-tool-execution time split, the real install command npx @alloevil/agent-xray, the Node.js 22.13+ requirement, and an explicit note that this deployment runs on synthetic sample logs. The only number is the 7 adapters, which is the published claim in frontend/public/claims.json; verified against the source:

$ node -e 'console.log(Object.keys(require("./lib/platforms/index.js").PLATFORMS).join(", "))'
openclaw, codex, claude-code, omp, dsh, gemini, hermes

Styling lives in src/index.css under #about (Tailwind @apply, existing dark-theme tokens only) rather than as inline style soup. Tailwind already scans ./index.html, so the rules land in the built stylesheet.

Making it genuinely reachable, not just present

This is the part that needed care, and it is the reason two commits are here.

src/index.css previously had html, body, #root { height: 100% } plus body { overflow: hidden }. Left alone, the new section would have been laid out below a viewport-height #root inside a body that cannot scroll: present in the DOM, visible to a text extractor, and unreachable for a human. That is cloaking by accident. So:

  • #root is pinned to exactly one viewport (height: 100vh; height: 100dvh) and body no longer hides its overflow. The dashboard therefore looks and behaves as before above the fold, and the section sits below it like an ordinary footer.
  • The first commit also added overscroll-behavior: contain to the dashboard's scroll panes, to stop a pane that hits its end from shoving the whole app up the page. The second commit removes it, because it was wrong. overscroll-behavior applies to every scroll container, including the overflow: hidden layout wrappers, and it blocks the chain even when a pane has nothing to scroll. Measured on the built page at 1440x900, where the sample data does not overflow any pane: wheeling over the centre of the app moved the page 0px with the rule in place and 1000px with it neutralised at runtime. The section was reachable only by scrollbar or keyboard — the exact failure this pass exists to prevent. Default chaining is the honest behaviour and is what ships.

No display: none, no <noscript>-only text, no keyword block, nothing that differs from what a visitor sees. Confirmed on the built page: display: block, visibility: visible, opacity: 1, 2319 chars of innerText.

Backlink

In the section's footer, beside the repo and npm links:

<a href="https://alloevil.github.io/projects/">More projects by allo</a>

Confirmed in the built DOM as exactly that. English, matching the section's lang="en" and the page's existing English demo banner / README / llms-full.txt.

Verification

Real build, not a simulation — cd frontend && npm ci && VITE_DEMO=1 npm run build, which is tsc -b && vite build, so TypeScript compiled clean too.

file before after
frontend/dist/index.html (built) 93 2402
frontend/index.html (source) 93 2402

Measured with the contract command (<script>/<style> stripped, tags removed, whitespace collapsed).

Served the built dist over plain HTTP at the /AgentXRay/ base and drove it in a real browser at 1440x900, 1440x420 and 390x780:

  • App still mounts and renders. #root has children, #root h1 is "AgentXRay", all four tabs render (会话 / 分析 / Prompts / 资产库), the demo banner and platform bar render, and the sessions view renders a sample session with tool calls and the per-turn ledger. Screenshots taken of the dashboard and of the section.
  • Dashboard geometry unchanged. #root computed height equals window.innerHeight at every viewport tested (900px / 420px / 780px), and the gap between #root's bottom edge and the section's top edge is exactly 0px. 100dvh is supported and applied; the 100vh line is the fallback for engines without dvh, and it reproduces the previous height: 100% behaviour exactly, since 100% of a 100%-height html/body chain resolved to the same large-viewport height.
  • No double scrollbar. Only the document scrolls: getComputedStyle(body).overflowY === 'visible' and body.scrollTop is not movable.
  • Reachable by the obvious gesture. Wheeling over the centre of the app scrolls the page to the bottom and brings the section into view at both 1440x900 (0 → 1007px, its full range) and 390x780 (0 → 1575px). End also works.
  • Panes scroll normally. At 1440x420 the message pane overflows by 299px: one 60px wheel notch moves it exactly 60px, continued wheeling reaches exactly its 299px maximum with no overshoot and no stuck boundary, and only once it is exhausted does the wheel carry on to the page. Nothing inside the dashboard is contained or blocked; overscroll-behavior no longer appears anywhere in the built CSS.

No formatters, no linters, no full suite. test/ contains only server-side tests; nothing in it references index.html, frontend/dist, or the site artifacts, and this change is frontend-only.

Not regressed

robots.txt, sitemap.xml, llms.txt, llms-full.txt, claims.json, the JSON-LD block, rel=canonical and the og:/twitter: tags are all unchanged and all present in the build. No page added, so sitemap.xml is untouched.

gaoruilin added 2 commits September 8, 2026 15:18
The Pages demo is a Vite SPA, so the raw HTML that non-JS fetchers see was
93 chars of text. LLM crawlers and answer-engine fetchers generally do not
execute JS, which made the landing page effectively blank for them.

Add a real "About AgentXRay" section to frontend/index.html, placed after
#root so React cannot overwrite it on mount. It is ordinary visible page
content, not hidden markup: the page now scrolls past the dashboard to reach
it. #root is pinned to one viewport (100dvh) so the dashboard itself looks
and behaves exactly as before, and its scroll panes get
overscroll-behavior: contain so they no longer drag the page along.

Content is drawn from README.md and frontend/public/llms-full.txt: what the
tool is, the seven registered log-format adapters with their default
directories, the per-turn token/cost/time ledger, the npx install command,
and the fact that this deployment runs on synthetic sample logs. No new
numbers beyond claims.json.

The section footer carries the repo link, the npm package and a link to the
portfolio hub at https://alloevil.github.io/projects/.

Built text length (frontend/dist/index.html, scripts and styles stripped):
93 -> 2402 chars.
Follow-up review question turned up a real defect in my own change. The
overscroll-behavior: contain rule was meant to stop the dashboard's scroll
panes from dragging the page when they hit their end. It also applies to
every other scroll container, including the overflow-hidden layout wrappers,
and it blocks the scroll chain even when a pane has nothing to scroll.

Measured on the built page at 1440x900, where the sample data does not
overflow any pane: wheeling over the centre of the app moved the page 0px
with the rule in place, and 1000px with it neutralised. The About section was
present, laid out and reachable only by scrollbar or keyboard — that is
exactly the present-but-unreachable shape this pass exists to avoid.

Default chaining is the honest behaviour: panes scroll normally, and once
exhausted the wheel carries on to the page and reveals #about.
@alloevil
alloevil merged commit 6bdb16a into master Sep 8, 2026
3 checks passed
@alloevil
alloevil deleted the crawl-text branch September 8, 2026 06:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant