Repository navigation
Prototype build-time Pele diagrams and measure build impact - #2058
David Pine (IEvangelist) wants to merge 1 commit into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
David Pine (IEvangelist)
added
the
build-profile
Frontend HTML artifact readyThe latest frontend build uploaded the This comment updates automatically when a new frontend build artifact is uploaded. |
There was a problem hiding this comment.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
Note
This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.
David Pine (IEvangelist)
left a comment
David Pine (IEvangelist)
left a comment
There was a problem hiding this comment.
[partial — phase A: 4 of 4 claims evaluated; phase B: seven affected documentation pages tested, sample README and inheritance SVG coverage incomplete]
Inspected head: e8dc9669af931220dd3c5399bc83273d37dd1c8a; target main at 264a00fcbf9e435082a687e46a1addd14d814b7d. This is a partial COMMENT, not approval. The independent tester exercised ten visible diagrams on seven affected documentation routes, but did not obtain actual sample README or generated C# inheritance SVG coverage. Those two changed rendering paths remain unverified, not passes. Production performance/size measurements were not independently reproduced.
Source refs: The changed prose describes the docs site's own renderer, owned by microsoft/aspire.dev, inspected at the PR head above and compared with the pinned target. No new core Aspire, CommunityToolkit, or Azure SDK claim was extracted, so no unrelated product-source ref was substituted. Trusted reviewer/tester/browser policies were read from default-branch commit 264a00fcbf9e435082a687e46a1addd14d814b7d. Native exact-URL skill invocation was unavailable; the frozen trusted repository-file workflow was followed.
Phase A: 4 site-renderer claims; 4 verified, 0 verified-with-nuance, 0 unverifiable, 0 contradicted, 0 narrative. All 17 changed-file diffs were read. Independent Phase B: 8 passed check groups, 0 critical issues observed, 3 warnings, and 2 affected content classes unverified. Browser observations do not establish baseline regression attribution.
Phase A — Claim verification
All four contributor-guide claims with inspected source evidence
- **A1 — verified**, contributor-guide.mdx:549: You can write Mermaid diagrams as code blocks.src/frontend/config/remark-mermaid.mjs:5-26 traverses nested nodes and handles node.type=code with node.lang=mermaid at :9. src/frontend/astro.config.mjs:13,94 imports and registers that plugin. Contributor-guide.mdx:551-571 includes the literal fence and rendered example. This is source verification of supported Mermaid fences, not exhaustive validation of every Mermaid dialect.
-
A2 — verified, contributor-guide.mdx:549: Pele renders these diagrams to SVG during page generation
src/frontend/src/utils/mermaid.mjs:4,12-29 imports render from pele and returns render(...).svg. config/remark-mermaid.mjs:15-16 inserts that SVG as HAST during Markdown transformation. src/frontend/src/components/SampleReadmeBlocks.astro:228 and api-reference/InheritanceDiagram.astro:75 call the same helper during Astro rendering. src/frontend/package.json pins pele1.12.0. No production build or full production route coverage was run.
-
A3 — verified, contributor-guide.mdx:549: their colors follow the selected site theme without browser-side re-rendering
src/frontend/src/styles/mermaid.css:15-27 binds Pele color variables to the site's theme variables, with existing light-theme overrides at :65-84. src/frontend/src/components/starlight/Head.astro:57-66 no longer imports the browser Mermaid script and preserves the selected theme on client swaps. The retained src/scripts/mermaid.ts is not imported by the changed site head. Browser behavior is an independent Phase B result, not inferred here.
-
A4 — verified, contributor-guide.mdx:549: Architecture diagrams support the
logos,iconoir, and customaspireicon packs.src/frontend/src/utils/mermaid.mjs:1-8 imports and registers logos/iconoir/aspire. :19-27 splits qualified names, resolves the pack icon, converts it to SVG, replaces IDs, and throws explicitly for an unknown icon. src/frontend/package.json and bun.lock contain the two Iconify packages; public/icons/aspire.json is the local custom pack. The registration is verified; every individual icon/diagram is not claimed browser-tested.
Audit links at the exact inspected head: Markdown fence transformer, Pele renderer and icon resolver, theme variables, shared head without browser renderer import, sample diagram call site, and inheritance diagram call site.
No contradicted or unverifiable source claim was found, so no claim-based inline finding is required. The benchmark workflow, comparison script, six changed test files, and dependency lock changes were read; their assertions and the PR body's CI/performance numbers were not credited as independently executed results.
Reader-facing observations and limits
The independent report below flags mobile labels shrinking to roughly 5.2–7.5px effective text, missing concise accessible diagram names/descriptions, and incomplete teaching for the newly named icon packs. It also explicitly records a sample index that returned a not-found page and a bounded C# type search that exposed HTML inheritance rows but no actual inheritance SVG. No source explanation, replacement route, baseline equivalence, or newly introduced regression is inferred from those observations. The full report preserves the source-blind tester's wording.
The exact head was served with root bun run dev in this automation's isolated checkout. Missing pele, logos, and iconoir packages were restored using the pinned lockfile with --frozen-lockfile --ignore-scripts; no package lifecycle hooks ran. Bun 1.4.4 was used versus declared 1.4.3; Astro was 7.4.1. Credentials were removed from the restore/dev subprocesses. No full site build, benchmark, comparison workflow, PR-provided test suite, product/AppHost command, deployment, global setup, or certificate operation was run. Development rendering is not production build-time/memory/output-size validation or exhaustive diagram compatibility proof.
Phase B — Complete independent source-blind doc-tester report
Documentation Test Report
Focus area: Independent, source-blind Phase B diagram testing for microsoft/aspire.dev PR #2058.
Date: 2026-11-05
Tester: doc-tester agent, browser-only behavioral observation
Requested inspected head: e8dc9669af931220dd3c5399bc83273d37dd1c8a
Target: main
Frozen policy baseline: 264a00fcbf9e435082a687e46a1addd14d814b7d
Origin: http://localhost:4321/
Summary
| Category | Passed check groups | Critical issues | Warnings |
|---|---|---|---|
| Actual affected diagram rendering | 1 | 0 | 0 |
| Desktop labels, icons, and arrows | 1 | 0 | 0 |
| Light/dark appearance and stable SVG markup | 1 | 0 | 0 |
| Mobile containment and readability | 1 | 0 | 1 |
| Diagram accessibility metadata | 0 | 0 | 1 |
| Contributor teaching and example copying | 2 | 0 | 1 |
| Repeated linked navigation/history | 2 | 0 | 0 |
| Total | 8 | 0 | 3 |
Coverage limits: 2 affected content classes remain unverified: sample README diagrams and generated C# inheritance diagrams. These are not passes. Ten visible, top-level documentation diagrams were exercised on the seven requested documentation routes. No renderer failure was observed in that tested subset. Findings below are observations, not claims that the PR introduced regressions.
Independence and runtime boundaries
- Read only the two supplied frozen policies before browser testing. Did not read repository files, product source, tests, PR body/diff/comments, Phase A results, GitHub reports, or previous review results.
- All site access and DOM/clipboard observations used
playwright-cliin the named, in-memory Edge sessionpr2058-doc-test. No MCP browser, direct Playwright program, curl, fetch, module inspection, or filesystem-based implementation inspection was used. - Initial browser navigation returned
net::ERR_CONNECTION_REFUSED. The first browser retry after a 15-second wait succeeded; the 120-second startup allowance was not exhausted. - Parent supplied the running server and attests the checkout/head. This tester did not independently verify the checkout; page-footer SHA text was not treated as checkout proof.
- Used desktop viewports of 1440×1000 and, for the tall architecture diagram, 1440×1400; mobile viewport was 390×844, normal browser scale.
- Tested diagrams only. No installation, build, benchmark, workflow, cloud/container operation, AppHost, project creation, or unrelated code/example execution. No old or unrelated links were fetched.
- No baseline build comparison, production/static-build run, full screen-reader session, quantitative contrast audit, browser zoom/pinch test, or exhaustive language/progressive-diagram variant test. Hidden and nested icon SVGs were excluded from top-level visible-diagram counts.
Exact routes and evidence
All paths below were loaded on the local origin, not the public production site.
| Route | Visible top-level diagrams | Observed content | Evidence stem |
|---|---|---|---|
/community/contributor-guide/#mermaid-diagrams |
1 | Six-node build/provision/deploy flow | contributor-diagram-*, contributor-0-mobile-dark.png |
/get-started/app-host/ |
2 | React/API/PostgreSQL topology; endpoint/connection-string reference chain | app-host-0-*, app-host-1-* |
/architecture/multi-language-architecture/ |
1 | Guest process → JSON-RPC/host → managed resources | multi-language-complete-*, multi-language-0-mobile-dark.png |
/architecture/resource-hierarchies/ |
1 | Web/API/PostgreSQL reference chain | resource-hierarchies-0-* |
/architecture/resource-model/ |
1 | Web/API/database topology | resource-model-0-* |
/deployment/pipelines/ |
3 | App topology; deployment dependency graph; four-step static-site chain | pipelines-0-*, pipelines-1-*, pipelines-2-* |
/get-started/first-app/ |
1 | React frontend → Node.js API | first-app-0-* |
The browser added /get-started/app-host/?lang=csharp and /get-started/first-app/?aspire-lang=typescript during ordinary UI state synchronization/history. These variants were also observed.
Evidence directory: C:\Users\dapine\.copilot\session-state\b1028536-a0cc-461b-98cb-21c8f8fc8a89\files\pr-2058-browser\
Evidence stems ending in -* refer to the saved -light.png, -dark.png, and/or -mobile-dark.png files in that directory. The complete tall-diagram desktop images are multi-language-complete-light.png and multi-language-complete-dark.png; earlier tall-diagram captures had development-page overlays at the viewport edge and are not relied on for clipping claims.
Supporting browser-result transcripts:
contributor-theme-copy.txtdiagram-desktop-a.txt,diagram-desktop-b.txtdiagram-mobile.txtaccessibility-and-teaching.txtnavigation-history-settled.txtsamples-unavailable.yml,samples-unavailable.pngapi-discovery.txt,api-container-discovery.txt,api-third-type.txt,api-third-type.yml,api-third-type.pngbrowser-cleanup.txt
Critical Issues
None observed within the tested diagram subset. This does not certify the two unverified content classes or prove absence of regressions.
Warnings
W1 — Responsive containment makes wide diagrams too small to read comfortably
Locations: /get-started/app-host/ second diagram; /architecture/resource-hierarchies/; /deployment/pipelines/ second and third diagrams; contributor-guide Mermaid example.
Type/severity: Rendering/readability, Medium.
Reader-facing labels include: “Endpoint Reference”, “Connection String Reference”, “configure-storage-permissions”.
Reproduction: Open an affected route at 390×844 and scroll to its diagram without browser zoom.
Expected: Relationships and labels should remain practically readable on a phone, not merely stay inside the page width.
Actual: All diagrams stay within the viewport, but wide charts shrink to a 324px-wide SVG. The AppHost/reference-hierarchy chains become approximately 48px tall with about 5.2px effective label text; the static-site pipeline becomes approximately 25px tall with about 6.2px text. The contributor/deployment graph has approximately 7.5px labels. The screenshot evidence visibly shows these very small labels. These numbers account for the SVG viewBox-to-display scaling, rather than reporting its nominal font size.
The tested diagram containers exposed no built-in button/link zoom control. Browser zoom or pinch behavior was not tested; this is not a claim that browser-level zoom is impossible.
Evidence: diagram-mobile.txt, app-host-1-mobile-dark.png, resource-hierarchies-0-mobile-dark.png, pipelines-2-mobile-dark.png, contributor-0-mobile-dark.png.
Recommendation: Preserve a readable minimum diagram scale with accessible horizontal scrolling or offer an accessible expand/full-size view. Retain the successful page-level overflow containment.
W2 — Diagrams lack concise accessible titles/descriptions
Locations: All ten tested visible documentation diagrams.
Type/severity: Accessibility metadata, Medium.
Reproduction: Inspect the diagram’s rendered DOM/accessibility representation.
Expected: A reader using assistive technology can identify the diagram’s purpose and obtain an equivalent explanation of its relationships.
Actual: The outer SVGs have role="graphics-document document" but no aria-label, aria-labelledby, SVG title, or SVG desc. Node text remains exposed; this is not a claim that the diagrams are entirely hidden from assistive technology. For example, the tall architecture diagram’s accessibility snapshot is a document containing the concatenated process/component labels, rather than a concise title plus relationship description.
Evidence: contributor-dom.txt, diagram-desktop-a.txt, diagram-desktop-b.txt, accessibility-and-teaching.txt.
Recommendation: Provide meaningful diagram titles and descriptions or adjacent equivalent prose tied to each diagram; document how contributors should supply them. A manual screen-reader check would be needed to establish the full impact.
W3 — New icon-pack guidance names a capability without teaching its use
Location: /community/contributor-guide/#mermaid-diagrams
Type/severity: Teaching/documentation gap, Low.
Exact rendered statement:
Architecture diagrams support the logos, iconoir, and custom aspire icon packs.
Expected: A new contributor can discover how to use the newly named packs from the section or a directly provided reference.
Actual: The section demonstrates only graph TD with ordinary text nodes. It gives no architecture/icon example, icon identifier format, available-icon reference, or link explaining the three packs. It also names Pele without a section-local explanatory/reference link. The basic fenced example is clear, but it does not teach the additional icon capability it announces.
Evidence: accessibility-and-teaching.txt, contributor-snapshot.yml, contributor-theme-copy.txt.
Recommendation: Add one minimal architecture diagram example using a supported icon and link to valid pack/icon names and syntax. Do not require contributors to infer these from other pages or source.
Passed Checks
- Actual affected rendering: All seven requested documentation routes loaded; ten visible top-level diagrams rendered as SVG rather than showing raw Mermaid text or a syntax-error placeholder.
- Desktop readability: Inspected the light/dark screenshots. Labels, recognizable React/C#/.NET/Node.js/PostgreSQL icons, reference/server icons, connecting lines, and arrowheads were visible. No missing-icon placeholder or persistent diagram clipping was observed in the complete captures. This is a visual check, not a validation of unrelated product architecture semantics.
- Theme behavior: Light/dark controls visibly changed diagram colors. Exact outer-SVG markup stayed unchanged across the theme switch for all ten tested diagrams. In the contributor example, observed text fill changed from
rgb(31, 30, 51)torgb(255, 255, 255). This supports the observable theme/no-replacement behavior, not an implementation claim about when SVG generation occurs. - Mobile containment: At 390px, all seven pages had a measured document width of 390px; diagram bounds were inside the viewport. Readability is separately qualified by W1.
- Literal copy/readability: The contributor example’s copy control placed the complete fenced Mermaid block on the browser clipboard, including all five declared arrows. It was not executed in a new project or fed into a separate renderer.
- Basic contributor teaching: The section clearly identifies Mermaid fenced code blocks and puts a matching six-node rendered example immediately after “Renders as:”. A contributor can copy this basic flow example without inventing code.
- Repeated navigation: Used an actual “AppHost” link from the first-app page, then back and forward, for two complete settled cycles. The first-app page consistently returned one visible diagram; AppHost consistently returned two, with the expected labels and no accumulating visible copies.
- Stable history content: After navigation settled, page title, URL, and diagram contents agreed on each of the six sampled link/back/forward states. Immediate pre-settle reads briefly saw outgoing/initial content; those transient readings were not reported as renderer failures. See
navigation-history-settled.txt, not the preliminarynavigation-history.txt.
Coverage Limits and Inaccessible Behavior
L1 — Sample README diagram coverage unavailable
The permitted discovery route /samples/ displayed “Page not found | Aspire” with no sample links in its main content. Preserved samples-unavailable.png and samples-unavailable.yml.
Discovery bound: Index visited once; 0 sample detail pages visited, below the maximum of three. No actual sample README diagram was found, so no sample-renderer pass is claimed. Did not guess replacement routes or follow unrelated old links. The cause of the unavailable index is unknown to this source-blind phase and is not attributed to the PR.
L2 — Generated C# inheritance SVG not found within the allowed type bound
Used only the rendered /reference/api/csharp/ search UI to discover candidate URLs. Inspected exactly these three type pages:
/reference/api/csharp/aspire.hosting/resource/— “IMPLEMENTS IResource”; no inheritance SVG./reference/api/csharp/aspire.hosting/containerresource/— text/HTML “INHERITS Resource → ContainerResource”; no actual inheritance SVG./reference/api/csharp/aspire.hosting.postgresql/postgresserverresource/— text/HTML “INHERITS ContainerResource → PostgresServerResource”; no actual inheritance SVG.
Stopped at the three-type limit. A readable HTML inheritance row is not a pass for the changed SVG inheritance renderer. No additional type pages or source were inspected. No C# inheritance diagram light/dark/mobile/accessibility pass is claimed.
Other runtime observations
Browser console evidence contained /api/live/ 404 errors and Astro development-audit “Failed to fetch” errors. These did not stop the tested diagrams from appearing. No source investigation or claim of PR causality was made. Evidence: .playwright-cli\console-2026-11-05T16-56-46-156Z.log.
Knowledge Gaps
- Architecture/icon authoring: A new contributor needs valid syntax and icon names. Supplying an example without a rendered reference would require prior Mermaid/icon knowledge. That knowledge was not silently substituted; W3 records the gap.
- Generation timing: The docs say Pele renders SVG during page generation. Browser observation verified stable markup and changing colors only; it cannot establish build-time/server implementation mechanics.
- Untested content classes: The sample index failure and absent inheritance SVGs prevent an informed conclusion for those render paths. No source-based explanation is invented.
Recommendations
- Address W1’s mobile legibility while preserving successful containment.
- Add concise diagram naming/equivalent descriptions and contributor instructions for them.
- Extend the icon-pack paragraph with an executable-looking, copyable architecture example and supported-icon references.
- Treat sample README and generated inheritance rendering as unresolved coverage, not approval evidence; parent may account for these limits without reinterpreting this report as a pass.
- Do not infer checkout provenance, generation timing, or regression causality from browser output.
Cleanup and Persistence
playwright-cli -s=pr2058-doc-test close succeeded: “Browser 'pr2058-doc-test' closed” (browser-cleanup.txt).
Only the owned named browser was closed. Server, checkout, dependencies, global configuration, and other browser sessions were not changed. Browser evidence and this frozen report are retained outside the repository. No GitHub submission was performed. Phase B is complete; no rerun is requested or performed.
Code reviewInspected head: The docs predecessor was confirmed before this stage. Its documentation phases and browser testing were not repeated. The actual trusted repository Result: one confirmed important test-coverage gap below; no runtime correctness defect was established by this bounded code review. High
Verification and limits
Test coverage and skill coverage
|
Summary
Prototype replacing browser-side Mermaid rendering with build-time SVG generation using Pele 1.12.0. Keep existing Mermaid fences, Aspire containers, light/dark theme colors, and local
logos,iconoir, and Aspire icons. Architecture service boxes are enlarged to 120px, preserving the boxed appearance selected during local evaluation.The shared renderer covers documentation fences, sample READMEs, and generated API inheritance diagrams. The previous browser renderer remains in the source tree for prototype comparison but is no longer imported by the site.
Measured full-build baseline
Mermaid build comparison run — both full production builds passed on Linux, with the same 33,356 HTML files. The Windows asset-staging failures did not reproduce. Baseline:
264a00fcbf9e435082a687e46a1addd14d814b7d; candidate synthetic merge:c24a4d93882c9d9daef9a73341120d308e5ec365.Interpretation: no build-time regression was observed in this run, but a 1.3% timing difference is not evidence of a speedup. The clearest measured tradeoffs are lower emitted JavaScript, slightly more HTML, and a higher peak memory reading. Memory remains a consideration before adopting this broadly.
The opt-in workflow runs with
build-profileand compares the exact PR base SHA with the synthetic PR merge, sequentially on the same 8-core Ubuntu runner, with each frozen lockfile and no restored Astro cache. Both usebun run build:production; installation time is excluded. Raw timing/resource logs and output manifests are preserved in themermaid-build-comparisonartifact.Limitations: one cold run per variant, always baseline first. Run order and OS/filesystem caches can affect measurements. Sizes are uncompressed build output, not measured browser transfer sizes. Warm/incremental-build impact is not yet measured. The early 163ms isolated diagram-rendering result is not used as a full-build performance estimate.
Behavioral tradeoffs: diagram rendering, icon resolution, and SVG-to-HAST conversion now run during page generation. Invalid diagram sources fail compilation/page generation rather than showing a browser-only error. Theme changes only change CSS variables and do not render another SVG. Retain draft status while evaluating these changes and completing the broader CI checks.
Third-party links and affiliations
None. This prototype adds the MIT-licensed Pele dependency and local Iconify packages but does not add or change third-party links on the site.
Validation
CI: both comparison builds (
bun run build:production) passed. The ordinary PR production build, repository lint/unit-test job, AppHost build/tests, and CodeQL passed. Full desktop/tablet/mobile E2E shards were still running when these measurements were recorded; their outcome is not yet claimed.Local commands run from
src/frontend:npm exec --yes --package=bun@1.4.3 -- bunx vitest run --config vitest.config.ts tests/unit/pele-mermaid.vitest.test.ts tests/unit/remark-mermaid.vitest.test.ts tests/unit/custom-components.vitest.test.ts tests/unit/navigation-theme.vitest.test.ts tests/unit/mermaid-build-comparison.vitest.test.ts— passed: 117 tests across five files. Covers all existing documentation fences, local icons, MDX nesting, generic inheritance, 120px boxes, unique accessibility IDs, and comparison calculations.npm exec --yes --package=bun@1.4.3 -- bunx vitest run --config vitest.config.ts tests/unit/mermaid-build-comparison.vitest.test.ts— passed: 2 tests, rerun after final comparison-report edits.npm exec --yes --package=bun@1.4.3 -- bunx eslint config/remark-mermaid.mjs src/utils/mermaid.mjs scripts/summarize-mermaid-build-comparison.mjs tests/unit/remark-mermaid.vitest.test.ts tests/unit/pele-mermaid.vitest.test.ts tests/unit/navigation-theme.vitest.test.ts tests/unit/mermaid-build-comparison.vitest.test.ts tests/e2e/site-ui-navigation.spec.ts --max-warnings 0— passed.mermaid-navigation.spec.tsandsite-ui-navigation.spec.ts, run against the existing local preview using a session-local Playwright config — 14 passed, 4 intentionally skipped. Covers desktop/tablet/mobile, both themes, repeated client navigation and history; SVG markup stays unchanged on theme switches.npm exec --yes --package=bun@1.4.3 -- bun run build:skip-search— failed before page generation at Astro image asset staging with WindowsEBUSYerrors. Retrying with isolated output and Node reproduced the same error class; a cross-drive output attempt instead hitEXDEV. The subsequent Linux baseline and candidate production builds both passed.git diff --check— passed.