Skip to content

Preserve partial API updates and reliably track package failures - #2048

Merged
David Pine (IEvangelist) merged 6 commits into
mainfrom
ievangelist-validate-tolerate-failed-packages
Nov 4, 2026
Merged

David Pine (IEvangelist) merged 6 commits into
mainfrom
ievangelist-validate-tolerate-failed-packages

Conversation

@IEvangelist

@IEvangelist David Pine (IEvangelist) commented Nov 4, 2026 •

Copy link
Copy Markdown
Member

Summary

Fix the complete package-failure path exposed by updater run 41827585564, rather than only suppressing its next validation error. Merge current main into this branch to resolve the conflict left by the squash merge of #2047.

  • Seed staging from committed API artifacts so failed packages actually retain their last-good data. When TypeScript generation fails after C# advances, restore both artifacts as a matching pair and rebuild the declarations.
  • Persist each failure's package, exact catalog version, generation stage, and reason in src/frontend/src/data/api-generation-failures.json. Both the updater and the generated data PR's CI use these narrowly deferred updates; other identity, provenance, exported-module, and declaration checks remain strict.
  • Retry pending failures even when catalog versions have not changed; clear the report on recovery.
  • Emit diagnostics immediately after each generator, before later phases can abort. Report package failures regardless of update-step success; do not close the issue after an incomplete generation pass.
  • Reuse one tracking issue, including reopening it on recurrence. Unchanged failures do not produce comments; changed diagnostics are preserved in issue comments, not just linked to expiring run logs.
  • Treat a completed C# batch's expected exit code 1 as partial success, preserving its per-package statuses. Crashes, incomplete batches, and contradictory statuses remain fatal. An all-package restore failure still reaches the final diagnostics and last-good-data preservation.
  • Add isolated tests that execute the actual PowerShell orchestrator and semantic validator with mocked external package generation; execute the actual C# batch reconciliation against completed-partial, crash, and incomplete fixtures; cover issue creation, deduplication, updates, closure, recurrence, and API errors.

Third-party links and affiliations

None. Example URLs in tests are non-network fixture data.

Validation

From src/frontend:

  • node node_modules\vitest\vitest.mjs run --config vitest.structured-data.config.ts — passed: 121 tests across 6 files, including 12 issue/orchestration regression tests.
  • node node_modules\eslint\bin\eslint.js scripts\report-api-generation-failures.mjs scripts\validate-generated-api-data.ts tests\unit\integration-data-updater.vitest.test.ts tests\unit\validate-generated-api-data.vitest.test.ts vitest.structured-data.config.ts --max-warnings 0 — passed.
  • node node_modules\tsx\dist\cli.mjs scripts\validate-generated-api-data.ts — passed on current committed API artifacts: 210 catalog identities and 140 TypeScript modules.

From the repository root:

  • pwsh -NoProfile -File tests\PackageJsonGenerator.Tests\generate-package-json.Tests.ps1 — passed: catalog-filter regression plus completed partial-batch, crash, and incomplete-output checks.
  • [System.Management.Automation.Language.Parser]::ParseFile(...) for all three modified generator/orchestrator PowerShell scripts — passed: no parse errors.
  • git diff --cached --check — passed before each commit.

No local production build or full live package regeneration was run. GitHub CI and an updater run on main after merge are still required before calling the live workflow fixed. The workflow was not dispatched on this PR branch, because doing so can update the real auto-merge data PR.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings November 4, 2026 12:24
…iled on aborts

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Critical carry-forward flaws can delete failed packages’ existing API data, misclassify successful packages as failed, and leave validation rejecting stale or duplicate outputs.

5 open issues
What changed in this PR

This PR aims to let integration-data updates continue when individual packages fail API generation, while keeping unrelated validation failures blocking.

Changes:

  • Records package failure reasons and adds preservation logic for existing C# API files.
  • Passes failed package names to validation and adds a tolerance regression test.
  • Reports generation failures in update PRs and a dedicated tracking issue.
File Description
src/​tools/​PackageJsonGenerator/​generate-package-json.ps1 Makes package failures non-fatal and adds failure details and preservation logic.
src/​frontend/​tests/​unit/​validate-generated-api-data.vitest.test.ts Adds a regression case for failed-package validation tolerance.
src/​frontend/​scripts/​validate-generated-api-data.ts Exempts named failed packages from C# missing/stale checks.
src/​frontend/​scripts/​update-integration-data.ps1 Collects failures, passes validation exemptions, and adds PR reporting.
.github/​workflows/​update-integration-data.yml Creates, updates, or closes an API-generation failure tracking issue.

Review effort: Balanced

@aspire-repo-bot

Copy link
Copy Markdown
Contributor

Frontend HTML artifact ready

The latest frontend build uploaded the frontend-dist artifact for PR #2048. Use the VS Code button below to open this PR with GitHub Artifacts Explorer and browse the built HTML locally.

VS Code: Open PR #2048 artifacts

This comment updates automatically when a new frontend build artifact is uploaded.

Persist exact package failures for CI and retries, retain matching last-good API artifacts, and report diagnostics independently of publication. Exercise the real orchestration and validation paths with isolated package-generation fixtures.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@IEvangelist David Pine (IEvangelist) changed the title Let API validation tolerate packages that failed generation Preserve partial API updates and reliably track package failures Nov 4, 2026
Honor per-package results when a completed batch returns its expected failure exit code. Keep missing completion, incomplete statuses, and crashes fatal; allow restore-only failures to reach diagnostics and preservation. Add direct batch reconciliation regressions and an all-package-failure orchestration case.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@IEvangelist
David Pine (IEvangelist) merged commit 555f41b into main Nov 4, 2026
20 checks passed
@IEvangelist
David Pine (IEvangelist) deleted the ievangelist-validate-tolerate-failed-packages branch November 4, 2026 13:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants