Skip to content

fix: restore community archive validation and compare submitted checksums - #4689

Merged
KSchlobohm merged 2 commits into
mainfrom
kschlobohm-sep-22-archive-fix-upstream
Sep 23, 2026
Merged

KSchlobohm merged 2 commits into
mainfrom
kschlobohm-sep-22-archive-fix-upstream

Conversation

@KSchlobohm

Copy link
Copy Markdown
Contributor

Description

Closes #4677.

Restore release-archive validation in the extension, preset, and bundle submission workflows. Hosted fork runs also exposed a checksum decision error: a submission containing an all-zero checksum was accepted despite the downloaded archive having a different digest.

This builds on #4622, which already added curl, HTTPS/timeout protections, direct-download evidence instructions, and GitHub archive firewall access. Those protections are retained, not replaced.

  • Add the missing explicit Copilot URL approvals, raw-content host access, and checksum-command permissions; align the bundle host list without unrestricted network or shell access.
  • Require actual archive-download evidence and distinguish environment blockers from invalid submissions. Metadata lookup is not a substitute for downloading the archive.
  • Preserve the optional submitted checksum separately from the computed digest, require an explicit comparison, reject malformed or conflicting submitted values, and record the downloaded archive's digest in new and updated catalog entries.
  • Apply bundle success labels in a deterministic conclusion step after successful catalog PR creation. Two valid-bundle runs omitted agent label requests; this step removes that dependency for the success-label update.

No submission-form changes, fork test catalog entries, or unrelated fork changes are included. This does not introduce an independent checksum enforcement gate.

Testing

Hosted fork evidence

Scenario Observed result
Preset, valid checksum Download and strict comparison succeeded; correct labels and catalog digest.
Preset, wrong checksum Rejected, success label removed, no catalog PR. See reporting limitation below.
Preset, no checksum Accepted; computed digest recorded in the catalog.
Extension, valid checksum Strict comparison succeeded; correct labels and catalog digest.
Extension, wrong checksum Strict comparison failed with exit 1; correct rejection labels and no catalog PR.
Bundle, wrong checksum Strict comparison failed with exit 1; correct rejection labels and no catalog PR.
Bundle, conflicting sources Explicitly rejected conflicting submitted checksums; no catalog PR.
Extension, conflicting sources Rejected the incorrect heading checksum; no catalog PR. See source-selection limitation below.
Bundle, valid checksum after label fix Strict comparison succeeded, correct catalog digest, deterministic conclusion step ran successfully, and final labels were correct.

The bundle label omission occurred in the initial valid case and an identical-input retry.

Local regression coverage

  • Tests execute each workflow's checksum command against matching, mismatching, and malformed inputs.
  • Static checks cover scoped permissions, issue-form/source documentation, outcome instructions, and catalog checksum handling.
  • Tests execute the actual bundle label-step JavaScript with mocked GitHub APIs, covering omitted agent updates, existing/unrelated labels, no-PR and failed-publication conditions, and surfaced API errors.
  • The final fork workflow suite reported 63 passed and 7 existing Windows Bash-dependent skips; Ruff 0.15.0 and strict gh-aw v0.88.7 compilation passed.
  • Upstream port validation: 61 passed, 7 existing Windows Bash-dependent skips; checksum command and Node label-step tests ran. Ruff 0.15.0, strict compilation of all three workflows, and git diff --check passed. The upstream test count differs because overlapping archive assertions were consolidated rather than duplicated.
  • All six upstream workflow artifacts match the tested fork revision after newline normalization. Job permissions, dependency edges, action/container pins, and runtime versions are unchanged from upstream; no dependency upgrade is included.

Limits of the evidence

  • Agent instructions are not an independent publishing barrier. The preset negative run correctly rejected the submission but skipped the explicit comparison command while claiming it returned FAILED. Other runs also differed from the prescribed ordering.
  • The extension conflicting-source case rejected an incorrect heading checksum, but did not explicitly diagnose the disagreement with the proposed catalog entry. It does not establish general enforcement of the source-agreement rule.
  • A local probe of the deployed firewall image/config denied an undeclared domain. End-to-end hosted handling of a blocked download was not retested.
  • In the final bundle run, the agent also emitted label requests. The hosted run confirms execution of the deterministic step; omitted-agent recovery is covered by local regression tests.

AI Disclosure

  • I did not use AI assistance for this contribution
  • I did use AI assistance

GitHub Copilot (model: GPT-6 Astra, autonomous; reasoning setting not exposed) authored the fork changes, upstream port, regression tests, test orchestration, and this draft on behalf of @KSchlobohm. The human directed the scope and approved the fork publications. No human line-by-line review is claimed.

…sums

Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@KSchlobohm
KSchlobohm requested a review from mnriem as a code owner September 23, 2026 01:23
Copilot AI balanced review requested due to automatic review settings September 23, 2026 01:23
Comment thread .github/workflows/add-community-bundle.md Outdated
Comment thread .github/workflows/add-community-extension.md Outdated
Comment thread .github/workflows/add-community-preset.md Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The scoped permissions, generated workflows, hosted evidence, and positive and negative regression coverage consistently support the intended fix.

Review effort: Balanced
Findings: None

What changed in this PR

Restores secure archive validation across all community submission workflows and closes #4677.

Changes:

  • Adds explicit GitHub URL and checksum-tool permissions.
  • Validates submitted checksums against downloaded archives.
  • Makes successful bundle labeling deterministic and adds regression coverage.
File Description
tests/​test_github_workflows.py Tests checksum, permissions, outcomes, and labels.
.github/​workflows/​add-community-preset.md Restores preset archive validation.
.github/​workflows/​add-community-preset.lock.yml Regenerates the preset workflow.
.github/​workflows/​add-community-extension.md Restores extension archive validation.
.github/​workflows/​add-community-extension.lock.yml Regenerates the extension workflow.
.github/​workflows/​add-community-bundle.md Adds bundle validation and deterministic labels.
.github/​workflows/​add-community-bundle.lock.yml Regenerates the bundle workflow.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Assisted-by: GitHub Copilot (model: GPT-6 Astra, autonomous)

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings September 23, 2026 01:45
@KSchlobohm

Copy link
Copy Markdown
Contributor Author

Commit: 42e8339

Replaced inline archive URL interpolation with an edit-tool data file and a fixed, double-quoted curl argument in all three submission workflows. Added strict URL character checks and regression coverage for shell metacharacters. Regenerated the workflow locks; runtime permissions are unchanged. Hosted workflows were not rerun.

Posted on behalf of @KSchlobohm by GitHub Copilot (model: GPT-6 Astra, autonomous); code and comment fully AI-drafted.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The scoped workflow changes align with the reported requirements and include strong positive, negative, security, and regression coverage.

Review effort: Balanced
Findings: None

@KSchlobohm
KSchlobohm merged commit 67ab049 into main Sep 23, 2026
18 checks passed
@KSchlobohm
KSchlobohm deleted the kschlobohm-sep-22-archive-fix-upstream branch September 23, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Community submission workflows cannot fetch release archives

3 participants