chore: add ASF source headers and a release header audit gate - #3397
chore: add ASF source headers and a release header audit gate#3397Astro-Han wants to merge 2 commits into
Conversation
66d3014 to
d3e0eef
Compare
jackwener
left a comment
There was a problem hiding this comment.
Automated review of exact head d3e0eefd1f8934d1e1ac94a1e7b3bbafcc634b69.
Blocking correctness finding: the mechanical sweep prepends a second ASF header to .asf.yaml. The parent already begins with the standard ASF header, but its license-URL indentation is four spaces while renderHeader("hash") emits two. Because hasHeader accepts only byte-exact rendered text, applyHeader treats the existing valid header as absent and adds another; the resulting file has two consecutive license blocks. The new audit then passes because it sees its exact block at offset zero, so neither the gate nor the tests catch the duplication.
Please make application recognize/replace an existing standard ASF header (or otherwise prevent duplicate semantic headers), clean the duplicated .asf.yaml header, and add a regression covering this pre-existing whitespace variant. This matters beyond one file: a future valid formatting variant would otherwise be silently doubled again.
Merge readiness: not ready. The PR is draft, required CI is still running, independent licensing/mentor review is required, and the duplicate-header blocker must be resolved on a new exact head.
c6f3418 to
f1d7d83
Compare
|
Thanks — good catch, and the failure mode was worse than the one file: my own mechanical-purity check proved Fixed at
Two checks added because the original one was not enough: a scan of the pre-sweep tree confirming Agreed on merge readiness. The |
The first Apache Maka (Incubating) source release has to apply the ASF header policy and enforce it, and nothing in the repository decided which files the policy covers or checked that they comply. `scripts/asf-license-headers.mjs` is that decision. It classifies every file exactly once: covered by a comment syntax and required to carry the ASF header, or matched by one reviewed exclusion rule that records why the header does not belong there. A file that is neither fails the audit, so a new file type or an unexpected path cannot reach a release candidate until someone writes down which of the two it is. The exclusions cover the ASF release documents themselves, third-party work kept under its own license, generated files that a `check:` script byte-compares against a fresh run, payloads republished verbatim as prompt text or as a pull request body, byte-significant fixtures, formats with no comment syntax, binaries, and version-control metadata. The same module applies the policy through `npm run write:asf-headers`, so the gate and the sweep cannot disagree about the header text or the covered set. `Prepare ASF source candidate` runs the audit inside the extracted archive before anything installs or builds into that tree, so the release gate reads the exact bytes a voter downloads instead of a checkout that resembles them. CI runs it unconditionally beside the other install-free checks, because a header gate that only sees the affected surfaces is not a gate. Apache RAT was the alternative. It would put a JVM and a downloaded jar into the audit path that reviewers are expected to reproduce, and its glob exclusions fail open: a new unexpected file inside an excluded directory is approved silently. This audit needs no toolchain beyond the Node the archive already requires, and an unexpected file fails it. The headers themselves are not in this commit. Until the sweep lands, the audit reports every existing source file as missing one. Refs #3271 Generated-by: Claude Code
`npm run write:asf-headers` produced this commit, and `npm run check:asf-headers` now passes on the checkout and on the extracted source archive. Nothing here was hand-edited: every changed file is exactly its previous content with the standard ASF header inserted at the top, below an interpreter shebang, an HTML doctype, or Markdown front matter where one of those has to open the file. `.asf.yaml`, which already carried the header, is untouched. `DISCLAIMER-WIP` no longer discloses that source files lack Apache license headers, because they no longer do. The remaining disclosures about `NOTICE` and about the software grant and ICLAs are unchanged and still open. `.git-blame-ignore-revs` gets an entry for this sweep. Following the convention the file documents, the placeholder records the landed hash in a follow-up right after the squash-merge, since a hash that never reaches main makes `git blame` fail outright. Merging this will require open pull requests to rebase. That is the cost of doing the sweep once; splitting it per directory would multiply the rebases instead of avoiding them. Refs #3271 Generated-by: Claude Code
f1d7d83 to
0f97abe
Compare
M4n5ter
left a comment
There was a problem hiding this comment.
English
Reviewed exact head 0f97abe69fca7b14fab6e0e057e59ae1ded73f12.
The mechanical sweep is internally consistent, the focused 35 tests pass, and all four CI checks are green. However, I found four policy-level gaps that should be addressed before this becomes the repository's release gate.
1. [Blocking] Keep known mixed-origin files out of the mechanical ASF-header sweep
The extension fallback adds whole-file ASF headers to:
packages/runtime/src/tool-output.tspackages/runtime/src/edit-replace.tspackages/runtime/src/model-protocol.ts
The first two explicitly state that they adapt opencode code; #3325 records the pinned MIT sources and describes them as mixed-origin. #2907 similarly records the Vercel AI SDK material adapted in model-protocol.ts.
The ASF source-header policy defines third-party work to include parts of a file not submitted by their copyright owner, says not to mechanically add the standard ASF header to third-party source, and leaves major modifications to a case-by-case PMC decision.
Please classify these paths as third-party/mixed-source exclusions for now and preserve their upstream attribution. If the PPMC later chooses a combined header for a heavily modified file, that decision can be applied explicitly. Landing #3325 or #2907 later would restore attribution, but would not remove the whole-file header added here.
2. [Blocking] Audit every source-candidate entry instead of skipping build-looking directory names
scripts/asf-license-headers.mjs:254-267,410-427 silently skips every archive directory named dist, release, test-results, and several others. asf-source-release.mjs does not reject most of those names, so an extracted source candidate can contain unaudited source files.
A concrete extracted-tree repro with release/unexpected.kt reports:
Audited 0 covered and 0 excluded file(s)
Every source file carries the ASF header or a reviewed exclusion.
The comment claiming that verifySourceCandidate already rejects these directories is therefore incorrect.
Please remove the build-output skip from archive mode and classify every regular file. Unsupported entry types such as symlinks should be explicitly rejected or classified rather than silently omitted. Since the workflow performs this audit before npm ci, legitimate checkout build output does not need an archive-mode exception. An extracted-candidate fixture containing dist/unexpected.kt should fail as unclassified.
3. [Important] Replace directory-wide exclusions with reviewed inventories
Several exclusions use an entire directory as the policy boundary:
- everything under
apps/desktop/resources/licenses - everything under
apps/desktop/src/renderer/assets/provider-brands - everything under
packages/runtime/resources/bundled-skills
This means a future Maka-authored .ts or .md file placed in any of those directories silently inherits a third-party/verbatim justification. It also already misdescribes git/NOTICE.txt and git/SOURCE_OFFER.txt: those are Maka-authored legal prose, not byte-identical upstream texts verified by the preparation scripts.
This contradicts the gate's fail-closed objective. Please use exact reviewed inventories or structurally provable shapes—for example, exact bundled payload paths and only *.svg provider-brand assets—and add negative tests showing that an unexpected .ts/README.md inside each directory remains unclassified.
4. [Important] Fail closed on existing ASF headers the writer cannot safely canonicalize
The formatting-variant fix still leaves several forms of the original duplicate-header failure open:
- A TypeScript file using a valid leading
//ASF header is parsed only asblock, classified asabsent, and receives a second header. - A common
/** ... */rendering behaves the same way. - The ASF policy explicitly accepts
httpsin the license URL, but that variant is also classified asabsentand duplicated. - Conversely, the exact-header fast path returns
canonicalbefore inspecting the complete leading hash/slash comment, so a directly adjacent third-party copyright line bypasses theentangledsafeguard.
In each duplication case, the next audit passes because the newly inserted canonical header is at offset zero.
The simpler and safer boundary is: keep the audit byte-exact, but if any leading ASF signature is present and is not the isolated canonical rendering, refuse to mutate it and require manual resolution. After this one-time sweep, consider removing the variant-rewriting state machine—or the write mode entirely—instead of continually expanding a semantic license-comment parser.
Simplify audit
The same fixes can materially reduce the policy surface:
- Delete the second archive inclusion authority (
buildOutputDirectories). - Replace broad prefix exclusions with explicit reviewed inventories.
- Retire the one-shot variant-rewriting machinery after the initial sweep, leaving one small exact audit gate.
Verdict: request changes. The current tree appears mechanically sound, but the gate does not yet enforce the fail-closed licensing boundary described by the PR.
简体中文
已审查精确 head 0f97abe69fca7b14fab6e0e057e59ae1ded73f12。
机械 header sweep 内部一致,35 项定向测试通过,四项 CI 也全部通过。不过,我确认了四个 policy 层面的缺口;在它成为仓库 release gate 前应当修复。
1. [Blocking] 不要对已知 mixed-origin 文件机械添加 whole-file ASF header
扩展名 fallback 给以下文件添加了整个文件的 ASF header:
packages/runtime/src/tool-output.tspackages/runtime/src/edit-replace.tspackages/runtime/src/model-protocol.ts
前两个文件明确说明其改编自 opencode;#3325 固定了 MIT 来源,并将其描述为 mixed-origin。#2907 同样记录了 model-protocol.ts 改编的 Vercel AI SDK 材料。
ASF source-header policy 明确将文件内并非由版权所有者提交的部分也视为 third-party work,要求不要机械地在第三方源码顶部添加标准 ASF header,并要求 PMC 对重大修改逐案决定。
请暂时将这些路径分类为 third-party/mixed-source exclusion,并保留上游 attribution。如果 PPMC 后续决定某个重度修改文件使用组合 header,可以再显式实施。之后合入 #3325 或 #2907 只能补回 attribution,无法移除本 PR 已经添加的 whole-file header。
2. [Blocking] 审计 source candidate 中的每个 entry,不要按看似 build output 的目录名跳过
scripts/asf-license-headers.mjs:254-267,410-427 会静默跳过 archive 中所有名为 dist、release、test-results 等的目录。asf-source-release.mjs 并不会拒绝其中大多数目录,因此解压后的 source candidate 可以包含完全未被审计的源码。
在解压目录中放入 release/unexpected.kt 后,实际输出为:
Audited 0 covered and 0 excluded file(s)
Every source file carries the ASF header or a reviewed exclusion.
所以,代码中声称 verifySourceCandidate 已经拒绝这些目录的注释并不成立。
请移除 archive mode 的 build-output skip,并分类每个普通文件。symlink 等不支持的 entry type 也应明确拒绝或分类,而不是静默忽略。workflow 在 npm ci 前运行该审计,因此 archive mode 不需要为 checkout build output 提供例外。建议增加包含 dist/unexpected.kt 的 extracted-candidate fixture,并断言其因 unclassified 而失败。
3. [Important] 用经过审查的文件清单替代整个目录级别的 exclusion
当前多个 exclusion 把整个目录作为 policy 边界:
apps/desktop/resources/licenses下的所有内容apps/desktop/src/renderer/assets/provider-brands下的所有内容packages/runtime/resources/bundled-skills下的所有内容
这会导致未来放入这些目录的 Maka 自有 .ts 或 .md 文件静默继承 third-party/verbatim 理由。当前理由也已经错误描述了 git/NOTICE.txt 和 git/SOURCE_OFFER.txt:它们是 Maka 编写的法律说明,不是由 preparation scripts 校验 digest 的上游逐字节文本。
这与 gate 的 fail-closed 目标相矛盾。请使用精确、经过审查的文件清单,或者能由结构严格证明的规则——例如精确列出 bundled payload,只允许 provider-brand 的 *.svg——并增加反向测试,确保这些目录中新出现的 .ts/README.md 仍然会被判为 unclassified。
4. [Important] 对 writer 无法安全 canonicalize 的已有 ASF header 应当 fail closed
当前 formatting-variant 修复仍留下了几种原始重复 header 问题:
- TypeScript 文件若使用有效的
//ASF header,只会按blockstyle 解析,因此被判为absent并叠加第二份 header。 - 常见的
/** ... */格式有同样的问题。 - ASF policy 明确接受 license URL 使用
https,但该变体仍会被判为absent并重复添加。 - 反过来,exact-header 快速路径会在检查完整的 hash/slash leading comment 前直接返回
canonical,导致紧邻 header 的第三方 copyright 行绕过entangled保护。
在所有重复场景中,下一次 audit 都会通过,因为新添加的 canonical header 位于 offset zero。
更简单且安全的边界是:audit 保持 byte-exact;但只要发现 leading ASF 标志,而它不是独立的 canonical rendering,就拒绝自动修改并要求人工处理。完成这次一次性 sweep 后,建议删除 variant rewrite 状态机——甚至整个 write mode——而不是继续扩展一个语义化的 license-comment parser。
简化审计
上述修复也能明显缩小 policy surface:
- 删除第二套 archive inclusion authority,即
buildOutputDirectories。 - 用明确的 reviewed inventory 替代宽泛的 prefix exclusion。
- 初次 sweep 完成后收回一次性的 variant rewrite 机制,只保留一个小型、精确的 audit gate。
结论:request changes。当前 tree 的机械改动本身没有发现问题,但这个 gate 尚未真正实现 PR 所描述的 fail-closed licensing boundary。
Summary
Applies the ASF source-header policy and enforces it with an automated audit of the source release candidate.
Nothing in the repository decided which files the header policy covers, and nothing checked compliance.
scripts/asf-license-headers.mjsis now that decision. It classifies every file exactly once:Unclassified is the design. A new file extension or an unexpected path cannot reach a release candidate until someone writes down which of the two it is, so the exclusion list stays a list of reasons rather than a list of paths that happened to be noisy. The audit also fails when an exclusion rule matches nothing in a checkout, so the reviewed list cannot accumulate dead rules.
.github/ASF_SOURCE_HEADERS.mddocuments the policy;npm run check:asf-headers -- --reportprints the resolved file list behind every rule.The same module applies the policy via
npm run write:asf-headers, so the gate and the sweep cannot disagree about the header text or the covered set.Why not Apache RAT
The issue allows "Apache RAT or an equivalent mentor-approved audit". This PR implements the equivalent, for three reasons:
apache-maka-<version>-incubating-src.tar.gz. RAT would put a JVM and a downloaded jar into the path a voter is expected to reproduce; this audit needs nothing beyond the Node the archive already requires.scripts/*.mjsregenerate-and-diff checks wired intocheck:npm scripts and run from the extracted candidate. This extends that seam.Mentors who prefer literal RAT output should say so on the issue — the policy data would carry over to a
rat-excludesfile, but the fail-closed property would not.Reviewed exclusions
not-in-source-releaseexport-ignorekeeps them out of the archiveasf-release-documentsLICENSE,NOTICE,DISCLAIMER-WIPare the license and notice themselvesthird-party-license-textsthird-party-sourcegenerated-filesverbatim-runtime-payloadsverbatim-github-templatesbyte-significant-fixturesno-comment-syntaxbinary-filesno-creative-contentThird-party license classification stays with #3270 / G3; this gate only decides where an ASF header may be asserted.
Two commits, on purpose
6c7445d— policy, audit, CI wiring, and release gate. On its own the audit reports every existing source file as missing a header, rather than passing silently.c6f3418— the sweep: 2,591 files, purely mechanical. Each changed file is exactly its previous content with the header inserted at the top, below a shebang, an HTML doctype, or Markdown front matter where one has to open the file. Verified programmatically: for every changed file,applyHeader(git show HEAD:<path>) === <working copy>..git-blame-ignore-revsrecords the sweep. Following the convention that file documents, this PR adds the placeholder entry and the landed squash hash goes in a follow-up immediately after merge — a hash that never reachesmainmakesgit blamefail outright.Merging this will require open pull requests to rebase. With ~60 open PRs that is a real cost, but it is paid once; splitting the sweep by directory would multiply the rebases rather than avoid them.
DISCLAIMER-WIPno longer discloses that source files lack Apache license headers. TheNOTICEand software-grant/ICLA disclosures are unchanged and still open.Refs #3271
Exit conditions satisfied
scripts/asf-license-headers.mjsand.github/ASF_SOURCE_HEADERS.md..asf.yamlalready carried the header and is untouched.Prepare ASF source candidateruns it insidecandidate-sourcebeforenpm ci.DISCLAIMER-WIPwhen its missing-header disclosure is no longer true..git-blame-ignore-revs— placeholder added; the hash follows the squash-merge per the file's own convention.Verification
Run in a clean worktree at
c6f3418, rebased ontoe955575:npm run check:asf-headers— 2,592 covered, 124 excluded, no unclassified file, no entangled header.apache-maka-0.1.11-incubating-src.tar.gzwithnpm run release:asf:source, extracted it, and ran the audit inside it atc6f3418— 2,592 covered, 114 excluded (the 10export-ignorefiles are correctly absent), exit 0..ktfile → reported unclassified, exit 1; a header removed frompackages/core/src/settings.ts→ reported missing, exit 1.npm run check:asf-source— 28 pass (includes the newasf-license-headers.test.mjsand the workflow-policy assertion that the audit precedesnpm ci).npm run lint,npm run format:check,npm run build,npm run typecheck— pass.npm run check:stale,npm run astryx:theme -- --check,npm run astryx:surface-inventory,npm run windows:inventory,npm run check:third-party-notices,npm run check:cli-third-party-notices,npm run check:windows-cargo-notices,npx knip(desktop and ui) — pass.astryx:themecaught a generated file the first pass had missed (maka.js); it is now in thegenerated-filesrule.check:staleneeded oneclean && buildof@maka/uifirst — a stale incrementaltsbuildinfo, not a code problem.npm --workspace @maka/desktop run test:dist— 1017 pass, 0 fail. This is the surface whose source-contract tests read.ts/.tsxbytes, so it is the one the sweep could plausibly have broken.npm --workspace @maka/core run test:dist— 579 pass, 0 fail.Not run locally: the remaining workspace suites, Electron e2e, Storybook, and the Windows lanes. CI covers them.
Review focus
The exclusion list is the part that needs human judgement, not the header insertion.
npm run check:asf-headers -- --reportprints every rule with its justification and the exact files it claims — one screen, and the part of this PR where being wrong has release consequences.The sweep does not need to be read file by file. Its claim is that every changed file equals its parent with the canonical header applied, and that claim is checkable:
That holds for all 2,591 files, and no file in the tree contains the license text more than once.
Addressed from review
@jackwener's automated review at
d3e0eeffound a real blocker:.asf.yamlalready carried an ASF header whose license-URL indentation differed from the rendering,hasHeadermatched byte-exactly, and the sweep prepended a second license block that the audit then accepted. Fixed at the class level rather than for the one file:classifyExistingHeaderrecognizes an existing header by its normalized prose, independent of comment syntax, indentation, and line wrapping. A formatting variant is replaced with the canonical rendering, never doubled.entangled;writerefuses and the audit fails. Separating those is a decision, not a rewrite..asf.yamlalready used, which is also the common ASF form, so that file is untouched by this PR.A repository-wide scan found
.asf.yamlwas the only pre-existing header.AI use
Select exactly one:
Tool(s) and scope: Claude Code (Opus) drafted the policy module, its tests, the CI and release-workflow wiring, the policy document, and ran the sweep and the verification above. Both commits carry
Generated-by: Claude Code. The exclusion rules and their justifications are AI-drafted and need a human contributor of record to confirm them; licensing conclusions remain with the PPMC, mentors, and IPMC.Checklist
Does this PR entail a change in behavior?