Skip to content

Make release archives repeatable #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dd3ok merged 1 commit into from
Jul 16, 2026
Merged

Make release archives repeatable #30

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,13 @@

## [Unreleased]

### Fixed

- Pinned standalone ZIP entry times, process time zone, and line-ending settings
so the same Git/platform toolchain produces committed bytes and a stable
SHA-256 across invocation times and host configurations, with toolchain limits
documented.

## [0.4.2] - 2026-07-17

### Added
Expand Down
43 changes: 40 additions & 3 deletions docs/install.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,13 +247,50 @@ Build from a committed tree so untracked files cannot leak into the archive:
```bash
(
set -euo pipefail
git archive --format=zip --prefix=watchlist-md/ \
python_check='import sys; raise SystemExit(sys.version_info < (3, 8))'
if python3 -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python3
elif python -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python
else
echo "Python 3.8 or newer is required" >&2
exit 1
fi
archive_ref=$(git rev-parse HEAD)
archive_mtime=$(git show -s --format=%cI "${archive_ref}")
archive_check_tree=$(mktemp -d)
mkdir "${archive_check_tree}/evals"
trap 'rm -f "${archive_check_tree}/evals/check_skill_package.py" "${archive_check_tree}/evals/runtime_package_files.txt"; rmdir "${archive_check_tree}/evals" "${archive_check_tree}"' EXIT
git show "${archive_ref}:evals/check_skill_package.py" \
>"${archive_check_tree}/evals/check_skill_package.py"
git show "${archive_ref}:evals/runtime_package_files.txt" \
>"${archive_check_tree}/evals/runtime_package_files.txt"
TZ=UTC git -c core.autocrlf=false -c core.eol=lf archive \
--format=zip --prefix=watchlist-md/ --mtime="${archive_mtime}" \
--output=watchlist-md-skill.zip \
HEAD:.agents/skills/watchlist-md
python3 evals/check_skill_package.py --archive watchlist-md-skill.zip
"${archive_ref}:.agents/skills/watchlist-md"
"${python_cmd}" "${archive_check_tree}/evals/check_skill_package.py" \
--archive watchlist-md-skill.zip
rm -f "${archive_check_tree}/evals/check_skill_package.py" \
"${archive_check_tree}/evals/runtime_package_files.txt"
rmdir "${archive_check_tree}/evals" "${archive_check_tree}"
trap - EXIT
)
```

This recipe requires Git 2.40 or newer with `git archive --mtime` support. The
explicit commit time, UTC process time zone, and disabled checkout line-ending
conversion keep committed file bytes and repeated output stable with the same
Git/platform toolchain. A subtree expression resolves to a tree object, so
omitting `--mtime` would stamp entries with the current time; omitting `TZ=UTC`
would use the host time zone; omitting `core.autocrlf=false` can rewrite line
endings. Git ZIP metadata and compression can still differ across other Git
builds, so cross-toolchain byte identity is not promised. Run the fenced recipe
in Bash (Git Bash on Windows); a native PowerShell translation must set
`$env:TZ = 'UTC'` for the archive command and restore the previous value afterward.
The checker and manifest are also read from the pinned commit, so concurrent
working-tree or `HEAD` changes cannot silently change the validation contract.

The archive contains `watchlist-md/SKILL.md` and `watchlist-md/LICENSE.txt` under
one top-level folder and remains Python-free. Repository `tools/`, `evals/`,
maintainer docs, transcripts, screenshots, and raw logs must not be packaged.
Expand Down
123 changes: 99 additions & 24 deletions docs/maintainers/release.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,25 @@ transcripts, screenshots, and raw logs.
Run:

```bash
PYTHONDONTWRITEBYTECODE=1 python3 -m unittest discover -s evals -p 'test_*.py'
python3 evals/check_policy_markers.py
python3 evals/check_semantic_cases.py
python3 evals/check_skill_package.py
python3 evals/check_release_metadata.py
python3 evals/check_watchlist.py examples/WATCHLIST.example.md --strict-format --strict-safety --require-archive-section
python3 tools/validate_watchlist.py .agents/skills/watchlist-md/assets/WATCHLIST.template.md --strict-format --strict-safety --require-archive-section
(
set -euo pipefail
python_check='import sys; raise SystemExit(sys.version_info < (3, 8))'
if python3 -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python3
elif python -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python
else
echo "Python 3.8 or newer is required" >&2
exit 1
fi
Comment on lines +36 to +44

@gemini-code-assist gemini-code-assist Bot Jul 16, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This Python version check logic is duplicated in four separate code blocks within this file. To improve maintainability and reduce redundancy, consider defining a shell function for this check at the beginning of each script block.

For example:

find_python() {
  python_check='import sys; raise SystemExit(sys.version_info < (3, 8))'
  if python3 -c "${python_check}" >/dev/null 2>&1; then
    python_cmd=python3
  elif python -c "${python_check}" >/dev/null 2>&1; then
    python_cmd=python
  else
    echo "Python 3.8 or newer is required" >&2
    exit 1
  fi
}

find_python
# Now use "${python_cmd}"

This would make each script block cleaner and easier to update if the Python requirement changes.

PYTHONDONTWRITEBYTECODE=1 "${python_cmd}" -m unittest discover -s evals -p 'test_*.py'
"${python_cmd}" evals/check_policy_markers.py
"${python_cmd}" evals/check_semantic_cases.py
"${python_cmd}" evals/check_skill_package.py
"${python_cmd}" evals/check_release_metadata.py
"${python_cmd}" evals/check_watchlist.py examples/WATCHLIST.example.md --strict-format --strict-safety --require-archive-section
"${python_cmd}" tools/validate_watchlist.py .agents/skills/watchlist-md/assets/WATCHLIST.template.md --strict-format --strict-safety --require-archive-section
)
```

Confirm no unintended runtime bundle change against the PR base and local tree:
Expand Down Expand Up @@ -67,7 +79,16 @@ Run the release-ready metadata check:
```bash
(
set -euo pipefail
python3 evals/check_release_metadata.py --release
python_check='import sys; raise SystemExit(sys.version_info < (3, 8))'
if python3 -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python3
elif python -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python
else
echo "Python 3.8 or newer is required" >&2
exit 1
fi
"${python_cmd}" evals/check_release_metadata.py --release
version=$(cat VERSION)
git fetch origin --tags
if git show-ref --verify --quiet "refs/tags/v${version}"; then
Expand Down Expand Up @@ -113,34 +134,60 @@ Build from the exact merged commit, not from an untracked working directory:
```bash
(
set -euo pipefail
python_check='import sys; raise SystemExit(sys.version_info < (3, 8))'
if python3 -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python3
elif python -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python
else
echo "Python 3.8 or newer is required" >&2
exit 1
fi
git fetch origin main --tags
release_sha=$(git rev-parse origin/main)
test "$(git rev-parse HEAD)" = "${release_sha}"
test -z "$(git status --porcelain)"
version=$(git show "${release_sha}:VERSION")
release_mtime=$(git show -s --format=%cI "${release_sha}")
release_tree=$(mktemp -d)
mkdir "${release_tree}/evals"
trap 'rm -f "${release_tree}/VERSION" "${release_tree}/CHANGELOG.md" "${release_tree}/evals/check_release_metadata.py"; rmdir "${release_tree}/evals" "${release_tree}"' EXIT
trap 'rm -f "${release_tree}/VERSION" "${release_tree}/CHANGELOG.md" "${release_tree}/evals/check_release_metadata.py" "${release_tree}/evals/check_skill_package.py" "${release_tree}/evals/runtime_package_files.txt"; rmdir "${release_tree}/evals" "${release_tree}"' EXIT
git show "${release_sha}:VERSION" >"${release_tree}/VERSION"
git show "${release_sha}:CHANGELOG.md" >"${release_tree}/CHANGELOG.md"
git show "${release_sha}:evals/check_release_metadata.py" \
>"${release_tree}/evals/check_release_metadata.py"
python3 "${release_tree}/evals/check_release_metadata.py" "${release_tree}" --release
rm -f "${release_tree}/VERSION" "${release_tree}/CHANGELOG.md" \
"${release_tree}/evals/check_release_metadata.py"
rmdir "${release_tree}/evals" "${release_tree}"
trap - EXIT
git show "${release_sha}:evals/check_skill_package.py" \
>"${release_tree}/evals/check_skill_package.py"
git show "${release_sha}:evals/runtime_package_files.txt" \
>"${release_tree}/evals/runtime_package_files.txt"
"${python_cmd}" "${release_tree}/evals/check_release_metadata.py" "${release_tree}" --release
mkdir -p dist
artifact="dist/watchlist-md-skill-v${version}.zip"
git archive --format=zip --prefix=watchlist-md/ --output="${artifact}" "${release_sha}:.agents/skills/watchlist-md"
python3 evals/check_skill_package.py --archive "${artifact}"
TZ=UTC git -c core.autocrlf=false -c core.eol=lf archive \
--format=zip --prefix=watchlist-md/ --mtime="${release_mtime}" \
--output="${artifact}" "${release_sha}:.agents/skills/watchlist-md"
"${python_cmd}" "${release_tree}/evals/check_skill_package.py" --archive "${artifact}"
sha256sum "${artifact}"
rm -f "${release_tree}/VERSION" "${release_tree}/CHANGELOG.md" \
"${release_tree}/evals/check_release_metadata.py" \
"${release_tree}/evals/check_skill_package.py" \
"${release_tree}/evals/runtime_package_files.txt"
rmdir "${release_tree}/evals" "${release_tree}"
trap - EXIT
)
```

The archive must contain one top-level `watchlist-md/` directory and the exact
seven runtime files. On PowerShell, use `Get-FileHash -Algorithm SHA256` instead
of `sha256sum`.
seven runtime files. This recipe requires Git 2.40 or newer with `git archive
--mtime` support. Pinning the entry time to the source commit and Git's process
time zone to UTC, and checkout line-ending conversion off makes committed file
bytes and repeated builds stable with the same Git/platform toolchain. It does
not promise identical bytes across every Git build or compression implementation.
Record the release OS, `git --version`, and SHA-256 with the release evidence.
Run the fenced recipe in Bash (Git Bash on Windows); a native PowerShell
translation must set `$env:TZ = 'UTC'` for the archive command, restore the
previous value afterward, and use `Get-FileHash -Algorithm SHA256` instead of
`sha256sum`.

## Publish And Verify

Expand All @@ -149,17 +196,39 @@ Only publish after required `main` CI checks succeed for `release_sha`:
```bash
(
set -euo pipefail
python_check='import sys; raise SystemExit(sys.version_info < (3, 8))'
if python3 -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python3
elif python -c "${python_check}" >/dev/null 2>&1; then
python_cmd=python
else
echo "Python 3.8 or newer is required" >&2
exit 1
fi
git fetch origin main --tags
release_sha=$(git rev-parse origin/main)
version=$(git show "${release_sha}:VERSION")
release_mtime=$(git show -s --format=%cI "${release_sha}")
artifact="dist/watchlist-md-skill-v${version}.zip"
test "$(git rev-parse HEAD)" = "${release_sha}"
test -z "$(git status --porcelain)"
python3 evals/check_release_metadata.py --release
release_tree=$(mktemp -d)
mkdir "${release_tree}/evals"
trap 'rm -f "${release_tree}/VERSION" "${release_tree}/CHANGELOG.md" "${release_tree}/evals/check_release_metadata.py" "${release_tree}/evals/check_skill_package.py" "${release_tree}/evals/runtime_package_files.txt"; rmdir "${release_tree}/evals" "${release_tree}"' EXIT
git show "${release_sha}:VERSION" >"${release_tree}/VERSION"
git show "${release_sha}:CHANGELOG.md" >"${release_tree}/CHANGELOG.md"
git show "${release_sha}:evals/check_release_metadata.py" \
>"${release_tree}/evals/check_release_metadata.py"
git show "${release_sha}:evals/check_skill_package.py" \
>"${release_tree}/evals/check_skill_package.py"
git show "${release_sha}:evals/runtime_package_files.txt" \
>"${release_tree}/evals/runtime_package_files.txt"
"${python_cmd}" "${release_tree}/evals/check_release_metadata.py" "${release_tree}" --release
mkdir -p dist
git archive --format=zip --prefix=watchlist-md/ \
TZ=UTC git -c core.autocrlf=false -c core.eol=lf archive \
--format=zip --prefix=watchlist-md/ --mtime="${release_mtime}" \
--output="${artifact}" "${release_sha}:.agents/skills/watchlist-md"
python3 evals/check_skill_package.py --archive "${artifact}"
"${python_cmd}" "${release_tree}/evals/check_skill_package.py" --archive "${artifact}"
sha256sum "${artifact}"
run_id=$(gh run list --repo dd3ok/WATCHLIST.md --workflow CI --event push \
--commit "${release_sha}" --limit 1 --json databaseId \
Expand All @@ -178,9 +247,15 @@ git fetch origin --tags
test "$(git rev-list -n 1 "v${version}")" = "${release_sha}"
gh release view "v${version}" --repo dd3ok/WATCHLIST.md \
--json tagName,publishedAt,targetCommitish,assets,url
rm -f "${release_tree}/VERSION" "${release_tree}/CHANGELOG.md" \
"${release_tree}/evals/check_release_metadata.py" \
"${release_tree}/evals/check_skill_package.py" \
"${release_tree}/evals/runtime_package_files.txt"
rmdir "${release_tree}/evals" "${release_tree}"
trap - EXIT
)
```

Verify the uploaded asset name, SHA-256 digest, tag target, and release URL. If
any check fails, stop and report it; do not move or recreate a published tag
silently.
Verify the uploaded asset name, SHA-256 digest, tag target, release URL, release
OS, and Git version. If any check fails, stop and report it; do not move or
recreate a published tag silently.
23 changes: 17 additions & 6 deletions evals/check_policy_markers.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,8 +54,13 @@
"Before updating, inspect whether the installed copy has local changes",
"backup_root=\"$HOME/.watchlist-md-skill-backups/claude\"",
"mktemp -d",
"git archive --format=zip --prefix=watchlist-md/",
"python3 evals/check_skill_package.py --archive watchlist-md-skill.zip",
"--format=zip --prefix=watchlist-md/",
"archive_ref=$(git rev-parse HEAD)",
"TZ=UTC git -c core.autocrlf=false -c core.eol=lf archive",
"Git 2.40 or newer",
'--mtime="${archive_mtime}"',
"check_skill_package.py",
"--archive watchlist-md-skill.zip",
"watchlist-md/SKILL.md",
"watchlist-md/LICENSE.txt",
"not the repository root",
Expand Down Expand Up @@ -86,13 +91,19 @@
"docs/maintainers/release.md": [
"Release Checklist",
"The installable skill bundle is intentionally Python-free",
"python3 evals/check_skill_package.py",
"python3 evals/check_release_metadata.py",
"python3 evals/check_release_metadata.py --release",
'"${python_cmd}" evals/check_skill_package.py',
'"${python_cmd}" evals/check_release_metadata.py',
'"${python_cmd}" evals/check_release_metadata.py --release',
"Python 3.8 or newer is required",
"gh release create \"v${version}\"",
"gh run watch \"${run_id}\"",
"set -euo pipefail",
"git archive --format=zip --prefix=watchlist-md/",
"--format=zip --prefix=watchlist-md/",
"TZ=UTC git -c core.autocrlf=false -c core.eol=lf archive",
"Git 2.40 or newer",
"same Git/platform toolchain",
'release_mtime=$(git show -s --format=%cI "${release_sha}")',
'--mtime="${release_mtime}"',
"git diff --name-only origin/main...HEAD -- .agents/skills/watchlist-md",
"git diff --name-only -- .agents/skills/watchlist-md",
"Repository-only files must stay outside `.agents/skills/watchlist-md/`",
Expand Down
2 changes: 1 addition & 1 deletion evals/check_skill_package.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ def main(argv: list[str]) -> int:
"Skill package manifest check failed:\n"
+ "\n".join(f"- {error}" for error in manifest_errors)
)
if not SKILL_DIR.is_dir():
if args.archive is None and not SKILL_DIR.is_dir():
return fail(f"Missing skill directory: {SKILL_DIR}")

if args.archive is not None:
Expand Down
Loading