From 0d408c6e2cc95401dbe2ad1a596122e03a8184f5 Mon Sep 17 00:00:00 2001 From: Nic-dorman Date: Thu, 22 Oct 2026 13:26:51 +0100 Subject: [PATCH 1/5] ci(release): fall back to generated notes, and add the changelog rotation script The release workflow ships the CHANGELOG section matching the tag's base version, but nothing at the train cut ever rotates `[Unreleased]` into that section, so every tag from ant-cli-v0.3.7 to ant-cli-v0.3.12-rc.1 shipped the "No changelog entry was recorded" placeholder. - scripts/rotate-changelog.sh X.Y.Z moves `[Unreleased]` into `## [X.Y.Z] - `, for the cut tooling to run on the rc branch. It is idempotent across rc re-cuts and never fails on an empty `[Unreleased]`. - With no matching section, the workflow now publishes GitHub's generated list of PRs merged since the previous stable ant-cli tag instead of the placeholder, still with the `::warning::` annotation. The placeholder is kept only for when that API call fails. - CLAUDE.md says where entries go and who writes the versioned heading. Refs V2-1107. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ant-cli-release.yml | 42 +++++++++- CLAUDE.md | 11 +++ scripts/rotate-changelog.sh | 107 ++++++++++++++++++++++++++ 3 files changed, 156 insertions(+), 4 deletions(-) create mode 100755 scripts/rotate-changelog.sh diff --git a/.github/workflows/ant-cli-release.yml b/.github/workflows/ant-cli-release.yml index f0771f5c..793231ef 100644 --- a/.github/workflows/ant-cli-release.yml +++ b/.github/workflows/ant-cli-release.yml @@ -349,6 +349,8 @@ jobs: - name: extract changelog entry id: changelog shell: bash + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | # Extract the changelog section for the version being released — NOT # "the first section". Blindly taking the first section shipped the @@ -357,8 +359,8 @@ jobs: # the dual-stack entry re-shipped for four months). See V2-1107. # # `-rc.N` / `-beta.N` builds share the base version's section: the - # section is created when the train cuts rc.1 (rotate `[Unreleased]` - # into `[X.Y.Z] - ` as part of the cut), so every prerelease + # section is created when the train cuts rc.1 (`scripts/rotate-changelog.sh + # X.Y.Z` moves `[Unreleased]` into `[X.Y.Z] - `), so every prerelease # and the final stable of one train ship the same, correct, notes. version="${{ steps.meta.outputs.version }}" base_version="${version%%-*}" @@ -373,11 +375,43 @@ jobs: # all". The `::warning::` annotation keeps the omission loud on the run. # Pre-tag enforcement, if we want it, belongs on the release-prep PR where # a fix is cheap and nothing is blocked. + # + # Without a section, the notes fall back to GitHub's generated list of the + # PRs merged since the previous stable `ant-cli-v*` release, so a missed + # rotation ships the right list of changes rather than none (0.3.7 to 0.3.11 + # all shipped the placeholder). The placeholder is only the last resort, for + # when that API call fails. if [ -z "$(echo "$changelog" | tr -d '[:space:]')" ]; then - echo "::warning title=Missing changelog section::CHANGELOG.md has no non-empty '## [${base_version}]' section. Publishing with placeholder notes; correct them with 'gh release edit' and rotate '## [Unreleased]' into '## [${base_version}] - $(date -u +%F)'. See V2-1107." - cat > /tmp/changelog_entry.md < /tmp/changelog_entry.md + else + echo "::warning title=Missing changelog section::CHANGELOG.md has no non-empty '## [${base_version}]' section, and GitHub's generated notes were unavailable. Publishing with placeholder notes; correct them with 'gh release edit' and rotate '## [Unreleased]' with 'scripts/rotate-changelog.sh ${base_version}' at the cut. See V2-1107." + cat > /tmp/changelog_entry.md < /tmp/changelog_entry.md diff --git a/CLAUDE.md b/CLAUDE.md index 0086c02d..ae6d9b94 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -205,3 +205,14 @@ CI enforces the Linear link and the template fields (`linear-link` and Release-promotion PRs — a `chore(release):` title from an `rc-*` head branch — are exempt from both checks (they are mechanical version-bump PRs generated by the release process). + +## Changelog + +A PR that changes what `ant` or `ant-core` users see adds an entry under +`## [Unreleased]` in `CHANGELOG.md` (Keep a Changelog headings: `Added`, +`Changed`, `Fixed`, `Security`, `Internal`; end the entry with the PR number). +Never write a versioned `## [X.Y.Z]` heading in a feature PR: the release train +creates it when it cuts rc.1, by running `scripts/rotate-changelog.sh X.Y.Z` on +the rc branch. The release workflow ships the `## [X.Y.Z]` section as the notes +of every `ant-cli-vX.Y.Z*` tag, and falls back to GitHub's generated PR list +when the section is missing (V2-1107). diff --git a/scripts/rotate-changelog.sh b/scripts/rotate-changelog.sh new file mode 100755 index 00000000..bd5a89e6 --- /dev/null +++ b/scripts/rotate-changelog.sh @@ -0,0 +1,107 @@ +#!/usr/bin/env bash +# Rotate CHANGELOG.md's `## [Unreleased]` entries into a `## [X.Y.Z] - ` section. +# +# Run this when a train cuts rc.1, on the rc branch, before tagging. The release workflow +# (`ant-cli-release.yml`) ships the section whose heading matches the tag's base version, +# so `ant-cli-vX.Y.Z-rc.N`, `-beta.N` and the final `ant-cli-vX.Y.Z` all read the +# `## [X.Y.Z]` section this script writes. See V2-1107. +# +# scripts/rotate-changelog.sh [] [] +# +# X.Y.Z, or a prerelease X.Y.Z-rc.N (the suffix is dropped) +# YYYY-MM-DD, defaults to today in UTC +# defaults to CHANGELOG.md +# +# Exit status: +# 0 rotated; or `## [X.Y.Z]` already has entries (an rc.2 re-cut), so nothing to do; +# or `[Unreleased]` is empty, so nothing to rotate (a warning is printed, and the +# release workflow publishes generated notes for that version instead) +# 1 bad arguments, or the changelog has no `## [Unreleased]` heading +# 2 `## [X.Y.Z]` exists with entries AND `[Unreleased]` has entries too: someone +# added entries after the section was rotated, and a human has to place them +# +# A changelog problem must never block a release (V2-1142): callers in the cut tooling +# should report a non-zero exit, not abort the cut on it. + +set -euo pipefail + +usage() { + sed -n '2,25p' "$0" | sed 's/^# \{0,1\}//' >&2 + exit 1 +} + +[ $# -ge 1 ] && [ $# -le 3 ] || usage + +version="${1#v}" +version="${version%%-*}" +date="${2:-$(date -u +%F)}" +changelog="${3:-CHANGELOG.md}" + +if ! [[ "$version" =~ ^[0-9]+\.[0-9]+\.[0-9]+$ ]]; then + echo "rotate-changelog: '$1' is not a version (want X.Y.Z or X.Y.Z-rc.N)" >&2 + exit 1 +fi +if ! [[ "$date" =~ ^[0-9]{4}-[0-9]{2}-[0-9]{2}$ ]]; then + echo "rotate-changelog: '$date' is not a date (want YYYY-MM-DD)" >&2 + exit 1 +fi +if [ ! -f "$changelog" ]; then + echo "rotate-changelog: $changelog not found" >&2 + exit 1 +fi +if ! grep -q '^## \[Unreleased\]' "$changelog"; then + echo "rotate-changelog: $changelog has no '## [Unreleased]' heading" >&2 + exit 1 +fi + +# Print the body of the section whose heading starts with `## []`. +section() { + awk -v name="$1" ' + index($0, "## [" name "]") == 1 { found = 1; next } + /^## \[/ { if (found) exit } + found' "$changelog" +} + +has_entries() { + [ -n "$(printf '%s' "$1" | tr -d '[:space:]')" ] +} + +unreleased="$(section Unreleased)" +existing="$(section "$version")" + +if has_entries "$existing"; then + if has_entries "$unreleased"; then + echo "rotate-changelog: ## [$version] already has entries, and so does ## [Unreleased]." >&2 + echo "rotate-changelog: move the [Unreleased] entries that ship in $version by hand." >&2 + exit 2 + fi + echo "rotate-changelog: ## [$version] already has entries; nothing to do." + exit 0 +fi + +if ! has_entries "$unreleased"; then + echo "rotate-changelog: WARNING: ## [Unreleased] is empty, so $version has no entries." >&2 + echo "rotate-changelog: the release will publish generated notes for it; add entries under" >&2 + echo "rotate-changelog: ## [$version] (and 'gh release edit' the release) to replace them." >&2 + exit 0 +fi + +# Keep the `[Unreleased]` heading, open `[X.Y.Z] - ` under it and move the entries +# there unchanged. An empty `## [X.Y.Z]` heading left by hand is dropped, not duplicated. +tmp="$(mktemp)" +trap 'rm -f "$tmp"' EXIT +awk -v ver="$version" -v date="$date" ' + index($0, "## [" ver "]") == 1 { skip_empty = 1; next } + skip_empty && /^[[:space:]]*$/ { next } + { skip_empty = 0 } + !done && /^## \[Unreleased\]/ { + print + print "" + print "## [" ver "] - " date + done = 1 + next + } + { print }' "$changelog" > "$tmp" +cat "$tmp" > "$changelog" + +echo "rotate-changelog: moved ## [Unreleased] into ## [$version] - $date in $changelog." From e2e3179c8731ccd9bb65ee5a852b9eb88f3932eb Mon Sep 17 00:00:00 2001 From: Nic-dorman Date: Thu, 22 Oct 2026 13:29:12 +0100 Subject: [PATCH 2/5] docs: move the changelog guidance to AGENTS.md so every coding agent sees it CLAUDE.md imports it with @AGENTS.md, so Claude Code still loads it. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 15 +++++++++++++++ CLAUDE.md | 15 +++++---------- 2 files changed, 20 insertions(+), 10 deletions(-) create mode 100644 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..6fc59e72 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,15 @@ +# AGENTS.md + +Instructions for any coding agent working in this repo (Claude Code, Codex, OpenCode, +pi and others). Claude Code loads this file through `CLAUDE.md`. + +## Changelog + +A PR that changes what `ant` or `ant-core` users see adds an entry under +`## [Unreleased]` in `CHANGELOG.md` (Keep a Changelog headings: `Added`, +`Changed`, `Fixed`, `Security`, `Internal`; end the entry with the PR number). +Never write a versioned `## [X.Y.Z]` heading in a feature PR: the release train +creates it when it cuts rc.1, by running `scripts/rotate-changelog.sh X.Y.Z` on +the rc branch. The release workflow ships the `## [X.Y.Z]` section as the notes +of every `ant-cli-vX.Y.Z*` tag, and falls back to GitHub's generated PR list +when the section is missing (V2-1107). diff --git a/CLAUDE.md b/CLAUDE.md index ae6d9b94..29f012d3 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -206,13 +206,8 @@ Release-promotion PRs — a `chore(release):` title from an `rc-*` head branch are exempt from both checks (they are mechanical version-bump PRs generated by the release process). -## Changelog - -A PR that changes what `ant` or `ant-core` users see adds an entry under -`## [Unreleased]` in `CHANGELOG.md` (Keep a Changelog headings: `Added`, -`Changed`, `Fixed`, `Security`, `Internal`; end the entry with the PR number). -Never write a versioned `## [X.Y.Z]` heading in a feature PR: the release train -creates it when it cuts rc.1, by running `scripts/rotate-changelog.sh X.Y.Z` on -the rc branch. The release workflow ships the `## [X.Y.Z]` section as the notes -of every `ant-cli-vX.Y.Z*` tag, and falls back to GitHub's generated PR list -when the section is missing (V2-1107). +## Shared agent instructions + +Instructions for every coding agent, not just Claude, live in `AGENTS.md`: + +@AGENTS.md From ccfdb4961fe92ab38460430b7e003e5b8686e889 Mon Sep 17 00:00:00 2001 From: Nic-dorman Date: Tue, 27 Oct 2026 14:49:09 +0000 Subject: [PATCH 3/5] fix(scripts): rename the rotated changelog into place instead of rewriting it rotate-changelog.sh wrote the rotated text back with `cat "$tmp" > "$changelog"`, so an interrupted run could leave CHANGELOG.md truncated. It now writes a temporary file next to the changelog, gives it the changelog's mode with `cp -p`, and renames it over the original. An interruption leaves the old file whole. scripts/test-rotate-changelog.sh covers: - the rotation, including an rc suffix; - an idempotent re-run; - exit 2 when both sections have entries; - an empty hand-written heading; - an empty [Unreleased]; - bad input; - the rename and the kept mode. Co-Authored-By: Claude Opus 5.5 --- scripts/rotate-changelog.sh | 7 +- scripts/test-rotate-changelog.sh | 112 +++++++++++++++++++++++++++++++ 2 files changed, 117 insertions(+), 2 deletions(-) create mode 100755 scripts/test-rotate-changelog.sh diff --git a/scripts/rotate-changelog.sh b/scripts/rotate-changelog.sh index bd5a89e6..583d8b5a 100755 --- a/scripts/rotate-changelog.sh +++ b/scripts/rotate-changelog.sh @@ -88,8 +88,11 @@ fi # Keep the `[Unreleased]` heading, open `[X.Y.Z] - ` under it and move the entries # there unchanged. An empty `## [X.Y.Z]` heading left by hand is dropped, not duplicated. -tmp="$(mktemp)" +# The result goes to a temporary file next to the changelog, which is then renamed over it, so +# an interrupted run leaves the old changelog whole. `cp -p` gives that file the changelog's mode. +tmp="$(mktemp "$(dirname "$changelog")/.rotate-changelog.XXXXXX")" trap 'rm -f "$tmp"' EXIT +cp -p "$changelog" "$tmp" awk -v ver="$version" -v date="$date" ' index($0, "## [" ver "]") == 1 { skip_empty = 1; next } skip_empty && /^[[:space:]]*$/ { next } @@ -102,6 +105,6 @@ awk -v ver="$version" -v date="$date" ' next } { print }' "$changelog" > "$tmp" -cat "$tmp" > "$changelog" +mv -f "$tmp" "$changelog" echo "rotate-changelog: moved ## [Unreleased] into ## [$version] - $date in $changelog." diff --git a/scripts/test-rotate-changelog.sh b/scripts/test-rotate-changelog.sh new file mode 100755 index 00000000..2cf284eb --- /dev/null +++ b/scripts/test-rotate-changelog.sh @@ -0,0 +1,112 @@ +#!/usr/bin/env bash +# Tests for scripts/rotate-changelog.sh. CI runs this in the `Release scripts` job. +# +# scripts/test-rotate-changelog.sh [] + +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +script="${1:-$here/rotate-changelog.sh}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT + +pass=0 fail=0 +ok() { echo "ok $*"; pass=$((pass + 1)); } +bad() { echo "FAIL $*"; fail=$((fail + 1)); } +mode() { stat -c %a "$1" 2> /dev/null || stat -f %Lp "$1"; } +inode() { stat -c %i "$1" 2> /dev/null || stat -f %i "$1"; } + +# fresh : writes a changelog with those entries under [Unreleased]. +fresh() { + rm -rf "$work/repo" + mkdir "$work/repo" + { + printf '# Changelog\n\n## [Unreleased]\n' + [ -n "$1" ] && printf '\n%s\n' "$1" + printf '\n## [0.3.12] - 2026-10-22\n\n### Fixed\n- An older fix (#232)\n' + } > "$work/repo/CHANGELOG.md" +} +# rotate : runs the script on the test changelog; sets $rc and $log. +rotate() { + rc=0 + log="$(bash "$script" "$@" "$work/repo/CHANGELOG.md" 2>&1)" || rc=$? +} +count() { grep -c -F -- "$1" "$work/repo/CHANGELOG.md" || true; } + +entries=$'### Added\n- A new thing (#300)' + +# 1. Rotation: [Unreleased] stays, empty, with the dated section and its entries under it. +fresh "$entries" +rotate 0.3.13 2026-11-05 +expected=$'# Changelog\n\n## [Unreleased]\n\n## [0.3.13] - 2026-11-05\n\n### Added\n- A new thing (#300)\n\n## [0.3.12] - 2026-10-22\n\n### Fixed\n- An older fix (#232)' +if [ "$rc" = 0 ] && [ "$(cat "$work/repo/CHANGELOG.md")" = "$expected" ]; then + ok "rotates [Unreleased] into a dated section" +else bad "rotates [Unreleased] into a dated section (rc=$rc): $(cat "$work/repo/CHANGELOG.md")"; fi + +# 2. Re-running is a no-op. +cp "$work/repo/CHANGELOG.md" "$work/before.md" +rotate 0.3.13-rc.2 2026-11-06 +if [ "$rc" = 0 ] && cmp -s "$work/before.md" "$work/repo/CHANGELOG.md" && printf '%s' "$log" | grep -q 'nothing to do'; then + ok "a re-run leaves the changelog alone" +else bad "a re-run leaves the changelog alone (rc=$rc, log=$log)"; fi + +# 3. Entries added after the rotation: exit 2, nothing changed. +fresh "$entries" +rotate 0.3.13 2026-11-05 +awk '{ print } /^## \[Unreleased\]/ { print ""; print "### Fixed"; print "- A late fix (#301)" }' \ + "$work/repo/CHANGELOG.md" > "$work/late.md" +cp "$work/late.md" "$work/repo/CHANGELOG.md" +rotate 0.3.13 2026-11-05 +if [ "$rc" = 2 ] && cmp -s "$work/late.md" "$work/repo/CHANGELOG.md"; then + ok "entries on both sides exit 2 and change nothing" +else bad "entries on both sides exit 2 and change nothing (rc=$rc)"; fi + +# 4. An rc version drops its suffix. +fresh "$entries" +rotate 0.3.13-rc.1 2026-11-05 +if [ "$rc" = 0 ] && [ "$(count '## [0.3.13] - 2026-11-05')" = 1 ] && [ "$(count 'rc.1')" = 0 ]; then + ok "an rc version writes the base version's heading" +else bad "an rc version writes the base version's heading (rc=$rc)"; fi + +# 5. An empty hand-written heading is replaced, not duplicated. +fresh "$entries" +awk '{ print } /^## \[Unreleased\]/ { print ""; print "## [0.3.13]" }' \ + "$work/repo/CHANGELOG.md" > "$work/hand.md" +cp "$work/hand.md" "$work/repo/CHANGELOG.md" +rotate 0.3.13 2026-11-05 +if [ "$rc" = 0 ] && [ "$(count '## [0.3.13]')" = 1 ] && [ "$(count '- A new thing (#300)')" = 1 ]; then + ok "an empty hand-written heading is replaced" +else bad "an empty hand-written heading is replaced (rc=$rc): $(cat "$work/repo/CHANGELOG.md")"; fi + +# 6. An empty [Unreleased] warns, exits 0 and changes nothing. +fresh "" +cp "$work/repo/CHANGELOG.md" "$work/before.md" +rotate 0.3.13 2026-11-05 +if [ "$rc" = 0 ] && cmp -s "$work/before.md" "$work/repo/CHANGELOG.md" && printf '%s' "$log" | grep -q 'WARNING'; then + ok "an empty [Unreleased] warns and changes nothing" +else bad "an empty [Unreleased] warns and changes nothing (rc=$rc, log=$log)"; fi + +# 7. The changelog is replaced by a rename, keeps its mode, and no temporary file is left behind. +fresh "$entries" +chmod 640 "$work/repo/CHANGELOG.md" +inode_before="$(inode "$work/repo/CHANGELOG.md")" +rotate 0.3.13 2026-11-05 +inode_after="$(inode "$work/repo/CHANGELOG.md")" +left="$(find "$work/repo" -mindepth 1 -not -name CHANGELOG.md | wc -l | tr -d ' ')" +if [ "$rc" = 0 ] && [ "$inode_before" != "$inode_after" ] && [ "$(mode "$work/repo/CHANGELOG.md")" = 640 ] && [ "$left" = 0 ]; then + ok "the changelog is renamed into place with its mode kept" +else bad "the changelog is renamed into place with its mode kept (rc=$rc, inode $inode_before -> $inode_after, mode $(mode "$work/repo/CHANGELOG.md"), $left left over)"; fi + +# 8. Bad input exits 1 and changes nothing. +fresh "$entries" +cp "$work/repo/CHANGELOG.md" "$work/before.md" +rotate 0.3 2026-11-05 +if [ "$rc" = 1 ]; then ok "a bad version exits 1"; else bad "a bad version exits $rc"; fi +rotate 0.3.13 05/11/2026 +if [ "$rc" = 1 ]; then ok "a bad date exits 1"; else bad "a bad date exits $rc"; fi +printf '# Changelog\n\n## [0.3.12] - 2026-10-22\n' > "$work/repo/CHANGELOG.md" +rotate 0.3.13 2026-11-05 +if [ "$rc" = 1 ]; then ok "no [Unreleased] heading exits 1"; else bad "no [Unreleased] heading exits $rc"; fi + +echo "$pass passed, $fail failed" +[ "$fail" = 0 ] From 369b10a9e25f6936dff5fd0c2bf00e728b0a7e04 Mon Sep 17 00:00:00 2001 From: Nic-dorman Date: Tue, 27 Oct 2026 14:49:09 +0000 Subject: [PATCH 4/5] fix(release): publish generated notes only when the API call succeeds On an HTTP error `gh api ... --jq .body` skips the filter, prints the raw JSON error body and exits non-zero. The step's `|| true` kept that body in `generated`, which then passed the non-empty check and would have been published as the release notes instead of the placeholder. Its output now counts only when the call succeeds. The notes logic moves from the workflow step into scripts/release-notes.sh, unchanged otherwise, so it can be tested: the changelog section, else the generated notes since the previous stable tag, else the placeholder. scripts/test-release-notes.sh runs it against fake `git` and `gh`, including a failed call that prints a JSON error body. A new `Release scripts` job in ci.yml runs that test and the rotation test on every PR. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/ant-cli-release.yml | 69 +------------ .github/workflows/ci.yml | 12 +++ scripts/release-notes.sh | 93 ++++++++++++++++++ scripts/test-release-notes.sh | 134 ++++++++++++++++++++++++++ 4 files changed, 244 insertions(+), 64 deletions(-) create mode 100755 scripts/release-notes.sh create mode 100755 scripts/test-release-notes.sh diff --git a/.github/workflows/ant-cli-release.yml b/.github/workflows/ant-cli-release.yml index 793231ef..5545038e 100644 --- a/.github/workflows/ant-cli-release.yml +++ b/.github/workflows/ant-cli-release.yml @@ -352,70 +352,11 @@ jobs: env: GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} run: | - # Extract the changelog section for the version being released — NOT - # "the first section". Blindly taking the first section shipped the - # ever-growing `[Unreleased]` blob as the notes of every release from - # 0.2.x through 0.3.4 (0.3.4's notes were byte-identical to 0.3.3's; - # the dual-stack entry re-shipped for four months). See V2-1107. - # - # `-rc.N` / `-beta.N` builds share the base version's section: the - # section is created when the train cuts rc.1 (`scripts/rotate-changelog.sh - # X.Y.Z` moves `[Unreleased]` into `[X.Y.Z] - `), so every prerelease - # and the final stable of one train ship the same, correct, notes. - version="${{ steps.meta.outputs.version }}" - base_version="${version%%-*}" - changelog=$(awk -v ver="$base_version" ' - BEGIN { gsub(/\./, "\\.", ver); pat = "^## \\[" ver "\\]" } - $0 ~ pat { found = 1; next } - /^## \[/ { if (found) exit } - found' CHANGELOG.md) - # A missing or empty section must never stop a release. The artifacts are - # fine and the notes are fixable after the fact with `gh release edit`, so - # failing closed here would only turn "missing notes" into "no release at - # all". The `::warning::` annotation keeps the omission loud on the run. - # Pre-tag enforcement, if we want it, belongs on the release-prep PR where - # a fix is cheap and nothing is blocked. - # - # Without a section, the notes fall back to GitHub's generated list of the - # PRs merged since the previous stable `ant-cli-v*` release, so a missed - # rotation ships the right list of changes rather than none (0.3.7 to 0.3.11 - # all shipped the placeholder). The placeholder is only the last resort, for - # when that API call fails. - if [ -z "$(echo "$changelog" | tr -d '[:space:]')" ]; then - tag="${{ steps.meta.outputs.tag }}" - # Highest stable tag below this base version. For `X.Y.Z-rc.N` and for - # `X.Y.Z` alike that is the previous train's stable, so every build of - # one train lists the same changes. - previous_tag=$(git ls-remote --tags --refs origin 'ant-cli-v*' \ - | sed 's|.*refs/tags/||' \ - | grep -E '^ant-cli-v[0-9]+\.[0-9]+\.[0-9]+$' \ - | { cat; echo "ant-cli-v${base_version}"; } \ - | sort -u -V \ - | grep -B1 -x "ant-cli-v${base_version}" \ - | grep -v -x "ant-cli-v${base_version}" || true) - generated="" - if [ -n "$previous_tag" ]; then - generated=$(gh api "repos/${GITHUB_REPOSITORY}/releases/generate-notes" \ - -f tag_name="$tag" -f previous_tag_name="$previous_tag" \ - --jq .body || true) - fi - if [ -n "$(echo "$generated" | tr -d '[:space:]')" ]; then - echo "::warning title=Missing changelog section::CHANGELOG.md has no non-empty '## [${base_version}]' section. Publishing GitHub's generated notes (${previous_tag}..${tag}); replace them with 'gh release edit' once the section exists, and rotate '## [Unreleased]' with 'scripts/rotate-changelog.sh ${base_version}' at the cut. See V2-1107." - { - echo "_No changelog entry was recorded for ${base_version}. These are the pull requests merged since ${previous_tag#ant-cli-v}; the notes will be corrected retroactively._" - echo - echo "$generated" - } > /tmp/changelog_entry.md - else - echo "::warning title=Missing changelog section::CHANGELOG.md has no non-empty '## [${base_version}]' section, and GitHub's generated notes were unavailable. Publishing with placeholder notes; correct them with 'gh release edit' and rotate '## [Unreleased]' with 'scripts/rotate-changelog.sh ${base_version}' at the cut. See V2-1107." - cat > /tmp/changelog_entry.md < /tmp/changelog_entry.md - fi + # The `## [X.Y.Z]` section of CHANGELOG.md for the tag's base version, else GitHub's + # generated list of the PRs since the previous stable release, else a placeholder. A + # missing section never stops the release. See scripts/release-notes.sh (V2-1107). + bash scripts/release-notes.sh "${{ steps.meta.outputs.version }}" \ + "${{ steps.meta.outputs.tag }}" /tmp/changelog_entry.md - name: generate release body shell: bash diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 6bbfa6df..dc7f6a74 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -20,6 +20,18 @@ jobs: components: rustfmt - run: cargo fmt --all -- --check + # The release workflow's notes fallback and the train cut's changelog rotation only run at + # release time, so their scripts are tested here, against fake `gh` and `git` (V2-1107). + release-scripts: + name: Release scripts + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - name: Test the release notes fallback + run: bash scripts/test-release-notes.sh + - name: Test the changelog rotation + run: bash scripts/test-rotate-changelog.sh + clippy: name: Clippy runs-on: ubuntu-latest diff --git a/scripts/release-notes.sh b/scripts/release-notes.sh new file mode 100755 index 00000000..51a00dc7 --- /dev/null +++ b/scripts/release-notes.sh @@ -0,0 +1,93 @@ +#!/usr/bin/env bash +# Write the GitHub release notes for an `ant-cli-v*` tag. The release workflow +# (`ant-cli-release.yml`) calls this; `scripts/test-release-notes.sh` tests it. See V2-1107. +# +# scripts/release-notes.sh [] +# +# the tag's version: X.Y.Z, or a prerelease X.Y.Z-rc.N / X.Y.Z-beta.N +# the tag being released, e.g. ant-cli-v0.3.12-rc.1 +# where to write the notes +# defaults to CHANGELOG.md +# +# The notes are, in order of preference: +# 1. The changelog section for the tag's base version, `## [X.Y.Z]`. NOT "the first section": +# blindly taking the first section shipped the ever-growing `[Unreleased]` blob as the notes +# of every release from 0.2.x through 0.3.4 (0.3.4's notes were byte-identical to 0.3.3's; +# the dual-stack entry re-shipped for four months). `-rc.N` / `-beta.N` builds share the +# base version's section, which the train creates when it cuts rc.1 +# (`scripts/rotate-changelog.sh X.Y.Z`), so every prerelease and the final stable of one +# train ship the same, correct, notes. +# 2. Without a section, GitHub's generated list of the PRs merged since the previous stable +# `ant-cli-v*` release, so a missed rotation ships the right list of changes rather than +# none (0.3.7 to 0.3.12 all shipped the placeholder). +# 3. A placeholder, when there is no previous stable release or the API call fails. +# +# A missing or empty section must never stop a release. The artifacts are fine and the notes are +# fixable after the fact with `gh release edit`, so failing closed here would only turn "missing +# notes" into "no release at all". The `::warning::` annotations keep the omission loud on the +# run. Pre-tag enforcement, if we want it, belongs on the release-prep PR where a fix is cheap and +# nothing is blocked. So this exits 0 whatever the API call does, and 1 only on bad arguments or +# a missing changelog. +# +# The API call needs GITHUB_REPOSITORY and GH_TOKEN, and the tags are listed from `origin`. + +set -euo pipefail + +if [ $# -lt 3 ] || [ $# -gt 4 ]; then + sed -n '5,10p' "$0" | sed 's/^# \{0,1\}//' >&2 + exit 1 +fi + +version="$1" +tag="$2" +out="$3" +changelog="${4:-CHANGELOG.md}" +base_version="${version%%-*}" + +if [ ! -f "$changelog" ]; then + echo "release-notes: $changelog not found" >&2 + exit 1 +fi + +section=$(awk -v ver="$base_version" ' + BEGIN { gsub(/\./, "\\.", ver); pat = "^## \\[" ver "\\]" } + $0 ~ pat { found = 1; next } + /^## \[/ { if (found) exit } + found' "$changelog") + +if [ -n "$(printf '%s' "$section" | tr -d '[:space:]')" ]; then + printf '%s\n' "$section" > "$out" + exit 0 +fi + +# Highest stable tag below this base version. For `X.Y.Z-rc.N` and for `X.Y.Z` alike that is the +# previous train's stable, so every build of one train lists the same changes. +previous_tag=$(git ls-remote --tags --refs origin 'ant-cli-v*' \ + | sed 's|.*refs/tags/||' \ + | grep -E '^ant-cli-v[0-9]+\.[0-9]+\.[0-9]+$' \ + | { cat; echo "ant-cli-v${base_version}"; } \ + | sort -u -V \ + | grep -B1 -x "ant-cli-v${base_version}" \ + | grep -v -x "ant-cli-v${base_version}" || true) + +generated="" +if [ -n "$previous_tag" ] && [ -n "${GITHUB_REPOSITORY:-}" ]; then + # Only a successful call counts. On an HTTP error `gh api` skips `--jq`, prints the raw JSON + # error body and exits non-zero, and that body would otherwise ship as the notes. + if ! generated=$(gh api "repos/${GITHUB_REPOSITORY}/releases/generate-notes" \ + -f tag_name="$tag" -f previous_tag_name="$previous_tag" --jq .body); then + generated="" + fi +fi + +if [ -n "$(printf '%s' "$generated" | tr -d '[:space:]')" ]; then + echo "::warning title=Missing changelog section::CHANGELOG.md has no non-empty '## [${base_version}]' section. Publishing GitHub's generated notes (${previous_tag}..${tag}); replace them with 'gh release edit' once the section exists, and rotate '## [Unreleased]' with 'scripts/rotate-changelog.sh ${base_version}' at the cut. See V2-1107." + { + echo "_No changelog entry was recorded for ${base_version}. These are the pull requests merged since ${previous_tag#ant-cli-v}; the notes will be corrected retroactively._" + echo + printf '%s\n' "$generated" + } > "$out" +else + echo "::warning title=Missing changelog section::CHANGELOG.md has no non-empty '## [${base_version}]' section, and GitHub's generated notes were unavailable. Publishing with placeholder notes; correct them with 'gh release edit' and rotate '## [Unreleased]' with 'scripts/rotate-changelog.sh ${base_version}' at the cut. See V2-1107." + echo "_No changelog entry was recorded for ${base_version}. See the commit history for the changes in this release; these notes will be corrected retroactively._" > "$out" +fi diff --git a/scripts/test-release-notes.sh b/scripts/test-release-notes.sh new file mode 100755 index 00000000..2e1268a1 --- /dev/null +++ b/scripts/test-release-notes.sh @@ -0,0 +1,134 @@ +#!/usr/bin/env bash +# Tests for scripts/release-notes.sh, with `git` and `gh` replaced by fakes on PATH. CI runs this +# in the `Release scripts` job. +# +# scripts/test-release-notes.sh [] + +set -euo pipefail + +here="$(cd "$(dirname "$0")" && pwd)" +script="${1:-$here/release-notes.sh}" +work="$(mktemp -d)" +trap 'rm -rf "$work"' EXIT +mkdir "$work/bin" + +# `git ls-remote` lists the tags in $FAKE_TAGS. +cat > "$work/bin/git" <<'EOF' +#!/usr/bin/env bash +[ "$1" = ls-remote ] || { echo "fake git: unexpected: $*" >&2; exit 2; } +for t in $FAKE_TAGS; do printf '%040d\trefs/tags/%s\n' 0 "$t"; done +EOF +# `gh` logs its arguments, prints $FAKE_GH_OUT and exits with $FAKE_GH_EXIT. +cat > "$work/bin/gh" <<'EOF' +#!/usr/bin/env bash +printf '%s\n' "$*" >> "$FAKE_GH_LOG" +printf '%s' "$FAKE_GH_OUT" +exit "$FAKE_GH_EXIT" +EOF +chmod +x "$work/bin/git" "$work/bin/gh" +export PATH="$work/bin:$PATH" GITHUB_REPOSITORY=WithAutonomi/ant-client FAKE_GH_LOG="$work/gh.log" +export FAKE_TAGS="ant-cli-v0.3.10 ant-cli-v0.3.11-rc.1 ant-cli-v0.3.11 ant-cli-v0.3.12-rc.1" + +cat > "$work/CHANGELOG.md" <<'EOF' +# Changelog + +## [Unreleased] + +### Added +- Something not released yet (#300) + +## [0.3.13] - 2026-11-05 + +## [0.3.11] - 2026-10-16 + +### Fixed +- A fix that shipped in 0.3.11 (#210) + +## [0.3.10] - 2026-10-09 + +### Added +- A feature that shipped in 0.3.10 (#200) +EOF + +pass=0 fail=0 +ok() { echo "ok $*"; pass=$((pass + 1)); } +bad() { echo "FAIL $*"; fail=$((fail + 1)); } + +# run : runs the script with the current fakes; sets $rc, $log and $notes. +run() { + : > "$FAKE_GH_LOG" + rm -f "$work/notes.md" + rc=0 + log="$(bash "$script" "$1" "$2" "$work/notes.md" "$work/CHANGELOG.md" 2>&1)" || rc=$? + notes="$(cat "$work/notes.md" 2> /dev/null || true)" +} +gh_called() { [ -s "$FAKE_GH_LOG" ]; } +placeholder() { [ "$notes" = "_No changelog entry was recorded for $1. See the commit history for the changes in this release; these notes will be corrected retroactively._" ]; } + +generated_body=$'## What\'s Changed\n* fix(core): something by @someone in #229' +error_body='{"message":"Not Found","documentation_url":"https://docs.github.com/rest/releases/releases#generate-release-notes-content-for-a-release","status":"404"}' + +# 1. A section for the version is used as it is, from the line after its heading, without +# calling the API. +FAKE_GH_OUT="$generated_body" FAKE_GH_EXIT=0 run 0.3.11 ant-cli-v0.3.11 +if [ "$rc" = 0 ] && [ "$notes" = $'\n### Fixed\n- A fix that shipped in 0.3.11 (#210)' ] && ! gh_called; then + ok "a stable tag ships its own section" +else bad "a stable tag ships its own section (rc=$rc, notes=$notes)"; fi + +# 2. A prerelease reads its base version's section. +FAKE_GH_OUT="$generated_body" FAKE_GH_EXIT=0 run 0.3.11-rc.1 ant-cli-v0.3.11-rc.1 +if [ "$rc" = 0 ] && printf '%s' "$notes" | grep -q '(#210)' && ! gh_called; then + ok "an rc tag ships its base version's section" +else bad "an rc tag ships its base version's section (rc=$rc, notes=$notes)"; fi + +# 3. 0.3.1 must not match the `## [0.3.11]` heading. +FAKE_GH_OUT="" FAKE_GH_EXIT=1 run 0.3.1 ant-cli-v0.3.1 +if [ "$rc" = 0 ] && ! printf '%s' "$notes" | grep -q '(#210)'; then + ok "0.3.1 does not take 0.3.11's section" +else bad "0.3.1 does not take 0.3.11's section (notes=$notes)"; fi + +# 4. No section: GitHub's generated notes since the previous stable tag. +FAKE_GH_OUT="$generated_body" FAKE_GH_EXIT=0 run 0.3.12-rc.1 ant-cli-v0.3.12-rc.1 +if [ "$rc" = 0 ] \ + && grep -q 'tag_name=ant-cli-v0.3.12-rc.1' "$FAKE_GH_LOG" \ + && grep -q 'previous_tag_name=ant-cli-v0.3.11' "$FAKE_GH_LOG" \ + && [ "$(printf '%s\n' "$notes" | head -1)" = "_No changelog entry was recorded for 0.3.12. These are the pull requests merged since 0.3.11; the notes will be corrected retroactively._" ] \ + && printf '%s' "$notes" | grep -qxF '* fix(core): something by @someone in #229' \ + && printf '%s' "$log" | grep -q "Publishing GitHub's generated notes"; then + ok "a missing section gets the generated notes since the previous stable" +else bad "a missing section gets the generated notes since the previous stable (rc=$rc, gh: $(cat "$FAKE_GH_LOG"), notes=$notes)"; fi + +# 5. An empty hand-written heading counts as missing. +FAKE_GH_OUT="$generated_body" FAKE_GH_EXIT=0 run 0.3.13 ant-cli-v0.3.13 +if [ "$rc" = 0 ] && gh_called && printf '%s' "$notes" | grep -q '#229'; then + ok "an empty section heading falls back to the generated notes" +else bad "an empty section heading falls back to the generated notes (notes=$notes)"; fi + +# 6. The API call fails with a JSON error body: placeholder, and the error is not published. +FAKE_GH_OUT="$error_body" FAKE_GH_EXIT=1 run 0.3.12-rc.1 ant-cli-v0.3.12-rc.1 +if [ "$rc" = 0 ] && gh_called && placeholder 0.3.12 \ + && ! printf '%s' "$notes" | grep -qE 'Not Found|documentation_url' \ + && printf '%s' "$log" | grep -q 'Publishing with placeholder notes'; then + ok "a failed API call gets the placeholder, not its error body" +else bad "a failed API call gets the placeholder, not its error body (rc=$rc, notes=$notes)"; fi + +# 7. A successful call with an empty body: placeholder. +FAKE_GH_OUT=$'\n' FAKE_GH_EXIT=0 run 0.3.12-rc.1 ant-cli-v0.3.12-rc.1 +if [ "$rc" = 0 ] && gh_called && placeholder 0.3.12; then + ok "empty generated notes get the placeholder" +else bad "empty generated notes get the placeholder (notes=$notes)"; fi + +# 8. No previous stable release: placeholder, and the API is not called. +FAKE_TAGS="ant-cli-v0.1.1-rc.1" FAKE_GH_OUT="$generated_body" FAKE_GH_EXIT=0 run 0.1.1 ant-cli-v0.1.1 +if [ "$rc" = 0 ] && ! gh_called && placeholder 0.1.1; then + ok "no previous stable release gets the placeholder without an API call" +else bad "no previous stable release gets the placeholder without an API call (notes=$notes)"; fi + +# 9. Bad arguments and a missing changelog exit 1. +rc=0; bash "$script" 0.3.12 ant-cli-v0.3.12 > /dev/null 2>&1 || rc=$? +if [ "$rc" = 1 ]; then ok "too few arguments exit 1"; else bad "too few arguments exit $rc"; fi +rc=0; bash "$script" 0.3.12 ant-cli-v0.3.12 "$work/notes.md" "$work/missing.md" > /dev/null 2>&1 || rc=$? +if [ "$rc" = 1 ]; then ok "a missing changelog exits 1"; else bad "a missing changelog exits $rc"; fi + +echo "$pass passed, $fail failed" +[ "$fail" = 0 ] From 94b01ec046cf974c47916c3bccf69c797be73104 Mon Sep 17 00:00:00 2001 From: Nic-dorman Date: Tue, 27 Oct 2026 14:49:10 +0000 Subject: [PATCH 5/5] docs: make AGENTS.md the full guide, and CLAUDE.md an import of it CLAUDE.md's guide moves into AGENTS.md unchanged, next to the changelog rule, so agents that read only AGENTS.md (Codex, OpenCode and others) get the whole guide and not just that one rule. CLAUDE.md is now `@AGENTS.md`, which Claude Code expands, so Claude still loads the same text and there is a single copy to keep current. The Claude Code E2E skill section is labelled as Claude-only. Co-Authored-By: Claude Opus 5.5 --- AGENTS.md | 209 ++++++++++++++++++++++++++++++++++++++++++++++++++++- CLAUDE.md | 211 +----------------------------------------------------- 2 files changed, 209 insertions(+), 211 deletions(-) diff --git a/AGENTS.md b/AGENTS.md index 6fc59e72..d350bc09 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,7 +1,212 @@ # AGENTS.md -Instructions for any coding agent working in this repo (Claude Code, Codex, OpenCode, -pi and others). Claude Code loads this file through `CLAUDE.md`. +This is the development guide for the `autonomi-client` project (the Unified CLI + `ant-core` +library), for people and for every coding agent working in this repo (Claude Code, Codex, +OpenCode, pi and others). Claude Code loads it through `CLAUDE.md`. + +## Project Overview + +A Rust workspace with two crates: +- **ant-core** (`ant-core/`) — Headless library containing all business logic. No UI code, no terminal output. +- **ant-cli** (`ant-cli/`) — Thin CLI adapter. Parses arguments via clap, calls `ant-core`, formats output. The binary is named `ant`. + +## Design Principles + +These are non-negotiable. Every contribution must follow them. + +### 1. No printing, no UI assumptions in ant-core +All `ant-core` functions return structured result types. Long-running operations report progress through the `ProgressReporter` trait and lifecycle events through the `EventListener` trait. Zero `println!`, `eprintln!`, or direct terminal output. + +### 2. Option structs replace long parameter lists +Every operation takes an options struct rather than many function parameters. This keeps APIs ergonomic and forward-compatible. + +### 3. Async-first +All I/O operations are `async`. The daemon runs on a tokio runtime. + +### 4. Serializable types +All result types and status models derive `serde::Serialize` + `serde::Deserialize`. This enables `--json` CLI output, REST API responses, and frontend integration. + +### 5. Cancellation support +Long-running operations accept a `CancellationToken` so any frontend can abort cleanly. + +### 6. No OS service managers +Nodes run as regular processes managed by the daemon. The only platform-specific code is process detachment, isolated in `ant-core::node::process::detach`. + +### 7. AI agent friendly +The REST API is the primary integration point for AI agents. All operations available through the CLI are also available through the REST API. The API is self-describing via OpenAPI spec at `/api/v1/openapi.json`. Real-time events are available via SSE. + +## Architecture + +``` +┌──────────┐ HTTP ┌──────────────────────────────────┐ +│ ant CLI │──────────────▶│ ant daemon │ +└──────────┘ 127.0.0.1 │ │ + │ ┌────────────┐ ┌────────────┐ │ +┌──────────┐ HTTP │ │ antnode 1 │ │ antnode 2 │ │ +│ Web UI │──────────────▶│ └────────────┘ └────────────┘ │ +└──────────┘ │ ┌────────────┐ ┌────────────┐ │ + │ │ antnode 3 │ │ antnode N │ │ +┌──────────┐ HTTP │ └────────────┘ └────────────┘ │ +│ AI Agent │──────────────▶│ │ +└──────────┘ └──────────────────────────────────┘ + │ + ▼ + node_registry.json +``` + +The daemon is a long-running process that manages all node processes on the machine, exposing a REST API on localhost. No admin privileges required on any platform. + +## Crate Structure + +``` +ant-core/src/ +├── lib.rs +├── error.rs # Unified error type (thiserror) +├── config.rs # Platform-appropriate data/log directory paths +├── data/ # Data storage and retrieval +│ ├── mod.rs # Re-exports (Client, DataMap, Wallet, PaymentMode, etc.) +│ ├── error.rs # Data operation errors +│ ├── network.rs # P2P network wrapper (DHT, peer discovery) +│ └── client/ # High-level client API +│ ├── mod.rs # Client, ClientConfig +│ ├── chunk.rs # chunk_put, chunk_get, chunk_exists +│ ├── data.rs # data_upload, data_download, data_map_store/fetch +│ ├── file.rs # file_upload, file_download (streaming self-encryption) +│ ├── payment.rs # pay_for_storage, approve_token_spend +│ ├── quote.rs # get_store_quotes from network peers +│ ├── merkle.rs # Merkle batch payment (PaymentMode enum) +│ ├── pointer.rs # pointer_create/update/get/resolve (ADR-0016), transfer/finality (ADR-0018) +│ └── cache.rs # In-memory LRU chunk cache +│ └── manifest/ # File manifests and ant:// links (ADR-0006) +│ ├── mod.rs # Manifest, ManifestEntry, ContentRef, encode/decode, limits +│ ├── path.rs # Portable path rules (cross-platform, enforced everywhere) +│ ├── link.rs # ant://
and ant://manifest/ links +│ ├── file.rs # .ant file read/write +│ ├── history.rs # Upload history: one .ant per upload under /uploads +│ ├── wire.rs # Manifest-owned DataMap layout; pre-decode structure check +│ ├── build.rs # ManifestBuilder (uploads files, embedded or compact refs) +│ ├── embed.rs # Root-vs-shrunk DataMap choice for embedding +│ ├── compact.rs # Plan/publish/apply compaction (embedded DataMaps -> addresses) +│ └── extract.rs # Containment-safe extraction with per-entry results +└── node/ # Node management + ├── mod.rs # add_nodes, remove_node, reset + ├── types.rs # DaemonConfig, DaemonStatus, NodeConfig, NodeInfo, AddNodeOpts, etc. + ├── events.rs # NodeEvent enum, EventListener trait + ├── binary.rs # Binary resolution (download/cache/validate), ProgressReporter trait + ├── registry.rs # Node registry (CRUD, JSON persistence, file locking) + ├── devnet.rs # LocalDevnet (local network + Anvil EVM blockchain) + ├── daemon/ + │ ├── mod.rs + │ ├── client.rs # Daemon client API (start/stop/status via HTTP) + │ ├── server.rs # HTTP server (axum), REST API handlers + │ ├── supervisor.rs # Process supervision with backoff + │ └── forward/ # Opt-in beta log forwarding (V2-1021) + │ ├── mod.rs # Status/result types, enable-request merging + │ ├── config.rs # Persisted opt-in + token (0600), LogLevel + │ ├── parse.rs # Log line -> LogEvent (text and JSON layouts) + │ ├── tail.rs # Rotation-aware tailing, multi-line events + │ ├── offsets.rs # Persisted tail positions + │ ├── document.rs # Tagging into the beta index's field names + │ ├── sink.rs # LogSink trait, bounded queue, batching, retry + │ ├── es.rs # Elasticsearch _bulk sink + │ └── runner.rs # The background forwarding task + └── process/ + ├── mod.rs + ├── spawn.rs # Spawning node processes + └── detach.rs # Platform-specific session detachment + +ant-cli/src/ +├── main.rs # Entry point, client/wallet/EVM initialization +├── cli.rs # Top-level clap definition +└── commands/ + ├── data/ + │ ├── file.rs # ant file upload/download + │ ├── chunk.rs # ant chunk put/get + │ ├── pointer.rs # ant pointer keygen/address/create/update/get/resolve/transfer/finality/controller + │ ├── manifest.rs # ant manifest create/list/show/link/export/download + │ └── wallet.rs # ant wallet address/balance + └── node/ + ├── mod.rs + ├── add.rs # ant node add command + ├── daemon.rs # daemon start/stop/status/info/run commands + ├── logs.rs # ant node logs forward enable/disable/status + ├── start.rs # ant node start + ├── stop.rs # ant node stop + ├── status.rs # ant node status + └── reset.rs # ant node reset +``` + +## Common Commands + +```bash +cargo check # Fast compilation check +cargo test --all # Run all unit and integration tests +cargo clippy --all-targets --all-features -- -D warnings # Lint +cargo fmt --all -- --check # Format check +cargo run --bin ant -- --help # Run the CLI +``` + +## Key Conventions + +- **Error handling**: Use `thiserror` in ant-core, `anyhow` in ant-cli. ant-core defines `error::Error` and `error::Result`. +- **CLI output**: All commands support `--json` global flag. Human-readable output by default, structured JSON when `--json` is passed. +- **Hidden commands**: `ant node daemon run` is hidden (runs daemon in foreground, used internally by `daemon start`). +- **REST API conflict responses**: All 409 responses include a `current_state` field so retrying clients can confirm desired state already exists. +- **Platform paths**: Use `ant_core::config::data_dir()` and `ant_core::config::log_dir()` for platform-appropriate paths. Never hardcode paths. +- **OpenAPI schema**: Types exposed in the REST API derive `utoipa::ToSchema`. Use `#[schema(value_type = String)]` for `PathBuf` fields. +- **Tests**: Unit tests go in `#[cfg(test)] mod tests` inside the source file. Integration tests go in `ant-core/tests/`. +- **Progress reporting**: Long-running operations (e.g., binary downloads) accept a `&dyn ProgressReporter` from `ant_core::node::binary`. CLI provides a terminal-printing implementation; use `NoopProgress` in tests and daemon API handlers. +- **Registry file locking**: Use `NodeRegistry::load_locked()` for read-modify-write operations to prevent concurrent CLI invocations from corrupting the registry. The returned `File` handle holds the lock until dropped. +- **Dual-path CLI commands**: Commands that modify the registry (like `ant node add`) check if the daemon is running. If so, they route through the REST API; otherwise, they operate directly on the registry file. +- **Binary source resolution**: Node binary sources are represented by the `BinarySource` enum (Latest, Version, Url, LocalPath). Download variants are stubbed until release infrastructure is available. +- **Manifests never touch the network**: a manifest (`.ant` file or `ant://manifest/...` link) + is shared off-network; only the files it references live on Autonomi. `ant://` is always + a plain file link. Every upload (`ant file upload`, `ant manifest create`) is also recorded as + a manifest under `/uploads/`; `ant manifest list` shows them and `show`/`link`/`export`/`download` + accept a record id. Entry paths obey the portable rules in `data/manifest/path.rs` at build, + decode and extract time, and extraction never follows symlinks inside the output directory. + See `docs/adr/ADR-0006-file-manifests-and-links.md`. +- **Log forwarding is opt-in and node-logging-dependent**: `ant node logs forward enable` is the + consent act. It only forwards nodes whose `NodeConfig.log_dir` is `Some` — node file logging is + off unless the node was added with `--log-dir-path` — and reports the nodes it is skipping. + Enabling never restarts a node or changes its arguments. See + `ant-core/src/node/daemon/forward/` and the V2-1016 ingest contract documented at the top of + `forward/es.rs` (`create` actions, per-position `items[].status`, deterministic `_id`s). + +## E2E Test Skill (Claude Code) + +The project includes an E2E test skill at `.claude/skills/e2e-node-management-test/SKILL.md` that tests all node management features against a real testnet (invoked via `/e2e-node-management-test`). When adding new `ant node` subcommands or changing existing node management behavior, update this command to cover the new or changed functionality. + +## Linear Project + +This project tracks work under the "Unified CLI" project in Linear (team: v2.0). + +## Pull requests: fill the template + +When you open a pull request, you MUST use `.github/PULL_REQUEST_TEMPLATE.md` as +the PR body and fill it in completely — do not open a PR with an empty or ad-hoc +description. + +- **Fill every field.** Leave nothing blank; if a value isn't determinable, ask + before opening the PR. +- **Link the Linear issue with a closing magic word** — write `Closes V2-123` in + the `## Linear issue` section, one line per issue. Any of Linear's closing + words works, in any tense (`close` / `fix` / `resolve` / `complete` / + `implement`, plus their `-s`, `-d` and `-ing` forms), and the key may be a + `linear.app` issue URL. A bare `V2-123` does **not** link the PR at all, and + the linking-only words (`ref`, `part of`, `towards`, `relates to`) attach it + without driving the Merged transition — CI rejects both. An issue key in the + branch name or PR title also links, but write the closing form anyway; it is + what moves the issue to Merged when the PR lands on `main`. +- **Check exactly one Risk tier box and exactly one Semver impact box.** Propose + them from the change; a human confirms them at review. +- An **ADR link is required for Tier 2/3**. + +CI enforces the Linear link and the template fields (`linear-link` and +`pr-template` status checks); a PR that omits them fails its checks. +Release-promotion PRs — a `chore(release):` title from an `rc-*` head branch — +are exempt from both checks (they are mechanical version-bump PRs generated by +the release process). ## Changelog diff --git a/CLAUDE.md b/CLAUDE.md index 29f012d3..8dca15fb 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,213 +1,6 @@ # CLAUDE.md -This is the development guide for the `autonomi-client` project (the Unified CLI + `ant-core` library). - -## Project Overview - -A Rust workspace with two crates: -- **ant-core** (`ant-core/`) — Headless library containing all business logic. No UI code, no terminal output. -- **ant-cli** (`ant-cli/`) — Thin CLI adapter. Parses arguments via clap, calls `ant-core`, formats output. The binary is named `ant`. - -## Design Principles - -These are non-negotiable. Every contribution must follow them. - -### 1. No printing, no UI assumptions in ant-core -All `ant-core` functions return structured result types. Long-running operations report progress through the `ProgressReporter` trait and lifecycle events through the `EventListener` trait. Zero `println!`, `eprintln!`, or direct terminal output. - -### 2. Option structs replace long parameter lists -Every operation takes an options struct rather than many function parameters. This keeps APIs ergonomic and forward-compatible. - -### 3. Async-first -All I/O operations are `async`. The daemon runs on a tokio runtime. - -### 4. Serializable types -All result types and status models derive `serde::Serialize` + `serde::Deserialize`. This enables `--json` CLI output, REST API responses, and frontend integration. - -### 5. Cancellation support -Long-running operations accept a `CancellationToken` so any frontend can abort cleanly. - -### 6. No OS service managers -Nodes run as regular processes managed by the daemon. The only platform-specific code is process detachment, isolated in `ant-core::node::process::detach`. - -### 7. AI agent friendly -The REST API is the primary integration point for AI agents. All operations available through the CLI are also available through the REST API. The API is self-describing via OpenAPI spec at `/api/v1/openapi.json`. Real-time events are available via SSE. - -## Architecture - -``` -┌──────────┐ HTTP ┌──────────────────────────────────┐ -│ ant CLI │──────────────▶│ ant daemon │ -└──────────┘ 127.0.0.1 │ │ - │ ┌────────────┐ ┌────────────┐ │ -┌──────────┐ HTTP │ │ antnode 1 │ │ antnode 2 │ │ -│ Web UI │──────────────▶│ └────────────┘ └────────────┘ │ -└──────────┘ │ ┌────────────┐ ┌────────────┐ │ - │ │ antnode 3 │ │ antnode N │ │ -┌──────────┐ HTTP │ └────────────┘ └────────────┘ │ -│ AI Agent │──────────────▶│ │ -└──────────┘ └──────────────────────────────────┘ - │ - ▼ - node_registry.json -``` - -The daemon is a long-running process that manages all node processes on the machine, exposing a REST API on localhost. No admin privileges required on any platform. - -## Crate Structure - -``` -ant-core/src/ -├── lib.rs -├── error.rs # Unified error type (thiserror) -├── config.rs # Platform-appropriate data/log directory paths -├── data/ # Data storage and retrieval -│ ├── mod.rs # Re-exports (Client, DataMap, Wallet, PaymentMode, etc.) -│ ├── error.rs # Data operation errors -│ ├── network.rs # P2P network wrapper (DHT, peer discovery) -│ └── client/ # High-level client API -│ ├── mod.rs # Client, ClientConfig -│ ├── chunk.rs # chunk_put, chunk_get, chunk_exists -│ ├── data.rs # data_upload, data_download, data_map_store/fetch -│ ├── file.rs # file_upload, file_download (streaming self-encryption) -│ ├── payment.rs # pay_for_storage, approve_token_spend -│ ├── quote.rs # get_store_quotes from network peers -│ ├── merkle.rs # Merkle batch payment (PaymentMode enum) -│ ├── pointer.rs # pointer_create/update/get/resolve (ADR-0016), transfer/finality (ADR-0018) -│ └── cache.rs # In-memory LRU chunk cache -│ └── manifest/ # File manifests and ant:// links (ADR-0006) -│ ├── mod.rs # Manifest, ManifestEntry, ContentRef, encode/decode, limits -│ ├── path.rs # Portable path rules (cross-platform, enforced everywhere) -│ ├── link.rs # ant://
and ant://manifest/ links -│ ├── file.rs # .ant file read/write -│ ├── history.rs # Upload history: one .ant per upload under /uploads -│ ├── wire.rs # Manifest-owned DataMap layout; pre-decode structure check -│ ├── build.rs # ManifestBuilder (uploads files, embedded or compact refs) -│ ├── embed.rs # Root-vs-shrunk DataMap choice for embedding -│ ├── compact.rs # Plan/publish/apply compaction (embedded DataMaps -> addresses) -│ └── extract.rs # Containment-safe extraction with per-entry results -└── node/ # Node management - ├── mod.rs # add_nodes, remove_node, reset - ├── types.rs # DaemonConfig, DaemonStatus, NodeConfig, NodeInfo, AddNodeOpts, etc. - ├── events.rs # NodeEvent enum, EventListener trait - ├── binary.rs # Binary resolution (download/cache/validate), ProgressReporter trait - ├── registry.rs # Node registry (CRUD, JSON persistence, file locking) - ├── devnet.rs # LocalDevnet (local network + Anvil EVM blockchain) - ├── daemon/ - │ ├── mod.rs - │ ├── client.rs # Daemon client API (start/stop/status via HTTP) - │ ├── server.rs # HTTP server (axum), REST API handlers - │ ├── supervisor.rs # Process supervision with backoff - │ └── forward/ # Opt-in beta log forwarding (V2-1021) - │ ├── mod.rs # Status/result types, enable-request merging - │ ├── config.rs # Persisted opt-in + token (0600), LogLevel - │ ├── parse.rs # Log line -> LogEvent (text and JSON layouts) - │ ├── tail.rs # Rotation-aware tailing, multi-line events - │ ├── offsets.rs # Persisted tail positions - │ ├── document.rs # Tagging into the beta index's field names - │ ├── sink.rs # LogSink trait, bounded queue, batching, retry - │ ├── es.rs # Elasticsearch _bulk sink - │ └── runner.rs # The background forwarding task - └── process/ - ├── mod.rs - ├── spawn.rs # Spawning node processes - └── detach.rs # Platform-specific session detachment - -ant-cli/src/ -├── main.rs # Entry point, client/wallet/EVM initialization -├── cli.rs # Top-level clap definition -└── commands/ - ├── data/ - │ ├── file.rs # ant file upload/download - │ ├── chunk.rs # ant chunk put/get - │ ├── pointer.rs # ant pointer keygen/address/create/update/get/resolve/transfer/finality/controller - │ ├── manifest.rs # ant manifest create/list/show/link/export/download - │ └── wallet.rs # ant wallet address/balance - └── node/ - ├── mod.rs - ├── add.rs # ant node add command - ├── daemon.rs # daemon start/stop/status/info/run commands - ├── logs.rs # ant node logs forward enable/disable/status - ├── start.rs # ant node start - ├── stop.rs # ant node stop - ├── status.rs # ant node status - └── reset.rs # ant node reset -``` - -## Common Commands - -```bash -cargo check # Fast compilation check -cargo test --all # Run all unit and integration tests -cargo clippy --all-targets --all-features -- -D warnings # Lint -cargo fmt --all -- --check # Format check -cargo run --bin ant -- --help # Run the CLI -``` - -## Key Conventions - -- **Error handling**: Use `thiserror` in ant-core, `anyhow` in ant-cli. ant-core defines `error::Error` and `error::Result`. -- **CLI output**: All commands support `--json` global flag. Human-readable output by default, structured JSON when `--json` is passed. -- **Hidden commands**: `ant node daemon run` is hidden (runs daemon in foreground, used internally by `daemon start`). -- **REST API conflict responses**: All 409 responses include a `current_state` field so retrying clients can confirm desired state already exists. -- **Platform paths**: Use `ant_core::config::data_dir()` and `ant_core::config::log_dir()` for platform-appropriate paths. Never hardcode paths. -- **OpenAPI schema**: Types exposed in the REST API derive `utoipa::ToSchema`. Use `#[schema(value_type = String)]` for `PathBuf` fields. -- **Tests**: Unit tests go in `#[cfg(test)] mod tests` inside the source file. Integration tests go in `ant-core/tests/`. -- **Progress reporting**: Long-running operations (e.g., binary downloads) accept a `&dyn ProgressReporter` from `ant_core::node::binary`. CLI provides a terminal-printing implementation; use `NoopProgress` in tests and daemon API handlers. -- **Registry file locking**: Use `NodeRegistry::load_locked()` for read-modify-write operations to prevent concurrent CLI invocations from corrupting the registry. The returned `File` handle holds the lock until dropped. -- **Dual-path CLI commands**: Commands that modify the registry (like `ant node add`) check if the daemon is running. If so, they route through the REST API; otherwise, they operate directly on the registry file. -- **Binary source resolution**: Node binary sources are represented by the `BinarySource` enum (Latest, Version, Url, LocalPath). Download variants are stubbed until release infrastructure is available. -- **Manifests never touch the network**: a manifest (`.ant` file or `ant://manifest/...` link) - is shared off-network; only the files it references live on Autonomi. `ant://` is always - a plain file link. Every upload (`ant file upload`, `ant manifest create`) is also recorded as - a manifest under `/uploads/`; `ant manifest list` shows them and `show`/`link`/`export`/`download` - accept a record id. Entry paths obey the portable rules in `data/manifest/path.rs` at build, - decode and extract time, and extraction never follows symlinks inside the output directory. - See `docs/adr/ADR-0006-file-manifests-and-links.md`. -- **Log forwarding is opt-in and node-logging-dependent**: `ant node logs forward enable` is the - consent act. It only forwards nodes whose `NodeConfig.log_dir` is `Some` — node file logging is - off unless the node was added with `--log-dir-path` — and reports the nodes it is skipping. - Enabling never restarts a node or changes its arguments. See - `ant-core/src/node/daemon/forward/` and the V2-1016 ingest contract documented at the top of - `forward/es.rs` (`create` actions, per-position `items[].status`, deterministic `_id`s). - -## E2E Test Skill - -The project includes an E2E test skill at `.claude/skills/e2e-node-management-test/SKILL.md` that tests all node management features against a real testnet (invoked via `/e2e-node-management-test`). When adding new `ant node` subcommands or changing existing node management behavior, update this command to cover the new or changed functionality. - -## Linear Project - -This project tracks work under the "Unified CLI" project in Linear (team: v2.0). - -## Pull requests: fill the template - -When you open a pull request, you MUST use `.github/PULL_REQUEST_TEMPLATE.md` as -the PR body and fill it in completely — do not open a PR with an empty or ad-hoc -description. - -- **Fill every field.** Leave nothing blank; if a value isn't determinable, ask - before opening the PR. -- **Link the Linear issue with a closing magic word** — write `Closes V2-123` in - the `## Linear issue` section, one line per issue. Any of Linear's closing - words works, in any tense (`close` / `fix` / `resolve` / `complete` / - `implement`, plus their `-s`, `-d` and `-ing` forms), and the key may be a - `linear.app` issue URL. A bare `V2-123` does **not** link the PR at all, and - the linking-only words (`ref`, `part of`, `towards`, `relates to`) attach it - without driving the Merged transition — CI rejects both. An issue key in the - branch name or PR title also links, but write the closing form anyway; it is - what moves the issue to Merged when the PR lands on `main`. -- **Check exactly one Risk tier box and exactly one Semver impact box.** Propose - them from the change; a human confirms them at review. -- An **ADR link is required for Tier 2/3**. - -CI enforces the Linear link and the template fields (`linear-link` and -`pr-template` status checks); a PR that omits them fails its checks. -Release-promotion PRs — a `chore(release):` title from an `rc-*` head branch — -are exempt from both checks (they are mechanical version-bump PRs generated by -the release process). - -## Shared agent instructions - -Instructions for every coding agent, not just Claude, live in `AGENTS.md`: +This repo's development guide is `AGENTS.md`, shared by every coding agent. Claude Code loads it +through the import below. Add new guidance to `AGENTS.md`, not here. @AGENTS.md