Skip to content

Discover nested source-package members so the alias-drift guard sees them - #338

Merged
MaxGhenis merged 3 commits into
mainfrom
nzhub/nested-package-discovery
Oct 11, 2026
Merged

MaxGhenis merged 3 commits into
mainfrom
nzhub/nested-package-discovery

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Follow-up to #335 (finding 1 of its review) and #336 (round-2 docstring nit).

The problem

discover_source_package_dirs in chronicle/source_package.py globbed only */*/source_package.yaml, so it couldn't see nested package members such as #335's stats_nz/census_2023_family_type/rc_01. Two checks rely on it: find_alias_map_drift (the alias-drift guard from #78) and the default-bundle coverage check.

The review of #335 showed the effect: removing a nested member's alias left the drift test green. Only #335's own package test caught it, so a future nested package added without an alias would be dropped from the bundle silently.

The fix

  • Discovery searches recursively, with os.walk(followlinks=True).
    • Symlinked directories are followed at every depth and reported under their link path. The old two-level glob followed them at its two levels; Path.rglob, used in this PR's first commit, does not follow them by default.
    • A directory whose real path is already in its own ancestry is not entered, so a symlink cycle ends the walk. Two links to one package are both reported, as under the old glob.
    • On main this finds the same 333 packages as before, with none missing and none unmapped. That equals the old glob and the alias targets.
    • On Add Stats NZ 2023 Census household and family counts by regional council #335's branch the review measured 367 discovered directories, exactly the 367 alias targets.
  • The _is_missing_object_diagnostic docstring in chronicle/artifacts.py now states the known limit: an error line that merely mentions a missing-object marker still counts as missing.

Tests

  • Five new cases in tests/test_source_package_alias_drift.py, each red under an in-memory mutant:
    • nested members are discovered, and an unmapped nested member is reported (mutant: the two-level glob);
    • a symlinked package and a symlinked family's nested member are found, and flagged when unaliased (mutant: plain rglob);
    • a directory and a link to it are both reported (mutants: plain rglob, and a global visited set in place of the ancestry check);
    • a cycle terminates with the one real package (mutant: no cycle check).
  • Test runs: tests/test_source_package_alias_drift.py 17 passed; tests/test_chronicle_verify_raw.py 37 passed.
  • Ruff: 0.14.10 check passes. ruff format --check flags chronicle/source_package.py for an alias line that main already has, which this PR doesn't touch; CI doesn't run a format check.

Repository tooling only: no source package, fact or bundle pin changes.

Review

An independent GPT-6.1 Sol review at 202f8211 returned APPROVE_WITH_NITS, "Needs Max: no". Its runs: 14 and 37 passed; both new tests red under the old glob; ruff 0.14.10 check passes; its discovery probe matched 333 packages to aliases and showed nested, hidden and vendor manifests tripping the guard.

  1. rglob skipped directory symlinks the old glob traversed, so a symlinked package without an alias escaped the guard. No package in the repo is a symlink today. Fixed in 866d9211, as above.
  2. Add Stats NZ 2023 Census household and family counts by regional council #335's historical review log isn't attached (informational). The reviewer reproduced the mechanism and confirmed 367 manifests and alias targets on Add Stats NZ 2023 Census household and family counts by regional council #335's branch independently.

🤖 Generated with Claude Code

MaxGhenis and others added 2 commits October 11, 2026 12:21
…them

discover_source_package_dirs globbed only */*/source_package.yaml, so nested
members (such as chronicle#335's rc_XX members) were invisible to
find_alias_map_drift and the default-bundle coverage check: removing a nested
member's alias was not reported (review of #335, finding 1). Search
recursively instead. On main this finds the same 333 packages as before (none
missing, none unmapped); on #335's branch the review measured 367, exactly the
alias targets.

Also correct the _is_missing_object_diagnostic docstring, which still said
incidental mentions stay a fetch error (review r2 of #336).

Tests: two new cases (nested members discovered; an unmapped nested member
reported), both red under the old two-level glob via an in-memory mutant.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Path.rglob does not follow directory symlinks by default, so the
recursive discovery from the previous commit lost a case the old
two-level glob covered: a symlinked package with no alias escaped the
drift guard (review finding on #338).

Discovery now walks with os.walk(followlinks=True) and reports each
package under its link path. It skips only a directory whose real path
is already in its own ancestry, so a cycle ends the walk while two
links to one package are both reported, as under the old glob.

Tests: a symlinked package and a symlinked family with a nested member
are found and flagged when unaliased; a directory and a link to it are
both reported; a cycle terminates. Each fails under its own in-memory
mutant (plain rglob; a global visited set; no cycle check). On the
real tree discovery still equals the old glob and the 333 alias
targets.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@MaxGhenis
MaxGhenis merged commit b58fb8b into main Oct 11, 2026
3 checks passed
@MaxGhenis

Copy link
Copy Markdown
Contributor Author

Merged by the NZ hub (session local_e407a527) at 18545cd with --merge. Gates: gh pr checks exit 0; MERGEABLE; not draft; no CHANGES_REQUESTED review. Reviewed head: 866d921; the merged head differs only by a base update, and the PR's own changed lines are identical to the reviewed head's. Independent GPT-6.1 Sol reviews: r1 at 202f821 APPROVE_WITH_NITS (Needs Max: no; one finding: rglob skipped directory symlinks the old glob followed, fixed in 866d921), r2 at 866d921 APPROVE_WITH_NITS (Needs Max: no; 17 + 37 passed; all five new tests red under mutants; seven symlink scenarios passed; only nit informational: the original #335 review log is not attached, and the 367 count was independently reproduced). 18545cd merges main (#335) with no change to this PR's lines; on main+#335 discovery equals the 369 alias targets with no drift, and removing a nested alias is now reported. Reports: ~/reviews/nz-hub/jobs/review-chronicle-338/.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant