Generate TUF test fixtures at test time instead of committing them - #2004
Open
nellshamrell wants to merge 1 commit into
Open
Generate TUF test fixtures at test time instead of committing them#2004nellshamrell wants to merge 1 commit into
nellshamrell wants to merge 1 commit into
Conversation
TUF metadata expires, so committing pre-built TUF repositories to the tree means the test suite breaks purely with the passage of time. Scaffolding hardcodes a six month expiry with no knob to change it, so regenerating the fixtures only ever buys another six months: these fixtures were already regenerated once (sigstore#1999) and, unaddressed, would have expired again and reddened CI in January 2027. Add internal/tuftest, which signs a fresh TUF repository on every call with an expiry relative to now, and use it in pkg/tuf, pkg/apis/policy/v1alpha1, and pkg/reconciler/trustroot in place of the inline base64 blobs and committed tarballs those packages used before. The reconciler's golden files are unaffected because they are derived from the committed keys and certificates rather than from the TUF metadata wrapping them, so only the TUF tarballs and root.json files go away there. The keys and certificates stay committed: they are valid for ten years and keeping them makes the expected reconciler output stable. hack/gentestdata now emits only those, and the TUF generation it used to do is removed. The helper deliberately does not import policy-controller/pkg/tuf, since pkg/tuf's tests are in-package (`package tuf`) and that would form an import cycle; it uses go-tuf directly and scaffolding's CompressFS. It also roots each repository at t.TempDir() rather than the single fixed /tmp/tuf path the old local helper in pkg/tuf used, which could not be called twice in one process. pkg/tuf's TestCompressUncompressFS takes a directory from the helper rather than a prebuilt tarball, so it keeps exercising this package's own CompressFS rather than scaffolding's. TestClientFromSerializedMirrorExpired signs a short-lived repository and waits for it to lapse, so that making fixtures permanently fresh cannot silently mask a loss of expiry validation; it waits on the expiry recorded in the generated metadata rather than the one requested, since go-tuf rounds to the nearest second. Finally, add a guardrail test that scans the tree for TUF metadata in any of the three shapes these fixtures have taken historically - metadata files, tarred repositories, and base64 blobs inlined in Go source - so an expiring fixture cannot quietly reappear. It lists files via `git ls-files` so local build output cannot fail it, and has its own tests covering all three shapes so it cannot decay into always passing. Fixes sigstore#2001 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: cf89d13a-ea80-4605-a098-9bfdf536903d Signed-off-by: Nell Shamrell <nellshamrell@gmail.com>
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #2004 +/- ##
==========================================
+ Coverage 30.73% 31.19% +0.46%
==========================================
Files 122 123 +1
Lines 7409 7443 +34
==========================================
+ Hits 2277 2322 +45
+ Misses 4897 4873 -24
- Partials 235 248 +13 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Contributor
Author
|
Feel free to close if this is not useful |
Contributor
|
@jku fyi, thought you might find this interesting |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
TUF test fixtures in this repo were committed as base64 blobs / tarballs whose
embedded metadata expires 6 months after generation, because
scaffolding/pkg/repo.CreateRepoWithMetadatahardcodestime.Now().AddDate(0, 6, 0)with no way to configure it. These fixtures were already regenerated once
(shipped via #2002) and, left alone, would expire again on 2027-01-28 and
redden CI purely from the passage of time.
This replaces the committed fixtures with a shared test-only helper,
internal/tuftest, that signs a fresh TUF repository at test time with anexpiry relative to
now, so it is never stale. It's used inpkg/tuf,pkg/apis/policy/v1alpha1, andpkg/reconciler/trustroot(the three packagesaffected). Six now-dead fixture files are removed from
pkg/reconciler/trustroot/testdata, andhack/gentestdatano longer generatesTUF repos (it still generates the keys/certs/golden JSON that remain
committed, since the reconciler's golden output is derived from those, not
from the TUF metadata).
Also adds a guardrail test (
internal/tuftest/guard_test.go) that scans thetree for TUF metadata in any of the shapes these fixtures have historically
taken - JSON metadata files, tarred repositories, and base64 blobs inlined in
Go source - so an expiring fixture can't quietly reappear. It has its own
tests covering all three shapes.
How to test:
go test ./...,go vet ./...,golangci-lint runall pass. Iadditionally stress-tested the previously-flaky expiry test 15 times in a row
(clean) and mutation-tested the guardrail with the real historical fixture
blobs from git history to confirm it actually detects them.
Fixes #2001
Release Note
NONE
Documentation
No user-facing behavior changes; this only affects test fixtures.