feat(coverage): gate generated workflow kinds on executing coverage - #546
Merged
Merged
Conversation
Add internal/coverage: a registry mapping every workflow kind the generator emits to the e2e scenarios, harness tests, and fleet lanes that run it, plus a test that derives the emitted-kind set from the generator source and fails when an emitted kind has no registry entry or a referenced scenario or lane does not exist. Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
…denominator Signed-off-by: Joshua Temple <joshua.temple@stablekernel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Nothing enforced that a newly generated workflow kind ships with executing coverage. A new emitted workflow could land with only generation-only assertions, or none, and CI would stay green. This operationalizes the "suites move with code" standard for the highest-risk axis: the set of workflow kinds the generator emits.
Fix
Adds a coverage registry and a gate test:
internal/coverage/registry.yaml(embedded): a positive map of each generated workflow kind to the executing coverage that exercises it (e2e scenarios and/or fleet lanes).internal/coverage/registry.go:workflowKindFromLiteralderives the kind set by AST-parsing the generator's own.github/workflows/<name>.yamloutput-path literals ininternal/generate(the same literalsverifytrusts), including templated forms (cascade-foo-%s.yamlresolves to stemcascade-foo), and rejects any phantom kind.internal/coverage/coverage_test.go: derives the kind set from source (never from the registry), fails if any kind lacks an executing-coverage entry, if a registry entry references a scenario/lane that does not exist, or if the registry lists a kind the generator no longer emits.A new generated workflow kind now fails CI until its registry entry points at an executing scenario or fleet lane.
Verification
go build ./...,go test ./...(2693 pass),golangci-lint run ./...all clean;go test ./internal/coverage/green including a test proving a templated-only kind is captured with no phantom kind. Red-proof: injecting a fake generated kind turns the gate red with a clear message. CONTRIBUTING documents the convention.