diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index bd09c1dd..0bb923fc 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -55,6 +55,7 @@ cascade holds to a few conventions in its own codebase and in the workflows it g - **Path fields reach every path sink**: a manifest field that widens which files a component reacts to must thread through all three places a path is consumed, or it is a silent bug. The emitted `on: push` paths filter fires the workflow, per-callback change detection decides which builds and deploys run, and the version commit range decides the bump. A field that reaches only some of these triggers a run that then no-ops, or bumps a version whose builds skip as unchanged. When you add such a field, add a test that asserts the shared path reaches each sink. - **A breaking generator or validation change moves with the fleet, in the same change**: a change that makes a previously valid manifest invalid, such as rejecting a field `parse-config` used to accept, is breaking even when it ships as a `fix:`. The fleet repin re-stamps every example repository onto the release-candidate binary and regenerates its workflows before any suite runs, so a manifest still carrying the now-rejected shape fails that repin, not the intended test. Before landing a validation change that can reject something that used to pass, scan the fleet example repos for that shape and migrate any that use it in the same pull request, alongside the doc's migration note. This generalizes the existing rule that a fleet suite and the eligibility logic it exercises are one coupled unit (see [Making a change](#making-a-change)); it applies to validation, not only to eligibility. - **Every generated workflow kind carries executing coverage**: each workflow the generator emits (`orchestrate`, `promote`, `external-update`, and the `cascade-` lanes) is mapped to the e2e scenarios and fleet lanes that run it in `internal/coverage/registry.yaml`. The coverage gate derives the emitted kinds straight from the generator source and fails when an emitted kind has no registry entry, so a new generated workflow cannot ship without a scenario or lane that exercises it. When you add a generated workflow kind, add its entry pointing at the scenario or fleet lane that runs it; a referenced scenario or lane that does not exist also fails the gate. +- **Assert a runtime outcome, never the script that produces it**: an `e2e/` or fleet-suite assertion for a load-bearing behavior must compare against a runtime artifact that differs when the behavior regresses: a state leaf (`state.` sha/version/ref), a job conclusion (`success`/`skipped`/`failure`), a preflight output, a tag, a release, a branch, a pull request, or a line the running job actually logged (`expect_log`). A passing assertion must be reachable only by the behavior working at run time. Never assert a behavior by grepping emitted script source for a marker that also appears in that source: a `workflow_files.contains`/`not_contains` check over a generated `.yaml` proves a string was rendered, not that the logic ran, and stays green when the runtime behavior is deleted because the marker text is still literally present in the file. As a cautionary example, the state-write retry loop was once "covered" by grepping the emitted `orchestrate.yaml` for `cascade-state-write: exhausted attempts=10`. That branch never runs on the happy path, yet the string is always present in the script, so the check was unconditionally green and a regression that replaced the whole loop with a single `git push` would have shipped green; the fix asserts the marker the running job emits (`cascade-state-write: ok attempt=1`) and proves it red-able by breaking the emission. Restrict `workflow_files` checks to behaviors whose entire effect is the generated shape (a `concurrency:` block, a `timeout-minutes:` value, a real-GitHub-only step act cannot execute) and label those scenarios generation-only in the header. When a behavior is genuinely un-runnable in act, its executing proof lives on the fleet, and a generation-only e2e cell is a ceiling that must be labeled as such, never credited as runtime coverage. The bar: a generator or behavior change adds or updates an assertion a reviewer can turn red by reverting the behavior alone, leaving the emitted string in place. - **Callback isolation**: generated workflows call your workflows via `workflow_call`, and cascade never reaches into your callback logic. - **Metadata courier**: cascade passes artifact identifiers and versions between stages. It never touches your container registry, package registry, or the systems you deploy to directly. diff --git a/e2e/harness/multistep.go b/e2e/harness/multistep.go index 8475d63e..68353cee 100644 --- a/e2e/harness/multistep.go +++ b/e2e/harness/multistep.go @@ -224,6 +224,18 @@ type CommitStep struct { // repo-wide orchestrate.yaml, byte-identical to an orchestrate step with no config. type OrchestrateStep struct { Component string `yaml:"component,omitempty"` + // Event overrides the GitHub event the orchestrate workflow runs under + // (default "push"). release_trigger: dispatch drops the push: trigger, so a + // scenario runs the same workflow under "push" (paired with ExpectNoRun to + // prove no job fires) and under "workflow_dispatch" (proving the dispatch path + // still advances state). + Event string `yaml:"event,omitempty"` + // ExpectNoRun asserts the orchestrate workflow produced no job run at all: act + // scheduled zero jobs because the event does not match any of the workflow's + // triggers. It is the runtime signal that a trigger was correctly suppressed. + // A bare source grep for the absent "push:" string cannot distinguish a + // suppressed trigger from a malformed on: block that would still fire. + ExpectNoRun bool `yaml:"expect_no_run,omitempty"` } // PromoteStep defines a promote action @@ -384,6 +396,15 @@ type StepExpect struct { // assert a config field survives a routine state write rather than being // dropped on finalize. Manifest *ManifestExpect `yaml:"manifest,omitempty"` + // ExpectLog asserts the last workflow run's logs contain this substring, so a + // scenario can prove a load-bearing runtime marker the running job actually + // emitted (for example the state-write loop's "cascade-state-write: ok + // attempt=1") instead of grepping the emitted script source, which stays green + // even when the loop is deleted because the marker text is still literally + // present in the file. Mirrors RollbackStep.ExpectLog and is evaluated against + // the same run result the Jobs assertion reads, so it applies to any step that + // ran a workflow (orchestrate, promote). + ExpectLog string `yaml:"expect_log,omitempty"` } // ManifestExpect asserts substrings against the live manifest read from Gitea. diff --git a/e2e/harness/multistep_test.go b/e2e/harness/multistep_test.go index 3c19952f..d8e67e72 100644 --- a/e2e/harness/multistep_test.go +++ b/e2e/harness/multistep_test.go @@ -67,6 +67,53 @@ steps: assert.Equal(t, "skipped", scenario.Steps[1].Expect.Jobs["build-worker"]) } +// TestParseMultiStepScenario_RuntimeAssertionFields round-trips the runtime +// de-vacuum surfaces (expect_log on a step expectation; event and expect_no_run +// on an orchestrate step) so a scenario that asserts a runtime log marker or a +// suppressed trigger parses into the fields the runner consumes. +func TestParseMultiStepScenario_RuntimeAssertionFields(t *testing.T) { + yaml := ` +name: "Runtime assertion fields" +config: + environments: [dev] +steps: + - name: "Orchestrate on push; assert state-write marker" + action: orchestrate + expect: + state: + dev: + version: "v0.1.0-rc.0" + expect_log: "cascade-state-write: ok attempt=1" + - name: "Push does not trigger a dispatch-only orchestrate" + action: orchestrate + orchestrate: + event: push + expect_no_run: true + - name: "Dispatch triggers the orchestrate" + action: orchestrate + orchestrate: + event: workflow_dispatch + expect: + state: + dev: + version: "v0.1.0-rc.0" +` + + scenario, err := ParseMultiStepScenario([]byte(yaml)) + require.NoError(t, err) + require.Len(t, scenario.Steps, 3) + + assert.Equal(t, "cascade-state-write: ok attempt=1", scenario.Steps[0].Expect.ExpectLog) + + require.NotNil(t, scenario.Steps[1].Orchestrate) + assert.Equal(t, "push", scenario.Steps[1].Orchestrate.Event) + assert.True(t, scenario.Steps[1].Orchestrate.ExpectNoRun) + + require.NotNil(t, scenario.Steps[2].Orchestrate) + assert.Equal(t, "workflow_dispatch", scenario.Steps[2].Orchestrate.Event) + assert.False(t, scenario.Steps[2].Orchestrate.ExpectNoRun) +} + func TestDiscoverMultiStepScenarios(t *testing.T) { // Create temp directory with test scenarios dir := t.TempDir() diff --git a/e2e/harness/runner.go b/e2e/harness/runner.go index 486126d7..3f52864c 100644 --- a/e2e/harness/runner.go +++ b/e2e/harness/runner.go @@ -1020,10 +1020,19 @@ func (r *Runner) executeOrchestrate(ctx context.Context, config Config, expectFa branch = "main" } + // The event defaults to push (a trunk merge). A dispatch-only orchestrate + // (release_trigger: dispatch) is run under "push" with ExpectNoRun to prove + // no job fires, and under "workflow_dispatch" to prove the dispatch path + // still advances state. + event := "push" + if orch != nil && orch.Event != "" { + event = orch.Event + } + // Run the actual orchestrate workflow via ActRunner result, err := r.harness.act.RunWorkflowFromRepo(ctx, RunOpts{ WorkflowPath: workflowPath, - Event: "push", + Event: event, Env: map[string]string{ "GITHUB_SHA": sha, "GITHUB_REF": fmt.Sprintf("refs/heads/%s", branch), @@ -1037,6 +1046,18 @@ func (r *Runner) executeOrchestrate(ctx context.Context, config Config, expectFa // Store workflow result for assertions r.lastWorkflowResult = result + // Handle the suppressed-trigger case: the workflow scheduled no jobs because + // the event matches none of its triggers. act marks a zero-job targeted run + // as a "failure" (missing/unloadable workflow); ExpectNoRun reinterprets that + // specific outcome as the success path, proving the trigger was dropped. + if orch != nil && orch.ExpectNoRun { + if len(result.Jobs) == 0 { + r.t.Logf(" Orchestrate: no job ran under event %q, as expected", event) + return nil + } + return fmt.Errorf("expected no orchestrate run under event %q but %d job(s) ran", event, len(result.Jobs)) + } + // Handle expected failures (mirrors executePromote's ExpectFailure path). if expectFailure { if result.Conclusion == "failure" { @@ -1540,6 +1561,16 @@ func (r *Runner) assertStep(ctx context.Context, step *Step, preState *Execution allErrs = append(allErrs, errs...) } + // Assert a runtime log marker the last workflow run actually emitted. This + // reads r.lastWorkflowResult (the same result the Jobs assertion consumes), + // so it proves a behavior ran rather than that its marker text was rendered + // into the workflow file. Skipped in unit-test mode where no workflow ran. + if expect.ExpectLog != "" && r.lastWorkflowResult != nil { + if !strings.Contains(r.lastWorkflowResult.Logs, expect.ExpectLog) { + allErrs = append(allErrs, fmt.Errorf("expected workflow logs to contain %q but did not", expect.ExpectLog)) + } + } + return allErrs } diff --git a/e2e/scenarios/08-state-push-retry.yaml b/e2e/scenarios/08-state-push-retry.yaml index 7ffcc3e6..31a71d08 100644 --- a/e2e/scenarios/08-state-push-retry.yaml +++ b/e2e/scenarios/08-state-push-retry.yaml @@ -1,14 +1,20 @@ name: "State Push Retry Loop" description: | - Verifies the generated orchestrate.yaml's Update Manifest step uses a - fetch+reset+reapply+push retry loop (#101) instead of a single git push. + Verifies the generated orchestrate.yaml's Update Manifest step runs its + fetch+reset+reapply+push retry loop (#101) at runtime instead of a single git + push. - Generator-output verification. Reproducing the actual concurrent-push race - needs real GHA and is out of scope for the act+gitea harness. + Runtime assertion. An orchestrate run advances dev state and the executed + finalize job logs the loop's success marker "cascade-state-write: ok + attempt=1". That marker is emitted only by the running loop, so it disappears + if the loop is replaced by a bare push, whereas a grep of the emitted script + source would stay green because the marker text is still literally present in + the file. Reproducing the actual concurrent-push 409 race needs real GHA and + stays a fleet concern. config: trunk_branch: main - environments: [] + environments: [dev] builds: - name: app workflow: build.yaml @@ -16,7 +22,7 @@ config: deploys: [] steps: - - name: "Initial commit; assert retry loop in Update Manifest step" + - name: "Initial commit" action: commit commit: message: "feat: add app" @@ -24,15 +30,19 @@ steps: src/app.go: | package main func main() {} + + - name: "Orchestrate; assert the retry loop ran and wrote state" + action: orchestrate expect: - workflow_files: - - path: ".github/workflows/orchestrate.yaml" - contains: - - "Update Manifest" - - "for attempt in 1 2 3 4 5 6 7 8 9 10" - - "cascade-state-write: attempt=$attempt/10" - - "cascade-state-write: ok attempt=$attempt" - - "cascade-state-write: exhausted attempts=10" - - "git fetch origin" - - "git reset --hard" - - "apply_state_edits" + # State advanced: the Update Manifest loop reached its push and committed + # the new leaf. A regressed loop that never pushes leaves this unchanged. + state: + dev: + sha: commit1 + version: "v0.1.0-rc.0" + jobs: + build-app: success + # The running loop's success marker. attempt=1 is the shell-expanded value + # the executed job logs on the first successful push, not the unexpanded + # "$attempt" that appears in the emitted script source. + expect_log: "cascade-state-write: ok attempt=1" diff --git a/e2e/scenarios/37-release-breaking-gate.yaml b/e2e/scenarios/37-release-breaking-gate.yaml index 65c45473..1ddcc4df 100644 --- a/e2e/scenarios/37-release-breaking-gate.yaml +++ b/e2e/scenarios/37-release-breaking-gate.yaml @@ -1,53 +1,105 @@ name: "Release workflow breaking-change gate, dry-run, concurrency" description: | - Verifies that the single-environment release workflow emits its - breaking-change gate, the dry_run skip wiring, and the serialized - non-cancelling concurrency block (#322). + Verifies at runtime that the single-environment Release workflow's + breaking-change gate blocks publishing a breaking (major) release and lets it + through once allow_breaking_changes is set (#322). - A single-environment manifest generates a Release workflow into - promote.yaml (see 09-single-env-repo.yaml). This asserts the generated - shape of that workflow. + A single-environment manifest generates a Release workflow into promote.yaml + (see 09-single-env-repo.yaml). From a published v0.1.0 baseline, a feat! + commit is orchestrated to a v1.0.0-rc.0 draft. Publishing it without the flag + fails and leaves the published pointer at v0.1.0; the same publish with + allow_breaking succeeds and cuts v1.0.0. The blocked/allowed pair is observed + through the run conclusion and the published release, both of which flip when + the gate is inverted, rather than by grepping the emitted can_proceed script. - Covers: - - allow_breaking_changes workflow_dispatch input - - the Check Breaking Changes preflight step and its can_proceed outputs - - the dry_run workflow_dispatch input and the release-job skip guard - - the top-level concurrency block with cancel-in-progress: false - - Generator-output verification only. + The dry_run skip wiring and the serialized non-cancelling concurrency block + are pure generated shape, asserted generation-only. config: trunk_branch: main environments: [prod] - deploys: + builds: - name: app - workflow: .github/workflows/deploy.yaml + workflow: build.yaml triggers: ["src/**"] + deploys: [] + +# Published v0.1.0 baseline so a subsequent major bump is gated. +setup: + state: + prod: + version: "v0.1.0" + tags: + - "v0.1.0" + releases: + - tag: "v0.1.0" + prerelease: false steps: - - name: "Initial commit generates release-flavored promote.yaml; assert the breaking gate" + - name: "Breaking change commit" action: commit commit: - message: "feat: add app source" + message: "feat!: major app refactor" files: src/app.go: | package main - func main() {} + func mainV2() {} + + - name: "Orchestrate; assert draft RC and the emitted gate/concurrency shape" + action: orchestrate expect: + state: + prod: + sha: commit1 + version: "v1.0.0-rc.0" + jobs: + build-app: success + releases: + - tag: "v1.0.0-rc.0" + prerelease: true + draft: true + # Generation-only: the dry_run skip guard and the serialized concurrency + # block are pure generated shape act does not execute here. The gate's + # RUNTIME behavior is proven by the blocked/allowed publish pair below. workflow_files: - path: ".github/workflows/promote.yaml" contains: - # Breaking-change gate input and detection step. - " allow_breaking_changes:\n" - " - name: Check Breaking Changes\n" - - " ALLOW_BREAKING: ${{ github.event.inputs.allow_breaking_changes }}\n" - - " echo \"has_breaking=$HAS_BREAKING\" >> \"$GITHUB_OUTPUT\"\n" - # The gate proceeds only when allowed, and blocks otherwise. - - " echo \"can_proceed=true\" >> \"$GITHUB_OUTPUT\"\n" - - " echo \"can_proceed=false\" >> \"$GITHUB_OUTPUT\"\n" - # dry_run input and the release-job skip guard. - " dry_run:\n" - " if: ${{ github.event.inputs.dry_run != 'true' }}\n" - # Serialized, non-cancelling concurrency block. - "concurrency:\n" - " cancel-in-progress: false\n" + + - name: "Publish without the flag is blocked" + action: promote + promote: + mode: default + expect_failure: true + expect: + # The published pointer never advanced to v1.0.0; the draft RC is intact. + state: + release: + unchanged: true + releases: + - tag: "v1.0.0" + deleted: true + + - name: "Publish with allow_breaking proceeds" + action: promote + promote: + mode: default + allow_breaking: true + expect: + state: + release: + sha: commit1 + version: "v1.0.0" + releases: + - tag: "v1.0.0" + prerelease: false + draft: false + latest: true + tags: + exist: ["v1.0.0"] + deleted: ["v1.0.0-rc.0"] diff --git a/e2e/scenarios/38-promote-breaking-gate-release-build.yaml b/e2e/scenarios/38-promote-breaking-gate-release-build.yaml index debfa854..c4565be1 100644 --- a/e2e/scenarios/38-promote-breaking-gate-release-build.yaml +++ b/e2e/scenarios/38-promote-breaking-gate-release-build.yaml @@ -1,51 +1,178 @@ name: "Promote breaking-change gate and release-build dispatch" description: | - Verifies that the multi-environment promote workflow emits its - CLI-driven breaking-change gate and the follow-on release-build dispatch - when release.workflow is configured (#322). + Verifies that the multi-environment promote workflow's CLI-driven + breaking-change gate blocks a breaking promotion into the release stage (#322). - Covers: - - allow_breaking_changes workflow_dispatch input - - the cascade promote preflight run with the --allow-breaking flag - - the dry_run input and the promote-job skip guard - - the Trigger Release Build step dispatching the configured release-build - workflow after a final-env publication + Load-bearing RUNTIME assertion (the block): from a published v0.1.0 baseline, a + feat! commit is orchestrated to a v1.0.0-rc.0 prerelease and cascaded through + the prerelease environments (no gate). Promoting uat into the release stage + WITHOUT the flag fails at the breaking-change gate: the CLI preflight rejects + the promotion before any publish, so the source and release state leaves stay + unchanged and no v1.0.0 tag or release is cut. This runs under act (the gate + fails in preflight, ahead of the gh-dependent publish path) and is red-able: + forcing preflight CanProceed=true makes the promote proceed instead of failing. - Generator-output verification only. + Generation-only (act cannot run the publish path): the allow_breaking side of + the gate and the release-build dispatch are asserted as EMITTED shape, not run. + The multi-env allow_breaking promote reaches a final-env publish entangled with + the gh-driven Trigger Release Build step; Publish Release cutting v1.0.0, + reaping the RC, advancing the release/prod state, and executing the dispatched + release-build workflow all need real GitHub, which act has no gh for. So this + scenario asserts only that the promote workflow EMITS the allow_breaking wiring + and the release-build dispatch. + + Residuals proven elsewhere: the runtime gate-BYPASS (a breaking promote + proceeding once allow_breaking is set) is proven by scenario 37 (single-env, + two-sided and red-able); the multi-env allow_breaking publish and the + release-build execution are proven on the fleet, where gh is present. config: trunk_branch: main - environments: [dev, test, prod] - deploys: + environments: [dev, qa, uat, prod] + builds: - name: app - workflow: .github/workflows/deploy.yaml + workflow: build.yaml triggers: ["src/**"] + deploys: + - name: cdk + workflow: deploy.yaml + triggers: ["cdk/**"] release: workflow: .github/workflows/release-build.yaml +# Published v0.1.0 baseline: prerelease envs hold the last RC, release and prod +# hold the published version. The RC tag was reaped at publish time. +setup: + state: + dev: + version: "v0.1.0-rc.0" + qa: + version: "v0.1.0-rc.0" + uat: + version: "v0.1.0-rc.0" + release: + version: "v0.1.0" + prod: + version: "v0.1.0" + tags: + - "v0.1.0" + releases: + - tag: "v0.1.0" + prerelease: false + steps: - - name: "Initial commit generates promote.yaml; assert gate and release-build dispatch" + - name: "Breaking change commit" action: commit commit: - message: "feat: add app source" + message: "feat!: major app refactor" files: - src/app.go: | - package main - func main() {} + src/app.ts: | + export function mainV2() { + console.log("App v1.0.0 - Breaking!"); + } + + - name: "Orchestrate dev; assert the emitted CLI gate and release-build shape" + action: orchestrate expect: + state: + dev: + sha: commit1 + version: "v1.0.0-rc.0" + jobs: + build-app: success + releases: + - tag: "v1.0.0-rc.0" + prerelease: true + draft: true + # Generation-only: these assert the emitted shape of behaviors act cannot + # execute (the CLI gate wiring and the real-GitHub-only release-build + # dispatch). The gate's RUNTIME behavior is proven by the blocked/allowed + # promote pair below; the release-build dispatch's execution is proven on + # the fleet, since act has no gh to run it. workflow_files: - path: ".github/workflows/promote.yaml" contains: - # Breaking-change gate input and CLI-driven preflight. - " allow_breaking_changes:\n" - " cascade promote preflight \\\n" - " --allow-breaking=\"${ALLOW_BREAKING:-false}\" \\\n" - # dry_run input and the promote-job skip guard. - - " dry_run:\n" - - " if: ${{ github.event.inputs.dry_run != 'true' }}\n" - # Follow-on release-build dispatch against the published tag. - " - name: Trigger Release Build\n" - " gh workflow run release-build.yaml \\\n" not_contains: - # The promote path uses the CLI gate, not the release bash step. + # The multi-env promote path uses the CLI gate, not the single-env + # release workflow's bash "Check Breaking Changes" step. - " - name: Check Breaking Changes\n" + + - name: "Cascade promote dev to qa (prerelease, no gate)" + action: promote + promote: + mode: cascade + target: qa + expect: + state: + qa: + sha: commit1 + version: "v1.0.0-rc.0" + dev: + unchanged: true + + - name: "Cascade promote qa to uat (prerelease, no gate)" + action: promote + promote: + mode: cascade + target: uat + expect: + state: + uat: + sha: commit1 + version: "v1.0.0-rc.0" + qa: + unchanged: true + + - name: "Promote uat to release without the flag is blocked at the gate" + action: promote + promote: + mode: default + expect_failure: true + expect: + # The gate blocks before Publish Release, so no v1.0.0 is cut and both the + # source state and the published release stay put. This is the load-bearing + # runtime blocked outcome, and it is red-able: forcing preflight + # CanProceed=true lets the promote proceed instead of failing. The + # gate-bypass counterpart (allow_breaking proceeding) is proven by + # scenario 37; the allow_breaking step below asserts only emitted shape. + state: + uat: + unchanged: true + release: + unchanged: true + releases: + - tag: "v1.0.0" + deleted: true + + - name: "Promote uat to release with allow_breaking: emitted allow-breaking wiring and release-build dispatch (generation-only)" + action: promote + promote: + mode: default + allow_breaking: true + # Under act this run clears the CLI gate but cannot complete: Publish + # Release and the gh-driven Trigger Release Build dispatch need real + # GitHub, which act has no gh for, so the run fails here for an + # infrastructure reason, NOT at the gate. The runtime gate-BYPASS (a + # breaking promote proceeding once allow_breaking is set) is proven by + # scenario 37; this step's surviving, act-runnable assertion is + # generation-only. + expect_failure: true + expect: + # Generation-only: the multi-env allow_breaking publish + release-build + # execution cannot run under act, so instead of asserting a published + # v1.0.0 (release/tag/rc-reap/state-advance), assert only that the promote + # workflow EMITS the allow_breaking wiring and the release-build dispatch. + # The publish + dispatch execution are proven on the fleet, where gh is + # present; the runtime gate-bypass is proven by scenario 37. + workflow_files: + - path: ".github/workflows/promote.yaml" + contains: + - " allow_breaking_changes:\n" + - " --allow-breaking=\"${ALLOW_BREAKING:-false}\" \\\n" + - " - name: Trigger Release Build\n" + - " gh workflow run release-build.yaml \\\n" diff --git a/e2e/scenarios/40-release-trigger-dispatch-only.yaml b/e2e/scenarios/40-release-trigger-dispatch-only.yaml index 4c7824f3..2e205e62 100644 --- a/e2e/scenarios/40-release-trigger-dispatch-only.yaml +++ b/e2e/scenarios/40-release-trigger-dispatch-only.yaml @@ -1,17 +1,19 @@ name: "Release trigger dispatch-only" description: | - Verifies that release_trigger: dispatch makes the generated orchestrate - workflow omit its push: trigger so it runs only on workflow_dispatch. A - maintainer-owned gate (for example a nightly schedule) then decides when a - release candidate is cut, instead of every trunk merge cutting one. + Verifies at runtime that release_trigger: dispatch makes the generated + orchestrate workflow run only on workflow_dispatch. A maintainer-owned gate + (for example a nightly schedule) then decides when a release candidate is cut, + instead of every trunk merge cutting one. - Covers: - - the push: trigger and its branches:/paths: filter are gone - - workflow_dispatch and its dry_run input survive - - the default (push) behavior is unaffected, asserted by 13-dispatch-inputs - and every other generator scenario that does not set release_trigger - - Generator-output verification only. + Runtime assertion. The same orchestrate workflow is run twice: under a push + event it schedules no job (the push: trigger is gone), and under a + workflow_dispatch event it runs and advances dev state to the first release + candidate. Running the dispatch path proves the workflow_dispatch trigger is + genuinely wired, which a grep for the "workflow_dispatch:" string cannot + establish, and the no-run push assertion proves the trigger was suppressed + rather than merely spelled differently in the on: block. The default (push) + behavior for repositories that do not set release_trigger is covered by every + other generator scenario. config: trunk_branch: main @@ -24,7 +26,7 @@ config: deploys: [] steps: - - name: "Initial commit; assert orchestrate omits push trigger" + - name: "Initial commit" action: commit commit: message: "feat: add app" @@ -32,13 +34,21 @@ steps: src/app.go: | package main func main() {} + + - name: "A push event triggers no orchestrate run" + action: orchestrate + orchestrate: + event: push + expect_no_run: true + + - name: "A workflow_dispatch event runs orchestrate and advances state" + action: orchestrate + orchestrate: + event: workflow_dispatch expect: - workflow_files: - - path: ".github/workflows/orchestrate.yaml" - contains: - - " workflow_dispatch:" - - " dry_run:" - not_contains: - - " push:" - - " branches: [main]" - - " paths:" + state: + dev: + sha: commit1 + version: "v0.1.0-rc.0" + jobs: + build-app: success diff --git a/e2e/scenarios/43-deploy-rollout-strategy.yaml b/e2e/scenarios/43-deploy-rollout-strategy.yaml index b153e551..a49e0468 100644 --- a/e2e/scenarios/43-deploy-rollout-strategy.yaml +++ b/e2e/scenarios/43-deploy-rollout-strategy.yaml @@ -7,7 +7,13 @@ description: | rollout, so a deploy that sets fail_fast: true and max_parallel: 2 proves the configured values flow through rather than the historical defaults. - Generator-output assertion only; no workflow run. + Generation-only. The whole effect of these knobs is the generated strategy + block, and the runtime behavior they configure (fail-fast cancelling a sibling + matrix leg, max-parallel throttling) is not observable in act: act does not + reliably implement matrix fail-fast cancellation, and max-parallel is a timing + property. The executing proof of a rollout that cancels on failure lives on the + fleet, not in an e2e grep. This cell is a ceiling, labeled as such, not runtime + coverage. config: trunk_branch: main