From 93698bdf357e20b075ebe4a6f8e989d8f2dd8cf4 Mon Sep 17 00:00:00 2001 From: Joshua Temple Date: Thu, 9 Jul 2026 12:06:55 -0400 Subject: [PATCH] docs: document the concurrency model, convergence guarantees, and non-goals Add a concurrency-and-convergence explanation to the internals architecture page: per-component concurrency-group isolation, the optimistic read-modify-write with re-apply that all finalize paths use, the bounded jittered retry and its convergence markers, why merge_group on the side-effecting orchestrate lane is rejected in favor of merge_queue.enabled, and how extra_paths and shared_paths thread a shared change to the components that consume it. State the deliberate non-goals: no intra-repo component ordering, no per-component state files, no same-component cross-lane serialization. Codify in CONTRIBUTING that a breaking generator or validation change must scan and migrate the fleet example repos in the same change, since the fleet repin regenerates every example repo with the release-candidate binary before any suite runs. Signed-off-by: Joshua Temple --- CONTRIBUTING.md | 1 + .../content/docs/internals/architecture.md | 71 +++++++++++++++++++ 2 files changed, 72 insertions(+) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 1ba9f109..e607a01e 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -53,6 +53,7 @@ cascade holds to a few conventions in its own codebase and in the workflows it g - **Additive manifest changes**: new fields are always optional with sensible defaults, so existing manifest files keep working across minor version bumps. - **Path fields reach every path sink**: a manifest field that widens which files a component reacts to must thread through all three places a path is consumed, or it is a silent bug. The emitted `on: push` paths filter fires the workflow, per-callback change detection decides which builds and deploys run, and the version commit range decides the bump. A field that reaches only some of these triggers a run that then no-ops, or bumps a version whose builds skip as unchanged. When you add such a field, add a test that asserts the shared path reaches each sink. +- **A breaking generator or validation change moves with the fleet, in the same change**: a change that makes a previously valid manifest invalid, such as rejecting a field `parse-config` used to accept, is breaking even when it ships as a `fix:`. The fleet repin re-stamps every example repository onto the release-candidate binary and regenerates its workflows before any suite runs, so a manifest still carrying the now-rejected shape fails that repin, not the intended test. Before landing a validation change that can reject something that used to pass, scan the fleet example repos for that shape and migrate any that use it in the same pull request, alongside the doc's migration note. This generalizes the existing rule that a fleet suite and the eligibility logic it exercises are one coupled unit (see [Making a change](#making-a-change)); it applies to validation, not only to eligibility. - **Callback isolation**: generated workflows call your workflows via `workflow_call`, and cascade never reaches into your callback logic. - **Metadata courier**: cascade passes artifact identifiers and versions between stages. It never touches your container registry, package registry, or the systems you deploy to directly. diff --git a/docs/src/content/docs/internals/architecture.md b/docs/src/content/docs/internals/architecture.md index 605296ca..56d92a9c 100644 --- a/docs/src/content/docs/internals/architecture.md +++ b/docs/src/content/docs/internals/architecture.md @@ -97,6 +97,77 @@ For pipelines that span more than one repository, one repository is the primary: See [Coordinate multiple repos](/cascade/guides/multi-repo/) for how to configure `external` and `notify` and for the operational detail this page does not repeat. +## Concurrency and convergence + +A busy trunk can fire several lanes at once: every component of a monorepo +orchestrating off the same commit, a promotion running while a hotfix lands, +several environments finalizing in parallel. All of that activity ultimately +writes into one place, the state block inside the manifest file, so cascade's +concurrency model is really a statement about how that one file converges +under concurrent writers. + +**Isolation by concurrency group.** Every lane derives its own GitHub Actions +`concurrency.group`, scoped per component where components are declared: +orchestrate, promote, and rollback each get a distinct group per component, +and hotfix groups per component and target environment. Two components never +share a queue and never cancel each other's runs; only repeat runs of the +*same* lane for the *same* component serialize or cancel, per that lane's +`cancel_in_progress` setting. See [`concurrency`](/cascade/reference/manifest/#concurrency) +and [Split a repo into components](/cascade/guides/components/#how-each-component-promotes-independently). + +**Convergence by optimistic re-apply, not by file separation.** State lives in +one manifest-embedded document, node-patched leaf by leaf +(`state.components..`, `latest_release.components.`), so a +concurrent writer's commit never overwrites a sibling leaf it does not own. +Every write path, the generated workflow's shell step and the Go `orchestrate`, +`promote`, `hotfix`, and `rollback` finalize paths alike, commits as a +read-modify-write: on a rejected push it re-reads the current trunk bytes, +re-applies its own leaf mutation on top of whatever the other writer already +committed, and retries. The retry ceiling is 10 attempts with exponential, +jittered backoff, sized to survive a realistic wave (every component of a +monorepo racing to write into the same file) rather than only a handful of +parallel environments. Each attempt and its outcome are logged with a stable, +greppable marker, `cascade-state-write: attempt=N/10`, `cascade-state-write: ok +attempt=N`, or `cascade-state-write: exhausted attempts=10`, so a live run's +convergence (or exhaustion) is provable from its logs. + +**Merge queues without a side-effecting speculative build.** A merge queue's +speculative build runs against a commit that may never land, so a lane that +cuts tags, publishes releases, or writes state must never run there. +`merge_queue.enabled` emits a read-only lane for the queued candidate; +attaching the raw `merge_group` event to the side-effecting orchestrate +workflow through `extra_triggers` is rejected at validate for exactly this +reason. See [`merge_queue`](/cascade/reference/manifest/#merge_queue). + +**Shared-path changes reach every consumer they should.** A component's +effective path set, its own `path` plus any `extra_paths` and top-level +`shared_paths`, reaches the emitted push filter, change detection, and the +version commit range alike, so a breaking change to code two components share +bumps exactly the components that declare a dependency on it. See [Share code +across components](/cascade/guides/components/#share-code-across-components). + +### What this deliberately does not do + +- **No intra-repo component ordering.** Components are independent by design, + and validate rejects overriding a per-component concurrency group precisely + to keep components from serializing against each other. See [What + components deliberately do not + do](/cascade/guides/components/#what-components-deliberately-do-not-do) for + the detail and the `external`/`notify` alternative for a component that + genuinely needs to react to another's deploy. +- **No per-component state files.** State stays one manifest-embedded + document with per-component leaves, not a file per component. The Contents + API commits to one branch ref regardless of how many files a commit + touches, so separate files would still collide on that ref; leaf-wise + optimistic re-apply gives sibling preservation without the extra surface. +- **No same-component cross-lane serialization.** A promote and a hotfix + targeting the same component and environment at the same instant are not + ordered against each other. Each converges its own leaf correctly (the + state write is what makes that safe), but cascade does not guarantee which + one lands first or block one while the other runs. Treat concurrent + operations on the same component and environment as a coordination problem + for the people running them, not one cascade arbitrates. + ## Security model Generated workflows run in the adopting repository's own context: secrets are passed with `secrets: inherit` or scoped per callback, environment protection comes from GitHub environments, and cross-repo dispatch requires an explicit token. Cascade's own repository stores no adopter secrets.