From fc61c8ab1c08c015d9f3cb17747261fb1cd8e3de Mon Sep 17 00:00:00 2001 From: David Elner Date: Wed, 17 Jun 2026 15:01:50 +0000 Subject: [PATCH] WIP: Plan --- PLAN.md | 458 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 458 insertions(+) create mode 100644 PLAN.md diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 0000000..de6ad27 --- /dev/null +++ b/PLAN.md @@ -0,0 +1,458 @@ +# Plan: Generate release workflows from templates + +## Context + +`.github/workflows/release-ruby.yml` is doing double duty: our **CI-test harness** (releases the +throwaway `bt-fake` gem, auto-dry-runs on PRs) **and** the de-facto **canonical workflow** other +Ruby SDK repos are told to copy. Those have diverged — a consumer copying it inherits CI-only and +bt-fake-only cruft (an extra `bump` job, `./actions` + per-job `checkout`, version patching, PR/Slack/ +notes plumbing). + +This extends the existing template system (`templates/` → generated output) to **workflows**: a +clean, parameterized **consumer** release-workflow template plus a CLI an agent calls to render it +into a repo (and re-render to diff for updates). It disambiguates the canonical template from our CI +harness and gives a low-magic update story (regenerate → diff → agentically apply), anchored by a +provenance header. + +**Scope:** Stage 1 ships the consumer template + the public CLI + validation + a committed reference +rendering. Driving our own `release-ruby.yml` from the same template (so it can't drift) is a planned +**Stage 2** — designed for now, decided from the real diff, optionally landed in this PR. + +## Interface model (settled) + +- **Public** = one binstub with subcommands: **`bin/workflow generate`** (scaffold / re-render), + **`bin/workflow validate`** (check a workflow before committing), **`bin/workflow compare`** (diff a + workflow against a freshly-rendered baseline), and **`bin/workflow update`** (3-way merge upstream + changes in — see *Update story*). `generate`/`validate`/`compare` land in Stage 1; `update` is a + designed-for fast-follow. Stdlib OptionParser subcommand dispatch — no Thor. The public tool is + workflow-scoped and mirrors the private `rake workflows:` namespace. +- **Action generation is internal**, not public — you only regenerate `actions/` when modifying + sdk-actions itself. It stays `scripts/generate.rb`, invoked by the private Rake tasks + CI. +- **Rake = private** (contributor/CI tasks), subject-first: `actions:generate`, `actions:validate`, + `actions:ci`, `workflows:validate`, `workflows:ci`, `ci`. The `workflows:*` validate tasks + **delegate to `bin/workflow validate`**, so the schema check lives in one place (the public tool). + +## Worked example (user story) — exercising the vision + +**Goal:** an agent sets up releases for gem `braintrust` in repo `braintrustdata/braintrust-ruby`. + +The binstub lives in **sdk-actions**, and it writes into the *consumer* repo — so the agent runs it +from an sdk-actions checkout. By default `--ref` resolves to the SHA of sdk-actions `origin/main` (the +latest released line), so it's usually omitted — you pin the released ref you generated against. + +**Scaffold (initial setup):** +```sh +# in an sdk-actions checkout +bin/workflow generate release/ruby \ + --gem-name braintrust \ + --version-module Braintrust \ + --dest ../braintrust-ruby/.github/workflows/release.yml +# smart defaults fill the rest: +# --ref (→ SHA of sdk-actions origin/main, the latest released line) +# --version-file (→ derived from --version-module: Braintrust → lib/braintrust/version.rb) +# --working-directory (.), --ruby-version (4.0), --emoji (:gem:), env/slack names +``` + +**Output** — `../braintrust-ruby/.github/workflows/release.yml`: +```yaml +# GENERATED by sdk-actions `bin/workflow` — update via `bin/workflow update`, not by hand. +# >>> sdk-actions >>> # sentinel-fenced; compare/update locate this, strip `# `, YAML.load it +# template: release/ruby +# ref: 9f3c1ab… +# params: # resolved params (non-secret: names only) +# gem_name: braintrust +# version_module: Braintrust +# version_file: lib/braintrust/version.rb +# working_directory: "." +# ruby_version: "4.0" +# emoji: ":gem:" +# publish_environment: publish +# dry_run_environment: publish-dry-run +# <<< sdk-actions <<< +name: Release Ruby +on: + workflow_dispatch: + inputs: + sha: { required: true } + prev_release: { required: false } + dry_run: { type: boolean, default: false } +jobs: + validate: + steps: + - uses: braintrustdata/sdk-actions/actions/release/lang/ruby/validate@9f3c1ab… + with: + sha: ${{ inputs.sha }} + version_file: lib/braintrust/version.rb + version_module: Braintrust + working_directory: . + prepare: # → braintrustdata/sdk-actions/actions/release/prepare@9f3c1ab… + notify-pending: # → …/release/notify-pending@9f3c1ab… (slack_token: ${{ secrets.SLACK_BOT_TOKEN }}) + publish: + environment: ${{ inputs.dry_run && 'publish-dry-run' || 'publish' }} + permissions: { contents: write, id-token: write } + steps: + - uses: braintrustdata/sdk-actions/actions/release/lang/ruby/publish@9f3c1ab… # checks out inputs.sha internally + with: { sha: ${{ inputs.sha }}, gem_name: braintrust, github_release: 'true', … } +``` +No `bump` job, no per-job `actions/checkout` (validate/publish check out internally), version from the file. + +**The binstub then prints a next-steps checklist** (generation ≠ full setup): +``` +✓ Wrote ../braintrust-ruby/.github/workflows/release.yml (release/ruby @ 9f3c1ab…) +Next, in braintrustdata/braintrust-ruby: + 1. Create environments: `publish` (required reviewers) and `publish-dry-run`. + 2. RubyGems trusted publisher for `braintrust`: repo=braintrustdata/braintrust-ruby, + workflow=release.yml, environment=publish. ← workflow filename must match your --dest + 3. (optional) Secrets/vars: SLACK_BOT_TOKEN (secret), SLACK_SDK_RELEASE_CHANNEL (var). + 4. Confirm lib/braintrust/version.rb defines Braintrust::VERSION. +``` + +**Validate before committing** (consumer-facing; works on a generated *or* hand-edited workflow): +```sh +bin/workflow validate ../braintrust-ruby/.github/workflows/release.yml +# → YAML parse + check-jsonschema against the vendored github-workflow schema; reports file:line errors or "ok" +``` + +**Compare / update (later, on a newer sdk-actions release):** first-class verbs instead of manual +`diff`-piping — the file's provenance header supplies the ref it was generated from: +```sh +# both read template + ref + params from the file's header — nothing to re-supply +# show what UPSTREAM changed since this file was generated (isolates upstream delta from your edits) +bin/workflow compare ../braintrust-ruby/.github/workflows/release.yml + +# apply that upstream delta via a 3-way merge (git merge-file), leaving conflict markers where you'd +# customized the same lines for the agent to resolve; rewrites the header ref to the new one +bin/workflow update ../braintrust-ruby/.github/workflows/release.yml +``` +Both read the header for template + ref + params, then re-render two baselines — the **old** ref +(`git show :templates/…`) and the **current** ref — so the diff/merge reflects only upstream +changes. `generate | diff` stays as the zero-dependency fallback. + +### Gaps the example surfaces (folded into the design) +- **Distribution:** the tool lives in sdk-actions and writes into the consumer repo → the agent needs + an sdk-actions checkout; `--ref` defaults to `origin/main`'s SHA (the released line you pin against). +- **Output target:** default to **stdout** (pipe/diff-friendly); `--dest FILE` writes to a file + (`mkdir -p`, and refuse to overwrite an existing file without `--force` so the update-diff flow + never clobbers a customized workflow). One binstub cleanly serves both scaffold (`--dest`) and + update (stdout | diff). +- **`version_file` derives from `version_module`** (`::`→`/`, CamelCase→snake_case) — mirrors Ruby's + layout (`Braintrust::OpenAI` → `lib/braintrust/open_ai/version.rb`), more reliable than from the gem + name; overridable for non-standard layouts. +- **`version_module` is required** (not derivable from the gem name). +- **Repo settings are NOT generated** (environments, RubyGems trusted publisher, secrets/vars) → the + binstub prints the checklist above; the workflow **filename** matters because the trusted publisher + pins it. +- **`publish` env** is the real-release gate — the consumer must add required reviewers. + +## Reuse (don't rebuild) +- **`Template` engine** (`render`/`render_step`/`with_if`/`indent_lines`) in `scripts/generate.rb` — + already supports `**locals` / `locals.fetch(:x){ default }`. Extract so the binstub shares it. + Workflows use it as a flat parameterized ERB (no `render_step` composition — keep them diffable). +- **check-jsonschema + the "not found → mise install" guard** from today's `validate:schema`. +- **`GENERATED_HEADER`** convention. + +## Approach (staged) + +### Stage 0 — Extract the render engine (no behavior change) +- New `scripts/lib/template.rb`: move `Template` + the `STEPS_DIR` it needs into a neutral module + (e.g. `Templating`). `scripts/generate.rb` `require_relative 'lib/template'`; keep action + orchestration/constants. **Gate:** `rake actions:generate` → `git diff --exit-code actions/` clean. + +### Stage 1 — Consumer template + public binstub + validation + rake reorg + docs +1. **`templates/workflows/release/ruby.yml.erb`** — category-A consumer workflow. **Opens with a + `<%# params: … -%>` frontmatter block** (YAML param schema: `required`/`default`/`desc` per param; + `desc` doubles as docs) that the binstub reads to build its CLI — so language/registry specifics + live in the template, not the binstub. ERB renders the block to nothing. Body parameterized via + `locals.fetch`; **derived defaults computed in-body** (e.g. `version_file` from `version_module`). + Lean on smart defaults so the typical call is just `--gem-name` + `--version-module` (+ `--dest`). + Declared params: `gem_name` (req), `version_module` (req), `version_file` + (default derived from `version_module`: `::`→`/`, CamelCase→snake_case, e.g. `Braintrust::OpenAI` → + `lib/braintrust/open_ai/version.rb`), `working_directory` (`.`), `ruby_version` (`4.0`), + `emoji` (`:gem:`), `ref` (default = SHA of sdk-actions `origin/main`, the latest released line), + `publish_environment` (`publish`), + `dry_run_environment` (`publish-dry-run`), `slack_token_secret` (`SLACK_BOT_TOKEN`), + `slack_channel_var` (`SLACK_SDK_RELEASE_CHANNEL`), `workflow_name` (`Release Ruby`). Emits external + `…/actions/release/…@` refs, no per-job checkout, no `bump` job, `workflow_dispatch` only, + plain `environment` ternary, plain secrets/vars. Header = `GENERATED_HEADER` + a **sentinel-fenced** + YAML metadata block (`# >>> sdk-actions >>>` … `# <<<`) capturing `template`, `ref`, and the + **resolved** non-secret `params`; read back by locate-via-sentinels + strip `# ` + `YAML.safe_load` + (so compare/update regenerate the exact baseline). Cohesive sections so a `profile: ci` (Stage 2) + layers cleanly. +2. **`bin/workflow`** — public binstub, stdlib `OptionParser` with manual subcommand dispatch + (`ARGV.shift`), executable. **Template-agnostic:** it reads the target template's frontmatter param + schema (grab the leading `<%# … -%>` block → `YAML.safe_load`) and **builds its `--` flags + dynamically** from it (so `bin/workflow generate release/python --help` lists *that* template's + params). Universal flags (`--dest`, `--ref`, `--force`) are binstub-level. Subcommands: + - `generate