diff --git a/README.md b/README.md index 932069e..fa247b9 100644 --- a/README.md +++ b/README.md @@ -110,6 +110,7 @@ with no production counterpart in a self-hosted stack. ## Documentation +- [Start here](docs/README.md) — what the toolbox is, map, commands, glossary - [Tour](docs/tour/) — how the pieces fit, nine pages - [Decisions](docs/decisions/) — why they fit that way - [Runbooks](docs/runbook/) — what to do when something specific happens diff --git a/docs/README.md b/docs/README.md new file mode 100644 index 0000000..c24ce1b --- /dev/null +++ b/docs/README.md @@ -0,0 +1,107 @@ +# scaffold documentation + +## What scaffold is + +- A bash toolbox: one `scaffold` script plus the libraries in `lib/`. +- It generates a client monorepo from `common/`, one adapter per application, and optional services. +- Every generated application implements the nine-task contract in `lib/contract.sh`. +- A generated project's CI calls this account's reusable workflows by the `@v1` tag and runs the contract per config root, so CI never learns the language. + +![System context](diagrams/system-context.svg) + +## Repository map + +| Path | Role | +| --- | --- | +| `scaffold` | Entry point: `main()` dispatches to one `cmd_*` function per command | +| `lib/` | Libraries `scaffold` sources into one shell, listed below | +| `adapters/` | One directory per framework: `adapter.env`, overlay files, a `lefthook.fragment.yml` | +| `services/` | One directory per database or cache: `service.env`, compose fragments, `drivers/`; `services/shared` is not a service but the driver bodies those `drivers/` source | +| `common/` | Copied into every new project, then rendered | +| `docs/` | This documentation: tour, decisions, runbooks, provenance, diagrams | +| `tests/` | bats suites and their fixtures | +| `scripts/` | CI helpers: adapter matrix, provenance check, deploy check | +| `.github/` | This toolbox's own workflows and pull request template | +| `.vscode/` | Editor settings and recommended extensions | +| `lib/log.sh` | `log`, `warn`, `die`, `step`, `run_quietly` | +| `lib/contract.sh` | The nine contract task names | +| `lib/lint.sh` | `scaffold lint`: adapters and services against the contract | +| `lib/adapter.sh` | Load an adapter, run its generator, overlay its files, wire its services | +| `lib/service.sh` | Compose services, host ports, service drivers | +| `lib/pnpm.sh` | The pnpm workspace and its supply-chain policy | +| `lib/manifest.sh` | `config_roots` and image targets, recorded once and derived everywhere else | +| `lib/project.sh` | Project skeleton, `.scaffold.toml`, the first commit | +| `lib/update.sh` | `scaffold update`: the patch from the recorded commit to this one | +| `lib/publish.sh` | `scaffold publish`: GitHub repository and settings | +| `lib/wizard.sh` | What the wizard asks and the command it builds | +| `lib/tui.sh` | The wizard's terminal screens | + +![Code layers](diagrams/code-layers.svg) + +## How `scaffold new` works + +![scaffold new](diagrams/scaffold-new.svg) + +- `scaffold:parse_new_flags` and `scaffold:resolve_service_selection` read `--web`, `--api`, `--app`, `--db`, `--cache` and refuse a service with no `--api` or `--app`. +- `lib/project.sh:init_project` copies `common/`, renders `you/` and `@PROJECT_NAME@`, and arms the cleanup trap; `init_scaffold_manifest` writes `.scaffold.toml`. +- `lib/service.sh:assemble_compose` and `assemble_example_env` merge each service into the compose lanes and `example.env`. +- `scaffold:install_adapters` runs, per app, `add_app_service`, `lib/adapter.sh:apply_adapter` and `record_scaffold_app`. +- `scaffold:settle_workspace_shape` joins one pnpm workspace when every app is TypeScript, keeps apps standalone otherwise, then runs `register_image_target`. +- `lib/project.sh:finalize_project` syncs the CI roots, runs `mise lock` and commits `feat: scaffold project`. + +## What a generated project contains + +![Generated project](diagrams/generated-project.svg) + +Measured on `scaffold new demo --api nestjs --web nextjs --db postgres`: 101 tracked files, one commit. + +## Commands + +| Command | Flags | Does | Decision | +| --- | --- | --- | --- | +| `scaffold` | none | On a terminal, a wizard for `new`, `update` or `publish`; elsewhere, prints usage and exits 1 | [09-wizard](tour/09-wizard.md) | +| `scaffold new ` | `--web`, `--api`, `--app `; `--db`, `--cache ` | Generates and commits a project | [ADR-0020](decisions/0020-database-default-is-derived-from-requested-adapters.md) | +| `scaffold add ` | `--adapter ` | Adds an app to an existing project and stages it | [ADR-0018](decisions/0018-add-does-not-recompute-the-typescript-workspace.md) | +| `scaffold update [dir]` | `--dry-run` | Applies toolbox changes since the commit in `.scaffold.toml`; never commits | [ADR-0023](decisions/0023-a-project-records-what-generated-it.md) | +| `scaffold publish [dir]` | `--public`, `--private` (default), `--no-protect`, `--dry-run` | Creates the GitHub repository and applies its settings | [ADR-0024](decisions/0024-publishing-a-project-is-part-of-generating-it.md) | +| `scaffold list` | `--adapters`, `--services` | Prints adapters with role and tier, services with kind | [ADR-0012](decisions/0012-tiered-adapter-support.md) | +| `scaffold lint` | none | Checks every adapter and service against the contract | [ADR-0011](decisions/0011-task-contract-names-follow-immich.md) | +| `scaffold --version` | also `-v` | `git describe` of this toolbox, `-dirty` for uncommitted edits | [ADR-0023](decisions/0023-a-project-records-what-generated-it.md) | +| `scaffold --help` | also `-h` | Prints usage | none | + +## Glossary + +| Term | Meaning | Where | +| --- | --- | --- | +| adapter | A framework's generator command plus the files overlaid on its output | `adapters/nestjs` | +| family | `ADAPTER_FAMILY`: which service driver an adapter uses (`laravel`, `nest`, `next`, `flask`) | `adapters/nestjs/adapter.env` | +| tier | `ADAPTER_TIER`: A, B or C, how often CI verifies the adapter | [ADR-0012](decisions/0012-tiered-adapter-support.md) | +| service | A database or cache: `SERVICE_KIND`, a pinned image, compose fragments | `services/postgres` | +| driver | Per-family script that wires a service into an app's code, Dockerfile and compose entry | `services/postgres/drivers` | +| config root | A directory with its own `mise.toml` that CI runs the contract in, listed in `config_roots` | [ADR-0013](decisions/0013-config-roots-is-the-manifest.md) | +| task contract | The nine tasks every app implements: `install` … `checklist` | `lib/contract.sh` | +| overlay | Copying an adapter's files over the generator's output | [ADR-0003](decisions/0003-adapter-overlay-instead-of-vendored-presets.md) | +| splice anchor | The `# @SERVICE_SETUP@` line in an adapter Dockerfile, replaced by the drivers' setup block | `adapters/nestjs/Dockerfile` | +| manifest | `config_roots` in the project's root `mise.toml`: recorded once, the CI matrix is derived from it | `lib/manifest.sh`, [ADR-0013](decisions/0013-config-roots-is-the-manifest.md) | +| `.scaffold.toml` | What generated the project: the toolbox commit and the adapter behind each app, read by `scaffold update` | [ADR-0023](decisions/0023-a-project-records-what-generated-it.md) | + +## Reading path + +| When | Read | +| --- | --- | +| Day one | This page, then [01-toolchain](tour/01-toolchain.md) through [03-ci](tour/03-ci.md) | +| First week | [04-guardrails](tour/04-guardrails.md) through [09-wizard](tour/09-wizard.md); ADR-0001, ADR-0003, ADR-0011 | +| On demand | The [runbook](runbook/) that names the situation | + +## Diagrams + +Each `.svg` is exported from the `.html` beside it. + +| Diagram | Shows | +| --- | --- | +| [system-context](diagrams/system-context.svg) | Toolbox, generated project, reusable workflows, images, releases, client host | +| [code-layers](diagrams/code-layers.svg) | `scaffold`, `lib/*.sh` and the directories they read | +| [scaffold-new](diagrams/scaffold-new.svg) | The `scaffold new` call order | +| [generated-project](diagrams/generated-project.svg) | The tree `scaffold new` produces | +| [release-flow](diagrams/release-flow.svg) | Continuous builds and cut releases of a generated project | +| [scaffold-update](diagrams/scaffold-update.svg) | How `scaffold update` patches an existing project | diff --git a/docs/diagrams/code-layers.html b/docs/diagrams/code-layers.html new file mode 100644 index 0000000..677ee01 --- /dev/null +++ b/docs/diagrams/code-layers.html @@ -0,0 +1,68 @@ + + + + + + How the toolbox code is layered + + + + +
+

Layer stack · scaffold

+

How the toolbox code is layered

+ + How the toolbox code is layered + Layer stack showing the scaffold entry point over the wizard, the project lifecycle libraries, the app and service assembly libraries, shared primitives, and the adapters, services and common directories they read. + + + + + + CALLS + + + READS + + + L1 + Entry point + scaffold · main() → cmd_new, cmd_add, cmd_update, cmd_publish, cmd_list, cmd_lint + + + L2 + Wizard + lib/wizard.sh · lib/tui.sh (no arguments, on a terminal) + + + L3 + Project lifecycle + lib/project.sh · lib/update.sh · lib/publish.sh · lib/lint.sh + + + L4 + Apps and services + lib/adapter.sh · lib/service.sh · lib/pnpm.sh · lib/manifest.sh + + + L5 + Shared primitives + lib/log.sh · lib/contract.sh + + + L6 + Data it reads + adapters/<name>/ · services/<name>/ · common/ + + Every lib/*.sh is sourced into one shell; L4 turns an adapter or a service into files in the project. + +
+ + diff --git a/docs/diagrams/code-layers.svg b/docs/diagrams/code-layers.svg new file mode 100644 index 0000000..aeb2b76 --- /dev/null +++ b/docs/diagrams/code-layers.svg @@ -0,0 +1,47 @@ + + + How the toolbox code is layered + Layer stack showing the scaffold entry point over the wizard, the project lifecycle libraries, the app and service assembly libraries, shared primitives, and the adapters, services and common directories they read. + + + + + + + CALLS + + + READS + + + L1 + Entry point + scaffold · main() → cmd_new, cmd_add, cmd_update, cmd_publish, cmd_list, cmd_lint + + + L2 + Wizard + lib/wizard.sh · lib/tui.sh (no arguments, on a terminal) + + + L3 + Project lifecycle + lib/project.sh · lib/update.sh · lib/publish.sh · lib/lint.sh + + + L4 + Apps and services + lib/adapter.sh · lib/service.sh · lib/pnpm.sh · lib/manifest.sh + + + L5 + Shared primitives + lib/log.sh · lib/contract.sh + + + L6 + Data it reads + adapters/<name>/ · services/<name>/ · common/ + + Every lib/*.sh is sourced into one shell; L4 turns an adapter or a service into files in the project. + diff --git a/docs/diagrams/generated-project.html b/docs/diagrams/generated-project.html new file mode 100644 index 0000000..c594c6a --- /dev/null +++ b/docs/diagrams/generated-project.html @@ -0,0 +1,84 @@ + + + + + + What a generated project contains + + + + +
+

Tree · generated project

+

What a generated project contains

+ + What a generated project contains + Tree of a project generated with nestjs, nextjs and postgres: apps with api and web, packages/types, docs, the GitHub workflows, and root files such as mise.toml and .scaffold.toml. + + + + + + + + + + + + + + + + + + + demo/ + one commit: feat: scaffold project + + + apps/ + one per role + + + packages/types + if every app is TS + + + docs/ + VitePress · ADRs + + + .github/workflows/ + 5 call sites + + + Root files + mise.toml + .scaffold.toml + + + apps/api + nestjs · Dockerfile + + + apps/web + nextjs · Dockerfile + + LEGEND + + Project root + + Directory or file group + + Depends on the apps + +
+ + diff --git a/docs/diagrams/generated-project.svg b/docs/diagrams/generated-project.svg new file mode 100644 index 0000000..bba4044 --- /dev/null +++ b/docs/diagrams/generated-project.svg @@ -0,0 +1,63 @@ + + + What a generated project contains + Tree of a project generated with nestjs, nextjs and postgres: apps with api and web, packages/types, docs, the GitHub workflows, and root files such as mise.toml and .scaffold.toml. + + + + + + + + + + + + + + + + + + + + demo/ + one commit: feat: scaffold project + + + apps/ + one per role + + + packages/types + if every app is TS + + + docs/ + VitePress · ADRs + + + .github/workflows/ + 5 call sites + + + Root files + mise.toml + .scaffold.toml + + + apps/api + nestjs · Dockerfile + + + apps/web + nextjs · Dockerfile + + LEGEND + + Project root + + Directory or file group + + Depends on the apps + diff --git a/docs/diagrams/release-flow.html b/docs/diagrams/release-flow.html new file mode 100644 index 0000000..ed65481 --- /dev/null +++ b/docs/diagrams/release-flow.html @@ -0,0 +1,104 @@ + + + + + + How a generated project releases + + + + +
+

Swimlane · release

+

How a generated project releases

+ + How a generated project releases + Swimlane showing every merge to main building main and sha image tags, Release Please opening a release pull request, and merging it publishing version, major.minor, latest and sha image tags, uploading compose.yaml, example.env and install.sh, and an optional deploy job, before a client host runs install.sh. + + + + + + DEVELOPER + BUILD.YML + app-build + RELEASE.YML + app-release + CLIENT HOST + + + + + + + + + + + + + PUSH + + PUSH + + RELEASE PR + + MERGE + + ASSETS + + + Merge to main + conventional commits + + + Build every image + main · sha-<sha> + + + Release Please + opens release PR + + + Merge release PR + released == true + + + image job + <ver> · <maj.min> + latest · sha-<sha> + + + assets job + compose.yaml + example.env · install.sh + + + deploy job + vars.DEPLOY_TARGET + + + Run install.sh + downloads, + compose up + + LEGEND + + Cut release image + + Step + + Publishes artifacts + + Stub, no target + +
+ + diff --git a/docs/diagrams/release-flow.svg b/docs/diagrams/release-flow.svg new file mode 100644 index 0000000..9eba4c6 --- /dev/null +++ b/docs/diagrams/release-flow.svg @@ -0,0 +1,83 @@ + + + How a generated project releases + Swimlane showing every merge to main building main and sha image tags, Release Please opening a release pull request, and merging it publishing version, major.minor, latest and sha image tags, uploading compose.yaml, example.env and install.sh, and an optional deploy job, before a client host runs install.sh. + + + + + + + DEVELOPER + BUILD.YML + app-build + RELEASE.YML + app-release + CLIENT HOST + + + + + + + + + + + + + PUSH + + PUSH + + RELEASE PR + + MERGE + + ASSETS + + + Merge to main + conventional commits + + + Build every image + main · sha-<sha> + + + Release Please + opens release PR + + + Merge release PR + released == true + + + image job + <ver> · <maj.min> + latest · sha-<sha> + + + assets job + compose.yaml + example.env · install.sh + + + deploy job + vars.DEPLOY_TARGET + + + Run install.sh + downloads, + compose up + + LEGEND + + Cut release image + + Step + + Publishes artifacts + + Stub, no target + diff --git a/docs/diagrams/scaffold-new.html b/docs/diagrams/scaffold-new.html new file mode 100644 index 0000000..fcaf1fe --- /dev/null +++ b/docs/diagrams/scaffold-new.html @@ -0,0 +1,94 @@ + + + + + + What scaffold new does + + + + +
+

Flowchart · scaffold new

+

What scaffold new does

+ + What scaffold new does + Flowchart of scaffold new: parse flags and services, copy and render common, add services, install each adapter app, join one pnpm workspace or keep apps standalone, register image targets, then lock the toolchain and commit. + + + + + + + + + + + + + + + + + YES + + NO + + register_image_target + + trap armed: a failure + removes <name> + + + scaffold new <name> + + + Parse flags, pick services + parse_new_flags · resolve_service_selection + + + Copy common/ and render it + init_project · init_scaffold_manifest + + + Add the database and cache + record_services · assemble_compose + + + Install each app + add_app_service · apply_adapter + + Every app + TypeScript? + + + One pnpm workspace + join_typescript_workspace + + + Apps stay standalone + keep_apps_standalone + + + + Lock and commit + finalize_project + + LEGEND + + Start / end + + Step + + Decision + +
+ + diff --git a/docs/diagrams/scaffold-new.svg b/docs/diagrams/scaffold-new.svg new file mode 100644 index 0000000..0a63b39 --- /dev/null +++ b/docs/diagrams/scaffold-new.svg @@ -0,0 +1,73 @@ + + + What scaffold new does + Flowchart of scaffold new: parse flags and services, copy and render common, add services, install each adapter app, join one pnpm workspace or keep apps standalone, register image targets, then lock the toolchain and commit. + + + + + + + + + + + + + + + + + + YES + + NO + + register_image_target + + trap armed: a failure + removes <name> + + + scaffold new <name> + + + Parse flags, pick services + parse_new_flags · resolve_service_selection + + + Copy common/ and render it + init_project · init_scaffold_manifest + + + Add the database and cache + record_services · assemble_compose + + + Install each app + add_app_service · apply_adapter + + Every app + TypeScript? + + + One pnpm workspace + join_typescript_workspace + + + Apps stay standalone + keep_apps_standalone + + + + Lock and commit + finalize_project + + LEGEND + + Start / end + + Step + + Decision + diff --git a/docs/diagrams/scaffold-update.html b/docs/diagrams/scaffold-update.html new file mode 100644 index 0000000..52f8473 --- /dev/null +++ b/docs/diagrams/scaffold-update.html @@ -0,0 +1,86 @@ + + + + + + What scaffold update does + + + + +
+

Flowchart · scaffold update

+

What scaffold update does

+ + What scaffold update does + Flowchart of scaffold update: read the toolbox commit recorded in .scaffold.toml, diff common and each adapter since then, stop if the patch is empty or print it on --dry-run, otherwise apply it hunk by hunk, re-derive computed files and record the new version. + + + + + + + + + + + + + + + YES + + NO + + YES + + NO + + + scaffold update [dir] + + + Read the recorded commit + resolve_project · update_source_commit + + + Diff the toolbox since then + update_patch: common/ + each adapter + + Patch + empty? + + + Already up to date + + --dry-run? + + + Print the patch + + + Apply what fits + clean tree · git apply --reject + + + Re-derive and record + resync_derived_files · version + + LEGEND + + Start / end + + Step + + Decision + +
+ + diff --git a/docs/diagrams/scaffold-update.svg b/docs/diagrams/scaffold-update.svg new file mode 100644 index 0000000..b256471 --- /dev/null +++ b/docs/diagrams/scaffold-update.svg @@ -0,0 +1,65 @@ + + + What scaffold update does + Flowchart of scaffold update: read the toolbox commit recorded in .scaffold.toml, diff common and each adapter since then, stop if the patch is empty or print it on --dry-run, otherwise apply it hunk by hunk, re-derive computed files and record the new version. + + + + + + + + + + + + + + + + YES + + NO + + YES + + NO + + + scaffold update [dir] + + + Read the recorded commit + resolve_project · update_source_commit + + + Diff the toolbox since then + update_patch: common/ + each adapter + + Patch + empty? + + + Already up to date + + --dry-run? + + + Print the patch + + + Apply what fits + clean tree · git apply --reject + + + Re-derive and record + resync_derived_files · version + + LEGEND + + Start / end + + Step + + Decision + diff --git a/docs/diagrams/system-context.html b/docs/diagrams/system-context.html new file mode 100644 index 0000000..42c4797 --- /dev/null +++ b/docs/diagrams/system-context.html @@ -0,0 +1,88 @@ + + + + + + Where scaffold sits + + + + +
+

Architecture · scaffold

+

Where scaffold sits

+ + Where scaffold sits + Architecture diagram showing the scaffold toolbox generating a project whose workflows call shared reusable workflows, which push container images and upload compose.yaml, example.env and install.sh to a GitHub release that a client host downloads before pulling the images. + + + + + + + + GITHUB · <OWNER> + + + + + + + + GENERATES + + USES @v1 + + PUSHES + + UPLOADS + + DOWNLOADS + + PULLS + + + scaffold toolbox + new · update · publish + + + Generated project + <owner>/<name> + + + Reusable workflows + <owner>/.github @v1 + + + Container images + ghcr.io/<owner>/<name>-<app> + + + GitHub release + compose.yaml · example.env + install.sh + + + Client host + runs install.sh + + LEGEND + + The toolbox + + Repository or host + + Shared CI, by tag + + Published artifact + +
+ + diff --git a/docs/diagrams/system-context.svg b/docs/diagrams/system-context.svg new file mode 100644 index 0000000..a83f9b1 --- /dev/null +++ b/docs/diagrams/system-context.svg @@ -0,0 +1,67 @@ + + + Where scaffold sits + Architecture diagram showing the scaffold toolbox generating a project whose workflows call shared reusable workflows, which push container images and upload compose.yaml, example.env and install.sh to a GitHub release that a client host downloads before pulling the images. + + + + + + + + + GITHUB · <OWNER> + + + + + + + + GENERATES + + USES @v1 + + PUSHES + + UPLOADS + + DOWNLOADS + + PULLS + + + scaffold toolbox + new · update · publish + + + Generated project + <owner>/<name> + + + Reusable workflows + <owner>/.github @v1 + + + Container images + ghcr.io/<owner>/<name>-<app> + + + GitHub release + compose.yaml · example.env + install.sh + + + Client host + runs install.sh + + LEGEND + + The toolbox + + Repository or host + + Shared CI, by tag + + Published artifact +