diff --git a/.chezmoidata/secrets.yaml b/.chezmoidata/secrets.yaml new file mode 100644 index 0000000..41a1b16 --- /dev/null +++ b/.chezmoidata/secrets.yaml @@ -0,0 +1,17 @@ +# 1Password reference catalog for chezmoi-native render-time secrets. +# Consumers reference paths via `.op_refs.` in target-file templates +# (e.g. private_dot_npmrc.tmpl). chezmoi does not evaluate template syntax +# inside .chezmoidata — values are plain `op://...` reference strings. +# +# Render mechanism: `{{ onepasswordRead .op_refs. }}` in a target-file +# template with `private_` prefix (mode 0600). Touch ID prompts once per +# `chezmoi apply` (1Password CLI 10-minute session covers later reads). +# +# NOT for install-time GITHUB_TOKEN — that's auto-picked from `gh auth token` +# if authed, else falls through to anonymous (usually fits 60/hr GH limit). +# Recovery if rate-limited: see README §"GitHub API rate limit". +# +# Populate when the first render-time consumer appears: +# 1. Add an entry below: `: "op://Vault/Item/field"` +# 2. Reference from a `private_dot_.tmpl` via `.op_refs.` +op_refs: {} diff --git a/.chezmoiscripts/run_onchange_after_50-install-packages.sh.tmpl b/.chezmoiscripts/run_onchange_after_50-install-packages.sh.tmpl index 97dd66c..7cd636e 100644 --- a/.chezmoiscripts/run_onchange_after_50-install-packages.sh.tmpl +++ b/.chezmoiscripts/run_onchange_after_50-install-packages.sh.tmpl @@ -31,25 +31,15 @@ export DOTFILES_GUI_MAC_CASKS={{ (.packages.gui.mac_casks | default (list)) | so export DOTFILES_GUI_LINUX_APT={{ (.packages.gui.linux_apt | default (list)) | sortAlpha | uniq | join " " | quote }} export DOTFILES_GUI_LINUX_DNF={{ (.packages.gui.linux_dnf | default (list)) | sortAlpha | uniq | join " " | quote }} -# GITHUB_TOKEN cascade: 1Password → gh → anon. Lifts mise's 60/hr anonymous -# GH API limit on fresh bootstrap. `op read` keeps token in process env only -# (chezmoi's native onepasswordRead would bake it into the rendered .sh under -# $TMPDIR). Template lookPath guards omit cascade steps for machines without -# the corresponding CLI. +# GITHUB_TOKEN: pre-exported env wins. Otherwise auto-pick from `gh auth token` +# (cached after `gh auth login`). Empty fallthrough = anonymous (60/hr GH limit +# — usually enough on a fast link). If exhausted, partial-install warning in +# lib/install-packages.sh::mise_install_tools() surfaces recovery hint. if [ -z "${GITHUB_TOKEN:-}" ]; then -{{- if lookPath "op" }} - # --no-newline: trailing \n would corrupt Authorization: Bearer headers. - # $DOTFILES_OP_GITHUB_REF overrides the default item path. - GITHUB_TOKEN="$(op read --no-newline \ - "${DOTFILES_OP_GITHUB_REF:-op://Personal/GitHub API Token/credential}" \ - 2>/dev/null || true)" -{{- end }} {{- if lookPath "gh" }} - if [ -z "${GITHUB_TOKEN:-}" ]; then - GITHUB_TOKEN="$(gh auth token 2>/dev/null || true)" - fi -{{- end }} + GITHUB_TOKEN="$(gh auth token 2>/dev/null || true)" [ -n "$GITHUB_TOKEN" ] && export GITHUB_TOKEN +{{- end }} fi export INSTALL_PACKAGES_INVOKE=1 diff --git a/AGENTS.md b/AGENTS.md index 4bc3a72..0fb81ba 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -218,16 +218,24 @@ match the canonical `id` list (Pop!_OS, Mint, Raspbian). ### Bootstrap script split — thin wrapper + `lib/` The install script (`run_onchange_after_50-install-packages.sh.tmpl`) is -intentionally a thin wrapper (~55 LOC including the 1Password→gh token -cascade) that: +intentionally a thin wrapper (~40 LOC) that: 1. Renders chezmoi facts (profile, osid, osRelease.idLike, every package list from `.chezmoidata/packages.yaml`) into `DOTFILES_*` env vars. -2. Runs the GITHUB_TOKEN cascade (template-time `lookPath` guards omit - unavailable backends). +2. Picks `GITHUB_TOKEN` from `gh auth token` if `gh` is authed (cheap + shell call, no disk persistence). Pre-exported env wins; empty + fallthrough is fine — anonymous 60/hr usually fits. 3. Sets `INSTALL_PACKAGES_INVOKE=1` and sources `{{ .chezmoi.sourceDir }}/lib/install-packages.sh`. +Secret-handling architecture (three layers, three mechanisms): + +| Concern | Mechanism | Where | +|---|---|---| +| Install-time | `gh auth token` auto-pickup (above) + recovery warning if rate-limited | wrapper + `mise_install_tools()` | +| Render-time | chezmoi-native `onepasswordRead` via `.chezmoidata/secrets.yaml` `op_refs:` catalog | per-file `private_dot_.tmpl` | +| Runtime CLI | 1Password Shell Plugins (`op plugin init `) | user-managed, per-CLI | + The library is pure POSIX `sh` with no template syntax — bats unit tests under `tests/unit/install-packages.bats` `source` it (with the invoke flag unset so `main` doesn't auto-run) and exercise each function with diff --git a/README.md b/README.md index 935a29b..0aecc97 100644 --- a/README.md +++ b/README.md @@ -81,44 +81,37 @@ by default — `sudo ufw enable` to activate). For a fresh machine (Mac or Linux), step-by-step: ```sh -# 1. (Optional but recommended) Pre-auth GitHub for mise tool downloads. -# Lifts mise's 60/hr anonymous limit to 1000/hr — important on slow -# links or shared IPs. -gh auth login # gh CLI's interactive auth -# OR if you use 1Password: create the item once -op item create --vault Personal --title 'GitHub API Token' \ - credential=ghp_yourToken # install-packages auto-reads via `op read` - -# 2. Bootstrap one-liner — installs chezmoi, clones source, runs apply. +# 1. Bootstrap one-liner — installs chezmoi, clones source, runs apply. sh -c "$(curl -fsLS get.chezmoi.io)" -- init --apply vaintrub -# 3. At the prompt, pick your profile (default core; pick workstation on +# 2. At the prompt, pick your profile (default core; pick workstation on # Mac laptop, dev on jetson / Linux server, core elsewhere). -# 4. After apply finishes, reload your shell so mise shims land on PATH: +# 3. After apply finishes, reload your shell so mise shims land on PATH: exec zsh -# 5. Verify the install: +# 4. Verify the install: mise ls # all rows should be without "(missing)" chezmoi diff # should be empty ``` -If step 5 shows any `(missing)` rows OR command-not-found errors, jump to +If step 4 shows any `(missing)` rows OR command-not-found errors, the +anonymous GitHub API rate limit was likely exhausted mid-install. Jump to the Recovery section below. ## Recovery (broken / partial first apply) ```sh -# Anonymous GitHub rate-limit ran out → some mise tools missing: -gh auth login # cache token (next apply auto-detects) -chezmoi apply # picks up token, retries missing tools -# OR seed token in 1Password (item: op://Personal/GitHub API Token/credential) +# Anonymous GitHub rate-limit ran out → some mise tools missing. +# Pick one (see §"1Password integration → Install-time" for full options): +gh auth login # cache token (subsequent applies auto-pick via `gh auth token`) +chezmoi apply # picks up token, retries missing tools idempotently # mise shims out of date / new tools not yet symlinked: mise reshim exec zsh -# `chezmoi.toml` cache has stale keys (old `personal`/`headless`, or `mac`): +# `chezmoi.toml` cache has stale keys (old `personal`/`headless`): chezmoi init --promptDefaults # rewrites from current template chezmoi apply @@ -148,31 +141,77 @@ to apply. ## 1Password integration -If you have the 1Password CLI (`op`) installed and signed in, the install- -packages wrapper auto-reads `GITHUB_TOKEN` from an item at the conventional -path: +Three distinct secret-handling concerns; each uses its own mechanism. - op://Personal/GitHub API Token/credential +### Install-time (e.g. `GITHUB_TOKEN` for mise rate limit) -`op` itself is installed cross-platform by mise (Mac + Linux arm64/amd64) at -`dev`/`workstation` profile — see `dot_config/mise/config.toml.tmpl` dev block -(unqualified `op` shortname → mise direct-URL backend against 1Password's CDN). +No prereq normally required — anonymous 60/hr GH API limit usually fits a +cold install on a fast link. The install-packages wrapper auto-picks +`GITHUB_TOKEN` from `gh auth token` if `gh` is authed; otherwise it +falls through to anonymous. -Mac `workstation` additionally installs the 1Password desktop app cask, which -enables Touch ID biometric unlock so `op read` works without `op signin`. +If you hit the rate limit (slow link, shared IP, multiple rebuilds in +one hour), `mise_install_tools()` surfaces a partial-install warning +with three recovery options: -Linux headless boxes use `OP_SERVICE_ACCOUNT_TOKEN` instead of biometric. +```sh +gh auth login # cached token, recommended +export GITHUB_TOKEN=ghp_yourPAT # one-off env +export GITHUB_TOKEN=$(op read 'op://Personal/GitHub API Token/credential') +``` + +Then `chezmoi apply` — mise retries missing tools idempotently. After +`gh auth login` no manual env-export is needed on subsequent applies. + +### Render-time (auth tokens that must live in target files) + +Tools that read credentials from on-disk files (npm, aws, rclone, docker, +git HTTP creds) need the secret baked into the target. Use chezmoi-native +render-time funcs against a single catalog at `.chezmoidata/secrets.yaml`. + +Workflow when a new render-time secret is needed: + +1. Add the 1Password reference to the catalog: + + ```yaml + # .chezmoidata/secrets.yaml + op_refs: + npm_token: "op://Personal/NPM Registry/token" + ``` + +2. Create a `private_dot_.tmpl` referencing it: + + ``` + //registry.npmjs.org/:_authToken={{ onepasswordRead .op_refs.npm_token }} + ``` + +3. `chezmoi apply` — file lands at `~/.npmrc` mode 0600 with the secret + baked in. Touch ID prompts once per apply (1Password CLI 10-minute + session covers subsequent reads). -Custom vault layout? Override the item reference per machine: +### Runtime CLI auth (gh, aws, npm CLIs themselves) + +Use [1Password Shell Plugins](https://developer.1password.com/docs/cli/shell-plugins/) +per CLI — biometric-gated, lazy, no disk exposure: ```sh -# ~/.zshrc_local -export DOTFILES_OP_GITHUB_REF='op://Private/My-GH-Token/value' +op plugin init gh +op plugin init aws +op plugin init npm ``` -The wrapper's cascade is `op` → `gh auth token` → anonymous. If `op` isn't -installed (Linux jetson, core profile) the `gh` path picks up; if neither, the -anonymous path still works (60/hr) for small bootstraps. +Not in any catalog — `op` manages auth per CLI internally. + +### `op` install + auth + +`op` itself is installed cross-platform by mise (Mac + Linux arm64/amd64) +at the `dev` / `workstation` profile — see `dot_config/mise/config.toml.tmpl` +dev block (unqualified `op` shortname → mise direct-URL backend against +1Password's CDN). + +Mac `workstation` additionally installs the 1Password desktop app cask, +which enables Touch ID biometric unlock so `op read` works without `op signin`. +Linux headless boxes use `OP_SERVICE_ACCOUNT_TOKEN` instead of biometric. ## Tools managed by mise (cross-platform via aqua + github + direct-URL backends) @@ -222,21 +261,7 @@ Tools NOT in mise's aqua registry — C apps with libpcap/curses/PAM deps, syste **First-apply latency** at `dev` or `workstation`: cold install downloads ~1.9 GB (4 language toolchains × ~300 MB + ~30 binaries via mise). Realistic ranges: 5-10 min on fast link, 15-30 min on residential, 30-60+ min on slow links / jetson. `core` profile only pulls fzf+zoxide (~5 MB). -**GitHub API rate limit on fresh bootstrap**: mise's aqua + github backends hit `api.github.com` once per tool (~29 tools × 1-2 calls ≈ 30-50 requests). Anonymous limit is 60/hr — easy to exhaust on a slow link or shared IP. Two ways to lift it: - -```sh -# Option 1 — set token directly before apply (one-off) -export GITHUB_TOKEN=ghp_yourPersonalAccessToken -chezmoi apply - -# Option 2 — `gh auth login` once, install-packages auto-picks it up on -# subsequent applies via `gh auth token` (works after first apply when -# gh is installed by mise). -gh auth login # interactive, browser flow -chezmoi apply # token auto-detected, ~5000/hr limit -``` - -Anonymous bootstrap on a fast link usually fits inside 60/hr — the auto-detect is a safety net for jetson / VPS / re-runs. +**GitHub API rate limit**: mise's aqua + github backends hit `api.github.com` ~30-50 times during a cold dev install. Anonymous limit is 60/hr — usually fits. If hit, the install-packages partial-install warning surfaces; see §"1Password integration → Install-time" above for recovery options. **Troubleshooting slow zsh prompt**: `MISE_TIMINGS=1 exec zsh` — total < 50ms per prompt is healthy. If consistently above, switch to mise shim mode in `dot_zshrc`: ```zsh diff --git a/lib/install-packages.sh b/lib/install-packages.sh index 3088193..6fffc88 100644 --- a/lib/install-packages.sh +++ b/lib/install-packages.sh @@ -138,7 +138,10 @@ mise_install_tools() { echo "" >&2 echo "[install-packages] WARNING: $missing_count mise tool(s) missing." >&2 echo " Likely cause: GitHub API rate-limit (60/hr anonymous)." >&2 - echo " Fix: gh auth login or op item create 'op://Personal/GitHub API Token/credential'" >&2 + echo " Fix (pick one):" >&2 + echo " gh auth login # cached token, recommended" >&2 + echo " export GITHUB_TOKEN=ghp_yourPAT # one-off env" >&2 + echo " export GITHUB_TOKEN=\$(op read 'op://Personal/GitHub API Token/credential')" >&2 echo " Then: chezmoi apply" >&2 echo "" >&2 fi