Simplify install-time secrets + scaffold render-time catalog

.chezmoidata/secrets.yaml

@@ -0,0 +1,17 @@
+# 1Password reference catalog for chezmoi-native render-time secrets.
+# Consumers reference paths via `.op_refs.<key>` in target-file templates
+# (e.g. private_dot_npmrc.tmpl). chezmoi does not evaluate template syntax
+# inside .chezmoidata — values are plain `op://...` reference strings.
+#
+# Render mechanism: `{{ onepasswordRead .op_refs.<key> }}` in a target-file
+# template with `private_` prefix (mode 0600). Touch ID prompts once per
+# `chezmoi apply` (1Password CLI 10-minute session covers later reads).
+#
+# NOT for install-time GITHUB_TOKEN — that's auto-picked from `gh auth token`
+# if authed, else falls through to anonymous (usually fits 60/hr GH limit).
+# Recovery if rate-limited: see README §"GitHub API rate limit".
+#
+# Populate when the first render-time consumer appears:
+#   1. Add an entry below: `<key>: "op://Vault/Item/field"`
+#   2. Reference from a `private_dot_<file>.tmpl` via `.op_refs.<key>`
+op_refs: {}

.chezmoiscripts/run_onchange_after_50-install-packages.sh.tmpl

@@ -31,25 +31,15 @@ export DOTFILES_GUI_MAC_CASKS={{ (.packages.gui.mac_casks | default (list)) | so
 export DOTFILES_GUI_LINUX_APT={{ (.packages.gui.linux_apt | default (list)) | sortAlpha | uniq | join " " | quote }}
 export DOTFILES_GUI_LINUX_DNF={{ (.packages.gui.linux_dnf | default (list)) | sortAlpha | uniq | join " " | quote }}
 
-# GITHUB_TOKEN cascade: 1Password → gh → anon. Lifts mise's 60/hr anonymous
-# GH API limit on fresh bootstrap. `op read` keeps token in process env only
-# (chezmoi's native onepasswordRead would bake it into the rendered .sh under
-# $TMPDIR). Template lookPath guards omit cascade steps for machines without
-# the corresponding CLI.
+# GITHUB_TOKEN: pre-exported env wins. Otherwise auto-pick from `gh auth token`
+# (cached after `gh auth login`). Empty fallthrough = anonymous (60/hr GH limit
+# — usually enough on a fast link). If exhausted, partial-install warning in
+# lib/install-packages.sh::mise_install_tools() surfaces recovery hint.
 if [ -z "${GITHUB_TOKEN:-}" ]; then
-{{- if lookPath "op" }}
-    # --no-newline: trailing \n would corrupt Authorization: Bearer headers.
-    # $DOTFILES_OP_GITHUB_REF overrides the default item path.
-    GITHUB_TOKEN="$(op read --no-newline \
-        "${DOTFILES_OP_GITHUB_REF:-op://Personal/GitHub API Token/credential}" \
-        2>/dev/null || true)"
-{{- end }}
 {{- if lookPath "gh" }}
-    if [ -z "${GITHUB_TOKEN:-}" ]; then
-        GITHUB_TOKEN="$(gh auth token 2>/dev/null || true)"
-    fi
-{{- end }}
+    GITHUB_TOKEN="$(gh auth token 2>/dev/null || true)"
     [ -n "$GITHUB_TOKEN" ] && export GITHUB_TOKEN
+{{- end }}
 fi
 
 export INSTALL_PACKAGES_INVOKE=1

AGENTS.md

@@ -218,16 +218,24 @@ match the canonical `id` list (Pop!_OS, Mint, Raspbian).
 ### Bootstrap script split — thin wrapper + `lib/`
 
 The install script (`run_onchange_after_50-install-packages.sh.tmpl`) is
-intentionally a thin wrapper (~55 LOC including the 1Password→gh token
-cascade) that:
+intentionally a thin wrapper (~40 LOC) that:
 
 1. Renders chezmoi facts (profile, osid, osRelease.idLike, every package
    list from `.chezmoidata/packages.yaml`) into `DOTFILES_*` env vars.
-2. Runs the GITHUB_TOKEN cascade (template-time `lookPath` guards omit
-   unavailable backends).
+2. Picks `GITHUB_TOKEN` from `gh auth token` if `gh` is authed (cheap
+   shell call, no disk persistence). Pre-exported env wins; empty
+   fallthrough is fine — anonymous 60/hr usually fits.
 3. Sets `INSTALL_PACKAGES_INVOKE=1` and sources
    `{{ .chezmoi.sourceDir }}/lib/install-packages.sh`.
 
+Secret-handling architecture (three layers, three mechanisms):
+
+| Concern | Mechanism | Where |
+|---|---|---|
+| Install-time | `gh auth token` auto-pickup (above) + recovery warning if rate-limited | wrapper + `mise_install_tools()` |
+| Render-time | chezmoi-native `onepasswordRead` via `.chezmoidata/secrets.yaml` `op_refs:` catalog | per-file `private_dot_<X>.tmpl` |
+| Runtime CLI | 1Password Shell Plugins (`op plugin init <cli>`) | user-managed, per-CLI |
+
 The library is pure POSIX `sh` with no template syntax — bats unit tests
 under `tests/unit/install-packages.bats` `source` it (with the invoke
 flag unset so `main` doesn't auto-run) and exercise each function with

README.md

@@ -81,44 +81,37 @@ by default — `sudo ufw enable` to activate).
 For a fresh machine (Mac or Linux), step-by-step:
 
 ```sh
-# 1. (Optional but recommended) Pre-auth GitHub for mise tool downloads.
-#    Lifts mise's 60/hr anonymous limit to 1000/hr — important on slow
-#    links or shared IPs.
-gh auth login                  # gh CLI's interactive auth
-# OR if you use 1Password: create the item once
-op item create --vault Personal --title 'GitHub API Token' \
-    credential=ghp_yourToken   # install-packages auto-reads via `op read`
-
-# 2. Bootstrap one-liner — installs chezmoi, clones source, runs apply.
+# 1. Bootstrap one-liner — installs chezmoi, clones source, runs apply.
 sh -c "$(curl -fsLS get.chezmoi.io)" -- init --apply vaintrub
 
-# 3. At the prompt, pick your profile (default core; pick workstation on
+# 2. At the prompt, pick your profile (default core; pick workstation on
 #    Mac laptop, dev on jetson / Linux server, core elsewhere).
 
-# 4. After apply finishes, reload your shell so mise shims land on PATH:
+# 3. After apply finishes, reload your shell so mise shims land on PATH:
 exec zsh
 
-# 5. Verify the install:
+# 4. Verify the install:
 mise ls                         # all rows should be without "(missing)"
 chezmoi diff                    # should be empty
 ```
 
-If step 5 shows any `(missing)` rows OR command-not-found errors, jump to
+If step 4 shows any `(missing)` rows OR command-not-found errors, the
+anonymous GitHub API rate limit was likely exhausted mid-install. Jump to
 the Recovery section below.
 
 ## Recovery (broken / partial first apply)
 
 ```sh
-# Anonymous GitHub rate-limit ran out → some mise tools missing:
-gh auth login                   # cache token (next apply auto-detects)
-chezmoi apply                   # picks up token, retries missing tools
-# OR seed token in 1Password (item: op://Personal/GitHub API Token/credential)
+# Anonymous GitHub rate-limit ran out → some mise tools missing.
+# Pick one (see §"1Password integration → Install-time" for full options):
+gh auth login                   # cache token (subsequent applies auto-pick via `gh auth token`)
+chezmoi apply                   # picks up token, retries missing tools idempotently
 
 # mise shims out of date / new tools not yet symlinked:
 mise reshim
 exec zsh
 
-# `chezmoi.toml` cache has stale keys (old `personal`/`headless`, or `mac`):
+# `chezmoi.toml` cache has stale keys (old `personal`/`headless`):
 chezmoi init --promptDefaults   # rewrites from current template
 chezmoi apply
 
@@ -148,31 +141,77 @@ to apply.
 
 ## 1Password integration
 
-If you have the 1Password CLI (`op`) installed and signed in, the install-
-packages wrapper auto-reads `GITHUB_TOKEN` from an item at the conventional
-path:
+Three distinct secret-handling concerns; each uses its own mechanism.
 
-    op://Personal/GitHub API Token/credential
+### Install-time (e.g. `GITHUB_TOKEN` for mise rate limit)
 
-`op` itself is installed cross-platform by mise (Mac + Linux arm64/amd64) at
-`dev`/`workstation` profile — see `dot_config/mise/config.toml.tmpl` dev block
-(unqualified `op` shortname → mise direct-URL backend against 1Password's CDN).
+No prereq normally required — anonymous 60/hr GH API limit usually fits a
+cold install on a fast link. The install-packages wrapper auto-picks
+`GITHUB_TOKEN` from `gh auth token` if `gh` is authed; otherwise it
+falls through to anonymous.
 
-Mac `workstation` additionally installs the 1Password desktop app cask, which
-enables Touch ID biometric unlock so `op read` works without `op signin`.
+If you hit the rate limit (slow link, shared IP, multiple rebuilds in
+one hour), `mise_install_tools()` surfaces a partial-install warning
+with three recovery options:
 
-Linux headless boxes use `OP_SERVICE_ACCOUNT_TOKEN` instead of biometric.
+```sh
+gh auth login                                             # cached token, recommended
+export GITHUB_TOKEN=ghp_yourPAT                           # one-off env
+export GITHUB_TOKEN=$(op read 'op://Personal/GitHub API Token/credential')
+```
+
+Then `chezmoi apply` — mise retries missing tools idempotently. After
+`gh auth login` no manual env-export is needed on subsequent applies.
+
+### Render-time (auth tokens that must live in target files)
+
+Tools that read credentials from on-disk files (npm, aws, rclone, docker,
+git HTTP creds) need the secret baked into the target. Use chezmoi-native
+render-time funcs against a single catalog at `.chezmoidata/secrets.yaml`.
+
+Workflow when a new render-time secret is needed:
+
+1. Add the 1Password reference to the catalog:
+
+   ```yaml
+   # .chezmoidata/secrets.yaml
+   op_refs:
+     npm_token: "op://Personal/NPM Registry/token"
+   ```
+
+2. Create a `private_dot_<file>.tmpl` referencing it:
+
+   ```
+   //registry.npmjs.org/:_authToken={{ onepasswordRead .op_refs.npm_token }}
+   ```
+
+3. `chezmoi apply` — file lands at `~/.npmrc` mode 0600 with the secret
+   baked in. Touch ID prompts once per apply (1Password CLI 10-minute
+   session covers subsequent reads).
 
-Custom vault layout? Override the item reference per machine:
+### Runtime CLI auth (gh, aws, npm CLIs themselves)
+
+Use [1Password Shell Plugins](https://developer.1password.com/docs/cli/shell-plugins/)
+per CLI — biometric-gated, lazy, no disk exposure:
 
 ```sh
-# ~/.zshrc_local
-export DOTFILES_OP_GITHUB_REF='op://Private/My-GH-Token/value'
+op plugin init gh
+op plugin init aws
+op plugin init npm
 ```
 
-The wrapper's cascade is `op` → `gh auth token` → anonymous. If `op` isn't
-installed (Linux jetson, core profile) the `gh` path picks up; if neither, the
-anonymous path still works (60/hr) for small bootstraps.
+Not in any catalog — `op` manages auth per CLI internally.
+
+### `op` install + auth
+
+`op` itself is installed cross-platform by mise (Mac + Linux arm64/amd64)
+at the `dev` / `workstation` profile — see `dot_config/mise/config.toml.tmpl`
+dev block (unqualified `op` shortname → mise direct-URL backend against
+1Password's CDN).
+
+Mac `workstation` additionally installs the 1Password desktop app cask,
+which enables Touch ID biometric unlock so `op read` works without `op signin`.
+Linux headless boxes use `OP_SERVICE_ACCOUNT_TOKEN` instead of biometric.
 
 ## Tools managed by mise (cross-platform via aqua + github + direct-URL backends)
 
@@ -222,21 +261,7 @@ Tools NOT in mise's aqua registry — C apps with libpcap/curses/PAM deps, syste
 
 **First-apply latency** at `dev` or `workstation`: cold install downloads ~1.9 GB (4 language toolchains × ~300 MB + ~30 binaries via mise). Realistic ranges: 5-10 min on fast link, 15-30 min on residential, 30-60+ min on slow links / jetson. `core` profile only pulls fzf+zoxide (~5 MB).
 
-**GitHub API rate limit on fresh bootstrap**: mise's aqua + github backends hit `api.github.com` once per tool (~29 tools × 1-2 calls ≈ 30-50 requests). Anonymous limit is 60/hr — easy to exhaust on a slow link or shared IP. Two ways to lift it:
-
-```sh
-# Option 1 — set token directly before apply (one-off)
-export GITHUB_TOKEN=ghp_yourPersonalAccessToken
-chezmoi apply
-
-# Option 2 — `gh auth login` once, install-packages auto-picks it up on
-# subsequent applies via `gh auth token` (works after first apply when
-# gh is installed by mise).
-gh auth login                 # interactive, browser flow
-chezmoi apply                 # token auto-detected, ~5000/hr limit
-```
-
-Anonymous bootstrap on a fast link usually fits inside 60/hr — the auto-detect is a safety net for jetson / VPS / re-runs.
+**GitHub API rate limit**: mise's aqua + github backends hit `api.github.com` ~30-50 times during a cold dev install. Anonymous limit is 60/hr — usually fits. If hit, the install-packages partial-install warning surfaces; see §"1Password integration → Install-time" above for recovery options.
 
 **Troubleshooting slow zsh prompt**: `MISE_TIMINGS=1 exec zsh` — total < 50ms per prompt is healthy. If consistently above, switch to mise shim mode in `dot_zshrc`:
 ```zsh

lib/install-packages.sh

@@ -138,7 +138,10 @@ mise_install_tools() {
         echo "" >&2
         echo "[install-packages] WARNING: $missing_count mise tool(s) missing." >&2
         echo "  Likely cause: GitHub API rate-limit (60/hr anonymous)." >&2
-        echo "  Fix: gh auth login   or   op item create 'op://Personal/GitHub API Token/credential'" >&2
+        echo "  Fix (pick one):" >&2
+        echo "    gh auth login                                              # cached token, recommended" >&2
+        echo "    export GITHUB_TOKEN=ghp_yourPAT                            # one-off env" >&2
+        echo "    export GITHUB_TOKEN=\$(op read 'op://Personal/GitHub API Token/credential')" >&2
         echo "  Then: chezmoi apply" >&2
         echo "" >&2
     fi