docs(deploy): add native Ubuntu RA and TL setup - #136
csnitker-godaddy wants to merge 3 commits into
Conversation
Signed-off-by: Connor Snitker <csnitker@godaddy.com>
Signed-off-by: Connor Snitker <csnitker@godaddy.com>
Signed-off-by: Connor Snitker <csnitker@godaddy.com>
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The RA configuration creates a critical unresolved privilege boundary through its administrator-level TL API key.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 1
What changed in this PR
Adds native Ubuntu deployment guidance and templates for ANS RA/TL using systemd, Caddy, OIDC, DNS, and Let's Encrypt.
Changes:
- Adds Ubuntu installation and operational documentation.
- Adds RA/TL configuration templates and systemd services.
- Adds Caddy HTTPS routing and maintenance guidance.
| File | Summary | Review notes |
|---|---|---|
README.md |
Links to the Ubuntu deployment guide. | — |
deploy/ubuntu/tl.yaml |
Provides TL configuration and producer-key setup. | — |
deploy/ubuntu/README.md |
Documents installation, bootstrap, HTTPS, and maintenance. | — |
deploy/ubuntu/ra.yaml |
Provides production-oriented RA configuration. | Critical: the RA process receives an administrator-level static TL API key. |
deploy/ubuntu/install-packages.sh |
Installs Ubuntu, Caddy, and Go dependencies. | — |
deploy/ubuntu/Caddyfile |
Configures HTTPS and restricted public TL routes. | — |
deploy/ubuntu/ans-tl.service |
Defines the hardened TL systemd service. | — |
deploy/ubuntu/ans-ra.service |
Defines the hardened RA systemd service. | — |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| tl-client: | ||
| base-url: "http://127.0.0.1:18081" | ||
| public-base-url: "https://tl.ans.example.com" | ||
| api-key: "__TL_SERVICE_KEY__" |
kperry-godaddy
left a comment
kperry-godaddy
left a comment
There was a problem hiding this comment.
This is a real deployment path, not a sketch: the systemd units hold up under ProtectSystem=strict (StateDirectory stays writable), secrets land O_EXCL 0640 root:group, the Go tarball is checksum-verified, and the TL Caddy allowlist matches every read route the TL registers while blocking every ingest and admin route by both path and method. I also checked the config templates key by key against the loader at this base, including vlei.type: "off" and the https-only rule on tl-client.public-base-url, and the note about verify-dns on an ACTIVE registration is accurate.
The one thing I'd want before merge is the run the description says hasn't happened yet: one pass on a clean Ubuntu 24.04 VM, with the release recorded at the top of the README. A 260-line runbook with three programs in it earns that, and it would turn several of the notes below from reasoning into observation.
Six notes inline (three with suggestions you can commit as-is), and a few more below because they are about the PR rather than a line.
Not blocking, worth a look
- "Fixes #133" won't auto-close from this base: GitHub only closes linked issues on merges into the default branch (https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue). The base is justified, since
ca.validation.roots-fileonly exists in the #125-#128 stack, but the merge order needs a plan: a squash merge of #128 that deletes its branch retargets this PR tomainwith #128's commits in its diff. - Because
/v2/admin/*is 404 through Caddy, the guide's smoke tests only work on the host. It would help to name what external monitoring should probe instead, for exampleGET /root-keyson the TL andGET /docson the RA. - README lines 50-52 record a one-off build failure ("Go 1.27 failed the current linter's type checks") rather than the rule. "Build with the Go release named by the
godirective ingo.mod" says what an operator needs and won't read as stale when the linter moves. no-storeon the TL host also covers/tile/*and/checkpoint. The freshness argument in the comment fits the badge, status and audit reads under/v1/; is covering the tile path intended?- The unit hardening is solid as far as it goes.
systemd-analyze security ans-rawill point at the rest:CapabilityBoundingSet=,ProtectClock,ProtectHostname,LockPersonality,RestrictRealtime,RestrictNamespaces,SystemCallFilter=@system-service,SystemCallArchitectures=native,PrivateDevices,ProtectProc=invisible.
| arm64) ans_arch=arm64 ;; | ||
| *) echo "Supported server architectures: amd64 and arm64" >&2; exit 1 ;; | ||
| esac | ||
| curl -fsSL --retry 3 'https://go.dev/dl/?mode=json' -o "$work/releases.json" |
There was a problem hiding this comment.
This has a date on it. ?mode=json returns only the two newest release lines (today that is go1.27.1 and go1.26.8), so the day Go 1.28 ships, 1.26 drops out of the list, line 42 finds nothing, and the installer exits with "No supported Go 1.26 patch release found". include=all keeps the full index:
| curl -fsSL --retry 3 'https://go.dev/dl/?mode=json' -o "$work/releases.json" | |
| curl -fsSL --retry 3 'https://go.dev/dl/?mode=json&include=all' -o "$work/releases.json" |
The .stable == true filter on line 42 still applies. Reading the line from the go directive in go.mod instead of hard-coding go1.26. would remove the second thing that has to move in lockstep.
| sudo python3 - <<'PYTRUST' | ||
| from pathlib import Path | ||
| import yaml | ||
| ra = yaml.safe_load(Path('/etc/ans/ra.yaml').read_text()) | ||
| p = Path('/etc/ans/tl.yaml') | ||
| tl = yaml.safe_load(p.read_text()) | ||
| kid = ra['signer']['keyId'] | ||
| entry = { | ||
| 'raId': ra['signer']['raId'], | ||
| 'keyId': kid, | ||
| 'algorithm': 'ES256', | ||
| 'publicKeyPem': (Path(ra['keys']['file']['path']) / (kid + '.pub')).read_text(), | ||
| } | ||
| existing = tl.setdefault('producerKeys', []) | ||
| matching = [e for e in existing if e['keyId'] == kid] | ||
| if matching and matching != [entry]: | ||
| raise SystemExit('Existing producer key differs; investigate before changing trust') | ||
| if not matching: | ||
| existing.append(entry) | ||
| p.write_text(yaml.safe_dump(tl, sort_keys=False)) | ||
| print('RA public key configured for TL bootstrap.') | ||
| PYTRUST |
There was a problem hiding this comment.
The TL already has an API for this step: POST /internal/v1/producer-keys on 127.0.0.1, authenticated with the same generated service key (static auth is admin), and it is the mechanism this README recommends for later rotations. Using it here removes the YAML rewrite of /etc/ans/tl.yaml, the TL restart, and the config-schema knowledge this script hard-codes (signer.keyId, keys.file.path, the <keyId>.pub layout), all of which drift silently when the RA changes. Roughly:
sudo curl -fsS -X POST http://127.0.0.1:18081/internal/v1/producer-keys \
-H "Authorization: Bearer $(sudo python3 -c 'import yaml;print(yaml.safe_load(open("/etc/ans/tl.yaml"))["auth"]["static"]["api-key"])')" \
-H 'Content-Type: application/json' \
-d "$(sudo jq -n --rawfile pem /var/lib/ans-ra/keys/ans-ra-example-signer.pub \
'{key_id:"ans-ra-example-signer", ra_id:"ans-ra-example", algorithm:"ES256", public_key_pem:$pem,
valid_from:(now|todate), expires_at:((now+315360000)|todate)}')"Same thought for the config installer at lines 93-112: both programs would be easier to keep honest as deploy/ubuntu/install-config.sh and bootstrap-trust.sh, where shellcheck and a smoke test can reach them. Not a blocker for the guide; it is a maintenance point.
| # Keep operational endpoints accessible only through localhost. | ||
| @private path /v2/admin /v2/admin/* | ||
| respond @private 404 | ||
| reverse_proxy 127.0.0.1:18080 |
There was a problem hiding this comment.
The verification section has operators paste an OIDC bearer token into the RA's Swagger UI, and this vhost serves /docs to the internet. The UI pulls its JavaScript from jsdelivr with no integrity hashes (that lives in docsui, not in this PR), so anyone able to tamper with that asset collects every token pasted into the page. The template can close that without touching the code, by keeping the docs off the public side:
@docs {
path /docs /docs/*
not remote_ip 203.0.113.0/24
}
respond @docs 404
or by telling operators to reach the UI through an SSH tunnel to 127.0.0.1:18080. Worth doing before this becomes the recommended production shape.
| ra.ans.example.com { | ||
| redir /docs/ /docs 308 | ||
| # Keep authenticated API responses out of shared proxy caches. | ||
| header Cache-Control "no-store" |
There was a problem hiding this comment.
Caddy doesn't send HSTS on its own, and both hosts are HTTPS-only with the redirect already in place, so the header is free here:
| header Cache-Control "no-store" | |
| header Cache-Control "no-store" | |
| header Strict-Transport-Security "max-age=31536000; includeSubDomains" |
Same line in the TL block (line 19).
| auth: | ||
| type: oidc | ||
| oidc: | ||
| issuer-url: "https://YOUR_INSTANCE.clerk.accounts.dev" |
There was a problem hiding this comment.
#133 asks for templates operators adapt to their own identity provider; with a Clerk-shaped issuer here and the Clerk-only paragraph at README lines 28-32, the guide reads as if Clerk were required. A neutral placeholder keeps the template generic, and Clerk stays useful as the worked example in the prose:
| issuer-url: "https://YOUR_INSTANCE.clerk.accounts.dev" | |
| issuer-url: "https://issuer.example.com" |
| before expiry. Post-renewal verification and sealing of changed DNS evidence | ||
| is not implemented: `verify-dns` on an ACTIVE registration currently returns | ||
| without refreshing its sealed DNS snapshot. Do not treat it as a DNS-update API. |
There was a problem hiding this comment.
Accurate: VerifyDNS returns early on an ACTIVE registration and never touches the sealed snapshot. Right now this paragraph is the only place that limitation is written down, and a deploy guide is an easy place to lose it in a later edit. An issue to link from here would give it a home, and the paragraph can then shrink to one sentence.
Add native Ubuntu deployment instructions and templates for RA/TL using systemd, Caddy, OIDC, real DNS, and Let's Encrypt.
Cover package installation, persistent storage, generated service secrets, RA producer-key bootstrap, and public TL routes including Swagger. Explain staging versus production trust, resolver caching, backups, certificate renewal, and the remaining post-renewal DNS refresh limitation.
Validation: shell syntax, duplicate-key-aware YAML parsing, configuration placeholder inspection, and patch applicability. The generalized instructions have not been installed on a fresh host.
Depends on #128; this PR targets
fix/renewal-publicationto keep the review diff scoped. Same-origin Swagger URLs are provided by a separate documentation fix.Fixes #133.
AI assistance
Assisted-by: Codex (GPT-6), under Connor Snitker's direction.