Issue the cloud cluster's bootstrap certificates after the apply - #93
Merged
Merged
Conversation
The vault role copies files/tls/<inventory_hostname>.{crt,key} to each
node. On a cloud profile the inventory is dynamic and a host is named by
its instance id, so those filenames are not knowable until after the
apply: there was nothing to pre-generate, and a first real apply would
have stalled with a cluster running and no way to configure it.
generate-cloud-certs.sh reads the hosts from the same inventory the
playbook will use, so it runs between terraform-to-ansible.sh and
ansible-playbook. Each leaf carries the private IP, the instance id, the
cluster servername and localhost -- deliberately the same set
issue-node-cert.sh and the vault_pki role issue on renewal, because if
they diverge a node that has renewed stops satisfying a check a node that
has not still passes.
The load balancer's name is not among them. It is AWS-generated and
unknowable before the apply, so --extra-san takes it; without one,
clients dialling `terraform output vault_addr` get a name mismatch
against a certificate that is otherwise correct.
Writing it turned up why this had never worked. The role verified a
delivered certificate with
openssl x509 -checkhost {{ ansible_default_ipv4.address }}
and -checkhost does hostname matching. Against a certificate carrying
IP:10.0.2.15 it reports "does NOT match", because it is looking for a DNS
SAN spelled like an address. So the check demanded a certificate nothing
here issues -- issue-node-cert.sh and the vault_pki role both put the
address in --ip-sans, which is what -checkip reads. A correctly issued
certificate failed, and the only way to pass was to issue a wrong one.
It had never run: the local profile is Docker Compose and does not use
this role, and neither cloud profile has been applied.
The tests take the flag out of the role rather than naming it, which is
the point of putting them in tests/pki rather than beside the generator.
A test that picked its own flag would have agreed with the generator and
missed exactly this.
ansible/files/tls/ is now gitignored. It was not, and this is the first
thing that writes private keys there -- including a CA key kept rather
than discarded, because an autoscaling group produces replacement nodes
without asking and each one needs a certificate.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Sep 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #92, which is stacked on #91. Base retargets as they merge.
Third and last of the blockers found while staging the first real AWS
apply.
The gap
The
vaultrole copiesfiles/tls/<inventory_hostname>.{crt,key}to eachnode. On a cloud profile the inventory is dynamic and a host is named by
its instance id, so those filenames are not knowable until after the
apply. There was nothing to pre-generate and commit, and a first real
apply would have stalled here with a cluster running and no way to
configure it.
scripts/generate-cloud-certs.shreads the hosts from the same inventorythe playbook will use, so it slots between
terraform-to-ansible.shandansible-playbook.IP:<private ip>DNS:<instance id>DNS:<cluster>.vault.internalDNS:localhost,IP:127.0.0.1Deliberately the same set
issue-node-cert.shand thevault_pkiroleissue on renewal: let them diverge and a node that has renewed stops
satisfying a check a node that has not still passes.
The load balancer's name is not among them — AWS generates it and it
cannot be known before the apply, so
--extra-santakes it. Without one,clients dialling
terraform output vault_addrget a name mismatchagainst a certificate that is otherwise correct.
Writing it turned up why this had never worked
The role verified a delivered certificate with:
-checkhostdoes hostname matching. Against a certificate carryingIP:10.0.2.15it reportsdoes NOT match, because it is looking for aDNS SAN spelled like an address:
So the check demanded a certificate nothing here issues —
issue-node-cert.shand thevault_pkirole both put the address in--ip-sans, which is what-checkipreads. A correctly issuedcertificate failed it, and the only way to pass was to issue a wrong one.
It had never run: the local profile is Docker Compose and does not use
this role, and neither cloud profile has been applied.
Why the tests live in
tests/pkiThey read the flag out of the role rather than naming it:
CHECK_FLAG="$(grep -oE '\-check(host|ip)' "$ROLE_TASKS" | head -1)"A test that picked its own flag would have agreed with the generator and
missed exactly this. Thirteen assertions, suite 75 → 89 — including that
another node's address fails the same check, that a host with no private
address is skipped rather than half-issued, that an empty inventory is an
error rather than a CA with no leaves, and that a second run refuses
without
--force(a new CA delivered to some nodes and not others leavesa cluster that cannot form).
Also
ansible/files/tls/is now gitignored. It was not, and this is thefirst thing that writes private keys there — including a CA key kept
rather than discarded, because an autoscaling group produces replacement
nodes without asking and each one needs a certificate.
Verified
shellcheckclean overscripts/*.shand every harness.pki89,ansible80,preflight-static30,cloud-preflight48,docs-indexand
lintgreen — run in a Linux checkout, not through/mnt/c, whereCRLF makes
tests/pkireport a false failure on an unrelated assertion.None of it has run against a real account. The tests assert what the
generator produces and that the role's own check accepts it; what they
cannot show is a node completing a Raft join with the result.
🤖 Generated with Claude Code