httpcaddyfile: new tls_automate_names global option

[IslamElsayed]

Assistance Disclosure

I used AI assistance throughout this PR — for exploring the existing tlsapp.go code, drafting the implementation and tests, and writing this description. I verified all of it myself: I read the surrounding code to work out why an automation policy was needed (see below), generated every expected adapt output from a build and checked each one by hand against what the config should say, and confirmed the tests actually fail when the implementation is broken in four different ways. The behaviour claims below are measured, not assumed.

---

Closes #7122.

The problem

Provisioning a certificate for a name you don't serve currently requires giving it a site block:

*.example.com {
}

foo.example.com {
	respond "Real site"
}

That gets the wildcard, but it also adds a route to the http app, so any name pointed at the server without its own site block — bar.example.com — gets an empty-but-valid response instead of no match. Wanting a certificate and wanting to serve a name are separate things, and the Caddyfile had no way to say only the first.

The change

{
	tls_automate_names *.example.com
}

foo.example.com {
	respond "Real site"
}

Per @francislavoie's suggestion on the issue. Adapting the two configs above, the tls app comes out identical — same policy, same subjects, same issuers — and the only difference is the route that is no longer there:

       "routes": [
         { "match": [{"host": ["foo.example.com"]}], "handle": [...], "terminal": true },
-        { "match": [{"host": ["*.example.com"]}], "terminal": true }
       ]
 ...
   "tls": {
+    "certificates": { "automate": ["*.example.com"] },
     "automation": { "policies": [
       { "subjects": ["foo.example.com", "*.example.com"], "issuers": [...] }
     ]}
   }

It also works with no site blocks at all, which #7122 and @felixauringer's comment both ask for — enough to keep certs renewed for a mail or XMPP server, or for names served by a layer 4 app:

{
	tls_automate_names mail.example.com xmpp.example.com
}

{"apps": {"tls": {"certificates": {"automate": ["mail.example.com", "xmpp.example.com"]}}}}

Notes on the implementation

The names need an automation policy, not just the automate loader. My first attempt only appended to the loader, and with email nobody@example.com set the wildcard silently came out with no issuers — it would not have used the configured ACME account. The catch-all policy that would normally carry global options gets dropped once every other policy names its subjects, so these names need a policy of their own; consolidation then folds it back into an identical one. This is what the tls_automate_names.caddyfiletest case pins, and it's the part most worth a careful look.

A policy is only created when it would carry something. Otherwise an inert {"subjects": [...]} with no issuers ends up in the output. I pulled the existing hasGlobalACMEDefaults expression out into a small helper to decide this, since newBaseAutomationPolicy's own "are there global options" check doesn't count email.

Names that can't get a public cert get the internal issuer, the same treatment a site block gives them.

A name that also has a site block stays listed here. It's redundant, and I'd initially planned to skip it (I said as much on the issue), but a site block for http:// only isn't managed by auto-HTTPS — dropping the name because some block exists could silently leave it without a certificate. Listing it twice is harmless; not listing it isn't. Happy to change this if you'd rather.

Repeating the option appends rather than replaces, so a long list can be split over lines.

On the name

I went with tls_automate_names (@francislavoie's suggestion) since @mholt's tls_certs/tls_certificates question was still open. Renaming is a one-line change plus the test filenames — say the word.

Testing

Three adapt tests: the issue's exact scenario, the no-site-block case, and internal/public name splitting with the option repeated. I checked they aren't vacuous by breaking the implementation four ways — dropping the internal split, never creating the policy, not adding to the loader, and making the parser overwrite instead of append — and confirming each mutation fails the tests that should catch it.

go test ./caddyconfig/... ./caddytest/... passes. TestACMEServerWithDefaults is flaky on my machine, but equally so on master (2 of 4 runs failed there, 1 of 3 on this branch), so it looks unrelated.

Docs for the new global option would need a PR to caddyserver/website — glad to write it once the name is settled.

[IslamElsayed]

Good catch, and it was worse than redundant — that combination didn't adapt at all. Reproducing it on the previous commit:

{
	email nobody@example.com
	tls_automate_names foo.example.com
}

foo.example.com {
	tls {
		ca https://acme.example.test/directory
	}
}

hostname appears in more than one automation policy, making certificate management ambiguous: foo.example.com

The adapter rejects overlapping subjects and my second policy collided with the site block's.

Fixed in cef0148: a name that already appears as a subject of some automation policy keeps that policy, and is still added to the automate loader. Same comparison the overlap check itself uses, so the two can't disagree. The config above now adapts, with the site block's issuer intact:

"certificates": { "automate": ["foo.example.com"] },
"automation": { "policies": [
  { "subjects": ["foo.example.com"], "issuers": [{"ca": "https://acme.example.test/directory", "email": "nobody@example.com", "module": "acme"}] }
]}

Regression test added as tls_automate_names_existing_policy.caddyfiletest. It carries both halves in one config — foo.example.com with a site block of its own keeps that policy, while bar.example.com has no site block and gets the one built from global options — so it pins the boundary rather than just the fixed case. I checked it fails when the new guard is removed.

go test ./caddyconfig/... ./caddytest/... is green.

[steadytao]

LGTM. Just need to deliberate on the option name since ya jumped the gun a bit.

[steadytao]

@mholt, are you happy with tls_automate_names, or do you still prefer tls_certs / tls_certificates ?

Once the name is settled, please add the global-option documentation in caddyserver/website @IslamElsayed - preferably human written :/ but I suppose if you can at least get a shape we can always refine it.

[mholt]

What about tls_certificates_for?

[steadytao]

That sounds pretty good to me. Happy for it to be changed and docs PR to be submitted.

[francislavoie]

I don't like _for, we have no precedence of config using that kind of language. I don't find it consistent with the rest of the config.

[steadytao]

Hmmmm that is also true, ill let you battle

[IslamElsayed]

Thanks for the review.

On the name — I'll implement whichever of the three you land on; it's a one-line change plus renaming four test files, so it costs nothing to decide late. One piece of data rather than an opinion, since @francislavoie's point is checkable: of the 41 currently registered global options, none contains a preposition.

acme_ca acme_ca_root acme_dns acme_eab admin auto_https cert_issuer cert_lifetime
debug default_bind default_sni dns ech email events fallback_sni filesystem
grace_period http_port https_port key_type local_certs log metrics ocsp_interval
ocsp_stapling on_demand_tls order persist_config pki preferred_chains renew_interval
renewal_window_ratio servers shutdown_delay skip_install_trust storage storage_check
storage_clean_interval tls_resolvers

on_demand_tls is the closest thing, and that's a compound adjective rather than a preposition linking two nouns. So "no precedence for that kind of language" holds as stated. Whether that should decide it is yours to call — tls_certificates_for does read more obviously than the alternatives.

On docs: yes, I'll open the PR to caddyserver/website. I'll write it myself rather than generate it, and I'd rather do that once the name is settled so the examples don't have to be rewritten — but say the word if you'd prefer a rough shape sooner to refine, and I'll put one up against whichever name is currently winning.

@steadytao one thing from your review I want to make sure I read right: "jumped the gun" — if you mean I should have waited for the name before implementing, that's fair, and I'll hold the rename until you two have settled rather than guessing again.

[steadytao]

That is precisely what I meant. It allows us to prevent this kind of noise in PRs and keep it to issues where it generally belongs. Appreciate it.

[francislavoie]

I still prefer tls_automate_names as I originally suggested. We already have tls_ for other options and this identifies which "app" the option belongs to, automate directly relates to what it produces in JSON, and names is the actual term for what the list contains.

[IslamElsayed]

@steadytao docs PR is up: caddyserver/website#570.

It uses tls_automate_names, per @francislavoie's call above, so nothing here needs changing — the implementation already matches.

One thing in it I flagged there and will repeat: I wrote "(Requires Caddy 2.12 or newer.)" purely because v2.11.4 is the latest release. If this lands in some other version, that line needs correcting and I have no way to know which.

[steadytao]

The implementation still looks correct. However; please add one adaptation fixture covering auto_https off with tls_automate_names since this option deliberately overrides that general certificate-management disable and the documentation should state the same behaviour as I mentioned there.

[IslamElsayed]

Added in 6c55051 — tls_automate_names_auto_https_off.caddyfiletest.

You were right that it needed pinning rather than changing: the option already overrides the disable. The hostless-key block above it is gated on !slices.Contains(autoHTTPS, "off"), but the tls_automate_names block is not, on the reasoning that naming a subject explicitly is a stronger signal than the general switch — the same way force_automate is not gated either. The fixture shows both facts in one adapt output:

"automatic_https": { "disable": true },
...
"tls": { "certificates": { "automate": ["mail.example.com"] } }

I checked it actually guards that: adding the auto_https off gate to the block makes this fixture the only one that fails.

[steadytao]

Website PR needs its updates but other then that; it may be worth including this in 2.12 alongside #7976 - @francislavoie, thoughts?

[francislavoie]

Since it's a low impact new feature and not an API change I'm fine with adding it into 2.11.5 maybe.

[steadytao]

Works for me.

[IslamElsayed]

Thanks — 2.11.5 answers the one thing I had flagged as a guess in the docs PR. Updated it there in caddyserver/website@30a0dc0: the entry now reads "(Requires Caddy 2.11.5 or newer.)" instead of 2.12.

@steadytao if "needs its updates" meant something beyond the version note, say what and I will fix it — that was the only open item I knew of on caddyserver/website#570.