Repository navigation
fix(harness): verify the manager by name; let rustls explain itself - #201
Merged
Merged
Conversation
…ature)
`default-features = false` on rustls dropped `logging`, one of its defaults.
Without it rustls cannot say WHY it rejects a chain, which is why three rounds
of debugging inferred its reasoning from the outside and kept landing on wrong
answers. Turning it back on took one Cargo change and produced the actual cause
in a single run.
What rustls reports from inside the guest:
Got CertificateRequest ... signature_algorithms: [ECDSA_NISTP384_SHA384,
ECDSA_NISTP256_SHA256, ..., RSA_PKCS1_SHA256]
Client auth requested but no cert/sigscheme available
No cached session for IpAddress(V4(Ipv4Addr([172, 16, 0, 1])))
Two findings, neither of which was visible before:
1. rustls is verifying against **IpAddress(172.16.0.1)**, not
DnsName("manager.lantern.internal"). The `domain_name` set on
ClientTlsConfig is not reaching the connection, so the server certificate is
checked against the VM's gateway address — which is not one of its SANs
(IP:127.0.0.1, DNS:localhost, DNS:manager.lantern.internal). This is the
name-pinning added for per-VM point-to-point links failing to take effect.
2. The client never presents its certificate: "Client auth requested but no
cert/sigscheme available", even though the server advertises RSA_PKCS1_SHA256
and the VM's key is RSA-2048. So mTLS client auth is also not working, and
this is a separate fault from (1).
The connection still dies reporting UnknownIssuer, so at least one of these is
mis-surfaced by tonic rather than reported as a name or client-auth failure.
VendSecret is NOT fixed by this commit.
Also noted, not fixed here: tracing is initialised BEFORE
`bootenv::bootstrap()`, so RUST_LOG arriving via boot-args can never affect the
filter — boot-args cannot configure logging in a microVM at all. Diagnosing this
needed the filter default patched instead, which is why it is worth fixing
properly.
…ns to have
rustls was checking the manager's server certificate against
IpAddress(172.16.x.1) — the VM's gateway — which is not, and can never be, one
of its SANs. Every microVM sits on its own point-to-point link, so the manager's
address differs per VM; only a NAME can be on the certificate.
`ClientTlsConfig::domain_name` was supposed to handle exactly this, and it is
set. It observably does not reach the connection: with rustls logging enabled
the ServerName was IpAddress on every attempt, never DnsName. Rather than
depend on that override taking effect, put the name in the URI so the TLS
server name is correct by construction.
The harness now pins `manager.lantern.internal` to its gateway in /etc/hosts at
boot and dials that name. One certificate with a DNS:manager.lantern.internal
SAN then covers every VM, with no per-VM IP SANs and no library override in the
path. Best-effort: an unparseable address or an unwritable /etc/hosts leaves the
original address in place, so the connection behaves exactly as before.
Verified on real KVM with rustls trace logging:
before: 5x IpAddress(V4(172.16.0.1)), 0x DnsName
after: 5x DnsName("manager.lantern.internal"), 0x IpAddress
VendSecret still fails, and NOT because of this. `UnknownIssuer` persists with
the name now correct, and the client still sends no certificate ("Client auth
requested but no cert/sigscheme available"). Taken together — wrong server name,
no trusted roots, no client identity — the evidence now points at the whole
ClientTlsConfig not reaching the connection, with tonic falling back to a
default. That the ServerName tracked the URI change while the roots stayed empty
fits that reading. It is a single remaining hypothesis rather than three
symptoms, but it is not yet proven, and no fix for it is attempted here.
Contributor
🔷 Gemini cross-audit (agent:audit-gemini)[Blocking] Soundness Issue: Concurrency & Undefined Behavior in
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two commits: re-enable rustls's own logging, and fix the server-name fault it exposed.
1. Let rustls explain itself
default-features = falseon rustls had droppedlogging, one of its defaults. Without it rustls cannot report why it rejects a chain — which is why several rounds of debugging inferred its reasoning from outside and kept landing on wrong answers (DER encoding, musl, entropy, clock skew, crypto provider — all checked, all wrong).One Cargo line, and it named the cause in a single run.
2. Verify the manager by name, not by address
rustls was checking the manager's certificate against
IpAddress(172.16.x.1)— the VM's gateway — which is not, and can never be, one of its SANs. Every microVM sits on its own point-to-point link, so the manager's address differs per VM; only a name can be on the certificate.ClientTlsConfig::domain_nameexists for exactly this and is set. It observably does not reach the connection: with logging on, the ServerName wasIpAddresson every attempt, neverDnsName. I verified the config is built correctly (identity + CA + domain_name) and that the running binary contains the change, so this isn't a stale build.Rather than depend on that override, the name now goes in the URI: the harness pins
manager.lantern.internalto its gateway in/etc/hostsat boot and dials that. The TLS server name is then correct by construction, one certificate with aDNS:manager.lantern.internalSAN covers every VM, and no library override sits in the path. Best-effort — an unparseable address or unwritable/etc/hostsleaves the original behaviour untouched.Verified on real KVM
VendSecret is still NOT fixed
Stating that plainly.
UnknownIssuerpersists with the name now correct, and the client still sends no certificate ("Client auth requested but no cert/sigscheme available").Those three facts — wrong server name, no trusted roots, no client identity — now point at one cause rather than three: the whole
ClientTlsConfignot reaching the connection, with tonic falling back to a default. That the ServerName tracked my URI change while the roots stayed empty fits that reading. It's a single hypothesis instead of a pile of symptoms, but it is not proven, and no fix for it is attempted here.153 harness tests pass, clippy and rustfmt clean.