fix: strict X.509 certificate chains for Hermes Python 3.13

.github/workflows/ci.yml

@@ -54,6 +54,13 @@ jobs:
       - run: cargo fmt --all -- --check
       - run: cargo clippy --all-targets --all-features -- -D warnings
       - run: cargo test --all-targets --all-features
+      - uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
+        with:
+          python-version: "3.13.5"
+      - name: Install Hermes-matched TLS client
+        run: python -m pip install httpx==0.28.1
+      - name: Test Python strict CONNECT certificates
+        run: CHARON_STRICT_TLS_PYTHON="$(command -v python)" cargo test --lib gateway::tests::python_313_strict_clients_accept_generated_chain -- --ignored --nocapture
       - name: Test deployment contract
         run: sh tests/deploy_redeploy.sh
 

Cargo.toml

@@ -41,6 +41,7 @@ zeroize = "1.8"
 [dev-dependencies]
 http-body-util = "0.1"
 tempfile = "3.23"
+x509-parser = "0.18"
 
 
 [lints.rust]

contracts/workload-gateway.md

@@ -29,6 +29,14 @@ or HTTP/2. CONNECT, TLS SNI, HTTP `Host`, and HTTP/2 `:authority` must agree;
 conflicting/malformed/duplicate authority forms fail closed. All permitted
 HTTPS terminates at Charon; no splice, direct tunnel, or TLS fallback exists.
 Every request on a reused or multiplexed connection is authorized afresh.
+Generated CAs carry critical CA Basic Constraints and signing Key Usage,
+plus noncritical subject/authority key identifiers. Interception leaves carry
+an exact DNS SAN, critical non-CA Basic Constraints and digital-signature Key
+Usage, server-authentication EKU, and noncritical subject/authority key identifiers.
+The leaf authority identifier matches the loaded issuer's subject identifier.
+These chains support Python 3.13/OpenSSL strict verification without relaxing
+certificate, hostname or upstream validation. A signing CA missing required
+extensions must be regenerated and its public trust distributed by Infra.
 CONNECT itself grants no operation or credential. Tunnels expire after one
 hour; each operation has its own shorter configured time limits.
 

docs/conventions.md

@@ -11,6 +11,9 @@ Python, and
 stable operator tasks across formatting, linting, tests, deployment-contract
 tests, and dependency policy. This cross-tool quality gate is why
 `mise run check` exists.
+Its `strict-tls-check` also selects Python 3.13.5 and HTTPX 0.28.1 in a
+disposable venv to verify generated CONNECT chains with Hermes-matched strict
+TLS clients. This client test does not change the admission package's Python pin.
 
 Repository-root `tmp/` is an ignored workspace for disposable artifacts. Use
 `tmp/<task-name>/` to avoid collisions. Never store credentials there. Move

docs/deployment.md

@@ -138,6 +138,14 @@ until the explicit owner security review covers ADR 0002 and ADR 0003.
 
 ### CA commands and rotation
 
+Generated deployment CAs include critical CA Basic Constraints and signing Key
+Usage, with subject/authority identifiers. Intercepted server leaves include
+the issuer authority identifier, non-CA constraints, digital-signature Key Usage
+and server-authentication EKU. These extensions support strict Python 3.13 TLS
+clients. Regenerate a CA missing required extensions and distribute its new
+public trust before using the corrected gateway; an image update cannot alter
+an existing CA certificate. Do not weaken client or upstream verification.
+
 Charon refuses to overwrite CA files. Generate a deployment CA only in an
 operator-owned protected directory:
 

docs/hermes-gateway.md

@@ -60,6 +60,22 @@ and ordinary public roots. Set `SSL_CERT_FILE`, `REQUESTS_CA_BUNDLE` and
 `NODE_EXTRA_CA_CERTS` if used. Confirm Go/gh and Git trust the installed root.
 Never use TLS verification disable flags.
 
+Hermes Python 3.13.5 and HTTPX 0.28.1 require the complete generated chain to
+pass OpenSSL strict X.509 validation. Charon generates CA signing Key Usage,
+CA Basic Constraints and key identifiers, and server leaves with exact DNS
+SAN, non-CA constraints, digital-signature Key Usage, server-authentication EKU
+and an authority identifier linked to the CA. Do not clear
+`VERIFY_X509_STRICT`, set `verify=False`, disable hostname checks or change
+upstream trust to work around certificate errors.
+
+Infra must regenerate a deployment CA that lacks required extensions using
+the corrected released binary, validate/distribute the new public CA to every
+client bundle, and select the matching signer/key before acceptance. Updating
+the image cannot add extensions to an installed certificate. CA generation
+refuses overwrites; use the [CA rotation procedure](deployment.md#ca-commands-and-rotation).
+Keep cutover guarded until strict urllib and HTTPX requests succeed through
+the actual gateway and the other isolation/admission/backup checks still pass.
+
 Set HTTP_PROXY/HTTPS_PROXY and lowercase equivalents to the exclusive listener,
 for example `http://charon:18080`, without proxy authentication. Set Telegram's
 explicit `TELEGRAM_PROXY` to that same URL where the Hermes adapter needs it.
@@ -81,13 +97,22 @@ just the new feature:
 ```sh
 mise exec -- cargo test gateway --lib
 mise exec -- cargo run -- gateway validate examples/hermes-gateway.toml
+mise run strict-tls-check
 ```
 
 The client fixture requires curl and, on Linux, gh. Linux CI exercises both
 unmodified clients. On macOS it exercises curl only: existing Go/gh builds use
 Keychain trust rather than the fixture's `SSL_CERT_FILE`. Deployment verification
 must still prove gh with the installed CA; tests do not change system trust.
 
+`strict-tls-check` selects Python 3.13.5 in a disposable venv with HTTPX 0.28.1
+and runs a real local intercepted CONNECT request against a generated CA and
+verified TLS fixture origin. It keeps Python's default strict flags, required
+certificate verification and hostname checks; checks trusted urllib/HTTPX,
+reuse, denied operations, untrusted CA and wrong hostname; and proves zero
+provider lookups. Linux CI runs the same required test. The normal Rust suite
+also checks the emitted CA/leaf roles, critical extensions and identifier link.
+
 The fixture upstream uses a separate synthetic CA and verified TLS; its local
 routing override is confined to tests. Runtime has no private-address exception
 or custom trust override. Verify the candidate deployment with synthetic

docs/security-review-workload-gateway.md

@@ -48,6 +48,17 @@ The complete Hermes check also runs against the pinned upstream plugin API.
 
 ## Residual risks and release scope
 
+The strict-certificate follow-up reproduces both missing leaf Authority Key
+Identifier and missing CA Key Usage with Python 3.13.5 before repairing the
+complete generated chain. Required local and Linux CI tests use urllib and
+HTTPX 0.28.1 with default strict verification, required certificates and
+hostname checking. They check successful interception, reuse/policy denial,
+untrusted CA/hostname rejection and zero provider lookups; the Rust suite
+checks actual emitted CA/leaf extensions and the AKI/SKI link. Trust boundaries,
+policy, mediation and upstream validation are unchanged. Infra must regenerate
+an installed CA lacking required extensions and distribute its public trust;
+no automatic CA replacement or live deployment is part of this repair.
+
 No local test can attest Infra's network policy or CA distribution. The
 listener is unsafe when another principal can reach it; TLS interception places
 all permitted traffic inside Charon's trusted boundary. Caller-owned token

mise.toml

@@ -8,7 +8,11 @@ disable_tools = []
 
 [tasks.check]
 description = "Run the complete local quality gate"
-depends = ["fmt-check", "clippy", "test", "deploy-test", "deny", "hermes-check"]
+depends = ["fmt-check", "clippy", "test", "deploy-test", "deny", "hermes-check", "strict-tls-check"]
+
+[tasks.strict-tls-check]
+description = "Prove generated CONNECT chains with Hermes Python 3.13.5 and HTTPX 0.28.1"
+run = "sh tests/strict_tls.sh"
 
 [tasks.hermes-check]
 description = "Test the first-party Hermes permission and receipt integration"

src/ca.rs

@@ -5,8 +5,10 @@ use std::{fs::OpenOptions, io::Write as _, path::Path};
 use anyhow::{Context, Result, bail};
 use rcgen::{
     BasicConstraints, CertificateParams, CertifiedIssuer, DistinguishedName, DnType, IsCa, KeyPair,
+    KeyUsagePurpose,
 };
 use rustls::pki_types::{CertificateDer, pem::PemObject as _};
+use secrecy::{ExposeSecret as _, SecretString};
 use sha2::{Digest as _, Sha256};
 
 use crate::{config::TlsConfig, tls::TlsAuthority};
@@ -21,14 +23,16 @@ pub fn generate(name: &str, certificate: &Path, private_key: &Path) -> Result<()
         bail!("CA name is invalid");
     }
     let key = KeyPair::generate().context("failed to generate CA key")?;
-    let key_pem = key.serialize_pem();
+    let key_pem = SecretString::from(key.serialize_pem());
     let mut params = CertificateParams::new(Vec::<String>::new())?;
     params.is_ca = IsCa::Ca(BasicConstraints::Unconstrained);
+    params.key_usages = vec![KeyUsagePurpose::KeyCertSign, KeyUsagePurpose::CrlSign];
+    params.use_authority_key_identifier_extension = true;
     let mut distinguished_name = DistinguishedName::new();
     distinguished_name.push(DnType::CommonName, name);
     params.distinguished_name = distinguished_name;
     let issuer = CertifiedIssuer::self_signed(params, key).context("failed to sign CA")?;
-    write_new(private_key, key_pem.as_bytes(), true)?;
+    write_new(private_key, key_pem.expose_secret().as_bytes(), true)?;
     if let Err(error) = write_new(certificate, issuer.pem().as_bytes(), false) {
         let _ = std::fs::remove_file(private_key);
         return Err(error);

src/gateway_tests.rs

@@ -721,6 +721,111 @@ fn executable(name: &str) -> Option<std::path::PathBuf> {
             .find(|p| p.is_file())
     })
 }
+
+#[tokio::test]
+#[ignore = "requires Hermes Python 3.13.5/HTTPX 0.28.1; mise run strict-tls-check"]
+async fn python_313_strict_clients_accept_generated_chain() -> Result<()> {
+    let python = std::env::var_os("CHARON_STRICT_TLS_PYTHON")
+        .context("strict Python client interpreter required")?;
+    let f = fixture().await?;
+    let output = timeout(
+        Duration::from_secs(60),
+        tokio::process::Command::new(python)
+            .env_clear()
+            .env("PYTHONDONTWRITEBYTECODE", "1")
+            .kill_on_drop(true)
+            .arg(concat!(
+                env!("CARGO_MANIFEST_DIR"),
+                "/tests/strict_tls_clients.py"
+            ))
+            .arg(format!("http://{}", f.address))
+            .arg(&f.gateway.config.tls.ca_certificate)
+            .output(),
+    )
+    .await??;
+    // This isolated script uses only synthetic hostnames/data; no ambient auth.
+    ensure!(
+        output.status.success(),
+        "strict TLS client proof failed: {}",
+        String::from_utf8_lossy(&output.stderr)
+    );
+    ensure!(
+        String::from_utf8(output.stdout)?.contains("PASS:"),
+        "client proof missing"
+    );
+    assert_eq!(f.provider.0.load(Ordering::SeqCst), 0);
+    Ok(())
+}
+
+#[tokio::test]
+async fn intercepted_chain_has_explicit_ca_and_server_certificate_roles() -> Result<()> {
+    use x509_parser::extensions::{GeneralName, ParsedExtension};
+    let f = fixture().await?;
+    let stream = tunnel(&f, "allowed.test", vec![b"http/1.1".to_vec()]).await?;
+    let chain = stream
+        .get_ref()
+        .1
+        .peer_certificates()
+        .context("chain missing")?;
+    assert_eq!(chain.len(), 2);
+    let (_, leaf) = x509_parser::parse_x509_certificate(chain[0].as_ref())?;
+    let (_, ca) = x509_parser::parse_x509_certificate(chain[1].as_ref())?;
+    // Duplicate extensions and ambiguous certificate roles must not be emitted.
+    leaf.extensions_map()?;
+    ca.extensions_map()?;
+    let ca_constraints = ca.basic_constraints()?.context("CA constraints missing")?;
+    assert!(ca_constraints.critical && ca_constraints.value.ca);
+    let ca_usage = ca.key_usage()?.context("CA key usage missing")?;
+    assert!(ca_usage.critical && ca_usage.value.key_cert_sign() && ca_usage.value.crl_sign());
+    let leaf_constraints = leaf
+        .basic_constraints()?
+        .context("leaf constraints missing")?;
+    assert!(leaf_constraints.critical && !leaf_constraints.value.ca);
+    let leaf_usage = leaf.key_usage()?.context("leaf key usage missing")?;
+    assert!(leaf_usage.critical && leaf_usage.value.digital_signature());
+    assert!(!leaf_usage.value.key_cert_sign() && !leaf_usage.value.crl_sign());
+    let eku = leaf.extended_key_usage()?.context("server EKU missing")?;
+    assert!(eku.value.server_auth && !eku.value.client_auth && !eku.value.any);
+    let san = leaf
+        .subject_alternative_name()?
+        .context("DNS SAN missing")?;
+    assert_eq!(
+        san.value.general_names,
+        vec![GeneralName::DNSName("allowed.test")]
+    );
+    let subject_id = |certificate: &x509_parser::certificate::X509Certificate<'_>| {
+        certificate
+            .extensions()
+            .iter()
+            .find_map(|extension| match extension.parsed_extension() {
+                ParsedExtension::SubjectKeyIdentifier(id) if !extension.critical => {
+                    Some(id.0.to_vec())
+                }
+                _ => None,
+            })
+    };
+    let authority_id = |certificate: &x509_parser::certificate::X509Certificate<'_>| {
+        certificate
+            .extensions()
+            .iter()
+            .find_map(|extension| match extension.parsed_extension() {
+                ParsedExtension::AuthorityKeyIdentifier(id) if !extension.critical => {
+                    id.key_identifier.as_ref().map(|key| key.0.to_vec())
+                }
+                _ => None,
+            })
+    };
+    let ca_id = subject_id(&ca).context("CA subject key ID missing")?;
+    assert_ne!(ca_id, [] as [u8; 0]);
+    assert_eq!(authority_id(&leaf), Some(ca_id.clone()));
+    assert_eq!(authority_id(&ca), Some(ca_id));
+    assert_ne!(
+        subject_id(&leaf).context("leaf subject key ID missing")?,
+        [] as [u8; 0]
+    );
+    assert_eq!(f.provider.0.load(Ordering::SeqCst), 0);
+    Ok(())
+}
 #[tokio::test]
 async fn unmodified_curl_and_gh_use_standard_proxy_and_credential_settings() -> Result<()> {
     let curl = executable("curl").context("curl required for ordinary client proof")?;

src/tls.rs

@@ -3,7 +3,7 @@
 use std::sync::Arc;
 
 use anyhow::{Context, Result, bail};
-use rcgen::{CertificateParams, Issuer, KeyPair};
+use rcgen::{CertificateParams, ExtendedKeyUsagePurpose, IsCa, Issuer, KeyPair, KeyUsagePurpose};
 use rustls::{
     ServerConfig,
     pki_types::{CertificateDer, PrivateKeyDer, PrivatePkcs8KeyDer, pem::PemObject as _},
@@ -59,8 +59,15 @@ impl TlsAuthority {
         let issuer = Issuer::from_ca_cert_pem(&self.certificate_pem, &self.private_key)
             .context("TLS CA certificate became invalid")?;
         let leaf_key = KeyPair::generate().context("failed to generate TLS leaf key")?;
-        let leaf = CertificateParams::new(vec![hostname.to_owned()])
-            .context("TLS leaf hostname is invalid")?
+        let mut params = CertificateParams::new(vec![hostname.to_owned()])
+            .context("TLS leaf hostname is invalid")?;
+        params.use_authority_key_identifier_extension = true;
+        // rcgen emits critical Basic Constraints and a Subject Key Identifier
+        // for ExplicitNoCa; AKI links this leaf to the loaded issuer's SKI.
+        params.is_ca = IsCa::ExplicitNoCa;
+        params.key_usages = vec![KeyUsagePurpose::DigitalSignature];
+        params.extended_key_usages = vec![ExtendedKeyUsagePurpose::ServerAuth];
+        let leaf = params
             .signed_by(&leaf_key, &issuer)
             .context("failed to sign TLS leaf certificate")?;
         let private_key = PrivateKeyDer::Pkcs8(PrivatePkcs8KeyDer::from(leaf_key.serialize_der()));

tests/strict_tls.sh

@@ -0,0 +1,12 @@
+#!/bin/sh
+set -eu
+
+# A separate interpreter preserves the Hermes admission package's tool pin.
+# The venv holds test dependencies only; all synthetic TLS keys stay in the
+# Rust fixture's disposable tempfile and are never emitted by this script.
+strict_venv=$(mktemp -d "${TMPDIR:-/tmp}/charon-strict-python.XXXXXX")
+trap 'rm -rf "$strict_venv"' EXIT HUP INT TERM
+mise exec python@3.13.5 -- python -m venv "$strict_venv"
+"$strict_venv/bin/python" -m pip --disable-pip-version-check install --quiet httpx==0.28.1
+CHARON_STRICT_TLS_PYTHON="$strict_venv/bin/python" cargo test --lib \
+    gateway::tests::python_313_strict_clients_accept_generated_chain -- --ignored --nocapture

tests/strict_tls_clients.py

@@ -0,0 +1,64 @@
+"""Hermes-matched clients against the real generated-CA CONNECT fixture."""
+
+import socket
+import ssl
+import sys
+import urllib.request
+
+import httpx
+
+
+assert sys.version_info[:3] == (3, 13, 5), "requires Hermes Python 3.13.5"
+assert httpx.__version__ == "0.28.1", "requires Hermes HTTPX 0.28.1"
+proxy, ca_file = sys.argv[1:]
+context = ssl.create_default_context(cafile=ca_file)
+# Python's default is TLS 1.2; state that floor explicitly for the raw probe
+# and static analysis. Verification flags and hostname checking stay intact.
+context.minimum_version = ssl.TLSVersion.TLSv1_2
+assert context.verify_flags & ssl.VERIFY_X509_STRICT
+assert context.verify_mode == ssl.CERT_REQUIRED and context.check_hostname
+url = "https://allowed.test/plain"
+
+opener = urllib.request.build_opener(
+    urllib.request.ProxyHandler({"https": proxy}),
+    urllib.request.HTTPSHandler(context=context),
+)
+with opener.open(url, timeout=10) as response:
+    assert response.status == 200
+    assert b"data: /plain" in response.read()
+
+# Match the cutover reproduction: HTTPX constructs its own default strict
+# context from the CA filename. Do not change verify flags or hostname checks.
+response = httpx.get(url, proxy=proxy, verify=ca_file, timeout=10, trust_env=False)
+assert response.status_code == 200 and "data: /plain" in response.text
+with httpx.Client(proxy=proxy, verify=context, timeout=10, trust_env=False) as client:
+    assert client.get(url).status_code == 200
+    assert client.get("https://allowed.test/not-granted").status_code == 403
+    assert client.get(url).status_code == 200
+
+try:
+    httpx.get(url, proxy=proxy, timeout=10, trust_env=False)
+except httpx.ConnectError as error:
+    assert "CERTIFICATE_VERIFY_FAILED" in str(error)
+else:
+    raise AssertionError("untrusted interception CA was accepted")
+
+host, port = proxy.removeprefix("http://").rsplit(":", 1)
+with socket.create_connection((host, int(port)), timeout=10) as tunnel:
+    tunnel.sendall(b"CONNECT allowed.test:443 HTTP/1.1\r\nHost: allowed.test:443\r\n\r\n")
+    headers = b""
+    while not headers.endswith(b"\r\n\r\n"):
+        chunk = tunnel.recv(1)
+        assert chunk and len(headers) < 4096
+        headers += chunk
+    assert headers.startswith(b"HTTP/1.1 200")
+    try:
+        context.wrap_socket(tunnel, server_hostname="wrong.test")
+    except ssl.SSLCertVerificationError:
+        pass
+    else:
+        raise AssertionError("hostname checking was bypassed")
+
+assert context.verify_flags & ssl.VERIFY_X509_STRICT
+assert context.verify_mode == ssl.CERT_REQUIRED and context.check_hostname
+print("PASS: Python 3.13.5 urllib and HTTPX 0.28.1 strict CONNECT trust, reuse and denials")