Publish the managed gateway's ports 80 and 443 only for HTTPS

deploy/install/README.md

@@ -65,8 +65,8 @@ This directory holds the Core/Web installer, the `oac` command and the node inst
 
 A default combined Docker installation adds two Compose services from one pinned image: `gateway` runs Caddy, and `installation` runs the packaged `oac domain-server`. The latter runs with the installing account's UID and its Docker socket access and calls the same locked apply implementation. Its only request surface is the private `ingress/api/api.sock`, with Core-key authentication and one typed domain action. Core and Web get no Docker socket, host process authority or writable installation configuration. Web gets only the private API socket directory, never Caddy's admin socket.
 
-- The gateway owns ports 80 and 443 and the initial Web port. Caddy issues and renews certificates and keeps its private data in `ingress/data`. `generated/Caddyfile` is derived from `config.json`, and apply reloads it through the private Caddy socket even when container inputs already match.
-- A domain change keeps the old entry point while it verifies a trusted certificate and an installation-specific response over HTTPS. Only then does it set `public_url` and call the common apply path. Failure restores the previous configuration and reports incomplete recovery; failed retries restore through the common apply even after a partial change.
+- The gateway publishes the initial Web port, and ports 80 and 443 only for HTTPS (`ingress_config.published`). Caddy issues and renews certificates and keeps its private data in `ingress/data`. `generated/Caddyfile` is derived from `config.json`, and apply reloads it through the private Caddy socket even when container inputs already match.
+- A domain change first checks that the hostname resolves and that no other program holds port 80 or 443; ports the gateway publishes while it runs as written are its own. It confirms a public URL change from the applied address, because it returns Core there before switching. It keeps the old entry point while the common apply renders the candidate address: the gateway publishes 80 and 443 and serves the candidate, and the verification needs a trusted certificate and an installation-specific response over HTTPS. Only then does it set `public_url` and apply again. Failure restores the previous configuration, including the gateway without 80 and 443 when HTTPS was off, and reports incomplete recovery; failed retries restore through the common apply even after a partial change.
 - The operation record keeps the last successfully applied public address. Apply and start update it after gateway verification and service health checks; generated files alone never prove that a new address is active. A successful apply reconciles the domain operation status after verifying the running services.
 - The domain operation refuses unrelated pending `config.json` edits and shares `.oac.lock` with the CLI. Its status file is bookkeeping and the recovery receipt; `config.json` stays the source of desired settings. An interrupted operation keeps its desired files and a visible failure and retry state; it never creates another service project or deletes execution data.
 - A Web restart ends console sessions, so the UI gives the new HTTPS sign-in address instead of treating a dropped request as success.

deploy/install/configuration.py

@@ -146,16 +146,18 @@ def service_address(config, service, connect=False):
 Listener = collections.namedtuple("Listener", "purpose host port setting")
 
 
-def listeners(config):
+def listeners(config, candidate=None):
     """Every host listener the services bind, from the addresses they are rendered with.
 
-    setting is the config.json key that owns the port.
+    setting is the config.json key that owns the port. candidate is the address domain
+    setup verifies, as in render.
     """
     mode, ports, result = config["mode"], config["ports"], []
     if ingress_config.enabled(config):
-        # The gateway publishes Web's port and the HTTPS ports; Web itself publishes none.
+        # The gateway publishes Web's port and, for HTTPS, 80 and 443; Web itself publishes none.
         result += [Listener("Web", config["host"], port, "ports.web") if port == ports["web"] else
-                   Listener("HTTPS", config["host"], port, "ingress") for port, _ in ingress_config.published(config)]
+                   Listener("HTTPS", config["host"], port, "public_url")
+                   for port, _ in ingress_config.published(config, candidate)]
     elif mode != "core-only":
         result.append(Listener("Web", service_host(config, "web"), ports["web"], "ports.web"))
     if mode != "web-only":
@@ -229,7 +231,7 @@ def settings_document(root, config, applied_at):
             "applied_at": applied_at, "settings": config_model.settings(config)}
 
 
-def compose_config(root, config, state):
+def compose_config(root, config, state, candidate=None):
     root = Path(root)
     mode, native = config["mode"], config.get("native_core", False)
     identity = f'{state["uid"]}:{state["gid"]}'
@@ -297,7 +299,7 @@ def compose_config(root, config, state):
             web["volumes"].append(bind(root / "ingress/api", "/installation"))
         services["web"] = web
     if ingress_config.enabled(config):
-        services.update(ingress_config.services(root, config, state, bind))
+        services.update(ingress_config.services(root, config, state, bind, candidate))
     return doc
 
 
@@ -327,7 +329,9 @@ def native_installer_inputs(root):
     return [sha256(catalog.read_bytes()), sorted(p.name for p in directory.glob("*.tar.gz") if p.is_file())]
 
 
-def render(root, config, state, applied_at):
+def render(root, config, state, applied_at, candidate=None):
+    """candidate is an HTTPS address domain setup verifies before public_url changes:
+    the gateway publishes 80 and 443 and serves it too."""
     root = Path(root)
     mode, native = config["mode"], config.get("native_core", False)
     secrets = secret_digests(root, mode)
@@ -354,15 +358,15 @@ def render(root, config, state, applied_at):
         }, sort_keys=True)
     if mode != "core-only":
         external["web"] = json.dumps({"core.key": secrets["core.key"]})
-    compose = compose_config(root, config, state)
+    compose = compose_config(root, config, state, candidate)
     services = {}
     for name, service in compose["services"].items():
         # Compose resolves env_file into the service configuration, so its content counts.
         text = json.dumps(service, sort_keys=True) + (core_env if "env_file" in service else "")
         services[name] = sha256(text + external.get("core" if name == "migrate" else name, ""))
         service["labels"] = {LABEL: services[name]}
     if ingress_config.enabled(config):
-        files["Caddyfile"] = ingress_config.caddyfile(config, state)
+        files["Caddyfile"] = ingress_config.caddyfile(config, state, candidate)
     files["compose.json"] = json.dumps(compose, indent=2) + "\n"
     if native:
         unit = native_service.unit_name(state)

deploy/install/ingress.py

@@ -6,6 +6,7 @@
 from http.server import BaseHTTPRequestHandler
 import json
 from pathlib import Path
+import socket
 import socketserver
 import ssl
 import threading
@@ -18,6 +19,10 @@
 import oac_cli
 
 
+FAILED = "Domain setup failed. Check gateway and installation logs, then retry."
+UNFINISHED = ("checking", "applying")
+
+
 class DomainError(oac_cli.OacError):
     def __init__(self, code, message, status=400):
         super().__init__(message)
@@ -28,6 +33,12 @@ def save(root, value):
     oac_cli.write_private(Path(root) / "ingress/status.json", json.dumps(value) + "\n")
 
 
+def unfinished(root):
+    """Whether status.json records a domain setup that has not finished."""
+    path = Path(root) / "ingress/status.json"
+    return path.exists() and json.loads(oac_cli.read_private(path, "domain setup status"))["state"] in UNFINISHED
+
+
 def status(root):
     config = oac_cli.load_config(root)
     supported = gateway.enabled(config)
@@ -40,14 +51,14 @@ def status(root):
         path = Path(root) / "ingress/status.json"
         if path.exists():
             result.update(json.loads(oac_cli.read_private(path, "domain setup status")))
-            if result["state"] in ("checking", "applying"):
+            if result["state"] in UNFINISHED:
                 try:
                     with oac_cli.locked(Path(root)):
                         # A live CLI or Web operation holds this same lock. Read
                         # again after acquiring it in case the operation just finished.
                         result.update(json.loads(oac_cli.read_private(path, "domain setup status")))
-                        if result["state"] in ("checking", "applying"):
-                            result.update(state="failed", message="Domain setup was interrupted. Retry the same hostname, or run oac apply and oac status on the server.")
+                        if result["state"] in UNFINISHED:
+                            result.update(state="failed", message="Domain setup was interrupted. Retry the same hostname, or run oac apply on the server: it restores the applied address, or finishes the switch if it had started.")
                 except oac_cli.OacError:
                     pass
     return result
@@ -84,17 +95,38 @@ def prepare(root, name, confirmation):
     rendered, disk, old_settings = oac_cli.render_now(root, config, state)
     if oac_cli.edited_files(state, disk, rendered):
         raise DomainError("generated_files_edited", "Resolve hand-edited generated files with oac apply before configuring the domain", 409)
-    old, base, answered = oac_cli.old_public_url(root, config, old_settings, disk, actual)
+    host = urlsplit(target).hostname
+    if not resolves(host):
+        raise DomainError("hostname_unresolved", f"{host} does not resolve. Add A/AAAA records for it that point at this server, then retry when DNS returns them.", 409)
+    # The gateway publishes 80 and 443 only for HTTPS; the ports it already publishes are its own.
+    busy = [str(listener.port) for listener in oac_cli.taken_listeners(candidate, oac_cli.own_listeners(config, old_settings, disk, actual))]
+    if busy:
+        named = f"Port {busy[0]} is" if len(busy) == 1 else "Ports " + " and ".join(busy) + " are"
+        raise DomainError("https_ports_unavailable", f"{named} already in use on this server. Automatic HTTPS cannot run beside another program on ports 80 and 443: free both and retry. Find the program with: sudo ss -ltnp '( sport = :80 or sport = :443 )'", 409)
+    _, base, answered = oac_cli.old_public_url(root, config, old_settings, disk, actual)
     args = argparse.Namespace(dry_run=False, yes=False, confirm_public_url_change=confirmation)
     try:
-        oac_cli.confirm_public_url(root, candidate, old, base, answered, args, False, lambda _: None)
+        # execute returns Core to the applied address first, even after an interrupted switch,
+        # so the change is confirmed from there.
+        oac_cli.confirm_public_url(root, candidate, configuration.local_public_url(config), base, answered, args, False, lambda _: None)
     except oac_cli.OacError:
         raise DomainError("public_url_confirmation_required", "Changing this address can disconnect existing nodes and executors. Confirm the new URL to continue; retain the old route while existing work uses it.", 409) from None
+    # The final apply makes the change confirmed here.
+    args = argparse.Namespace(dry_run=False, yes=False, confirm_public_url_change=target)
     job = {"state": "checking", "target_url": target, "public_url": config["public_url"], "message": None}
     save(root, job)
     return config, state, candidate, args, job
 
 
+def resolves(host):
+    """Whether DNS answers with an A or AAAA record."""
+    try:
+        # The trailing dot keeps the resolver from appending its search domains.
+        return bool(socket.getaddrinfo(host + ".", 443, type=socket.SOCK_STREAM))
+    except (OSError, UnicodeError):
+        return False
+
+
 def verify(target, installation_id, timeout=180):
     """A trusted certificate and this gateway's identity must both answer at the public address."""
     host = urlsplit(target).hostname
@@ -111,14 +143,21 @@ def verify(target, installation_id, timeout=180):
         except (OSError, http.client.HTTPException):
             pass
         if time.monotonic() >= deadline:
-            raise DomainError("https_not_ready", "HTTPS verification failed. Point this hostname's A/AAAA records at the server, allow inbound ports 80 and 443, and check gateway logs. The previous address is kept; retry after correcting DNS or connectivity.")
+            raise DomainError("https_not_ready", f"HTTPS verification failed: {host} did not reach this installation with a trusted certificate. Check that its A/AAAA records point at this server, that no firewall or NAT blocks inbound ports 80 and 443, and the gateway logs for certificate errors. The previous address is kept; retry after the fix.")
         time.sleep(2)
 
 
 def execute(root, prepared):
     config, state, candidate, args, job = prepared
+    # Core and Web keep the applied address until the candidate is verified.
+    keep = argparse.Namespace(dry_run=False, yes=False, confirm_public_url_change=configuration.local_public_url(config))
     try:
-        gateway.reload(root, gateway.caddyfile(config, state, candidate=candidate["public_url"]))
+        # Through common apply, the gateway publishes 80 and 443 and serves the candidate.
+        oac_cli.write_private(Path(root) / "config.json", json.dumps(config, indent=2) + "\n")
+        try:
+            oac_cli._apply(Path(root), keep, False, False, False, lambda _: None, candidate=candidate["public_url"])
+        except oac_cli.OacError:
+            raise DomainError("domain_setup_failed", FAILED) from None  # Its message addresses oac apply users.
         verify(candidate["public_url"], state["installation_id"])
         job["state"] = "applying"
         save(root, job)
@@ -131,16 +170,14 @@ def execute(root, prepared):
     except Exception as error:
         recovery_failed = False
         try:
-            # A retry may inherit partially applied services from an interrupted
-            # attempt. Restore through common apply even if this attempt only probed TLS.
+            # Restore through common apply: the gateway without the candidate, and without
+            # 80 and 443 unless HTTPS was already on, and any partially applied services.
             oac_cli.write_private(Path(root) / "config.json", json.dumps(config, indent=2) + "\n")
-            restore = argparse.Namespace(dry_run=False, yes=False,
-                                         confirm_public_url_change=configuration.local_public_url(config))
-            oac_cli._apply(Path(root), restore, False, False, False, lambda _: None)
+            oac_cli._apply(Path(root), keep, False, False, False, lambda _: None)
             gateway.reload(root, gateway.caddyfile(config, state))
         except Exception:
             recovery_failed = True
-        message = str(error) if isinstance(error, (DomainError, oac_cli.OacError)) else "Domain setup failed. Check gateway and installation logs, then retry."
+        message = str(error) if isinstance(error, (DomainError, oac_cli.OacError)) else FAILED
         if recovery_failed:
             message += " Recovery was incomplete; run oac apply and oac status on the server."
         job.update(state="failed", message=message)
@@ -232,14 +269,14 @@ def setup(self):
 
 def serve(root):
     root = Path(root)
-    socket = Path("/control/api.sock")
-    socket.unlink(missing_ok=True)
+    path = Path("/control/api.sock")
+    path.unlink(missing_ok=True)
     import os
     previous = os.umask(0o077)
     try:
-        with Server(str(socket), Handler) as server:
+        with Server(str(path), Handler) as server:
             server.root = root
-            os.chmod(socket, 0o600)
+            os.chmod(path, 0o600)
             server.serve_forever()
     finally:
         os.umask(previous)

deploy/install/ingress_config.py

@@ -107,12 +107,27 @@ def reload(root, document):
         raise RuntimeError("HTTPS gateway is unavailable; inspect gateway logs and retry") from None
 
 
-def published(config):
-    """(host port, gateway port) of each port the gateway publishes on config["host"]."""
-    return [(config["ports"]["web"], 8080), (80, 80), (443, 443)]
+def published(config, candidate=None):
+    """(host port, gateway port) of each port the gateway publishes on config["host"].
 
+    Automatic HTTPS needs ports 80 and 443, so the gateway publishes them only once
+    public_url is set, or while domain setup verifies a candidate address.
+    """
+    https = [(80, 80), (443, 443)] if config["public_url"] or candidate else []
+    return [(config["ports"]["web"], 8080)] + https
 
-def services(root, config, state, bind):
+
+def written_listeners(compose):
+    """(address, port) of each host port the gateway publishes in a written compose.json."""
+    try:
+        ports = json.loads(compose)["services"]["gateway"]["ports"]
+        return {(ipaddress.ip_address(host.strip("[]")), int(port))
+                for host, port, _ in (item.rsplit(":", 2) for item in ports)}
+    except (TypeError, ValueError, KeyError, AttributeError):
+        return set()
+
+
+def services(root, config, state, bind, candidate=None):
     root = Path(root)
     identity = f'{state["uid"]}:{state["gid"]}'
     common = {"image": state["images"]["ingress"], "user": identity, "restart": "unless-stopped",
@@ -121,7 +136,7 @@ def services(root, config, state, bind):
     if ":" in host:
         host = "[" + host + "]"
     gateway = dict(common, command=["caddy", "run", "--config", "/generated/Caddyfile", "--adapter", "caddyfile"],
-                   ports=[f"{host}:{port}:{target}" for port, target in published(config)],
+                   ports=[f"{host}:{port}:{target}" for port, target in published(config, candidate)],
                    environment={"XDG_DATA_HOME": "/data", "XDG_CONFIG_HOME": "/data/config",
                                 "NO_PROXY": "core,web,localhost,127.0.0.1,::1",
                                 "no_proxy": "core,web,localhost,127.0.0.1,::1"},

deploy/install/install_output.py

@@ -56,7 +56,7 @@ def summary(root, config, addresses, fresh, selection, deployment, reachable, in
         paragraph("Use this key for the Core management API. Keep it private.")
     heading("Next")
     if ingress_config.enabled(config) and not config["public_url"]:
-        paragraph("Open Web at the server IP and sign in. In System → Domain and HTTPS, enter your DNS hostname; the installation requests and renews its certificate. DNS must point to this server and ports 80 and 443 must be reachable.")
+        paragraph("Open Web at the server IP and sign in. In System → Domain and HTTPS, enter your DNS hostname; the installation requests and renews its certificate. HTTPS then uses ports 80 and 443: DNS must point to this server, no other program on it may use those ports, and they must be reachable from the internet. Web checks DNS and the ports before it starts.")
     if mode != "core-only":
         paragraph("Create a Project and its API key on the Projects and keys page.")
     else: