Run the emitter host from an installed microcosm-emitter command

README.md

@@ -13,6 +13,20 @@ Installing `microcosm-emitter` always installs the telemetry library and all its
 
 The build process imports only the adapter, not the service implementation. It supplies run metadata, its existing queue path, and build identity. The adapter starts a separate local telemetry emitter service process automatically and sends requests over a private Unix socket. The adapter does not track stages, construct events, or redact telemetry. Imports alone create no processes, sockets, files, or network requests.
 
+## Installing the emitter on its own
+
+`uv tool install microcosm-emitter` installs the `microcosm-emitter` command in its own environment, separately from the build's Python environment. A build then starts `microcosm-emitter host` as its own child process, exactly as the in-process launcher starts `python -m microcosm_emitter.host`:
+
+- the build passes its process ID with `--parent-pid` and the module configuration on standard input, never in process-list arguments;
+- the host inherits the build's environment variables and runs as the same user, so it finds the same Hugging Face credential (`HF_TOKEN`, then `HUGGINGFACE_TOKEN`, then the token saved by `hf auth login`);
+- the host exits after the build exits. Without `--parent-created-at`, the host reads the build's creation time itself, and refuses to start if the named process is not its parent.
+
+The host is never a separately started system service: such a service would not see the build's environment variables. `--skip-unavailable-modules` starts the host without requested modules that this emitter release does not install, so a newer build can still use an older emitter's modules.
+
+Only a host started by `microcosm-emitter host` answers a `hello` request on the socket. It returns the emitter version, the socket protocol versions it accepts, the modules it loaded and the requested modules it skipped (`microcosm_emitter.compatibility.HelloResponse`). A host started with `python -m microcosm_emitter.host`, as by the in-process launcher, refuses `hello` like any unknown action. `microcosm-emitter version` prints the emitter version and protocol versions as JSON. Every emitter release must keep accepting every protocol version that builds still send (`microcosm_emitter.compatibility`).
+
+With `--skip-unavailable-modules`, the host still refuses to start if none of the requested modules is installed, and refuses each request to a skipped module as it refuses one to an unknown module.
+
 ## Delivery and persistence
 
 The service interprets requests such as a stage change or build failure, timestamps and redacts the resulting events, and saves them before acknowledging the socket request. A stage change can complete one stage and start another; both events commit in one SQLAlchemy ORM transaction, and the service changes its stage state only after that transaction succeeds. Periodic retention maintenance runs outside this acknowledgement path.

changelog.d/installed-emitter-command.added.md

@@ -0,0 +1 @@
+Add the `microcosm-emitter` command, so `uv tool install microcosm-emitter` installs the emitter outside the build's Python environment. A build starts `microcosm-emitter host` as its own child process, which inherits the build's environment and user. The host can read its parent's creation time itself, and can skip requested modules that it does not install (it still refuses to start if none is installed). Only a host started by `microcosm-emitter host` answers `hello`, with the emitter version, accepted socket protocol versions and loaded and skipped modules; a host started with `python -m microcosm_emitter.host` refuses it.

pyproject.toml

@@ -21,6 +21,7 @@ telemetry = "microcosm_emitter.telemetry.service:create_module"
 orrery = "microcosm_emitter.orrery.service:create_module"
 
 [project.scripts]
+microcosm-emitter = "microcosm_emitter.cli:main"
 microcosm-emitter-publish-graph = "microcosm_emitter.orrery.cli:main"
 
 [build-system]

src/microcosm_emitter/cli.py

@@ -0,0 +1,54 @@
+"""The `microcosm-emitter` command installed with `uv tool install microcosm-emitter`.
+
+`microcosm-emitter host` runs the local service host. A build starts it as its
+own child process, so the host inherits the build's environment variables and
+user, including any Hugging Face credential. It is never a system service.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from importlib.metadata import version
+
+from microcosm_emitter.compatibility import (
+    SUPPORTED_PROTOCOL_VERSIONS,
+    EmitterDescription,
+)
+
+DISTRIBUTION_NAME = "microcosm-emitter"
+
+
+def describe() -> EmitterDescription:
+    """Versions a client checks before sending any hand-over."""
+    return {
+        "emitter_version": version(DISTRIBUTION_NAME),
+        "protocol_versions": list(SUPPORTED_PROTOCOL_VERSIONS),
+    }
+
+
+def main(argv: list[str] | None = None) -> int:
+    arguments = sys.argv[1:] if argv is None else list(argv)
+    parser = argparse.ArgumentParser(prog="microcosm-emitter", description=__doc__)
+    commands = parser.add_subparsers(dest="command", required=True)
+    commands.add_parser(
+        "host",
+        help="Run the local service host; its configuration arrives on stdin.",
+        add_help=False,
+    )
+    commands.add_parser(
+        "version", help="Print the emitter and socket protocol versions as JSON."
+    )
+    selected, _ = parser.parse_known_args(arguments[:1])
+    if selected.command == "host":
+        from microcosm_emitter.host.__main__ import main as host_main
+
+        return host_main(arguments[1:], description=describe())
+    parser.parse_args(arguments)
+    print(json.dumps(describe()))
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/microcosm_emitter/compatibility.py

@@ -0,0 +1,28 @@
+"""Socket protocol versions this emitter accepts, and how it describes itself.
+
+Builds start whichever emitter is installed, so a release must keep accepting
+every protocol version that the `microcosm-emitter-interface` releases still in
+use send. Remove a version only after no supported Microcosm release sends it.
+"""
+
+from typing import Final, TypedDict
+
+SUPPORTED_PROTOCOL_VERSIONS: Final = (1,)
+
+
+class EmitterDescription(TypedDict):
+    """What `microcosm-emitter version` prints."""
+
+    emitter_version: str
+    protocol_versions: list[int]
+
+
+class HelloResponse(EmitterDescription):
+    """What a host started by `microcosm-emitter host` answers to `hello`.
+
+    A host started with `python -m microcosm_emitter.host` has no description
+    and refuses `hello` like any unknown action.
+    """
+
+    modules: list[str]
+    skipped_modules: list[str]

src/microcosm_emitter/host/__main__.py

@@ -5,35 +5,77 @@
 import sys
 from pathlib import Path
 
+from microcosm_emitter.compatibility import EmitterDescription, HelloResponse
 from microcosm_emitter.host.constants import MAX_MESSAGE_BYTES
 from microcosm_emitter.host.contracts import ModuleContext
-from microcosm_emitter.host.errors import safe_startup_error
+from microcosm_emitter.host.errors import ModuleNotInstalledError, safe_startup_error
 from microcosm_emitter.host.loading import load_module
-from microcosm_emitter.host.process import ParentProcess
+from microcosm_emitter.host.process import parent_process
 from microcosm_emitter.host.runtime import ServiceRuntime
 
 
-def main() -> int:
-    parser = argparse.ArgumentParser()
+def main(
+    argv: list[str] | None = None,
+    *,
+    description: EmitterDescription | None = None,
+) -> int:
+    """Run the host.
+
+    With ``description`` (given only by `microcosm-emitter host`), the host
+    answers ``hello`` with it and the loaded and skipped modules. Without it,
+    as when started with `python -m microcosm_emitter.host`, ``hello`` is
+    refused like any unknown action.
+    """
+    parser = argparse.ArgumentParser(prog="microcosm-emitter host")
     parser.add_argument("--socket", type=Path, required=True)
     parser.add_argument("--parent-pid", type=int, required=True)
-    parser.add_argument("--parent-created-at", type=float, required=True)
-    args = parser.parse_args()
+    # Earlier launchers pass the parent's creation time. Without it, the host
+    # reads it from its own parent, which must be the process named here.
+    parser.add_argument("--parent-created-at", type=float)
+    parser.add_argument(
+        "--skip-unavailable-modules",
+        action="store_true",
+        help="Start without requested modules that this emitter does not install.",
+    )
+    args = parser.parse_args(argv)
     try:
         payload = sys.stdin.buffer.read(MAX_MESSAGE_BYTES + 1)
         if len(payload) > MAX_MESSAGE_BYTES:
             raise ValueError("configuration is too large")
         configurations = json.loads(payload)
         if not isinstance(configurations, dict) or not configurations:
             raise ValueError("module configurations must be a nonempty object")
-        parent = ParentProcess(args.parent_pid, args.parent_created_at)
+        parent = parent_process(args.parent_pid, args.parent_created_at)
         context = ModuleContext(args.parent_pid)
         modules = {}
+        skipped = []
         for name, settings in configurations.items():
             if not isinstance(name, str) or not isinstance(settings, dict):
                 raise ValueError("invalid module configuration")
-            modules[name] = load_module(name, settings, context)
-        ServiceRuntime(args.socket, modules=modules, is_parent_alive=parent.alive).run()
+            try:
+                modules[name] = load_module(name, settings, context)
+            except ModuleNotInstalledError:
+                if not args.skip_unavailable_modules:
+                    raise
+                skipped.append(name)
+        if not modules:
+            raise ValueError("no requested module is installed")
+        hello_response: HelloResponse | None = (
+            None
+            if description is None
+            else {
+                "emitter_version": description["emitter_version"],
+                "protocol_versions": list(description["protocol_versions"]),
+                "modules": sorted(modules),
+                "skipped_modules": sorted(skipped),
+            }
+        )
+        ServiceRuntime(
+            args.socket,
+            modules=modules,
+            is_parent_alive=parent.alive,
+            description=hello_response,
+        ).run()
         return 0
     except Exception as error:
         # Never include configuration or module exception payloads in stderr.

src/microcosm_emitter/host/errors.py

@@ -7,6 +7,10 @@ class LocalServiceError(RuntimeError):
     """A local service could not start, accept a message, or shut down."""
 
 
+class ModuleNotInstalledError(LocalServiceError):
+    """No installed entry point provides the requested module."""
+
+
 StartupFailureReason = Literal[
     "readiness_timeout",
     "configuration_timeout",

src/microcosm_emitter/host/loading.py

@@ -8,13 +8,15 @@
     ModuleContext,
     ServiceModule,
 )
-from microcosm_emitter.host.errors import LocalServiceError
+from microcosm_emitter.host.errors import LocalServiceError, ModuleNotInstalledError
 
 
 def load_module(
     name: str, configuration: JsonObject, context: ModuleContext
 ) -> ServiceModule:
     matches = list(entry_points(group=MODULE_GROUP, name=name))
+    if not matches:
+        raise ModuleNotInstalledError(f"No installed module is named {name!r}.")
     if len(matches) != 1:
         raise LocalServiceError(
             f"Expected exactly one installed module named {name!r}."

src/microcosm_emitter/host/process.py

@@ -1,5 +1,7 @@
 """Monitor a process identity, not just a reusable numeric PID."""
 
+import os
+
 import psutil
 
 
@@ -18,3 +20,23 @@ def alive(self) -> bool:
             )
         except (psutil.Error, OSError):
             return False
+
+
+def parent_process(pid: int, created_at: float | None) -> ParentProcess:
+    """Identify the build that owns this host.
+
+    A launcher may pass the creation time it measured. Otherwise the host reads
+    the creation time of the process named ``pid`` and then checks that this
+    process is still its parent. A parent that exits is replaced as parent by
+    another, still running process, so a match after the read proves the
+    creation time is the parent's, not that of an unrelated process that was
+    given the same process ID after the parent exited.
+    """
+    if created_at is not None:
+        return ParentProcess(pid, created_at)
+    if os.getppid() != pid:
+        raise ValueError("the named parent process is not this host's parent")
+    parent_created_at = psutil.Process(pid).create_time()
+    if os.getppid() != pid:
+        raise ValueError("the named parent process is not this host's parent")
+    return ParentProcess(pid, parent_created_at)

src/microcosm_emitter/host/runtime.py

@@ -7,6 +7,7 @@
 import time
 from collections.abc import Callable, Mapping
 from pathlib import Path
+from typing import Any
 
 from microcosm_emitter.host.constants import (
     ACK_ERROR,
@@ -37,8 +38,11 @@ def __init__(
         is_parent_alive: Callable[[], bool],
         tick_seconds: float = TICK_SECONDS,
         drain_seconds: float = DRAIN_SECONDS,
+        description: Mapping[str, Any] | None = None,
     ):
         self.socket_path = socket_path
+        # Returned to ``hello`` requests; the host does not interpret it.
+        self.description = None if description is None else dict(description)
         if (module is None) == (modules is None):
             raise ValueError("Select one module or an explicit module mapping.")
         self.modules = dict(modules) if modules is not None else {"default": module}
@@ -164,6 +168,13 @@ def _respond(self, connection: socket.socket) -> bytes | None:
                 ):
                     raise ValueError("module request queue is unavailable")
                 return None  # The module worker now owns this connection.
+            elif action == "hello" and self.description is not None:
+                response = (
+                    json.dumps(self.description, allow_nan=False).encode() + DELIMITER
+                )
+                if len(response) > MAX_MESSAGE_BYTES:
+                    raise ValueError("description too large")
+                return response
             elif action == "close":
                 self.stop()
             elif action != "ping":