feat: driver Pkl module loader (driver:<name>) + side-car manifests#9
Merged
feat: driver Pkl module loader (driver:<name>) + side-car manifests#9
driver:<name>) + side-car manifests#9Conversation
Spec for resolving the `driver:hue`-style Pkl import scheme that's documented but unimplemented. Implements DR-7 from the C4 design plan. Side-car layout (one driver per directory under `<data-dir>/drivers/`), manifests amend a new `switchyard:driver` base module, drivers.toml retired in the same change.
… in internal/config While starting the implementation plan, an exploration of the current code surfaced that: - there is no `drivers.toml` loader to delete (supervisor is already fully Pkl-driven via `Manager.Apply` → `CarportManager.RegisterInstance`), - `internal/cli/cmd_mcp.go` doesn't construct an evaluator (it talks to a running daemon), - the driver registry is better placed in `internal/config/` than `internal/carport/` because binary-path resolution must happen before the supervisor is called. Spec updated to reflect actual file boundaries.
13 bite-sized tasks following TDD: Pkl base modules first, then driverModuleReader, registry, Manager.Apply changes, carport interface, daemon/CLI flags, e2e smoke, docs, and a one-shot operator hand-migration of the local install. Implements docs/design/specs/2026-05-02-driver-pkl-modules-design.md.
Verified empirically with Pkl 0.31.1:
1. amends + non-local class declarations is rejected. Switched
switchyard:driver to `open module` and manifests use `extends`.
`name`/`version` must be `const` so per-driver classes can
reference them in class-level defaults.
2. Lexical scoping pins `Instance.driverName = name` to the base
module's empty `name`. Moved the auto-derivation to per-driver
instance classes (`class HueInstance extends Instance {
driverName = name; ... }`) — one line of boilerplate per driver,
consumer code unchanged.
3. Concern 1 fix: lifecycleDefaults uses LifecycleOverride
(all-nullable) instead of LifecycleConfig. Pkl JSON renderer
omits null fields, so unset entries cleanly fall through to the
Go-side carport.DefaultLifecycleConfig() — symmetric three-layer
merge.
- DriverInstance gains `enabled: Boolean = true` and `lifecycle: LifecycleOverride? = null`; binary moves to per-driver manifest in a follow-on task. - LifecycleOverride is all-nullable so unset fields fall through to Go-side carport.DefaultLifecycleConfig() — symmetric merge with the per-driver and per-instance override layers. - No Pkl-side LifecycleConfig: concrete defaults live exactly once, in Go. Per docs/design/specs/2026-05-02-driver-pkl-modules-design.md.
Provides the typed surface for driver manifest.pkl files (open module that manifests `extends`) and the abstract `Instance` class each driver's instance subclass extends. Per-driver classes add `driverName = name` themselves (Pkl lexical scoping forbids inheriting that default from the base). No Go consumers yet — Task 3 wires the driver: URI reader.
Reads <root>/<name>/manifest.pkl. validDriverName rejects names that contain path components, uppercase letters, dots, spaces, or exceed 64 chars — keeping arbitrary paths out of driver: URIs. Not yet registered on the Pkl evaluator; Task 4 plumbs driversRoot.
- newPklEvaluator(ctx, driversRoot) registers driverModuleReader alongside switchyardModuleReader. - ValidateOffline + NewManager + daemon.Config gain driversRoot. - switchyardd grows --drivers-dir; daemon defaults to <data-dir>/drivers. - switchyard config validate grows --drivers-dir; defaults to <data-dir>/drivers. - Daemon emits a one-shot warn-level log on startup if a leftover drivers.toml is present so operators notice it's no longer read. driver: imports won't actually do anything useful until the registry (Task 5) wires the per-instance binary/lifecycle resolution; this commit just makes the reader available.
DriverRegistry scans <root>/<name>/manifest.pkl, evaluates each through Pkl, validates that the directory name matches the manifest's `name` field, and resolves the binary path (default <dir>/<name>-driver, optional `binary` override absolute or relative). LifecycleOverride is a single Go type used for both manifest defaults and per-instance overrides — symmetric merge layers, single decode path. switchyard:driver gains a JsonRenderer with the same Duration converter as switchyard:config so EvaluateOutputText returns parseable JSON.
Drops the per-instance Binary field from the evaluator's parse path (no longer rendered by Pkl after Task 1). Adds parseInstanceOptions which decodes enabled and the LifecycleOverride out of the raw Params JSON — Manager.Apply will consume this in Task 7.
- CarportManager.RegisterInstance gains enabled + lifecycle params. - *carport.Host implements the new interface; old call sites (RegisterInstanceWithLifecycle, dynamic_test.go) updated to forward through the new signature. - carport.defaultLifecycleConfig renamed to DefaultLifecycleConfig (exported so config.MergeLifecycle can seed the merge bottom). - config.MergeLifecycle: defaults ← manifest ← per-instance override using the symmetric LifecycleOverride shape at both layers. - Manager.NewManager now builds the DriverRegistry at construction. - Manager.registerInstance: lookup driver entry, parse instance options, skip on enabled=false, merge lifecycle, forward to carport. - New integration tests under //go:build integration cover the four paths (resolution, override, disabled, missing).
Drives the full stack: hand-rolled fake driver in --drivers-dir, main.pkl that imports driver:fake, validate succeeds with the "Config valid" output line. Catches regressions in the joined evaluator + driverModuleReader + driverRegistry path.
- configuration/drivers.md: drop the C4-deferred caveat, document the <data-dir>/drivers/<name>/ layout, the --drivers-dir flag, the new enabled and lifecycle fields. Remove driverName from examples — auto-derived now. - drivers/building/manifest.md: replace the speculative manifest shape with the real one. extends "switchyard:driver", const name/version, the driverName = name boilerplate, and the lifecycleDefaults overlay. Explain the Pkl semantic constraints (extends + const) so authors aren't surprised.
fdatoo
commented
May 2, 2026
| Two Pkl semantics constraints shape the manifest's outer form: | ||
|
|
||
| --- | ||
| - **`open module` + `extends`.** The manifest needs to declare its own classes (like `HueInstance`). Pkl rejects non-local class declarations under `amends` — `Class needs a 'local' modifier. To define a non-local class, extend rather than amend the parent module (which must be 'open' for extension)`. So `switchyard:driver` is `open module` and manifests use `extends`. |
Owner
Author
There was a problem hiding this comment.
Including the error message from Pkl's interpreter is unnecessary.
Owner
Author
There was a problem hiding this comment.
Removed the verbatim error quote in ac700b6. Prose stands without it.
| produces = new { "light"; "scene"; "sensor" } | ||
| driverEventTypes = new { "bridge_unavailable"; "bridge_reconnected" } | ||
| lifecycleDefaults { | ||
| handshakeDeadline = 30.s // CLIP v2 first-time discovery is slow |
Owner
Author
There was a problem hiding this comment.
CLIP v2 is specific to the Hue driver, but this doc is for all drivers' manifests. Remove the comment. Also, is this section supposed to be a full manifest example? If so, why remove the heading?
Owner
Author
There was a problem hiding this comment.
Both fixed in ac700b6:
- Dropped the CLIP v2 comment.
- The lifecycle snippet here was meant as a partial example showing just the
lifecycleDefaultsblock, not a full manifest — the missing heading was a giveaway. Renamed the section near the top of the page from "Hue example" to "Full manifest example" with a sentence calling out that it's the canonical complete reference, so the per-section snippets later on are clearly partial.
- Drop the verbatim Pkl error quote in the "Why extends" section; the surrounding prose stands without it. - Drop the Hue-specific "CLIP v2 first-time discovery is slow" comment from the lifecycleDefaults snippet (this page is generic to all drivers). - Rename "Hue example" → "Full manifest example" so its role as the canonical complete reference is explicit; the per-section snippets later on are intentionally partial.
CI on busy macOS race-mode runners observed ok=3 (PR #9 first run); local 5x reproduction passes. The fundamental MODE_SINGLE check (skipped >> ok, ok+skipped == n) still validates the semantics — the upper bound was previously tight enough to flake under load.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
driver:Pkl URI scheme —import "driver:hue"resolves to<data-dir>/drivers/hue/manifest.pkl. Each driver lives in its own subdir under a configurable drivers root (--drivers-dir, defaults to<data-dir>/drivers/).switchyard:driverbase Pkl module that manifestsextends. Manifests declareconst name/const version/produces/optionalbinaryoverride +lifecycleDefaults: LifecycleOverride, plus a per-driverInstancesubclass operators reference (new hue.HueInstance { … }).switchyard:carport.DriverInstancedrops the per-instancebinaryfield and gainsenabled: Boolean = trueandlifecycle: LifecycleOverride? = null. Binary path is now resolved server-side inManager.Applyfrom the driver registry; lifecycle is merged through three layers (Go defaults ← manifest defaults ← per-instance override) using a single all-nullable shape so unset fields fall through cleanly.CarportManager.RegisterInstanceextended withenabled+lifecycleparameters;*carport.Hostupdated;RegisterInstanceWithLifecyclebecomes a thin wrapper for tests.<data-dir>/drivers.toml(no code path reads it).docs/design/specs/2026-04-22-c4-pkl-config-design.md).Spec + plan committed under
docs/design/{specs,plans}/2026-05-02-driver-pkl-modules*. Spec was corrected mid-flight after empirical Pkl verification turned up two semantic constraints (open module+extendsrequired for in-module class declarations;const name/versionrequired so per-driver classes can reference them in defaults).Test plan
go test ./...(unit) — all greengo test ./... -tags integration— all green, including the four newTestManagerApply_*cases (binary resolution, per-instance lifecycle override,enabled = false, missing-driver error) and the new e2eTestConfigValidate_OfflineWithDriverImportgo build -o dist/switchyardd ./cmd/switchyardd— builds clean~/.local/share/switchyard/drivers/hue/{hue-driver,manifest.pkl}+ updated~/.local/share/switchyard/config/main.pkl; daemon starts cleanly, spawns hue-driver, reacheskind=started detail=0.1.0(manifest version), shuts down gracefully🤖 Generated with Claude Code