F-157: Widget pack install — OCI pull + cosign verification#10
Open
F-157: Widget pack install — OCI pull + cosign verification#10
Conversation
Design for implementing OCI pull + cosign verification for widget pack install (replacing the metadata-registration stub in internal/widgetpack/install.go), exposed via a new admin-authz'd WidgetPackService Connect-RPC and the existing CLI scaffolding. Covers full C10 spec §15.4 install flow, §15.2 manifest schema extension, §15.7 runtime serving. Lands independently of F-156. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Document that F-157's procedure-catalog entries are inert until F-184 wires a ProcedureCatalog implementation into the daemon's authz interceptor. F-185 tracks populating entries for the rest of the API. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
15 tasks decomposing the F-157 design into TDD-ordered units: Pkl schema, proto, Store persistence, cosign verifier, OCI fetcher, manifest validation, bundle handler, install/uninstall flow, RPC service, procedure-catalog registrar (inert until F-184), daemon+listener wiring, CLI replacement, end-to-end integration test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ets.pkl - Rewrites widgets.pkl with the complete §15.2 PackManifest schema (name, version, protocol, sdkVersion, bundle, bundleHash, classes, description?, homepage?, license?) with appropriate constraints. - Relocates Position, Grid, and abstract WidgetInstance from dashboards.pkl into widgets.pkl so pack authors can extend WidgetInstance without importing dashboards.pkl. - Updates dashboards.pkl to import switchyard:widgets and reference widgets.WidgetInstance, widgets.Grid, widgets.Position throughout. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…-export Architecture table contradicted §4.1's explicit decision to keep ContainerWidget in dashboards.pkl. Pack authors don't author containers in v1.0 (only the builtin GroupCard is a container).
- Adds `import "switchyard:widgets" as widgets` and
`widgetPackPolicy: widgets.PackPolicy = new {}` to config.pkl
(the top-level config module, where all aggregated fields live).
- Adds `WidgetPackPolicy` proto message with `allowed_signers` and
`allow_unsigned` fields; adds `widget_pack_policy = 18` to
`ConfigSnapshot`.
- Adds `widgetPackPolicyJSON` struct and decodes the field into
`snap.WidgetPackPolicy` in `parseConfigJSON`.
- Regenerates `gen/switchyard/config/v1/snapshot.pb.go` via `task proto`.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Define WidgetPackService with Install/List/Uninstall/Watch RPCs and all associated message types (InstalledPack, UninstalledPack, WidgetPackEvent). Reuses the existing SignatureStatus enum from dashboard.proto (same package). Regenerate Go + Connect bindings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Task 3 implementation reused the existing SignatureStatus enum from dashboard.proto rather than declaring a duplicate. The existing enum has SIGNATURE_EXPIRED which the original spec didn't map to a Connect error code; add that to §5.2.
Replaces the in-memory Store stub with a fully-persistent implementation: .registry.json written atomically on every Add/Remove, stale-entry pruning on Load, multi-version keying (name@version), and non-blocking Subscribe fan-out for install/uninstall WatchEvents. Also updates install_test.go call sites to supply the required root arg. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
… stale log - Remove now restores the in-memory entry if persistLocked fails (mirror of Add) - Add no longer leaks the store's *InstalledPack into WatchEvent — a subscriber mutating the event payload could silently corrupt live state - Load logs a warning when pruning stale registry entries (spec §6.4)
Replaces the placeholder string-switch TrustPolicy.Verify with a real sigstore-go-backed verifier. The new surface (Verifier, TrustPolicy, VerificationResult) is what later F-157 tasks (install flow, daemon wiring, integration test) will consume. - TrustPolicy now stores allowed-signer globs and an AllowUnsigned flag behind an RWMutex; Set replaces both atomically. Glob matching uses path.Match against the cert SAN URI. - NewVerifier accepts an injectable root.TrustedMaterial. Tests pass a ca.VirtualSigstore directly; production will use NewProductionVerifier (currently stubbed pending TUF wiring). - Verify decodes a sigstore JSON bundle and runs sigstore-go's *verify.Verifier with WithTransparencyLog(1) + WithObserverTimestamps(1). Identity matching is done outside sigstore-go so we can apply our own glob policy against the SAN URI. testutil_test.go provides newTestTrustRoot + signBlobEntity, shared with the upcoming Task 15 OCI integration test. Unit tests feed entities to the package-internal verifyEntity hook to exercise the full sigstore-go pipeline (cert chain + Rekor + RFC3161 timestamp) without serialising to JSON; the JSON path is covered by a garbage-bundle reject test plus the forthcoming Task 15 integration test. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…hecks - TrustPolicy.Set returns an error if any signer pattern is malformed (was: silently failing to match at verify time, looking like 'no signers') - Verify godoc tightened: pol must be non-nil, nil pol rejects signed bundles - NewVerifier returns an error on nil TrustedMaterial - Comment on the unused ctx parameter clarifying it's reserved for TUF refresh
Adds Fetcher type that pulls widget pack OCI artifacts plus their cosign signature artifacts (best-effort) into memory. Rejects multi-layer artifacts and wrong media types. Tarball extraction and signature verification are owned by Install (Task 9). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Wire retry.DefaultClient into auth.Client so 429/Retry-After are honored (modern registries rate-limit unauthenticated pulls aggressively) - go mod tidy: promote oras-go and image-spec to direct deps - Document OCI 1.1 Referrers signature gap as a known limitation - Add zero-layer manifest test case
…erty to widgets.pkl Add `SwitchyardSchemeReaderOption()` to `internal/config` so external packages can register the embedded switchyard: Pkl module reader without importing unexported internals. Add a top-level `manifest: PackManifest?` property to `widgets.pkl` so pack manifest.pkl files can amend the module and be rendered to JSON by the Pkl evaluator. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Add EvalManifest(ctx, path) which creates a fresh Pkl evaluator with the switchyard: scheme reader, renders the manifest as JSON, and decodes into a Manifest struct. Pkl constraints in PackManifest (protocol == "v1", bundleHash startsWith "sha256:", name not empty, classes not empty) act as the validation layer — constraint violations surface as evaluator errors. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Manifest sources come from extracted-from-tarball Pkl files (Task 9 install flow). The Pkl evaluator must be sandboxed accordingly. - Set RootDir to the manifest's directory (spec §6 step 4) - Drop WithOsEnv so manifests can't read host environment variables - Errcheck-clean ev.Close() - Better error when 'manifest' property is null (vs. misleading "missing required field: name") - Tighten test error handling and add coverage for null + optional fields Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Serves /widgets/<pack>/<version>/<file> with immutable Cache-Control, ETag from pack SHA256, correct Content-Type, 404 for unknown packs, 405 for non-GET/HEAD, 304 for If-None-Match, and two-layer path traversal defence (path.Clean + post-Join prefix check). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Rewrites Installer.Install to chain pull → verify → stage → manifest validate → hash → SDK check → collisions → atomic commit, with stable FailureReason tokens and per-(name@version) install-mutex serialization. Tarball extraction uses an Abs-prefix path-traversal defense. The previous stub installer's tests are removed; Task 15's integration test will exercise the real pipeline end-to-end. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add Installer.Uninstall (os.RemoveAll → store.Remove) with optional DashboardLister guard that blocks removal when pack classes are in use. Defaults to emptyDashboardLister (no-op) until F-156 lands. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Implements the four-method Connect-RPC handler (Install, List, Uninstall, Watch) together with proto-conversion helpers and full FailureReason→code error mapping; includes two unit tests for the List and Uninstall paths. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Construct the F-157 widget pack subsystem (Store, Fetcher, TrustPolicy, Installer, Service, BundleHandler) inside the daemon's Run flow, mount the WidgetPackService Connect handler on the API listener, and pass the bundle handler to listener.Deps.WidgetsHandler so /widgets/<pack>/... serves real bundles. The trust policy is initialised from the current Pkl ConfigSnapshot's WidgetPackPolicy and hot-reloads via cfgManager.OnApplied; bad signer patterns log a warning instead of crashing. NewProductionVerifier is currently stubbed (the production TUF root is not yet wired). The daemon tolerates a nil verifier: install.go's Step 2 treats it as "no verifier configured" and rejects signed packs with ReasonSignatureInvalid while still permitting the policy.AllowUnsigned path. This is the chosen v1 behaviour until the trust root lands. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace no-op RunE stubs with real Connect-RPC client calls to WidgetPackService; add --version/--force flags on uninstall and styled output via existing helpers. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
5 passing tests + 2 skipped (signed paths blocked on sigstore Bundle inclusion-proof construction; signed verification is unit-tested via verifyEntity in trust_test.go). Adds Fetcher.WithPlainHTTP option (test-only) so the in-process registry can serve plain HTTP. Acceptance: unsigned-rejected, unsigned-allowed (full happy path), hash-mismatch, class-collision-with-builtin, already-exists. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds Store.ClassesView() + PackView/PackClass snapshot types to widgetpack. Updates dashboardBackend to accept *widgetpack.Store and rebuild the catalog on each WidgetCatalog call, joining pack classes with builtins. Fixes F-157 acceptance criterion 5: catalog now reflects installed widget packs. Adds TestStore_ClassesView and TestDashboardBackend_WidgetCatalog_ReflectsStore. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Removes Installer.dataDir (set but never read) and its NewInstaller parameter.
Drops SetDashboardLister (unsynchronized write); moves dl into the constructor
with nil defaulting to emptyDashboardLister{}, matching the rest of the
constructor pattern and eliminating the race condition. Updates all callers.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes F-157.
Summary
Replaces
internal/widgetpack/install.go's metadata-registration stub with the full C10 spec §15.4 install flow: OCI pull (oras-go), cosign keyless verification (sigstore-go), tarball extraction with path-traversal defense, Pkl manifest evaluation, bundle SHA-256 verification, SDK compatibility check, class-collision check, atomic-rename commit, and event emission. Exposed via a new admin-authz-readyWidgetPackServiceConnect-RPC and the existingswitchyard widget {install,list,uninstall}CLI.Also extends
widgets.pklwith the §15.2PackManifestschema, addswidgetPackPolicyto the config snapshot (Pkl + proto + decoder), wireswidgetpack.Storeintodashboard.Catalogso installed pack classes surface inGetWidgetCatalog, and serves bundles at/widgets/<pack>/<version>/<file>with immutable cache headers + ETag.Architecture
32 commits, ~4.8k lines added across 40 files.
Spec:
docs/design/specs/2026-05-04-f157-widget-pack-install-design.mdPlan:
docs/design/plans/2026-05-04-f157-widget-pack-install.mdAcceptance criteria
go-containerregistry/pkg/registryin tests)/widgets/<pack>/<version>/<file>?h=<sha>)manifest.pklTest plan
go test ./... -race -count=1— all green except a pre-existing flake ininternal/mcp/resources/TestEntityWatch_CoalescesOnOverflowunrelated to this PR (passes 2/3 attempts)go vet ./...— cleango build ./...— cleanswitchyardd --help— daemon starts cleanly, logs the expectedwidget pack: production verifier unavailablewarning (TUF wiring deferred)switchyard widget --help— install/list/uninstall subcommands presentKnown limitations and tracked follow-ups
All deferred work is filed:
WidgetPackService.Watchfor catalog cache invalidationwidgets-lock.pklwidget updatecommand + install progress streamingwidget searchcommandProcedureCataloginto daemon authz interceptor (this PR's catalog registrar is inert until then)ProcedureCatalogfor all existing RPCsEvalManifestAllowedModulesto excludehttp(s)Other deferrals documented in code:
verifyEntity)NewProductionVerifieris stubbed (returns explanatory error); daemon tolerates and logsAuthz posture
WidgetPackServiceprocedures are registered but theProcedureCatalogis not wired into the daemon's authz interceptor (F-184 blocker; daemon passesnilcatalog atdaemon.go:408). Today every RPC is effectively unauthenticated, the same posture as every other write RPC in the codebase. Closing this gap is F-184/F-185.🤖 Generated with Claude Code