diff --git a/dev-docs/adr/0050-a-component-call-is-a-transaction-over-a-snapshot.md b/dev-docs/adr/0050-a-component-call-is-a-transaction-over-a-snapshot.md new file mode 100644 index 00000000..9442b798 --- /dev/null +++ b/dev-docs/adr/0050-a-component-call-is-a-transaction-over-a-snapshot.md @@ -0,0 +1,54 @@ +# ADR-0050: A component call is a transaction over a snapshot, within the command's lifetime + +- **Date:** 2026-10-25 +- **Status:** Accepted + +## Context + +The same SDK module behaved differently depending on how it ran. A managed +plugin's request crosses the wire as a copy, and its result is applied only +when the RPC succeeds. An embedded matcher or analyzer received the host's own +graph, registry and target pointers, so one that changed them and then failed +-- the reachability analyzers enrich `req.Registry` in place -- left the +pipeline half-enriched. Lifetimes diverged too: embedded constructors ran +under `context.Background()` with no way to release what they acquired, and a +managed component was torn down only as a side effect of killing its process. + +## Decision + +- Every `Ready`, `Applicable` and `Match`/`Analyze` call receives its own + `MatchRequest.Clone` / `AnalyzeRequest.Clone`. No component holds the host's + state. +- Only a successful action is committed: a returned `Registry` wins, otherwise + `PackageUpdates` merge through `model.ApplyPackageUpdates`. A failed call, or + one that returns after the command was cancelled, commits nothing. Rollback + is per call; earlier successes stand. +- Changes a component makes to the graph or target are never committed, as + they never were for a managed plugin. +- An embedded component is constructed under the command's context and gets a + `plugin.ComponentScope`. `Registry.Close` ends every scope, newest first, in + the command's cleanup chain ahead of temp-dir cleanup. A managed component's + scope closes when go-plugin's `Kill` shuts the plugin down gracefully, and the + pool shuts plugins down concurrently, so the command waits one grace period. + +## Rejected + +Letting embedded components keep mutating the request, documented as a +feature, was rejected: it makes rollback depend on how a plugin is packaged. +Instead the cost of one snapshot per call is accepted, measured at under 1 ms +for 500 packages. + +## Consequences + +- A component that relied on mutating the graph or target to communicate must + return a registry or deltas instead. +- Snapshots cost time and memory per call. That is the price of parity, and + it is the first place to look if a very large scan slows. +- A component's cleanups run even when its constructor fails after + registering them. + +Enforced by: `TestEngineMatchCommitsOnlySuccessfulCalls`, +`TestEngineAnalyzeCommitsOnlySuccessfulCalls` (embedded and simulated-wire +parity), `TestEngineMatchDiscardsWorkFinishedAfterCancellation`, +`TestRegistryComponentLifetimeIsTheCommands`, and bomly-sdk's conformance +probe of a managed cleanup on shutdown. diff --git a/dev-docs/adr/README.md b/dev-docs/adr/README.md index 444bb43d..ea67294f 100644 --- a/dev-docs/adr/README.md +++ b/dev-docs/adr/README.md @@ -75,6 +75,7 @@ The rule itself is [ADR-0034](0034-decisions-are-recorded-as-individual-adrs.md) | ADR-0047 | 2026-10-03 | [Release assurance is a catalog plus a per-check result contract](0047-release-assurance-is-a-catalog-and-a-result-contract.md) | | ADR-0048 | 2026-10-19 | [OSV stays opt-in; Grype keeps its database fresh](0048-osv-stays-opt-in-and-grype-keeps-its-database-fresh.md) | | ADR-0049 | 2026-10-19 | [A pinned origin is one VCS locator in every export](0049-a-pinned-origin-is-one-vcs-locator-in-every-export.md) | +| ADR-0050 | 2026-10-25 | [A component call is a transaction over a snapshot, within the command's lifetime](0050-a-component-call-is-a-transaction-over-a-snapshot.md) | ## Retired diff --git a/docs/PLUGINS.md b/docs/PLUGINS.md index 69521054..42547f75 100644 --- a/docs/PLUGINS.md +++ b/docs/PLUGINS.md @@ -37,6 +37,10 @@ Installed plugins are disabled until you enable them. Enabling a plugin is the t A plugin is a Go binary built on Bomly's public SDK. Bomly starts each enabled plugin as a separate process and talks to it over a versioned protocol. During one command, Bomly keeps one process per enabled plugin, starts it on first use, and stops it when the command ends. If the process dies, Bomly restarts it once; a second death disables the plugin for the rest of the command with a warning. Plugin processes never outlive the Bomly command. +When the command ends, Bomly closes each plugin's connection and gives it a short grace period before stopping the process, so cleanups the plugin registered with `host.OnCleanup` run, all plugins at once. A component compiled into Bomly gets the same lifetime: it is built under the command's context and its cleanups run when the command ends. + +Each matcher and analyzer call receives a copy of the dependency graph and package registry, and Bomly keeps a call's result only when it succeeds. A plugin that fails or is cancelled partway leaves the scan exactly as the previous component left it. Changes a plugin makes to the graph or the target dependency are never kept: return enrichment as a registry or as package updates. + Bomly owns installing plugin packages, validating their manifest, checking recorded checksums, storing plugins under `~/.bomly/plugins`, enabling and disabling them, and loading enabled plugins at the start of a command. Plugins have no install hooks or post-install scripts, and Bomly never runs a plugin it finds in a repository checkout. ## Install sources diff --git a/go.mod b/go.mod index 1cf8e892..19617308 100644 --- a/go.mod +++ b/go.mod @@ -14,7 +14,7 @@ require ( github.com/bomly-dev/bomly-plugin-pyreach-analyzer v0.4.0 github.com/bomly-dev/bomly-plugin-scorecard-matcher v0.3.0 github.com/bomly-dev/bomly-plugin-syft-detector v0.6.0 - github.com/bomly-dev/bomly-sdk v0.14.10 + github.com/bomly-dev/bomly-sdk v0.16.0 github.com/charmbracelet/bubbles v1.0.0 github.com/charmbracelet/bubbletea v1.3.10 github.com/charmbracelet/lipgloss v1.1.0 diff --git a/go.sum b/go.sum index a639fad6..05ba7ae5 100644 --- a/go.sum +++ b/go.sum @@ -263,8 +263,8 @@ github.com/bomly-dev/bomly-plugin-scorecard-matcher v0.3.0 h1:CB4M5/N7vR2OlMuxKs github.com/bomly-dev/bomly-plugin-scorecard-matcher v0.3.0/go.mod h1:g62nlreSY249tHBl8uoECIan2FW8r+S1xNFZkbdqSBw= github.com/bomly-dev/bomly-plugin-syft-detector v0.6.0 h1:4U/YOzWujQx1LK+VpPhPEpt2ikwiBciDzaE0V5iNN6M= github.com/bomly-dev/bomly-plugin-syft-detector v0.6.0/go.mod h1:B17C01ssOWdoUGa3geCaouPSge+23QiD2ssFLao0g94= -github.com/bomly-dev/bomly-sdk v0.14.10 h1:ZFb4pXYofXeB6Zd8Rh36vj64X31u9MuD7TB5LwU11hw= -github.com/bomly-dev/bomly-sdk v0.14.10/go.mod h1:XzJZ6B9fNewalouISppzR52Fev4XGIDPElO9OjJTU5M= +github.com/bomly-dev/bomly-sdk v0.16.0 h1:czCVZRiZ2TDGMRQwy1a6jxbS+/3ypoaMYcP0/i7Boo0= +github.com/bomly-dev/bomly-sdk v0.16.0/go.mod h1:XzJZ6B9fNewalouISppzR52Fev4XGIDPElO9OjJTU5M= github.com/bradleyjkemp/cupaloy/v2 v2.8.0 h1:any4BmKE+jGIaMpnU8YgH/I2LPiLBufr6oMMlVBbn9M= github.com/bradleyjkemp/cupaloy/v2 v2.8.0/go.mod h1:bm7JXdkRd4BHJk9HpwqAI8BoAY1lps46Enkdqw6aRX0= github.com/bufbuild/protocompile v0.14.1 h1:iA73zAf/fyljNjQKwYzUHD6AD4R8KMasmwa/FBatYVw= diff --git a/internal/cli/opts/options.go b/internal/cli/opts/options.go index c9db984f..e02ee74d 100644 --- a/internal/cli/opts/options.go +++ b/internal/cli/opts/options.go @@ -2,6 +2,7 @@ package opts import ( "context" + "errors" "fmt" "io" "os" @@ -257,7 +258,11 @@ func (o *Options) PrepareForExecutionTarget(ctx context.Context, logger *zap.Log registryConfigs := RegistryConfigsFromResolved(resolved) registryConfigs.HTTPClientProvider = httpProvider scanRegistry := engine.NewRegistry(registryConfigs, *logger) - scanRegistry.Build() + scanRegistry.BuildContext(ctx) + // The registry's embedded components live as long as the command: their + // cleanups join the command cleanup chain ahead of the temp-dir cleanup + // they may still depend on, on every path out of here. + cleanup = joinCleanups(scanRegistry.Close, cleanup) if err := o.registerInstalledPluginDescriptors(ctx, scanRegistry); err != nil { if cleanup != nil { @@ -578,6 +583,21 @@ func (o *Options) Writer(stdout io.Writer) (io.Writer, func() error, error) { } // Close performs any necessary cleanup for the command options. +// joinCleanups returns a cleanup that runs first, then rest, and joins +// their errors: one failing never skips the other. Either may be nil. +func joinCleanups(first, rest func() error) func() error { + return func() error { + var errs []error + if first != nil { + errs = append(errs, first()) + } + if rest != nil { + errs = append(errs, rest()) + } + return errors.Join(errs...) + } +} + func (o *Options) Close() error { if o.cleanup == nil { return nil diff --git a/internal/cli/opts/options_test.go b/internal/cli/opts/options_test.go index 3c5d3665..63ba5769 100644 --- a/internal/cli/opts/options_test.go +++ b/internal/cli/opts/options_test.go @@ -3,6 +3,7 @@ package opts import ( "bytes" "context" + "errors" "os" "path/filepath" "slices" @@ -1089,3 +1090,25 @@ func TestDetachPluginPoolIsolatesClones(t *testing.T) { t.Fatal("DetachPluginPool on the clone must not clear the base pool") } } + +// TestJoinCleanupsRunsBothInOrder: the registry's component cleanups run +// before the temp-dir cleanup they may depend on, and one failing never +// skips the other. +func TestJoinCleanupsRunsBothInOrder(t *testing.T) { + var order []string + errFirst, errRest := errors.New("component cleanup"), errors.New("temp dir") + joined := joinCleanups( + func() error { order = append(order, "components"); return errFirst }, + func() error { order = append(order, "temp"); return errRest }, + ) + err := joined() + if !slices.Equal(order, []string{"components", "temp"}) { + t.Errorf("order = %v", order) + } + if !errors.Is(err, errFirst) || !errors.Is(err, errRest) { + t.Errorf("joined error %v lost a failure", err) + } + if err := joinCleanups(nil, nil)(); err != nil { + t.Errorf("two nil cleanups = %v", err) + } +} diff --git a/internal/composition/composition_test.go b/internal/composition/composition_test.go index 328c7c99..ff9badf4 100644 --- a/internal/composition/composition_test.go +++ b/internal/composition/composition_test.go @@ -15,7 +15,8 @@ import ( // fakeHost is a minimal plugin.HostContext for construction tests. type fakeHost struct{} -func (fakeHost) Logger() *zap.Logger { return zap.NewNop() } +func (fakeHost) Logger() *zap.Logger { return zap.NewNop() } +func (fakeHost) OnCleanup(func() error) {} func (fakeHost) HTTPClient() *httpkit.ClientProvider { provider, _ := httpkit.NewClientProvider(httpkit.ClientConfig{}) return provider diff --git a/internal/engine/engine.go b/internal/engine/engine.go index 28fa75ef..d5ecbd58 100644 --- a/internal/engine/engine.go +++ b/internal/engine/engine.go @@ -92,7 +92,8 @@ func cloneAuditRequest(req plugin.AuditRequest) plugin.AuditRequest { } // Analyze runs registered analyzers against the graph and returns the -// reachability-annotated graph. Unlike Audit, Analyze does NOT error when +// reachability-annotated graph. Each call is a transaction over its own +// AnalyzeRequest.Clone, committed only on success, exactly as Match's are. Unlike Audit, Analyze does NOT error when // zero analyzers apply — reachability is opt-in and a request with no // applicable analyzers is a normal outcome. func (e *Engine) Analyze(ctx context.Context, req plugin.AnalyzeRequest) (plugin.AnalyzeResult, error) { @@ -114,11 +115,11 @@ func (e *Engine) Analyze(ctx context.Context, req plugin.AnalyzeRequest) (plugin descriptor := analyzer.Descriptor() name := descriptor.Name label := descriptor.Label() - if err := analyzer.Ready(ctx, req); err != nil { + if err := analyzer.Ready(ctx, req.Clone()); err != nil { errs = append(errs, fmt.Errorf("analyzer %s: not ready: %w", name, err)) continue } - applicable, err := analyzer.Applicable(ctx, req) + applicable, err := analyzer.Applicable(ctx, req.Clone()) if err != nil { errs = append(errs, fmt.Errorf("analyzer %s: applicability check failed: %w", name, err)) continue @@ -127,8 +128,8 @@ func (e *Engine) Analyze(ctx context.Context, req plugin.AnalyzeRequest) (plugin continue } - result, err := analyzer.Analyze(ctx, req) - if err != nil { + result, err := analyzer.Analyze(ctx, req.Clone()) + if err = committable(ctx, err); err != nil { errs = append(errs, fmt.Errorf("analyzer %s: %w", name, err)) continue } @@ -155,6 +156,16 @@ func (e *Engine) Analyze(ctx context.Context, req plugin.AnalyzeRequest) (plugin } // Match runs registered matchers against the graph and returns the enriched graph. +// +// Each call is a transaction over a snapshot (ADR-0050): every Ready, +// Applicable and Match call receives its own MatchRequest.Clone, so a matcher +// never holds the host's graph, registry or target. Only a successful Match +// is committed -- its Registry if it returned one, otherwise its +// PackageUpdates merged into the canonical registry -- and a failed or +// cancelled one leaves the canonical state exactly as the previous matcher +// left it. Changes a matcher makes to the graph or target are never +// committed, as they never were for a managed plugin, whose request crosses +// the wire as a copy. func (e *Engine) Match(ctx context.Context, req plugin.MatchRequest) (MatchResult, error) { originalGraphSize := 0 if req.Graph != nil { @@ -189,11 +200,11 @@ func (e *Engine) Match(ctx context.Context, req plugin.MatchRequest) (MatchResul for _, matcher := range matcherList { descriptor := matcher.Descriptor() name := descriptor.Name - if err := matcher.Ready(ctx, req); err != nil { + if err := matcher.Ready(ctx, req.Clone()); err != nil { errs = append(errs, fmt.Errorf("matcher %s: not ready: %w", name, err)) continue } - applicable, err := matcher.Applicable(ctx, req) + applicable, err := matcher.Applicable(ctx, req.Clone()) if err != nil { errs = append(errs, fmt.Errorf("matcher %s: applicability check failed: %w", name, err)) continue @@ -203,8 +214,8 @@ func (e *Engine) Match(ctx context.Context, req plugin.MatchRequest) (MatchResul continue } - result, err := matcher.Match(ctx, req) - if err != nil { + result, err := matcher.Match(ctx, req.Clone()) + if err = committable(ctx, err); err != nil { errs = append(errs, fmt.Errorf("matcher %s: %w", name, err)) continue } @@ -230,6 +241,20 @@ func (e *Engine) Match(ctx context.Context, req plugin.MatchRequest) (MatchResul return aggregated, nil } +// committable reports whether a component call's result may be committed: +// not when the call failed, and not when the command was cancelled while it +// ran, even if the component returned without an error -- a component that +// ignored its context may have returned partial work. +func committable(ctx context.Context, err error) error { + if err != nil { + return err + } + if ctxErr := ctx.Err(); ctxErr != nil { + return fmt.Errorf("cancelled: %w", ctxErr) + } + return nil +} + func finalizeMatchResult(result *MatchResult) { if result == nil || result.Registry == nil { return diff --git a/internal/engine/engine_test.go b/internal/engine/engine_test.go index 4cf9f6fd..34d15d85 100644 --- a/internal/engine/engine_test.go +++ b/internal/engine/engine_test.go @@ -2,7 +2,10 @@ package engine import ( "context" + "encoding/json" "errors" + "slices" + "strings" "testing" "github.com/bomly-dev/bomly-cli/internal/testnodes" @@ -246,3 +249,324 @@ func TestEngineAudit_SkipsNotReadyOrNotApplicableAuditors(t *testing.T) { t.Fatalf("expected applicable ready auditor result, got %#v", result.Findings) } } + +// --- Transactional component calls (ADR-0050) ------------------------------- + +// wireCopy round-trips v through JSON into out, which is what the managed +// transport does to every request and result. +func wireCopy(t *testing.T, v, out any) { + t.Helper() + data, err := json.Marshal(v) + if err != nil { + t.Fatalf("encode for the wire: %v", err) + } + if err := json.Unmarshal(data, out); err != nil { + t.Fatalf("decode off the wire: %v", err) + } +} + +// wireMatcher runs a matcher as a managed plugin sees the world: each request +// arrives as a copy, and the result leaves as one. +type wireMatcher struct { + t *testing.T + inner plugin.Matcher +} + +func (w wireMatcher) Descriptor() MatcherDescriptor { return w.inner.Descriptor() } + +func (w wireMatcher) Ready(ctx context.Context, req MatchRequest) error { + var copied MatchRequest + wireCopy(w.t, req, &copied) + return w.inner.Ready(ctx, copied) +} + +func (w wireMatcher) Applicable(ctx context.Context, req MatchRequest) (bool, error) { + var copied MatchRequest + wireCopy(w.t, req, &copied) + return w.inner.Applicable(ctx, copied) +} + +func (w wireMatcher) Match(ctx context.Context, req MatchRequest) (plugin.MatchResult, error) { + var copied MatchRequest + wireCopy(w.t, req, &copied) + result, err := w.inner.Match(ctx, copied) + if err != nil { + return plugin.MatchResult{}, err + } + var returned plugin.MatchResult + wireCopy(w.t, result, &returned) + return returned, nil +} + +// wireAnalyzer is wireMatcher's analyzer twin. +type wireAnalyzer struct { + t *testing.T + inner plugin.Analyzer +} + +func (w wireAnalyzer) Descriptor() plugin.AnalyzerDescriptor { return w.inner.Descriptor() } + +func (w wireAnalyzer) Ready(ctx context.Context, req plugin.AnalyzeRequest) error { + var copied plugin.AnalyzeRequest + wireCopy(w.t, req, &copied) + return w.inner.Ready(ctx, copied) +} + +func (w wireAnalyzer) Applicable(ctx context.Context, req plugin.AnalyzeRequest) (bool, error) { + var copied plugin.AnalyzeRequest + wireCopy(w.t, req, &copied) + return w.inner.Applicable(ctx, copied) +} + +func (w wireAnalyzer) Analyze(ctx context.Context, req plugin.AnalyzeRequest) (plugin.AnalyzeResult, error) { + var copied plugin.AnalyzeRequest + wireCopy(w.t, req, &copied) + result, err := w.inner.Analyze(ctx, copied) + if err != nil { + return plugin.AnalyzeResult{}, err + } + var returned plugin.AnalyzeResult + wireCopy(w.t, result, &returned) + return returned, nil +} + +const ( + txPURL = "pkg:npm/tx-target@1.0.0" + txOtherPURL = "pkg:npm/tx-other@1.0.0" +) + +// txState is the canonical host state a component call must leave alone +// unless it succeeds: the registry, the graph and the target. +type txState struct { + graph *model.Graph + registry *model.PackageRegistry + target *model.DependencyNode +} + +func newTxState(t *testing.T) txState { + t.Helper() + graph := model.New() + target := matchTestDependency("tx-target", "1.0.0", "", model.DependencySourceRegistry) + target.Metadata = map[string]any{"seed": "kept"} + other := matchTestDependency("tx-other", "1.0.0", "", model.DependencySourceRegistry) + for _, node := range []model.GraphNode{target, other} { + if err := graph.AddNode(node); err != nil { + t.Fatal(err) + } + } + if err := graph.AddEdge(target.NodeID(), other.NodeID()); err != nil { + t.Fatal(err) + } + registry := model.NewPackageRegistry() + registry.Add(&model.Package{ + Coordinates: model.Coordinates{PURL: txPURL, Ecosystem: model.EcosystemNPM, Name: "tx-target", Version: "1.0.0"}, + Vulnerabilities: []model.Vulnerability{{Source: "osv", ID: "GHSA-seed-seed-seed"}}, + Metadata: map[string]any{"seed": map[string]any{"nested": []any{"kept"}}}, + }) + registry.Add(&model.Package{Coordinates: model.Coordinates{PURL: txOtherPURL, Ecosystem: model.EcosystemNPM, Name: "tx-other", Version: "1.0.0"}}) + return txState{graph: graph, registry: registry, target: target} +} + +// encoded is the state no component call may commit to: the graph and the +// target. The registry is where successful calls commit, so it is checked +// for the failure's marks instead. +func (s txState) encoded(t *testing.T) string { + t.Helper() + data, err := json.Marshal(map[string]any{"graph": s.graph, "target": s.target}) + if err != nil { + t.Fatal(err) + } + return string(data) +} + +// mutateEverything is what a careless component does to its request: it +// rewrites a package, its nested advisories and metadata, the graph, and +// the target in place. +func mutateEverything(graph *model.Graph, registry *model.PackageRegistry, target *model.DependencyNode) { + if pkg, ok := registry.Get(txPURL); ok { + pkg.Matched = true + pkg.Vulnerabilities[0].ID = "GHSA-leak-leak-leak" + pkg.Vulnerabilities = append(pkg.Vulnerabilities, model.Vulnerability{Source: "osv", ID: "GHSA-half-done-yet"}) + pkg.Metadata["seed"].(map[string]any)["nested"] = []any{"leaked"} + } + registry.Ensure("pkg:npm/leaked@1.0.0") + if graph != nil { + graph.RemoveNode(txOtherPURL) + } + if target != nil { + target.Metadata["seed"] = "leaked" + } +} + +// assertNoFailureMarks fails on any trace mutateEverything leaves in a +// registry: every one of its changes is distinguishable from committed work. +func assertNoFailureMarks(t *testing.T, registry *model.PackageRegistry) { + t.Helper() + if _, leaked := registry.Get("pkg:npm/leaked@1.0.0"); leaked { + t.Error("a failed call's new package reached the registry") + } + pkg, _ := registry.Get(txPURL) + if pkg.Matched { + t.Error("a failed call's flag reached the registry") + } + for _, v := range pkg.Vulnerabilities { + if v.ID == "GHSA-leak-leak-leak" || v.ID == "GHSA-half-done-yet" { + t.Errorf("a failed call's advisory %s reached the registry", v.ID) + } + } + if nested := pkg.Metadata["seed"].(map[string]any)["nested"].([]any); nested[0] != "kept" { + t.Error("a failed call's nested metadata change reached the registry") + } +} + +// TestEngineMatchCommitsOnlySuccessfulCalls runs one pipeline through +// three matchers -- one that enriches by delta, one that mutates every +// class of request data and then fails, one that returns a full registry +// alongside deltas -- embedded and over a simulated wire, and requires the +// same canonical outcome both ways: earlier success kept, the failure +// leaving no trace, the full registry winning over its own deltas, nothing +// applied twice, and the graph and target untouched. +func TestEngineMatchCommitsOnlySuccessfulCalls(t *testing.T) { + matchers := []plugin.Matcher{ + deltaMatcher{name: "enricher", match: func(MatchRequest) (plugin.MatchResult, error) { + return plugin.MatchResult{PackageUpdates: []*model.Package{{ + Coordinates: model.Coordinates{PURL: txPURL}, + Vulnerabilities: []model.Vulnerability{{Source: "osv", ID: "GHSA-from-enricher"}}, + }}}, nil + }}, + deltaMatcher{name: "careless", match: func(req MatchRequest) (plugin.MatchResult, error) { + mutateEverything(req.Graph, req.Registry, req.Target) + return plugin.MatchResult{}, errors.New("upstream timed out") + }}, + deltaMatcher{name: "full-registry", match: func(req MatchRequest) (plugin.MatchResult, error) { + pkg, _ := req.Registry.Get(txOtherPURL) + pkg.Licenses = []model.PackageLicense{{Value: "MIT", SPDXExpression: "MIT", Type: model.LicenseTypeDeclared}} + return plugin.MatchResult{ + Registry: req.Registry, + PackageUpdates: []*model.Package{{ + Coordinates: model.Coordinates{PURL: txPURL}, + Vulnerabilities: []model.Vulnerability{{Source: "osv", ID: "GHSA-ignored-delta"}}, + }}, + }, nil + }}, + } + run := func(t *testing.T, wrap func(plugin.Matcher) plugin.Matcher) (string, txState) { + t.Helper() + state := newTxState(t) + components := newTestRegistry() + for _, matcher := range matchers { + components.registerMatcher(wrap(matcher)) + } + before := state.encoded(t) + result, err := NewEngine(components).Match(context.Background(), plugin.MatchRequest{Graph: state.graph, Registry: state.registry, Target: state.target}) + if err == nil || !strings.Contains(err.Error(), "matcher careless: upstream timed out") { + t.Fatalf("Match error = %v, want the careless matcher's failure", err) + } + if after := state.encoded(t); after != before { + t.Fatalf("the host's graph or target was changed in place:\nbefore %s\nafter %s", before, after) + } + assertNoFailureMarks(t, state.registry) + pkg, _ := result.Registry.Get(txPURL) + var ids []string + for _, v := range pkg.Vulnerabilities { + ids = append(ids, v.ID) + } + slices.Sort(ids) + if want := []string{"GHSA-from-enricher", "GHSA-seed-seed-seed"}; !slices.Equal(ids, want) { + t.Errorf("advisories = %v, want %v: the earlier success kept once, the failure and the ignored delta absent", ids, want) + } + if _, leaked := result.Registry.Get("pkg:npm/leaked@1.0.0"); leaked || pkg.Matched { + t.Error("the failed matcher's changes reached the result") + } + other, _ := result.Registry.Get(txOtherPURL) + if len(other.Licenses) != 1 { + t.Errorf("the full-registry matcher's license was not committed: %v", other.Licenses) + } + data, err := json.Marshal(result.Registry) + if err != nil { + t.Fatal(err) + } + return string(data), state + } + embedded, _ := run(t, func(m plugin.Matcher) plugin.Matcher { return m }) + managed, _ := run(t, func(m plugin.Matcher) plugin.Matcher { return wireMatcher{t: t, inner: m} }) + if embedded != managed { + t.Errorf("embedded and managed execution disagree:\nembedded %s\nmanaged %s", embedded, managed) + } +} + +// TestEngineAnalyzeCommitsOnlySuccessfulCalls is the analyzer twin. +func TestEngineAnalyzeCommitsOnlySuccessfulCalls(t *testing.T) { + reachable := &model.Reachability{Status: model.ReachabilityReachable} + analyzers := []plugin.Analyzer{ + deltaAnalyzer{name: "careless", analyze: func(req plugin.AnalyzeRequest) (plugin.AnalyzeResult, error) { + mutateEverything(req.Graph, req.Registry, req.Target) + return plugin.AnalyzeResult{}, errors.New("toolchain missing") + }}, + deltaAnalyzer{name: "reach", analyze: func(req plugin.AnalyzeRequest) (plugin.AnalyzeResult, error) { + pkg, _ := req.Registry.Get(txPURL) + update := pkg.Clone() + update.Vulnerabilities[0].Reachability = reachable.Clone() + return plugin.AnalyzeResult{PackageUpdates: []*model.Package{update}}, nil + }}, + } + run := func(t *testing.T, wrap func(plugin.Analyzer) plugin.Analyzer) string { + t.Helper() + state := newTxState(t) + components := newTestRegistry() + for _, analyzer := range analyzers { + components.RegisterAnalyzer(wrap(analyzer)) + } + before := state.encoded(t) + result, err := NewEngine(components).Analyze(context.Background(), plugin.AnalyzeRequest{Graph: state.graph, Registry: state.registry, Target: state.target}) + if err == nil || !strings.Contains(err.Error(), "analyzer careless: toolchain missing") { + t.Fatalf("Analyze error = %v, want the careless analyzer's failure", err) + } + if after := state.encoded(t); after != before { + t.Fatalf("the host's graph or target was changed in place:\nbefore %s\nafter %s", before, after) + } + assertNoFailureMarks(t, state.registry) + assertNoFailureMarks(t, result.Registry) + pkg, _ := result.Registry.Get(txPURL) + if len(pkg.Vulnerabilities) != 1 || pkg.Vulnerabilities[0].ID != "GHSA-seed-seed-seed" || pkg.Vulnerabilities[0].Reachability == nil { + t.Errorf("want the seed advisory annotated by the later success and nothing from the failure, got %+v", pkg.Vulnerabilities) + } + data, err := json.Marshal(result.Registry) + if err != nil { + t.Fatal(err) + } + return string(data) + } + embedded := run(t, func(a plugin.Analyzer) plugin.Analyzer { return a }) + managed := run(t, func(a plugin.Analyzer) plugin.Analyzer { return wireAnalyzer{t: t, inner: a} }) + if embedded != managed { + t.Errorf("embedded and managed execution disagree:\nembedded %s\nmanaged %s", embedded, managed) + } +} + +// TestEngineMatchDiscardsWorkFinishedAfterCancellation: a matcher that +// returns success after the command was cancelled has its result discarded, +// since a component that ignored its context may have returned partial work. +func TestEngineMatchDiscardsWorkFinishedAfterCancellation(t *testing.T) { + ctx, cancel := context.WithCancel(context.Background()) + defer cancel() + state := newTxState(t) + components := newTestRegistry() + components.registerMatcher(deltaMatcher{name: "ignores-context", match: func(req MatchRequest) (plugin.MatchResult, error) { + cancel() + mutateEverything(req.Graph, req.Registry, req.Target) + return plugin.MatchResult{Registry: req.Registry}, nil + }}) + before := state.encoded(t) + result, err := NewEngine(components).Match(ctx, plugin.MatchRequest{Graph: state.graph, Registry: state.registry, Target: state.target}) + if !errors.Is(err, context.Canceled) { + t.Fatalf("Match error = %v, want the cancellation", err) + } + if _, leaked := result.Registry.Get("pkg:npm/leaked@1.0.0"); leaked { + t.Error("work finished after cancellation was committed") + } + if after := state.encoded(t); after != before { + t.Error("the host's graph or target was changed in place") + } + assertNoFailureMarks(t, state.registry) +} diff --git a/internal/engine/match_eligibility_test.go b/internal/engine/match_eligibility_test.go index 827919b5..dfb33666 100644 --- a/internal/engine/match_eligibility_test.go +++ b/internal/engine/match_eligibility_test.go @@ -119,8 +119,13 @@ func TestEngineMatchFiltersOccurrencesButPreservesGraphAndRegistry(t *testing.T) if err != nil || len(children) != 1 || !testnodes.Is(children[0], legacy.NodeID()) { t.Fatalf("expected eligible internal edge to survive, children=%#v err=%v", children, err) } - if matcher.registry != registry || result.Registry != registry || matcher.registry.Len() != registry.Len() { - t.Fatal("expected the complete package registry to remain shared with matchers") + // Matchers see the complete registry, not one filtered like the graph -- + // as a snapshot (ADR-0050), never the host's own pointer. + if matcher.registry == registry { + t.Fatal("the matcher received the host's registry instead of a snapshot") + } + if matcher.registry.Len() != registry.Len() || result.Registry.Len() != registry.Len() { + t.Fatalf("expected the complete package registry: matcher saw %d, result has %d, want %d", matcher.registry.Len(), result.Registry.Len(), registry.Len()) } if graph.Size() != len(all) { t.Fatalf("complete graph was mutated: size=%d want=%d", graph.Size(), len(all)) diff --git a/internal/plugin/pool.go b/internal/plugin/pool.go index bd5a1d3a..d85c058a 100644 --- a/internal/plugin/pool.go +++ b/internal/plugin/pool.go @@ -93,8 +93,14 @@ func (p *ClientPool) Acquire(ctx context.Context, executable, pluginID string, k return client.Raw(), nil } -// Shutdown terminates every pooled subprocess and runs config-file cleanups. -// The pool remains usable afterwards, but callers are expected to drop it. +// Shutdown ends every pooled subprocess. Close is go-plugin's Kill, which +// closes the connection first and gives the plugin a bounded grace period to +// run its components' cleanups (bomly-sdk ServeModule closes the component +// scope when serving stops) before it forces the process down; an old plugin +// simply exits. The subprocesses are shut down concurrently, so the command +// waits one grace period, not one per plugin. A subprocess that already died +// is closed once and harmlessly. The pool remains usable afterwards, but +// callers are expected to drop it. func (p *ClientPool) Shutdown() { if p == nil { return @@ -103,12 +109,16 @@ func (p *ClientPool) Shutdown() { entries := p.entries p.entries = make(map[string]*poolEntry) p.mu.Unlock() + var wg sync.WaitGroup for _, entry := range entries { - entry.mu.Lock() - if entry.client != nil { - entry.client.Close() - entry.client = nil - } - entry.mu.Unlock() + wg.Go(func() { + entry.mu.Lock() + defer entry.mu.Unlock() + if entry.client != nil { + entry.client.Close() + entry.client = nil + } + }) } + wg.Wait() } diff --git a/internal/plugin/pool_test.go b/internal/plugin/pool_test.go index f73df19a..b79a3bf4 100644 --- a/internal/plugin/pool_test.go +++ b/internal/plugin/pool_test.go @@ -6,6 +6,7 @@ import ( "sync" "sync/atomic" "testing" + "time" sdkplugin "github.com/bomly-dev/bomly-sdk/plugin" "github.com/bomly-dev/bomly-sdk/runtime" @@ -114,3 +115,49 @@ func TestClientPoolShutdownClosesClients(t *testing.T) { } } } + +// slowClosingClient models go-plugin's Kill on a plugin that uses its whole +// grace period to run cleanups. +type slowClosingClient struct { + fakePooledClient + delay time.Duration +} + +func (c *slowClosingClient) Close() { + time.Sleep(c.delay) + c.closed.Store(true) +} + +// TestClientPoolShutdownIsBoundedByOneGracePeriod: plugins shut down +// concurrently, so the command waits about one grace period, not one per +// plugin, and every one is closed exactly when Shutdown returns -- a dead +// subprocess included. +func TestClientPoolShutdownIsBoundedByOneGracePeriod(t *testing.T) { + const delay = 150 * time.Millisecond + var clients []*slowClosingClient + pool := NewClientPool() + pool.startFn = func(context.Context, string, string, sdkplugin.PluginKind) (pooledClient, error) { + client := &slowClosingClient{delay: delay} + clients = append(clients, client) + return client, nil + } + for _, id := range []string{"a.plugin", "b.plugin", "c.plugin", "d.plugin"} { + if _, err := pool.Acquire(context.Background(), "/bin/fake", id, ""); err != nil { + t.Fatalf("Acquire(%s): %v", id, err) + } + } + clients[1].exited.Store(true) + + started := time.Now() + pool.Shutdown() + elapsed := time.Since(started) + if elapsed > 3*delay { + t.Errorf("Shutdown took %v for %d plugins; it should be about one %v grace period", elapsed, len(clients), delay) + } + for i, client := range clients { + if !client.closed.Load() { + t.Errorf("client %d was not closed by the time Shutdown returned", i) + } + } + pool.Shutdown() +} diff --git a/internal/registry/builder.go b/internal/registry/builder.go index 16aa21aa..ed3ccfb3 100644 --- a/internal/registry/builder.go +++ b/internal/registry/builder.go @@ -127,6 +127,12 @@ type Registry struct { defaultEnabled map[string]bool componentOrigins map[string]plugin.DetectorOrigin httpProvider *httpkit.ClientProvider + // lifetime is the context embedded components are constructed under: + // the owning command's, set by BuildContext. scopes are the lifetimes + // RegisterModule started from it, in construction order; Close ends + // them. + lifetime context.Context + scopes []*plugin.ComponentScope } // ComponentOptions records Bomly-owned registry behavior that plugin authors @@ -214,8 +220,19 @@ func NewRegistry(configs Configs, logger zap.Logger) *Registry { } } -// Build registers detectors, auditors, matchers, and analyzers. +// Build registers detectors, auditors, matchers, and analyzers, with no +// command lifetime: for a registry built only to inspect descriptors. A +// registry whose components run uses BuildContext. func (r *Registry) Build() { + r.BuildContext(context.Background()) +} + +// BuildContext registers detectors, auditors, matchers, and analyzers, and +// constructs every embedded module component under ctx -- the owning +// command's context -- so a component's lifetime ends with the command's. +// The owner calls Close when the command is done. +func (r *Registry) BuildContext(ctx context.Context) { + r.lifetime = ctx r.logger.Debug("Building scan registry") r.registerDetectors() r.registerMatchers() diff --git a/internal/registry/module.go b/internal/registry/module.go index d2ebf6ff..dea8e27f 100644 --- a/internal/registry/module.go +++ b/internal/registry/module.go @@ -3,6 +3,7 @@ package registry import ( "context" "encoding/json" + "errors" "fmt" "go.uber.org/zap" @@ -19,6 +20,40 @@ type embeddedHostContext struct { registry *Registry kind plugin.PluginKind name string + scope *plugin.ComponentScope +} + +// OnCleanup registers fn on the component's lifetime, which ends when the +// registry is closed. +func (c *embeddedHostContext) OnCleanup(fn func() error) { + c.scope.OnCleanup(fn) +} + +// Close ends the lifetime of every embedded component the registry +// constructed, newest first, running each one's cleanups at most once. Every +// scope is closed even when an earlier one fails; the failures are joined, +// each naming its component. Safe to call more than once. +func (r *Registry) Close() error { + if r == nil { + return nil + } + var errs []error + for i := len(r.scopes) - 1; i >= 0; i-- { + if err := r.scopes[i].Close(); err != nil { + errs = append(errs, err) + } + } + return errors.Join(errs...) +} + +// newEmbeddedHost starts a component lifetime under parent and returns the +// host context that carries it. The scope is recorded before the constructor +// runs, so a constructor that registers a cleanup and then fails is still +// cleaned up when the registry closes. +func (r *Registry) newEmbeddedHost(parent context.Context, kind plugin.PluginKind, name string) *embeddedHostContext { + scope := plugin.NewComponentScope(parent, string(kind)+" "+name) + r.scopes = append(r.scopes, scope) + return &embeddedHostContext{registry: r, kind: kind, name: name, scope: scope} } func (c *embeddedHostContext) Logger() *zap.Logger { @@ -74,32 +109,35 @@ func (r *Registry) RegisterModule(m plugin.Module, opts ComponentOptions) error if err := plugin.ValidateModule(m); err != nil { return fmt.Errorf("register module: %w", err) } - ctx := context.Background() + ctx := r.lifetime + if ctx == nil { + ctx = context.Background() + } switch m.Kind { case plugin.PluginKindDetector: - host := &embeddedHostContext{registry: r, kind: m.Kind, name: m.Detector.Descriptor.Name} - detector, err := m.Detector.New(ctx, host) + host := r.newEmbeddedHost(ctx, m.Kind, m.Detector.Descriptor.Name) + detector, err := m.Detector.New(host.scope.Context(), host) if err != nil { return fmt.Errorf("construct detector %q: %w", m.Detector.Descriptor.Name, err) } r.RegisterDetectorWithOptions(detector, opts) case plugin.PluginKindMatcher: - host := &embeddedHostContext{registry: r, kind: m.Kind, name: m.Matcher.Descriptor.Name} - matcher, err := m.Matcher.New(ctx, host) + host := r.newEmbeddedHost(ctx, m.Kind, m.Matcher.Descriptor.Name) + matcher, err := m.Matcher.New(host.scope.Context(), host) if err != nil { return fmt.Errorf("construct matcher %q: %w", m.Matcher.Descriptor.Name, err) } r.RegisterMatcherWithOptions(matcher, opts) case plugin.PluginKindAuditor: - host := &embeddedHostContext{registry: r, kind: m.Kind, name: m.Auditor.Descriptor.Name} - auditor, err := m.Auditor.New(ctx, host) + host := r.newEmbeddedHost(ctx, m.Kind, m.Auditor.Descriptor.Name) + auditor, err := m.Auditor.New(host.scope.Context(), host) if err != nil { return fmt.Errorf("construct auditor %q: %w", m.Auditor.Descriptor.Name, err) } r.RegisterAuditorWithOptions(auditor, opts) case plugin.PluginKindAnalyzer: - host := &embeddedHostContext{registry: r, kind: m.Kind, name: m.Analyzer.Descriptor.Name} - analyzer, err := m.Analyzer.New(ctx, host) + host := r.newEmbeddedHost(ctx, m.Kind, m.Analyzer.Descriptor.Name) + analyzer, err := m.Analyzer.New(host.scope.Context(), host) if err != nil { return fmt.Errorf("construct analyzer %q: %w", m.Analyzer.Descriptor.Name, err) } diff --git a/internal/registry/module_test.go b/internal/registry/module_test.go index e3cfe036..2decd4fa 100644 --- a/internal/registry/module_test.go +++ b/internal/registry/module_test.go @@ -3,6 +3,7 @@ package registry import ( "context" "errors" + "slices" "strings" "testing" @@ -169,3 +170,79 @@ func TestCompositionAnalyzerEntriesStayInDocsCatalog(t *testing.T) { } } } + +// lifecycleModule is a matcher module whose constructor records its context +// and registers cleanups that append to a shared trace, and optionally fails +// after registering them. +func lifecycleModule(name string, trace *[]string, constructed *context.Context, failAfterRegister bool) plugin.Module { + return plugin.Module{Kind: plugin.PluginKindMatcher, Matcher: &plugin.MatcherModule{ + Descriptor: plugin.MatcherDescriptor{Name: name}, + New: func(ctx context.Context, host plugin.HostContext) (plugin.Matcher, error) { + if constructed != nil { + *constructed = ctx + } + host.OnCleanup(func() error { *trace = append(*trace, name+":first"); return nil }) + host.OnCleanup(func() error { *trace = append(*trace, name+":second"); return nil }) + if failAfterRegister { + return nil, errors.New("config invalid") + } + return &moduleTestMatcher{name: name}, nil + }, + }} +} + +// TestRegistryComponentLifetimeIsTheCommands holds the embedded half of +// cli#415: a constructor's context is the command's, cleanups run newest +// first and at most once when the registry closes -- including those a +// failed constructor registered before failing -- and a cleanup error names +// its component without stopping the rest. +func TestRegistryComponentLifetimeIsTheCommands(t *testing.T) { + command, cancel := context.WithCancel(context.Background()) + registry := NewRegistry(Configs{}, *zap.NewNop()) + registry.lifetime = command + + var trace []string + var constructed context.Context + if err := registry.RegisterModule(lifecycleModule("alpha", &trace, &constructed, false), ComponentOptions{}); err != nil { + t.Fatalf("RegisterModule(alpha): %v", err) + } + if err := registry.RegisterModule(lifecycleModule("broken", &trace, nil, true), ComponentOptions{}); err == nil { + t.Fatal("RegisterModule(broken) succeeded") + } + failing := plugin.Module{Kind: plugin.PluginKindMatcher, Matcher: &plugin.MatcherModule{ + Descriptor: plugin.MatcherDescriptor{Name: "leaky"}, + New: func(_ context.Context, host plugin.HostContext) (plugin.Matcher, error) { + host.OnCleanup(func() error { return errors.New("socket already closed") }) + return &moduleTestMatcher{name: "leaky"}, nil + }, + }} + if err := registry.RegisterModule(failing, ComponentOptions{}); err != nil { + t.Fatalf("RegisterModule(leaky): %v", err) + } + + if constructed.Err() != nil { + t.Fatal("the constructor's context is already done") + } + cancel() + if constructed.Err() == nil { + t.Error("cancelling the command did not reach the constructor's context") + } + if len(trace) != 0 { + t.Fatalf("cancellation ran cleanups; only Close may: %v", trace) + } + + err := registry.Close() + if want := []string{"broken:second", "broken:first", "alpha:second", "alpha:first"}; !slices.Equal(trace, want) { + t.Errorf("cleanup order = %v, want %v", trace, want) + } + if err == nil || !strings.Contains(err.Error(), "matcher leaky: cleanup: socket already closed") { + t.Errorf("Close error = %v, want the leaky component's failure by name", err) + } + again := registry.Close() + if len(trace) != 4 { + t.Errorf("a second Close ran cleanups again: %v", trace) + } + if again == nil || again.Error() != err.Error() { + t.Errorf("a second Close = %v, want the first Close's %v", again, err) + } +}