Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/CI_INTEGRATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ Only the packages the PR touched are audited, so findings elsewhere never fail t

### Cache enrichment data

Bomly keeps enrichment responses under `~/.bomly/cache` and the bundled Grype vulnerability database under the operating system's cache directory (`~/.cache/grype/db` on Linux). Caching both between runs cuts enrichment time from minutes to seconds:
Bomly keeps enrichment responses under `~/.bomly/cache` and the bundled Grype vulnerability database under the operating system's cache directory (`~/.cache/grype/db` on Linux, `~/Library/Caches/grype/db` on macOS). Caching both between runs cuts enrichment time from minutes to seconds:

```yaml
- name: Cache Bomly enrichment data
Expand Down
4 changes: 2 additions & 2 deletions docs/DETECTORS.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,9 @@ Matchers only call out when you pass `--enrich`, but a detector that runs a buil
| Lockfile parser | `npm`, `pnpm`, `yarn` (lockfile strategy), `bundler-detector`, `composer-detector`, `nuget-detector`, `github-actions-detector`, SBOM ingest | None. The detector only reads files. |
| Lockfile-first hybrid | `cargo-detector`, `poetry-detector`, `uv-detector` | None when the lockfile is present. The build-tool fallback passes `--locked` or `--no-sync` to stay offline. |
| `pip inspect` | `pip-detector`, `pipenv-detector` | None when a lockfile is present. Otherwise the detector **may download** packages while it fills the isolated environment it inspects. |
| Build-tool primary | `go-detector`, `maven-detector`, `gradle-detector`, `sbt-native-detector` | **May download** artifacts that are not in the local cache. |
| Build-tool primary | `go-detector`, `maven-detector`, `gradle-detector`, `sbt-native-detector`, `pub-native-detector`, `swiftpm-native-detector` | **May download** artifacts that are not in the local cache. |

The build-tool detectors run commands you would already run locally: `go list`, `mvn dependency:tree`, `gradle dependencies`, and `sbt dependencyTree`. Whether those commands use the network depends on the tool and on your local cache, not on Bomly. To keep these scans offline, warm the local cache first (for example with `go mod download` or `mvn dependency:go-offline`) or commit a lockfile when the ecosystem supports one.
The build-tool detectors run commands you would already run locally: `go list`, `mvn dependency:tree`, `gradle dependencies`, `sbt dependencyTree`, `dart pub deps`, and `swift package show-dependencies`. Whether those commands use the network depends on the tool and on your local cache, not on Bomly. To keep these scans offline, warm the local cache first (for example with `go mod download` or `mvn dependency:go-offline`) or commit a lockfile when the ecosystem supports one.

Each page under [`detectors/`](detectors/) documents the exact command that detector runs and whether it uses the network.

Expand Down
9 changes: 6 additions & 3 deletions docs/MATCHERS.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,16 +61,19 @@ Every matcher caches its responses on disk, so repeated scans are fast and survi
| --- | --- |
| Cache root (Unix/macOS) | `$HOME/.bomly/cache/` |
| Cache root (Windows) | `%USERPROFILE%\.bomly\cache\` |
| Fallback when no home dir | `./.bomly-cache/` |
| No home directory | No cache. Every run queries the services. |

Each matcher uses its own subdirectory and time to live (TTL):

| Matcher | Subdirectory | Default TTL |
| --- | --- | --- |
| OSV (queries) | `osv/` | 24h |
| OSV (vulnerability details) | `osv-vulns/` | 7d |
| CISA KEV | `kev/` | 6h |
| CISA KEV | `kev/` | 24h |
| deps.dev | `licenses/depsdev/` | 24h |
| Scorecard | `scorecard/` | 24h |

The bundled Grype keeps its vulnerability database outside this cache, in the operating system's cache directory. See [Grype](matchers/grype.md#network).

To clear the cache, delete the directory:

Expand All @@ -89,7 +92,7 @@ Matchers degrade rather than abort. A failed enrichment never fails the scan:
- **Cache write error**: the response is still applied, and a warning is logged.
- **Rate limit or server error**: Bomly retries with backoff inside the matcher, then degrades.

So a scan with `--enrich` exits 0 on a healthy graph even when some lookups failed. A policy gate only sees the data that did arrive, so it cannot tell a clean package from one whose lookup failed. Run with `-v` to see the warning for each failed lookup. To fail CI on the findings enrichment does return, add `--audit --fail-on <severity>`; see [Auditors](AUDITORS.md).
So a scan with `--enrich` exits 0 on a healthy graph even when some lookups failed. A policy gate only sees the data that did arrive, so it cannot tell a clean package from one whose lookup failed. A matcher that returned no data at all, for example Grype without its database, prints a warning line at the top of the text report. Run with `-v` to see the warning for each failed lookup. To fail CI on the findings enrichment does return, add `--audit --fail-on <severity>`; see [Auditors](AUDITORS.md).

## See also

Expand Down
26 changes: 24 additions & 2 deletions docs/NETWORK.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,11 +34,33 @@ For an air-gapped or restricted pipeline:
1. Do not pass `--enrich`, `--url`, `--image`, or `--install-first`. Scan a local checkout.
2. Commit lockfiles, so the lockfile detectors resolve the graph and no build tool runs. In CI, pin `--detectors` to the lockfile detectors, so an unexpected fallback to a build-tool detector fails the job instead of running the tool. See [CI-readiness warnings](CI_READINESS.md).
3. If you do rely on a build-tool detector, that tool's own offline settings (Go module proxy settings, Maven offline mode, a local registry mirror) decide whether it downloads anything.
4. Enrichment responses are cached under `~/.bomly/cache/` with a lifetime per matcher, so a previously enriched environment keeps working through an outage. Offline, an expired entry is a miss, not stale data. See [Matchers](MATCHERS.md#cache).
4. Enrichment responses are cached under `~/.bomly/cache/` with a lifetime per matcher. Within that lifetime, a repeated scan of the same packages reads the cache. Offline, an expired entry is a miss, not stale data. See [Matchers](MATCHERS.md#cache). A failed lookup is a warning in the text report, not a finding, and the JSON document does not record it; see [failure semantics](MATCHERS.md#failure-semantics).

### Carry the Grype database to an offline host

Grype's vulnerability database is one bulk download, so it is the only enrichment data you can carry to a host with no network. The OSV, KEV, license, and Scorecard caches hold answers for packages already scanned, and only until their lifetime ends, so they do not replace a database.

On a connected host, run one enriched scan to download the database:

```bash
bomly scan --enrich --matchers grype
```

Copy the database directory into the Grype cache directory of the offline host. The full `bomly` binary reads it from the operating system's cache directory, so the destination depends on the offline host, not on where it came from: `~/.cache/grype/db` on Linux (or under `XDG_CACHE_HOME`) and `~/Library/Caches/grype/db` on macOS.

The lite binary uses the external `grype` command and its own settings. On the connected host, run `grype db update`, then copy the directory that `grype db status` reports to the same location on the offline host, and set `GRYPE_DB_AUTO_UPDATE=false` there instead of the Bomly variable below.

On the offline host, turn the update check off and run Grype alone, with `--detectors` pinned to lockfile detectors (step 2 above). A build-tool detector such as `go-detector` runs the tool, which may download modules, so leave it out unless the tool's own cache and offline settings cover it:

```bash
BOMLY_GRYPE_DB_AUTO_UPDATE=false bomly scan --enrich --matchers grype --detectors npm --audit --fail-on high
```

With the check off, the scan still warns once the database is older than `matchers.grype.db_max_age` (five days by default). Repeat the download on the connected host and copy the directory again to refresh it. The [Grype matcher](matchers/grype.md) page lists the settings.

## Proxies, custom endpoints, and CAs

Bomly's built-in HTTP clients honor the configured proxy, no-proxy rules, and extra CA certificates, and accept custom OSV and Scorecard endpoints, including private addresses. These are trusted settings: set them in the user config, in a file you select with `--config` or `BOMLY_CONFIG`, or in environment variables. A config file inside a repository is never loaded on its own. Redirects follow Go's standard rules, and Bomly does not block private addresses, so self-hosted mirrors work. [Security and trust boundaries](SECURITY.md#network-trust) explains the trade-offs of that choice, and the [Config reference](CONFIG_REFERENCE.md) lists the keys.
Bomly's built-in HTTP clients honor the configured proxy, no-proxy rules, and extra CA certificates, and accept custom OSV and Scorecard endpoints, including private addresses. The Grype database download is the exception: the bundled Grype reads the standard `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` variables and the system certificate store, not Bomly's settings. These are trusted settings: set them in the user config, in a file you select with `--config` or `BOMLY_CONFIG`, or in environment variables. A config file inside a repository is never loaded on its own. Redirects follow Go's standard rules, and Bomly does not block private addresses, so self-hosted mirrors work. [Security and trust boundaries](SECURITY.md#network-trust) explains the trade-offs of that choice, and the [Config reference](CONFIG_REFERENCE.md) lists the keys.

## See also

Expand Down
2 changes: 1 addition & 1 deletion docs/SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ These defaults limit what Bomly may do. They do not make every scan offline: a b

## Network trust

Bomly's built-in HTTP clients honor the configured proxy, no-proxy rules, and extra CA certificates. Redirects follow Go's standard rules, including its redirect limit and the removal of credentials on a cross-host redirect. Errors and logs never include endpoint or proxy credentials.
Bomly's built-in HTTP clients honor the configured proxy, no-proxy rules, and extra CA certificates. The bundled Grype's database download reads the standard proxy variables and the system certificate store instead. Redirects follow Go's standard rules, including its redirect limit and the removal of credentials on a cross-host redirect. Errors and logs never include endpoint or proxy credentials.

A trusted configuration may point a matcher at a custom endpoint on a public or private network, including plain HTTP, and a standard redirect may move an HTTPS request to HTTP. Bomly does not block private addresses, pin DNS answers, prevent DNS rebinding, or block that downgrade. In practice this means that whoever controls your configuration, or a server it names, controls where enrichment requests go. Keep configuration files under the same review as code.

Expand Down
2 changes: 1 addition & 1 deletion docs/TROUBLESHOOTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ The scan exited `0` with no findings. Check that you passed `--enrich` (without

## Enrichment is slow on first run

This is expected. The bundled Grype downloads its vulnerability database on first use, and matcher responses are cached under `~/.bomly/cache/`. Later runs read the cache and finish in seconds. In CI, cache that directory between runs, and schedule a nightly run on `main` to keep it warm. See [CI integration](CI_INTEGRATION.md#cache-enrichment-data).
This is expected. The bundled Grype downloads its vulnerability database on first use into the operating system's cache directory, and matcher responses are cached under `~/.bomly/cache/`. Later runs read the cache and finish in seconds. In CI, cache that directory between runs, and schedule a nightly run on `main` to keep it warm. See [CI integration](CI_INTEGRATION.md#cache-enrichment-data).

## Enrichment fails with rate-limit or network errors

Expand Down
2 changes: 1 addition & 1 deletion docs/USE_CASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ bomly scan # no enrichment calls
bomly scan --sbom --path sbom.json # read an SBOM you already have
```

**You get:** the dependency graph with no license or vulnerability data. Without `--enrich` Bomly makes no enrichment calls, but a detector that runs your build tool (Go, Maven, Gradle) may download packages. Commit a lockfile or fill the build tool's cache first to stay fully offline. See [Network and privacy](NETWORK.md).
**You get:** the dependency graph with no license or vulnerability data. Without `--enrich` Bomly makes no enrichment calls, but a detector that runs your build tool (Go, Maven, Gradle) may download packages. Commit a lockfile or fill the build tool's cache first to stay fully offline. To match vulnerabilities on a host with no network, carry the Grype database over; see [Network and privacy](NETWORK.md#carry-the-grype-database-to-an-offline-host).

## Scan and audit a container image

Expand Down
6 changes: 3 additions & 3 deletions docs/matchers/grype.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,10 +37,10 @@ Grype reads from a local vulnerability database. The full `bomly` binary downloa

| Mode | Database sync source | Database location |
| --- | --- | --- |
| Full `bomly` binary | Anchore's database service | Managed by the built-in Grype; default `~/.cache/grype/db/` |
| Full `bomly` binary | Anchore's database service | The operating system's cache directory: `~/.cache/grype/db/` on Linux (or under `XDG_CACHE_HOME`), `~/Library/Caches/grype/db/` on macOS |
| `bomly-lite` | The external `grype` command | Wherever that `grype` stores it |

The database sync is the only network call Grype makes. Once synced, matching is local. Bomly's own enrichment cache does not wrap Grype, because Grype manages its own.
The database sync is the only network call Grype makes. Once synced, matching is local. Bomly's own enrichment cache does not wrap Grype, because Grype manages its own. The download reads the standard `HTTPS_PROXY`, `HTTP_PROXY`, and `NO_PROXY` variables and the system certificate store, not Bomly's `network.proxy` settings or CA file. To run with no network at all, copy the database directory from a connected host; see [Network and privacy](../NETWORK.md#carry-the-grype-database-to-an-offline-host).

Three settings control the refresh in the full binary. See the [config reference](../CONFIG_REFERENCE.md) for the full list:

Expand Down Expand Up @@ -92,7 +92,7 @@ GRYPE_DB_AUTO_UPDATE=false bomly-lite scan --enrich # lite: the externa
BOMLY_GRYPE_DB_AUTO_UPDATE=false bomly scan --enrich # full: keep the local database as it is
```

With updates off, the full binary still warns once the database is older than `matchers.grype.db_max_age`. Delete `~/.cache/grype/db` to download a current one.
With updates off, the full binary still warns once the database is older than `matchers.grype.db_max_age`. Delete the database directory to download a current one.

## Limitations

Expand Down
28 changes: 28 additions & 0 deletions internal/cli/cmd_progress.go
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,34 @@ func warningProgressChildren(warnings []engine.PipelineWarning) []progress.Child
return children
}

// pipelineWarningNotices renders stage warnings as report notices, one
// "source: message" line per distinct warning, so a run without a terminal
// (CI logs, -q, piped output) still says that a matcher produced no data.
// The progress stream shows the same warnings only when it can draw.
func pipelineWarningNotices(warnings []engine.PipelineWarning) []string {
notices := make([]string, 0, len(warnings))
seen := make(map[string]struct{}, len(warnings))
for _, warning := range warnings {
message := render.SanitizeUntrusted(strings.ReplaceAll(warning.Message, "\n", " "))
if message == "" {
continue
}
notice := message
if source := render.SanitizeUntrusted(warning.Source); source != "" {
notice = source + ": " + message
}
if _, ok := seen[notice]; ok {
continue
}
seen[notice] = struct{}{}
notices = append(notices, notice)
}
if len(notices) == 0 {
return nil
}
return notices
}

// detectorWarningProgressChildren converts detection warnings into ⚠ children
// using the warning source as Label and the message as Detail.
func detectorWarningProgressChildren(warnings []plugin.DetectorWarning) []progress.Child {
Expand Down
31 changes: 31 additions & 0 deletions internal/cli/pipeline_warning_notices_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
package cli

import (
"reflect"
"testing"

"github.com/bomly-dev/bomly-cli/internal/engine"
)

// A matcher that returned no data must leave a line in the text report. The
// progress stream carries the same warning only on a terminal, so a CI log of
// a scan with a missing vulnerability database read as a clean scan.
func TestPipelineWarningNoticesNameTheSourceAndDeduplicate(t *testing.T) {
warnings := []engine.PipelineWarning{
{Source: "grype", Message: "grype vulnerability DB unavailable: database does not exist"},
{Source: "grype", Message: "grype vulnerability DB unavailable: database does not exist"},
{Source: "osv", Message: "osv batch query:\nconnection refused\x1b[31m"},
{Source: "", Message: " "},
}
got := pipelineWarningNotices(warnings)
want := []string{
"grype: grype vulnerability DB unavailable: database does not exist",
"osv: osv batch query: connection refused",
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("notices = %#v, want %#v", got, want)
}
if got := pipelineWarningNotices(nil); got != nil {
t.Fatalf("expected nil for no warnings, got %#v", got)
}
}
7 changes: 4 additions & 3 deletions internal/cli/scan_cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -117,9 +117,10 @@ func newScanCmd() *cobra.Command {
}
scanManifests := output.ScanManifestsFromConsolidated(consolidated, pipeResult.Registry)
// Notices ride the report itself, not just the progress stream, so
// -q and non-terminal CI runs still surface degraded resolution and
// CI-readiness problems.
notices := render.WarningNotices(payload.Warnings)
// -q and non-terminal CI runs still surface degraded resolution,
// CI-readiness problems, and a matcher that returned no data (a
// missing vulnerability database reads as a clean scan otherwise).
notices := append(render.WarningNotices(payload.Warnings), pipelineWarningNotices(pipeResult.MatchWarnings)...)
textRenderer := func(w io.Writer) error {
if _, err := io.WriteString(w, render.Scan(selectedGraph, pipeResult.Registry, findings, pipeResult.MatcherStats, commandCtx.ResolvedConfig.Enrich, commandCtx.ResolvedConfig.Audit, commandCtx.ResolvedConfig.Analyze, commandCtx.ResolvedConfig.FailOn, scanManifests, notices)); err != nil {
return fmt.Errorf("write scan text output: %w", err)
Expand Down
Loading
Loading