Labeled vulnerability matches: for a package in a known image, is a given vulnerability really there? Each label says tp (the installed code is vulnerable), fp (it isn't) or unclear (nobody has shown either), with a justification and the evidence behind it.
This is the ground truth yardstick scores grype against. A yardstick gate scans a set of images with two grype setups and uses these labels to tell whether a change made matching better or worse.
It replaces vulnerability-match-labels, whose labels were ported here.
What it is not:
- not a vulnerability database or feed: a label is about one package version in one image, not about a CVE in general
- not a risk or severity rating: a label states whether the code is vulnerable, never whether anyone should care (see P-1 in the labeling policy)
- not where SBOMs are stored: they live in R2 (see below)
yardstick reads the published snapshot by default (github.com/anchore/claims@main as a claims source), so nobody needs a checkout just to run a gate. To add labels:
- run a gate and label what changed with
yardstick claims serve(oryardstick claims add), which writes to a local scratch directory yardstick claims promote <scratch dir> --to <this checkout>to move the labels heremake validate-changed, then open a pull request
The walkthrough in yardstick's demo does all of it with real output.
pkg/oci/<registry>/<repository>/sha256-<digest>/is one image: its labels sit in a directory per kind of label, and today the only kind ismatches/, with<type>.jsonper package type (apk.json,rpm.json,go-module.json, ...) andimage.jsonfor labels that name no package. The image's SBOM, when fetched, sits besidematches/assbom.syft.jsonsbom/sha256-<digest>/matches/is the same for a subject that is an SBOM rather than an image, andglobal/matches/<type>.jsonholds labels that apply to every subject. Neither has any labels yetcatalog.yamlpins the syft version and platform every image's SBOM is built withvocabulary.yamllists the allowed justifications, fact sources and namespaces, and the labeling rules: each rule's verdict, confidence ceiling and the policy decision behind itlabeling-policy.mdsays what a label means, what each kind of evidence can support, and the decisions (P-1, P-2, ...) the rules cite. Change a rule invocabulary.yamland its decision here in the same pull requestpublish.yamlnames the label file format this corpus is in (v1) and which snapshotlatestpoints atschema/matches/holds the JSON schema match label files are validated against (yardstick claims schemawrites it)scripts/holds the bundling, publishing, SBOM and validation scripts, plus the one-time port from vulnerability-match-labelsTaskfile.yaml,Makefileand.binny.yamlhold the tasks and pinned tools, including the yardstick that validates the corpus.make helplists the tasks
Every merge to main that touches the corpus is validated, bundled and published to the oss-prod-yardstick R2 bucket, served at https://yardstick.anchore.io:
claims/matches/snapshots/<sha256>.zipis the corpus at that commit, named by its digest and never overwrittenclaims/matches/commits/<sha>.jsonnames the snapshot for a commitclaims/matches/latest-v1.jsonnames the snapshot yardstick's@mainreads. It follows the tip ofmainunlesspublish.yamlpins it to a commitclaims/matches/schema/schema-<version>.jsonis the match label file schemaclaims/sboms/pkg/oci/.../sbom.syft-v<version>.jsonis each image's SBOM, built by the syft incatalog.yamland uploaded once. It sits undersboms/rather than besidematches/, since every kind of label reads it
A snapshot holds only the corpus (catalog.yaml, vocabulary.yaml, pkg/, sbom/, global/). make sboms fetches the SBOMs into a checkout, building any R2 doesn't have yet.