Materialize per-layer artifacts with explicit whiteout handling

[chruffins]

tldr

Content-addressed per-layer artifact store at images/layers/<layer-blob-digest>/. Pulls and composition (next PR in the stack) will share it. Not yet called by production code — no behavior change today.

what this adds

1. Materialization (materializeLayerArtifact) — unpacks a layer blob from the existing shared OCI cache with umoci, converts to erofs, installs layer.erofs + an artifact.json record atomically. Same layer shared across images materializes once; concurrent callers share one build (singleflight).
2. Whiteouts as overlayfs — .wh.<name> → 0:0 char device, .wh..wh..opq → trusted.overlay.opaque xattr. Standard representation; no private marker format.
3. Hardened extraction — umoci confines entries, blob digest + diff ID verified, stream capped at 100 GiB, detached builds bounded by a 1h deadline.
4. Disk accounting — materialized artifacts counted in TotalOCICacheBytes (conservative: blob + artifact of the same layer may transiently coexist).

review status: done

Two independent review passes (correctness + code-quality). All actionable findings fixed in 46d02f3. Notables:

• Hung blob read could wedge the singleflight key forever → 1h deadline on detached builds
• Cache blob integrity was never verified → unpack now checks the compressed digest
• Dead/simplification cleanup: composeOnDiskFormat + whiteoutPrefix moved to tests, multiCloser deleted, blob path via paths.OCICacheBlob

Verified empirically on a dev host: mkfs.erofs 1.8.10 and mkfs.ext4 both preserve whiteout char devices and opaque xattrs through ExportRootfs — the stacking design's premise holds. Depends on host erofs-utils version; check it in the pull-integration PR.

reviewer focus (3 things)

1. layerBuildTimeout = 1h is a judgment call — flag if hypeman has a convention for bounding background work
2. Unpack now hard-fails on blob digest mismatch — new error if any path intentionally writes unverified blobs
3. Store machinery lands ahead of its production caller. The pull-integration PR should follow soon: it sweeps stale .unpack-* dirs, decides on blob retirement (double-counting), and decomposes manager.go (~1000 lines)

validation

Full lib/images + lib/paths green on a dev host, plus a root pass for the root-gated tests (overlayfs whiteout inodes, device/fifo nodes). Tests cover: materialize/reuse/rebuild, corrupt-record recovery, invalid digest rejection (sha256:.., sha256:a/b), whiteout/opaque output and cross-layer semantics, symlink confinement and replacement, hardlinks, cancellation, zstd + docker media types, concurrent singleflight (8 callers, one build), trailing-padding diff IDs. Disk-usage exclusions covered by existing TestTotalLayerArtifactBytesFromFilesystem.

CI note: the test workflow is flaky on this branch — failures are unrelated integration tests (TestEgressProxyRewritesHTTPSHeaders timeout, docker/network tests needing nginx:alpine), failing before the latest commits too. Layer-artifact tests pass consistently.

next step

Review the 3 focus items above, then approve. The pull-integration PR is where the store gets its production caller.

---

Note

Medium Risk
New layer unpack and filesystem materialization touches tar extraction and external mkfs tools; incorrect whiteout or path handling could affect future image composition, though pull behavior is unchanged until wired in.

Overview
Introduces a content-addressed per-layer artifact store under images/layers/<blob-digest>/, keyed by compressed layer digest plus format (erofs/ext4). materializeLayerArtifact reads blobs from the existing OCI cache, unpacks with umoci into a temp dir, converts to the default disk format, and installs layer.* plus an artifact.*.json record atomically; duplicate work is deduped with singleflight and valid on-disk records are reused.

Whiteouts are modeled explicitly: artifact extraction uses umoci OverlayfsRootfs (character-device whiteouts and opaque xattrs) so stacked layers can be mounted later; compose-oriented unpacking uses DirRootfs. Layer unpack supports gzip/zstd, verifies diff IDs, caps unpacked size at 100 GiB, and honors context cancellation via a contextReader and CommandContext for mkfs.erofs.

Disk accounting now walks layer.* files (skipping .unpack-* temps) and folds that into TotalOCICacheBytes / resource admission alongside OCI blobs. New path helpers live in lib/paths. The materialization API is implemented and tested but not yet called from the image pull/build path (follow-up PR).

^{Reviewed by Cursor Bugbot for commit d4ed105. Configure here.}

[github-actions]

Decompression bomb: layer extraction has no cumulative size cap

Semgrep rule: go.lang.security.decompression_bomb.potential-dos-via-decompression-bomb

The rule fired on line 291 (io.Copy(io.Discard, reader)), but that copy targets io.Discard — it burns CPU, not disk. The real instance of the same class is here: every tar.TypeReg entry is written through this unbounded io.Copy(file, tr), and neither extractTarFile nor the loop in unpackLayerBlob enforces any ceiling on total unpacked bytes or entry count.

Why this is a true positive

• Layer blobs and media types come from a remote OCI manifest (lib/images/oci.go:474), i.e. a customer-supplied image reference — the compressed input is attacker-controlled.
• stats.unpackedBytes is accumulated but never compared against a limit, and header.Size is trusted for accounting only.
• The diffID check in materializeLayerArtifact (lib/images/layer_artifact.go:173) runs after extraction finishes, and is skipped entirely when desc.DiffID == "", so it cannot bound the write.
• Disk admission control in lib/resources/resource.go sums images already marked ready; it does not throttle an in-progress unpack. On a shared hypervisor host, a ~1 KB gzip layer expanding to hundreds of GB fills the data partition for every co-tenant VM.

Note this path is currently reached only from layer_artifact_test.go — nothing else calls materializeLayerArtifact yet — so this is latent, and worth fixing before it is wired into the pull path.

Recommended fix — thread an explicit budget through the unpack loop and bound the per-file copy:

const (
    maxUnpackedBytes = 32 << 30 // 32 GiB per layer
    maxEntries       = 1 << 20
)

// unpackLayerBlob: budget := int64(maxUnpackedBytes), and in the loop
if stats.entries++; stats.entries > maxEntries {
    return nil, fmt.Errorf("layer exceeds %d entries", maxEntries)
}

// extractTarFile: refuse to write past the remaining budget
func extractTarFile(tr *tar.Reader, target string, header *tar.Header, budget *int64) error {
    // ...
    n, err := io.CopyN(file, tr, *budget+1)
    if err != nil && err != io.EOF {
        _ = file.Close()
        return err
    }
    if n > *budget {
        _ = file.Close()
        return fmt.Errorf("layer exceeds unpacked size limit of %d bytes", maxUnpackedBytes)
    }
    *budget -= n
    // ...
}

Do not apply Semgrep's suggested autofix at line 291 (io.CopyN(io.Discard, reader, 1024*1024*256)). Truncating that drain leaves the TeeReader hash incomplete, yielding a wrong stats.diffID and spurious "diff id mismatch" failures for any layer over 256 MB. If you want the trailing-data read bounded too, cap it well above the largest expected layer and treat hitting the cap as an error rather than silently stopping.

If you consider this an accepted risk, suppress with either:

• inline on the flagged line:
  // nosemgrep: go.lang.security.decompression_bomb.potential-dos-via-decompression-bomb (or bare // nosemgrep to silence all rules on that line)
• or exclude the file by adding lib/images/layer_artifact.go to .semgrepignore

[chruffins]

@cursor review

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

^{Reviewed by Cursor Bugbot for commit d4ed105. Configure here.}