feat: generalize framework artifact pipeline foundations

.github/workflows/layered-workspaces.yml

@@ -62,6 +62,9 @@ jobs:
       - run: cargo test -p trail fuse_adapter_runs_shared_mounted_view_suite -- --nocapture
         env:
           TRAIL_RUN_FUSE_COW_TESTS: "1"
+      - run: cargo test -p trail fuse_mount_reads_artifact_manifest_without_layer_materialization -- --nocapture
+        env:
+          TRAIL_RUN_FUSE_COW_TESTS: "1"
       - run: cargo test -p trail real_cmake_configure_build_and_clean_stay_lane_private -- --nocapture
         env:
           TRAIL_RUN_FUSE_COW_TESTS: "1"
@@ -132,6 +135,9 @@ jobs:
       - run: cargo test -p trail nfs_adapter_runs_shared_mounted_view_suite -- --nocapture
         env:
           TRAIL_RUN_NFS_COW_TESTS: "1"
+      - run: cargo test -p trail nfs_mount_reads_artifact_manifest_without_layer_materialization -- --nocapture
+        env:
+          TRAIL_RUN_NFS_COW_TESTS: "1"
       - run: cargo test -p trail real_nfs_mount_records_new_modified_and_renamed_files -- --nocapture
         env:
           TRAIL_RUN_NFS_COW_TESTS: "1"
@@ -185,6 +191,7 @@ jobs:
         run: |
           $env:TRAIL_RUN_DOKAN_COW_TESTS = "1"
           cargo test -p trail dokan_adapter_runs_shared_mounted_view_suite -- --nocapture
+          cargo test -p trail dokan_mount_reads_artifact_manifest_without_layer_materialization -- --nocapture
           cargo test -p trail foreground_dokan_mount_stops_through_a_separate_trail_handle -- --nocapture
           cargo test -p trail daemon_owned_dokan_mount_returns_ready_and_unmounts_asynchronously -- --nocapture
           cargo test -p trail real_windows_cmake_build_and_clean_stay_lane_private -- --nocapture

CHANGELOG.md

@@ -5,6 +5,75 @@ All notable changes to Trail are documented in this file. Trail follows
 
 ## [Unreleased]
 
+### Added
+
+- Added Rust library artifact resolution component/batch operations with durable fenced
+  attempts, content-addressed snapshot reuse, explicit-only refresh, bounded redacted
+  evidence, and deterministic reports.
+- Workspace-layer singleflight now records durable generation-fenced owner phases and
+  waiter outcomes, and only recovers a lock when the exact PID/start identity is proven
+  dead or mismatched.
+- Workspace open now recovers dead artifact constructors and exact owned staging;
+  doctor and fsck validate raw CAS objects, snapshots, envelopes, attempt coherence,
+  legacy/CAS layouts, and orphan materializations with repair guidance.
+- Backup/restore validation now treats omitted materialization caches as disposable,
+  rebases restored layer paths before publication, and parallel environment builders
+  use a bounded SQLite wait during short WAL publication overlap.
+- Environment discovery now reports marker-recognized plugins that do not support the
+  current host as typed `unsupported` proposals without launching plugin code.
+- Native lane views now resolve verified immutable artifact manifests lazily, read only
+  requested blob/chunk ranges, and materialize only touched files during copy-up while
+  preserving shared FUSE, NFS, and Dokan upper/whiteout semantics.
+- Real-directory artifact consumers now reuse tree-root/backend-keyed verified
+  materialization caches that rebuild from authoritative CAS, restore immutable
+  permissions on reuse, and clone/reflink or independently copy into mutable state.
+- Lane forks now inherit only individually verified CAS-backed outputs after desired-key,
+  envelope/tree, current adapter package, scope, portability, and backend checks, while
+  allocating fresh artifact bindings and private workspace identities.
+- Portable backups now retain source uppers and authoritative artifact snapshots,
+  objects, envelopes, attestations, historical generations, and exact bindings while
+  reporting omitted materializations and performance caches as rebuildable.
+- Object GC now traces artifact envelopes through deterministic directory, file,
+  blob, chunk-list, and chunk edges from generation, attempt, snapshot, attestation,
+  quarantine, hold, layer, and materialization roots, then reclaims last-reference
+  content in restartable deterministic batches.
+- Lane-space and cache-GC reports now expose artifact logical, unique authoritative,
+  cross-artifact shared, materialized, lane-private, persisted-prefetch,
+  demand-loaded, reclaimable, and unknown byte accounting without counting a CAS
+  object more than once.
+- Object GC now orders unreachable artifact DAGs parent-before-child across
+  transaction batches, allowing an interrupted collection to reopen and resume
+  without leaving the remaining CAS graph invalid.
+- Artifact validations now distinguish structural, loadability, framework,
+  policy, gate, and reproducibility declarations and produce deterministic,
+  secret-rejected receipts bound to the exact desired identity and tree.
+- Workspace-layer publication now rechecks exact construction pins, freezes and
+  rescans Trail-owned candidate output, and requires structural and policy host-seal
+  receipts before a ready artifact envelope can be published or attached.
+- Artifact producers now use a host-selected phase/trust-tier capability ceiling for
+  reviewed built-ins, certified signed plugins, locally trusted plugins, and repository
+  declarations; signatures authenticate origin without implicitly elevating authority.
+- Secret-consuming artifact producers now carry typed non-secret taint evidence;
+  resolver candidates stay out of shared CAS, runtime-secret generations cannot promote
+  private output, and producer receipts are rejected if tainted or sensitive while
+  bounded failure evidence remains exact-value redacted.
+- Ready artifact envelopes now receive deterministic content-addressed host attestations
+  with typed producer, capability, policy, validation, portability, and taint evidence;
+  inspection and attachment verification detect state/signature tampering and recheck
+  current plugin package and publisher revocation.
+- Resolver plans now fail before attempt publication when paths, arguments, or declared
+  resource limits exceed host ceilings; native command-recipe tests also prove nested
+  child execution remains denied.
+- Repository environment parsing now recognizes an explicit `trail.environment/v2`
+  header without changing v1 command semantics, and rejects mixed schema versions across
+  one local include/profile graph.
+- Version-2 repository documents now retain typed resolver, action-phase, validation,
+  capability, heterogeneous-output, and source-export declarations with strict nested
+  unknown-field rejection; v1 documents cannot opt into those fields implicitly.
+- Repository v2 pipelines now compile into Trail's shared discovery, resolution,
+  component-graph, desired-key v2, output, validation, and source-export models instead
+  of introducing a parallel framework-specific execution representation.
+
 ### Changed
 
 - Changed omitted lane workdir mode to lazy qualified transparent `auto`.

Cargo.toml

@@ -62,6 +62,7 @@ axoupdater = { version = "0.10.0", features = ["blocking"] }
 bzip2 = "0.5"
 ed25519-dalek = "2.2"
 flate2 = "1.0"
+fastcdc = "4.0.1"
 rayon = "1.10"
 rocksdb = "0.22"
 reqwest = { version = "0.12.28", default-features = false, features = ["blocking", "rustls-tls"] }
@@ -100,6 +101,7 @@ terminal_size = "0.4.4"
 anstyle = "1.0.14"
 time = { version = "=0.3.36", features = ["formatting"] }
 walkdir = "2.5"
+xattr = "1.6"
 zip = { version = "0.6.6", default-features = false, features = ["bzip2", "deflate"] }
 uniffi = "=0.31.0"
 wasm-bindgen = "0.2.126"

docs/concepts/storage-indexes-and-backups.md

@@ -59,7 +59,13 @@ trail doctor
 trail fsck
 ```
 
-`doctor` checks operational readiness, schema version, current branch, `.trailignore` defaults, runtime integration state, and pending approvals. `fsck` verifies structural integrity.
+`doctor` checks operational readiness, schema version, current branch,
+`.trailignore` defaults, runtime integration state, pending approvals, and
+artifact/materialization health. `fsck` verifies structural integrity including
+raw artifact object identity and edges, resolution snapshots, envelopes,
+construction evidence, and owned versus orphan materializations. Reopening Trail
+recovers only staging owned by a provably dead exact process fence; unknown
+materializations are reported for review instead of deleted automatically.
 
 ## Backups
 
@@ -71,11 +77,21 @@ trail backup verify /tmp/trail-backup
 trail backup restore /tmp/trail-backup
 ```
 
-Restore rewrites materialized lane workdir paths so they point inside the restored workspace,
-re-secures the private `.trail` and `.trail/index` directories, and rotates the
-changed-path filesystem identity. The next daemon-backed command rebinds the
-observer to the restored host and reconciles the workspace before trusting its
-incremental ledger again.
+Backups retain source uppers and recovery journals plus authoritative artifact
+snapshots, objects, envelopes, attestations, historical generations, and exact
+generation bindings. They omit mounted projections, generated/scratch uppers,
+artifact materializations, and performance caches. Create, verify, and restore
+reports expose retained private bytes and the count/known bytes of omitted state
+as rebuildable. Verification seals the retained private tree by normalized path,
+entry type, symlink target, and file content.
+
+Restore rewrites lane workdir and retained-view paths so they point inside the
+restored workspace, re-secures the private `.trail` and `.trail/index`
+directories, retires copied active environment pointers, and rotates the
+changed-path filesystem identity. The next environment sync reconstructs
+materializations and caches from the retained authority. The next daemon-backed
+command rebinds the observer to the restored host and reconciles the workspace
+before trusting its incremental ledger again.
 
 ## Garbage Collection
 
@@ -86,6 +102,45 @@ trail gc --dry-run
 trail gc
 ```
 
+Object GC now understands artifact CAS graphs. It retains content reachable
+from generation bindings, layer shadows and pins, durable attempts and
+resolution snapshots, attestations, quarantines, active holds, and recorded
+materialization leases. It follows envelope, tree, directory, file, blob,
+chunk-list, and chunk edges, so a chunk shared by several artifacts is removed
+only after the last retained graph disappears.
+
+Collection is deterministic and restartable: unreachable artifact DAGs are
+ordered parent-before-child with object-ID tie breaking, and live deletion uses
+256-object transactions. Every committed batch leaves the uncollected graph
+valid, so an interrupted process can reopen and resume. Corrupt or ambiguous
+reachability stops the operation without treating missing evidence as
+permission to delete. Run
+the cache collector before object GC when you also want an unused verified
+materialization to stop retaining its reconstructible tree:
+
+```sh
+trail cache gc
+trail gc
+```
+
+`trail lane space` and the structured `trail cache gc` report include an
+`artifact_storage` object. It separates logical artifact content, authoritative
+CAS bytes unique to one envelope, CAS bytes shared across envelopes, physical
+materializations, lane-private allocation, demand-loaded projections,
+persisted prefetch allocation, reclaimable bytes, and allocation that Trail
+cannot safely attribute. These are multiple views of storage, not values to
+sum: reclaimable bytes can also be materialized or demand-loaded, and logical
+bytes are independent of both CAS encoding and filesystem allocation.
+
+Trail deduplicates authoritative bytes by object ID and logical bytes by tree
+root. Hot-set prefetch currently performs bounded reads into the operating
+system page cache and therefore reports zero persisted prefetch bytes. Native
+clone/reflink reports leave filesystem extents under `unknown_bytes` unless the
+platform can prove their ownership; they do not invent shared savings.
+
+Backup archives are self-contained and are created under the workspace write
+lock; they do not pin the source workspace after publication.
+
 ## Code Facts Used
 
 - Storage schema: `trail/src/db/storage/schema`

docs/design/data-model.md

@@ -199,7 +199,9 @@ Reports are deliberately not the same as storage rows. Examples:
 
 - `LaneReadinessReport` aggregates branch status, workdir state, approvals, conflicts, and gate status.
 - `LaneHandoffReport` bundles readiness, current session context, recent events, spans, operations, and next steps.
-- `BackupRestoreReport` includes restore effects such as rewritten workdirs.
+- Backup reports distinguish retained private view bytes from omitted
+  rebuildable materializations/performance caches; `BackupRestoreReport` also
+  reports rewritten workdirs and restored private views.
 - `GuardrailCheckReport` includes decision, reasons, path checks, approvals, and optional approval request instructions.
 
 This keeps command/API consumers from having to join internal tables themselves.

docs/design/environment-adapter-contract.md

@@ -59,6 +59,14 @@ reports logical component identity separately from adapter identity. CLI, HTTP/O
 MCP, and Rust APIs share this state. Existing `trail deps` behavior remains a Node
 compatibility surface.
 
+The Rust library also has a first common resolution-snapshot boundary:
+`resolve_artifact_component` and `resolve_all_artifact_components` accept normalized
+executor candidates, validate pinned source/tool/policy identity, reuse snapshots until
+an explicit refresh, and own durable attempt evidence and publication. This is not yet
+the public `trail env resolve` workflow: resolver process launch, provider integration,
+and CLI/HTTP/MCP operations remain planned. Discovery reports unsupported installed
+plugins from their pinned marker and package metadata without launching the plugin.
+
 Command recipes and v1/v2 plugins may declare stable logical component dependencies.
 `sync all` validates missing nodes, duplicate/self edges, complete cycles, and mount
 collisions before running a command; it then runs a bounded deterministic ready queue,
@@ -194,6 +202,31 @@ that needs semantic code can become an isolated plugin; a widely used, fully cer
 plugin can be proposed as a built-in. Component and adapter identities change explicitly
 when semantics change, so promotion never silently reinterprets an existing generation.
 
+Repository documents at `trail.environment.toml` or `.trail/environment.toml` now
+recognize both `trail.environment/v1` and the explicitly opted-in
+`trail.environment/v2` header. Version 1 retains its existing paths, defaults,
+includes/profiles, validation, and normalized identity. Every file in one include graph
+must declare the same version. The v2 header initially accepts the existing restricted
+command shape and adds strictly typed `component.resolve`, repeated `component.action`,
+repeated `component.validation`, `component.capabilities`, heterogeneous
+`component.output`, and repeated `component.source_export` sections. Resolver and action
+commands remain argv arrays. Network authority lists, candidate snapshot format/path,
+phase names, capability requests, validation requirements, output policies, and export
+collision intent are represented as separate fields. Unknown fields fail deserialization,
+and a v1 document cannot gain any of these authorities by spelling a v2-only section.
+Compilation and execution still pass through the host-owned normalized models and
+capability checks; parsing a declaration never grants authority by itself.
+
+The v2 compiler does not create a second repository execution graph. It projects one
+validated document into the existing `EnvironmentDiscoveredComponentReport`, optional
+`ArtifactResolutionPlanV1`, `WorkspaceEnvironmentPlan`, `ArtifactDesiredKeyMaterialV2`
+and desired key, `ArtifactOutputContractV2`, `ArtifactValidationV1`, and
+`ArtifactSourceExportContractV2` types. Exact executable identities, selected pinned
+inputs, normalized output policy, platform identity, policy identity, and requested
+network authorities therefore participate in the same host-owned reports and identities
+as built-in and plugin adapters. A declared resolver produces a `resolvable` proposal;
+it does not run during discovery or compilation.
+
 ### Add a repository adapter without Rust
 
 Use `trail/command@1` when the adapter can be represented as pinned byte inputs, an argv
@@ -319,6 +352,14 @@ copy, while removal remains available for recovery. Trust revocation also fails
 packages closed immediately. Unsigned packages are visibly `local-experimental`; a
 signature authenticates origin but does not grant stable certification.
 
+The host classifies current publisher-authenticated experimental packages and unsigned
+local packages as `locally_trusted_plugin` for execution. Only a separate durable
+conformance result may select `certified_signed_plugin`. That host-selected tier is
+combined with the action phase to produce `ArtifactCapabilityCeilingV1`; the normalized
+plan is rejected if its inferred reads, writes, caches, processes, provider use, mounted
+execution, network, or secret channels exceed that ceiling. No tier can request direct
+publication authority.
+
 The adapter catalog reports each package's planner protocols, supported operating systems,
 and architectures. Unsupported plugins remain inspectable but are not auto-discovered or
 executed on the current host. `lane readiness` replans every installed component from
@@ -902,6 +943,30 @@ Adapters declare which command argument or output fields may echo credentials so
 host can add structured redaction. This declaration supplements, but never replaces,
 host-wide exact-value redaction and secret scanning.
 
+Receiving a secret value taints the producer output; a non-secret provider/version
+reference alone does not. Taint metadata contains only canonical channel classes, never
+provider names, handles, paths, or values. Tainted candidates may remain in lane-private
+storage, but the host rejects resolution-snapshot publication, shared artifact sealing,
+private-output promotion, and source export. Bounded attempt evidence is exact-value
+redacted before storage. In the current resolver API a declared credential handle cannot
+be distinguished from one actually consumed, so shared resolution fails closed. A future
+private resolver-output store may preserve such output without weakening this rule.
+
+Resolver declarations are also capped by host ceilings before an attempt record or
+subprocess can be created: one hour of wall time, 16 MiB for each captured stream,
+1 GiB and 1,000,000 entries for the candidate, and 256 child processes. Declared limits
+must be non-zero and may narrow these ceilings, never expand them. Repository-relative
+path and control-character validation runs in the same pre-attempt boundary.
+
+Adapters do not author or publish artifact attestations. After host sealing, Trail creates
+the deterministic attestation from validated host evidence and stores it as a separate
+content-addressed object so signatures and trust observations do not alter artifact tree
+identity. Attachment requires an exact statement/envelope match and current producer
+trust. For plugin producers, the installed distribution digest, publisher, and publisher
+key must still match; tombstoned packages and removed publisher trust fail closed.
+Optional signatures cover the canonical attestation statement and are never interpreted
+as adapter certification or publication authority.
+
 ## Plugin protocol and capabilities
 
 The implemented v1/v2 transport is a natively sandboxed subprocess using one