Skip to content

fix(auth): isolate capsule verification and page catalogs - #583

Closed
forhappy wants to merge 1 commit into
codex/xorb-chunk-count-limitfrom
codex/protected-catalog-lookup
Closed

forhappy wants to merge 1 commit into
codex/xorb-chunk-count-limitfrom
codex/protected-catalog-lookup

Conversation

@forhappy

Copy link
Copy Markdown
Contributor

Protected capsule receive materialized complete Xet catalogs, rejected checkpoint-backed Git workspaces, and could satisfy a scoped candidate's omitted Git blob from the private source repository. The latter was reproduced by a candidate that declared an allowed-path file but omitted its source-only blob body: verification incorrectly succeeded.

Validate scoped candidates using only filtered and incoming objects before installing the private source packs. Scoped and unrestricted capsule receive now share one graph/visibility verifier. Newly reachable objects must belong to the captured visible-ref union or the candidate's verified pack indexes; retained physical objects do not grant access. The admission union uses raw OIDs and avoids per-object scans across every ref. Existing changed-path policy and source-commit synthesis remain in their owners.

Receive also captures paged source and filtered catalog views separately. After Git proof it selects outgoing file bindings plus candidate shard/xorb identities under each authority, then applies the candidate delta. File-only deltas can reuse historical filtered artifacts. Source-first file selection, immutable identity conflicts, physical payload verification and copy-before-ref promotion remain intact. Prepare reads only ref authority.

The shared origin-backed installer now accepts candidate capsules together with captured packs, under the same pack intake limit and external-delta ordering. Candidate-free callers delegate to this implementation and retain cold-clone/cache behavior. Installing candidate packs never grants complete visible-closure authorization.

Stacked on #580 (codex/xorb-chunk-count-limit), which fixes the independently reproduced 8,192-chunk writer/reader mismatch found by this qualification. No format, dependency or configuration changes here.

Validation:

  • All 81 receive tests pass on the final implementation, including unrestricted staged-Xet publication and retries.
  • Four scoped workflow cases vary 100/10,000 real historical files and run/checkpoint storage. Each measured verification stays within 64 origin reads / 512 KiB. Recipes, shard bodies and xorb bodies are real, including the split required by fix(xet): split xorbs at the protocol chunk limit #580.
  • These cases preserve hidden source-only text/Xet content, apply a file-only delta referencing historical filtered artifacts, promote Xet/LFS dependencies, commit and retry, verify catalog objects, and hydrate both selected files byte-for-byte.
  • The source-only omitted-blob regression fails before the authority fix and passes after it. A separate checkpointed retained-but-invisible blob case is rejected without moving the source ref.
  • The original checkpointed workflow fails before the installer change with layered checkpoint sources require object-store-backed installation.
  • All four external-delta/installer tests pass on the current source: combined intake rejection when base and candidate fit individually, thin-delta reconstruction, missing/cyclic-base rejection, and pre-cancellation without I/O.
  • All three native pack-cache sibling tests pass: reuse, corruption rejection and cancellation cleanup.
  • Strict all-target selected-package Clippy passed for read/remote/auth. The final auth-only refresh after the authority fix also passes; whole-workspace formatting and diff whitespace checks pass.

Evidence map:

  • Entry points: verify_receive and commit re-verification call verify_capsule_candidate in crates/crab-auth-server/src/receive/capsule.rs.
  • Owner boundary: receive owns source/filtered authorization, graph admission and candidate selection; metadata authenticates selected dependency closure; read owns origin-backed pack intake; write retains ref publication.
  • Callees: verify_capsule_candidate_refs checks actual bounded Git reachability against captured visibility and incoming index membership. The native workspace first installs and validates only filtered/incoming packs. select_pointer_catalog calls the captured PointerCatalogLookup::dependencies; existing source delta and content-proof functions retain their policy.
  • Siblings: both capsule receive paths share the new proof. Manifest receive regression tests pass, but equivalent private-source-object isolation in the v1 manifest workflow still needs a separate audit. Candidate-free native cache installation has explicit sibling proof. CLI/shared publication already use selected catalogs; current-state hydration still needs migration. Full maintenance remains explicit.
  • Current main: fetched origin/main at cc2e4e25f46eeaee3cfd30025d257397c17a6049 lacks this capsule receive module. The stacked parent uses full catalog views and the resident-only installer; this PR depends on the preceding protocol series. These regressions concern that unreleased capsule path.

Is this the best fix? It removes full catalog reconstruction from this foreground consumer, isolates candidate proof before broader source access, reuses one graph verifier and one origin-backed installer, and retains separate captured authorities. Approximately 159 net production lines cover key selection, candidate intake and the missing authority proof; the larger diff is qualification coverage.

Remaining gates: deployed protected-receive/S3 qualification, publication/GC races, abandoned-operation cleanup across the full receive lifecycle, 10 GiB multi-edit integration, v1 sibling isolation audit and hosted-S3 latency. Existing receive APIs do not expose caller cancellation; installer cancellation tests do not establish cancellation of every workflow phase. The separate cluster immutable-write-denial qualification failure remains open. No merge or production rollout is included.

The two large-file audit reports and unrelated advisor-plan edits are excluded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant