Skip to content

simplify(search-eval): keep one canonical corpus tree instead of a separately maintained packaged mirror #4449

Closed

Description

@ScriptedAlchemy

Priority / target

P3 · build/data ownership consolidation · medium reproducibility risk. Preserve the packaged search evaluator and its exact runtime qualification corpus. Remove the second checked-in copy of the same fixture bytes and the associated source/package synchronization burden.

Evidence

Audited at 1bf9f0819465b2720241b20057510ff01dfa633a:

  • search_quality/packaged.rs manually maps logical root-relative fixture paths to include_bytes! from crates/tracedecay-query/assets/runtime-root/.... Its runtime loader validates the semantic workload pin and quoted need provenance against those embedded bytes. This is real production qualification material, not disposable test noise.
  • The root corpus and packaged tree both contain the same inspected time.rs fixture and packaged time.rs copy. Other paths in the explicit FILES list follow the same mirrored layout. This audit verified that example, not byte equality of every file in both trees.
  • candidate_output.rs defines workload and corpus identities using the original logical paths. Those identities need not require two physical source trees.
  • The generated tracedecay-query/BUILD.bazel already declares asset compile inputs. .gitattributes separately enforces LF on the packaged mirror; tracedecay-search-eval/src/report_tests.rs points maintainers to its workload copy for receipt updates.

Exact implementation

  1. Inventory the FILES list against root fixtures and record exact byte differences and every consumer. Preserve genuinely distinct/versioned fixtures as distinct assets; do not overwrite a golden snapshot with current application source or declare all similarly named files duplicates.
  2. Choose one canonical location for each shared workload/corpus byte sequence. Make both source-tree tests and compile-time embedding consume it through existing Bazel data/compile_data and fixture helpers. A generated build-output staging layout is acceptable where a tool needs physical paths; a second editable checked-in mirror is not.
  3. Preserve logical corpus paths, source commit/tree provenance, content digests, quoted need provenance, labels and train/validation boundaries exactly. Physical relocation alone must not change WORKLOAD_SHA256, ranking receipts, query labels or qualification outcome. Never re-pin receipts just to hide an accidental byte change.
  4. Update the existing build generator rather than hand-editing BUILD.bazel. Keep native Windows line endings/path handling, sandboxed/hermetic builds, and installed evaluator behavior outside a repository checkout. Do not add a new asset manifest authority, copy daemon, archive download or build-time network source.
  5. Delete redundant committed copies, their copy/synchronization steps if present, duplicate path policies and tests whose only purpose was keeping two physical mirrors synchronized. Keep a narrow real release smoke proving the installed binary contains the required immutable corpus.
  6. Source-crate distribution is not a reason to preserve hypothetical layout reconstruction: Remove the unpublished-crate distribution battery and retain real release smoke coverage #4218 is already closed as completed. If a currently supported distributable genuinely requires staging, produce it from the canonical bytes at packaging time through its existing owner.

Acceptance

The complete source-backed and installed packaged evaluator runs see identical corpus bytes and logical identities, including from a temporary directory without the checkout. Existing workload, provenance and fallback-receipt validation still reject corruption. Windows/Linux builds remain hermetic. Show the exact deleted mirror files and unchanged semantic pins; do not claim binary-size reduction since the runtime still needs the corpus. Run owning search-eval/build tests and the actual packaged smoke with nonzero behavioral evidence.

This is a residual source-asset ownership cleanup, not a repeat of #4218's deleted distribution battery and not a proposal to remove runtime evaluation. Static inspection and one matching-file comparison only; no full corpus equality check, Rust build or evaluator run was performed while filing.

Activity

  1. ScriptedAlchemy commented on Oct 30, 2026

    @ScriptedAlchemy
    OwnerAuthor

    Closing as completed by #4530, restored onto master by #4587 after the generated release branch dropped the original stack commits.

    Verified current master 64a04ef: crates/tracedecay-query/src/search_quality/packaged.rs embeds the canonical repository fixtures directly and retains workload digest sha256:b40c4c470dc2afae55debd2930d39f9f5d56494afd8a010672fb53bceb0295ec. #4530 removed 15 byte-identical mirrors, moved the workload without rewriting its bytes, and updated the maintained Bazel generator and LF policy.

    Recorded validation includes 28 evaluator tests, 15 search-quality tests and 8 generator tests, with source/packaged corpus, provenance and fallback-receipt parity. This closes duplicate asset ownership, not the evaluator's separate conceptual search-quality misses, and makes no binary-size or new-test-run claim.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions