Skip to content

[CI] Give the Clang producer cache one narrow build owner #170

Description

@samchon

Problem

The experiment and index-time workflows key the pinned Clang producer cache by
hashing the complete tests/experiment/src/catalog.mjs and
tests/experiment/src/setup-language.mjs files. Most changes in those files
cannot affect the LLVM build: a Java catalog row, a LuaLS digest, or a comment
in another language's installer nevertheless invalidates the cache and makes
both C and C++ lanes rebuild LLVM.

The cost is material. A cold producer build has taken 56 to 107 minutes. The C
and C++ jobs may also miss the same new key concurrently, build identical
bytes twice, and race to reserve one cache entry; one full build is then
discarded.

The current broad key was chosen to prevent a different failure: a key over
only CPP_CLANG_PRODUCER_COMMIT.ts could hit while the catalog/setup recipe
points at different bytes, fail the version check, rebuild, and never save the
replacement. Narrowing the key without giving the build recipe one owner would
reintroduce that permanent miss.

Consequence surface

  • C/C++ real-provider experiments and Redis/LevelDB index-time measurements.
  • Producer pin equality between the TypeScript adapter, experiment catalog,
    setup recipe, cache restore/save, and recorded tool manifest.
  • Cold-miss concurrency and the evidence available when a later corpus step
    fails.

Approach

Extract the Clang producer's repository, commit, build inputs, and provisioning
recipe into a dedicated source consumed by setup and mechanically checked
against the adapter constant. Hash only that owner and the adapter pin. Arrange
each workflow so one cold key is prepared before both C-family consumers read
it; a later experiment failure must not discard a successfully built producer.

Acceptance and verification

  • Editing an unrelated language's catalog/setup data leaves the Clang cache key
    unchanged.
  • Changing the producer commit or any byte that affects its build changes the
    key.
  • Catalog, setup, adapter, version check, and tool manifest cannot disagree on
    the producer generation.
  • One workflow execution performs at most one cold build for the shared C/C++
    producer, and both lanes restore the same verified result.
  • A producer build is saved before a later corpus assertion can fail.
  • Deterministic workflow tests cover unrelated edits, relevant edits,
    divergence refusal, cold miss, cache hit, and failed consumer steps.
  • pnpm --filter @samchon/graph-benchmark test, focused workflow tests,
    pnpm build, pnpm test, and pnpm coverage pass.

Coordination

Do not weaken the exact commit pin or use a mutable release channel. This issue
changes build ownership and cache invalidation, not the measured product or the
published benchmark cells.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions