ci(vcpkg): Fix recurring binary cache invalidation from runner image updates#2371
ci(vcpkg): Fix recurring binary cache invalidation from runner image updates#2371bobtista wants to merge 3 commits intoTheSuperHackers:mainfrom
Conversation
|
| Filename | Overview |
|---|---|
| triplets/x86-windows.cmake | Created custom x86-windows triplet with VCPKG_DISABLE_COMPILER_TRACKING to prevent weekly MSVC updates from invalidating the binary cache |
| .github/workflows/build-toolchain.yml | Removed MSVC version tracking from cache key, bumped to v3, added triplet hash tracking, and configured VCPKG_OVERLAY_TRIPLETS and VCPKG_INSTALL_OPTIONS |
| CMakePresets.json | Added VCPKG_OVERLAY_TRIPLETS to default-vcpkg preset to ensure custom triplet is used in both CI and local builds |
Last reviewed commit: 7eb4dca
|
What is triplets? The build time still shows with 23 minutes. |
A vcpkg triplet is a config file that tells vcpkg how to build packages (32 vs 64-bit, static vs dynamic linking, etc). We add a custom one that also tells vcpkg to ignore compiler version changes when caching. GitHub's runner images update MSVC weekly. vcpkg includes the exact compiler version in its cache hash by default. So every week the compiler version bumps, the hash changes, the cached binaries don't match, and ffmpeg gets rebuilt from source (~17 min) even though nothing in our code or dependencies changed. Disabling compiler tracking stops that cycle. The build is like 6 minutes - I only re-ran one of the vcpkg jobs, I can run all of them again now though |
| # Exclude compiler version from ABI hash so that weekly GitHub runner image | ||
| # updates don't invalidate the binary cache. Minor MSVC version bumps do not | ||
| # cause ABI incompatibilities for this project. | ||
| set(VCPKG_DISABLE_COMPILER_TRACKING ON) |
There was a problem hiding this comment.
Can this possibly cause other issues in the future? If the binary cache is invalidated once a week, then why does it not keep the new binary cache after 1 build? Does that not need looking into?
Do we perhaps need a CI trigger to manually force purge and refresh the cache in case it is accidentially broken?
Do we know why exactly the cache is broken right now?
There was a problem hiding this comment.
There are two separate layers of caching.
- GitHub Actions cache — stores/restores a directory of files based on a key we define in the workflow. It just zips up the folder and gives it back when the key matches. It knows nothing about vcpkg.
- vcpkg binary cache — inside that directory, each package is a .zip file named by vcpkg's own ABI hash (which includes the exact compiler version like 14.44.35207). When vcpkg runs, it computes its hash and looks for a matching .zip in the directory.
So the problem was GitHub Actions says: "key v2-msvc14.44-... matches, here's your directory" and says cach-hit was true. Then vcpkg opens that directory, computes its hash with the full compiler version 14.44.35207, finds no matching .zip files (they were built with 14.43.xxxxx) and re-builds ffmpeg which takes forever (17 min). Because cache-hit was true as far as github is concerned, it doesn't save the new build. This fix removes the compiler version from vcpkg's hash, so both layers should stay in sync.
RE future issues, MSVC maintains ABI compatibility within the same major version (14.x), and the cache still invalidates when the vcpkg baseline or triplet config changes. We'd be very unlikely to have no other changes before the next major MSVC version - and yes we can always just bump the version in build-toolchain.yml to force purge and refresh the cache.
Summary
VCPKG_DISABLE_COMPILER_TRACKINGso weekly MSVC version bumps on GitHub-hosted runners don't invalidate the vcpkg binary cache--x-abi-tools-use-exact-versionsto pin tool versions from vcpkg's own manifest instead of using system-installed versionsv3to flush the current stale cache and remove MSVC version from the key since compiler tracking is now disabledVCPKG_OVERLAY_TRIPLETSto CMakePresets.json so the custom triplet is used in both CI and local buildsTest plan
v3key prefix), build from source, and save cache