Skip to content

test: eliminate intermittent macOS gtest-discovery build failure - #556

Merged
gmarzot merged 1 commit into
mainfrom
fix/gtest-discovery-pre-test
Aug 11, 2026
Merged

test: eliminate intermittent macOS gtest-discovery build failure#556
gmarzot merged 1 commit into
mainfrom
fix/gtest-discovery-pre-test

Conversation

@gmarzot

@gmarzot gmarzot commented Aug 10, 2026

Copy link
Copy Markdown
Contributor

Problem

The macos job intermittently fails at build time with:

CMake Error at /opt/homebrew/share/cmake/Modules/GoogleTest/ParseTestList.cmake:96 (string):
  string sub-command JSON failed parsing json string:
  * Line 1, Column 1
    Syntax error: value, object or array expected.

(e.g. this run on #533 — passed on rerun, unrelated to the PR's changes.)

Root cause

CMake 4.4.0 (current Homebrew) has a regression in the GoogleTest module refactor: the generated *_discovery.cmake scripts omit TEST_TARGET, so the per-target hash that keys the discovery JSON file is computed from the empty string. Every test target in build/test therefore shares one file, cmake_test_discovery_e3b0c44298.json (sha256("")), defeating the race protection that hash exists for.

In default POST_BUILD mode, discovery runs right after each test binary links — and Ninja links many test binaries near-simultaneously (the failing run linked three within 0.2 s). Concurrent discoveries then race on the shared file: one truncates/removes it while another parses → empty JSON → the error above. Intermittent by nature; the losing target varies. Fixed upstream in cmake 4.4.1, which Homebrew doesn't ship yet.

Fix

Switch discovery to PRE_TEST mode (one set() before the gtest_discover_tests() calls). Discovery then runs serially at ctest startup — immune to the race on any cmake version.

Cost

  • Build step: slightly faster — the ~28 post-link discovery executions disappear from the build.
  • Test step: +0.8 s measured to enumerate all 772 tests on first ctest invocation; cached via an IS_NEWER_THAN guard thereafter (measured 0.12 s), re-discovering only binaries that actually relinked.

Validation (local, linux, cmake 4.2.3)

  • Reconfigured + full build: discovery steps gone from the build graph; generated *_include.cmake files now carry the guarded gtest_discover_tests_impl call.
  • ctest -N: all 772 tests enumerated.
  • Targeted run of 306 tests: all pass. (UpstreamProviderTest network tests fail/hang on this dev box with or without the change — pre-existing local-env issue; CI is the gate for those.)

Note for #519: moqx_add_gtest() there still uses default POST_BUILD discovery, so this line should survive the rework (or move into the helper as DISCOVERY_MODE PRE_TEST).


This change is Reviewable

@michalhosna michalhosna left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@michalhosna made 1 comment.
Reviewable status: 0 of 1 files reviewed, 1 unresolved discussion (waiting on gmarzot).


-- commits line 14 at r1:
Homebrew now ships 4.4.2 (See https://formulae.brew.sh/formula/cmake).

So, is this still a problem?

@afrind afrind left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with this, but maybe it's not actually going to fix it?

@afrind reviewed 1 file and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on michalhosna).


-- commits line 14 at r1:

Previously, michalhosna (Michal Hošna) wrote…

Homebrew now ships 4.4.2 (See https://formulae.brew.sh/formula/cmake).

So, is this still a problem?

What's on our CI runner?

@gmarzot

gmarzot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

I'm ok with this, but maybe it's not actually going to fix it?

@afrind reviewed 1 file and all commit messages, and made 2 comments.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on michalhosna).

-- commits line 14 at r1:

Previously, michalhosna (Michal Hošna) wrote…

Homebrew now ships 4.4.2 (See https://formulae.brew.sh/formula/cmake).
So, is this still a problem?

What's on our CI runner?

I am reading now that this problem will fade out in a few weeks. the runners are mixed and the bad version can still crop up.

the restructure is slightly better even after 4.4.2 roles out... conserves runner cycles.. but doesnt really translate into any observable build perf benefit ~1sec

we can close or merge.. you guys decide? eh i see a check mark.. will merge

@gmarzot gmarzot self-assigned this Aug 11, 2026
@gmarzot

gmarzot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor Author

we should not see it again w/ this change and some minor optimization bonus

@gmarzot gmarzot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

@gmarzot made 1 comment.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on afrind and michalhosna).

CMake 4.4.0's generated discovery scripts omit TEST_TARGET, so every
target's POST_BUILD discovery shares one JSON file
(cmake_test_discovery_e3b0c44298.json, sha256 of the empty string).
Ninja links test binaries in parallel and their discovery steps race on
that file: one truncates it while another parses, producing the
intermittent macOS failure

  ParseTestList.cmake:96: string sub-command JSON failed parsing json
  string ... Line 1, Column 1: Syntax error

Fixed upstream in cmake 4.4.1, but Homebrew CI runners ship 4.4.0.
PRE_TEST discovery runs serially at ctest startup, immune to the race
on any cmake version. Measured cost: 0.8s to enumerate all 772 tests,
cached thereafter (0.12s); build step drops its ~28 post-link discovery
executions.
@gmarzot
gmarzot force-pushed the fix/gtest-discovery-pre-test branch from 6c89931 to 1619dbb Compare August 11, 2026 12:27

@gmarzot gmarzot left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gmarzot resolved 1 discussion.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on gmarzot).

@gmarzot
gmarzot merged commit 6fc9473 into main Aug 11, 2026
16 checks passed
@gmarzot
gmarzot deleted the fix/gtest-discovery-pre-test branch August 11, 2026 15:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants