Skip to content

CI: investigate nondeterministic Stage C /preview/ screenshot parity on identical Git tree #778

Open

Description

@masarray

Observed nondeterministic Stage C screenshot parity on byte-identical source

Evidence (2026-11-18, after MV-3 #777 merged):

  • Same content tree SHA adbe35762b3040644f2bf7bfbfdb77d001ea8f6b for feature HEAD 1e1162c5f653946d7520ef0d0961a612720b98ad and squash main HEAD 3307c4bbb141f2a045ae42a83d7f3b6cd477657a. Git trees were fetched from GitHub and compared, not assumed.
  • Exact-tree pre-merge Full Chromium #44235017840: SUCCESS 109/109 steps; Stage C passed.
  • Main push UI run #44236089782, attempt 1: FAILURE at Verify Stage C shipped CSS versus original-source computed and screenshot parity. tests/approved_stage_c_css_visual_parity_smoke.py reported AssertionError: ('approved GUI pixel screenshot changed', (1366,768), 'default', 'a34d453d40c94ab1beb50572d9bec65edc6756ed3d86e2351f389261a8485952', '1e980c31fb0de759eec773b746394efeec7e7aef3ce324a1e51066c3b0b638a5'). The computed CSS equality assertion just before the PNG assertion did not fail.
  • Single controlled rerun of the failed Chromium job on exact same main SHA, no source/YAML/baseline edits: attempt 2 passed the Stage C step and proceeded to later WIRE/UI checks. Final run conclusion to be collected separately. Do not treat repeat success as a diagnosis of pixel cause.
  • Fast UI/Fast Native and required gate passed; Windows MSVC/CTest (automatically triggered) independent. No mixer/DSP audio claim follows from screenshot evidence.

Goal: Investigate and eliminate the intermittent visual comparison failure without weakening parity assurance.

Suggested bounded investigation:

  1. On failure, preserve both original and compiled screenshots, a pixel-diff diagnostic, computed style evidence and browser/runtime versions as run artifacts; preserve strict failure exit status.
  2. Determine whether asynchronous JS requestAnimationFrame, font/image paint readiness, transient hover/tooltip focus, or actual CSS compilation creates the difference. Today render() waits for font readiness and fixed 100ms/130ms; that is not an evidence-based stable-frame barrier.
  3. Add a deterministic visual-settle condition scoped to the test (if proven), with negative fixtures that deliberately change CSS and still FAIL. Never merely increase tolerated pixel mismatch, hide CI failure, or skip Stage C.
  4. Measure failure frequency on exact tree and runner image before/after. If changes are needed, use one CI-only PR; do not mix with MVVM Device/FX/WIRE/native graph changes.

Owner impact: GUI source refactor #777 is merged; this is an independent regression-test reliability follow-up, not a claim that new /preview/ visuals changed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions