Skip to content

BLOCKER: Strix full self-test baseline diverges after b50e4f7 #571

Open

Description

@seonghobae

증거\n\nA full local run of scripts/ci/test_strix_quick_gate.sh from main commit dd078cc plus only the executable-permission fix ran for about 58 minutes and exposed pre-existing baseline divergences introduced by or visible after b50e4f7. It then ended 137 under memory pressure.\n\nObserved independent failures:\n\n- nonrecoverable expected the generic non-recoverable message, while the gate selected the stricter provider infrastructure or failure-signal branch.\n- provider-prefix-required-resource-path-primary-explicit-empty-default-provider still expects implicit vertex_ai normalization, while current policy rejects resource paths without an explicit Vertex provider.\n- two unchanged-file baseline scenarios expect a later successful fallback to continue, while the new success-with-threshold-report check fails before PR changed-file baseline attribution can permit continuation.\n- vertex-primary-hallucinated-endpoint-fallback-success expects a generic terminal message, while current PR attribution fails closed earlier.\n\n## 범위 경계\n\nThis blocker is separate from #569. Do not weaken executable integrity to resolve it. Reconcile the intended policy order and update both implementation and scenario expectations together.\n\n## 완료 기준\n\n- Each failing scenario is reproducible through STRIX_TEST_CASE_FILTER.\n- Policy precedence for successful process plus threshold report versus unchanged-file PR baseline is explicit.\n- The full self-test completes without baseline assertion failures or resource exhaustion.\n\nAgent: Codex\nRecorded: 2026-07-17T01:42:15+09:00\nPhase: Ops

Activity

  1. added
    area: apiAPI, protocol, event, or external contract
    area: authAuthentication, authorization, identity, or tenant isolation
    area: performancePerformance, resource use, scalability, or benchmarking
    priority: criticalImmediate blocker, P0, urgent deadlock, or critical incident
    status: triagedOpen issue has an organization taxonomy assignment
    type: featureNew or expanded product capability
    on Aug 22, 2026
  2. moved this from Todo to In Progress in naruon Platform Roadmapon Aug 25, 2026
  3. seonghobae commented on Aug 25, 2026

    @seonghobae
    ContributorAuthor

    Codex pickup (2026-08-25 10:22 +0900): current central main commit a724582 changed Strix fallback models from gpt-5.6-luna to gpt-5.4 in .github/workflows/strix.yml, but scripts/ci/strix_required_workflow_smoke.sh still asserts the retired gpt-5.6-luna contract. This exact smoke mismatch fails every consumer Strix check (observed on ContextualWisdomLab/disksage#247 exact head a9c868a6e9c8d68a9c6ea6de381e188740b8f5db). I am reconciling the workflow/smoke contract with a focused test before considering the blocker resolved.

  4. seonghobae commented on Aug 25, 2026

    @seonghobae
    ContributorAuthor

    Clarification (2026-08-25): PR #1316 tracks the separate trusted-smoke/model-contract mismatch in new issue #1317. This issue #571 remains scoped to the independent full self-test baseline divergences and is not being closed by that PR.

  5. added
    bugSomething isn't working
    type: bugDefect or incorrect behavior
    on Sep 7, 2026
  6. seonghobae commented on Sep 26, 2026

    @seonghobae
    ContributorAuthor

    Fresh protected-branch reproduction (2026-09-27 evidence): a clean detached worktree at protected main@e6334e229581a918e2f22de18733b76fa65d7e71 ran scripts/ci/test_strix_quick_gate.sh and failed immediately because the self-test still requires TARGET_REPOSITORY: ${{ needs.validate-pr-metadata.outputs.target_repository }}, which the current OpenCode workflow no longer contains. The 12-second bounded probe exited 124 only because the monolithic suite kept running after recording the failure.

    A longer run on the #2382/#2385 convergence tree reproduced the same inherited assertion and then many fixture failures where temporary repositories omit scripts/ci/strix_evidence_binding.py; the gate exits 2 before scenario-specific expectations. The same first failure on protected main proves this is not introduced by either PR's delta.

    No expectation was weakened and no leaf workaround was added. This remains a canonical .github baseline defect: update the self-test fixtures and asserted workflow shape together with the intended trust boundary, then require the full script to finish GREEN without resource exhaustion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area: apiAPI, protocol, event, or external contractarea: authAuthentication, authorization, identity, or tenant isolationarea: performancePerformance, resource use, scalability, or benchmarkingbugSomething isn't workingpriority: criticalImmediate blocker, P0, urgent deadlock, or critical incidentstatus: triagedOpen issue has an organization taxonomy assignmenttype: bugDefect or incorrect behaviortype: featureNew or expanded product capability

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions