test(knowledge): quarantine the RAG pipeline ingest test for #1667

[Victor-w-Madeira]

Quarantines Full RAG pipeline grounds the model answer on the retrieved chunk as prevention, from the triage of daily-failure #1665 (run 33511210195, 2026-09-01). It is not a fix — the investigation is #1667.

Why

The test flaked with the same signature it carried on 2026-08-18 — expect(locator).toBeVisible() failed — and the call log is specific about which way:

Locator: locator('[data-id="Knowledge-ingest"]').locator('[data-testid^="node_duration"]')
Expected: visible
Timeout: 90000ms
Error: element(s) not found

The Knowledge (Ingest) node never rendered a duration badge, so the ingest never reported completing and the test never reached the answer path it is named for. Four occurrences in reports/daily-history.jsonl since 2026-07-16 (07-16, 07-22, 08-18, 09-01), every one recovering on retry, which is why it has never gone hard.

Why it is filed on an environmental day

Daily 2026-09-01 tripped the mass-failure guard (8 hard failures against a threshold of 5) and the triage's verdict for the day is environmental — 4/4 shards needing the post-sweep health gate (91–110 s), 19 outages, 25.4 min of measured unreachable backend. The guard governs hard failures only, and the flake criterion has no guard-day exemption (CONTRIBUTING.md → Triage protocol, step 2).

The day's verdict does not cover this test either way. Attempt 0 ran 13:11:52 → 13:13:30 UTC on shard 3, where the in-run liveness recorder measured 1 of 91 probes down and no probe slower than 2 s. It is the only failing attempt of the entire run on a demonstrably healthy backend.

One caveat carried into #1667 rather than assumed here: the recorded signature is the generic expect(locator).toBeVisible() failed, which #1626 documents as too coarse to distinguish two locators inside one spec. The recurrence claim rests on the second shared dimension — same test, same step — and #1667 is directed to confirm the 2026-08-18 locator from that run's Playwright JSON.

The change

Both edits together, per CONTRIBUTING.md → Quarantine mechanism:

• @stable removed — Phase 0 of QA-CHECKLIST.md is generated from @stable test() calls, so leaving it would keep counting a quarantined test as validated.
• test.fixme added — @stable removal alone only stops the daily; the test would keep going red on the pr-validation.yml impacted-specs gate, which selects by file diff rather than by tag (Triage quarantine PRs go red on the impacted-specs gate — quarantine via test.fixme, not @stable removal alone #871). test.fixme skips it in every context.

@release @components @files are kept — the test is quarantined, not reclassified.

No QA-CHECKLIST.md change: the spec still has a doc under docs/, so its manual Part II bullet must stay, and the generated blocks are regenerated on merge to main.

Verification

• npm run typecheck — clean
• npm run lint — 0 errors
• node scripts/check-checklist-guard.mjs — no generated block edited
• npm run check:checklist-coverage — every @stable and documented spec still referenced by a Part II bullet
• npx playwright test <spec> --grep @stable --list → Total: 0 tests, and the unfiltered list shows the test present but not executed

Lifting the quarantine — remove test.fixme + restore @stable, re-validated per CONTRIBUTING.md — is a deliverable of #1667.

Refs #1667, #1665

[Copilot]

Pull request overview

This PR quarantines the flaky RAG pipeline ingest test in the Langflow E2E regression suite to prevent repeated noise in CI while #1667 investigates the underlying failure.

Changes:

• Converted the affected test to test.fixme(...) so it is skipped in all run contexts.
• Removed @stable from the test tags so it no longer counts toward the generated “Phase 0 — Validated” list.
• Added an in-file quarantine rationale comment referencing the originating daily triage and follow-up issue.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.