fix(llm-agents): free the canvas bottom overlay before the memory inspect click (#1643)

[Victor-w-Madeira]

Closes #1643.

Problem

Both context-id retrieval specs hard-failed all 3 attempts of the 2026-08-31 daily (run 33410643882, triage #1642) on a 20 s locator.click timeout against output-inspection-messages-memory — the button resolving as visible, enabled and stable and every retry refused by <div class="absolute bottom-16 left-1/2 z-50 w-[530px]">… subtree intercepts pointer events. Quarantined by hand in #1647 because the mass-failure guard tripped and the workflow auto-removed nothing.

Root cause (test defect — confirmed live, not a product regression). Langflow renders two different components into one fixed container over the canvas, absolute bottom-16 left-1/2 z-50 w-[530px] -translate-x-1/2:

Component	Behaviour in the slot
flowBuildingComponent — the build-status bar	transient on success: handleDismiss fires 2 s after "Flow built successfully"
UpdateAllComponents — the "Flow needs review / N components need updates" banner	its shouldHide includes buildInfo?.success, so it is hidden while the bar is up, retakes the slot when the bar's state is cleared, and then never leaves

Measured at the default 1280×720 viewport, the inspect button's box is y 585.6–601.0, centre 593.3. The build bar's top edge is y 598 — the click clears it by ~5 px. The banner is 12 px taller, top edge y ~586 — above the centre. So the identical click passed or was refused purely on which component owned the slot, and once the banner owned it no amount of retrying could help, which is exactly what the call log shows: first interceptor the bar's inner flex min-h-10 w-full items-center justify-between gap-2, second the banner's flex items-center justify-between gap-6 rounded-lg border bg-background px-4 py-3 text-sm shadow-md.

The banner is present because these specs seed their flow from tests/assets/flows/chat-io-ok-trace-fixture.json, whose nodes carry lf_version: 1.7.0; the nightly reports one of them as outdated.

Reproduced deterministically: dwelling 4 s before the click hands the slot to the banner, and the failure returns with the interceptor byte-identical to the daily's. With the fix the same dwell passes — probe reads Flow needs review … → EMPTY.

The environmental hypothesis is ruled out as the issue asked: this reproduces on an idle local nightly with no wedge, and the daily's own liveness recorder had both specs' final attempt at 0–3 % of probes down.

Fix

New shared helper tests/helpers/ui/clear-canvas-bottom-overlay.ts, called between the run and the inspect click. It frees the slot instead of clicking into it, and fails closed in three directions (the second and third came out of review):

• Timeout → throws naming what is still in the slot and whether Dismiss was tried, so a build that stops honouring its own Dismiss fails attributed rather than as an unexplained locator.click timeout at the call site.
• Lost selector → a slot that was never occupied is an error, not success. Callers reach this right after waitForSelector("text=built successfully"), which proves the bar is on screen, so "nothing ever matched" can only mean an upstream Tailwind edit (w-[530px] → w-[560px]). Reporting "clear" there would bring [Daily #1642] agent-context-id-{isolation,continuity} — an overlay at the canvas bottom intercepts the memory-output inspect click, and mode: 'serial' skips each file's sibling #1643 back as precisely the unattributed timeout the helper exists to prevent — with a helper call in the trace implying otherwise. { allowAlreadyClear: true } opts out.
• Failed-build bar → refused by name, never dismissed. It renders Retry + Dismiss and has no timer, so it satisfies the "will not leave on its own" predicate — but dismissing it erases the only UI evidence of a failed run, and flow-error-policy's v2 verdict is advisory on 1.12.x, so a caller could go green on a build that failed.

The empty-slot window is sized against the real constant, not the observed gap. handleDismiss sets dismissed = true (framer exit ~340 ms) and separately schedules setBuildInfo(null) at +500 ms, which is what un-hides the banner — so the gap is 500 ms − exit, measured ~89 ms with a 10 ms sampler. Two 200 ms-spaced reads cleared that by ~2.2×, but drop the exit animation upstream and the window fits inside the gap. EMPTY_CONFIRMATIONS = 5 (an 800 ms quiet window), and since the fake counts ticks and ignores durations (a POLL_MS = 0 survived every behavioural test), the relationship POLL_MS × (EMPTY_CONFIRMATIONS − 1) > BANNER_UNHIDE_DELAY_MS is asserted on the constants.

Also documented, because it was not obvious: the helper writes to the flow. handleDismissAllComponents calls setNodes marking each flagged node edited: true, and the editor persists it with a PATCH /api/v1/flows/{id} (observed on the wire). Harmless for a caller that seeds a throwaway flow and deletes it in teardown; a trap for one that asserts on the persisted graph.

Rejected alternatives: refreshing chat-io-ok-trace-fixture.json (shared by many specs, silences this only until the next template bump, leaves the build bar's ~5 px margin as fragile); click({ force: true }) (force skips the actionability check but the browser still dispatches at the point — the overlay swallows it and the test fails later, worse attributed); panning/zooming the node out of the slot (trades one geometric coincidence for another).

Third file: agent-n-messages-limit.spec.ts — hardening, and the distinction is measured

Same fixture, same node, same five lines, both tests @stable; #989 treated the three sibling files together. But it is not currently intercepted, and that was checked rather than assumed. The banner is present on its flow (1.13.0.dev0, slot y 586–656) — what differs is the margin: this node exposes two advanced fields where the context-id specs expose three, so it is shorter and its inspect button sits at y 541.0–556.9, ~37 px clear against ~5 px. Forcing the failing condition does not reproduce an interception there. Added anyway because 37 px is a coincidence of node height and node height is exactly what upstream moves (dev46 already changed how these fields are exposed); it costs ~2 s and removes the dependence.

Four further specs share the exposure in a different shape (split-text-chunking, and byte-identical private dismissUpdateBannerIfPresent copies in rag-pipeline / vector-store-index-query, plus a bare waitForTimeout(600) in chatInputOutputUser-shard-1). Folding those into the helper is a dedup refactor, not this fix — follow-up issue.

Scope note

No assertion changed anywhere. What each test proves is untouched; only the path to the output inspector is. The parametrized agent tests in these files are unchanged.

All three files stay mode: "serial" (the agent-area rule). With the retrieval tests passing, each file's sibling runs again — the 2 skips this cluster caused are gone, verified from the runner. The coupling itself is already mitigated the way these files can mitigate it: the model-free describe is declared before the parametrized loop, so a weak-model failure cannot skip the half that needs no provider. That ordering cannot help when the failure is in the first describe, which is what happened here.

Validation

Run on nightly 1.13.0.dev0 — langflowai/langflow-nightly:latest cut over to the 1.13 cycle on 2026-09-02T03:54Z, and that is the image daily-stable.yml pulls, so restoring @stable on a 1.12-only validation would ship these into a lane running an image they were never run against. Root cause and geometry were measured on 1.12.0.dev39.

• npm run typecheck ✅ · npm run lint ✅ (0 errors) · npm run test:units ✅ (893 pass / 0 fail, incl. 9 new) · npm run check:checklist-coverage ✅ · check-checklist-guard.mjs ✅ · watch-upstream-areas --mode=check-docs ✅ (no unresolved path in the changed docs)
• 2 clean full-file runs on 1.13.0.dev0 of both context-id specs: 6 passed, 2 skipped (1.6–3.8 min). 2 clean runs of agent-n-messages-limit: 2 passed (~18 s).
• 3 further clean full-file runs on 1.12.0.dev39 plus 3 isolated --retries=0 runs of the isolation retrieval test.
• The 2 skips are the anthropic parametrized targets — providers.json records the provider inactive ("Your credit balance is too low to access the Anthropic API"), a pre-existing key state unrelated to this issue. openai and google pass.
• Adversarial: the deterministic repro (4 s dwell, banner owning the slot) fails on main and passes with the fix.
• The green run now proves the selector too — with the fail-closed guard, a run that passes cannot have silently skipped a present overlay.
• Zero 🚨 Backend Error on the final runs. (An earlier 1.12 run logged one 404 on GET /api/v1/flows/{id} — that is resolveLiveFlowId probing the collected transient ids on purpose, documented in the spec.)
• Flow-orphan check: the fresh 1.13 instance holds 26 flows after every run — the starter templates only, zero leaked. On the 1.12 instance, 27 before → 27 after.
• --trace=on not run: known limitation on the Simple Agent template family (Create agent-tool-name-validation.spec.ts — invalid tool name blocked with clear message #490/flaky/bug: export-import-flow export produces empty flow (nodes: []) — regression of #384 #518), where tracing hangs indefinitely. Covered by the --retries=0 bursts, the force-fails and live scout probes instead.

Force-fails — all EXECUTED and observed failing, then reverted (0 mutation markers left)

#	Target	Mutation	Observed
FF-1	agent-context-id-isolation retrieval	invert the cross-context negative	Expected substring: "CTXISO-…-B-" ⇒ failed
FF-2	agent-context-id-continuity retrieval	invert the untagged-control negative	Expected substring: "CTXCONT-…-CTRL" ⇒ failed
FF-3	agent-context-id-isolation parametrized	assert a context never set	message(s) with wrong context_id: […] ⇒ failed
FF-4	agent-context-id-continuity parametrized	same on the session assertion	same shape ⇒ failed
FF-5	helper, unit	EMPTY_CONFIRMATIONS 2 → 1	only the handover test fails (5 pass / 1 fail)
FF-6	helper, end-to-end	pre-fix code under the deterministic repro	fails with the daily's exact interceptor
FF-7	agent-n-messages-limit truncation	expect 3 instead of 2	Expected: 3, Received: 2 ⇒ failed
FF-8	agent-n-messages-limit causal control	expect SEEDED_MESSAGES + 1	Expected: 11, Received: 10 ⇒ failed

Related: #1642 (triage), #1647 (quarantine), #989 (the previous fix of this same three-file cluster).

[Copilot]

🟢 Approval recommended

The fix is localized, restores quarantined coverage safely, and includes targeted unit tests that pin the previously flaky handover edge case.

Pull request overview

This PR fixes a deterministic E2E test defect where Langflow’s canvas bottom overlay intermittently (and sometimes permanently) intercepted pointer events, causing the context-id retrieval specs to hard-fail on the memory-output inspect click. The change adds a shared UI helper to proactively clear that overlay slot before clicking, lifts the prior quarantine by restoring @stable and removing test.fixme, and updates the associated spec docs accordingly.

Changes:

• Add clearCanvasBottomOverlay(page) helper (with unit tests + a fake overlay timeline) to wait out transient overlays and dismiss persistent ones.
• Call the helper in both context-id retrieval helpers immediately before clicking output-inspection-messages-memory.
• Restore @stable and remove the quarantine test.fixme in both specs; update spec docs’ “Last validated” and add the overlay note.

File summaries

File	Description
tests/tests-automations/regression/core-functionality/llm-agents/agent-context-id-isolation.spec.ts	Uses the new overlay-clearing helper before the memory inspector click; restores @stable and removes quarantine skip.
tests/tests-automations/regression/core-functionality/llm-agents/agent-context-id-continuity.spec.ts	Same overlay-clearing integration before the inspect click; restores @stable and removes quarantine skip.
tests/helpers/ui/clear-canvas-bottom-overlay.ts	New UI helper that polls the shared bottom overlay slot, dismissing when possible and failing with an attributed error if it can’t clear.
tests/helpers/ui/clear-canvas-bottom-overlay.test.ts	Unit coverage for the helper’s key behaviors, including the one-tick empty handover edge case and “dismiss no longer works” attribution.
tests/helpers/ui/clear-canvas-bottom-overlay.fake.ts	Fake Page/Locator implementation to deterministically simulate the overlay handover timeline and non-dismissible variants.
docs/core-functionality/llm-agents/agent-context-id-isolation.md	Updates “Last validated” and documents the overlay mechanism + the new required step (freeing the slot) before inspection.
docs/core-functionality/llm-agents/agent-context-id-continuity.md	Same as isolation doc: updates “Last validated” and adds the overlay note + procedural step.

Review details

• Files reviewed: 7/7 changed files
• Comments generated: 0
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.