Provide reusable Playwright journeys for UX tests and communication captures

[gbaudrit]

Area

Cross-cutting

Feature type

Developer experience

Problem or need

Agentstration needs browser-level UX validation while the separate communication repository needs reproducible product screenshots and video source material for newsletters, release communication, and other editorial content. Both usages exercise the same product interactions, and the richest knowledge about those interactions naturally accumulates in UX tests.

Maintaining independent Playwright scripts in both repositories would duplicate selectors, navigation, waits, test data setup, and localization assumptions. Those copies would drift as the Console and Workplace evolve, making captures unreliable and forcing communication contributors to rediscover product behavior already encoded by product tests.

The automation must also work against different Agentstration instances without silently targeting the wrong data. Authentication, instance URLs, browser selection, and campaign Workspace selection therefore need explicit and reproducible configuration.

Expected outcome

Agentstration owns versioned, reusable Playwright journeys that are shared by its UX tests and external communication workflows. A communication workflow can check out the exact product revision it documents and invoke the matching journeys directly, without copying browser scripts or depending on a published npm package.

The same journey can drive assertions, screenshots, or video recording. Capture consumers select stable semantic checkpoints instead of browser selectors. Campaign data is isolated in a dedicated, explicitly prepared Workspace, while locale, theme, viewport, product revision, capture selection, and editorial output remain independent dimensions.

Proposed solution

Maintain a standalone TypeScript Playwright workspace under automation/playwright with:

• page objects that own accessible selectors, scoped Test IDs, stable waits, and product-surface operations;
• typed journeys that compose meaningful user actions and emit named checkpoints;
• UX test specifications that wrap journeys with assertions;
• a capture CLI that wraps the same journeys with screenshot recording and a reproducibility manifest;
• isolated managed local hosts for offline deterministic tests, plus an external-instance mode;
• explicit environment configuration for Console and Workplace URLs, credentials, browser channel, and campaign Workspace;
• example capture plans for the main creation paths.

Prefer semantic selectors first. Use centralized data-testid values only for cross-locale journeys, ambiguous controls, stable capture boundaries, or non-visible readiness state. Keep expected localized text independent from product RESX files so localization tests do not become circular.

Treat the Workspace as the campaign data boundary. The create-workspace journey creates it explicitly. Agent, Flow, and Entry journeys select an existing Workspace from the global Console header by stable technical name and fail clearly if it is unavailable. They must never silently create, replace, or delete campaign data.

Scope

• Reusable authentication and Console preparation.
• Explicit campaign Workspace creation and header-based selection.
• Agent creation and deployment journey.
• Handoff Flow creation, participant and route configuration, publication, and activation journey.
• Entry creation, Flow binding, presentation configuration, suggestions, and publication journey.
• Stable Test ID, checkpoint, and independent localization expectation contracts.
• Screenshot capture plans supporting full-page or checkpoint-target captures.
• Capture manifests recording the product commit, dirty state, browser version, locale, theme, viewport, and asset checksums.
• Command-line Console and Workplace URL overrides for captures.
• Environment-driven external-instance URLs, username, password, browser channel, and Workspace name for Playwright UI and tests.
• Offline deterministic smoke coverage and contributor documentation.
• Architectural records describing ownership of browser journeys and campaign Workspace isolation.

Out of scope

• Publishing the Playwright workspace as an npm package in this increment.
• Copying journeys or page objects into the communication repository.
• Newsletter rendering, editorial copy, storyboards, video editing, or final communication assets.
• Automatic creation, replacement, deletion, or cleanup of a Workspace from dependent Agent, Flow, or Entry journeys.
• Encoding locale, theme, viewport, or product version into Workspace identifiers.
• Generating localization expectations from product RESX catalogs.
• Requiring an external model provider, Internet access, Docker, Azure, Ollama, or another non-local dependency for the default UX suite.
• Comprehensive mobile journey coverage in this increment.

Dependencies

• Implemented by feat(testing): add reusable Playwright browser journeys #170.
• Architecture decisions: ADR-0079, product-owned browser journeys; ADR-0080, dedicated Workspaces for browser campaigns.
• The communication repository consumes the automation from a checkout of the exact Agentstration tag or commit being documented.

Alternatives considered

• Duplicate Playwright scripts in the communication repository: rejected because selectors, waits, and workflow knowledge would drift away from the product and its UX tests.
• Publish an npm package immediately: deferred because consumers can invoke the versioned workspace from the exact product checkout, avoiding package release and compatibility overhead during the current alpha phase.
• Use localized UI text as the primary cross-locale selector contract: rejected because translations are presentation details and can change independently; semantic roles and narrowly scoped stable Test IDs provide a more durable contract.
• Create one Workspace per language: rejected because localization is a campaign dimension, not a data-isolation identity. The same locale-neutral Workspace technical name can be exercised across languages when the surrounding campaign controls state appropriately.
• Let each dependent journey create missing prerequisites: rejected because hidden setup makes captures non-reproducible and can document the wrong topology.

Deployment impact

None

Acceptance criteria

• A single typed journey can be executed by a Playwright test and by the capture CLI without duplicating selectors or navigation logic.
• Capture plans select journeys and named checkpoints but contain no DOM selectors or localized product strings.
• The managed test fixture starts isolated local Console and Workplace hosts with deterministic AI and no network or live-model requirement.
• Capture callers can override the Console URL directly on the command line and optionally provide a distinct Workplace URL.
• Playwright UI and tests can target an existing instance through environment variables without starting or stopping that instance.
• External-instance authentication requires an explicit username/password pair and never silently falls back to development fixture credentials.
• A dedicated journey creates and selects a new campaign Workspace and fails on a technical-name collision.
• Agent, Flow, and Entry journeys select an existing campaign Workspace from the global header using explicit journey input or AGENTSTRATION_WORKSPACE_NAME.
• A configured but unavailable Workspace causes an immediate diagnostic error listing available candidates; dependent journeys never create it implicitly.
• Reusable Agent, handoff Flow, and Entry journeys cover their meaningful functional blocks and emit stable checkpoints after observable ready states.
• Test IDs, checkpoint identifiers, and independent en-US/fr-FR expectations have documented centralized ownership.
• Generated capture manifests identify the exact product revision and include browser and capture dimensions required for reproduction.
• Contributor documentation explains local setup, installed-browser fallback, external-instance configuration, campaign ordering, and use from another repository.
• TypeScript compilation, non-browser tests, browser UX smoke tests, .NET build/tests, and documentation build pass in CI.

Additional context

The intended campaign sequence is explicit and dependency ordered:

1. Create the campaign Workspace.
2. Prepare required profiles, Packs, or other shared resources.
3. Create and deploy participant Agents.
4. Create, configure, publish, and activate the Flow.
5. Create and publish the Entry bound to the active Flow.
6. Reuse journey checkpoints for UX evidence, newsletter screenshots, or video source recording.

Typical external Playwright UI configuration:

$env:AGENTSTRATION_CONSOLE_URL = "http://localhost:53400"
$env:AGENTSTRATION_WORKPLACE_URL = "http://localhost:53401"
$env:AGENTSTRATION_USERNAME = "admin"
$env:AGENTSTRATION_PASSWORD = "admin"
$env:AGENTSTRATION_WORKSPACE_NAME = "capture-handoff"
$env:AGENTSTRATION_PLAYWRIGHT_CHANNEL = "chrome"

Set-Location automation/playwright
npx playwright test --ui

The fixture credential shown above is public disposable Development data and is not suitable for an exposed environment.

Attachments

None.

Preliminary checks

• I searched for an existing issue covering this request.

[gbaudrit]

Implementation is now organized as squashable child increments targeting a dedicated integration branch:

• Make Playwright route coverage explicit and enforceable #285 — executable route/surface coverage catalog and measurable definition of 100%
• Cover Workplace conversations, tasks, and notifications #286 — Workplace conversations, tasks, notifications, and responsive routes
• Cover extension, source, bootstrap, and Pack journeys #287 — extensions, Sources, registries, Bootstrap, Packs, and resource scopes
• Cover Flow authoring, execution, and observability journeys #288 — Flow authoring, execution, runs, causality, and observability
• Cover Console identity, organization, and navigation journeys #289 — identity, organization, account, preferences, and global navigation
• Enforce cross-cutting browser quality gates #290 — cross-cutting localization, responsive, health, diagnostics, isolation, and CI gates
• Cover Console resource lifecycle journeys #291 — Console management-resource lifecycle journeys

The first catalog pass inventories 74 functional surfaces and 97 route declarations. The current PR #170 baseline fully covers 5 surfaces (6 route declarations), with Flow orchestration and Entry editor coverage marked partial until their namespaced aliases and remaining critical behavior are exercised.

The review also found #294: Flow and Entry scope selectors currently bind a value that is not propagated by their create/save operations.

Branch strategy:

1. update PR feat(testing): add reusable Playwright browser journeys #170 with current main;
2. use codex/playwright-application-coverage as the integration base;
3. merge each child PR into that branch with squash after review;
4. merge the completed integration branch into main only after maintainer validation.

[gbaudrit]

First child increment completed: #297 was squash-merged into codex/playwright-application-coverage after Documentation and Browser UX passed. The executable catalog now tracks 74 functional surfaces / 97 route declarations, with 5 surfaces initially fully covered. During validation, the managed model fixture and Flow handoff route reconciliation were updated for current main. Issue #285 is complete; the remaining coverage work stays split across #286–#291. No changes were merged into main.

[gbaudrit]

Child increment #286 is complete and squash-merged into codex/playwright-application-coverage via #305. All 8 managed Browser UX smoke tests pass. The executable catalog now reports 10/74 covered surfaces and 14/97 covered route declarations. No change has been merged to main.

[gbaudrit]

Child increment #287 is complete and squash-merged into codex/playwright-application-coverage via #308 (a8c86e7). Browser UX run 34589963425 passed all 14 smoke scenarios. The distribution increment covers Extensions/AEP discovery and detail states, Source Providers, Source Registries and discovery, pasted Source import/details, bootstrap catalog, Packs/Pack Projects, and resource scopes. It also fixed Development platform-administrator provisioning and removed a parallel .NET build race in the managed browser fixture. main remains untouched. Next: #288 (Flow authoring, execution, and observability).

[gbaudrit]

Repository-hygiene follow-up #314 is complete via #327 (c4acefa on the integration branch). Browser UX run 34590717888 passed. This contains line-ending normalization only; there are no logical code changes and main remains untouched.

[gbaudrit]

Child increment #288 is complete and squash-merged into codex/playwright-application-coverage via #336 (945dae0). Browser UX run 34593364048 passed all 16 smoke scenarios. Coverage now includes Flow lists/details, immutable namespaced views, designer routes, handoff authoring/publication, explicit Agent Runs, global and task-scoped Flow Runs, task details, and retained run events through the reusable inspect-flow-observability journey. Executable coverage is 34/74 surfaces and 48/97 route declarations. main remains untouched. Next: #289.

[gbaudrit]

FR #289 is complete and squash-merged into codex/playwright-application-coverage via PR #350. Browser UX run 34598312620 passed 19/19 Chromium smoke tests. Coverage is now 46/74 surfaces and 60/97 route declarations. main remains untouched.

[gbaudrit]

FR #290 is complete and squash-merged into codex/playwright-application-coverage via #356 (c268406). The strengthened Browser UX gate passed 22/22 checks across desktop, mobile, and independent English/French locale projects, with automatic page-health monitoring and always-published evidence. Current catalog: 49/74 surfaces, 63/97 route declarations. Continuing with #291.

[gbaudrit]

The first #291 increment is squash-merged via #358 (c91162d): stable contracts plus a reusable/capture-ready inventory journey for 12 resource lists and 8 creation editors. Browser UX is green 23/23. Continuing #291 with real model/runtime, secret/vault, tool, and trigger lifecycle mutations before promoting partial surfaces to covered.