Skip to content

Implement first-class UAT workflow support with explicit UAT policy and routing #213

Description

@fujiwaranosai850

From research #212

Overview

Add first-class UAT support to DevClaw as a distinct workflow phase, without overloading ordinary tester semantics. The recommended v1 design is to introduce explicit UAT workflow/config/routing support while reusing the existing tester worker infrastructure for agent-run UAT in the short term. UAT must support three modes: disabled/backward-compatible, human-only approval gate, and agent-run lane with optional final human approval.

Keep #207 as the canonical feature tracker for the product feature.

Implementation Checklist

Phase 1: Config and workflow model (~1.5-2 days)

  • Extend workflow typing in lib/workflow/types.ts to add a first-class uatPolicy enum, with v1 values covering disabled plus human/agent/hybrid behavior.
  • Update validation in lib/config/schema.ts so workflow.uatPolicy is accepted and validated alongside reviewPolicy and testPolicy.
  • Update built-in defaults in lib/workflow/defaults.ts to include UAT-aware state definitions and transitions while preserving current behavior when UAT is not enabled.
  • Update the user-facing template in defaults/devclaw/workflow.yaml to document the optional UAT phase, recommended state shapes, and backward-compatible defaults.
  • Add or update integrity/query helpers in lib/workflow/queries.ts so status/reporting can distinguish test-phase support from UAT-phase support.

Phase 2: Routing, dispatch, and heartbeat behavior (~2-3 days)

  • Extend step routing labels in lib/workflow/labels.ts to include explicit uat:* labels and any helper needed to resolve default UAT routing from policy.
  • Update dispatch logic in lib/dispatch/index.ts so issues entering the UAT-producing portion of the workflow receive the correct uat:* routing label without breaking existing review:* and test:* behavior.
  • Update queue gating in lib/services/tick.ts so agent-run UAT can dispatch safely through the short-term reused tester lane, while human-only UAT remains a hold gate.
  • Add heartbeat support for UAT auto-routing, including a pass analogous to lib/services/heartbeat/test-skip.ts / review-skip.ts for any uat:skip or human-gate transition behavior required by the chosen workflow design.
  • Ensure completion semantics still work cleanly for UAT states through lib/workflow/completion.ts and lib/services/pipeline.ts, including fail/refine/block flows back to To Improve / Refining.

Phase 3: Reporting, docs, and regression coverage (~1.5-2 days)

  • Update project/status surfaces such as lib/tools/admin/project-status.ts and any related summaries so UAT is reported distinctly rather than being collapsed into generic testing.
  • Update workflow guidance and docs in lib/tools/admin/workflow-guide.ts, docs/ONBOARDING.md, docs/ROADMAP.md, and any workflow docs that currently only describe review + test.
  • Add focused tests covering config validation, routing labels, dispatch gating, and heartbeat transitions for the three supported UAT modes.
  • Add an end-to-end workflow test, likely in lib/services/pipeline.e2e.test.ts, covering at least one enabled UAT path and one backward-compatible no-UAT path.
  • Document explicitly that v1 uses the existing tester worker infrastructure for agent-run UAT, and note dedicated uat role support as future follow-on work rather than part of this issue.

Dependencies & Blockers

  • Root feature scope should continue to live in Add first-class UAT phase support to DevClaw workflow #207; this task should reference it in comments/PR text without auto-closing it.
  • Decide and lock the exact uatPolicy enum names before broad code/docs changes, to avoid churn across workflow defaults and docs.
  • Be careful not to regress existing projects that rely on testPolicy: skip and current review/test routing labels.
  • Avoid introducing a dedicated uat role in this task unless implementation reveals a hard blocker; that would materially widen scope into registry, prompt scaffolding, model assignment, and worker-state surfaces.

Estimated Total: 5-7 days

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions