Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
dd4dc61
Add controlled inline worker picker and readiness helpers
raphael Nov 19, 2026
658dd56
Report the worker roles required by a new build
raphael Nov 19, 2026
77f4f20
Present approved merge actions and retain saved evidence
raphael Nov 19, 2026
7fc5b8f
Supply installed agent metadata in control plane fixtures
raphael Nov 19, 2026
460a06c
Connect worker choices and approved merge actions to page state
raphael Nov 19, 2026
76a19c1
Keep merge progress bound to its change
raphael Nov 19, 2026
33a064b
Document the agent-driven Surla journey
raphael Nov 19, 2026
5288182
Decode agent discovery proof through generated HTTP response
raphael Nov 19, 2026
8789708
Show merge controls only for an approved build or its outcome
raphael Nov 19, 2026
c4eb42c
test(views): cover inline workers and approved browser merge
raphael Nov 19, 2026
b37b07c
Exercise inline worker choices through served views
raphael Nov 19, 2026
e5f49e7
Cover notice persistence and approval-bound merge recovery
raphael Nov 19, 2026
d7e7a98
Keep worker drafts through saved card refreshes
raphael Nov 19, 2026
a2113b6
Prove pending merge prevents duplicate requests
raphael Nov 19, 2026
6507538
Keep Check evidence separate from approved merge controls
raphael Nov 19, 2026
7c697cd
Exercise controlled input events and await worker discovery
raphael Nov 19, 2026
257d9a2
test(views): read approval through the Build primary action
raphael Nov 19, 2026
bf5c611
Keep merge recovery scoped to approved builds and pending changes
raphael Nov 19, 2026
f9fd168
Observe selected Build freshness through the approval review owner
raphael Nov 19, 2026
e2c1f42
Clarify view recovery and exact approval requirements
raphael Nov 19, 2026
cbaf6a6
Correct setup and worker integration documentation
raphael Nov 19, 2026
d58f7eb
Keep browser recovery in the integration and accept prose capitalization
raphael Nov 19, 2026
f254a55
Prove session recovery through coding agents and the companion
raphael Nov 19, 2026
ee9d7de
Check session lifetime guidance against its named reopen routes
raphael Nov 19, 2026
3490c2e
test(views): follow agent and companion session recovery
raphael Nov 19, 2026
b3e04da
Match saved-result assertions to current approval guidance
raphael Nov 19, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
118 changes: 77 additions & 41 deletions .agents/skills/surla-ux-review/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,14 @@ description: Run a moderated usability study of Surla. One subagent plays a firs
# Surla UX review

This skill runs a moderated, think-aloud usability test of Surla and turns it
into a UX research report. Two subagents take part:
into a UX research report. The product goal is a journey driven by the
participant's existing coding-agent session: after installation and one-time
`surla up`, the participant describes the goal through `/surla` (`$surla`),
uses the browser for human decisions, and reaches an approved merge without
typing terminal commands to move between stages. Commands the coding agent
runs internally are distinct from commands the user must enter. Observe whether the product
actually enables that journey; never supply missing steps as hidden coaching.
Two subagents take part:

- **The participant** plays a developer who has never used Surla. It may read
only Surla's `README.md` and whatever Surla itself prints or generates, and
Expand Down Expand Up @@ -69,45 +76,59 @@ source out of its view.
`attempt-1/transcript.md`:

- the Surla commit, the date, and the OS;
- whether the folder in the README's `surla init -dir` example (such as
`/tmp/shop`) already exists. `surla init` refuses an existing folder, so a
leftover would look like a usability problem. Do not delete it. Tell the
tester it exists, labeled "ownership unconfirmed";
- the `-surla-version` that the README tells `surla init` to use. A scaffolded
project downloads that version, not the commit under test;
- whether the participant's `work/` folder is empty apart from permitted
coding-agent settings, and any known pre-existing example destinations.
Leave existing folders alone and label their ownership unconfirmed;
- the global setup writes this attempt may make. `surla up` and
`surla setup -global` update the real machine's detected coding-agent MCP
configuration and skill files, and may install the Herdr mod and add its
free macOS shortcut in `keys.json`. Record the paths setup reports and
pre-existing registrations or bindings without recording credentials.
These files are outside `work/` and may be used by other sessions; do not
delete or restore them as attempt cleanup;
- the version of Surla installed by the README and the version its starter
module will download. Record whether they match the commit under test;
- the versions of `go`, `goa`, and `temporal`, or that they are missing;
- which credentials are set, such as `ANTHROPIC_API_KEY`. Record that each is
set, never its value;
- any personal instruction files your harness loads into every subagent, such
as a global `CLAUDE.md` or `AGENTS.md`. These can tell the participant
things the README does not, so the report lists them as a limitation;
- whether commands run inside your coding agent's own sandbox, and whether
they have network access. `surla init` downloads modules, and Surla builds
they have network access. Starter initialization downloads modules, and Surla builds
and runs the code under test as ordinary processes with the user's file
and network access. Working copies and process groups do not confine that
code. Keep any coding-agent sandbox restrictions separate from Surla's
documented execution policy. Give the participant the host access you are
authorized to give. Trusted Surla commands run as the docs show, including
`surla init`, which generates code from Surla's own unmodified starter
templates. The product's own code is never compiled, generated from, or run
except through Surla's commands. That
includes `goa gen` on its design, which runs that design code. If the
documented workflow needs a step Surla offers no command for, the study
authorized to give. Trusted Surla operations run as the docs show,
including initialization from Surla's own unmodified starter templates.
The product's own code is never compiled, generated from, or run except
through Surla's documented operations. That includes `goa gen` on its
design, which runs that design code. If the
documented workflow needs a step Surla offers no operation for, the study
reports it as a missing capability. If the access Surla needs is not
available, that is an environment limitation: the tester marks the
milestones it blocks as not supported;
- **how the participant reaches Surla's MCP server.** The README tells a user
to point a coding agent at `http://<addr>/mcp`. A subagent cannot add an MCP
server to its own session. The participant therefore runs its own
coding-agent CLI non-interactively in `work/`, with an MCP configuration it
writes itself, for example `claude -p`, `codex exec`, or `gemini -p`.
Record which CLIs it may use, or that none can run here. Do not hand it an
undocumented HTTP route, because working that out from outside the README
is coaching.
- **how the participant reaches Surla's MCP server.** The current README
uses one-time `surla up` (or `surla setup -global`) to register `surla stdio`
and the `/surla` (`$surla`) skill. A subagent cannot add an MCP server to
its own session, so the participant may run an installed coding-agent CLI
in `work/` and invoke the skill there. Record which clients can run, which
load the global registration, whether a session reload is needed, and
whether the environment can show the browser for human decisions. Record
harness-only CLI launches separately from user workflow steps; a harness
that cannot preserve the coding session is a study limitation. Do not
hand it an HTTP route, credential, prewritten MCP configuration or lifecycle
command. Failure to discover or use the documented integration is evidence;
- whether Herdr and Chorus are installed, whether the agent starts inside a
live Herdr pane, whether its client has been observed forwarding
`HC_TERMINAL_ID` and provider settings to stdio, the installed agent
families and background-worker support, and any project choices or
dismissed recommendations already present. Do not install a second family just to hide the one-family path;
- **what each mid-build change needs** (see the tester brief's "Mid-build
changes"), recorded as two separate facts: what this machine provides,
such as model credentials or the access a live demo needs, and what the
commit under test provides, such as a command that regenerates code after a
commit under test provides, such as an operation that regenerates code after a
design change. The tester uses them to tell "not supported on this
machine" apart from "blocked by a missing Surla capability".

Expand Down Expand Up @@ -157,8 +178,8 @@ participant.
Each round trip works like this:

1. The participant works until a natural pause and replies with a step
report. Its brief lists when it pauses, including before the first run of
each new `surla` command.
report. Its brief lists when it pauses, including before one-time setup,
its first skill invocation and each new human decision on the page.
2. Append the report to the transcript and send it unchanged to the tester,
together with the elapsed time since the previous report.
3. The tester replies with up to three questions, an optional hint, an
Expand Down Expand Up @@ -190,7 +211,7 @@ harness keeps a subagent transcript. Also read the logs its coding agent saved
in `work/agent-logs/`. A leak is reading anything outside the participant's
allowed list, loading this skill, building the product without Surla, or
compiling, generating from, or running the product's code except through
Surla's commands.
Surla's documented operations.

A participant cannot unlearn what it read, so a leak ends the attempt. Tell
the tester what was read and when. The tester records the leak as a finding,
Expand All @@ -207,15 +228,16 @@ limitations that leaks were only self-reported.
`ps -p <pid> -o command=` shows the reported binary and folder. PIDs get
reused. Leave running any process you cannot match.
- Delete nothing else. Every project folder the participant created is
evidence for the report, including folders outside `work/` such as
`/tmp/shop`, and may hold a Git repository with unpushed work. Each attempt
evidence for the report, including a project initialized outside `work/`,
and may hold a Git repository with unpushed work. Keep machine-wide setup
changes recorded above; they are not disposable attempt files. Each attempt
gets a new folder, so nothing needs to be cleared for the next one.

In the next transcript, record the exact path of every folder outside `work/`
that the participant reported or that appears in `work/agent-logs/`, and
every process left running. Label them "ownership unconfirmed". Tell the
tester when the next attempt starts, so it does not log a leftover, such as
`surla init` refusing an existing `-dir` or a port already in use, as a
initialization refusing a nonempty folder or a port already in use, as a
Surla bug. If someone later asks for cleanup, inspect each exact target
first: nested repositories, `git status`, unpushed commits, and whether
deleting it is authorized.
Expand All @@ -224,17 +246,18 @@ deleting it is authorized.

When the tester decides `stop_for_bug`, it names the bug and gives the steps
to reproduce it. A blocking bug is one where Surla crashes, behaves
differently from its documentation, or offers no command to run
something its documented workflow needs, and no documented workaround
exists. The participant cannot get past it however well it uses Surla.
differently from its documentation, or lacks an MCP/browser operation its
documented workflow needs, and no documented workaround exists. A terminal
workaround can unblock the product task while still failing the zero-command
journey goal; record both outcomes rather than silently counting success.

1. Reproduce the bug yourself with the trusted Surla CLI, using the host and
network access you are authorized to use, so a restriction in your coding
agent is not mistaken for a Surla bug. The product's code is still
compiled, generated from, and run only through Surla's commands.
compiled, generated from, and run only through Surla's documented operations.
- If the failure comes from the harness, tell the tester "environment
limitation, not a Surla bug" and which milestones it affects.
- A missing capability, meaning no Surla command for a step the
- A missing capability, meaning no documented Surla operation for a step the
documented workflow needs, counts as a real bug even when Surla lacks
it by design. Go to step 2.
- If Surla behaves as designed, tell the tester "reproduced; behavior is as
Expand All @@ -244,17 +267,18 @@ exists. The participant cannot get past it however well it uses Surla.
separate clone and branch, the design gate, tests, an independent review,
and a pull request. Fix the cause, not the symptom the participant
happened to hit.
3. If the fix is in code that a scaffolded project downloads, the participant
would still get the old version through the README's `-surla-version`.
Push the fix. Then add a follow-up commit that sets `-surla-version` to the
fix commit's Go pseudo-version (`v0.0.0-<yyyymmddhhmmss>-<12-char-sha>`,
as `go list -m github.com/goadesign/surla@<sha>` prints it). This is the
same way main updates `examples/orders` after a release.
3. If the fix is in code a starter downloads, publish a version containing
it before restarting. Confirm that the documented installation and starter
initialization resolve that version, rather than an earlier dependency.
An exact Go pseudo-version is printed by
`go list -m github.com/goadesign/surla@<sha>`; update a documented version
pin when that is what selects the dependency. Do not patch the participant's
project or supply an undocumented override to make the attempt pass.
4. Restart from scratch:
1. Stop processes and record leftovers as in section 3.
2. Create a new participant folder, `~/src/surla-ux-<id>-a<N+1>/`, holding
a clone at the newest commit of the fix, including any
`-surla-version` follow-up, and an empty `work/`. Leave the previous
version-pin follow-up, and an empty `work/`. Leave the previous
attempt's folder in place as evidence.
3. Copy the user docs from that commit into the tester's `docs/` again, so
the tester judges the participant against the docs it is reading.
Expand All @@ -271,6 +295,18 @@ set a limit on attempts, stop and report when it is reached.

## 5. Debrief and report

Keep these outcomes separate: product built, verified, approved, and merged;
one-time installation/setup; later user-entered terminal commands; commands
the coding agent runs internally; harness-only invocations; and steps
completed with help. For each applicable path, record observed success or why it was not
exercised: live guide adoption, page recovery through the guide or companion,
inline agent choices, builder-only builds with configured tests, one-family
operation, missing Herdr, persisted choices/notices after reload, and human
Merge with refusal recovery. Do not add fixtures or hints to force coverage.
A source claim, successful command or assignment is not proof of a visible
outcome. The report's severity ratings and transcript evidence remain the
moderator's responsibility.

When the tester decides `end_session`, it sends the participant its debrief
questions. Relay the questions and the answers. Then send the tester the text
of `references/report-template.md` and ask it to write `report.md`.
Expand Down
32 changes: 23 additions & 9 deletions .agents/skills/surla-ux-review/references/participant-brief.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,16 +31,26 @@ You are taking part in a usability study of a developer tool called
real first-time user works from the README and the tool, and that is what
this study measures. Documents the README links to are not available. If
you want one, say so in your report, because that is useful evidence.
- Run Surla itself the way the README shows, including `surla init`, which
sets up your starter project.
- Install and set up Surla the way the README shows. After one-time setup,
use your coding-agent conversation and Surla's browser page to reach the
goal. Setup changes global coding-agent configuration and skill files,
and may install the Herdr companion and its free macOS shortcut outside
`work/`. Report the paths it changes; do not remove shared settings during
cleanup. Report every terminal command you still need to enter as the user;
distinguish commands your coding agent runs internally and harness-only
agent launches. Do not hide a manual workaround as ordinary progress.
- You may read and invoke the runtime `/surla` (`$surla`) skill that Surla
installs for your coding agent. This is separate from the forbidden
`surla-ux-review` study skill. Learn the entrypoint from the README; the
facilitator will not configure a connection or supply a page credential.
- Surla runs your product's code as ordinary processes with your file and
network access. Its working copies and process groups do not confine that
code. Any sandbox imposed by your coding-agent environment is a separate
environment restriction.
- Do not compile, generate code from, or run your product's code yourself,
for example with `goa gen`, `go build`, `go run`, or `go test` in your
project. Your product's code runs only through Surla's commands, such as
`surla verify`. If Surla's README or generated files tell you to run one
project. Your product's code runs only through Surla's documented
operations. If Surla's README or generated files tell you to run one
of those commands yourself, or you need a step Surla gives you no way to
run, do not run it. Say so in your report instead.
- This machine has: {{ENVIRONMENT_FACTS}}. Never print the value of a
Expand All @@ -49,9 +59,9 @@ You are taking part in a usability study of a developer tool called
{{CODING_AGENTS}}. If you start one, save its full output under
`{{PARTICIPANT}}/work/agent-logs/`. Tell it to use Surla, and tell it not to
read Surla's source, use the `surla-ux-review` skill, or compile, generate
code from, or run the product's code except through Surla's commands. If
it needs such a step, it must stop and say so in its output, not skip the
step or edit generated files by hand. Surla's source is
code from, or run the product's code except through Surla's documented
operations. If it needs such a step, it must stop and say so in its output,
not skip the step or edit generated files by hand. Surla's source is
the clone apart from its README, and `github.com/goadesign/surla` in the Go
module cache. It may read other libraries it needs to write code.
- Run long-running processes, such as servers, in the background. Report
Expand All @@ -70,8 +80,12 @@ Send a step report when any of these happens:
- you finish a meaningful part of the goal;
- you are stuck;
- you are choosing between different ways of using Surla;
- you are about to run a `surla` command for the first time. Report before
running it, and say what you expect it to do;
- you are about to install or set up Surla, invoke its skill for the first
time, or make a new human decision in its browser page. Report before the
action and say what you expect;
- you need to enter a terminal command as the user after one-time setup to
move the product forward; report the reason and what the interface made
available;
- about ten actions have passed since your last report.

Use this format:
Expand Down
Loading
Loading