Skip to content

Commit 525765d

Browse files
committed
🚀 Start and resume a workflow run from the CLI
`xmd workflow start <definition>` creates a run from committed Git bytes and executes it; `xmd workflow resume <run-id>` continues one from its retained definition and metadata rather than from whatever the checkout holds now. A start may name its own id with `--id`, and reuses a compatible run rather than creating a second one. The run reaches canonical core the way #426 and #433 settled: as an `ExecutionInstallation` the trusted host passes to `executeInstalled()`, carrying the retained-run admission core applies inside its own journal read and the `prepare` hook it invokes inside the durable root. One execution, not an installed call followed by an ordinary one — `xmd run` and `xmd test` pass an empty installation list, which is exactly what `execute()` already does. A run that ended is not a run to continue. `resume` admits interrupted, suspended and — as a full replay — completed; failed and cancelled are refused before the definition is fetched, before an orphaned execution is closed, before a record is begun, before a Workspace is attached and before anything is appended. Reusing a compatible id through `start` still replays a failed run's retained failure; that is a separate rule. A status line says what was retained, so it is published only once both lifecycle writes have persisted — the completion record, then the run state. The first refusal is the answer: the intended status is neither published nor claimed, the invocation exits 1, and a document failure that also occurred is still reported. An explicit lifecycle phase keeps a post-execution storage refusal from being republished as a host interruption. The definition is pinned to a Git object, so an uncommitted edit beside it never runs. A component search path would read that mutable checkout, so there is none. Service denial is installed in the execution's own scope, before the root import. A run's Workspace is attached only for live and partial work: a completed replay has a recorded root result and is given no filesystem to open a transaction against. Node and Bun refuse to host a workflow with one settled sentence, before reading a definition or creating a store, while rendering the same grammar every host does.
1 parent 0b73cbb commit 525765d

29 files changed

Lines changed: 3144 additions & 67 deletions

‎.github/workflows/publish-packages.yml‎

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -30,7 +30,7 @@ jobs:
3030
- name: Validate the manifests declare this version
3131
run: |
3232
VERSION="${{ steps.resolve.outputs.value }}"
33-
for f in packages/durable-streams/deno.json packages/runtime/deno.json packages/core/deno.json packages/acp/deno.json packages/testing/deno.json packages/test-agent/deno.json packages/web/deno.json packages/cli/deno.json packages/code-review-agent/deno.json packages/workflow/deno.json; do
33+
for f in packages/durable-streams/deno.json packages/runtime/deno.json packages/core/deno.json packages/acp/deno.json packages/testing/deno.json packages/test-agent/deno.json packages/web/deno.json packages/workflow/deno.json packages/cli/deno.json packages/code-review-agent/deno.json; do
3434
declared="$(jq -r .version "$f")"
3535
if [ "$declared" != "$VERSION" ]; then
3636
echo "::error::$f declares $declared, not $VERSION — the tag does not match the manifests"
@@ -110,8 +110,15 @@ jobs:
110110
package: packages/web
111111
version: ${{ needs.version.outputs.value }}
112112

113+
workflow:
114+
needs: [version, core, durable-streams, runtime]
115+
uses: ./.github/workflows/publish-one.yml
116+
with:
117+
package: packages/workflow
118+
version: ${{ needs.version.outputs.value }}
119+
113120
cli:
114-
needs: [version, acp, core, durable-streams, runtime, test-agent, testing, web]
121+
needs: [version, acp, core, durable-streams, runtime, test-agent, testing, web, workflow]
115122
uses: ./.github/workflows/publish-one.yml
116123
with:
117124
package: packages/cli
@@ -124,13 +131,6 @@ jobs:
124131
package: packages/code-review-agent
125132
version: ${{ needs.version.outputs.value }}
126133

127-
workflow:
128-
needs: [version, core, durable-streams, runtime]
129-
uses: ./.github/workflows/publish-one.yml
130-
with:
131-
package: packages/workflow
132-
version: ${{ needs.version.outputs.value }}
133-
134134
jsr:
135135
needs: [version]
136136
runs-on: ubuntu-latest

‎README.md‎

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -113,6 +113,58 @@ Useful flags:
113113
- `--verbose`, `-V` - print durable journal entries to stderr while running.
114114
- `--component-dir` - add component search directories. Defaults to `components` and `.`.
115115

116+
## Run a workflow
117+
118+
`xmd run` executes against the directory you are in and promises nothing
119+
afterwards. `xmd workflow` executes against a **run**: one retained Workspace and
120+
one filtered journal, in a database that outlives the process, so an interrupted
121+
procedure resumes from where it stopped instead of starting again.
122+
123+
```bash
124+
xmd workflow start flows/prepare-release.md
125+
xmd workflow start --id=release-1.4 --props-channel=stable flows/prepare-release.md
126+
xmd workflow resume release-1.4
127+
```
128+
129+
`start` names a document; `resume` names a run. A document path locates a
130+
definition and never selects a previous run, so starting the same document twice
131+
without `--id` creates two runs. Reusing an `--id` addresses the same run when
132+
the definition, base and normalized properties all agree, and is refused when
133+
any of them differ. `resume` takes no document and no properties: it uses the
134+
ones its run retained.
135+
136+
What a run is, is a Git object: the repository containing the document, the
137+
commit `HEAD` resolves to, and the document's path inside it. **The committed
138+
document runs**, so uncommitted edits in your working tree do not change what a
139+
run is a run of, and a resume months later loads the same object rather than
140+
whatever the file says now.
141+
142+
Inside a run, `<File>` and `<Glob>` name entries in the run's own logical
143+
filesystem rather than yours. Each read, write and search is one durable effect:
144+
the mutation, the Workspace root it produces and the journal result commit
145+
together, so a crash leaves all three or none, and a resume restores what was
146+
recorded instead of doing it again. Operations a run does not have — a temporary
147+
directory, a native service — fail explicitly rather than reaching your machine.
148+
149+
Identity and outcome go to standard error, so piping stdout still gives you the
150+
document:
151+
152+
```text
153+
workflow run: release-1.4
154+
workflow status: completed
155+
```
156+
157+
Only a completed run exits `0`. Failed exits `1`, suspended `2`, cancelled `3`
158+
and interrupted `130`, so a script cannot mistake an incomplete workflow for a
159+
finished one.
160+
161+
Runs live under `~/.xmd/runs`; set `XMD_WORKFLOW_RUNS` to an absolute directory
162+
to keep them somewhere else. `xmd workflow` is available through the Deno
163+
entrypoint and the compiled binary; under Node and Bun the command exists and
164+
refuses before creating anything.
165+
166+
Status, list, history, cancel and fork are designed but not yet shipped.
167+
116168
## Coding agents
117169

118170
Run ACP-compatible coding agents directly from a document with `<Agent>`,

‎architecture.md‎

Lines changed: 80 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -154,9 +154,8 @@ The `@executablemd/workflow` package owns `WorkflowRun`,
154154
and the Git capability. It depends on `@executablemd/core`,
155155
`@executablemd/durable-streams` and `@executablemd/runtime`, whose contextual
156156
`exec()` and `cwd()` the Git provider invokes; core never imports workflow or
157-
Git. The future CLI lifecycle is `xmd workflow start` and `xmd workflow resume`;
158-
there is no workflow CLI execution branch yet. The durable lookup that resume
159-
will require is the run storage below. Ordinary `xmd run` remains unchanged.
157+
Git. `xmd workflow start` and `xmd workflow resume` are the CLI lifecycle, and
158+
they resume through the run storage below. Ordinary `xmd run` remains unchanged.
160159

161160
## Workflow run storage
162161

@@ -352,6 +351,68 @@ describe what failed without repeating retained props or journal payloads —
352351
including their member *names*, which can carry a credential as readily as a
353352
member value can.
354353

354+
## The workflow lifecycle
355+
356+
`xmd workflow start [--id=<run-id>] [--props-*=…] <definition>` and
357+
`xmd workflow resume <run-id>` are the two commands. Both run in the foreground,
358+
stream the document's own output to standard output, and report identity and
359+
outcome on standard error as two stable lines — `workflow run: <run-id>` once
360+
the run has been created or found, and `workflow status: <status>` once the
361+
execution settles. Only a completed run exits zero: failed exits 1, suspended 2,
362+
cancelled 3 and interrupted 130, so shell automation cannot mistake an
363+
incomplete workflow for a finished one. A request the command refuses — bad
364+
grammar, a missing run, an incompatible reuse, damaged storage, an unsupported
365+
host — exits 1.
366+
367+
`start` establishes an immutable definition from Git rather than identifying a
368+
working-tree file. It locates the repository containing the supplied path,
369+
resolves `HEAD^{commit}` once because the command has no base option, reads the
370+
repository's object format, and stores version 1 of the descriptor with that
371+
format, the lowercase commit ID and the normalized repository-relative POSIX
372+
path. **The bytes that execute are the ones that commit holds**, so a working
373+
tree with uncommitted edits runs the committed document. Where the repository is
374+
checked out is retrieval metadata: replaceable, credential-free, excluded from
375+
identity, and reauthorized before it is used again. `resume` loads exactly the
376+
retained object through that locator and never substitutes the current `HEAD` or
377+
a same-named working-tree file. A missing object, missing or unreadable
378+
retrieval metadata, a path outside the repository, or a root that is not
379+
Markdown fails explicitly; none of them creates a replacement run or an empty
380+
definition. A workflow definition is one immutable object, so the component
381+
search path is empty and a repository component fails to resolve rather than
382+
resolving to content beside the definition in a mutable checkout.
383+
384+
Every actual execution opens or creates the run's database, begins a
385+
document-execution record, installs the exact retained WorkflowRun, installs the
386+
service denial before the root is imported, and — for a live or partial
387+
execution — installs the logical working directory `/`, the transaction-bound
388+
Files provider and that database's Workspace effect coordinator. It executes
389+
against the database's own journal with the retained props and secret detection,
390+
then finishes the execution record and publishes the run's status. A completed
391+
run still replays, so its retained output and result are emitted, but it
392+
attaches no Workspace provider or coordinator and performs no filesystem
393+
mutation.
394+
395+
A failure retains `failed` and uses a journal stop reason when a retained event
396+
identifies it, and a categorical host code otherwise; no exception text is
397+
retained beside the journal that filtered it. Graceful foreground interruption
398+
finishes the execution `interrupted` and exits 130. `suspended` is resumable;
399+
`failed` and `cancelled` are refused by `resume`, and `completed` replays under
400+
either command.
401+
402+
The initial host is Deno-local. The Deno entrypoints — source and the compiled
403+
binary — install the local run store, beneath `~/.xmd/runs` unless
404+
`XMD_WORKFLOW_RUNS` names another absolute directory. Node and Bun expose the
405+
same grammar and refuse before creating or executing anything. The shared CLI
406+
module imports no SQLite, no DOFS and no runtime detection: it asks a host
407+
adapter to open storage and to attach a run's Workspace, and the entrypoints
408+
decide which adapter exists.
409+
410+
Durable ownership and concurrent-executor enforcement belong to #367. Until it
411+
lands, a run left `running` because its host disappeared is treated as an
412+
orphaned interrupted execution by the next resume, which closes that unfinished
413+
execution record as `interrupted` before beginning its own. Nothing here claims
414+
concurrent resume is safe.
415+
355416
## Workflow Workspace
356417

357418
The command selects the environment; the document describes the procedure.
@@ -480,8 +541,8 @@ commit form one boundary. The Workspace filesystem uses the pinned synchronous D
480541
entry points for its string and byte-array surface, so cancellation leaves no
481542
eager promise or stream pull able to reach the connection after transaction
482543
authority ends. The transaction-bound Files provider selects that operation for
483-
every document filesystem read, write and search; workflow lifecycle commands do
484-
not select it yet.
544+
every document filesystem read, write and search, and the workflow lifecycle
545+
commands install that provider.
485546

486547
An external provider cannot join that transaction. Prompt, Git push and pull
487548
request effects derive a stable identity from the run and expansion, ask the
@@ -557,8 +618,8 @@ collection is not in the production closure and is never invoked. The provider
557618
exposes no public history selection or fork operation at this layer. Its
558619
coordinator combines one mutation, immutable-root publication and one filtered
559620
journal result atomically, and the transaction-bound Files provider is what
560-
routes a document's `<File>` and `<Glob>` to it. Workflow start and resume do
561-
not reach it yet.
621+
routes a document's `<File>` and `<Glob>` to it. `xmd workflow start` and
622+
`xmd workflow resume` install that provider around each execution.
562623

563624
The coordinator treats only errors produced through its private filesystem
564625
adapter's documented path and mutation refusals as journalable operation
@@ -1034,12 +1095,12 @@ consumed. If collision handling terminates immediately,
10341095
receives no terminal event; restoring the compatible definition can still
10351096
replay it.
10361097

1037-
#390 provides and tests the non-delegating `useWorkflowServiceDenial()` provider.
1038-
#366 will install it in the future `xmd workflow start` and `xmd workflow resume`
1039-
scopes. No workflow CLI execution branch exists yet. The provider prevents a
1040-
workflow from reaching an inherited host adapter, because a run-owned durable
1041-
service requires stable identity and reconciliation rather than an
1042-
execution-owned live process.
1098+
`xmd workflow start` and `xmd workflow resume` install the non-delegating
1099+
`useWorkflowServiceDenial()` provider inside each execution scope, before the
1100+
root document is imported — in the same place `xmd run` installs its host
1101+
service adapter. The provider prevents a workflow from reaching an inherited
1102+
host adapter, because a run-owned durable service requires stable identity and
1103+
reconciliation rather than an execution-owned live process.
10431104

10441105
## State ownership
10451106

@@ -1412,7 +1473,7 @@ Status is measured against main.
14121473
| `workflowInstallation()` / `getWorkflowRun()` | associates one document execution with a workflow run, through an `ExecutionInstallation` the trusted host passes to `executeInstalled()` | built on the #366 stack |
14131474
| `retainedWorkflowInstallation()` | associates one document execution with a run storage already created, requiring exact journal agreement | built on the #366 stack |
14141475
| `Git.revParse()` | verifies and resolves one Git revision expression contextually | built on main |
1415-
| workflow run storage | creates or compatibly finds one run by public run ID, retains its identity, state, document executions and filtered journal, and validates immutable Workspace roots through one provider-owned connection entry | built on the #365 stack; public workflow execution is unbuilt |
1476+
| workflow run storage | creates or compatibly finds one run by public run ID, retains its identity, state, document executions and filtered journal, and validates immutable Workspace roots through one provider-owned connection entry | built on the #365 stack; the CLI lifecycle reaches it on the #366 stack |
14161477
| caller-owned storage transaction | publishes several changes, including journal events, in one transaction nothing else enlists in | built on main |
14171478
| live durable-operation coordinator | explicitly coordinates structured live execution with existing Yield publication while leaving replay and callback effects unchanged | built on the #365 stack |
14181479
| Workspace coordination API | fails closed by default; replaceable context routes only a one-use provider selection, while the selected provider directly invokes an execution-owned credentialed capability for execution, publication and failure activation | built on the #365 stack; the Deno provider installs an adapter-private atomic handler |
@@ -1421,16 +1482,16 @@ Status is measured against main.
14211482
| `Config` run deadline / exec default / Fetch default | three independently owned contextual timeouts, absent unless configured, each read by exactly one consumer | built on main |
14221483
| `API.Files` | routes every document filesystem operation to the installed provider, with no host default and structural failure data | built on the #227 stack |
14231484
| host Files provider / `useHostFiles()` | resolves document paths in the caller's filesystem, containing them while the host namespace is stable; installed by all four CLI entrypoints | built on the #227 stack |
1424-
| transaction-bound Files provider | resolves document paths in the run-owned logical Workspace inside the caller-owned transaction | built on the #366 stack; CLI reachability remains unbuilt |
1485+
| transaction-bound Files provider | resolves document paths in the run-owned logical Workspace inside the caller-owned transaction | built on the #366 stack |
14251486
| `service=<binding>` | publishes the attachment's endpoint into the live binding overlay for its invocation | built on main |
14261487
| `ephemeral eval` | reconstructs live middleware and bindings without a journal entry | built on main |
1427-
| `useWorkflowServiceDenial()` | provides and tests a non-delegating workflow service denial provider; #366 will install it in future start and resume scopes | built on main; no workflow CLI execution branch exists yet |
1428-
| `xmd workflow start` / `xmd workflow resume` | starts or resumes a workflow run from the CLI | defined in `specs/workflow-workspace-spec.md`, unbuilt; the lookup it resumes through is built |
1429-
| implicit workflow Workspace | retains provider-neutral filesystem, repository and attachment state by run ID | defined in `specs/workflow-workspace-spec.md`, unbuilt (#218) |
1488+
| `useWorkflowServiceDenial()` | provides a non-delegating workflow service denial provider, installed inside every start and resume execution scope | built on the #366 stack |
1489+
| `xmd workflow start` / `xmd workflow resume` | starts or resumes a workflow run from the CLI, under the Deno entrypoints only | built on the #366 stack; status, list, history, cancel, fork and delete are unbuilt |
1490+
| implicit workflow Workspace | retains provider-neutral filesystem, repository and attachment state by run ID | document filesystem built on the #366 stack; repository, process and attachment capabilities unbuilt (#218) |
14301491
| Repository / Worktree / transactional Git effects | compose named checkouts and publish local mutations with their journal result | defined in `specs/workflow-workspace-spec.md`, unbuilt |
14311492
| workflow inspection and history fork | reads status/history without advancing a run and creates a new run from a checkpoint | defined in `specs/workflow-workspace-spec.md`, unbuilt |
14321493
| read-only workflow Agent / generated XMD | lets an Agent inspect a derived view and propose constrained executable changes | defined in `specs/workflow-workspace-spec.md`, unbuilt |
1433-
| Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects route to it on the #366 stack, and workflow lifecycle reachability is unbuilt |
1494+
| Deno-local DOFS provider | owns one authoritative SQLite/DOFS connection per run path, captures arbitrary canonical retained roots, privately restores them, and atomically coordinates one Workspace mutation with its filtered Yield | built on the #365 stack; public document filesystem effects and the CLI lifecycle route to it on the #366 stack |
14341495
| scoped Worker Shell | executes `just-bash` through the Workspace adapter inside a Deno Worker | containment and effect-transaction POCs complete (#351, #357); production integration unbuilt |
14351496
| `<Retry max timeout>` | retry a region until it completes | defined, unbuilt |
14361497
| suspension effect | suspend durably | defined, unbuilt |

‎bun.lock‎

Lines changed: 2 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎packages/cli/package.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@
1919
"@executablemd/test-agent": "workspace:*",
2020
"@executablemd/testing": "workspace:*",
2121
"@executablemd/web": "workspace:*",
22+
"@executablemd/workflow": "workspace:*",
2223
"@standard-schema/spec": "^1.0.0",
2324
"configliere": "^0.4.0",
2425
"effection": "4.1.0",

‎packages/cli/src/bun.ts‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,7 @@ import process from "node:process";
1111
import { API, useHostFiles } from "@executablemd/runtime";
1212
import { compileDataUri } from "@executablemd/core";
1313
import { runXmd } from "./cli.ts";
14+
import { unsupportedWorkflowHost } from "./workflow.ts";
1415
import { useBunService } from "./bun-service.ts";
1516

1617
const ENTRYPOINT = fileURLToPath(import.meta.url);
@@ -35,5 +36,5 @@ await main(function* (args) {
3536
// no host default: a run with no provider must fail rather than reach the
3637
// host by accident.
3738
yield* useHostFiles();
38-
yield* runXmd(args, useBunService);
39+
yield* runXmd(args, useBunService, unsupportedWorkflowHost);
3940
});

0 commit comments

Comments
 (0)