✨ Execute Markdown workflows without Git (#443)

[taras]

Closes #443.

Why

xmd workflow start located a repository, resolved HEAD and read the document through repositoryRoot(), revParse() and readGitObject(). A file outside Git could not start, an untracked file could not start, and a modified tracked file executed its older committed bytes. That dependency also blocks #822: Workflow needs Git to load its own definition, while the Git Plugin needs Workflow's durable execution boundaries.

What changes

Before:

$ xmd workflow start ./prepare-release.md
error: not a git repository

A tracked file with unstaged edits started, but executed the bytes HEAD held rather than the ones on disk.

After:

$ xmd workflow start ./prepare-release.md
workflow run: prepare-release-8f2a

The command snapshots the file's exact current bytes before the run becomes durable. The first execution and every resume use that retained snapshot even if the original file is edited, moved or deleted. Existing version-1 runs keep executing from their pinned commit, byte for byte, through a reader the host supplies.

How it works

xmd workflow start ./doc.md#Target
  → establishDefinition() parses the reference, reads the bytes once, resolves
    the selector through Core against that retained text
  → buildSourceBundle() hashes each source and the manifest over them
  → begin() writes definition, manifest and BLOBs in one transaction
  → the authenticated retained closure it returns is what executes

A resume never reads the filesystem again. Under the executor lock, before any lifecycle mutation, it recomputes every retained length and content hash from the BLOBs it holds, and the bundle hash over the resulting manifest and mapping.

Two hashes, domain-separated. A source's content hash is SHA-256(field("executablemd.workflow.source.v2") || u64(len) || bytes). The bundle hash commits to the entrypoint, the ordered manifest and the component mapping under its own domain. field(v) = u32(utf8(v).length) || utf8(v), so no length-extension or boundary ambiguity between adjacent fields.

Logical paths are NFC Unicode scalar values, /-separated, with no leading or trailing /, no NUL, C0, DEL, backslash or #, and no empty, . or .. segment. They are compared and ordered by UTF-8 bytes — not by JavaScript <, which orders UTF-16 code units and would place U+FB01 after U+1F600.

Review guide

Start with: packages/workflow/src/storage/source-bundle.ts

Then review:

1. packages/workflow/src/storage/definition.ts — parseWorkflowDefinition dispatches on kind, not version, which is what preserves every existing WD1–WD35 diagnostic including WD4's $.version.
2. packages/workflow/src/lifecycle/source.ts and packages/workflow/src/deno/definition-source.ts — the retained-source union and its authentication.
3. packages/workflow/src/deno/transitions.ts and packages/workflow/src/deno/schema.ts — source verification before lifecycle mutation, and live schema version 2.
4. packages/cli/src/workflow-definition.ts — how a document reference becomes a bundle.

Look carefully at:

• settleSources() in transitions.ts: source verification happens under the executor lock and before stale recovery, attachment, replay or any lifecycle write. There is no rollback path, because nothing is written to roll back.
• validateLegacySources() in definition-source.ts: Workflow recomputes every returned blob identity from the bytes that came back. A reader that answers with different content than the descriptor names is refused, not trusted.

What must stay true

• A v1 run executes the same bytes it always did. Enforced by keeping schema version 1 and artifact format 1 byte-identical; checked by the released fixtures in workflow-run-storage.test.ts and xmd-artifact.test.ts.
• After begin succeeds, only its authenticated closure reaches import. Enforced by begin returning the closure and the CLI having no second path to a candidate; checked by WFK42, which reads the live staged workflow_definition_blob.content and asserts it equals the candidate bytes.
• packages/workflow/src/run.ts is the only Workflow module importing Git. Enforced by the import-boundary test, which pins that one existing import and its sole revParse() use and also catches dynamic and bare imports.
• Two descriptors of different versions are never one run. Enforced by compatibility.ts comparing kind before anything else; checked by the cross-version reuse case in workflow-run-storage.test.ts.
• Mutating the caller's buffer after transition entry is inert. Enforced by copying before validation and persisting only owned bytes; checked by WD49.

How to verify it

• scripts/tests/plugin-compiled.test.ts — "starts and resumes a document outside any repository" proves the compiled binary needs no repository at all, and fails if any Git lookup remains on the start or resume path.
• WD36–WD52 in workflow-definition.test.ts prove the hash vectors against fixed independent values, and fail if domain separation, field framing or UTF-8 ordering changes. The U+FB01 / U+1F600 pair fails specifically if ordering regresses to JavaScript <.
• WFC9b in workflow-cli.test.ts starts with #Pub* and expects the reported target #Publish, proving the selector is canonicalized through Core before storage rather than retained as the caller wrote it.
• WFD47 proves a file named od#d.md is addressable inside a bundle as od%23d.md but refused as an entrypoint, because # cannot be a logical path character.
• WS44 proves a start executes the closure begin returned rather than the candidate, which is unobservable at the storage layer because compatible reuse compares the whole manifest.
• WFF3 proves a fork's source is independent of its parent's.
• XE40 proves a v2 record is read at its own schema version, and fails if readRunRow reads every record as v1.
• WFK42 proves staged bytes are the candidate's, reading them live from SQLite rather than inferring them from a replay.

Manual check:

$ cd "$(mktemp -d)" && printf '# Hello\n' > doc.md
$ xmd workflow start ./doc.md      # no repository anywhere above this directory

Scope

Included

• Source-bundle definition v2, its closed parser, framing and hashing.
• Live schema version 2 and artifact format 2, alongside unchanged 1.
• Version-aware records, compatibility, lifecycle transitions, journals, public WorkflowRun, forks and artifacts.
• LegacyWorkflowSourceReader, the direct non-contextual closure a trusted host supplies for v1.
• CLI start, resume and fork from retained source, including target resolution.
• The five authoritative documents reconciled to the delivered contract.

Intentionally unchanged

• The artifact-backed fork. Deliberately out of scope; the plan names it undelivered.
• The Make Git and GitHub available in XMD runs and workflows #822 extraction. The legacy Git seam stays here and stays non-contextual; Make Git and GitHub available in XMD runs and workflows #822 consumes and moves it after this merges.
• Physical artifact container schema 1. Format 2 is a payload format inside it; it does not mean user_version = 2.
• Version-1 storage and artifact bytes. Preserved exactly, with released fixtures as golden evidence.

New abstractions

• SourceBundleWorkflowDefinitionV2 and its parser exist because identity must come from retained bytes rather than a Git object; consumed by storage, lifecycle, journals, artifacts and the CLI.

• LegacyWorkflowSourceReader exists because a v1 run's bytes live in a repository Workflow must not reach into. It is a direct closure rather than a context so that Make Git and GitHub available in XMD runs and workflows #822 can move it without leaving a contextual seam behind.

• RetainedDefinitionSources exists as the closed v1/v2 union every executable path consumes, so a candidate cannot remain a second route to execution.

• Each new abstraction has multiple concrete uses or a clear justification.

• No speculative functionality is included.

New dependencies

None. No manifest or lockfile changed.

Generated or mechanical changes

• The last two commits are test-host corrections with no production behavior:
  • 🐛 Narrow v1-only test hosts replaces union member reads in five v1-only test hosts with the already-exported isGitWorkflowDefinition / isGitWorkflowRunRecord predicates. No casts.
  • 🐛 Install the legacy source reader in v1 CLI test hosts passes legacySource: readLegacyDefinitionSource at twelve installation sites in two suites whose helpers call themselves "the production host" while omitting what production installs.

Risks and limitations

• V1 compatibility is byte-sensitive. The v1 schema and artifact declarations are immutable and must not be "shared" with v2 by widening an inventory or a serializer.
• The legacy reader is a real trust boundary. It returns content from outside Workflow's durable state. Workflow recomputes every blob identity it returns; a change that trusts the descriptor instead would silently accept substituted content.
• Recovery or rollback: revert the branch. Version-1 runs are untouched by it, and no version-2 run exists before it merges.

Scope confirmation

• Every changed file supports the purpose described above.
• Unrelated cleanup and formatting changes are excluded.
• Generated or mechanical changes are clearly identified.
• The description matches the final diff and test results.