✨ Slice E of #848: run and reconstruct one XMD entry in a REPL

architecture.md

@@ -5402,15 +5402,69 @@ below it keeps a clock of its own. The stream carries presentation time only —
 Journal records, model selection, routes and execution pause state are outside
 it.
 
+A frame that cannot be drawn ends the screen. A refused subscription, a refused
+reconcile and a refused render all raise, because the scope that owns the
+terminal is the only thing that restores its modes: a command that kept raw mode
+and the alternate screen while showing a picture it can no longer update has
+taken the terminal and stopped saying anything.
+
 The terminal itself is a contextual Api, and a runtime-named adapter installs
-it. Shared code never asks which runtime it is on; it asks for the size, bytes
-in, bytes out, raw mode and resize notifications. Whoever opens the terminal
+it. Shared code never asks which runtime it is on; it asks whether a person is
+at it, and for the size, bytes
+in, bytes out, raw mode and resize notifications. Whether there is a terminal is
+settled before any path is formed or any history created, so a piped invocation
+refuses instead of leaving an execution nobody can open. Whoever opens the terminal
 registers the release of each of those before taking it, so a cancellation
 between the two still gives the terminal back — and every exit, whether the run
 finished, refused, failed, was cancelled or reached end of input, stops the
 reader, removes every listener, restores the modes and writes the final reset
 exactly once.
 
+### The whole of it, and where the pieces are
+
+```text
+DurableEvents -> frozen ReplModel -> resolved immutable view
+  -> keyed Freedom tree -> semantic layout -> tty frame
+  -> typed action back to the root
+```
+
+| module | what it owns |
+| --- | --- |
+| `model.ts` | projecting one validated Journal prefix into a frozen model |
+| `route.ts` | the location grammar, and resolving one against a model |
+| `journal.ts` | the retained NDJSON stream and the repository over a root |
+| `session.ts` | admitting one entry, reopening one history, and the live overlay |
+| `expansion.ts` | pausing and continuing expansion over the public seams |
+| `elicitation.ts` | the one question shape this REPL presents |
+| `description.ts` | opaque immutable descriptions and the closed action boundary |
+| `reconcile.ts` | reconciling descriptions into one mounted Freedom tree |
+| `handoff.ts` | the acknowledged commit boundary |
+| `layout.ts` | deterministic placement at four sizes |
+| `renderer.ts` | drawing the mounted tree, and the frame map |
+| `frame.ts` | the one acknowledged frame stream |
+| `input.ts` | normalizing bytes into the closed event union |
+| `terminal.ts` · `terminal-host.ts` · `screen.ts` | the terminal Api, its portable half, and the one owner of its modes |
+| `components/` · `application.ts` | the screens, and the one state transition boundary |
+| `storage.ts` · `program.ts` | where histories live, and the command as one scope |
+
+A location is resolved against the retained history before a session opens.
+Opening one starts or resumes the execution, and replay past the retained prefix
+appends — so a route naming a scope or drawer the file never held is answered
+from the records already on disk, where being wrong costs nothing. Resolution
+takes one fact the model cannot hold: whether this process is asking a question.
+A waiting question is the one thing a Journal never records, so neither
+`settled` nor the recorded elicitations distinguish a history that is about to
+ask from one that never will, and a live drawer resolved from a history alone
+would replay a whole execution to mount nothing. It defaults to *not* asking, so
+a caller that cannot say refuses. The overlay's
+output is announced on its own stream, because it is the one thing a frame shows
+that leaves no record for anything else to observe.
+
+`specs/repl-spec.md` describes the product this assembles. The runtime-named
+`{deno,node,bun,compiled}-repl.ts` modules state their own platform's data
+directory, identifier and filesystem operations; nothing under `repl/` names a
+runtime.
+
 ## Changing these rules
 
 Spec, tests, and mechanics move together, in the same PR. If a workaround

packages/cli/src/bun-repl-terminal.ts

@@ -21,6 +21,11 @@ import type { ReplTerminalSize } from "./repl/terminal.ts";
 /** Install the Bun-backed terminal for the calling scope. */
 export function useBunReplTerminal(): Operation<void> {
   return installReplTerminal({
+    interactive(): boolean {
+      // Both ends, because the REPL reads keys from one and draws frames on
+      // the other: a redirected half is a half this product cannot run on.
+      return process.stdin.isTTY === true && process.stdout.isTTY === true;
+    },
     size(): ReplTerminalSize {
       return { columns: process.stdout.columns, rows: process.stdout.rows };
     },

packages/cli/src/bun-repl.ts

@@ -0,0 +1,31 @@
+/**
+ * The REPL's host on Bun.
+ *
+ * Named for its runtime because every answer here is one only the running
+ * process can give: the platform and home directory it is standing on, a random
+ * identifier, and the two filesystem operations `@effectionx/fs` does not offer.
+ * Nothing registers it — `bun.ts` installs it when `xmd repl` is the command
+ * that was selected.
+ */
+
+import { randomBytes } from "node:crypto";
+import { appendFile, open } from "node:fs/promises";
+import { homedir } from "node:os";
+import { sep } from "node:path";
+import process from "node:process";
+import type { Operation } from "effection";
+import { installReplHost, platformDataRoot } from "./repl-assembly.ts";
+import { useBunReplTerminal } from "./bun-repl-terminal.ts";
+
+/** Install everything the REPL needs from this host. */
+export function* useBunRepl(): Operation<void> {
+  yield* installReplHost({
+    dataRoot: () => platformDataRoot(process.platform, homedir(), process.env, sep),
+    // Sixteen random bytes as hex: URL-safe, opaque, and not derived from a
+    // path, a clock or a counter, none of which a location should leak.
+    identify: () => randomBytes(16).toString("hex"),
+    createExclusive: (path) => open(path, "wx").then((handle) => handle.close()),
+    appendRecord: (path, record) => appendFile(path, record),
+  });
+  yield* useBunReplTerminal();
+}

packages/cli/src/bun.ts

@@ -11,6 +11,7 @@ import process from "node:process";
 import { API, useHostFiles } from "@executablemd/runtime";
 import { compileTempFile } from "@executablemd/core";
 import { runXmd, XMD_VERSION } from "./cli.ts";
+import { useBunRepl } from "./bun-repl.ts";
 import { readInputStream } from "./standard-input.ts";
 import { importPluginModule } from "./host-plugin-modules.ts";
 import type { UpgradeAssembly } from "./upgrade.ts";
@@ -76,5 +77,6 @@ await main(function* (args) {
     importPluginModule,
     unsupportedWorkflowHost,
     unassembledMachineSessions(),
+    useBunRepl,
   );
 });

packages/cli/src/cli.ts

@@ -83,6 +83,8 @@ import type {
 } from "@executablemd/core";
 import { useAcpxProvider } from "@executablemd/acp";
 import { command as hostCommand, cwd } from "@executablemd/runtime";
+import { runReplProgram } from "./repl/program.ts";
+import { decodeLocation } from "./repl/route.ts";
 import { renderAgentOptions, renderAgentOptionsJson, scanAgentArgs } from "./agent-options.ts";
 import type { MachineSessionAssembly } from "./session-coordinator.ts";
 import {
@@ -278,6 +280,30 @@ const runConfig = object({
   ...executionFields,
 });
 
+/** What `xmd --help` says the repl command is for. */
+const REPL_DESCRIPTION = "Run and reconstruct one XMD entry in an interactive terminal.";
+
+/**
+ * `xmd repl` — one entry, one terminal, one retained history.
+ *
+ * The optional positional is a location this command printed earlier. With one,
+ * the command reopens exactly that retained history and selects exactly what the
+ * location names; with none, it starts a fresh execution with an empty draft.
+ *
+ * No option a run configures appears here. This command renders no file, takes
+ * no document reference and spends no model turn beyond what the one entry a
+ * person types asks for, so a permission mode, an exec deadline, a props file
+ * and a journal each configure work this command never performs.
+ */
+const replConfig = object({
+  location: {
+    description:
+      "a location this REPL printed earlier — `xmd://repl/<execution>/<surface>...` — to reopen " +
+      "that retained history and the exact view it names",
+    ...field(z.string().optional(), cli.argument()),
+  },
+});
+
 /** What `xmd --help` says the plan command is for. */
 const PLAN_DESCRIPTION =
   "Turn a request into an XMD Plan, review it, and write the approved source.";
@@ -503,6 +529,7 @@ const xmd = program({
       syntax: syntaxConfig,
       agent: { ...agentConfig, description: AGENT_DESCRIPTION },
       upgrade: { ...upgradeConfig, description: UPGRADE_DESCRIPTION },
+      repl: { ...replConfig, description: REPL_DESCRIPTION },
       "test-agent": testAgentConfig,
       workflow: workflowConfig,
     },
@@ -520,6 +547,77 @@ const UPGRADE_JOURNAL_ALIAS = "-j";
 /** Everything the command accepts, as help and refusals name it. */
 const UPGRADE_OPTIONS: readonly string[] = [...UPGRADE_SWITCHES, UPGRADE_JOURNAL];
 
+/**
+ * What `xmd repl --help` says beyond its argument list.
+ *
+ * The one-entry limit is stated because it is the product, not a restriction to
+ * be worked around: an execution admits one entry, and reopening its location
+ * shows that entry's history rather than offering a second.
+ */
+const REPL_HELP = [
+  "ONE ENTRY",
+  "",
+  "  xmd repl opens an empty draft in a full-screen terminal. Type or paste one",
+  "  XMD entry and submit it; the REPL runs it, and you can pause its expansion,",
+  "  look at an earlier point in its history, answer a question it asks and read",
+  "  what it produced.",
+  "",
+  "  An execution admits exactly one entry. The screen always shows a location",
+  "  of the form xmd://repl/<execution>/<surface>..., and the command prints the",
+  "  one it ended at. Passing that location back reopens the same retained",
+  "  history and selects the same view — in this process or another one, from the",
+  "  history file alone.",
+].join("\n");
+
+/**
+ * How a host installs everything the REPL needs from it.
+ *
+ * A value the entrypoint supplies rather than something shared code reaches for,
+ * exactly as the standard-input reader and the plugin loader are: a per-user data
+ * directory, an exclusive create, an append and a terminal are host capabilities,
+ * and the shared command names none of them.
+ */
+export type ReplHostInstaller = () => Operation<void>;
+
+/**
+ * What fixed grammar refuses about one `xmd repl` command line.
+ *
+ * Pure over argv: it reads nothing, so a malformed line is answered before this
+ * command has taken a directory, a file or a terminal. `xmd repl` defines no
+ * option of its own, which is why anything option-shaped is refused rather than
+ * ignored — a caller who wrote `--json` asked for something, and silence would
+ * let them believe they got it.
+ */
+export function replGrammarError(
+  args: readonly string[],
+  location: string | undefined,
+): string | undefined {
+  // The command's own name is the first token, as it is for every command that
+  // reads its line directly.
+  const rest = args.slice(1);
+  const option = rest.find((token) => token.startsWith("-") && token !== "-");
+  if (option !== undefined) {
+    return (
+      `unrecognized option for xmd repl: ${option} — xmd repl takes one optional location ` +
+      "and no options"
+    );
+  }
+  const positional = rest.filter((token) => !token.startsWith("-"));
+  if (positional.length > 1) {
+    return (
+      "xmd repl takes at most one location. This REPL admits one entry per execution, so " +
+      "there is no second document to name."
+    );
+  }
+  if (location !== undefined) {
+    const decoded = decodeLocation(location);
+    if (!decoded.ok) {
+      return `xmd repl: ${decoded.error.message}`;
+    }
+  }
+  return undefined;
+}
+
 /** What fixed grammar establishes about one `xmd upgrade` command line. */
 interface UpgradeScan {
   /** The exact tag the caller named, or `null` for the latest stable release. */
@@ -2303,6 +2401,7 @@ const COMMAND_NAMES = [
   "syntax",
   "upgrade",
   "agent",
+  "repl",
   "test-agent",
   "workflow",
 ];
@@ -2429,7 +2528,9 @@ function renderHelp(phase: PropsPhase): string {
           ? UPGRADE_HELP
           : command === "agent"
             ? AGENT_OPTIONS_HELP
-            : "";
+            : command === "repl"
+              ? REPL_HELP
+              : "";
   const withSource = epilogue === "" ? base : `${base}\n\n${epilogue}`;
 
   if (!phase.root) {
@@ -2516,6 +2617,7 @@ function* dispatch(
   readStandardInput: StandardInputReader,
   workflowHost: WorkflowHost | undefined,
   sessions: MachineSessionAssembly | undefined,
+  installRepl: ReplHostInstaller | undefined,
   /**
    * What this invocation's Plugins installed.
    *
@@ -2721,6 +2823,44 @@ function* dispatch(
       }
       break;
     }
+    case "repl": {
+      // The command's whole grammar is one optional location, so anything else
+      // on the line is refused here — before a per-user directory is formed,
+      // before a history file is created or opened, and before the terminal's
+      // modes are touched. A refusal that had already taken the terminal would
+      // print into an alternate screen nobody is looking at.
+      const stray = replGrammarError(helpRequest.args, command.config.location);
+      if (stray !== undefined) {
+        console.error(stray);
+        yield* exit(1);
+        break;
+      }
+      if (installRepl === undefined) {
+        console.error(
+          "xmd repl: this host assembles no interactive terminal, so there is nothing to open.",
+        );
+        yield* exit(1);
+        break;
+      }
+      yield* installRepl();
+      const ran = yield* runReplProgram(
+        command.config.location === undefined ? {} : { location: command.config.location },
+      );
+      if (!ran.ok) {
+        console.error(`xmd repl: ${ran.error.message}`);
+        yield* exit(1);
+        break;
+      }
+      // The location it ended at, so a person can reopen exactly this view.
+      const written = yield* deliverWhole(`${ran.value.location}\n`, process.stdout);
+      if (!written.ok) {
+        console.error(
+          `xmd repl: stdout did not accept the whole location: ${describeError(written.error)}`,
+        );
+        yield* exit(1);
+      }
+      break;
+    }
     case "upgrade": {
       // Fixed grammar first, and it reads nothing: a command line this command
       // does not define is answered before the packaged policy exists, before
@@ -3080,6 +3220,13 @@ export function* runXmd(
   // owns the session or which build it belongs to. A caller that names none
   // gets no machine sessions at all, which is the ordinary ACP behaviour.
   sessions?: MachineSessionAssembly,
+  // How this host assembles the interactive REPL: a per-user data directory, an
+  // opaque execution name, exclusive create and append, and a terminal. Only the
+  // `repl` command calls it, so help and every other command reach none of it —
+  // which is what keeps raw mode and a data directory off their path. A host that
+  // names none has no REPL, and says so rather than opening a terminal it cannot
+  // restore.
+  installRepl?: ReplHostInstaller,
 ): Operation<void> {
   // Before every scanner and before anything reads a path: what a command line
   // selects is read from the argv the caller wrote, and the tokens that
@@ -3113,6 +3260,7 @@ export function* runXmd(
     loadPluginModule,
     installWorkflowHost,
     sessions,
+    installRepl,
   );
 }
 
@@ -3174,6 +3322,7 @@ function* runCommand(
   loadPluginModule: PluginModuleLoader,
   installWorkflowHost: HostWorkflowInstaller,
   sessions: MachineSessionAssembly | undefined,
+  installRepl: ReplHostInstaller | undefined,
 ): Operation<void> {
   // First, so that no later scanner — help, properties, agent flags — can
   // mistake the inline document's own text for an option.
@@ -3238,6 +3387,7 @@ function* runCommand(
       readStandardInput,
       workflowHost,
       sessions,
+      installRepl,
       plugins,
     );
 

packages/cli/src/compiled-repl-terminal.ts

@@ -33,6 +33,7 @@ export function* useCompiledReplTerminal(): Operation<void> {
     throw new Error("this host is not Deno, so it has no Deno terminal to install");
   }
   yield* installReplTerminal({
+    interactive: () => host.interactive(),
     size(): ReplTerminalSize {
       return host.consoleSize();
     },