diff --git a/architecture.md b/architecture.md index 3b6ccce5..fa26f8e2 100644 --- a/architecture.md +++ b/architecture.md @@ -5402,15 +5402,69 @@ below it keeps a clock of its own. The stream carries presentation time only — Journal records, model selection, routes and execution pause state are outside it. +A frame that cannot be drawn ends the screen. A refused subscription, a refused +reconcile and a refused render all raise, because the scope that owns the +terminal is the only thing that restores its modes: a command that kept raw mode +and the alternate screen while showing a picture it can no longer update has +taken the terminal and stopped saying anything. + The terminal itself is a contextual Api, and a runtime-named adapter installs -it. Shared code never asks which runtime it is on; it asks for the size, bytes -in, bytes out, raw mode and resize notifications. Whoever opens the terminal +it. Shared code never asks which runtime it is on; it asks whether a person is +at it, and for the size, bytes +in, bytes out, raw mode and resize notifications. Whether there is a terminal is +settled before any path is formed or any history created, so a piped invocation +refuses instead of leaving an execution nobody can open. Whoever opens the terminal registers the release of each of those before taking it, so a cancellation between the two still gives the terminal back — and every exit, whether the run finished, refused, failed, was cancelled or reached end of input, stops the reader, removes every listener, restores the modes and writes the final reset exactly once. +### The whole of it, and where the pieces are + +```text +DurableEvents -> frozen ReplModel -> resolved immutable view + -> keyed Freedom tree -> semantic layout -> tty frame + -> typed action back to the root +``` + +| module | what it owns | +| --- | --- | +| `model.ts` | projecting one validated Journal prefix into a frozen model | +| `route.ts` | the location grammar, and resolving one against a model | +| `journal.ts` | the retained NDJSON stream and the repository over a root | +| `session.ts` | admitting one entry, reopening one history, and the live overlay | +| `expansion.ts` | pausing and continuing expansion over the public seams | +| `elicitation.ts` | the one question shape this REPL presents | +| `description.ts` | opaque immutable descriptions and the closed action boundary | +| `reconcile.ts` | reconciling descriptions into one mounted Freedom tree | +| `handoff.ts` | the acknowledged commit boundary | +| `layout.ts` | deterministic placement at four sizes | +| `renderer.ts` | drawing the mounted tree, and the frame map | +| `frame.ts` | the one acknowledged frame stream | +| `input.ts` | normalizing bytes into the closed event union | +| `terminal.ts` · `terminal-host.ts` · `screen.ts` | the terminal Api, its portable half, and the one owner of its modes | +| `components/` · `application.ts` | the screens, and the one state transition boundary | +| `storage.ts` · `program.ts` | where histories live, and the command as one scope | + +A location is resolved against the retained history before a session opens. +Opening one starts or resumes the execution, and replay past the retained prefix +appends — so a route naming a scope or drawer the file never held is answered +from the records already on disk, where being wrong costs nothing. Resolution +takes one fact the model cannot hold: whether this process is asking a question. +A waiting question is the one thing a Journal never records, so neither +`settled` nor the recorded elicitations distinguish a history that is about to +ask from one that never will, and a live drawer resolved from a history alone +would replay a whole execution to mount nothing. It defaults to *not* asking, so +a caller that cannot say refuses. The overlay's +output is announced on its own stream, because it is the one thing a frame shows +that leaves no record for anything else to observe. + +`specs/repl-spec.md` describes the product this assembles. The runtime-named +`{deno,node,bun,compiled}-repl.ts` modules state their own platform's data +directory, identifier and filesystem operations; nothing under `repl/` names a +runtime. + ## Changing these rules Spec, tests, and mechanics move together, in the same PR. If a workaround diff --git a/packages/cli/src/bun-repl-terminal.ts b/packages/cli/src/bun-repl-terminal.ts index b4bf3f5e..8ddd05cc 100644 --- a/packages/cli/src/bun-repl-terminal.ts +++ b/packages/cli/src/bun-repl-terminal.ts @@ -21,6 +21,11 @@ import type { ReplTerminalSize } from "./repl/terminal.ts"; /** Install the Bun-backed terminal for the calling scope. */ export function useBunReplTerminal(): Operation { return installReplTerminal({ + interactive(): boolean { + // Both ends, because the REPL reads keys from one and draws frames on + // the other: a redirected half is a half this product cannot run on. + return process.stdin.isTTY === true && process.stdout.isTTY === true; + }, size(): ReplTerminalSize { return { columns: process.stdout.columns, rows: process.stdout.rows }; }, diff --git a/packages/cli/src/bun-repl.ts b/packages/cli/src/bun-repl.ts new file mode 100644 index 00000000..0a729d0f --- /dev/null +++ b/packages/cli/src/bun-repl.ts @@ -0,0 +1,31 @@ +/** + * The REPL's host on Bun. + * + * Named for its runtime because every answer here is one only the running + * process can give: the platform and home directory it is standing on, a random + * identifier, and the two filesystem operations `@effectionx/fs` does not offer. + * Nothing registers it — `bun.ts` installs it when `xmd repl` is the command + * that was selected. + */ + +import { randomBytes } from "node:crypto"; +import { appendFile, open } from "node:fs/promises"; +import { homedir } from "node:os"; +import { sep } from "node:path"; +import process from "node:process"; +import type { Operation } from "effection"; +import { installReplHost, platformDataRoot } from "./repl-assembly.ts"; +import { useBunReplTerminal } from "./bun-repl-terminal.ts"; + +/** Install everything the REPL needs from this host. */ +export function* useBunRepl(): Operation { + yield* installReplHost({ + dataRoot: () => platformDataRoot(process.platform, homedir(), process.env, sep), + // Sixteen random bytes as hex: URL-safe, opaque, and not derived from a + // path, a clock or a counter, none of which a location should leak. + identify: () => randomBytes(16).toString("hex"), + createExclusive: (path) => open(path, "wx").then((handle) => handle.close()), + appendRecord: (path, record) => appendFile(path, record), + }); + yield* useBunReplTerminal(); +} diff --git a/packages/cli/src/bun.ts b/packages/cli/src/bun.ts index ce5cc9e2..886c6b69 100644 --- a/packages/cli/src/bun.ts +++ b/packages/cli/src/bun.ts @@ -11,6 +11,7 @@ import process from "node:process"; import { API, useHostFiles } from "@executablemd/runtime"; import { compileTempFile } from "@executablemd/core"; import { runXmd, XMD_VERSION } from "./cli.ts"; +import { useBunRepl } from "./bun-repl.ts"; import { readInputStream } from "./standard-input.ts"; import { importPluginModule } from "./host-plugin-modules.ts"; import type { UpgradeAssembly } from "./upgrade.ts"; @@ -76,5 +77,6 @@ await main(function* (args) { importPluginModule, unsupportedWorkflowHost, unassembledMachineSessions(), + useBunRepl, ); }); diff --git a/packages/cli/src/cli.ts b/packages/cli/src/cli.ts index 432fec55..6a5b26ff 100755 --- a/packages/cli/src/cli.ts +++ b/packages/cli/src/cli.ts @@ -83,6 +83,8 @@ import type { } from "@executablemd/core"; import { useAcpxProvider } from "@executablemd/acp"; import { command as hostCommand, cwd } from "@executablemd/runtime"; +import { runReplProgram } from "./repl/program.ts"; +import { decodeLocation } from "./repl/route.ts"; import { renderAgentOptions, renderAgentOptionsJson, scanAgentArgs } from "./agent-options.ts"; import type { MachineSessionAssembly } from "./session-coordinator.ts"; import { @@ -278,6 +280,30 @@ const runConfig = object({ ...executionFields, }); +/** What `xmd --help` says the repl command is for. */ +const REPL_DESCRIPTION = "Run and reconstruct one XMD entry in an interactive terminal."; + +/** + * `xmd repl` — one entry, one terminal, one retained history. + * + * The optional positional is a location this command printed earlier. With one, + * the command reopens exactly that retained history and selects exactly what the + * location names; with none, it starts a fresh execution with an empty draft. + * + * No option a run configures appears here. This command renders no file, takes + * no document reference and spends no model turn beyond what the one entry a + * person types asks for, so a permission mode, an exec deadline, a props file + * and a journal each configure work this command never performs. + */ +const replConfig = object({ + location: { + description: + "a location this REPL printed earlier — `xmd://repl//...` — to reopen " + + "that retained history and the exact view it names", + ...field(z.string().optional(), cli.argument()), + }, +}); + /** What `xmd --help` says the plan command is for. */ const PLAN_DESCRIPTION = "Turn a request into an XMD Plan, review it, and write the approved source."; @@ -503,6 +529,7 @@ const xmd = program({ syntax: syntaxConfig, agent: { ...agentConfig, description: AGENT_DESCRIPTION }, upgrade: { ...upgradeConfig, description: UPGRADE_DESCRIPTION }, + repl: { ...replConfig, description: REPL_DESCRIPTION }, "test-agent": testAgentConfig, workflow: workflowConfig, }, @@ -520,6 +547,77 @@ const UPGRADE_JOURNAL_ALIAS = "-j"; /** Everything the command accepts, as help and refusals name it. */ const UPGRADE_OPTIONS: readonly string[] = [...UPGRADE_SWITCHES, UPGRADE_JOURNAL]; +/** + * What `xmd repl --help` says beyond its argument list. + * + * The one-entry limit is stated because it is the product, not a restriction to + * be worked around: an execution admits one entry, and reopening its location + * shows that entry's history rather than offering a second. + */ +const REPL_HELP = [ + "ONE ENTRY", + "", + " xmd repl opens an empty draft in a full-screen terminal. Type or paste one", + " XMD entry and submit it; the REPL runs it, and you can pause its expansion,", + " look at an earlier point in its history, answer a question it asks and read", + " what it produced.", + "", + " An execution admits exactly one entry. The screen always shows a location", + " of the form xmd://repl//..., and the command prints the", + " one it ended at. Passing that location back reopens the same retained", + " history and selects the same view — in this process or another one, from the", + " history file alone.", +].join("\n"); + +/** + * How a host installs everything the REPL needs from it. + * + * A value the entrypoint supplies rather than something shared code reaches for, + * exactly as the standard-input reader and the plugin loader are: a per-user data + * directory, an exclusive create, an append and a terminal are host capabilities, + * and the shared command names none of them. + */ +export type ReplHostInstaller = () => Operation; + +/** + * What fixed grammar refuses about one `xmd repl` command line. + * + * Pure over argv: it reads nothing, so a malformed line is answered before this + * command has taken a directory, a file or a terminal. `xmd repl` defines no + * option of its own, which is why anything option-shaped is refused rather than + * ignored — a caller who wrote `--json` asked for something, and silence would + * let them believe they got it. + */ +export function replGrammarError( + args: readonly string[], + location: string | undefined, +): string | undefined { + // The command's own name is the first token, as it is for every command that + // reads its line directly. + const rest = args.slice(1); + const option = rest.find((token) => token.startsWith("-") && token !== "-"); + if (option !== undefined) { + return ( + `unrecognized option for xmd repl: ${option} — xmd repl takes one optional location ` + + "and no options" + ); + } + const positional = rest.filter((token) => !token.startsWith("-")); + if (positional.length > 1) { + return ( + "xmd repl takes at most one location. This REPL admits one entry per execution, so " + + "there is no second document to name." + ); + } + if (location !== undefined) { + const decoded = decodeLocation(location); + if (!decoded.ok) { + return `xmd repl: ${decoded.error.message}`; + } + } + return undefined; +} + /** What fixed grammar establishes about one `xmd upgrade` command line. */ interface UpgradeScan { /** The exact tag the caller named, or `null` for the latest stable release. */ @@ -2303,6 +2401,7 @@ const COMMAND_NAMES = [ "syntax", "upgrade", "agent", + "repl", "test-agent", "workflow", ]; @@ -2429,7 +2528,9 @@ function renderHelp(phase: PropsPhase): string { ? UPGRADE_HELP : command === "agent" ? AGENT_OPTIONS_HELP - : ""; + : command === "repl" + ? REPL_HELP + : ""; const withSource = epilogue === "" ? base : `${base}\n\n${epilogue}`; if (!phase.root) { @@ -2516,6 +2617,7 @@ function* dispatch( readStandardInput: StandardInputReader, workflowHost: WorkflowHost | undefined, sessions: MachineSessionAssembly | undefined, + installRepl: ReplHostInstaller | undefined, /** * What this invocation's Plugins installed. * @@ -2721,6 +2823,44 @@ function* dispatch( } break; } + case "repl": { + // The command's whole grammar is one optional location, so anything else + // on the line is refused here — before a per-user directory is formed, + // before a history file is created or opened, and before the terminal's + // modes are touched. A refusal that had already taken the terminal would + // print into an alternate screen nobody is looking at. + const stray = replGrammarError(helpRequest.args, command.config.location); + if (stray !== undefined) { + console.error(stray); + yield* exit(1); + break; + } + if (installRepl === undefined) { + console.error( + "xmd repl: this host assembles no interactive terminal, so there is nothing to open.", + ); + yield* exit(1); + break; + } + yield* installRepl(); + const ran = yield* runReplProgram( + command.config.location === undefined ? {} : { location: command.config.location }, + ); + if (!ran.ok) { + console.error(`xmd repl: ${ran.error.message}`); + yield* exit(1); + break; + } + // The location it ended at, so a person can reopen exactly this view. + const written = yield* deliverWhole(`${ran.value.location}\n`, process.stdout); + if (!written.ok) { + console.error( + `xmd repl: stdout did not accept the whole location: ${describeError(written.error)}`, + ); + yield* exit(1); + } + break; + } case "upgrade": { // Fixed grammar first, and it reads nothing: a command line this command // does not define is answered before the packaged policy exists, before @@ -3080,6 +3220,13 @@ export function* runXmd( // owns the session or which build it belongs to. A caller that names none // gets no machine sessions at all, which is the ordinary ACP behaviour. sessions?: MachineSessionAssembly, + // How this host assembles the interactive REPL: a per-user data directory, an + // opaque execution name, exclusive create and append, and a terminal. Only the + // `repl` command calls it, so help and every other command reach none of it — + // which is what keeps raw mode and a data directory off their path. A host that + // names none has no REPL, and says so rather than opening a terminal it cannot + // restore. + installRepl?: ReplHostInstaller, ): Operation { // Before every scanner and before anything reads a path: what a command line // selects is read from the argv the caller wrote, and the tokens that @@ -3113,6 +3260,7 @@ export function* runXmd( loadPluginModule, installWorkflowHost, sessions, + installRepl, ); } @@ -3174,6 +3322,7 @@ function* runCommand( loadPluginModule: PluginModuleLoader, installWorkflowHost: HostWorkflowInstaller, sessions: MachineSessionAssembly | undefined, + installRepl: ReplHostInstaller | undefined, ): Operation { // First, so that no later scanner — help, properties, agent flags — can // mistake the inline document's own text for an option. @@ -3238,6 +3387,7 @@ function* runCommand( readStandardInput, workflowHost, sessions, + installRepl, plugins, ); diff --git a/packages/cli/src/compiled-repl-terminal.ts b/packages/cli/src/compiled-repl-terminal.ts index 48baf6ee..ca0a1d3f 100644 --- a/packages/cli/src/compiled-repl-terminal.ts +++ b/packages/cli/src/compiled-repl-terminal.ts @@ -33,6 +33,7 @@ export function* useCompiledReplTerminal(): Operation { throw new Error("this host is not Deno, so it has no Deno terminal to install"); } yield* installReplTerminal({ + interactive: () => host.interactive(), size(): ReplTerminalSize { return host.consoleSize(); }, diff --git a/packages/cli/src/compiled-repl.ts b/packages/cli/src/compiled-repl.ts new file mode 100644 index 00000000..ad106e71 --- /dev/null +++ b/packages/cli/src/compiled-repl.ts @@ -0,0 +1,31 @@ +/** + * The REPL's host inside the compiled binary. + * + * Named for its runtime because every answer here is one only the running + * process can give: the platform and home directory it is standing on, a random + * identifier, and the two filesystem operations `@effectionx/fs` does not offer. + * Nothing registers it — `compiled.ts` installs it when `xmd repl` is the command + * that was selected. + */ + +import { randomBytes } from "node:crypto"; +import { appendFile, open } from "node:fs/promises"; +import { homedir } from "node:os"; +import { sep } from "node:path"; +import process from "node:process"; +import type { Operation } from "effection"; +import { installReplHost, platformDataRoot } from "./repl-assembly.ts"; +import { useCompiledReplTerminal } from "./compiled-repl-terminal.ts"; + +/** Install everything the REPL needs from this host. */ +export function* useCompiledRepl(): Operation { + yield* installReplHost({ + dataRoot: () => platformDataRoot(process.platform, homedir(), process.env, sep), + // Sixteen random bytes as hex: URL-safe, opaque, and not derived from a + // path, a clock or a counter, none of which a location should leak. + identify: () => randomBytes(16).toString("hex"), + createExclusive: (path) => open(path, "wx").then((handle) => handle.close()), + appendRecord: (path, record) => appendFile(path, record), + }); + yield* useCompiledReplTerminal(); +} diff --git a/packages/cli/src/compiled.ts b/packages/cli/src/compiled.ts index 34f62cd5..5885be25 100644 --- a/packages/cli/src/compiled.ts +++ b/packages/cli/src/compiled.ts @@ -10,6 +10,7 @@ import process from "node:process"; import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd, XMD_VERSION } from "./cli.ts"; +import { useCompiledRepl } from "./compiled-repl.ts"; import { readInputStream } from "./standard-input.ts"; import { importPluginModule } from "./host-plugin-modules.ts"; import { compiledUpgradeAssembly } from "./compiled-upgrade.ts"; @@ -92,6 +93,7 @@ if (isCredentialHelperMode(process.argv.slice(2))) { importPluginModule, () => useDenoWorkflowHost(HELPER), useMachineSessions(), + useCompiledRepl, ); }); } diff --git a/packages/cli/src/deno-repl-terminal.ts b/packages/cli/src/deno-repl-terminal.ts index d5024aea..11e0f15e 100644 --- a/packages/cli/src/deno-repl-terminal.ts +++ b/packages/cli/src/deno-repl-terminal.ts @@ -28,6 +28,7 @@ export function* useDenoReplTerminal(): Operation { throw new Error("this host is not Deno, so it has no Deno terminal to install"); } yield* installReplTerminal({ + interactive: () => host.interactive(), size(): ReplTerminalSize { return host.consoleSize(); }, diff --git a/packages/cli/src/deno-repl.ts b/packages/cli/src/deno-repl.ts new file mode 100644 index 00000000..a8cd25ac --- /dev/null +++ b/packages/cli/src/deno-repl.ts @@ -0,0 +1,31 @@ +/** + * The REPL's host on Deno. + * + * Named for its runtime because every answer here is one only the running + * process can give: the platform and home directory it is standing on, a random + * identifier, and the two filesystem operations `@effectionx/fs` does not offer. + * Nothing registers it — `deno.ts` installs it when `xmd repl` is the command + * that was selected. + */ + +import { randomBytes } from "node:crypto"; +import { appendFile, open } from "node:fs/promises"; +import { homedir } from "node:os"; +import { sep } from "node:path"; +import process from "node:process"; +import type { Operation } from "effection"; +import { installReplHost, platformDataRoot } from "./repl-assembly.ts"; +import { useDenoReplTerminal } from "./deno-repl-terminal.ts"; + +/** Install everything the REPL needs from this host. */ +export function* useDenoRepl(): Operation { + yield* installReplHost({ + dataRoot: () => platformDataRoot(process.platform, homedir(), process.env, sep), + // Sixteen random bytes as hex: URL-safe, opaque, and not derived from a + // path, a clock or a counter, none of which a location should leak. + identify: () => randomBytes(16).toString("hex"), + createExclusive: (path) => open(path, "wx").then((handle) => handle.close()), + appendRecord: (path, record) => appendFile(path, record), + }); + yield* useDenoReplTerminal(); +} diff --git a/packages/cli/src/deno-terminal-surface.ts b/packages/cli/src/deno-terminal-surface.ts index 18332fe0..8a7df31e 100644 --- a/packages/cli/src/deno-terminal-surface.ts +++ b/packages/cli/src/deno-terminal-surface.ts @@ -15,6 +15,8 @@ /** What a Deno host offers a terminal. */ export interface DenoTerminalSurface { + /** Whether both of this host's standard streams are a terminal. */ + interactive(): boolean; consoleSize(): { readonly columns: number; readonly rows: number }; /** Hand bytes over, resolving with how many it took. */ write(bytes: Uint8Array): Promise; @@ -73,11 +75,15 @@ export function denoTerminalSurface(): DenoTerminalSurface | undefined { const write = callable(stdout, "write"); const writeSync = callable(stdout, "writeSync"); const setRaw = callable(stdin, "setRaw"); + const readingTerminal = callable(stdin, "isTerminal"); + const writingTerminal = callable(stdout, "isTerminal"); const readable: unknown = Reflect.get(stdin, "readable"); if ( write === undefined || writeSync === undefined || setRaw === undefined || + readingTerminal === undefined || + writingTerminal === undefined || typeof readable !== "object" || readable === null || !(Symbol.asyncIterator in readable) @@ -87,6 +93,11 @@ export function denoTerminalSurface(): DenoTerminalSurface | undefined { const source = readable; return { + interactive(): boolean { + // Both ends, because the REPL reads keys from one and draws frames on + // the other: a redirected half is a half this product cannot run on. + return readingTerminal() === true && writingTerminal() === true; + }, consoleSize(): { readonly columns: number; readonly rows: number } { const reported: unknown = consoleSize(); if (typeof reported !== "object" || reported === null) { diff --git a/packages/cli/src/deno.ts b/packages/cli/src/deno.ts index d13039f8..a0b7ff5c 100644 --- a/packages/cli/src/deno.ts +++ b/packages/cli/src/deno.ts @@ -13,6 +13,7 @@ import process from "node:process"; import { API, useHostFiles } from "@executablemd/runtime"; import { compileDataUri } from "@executablemd/core"; import { runXmd, XMD_VERSION } from "./cli.ts"; +import { useDenoRepl } from "./deno-repl.ts"; import { readInputStream } from "./standard-input.ts"; import { importPluginModule } from "./host-plugin-modules.ts"; import type { UpgradeAssembly } from "./upgrade.ts"; @@ -111,6 +112,7 @@ if (isCredentialHelperMode(process.argv.slice(2))) { importPluginModule, () => useDenoWorkflowHost(HELPER), useMachineSessions(), + useDenoRepl, ); }); } diff --git a/packages/cli/src/node-repl-terminal.ts b/packages/cli/src/node-repl-terminal.ts index 58d55e96..0bef3c33 100644 --- a/packages/cli/src/node-repl-terminal.ts +++ b/packages/cli/src/node-repl-terminal.ts @@ -21,6 +21,11 @@ import type { ReplTerminalSize } from "./repl/terminal.ts"; /** Install the Node-backed terminal for the calling scope. */ export function useNodeReplTerminal(): Operation { return installReplTerminal({ + interactive(): boolean { + // Both ends, because the REPL reads keys from one and draws frames on + // the other: a redirected half is a half this product cannot run on. + return process.stdin.isTTY === true && process.stdout.isTTY === true; + }, size(): ReplTerminalSize { return { columns: process.stdout.columns, rows: process.stdout.rows }; }, diff --git a/packages/cli/src/node-repl.ts b/packages/cli/src/node-repl.ts new file mode 100644 index 00000000..630d61a6 --- /dev/null +++ b/packages/cli/src/node-repl.ts @@ -0,0 +1,31 @@ +/** + * The REPL's host on Node. + * + * Named for its runtime because every answer here is one only the running + * process can give: the platform and home directory it is standing on, a random + * identifier, and the two filesystem operations `@effectionx/fs` does not offer. + * Nothing registers it — `node.ts` installs it when `xmd repl` is the command + * that was selected. + */ + +import { randomBytes } from "node:crypto"; +import { appendFile, open } from "node:fs/promises"; +import { homedir } from "node:os"; +import { sep } from "node:path"; +import process from "node:process"; +import type { Operation } from "effection"; +import { installReplHost, platformDataRoot } from "./repl-assembly.ts"; +import { useNodeReplTerminal } from "./node-repl-terminal.ts"; + +/** Install everything the REPL needs from this host. */ +export function* useNodeRepl(): Operation { + yield* installReplHost({ + dataRoot: () => platformDataRoot(process.platform, homedir(), process.env, sep), + // Sixteen random bytes as hex: URL-safe, opaque, and not derived from a + // path, a clock or a counter, none of which a location should leak. + identify: () => randomBytes(16).toString("hex"), + createExclusive: (path) => open(path, "wx").then((handle) => handle.close()), + appendRecord: (path, record) => appendFile(path, record), + }); + yield* useNodeReplTerminal(); +} diff --git a/packages/cli/src/node.ts b/packages/cli/src/node.ts index ad8041b1..d72a3b86 100755 --- a/packages/cli/src/node.ts +++ b/packages/cli/src/node.ts @@ -17,6 +17,7 @@ import process from "node:process"; import { API, useHostFiles } from "@executablemd/runtime"; import { compileTempFile } from "@executablemd/core"; import { runXmd, XMD_VERSION } from "./cli.ts"; +import { useNodeRepl } from "./node-repl.ts"; import { readInputStream } from "./standard-input.ts"; import { importPluginModule } from "./host-plugin-modules.ts"; import type { UpgradeAssembly } from "./upgrade.ts"; @@ -84,5 +85,6 @@ await main(function* (args) { importPluginModule, unsupportedWorkflowHost, unassembledMachineSessions(), + useNodeRepl, ); }); diff --git a/packages/cli/src/repl-assembly.ts b/packages/cli/src/repl-assembly.ts new file mode 100644 index 00000000..4a28e907 --- /dev/null +++ b/packages/cli/src/repl-assembly.ts @@ -0,0 +1,86 @@ +/** + * The portable half of a REPL host, over what a runtime states about itself. + * + * Three things only a host can answer: where this person's data lives, what to + * call a new execution, and how to create a file that must not already exist and + * append a line to it. `@effectionx/fs` has neither of those last two — an + * exclusive create and an append are the two operations it does not offer — so + * they arrive here as asynchronous primitives the entrypoint supplies, adapted + * with `until` and never called synchronously. + * + * Nothing registers this. A runtime entrypoint installs it when the REPL command + * is the one selected, which is what keeps a per-user directory and a terminal + * off the path of `xmd run`. + */ + +import { type Operation, until } from "effection"; +import { ReplHost } from "./repl/host.ts"; +import { ReplStorage } from "./repl/storage.ts"; + +/** What a runtime states about itself for the REPL to use. */ +export interface ReplHostCapabilities { + /** This platform's per-user data directory. */ + dataRoot(): string; + /** A fresh opaque, URL-safe name for one execution. */ + identify(): string; + /** Create this file, failing if it already exists. */ + createExclusive(path: string): Promise; + /** Append this already-terminated record to that file. */ + appendRecord(path: string, record: string): Promise; +} + +/** Install one runtime's REPL host for the calling scope. */ +export function* installReplHost(host: ReplHostCapabilities): Operation { + yield* ReplStorage.around( + { + // deno-lint-ignore require-yield + *dataRoot(): Operation { + return host.dataRoot(); + }, + }, + { at: "min" }, + ); + yield* ReplHost.around( + { + // deno-lint-ignore require-yield + *identify(): Operation { + return host.identify(); + }, + *createExclusive([path]: [string]): Operation { + yield* until(host.createExclusive(path)); + }, + *appendRecord([path, record]: [string, string]): Operation { + yield* until(host.appendRecord(path, record)); + }, + }, + { at: "min" }, + ); +} + +/** + * Where a platform keeps per-user application data. + * + * Stated from what the process says about itself, at the entrypoint that knows + * it is a process. The three answers are the platform conventions: macOS keeps + * application support beside the user's library, Windows keeps local application + * data in its own variable, and everything else follows the XDG base directory + * specification with its documented default. + */ +export function platformDataRoot( + platform: string, + home: string, + environment: { readonly [name: string]: string | undefined }, + separator: string, +): string { + if (platform === "darwin") { + return [home, "Library", "Application Support"].join(separator); + } + if (platform === "win32") { + const local = environment["LOCALAPPDATA"]; + return local !== undefined && local.length > 0 + ? local + : [home, "AppData", "Local"].join(separator); + } + const xdg = environment["XDG_DATA_HOME"]; + return xdg !== undefined && xdg.length > 0 ? xdg : [home, ".local", "share"].join(separator); +} diff --git a/packages/cli/src/repl/application.ts b/packages/cli/src/repl/application.ts new file mode 100644 index 00000000..4651555d --- /dev/null +++ b/packages/cli/src/repl/application.ts @@ -0,0 +1,1028 @@ +/** + * The product: what is on screen, and what every action changes. + * + * One boundary owns state. Everything a frame shows is derived here from a + * resolved model prefix plus this process's explicit overlay, and every action + * the tree returns is answered here. Components receive detached view data and + * hand back a semantic action; they hold no session, no repository, no route + * mutator and no host operation, so a keystroke cannot reach the Journal except + * through this reduction. + * + * ## A prefix is not the present + * + * A view frozen at a history marker is read only and fills nothing from the + * live head: no live output, no waiting question, no pause capability. That is + * why the overlay is a separate member rather than merged into the model — + * merging them is exactly the mistake that shows somebody the present while + * they are looking at the past. + * + * ## Navigation that fails changes nothing + * + * Every navigating action builds a candidate route and resolves it against the + * model before it is adopted. A candidate that does not resolve leaves the + * standing route, selection and draft exactly as they were and reports why. + * There is no half-applied navigation, because a route is adopted whole or not + * at all. + */ + +import { Ok, type Result } from "effection"; +import type { Json } from "@executablemd/durable-streams"; + +import { describe as describeNode } from "./description.ts"; +import type { ReplDescription } from "./description.ts"; +import { drawerWidth, HISTORY_ROWS, NARROW, surfaceWidth } from "./layout.ts"; +import type { ReplSurface as ReplPlacedSurface, ReplSurfaceCell } from "./layout.ts"; +import type { ReplTerminalSize } from "./terminal.ts"; +import { decodeLocation, encodeLocation, resolveLocation } from "./route.ts"; +import type { ReplDrawerRef, ReplRoute, ReplSelection, ReplSurface } from "./route.ts"; +import type { ReplModel, ReplRow, ReplScope } from "./model.ts"; +import type { ReplQuestion } from "./elicitation.ts"; +import type { ExpansionState } from "./expansion.ts"; +import type { ReplTree } from "./reconcile.ts"; +import { DRAWER, FIELD, LINE, REFUSAL, SELECT_ROW } from "./components/rows.ts"; +import type { ReplAction } from "./components/actions.ts"; + +export type { ReplAction }; + +/** What this process holds that the Journal does not. */ +export interface ReplLive { + /** Output no recorded outcome has replaced yet. */ + readonly output: string; + /** The question waiting right now, or none. */ + readonly question: ReplQuestion | undefined; + readonly expansion: ExpansionState; + /** Whether this process holds the continuations, and so may pause at all. */ + readonly pausable: boolean; +} + +/** Everything typed and not yet committed anywhere. */ +export interface ReplState { + readonly route: ReplRoute; + /** The entry draft, before an entry exists. */ + readonly draft: string; + /** The answer being typed into the waiting question. */ + readonly answer: string; + /** Why the last action changed nothing, or none. */ + readonly refusal: string | undefined; +} + +/** What the root must perform, because a component cannot. */ +export type ReplIntent = + | { readonly kind: "none" } + | { readonly kind: "submit"; readonly source: string } + | { readonly kind: "pause" } + | { readonly kind: "continue" } + | { readonly kind: "answer"; readonly choice: string }; + +/** One reduction: the state that stands now, and what the root owes. */ +export interface ReplTransition { + readonly state: ReplState; + readonly intent: ReplIntent; +} + +/** One complete reading of the product, ready to describe. */ +export interface ReplView { + readonly state: ReplState; + readonly model: ReplModel; + readonly selection: ReplSelection; + readonly live: ReplLive; + /** The canonical location this view is at. */ + readonly location: string; + /** + * Why there is no view at all, when that is what happened. + * + * Only that. A refusal of one *action* is `state.refusal` and belongs in the + * footer beside the control that was refused — replacing the whole screen with + * it would take away the thing the person was working on in order to explain + * why it did not change. + */ + readonly refusal: string | undefined; + /** + * How much terminal there is. + * + * Carried because what a row may contain depends on where it will be put: a + * row longer than its region is reflowed into rows the layout never allocated, + * so whoever writes one has to know how wide it will be. + */ + readonly size: ReplTerminalSize; + /** + * The key of the control that held focus when this view was built. + * + * Derived, and therefore one frame behind: focus belongs to the mounted tree, + * and the tree is what answers where it is. A person has to be able to see + * which control their next keystroke reaches, so the marker is part of the view + * rather than something the renderer decorates. + */ + readonly focused: string | undefined; +} + +/** + * The state after an entry was admitted. + * + * The draft leaves the state and the location together, because what was typed is + * now the entry and a location carrying both would describe two different things + * at once. + */ +export function admitted(state: ReplState): ReplState { + return Object.freeze({ + ...state, + draft: "", + route: Object.freeze({ ...state.route, draft: undefined }), + refusal: undefined, + }); +} + +/** + * The state after a question accepted its answer. + * + * The question is over, so the drawer that was asking it is over too. Leaving + * `+elicit` in the route would print a location naming a drawer the topology no + * longer mounts — a URL that describes a screen nobody can be shown — and + * leaving the typed text in `answer` would offer it again as though it were + * still waiting to be sent. + */ +export function answered(state: ReplState): ReplState { + return Object.freeze({ + ...state, + answer: "", + route: Object.freeze({ + ...state.route, + drawers: Object.freeze(state.route.drawers.filter((drawer) => drawer.kind !== "live-elicit")), + }), + refusal: undefined, + }); +} + +/** The empty route one fresh execution starts at. */ +export function initialRoute(execution: string): ReplRoute { + return Object.freeze({ + execution, + surface: "repl", + scopes: Object.freeze([]), + drawers: Object.freeze([]), + at: undefined, + inspect: false, + draft: undefined, + }); +} + +/** The state one fresh execution starts in. */ +export function initialState(execution: string): ReplState { + return Object.freeze({ + route: initialRoute(execution), + draft: "", + answer: "", + refusal: undefined, + }); +} + +/** Read one location into the state it names. */ +export function stateFor(location: string): Result { + const decoded = decodeLocation(location); + if (!decoded.ok) { + return decoded; + } + return Ok( + Object.freeze({ + route: decoded.value, + draft: decoded.value.draft ?? "", + answer: "", + refusal: undefined, + }), + ); +} + +/** + * Build one view, or say why there is none. + * + * The selection is resolved here and nowhere else, so every surface below reads + * the same exact model objects rather than looking them up again and possibly + * differently. + */ +export function viewFor( + state: ReplState, + model: ReplModel, + live: ReplLive, + size: ReplTerminalSize, + focused?: string, +): Result { + // This process is the only thing that can say a question is waiting, so it + // says so here rather than leaving resolution to infer it from a history that + // does not record it. + const resolved = resolveLocation(model, state.route, live.question !== undefined); + if (!resolved.ok) { + return resolved; + } + // A frozen prefix shows nothing of the present. Stated once, here, rather + // than remembered at each surface that would otherwise reach for the overlay. + const shown: ReplLive = + state.route.at === undefined + ? live + : { + output: "", + question: undefined, + expansion: live.expansion, + pausable: false, + }; + return Ok( + Object.freeze({ + state, + model, + selection: resolved.value, + live: shown, + location: encodeLocation(state.route), + // Not `state.refusal`: a view exists, and what one action refused is said in + // the footer rather than in place of everything. + refusal: undefined, + size, + focused, + }), + ); +} + +/** The one view a cold open that cannot be projected gets. */ +export function refusedView( + state: ReplState, + reason: string, + size: ReplTerminalSize = NARROW, +): ReplView { + return Object.freeze({ + state, + model: EMPTY_MODEL, + selection: Object.freeze({ + route: state.route, + surface: state.route.surface, + entry: undefined, + ancestry: Object.freeze([]), + scope: undefined, + drawers: Object.freeze([]), + }), + live: Object.freeze({ + output: "", + question: undefined, + expansion: "playing", + pausable: false, + }), + location: encodeLocation(state.route), + refusal: reason, + size, + focused: "refusal", + }); +} + +const EMPTY_MODEL: ReplModel = Object.freeze({ + selection: undefined, + head: true, + entry: undefined, + settled: false, + terminal: undefined, + checkpoints: Object.freeze([]), + transcript: Object.freeze([]), +}); + +/** + * Answer one action. + * + * Total over the union, and pure: nothing here appends, opens, pauses or + * answers. What needs doing comes back as an intent for the root to perform, so + * the decision and the effect are separable and the decision is testable alone. + */ +export function reduceRepl( + state: ReplState, + action: ReplAction, + model: ReplModel, + live: ReplLive, +): ReplTransition { + const answering = state.route.drawers.some((drawer) => drawer.kind === "live-elicit"); + + switch (action.kind) { + case "type": { + if (answering) { + return settled({ ...state, answer: state.answer + action.text, refusal: undefined }); + } + if (model.entry !== undefined) { + return refuse(state, "this execution has admitted its entry, and an entry is immutable."); + } + return drafting(state, state.draft + action.text); + } + case "erase": { + if (answering) { + return settled({ ...state, answer: shortened(state.answer), refusal: undefined }); + } + if (model.entry !== undefined) { + return refuse(state, "this execution has admitted its entry, and an entry is immutable."); + } + return drafting(state, shortened(state.draft)); + } + case "submit": { + if (model.entry !== undefined) { + return refuse(state, "this REPL admits one entry, and this execution has admitted it."); + } + if (state.draft.length === 0) { + return refuse(state, "there is nothing to submit yet."); + } + // The draft stays until the entry exists. Clearing it here would lose + // somebody's document to a preflight refusal, which is the one moment they + // most need it back. + return { + state: Object.freeze({ ...state, refusal: undefined }), + intent: { kind: "submit", source: state.draft }, + }; + } + case "select-surface": { + return navigate( + state, + model, + { ...state.route, surface: action.surface, drawers: [] }, + live.question !== undefined, + ); + } + case "select-scope": { + return navigate( + state, + model, + { + ...state.route, + surface: "repl", + scopes: Object.freeze([...action.scopes]), + // A drawer named a thing inside the scope that was open. Selecting a + // different scope cannot keep it. + drawers: Object.freeze([]), + }, + live.question !== undefined, + ); + } + case "open-drawer": { + if (action.drawer.kind === "live-elicit") { + if (state.route.at !== undefined) { + return refuse( + state, + "a historical view cannot answer the question this process is asking.", + ); + } + if (live.question === undefined) { + return refuse(state, "nothing is being asked right now."); + } + } + return navigate( + state, + model, + { ...state.route, drawers: Object.freeze([...state.route.drawers, action.drawer]) }, + live.question !== undefined, + ); + } + case "close-drawer": { + if (state.route.drawers.length === 0) { + return refuse(state, "no drawer is open."); + } + const closing = state.route.drawers[state.route.drawers.length - 1]; + const remaining = Object.freeze(state.route.drawers.slice(0, -1)); + const closed = navigate( + state, + model, + { ...state.route, drawers: remaining }, + live.question !== undefined, + ); + // Dismissing the question's drawer discards what was typed into it. It is + // not an answer, and keeping it would offer it again as though it were. + return closing.kind === "live-elicit" + ? { ...closed, state: Object.freeze({ ...closed.state, answer: "" }) } + : closed; + } + case "select-marker": { + // Adopted rather than resolved here: a position is a *different reading* of + // the file, and this model is the one the view being left was built from. + // The root reprojects at the named position and verifies there, which is + // the only place the answer exists. + return settled({ + ...state, + route: Object.freeze({ ...state.route, at: action.marker, inspect: true }), + refusal: undefined, + }); + } + case "go-live": { + return settled({ + ...state, + route: Object.freeze({ + ...state.route, + at: undefined, + inspect: false, + // A recorded drawer may name something the head no longer selects. + drawers: Object.freeze([]), + }), + refusal: undefined, + }); + } + case "pause": { + if (!live.pausable) { + return refuse(state, "this process holds no live expansion to pause."); + } + return { state: Object.freeze({ ...state, refusal: undefined }), intent: { kind: "pause" } }; + } + case "continue": { + if (!live.pausable) { + return refuse(state, "this process holds no live expansion to continue."); + } + if (live.expansion !== "paused") { + // Nothing is held yet, so there is nothing to release. Resuming here + // would withdraw a pause that has not finished taking effect. + return refuse(state, "expansion is not paused, so nothing is held to continue."); + } + return { + state: Object.freeze({ ...state, refusal: undefined }), + intent: { kind: "continue" }, + }; + } + case "answer": { + if (state.route.at !== undefined) { + return refuse( + state, + "a historical view cannot answer the question this process is asking.", + ); + } + if (live.question === undefined) { + return refuse(state, "nothing is being asked right now."); + } + if (state.answer.length === 0) { + return refuse(state, "type one of the offered choices first."); + } + return { + state: Object.freeze({ ...state, refusal: undefined }), + intent: { kind: "answer", choice: state.answer }, + }; + } + } +} + +function settled(state: ReplState): ReplTransition { + return { state: Object.freeze(state), intent: { kind: "none" } }; +} + +function refuse(state: ReplState, reason: string): ReplTransition { + return { state: Object.freeze({ ...state, refusal: reason }), intent: { kind: "none" } }; +} + +/** The draft, and the canonical query that carries it. */ +function drafting(state: ReplState, draft: string): ReplTransition { + return settled({ + ...state, + draft, + // The draft is route state, so the location always says what would be + // submitted. It is the one thing in the URL that is not yet durable. + route: Object.freeze({ ...state.route, draft: draft.length === 0 ? undefined : draft }), + refusal: undefined, + }); +} + +/** + * Adopt a candidate route, or keep the one that works. + * + * Resolved against the model first: a route that selects nothing is not a view + * with empty regions, it is a navigation that did not happen. + */ +function navigate( + state: ReplState, + model: ReplModel, + candidate: ReplRoute, + asking: boolean, +): ReplTransition { + const route = Object.freeze({ ...candidate }); + const resolved = resolveLocation(model, route, asking); + if (!resolved.ok) { + return refuse(state, resolved.error.message); + } + return settled({ ...state, route, refusal: undefined }); +} + +/** One text unit shorter, counted in scalar values rather than code units. */ +function shortened(text: string): string { + const units = [...text]; + units.pop(); + return units.join(""); +} + +/** How a value reads in one line of a list. */ +function summarize(value: Json): string { + const written = JSON.stringify(value) ?? "null"; + return written.length <= SUMMARY_WIDTH ? written : `${written.slice(0, SUMMARY_WIDTH - 1)}…`; +} + +/** How much of a value one line of a list shows. */ +const SUMMARY_WIDTH = 20; + +/** + * One long string as rows that fit, in order, losing nothing. + * + * Each row is padded to the full width. A renderer writes what changed, so a row + * whose text got shorter would otherwise keep the tail of what used to be there — + * and a location with somebody else's characters on the end of it is worse than + * no location at all. + */ +function chunked(text: string, width: number): readonly string[] { + if (width < 1) { + // No surface to write on. A terminal too small to draw in draws a refusal + // and nothing else, so this row is never placed — but it must still be a row. + return [text]; + } + const parts: string[] = []; + for (let at = 0; at < text.length; at += width) { + parts.push(text.slice(at, at + width).padEnd(width, " ")); + } + return parts.length === 0 ? ["".padEnd(width, " ")] : parts; +} + +/** A cell the layout can place, with the key its description was given. */ +interface Described { + readonly key: string; + readonly description: ReplDescription; +} + +function row( + key: string, + label: string, + select: { readonly [name: string]: Json }, + options: { readonly focus?: true; readonly here?: string | undefined } = {}, +): Described { + return { + key, + description: describeNode({ + key, + component: SELECT_ROW, + input: { label, ...select, ...(options.here === key ? { focused: true } : {}) }, + ...(options.focus === undefined ? {} : { focus: options.focus }), + }), + }; +} + +function line(key: string, label: string): Described { + return { + key, + description: describeNode({ key, component: LINE, input: { label } }), + }; +} + +/** + * A row of a drawer, padded so the drawer covers what it is in front of. + * + * A renderer writes what changed, so a row that only writes its own text leaves + * whatever was underneath it visible from where its text ends — and a modal you + * can read the transcript through is not a modal. + */ +function drawerLine(key: string, label: string, width: number): Described { + return line(key, width < 1 ? label : label.padEnd(width, " ")); +} + +function field( + key: string, + prompt: string, + text: string, + purpose: "draft" | "answer", + options: { readonly focus?: true; readonly here?: string | undefined } = {}, +): Described { + return { + key, + description: describeNode({ + key, + component: FIELD, + input: { prompt, text, purpose, ...(options.here === key ? { focused: true } : {}) }, + ...(options.focus === undefined ? {} : { focus: options.focus }), + }), + }; +} + +/** + * Describe the whole screen. + * + * One flat set with keyed children for the drawer, because placement is not + * nesting: where a row appears is the layout's decision, and the only nesting + * that matters to the tree is what a modal must contain. + */ +export function describeApplication(view: ReplView): readonly ReplDescription[] { + return described(view).map((one) => one.description); +} + +function described(view: ReplView): readonly Described[] { + if (view.refusal !== undefined) { + return [ + { + key: "refusal", + description: describeNode({ + key: "refusal", + component: REFUSAL, + input: { + label: view.refusal, + // Somewhere to go only when there is somewhere: a cold open of a + // history that cannot be read has no earlier view to return to. + ...(view.state.route.at === undefined ? {} : { back: "live" }), + }, + focus: true, + }), + }, + ]; + } + + const items: Described[] = []; + const { model, selection, live, state } = view; + + items.push( + row( + "sessions:heading", + "Sessions", + { select: "surface", surface: "sessions" }, + { here: view.focused }, + ), + ); + // Present and empty. This REPL keeps one execution per invocation, and a + // Sessions surface that vanished when it held nothing would read as a feature + // that does not exist. + items.push(line("sessions:empty", " (none retained)")); + items.push( + row( + "entries:heading", + "Entries", + { select: "surface", surface: "repl" }, + { here: view.focused }, + ), + ); + + const entry = model.entry; + if (entry === undefined) { + items.push(line("entry:none", " 1. (not submitted)")); + } else { + items.push( + row( + "entry:1", + ` 1. ${entry.name}`, + { select: "scope", scopes: [entry.key] }, + { here: view.focused }, + ), + ); + for (const scope of nested(entry, [entry.key])) { + items.push( + row( + `scope:${scope.path.join("/")}`, + ` ${scope.label}`, + { + select: "scope", + scopes: scope.path, + }, + { here: view.focused }, + ), + ); + } + } + + for (const [index, transcript] of model.transcript.entries()) { + // One cell is one row, so a recorded row that holds several lines of output + // becomes several cells. A cell given more than one line would show only the + // first, which is the whole of what a reader would then believe was there. + for (const [offset, text] of describeRow(transcript).split("\n").entries()) { + items.push(line(`line:${index}:${offset}`, text)); + } + } + // The live overlay, explicitly below the recorded rows and explicitly labelled. + // Once the durable close exists its recorded output is in the transcript and + // this is empty, so the two never both claim to be the output. + if (live.output.length > 0) { + for (const [offset, text] of live.output.split("\n").entries()) { + items.push(line(`line:live:${offset}`, `… ${text}`)); + } + } + + const scope = selection.scope; + if (scope !== undefined) { + for (const binding of scope.bindings) { + items.push( + row( + `binding:${binding.name}`, + `${binding.name} = ${summarize(binding.value)}`, + { + select: "binding", + name: binding.name, + }, + { here: view.focused }, + ), + ); + } + for (const elicitation of scope.elicitations) { + items.push( + row( + `elicit:${elicitation.marker}`, + `answered ${elicitation.location}`, + { + select: "recorded-elicit", + marker: elicitation.marker, + }, + { here: view.focused }, + ), + ); + } + } + + // The way into history. The band above shows where the positions are; choosing + // an exact one is a drawer, because a position is something you select and the + // band is something you read. + items.push(row("footer:history", "[history]", { select: "history" }, { here: view.focused })); + if (state.route.at !== undefined) { + items.push(row("footer:live", "[live]", { select: "live" }, { here: view.focused })); + } + if (live.pausable) { + items.push( + // The control is what it does; the state is what expansion is doing. One + // label that changed between them would rename a control out from under + // whoever was reaching for it. + row( + "footer:pause", + live.expansion === "playing" ? "[pause]" : `[pause] ${live.expansion}`, + { select: "pause" }, + { here: view.focused }, + ), + ); + // Continue releases a continuation, so it exists exactly while one is + // held. Expansion that is *pausing* holds nothing yet — the walks it asked + // to stop have not all stopped — and a Continue offered there would cancel + // the pause somebody just asked for rather than resume anything. + if (live.expansion === "paused") { + items.push( + row("footer:continue", "[continue]", { select: "continue" }, { here: view.focused }), + ); + } + } + if (live.question !== undefined && state.route.at === undefined) { + items.push( + row( + "footer:asked", + `? ${live.question.message}`, + { select: "live-elicit" }, + { here: view.focused }, + ), + ); + } + + // The canonical location, as it stands and in full. It is the one thing a + // person copies out of this screen — how they come back to exactly this view, + // here or in another process — so a prefix of it is no use to them. A location + // carrying a draft is longer than a row, so it is as many rows as it needs, + // above whatever surface is being shown rather than in the footer, which is + // seven rows and has controls in them. + for (const [offset, part] of chunked(view.location, surfaceWidth(view.size)).entries()) { + items.push(line(`location:${offset}`, part)); + } + + // Why the last thing asked for changed nothing. Shown rather than swallowed: a + // refusal nobody can read is a keystroke that appeared to do nothing. + if (state.refusal !== undefined) { + // One line: the footer is seven rows and the controls live in them, so a + // refusal that wrapped would push the draft off the screen it is about. + items.push(line("footer:refused", `! ${state.refusal.split("\n").join(" ")}`)); + } + + // The draft, which is where typing goes until an entry exists. + // + // It claims focus only while nothing holds it. A claim is where focus *starts*, + // not an assertion repeated every frame: the tree re-reads the claim at each + // commit, and this screen commits on every frame, so a standing claim would + // drag focus back here after every Tab and traversal would never move at all. + const claiming = state.route.drawers.length === 0 && view.focused === undefined; + items.push( + field( + "footer:input", + entry === undefined ? "> " : " ", + entry === undefined ? state.draft : "(one entry admitted)", + "draft", + claiming ? { focus: true, here: view.focused } : { here: view.focused }, + ), + ); + + const drawer = drawerFor(view); + if (drawer !== undefined) { + items.push(drawer); + } + return items; +} + +/** + * The innermost open drawer, as a modal branch holding its own controls. + * + * Its detail is one child per line rather than one multi-line label, because a + * cell is a row: a label holding three lines would show one of them, and a reader + * would have no way to know the other two existed. + */ +function drawerFor(view: ReplView): Described | undefined { + const open = view.selection.drawers[view.selection.drawers.length - 1]; + if (open === undefined) { + return undefined; + } + const children: ReplDescription[] = []; + // Every row of this drawer reaches its own right edge. A modal that wrote only + // its own text would let what it is in front of show through from where that + // text stopped, because a renderer writes what changed and nothing else. + const width = drawerWidth(view.size); + let title: string; + + if (open.kind === "binding") { + title = open.name; + for (const [offset, text] of detail(open.binding.value).entries()) { + children.push(drawerLine(`drawer:value:${offset}`, text, width).description); + } + } else if (open.kind === "recorded-elicit") { + title = open.elicitation.location; + // The whole of what was asked and the whole of what was answered. A drawer is + // where the retained value is, so a summary here would leave a reader with no + // way to see what the record actually holds. + children.push(drawerLine("drawer:schema", "schema", width).description); + for (const [offset, text] of detail(open.elicitation.schema).entries()) { + children.push(drawerLine(`drawer:schema:${offset}`, text, width).description); + } + children.push(drawerLine("drawer:answered", "answer", width).description); + for (const [offset, text] of detail(open.elicitation.answer).entries()) { + children.push(drawerLine(`drawer:answer:${offset}`, text, width).description); + } + } else if (open.kind === "history") { + title = "History"; + for (const checkpoint of view.model.checkpoints) { + children.push( + row( + `drawer:marker:${checkpoint.marker}`, + width < 1 ? checkpoint.label : checkpoint.label.padEnd(width, " "), + { + select: "marker", + marker: checkpoint.marker, + }, + { here: view.focused }, + ).description, + ); + } + } else { + const question = view.live.question; + if (question === undefined) { + return undefined; + } + title = question.message; + // The retained normalized schema, as a form: the one field it asks for and + // the exact values it will accept. + children.push( + drawerLine( + "drawer:form", + `${question.form.field}: ${question.form.choices.join(" | ")}`, + width, + ).description, + ); + // The same rule inside the modal: the field claims focus when the drawer + // opens, and afterwards traversal inside the drawer owns it. + const entering = view.focused === undefined || !view.focused.startsWith("drawer:"); + const claim: { readonly focus?: true } = entering ? { focus: true } : {}; + children.push( + field("drawer:answer", "= ", view.state.answer, "answer", { + ...claim, + here: view.focused, + }).description, + ); + } + + children.push( + row( + "drawer:close", + width < 1 ? "[close]" : "[close]".padEnd(width, " "), + { select: "close" }, + { + here: view.focused, + }, + ).description, + ); + return { + key: "drawer:open", + description: describeNode({ + key: "drawer:open", + component: DRAWER, + input: { label: width < 1 ? title : title.padEnd(width, " ") }, + children, + modal: true, + }), + }; +} + +/** One value, as the lines a drawer shows it on. */ +function detail(value: Json): readonly string[] { + return (JSON.stringify(value, undefined, 2) ?? "null").split("\n"); +} + +/** Every nested scope beneath one, with the key path that selects it. */ +/** One selectable nested scope: what to call it, and the path that selects it. */ +interface NestedScope { + readonly label: string; + readonly path: string[]; +} + +function nested(scope: ReplScope, path: readonly string[]): readonly NestedScope[] { + const found: NestedScope[] = []; + for (const child of scope.scopes) { + const here = [...path, child.key]; + found.push({ label: `${child.kind} ${child.name}`, path: here }); + found.push(...nested(child, here)); + } + return found; +} + +/** One transcript row, as a line. */ +function describeRow(entry: ReplRow): string { + switch (entry.kind) { + case "entry": + return `entry ${entry.path}`; + case "scope": + return `${entry.scope} ${entry.name}`; + case "binding": + return `${entry.scope} bound ${entry.names.join(", ")}`; + case "output": + return entry.text; + case "generated": + return `generated ${entry.decision}${entry.source === undefined ? "" : `: ${entry.source}`}`; + case "elicit": + return `answered ${entry.location} ${summarize(entry.answer)}`; + case "effect": + return `${entry.type} ${entry.status}`; + case "terminal": + return entry.output.length > 0 ? entry.output : `closed ${entry.status}`; + } +} + +/** + * The surface one committed tree presents. + * + * Read from the frame, so a node the tree removed contributes nothing, and keyed + * by the description's own key so the region a row belongs to is decided in one + * place. The live node id travels with every cell, because that id is what a + * pointer resolved against the drawn frame has to name. + */ +export function replSurface(tree: ReplTree, view: ReplView): ReplPlacedSurface { + const sessions: ReplSurfaceCell[] = []; + const entries: ReplSurfaceCell[] = []; + const transcript: ReplSurfaceCell[] = []; + const inspection: ReplSurfaceCell[] = []; + const drawer: ReplSurfaceCell[] = []; + const footer: ReplSurfaceCell[] = []; + /** The location rows, which sit above whatever surface is being shown. */ + const located: ReplSurfaceCell[] = []; + for (const cell of tree.frame().cells) { + const key = tree.keyOf(cell.node); + if (key === undefined) { + continue; + } + const placed: ReplSurfaceCell = { + node: cell.node, + text: cell.cell, + ...(targetable(key) ? { targetable: true } : {}), + }; + if (key.startsWith("drawer:")) { + drawer.push(placed); + } else if (key.startsWith("location:")) { + located.push(placed); + } else if (key.startsWith("sessions:")) { + sessions.push(placed); + } else if (key.startsWith("entries:") || key.startsWith("entry:") || key.startsWith("scope:")) { + entries.push(placed); + } else if (key.startsWith("line:")) { + transcript.push(placed); + } else if (key.startsWith("binding:") || key.startsWith("elicit:")) { + inspection.push(placed); + } else if (key.startsWith("footer:") || key === "refusal") { + footer.push(placed); + } + } + + return { + // Narrow shows exactly the surface the route selected. + content: [...located, ...(view.state.route.surface === "sessions" ? sessions : entries)], + sessions, + entries, + transcript: [...located, ...transcript], + inspection, + drawer, + footer, + // The band's labels come from the model rather than from mounted nodes: a + // history position is a place in the file, and offering twenty of them as + // twenty focus stops in a seven-row footer would bury the controls that are + // actually there. Selecting an exact one is what the History drawer is for. + history: view.model.checkpoints.map((checkpoint) => ({ + marker: checkpoint.marker, + label: checkpoint.label, + })), + }; +} + +/** Whether a pointer may activate the row this key names. */ +function targetable(key: string): boolean { + // A line is text. Everything a pointer may activate is a control, and the two + // footer lines that are not — the location somebody copies and the reason the + // last action changed nothing — are lines. + return ( + !key.startsWith("line:") && + key !== "sessions:empty" && + key !== "entry:none" && + key !== "footer:location" && + key !== "footer:refused" + ); +} + +export { HISTORY_ROWS }; +export type { ReplDrawerRef, ReplSurface }; diff --git a/packages/cli/src/repl/components/actions.ts b/packages/cli/src/repl/components/actions.ts new file mode 100644 index 00000000..75e5de45 --- /dev/null +++ b/packages/cli/src/repl/components/actions.ts @@ -0,0 +1,37 @@ +/** + * Everything this product can be asked to do. + * + * Closed, and the only vocabulary that crosses from the component tree to the + * root. A component turns a normalized key, a piece of text or a pointer into + * one of these; the root is the only thing that decides what any of them changes. + * Nothing below the root holds the session, the repository, the route or a host + * operation, so a component cannot act — it can only ask. + * + * Every member says what was asked for and names the thing it was asked about. + * None of them carries a model object, because the root already has the model + * and a component's copy of one could be a frame out of date. + */ + +import type { ReplDrawerRef, ReplSurface } from "../route.ts"; + +export type ReplAction = + /** Text for whichever field has focus. */ + | { readonly kind: "type"; readonly text: string } + /** Remove the last decoded text unit from whichever field has focus. */ + | { readonly kind: "erase" } + /** Admit the draft as this execution's one entry. */ + | { readonly kind: "submit" } + | { readonly kind: "select-surface"; readonly surface: ReplSurface } + /** Select a structural scope by its key path, outermost first. */ + | { readonly kind: "select-scope"; readonly scopes: readonly string[] } + | { readonly kind: "open-drawer"; readonly drawer: ReplDrawerRef } + /** Close the innermost open drawer. Never an answer to anything. */ + | { readonly kind: "close-drawer" } + /** Freeze the view at one history position. */ + | { readonly kind: "select-marker"; readonly marker: string } + /** Return to the Journal head. */ + | { readonly kind: "go-live" } + | { readonly kind: "pause" } + | { readonly kind: "continue" } + /** Answer the question waiting right now. */ + | { readonly kind: "answer" }; diff --git a/packages/cli/src/repl/components/rows.ts b/packages/cli/src/repl/components/rows.ts new file mode 100644 index 00000000..e75b6e77 --- /dev/null +++ b/packages/cli/src/repl/components/rows.ts @@ -0,0 +1,256 @@ +/** + * The things a REPL screen is made of. + * + * Each one renders exactly what its input says and claims exactly the input it + * has a meaning for. A row that has no use for text lets text pass, so the + * draft below still receives it; a row that has no use for Escape lets Escape + * pass, so the drawer above it still closes. That is the whole reason claims are + * per-node rather than a table somewhere: the answer to "what does this + * keystroke mean" depends on what is under the cursor. + * + * No component here holds a `DurableEvent`, a stream, a session, a repository, + * the route or a host operation. They are given detached view data and they + * answer with a semantic action. + */ + +import { component, fields } from "../description.ts"; +import type { ReplComponent, ReplInputEvent, ReplNode, ReplViewData } from "../description.ts"; +import type { ReplAction } from "./actions.ts"; + +/** Read one string field, by parsing rather than by assertion. */ +export function textOf(input: ReplViewData, name: string): string { + const value = fields(input)?.[name]; + if (typeof value !== "string") { + throw new Error(`this component is given a { ${name}: string } input`); + } + return value; +} + +/** + * Whether this control holds focus, as the view said when it was built. + * + * Marked so a person can see which control their next keystroke reaches. It is + * one frame behind the tree's own answer, because focus is derived from the tree + * and a description is written before the commit that settles it — which is + * exactly why the marker is redrawn after every commit rather than remembered. + */ +function marked(input: ReplViewData, label: string): string { + return fields(input)?.["focused"] === true ? `> ${label}` : ` ${label}`; +} + +/** Read an optional string field. */ +function optional(input: ReplViewData, name: string): string | undefined { + const value = fields(input)?.[name]; + return typeof value === "string" ? value : undefined; +} + +/** Text nobody can select: a transcript line, a binding summary, a heading. */ +export const LINE: ReplComponent = component({ + name: "line", + attach(node: ReplNode): void { + node.render(textOf(node.input, "label")); + node.onInput((input: ReplViewData) => node.render(textOf(input, "label"))); + }, +}); + +/** + * A row that selects something when it is activated. + * + * `select` names what activating it asks for. The action is built from the + * input's own fields rather than from the node's key, because a key is a + * reconciliation identity and reading product meaning out of one would make + * renaming a key a behavior change. + */ +export const SELECT_ROW: ReplComponent = component({ + name: "select-row", + attach(node: ReplNode): void { + node.focusable(); + node.render(marked(node.input, textOf(node.input, "label"))); + node.onInput((input: ReplViewData) => node.render(marked(input, textOf(input, "label")))); + node.claim((event: ReplInputEvent): ReplAction | undefined => { + if (event.kind === "text") { + return undefined; + } + if (event.kind !== "pointer" && event.key !== "Enter") { + return undefined; + } + return activation(node.input); + }); + }, +}); + +/** What activating one row asks for, read from its own input. */ +function activation(input: ReplViewData): ReplAction | undefined { + const named = fields(input); + if (named === undefined) { + return undefined; + } + const select = named["select"]; + if (select === "surface") { + const surface = named["surface"]; + return surface === "repl" || surface === "sessions" + ? { kind: "select-surface", surface } + : undefined; + } + if (select === "scope") { + const scopes = named["scopes"]; + if (!Array.isArray(scopes)) { + return undefined; + } + const keys: string[] = []; + for (const key of scopes) { + if (typeof key !== "string") { + return undefined; + } + keys.push(key); + } + return { kind: "select-scope", scopes: keys }; + } + if (select === "marker") { + const marker = named["marker"]; + return typeof marker === "string" ? { kind: "select-marker", marker } : undefined; + } + if (select === "binding") { + const name = named["name"]; + return typeof name === "string" + ? { kind: "open-drawer", drawer: { kind: "binding", name } } + : undefined; + } + if (select === "recorded-elicit") { + const marker = named["marker"]; + return typeof marker === "string" + ? { kind: "open-drawer", drawer: { kind: "recorded-elicit", marker } } + : undefined; + } + if (select === "live-elicit") { + return { kind: "open-drawer", drawer: { kind: "live-elicit" } }; + } + if (select === "history") { + return { kind: "open-drawer", drawer: { kind: "history" } }; + } + if (select === "live") { + return { kind: "go-live" }; + } + if (select === "pause") { + return { kind: "pause" }; + } + if (select === "continue") { + return { kind: "continue" }; + } + if (select === "close") { + return { kind: "close-drawer" }; + } + return undefined; +} + +/** + * A field text goes into: the entry draft, and the live Elicit answer. + * + * The same component for both, because typing is typing. Which field the text + * reaches is decided by focus, and which field has focus is decided by the + * description — so opening the live Elicit drawer moves the answer into the + * focus root and the draft stops receiving anything, without either component + * knowing the other exists. + */ +export const FIELD: ReplComponent = component({ + name: "field", + attach(node: ReplNode): void { + node.focusable(); + node.render(rendered(node.input)); + node.onInput((input: ReplViewData) => node.render(rendered(input))); + node.claim((event: ReplInputEvent): ReplAction | undefined => { + if (event.kind === "text") { + return { kind: "type", text: event.text }; + } + if (event.kind === "pointer") { + return undefined; + } + if (event.key === "Backspace") { + return { kind: "erase" }; + } + if (event.key === "Enter") { + return submission(node.input); + } + return undefined; + }); + }, +}); + +/** What submitting this field asks for. */ +function submission(input: ReplViewData): ReplAction | undefined { + const purpose = optional(input, "purpose"); + if (purpose === "draft") { + return { kind: "submit" }; + } + if (purpose === "answer") { + return { kind: "answer" }; + } + return undefined; +} + +/** + * A field shows its prompt and the line being edited. + * + * One line, because a field is one line. A pasted document is many, and the + * earlier ones are said to be there rather than drawn into a footer that is a + * row tall — what the whole draft is remains exactly readable in the canonical + * location, which is where a caller reopens it from. + */ +function rendered(input: ReplViewData): string { + const text = textOf(input, "text"); + const here = fields(input)?.["focused"] === true; + const lines = text.split("\n"); + const last = lines[lines.length - 1]; + const earlier = lines.length - 1; + const body = earlier === 0 ? last : `[${earlier} line${earlier === 1 ? "" : "s"}] ${last}`; + return `${here ? ">" : " "}${textOf(input, "prompt")}${body}`; +} + +/** + * A drawer: a modal branch, closed by Escape. + * + * It claims Escape and nothing else. Escape closes the drawer and never answers + * the question inside it, because a person dismissing a prompt has not chosen + * one of its options. + */ +export const DRAWER: ReplComponent = component({ + name: "drawer", + attach(node: ReplNode): void { + node.render(textOf(node.input, "label")); + node.onInput((input: ReplViewData) => node.render(textOf(input, "label"))); + node.claim((event: ReplInputEvent): ReplAction | undefined => + event.kind === "key" && event.key === "Escape" ? { kind: "close-drawer" } : undefined, + ); + }, +}); + +/** + * A refusal, and the route back when there is one. + * + * Focusable only when it offers somewhere to go: a refusal with no remedy is + * something to read, and making it focusable would put the focus chain on a + * control that does nothing. + */ +export const REFUSAL: ReplComponent = component({ + name: "refusal", + attach(node: ReplNode): void { + const back = optional(node.input, "back"); + if (back !== undefined) { + node.focusable(); + } + node.render(textOf(node.input, "label")); + node.onInput((input: ReplViewData) => node.render(textOf(input, "label"))); + if (back === undefined) { + return; + } + node.claim((event: ReplInputEvent): ReplAction | undefined => { + if (event.kind === "text") { + return undefined; + } + if (event.kind !== "pointer" && event.key !== "Enter") { + return undefined; + } + return back === "live" ? { kind: "go-live" } : { kind: "select-surface", surface: "repl" }; + }); + }, +}); diff --git a/packages/cli/src/repl/input.ts b/packages/cli/src/repl/input.ts index 5f8f5375..2f4d542a 100644 --- a/packages/cli/src/repl/input.ts +++ b/packages/cli/src/repl/input.ts @@ -29,6 +29,15 @@ * join the next keystroke and turn an `a` into an Alt-`a`. There is no way to * clear that buffer, so the decoder is replaced instead. That costs one * instantiation, and only when somebody actually presses Escape by itself. + * + * ## One scan is not one chunk + * + * The same scanner returns at most 128 events per call and keeps the rest of the + * bytes buffered, so a pasted document arrives over several scans. A host that + * scanned once per chunk would silently drop everything past the 128th + * character of a paste — which for this REPL means most of an entry. So a scan + * here means "scan until the buffer stops producing", and the events of all + * those passes are one result in the order the terminal sent them. */ import { createInput, type Input, type InputEvent, type ScanResult } from "@bomb.sh/tty"; @@ -189,14 +198,14 @@ export function useReplDecoder( let input: Input = yield* until(createInput(settings)); yield* provide({ *scan(bytes?: Uint8Array): Operation { - const scanned = scanInput(input, bytes); + const drained = drain(input, bytes); const settling = bytes === undefined && - scanned.pendingFor !== undefined && - scanned.events.length === 0 && - scanned.pointers.length === 0; + drained.pendingFor !== undefined && + drained.events.length === 0 && + drained.pointers.length === 0; if (!settling) { - return scanned; + return drained; } // Nothing followed the held ESC, so it was the key. A fresh scanner, // because the spent byte is still in this one's buffer and would @@ -205,10 +214,42 @@ export function useReplDecoder( return Object.freeze({ events: Object.freeze([ESCAPE]), pointers: Object.freeze([]), - resized: scanned.resized, + resized: drained.resized, pendingFor: undefined, }); }, }); }); } + +/** + * Everything the scanner has, not just the first 128 of it. + * + * Each pass after the first is a scan with no new bytes, which is how the + * scanner hands over what it still holds. Draining stops when a pass produces + * nothing, and the last pass's pending report is the one that is still true. + */ +function drain(input: Input, bytes?: Uint8Array): ReplInputScan { + const events: ReplInputEvent[] = []; + const pointers: ReplPointerAt[] = []; + let resized = false; + let pendingFor: number | undefined; + let first = true; + + while (true) { + const pass = first && bytes !== undefined ? scanInput(input, bytes) : scanInput(input); + first = false; + events.push(...pass.events); + pointers.push(...pass.pointers); + resized = resized || pass.resized; + pendingFor = pass.pendingFor; + if (pass.events.length === 0 && pass.pointers.length === 0 && !pass.resized) { + return Object.freeze({ + events: Object.freeze(events), + pointers: Object.freeze(pointers), + resized, + pendingFor, + }); + } + } +} diff --git a/packages/cli/src/repl/layout.ts b/packages/cli/src/repl/layout.ts index 91f845b3..22c4b587 100644 --- a/packages/cli/src/repl/layout.ts +++ b/packages/cli/src/repl/layout.ts @@ -136,6 +136,40 @@ interface ColumnWidths { const SIDEBAR: ColumnWidths = { wide: 32, medium: 28 }; const INSPECTION: ColumnWidths = { wide: 36, medium: 28 }; +/** + * How wide the surface that carries content is, at one size. + * + * Asked by whoever has to *write* something that must fit: a row longer than the + * region it lands in is reflowed into rows the layout never allocated, so the + * width has to be a question with one answer rather than a constant each caller + * guesses at. Zero when there is no such surface, which is the refusal. + */ +export function surfaceWidth(size: ReplTerminalSize): number { + const profile = profileFor(size); + if (profile === "too-small") { + return 0; + } + if (profile === "narrow") { + return size.columns; + } + const sidebar = profile === "wide" ? SIDEBAR.wide : SIDEBAR.medium; + const inspection = profile === "wide" ? INSPECTION.wide : INSPECTION.medium; + return size.columns - sidebar - inspection; +} + +/** + * How wide the drawer layer is, at one size. + * + * The same question for a modal: what it writes has to reach its own edges, or + * what it is in front of shows through from where its text stops. + */ +export function drawerWidth(size: ReplTerminalSize): number { + if (profileFor(size) === "too-small") { + return 0; + } + return size.columns - 2 * Math.floor(size.columns / 8); +} + /** Which profile a size gets. */ export function profileFor(size: ReplTerminalSize): ReplProfile { if (size.columns < NARROW.columns || size.rows < NARROW.rows) { diff --git a/packages/cli/src/repl/program.ts b/packages/cli/src/repl/program.ts new file mode 100644 index 00000000..32ebd6a8 --- /dev/null +++ b/packages/cli/src/repl/program.ts @@ -0,0 +1,745 @@ +/** + * The command, as one scope. + * + * One scope owns the repository, the session, the keyed tree, the acknowledged + * frame stream, the renderer and the terminal, and it ends all of them together. + * Everything below it is either a kernel that was settled in an earlier slice or + * the pure application reduction — this module only arranges them, decides when a + * frame is owed, and performs the effects a component asked for and cannot do. + * + * ## Nothing is drawn on a clock + * + * A frame is owed when something that a frame shows has changed: the session + * reprojected, the document printed, a question arrived, expansion moved, the + * terminal resized, or a normalized event + * produced an action. Each of those wakes the loop; nothing polls, and when + * nothing has changed no subscription is held and no timer exists. The + * subscription is taken for the frame it is about to draw and released after it, + * so demand and drawing are the same fact. + * + * ## Acknowledged after present, never before + * + * The order is snapshot, commit, lay out, render, present, retain the frame map, + * then acknowledge. Acknowledging earlier would tell the stream a timestamp had + * been applied while the bytes for it were still being written, which is the + * drift the acknowledgement exists to prevent. Anything that fails before the + * present releases demand without applying the frame — and raises, because the + * scope that owns the terminal is the only thing that puts its modes back, and + * a screen that cannot be redrawn must not be left in raw mode showing a + * picture that will never change again. + */ + +import { + Err, + type Operation, + Ok, + type Result, + scoped, + spawn, + type Subscription, + withResolvers, +} from "effection"; +import type { ExecutionInstallation } from "@executablemd/core/host"; + +import { + admitted, + answered, + describeApplication, + initialState, + reduceRepl, + refusedView, + replSurface, + stateFor, + viewFor, +} from "./application.ts"; +import type { ReplAction, ReplIntent, ReplLive, ReplState, ReplView } from "./application.ts"; +import { decodeLocation, encodeLocation, resolveLocation } from "./route.ts"; +import { replRepository } from "./journal.ts"; +import type { ReplExecution } from "./journal.ts"; +import { openReplSession, submitReplEntry } from "./session.ts"; +import type { ReplSession } from "./session.ts"; +import { useReplFrames } from "./frame.ts"; +import type { ReplFrames } from "./frame.ts"; +import { layout } from "./layout.ts"; +import { resolvePointer, snapshotRender, useReplRenderer } from "./renderer.ts"; +import type { ReplRendered, ReplRenderer } from "./renderer.ts"; +import { useReplScreen } from "./screen.ts"; +import type { ReplScreen, ReplScreenEvent } from "./screen.ts"; +import { useReplTree } from "./reconcile.ts"; +import type { ReplTree } from "./reconcile.ts"; +import { projectRepl } from "./model.ts"; +import type { ReplModel } from "./model.ts"; +import { useReplRoot } from "./storage.ts"; +import { ReplTerminal } from "./terminal.ts"; +import type { ReplTerminalSize } from "./terminal.ts"; + +/** What one `xmd repl` invocation was asked to do. */ +export interface ReplProgramOptions { + /** The location to reopen, or none to start a fresh execution. */ + readonly location?: string; + /** Where components are looked for. */ + readonly includes?: readonly string[]; + /** What this host installs around the document. */ + readonly installations?: readonly ExecutionInstallation[]; + /** The repository root. Defaults to this host's per-user data directory. */ + readonly root?: string; +} + +/** What the command reports when it ends. */ +export interface ReplOutcome { + /** The canonical location the screen was showing when it ended. */ + readonly location: string; + /** Why it ended without a session, when that is what happened. */ + readonly refusal: string | undefined; +} + +/** + * Run one REPL. + * + * Refusals come back as `Err` rather than as a thrown error, because a person + * naming an unreadable location has not caused a failure — they have asked for + * something this command cannot show, and the difference decides whether a + * terminal was ever opened. + */ +export function* runReplProgram(options: ReplProgramOptions = {}): Operation> { + // Before a path is formed, a file is opened or a terminal is touched: what the + // caller named has to be a location this grammar defines. + if (options.location !== undefined) { + const decoded = decodeLocation(options.location); + if (!decoded.ok) { + return decoded; + } + } + + let state: ReplState = + options.location === undefined ? initialState("pending") : yield* stateOf(options.location); + + // A terminal, before a directory is formed or a file exists. Raw mode is the + // first thing that fails over a pipe, and it fails several steps after this + // command has already created an execution — leaving an empty history behind + // for a session that was never going to open. Asked here, "there is no + // terminal" is a refusal instead of wreckage. + if (!(yield* ReplTerminal.operations.interactive())) { + return Err( + new ReplRefusedError( + "the REPL runs where a terminal is: it is not available over a pipe.", + encodeLocation(state.route), + ), + ); + } + + const root = options.root ?? (yield* useReplRoot()); + const repository = replRepository(root); + + // Opening a retained history reads and validates the whole file, and a file + // that is not one raises. That is a location this command cannot show rather + // than a failure of the command, so it becomes the refusal screen — with no + // session, no provider and no observer ever created. + let execution: ReplExecution; + try { + execution = + options.location === undefined + ? yield* repository.create() + : yield* repository.open(requireExecution(options.location)); + } catch (error) { + return yield* refuse(state, error instanceof Error ? error.message : String(error)); + } + if (options.location === undefined) { + state = initialState(execution.id); + } + + // The route, against what the file already holds — *before* anything replays. + // + // Opening a session starts or resumes the execution, and replay that gets + // past the retained prefix appends. A location naming a scope this history + // never had is a typo, and discovering it after replay has run means the typo + // grew the file: the run happened, records landed, and the answer the person + // gets is still that their URL was wrong. Everything the route names comes + // from records that are already on disk, so the question has an answer here, + // where the only cost of the wrong URL is being told so. + if (options.location !== undefined) { + const reachable = yield* resolvable(execution, state); + if (!reachable.ok) { + return yield* refuse(state, reachable.error.message); + } + } + + const opened = yield* openReplSession({ + execution, + ...(options.includes === undefined ? {} : { includes: options.includes }), + ...(options.installations === undefined ? {} : { installations: options.installations }), + ...(state.route.at === undefined ? {} : { selection: state.route.at }), + }); + + if (!opened.ok) { + // A history that cannot be projected mounts the refusal and nothing else: + // no provider, no observer, no execution task, and no append. + return yield* refuse(state, opened.error.message); + } + + return yield* drive(opened.value, state, execution, options); +} + +/** + * Whether the retained history can answer everything this route names. + * + * Projected from the file as it stands rather than from a session, because a + * session is the thing this is deciding whether to open. The prefix a route + * selects, the scopes it walks and the drawers it opens are all answers the + * retained records already hold. + */ +function* resolvable(execution: ReplExecution, state: ReplState): Operation> { + const projected = projectRepl(yield* execution.stream.readAll(), state.route.at); + if (!projected.ok) { + return projected; + } + const resolved = resolveLocation(projected.value, state.route); + return resolved.ok ? Ok(undefined) : Err(resolved.error); +} + +/** The execution segment of a location that has already been decoded once. */ +function requireExecution(location: string): string { + const decoded = decodeLocation(location); + if (!decoded.ok) { + throw decoded.error; + } + return decoded.value.execution; +} + +function* stateOf(location: string): Operation { + const read = stateFor(location); + if (!read.ok) { + throw read.error; + } + return read.value; +} + +/** + * Show one refusal until the person leaves. + * + * A screen, because a refusal a caller cannot see is a command that exited + * silently; and only a screen, because there is nothing behind it to act on. + */ +function* refuse(state: ReplState, reason: string): Operation> { + const view = refusedView(state, reason); + yield* scoped(function* (): Operation { + const screen = yield* useReplScreen(); + const events: Subscription = yield* screen.events(); + const tree = yield* useReplTree(); + const frames = yield* useReplFrames(); + const size = yield* screen.size(); + const renderer = yield* useReplRenderer({ columns: size.columns, rows: size.rows }); + + yield* paint(frames, tree, renderer, screen, view); + while (true) { + const next = yield* events.next(); + if (next.done === true || next.value.kind === "eof") { + return; + } + if (next.value.kind === "resize") { + // A refusal recovers on resize like any other screen: the terminal + // growing is the remedy for the one refusal that has no other. + renderer.resize(next.value.size); + yield* paint(frames, tree, renderer, screen, view); + continue; + } + if ( + next.value.kind === "input" && + next.value.event.kind === "key" && + next.value.event.key === "Escape" + ) { + return; + } + } + }); + return Err(new ReplRefusedError(reason, encodeLocation(state.route))); +} + +/** A location this command cannot show. */ +export class ReplRefusedError extends Error { + readonly location: string; + constructor(message: string, location: string) { + super(message); + this.name = "ReplRefusedError"; + this.location = location; + } +} + +/** + * Everything that has changed since the last frame. + * + * A queue rather than a stream subscription, because what the loop needs is not + * the next event but *every* event that has arrived: one burst of input is one + * frame, and a subscription hands them over one at a time with no way to ask + * whether more are already waiting. + */ +interface Wakes { + send(wake: Wake): void; + close(): void; + /** Everything waiting, or nothing at all when the sources have ended. */ + take(): Operation; +} + +function queued(): Wakes { + const waiting: Wake[] = []; + let ended = false; + let ready = withResolvers(); + return { + send(wake: Wake): void { + waiting.push(wake); + ready.resolve(); + }, + close(): void { + ended = true; + ready.resolve(); + }, + *take(): Operation { + while (waiting.length === 0) { + if (ended) { + return []; + } + yield* ready.operation; + ready = withResolvers(); + } + return waiting.splice(0, waiting.length); + }, + }; +} + +/** What woke the loop. */ +type Wake = + | { readonly kind: "screen"; readonly event: ReplScreenEvent } + | { readonly kind: "session" }; + +/** Run the screen for one opened session. */ +function* drive( + session: ReplSession, + start: ReplState, + execution: ReplExecution, + options: ReplProgramOptions, +): Operation> { + let state = start; + let current: ReplSession = session; + let model: ReplModel = session.model; + let rendered: ReplRendered | undefined; + /** The question whose drawer has already been offered. */ + let offered: unknown; + + const outcome = yield* scoped(function* (): Operation> { + const screen = yield* useReplScreen(); + const events: Subscription = yield* screen.events(); + const tree = yield* useReplTree(); + const frames = yield* useReplFrames(); + const size = yield* screen.size(); + const renderer = yield* useReplRenderer({ columns: size.columns, rows: size.rows }); + + // One queue, so the loop has exactly one place it waits — and so a burst is + // one frame. A paste arrives as hundreds of text events in a few chunks; a + // loop that drew after each of them would draw hundreds of frames nobody + // sees, and the person would watch their document appear a character at a + // time. + const wakes = queued(); + + yield* spawn(function* reading(): Operation { + while (true) { + const next = yield* events.next(); + if (next.done === true) { + wakes.close(); + return; + } + wakes.send({ kind: "screen", event: next.value }); + } + }); + + yield* watch(current, wakes); + + let focused: string | undefined; + rendered = yield* paint( + frames, + tree, + renderer, + screen, + yield* build(state, model, current, yield* screen.size(), focused), + ); + focused = keyOfFocus(tree); + // The first frame has the same obligation as every other one. + rendered = yield* paint( + frames, + tree, + renderer, + screen, + yield* build(state, model, current, yield* screen.size(), focused), + ); + + while (true) { + const taken = yield* wakes.take(); + if (taken.length === 0) { + return Ok({ location: encodeLocation(state.route), refusal: undefined }); + } + + // What stands now, to return to if what is attempted does not resolve. + const standing = state; + let ended = false; + for (const wake of taken) { + if (wake.kind === "session") { + model = current.model; + } else if (wake.event.kind === "eof") { + ended = true; + } else if (wake.event.kind === "resize") { + renderer.resize(wake.event.size); + } else if (wake.event.kind === "pointer") { + // Resolved only against the frame that produced these coordinates. A + // stale frame, a node the tree has removed and a target behind a drawer + // all reach nothing, and the drop happens without an action. + const aimed = + rendered === undefined ? undefined : resolvePointer(rendered, wake.event.at); + if (aimed !== undefined) { + const dispatched = yield* tree.dispatch(aimed); + if (dispatched.ok && dispatched.value.outcome === "action") { + const transition = reduceRepl(state, dispatched.value.action, model, liveOf(current)); + state = transition.state; + const performed = yield* perform( + transition.intent, + current, + execution, + options, + wakes, + ); + if (performed.session !== undefined) { + current = performed.session; + model = performed.session.model; + // The entry exists now, so the draft that became it is finished. + state = admitted(state); + } + if (performed.answered === true) { + // The question took it, so the drawer that was asking is over. + state = answered(state); + } + if (performed.refusal !== undefined) { + state = Object.freeze({ ...state, refusal: performed.refusal }); + } + } + } + } else if (wake.event.kind === "input") { + const delivered = wake.event.event; + if (delivered !== undefined) { + const dispatched = yield* tree.dispatch(delivered); + if (dispatched.ok && dispatched.value.outcome === "action") { + const transition = reduceRepl(state, dispatched.value.action, model, liveOf(current)); + state = transition.state; + const performed = yield* perform( + transition.intent, + current, + execution, + options, + wakes, + ); + if (performed.session !== undefined) { + current = performed.session; + model = performed.session.model; + // The entry exists now, so the draft that became it is finished. + state = admitted(state); + } + if (performed.answered === true) { + // The question took it, so the drawer that was asking is over. + state = answered(state); + } + if (performed.refusal !== undefined) { + state = Object.freeze({ ...state, refusal: performed.refusal }); + } + } + } + } + } + + // A question is the interaction, not a place to go looking for one: when + // this process starts asking, its drawer is offered once. Dismissing it + // with Escape is final for that question, because the offer is remembered + // by the question itself rather than by a flag somebody has to clear. + const asking = current.overlay.question; + if (asking !== undefined && asking !== offered && state.route.at === undefined) { + offered = asking; + const opening = reduceRepl( + state, + { kind: "open-drawer", drawer: { kind: "live-elicit" } }, + model, + liveOf(current), + ); + state = opening.state; + } + + // A route that names a history position needs the model projected *at* that + // position: a prefix is a different reading of the same file rather than a + // filter over the head. Reprojected, then verified — and a route that does + // not resolve there leaves the standing one exactly as it was. + let view: ReplView; + const attempted = yield* reproject(state, model, execution); + const drawnAt = yield* screen.size(); + const built = viewFor(attempted.state, attempted.model, liveOf(current), drawnAt, focused); + if (built.ok) { + state = attempted.state; + model = attempted.model; + view = built.value; + } else { + // The reason the *reprojection* refused, when there was one: it is the + // first thing that went wrong, and the resolution failure below it is a + // consequence of still holding the older reading. + state = Object.freeze({ + ...standing, + refusal: attempted.state.refusal ?? built.error.message, + }); + const back = yield* reproject(state, model, execution); + model = back.model; + const again = viewFor(state, model, liveOf(current), drawnAt, focused); + view = again.ok ? again.value : refusedView(state, again.error.message, drawnAt); + } + rendered = yield* paint(frames, tree, renderer, screen, view); + // Focus is the tree's answer, and the tree only answers after the commit — + // so a frame built before it can be marking the wrong control. When it has + // moved, draw once more with where it actually is. Otherwise the marker is + // always one keystroke behind, and a person reaching for a control would be + // acting on the one after it. + const settledFocus = keyOfFocus(tree); + if (settledFocus !== focused) { + focused = settledFocus; + rendered = yield* paint( + frames, + tree, + renderer, + screen, + yield* build(state, model, current, yield* screen.size(), focused), + ); + } + if (ended) { + // End of input is a lifecycle outcome: the last frame is drawn, and then + // the command is over. + return Ok({ location: encodeLocation(state.route), refusal: undefined }); + } + } + }); + + return outcome; +} + +/** + * The model the route asks for, projected at the position it names. + * + * Returns the state unchanged when the file cannot be read at that position, with + * the reason, so a caller can decide between adopting and restoring rather than + * being handed a half-applied navigation. + */ +function* reproject( + state: ReplState, + model: ReplModel, + execution: ReplExecution, +): Operation<{ readonly state: ReplState; readonly model: ReplModel }> { + if (state.route.at === model.selection) { + return { state, model }; + } + const projected = projectRepl(yield* execution.stream.readAll(), state.route.at); + return projected.ok + ? { state, model: projected.value } + : { state: Object.freeze({ ...state, refusal: projected.error.message }), model }; +} + +/** The key of whatever holds focus now, or none. */ +function keyOfFocus(tree: ReplTree): string | undefined { + const node = tree.focused(); + return node === undefined ? undefined : tree.keyOf(node); +} + +/** This process's overlay, as the application reads it. */ +function liveOf(session: ReplSession): ReplLive { + return { + output: session.overlay.output, + question: session.overlay.question, + expansion: session.expansion.state, + pausable: session.controller !== undefined, + }; +} + +/** The view for the state that stands, or a refusal naming why there is none. */ +function* build( + state: ReplState, + model: ReplModel, + session: ReplSession, + size: ReplTerminalSize, + focused: string | undefined, +): Operation { + const view = viewFor(state, model, liveOf(session), size, focused); + return view.ok ? view.value : refusedView(state, view.error.message, size); +} + +/** Wake the loop whenever this session's history or overlay moves. */ +function* watch(session: ReplSession, wakes: Wakes): Operation { + yield* spawn(function* projections(): Operation { + const changes = yield* session.changes; + while (true) { + const next = yield* changes.next(); + if (next.done === true) { + return; + } + wakes.send({ kind: "session" }); + } + }); + yield* spawn(function* printing(): Operation { + // Output is the one thing a frame shows that leaves no record behind, so + // nothing else will ever announce it. A document that only writes + // reprojects nothing, asks nothing and expands nothing; without this its + // text would sit in the overlay until some unrelated event happened to + // draw a frame, and a run that printed and then finished would show it all + // at the end, which is the opposite of what an overlay is for. + const changes = yield* session.outputs; + while (true) { + const next = yield* changes.next(); + if (next.done === true) { + return; + } + wakes.send({ kind: "session" }); + } + }); + yield* spawn(function* questions(): Operation { + const changes = yield* session.elicitation.changes; + while (true) { + const next = yield* changes.next(); + if (next.done === true) { + return; + } + wakes.send({ kind: "session" }); + } + }); + yield* spawn(function* expansion(): Operation { + const changes = yield* session.expansion.states; + while (true) { + const next = yield* changes.next(); + if (next.done === true) { + return; + } + wakes.send({ kind: "session" }); + } + }); +} + +/** What performing one intent produced. */ +interface Performed { + /** The session that stands now, when submitting produced a different one. */ + readonly session?: ReplSession; + /** Whether a question took the answer it was given and is now over. */ + readonly answered?: boolean; + /** Why it could not be done, for the screen to say. */ + readonly refusal?: string; +} + +/** + * Do the one thing a component asked for and cannot do itself. + * + * The session is the authority for its own history, so admitting an entry is the + * one action that produces a different session rather than a different view. A + * refusal comes back rather than being swallowed: a person who pressed Enter is + * owed either an entry or a reason. + */ +function* perform( + intent: ReplIntent, + session: ReplSession, + execution: ReplExecution, + options: ReplProgramOptions, + wakes: Wakes, +): Operation { + switch (intent.kind) { + case "none": + return {}; + case "pause": + session.controller?.pause(); + wakes.send({ kind: "session" }); + return {}; + case "continue": + session.controller?.resume(); + wakes.send({ kind: "session" }); + return {}; + case "answer": { + // A choice the form does not offer is not an answer: the question stays + // open, nothing is appended, and the drawer stays up holding what was + // typed so it can be corrected. Only an accepted answer ends the + // question, and the route has to end with it. + const accepted = session.overlay.question?.answer(intent.choice) ?? false; + wakes.send({ kind: "session" }); + return accepted ? { answered: true } : {}; + } + case "submit": { + const submitted = yield* submitReplEntry({ + execution, + source: intent.source, + ...(options.includes === undefined ? {} : { includes: options.includes }), + ...(options.installations === undefined ? {} : { installations: options.installations }), + }); + if (!submitted.ok) { + // A preflight refusal leaves the draft exactly as it was and the history + // empty: nothing was admitted, so there is nothing to undo. + return { refusal: submitted.error.message }; + } + yield* watch(submitted.value, wakes); + wakes.send({ kind: "session" }); + return { session: submitted.value }; + } + } +} + +/** + * Draw one frame, in the one order this product allows. + * + * The subscription is taken for this frame and released with the scope, so + * holding one *is* owing a frame: nothing is subscribed while the screen is + * quiet, and a failure before the present releases the demand without applying + * the timestamp. + * + * ## A frame that cannot be drawn ends the screen + * + * Failing to subscribe, to reconcile or to render raises. It does not hand back + * the frame before it: the terminal is in raw mode and on the alternate screen, + * and a command that keeps both while showing a picture it can no longer + * update has taken the person's terminal and stopped telling them anything. + * Raising unwinds the scope that owns the terminal, which is the only thing + * that puts the modes back. + */ +function* paint( + frames: ReplFrames, + tree: ReplTree, + renderer: ReplRenderer, + screen: ReplScreen, + view: ReplView, +): Operation { + return yield* scoped(function* (): Operation { + const held = yield* frames.subscribe({ owner: "repl", participants: [["repl"]] }); + if (!held.ok) { + throw held.error; + } + const tick = yield* held.value.next(); + + // 1. The immutable view is already in hand. 2. Commit it and wait for that + // exact handoff. + const committed = yield* tree.apply(describeApplication(view)); + if (!committed.ok) { + throw committed.error; + } + // 3. Only the mounted tree, laid out for the terminal as it is now. + const size = yield* screen.size(); + const frame = layout(size, replSurface(tree, view)); + const drawn = yield* renderer.render( + snapshotRender({ + frame, + tree: tree.frame().id, + mounted: tree.mounted(), + deltaTime: tick.delta, + pointer: undefined, + }), + ); + if (!drawn.ok) { + throw drawn.error; + } + // 4. Present the copied bytes. 5. The caller retains the returned map. And + // only then 6. acknowledge that this timestamp has been applied. + yield* screen.present(drawn.value.output); + held.value.acknowledge(); + return drawn.value; + }); +} diff --git a/packages/cli/src/repl/route.ts b/packages/cli/src/repl/route.ts index fb4e8a5e..82cddf2c 100644 --- a/packages/cli/src/repl/route.ts +++ b/packages/cli/src/repl/route.ts @@ -223,8 +223,20 @@ export function encodeLocation(route: ReplRoute): string { * did not record. A target that is absent refuses; so does a drawer stack whose * earlier members were left out, because a stack with a hole in it does not * describe a surface anybody saw. + * + * `asking` is the one thing the model cannot answer. A question that is waiting + * has by definition not been recorded — a Journal holds answered questions, not + * pending ones — so no reading of any history can establish that a live + * question's drawer is mountable. The process that is asking says so. It + * defaults to false, because a caller that cannot say is a caller with no + * question: resolving one it does not have would accept a URL naming a drawer + * nothing will mount. */ -export function resolveLocation(model: ReplModel, route: ReplRoute): Result { +export function resolveLocation( + model: ReplModel, + route: ReplRoute, + asking = false, +): Result { if (route.at !== model.selection) { return Err( new ReplRouteError( @@ -280,7 +292,7 @@ export function resolveLocation(model: ReplModel, route: ReplRoute): Result { if (reference.kind === "history") { return Ok({ kind: "history" }); @@ -329,6 +342,21 @@ function openDrawer( ), ); } + if (!asking) { + // Nothing retained can establish this drawer. A waiting question is the + // one thing a Journal never holds — it records answers — so `settled`, + // the recorded elicitations and every other fact in the model are all + // consistent with no question ever arriving. A route resolved here + // without one replays a whole execution, mounts nothing and reports + // success. Only the process actually asking may open it. + return Err( + new ReplRouteError( + "nothing is being asked. A live question's drawer belongs to the process holding the " + + "question, and no history records one that is still waiting — so this location " + + "cannot be reopened into answering it.", + ), + ); + } return Ok({ kind: "live-elicit" }); } if (scope === undefined) { diff --git a/packages/cli/src/repl/session.ts b/packages/cli/src/repl/session.ts index 6351e524..e637287c 100644 --- a/packages/cli/src/repl/session.ts +++ b/packages/cli/src/repl/session.ts @@ -147,6 +147,16 @@ export interface ReplSession { readonly live: boolean; /** Each reprojection, as the history grows under it. */ readonly changes: Stream; + /** + * The overlay's output, each time the document adds to it. + * + * Separate from `changes` because plain output is precisely what the Journal + * does not record: a document that only writes appends nothing, reprojects + * nothing, and would otherwise be invisible to anything watching the + * history. A reader that shows `overlay.output` has to watch this too, or it + * shows text only when something unrelated happens to move. + */ + readonly outputs: Stream; /** Wait for this process's execution to finish, and answer how it finished. */ join(): Operation>; } @@ -219,6 +229,7 @@ function* start( } const changes = createSignal(); + const outputs = createSignal(); let output = ""; let live = true; const admission = withResolvers>(); @@ -288,6 +299,7 @@ function* start( return live; }, changes, + outputs, join(): Operation> { return task; }, @@ -342,6 +354,10 @@ function* start( let next = yield* chunks.next(); while (!next.done) { output += next.value; + // Announced, not merely accumulated: nothing else will say that the + // overlay moved, because output the Journal has not settled leaves + // no record to reproject from. + outputs.send(output); next = yield* chunks.next(); } }); @@ -411,6 +427,8 @@ function idle(execution: string, model: ReplModel): ReplSession { elicitation, live: false, changes, + // Nothing is running, so nothing will ever write. + outputs: createSignal(), // deno-lint-ignore require-yield *join(): Operation> { return Ok(undefined); diff --git a/packages/cli/src/repl/storage.ts b/packages/cli/src/repl/storage.ts new file mode 100644 index 00000000..49575d5b --- /dev/null +++ b/packages/cli/src/repl/storage.ts @@ -0,0 +1,57 @@ +/** + * Where a person's REPL history lives, and who knows that. + * + * Only the runtime knows where a per-user data directory is: it is a different + * path on macOS, Linux and Windows, and a compiled binary answers it the same + * way the source runtime does but says so for itself. So the path is an Api a + * runtime-named entrypoint installs, and shared code asks rather than inspects. + * + * Nothing here decides *what* is stored. The repository beneath this root holds + * ordinary serialized DurableEvents and nothing else — no manifest, no cache, no + * materialized model, no checkpoint file. A reader with the file and the URL has + * everything, which is only true while that stays true. + */ + +import { type Api, createApi } from "@effectionx/context-api"; +import { ensureDir } from "@effectionx/fs"; +import { join } from "node:path"; +import type { Operation } from "effection"; + +/** No adapter said where this host keeps a person's data. */ +export class ReplStorageError extends Error { + constructor() { + super( + "the REPL has no data directory on this host. A runtime adapter installs one with " + + 'ReplStorage.around({ ... }, { at: "min" }) before the command runs.', + ); + this.name = "ReplStorageError"; + } +} + +export interface ReplStorageApi { + /** The per-user data directory this runtime keeps application state in. */ + dataRoot(): Operation; +} + +export const ReplStorage: Api = createApi("ReplStorage", { + // deno-lint-ignore require-yield + *dataRoot(): Operation { + throw new ReplStorageError(); + }, +}); + +/** The directory beneath a data root that this REPL's histories live in. */ +export const REPL_DIRECTORY: readonly string[] = ["xmd", "repl"]; + +/** + * The repository root, created if this is the first time. + * + * Created rather than required, because the first `xmd repl` on a machine is + * the ordinary case. Nothing is ever removed from it: what is in one of these + * files is somebody's work. + */ +export function* useReplRoot(): Operation { + const root = join(yield* ReplStorage.operations.dataRoot(), ...REPL_DIRECTORY); + yield* ensureDir(root); + return root; +} diff --git a/packages/cli/src/repl/terminal-host.ts b/packages/cli/src/repl/terminal-host.ts index 6013e4c2..45bfa7bb 100644 --- a/packages/cli/src/repl/terminal-host.ts +++ b/packages/cli/src/repl/terminal-host.ts @@ -32,6 +32,14 @@ import { ReplTerminal, type ReplTerminalSize } from "./terminal.ts"; * takes a stream (Code Rule 12). */ export interface ReplTerminalCapabilities { + /** + * Whether both ends of this terminal are a terminal. + * + * Required rather than assumed: a host that cannot say is a host whose + * answer would have to be guessed, and the guess that costs something is the + * optimistic one — it creates a history file and then fails on raw mode. + */ + interactive(): boolean; /** The size right now. */ size(): ReplTerminalSize; /** @@ -60,6 +68,10 @@ export interface ReplTerminalCapabilities { export function installReplTerminal(host: ReplTerminalCapabilities): Operation { return ReplTerminal.around( { + // deno-lint-ignore require-yield + *interactive(): Operation { + return host.interactive(); + }, // deno-lint-ignore require-yield *size(): Operation { return host.size(); diff --git a/packages/cli/src/repl/terminal.ts b/packages/cli/src/repl/terminal.ts index 276c9e23..759abc12 100644 --- a/packages/cli/src/repl/terminal.ts +++ b/packages/cli/src/repl/terminal.ts @@ -38,6 +38,15 @@ export class ReplTerminalError extends Error { } export interface ReplTerminalApi { + /** + * Whether a person is at this terminal. + * + * Asked before anything is created, because the REPL is not available over a + * pipe and finding that out by calling `setRaw` on one is finding it out + * after a history file already exists. A host answers from what it knows + * about its own descriptors; nothing here guesses. + */ + interactive(): Operation; /** The size as it stands. */ size(): Operation; /** Write bytes, returning once the terminal has them. */ @@ -59,6 +68,10 @@ export interface ReplTerminalApi { } export const ReplTerminal: Api = createApi("ReplTerminal", { + // deno-lint-ignore require-yield + *interactive(): Operation { + throw new ReplTerminalError("asking whether a person is at it"); + }, // deno-lint-ignore require-yield *size(): Operation { throw new ReplTerminalError("reading its size"); diff --git a/packages/cli/tests/cli-help.test.ts b/packages/cli/tests/cli-help.test.ts index 49ac8da4..907e70e3 100644 --- a/packages/cli/tests/cli-help.test.ts +++ b/packages/cli/tests/cli-help.test.ts @@ -232,4 +232,39 @@ describe("Tier CH — xmd help", { sanitizeOps: false, sanitizeResources: false expect(stdout).not.toContain("%23"); expect(stdout).not.toContain("document reference"); }); + + it("CH12: program help lists repl, and its own help describes the one entry", function* () { + const listed = yield* runCli(["--help"]).expect(); + expect(listed.stdout).toContain("repl"); + expect(listed.stdout).toContain( + "Run and reconstruct one XMD entry in an interactive terminal.", + ); + + const own = yield* runCli(["repl", "--help"]).expect(); + expect(own.stdout).toContain("Usage: xmd repl [OPTIONS] [location]"); + expect(own.stdout).toContain("a location this REPL printed earlier"); + expect(own.stdout).toContain("ONE ENTRY"); + expect(own.stdout).toContain("An execution admits exactly one entry."); + // Describing a command is not running one: help reaches no data directory, + // no history file and no terminal mode. + expect(own.stdout).not.toContain("xmd/repl/"); + }); + + it("CH13: xmd repl refuses a bad command line before it opens anything", function* () { + // Each of these is refused by fixed grammar, which reads nothing — so the + // refusal arrives before a per-user directory is formed, a history file is + // created or the terminal's modes are touched. + const second = yield* runCli(["repl", "one", "two"]).join(); + expect(second.code).toBe(1); + expect(second.stderr).toContain("xmd repl takes at most one location"); + + const unknown = yield* runCli(["repl", "--json"]).join(); + expect(unknown.code).toBe(1); + expect(unknown.stderr).toContain("unrecognized option for xmd repl: --json"); + + const malformed = yield* runCli(["repl", "xmd://nope"]).join(); + expect(malformed.code).toBe(1); + expect(malformed.stderr).toContain("xmd repl:"); + expect(malformed.stderr).toContain("xmd://repl/"); + }); }); diff --git a/packages/cli/tests/repl-boundaries.test.ts b/packages/cli/tests/repl-boundaries.test.ts new file mode 100644 index 00000000..1747163c --- /dev/null +++ b/packages/cli/tests/repl-boundaries.test.ts @@ -0,0 +1,461 @@ +/** + * The REPL's source boundaries (#848 S1). + * + * Read from the source rather than from behaviour, because these are claims about + * what a module *can* reach. A component that never happens to touch the Journal + * in one test is not a component that cannot; an import is. + * + * Scoped deliberately. Vendored Freedom is somebody else's code under a recorded + * provenance, fixtures model bad input on purpose, and generated output is not + * authored — none of them is a production violation, and a scan broad enough to + * catch them would be a scan nobody could keep green. + */ + +import { describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { readTextFile } from "@effectionx/fs"; +import { readdir } from "node:fs/promises"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { createContext, ensure, type Operation, scoped, until } from "effection"; + +import { useHostFiles } from "@executablemd/runtime"; +import { runXmd } from "../src/cli.ts"; +import { unsupportedRepositories } from "../src/run-repositories.ts"; +import { unsupportedWorkflowHost } from "../src/workflow.ts"; +import type { UpgradeAssembly } from "../src/upgrade.ts"; +import { refusedStandardInput } from "./support/standard-input.ts"; +import { refusedPluginModules } from "./support/plugin-modules.ts"; + +import { replGrammarError } from "../src/cli.ts"; +import { readQuestionForm } from "../src/repl/elicitation.ts"; +import { decodeLocation, encodeLocation } from "../src/repl/route.ts"; + +const CLI = fileURLToPath(new URL("../", import.meta.url)); +const REPL = join(CLI, "src", "repl"); + +/** Every authored production source under the REPL, by path. */ +function* authored(): Operation { + const found: string[] = []; + const walk = function* (directory: string): Operation { + for (const entry of yield* until(readdir(directory, { withFileTypes: true }))) { + const path = join(directory, entry.name); + if (entry.isDirectory()) { + // Vendored code is not authored here, and its provenance is recorded in + // its own manifest rather than asserted by this checker. + if (entry.name === "vendor") { + continue; + } + yield* walk(path); + continue; + } + if (entry.name.endsWith(".ts")) { + found.push(path); + } + } + }; + yield* walk(REPL); + return found.sort(); +} + +/** One file's text. */ +function read(path: string): Operation { + return readTextFile(path); +} + +/** + * One file's code, without its prose. + * + * A module that *says* it holds no `DurableEvent` contains the word, and a + * checker reading the comment would fail the file for documenting the rule it + * follows. Classification has to be about what the source does. + */ +function* code(path: string): Operation { + const text = yield* read(path); + return text + .replace(/\/\*[\s\S]*?\*\//g, "") + .split("\n") + .map((line) => line.replace(/(^|\s)\/\/.*$/, "")) + .join("\n"); +} + +/** The runtime globals and environment reads no shared module may reach. */ +const RUNTIME_ACCESS: readonly { readonly name: string; readonly pattern: RegExp }[] = [ + { name: "the Deno global", pattern: /(^|[^.\w])Deno\s*\./ }, + { name: "the Bun global", pattern: /(^|[^.\w])Bun\s*\./ }, + { name: "node:process", pattern: /from "node:process"/ }, + { name: "process.env", pattern: /process\s*\.\s*env/ }, + { name: "process.platform", pattern: /process\s*\.\s*platform/ }, + { name: "node:os", pattern: /from "node:os"/ }, + { name: "a synchronous filesystem call", pattern: /from "node:fs"/ }, +]; + +/** Dependencies this slice's production code may not have acquired. */ +const REJECTED: readonly string[] = [ + "starfx", + "@bomb.sh/input", + "path-to-regexp", + "crank", + "revolution", +]; + +describe("REPL boundaries: what production code cannot reach", () => { + it("S1: no shared REPL module reaches a runtime global or the environment", function* () { + const offences: string[] = []; + for (const path of yield* authored()) { + const text = yield* code(path); + for (const { name, pattern } of RUNTIME_ACCESS) { + if (pattern.test(text)) { + offences.push(`${path.slice(CLI.length)} reaches ${name}`); + } + } + } + expect(offences).toEqual([]); + }); + + it("S1: the checker reads code and not prose", function* () { + // A module documenting the rule it follows names the thing it does not hold. + const prose = + "/** No component holds a DurableEvent or a ReplSession. */\nexport const x = 1;\n"; + expect(prose.includes("DurableEvent")).toBe(true); + const stripped = prose + .replace(/\/\*[\s\S]*?\*\//g, "") + .split("\n") + .map((line) => line.replace(/(^|\s)\/\/.*$/, "")) + .join("\n"); + expect(stripped.includes("DurableEvent")).toBe(false); + // And it still sees the import it is there to catch. + const real = + '// a comment\nimport type { DurableEvent } from "@executablemd/durable-streams";\n'; + const keeps = real + .replace(/\/\*[\s\S]*?\*\//g, "") + .split("\n") + .map((line) => line.replace(/(^|\s)\/\/.*$/, "")) + .join("\n"); + expect(keeps.includes("DurableEvent")).toBe(true); + }); + + it("S1: the checker rejects a module that reaches one", function* () { + // The same patterns against text that deliberately contains them, so a + // checker that matched nothing could not pass by being vacuous. + const seeded = [ + "const size = Deno.consoleSize();", + 'import process from "node:process";', + 'const home = process.env["HOME"];', + 'if (process.platform === "darwin") {}', + 'import { homedir } from "node:os";', + 'import { readFileSync } from "node:fs";', + 'const there = Bun.file("x");', + ]; + for (const line of seeded) { + const matched = RUNTIME_ACCESS.some(({ pattern }) => pattern.test(line)); + expect(matched).toBe(true); + } + // And it does not reject the forms that are fine. + for (const line of [ + 'import { readTextFile } from "@effectionx/fs";', + 'import { appendFile } from "node:fs/promises";', + "const processed = { env: 1 };", + ]) { + expect(RUNTIME_ACCESS.some(({ pattern }) => pattern.test(line))).toBe(false); + } + }); + + it("S1: application components import no journal, session or host authority", function* () { + const forbidden = [ + "durable-streams", + "./journal.ts", + "./session.ts", + "./storage.ts", + "./host.ts", + "./program.ts", + "@effectionx/fs", + ]; + const directory = join(REPL, "components"); + for (const entry of yield* until(readdir(directory))) { + if (!entry.endsWith(".ts")) { + continue; + } + const text = yield* code(join(directory, entry)); + for (const name of forbidden) { + expect(text.includes(`from "${name}"`)).toBe(false); + } + // A component receives view data and answers with an action. Nothing else. + expect(text.includes("DurableEvent")).toBe(false); + expect(text.includes("ReplSession")).toBe(false); + } + }); + + it("S1: the terminal, decoder and renderer know no application action name", function* () { + // The host says what happened and where. What it means is decided above it, + // so the names of this product's actions cannot appear down here. + const actions = [ + "select-surface", + "select-scope", + "open-drawer", + "select-marker", + "go-live", + "insert", + ]; + for (const module of [ + "terminal.ts", + "terminal-host.ts", + "input.ts", + "renderer.ts", + "layout.ts", + ]) { + const text = yield* code(join(REPL, module)); + for (const action of actions) { + expect(text.includes(`"${action}"`)).toBe(false); + } + } + }); + + it("S1: runtime-specific access lives only in runtime-named modules", function* () { + // The four adapters are the only files that may name a runtime, and each one + // names its own. + for (const [file, mentions] of [ + ["deno-repl.ts", ["node:os", "node:process"]], + ["node-repl.ts", ["node:os", "node:process"]], + ["bun-repl.ts", ["node:os", "node:process"]], + ["compiled-repl.ts", ["node:os", "node:process"]], + ] as const) { + const text = yield* code(join(CLI, "src", file)); + for (const mention of mentions) { + expect(text.includes(mention)).toBe(true); + } + } + // And the portable installer names none of them. + const portable = yield* code(join(CLI, "src", "repl-assembly.ts")); + for (const { pattern } of RUNTIME_ACCESS) { + expect(pattern.test(portable)).toBe(false); + } + }); + + it("S1: no rejected dependency entered production", function* () { + const manifest = yield* read(join(CLI, "deno.json")); + const packaged = yield* read(join(CLI, "package.json")); + for (const rejected of REJECTED) { + expect(manifest.includes(rejected)).toBe(false); + expect(packaged.includes(rejected)).toBe(false); + } + // The one dependency this stack did add, at the exact version it pinned. + expect(manifest).toContain('"@bomb.sh/tty": "npm:@bomb.sh/tty@0.9.0"'); + expect(packaged).toContain('"@bomb.sh/tty": "0.9.0"'); + }); + + it("S1: no REPL record type exists in production", function* () { + // Every retained line is an ordinary durable event. A record shape of this + // slice's own would be a second protocol nobody else can read. + for (const path of yield* authored()) { + const text = yield* code(path); + expect(text.includes('type: "repl')).toBe(false); + expect(text.includes("REPL_RECORD")).toBe(false); + } + }); +}); + +describe("REPL documentation: what it says is what the code does", () => { + it("D1: every command example in the spec is one the parser accepts", function* () { + const spec = yield* read(join(CLI, "..", "..", "specs", "repl-spec.md")); + const commands = [...spec.matchAll(/^xmd repl(.*)$/gm)].map((match) => match[1].trim()); + expect(commands.length).toBeGreaterThan(0); + + for (const rest of commands) { + // Stripped of the shell's quoting and of the comment beside it, which is + // prose rather than argv. + const argument = rest + .replace(/#.*$/, "") + .trim() + .replace(/^'(.*)'$/, "$1"); + const args = argument === "" ? ["repl"] : ["repl", argument]; + const location = argument === "" ? undefined : argument; + // A placeholder is not a location; what is being checked is the shape. + const concrete = location?.replace("", "kf39sla2"); + expect(replGrammarError(args, concrete)).toBeUndefined(); + } + }); + + it("D1: the spec's route grammar is the one the codec implements", function* () { + const spec = yield* read(join(CLI, "..", "..", "specs", "repl-spec.md")); + const written = /xmd:\/\/repl\/\/repl/; + expect(written.test(spec)).toBe(true); + + // The same shape, through the real codec. + const encoded = encodeLocation({ + execution: "kf39sla2", + surface: "repl", + scopes: [], + drawers: [], + at: undefined, + inspect: false, + draft: undefined, + }); + expect(encoded).toBe("xmd://repl/kf39sla2/repl"); + const decoded = decodeLocation(encoded); + expect(decoded.ok).toBe(true); + }); + + it("D1: a drifted example is caught", function* () { + // The same two checks against text that is wrong on purpose, so a checker + // that matched nothing could not pass by being vacuous. + expect(replGrammarError(["repl", "one", "two"], undefined)).toBeDefined(); + expect(replGrammarError(["repl", "--json"], undefined)).toBeDefined(); + expect(replGrammarError(["repl", "xmd://repl/"], "xmd://repl/")).toBeDefined(); + // And a grammar the codec does not produce. + expect( + encodeLocation({ + execution: "kf39sla2", + surface: "repl", + scopes: [], + drawers: [], + at: undefined, + inspect: false, + draft: undefined, + }), + ).not.toBe("xmd://repl/kf39sla2/entries"); + }); + + it("D1: the spec's Elicit metadata statement matches the real reader", function* () { + const spec = yield* read(join(CLI, "..", "..", "specs", "repl-spec.md")); + // The spec says the drawer shows the one field the schema asks for and the + // values it will accept. The reader is what decides that. + expect(spec).toContain("the one field the\nschema asks for"); + const form = readQuestionForm({ + type: "object", + properties: { decision: { type: "string", enum: ["approve", "decline"] } }, + required: ["decision"], + additionalProperties: false, + }); + expect(form).toEqual({ field: "decision", choices: ["approve", "decline"] }); + // And a schema of another shape is not presentable, which is why the spec + // says one field rather than a form in general. + expect(readQuestionForm({ type: "string" })).toBeUndefined(); + }); + + it("D1: Freedom's recorded provenance still matches its manifest", function* () { + const manifest: unknown = JSON.parse( + yield* read(join(REPL, "vendor", "freedom", "MANIFEST.json")), + ); + const provenance = yield* read(join(REPL, "vendor", "freedom", "PROVENANCE.md")); + expect(typeof manifest).toBe("object"); + if (typeof manifest !== "object" || manifest === null || !("commit" in manifest)) { + throw new Error("the vendor manifest records the commit it was taken from"); + } + const commit = manifest.commit; + expect(typeof commit).toBe("string"); + // The provenance document names the same revision the manifest pins. + expect(provenance).toContain(String(commit)); + + // A drifted hash is caught: the same comparison against a revision the + // manifest does not name. + expect(provenance.includes("0000000000000000000000000000000000000000")).toBe(false); + }); +}); + +/** + * The exit continuation `exit()` reaches for. + * + * `main()` installs one under this name; a suite driving `runXmd` directly + * installs its own, so a command's status is a value rather than this process + * ending. + */ +const ExitContext = createContext<(result: { status: number }) => Operation>("exit"); + +/** What one in-process `xmd` invocation did, and whether it reached the REPL. */ +interface Invocation { + readonly status: number; + /** How many times the host's REPL assembly was entered. */ + readonly installed: number; +} + +/** + * Drive `runXmd` in this process with a REPL installer that counts. + * + * The lifecycle claim is not about what the output says: it is that describing a + * command and refusing one never reach the host at all — no per-user directory, + * no history file, no terminal mode. An installer that is never entered is the + * only way to see that from outside. + */ +function* invoke(args: readonly string[]): Operation { + let status = 0; + let installed = 0; + const wrote = console.log; + const warned = console.error; + return yield* scoped(function* (): Operation { + yield* ensure(() => { + console.log = wrote; + console.error = warned; + }); + console.log = () => {}; + console.error = () => {}; + yield* ExitContext.set(function* (result) { + status = result.status; + }); + yield* useHostFiles(); + try { + yield* runXmd( + [...args], + function* () {}, + UPGRADE, + unsupportedRepositories, + refusedStandardInput, + refusedPluginModules, + unsupportedWorkflowHost, + undefined, + function* (): Operation { + installed += 1; + }, + ); + } catch { + // This host assembles no terminal and no data directory, so a command line + // that really runs fails once it asks for one. What is being counted is + // whether it got that far, so the failure is an outcome rather than an end. + status = 1; + } + return { status, installed }; + }); +} + +/** What this host says it is. Nothing here upgrades anything. */ +const UPGRADE: UpgradeAssembly = { + provenance: "deno-source", + currentVersion: "0.0.0-test", + executablePath: "/nonexistent", + platform: "darwin", + architecture: "arm64", +}; + +describe("REPL command: what runs before the host is reached", () => { + it("S1: help describes the command without assembling a REPL", function* () { + for (const args of [["--help"], ["repl", "--help"]]) { + const ran = yield* invoke(args); + expect(ran.status).toBe(0); + // Describing a command is not running one. + expect(ran.installed).toBe(0); + } + }); + + it("S1: every refusal is decided before the host is asked for anything", function* () { + for (const args of [ + ["repl", "one", "two"], + ["repl", "--json"], + ["repl", "xmd://nope"], + ]) { + const ran = yield* invoke(args); + expect(ran.status).toBe(1); + // No data directory, no history file, no terminal: the command line was + // answered before any of them could be reached for. + expect(ran.installed).toBe(0); + } + }); + + it("S1: the counter is entered when the command really runs", function* () { + // The same harness against a command line that is *not* refused, so a counter + // that could never increment cannot pass the two rows above. This host + // assembles nothing, so the command fails once it asks for a data directory — + // which is the point: it got that far. + const ran = yield* invoke(["repl"]); + expect(ran.installed).toBe(1); + }); +}); diff --git a/packages/cli/tests/repl-execution.test.ts b/packages/cli/tests/repl-execution.test.ts index 74026eaa..41f863d9 100644 --- a/packages/cli/tests/repl-execution.test.ts +++ b/packages/cli/tests/repl-execution.test.ts @@ -18,7 +18,7 @@ import { expect } from "@executablemd/test-support/expect"; import { useTempFileCompiler } from "@executablemd/core"; import { InMemoryStream, serializeDurableEvent } from "@executablemd/durable-streams"; import type { DurableEvent } from "@executablemd/durable-streams"; -import { ensure, race, scoped, sleep, until } from "effection"; +import { ensure, race, scoped, sleep, spawn, until } from "effection"; import type { Operation, Result } from "effection"; import { ensureDir, rm, writeTextFile } from "@effectionx/fs"; import { API } from "@executablemd/runtime"; @@ -249,6 +249,40 @@ describe("REPL execution: submitting one entry", () => { expect(session.overlay.output).toBe(session.model.terminal?.output); }); + it("X1: the overlay reports its output as the document prints it", function* () { + const holder = execution(); + const source = yield* referenceSource(); + const session = opened(yield* submitReplEntry({ ...options(holder), source })); + + // Output is the one thing in the overlay that no record describes: a + // document that prints and nothing else reprojects nothing, so a reader + // watching the history would never learn that the overlay had moved. + const reported: string[] = []; + yield* spawn(function* (): Operation { + const outputs = yield* session.outputs; + let next = yield* outputs.next(); + while (!next.done) { + reported.push(next.value); + next = yield* outputs.next(); + } + }); + + const question = yield* nextQuestion(session); + + // Reported while the run is still holding at its question, rather than + // collected and handed over once it finished. + expect(reported.length).toBeGreaterThan(0); + const latest = reported[reported.length - 1]; + expect(latest).toContain("About to evaluate:"); + // What it reported is what the overlay holds: the same text, not a chunk + // the reader would have to accumulate itself. + expect(latest).toBe(session.overlay.output); + expect(session.model.terminal).toBe(undefined); + + question.answer("approve"); + yield* session.join(); + }); + it("X1: the generated fragment is shown before it is admitted, and follows the binding", function* () { const holder = execution(); const source = yield* referenceSource(); diff --git a/packages/cli/tests/repl-journey.test.ts b/packages/cli/tests/repl-journey.test.ts new file mode 100644 index 00000000..9ba2944d --- /dev/null +++ b/packages/cli/tests/repl-journey.test.ts @@ -0,0 +1,2361 @@ +/** + * The one-entry product journey (#848 J1, S1, D1). + * + * A real terminal's worth of bytes in, a real terminal's worth of bytes out, and + * a real durable execution in between. The terminal is injected, so the same + * corpus runs under Deno, Node and Bun; everything else — the repository, the + * session, the keyed tree, the layout, the renderer — is the production assembly + * the command uses. + * + * Output is asserted as the text the screen shows, with escape sequences + * stripped. What a terminal's cursor addressing looks like is the renderer's + * business and changes when it optimizes; what a person can read on the screen + * is the product. + */ + +import { beforeAll, describe, it } from "@executablemd/test-support/bdd"; +import { expect } from "@executablemd/test-support/expect"; +import { type Operation, type Result, scoped, sleep, spawn, withResolvers } from "effection"; +import { appendFile, mkdir, mkdtemp, open, readFile, readdir, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { randomBytes } from "node:crypto"; +import { until } from "effection"; + +import { API } from "@executablemd/runtime"; +import { Elicitation, useTempFileCompiler } from "@executablemd/core"; +import { ordinaryEvaluationProfile } from "../src/evaluation-profile.ts"; +import { installReplHost } from "../src/repl-assembly.ts"; +import { installReplTerminal } from "../src/repl/terminal-host.ts"; +import type { ReplTerminalCapabilities } from "../src/repl/terminal-host.ts"; +import type { ReplTerminalSize } from "../src/repl/terminal.ts"; +import { ReplClock } from "../src/repl/frame.ts"; +import { initialState, reduceRepl } from "../src/repl/application.ts"; +import { runReplProgram } from "../src/repl/program.ts"; +import type { ReplOutcome } from "../src/repl/program.ts"; +import { parseDurableEvent, serializeDurableEvent } from "@executablemd/durable-streams"; +import { drawerWidth, NARROW, surfaceWidth } from "../src/repl/layout.ts"; +import { projectRepl } from "../src/repl/model.ts"; +import type { ReplModel } from "../src/repl/model.ts"; +import { decodeLocation, encodeLocation } from "../src/repl/route.ts"; +import { + REFERENCE_DIRECTORY, + referenceEvents, + referenceSource, +} from "./fixtures/repl/reference.ts"; + +const BYTES = new TextEncoder(); + +/** A model with nothing in it, which is what a fresh execution projects. */ +const EMPTY_MODEL_FOR_TEST = Object.freeze({ + selection: undefined, + head: true, + entry: undefined, + settled: false, + terminal: undefined, + checkpoints: Object.freeze([]), + transcript: Object.freeze([]), +}); +const TEXT = new TextDecoder(); + +/** What this host installs around the reference entry. */ +const INSTALLATIONS = [{ evaluation: ordinaryEvaluationProfile() }]; + +/** What the reference entry's eval publishes, as the model retains it. */ +const PLAN = { title: "Ship the REPL", steps: 2 }; + +/** The normalized schema the reference entry's question is judged against. */ +const SCHEMA = { + type: "object", + properties: { decision: { type: "string", enum: ["approve", "decline"] } }, + required: ["decision"], + additionalProperties: false, +}; + +/** The answer this journey gives it. */ +const ANSWER = { decision: "approve" }; + +/** Let every task that is ready take its turn. */ +function* settled(turns = 8): Operation { + for (let turn = 0; turn < turns; turn += 1) { + yield* sleep(0); + } +} + +/** A clock the test moves, so nothing in this suite waits on real time. */ +function immediateClock(): Operation { + return ReplClock.around( + { + // deno-lint-ignore require-yield + *now(): Operation { + return 0; + }, + // deno-lint-ignore require-yield + *wait(): Operation { + // Returns at once: this product draws when something changed, so the + // frame interval is the only thing being skipped. + }, + }, + { at: "min" }, + ); +} + +/** A clock that records what was scheduled and releases it only when told. */ +interface CountingClock { + waits: number; + install(): Operation; + release(): void; +} + +function countingClock(): CountingClock { + let pending: (() => void)[] = []; + const clock: CountingClock = { + waits: 0, + install(): Operation { + return ReplClock.around( + { + // deno-lint-ignore require-yield + *now(): Operation { + return 0; + }, + *wait(): Operation { + clock.waits += 1; + const waiter = withResolvers(); + pending.push(() => waiter.resolve()); + yield* waiter.operation; + }, + }, + { at: "min" }, + ); + }, + release(): void { + const releasing = pending; + pending = []; + for (const one of releasing) { + one(); + } + }, + }; + return clock; +} + +/** A terminal the test drives completely. */ +interface Terminal { + /** Everything ever presented, in order. */ + readonly presented: Uint8Array[]; + /** + * When set, the next presentation blocks here until it is released. + * + * The seam the frame-order control needs: while a frame is being written, the + * stream must not have been told that frame was applied. + */ + holdPresent: { release(): void } | undefined; + size: ReplTerminalSize; + readonly raw: boolean[]; + resets: number; + listeners: number; + readers: number; + feed(text: string): void; + bytes(raw: Uint8Array): void; + /** Make the next presentation block, so a test can look at the frame stream. */ + holdNextPresent(): void; + resized(size: ReplTerminalSize): void; + end(): void; +} + +function recordingTerminal( + size: ReplTerminalSize = { columns: 160, rows: 36 }, + interactive = true, +): { + terminal: Terminal; + install(): Operation; +} { + const queue: Uint8Array[] = []; + const watchers = new Set<() => void>(); + let waiting: ((result: IteratorResult) => void) | undefined; + let ended = false; + + let holding = false; + const terminal: Terminal = { + presented: [], + holdPresent: undefined, + size, + raw: [], + resets: 0, + listeners: 0, + readers: 0, + feed(text: string): void { + terminal.bytes(BYTES.encode(text)); + }, + holdNextPresent(): void { + holding = true; + }, + bytes(raw: Uint8Array): void { + const resolve = waiting; + if (resolve === undefined) { + queue.push(raw); + return; + } + waiting = undefined; + resolve({ done: false, value: raw }); + }, + resized(next: ReplTerminalSize): void { + terminal.size = next; + for (const watcher of watchers) { + watcher(); + } + }, + end(): void { + ended = true; + const resolve = waiting; + if (resolve !== undefined) { + waiting = undefined; + resolve({ done: true, value: undefined }); + } + }, + }; + + const host: ReplTerminalCapabilities = { + interactive: () => interactive, + size: () => terminal.size, + write(bytes: Uint8Array): Promise { + terminal.presented.push(new Uint8Array(bytes)); + if (!holding) { + return Promise.resolve(); + } + holding = false; + return new Promise((resolve) => { + terminal.holdPresent = { release: resolve }; + }); + }, + writeNow(): void { + terminal.resets += 1; + }, + setRaw(raw: boolean): void { + terminal.raw.push(raw); + }, + bytes(): AsyncIterable { + return { + [Symbol.asyncIterator](): AsyncIterator { + terminal.readers += 1; + return { + next(): Promise> { + const head = queue.shift(); + if (head !== undefined) { + return Promise.resolve({ done: false, value: head }); + } + if (ended) { + return Promise.resolve({ done: true, value: undefined }); + } + return new Promise((resolve) => { + waiting = resolve; + }); + }, + return(): Promise> { + terminal.readers -= 1; + const resolve = waiting; + waiting = undefined; + resolve?.({ done: true, value: undefined }); + return Promise.resolve({ done: true, value: undefined }); + }, + }; + }, + }; + }, + onResize(listener: () => void): () => void { + watchers.add(listener); + terminal.listeners += 1; + return () => { + watchers.delete(listener); + terminal.listeners -= 1; + }; + }, + }; + + return { terminal, install: () => installReplTerminal(host) }; +} + +/** A REPL host over a temporary directory nothing else uses. */ +function* useTemporaryHost(): Operation { + const root = yield* until(mkdtemp(join(tmpdir(), "xmd-repl-journey-"))); + yield* installReplHost({ + dataRoot: () => root, + identify: () => randomBytes(8).toString("hex"), + createExclusive: (path) => open(path, "wx").then((handle) => handle.close()), + appendRecord: (path, record) => appendFile(path, record), + }); + return root; +} + +/** + * What the screen says, by replaying what was written to it. + * + * A real buffer rather than the bytes with escapes stripped, because this + * renderer writes *diffs*: it moves the cursor to what changed and writes only + * that. Concatenating the diffs gives characters in the order they were written + * rather than the order they appear, and a character the previous frame already + * had is not written again at all — so stripped bytes read as words with letters + * missing. Interpreting the cursor moves is what makes an assertion about the + * screen an assertion about the screen. + */ +function screenOf(terminal: Terminal): string[] { + const rows: string[][] = []; + let row = 0; + let column = 0; + + const put = (character: string): void => { + while (rows.length <= row) { + rows.push([]); + } + const line = rows[row]; + while (line.length < column) { + line.push(" "); + } + line[column] = character; + column += 1; + }; + + const written = terminal.presented.map((bytes) => TEXT.decode(bytes)).join(""); + for (let index = 0; index < written.length; index += 1) { + const character = written[index]; + if (character !== "\u001B") { + if (character === "\n") { + row += 1; + column = 0; + } else if (character === "\r") { + column = 0; + } else { + put(character); + } + continue; + } + // CSI: the only sequences this renderer uses to position and to clear. + const csi = /^\u001B\[([0-9;]*)([@-~])/.exec(written.slice(index)); + if (csi !== null) { + const parameters = csi[1].split(";").map((one) => (one === "" ? 0 : Number(one))); + if (csi[2] === "H") { + row = Math.max(0, (parameters[0] ?? 1) - 1); + column = Math.max(0, (parameters[1] ?? 1) - 1); + } else if (csi[2] === "J") { + rows.length = 0; + row = 0; + column = 0; + } + index += csi[0].length - 1; + continue; + } + // OSC, and the two-byte escapes. Neither carries anything readable. + const osc = /^\u001B\][^\u0007\u001B]*(?:\u0007|\u001B\\)/.exec(written.slice(index)); + if (osc !== null) { + index += osc[0].length - 1; + continue; + } + index += 1; + } + return rows.map((line) => line.join("")); +} + +/** What a run really performed, as opposed to what it restored. */ +interface Performed { + /** Component sources actually read from disk. */ + reads: string[]; + /** Eval blocks actually compiled, which is where a block really runs. */ + compiles: number; + /** Questions a provider was actually asked. */ + asked: number; +} + +/** + * Count the work a run performs, at the seams where performing it happens. + * + * Not at the durable operations: replay enters those and hands back what was + * recorded, so counting them would count restoration as work. A component's + * source is read inside the recorded selection and an eval block is compiled + * inside the recorded evaluation, so these counts are zero for anything a cold + * open restored rather than ran. + */ +function* countPerformed(): Operation { + const performed: Performed = { reads: [], compiles: 0, asked: 0 }; + yield* API.Fs.around({ + *readTextFile([path], next) { + performed.reads.push(path); + return yield* next(path); + }, + }); + yield* API.Env.around({ + *compile([source, options], next) { + performed.compiles++; + return yield* next(source, options); + }, + }); + yield* Elicitation.around({ + *elicit([request], next) { + performed.asked++; + return yield* next(request); + }, + }); + return performed; +} + +/** Whether any row of the screen contains this text. */ +function shows(terminal: Terminal, expected: string): boolean { + return screenOf(terminal).some((line) => line.includes(expected)); +} + +/** + * Tab until the control this key names holds focus. + * + * Traversal through the ordinary normalized boundary, exactly as a person does + * it: there is no host shortcut that jumps to a control, and the marker on the + * focused control is how anybody — a person or this test — knows where they are. + */ +function* focusOn(terminal: Terminal, label: string, limit = 240): Operation { + if (focusedOn(terminal, label)) { + return; + } + for (let press = 0; press < limit; press += 1) { + terminal.feed("\t"); + yield* settled(12); + if (focusedOn(terminal, label)) { + return; + } + } + throw new Error(`focus never reached ${label} in ${limit} presses`); +} + +/** + * Whether the control holding focus is the one this label names. + * + * Anchored to the marker rather than matched anywhere on the line, because a + * line of this screen crosses three columns: the sidebar, the transcript and the + * inspection column all write to the same rows, so a label found *somewhere* on + * a line with a marker on it is usually a different control in a different column. + * The marker is searched for at any position for the same reason — a focused + * control in the inspection column has the sidebar's text to the left of it. + */ +function focusedOn(terminal: Terminal, label: string): boolean { + for (const line of screenOf(terminal)) { + for (let at = line.indexOf(">"); at !== -1; at = line.indexOf(">", at + 1)) { + if ( + line + .slice(at + 1) + .trimStart() + .startsWith(label) + ) { + return true; + } + } + } + return false; +} + +/** + * The canonical location the screen is showing, if it has drawn one yet. + * + * Reassembled, because a location carrying a draft is longer than a row and the + * screen shows it as consecutive rows. Which rows belong to it is decided by the + * grammar rather than by counting: the longest run that decodes *is* the location, + * and a shorter prefix of it decodes to a different route or to nothing. + */ +function maybeLocation(terminal: Terminal): string | undefined { + const rows = screenOf(terminal); + const first = rows.findIndex((line) => line.includes("xmd://repl/")); + if (first === -1) { + return undefined; + } + const at = rows[first].indexOf("xmd://repl/"); + const parts: string[] = []; + for (let row = first; row < rows.length && row < first + 24; row += 1) { + const part = (rows[row] ?? "").slice(at, at + surfaceWidth(terminal.size)); + if (part.trim().length === 0) { + break; + } + parts.push(part.trimEnd()); + } + + // The rows below a location belong to whatever is drawn under it, and a row that + // used to hold a longer location can still have that tail on the end. So the + // answer is the longest prefix that *round-trips*: the grammar accepts some + // trailing junk inside a drawer segment, but re-encoding what it decoded only + // reproduces the prefix that really was the location. + const joined = parts.join(""); + let found: string | undefined; + for (let length = joined.length; length > "xmd://repl/".length; length -= 1) { + const candidate = joined.slice(0, length); + const decoded = decodeLocation(candidate); + if (decoded.ok && encodeLocation(decoded.value) === candidate) { + found = candidate; + break; + } + } + return found; +} + +/** The canonical location the screen is showing. */ +function locationOn(terminal: Terminal): string { + const shown = maybeLocation(terminal); + if (shown === undefined) { + throw new Error( + "the screen shows its canonical location. rows=" + + JSON.stringify( + screenOf(terminal) + .slice(0, 5) + .map((l) => l.trimEnd()), + ), + ); + } + return shown; +} + +/** + * Wait until the first frame has been drawn. + * + * The command opens a terminal, a repository and a session before it can draw + * anything, and how long that takes is not a number of turns — so every test + * that reads the screen waits for it rather than assuming. + */ +function* untilDrawn(terminal: Terminal): Operation { + for (let attempt = 0; attempt < 40; attempt += 1) { + if (maybeLocation(terminal) !== undefined) { + return; + } + yield* sleep(10); + yield* settled(10); + } + throw new Error( + `the screen never drew its first frame. frames=${terminal.presented.length} rows=` + + JSON.stringify( + screenOf(terminal) + .map((l) => l.trimEnd()) + .filter((l) => l.trim().length > 0), + ), + ); +} + +/** + * Dismiss the question's drawer, which opens itself when the question appears. + * + * A modal owns focus while it is up, so anything that means to reach a control + * beneath it has to close it first — and it may not have opened yet when the + * submission settles, so this waits for it rather than assuming. + */ +function* dismissQuestion(terminal: Terminal): Operation { + for (let attempt = 0; attempt < 40; attempt += 1) { + // Keyed to the drawer's own form, because while it is up it covers the rows + // the location is drawn on — a modal is drawn over what it is in front of. + if (shows(terminal, "decision: approve | decline")) { + terminal.feed("\x1b"); + yield* settled(30); + return; + } + // Real time, not only turns: reaching the question compiles an eval block, + // and a compile is work off this interpreter rather than a turn on it. + yield* sleep(10); + yield* settled(20); + } + throw new Error("the question's drawer never opened"); +} + +/** + * Click one control, by finding it on the screen and pressing there. + * + * The other way in, and the fast one: activating a control by pointer needs no + * focus traversal, so it can reach a control while an execution is still moving. + * The protocol counts from one and the screen counts from zero. + */ +function* click(terminal: Terminal, label: string): Operation { + const at = coordinateOf(terminal, label); + if (at === undefined) { + throw new Error(`no control labelled ${label} is on the screen`); + } + yield* clickAt(terminal, at); +} + +/** More than any label this REPL puts in a drawer, and less than any box. */ +const LABEL_ROOM = 48; + +/** Where on the screen one label is, if it is there. */ +function coordinateOf( + terminal: Terminal, + label: string, +): { readonly column: number; readonly row: number } | undefined { + for (const [row, line] of screenOf(terminal).entries()) { + const column = line.indexOf(label); + if (column !== -1) { + return { column, row }; + } + } + return undefined; +} + +/** Press at exactly this cell. The protocol counts from one; the screen from zero. */ +function* clickAt( + terminal: Terminal, + at: { readonly column: number; readonly row: number }, +): Operation { + terminal.feed(`\x1b[<0;${at.column + 1};${at.row + 1}M`); + yield* settled(30); +} + +/** Wait until the screen says what it is asked about, in real time. */ +function* until_( + terminal: Terminal, + what: string, + says: (terminal: Terminal) => boolean, +): Operation { + for (let attempt = 0; attempt < 60; attempt += 1) { + if (says(terminal)) { + return; + } + yield* sleep(10); + yield* settled(20); + } + throw new Error( + `the screen never said ${what}. footer=` + + JSON.stringify( + screenOf(terminal) + .slice(-9) + .map((line) => line.trim()), + ), + ); +} + +/** + * Every position the open History drawer lists, in the order it lists them. + * + * Read from the drawer rather than from the compact band: the band shares labels + * when space is short, and what is being compared is the exact set. + */ +function drawerMarkers(terminal: Terminal): string[] { + const found: string[] = []; + const rows = screenOf(terminal); + const title = rows.findIndex((line) => line.includes("History")); + if (title === -1) { + return found; + } + const at = rows[title].indexOf("History"); + for (let row = title + 1; row < rows.length; row += 1) { + const text = (rows[row] ?? "").slice(at, at + drawerWidth(terminal.size)).trimEnd(); + const label = text.replace(/^>\s*/, "").trim(); + if (label.length === 0 || label === "[close]") { + break; + } + found.push(label); + } + return found; +} + +/** Focus one control and activate it, the way a person does. */ +function* activate(terminal: Terminal, marker: string): Operation { + yield* focusOn(terminal, marker); + terminal.feed("\r"); + yield* settled(30); +} + +/** Every history file the repository holds, by name. */ +function* histories(root: string): Operation { + const entries = yield* until(readdir(join(root, "xmd", "repl"))); + return entries.filter((name) => name.endsWith(".jsonl")).sort(); +} + +/** The last row of the screen that contains this text, or none. */ +function lastRowContaining(rows: readonly string[], text: string): number { + for (let row = rows.length - 1; row >= 0; row -= 1) { + if (rows[row].includes(text)) { + return row; + } + } + return -1; +} + +/** One history file's lines, projected as the model reads them. */ +function* projectionOf(root: string, file: string): Operation { + const projected = projectRepl( + (yield* records(root, file)).map((line) => { + const parsed = parseDurableEvent(line); + if (!parsed.ok) { + throw parsed.error; + } + return parsed.value; + }), + ); + if (!projected.ok) { + throw projected.error; + } + return projected.value; +} + +/** One history file's lines, without the trailing empty one. */ +function* records(root: string, file: string): Operation { + const text = yield* until(readFile(join(root, "xmd", "repl", file), "utf8")); + return text.split("\n").filter((line) => line.length > 0); +} + +describe("REPL journey: one entry, from raw bytes", () => { + // The same compiler an entrypoint installs: an eval block is compiled, and a + // host that supplies none refuses rather than evaluating nothing. + beforeAll(() => useTempFileCompiler()); + + it("J1: pasting the reference entry puts all of it in the draft and the location", function* () { + const { terminal, install } = recordingTerminal(); + const source = yield* referenceSource(); + + let outcome: ReplOutcome | undefined; + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + outcome = ran.value; + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + // Pasted, as a terminal delivers a paste: one burst of bytes carrying + // every line, including the newlines between them. The scanner hands over + // 128 events per call and buffers the rest, so proving the whole document + // arrived is proving the host drained it. + const before = terminal.presented.length; + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + + // One burst is one frame, not one frame per character. + expect(terminal.presented.length - before).toBeLessThan(4); + // The field is one line and says how many precede it. + expect(shows(terminal, `[${source.split("\n").length - 1} lines]`)).toBe(true); + // Nothing durable yet: a draft is this process's, and the file is empty. + expect(yield* records(root, files[0])).toEqual([]); + + terminal.end(); + yield* running; + }); + + // The canonical location carries the draft exactly — every line, every + // character — which is what makes a draft reopenable before it is durable. + const location = outcome?.location; + expect(location).toBeDefined(); + const decoded = decodeLocation(location ?? ""); + expect(decoded.ok).toBe(true); + if (decoded.ok) { + expect(decoded.value.draft).toBe(source); + expect(decoded.value.surface).toBe("repl"); + expect(decoded.value.at).toBeUndefined(); + } + }); + + it("J1: submitting admits exactly that source, and the run settles", function* () { + const { terminal, install } = recordingTerminal(); + const source = yield* referenceSource(); + + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + terminal.feed("\r"); + yield* settled(400); + + const lines = yield* records(root, files[0]); + expect(lines.length).toBeGreaterThan(0); + // Every line is an ordinary durable record. There is no REPL record type, + // no manifest and no cache: the file and the URL are the whole state. + for (const line of lines) { + const parsed: unknown = JSON.parse(line); + expect(typeof parsed).toBe("object"); + if (typeof parsed === "object" && parsed !== null && "type" in parsed) { + expect(["yield", "close"]).toContain(parsed.type); + } + } + + // The entry is admitted and immutable, and the question it reaches is + // being asked. + expect(shows(terminal, "(one entry admitted)")).toBe(true); + expect(shows(terminal, "Approve Ship the REPL?")).toBe(true); + + terminal.end(); + yield* running; + }); + + expect(terminal.resets).toBe(1); + expect(terminal.readers).toBe(0); + expect(terminal.listeners).toBe(0); + }); + + it("J1: the question is answered through the terminal, and the answer settles", function* () { + const { terminal, install } = recordingTerminal(); + const source = yield* referenceSource(); + let location: string | undefined; + + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + location = ran.value.location; + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + terminal.feed("\r"); + yield* settled(400); + expect(shows(terminal, "Approve Ship the REPL?")).toBe(true); + + // The question's drawer was offered when the question appeared: a blocked + // document is the interaction, not something to go hunting for. + // The drawer shows the retained schema as a form: one field, its choices. + expect(shows(terminal, "decision: approve | decline")).toBe(true); + + // Typed, as a person types it, into the field the drawer focused. + terminal.bytes(BYTES.encode("approve")); + yield* settled(30); + terminal.feed("\r"); + yield* settled(400); + + // The answer is recorded as an ordinary elicit event, and what the document + // rendered after it changed because of the stored answer. + const lines = yield* records(root, files[0]); + const recorded = lines.filter((line) => line.includes("elicit")); + expect(recorded.length).toBeGreaterThan(0); + expect(shows(terminal, "Decision: approve")).toBe(true); + // Settled: the root closed, so the answer is in the history and the entry + // is immutable. + expect(shows(terminal, "(one entry admitted)")).toBe(true); + + terminal.end(); + yield* running; + }); + + // The Sessions surface is still there, and still explicitly empty. + expect(shows(terminal, "(none retained)")).toBe(true); + expect(location).toBeDefined(); + expect(terminal.resets).toBe(1); + }); + + it("J1: a second process reconstructs the view from the URL and the journal alone", function* () { + const source = yield* referenceSource(); + const first = recordingTerminal(); + let root: string | undefined; + let files: string[] = []; + + // One complete journey, then the process is over. + yield* scoped(function* (): Operation { + yield* first.install(); + yield* immediateClock(); + root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* settled(); + files = yield* histories(root); + + first.terminal.bytes(BYTES.encode(source)); + yield* settled(60); + first.terminal.feed("\r"); + yield* settled(400); + first.terminal.bytes(BYTES.encode("approve")); + yield* settled(30); + first.terminal.feed("\r"); + yield* settled(400); + + first.terminal.end(); + yield* running; + }); + // Halted completely: no session, no provider, no observer, no view cache. + expect(first.terminal.resets).toBe(1); + expect(first.terminal.readers).toBe(0); + + const retained = root; + expect(retained).toBeDefined(); + if (retained === undefined) { + throw new Error("the first process created a repository"); + } + const execution = files[0].replace(/\.jsonl$/, ""); + const lines = yield* records(retained, files[0]); + + // A location that selects something meaningful: a structural scope inside the + // entry, a historical position, and a recorded drawer. + const projected = projectRepl( + lines.map((line) => { + const parsed = parseDurableEvent(line); + if (!parsed.ok) { + throw parsed.error; + } + return parsed.value; + }), + ); + if (!projected.ok) { + throw projected.error; + } + const entry = projected.value.entry; + expect(entry).toBeDefined(); + const nested = entry?.scopes[0]; + expect(nested).toBeDefined(); + const binding = entry?.bindings[0]; + expect(binding).toBeDefined(); + const marker = projected.value.checkpoints[projected.value.checkpoints.length - 1]?.marker; + expect(marker).toBeDefined(); + if ( + entry === undefined || + nested === undefined || + binding === undefined || + marker === undefined + ) { + throw new Error("the retained history holds an entry, a nested scope, a binding and markers"); + } + + // A drawer names something inside the scope the location selects, which is + // why this selects the entry: the binding is the entry's, and a location that + // opened it beside a nested scope would be describing two different places. + const location = encodeLocation({ + execution, + surface: "repl", + scopes: [entry.key], + drawers: [{ kind: "binding", name: binding.name }], + at: marker, + inspect: true, + draft: undefined, + }); + + // A fresh host: a new terminal, a new repository handle, and nothing carried + // over but that URL and the file it names. + const second = recordingTerminal(); + let performed: Performed | undefined; + yield* scoped(function* (): Operation { + yield* second.install(); + yield* immediateClock(); + yield* installReplHost({ + dataRoot: () => retained, + identify: () => { + throw new Error("a reopened execution mints no identifier"); + }, + createExclusive: () => Promise.reject(new Error("a reopened execution creates no file")), + appendRecord: (path, record) => appendFile(path, record), + }); + performed = yield* countPerformed(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + location, + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(second.terminal); + + // The same topology, the same binding value, the same transcript and the + // same recorded answer — read from the file, not re-run. + // The drawer the location named is open on the retained value. + expect(shows(second.terminal, "[close]")).toBe(true); + + // Out from under the drawer first: a modal covers what it is in front of, so + // the columns beneath it are read once it is closed. + second.terminal.feed("\x1b"); + yield* settled(30); + + // The same topology: the entry, and the nested component occurrence beneath + // it, at the exact keys the first process recorded. + expect(shows(second.terminal, entry.name)).toBe(true); + expect(shows(second.terminal, nested.name)).toBe(true); + // The same binding value, read from the file. + expect(shows(second.terminal, `${binding.name} = `)).toBe(true); + // The same transcript, including the answer the first process gave and the + // output that answer produced. + expect(shows(second.terminal, "Decision: approve")).toBe(true); + + second.terminal.end(); + yield* running; + }); + + // No completed work was performed again: no component source was read, no + // eval block was compiled, and nobody was asked anything. + expect(performed?.reads ?? ["unmeasured"]).toEqual([]); + expect(performed?.compiles).toBe(0); + expect(performed?.asked).toBe(0); + // And nothing was appended: the file is exactly what the first process left. + expect(yield* records(retained, files[0])).toEqual(lines); + expect(second.terminal.resets).toBe(1); + }); + + it("J1: starting with no location creates one execution and focuses an empty draft", function* () { + const { terminal, install } = recordingTerminal(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + let outcome: ReplOutcome | undefined; + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + outcome = ran.value; + }); + yield* settled(); + + // One retained execution, created exclusively, and named opaquely. + const files = yield* histories(root); + expect(files).toHaveLength(1); + const execution = files[0].replace(/\.jsonl$/, ""); + expect(execution).toMatch(/^[A-Za-z0-9][A-Za-z0-9_-]{0,63}$/); + // Nothing is in it yet: no entry, so no record. + expect(yield* records(root, files[0])).toEqual([]); + + // The screen says where it is, what it holds and what it is waiting for. + expect(shows(terminal, "Sessions")).toBe(true); + expect(shows(terminal, "(none retained)")).toBe(true); + expect(shows(terminal, "Entries")).toBe(true); + expect(shows(terminal, "(not submitted)")).toBe(true); + + terminal.end(); + yield* running; + expect(outcome?.location).toBe(`xmd://repl/${execution}/repl`); + expect(outcome?.refusal).toBeUndefined(); + }); + + // And it gave the terminal back. + expect(terminal.resets).toBe(1); + expect(terminal.readers).toBe(0); + expect(terminal.listeners).toBe(0); + expect(terminal.raw[terminal.raw.length - 1]).toBe(false); + }); +}); + +describe("REPL journey: what it refuses, and what it leaves alone", () => { + it("J1: an invalid navigation leaves the standing route, tree, focus and targets", function* () { + const { terminal, install } = recordingTerminal(); + const source = yield* referenceSource(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + // This one runs the document for real, so it needs the compiler an + // entrypoint installs. + yield* useTempFileCompiler(); + const root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + terminal.feed("\r"); + yield* settled(400); + yield* dismissQuestion(terminal); + + // A standing, meaningful selection, reached through the product: the nested + // component occurrence, which exists only after the run admitted it. + yield* activate(terminal, "component Checklist"); + const standing = locationOn(terminal); + expect(standing).toContain("/Checklist-1"); + const before = screenOf(terminal); + const admitted = yield* records(root, files[0]); + + // Now a history position from *before* that scope was admitted. Both + // controls are ordinary mounted controls, and the combination is a view that + // cannot exist: the route selects a scope the prefix does not hold. + yield* activate(terminal, "[history]"); + yield* focusOn(terminal, "Entry 1 admitted"); + // What holds focus at the moment of the refused navigation, so the claim + // afterwards is about the same control rather than about there being one. + const focusedBefore = "Entry 1 admitted"; + expect(focusedOn(terminal, focusedBefore)).toBe(true); + // And where its close control is, recorded now. + const closeAt = coordinateOf(terminal, "[close]"); + terminal.feed("\r"); + yield* settled(60); + + // The route that stands is the one that worked. The drawer it was asked + // from is still open — the navigation did not happen, so nothing it would + // have changed changed — and the file is untouched. + expect(locationOn(terminal)).toContain("/Checklist-1"); + expect(locationOn(terminal)).not.toContain("at="); + expect(shows(terminal, "History")).toBe(true); + expect(yield* records(root, files[0])).toEqual(admitted); + // And the screen says why, which is the only difference from before. + expect(shows(terminal, "!")).toBe(true); + + // The same control still holds focus, in the same place. + expect(focusedOn(terminal, focusedBefore)).toBe(true); + // And the same pointer target is still the same reachable node: the exact + // coordinate recorded *before* the refusal still reaches the same control + // and still does what it did. Sent to that coordinate rather than to + // wherever the control is now, because "the target moved" and "the target + // survived" are different answers. + expect(closeAt).toBeDefined(); + if (closeAt === undefined) { + throw new Error("the open drawer has a close control"); + } + yield* clickAt(terminal, closeAt); + expect(locationOn(terminal)).not.toContain("+history"); + expect(locationOn(terminal)).toContain("/Checklist-1"); + expect(before.length).toBeGreaterThan(0); + + terminal.end(); + yield* running; + }); + }); + + it("J1: a document that cannot be admitted leaves the draft and an empty journal", function* () { + const { terminal, install } = recordingTerminal(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + // A component nothing supplies: refused before anything is admitted. + terminal.bytes(BYTES.encode("")); + yield* settled(30); + terminal.feed("\r"); + yield* settled(80); + + // The journal is empty, and the draft is exactly what was typed. + expect(yield* records(root, files[0])).toEqual([]); + expect(shows(terminal, "")).toBe(true); + // And the refusal says so rather than the keystroke seeming to do nothing. + expect(screenOf(terminal).some((line) => line.includes("!"))).toBe(true); + + terminal.end(); + yield* running; + }); + expect(terminal.resets).toBe(1); + }); + + it("J1: a second entry is refused, and history does not change", function* () { + const { terminal, install } = recordingTerminal(); + const source = yield* referenceSource(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + // This one runs the document for real, so it needs the compiler an + // entrypoint installs. + yield* useTempFileCompiler(); + const root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + terminal.feed("\r"); + yield* settled(400); + const admitted = yield* records(root, files[0]); + expect(admitted.length).toBeGreaterThan(0); + + // The question's drawer owns focus while it is up, so typing would reach its + // answer field rather than the draft. Dismissed first, and focus put back on + // the draft, so what follows really is an attempt to submit a second entry. + yield* dismissQuestion(terminal); + yield* focusOn(terminal, "(one entry admitted)"); + + terminal.bytes(BYTES.encode("")); + yield* settled(30); + terminal.feed("\r"); + yield* settled(120); + + // The submission path itself refused, history did not change, and the + // screen says why rather than appearing to do nothing. + expect(yield* records(root, files[0])).toEqual(admitted); + expect(shows(terminal, "(one entry admitted)")).toBe(true); + expect(shows(terminal, "admits one entry")).toBe(true); + + terminal.end(); + yield* running; + }); + }); + + it("J1: a corrupt history refuses atomically, with no append and no session", function* () { + const { terminal, install } = recordingTerminal(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + const performed = yield* countPerformed(); + + // A file that is not a projectable history. + const directory = join(root, "xmd", "repl"); + yield* until(mkdir(directory, { recursive: true })); + const path = join(directory, "broken.jsonl"); + yield* until(writeFile(path, "{not a record}\n")); + const before = yield* until(readFile(path, "utf8")); + + // The refusal is a screen, so it is held until the person leaves it. + let refused: boolean | undefined; + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + location: "xmd://repl/broken/repl", + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + refused = !ran.ok; + }); + yield* settled(20); + // What it says, rather than a crash: the command cannot show this history. + expect(shows(terminal, "cannot read")).toBe(true); + terminal.end(); + yield* running; + + // Refused, not thrown: a location this command cannot show is an outcome. + expect(refused).toBe(true); + // Nothing ran, nothing was asked, and nothing was appended. + expect(performed.reads).toEqual([]); + expect(performed.compiles).toBe(0); + expect(performed.asked).toBe(0); + expect(yield* until(readFile(path, "utf8"))).toBe(before); + }); + expect(terminal.resets).toBe(1); + expect(terminal.readers).toBe(0); + expect(terminal.listeners).toBe(0); + }); + + it("J1: a malformed location reaches no path, no file and no terminal", function* () { + const { terminal, install } = recordingTerminal(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + const ran = yield* runReplProgram({ location: "xmd://repl//repl" }); + expect(ran.ok).toBe(false); + + // No directory was formed, no file was created, and the terminal's modes + // were never touched. + expect(yield* until(readdir(root))).toEqual([]); + expect(terminal.raw).toEqual([]); + expect(terminal.resets).toBe(0); + expect(terminal.readers).toBe(0); + }); + }); + + it("J1: a control chord inserts nothing into the draft", function* () { + const { terminal, install } = recordingTerminal(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + yield* useTemporaryHost(); + + let outcome: ReplOutcome | undefined; + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + outcome = ran.value; + }); + yield* settled(); + + // Control-C, Alt-a, Control-H and F5: each decodes to a letter or a key + // this product has no meaning for, and none of them is text. + terminal.bytes(new Uint8Array([0x03])); + terminal.bytes(BYTES.encode("\x1ba")); + terminal.bytes(new Uint8Array([0x08])); + terminal.bytes(BYTES.encode("\x1b[15~")); + yield* settled(30); + terminal.end(); + yield* running; + + // The draft is still empty, so the canonical location carries none. + const decoded = decodeLocation(outcome?.location ?? ""); + expect(decoded.ok).toBe(true); + if (decoded.ok) { + expect(decoded.value.draft).toBeUndefined(); + } + }); + }); + + it("J1: a terminal below the minimum refuses, and recovers when it grows", function* () { + const { terminal, install } = recordingTerminal({ columns: 40, rows: 10 }); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* settled(20); + + // The one refusal with a remedy the person already has. + expect(shows(terminal, "at least 72x20")).toBe(true); + // And nothing behind it: no control was drawn, so none can be pointed at. + expect(shows(terminal, "Sessions")).toBe(false); + + terminal.resized({ columns: 160, rows: 36 }); + yield* settled(30); + // Recovered without restarting anything. + expect(shows(terminal, "Sessions")).toBe(true); + + terminal.end(); + yield* running; + }); + expect(terminal.resets).toBe(1); + }); +}); + +describe("REPL journey: the whole of it, from raw bytes", () => { + it("J1: run, hold at a gate, inspect a prefix, return live, continue, answer, settle", function* () { + const source = yield* referenceSource(); + const first = recordingTerminal(); + let root: string | undefined; + let files: string[] = []; + let captured: string | undefined; + let liveMarkers: string[] = []; + + yield* scoped(function* (): Operation { + yield* first.install(); + yield* immediateClock(); + yield* useTempFileCompiler(); + root = yield* useTemporaryHost(); + const terminal = first.terminal; + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + files = yield* histories(root); + + // 1. The draft, and the exact canonical location that carries it. Decoded + // from what is on the terminal, and compared to the whole pasted source. + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + const drafting = decodeLocation(locationOn(terminal)); + expect(drafting.ok).toBe(true); + if (drafting.ok) { + expect(drafting.value.draft).toBe(source); + } + + // 2. Admitted, and then held before it can reach the question. + // + // Pause has to be *requested* before the document gets as far as the + // question, because the provider's outstanding request is work that keeps a + // walk from ever becoming satisfied — a pause asked for then is a request + // that never becomes a hold. So focus is put on the control that will sit + // directly above Pause first, and the submit is followed immediately by the + // two keystrokes that reach it: no screen is read in between, and nothing + // waits. + // Enter activates whatever holds focus, and what holds it is the draft — so + // the submit goes first, and Pause is reached by walking *backwards* from + // the draft: while expansion is playing nothing is held, so Continue is not + // mounted and Pause is the one control before the draft. All of it in one + // burst, so nothing is read and nothing waits. + expect(focusedOn(terminal, "[20 lines]")).toBe(true); + terminal.feed("\r"); + terminal.feed("\x1b[Z\r"); + + // Exactly paused: expansion reached a gate and stopped there. `pausing` is a + // request that has not been met, and continuing one of those continues + // nothing. + yield* until_(terminal, "expansion paused", (t) => shows(t, "[pause] paused")); + + // 3. Held: over many turns and clock releases, neither the file nor the + // screen moves. + const heldRecords = yield* records(root, files[0]); + const heldScreen = screenOf(terminal); + for (let turn = 0; turn < 6; turn += 1) { + yield* sleep(10); + yield* settled(30); + } + expect(yield* records(root, files[0])).toEqual(heldRecords); + expect(screenOf(terminal)).toEqual(heldScreen); + // And it is still held rather than having drifted on. + expect(shows(terminal, "[pause] paused")).toBe(true); + expect(shows(terminal, "Approve Ship the REPL?")).toBe(false); + + // 4. An earlier prefix, chosen through History. + yield* activate(terminal, "[history]"); + expect(locationOn(terminal)).toContain("+history"); + yield* focusOn(terminal, "Entry 1 admitted"); + terminal.feed("\r"); + yield* settled(40); + + const frozen = locationOn(terminal); + expect(frozen).toContain("at="); + expect(frozen).toContain("inspect"); + // Read only, and nothing of the present in it. + expect(shows(terminal, "[pause]")).toBe(false); + expect(screenOf(terminal).some((line) => line.trim().startsWith("…"))).toBe(false); + + // 5. Back to the head, and Continue — once — releases the hold. + terminal.feed("\x1b"); + yield* settled(30); + yield* activate(terminal, "[live]"); + expect(locationOn(terminal)).not.toContain("at="); + expect(shows(terminal, "Approve Ship the REPL?")).toBe(false); + + yield* activate(terminal, "[continue]"); + yield* until_(terminal, "the question", (t) => shows(t, "Approve Ship the REPL?")); + + // 6. The same execution takes the typed answer and settles. + yield* until_(terminal, "the question's drawer", (t) => + shows(t, "decision: approve | decline"), + ); + terminal.bytes(BYTES.encode("approve")); + yield* settled(30); + terminal.feed("\r"); + yield* until_(terminal, "the answer's output", (t) => shows(t, "Decision: approve")); + expect(shows(terminal, "(one entry admitted)")).toBe(true); + + // The generated fragment's source is on screen above the row that admits it. + // Which two rows those are comes from the model rather than from a guess at + // their wording; which order they are in comes from the screen. + const settledModel = yield* projectionOf(root, files[0]); + const at = settledModel.transcript.findIndex((row) => row.kind === "generated"); + expect(at).toBeGreaterThanOrEqual(0); + const sourceRow = settledModel.transcript[at]; + const admission = settledModel.transcript.slice(at + 1).find((row) => row.kind === "effect"); + if (sourceRow?.kind !== "generated" || admission?.kind !== "effect") { + throw new Error("the settled history holds a generated fragment and its admission"); + } + const rows = screenOf(terminal).map((line) => line.trim()); + const sourceAt = rows.findIndex((line) => line.includes(sourceRow.source ?? "«none»")); + const admittedAt = lastRowContaining(rows, `${admission.type} ${admission.status}`); + expect(sourceAt).toBeGreaterThanOrEqual(0); + expect(admittedAt).toBeGreaterThanOrEqual(0); + expect(sourceAt).toBeLessThan(admittedAt); + + // 7. The complete binding value, from the drawer that holds it. + yield* activate(terminal, "1. entry-1"); + yield* activate(terminal, "plan"); + expect(locationOn(terminal)).toContain("binding:plan"); + for (const line of JSON.stringify(PLAN, undefined, 2).split("\n")) { + expect(shows(terminal, line)).toBe(true); + } + terminal.feed("\x1b"); + yield* settled(30); + + // 8. The complete ordered set of History positions, as the drawer lists them. + yield* activate(terminal, "[history]"); + liveMarkers = drawerMarkers(terminal); + expect(liveMarkers.length).toBeGreaterThan(4); + + // 9. The historical state a reader would want back: the terminal position, + // read only, with the recorded question's drawer open on it. + const terminalMarker = liveMarkers[liveMarkers.length - 1]; + yield* focusOn(terminal, terminalMarker); + terminal.feed("\r"); + yield* settled(40); + terminal.feed("\x1b"); + yield* settled(30); + yield* activate(terminal, "answered"); + + captured = locationOn(terminal); + const decoded = decodeLocation(captured); + expect(decoded.ok).toBe(true); + if (decoded.ok) { + expect(decoded.value.scopes).toEqual(["entry-1"]); + expect(decoded.value.at).toBeDefined(); + expect(decoded.value.inspect).toBe(true); + expect(decoded.value.drawers.map((one) => one.kind)).toEqual(["recorded-elicit"]); + } + // The whole retained schema and the whole retained answer, in rows. + for (const line of JSON.stringify(SCHEMA, undefined, 2).split("\n")) { + expect(shows(terminal, line)).toBe(true); + } + for (const line of JSON.stringify(ANSWER, undefined, 2).split("\n")) { + expect(shows(terminal, line)).toBe(true); + } + + terminal.end(); + yield* running; + }); + + const retained = root; + if (retained === undefined || captured === undefined) { + throw new Error("the first process ran and reported a location"); + } + const lines = yield* records(retained, files[0]); + expect(lines.length).toBeGreaterThan(0); + + // 10. A fresh host, with nothing but that location and the file. + const second = recordingTerminal(); + let performed: Performed | undefined; + yield* scoped(function* (): Operation { + yield* second.install(); + yield* immediateClock(); + yield* installReplHost({ + dataRoot: () => retained, + identify: () => { + throw new Error("a reopened execution mints no identifier"); + }, + createExclusive: () => Promise.reject(new Error("a reopened execution creates no file")), + appendRecord: (path, record) => appendFile(path, record), + }); + performed = yield* countPerformed(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + location: captured, + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(second.terminal); + + // The same location, unchanged, rendered from the URL it was given. + expect(locationOn(second.terminal)).toBe(captured); + + // The same retained question: the whole schema and the whole answer, not a + // label that happens to contain the word. + for (const line of JSON.stringify(SCHEMA, undefined, 2).split("\n")) { + expect(shows(second.terminal, line)).toBe(true); + } + for (const line of JSON.stringify(ANSWER, undefined, 2).split("\n")) { + expect(shows(second.terminal, line)).toBe(true); + } + + // Then out from under the drawer, for the rest of what the file holds. + second.terminal.feed("\x1b"); + yield* settled(30); + expect(shows(second.terminal, "component Checklist")).toBe(true); + expect(shows(second.terminal, "generated admitted:")).toBe(true); + expect(shows(second.terminal, "Decision: approve")).toBe(true); + + // The same complete binding value. + yield* activate(second.terminal, "plan"); + for (const line of JSON.stringify(PLAN, undefined, 2).split("\n")) { + expect(shows(second.terminal, line)).toBe(true); + } + second.terminal.feed("\x1b"); + yield* settled(30); + + // And the same History positions, in the same order. + yield* activate(second.terminal, "[history]"); + expect(drawerMarkers(second.terminal)).toEqual(liveMarkers); + + second.terminal.end(); + yield* running; + }); + + // Nothing was performed again, and nothing was written. + expect(performed?.reads ?? ["unmeasured"]).toEqual([]); + expect(performed?.compiles).toBe(0); + expect(performed?.asked).toBe(0); + expect(yield* records(retained, files[0])).toEqual(lines); + expect(second.terminal.resets).toBe(1); + expect(second.terminal.readers).toBe(0); + expect(second.terminal.listeners).toBe(0); + }); +}); + +describe("REPL journey: when a frame counts as applied", () => { + it("J1: a frame is not acknowledged until it has been presented", function* () { + const { terminal, install } = recordingTerminal(); + const clock = countingClock(); + + yield* scoped(function* (): Operation { + yield* install(); + yield* clock.install(); + yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + + // Let the first frames through, then block inside the next presentation. + for (let turn = 0; turn < 30; turn += 1) { + clock.release(); + yield* settled(6); + } + // Counted before the frame is drawn, so what is measured is what *this* + // frame caused rather than whatever had already happened. + const before = clock.waits; + terminal.holdNextPresent(); + // A resize, because it causes exactly one frame: a keystroke that moves + // focus causes a second one to redraw the marker, and a count has to have a + // baseline it can name. + terminal.resized({ columns: 150, rows: 34 }); + for (let turn = 0; turn < 20 && terminal.holdPresent === undefined; turn += 1) { + clock.release(); + yield* settled(6); + } + const held = terminal.holdPresent; + expect(held).toBeDefined(); + if (held === undefined) { + throw new Error("a presentation was blocked so the frame stream could be read"); + } + + // The bytes are being written. The tick this frame is drawing with came from + // a wait already counted, and because the frame has not been *applied* yet, + // the stream has scheduled nothing behind it. + yield* settled(20); + clock.release(); + yield* settled(20); + expect(clock.waits).toBe(before); + + // Released, and only now does the stream move on. + held.release(); + yield* settled(20); + clock.release(); + yield* settled(20); + expect(clock.waits).toBeGreaterThan(before); + + terminal.end(); + yield* running; + }); + }); +}); + +describe("REPL journey: the same product at every size", () => { + it("J1: a long draft location stays exact at medium, and no row crosses its region", function* () { + const source = yield* referenceSource(); + // Medium: a narrower content surface than wide, and an inspection column + // beside it — so a location row written at the wide width would run into it. + const { terminal, install } = recordingTerminal({ columns: 120, rows: 30 }); + + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + + // Exact, all of it, at this size. + const decoded = decodeLocation(locationOn(terminal)); + expect(decoded.ok).toBe(true); + if (decoded.ok) { + expect(decoded.value.draft).toBe(source); + } + + // And no row of it reaches past the surface it was placed in: the columns + // to its right belong to the inspection column, and the layout gave them + // to something else. + const surface = surfaceWidth(terminal.size); + expect(surface).toBe(64); + const rows = screenOf(terminal); + const first = rows.findIndex((line) => line.includes("xmd://repl/")); + expect(first).toBeGreaterThanOrEqual(0); + const at = rows[first].indexOf("xmd://repl/"); + for (let row = first; row < rows.length; row += 1) { + const inside = (rows[row] ?? "").slice(at, at + surface); + if (inside.trim().length === 0) { + break; + } + // Whatever is past the surface's right edge is not this row's. + expect((rows[row] ?? "").slice(at + surface, at + surface + 4).trim()).toBe(""); + } + + terminal.end(); + yield* running; + }); + }); + + it("J1: a drawer covers what it is in front of, at medium and at wide", function* () { + const source = yield* referenceSource(); + + for (const size of [ + { columns: 120, rows: 30 }, + { columns: 160, rows: 36 }, + ]) { + const { terminal, install } = recordingTerminal(size); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + yield* useTempFileCompiler(); + yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + terminal.feed("\r"); + yield* dismissQuestion(terminal); + yield* until_(terminal, "the transcript", (t) => shows(t, "component Checklist")); + + // A transcript with something in it — recorded before anything covers it, + // so what follows is a claim about coverage rather than about absence. + // Texts that belong to the transcript and to nothing the drawer lists, so + // finding one inside the drawer means it showed through. + const underneath = ["import_component ok", "About to evaluate:"]; + for (const text of underneath) { + expect(shows(terminal, text)).toBe(true); + } + + // And then a drawer over it. + yield* activate(terminal, "[history]"); + expect(shows(terminal, "History")).toBe(true); + + // None of it shows through: every row of the drawer's box reaches the + // box's own right edge, so what it is in front of is behind it. + const rows = screenOf(terminal); + const title = rows.findIndex((line) => line.includes("History")); + expect(title).toBeGreaterThanOrEqual(0); + const left = Math.floor(size.columns / 8); + const right = left + drawerWidth(terminal.size); + // The drawer's own rows: its title, one per position it lists, and its + // close control. A drawer draws rows rather than filling its box, so the + // rows below its last one are the transcript and are meant to be. + const drawn = drawerMarkers(terminal).length + 2; + expect(drawn).toBeGreaterThan(4); + for (let row = title; row < title + drawn; row += 1) { + const inside = (rows[row] ?? "").slice(left, right); + for (const text of underneath) { + expect(inside.includes(text)).toBe(false); + } + // And positionally: no label this drawer lists is anywhere near this + // long, so every column past it belongs to the drawer and must have + // been painted by it. A drawer that stopped short would leave whatever + // the transcript has out there exactly where it was. + expect((inside.slice(LABEL_ROOM) ?? "").trim()).toBe(""); + } + + terminal.end(); + yield* running; + }); + } + }); + + it("J1: narrow still routes one surface, with the location above it", function* () { + const { terminal, install } = recordingTerminal(NARROW); + + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + + // The routed surface, and the location above it — both, at the smallest + // size this REPL draws at. + expect(shows(terminal, "Entries")).toBe(true); + expect(locationOn(terminal)).toMatch(/^xmd:\/\/repl\/[A-Za-z0-9_-]+\/repl$/); + expect(surfaceWidth(terminal.size)).toBe(NARROW.columns); + + terminal.end(); + yield* running; + }); + }); +}); + +/** + * What this command settles before it is allowed to do anything. + * + * Each of these is about *order*. The command's refusals are cheap only while + * nothing has happened yet: a terminal that is not one, a URL that names + * something the file never held, a question that has already been answered. + * Discovered late, each of them costs something that cannot be taken back — an + * empty history nobody asked for, records a typo caused, a location naming a + * drawer that is not there. + */ +describe("REPL journey: what it settles before it acts", () => { + beforeAll(() => useTempFileCompiler()); + + it("J1: a piped invocation refuses before a history file exists", function* () { + // The one thing that differs from every other test here: this host's + // standard streams are not a terminal. + const { terminal, install } = recordingTerminal({ columns: 160, rows: 36 }, false); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + // Spawned rather than awaited, so that a command which *fails* to refuse + // is caught as a command still running instead of as a hanging test. + let ran: Result | undefined; + const running = yield* spawn(function* (): Operation { + ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + }); + yield* settled(60); + + // It came straight back. No screen was opened, so there is nothing to + // drive and nothing to wait for. + expect(ran).toBeDefined(); + expect(ran?.ok).toBe(false); + if (ran !== undefined && !ran.ok) { + expect(ran.error.message).toContain("not available over a pipe"); + } + // Nothing was created. Not an empty history file, not the directory that + // would hold one — the refusal happened before the repository did. + expect(yield* until(readdir(root))).toEqual([]); + // And nothing touched the terminal: no raw mode, no alternate screen, and + // so nothing to reset. + expect(terminal.raw).toEqual([]); + expect(terminal.presented).toEqual([]); + expect(terminal.resets).toBe(0); + + terminal.end(); + yield* running; + }); + }); + + it("J1: a route naming a scope the history never had refuses before anything replays", function* () { + const source = yield* referenceSource(); + const first = recordingTerminal(); + let retained = ""; + let files: string[] = []; + let entryKey = ""; + + // A history that is deliberately *unfinished*: the entry is admitted and + // the run is holding at its question. Replaying one of these has somewhere + // to go — it resumes, asks again and appends — which is exactly what a + // route that cannot resolve must not be allowed to cause. + yield* scoped(function* (): Operation { + yield* first.install(); + yield* immediateClock(); + retained = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(first.terminal); + files = yield* histories(retained); + + first.terminal.bytes(BYTES.encode(source)); + yield* settled(60); + first.terminal.feed("\r"); + yield* until_(first.terminal, "the question", (t) => shows(t, "Approve Ship the REPL?")); + + // Left unanswered on purpose. + first.terminal.end(); + yield* running; + }); + + const model = yield* projectionOf(retained, files[0]); + expect(model.entry).toBeDefined(); + entryKey = model.entry?.key ?? ""; + expect(model.settled).toBe(false); + const lines = yield* records(retained, files[0]); + + // A URL a person could plausibly type: the execution is real, the entry is + // real, and one segment of the path is a typo. + const mistyped = encodeLocation({ + execution: files[0].replace(/\.jsonl$/, ""), + surface: "repl", + scopes: [entryKey, "Nowhere-9"], + drawers: [], + at: undefined, + inspect: false, + draft: undefined, + }); + + const second = recordingTerminal(); + let performed: Performed | undefined; + let outcome: Result | undefined; + yield* scoped(function* (): Operation { + yield* second.install(); + yield* immediateClock(); + yield* installReplHost({ + dataRoot: () => retained, + identify: () => { + throw new Error("a reopened execution mints no identifier"); + }, + createExclusive: () => Promise.reject(new Error("a reopened execution creates no file")), + appendRecord: (path, record) => appendFile(path, record), + }); + performed = yield* countPerformed(); + + const running = yield* spawn(function* (): Operation { + outcome = yield* runReplProgram({ + location: mistyped, + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + }); + // A refusal screen, not a reconstruction: waited for by what it says + // rather than by a full frame, because a refusal *is* one row. + yield* until_(second.terminal, "the refusal", (t) => shows(t, "Nowhere-9")); + + // It names the segment it could not follow, rather than showing a view + // of something the URL did not ask for. + expect(shows(second.terminal, "holds no Nowhere-9")).toBe(true); + + second.terminal.end(); + yield* running; + }); + + expect(outcome?.ok).toBe(false); + // Nothing replayed. Nobody was asked the question a resumed run would have + // asked again, no eval block was compiled, and the file is byte for byte + // what the first process left behind. + expect(performed?.asked).toBe(0); + expect(performed?.compiles).toBe(0); + expect(yield* records(retained, files[0])).toEqual(lines); + }); + + it("J1: a stale live-question URL refuses against a finished execution", function* () { + const source = yield* referenceSource(); + const first = recordingTerminal(); + let retained = ""; + let files: string[] = []; + + // A history that ran to the end: the question was asked, answered and + // recorded, and the root closed. + yield* scoped(function* (): Operation { + yield* first.install(); + yield* immediateClock(); + retained = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(first.terminal); + files = yield* histories(retained); + + first.terminal.bytes(BYTES.encode(source)); + yield* settled(60); + first.terminal.feed("\r"); + yield* until_(first.terminal, "the question's drawer", (t) => + shows(t, "decision: approve | decline"), + ); + first.terminal.bytes(BYTES.encode("approve")); + yield* settled(30); + first.terminal.feed("\r"); + yield* until_(first.terminal, "the answer", (t) => shows(t, "Decision: approve")); + + first.terminal.end(); + yield* running; + }); + + const model = yield* projectionOf(retained, files[0]); + expect(model.settled).toBe(true); + const lines = yield* records(retained, files[0]); + + // The URL somebody kept from while the question was up. The execution it + // names is real and the drawer it names was real — and the question is gone, + // which no history records, so nothing this file holds could say otherwise. + const stale = encodeLocation({ + execution: files[0].replace(/\.jsonl$/, ""), + surface: "repl", + scopes: [model.entry?.key ?? ""], + drawers: [{ kind: "live-elicit" }], + at: undefined, + inspect: false, + draft: undefined, + }); + expect(stale).toContain("+elicit"); + + const second = recordingTerminal(); + let performed: Performed | undefined; + let outcome: Result | undefined; + yield* scoped(function* (): Operation { + yield* second.install(); + yield* immediateClock(); + yield* installReplHost({ + dataRoot: () => retained, + identify: () => { + throw new Error("a reopened execution mints no identifier"); + }, + createExclusive: () => Promise.reject(new Error("a reopened execution creates no file")), + appendRecord: (path, record) => appendFile(path, record), + }); + performed = yield* countPerformed(); + + const running = yield* spawn(function* (): Operation { + outcome = yield* runReplProgram({ + location: stale, + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + }); + yield* until_(second.terminal, "the refusal", (t) => shows(t, "nothing is being asked")); + + second.terminal.end(); + yield* running; + }); + + // A refusal rather than a success with nothing on the screen, and the run it + // would have replayed never happened. + expect(outcome?.ok).toBe(false); + expect(performed?.asked).toBe(0); + expect(performed?.compiles).toBe(0); + expect(yield* records(retained, files[0])).toEqual(lines); + }); + + it("J1: an answered history with its root still open refuses the same URL", function* () { + // The shape `settled` cannot speak for: the question was asked *and + // answered*, and only the root close is missing. Reopening it asks nobody + // anything — it finishes the root and stops — so a route that resolved its + // live drawer here would replay, append that close, and only then discover + // there was no drawer to mount. + const events = yield* referenceEvents(); + const unclosed = events.slice(0, -1); + expect(events[events.length - 1]?.type).toBe("close"); + + const { terminal, install } = recordingTerminal(); + let outcome: Result | undefined; + let performed: Performed | undefined; + let before = ""; + let path = ""; + + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + const directory = join(root, "xmd", "repl"); + yield* until(mkdir(directory, { recursive: true })); + path = join(directory, "unclosed.jsonl"); + yield* until(writeFile(path, unclosed.map((event) => serializeDurableEvent(event)).join(""))); + before = yield* until(readFile(path, "utf8")); + performed = yield* countPerformed(); + + const model = yield* projectionOf(root, "unclosed.jsonl"); + expect(model.settled).toBe(false); + expect(model.entry?.elicitations[0].answer).toEqual({ decision: "approve" }); + + const running = yield* spawn(function* (): Operation { + outcome = yield* runReplProgram({ + location: `xmd://repl/unclosed/repl/${model.entry?.key ?? ""}/+elicit`, + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + }); + yield* until_(terminal, "the refusal", (t) => shows(t, "nothing is being asked")); + + terminal.end(); + yield* running; + }); + + expect(outcome?.ok).toBe(false); + // Nothing replayed, and the close this reopen would have written is not + // there: the file is exactly the prefix that was handed to it. + expect(performed?.asked).toBe(0); + expect(performed?.compiles).toBe(0); + expect(yield* until(readFile(path, "utf8"))).toBe(before); + }); + + it("J1: an accepted answer closes the drawer and clears it from the location", function* () { + const { terminal, install } = recordingTerminal(); + const source = yield* referenceSource(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + terminal.feed("\r"); + yield* until_(terminal, "the question", (t) => shows(t, "Approve Ship the REPL?")); + + // The drawer is offered on its own, and the URL says so while it is up. + yield* until_( + terminal, + "the question's drawer", + (t) => maybeLocation(t)?.includes("+elicit") === true, + ); + expect(shows(terminal, "[close]")).toBe(true); + + terminal.bytes(BYTES.encode("approve")); + yield* settled(30); + terminal.feed("\r"); + yield* until_(terminal, "the answer", (t) => shows(t, "Decision: approve")); + + // The question is over, so the drawer is over: it is off the screen, and + // it is out of the URL. A location still naming `+elicit` would name a + // drawer nothing mounts, which is a view nobody can be shown. + expect(shows(terminal, "[close]")).toBe(false); + expect(locationOn(terminal)).not.toContain("+elicit"); + // And the form it was typed into is gone with it, rather than standing + // there still offering the choices. + expect(shows(terminal, "decision: approve | decline")).toBe(false); + + expect((yield* records(root, files[0])).length).toBeGreaterThan(0); + terminal.end(); + yield* running; + }); + }); + + it("J1: Continue is mounted only while a continuation is held", function* () { + const { terminal, install } = recordingTerminal(); + const source = yield* referenceSource(); + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + yield* useTemporaryHost(); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + expect(focusedOn(terminal, "[20 lines]")).toBe(true); + // Submit, then reach Pause by walking backwards one control — which is + // the whole claim: while expansion is playing, Pause is the control + // immediately before the draft because Continue is not mounted at all. + terminal.feed("\r"); + terminal.feed("\x1b[Z\r"); + + yield* until_(terminal, "expansion paused", (t) => shows(t, "[pause] paused")); + // Now a continuation is held, so the control that releases it exists. + expect(shows(terminal, "[continue]")).toBe(true); + + terminal.end(); + yield* running; + }); + }); + + it("J1: Continue is refused while a pause is still being taken", function* () { + // The state between asking and holding. A Continue accepted here would + // withdraw the pause instead of releasing anything, so the reduction + // refuses it — and says which of the two it is. + const pausing = reduceRepl(initialState("abc"), { kind: "continue" }, EMPTY_MODEL_FOR_TEST, { + output: "", + question: undefined, + expansion: "pausing", + pausable: true, + }); + expect(pausing.intent.kind).toBe("none"); + expect(pausing.state.refusal).toContain("not paused"); + + const held = reduceRepl(initialState("abc"), { kind: "continue" }, EMPTY_MODEL_FOR_TEST, { + output: "", + question: undefined, + expansion: "paused", + pausable: true, + }); + expect(held.intent.kind).toBe("continue"); + expect(held.state.refusal).toBe(undefined); + }); +}); + +/** + * What a held run has already printed is on the screen while it is held. + * + * The elicitation provider is stopped before it registers its question, so the + * run is inside the document with nothing pending: no question to show, no + * answer to append, and the last thing it did was print. What it printed has + * to be readable *then* rather than when the run finishes, because the overlay + * exists precisely to show what the Journal has not settled yet. + * + * This pins the product's behaviour, not the wake that delivers it: expansion + * moves around every element, so a frame is owed at nearly the same instant + * for a second reason. `repl-execution.test.ts` covers the overlay's own + * report of its output, which is the part that has no other announcer. + */ +describe("REPL journey: output nothing records still reaches the screen", () => { + beforeAll(() => useTempFileCompiler()); + + /** Distinctive, so finding it on the screen cannot be finding something else. */ + const PRINTED = "the-last-thing-this-document-prints"; + + it("J1: text the Journal has not settled is drawn while the run is held", function* () { + const { terminal, install } = recordingTerminal(); + const gate = withResolvers(); + let held = 0; + // One durable record, then plain text, then the question. The text is the + // last thing this document does before it blocks, so no record follows it + // and nothing but the text itself can ask for the frame that shows it. + const source = + "```js eval\n" + + 'const schema = { type: "object", properties: ' + + '{ decision: { type: "string", enum: ["yes"] } }, ' + + 'required: ["decision"], additionalProperties: false };\n' + + "```\n\n" + + `${PRINTED}\n\n` + + 'Decide?\n'; + + yield* scoped(function* (): Operation { + yield* install(); + yield* immediateClock(); + const root = yield* useTemporaryHost(); + + // Held *before* the question is registered, so nothing about it has + // reached this process yet: no pending question, no wake. + yield* Elicitation.around({ + *elicit([request], next) { + held += 1; + yield* gate.operation; + return yield* next(request); + }, + }); + + const running = yield* spawn(function* (): Operation { + const ran = yield* runReplProgram({ + includes: [REFERENCE_DIRECTORY], + installations: INSTALLATIONS, + }); + if (!ran.ok) { + throw ran.error; + } + }); + yield* untilDrawn(terminal); + const files = yield* histories(root); + + terminal.bytes(BYTES.encode(source)); + yield* settled(60); + terminal.feed("\r"); + + // Wait for the run to reach the question and stop there. + yield* until_(terminal, "the run reaching its question", () => held > 0); + // Then let it sit, with nothing else able to move. + for (let turn = 0; turn < 4; turn += 1) { + yield* sleep(10); + yield* settled(30); + } + + // No question is pending, so nothing but output has woken this screen. + expect(shows(terminal, "Decide?")).toBe(false); + // The text after the last record this document writes. Nothing recorded + // it, nothing reprojected because of it, and it is on the screen. + expect(shows(terminal, PRINTED)).toBe(true); + + gate.resolve(); + yield* until_(terminal, "the question", (t) => shows(t, "Decide?")); + expect((yield* records(root, files[0])).length).toBeGreaterThan(0); + + terminal.end(); + yield* running; + }); + }); +}); diff --git a/packages/cli/tests/repl-route.test.ts b/packages/cli/tests/repl-route.test.ts index bc7329fe..c6c4f755 100644 --- a/packages/cli/tests/repl-route.test.ts +++ b/packages/cli/tests/repl-route.test.ts @@ -44,16 +44,16 @@ function projected(events: readonly DurableEvent[], selection?: string): ReplMod return result.value; } -function resolved(model: ReplModel, location: string): ReplSelection { - const result = resolveLocation(model, decoded(location)); +function resolved(model: ReplModel, location: string, asking = false): ReplSelection { + const result = resolveLocation(model, decoded(location), asking); if (!result.ok) { throw result.error; } return result.value; } -function unresolved(model: ReplModel, location: string): string { - const result = resolveLocation(model, decoded(location)); +function unresolved(model: ReplModel, location: string, asking = false): string { + const result = resolveLocation(model, decoded(location), asking); if (result.ok) { throw new Error(`${location} resolved, and this model cannot answer it`); } @@ -249,9 +249,26 @@ describe("REPL route: resolving against one model", () => { `xmd://repl/${EXECUTION}/repl/entry-1/+elicit?at=${answered.marker}&inspect`, ), ).toContain("live question"); - expect(resolved(head, `xmd://repl/${EXECUTION}/repl/entry-1/+elicit`).drawers[0].kind).toBe( - "live-elicit", - ); + // No reading of any history can mount a live question's drawer. A waiting + // question is the one fact a Journal never holds: it records answers. So a + // settled history, and an unsettled one that has already recorded this + // answer and merely has its root left to close, are both consistent with no + // question ever arriving — and `settled` cannot tell them apart from one + // that is really asking. + const live = `xmd://repl/${EXECUTION}/repl/entry-1/+elicit`; + expect(head.settled).toBe(true); + expect(unresolved(head, live)).toContain("nothing is being asked"); + + const unclosed = projected(events.slice(0, -1)); + expect(unclosed.settled).toBe(false); + // Already answered: reopening this shape asks nobody anything and only + // finishes the root, so a route resolved here would replay and append + // before discovering that no drawer can mount. + expect(unclosed.entry?.elicitations[0].answer).toEqual({ decision: "approve" }); + expect(unresolved(unclosed, live)).toContain("nothing is being asked"); + + // Only the process actually holding the question may open it, and it says so. + expect(resolved(head, live, true).drawers[0].kind).toBe("live-elicit"); }); it("R2: refuses a missing entry, scope, binding or incomplete path", function* () { diff --git a/packages/cli/tests/repl-terminal.test.ts b/packages/cli/tests/repl-terminal.test.ts index 458cf4c5..63064bbb 100644 --- a/packages/cli/tests/repl-terminal.test.ts +++ b/packages/cli/tests/repl-terminal.test.ts @@ -853,6 +853,7 @@ function recordingTerminal(size: ReplTerminalSize = { columns: 160, rows: 36 }): }; const host: ReplTerminalCapabilities = { + interactive: () => true, size(): ReplTerminalSize { return log.size; }, diff --git a/specs/repl-spec.md b/specs/repl-spec.md new file mode 100644 index 00000000..80bf47e8 --- /dev/null +++ b/specs/repl-spec.md @@ -0,0 +1,141 @@ +# The XMD REPL + +`xmd repl` opens one XMD entry in a full-screen terminal, runs it for real, and +lets you look at what it did — while it is running and afterwards, in this +process or in another one. + +```bash +xmd repl # a fresh execution with an empty draft +xmd repl 'xmd://repl//repl' # reopen exactly that retained history +``` + +## One live journey + +Run `xmd repl`. The screen shows an empty draft, a Sessions list that says it is +empty, an Entries list with nothing in it yet, and a location of the form +`xmd://repl//repl` — the execution already exists, as an empty history +file, before you have typed anything. + +Type or paste one XMD entry and press Enter. What was in the draft is now the +entry, and the entry is immutable: this execution admits one, and typing after +that changes nothing. The run begins, and the screen fills in as it goes — the +scopes the entry admitted, the bindings its `eval` blocks published, the source a +generated fragment produced before that fragment was admitted, and each line the +document rendered. + +When the entry asks a question, its drawer opens: the message, the one field the +schema asks for, and exactly the values it will accept. Type one and press Enter. +The answer is recorded, and what the document renders after it changes because of +the recorded answer rather than because of anything this process remembered. +Escape closes the drawer without answering; the question stays open. An answer +the schema accepts ends the question, and the drawer goes with it: it leaves the +screen and it leaves the location, because a URL naming a drawer nothing mounts +describes a view nobody can be shown. + +Press Enter on `[pause]` to stop expansion at its next boundary. `[continue]` +exists only while a continuation is actually held — not while a pause is still +being taken — and releases the holds. Pausing cancels nothing: the execution is +suspended, not abandoned. Open the History drawer to select an +earlier position; the whole view freezes there and is read only. `[live]` returns +to the head. + +When the root settles, its recorded output is what the transcript shows, and the +screen stops asking for frames. + +## One cold journey + +The command prints the location it ended at. Pass that location to a new +`xmd repl` — on this machine, in a new process, with nothing carried over — and +the same view comes back: the same selected scope, the same binding values, the +same transcript, the same generated source, the same recorded question and answer, +the same History positions and the same terminal output. + +Nothing is re-run to do it. No component source is read, no `eval` block is +compiled, and nobody is asked anything: everything on that screen was +reconstructed from the location and the retained events. A location naming a +history this version cannot read refuses, with nothing appended and no execution +started. + +## What the screen does at each size + +| terminal | what it shows | +| --- | --- | +| `160x36` and larger | Sessions/Entries sidebar, transcript, bindings and recorded questions, the drawer layer, and a fixed full-width footer holding five History rows and the input | +| `120x30` and larger | the same, with a narrower sidebar and inspection column | +| `72x20` and larger | one routed surface — the one the route selected — with the same drawer and footer | +| smaller than `72x20` | a refusal saying the minimum, showing nothing else; it recovers when the window grows | + +At narrow, the surfaces the route did not select are still there in the model and +are not on the screen: they are in no cell, in no target map, and no pointer +reaches them. The History band is five rows at every size; when the labels of +several positions cannot all fit, they share a label and every position keeps its +own identity. + +A refusal of one action — a second entry, a navigation that selects nothing, a +pause this process cannot perform — appears in the footer beside the control that +was refused. It does not replace the screen. + +## One entry, and nothing else + +This product admits zero or one entry per execution. There is no second entry, no +catalog of past runs, no fork, no agent, no snapshot and no sidecar file. The +Sessions surface exists and says it is empty. + +## What is retained, and what is not + +Each execution is one file of serialized ordinary `DurableEvent`s — the same +records any other XMD run writes. There is no manifest, no cache, no materialized +model, no checkpoint file and no record type of the REPL's own. The file and the +location are the whole of the state, which is what makes reconstruction from them +possible at all. + +The location carries route state only: which surface, which scopes, which +drawers, which history position, and the draft before an entry exists. It never +carries focus, pause state, form internals, rendered cells or this process's +overlay. + +## History is immutable; the present is explicit + +A view frozen at a history position shows what the file held at that position and +nothing else. It fills nothing from the live head: no live output, no waiting +question, no pause capability, and it cannot open the question this process is +asking. Before the root closes, output this process has produced but the file has +not recorded appears as an explicit overlay; once the durable close exists, the +recorded output replaces it. + +## Who owns what + +Components receive immutable view data and answer with a typed semantic action. +They hold no events, no history, no session, no repository and no host operation, +so a keystroke cannot reach the file except through the one place that owns the +product's state. Focus belongs to the mounted tree: Tab and Backtab traverse it, +a drawer contains it while it is open, and the control that has it is marked. + +Normalized input is decided once, at the host, which names an event shape and a +position and never an action. Text is its own event and goes to whichever field +has focus; a Control or Alt chord is dropped whole rather than typed as the letter +it was pressed with. A pointer is resolved against the exact frame that produced +its coordinates, so a stale frame, a node that has gone and a target behind a +drawer all reach nothing — and activating a control with a pointer produces the +same action as pressing Enter on it. + +Presentation time has one owner: a single acknowledged frame stream, advanced only +once every subscriber has applied the previous timestamp. A settled screen +schedules no timer. + +## Exclusions + +- Two processes writing one execution is unsupported. There is no lease protocol; + do not open the same location twice for writing. +- `xmd repl` takes one optional location and no options. It renders no file and + takes no document reference. +- The REPL runs where a terminal is: it is not available over a pipe, and help + acquires no terminal or filesystem capability. Whether there is a terminal is + settled before a history exists, so a piped invocation leaves nothing behind. +- A location is resolved against the retained history before anything replays. A + route naming a scope or a drawer the file does not hold is refused without the + execution being started, so a typo cannot cause a record. +- A waiting question's drawer cannot be reopened from a URL. A Journal records + answers, never a question that is still waiting, so no history can establish + that drawer — it exists only while the process holding the question is running. + A location carrying one is refused before anything replays.