Repository navigation
feat(history): per-instance JSONL store, project identity, and storage tests (slice 1/6) #1390
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
9b56e0b
2b90751
84c1232
31e7d50
892da55
6e27038
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # Prompt history | ||
|
|
||
| Slice 1 of the prompt-history extension (#819 split) ships the storage layer only: | ||
| a per-instance JSONL capture store, project identity, and the read/write | ||
| primitives later slices build on. The selector UI, deletion/scope drains, and GC | ||
| arrive in later slices of the chain. | ||
|
|
||
| ## Capture is opt-in | ||
|
|
||
| Recording is **off by default**. Delivered prompts can contain secrets, and the | ||
| deletion UI is not shipped yet, so nothing is stored unless you explicitly opt in: | ||
|
|
||
| ```bash | ||
| GENTLE_PI_HISTORY_CAPTURE=1 pi | ||
| ``` | ||
|
|
||
| - Enabled by `1`, `true`, or `on` (case-insensitive). Unset, empty, or any other | ||
| value means **off** — the same switch is the disable path. | ||
| - The check runs per prompt: unsetting the switch (or setting it to `0`) stops | ||
| new captures immediately, no pi restart needed. | ||
| - With capture off the extension is inert: no registry entry, no files, and | ||
| prompts are never written. | ||
|
|
||
| ## Where the files live | ||
|
|
||
| Everything sits under `~/.pi/agent/history/`: | ||
|
|
||
| - `registry.json` — advisory map of project hash → cwd, used for display | ||
| labels. | ||
| - `projects/<hash>/<instance>.jsonl` — one append-only capture file per pi | ||
| process. | ||
|
|
||
| `<hash>` is the first 16 hex chars of the SHA-256 of the canonicalized project | ||
| cwd; `<instance>` is a per-process UUID. Each line is one delivered prompt: | ||
|
|
||
| ```json | ||
| {"v":1,"text":"the prompt as delivered","ts":1700000000000} | ||
| ``` | ||
|
|
||
| UI command-like prompts (`/name ...`) and empty lines are never stored. Later | ||
| slices add the rebuildable `seed.jsonl`, scope drains/deletes, and GC. | ||
|
|
||
| ## Who can read them | ||
|
|
||
| The store is plain JSONL on your local disk, not encrypted. Files are created by | ||
| the pi process with default umask permissions (typically `0644` files inside | ||
| `0755` directories), so any process running as your OS user can read them, and | ||
| other local accounts can too wherever they can traverse your home directory. | ||
| Treat the store as sensitive: it holds your prompts verbatim. | ||
|
|
||
| ## What disabling capture does | ||
|
|
||
| Turning the switch off only stops **new** captures. Nothing is deleted: files | ||
| already written — and the registry entry — stay on disk until you remove them or | ||
| the deletion UI ships. To erase the store manually while capture is off (or pi | ||
| is not running): | ||
|
|
||
| ```bash | ||
| rm -rf ~/.pi/agent/history # whole store | ||
| rm -rf ~/.pi/agent/history/projects/<hash> # one project (see registry.json) | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win Make the single-project removal command executable. If a user copies this command literally, Bash interprets 🤖 Prompt for AI Agents |
||
| ``` | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| import path from "node:path"; | ||
| // SPDX-FileCopyrightText: 2026 ExoPro. Inspired by @jasonish/pi-prompt-history | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| import fs from "node:fs"; | ||
|
|
||
| /** | ||
| * Shared atomic JSON writer (design §D3): serialize to a `.tmp` file in the | ||
| * SAME directory as the target, then renameSync it into place — a same-dir | ||
| * rename is atomic on POSIX/APFS, so readers see the old or the new file, | ||
| * never a partial write. Any error returns false and never throws. | ||
| * | ||
| * No fsync: both consumers treat lost writes as derived cache (a lost index | ||
| * rebuilds on the next open; a lost tombstone resurfaces a prompt the user | ||
| * can re-delete), so the per-write fsync cost is not justified — the crash | ||
| * window is documented, not fixed. The staging name is unique per write | ||
| * (`.tmp-<pid>-<ts>`, the same convention as the store.ts writers): the | ||
| * state dir is shared across concurrent pi instances, so a fixed | ||
| * `${filePath}.tmp` would let two writers clobber the same staging file | ||
| * (torn target JSON, spurious rename failures). A failed write unlinks its | ||
| * staging file, so orphaned `.tmp` files do not accumulate. | ||
| */ | ||
| export function writeJsonAtomic(filePath: string, value: unknown): boolean { | ||
| const tmpPath = `${filePath}.tmp-${process.pid}-${Date.now()}`; | ||
| try { | ||
| fs.mkdirSync(path.dirname(filePath), { recursive: true }); | ||
| fs.writeFileSync(tmpPath, JSON.stringify(value), "utf8"); | ||
| fs.renameSync(tmpPath, filePath); | ||
| return true; | ||
| } catch { | ||
| try { | ||
| fs.unlinkSync(tmpPath); | ||
| } catch { | ||
| // staging file never created or already renamed | ||
| } | ||
| return false; | ||
| } | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,89 @@ | ||
| // SPDX-FileCopyrightText: 2026 ExoPro. Inspired by @jasonish/pi-prompt-history | ||
| // SPDX-License-Identifier: MIT | ||
|
|
||
| // Prompt-history extension entry (slice 1): identity constants, the | ||
| // per-instance writer lifecycle, and the before_agent_start capture | ||
| // handler. Selector UI, shortcut/command, scope drains, legacy migration | ||
| // and seed bootstrap, and GC arrive in later slices. | ||
| // | ||
| // Capture is OPT-IN while the deletion/privacy behavior is unshipped: | ||
| // nothing is recorded unless GENTLE_PI_HISTORY_CAPTURE=1|true|on. With the | ||
| // switch off the handler is a no-op — no registry entry, no files, and | ||
| // prompts are never written. Unsetting the switch only stops NEW captures; | ||
| // files already written stay on disk (docs/prompt-history.md). | ||
|
|
||
| import { randomUUID } from "node:crypto"; | ||
| import { homedir } from "node:os"; | ||
| import { join } from "node:path"; | ||
| import type { ExtensionAPI } from "@earendil-works/pi-coding-agent"; | ||
| import { | ||
| appendSessionCapture, | ||
| ensureRegistryEntry, | ||
| openSessionWriter, | ||
| type SessionWriterState, | ||
| } from "./store.ts"; | ||
|
|
||
| // v2 multi-concurrency store root (design: tmp/multi-concurrency-design.md). | ||
| const PI_HISTORY_ROOT = join(homedir(), ".pi", "agent", "history"); | ||
|
|
||
| export interface HistoryDeps { | ||
| env?: NodeJS.ProcessEnv; | ||
| root?: string; | ||
| cwd?: string; | ||
| instanceId?: string; | ||
| now?: () => number; | ||
| } | ||
|
|
||
| /** | ||
| * Strict opt-in: capture stays off unless GENTLE_PI_HISTORY_CAPTURE is | ||
| * explicitly 1, true, or on (case-insensitive). The same switch is the | ||
| * disable path — unsetting it stops new captures; files already on disk | ||
| * are left untouched until the deletion tooling lands. | ||
| */ | ||
| export function captureEnabled(env: NodeJS.ProcessEnv = process.env): boolean { | ||
| const value = env.GENTLE_PI_HISTORY_CAPTURE?.trim().toLowerCase(); | ||
| return value === "1" || value === "true" || value === "on"; | ||
| } | ||
|
|
||
| export default function promptHistoryExtension( | ||
| pi: ExtensionAPI, | ||
| deps: HistoryDeps = {}, | ||
| ): void { | ||
| const env = deps.env ?? process.env; | ||
| const root = deps.root ?? PI_HISTORY_ROOT; | ||
| const cwd = deps.cwd ?? process.cwd(); | ||
| const instanceId = deps.instanceId ?? randomUUID(); | ||
| const now = deps.now ?? Date.now; | ||
| let writerState: SessionWriterState | null = null; | ||
|
|
||
| /** | ||
| * One-time init per extension load: register the project in the advisory | ||
| * registry, then open this instance's exclusive capture file. Legacy | ||
| * migration and seed bootstrap join this init order in a later slice. | ||
| */ | ||
| const getWriter = (): SessionWriterState => { | ||
| if (!writerState) { | ||
| try { | ||
| ensureRegistryEntry(root, cwd); | ||
| } catch { | ||
| // registry is advisory | ||
| } | ||
| writerState = openSessionWriter(root, cwd, instanceId); | ||
| } | ||
| return writerState; | ||
| }; | ||
|
|
||
| // Persist every delivered user prompt (write-through, append-only JSONL), | ||
| // but only for opted-in sessions — see captureEnabled(). The local | ||
| // ExtensionAPI stub types handler args as unknown; narrow here. | ||
| pi.on("before_agent_start", (...args: unknown[]) => { | ||
| if (!captureEnabled(env)) return; | ||
| try { | ||
| const event = args[0] as { prompt?: string } | undefined; | ||
| appendSessionCapture(getWriter(), event?.prompt ?? "", now()); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🔒 Security & Privacy | 🛡️ Detected with Advanced Tier | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- changed files ---'
git diff --stat 777238324bd729a5a9c5817fe00373002ef666a4 84c1232361ca221e4b77b7aa8d2d0ebb6ddf3097 -- extensions/history
printf '%s\n' '--- history store definitions and permission-related references ---'
rg -n -C 3 'PI_HISTORY_ROOT|history|mkdirSync|appendFileSync|chmod|mode|permission|0o7|0o6|umask' extensions/history docs tests --glob '*.ts' --glob '*.md' | head -240
printf '%s\n' '--- changed store/index hunks ---'
git diff --unified=35 777238324bd729a5a9c5817fe00373002ef666a4 84c1232361ca221e4b77b7aa8d2d0ebb6ddf3097 -- extensions/history/store.ts extensions/history/index.ts | head -320Repository: Gentleman-Programming/gentle-shell Length of output: 30677 🏁 Script executed: #!/bin/bash
set -eu
printf '%s\n' '--- path and permission references ---'
rg -n -C 3 'PI_HISTORY_ROOT|mkdirSync|appendFileSync|chmod|mode|permission|umask' extensions/history docs tests --glob '*.ts' --glob '*.md' | head -240
printf '%s\n' '--- changed history hunks ---'
git diff --unified=25 777238324bd729a5a9c5817fe00373002ef666a4 84c1232361ca221e4b77b7aa8d2d0ebb6ddf3097 -- extensions/history/store.ts extensions/history/index.ts | head -320Repository: Gentleman-Programming/gentle-shell Length of output: 31314 Sensitive Data Exposure Reachability: External Create prompt-history files with restrictive permissions. The documented store uses default umask permissions, typically Set restrictive creation modes- fs.mkdirSync(path.dirname(state.filePath), { recursive: true });
- fs.appendFileSync(state.filePath, serializeEntry(entry) + "\n", "utf8");
+ fs.mkdirSync(path.dirname(state.filePath), { recursive: true, mode: 0o700 });
+ fs.appendFileSync(state.filePath, serializeEntry(entry) + "\n", {
+ encoding: "utf8",
+ mode: 0o600,
+ });🤖 Prompt for AI AgentsSource: Learnings |
||
| } catch { | ||
| // A capture failure must never break the agent loop or unregister | ||
| // the handler - swallow and keep the next prompt capturable. | ||
| } | ||
| }); | ||
| } | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🎯 Functional Correctness | 🟠 Major | ⚡ Quick win
Correct the no-restart disable instruction.
If a user launches
GENTLE_PI_HISTORY_CAPTURE=1 pi, unsetting that variable in a shell does not change the running Pi process’s environment.captureEnabledtherefore remains true, and later prompts can still be stored. The test changes an injected object inside the process; it does not test the documented action. Document that disabling capture requires a restart, or provide an in-process disable control. (gnu.org)🤖 Prompt for AI Agents