feat(history): tombstones, deletion, and privacy semantics (slice 5/6)

docs/prompt-history.md

@@ -1,13 +1,13 @@
 # Prompt history
 
-Prompt history stores captured prompts per pi instance and can import older
-history and project session transcripts. Deletion and compaction arrive in
-later slices of the chain.
+Prompt history stores captured prompts per pi instance, can import older
+history and project session transcripts, and lets you delete prompts from the
+history selector. Compaction arrives in a later slice of the chain.
 
 ## Capture is opt-in
 
-Recording is **off by default**. Delivered prompts can contain secrets, and the
-deletion UI is not shipped yet, so nothing is stored unless you explicitly opt in:
+Recording is **off by default**. Delivered prompts can contain secrets, so
+nothing is stored unless you explicitly opt in:
 
 ```bash
 GENTLE_PI_HISTORY_CAPTURE=1 pi
@@ -18,13 +18,15 @@ GENTLE_PI_HISTORY_CAPTURE=1 pi
 - The check runs per prompt: unsetting the switch (or setting it to `0`) stops
   new captures immediately, no pi restart needed.
 - With capture off the extension is inert: no registry entry, no files, and
-  prompts are never written.
+  prompts are never written. The history selector only warns; it reads,
+  imports, and deletes nothing.
 
 ## Legacy migration and seeding are opt-in
 
-Importing past prompts is part of capture: the first delivered prompt in an
-opted-in session attempts legacy migration and one-time bootstrap from project
-session transcripts. The selector reads the store but does not initiate import.
+Importing past prompts is part of capture: an opted-in session attempts legacy
+migration and one-time bootstrap from project session transcripts shortly
+after the extension loads, or at its first delivered prompt if that comes
+first. The selector reads the store but does not initiate import.
 With capture off, both capture and the selector leave the store untouched.
 Failed migration reads can be retried on a later session; untrusted deletion
 records defer transcript bootstrap until they can be read safely.
@@ -44,6 +46,7 @@ Everything sits under `~/.pi/agent/history/`:
   process.
 - `projects/<hash>/seed.jsonl` — one-time transcript import for this project.
 - `history-global.jsonl` — imported legacy editor-history prompts.
+- `hidden.json` — deletion records (tombstones); see "Delete" below.
 
 `<hash>` is the first 16 hex chars of the SHA-256 of the canonicalized project
 cwd; `<instance>` is a per-process UUID. Each line is one delivered prompt:
@@ -67,11 +70,80 @@ Treat the store as sensitive: it holds your prompts verbatim.
 ## What disabling capture does
 
 Turning the switch off only stops **new** captures. Nothing is deleted: files
-already written — and the registry entry — stay on disk until you remove them or
-the deletion UI ships. To erase the store manually while capture is off (or pi
-is not running):
+already written — and the registry entry — stay on disk until you remove them.
+Individual prompts can be deleted from the history selector while capture is
+on (see "Delete" below); the store directory itself is removed by hand:
 
 ```bash
 rm -rf ~/.pi/agent/history            # whole store
 rm -rf ~/.pi/agent/history/projects/<hash>   # one project (see registry.json)
 ```
+
+## Delete
+
+The selector's delete key (`ctrl+shift+backspace`) is a two-step y/n
+confirmation:
+
+1. The first press **arms** the delete for the selected row: the footer
+   shows "Delete this prompt from history (y/n)? Prompt stays in session
+   log" and the row highlights in red.
+2. While armed, the next key decides: `y` executes the delete, `n` or
+   `Esc` cancels, and any other key is ignored — nothing is typed into the
+   search box and the overlay stays open.
+
+A delete removes the prompt by its identity: whitespace runs collapsed,
+leading and trailing whitespace trimmed, letter case ignored. Only that exact
+prompt is affected — prompts that merely share a beginning stay.
+
+1. **Store copies are removed.** In the project scope, every copy in the
+   current project's files (`<instance>.jsonl` and `seed.jsonl`) is removed;
+   in the global scope, every copy in every project's files and in
+   `history-global.jsonl`. Each affected file is rewritten atomically (temp
+   file + rename). Files are never removed, even when they end up empty.
+   Lines that another pi instance appends while a file is being rewritten
+   are carried over into the new file.
+2. **A tombstone is written** to `hidden.json`, so the prompt stays hidden
+   everywhere the selector reads, and a later transcript bootstrap does not
+   import it again. The session transcripts themselves are never modified.
+
+Deletes only run from the selector, so they need capture enabled.
+
+### What `hidden.json` contains
+
+`hidden.json` is a JSON array of strings, oldest first. Each deletion adds
+`sha256:` followed by the SHA-256 hex digest of the normalized prompt, so the
+file does not hold the text of deleted prompts. Plain-text entries written by
+earlier builds (a prompt's first 120 normalized characters) are still read
+and keep their original meaning: an entry shorter than 120 characters hides
+that exact prompt, and a 120-character entry hides every prompt that begins
+with it. They are kept as they are, and new deletions never add them.
+
+A tombstone hides every copy of its prompt, including one you type again
+later: that prompt is captured, but it stays hidden while the tombstone
+exists.
+
+The file is a bounded cache, not a retention guarantee: it holds at most
+**1000 entries** in recency order, and deleting the same prompt again moves
+its entry to the end. Past the cap, the oldest entry is dropped. Its prompt
+can reappear if a copy is still on disk (for example, in a file that could
+not be rewritten), and can be deleted again.
+
+### Failures
+
+Failures surface an error notification and never report a clean delete:
+
+- If the store delete fails before touching any file, nothing is removed and
+  no tombstone is written ("Store delete failed; nothing was removed.").
+- If some files cannot be read or rewritten, the others are still cleaned,
+  temp files are removed, and the tombstone is still written ("Some history
+  files could not be rewritten; the prompt is hidden, but copies may remain
+  on disk.").
+- If the tombstone write fails after store copies were removed, the prompt
+  may reappear from session transcripts ("Deleted from the store, but
+  hiding failed — the prompt may reappear from session transcripts.").
+
+`hidden.json` fails closed: if it exists but cannot be trusted (unreadable,
+corrupt, or not an array), history is blocked with a recovery warning
+instead of resurfacing hidden prompts, transcript bootstrap waits, and
+deletes refuse to rewrite it. Recovery is explicit — restore the file or
+delete it yourself (hidden prompts may then reappear).

extensions/history/hide-prompts.ts

@@ -1,6 +1,7 @@
 // SPDX-FileCopyrightText: 2026 ExoPro. Inspired by @jasonish/pi-prompt-history
 // SPDX-License-Identifier: MIT
 
+import { createHash } from "node:crypto";
 import fs from "node:fs";
 import path from "node:path";
 import { writeJsonAtomic } from "./atomic-write.ts";
@@ -9,6 +10,50 @@ import { promptDedupKey } from "./selector-helpers.ts";
 /** Name of the tombstone file inside the injected state dir (spec C4). */
 const HIDE_FILE_NAME = "hidden.json";
 
+/** Marks tombstone entries written in the exact (hashed) key format. */
+const TOMBSTONE_KEY_PREFIX = "sha256:";
+
+/**
+ * UI-level prompt identity: whitespace-collapsed, trimmed, case-insensitive,
+ * never truncated. The store's scope deletes sweep by this same identity,
+ * so a tombstone hides exactly the copies a delete removes.
+ */
+export function promptIdentity(text: string): string {
+  return text.replace(/\s+/g, " ").trim().toLowerCase();
+}
+
+/**
+ * Tombstone key for `text`: a SHA-256 of its full prompt identity. Exact —
+ * two prompts that merely share a prefix get different keys — and hashed,
+ * so hidden.json never holds the text of a deleted prompt.
+ */
+export function tombstoneKey(text: string): string {
+  return (
+    TOMBSTONE_KEY_PREFIX +
+    createHash("sha256").update(promptIdentity(text)).digest("hex")
+  );
+}
+
+/**
+ * Whether `text` is hidden by the tombstone set `keys`. Hashed keys match
+ * the exact prompt identity. Plaintext entries from the earlier prefix
+ * format (`promptDedupKey`: the first 120 normalized characters) stay
+ * honored as written, so upgrading never resurfaces a hidden prompt; only
+ * new deletions use the exact format.
+ */
+export function isPromptHidden(keys: ReadonlySet<string>, text: string): boolean {
+  if (keys.size === 0) return false;
+  return keys.has(tombstoneKey(text)) || keys.has(promptDedupKey(text));
+}
+
+/**
+ * Retention cap for hidden.json (slice-05 D5): the tombstone file is a
+ * rebuildable derived cache, not a retention guarantee, so it holds at
+ * most this many keys in recency order; hiding past the cap drops the
+ * OLDEST keys from the front.
+ */
+export const HIDE_FILE_MAX_ENTRIES = 1000;
+
 /**
  * Shared recovery warning for a file that exists but cannot be trusted
  * (spec C4, fail-closed READ half): toast-suitable, names hidden.json, and
@@ -48,8 +93,11 @@ export type HiddenRead =
  * with the recovery warning so callers block the drain; it never degrades
  * to an empty trusted set. A MISSING file — before any deletion — is the
  * safe empty case and reads `trusted` with no keys. A valid array is
- * trusted; junk items inside it are ignored, never trusted. Keys are
- * `promptDedupKey` strings written by `hidePrompt`; the call never throws.
+ * trusted; junk items inside it are ignored, never trusted. Keys are the
+ * `tombstoneKey` hashes written by `hidePrompt`, or plaintext entries from
+ * the earlier prefix format (see `isPromptHidden`); the call never throws.
+ * A valid array's stored order is preserved (the recency order — oldest
+ * first — that `hidePrompt` maintains and caps).
  */
 export function readHiddenPrompts(stateDir: string): HiddenRead {
   let raw: string;
@@ -90,25 +138,41 @@ export function readHiddenPrompts(stateDir: string): HiddenRead {
 /**
  * Write the tombstone key for `text` into `stateDir/hidden.json` — the
  * WRITE half of the hide-file contract (spec C4). The key is the shared
- * `promptDedupKey` (byte-match normative with the merge filter — never a
- * re-implementation); the set compacts on write and persists as a SORTED
- * array via the shared atomic tmp+rename writer. An untrusted existing file
- * is never silently reset (a clean rewrite would clear the blocked state
- * one hide later): hidePrompt refuses with the recovery warning until the
- * user restores or deletes the file. A missing file is the clean baseline;
- * any write failure returns an error object for the delete-flow toast; the
- * call never throws.
+ * `tombstoneKey` (byte-match normative with the drain and seed filters via
+ * `isPromptHidden` — never a re-implementation). The file array is RECENCY-ordered — oldest key
+ * first, newest key appended last — and re-hiding an existing key
+ * refreshes it to the end (delete + add, since Set.add on a present
+ * member keeps its old position). The file is capped at
+ * `HIDE_FILE_MAX_ENTRIES` (1000): after the append, keys drop from the
+ * FRONT until the file fits, so hidden.json stays a bounded cache — a
+ * dropped (oldest) prompt may reappear in the list and can be deleted
+ * again. Keys persist in that insertion order — NO sort — via the shared
+ * atomic tmp+rename writer. An untrusted existing file is never silently
+ * reset (a clean rewrite would clear the blocked state one hide later):
+ * hidePrompt refuses with the recovery warning until the user restores or
+ * deletes the file. A missing file is the clean baseline; any write
+ * failure returns an error object for the delete-flow toast; the call
+ * never throws.
  */
 export function hidePrompt(stateDir: string, text: string): HideResult {
   const read = readHiddenPrompts(stateDir);
   if (read.status === "untrusted") {
     // Refuse without writing: never reset the untrusted state silently.
     return { status: "error", message: read.message };
   }
-  read.keys.add(promptDedupKey(text));
+  // Recency order (slice-05 D5): the set iterates in stored file order
+  // (oldest first); delete+add refreshes a re-hidden key to the END.
+  const key = tombstoneKey(text);
+  read.keys.delete(key);
+  read.keys.add(key);
+  // Cap: drop the OLDEST keys from the front once over the limit.
+  const ordered = [...read.keys];
+  if (ordered.length > HIDE_FILE_MAX_ENTRIES) {
+    ordered.splice(0, ordered.length - HIDE_FILE_MAX_ENTRIES);
+  }
   const written = writeJsonAtomic(
     path.join(stateDir, HIDE_FILE_NAME),
-    [...read.keys].sort(),
+    ordered,
   );
   return written
     ? { status: "written" }