Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 41 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,17 @@
# pi-background-run

Run long shell commands (test suites, builds, linters) as detached background jobs
so your pi agent session stays unblocked and its context stays clean. Output lands
on disk — the full log plus a trailing exit marker — so nothing large ever enters
the conversation; the command returns immediately. When the job finishes,
pi-background-run **wakes the live agent session** so it proactively reads a
condensed digest of the results and continues — no polling, no human intervention.
Run genuinely asynchronous shell commands as detached background jobs so your pi
agent session stays unblocked and its context stays clean. Output lands on disk —
the full log plus a trailing exit marker — so nothing large enters the conversation;
the command returns immediately. Each job chooses whether completion wakes the
live agent always, only on failure, or never. Human toast and widget updates remain
enabled for every policy.

Built as a [pi](https://github.com/earendil-works/pi-coding-agent) extension. No
shell runner and no external daemon — the extension spawns the job in-process,
detects completion via the child `exit` event, and calls `pi.sendUserMessage` to wake
the agent. The log file is self-describing (full output + a trailing
detects completion via the child `exit` event, and conditionally calls
`pi.sendUserMessage` when the job's wake policy requests a model turn. The log
file is self-describing (full output + a trailing
`__BGRUN_EXIT__=N` marker), so exit codes survive pi restarting. Two small pieces
exist beyond the spawn: a 30s timer that only re-checks jobs whose live child handle
is gone (reconstructed from a restart, or adopted from another session), and a
Expand All @@ -33,12 +34,32 @@ Restart pi after install so the extension loads.

| Tool | Purpose |
| ------ | --------- |
| `bgrun` | Launch a command detached in the background. Optional `name` gives the job a short human-readable label. Returns `started: <job-id>` immediately. Wakes the session automatically on completion. |
| `bgrun` | Launch a command detached in the background. Optional `name` gives the job a short human-readable label. `wake` selects `never`, `failure`, or `always` model-turn delivery. Returns `started: <job-id>` immediately. |
| `bgstatus` | Show job status. With an id: any job's state + exit code. Without: this session's running jobs (finished jobs hidden by default — pass `includeDone: true` or set `showCompletedJobs`). Other sessions' *running* jobs are listed only when `adoptForeignJobs` is enabled; finished foreign logs from the shared dir can also appear when finished jobs are included. |
| `bgtail` | Read the newest lines of a job's log (default 40; it reads the log's **last 2 MB** — widen with `bytes`, max 64 MiB), **condensed for context**: ANSI escapes stripped, repeated lines collapsed, long lines and total size capped. First read = full last-N tail; repeat reads return **only lines appended since your last read** (delta tailing) — polling a running job never re-pays for lines already seen. Pass `raw: true` for the unprocessed last-N window (still advances the bookmark). |
| `bggrep` | Regex search over the **last 2 MB** of a job's log (`bytes` widens the window, max 64 MiB): line-numbered matches, optional `context` lines, each line pre-truncated to 10 000 chars before matching, results capped (~50 matches, ~8KB) and condensed. Resolves the job id to the configured jobs dir itself — no log path to reconstruct. `ctx_execute_file` can read the same file (it takes an absolute path; only your Read-deny rules apply), but it needs that path. Matching runs under a wall-clock budget ([Bounded matching](#bounded-matching)). With no `pattern`, a generic failure-signature default is used (override it — convenience, not guarantee). |
| `bgclean` | Remove old job logs. **Default scope: this session's jobs only** — other sessions' logs are untouched — and it also drops stale per-project digest markers (`.bgrun-used-*`, `.digest-nudge-*`) in the session's jobs dir (markers are not session data). Pass `all: true` to sweep every shared jobs dir — under the project-local default that is the project's dir plus the machine-global one, while an explicit absolute `jobsDir` is swept alone — and do the same marker sweep across them. Retention: `cleanupDays` config (7 days); `days` must be a positive number (`days: 0` is rejected rather than purging everything). Never removes a running job's log. |

## Completion wake policy

Every job accepts `wake: "never" | "failure" | "always"`:

- `never` keeps model context quiet; completion still updates the toast/widget and persists status/logs.
- `failure` wakes only for a non-zero exit or spawn failure.
- `always` preserves the original behavior and wakes on every completion.

Omitting `wake` uses `defaultWake` from layered configuration (`PI_BGRUN_WAKE`
overrides it). The package default remains `always` for backward compatibility;
users who want opt-in model turns can set `"defaultWake": "never"`. Per-job
policy is persisted in transcript entries and displayed by `bgstatus` after a
session reload.

Use `always` for deployment/eval monitors whose completion requires immediate
follow-up, `failure` for long checks whose successful completion needs no model
turn, and `never` for independent work. Foreground execution remains the default
for routine focused commands; do not choose background execution from command
category alone.

## Slash commands

Human-facing mirrors of the read/clean tools, usable directly in the TUI
Expand All @@ -64,7 +85,7 @@ wake messages) is the agent's workflow.
## How it works

```text
agent calls bgrun(command: "make test-short", name: "unit-tests")
agent calls bgrun(command: "gh run watch …", name: "deploy-monitor", wake: "always")
→ extension resolves log path: <jobsDir>/<slug>-<ts>-<pid>.log (default <project>/.pi-bgrun/jobs/ in a repo, else ~/.pi-bgrun/jobs/)
→ spawn('sh', ['-c', <wrapper>, 'bgrun', '<cmd>'],
{ stdio: ['ignore', logFd, logFd], detached: true }).unref()
Expand All @@ -76,9 +97,9 @@ agent calls bgrun(command: "make test-short", name: "unit-tests")

child 'exit' event fires:
→ extension records exit code, appends a done entry
pi.sendUserMessage(wake) when idle (triggers a turn)
or pi.sendUserMessage(wake, { deliverAs: 'followUp' }) when busy
→ ctx.ui.notify(...) — toast for the human
when the per-job wake policy matches the outcome, pi.sendUserMessage(wake)
triggers a turn when idle or queues a follow-up when busy
→ ctx.ui.notify(...) — toast for the human, regardless of wake policy
→ ctx.ui.setWidget("bgrun", ...) — updates/clears the live status widget
```

Expand Down Expand Up @@ -218,6 +239,7 @@ run locally (completed jobs visible, a scorecard on `bun test` runs).
{
"adoptForeignJobs": false,
"showCompletedJobs": false,
"defaultWake": "always",
"cleanupDays": 7,
"maxLogBytes": 67108864,
"globalAutoClean": true,
Expand Down Expand Up @@ -317,6 +339,7 @@ Environment variables (same knobs, handy for one-off overrides):
| `PI_BGRUN_GLOBAL_DIR` | `~/.pi-bgrun/jobs` | **Deprecated.** Overrides the machine-global jobs base — the fallback used only when the cwd has no project root (see [deprecation](#deprecated-machine-global-jobs-dir)). A leading `~` or `~/` is expanded to the home dir; `~user` is not. |
| `PI_BGRUN_FOREIGN_JOBS` | `false` | Adopt other sessions' running jobs into this session's widget and job list. Adopted jobs are polled so they leave the widget when they finish. |
| `PI_BGRUN_SHOW_COMPLETED` | `false` | Include finished jobs in `bgstatus` listings by default. |
| `PI_BGRUN_WAKE` | `always` | Default model-turn completion policy when a job omits `wake`: `never`, `failure`, or `always`. Per-job `wake` takes precedence. Toast/widget updates are unaffected. |
| `PI_BGRUN_CLEANUP_DAYS` | `7` | Log retention for cleanup sweeps and the `bgclean` default. |
| `PI_BGRUN_MAX_LOG_BYTES` | `67108864` (64 MiB) | Byte ceiling for a job's log (stdout+stderr). `0` disables it (unlimited). See [Log size ceiling](#log-size-ceiling). |
| `PI_BGRUN_GLOBAL_AUTO_CLEAN` | `true` | Set `0`/`false` to disable the automatic orphan sweep (see below). |
Expand All @@ -325,10 +348,11 @@ Environment variables (same knobs, handy for one-off overrides):

### Digest scorecard (opt-in)

Wake messages always lead with universal facts — exit code, duration, and the
command's own log line count (the internal exit marker is excluded). A project
can additionally opt into a **digest scorecard**: a one-line pass/fail summary
extracted from the log and appended to the wake.
When a job's wake policy requests a model turn, its message leads with universal
facts — exit code, duration, and the command's own log line count (the internal
exit marker is excluded). A project can additionally opt into a **digest
scorecard**: a one-line pass/fail summary extracted from the log and appended
to that wake.

#### Job identity: name, type, command

Expand Down
99 changes: 96 additions & 3 deletions extension/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -311,6 +311,27 @@ async function withJobsDir<T>(

// Default below Bun's 5s test timeout so a stuck wait rejects with a clear
// message instead of racing the harness kill (a flake-masking failure mode).
function waitForLogExit(
logPath: string,
timeoutMs = 4000,
): Promise<void> {
return new Promise((resolve, reject) => {
const start = Date.now();
const tick = () => {
try {
if (readFileSync(logPath, "utf8").includes("__BGRUN_EXIT__="))
return resolve();
} catch {
// The child may not have created/renamed the final log yet.
}
if (Date.now() - start > timeoutMs)
return reject(new Error(`timed out waiting for ${logPath} to finish`));
setTimeout(tick, 50);
};
tick();
});
}

function waitForWakes(
wakes: CapturedWake[],
count: number,
Expand Down Expand Up @@ -412,6 +433,60 @@ test("bgrun: successful command writes log + exit marker and wakes with ✅", as
});
});

test("bgrun: wake never keeps model context quiet while persisting completion", async () => {
await withJobsDir(async (dir, h) => {
const { wakes, entries, tools, ctx } = h;
const bgrun = tools.get("bgrun")!;
const res = await bgrun.execute(
"call-never",
{ command: "echo quiet-success", wake: "never" },
undefined,
undefined,
ctx,
);
const text = res.content[0].text as string;
const id = text.match(/^started: ([^\n]+)/)![1];
assert.match(text, /wake: never/);
assert.match(text, /without waking the agent/);
await waitForLogExit(join(dir, `${id}.log`));
await new Promise((resolve) => setTimeout(resolve, 50));
assert.equal(wakes.length, 0);
const records = entries.filter((entry) => entry.data?.id === id);
assert.equal(records.length, 2, "running + done entries persisted");
assert.ok(records.every((entry) => entry.data?.wake === "never"));
});
});

test("bgrun: wake failure ignores success and wakes on non-zero exit", async () => {
await withJobsDir(async (dir, h) => {
const { wakes, tools, ctx } = h;
const bgrun = tools.get("bgrun")!;
const success = await bgrun.execute(
"call-failure-success",
{ command: "echo pass", wake: "failure" },
undefined,
undefined,
ctx,
);
const successId = (success.content[0].text as string).match(
/^started: ([^\n]+)/,
)![1];
await waitForLogExit(join(dir, `${successId}.log`));
await new Promise((resolve) => setTimeout(resolve, 50));
assert.equal(wakes.length, 0);

await bgrun.execute(
"call-failure-error",
{ command: "echo failed; exit 9", wake: "failure" },
undefined,
undefined,
ctx,
);
await waitForWakes(wakes, 1);
assert.match(wakes[0].text, /exit 9/);
});
});

test("bgrun: failing command wakes with ❌ and the non-zero exit code", async () => {
await withJobsDir(async (_dir, h) => {
const { wakes, tools, ctx } = h;
Expand Down Expand Up @@ -5215,7 +5290,7 @@ test("wake digest: invalid type entry dropped, other entries still work", async
}
});

test("bgrun: type flows into the started result, entries, and resume reconstruction", async () => {
test("bgrun: type and wake policy survive entry persistence and reconstruction", async () => {
const dir = mkTmp("pi-bgrun-test-");
process.env.PI_BGRUN_DIR = dir;
try {
Expand All @@ -5224,7 +5299,12 @@ test("bgrun: type flows into the started result, entries, and resume reconstruct
const bgrun = tools.get("bgrun")!;
const res = await bgrun.execute(
"call-ty1",
{ command: "echo typed", name: "unit-tests", type: "Test" },
{
command: "echo typed; exit 1",
name: "unit-tests",
type: "Test",
wake: "failure",
},
undefined,
undefined,
ctx,
Expand All @@ -5235,12 +5315,15 @@ test("bgrun: type flows into the started result, entries, and resume reconstruct
assert.match(started, /^ {2}name: unit-tests$/m);
// Types are lowercase-normalized so selection is an exact compare.
assert.match(started, /^ {2}type: test$/m);
assert.match(started, /^ {2}wake: failure$/m);
assert.equal((res.details as any).type, "test");
assert.equal((res.details as any).wake, "failure");
await waitForWakes(wakes, 1);

// The persisted done entry carries the type.
const done = entries.filter((e) => e.customType === "bgrun-job").at(-1);
assert.equal(done?.data?.type, "test");
assert.equal(done?.data?.wake, "failure");

// Resume: a fresh instance reconstructs the in-memory map from entries.
const {
Expand All @@ -5266,6 +5349,8 @@ test("bgrun: type flows into the started result, entries, and resume reconstruct
"reconstructed record carries the type",
);
assert.equal((status.details as any).type, "test");
assert.match(text, /^ {2}wake: failure$/m);
assert.equal((status.details as any).wake, "failure");
} finally {
delete process.env.PI_BGRUN_DIR;
rmSync(dir, { recursive: true, force: true });
Expand Down Expand Up @@ -5668,21 +5753,29 @@ test("resolveConfig: env vars override config files", async () => {
const userCfg = join(userDir, "pi-bgrun.json");
const prevDir = process.env.PI_BGRUN_DIR;
const prevDays = process.env.PI_BGRUN_CLEANUP_DAYS;
const prevWake = process.env.PI_BGRUN_WAKE;
delete process.env.PI_BGRUN_DIR;
try {
writeFileSync(userCfg, JSON.stringify({ cleanupDays: 11 }));
writeFileSync(
userCfg,
JSON.stringify({ cleanupDays: 11, defaultWake: "failure" }),
);
process.env.PI_BGRUN_CLEANUP_DAYS = "3";
process.env.PI_BGRUN_WAKE = "never";

const cfg = mod.resolveConfig({
cwd: proj,
isProjectTrusted: () => true,
userConfigPath: userCfg,
});
assert.equal(cfg.cleanupDays, 3, "env beats both config files");
assert.equal(cfg.defaultWake, "never", "wake env beats config file");
} finally {
if (prevDir !== undefined) process.env.PI_BGRUN_DIR = prevDir;
if (prevDays === undefined) delete process.env.PI_BGRUN_CLEANUP_DAYS;
else process.env.PI_BGRUN_CLEANUP_DAYS = prevDays;
if (prevWake === undefined) delete process.env.PI_BGRUN_WAKE;
else process.env.PI_BGRUN_WAKE = prevWake;
rmSync(proj, { recursive: true, force: true });
rmSync(userDir, { recursive: true, force: true });
}
Expand Down
Loading