diff --git a/PERF.md b/PERF.md index 2984ed7b3f..c17ee82145 100644 --- a/PERF.md +++ b/PERF.md @@ -1232,6 +1232,32 @@ only omit what the reader is already holding. On the inherited-brief road the address rides the prerequisite's HEADER, which the shared pot above does not clip. Pinned by `internal/session/task_result_e2e_test.go`. +## A background job's disk spool is bounded + +`internal/session/jobs.go` used to spool everything a background job wrote to +one `.log` with no ceiling: a watcher printing for a week filled the disk +at whatever rate it printed, and the in-memory ring appended one Write before +trimming, so a single multi-megabyte Write grew a temporary to match. + +The spool is now a window of at most **jobSpoolChunks (2) chunks of +jobSpoolChunkBytes (4MB)** — `.log` live and `.log.1` kept — rotated by +copying a full chunk once at its boundary, then truncating and seeking the +same live inode. The previous backup is removed before copying, keeping even +transient usage within two chunks. No job ID or writer lock is released during +rotation. One huge Write spools in chunk-sized pieces and hands +only its newest **64KB** (`jobRingBytes`) to the ring. The retained output +stays addressable by the read tool exactly as before, so no limit grows for +the reader. + +The honesty is the point, and it is pinned: a spool that has discarded +anything — or a spool write, short write or close that failed — sets the +sink's notice, and every footer a model reads stops saying `full log:` and +names the truncation or the failure beside the file instead +(`TestJobFooterNamesTruncationInsteadOfFullLog`); rotation, the discard, the +huge-Write tail and the injected failure are pinned by +`internal/session/jobspool_test.go`. The bound is a fact about the code, not +about the box: the window is fixed bytes per job, not a disk-filling rate. + ## Specialist tool discovery Chat starts with core tools and one local `load_capability` registry operation @@ -2615,3 +2641,73 @@ Rendering selects a phrase by elapsed ten-second interval and samples the existi decoding ripple with 240 ms letter steps and a 1.8-second pause per pass. The 28-column caption and nine-column mark have fixed widths. This uses the existing clock and one foreground span; it adds no timer, I/O, model call or per-frame randomness. + +## Completed job log retention + +`internal/session/jobretention.go` limits eligible completed managed spools in +one jobs directory to **128 MiB and 64 job groups**, counting the base and +rotation together. Active spools have separate per-job limits; unmarked legacy +logs and unsafe files remain outside the budget because older writers may not +hold leases. This is not a machine-wide bound. Startup retains the existing +seven-day (`sweepTTL`) expiry for eligible inactive groups only: both chunks +must be older than the cutoff. Expired groups are removed before applying the +byte/count budget to fresh groups; metadata and active/legacy logs do not expire. + +Maintenance runs at log creation, sink close, and the existing startup sweep, +never per output write. It +lists one jobs directory, sorts candidates by allocated ID, and takes +nonblocking independent file leases; a deletion holds its lease through unlink. +Directory locks serialize allocation and maintenance across processes. Counter +and ownership metadata reads are capped at 256 bytes. ID allocation persists a +high-water value before cleanup and keeps the writer lease before publishing +the marker, preventing both reused IDs and newborn-log eviction. The stable +lock file remembers initialization if a counter later disappears. + +A claim with damaged metadata fails explicitly. Cleanup failures remain +retryable and are surfaced in the sink notice. Existing journals, worktrees, +legacy logs, and unrelated directories are not retention candidates. Tests use +explicit byte/count budgets with small payloads, avoiding mutable global limits. + +The startup TTL sweeper delegates `logs/jobs/` to this retention instead of +expiring the stable allocation metadata or ownership markers by age. +## Runtime-safe file search + +The structured `grep` tool excludes known codeaf runtime output for both engines, +including custom state homes and searches starting inside those directories. +The policy preserves source under `work/`, `trees/`, and ordinary user `logs/` +directories. Ripgrep receives exclusions after user globs and runs without user +config or symlink traversal. Shell commands do not inherit these protections. + +### Foreground bash snapshots + +Foreground bash retains at most **8 MiB** of initial output per spill under the +state home's `logs/bash/`, separately from its bounded latest-result tail. +Completed snapshots share **128 MiB / 64 files** and expire after **seven days**; +retention runs at creation and close. Active writers hold independent leases +and are outside the completed-file budget. A directory lock serializes cleanup +and publication. Unknown and unsafe linked files, and old `pi-bash-*.log` +temporary files, remain outside retention. These are per-directory bounds. + +Write and close failures stop spooling, not draining. Incomplete output never +claims to be full. Promotion closes the foreground snapshot and sends later +bytes only to the job sink. Structured recursive search excludes new snapshot +directories and legacy pi-bash files in the current temporary directory. + +`internal/exec/bare/bash_spill_test.go` covers the byte bound, initial-prefix +preservation, real foreground shell output, disk failures, promotion, retention +budgets, expiry, cross-process active leases, and linked-path refusal. + +Recursive search skips files above **8 MiB**. The walking engine and explicit +single-file inspection read a snapshot bounded by `min(size-at-open, 8 MiB)`; +one **64 KiB** line buffer drains and skips oversized lines, with an incomplete +result notice. Stored matches are clipped to the existing 500-byte display cap, +with at most **1000 matches** and **20 context lines per side**. Context rendering +streams the same bounded snapshot reader and stops accumulating output once the +result byte budget has been reached. Directory traversal and total files searched +remain governed by the caller's context, rather than a machine-wide byte cap. + +Ripgrep JSON records are limited to **1 MiB**; scanner errors and match limits +kill and reap the child so `Wait` cannot hang behind a full stdout pipe. Stderr +capture retains at most **4 KiB** while continuing to drain. Regression fixtures +cover both engines, direct runtime roots, aliases, broad globs, subprocess +termination, source worktrees, and growth during a snapshot read. diff --git a/cmd/codeaf/brief_test.go b/cmd/codeaf/brief_test.go index 5394ec5bfe..f3586e2f5a 100644 --- a/cmd/codeaf/brief_test.go +++ b/cmd/codeaf/brief_test.go @@ -140,3 +140,13 @@ func TestABriefMayArriveOnStandardInput(t *testing.T) { t.Fatalf("brief = %q, want the piped text", text) } } + +func TestReadTextRejectsAnAllBlankArgumentList(t *testing.T) { + want := noGoalGiven("do").Error() + for _, args := range [][]string{{""}, {" "}, {"", "\t"}} { + text, err := readText("do", args) + if text != "" || err == nil || err.Error() != want { + t.Errorf("readText(%q) = %q, %v; want noGoalGiven", args, text, err) + } + } +} diff --git a/cmd/codeaf/carried.go b/cmd/codeaf/carried.go index 90ed2e2e47..1bf0a7c56f 100644 --- a/cmd/codeaf/carried.go +++ b/cmd/codeaf/carried.go @@ -317,12 +317,13 @@ func runCarriedHost(ctx context.Context, inv *delegate.Invocation) error { // name it has, so its rows are filed as one piece of work under it. subject := filepath.Base(record) api, err := modelapi.Open(modelapi.Config{ - TaskDir: record, - CompleterFor: road.completerFor, - Serves: road.serves, - ModelPrice: road.modelPrice, - Seat: road.seat, - Ceiling: inv.Ceilings.CostUSD, + TaskDir: record, + CompleterFor: road.completerFor, + AuthKeySource: config.APIKeySourceAt(config.ProfileDir()), + Serves: road.serves, + ModelPrice: road.modelPrice, + Seat: road.seat, + Ceiling: inv.Ceilings.CostUSD, Bank: func(charge modelapi.Charge) { // THE MACHINE'S SPENDING LEDGER, one row per call, written here and // nowhere else: nothing else in this process meters these calls. diff --git a/cmd/codeaf/chatv3_host.go b/cmd/codeaf/chatv3_host.go index 194255e9c7..06ca2a6a74 100644 --- a/cmd/codeaf/chatv3_host.go +++ b/cmd/codeaf/chatv3_host.go @@ -200,7 +200,7 @@ func (l *engineLink) spawn() (io.ReadWriteCloser, error) { // is how a passphrase prompt and a host-key question reach the person — and // the tail is kept so that a handshake failure can name the likely cause. tail := &tailWriter{} - process.Stderr = io.MultiWriter(os.Stderr, tail) + process.Stderr = l.stderrWriter(tail, os.Stderr) if err := process.Start(); err != nil { if strings.Contains(err.Error(), "executable file not found") { return nil, fmt.Errorf("this machine has no ssh on its path, and --host is ssh") @@ -211,6 +211,19 @@ func (l *engineLink) spawn() (io.ReadWriteCloser, error) { return pipePair{r: stdout, w: stdin}, nil } +// stderrWriter keeps the launch-time prompts visible but keeps redial output +// inside the session's diagnostic tail. A reconnect happens while Bubble Tea +// owns the terminal's alternate screen, so writing ssh's transient errors to +// os.Stderr would paint over the frame instead of becoming a status detail. +func (l *engineLink) stderrWriter(tail *tailWriter, terminal io.Writer) io.Writer { + l.mu.Lock() + defer l.mu.Unlock() + if l.process == nil { + return io.MultiWriter(terminal, tail) + } + return tail +} + // sshTransportArgs keeps the carrier's latency policy in one place. -T remains // first because a pseudo-terminal changes bytes; the other options keep a warm // connection available for redials, notice a machine that stopped answering, @@ -250,15 +263,22 @@ func sshControlPath() string { return "" } path := filepath.Join(dir, "ctl-%C") - // OpenSSH expands %C to a 40-character SHA-1 digest before bind(2), so the - // expanded path is the one that must fit the shared macOS/Linux ceiling. - expanded := strings.Replace(path, "%C", strings.Repeat("0", 40), 1) - if !enginehost.SocketPathFits(expanded) { + if !sshControlPathFits(path) { return "" } return path } +// sshControlPathFits accounts for OpenSSH's temporary control-master name as +// well as the final hashed path, so a path accepted here cannot fail at bind. +func sshControlPathFits(path string) bool { + // OpenSSH expands %C to a 40-character SHA-1 digest and briefly appends a + // 17-character suffix before bind(2), so both forms must fit the shared + // macOS/Linux ceiling. + expanded := strings.Replace(path, "%C", strings.Repeat("0", 40), 1) + return enginehost.SocketPathFits(expanded) && enginehost.SocketPathFits(expanded+strings.Repeat("0", 17)) +} + // hold takes the new child and lets go of the old one. THE PREVIOUS SSH IS // REAPED IN THE BACKGROUND, because a redial happens after its pipe died and a // child nobody waits on is a zombie for as long as this terminal is open — five @@ -647,7 +667,7 @@ func hostOptions(fleet *engineFleet, welcome remote.Welcome, pick bool) (tui3.Op world.prime() ledger := newHostLedger(far) ledger.prime() - memory := newHostMemory(far) + memory := newHostMemory(far, welcome.Memory) memory.prime() // The counting gate is here and not on the roads: a session this window @@ -1557,11 +1577,12 @@ type hostMemory struct { mu sync.Mutex shelves store.MemoryShelves learned, letGo int + enabled bool known bool } -func newHostMemory(far hostFar) *hostMemory { - h := &hostMemory{client: far.client} +func newHostMemory(far hostFar, enabled bool) *hostMemory { + h := &hostMemory{client: far.client, enabled: enabled, known: true} far.arm(&h.duty, "reading what is remembered") return h } @@ -1618,3 +1639,13 @@ func (h *hostMemory) RestoreMemory(id string) error { func (h *hostMemory) MemoryProvenance(id string) (string, string, time.Time, error) { return h.client.MemoryProvenance(id) } +func (h *hostMemory) Remembers() bool { return h.enabled } +func (h *hostMemory) Remember(text string) (string, error) { + return h.client.Remember(text) +} +func (h *hostMemory) Forget(query string) (string, error) { + return h.client.ForgetQuery(query) +} +func (h *hostMemory) Memories(query string) ([]session.MemoryLine, error) { + return h.client.Memories(query) +} diff --git a/cmd/codeaf/chatv3_host_test.go b/cmd/codeaf/chatv3_host_test.go index 152a66626a..d16f486fba 100644 --- a/cmd/codeaf/chatv3_host_test.go +++ b/cmd/codeaf/chatv3_host_test.go @@ -1,9 +1,11 @@ package main import ( + "bytes" "context" "errors" "os" + "os/exec" "path/filepath" "strings" "testing" @@ -20,6 +22,30 @@ import ( // of a remote session. var _ tui3.Agent = (*remote.Agent)(nil) +func TestRedialSSHStderrStaysOutOfTheTerminal(t *testing.T) { + link := &engineLink{} + terminal := new(bytes.Buffer) + first := &tailWriter{} + if _, err := link.stderrWriter(first, terminal).Write([]byte("host-key prompt\n")); err != nil { + t.Fatal(err) + } + if terminal.String() != "host-key prompt\n" { + t.Fatalf("the first ssh prompt was not shown: %q", terminal.String()) + } + + link.process = &exec.Cmd{} + redial := &tailWriter{} + if _, err := link.stderrWriter(redial, terminal).Write([]byte("connection refused\n")); err != nil { + t.Fatal(err) + } + if terminal.String() != "host-key prompt\n" { + t.Fatalf("redial stderr painted the terminal: %q", terminal.String()) + } + if got := redial.String(); got != "connection refused\n" { + t.Fatalf("redial stderr was not retained: %q", got) + } +} + func TestParseHostTarget(t *testing.T) { cases := []struct { raw string diff --git a/cmd/codeaf/do.go b/cmd/codeaf/do.go index 407037eb3d..1ded4b0396 100644 --- a/cmd/codeaf/do.go +++ b/cmd/codeaf/do.go @@ -252,10 +252,11 @@ type headlessOutcome struct { // callers must never parse the bounded person's account, and it is never // clipped. Checklist []revision.PointOutcome `json:"checklist,omitempty"` - // Error is the sentence a run that never reached an outcome left behind: + // Error is the sentence a run that could not finish left behind: // the store that would not open, the working directory that could not be // made, the resident that never picked the command up, a journal read that - // failed mid-flight. It is empty on every run that produced an answer. + // failed mid-flight, or a worker's reason it could not continue. It is + // empty on every run that produced an answer. // // It exists because --json's whole promise is one object on stdout, and a // promise that only holds when the work succeeds is not one a script can be @@ -3403,6 +3404,9 @@ func withoutSummaryFileList(deliverable string, artifacts []string) string { // most common thing anyone does with a one-shot is pipe it somewhere. func reportErrand(request doRequest, outcome headlessOutcome) error { sayBlocked(request.stderr, outcome) + if !request.asJSON && request.stderr != nil && strings.TrimSpace(outcome.Error) != "" { + fmt.Fprintln(request.stderr, "error: "+outcome.Error) + } if outcome.recordKept != "" && request.stderr != nil { fmt.Fprintf(request.stderr, "record kept at %s\n", outcome.recordKept) } @@ -3691,6 +3695,9 @@ func runErrand(request doRequest, seats config.Seats) (outcome headlessOutcome, defer stopSignals() ctx, cancel := context.WithTimeout(signalled, request.timeout) defer cancel() + if spendPreauthorized(request.yesSpend, env.Value) { + ctx = session.WithDailySpendPreauthorized(ctx) + } // THE MACHINE'S HOLD IS SAID, because nothing else here would say it: the // chat draws `waiting · machine busy` on the run's rail, and a headless run @@ -3719,6 +3726,8 @@ func runErrand(request doRequest, seats config.Seats) (outcome headlessOutcome, Nodes: summary.Nodes, Seconds: summary.Seconds, Spend: summary.USD, + tokensIn: summary.TokensIn, + tokensOut: summary.TokensOut, } switch summary.Outcome { case runengine.OutcomeDone: @@ -3735,6 +3744,9 @@ func runErrand(request doRequest, seats config.Seats) (outcome headlessOutcome, default: // ran and did not finish: a leaf failed, or the clock arrived. errand.stop, errand.Settled = stopIncomplete, true + // The run already kept the worker's reason; both receipt formats + // need it to explain an incomplete ending. + errand.Error = plainWords(strings.TrimSpace(summary.Failure)) } // THE RUN'S OWN CLOCK, read off the context because the summary's word is // the same one a failed leaf leaves and a caller raising a timeout has to be @@ -3927,44 +3939,31 @@ func runSpendBound(request doRequest, profileDir string, now time.Time) (runSpen if err != nil { return runSpend{}, err } - daily, err := config.DailyBudgetUSDAt(profileDir) - if err != nil { - return runSpend{}, err - } bound := runSpend{} if consent > 0 { bound = runSpend{usd: consent, stop: stopPrice, words: fmt.Sprintf( "the run reached $%.2f, the price above which codeaf asks before it spends more; "+ "rerun with --yes-spend to let it go past that", consent)} } - if daily > 0 { - left := daily - spentToday(now) - if left <= 0 { - return runSpend{refused: true, stop: stopBudget, words: fmt.Sprintf( - "today's spending limit of $%.2f is spent, so nothing was started; "+ - "rerun with --yes-spend to spend past it", daily)}, nil + daily, err := session.DailySpendAt(profileDir, now) + if err != nil { + return runSpend{}, err + } + if daily.Limit > 0 { + left := daily.Limit - daily.Spent + if daily.Reached || left <= 0 { + return runSpend{refused: true, stop: stopBudget, words: session.DailySpendAction(daily.Limit) + + "; rerun with --yes-spend to spend past it"}, nil } if bound.usd == 0 || left < bound.usd { bound = runSpend{usd: left, stop: stopBudget, words: fmt.Sprintf( "the run reached what was left of today's spending limit of $%.2f; "+ - "rerun with --yes-spend to spend past it", daily)} + "rerun with --yes-spend to spend past it", daily.Limit)} } } return bound, nil } -// spentToday is what today has cost on this machine, read off the usage ledger -// every conversation and every run worker writes ([session.SpendToday]). A -// ledger that cannot be read is a day that has spent nothing as far as this -// door can tell; the plan-price rung still bounds the run. -func spentToday(now time.Time) float64 { - lines, err := session.ReadUsage(session.UsageLedgerPath(), now.Add(-48*time.Hour)) - if err != nil { - return 0 - } - return session.SpendToday(lines, now) -} - // crewCompleters turns the run road's provider seam into the per-model // completer [runengine.CrewFactory] asks for. The factory reads the crew at every // launch, so a task's seat model is not known until the task is handed over; diff --git a/cmd/codeaf/do_engine_contract_test.go b/cmd/codeaf/do_engine_contract_test.go index abe48ecabc..beec70ba78 100644 --- a/cmd/codeaf/do_engine_contract_test.go +++ b/cmd/codeaf/do_engine_contract_test.go @@ -36,6 +36,8 @@ func finishingSeat(usd float64) *beltSeat { if usd > 0 { cost := usd reply.Usage.Cost = &cost + reply.Usage.PromptTokens = 100 + reply.Usage.CompletionTokens = 20 } return reply } @@ -336,6 +338,11 @@ func TestDoOnTheRunEngineYesSpendRunsPastThePlanPrice(t *testing.T) { t.Fatalf("a run with --yes-spend left with %v, want 0\nstdout:\n%s\nstderr:\n%s", err, stdout.String(), stderr.String()) } + fields := doEnvelopeFields(t, stdout.String()) + tokens, ok := fields["tokens"].(map[string]any) + if !ok || tokens["in"] == float64(0) || tokens["out"] == float64(0) { + t.Fatalf("non-zero spend carried empty token counts: %v", fields["tokens"]) + } } // --db IS REFUSED IN WORDS. The run keeps its plan in a private folder, diff --git a/cmd/codeaf/do_engine_test.go b/cmd/codeaf/do_engine_test.go index 90df20d0dc..42dc6ad99e 100644 --- a/cmd/codeaf/do_engine_test.go +++ b/cmd/codeaf/do_engine_test.go @@ -847,3 +847,48 @@ func TestDoOnTheRunEngineSeatsAnUnpinnedCheckOnTheCrewsChecker(t *testing.T) { // bound is a named slot count for a request, the way the flag would name one. func bound(n int) *int { return &n } + +func TestDoAuthenticationFailureReachesHumanAndJSON(t *testing.T) { + for _, asJSON := range []bool{false, true} { + name := "human" + if asJSON { + name = "json" + } + t.Run(name, func(t *testing.T) { + beltRunEnv(t) + seat := &beltSeat{ever: func(context.Context, []ai.Message) (*ai.Response, error) { + return nil, errors.New("API error (401): Missing Authentication header") + }} + var stdout, stderr strings.Builder + err := doErrand(doRequest{ + task: "write out.txt", workspace: t.TempDir(), asJSON: asJSON, + timeout: 10 * time.Second, slots: bound(1), stdout: &stdout, stderr: &stderr, + newBeltCompleter: func(string) session.Completer { return seat }, + }) + if code := exitCodeOf(err); code != 2 { + t.Fatalf("auth refusal exited %d; want 2: %v", code, err) + } + want := "your key was not accepted for this model — the shell's OPENROUTER_API_KEY" + if asJSON { + var envelope struct { + Error string `json:"error"` + Stop string `json:"stop"` + } + if err := json.Unmarshal([]byte(stdout.String()), &envelope); err != nil { + t.Fatal(err) + } + if envelope.Error != want || envelope.Stop != "incomplete" { + t.Errorf("receipt error=%q stop=%q; want %q and incomplete", envelope.Error, envelope.Stop, want) + } + } else if !strings.Contains(stderr.String(), want) { + t.Errorf("human stderr lacks auth ending %q: %q", want, stderr.String()) + } + if seat.seen != 1 { + t.Errorf("auth refusal made %d calls; want 1", seat.seen) + } + if strings.Contains(stdout.String()+stderr.String(), "test-key") { + t.Error("the key reached the output") + } + }) + } +} diff --git a/cmd/codeaf/main.go b/cmd/codeaf/main.go index 16f3dd417d..063410178b 100644 --- a/cmd/codeaf/main.go +++ b/cmd/codeaf/main.go @@ -1099,11 +1099,15 @@ func emit(graph *plan.Graph, output string, asJSON bool) error { // lines, the environment included — for the sake of one missing quoted string, // and the one line that mattered scrolled off the top of the terminal. func readText(name string, args []string) (string, error) { - if len(args) == 1 && args[0] == "-" { - return readPipedText(name) - } if len(args) > 0 { - return strings.TrimSpace(strings.Join(args, " ")), nil + text := strings.TrimSpace(strings.Join(args, " ")) + if text == "" { + return "", noGoalGiven(name) + } + if len(args) == 1 && args[0] == "-" { + return readPipedText(name) + } + return text, nil } if stdinIsTerminal(os.Stdin) { return "", noGoalGiven(name) diff --git a/cmd/codeaf/sshcontrol_regression_test.go b/cmd/codeaf/sshcontrol_regression_test.go new file mode 100644 index 0000000000..9c0ef2ec01 --- /dev/null +++ b/cmd/codeaf/sshcontrol_regression_test.go @@ -0,0 +1,26 @@ +package main + +import ( + "os" + "path/filepath" + "strings" + "testing" + + "github.com/Agent-Field/codeaf/internal/enginehost" +) + +func TestSSHControlPathFitsOpenSSHTemporarySuffix(t *testing.T) { + const hashLength = 40 + const temporarySuffix = 17 + prefix, err := os.MkdirTemp("/tmp", "s") + if err != nil { + t.Fatal(err) + } + t.Cleanup(func() { _ = os.RemoveAll(prefix) }) + rootLength := enginehost.SocketLimit - len("/v3/ssh/ctl-") - hashLength + root := filepath.Join(prefix, strings.Repeat("a", rootLength-len(prefix)-1)) + t.Setenv("CODEAF_HOME", root) + if got := sshControlPath(); got != "" { + t.Fatalf("control path %q was accepted without room for OpenSSH's %d-byte temporary suffix", got, temporarySuffix) + } +} diff --git a/docs/changes/unreleased/1562-plandb-done-terminal-and-manager-ground.md b/docs/changes/unreleased/1562-plandb-done-terminal-and-manager-ground.md new file mode 100644 index 0000000000..d44a548477 --- /dev/null +++ b/docs/changes/unreleased/1562-plandb-done-terminal-and-manager-ground.md @@ -0,0 +1,27 @@ +--- +kind: fixed +title: plandb done refuses terminal tasks and manager tasks derive project workspace +pr: 1562 +surface: [engine, resident] +invalidates: + - "plandb done on a cancelled task failed with 'task is not claimed'; it now explicitly reports that the task is already terminal." + - "a task proposed by a manager whose workspace was ~ inherited ~ as folder ground; it now falls back to Place.Workspace." +--- + +When a task was cancelled upstream, plandb done called requireOwner first, which failed with a misleading 'task is not claimed' error because cancelled tasks clear their claim. Now terminal status is checked before owner verification (preserving placeholder auto-completion). + +In addition, groundLadder now falls back from a non-repository workspace (such as ~) to a configured project Place.Workspace rather than landing tasks directly in ~. + +The default bash run door now uses the same ground resolver as proposed and +legacy tasks. Previously a typed `/task` silently bypassed that resolver and +copied the conversation directory even when its brief named a repository. +The configured-project fallback applies only to roots; children keep their +parent's folder. Bash guidance uses bounded project discovery and directs +workers to `plandb --help` instead of hunting for missing design documents. +Cancelled review tasks report their terminal state before validating review +results, and rejected completion leaves the stored task unchanged. + +The shared bash-worker brief now names the actual assigned working directory +before the work order, for workers and checkers alike. A request can quote the +source checkout without inviting the worker to leave its isolated copy; the +request and unrelated read-only reference paths remain unchanged. diff --git a/docs/changes/unreleased/1603-bounded-foreground-output.md b/docs/changes/unreleased/1603-bounded-foreground-output.md new file mode 100644 index 0000000000..6bc52a333a --- /dev/null +++ b/docs/changes/unreleased/1603-bounded-foreground-output.md @@ -0,0 +1,21 @@ +--- +kind: fixed +title: bound foreground bash spill files and preserve their initial output +pr: 1603 +surface: [chat, engine] +invalidates: + - "Only background job logs were bounded; foreground bash could still fill the temporary disk through unbounded pi-bash spill files." + - "A spill beginning after the result limit silently lost its initial bytes while claiming Full output." +--- + +Fixes #1606. Foreground output now retains an 8 MiB initial snapshot under the +state home's logs/bash directory while the result continues to show the latest +in-memory tail. Incomplete snapshots and write/close errors are named honestly. +Promotion closes the foreground spill before forwarding output to the job log. + +Completed snapshots have a 128 MiB, 64-file, seven-day retention policy. A +directory lock serializes publication and cleanup; independent file leases +protect active writers across processes. Unknown, linked and legacy files are +preserved. Recursive structured search excludes both new snapshots and old +pi-bash files in the current temporary directory. Explicit file reads remain +bounded and shell commands do not inherit these exclusions. diff --git a/docs/changes/unreleased/1603-bounded-job-logs.md b/docs/changes/unreleased/1603-bounded-job-logs.md new file mode 100644 index 0000000000..8f6b1e1df1 --- /dev/null +++ b/docs/changes/unreleased/1603-bounded-job-logs.md @@ -0,0 +1,28 @@ +--- +kind: fixed +title: bound background logs and keep recursive searches out of runtime output +pr: 1603 +surface: [chat, engine] +invalidates: + - "Background jobs wrote unbounded logs and promised full output. Each job now retains two 4 MiB chunks, reports discarded output and disk failures, and continues draining the child process." + - "Completed job logs could accumulate without an aggregate budget. Eligible completed managed logs now share a 128 MiB and 64-job budget per directory, while startup preserves seven-day expiry and durable job ID history. Active and unmarked legacy logs are protected from cleanup." + - "Recursive grep could read its own runtime output and the fallback loaded whole files. Both engines now exclude known runtime output, while explicit file inspection and fallback reads use bounded snapshots. Shell search guidance now requires explicit exclusions and byte-limited log reads." +--- + +The spool retains the same active file and lease through rotation, so recursive +readers cannot chase an ever-growing old inode. Write, rotation and close errors +remain visible without failing the child process. Retention uses durable ID +allocation, ownership markers and independent file leases. Unsafe linked files +or descendants of the selected storage root are refused; legacy output remains +outside the budget. Selected workspace and session folder aliases are resolved +before creating logs, so a symlinked login home does not prevent jobs starting. +The bounds apply per job and per jobs directory, not across the machine. + +Structured search preserves source worktrees, including those inside custom +state homes. Input-history journals and project task indexes are excluded too, +so the current question and generated task title cannot re-enter source results. +Recursive searches skip files above 8 MiB. Explicit inspection reads +at most the first 8 MiB present at open; oversized 64 KiB lines are skipped with +an incomplete-result notice. Requests allow at most 1000 matches and 20 context +lines per side. Oversized ripgrep records terminate and reap the child with an +explicit error. Arbitrary shell commands do not inherit structured exclusions. diff --git a/docs/changes/unreleased/1604-docs-audit-batch.md b/docs/changes/unreleased/1604-docs-audit-batch.md new file mode 100644 index 0000000000..3b5d89cc6a --- /dev/null +++ b/docs/changes/unreleased/1604-docs-audit-batch.md @@ -0,0 +1,37 @@ +--- +kind: fixed +title: A batch of fixes from the docs audit — headless, tasks, standing, teams, runs, media, remote +pr: 1604 +surface: [chat, engine, remote, docs] +invalidates: + - "The restart manual incorrectly put interrupted and failed tasks under a separate Incomplete rail group. Inactive rows remain in the Done fold with truthful interrupted/incomplete labels; failed rows sort first. Valid saved pending requests return to Queued, and admitted ordinary tasks reconnect to their recorded copy." + - "A turn waiting on the daily-budget card could outlive cancellation, and `codeaf do --yes-spend` workers could wait on that card despite the run being authorized. Cancelled waits now close their streams, and child workers and checkers inherit only the run-scoped daily-limit permission; task, time and approval limits remain in force." + - "`codeaf do \"\"` (or an all-blank brief) ran a paid job on a goal the planner invented. It is refused with the same usage as `codeaf do` with no argument, before anything is spent." + - "Flags after the brief (`codeaf senior-dev \"\" --max-cost 0.5`) were folded into the brief and the default ceiling applied. They are parsed wherever they sit for every delegate program, an unknown flag is refused, and `--` still ends flags." + - "`codeaf do --json` on the run engine reported zero tokens beside a non-zero spend. The run engine's workers now carry input and output tokens to the receipt." + - "A read hand-off (`◆ reading: …`) ran as an ordinary quick task with bash and commit, so it could write, commit and merge. It now gets the audit read-only belt, hands anything it cannot do back to the conversation, and a failed hand-off leaves no stopped card." + - "A task held by the busy-machine gate had already taken its folder lock and branch, and `/stop` could not end it. A held task owns only its plan row (`queued · machine busy`), stops at once, and prepares its folder only after admission; an admitted task's refusals are unchanged." + - "After an engine restart ordinary planned tasks stayed working without a driver. Reopening reconnects the same plan and recorded copy, or readmits the exact queued request before preparing any folder; missing older ground becomes explicitly interrupted, never done. `/stop` also reaches a sole queued plan from the main box." + - "An explicit stop was recorded as a failed crew, so the crew line said `failed … /redo stronger` and the router learned a failure. A stop reads `stopped`, offers no redo and teaches the router nothing." + - "Plan-born task workers were briefed without the project's standing orders. Every worker a run seats carries the one standing section." + - "A project standing order stored with a trailing separator was listed under other projects; the standing card quoted the allowance the session started with; a firing that ran a task read `said:`. Paths are compared cleaned, the card quotes the live allowance the rail enforces, and task firings read `task:`." + - "A manager started with `M` had no handle until its first turn, so `team_send` could not reach it. It gets a unique fallback handle when it is registered." + - "A sub-team started by `team_start` did not inherit the starting manager's approval posture, and the start card showed the rule word `default`. The posture is carried to the child, and a fallback rule shows the cost clause." + - "`team_start`'s description said the person is always asked first, so a manager under an allowing posture reported an approval nobody gave. It now says approval follows the posture." + - "Every check in a fan-out run drew from one checker-seat tally, so later checks stopped at a ceiling other checks had spent, and the run still said done. Each check has its own ceiling, and a run with unfinished checks does not end done." + - "A run branch kept on a protected or moved checkout was invisible to `/land`, and `.orig` backups could reach a task branch. `/land` lists and lands kept run branches, and `.orig` files are droppings." + - "A Home conversation could be opened on another project's live engine. A conversation whose engine answers for a different workspace is refused visibly before anything is sent." + - "The wall showed other windows' conversations as open here, and a tile opened from Teams landed back on Teams. It shows only this window's conversations, and a tile lands on its conversation." + - "`/memory`, `/remember`, `/memories` and `/forget` said memory is off on the hosted engine road. They reach the engine's memory over the wire." + - "Generated JPEG bytes were saved under a `.png` name. Images are named by their sniffed bytes." + - "Speech spend had no role in usage.jsonl and media calls wrote no call-log rows. Speech records under its own role, and media requests write a call-log pair with the provider's cost." + - "After a `--host` redial the window became a watcher of its own dead pipe, and ssh stderr painted over the frame. The same window takes its keyboard back by client id, and a redial's stderr goes to the diagnostic tail." + - "`/drafts` drew nothing while it held the keyboard, `/skill` on an empty shelf left `/skill ` in the box, and esc could not cancel a pending browser sign-in. All three are fixed." +--- + +One entry for the whole batch. The issues it closes, the ones it only narrows, and the ones left for a ruling are listed in the pull request. + +The new-direction completion fixture now observes its asynchronous readings before +advancing the instant script. It still requires two accepted completion endings +and no task admission; exhausting the script is no longer mistaken for a +production completion failure. No runtime timing or completion policy changed. diff --git a/docs/design/conversations-and-teams/DESIGN.md b/docs/design/conversations-and-teams/DESIGN.md index 5947b470ed..b8c3836971 100644 --- a/docs/design/conversations-and-teams/DESIGN.md +++ b/docs/design/conversations-and-teams/DESIGN.md @@ -664,7 +664,7 @@ Everything below is `internal/teams` unless named otherwise. | teams.json field | Go | Meaning | Band | |---|---|---|---| | `questions_up` | `*bool` | members' questions go to the home manager first | | -| `cap_usd_day` | `*float64` | dollars per local day for the team and everything under it; `0` is an explicit no cap | `>= 0` | +| `cap_usd_day` | `*float64` | an explicit dollars-per-day cap for the team and everything under it; `0` is an explicit no cap | `>= 0` | | `depth_limit` | `*int` | levels of teams, the top counting as one | 1 to 10 | | `sub_share` | `*float64` | fraction of this team's cap a new sub-team is made with | (0, 1] | | `wake` | `*bool` | team traffic wakes idle conversations (section 5); the old `"wake": false` reads as off | | @@ -684,13 +684,16 @@ Everything below is `internal/teams` unless named otherwise. - `(*File).Depth(id) int` (the root is 0, top level 1), `(*File).CanNest(parent, Defaults) bool` (parent open and one more level inside its effective limit), `(*File).SubTeamCap(parent, Defaults) float64` (parent's effective cap times its share, to the cent; 0 when the parent - has none). The session writes that figure on the new sub-team with `SetSettings`, so a later + has none). With no derived cap, the new sub-team follows the ordinary effective-cap walk. + The session writes a positive figure on the new sub-team with `SetSettings`, so a later change of the share moves no team that exists. -**A cap is a pool.** A team's spend counts every team under it, so an inherited cap is the -ancestor's one pool, shared, never a second allowance of the same size. `Effective.CapFrom.Team` -names the pool's owner; when the cap comes from Settings, the owner is the top of the chain -(the root when there is one). The spend drawn beside a cap is always the owner's. +**A cap is a pool.** An explicit cap on a team counts that team and every team under it, so an +inherited cap is the ancestor's one pool, shared, never a second allowance of the same size. +The profile's default cap is different: it gives each ordinary team its own pool, while the +`All teams` root has no cap unless it is explicitly set. `Effective.CapFrom.Team` names the +pool's owner; when the cap comes from Settings, the owner is the team itself for an ordinary +team. The spend drawn beside a cap is always the owner's. **Config keys** (`internal/config/teamdefaults.go`, flat dotted keys, category `teams`, one settings tab `Teams`, each a row with a named reader `TeamDefaultsAt` and a ledger line): @@ -867,8 +870,8 @@ forbids every new store door in a frame (`framedisk_law_test.go`). - **One level.** `team_send`, `team_stop` and `team_start` reach only the manager's own team's members (a sub-team's manager is one). Links may send fyi notes (`KindNote`) and nothing else. - **Caps.** Before each model request of a member, the session compares - `TeamSpend(owner, Today())` with the pool owner's effective cap (`Effective.CapFrom.Team`, or - the top of the chain). Reached: it raises one `cap` packet to `Person` for that team and day + `TeamSpend(owner, Today())` with the pool owner's effective cap (`Effective.CapFrom.Team`). + Reached: it raises one `cap` packet to `Person` for that team and day (not one per request; look for an open one first) with options `raise` (`Raise to $10`, twice the cap) and `stop` (`Stop for today`) and a recommendation, and holds new member turns in that pool until it is decided. Managers never raise a cap: money is the person's. @@ -1018,13 +1021,15 @@ its tab stays open. Other shared conversations keep the prior rule.) - **The settings group is its own tab, `Teams`.** The settings tabs are one-to-one with their categories for the four newer tabs; a group inside Tasks would be a row filed under one category and drawn under another. -- **A cap is a pool, and the header says whose.** `$1.20 of $5 today · from harbor` beside a +- **A cap is a pool, and the header says whose.** `$1.20 of $5 today · harbor's cap` beside a sub-team would read as a second $5; the header shows the pool owner's spend and says `harbor's cap`. The settings card still says `from harbor`, which is true of the value. - **Cap packets always go to the person.** A manager that could raise its own cap would make the cap advice. Managers may stop their own team early; they may not spend more. -- **The global manager is a real root team**, not a special case beside the tree, so every - rule above holds for it unchanged; the root is not a level and cannot close. +- **The global manager is a real root team**, not a special case beside the tree, so its + membership, authority and spend walk are ordinary; the root is not a level and cannot close. + Its profile-default cap is the deliberate exception: `All teams` has no cap until the person + explicitly sets one on it. - **Team defaults are rails**, refused to model self-service like the spending and consent rows. A team's own overrides are written only by the interface for the person; the team tools must not write them (d1). @@ -1095,8 +1100,8 @@ without waking. `Decide` on a question now logs `answered @web: