Skip to content

Trim the tool roster, guard runaway tool calls, and release 0.3.4 - #18

Merged
Justar96 merged 2 commits into
developfrom
chore/post-0.3.3-debt
Oct 3, 2026
Merged

Justar96 merged 2 commits into
developfrom
chore/post-0.3.3-debt

Conversation

@Justar96

@Justar96 Justar96 commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Summary

  • Leaner tools and prompt. Tool and parameter descriptions are shorter, and the persona asks for independent tool calls in one response, an edit sent with its check, one edit call per file, and no confirming re-read.
  • Removed tools: todo_write (and the TUI task list, Ctrl+T, compaction <todo-list>), workflow and ralph, present, and subagent_fork. subagent, send_message, list_agents, interrupt_agent, and list_subagent_models are unchanged.
  • Removed unused packages: webhook and workspace registry, str-replace-editor, Exa search, time/tmux context, Python PTC runtime, per-message feedback, the Node PTC runtime, tool_help/details, and subagent parent-history seeding.
  • Runaway tool-call guard. GPT 6.1 Sol sometimes finishes a tool call's arguments and then streams whitespace until timeout. llm-pi-ai now aborts once more than 2,000 whitespace characters follow outside JSON strings and fails the step with retryable TRANSPORT. New opt-in strictTools route setting sends strict tool schemas on OpenAI wires.
  • Core cleanup, no behaviour change: shared helpers moved to util-values, ToolRuntime split (schemas byte-identical), dead launcher-identity code removed, stale ACP/web-client wording fixed.
  • Eval runner: pi agent arm, @effort suffix, EVAL_GATEWAY, per-request stream timing and tool-argument capture, deep-merged EVAL_SETTINGS_<ARM>.
  • Release 0.3.4: version bump, CHANGELOG, eval records renamed to evals/agent-loop/versions/v0.3.4.

Session format

No version bump and no persistence digest moved. Event types of removed plugins (todo/write, tool-workflow/*, deliverables/presented, feedback/message-*, webhook sources) moved to the retired vocabulary, so released sessions still open. bun scripts/persistence-changes.ts --check: 63 roots match 5 history records.

Eval

Records in evals/agent-loop/versions/v0.3.4/:

  • 2026-10-03-trim-and-removals (base: pre-change tree, 4 models, 3 trials): total tokens −42% Opus 5.5, −39% DeepSeek V4 Pro, −67% Gemini 3.8 Flash, −42% GPT 6.1 Sol, with fewer requests and no tool errors. Flagged: GPT failures 1 → 4. Accepted: follow-up probes with stream capture showed all of them are GPT's whitespace runaway in an unclosed tool call, which also happens on the base tree. It is fixed by the next record.
  • 2026-10-04-runaway-guard (base: tree just before the guard, every arm one retry): no regressions. GPT failures 11 → 1 of 40; Opus, Gemini and DeepSeek unchanged and the guard never fired.
  • steps/strict-tools: GPT 40/40, with no runaways. Flagged: +6.3% tokens vs the guard arm, from the larger strict schemas. Accepted: strictTools is off by default.
  • Also: 2026-10-03-trim-and-batch (with attribution steps), 2026-10-03-bake-vs-pi, 2026-10-02-text-block-parts.

Checks

  • bun run preflight: 27 passed, 1 skipped (actionlint not installed); includes runtime, integration, and e2e PTY.
  • bun run release:preflight: ready to release 0.3.4.
  • Not run: Windows/macOS or a packed install.

Shorter tool descriptions and round-trip persona guidance; remove the
todo_write, workflow, ralph, present, and subagent_fork tools, the
webhook and workspace packages, and unmounted packages (retired event
types kept so released sessions open); abort tool calls that stream
runaway whitespace and add opt-in strict tool schemas; dedupe core
helpers and split ToolRuntime; eval runner gains a pi arm, stream
timing, and EVAL_GATEWAY. Eval records under versions/unreleased.
@Justar96
Justar96 merged commit ca5f4ea into develop Oct 3, 2026
5 checks passed
@Justar96 Justar96 mentioned this pull request Oct 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant