Skip to content

feat: smart tool shortlisting per turn - #9

Open
Shifat7 wants to merge 1 commit into
clutchpbcfo:mainfrom
Shifat7:pr-02-smart-tool-shortlisting
Open

Shifat7 wants to merge 1 commit into
clutchpbcfo:mainfrom
Shifat7:pr-02-smart-tool-shortlisting

Conversation

@Shifat7

@Shifat7 Shifat7 commented Aug 22, 2026

Copy link
Copy Markdown

What changed

Before buildRelayPrompt(), each turn is classified locally into one of final_answer_only / read_or_search / edit_code / run_command / debug_failure / unknown, and only the client tool categories that task type plausibly needs are forwarded to Hyperagent. Function tool schemas are minimised to argument names, types, and required lists; descriptions are capped (160 chars default).

  • New exports in src/protocol.mjs: classifyTurn(input), shortlistTools(tools, taskType, config); extractClientTools applies them when enabled.
  • Config: enableSmartToolSelection (default true), forwardFullToolSchemas (default false), maxToolDescriptionChars (default 160). Old behaviour available via enableSmartToolSelection: false.

Why it saves credits

Tool schemas are a large fixed cost on every request of every tool loop.

Measured locally (synthetic 32-tool toolbox, request "Run the test suite."):

Before: 32 tools forwarded, 10,431 tool-schema chars
After:   4 tools forwarded,  1,127 tool-schema chars

Safety rules (all test-enforced)

  • Exact tool names preserved; nothing invented.
  • Never removes all tools when the user asked to inspect/edit/run/patch; edit and debug tasks keep patch + read + shell chains.
  • Conversations that already used tools are never reduced to zero.
  • Unknown turns keep the full inventory; disabling the flag restores byte-for-byte prior behaviour.

Risks/caveats

A mis-classified first turn can omit a specialised MCP tool for that one turn; the relay falls back to plain text naming the missing tool (existing parseRelayOutput behaviour) instead of executing it. Rephrase or disable if that bites.

Testing

npm test: 82 pass, 0 fail (3 pre-existing skips). New tests cover classification taxonomy, per-task shortlists, edit/debug chain preservation, mid-task retention, disable/unknown pass-through, and schema minimisation shape.

Classify each turn locally into final_answer_only, read_or_search,
edit_code, run_command, debug_failure, or unknown, then forward only
the client tool categories that task type needs. Function schemas are
minimised to argument names, types, and required lists with capped
descriptions unless forwardFullToolSchemas is enabled.

Safety rules enforced by tests: exact tool names preserved, no tools
invented, mid-task conversations keep their tools, unknown turns keep
the full inventory, and disabling enableSmartToolSelection restores
unconditional forwarding. Defaults: on, minimal schemas, 160-char
descriptions.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant