Skip to content

refactor(api/nlq): convert UI actions from marker system to tool definitions#307

Merged
SimplicityGuy merged 2 commits intomainfrom
feat/nlq-actions-as-tools
Apr 15, 2026
Merged

refactor(api/nlq): convert UI actions from marker system to tool definitions#307
SimplicityGuy merged 2 commits intomainfrom
feat/nlq-actions-as-tools

Conversation

@SimplicityGuy
Copy link
Copy Markdown
Owner

Summary

  • Converts the NLQ engine's 10 UI action types (seed_graph, switch_pane, filter_graph, ui_find_path, highlight_path, focus_node, show_credits, open_insight_tile, set_trend_range, suggest_followups) from a markdown HTML-comment marker (``) to real Anthropic tool-use definitions. Each action tool validates its payload via pydantic and records the Action into a per-request list that populates `NLQResult.actions`.
  • Removes `_ACTIONS_INSTRUCTION`, `_ACTIONS_MARKER_RE`, and `_extract_actions` from `api/nlq/engine.py`. The system prompt now contains one short paragraph pointing the model at the action tools; the rest is self-documenting via Anthropic's tool schemas.
  • `ui_find_path` is the tool name for the client action type `find_path` because the data tool `find_path` already owns that name in the Anthropic tool namespace. The internal mapping in `_ACTION_TOOL_TO_TYPE` rewrites it back to `find_path` when building the Action, so the client-visible wire format is unchanged.

Why

The marker system was load-bearing on LLM discipline and failed on complex queries in practice. Users asked things like "what's the biggest Trance label" and got a perfect markdown answer but zero actions fired — the model forgot the marker, or the regex silently dropped malformed JSON. Tool-use makes the action emission a structural contract: the model sees each action as a first-class tool with a JSON schema, and the engine assembles `NLQResult.actions` from the actual tool invocations instead of scraping text.

Contract preservation

  • `NLQResult.actions` still ships as `[{type: str, ...payload}, ...]` on the wire — the NLQ router already serializes via `action.model_dump(by_alias=True, mode="json")`, and the Action pydantic models are unchanged.
  • The frontend `NlqActionApplier` and handler shapes in `explore/static/js/nlq-handlers.js` and `explore/static/js/nlq-action-applier.js` are untouched — the tool input schemas match their expected payload shapes 1:1.
  • Data tools are untouched. Auth-aware tool selection is untouched. `_AUTH_ADDENDUM` is untouched.

Test plan

  • `uv run pytest tests/api/nlq/` — 110 passing (14 new action-tool tests plus existing)
  • `uv run pytest tests/api/` — 1551 passing
  • `uv run mypy api/nlq/` — clean
  • `uv run ruff check api/nlq/ tests/api/nlq/` — clean
  • Each of the 10 action tools exercised end-to-end (single-tool invocation → Action recorded in `NLQResult.actions`)
  • Mixed data + action tool in the same iteration (search + seed_graph)
  • No-action data-only flow leaves `actions` empty
  • Invalid action payload (`switch_pane` with unknown pane) returns an error result and is NOT recorded
  • Multiple distinct actions recorded in order (switch_pane + set_trend_range)
  • Action tool names disjoint from data tool names

🤖 Generated with Claude Code

…nitions

Previously the NLQ engine emitted UI mutation actions via a markdown HTML
comment marker (<!--actions:[...]-->) appended to the LLM's response text.
This design was load-bearing on LLM discipline and failed on complex
queries (e.g. "what's the biggest Trance label") — users got perfect
answers but zero actions fired because the marker was missing or malformed.

Each of the ten action types is now a real Anthropic tool definition:
seed_graph, highlight_path, focus_node, filter_graph, ui_find_path,
show_credits, switch_pane, open_insight_tile, set_trend_range, and
suggest_followups. The tool loop validates each invocation via pydantic
and records the resulting Action into a per-request list that populates
NLQResult.actions. Client wire format is unchanged.

The ui_find_path tool records actions as client type "find_path" — the
tool name is distinct because the data tool find_path already owns that
name in the Anthropic tool namespace.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@codecov
Copy link
Copy Markdown

codecov bot commented Apr 15, 2026

Codecov Report

❌ Patch coverage is 96.87500% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
api/nlq/tools.py 95.83% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

PR #307 introduced action tools with unprefixed names (seed_graph,
switch_pane, etc.) and resolved the find_path collision by prefixing only
that one as ui_find_path. Review feedback: prefer a consistent ui_ prefix
on all 10 action tools for visual clarity in the model's tool list.

Client wire types (NLQResult.actions[i].type) stay unprefixed so the
frontend applier contract is unchanged — _ACTION_TOOL_TO_TYPE maps each
ui_* tool name to its wire type.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@github-actions
Copy link
Copy Markdown
Contributor

E2E Coverage (webkit)

Totals Coverage
Statements: 46.5% ( 1241 / 2669 )
Lines: 46.5% ( 1241 / 2669 )

StandWithUkraine

@github-actions
Copy link
Copy Markdown
Contributor

E2E Coverage (chromium)

Totals Coverage
Statements: 46.5% ( 1241 / 2669 )
Lines: 46.5% ( 1241 / 2669 )

StandWithUkraine

@github-actions
Copy link
Copy Markdown
Contributor

E2E Coverage (firefox)

Totals Coverage
Statements: 46.5% ( 1241 / 2669 )
Lines: 46.5% ( 1241 / 2669 )

StandWithUkraine

@github-actions
Copy link
Copy Markdown
Contributor

E2E Coverage (webkit - iPhone 15)

Totals Coverage
Statements: 46.5% ( 1241 / 2669 )
Lines: 46.5% ( 1241 / 2669 )

StandWithUkraine

@github-actions
Copy link
Copy Markdown
Contributor

E2E Coverage (webkit - iPad Pro 11)

Totals Coverage
Statements: 46.5% ( 1241 / 2669 )
Lines: 46.5% ( 1241 / 2669 )

StandWithUkraine

@SimplicityGuy SimplicityGuy merged commit 3b3d07c into main Apr 15, 2026
57 checks passed
@SimplicityGuy SimplicityGuy deleted the feat/nlq-actions-as-tools branch April 15, 2026 22:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant