Skip to content

feat(agent-service): use the AI SDK's native message history - #9494

Open
bobbai00 wants to merge 1 commit into
apache:mainfrom
bobbai00:feat/agent-native-messages
Open

bobbai00 wants to merge 1 commit into
apache:mainfrom
bobbai00:feat/agent-native-messages

Conversation

@bobbai00

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

Before every model call, the agent dropped the AI SDK's messages and wrote one user message in their place: a summary of the conversation, then the current dataflow. This PR keeps the conversation as the SDK's own messages.

main This PR
What the model reads One user message the service writes again before every call The conversation's messages: the user's, the assistant's with its tool calls, and the tool results
A past tool call Its name, its summary and the first line of its result Its arguments and its whole result
The current dataflow Inside that message One message at the end of each call's input, written again each time and never kept
The loop generateText with onStepFinish, an alias AI SDK 7 deprecates The SDK's ToolLoopAgent with onStepEnd
The system prompt Describes the summary's format (1.4 KB) Describes the dataflow block only

Each step keeps the messages it added, and the model reads them along the path to HEAD, so a branch HEAD has left stays out of sight as before. A stopped or failed turn ends with the note the chat already shows ("Generation stopped by user.", "Error: ..."), so the next turn knows. The chat looks the same; only its step details change, since they list what the model received:

Step details on main Step details with this PR
One rebuilt message The conversation as native messages

Any related issues, documentation, discussions?

Closes #9455
Part of #9453

How was this PR tested?

  • cd agent-service && bun test (341 pass), bun run typecheck and bun run format:check. The new tests check, with the SDK's mock model, the exact input of each call: a tool call and its whole result reach the next step, the dataflow block is last and appears once, a second turn reads the first as it happened, an abandoned branch stays out, a stopped or failed turn is part of the next turn's history, and clearHistory leaves nothing behind.

  • End to end with Claude Haiku 5.5 through LiteLLM, on a single-node stack of today's images with this branch's agent-service and the Angular app. Three prompts on the Iris example dataset (load and count it, the species with the largest average petal length, then add the average sepal length) on main and on this branch:

    Answers correct Model calls per turn Input tokens
    main 3 of 3 2, 2, 2 79,194
    This PR 3 of 3 2, 2, 2 78,384
  • A turn stopped after its first step, followed by "Go on: finish what I asked before", continued from the stopped turn's tool result and added only the missing operator.

Was this PR authored or co-authored using generative AI tooling?

Generated-by: Claude Opus 5.5

@bobbai00

Copy link
Copy Markdown
Contributor Author

The evaluation from #9495 on this branch, against main, with Claude Haiku 5.5 and three runs per case:

main This PR
Runs passed 9 of 18 9 of 18
Model calls per run 2.5 2.4
Input tokens per run 32,506 31,180
Output tokens per run 964 721
KB per call 37.6 36.9

The same cases pass and fail on both: the docs question, a change the user makes between two prompts, and the user's own execution are what later sub-issues of #9453 address. Most of each call is still the system prompt, which #9456 and #9481 shrink.

@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • Contributors with relevant context: @oliarmak-web, @aglinxinyuan
    You can notify them by mentioning @oliarmak-web, @aglinxinyuan in a comment.

@codecov-commenter

codecov-commenter commented Nov 16, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.56%. Comparing base (4db10cf) to head (bea6728).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #9494      +/-   ##
============================================
- Coverage     91.57%   91.56%   -0.02%     
  Complexity     5159     5159              
============================================
  Files          1300     1300              
  Lines         58092    58019      -73     
  Branches       7615     7615              
============================================
- Hits          53198    53125      -73     
  Misses         2958     2958              
  Partials       1936     1936              
Flag Coverage Δ *Carryforward flag
access-control-service 77.44% <ø> (ø) Carriedforward from 4db10cf
agent-service 99.55% <100.00%> (-0.02%) ⬇️
amber 85.44% <ø> (ø) Carriedforward from 4db10cf
computing-unit-managing-service 63.88% <ø> (ø) Carriedforward from 4db10cf
config-service 87.50% <ø> (ø) Carriedforward from 4db10cf
file-service 82.16% <ø> (ø) Carriedforward from 4db10cf
frontend 96.45% <ø> (ø) Carriedforward from 4db10cf
notebook-migration-service 83.77% <ø> (ø) Carriedforward from 4db10cf
pyamber 98.38% <ø> (ø) Carriedforward from 4db10cf
result-service 85.71% <ø> (ø) Carriedforward from 4db10cf
workflow-compiling-service 78.30% <ø> (ø) Carriedforward from 4db10cf

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Before every model call, the agent dropped the SDK's messages and wrote one
user message: a summary of the conversation followed by the current
dataflow. A past tool call appeared as its name, its summary and the first
line of its result.

Each step now keeps the model messages it added: the user's message, or the
assistant message with its tool calls and the tool results. The model reads
these along the path to HEAD, so a branch HEAD has left stays out of sight.
The current dataflow is one message at the end of every step's input,
written again each time and never kept. The loop runs as the SDK's
ToolLoopAgent, and the system prompt describes only the dataflow block.

A tool call the SDK did not run, because its response was cut off, is kept
with a failed result: the SDK refuses a conversation in which a call has no
result, so every later turn would fail.
@bobbai00
bobbai00 force-pushed the feat/agent-native-messages branch from 9ff9816 to bea6728 Compare November 19, 2026 08:38
@bobbai00

Copy link
Copy Markdown
Contributor Author

@yrenat @sshiv012 Can you guys review this PR ? I feel that both of you have first-hand experiences with developing TexeraAgent

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Use the AI SDK's native message history in the agent

2 participants