Skip to content

fix(ai): a retried LLM chat task should keep its conversation history - #1480

Open
ling-senpeng13 wants to merge 1 commit into
mainfrom
fix/3876-mapper-snapshots-messages
Open

fix(ai): a retried LLM chat task should keep its conversation history#1480
ling-senpeng13 wants to merge 1 commit into
mainfrom
fix/3876-mapper-snapshots-messages

Conversation

@ling-senpeng13

@ling-senpeng13 ling-senpeng13 commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

The bug

Retrying an LLM_CHAT_COMPLETE task in an agent workflow loses the entire conversation. The retried attempt sees only the static [system, user] template from the workflow definition, not the history the first attempt had.

Why it happens

Conversation history is never expressed as a ${...} reference in the workflow definition. AgentChatCompleteTaskMapper assembles it imperatively in Java at scheduling time — walking the workflow's completed tasks, extracting tool results, condensing if the context window demands it — and writes the result into the task's inputData.

Retry doesn't re-run the mapper. It copies the task and re-resolves the definition:

TaskModel taskToBeRetried = task.copy();              // history is here...
taskToBeRetried.getInputData().putAll(taskInput);     // ...and clobbered by re-resolving
                                                      //    the definition's inputParameters

Since the definition's inputParameters contain only the static template, putAll overwrites the assembled conversation with it. The history existed solely in the first attempt's inputData, and that's what gets replaced.

Fix

The mapper strips messages and tools from the task's own copy of the definition before it's persisted, so at retry time getWorkflowTask().getInputParameters() doesn't have them, the re-resolved taskInput in taskToBeRescheduled() doesn't have them, and putAll can't overwrite the assembled values which is the root cause of this bug.

AIModelTaskMapper#detachAssembledInputFromDefinition(TaskModel, String... keys), shared by both chat-complete mappers.

Testing

@ling-senpeng13 ling-senpeng13 self-assigned this Aug 5, 2026
@ling-senpeng13
ling-senpeng13 marked this pull request as ready for review August 5, 2026 03:07
@ling-senpeng13
ling-senpeng13 requested a review from v1r3n August 5, 2026 03:45

@manan164 manan164 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this happen with all worker task that we lose input parameters on retry?

@ling-senpeng13
ling-senpeng13 requested a review from manan164 August 5, 2026 14:52
@ling-senpeng13
ling-senpeng13 force-pushed the fix/3876-mapper-snapshots-messages branch 2 times, most recently from 7805f7e to c621c44 Compare August 5, 2026 20:49
@ling-senpeng13

Copy link
Copy Markdown
Contributor Author

Does this happen with all worker task that we lose input parameters on retry?

Yes — re-resolving inputParameters on retry is the designed behaviour for every worker task, not
something specific to LLM tasks:

taskToBeRetried.getInputData().putAll(taskInput);   // taskToBeRescheduled

@NicholasDCole NicholasDCole left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple suggestions. Please check

@ling-senpeng13
ling-senpeng13 force-pushed the fix/3876-mapper-snapshots-messages branch 2 times, most recently from 5a7a242 to e6eaf3c Compare August 6, 2026 00:22
@ling-senpeng13 ling-senpeng13 changed the title fix(agentspan): snapshot assembled messages so a retry keeps conversation history fix(ai): a retried LLM chat task keeps its conversation history Aug 6, 2026
@ling-senpeng13
ling-senpeng13 force-pushed the fix/3876-mapper-snapshots-messages branch from ace2b3e to 5c5d139 Compare August 6, 2026 15:55
@ling-senpeng13
ling-senpeng13 force-pushed the fix/3876-mapper-snapshots-messages branch from 5c5d139 to 6a1976b Compare August 6, 2026 16:23
…on — both mappers, shared mechanism

Supersedes the snapshot approach with its inverse, and extends the fix to
the plain LLM_CHAT_COMPLETE mapper, which has the identical bug (its
getHistory() walks prior turns and f-string substitutes message text —
all imperative, all clobbered by retry's template re-resolution).

Retry and rerun do not re-run task mappers: they copy the task and
re-resolve the carried WorkflowTask definition's inputParameters over it,
overwriting mapper-assembled input with the static template. Instead of
snapshotting the assembled messages INTO a task-owned definition copy —
which round-trips conversation content through the template engine on
retry (text containing ${...} would be re-interpreted) and doubles the
task payload — DETACH the assembled keys from the task-owned copy:
re-resolution then never produces those keys, and the copied attempt's
own inputData, the exact conversation the first attempt used, survives
retry and rerun untouched.

The shared helper lives in AIModelTaskMapper and is called by both chat
mappers; the deep copy protects the cached WorkflowDef's shared instance,
and new schedules (including DO_WHILE iterations) map from the
definition's own instance, so they are unaffected.

Tests: detach semantics, retry simulation (replicating
taskToBeRescheduled's copy+putAll) preserving the conversation, a
negative control documenting the pre-fix clobber, template-pattern
content surviving verbatim, shared-definition immutability, no-op and
null paths. ai+agentspan suites: 1301 tests green.
@ling-senpeng13
ling-senpeng13 force-pushed the fix/3876-mapper-snapshots-messages branch from 82f6038 to 4a401f4 Compare August 6, 2026 19:06
@ling-senpeng13 ling-senpeng13 changed the title fix(ai): a retried LLM chat task keeps its conversation history fix(ai): a retried LLM chat task should keep its conversation history Aug 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants