Skip to content

Conversation

@kiranandcode
Copy link
Contributor

This PR reimplements RetryLLMHandler on top of the updated internal API.

When decoding fails, this handler:
1. Adds an error message to the conversation
2. Retries the completion with the error feedback
3. On success, returns the clean final message (without error messages)

The code is a bit long because call_assistant does several logical steps, but otherwise encapsulates all the relevant logic to a single place.

Copy link
Contributor

@eb8680 eb8680 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are at least a couple other points of failure that we'd want to cover with retrying logic:

  • Server-side LLM API/network failures - this should be handleable within litellm (#494)
  • Runtime tool call failure, i.e. a tool raises an unhandled exception - we'd need to separately handle call_tool and have it return an appropriate failure Message

@kiranandcode
Copy link
Contributor Author

@eb8680 updated!

Copy link
Contributor

@eb8680 eb8680 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might also want to include the traceback in the feedback message. I think @datvo06 had done that at some point.

@eb8680 eb8680 linked an issue Jan 31, 2026 that may be closed by this pull request
Copy link
Contributor

@eb8680 eb8680 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to merge once tests pass. Nice work! This is much more robust and comprehensive than what we had before, and it should work really nicely in conjunction with the updated synthesis machinery #519 #520

@kiranandcode
Copy link
Contributor Author

Awesome!

@eb8680 eb8680 merged commit 5d44086 into master Jan 31, 2026
6 checks passed
@eb8680 eb8680 deleted the kg-retry-handler-new branch January 31, 2026 04:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

2 participants