Don't replay image attachments to models without image input - #98
Open
hmk wants to merge 1 commit into
Open
Conversation
History replay converted image attachments to image content parts unconditionally, so an image sent to (or a chat later switched to) a model without image input made every request 400 with the provider's "does not support image input" error -- permanently wedging the chat, since replay re-sends history on every turn. Extract the attachment-to-content-parts conversion into chat-attachment-parts.ts (following chat-attachment-validation.ts / chat-attachment-pdf.ts) and gate the image part on the model's declared input modalities, degrading to the existing omitted-file text marker otherwise. PDFs take the same gate, since they ride an image content part (see chat-attachment-pdf.ts). Because replay rebuilds from the log each turn, already-wedged chats recover on their next message. Claude tracked down the replay path and wrote the fix and tests.
|
All contributors have signed the CLA ✍️ ✅ |
Author
|
I have read the CLA Document and I hereby sign the CLA |
Author
|
for posterity: this PR was built with the assistance of Claude Code and Fable 5 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sending an image attachment to a model without image input fails with the provider's 400 — and because history replay rebuilds the model request from the chat log on every turn, the image is re-sent on every subsequent message, permanently wedging the chat.
This gates the image content part on the model's declared input modalities (
model.input), degrading to the existing[Attached file … omitted]text marker instead — the same path other unsupported attachment types already take. PDFs get the same gate since they ride an image content part. The conversion is extracted intochat-attachment-parts.tswith tests, following the existingchat-attachment-validation.ts/chat-attachment-pdf.tspattern.Since replay rebuilds from the log each turn, chats that are already stuck recover on their next message.
Known limitation: custom/uncatalogued models default to
input: ["text", "image"], so a text-only custom model can still hit the 400. Discussion of how to handle that is on the issue.Fixes #97