Repository navigation
feat: per-model image limits in model capabilities - #244
Draft
erzaehlsalex wants to merge 8 commits into
Draft
erzaehlsalex wants to merge 8 commits into
erzaehlsalex wants to merge 8 commits into
Conversation
Add optional capabilities.input with maxImages and maxMedia, present only when a provider has a verified per-request limit for the model id on the host it serves. Mirrors output.maxTokens and the ImageCountExceededError / MediaCountExceededError fields. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
600 for models with a 1M-token context window, 100 for 200k models, per Anthropic's vision and context-windows docs. Unknown ids report no limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Known Gemini models report 3,600 images on the Gemini API and 3,000 on Vertex AI. getGoogleModelCapabilities and createGoogleGenAIProvider take a platform; google-vertex passes 'vertex' and its getGoogleModelCapabilities export returns Vertex limits. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Image-capable registry models report 1,500 images per request. Providers that fall back to the OpenAI registry on another host (Azure OpenAI, OpenAI-compatible servers, Omnifact) drop it via omitOpenAIInputLimits, as do unknown ids. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Mistral's vision families report 8 images per request (vision docs FAQ, matching the API rejection). Unknown and self-hosted ids keep vision input without a limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both providers document no image count limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe capabilities.input in the types reference, add an image limits section to the multi-modal guide, and note Vertex AI's limit on the provider page. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Preview approval requiredThis pull request comes from a fork, so Mintlify won't build a preview until someone with write access approves commit |
erzaehlsalex
marked this pull request as draft
October 27, 2026 20:55
Google GenAI chat models now pick Vertex AI's limit when the client was built with `vertexai: true`. OpenAI's 1,500-image limit is kept only when the client's baseURL is api.openai.com or a regional OpenAI host, instead of being inferred from whether a registry was passed. createOpenAI drops its duplicate registry, so fine-tuned ids resolve to their base model. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
erzaehlsalex
commented
Oct 28, 2026
| * context-windows docs). Anthropic documents the same split on Vertex AI and | ||
| * Bedrock. | ||
| */ | ||
| const MAX_MEDIA_PER_REQUEST: Record<string, number> = { |
Contributor
Author
There was a problem hiding this comment.
We can add 600 and 100 into constants
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
ModelCapabilities.inputwithmaxImagesand an optionalmaxMedia. Each chat model now reports the most images it accepts in one request, so apps can trim or batch images before sending instead of finding out fromImageCountExceededError(#229).Like
output.maxTokens, the field is present only when the provider has a verified limit for the model id on the host it serves. If it is missing, the limit is unknown. It does not mean there is no limit.maxImagesmaxMediais set too… > 100errorsDesign notes
ImageCountExceededErrorhandling stays as it is.getGoogleModelCapabilities(modelId, platform)and aplatformfactory option oncreateGoogleGenAIProvider.@core-ai/google-vertexpasses'vertex', and itsgetGoogleModelCapabilitiesexport now returns Vertex AI's limits.omitOpenAIInputLimits. This covers Azure v1, openai-compat and Omnifact. Azure classic applies it explicitly.Not covered
Testing
npm run build,turbo lint check-types test,npm run docs:check-links🤖 Generated with Claude Code