General gateway fixes from the Atlas integration branch - #38
Merged
Merged
Conversation
Responses input_image/input_video parts keep their detail and descriptor fields, and video payloads get the same opaque-media token estimate as images. Split from the Atlas integration branch (1e8d984) without its Atlas-specific recipe and docs.
The Atlas engine renders a chat template whose exact token count decides whether a long-context request fits, but the gateway had no way to ask for it: `/v1/tokenize` was not routed, so an exact-count client could only estimate. Add the route as a thin pass-through. The body is forwarded untouched apart from the upstream model name, including `chat_template_kwargs`, because rewriting it would change the count the endpoint exists to report. `prompt` and `messages` are both accepted, matching the backend's own contract. Missing input and unknown models are rejected as client errors before any upstream call so a bad request cannot occupy a backend slot. Atlas serves its OpenAI surface under `/v1` but token counting at the bare `/tokenize`, and an OpenAI-typed backend's baseUrl already ends in `/v1`. Joining the route to that base would request `/v1/tokenize` and the engine answers 404, so the handler resolves the backend origin explicitly and `upstreamUrl` now passes an already-absolute URL through instead of prepending the baseUrl to it. Tests cover the forwarded route and absolute resolution for baseUrls with and without `/v1`, upstream-model substitution, template kwargs preservation, input validation, and verbatim propagation of a backend rejection. Verified: node --check, prettier --check, tokenize route tests, security, model failover, server resilience, smoke, and a live request through the gateway to the Atlas engine.
A caller-supplied `response_format.json_schema.schema` was forwarded to the backend untouched. When it carried `type: "invalid"` the backend ignored the constraint it could not compile and answered 200 with ordinary text, so a client that asked for schema-constrained output silently got an unconstrained response and no error. Reject the request instead, before any backend call, via `assertCompilableSchema`. The check is deliberately conservative: only a `type` that is not one of the seven JSON Schema types is refused, at the root or at any place the specification puts a subschema (properties, items, combinators, definitions, `$defs`, `dependentSchemas`, and the conditional keywords). `$ref`, `enum`, `const`, `format` and unknown keywords pass through, and the failing path is named in the error. The same check now guards `lloom.outputSchema`, which reaches the same construction by a different route. Verified: node test/protocol.test.mjs, protocol-stream, security, responses-codex, model-failover and tokenize-route suites, plus prettier --check.
…re' into fix/atlas-general-fixes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Five commits from
atlas/installable-20260924that aren't Atlas-specific, split out so they land now. The Atlas recipe, backend and image pins stay on that branch.input_image/input_videoparts keep theirdetailand descriptor fields. Video payloads get the same opaque-media token estimate as images, so base64 video is no longer counted as prompt text. The Atlas recipe and docs parts of the original commit were dropped.lloom runtime-policysets explicit memory limits through validated CLI writes./tokenizeproxies exact token counts to the serving backend.Test plan
npm run test:unit(full chain, pass)node --test test/runtime-policy-config.test.mjs test/tokenize-route.test.mjsbin/lloom.mjsis already unformatted on main and was left as is.🤖 Generated with Claude Code