Repository navigation
feat(models): add native Google GenAI generateContent dialect - #147
Open
jack-scitix-ai wants to merge 11 commits into
Open
jack-scitix-ai wants to merge 11 commits into
jack-scitix-ai wants to merge 11 commits into
Conversation
Allow dialect evidence to verify the final PreparedRequest rather than only JSON body fields. This lets operation-selected protocols prove stream and endpoint control flow without inventing a body parameter.
Keep Google generateContent finish reasons provider-verbatim while ensuring MAX_TOKENS contributes to both anomaly detection and truncated-rollout reporting.
Bind the native generateContent protocol with audited request lowering, strict streaming and non-streaming lifting, capability declarations, deterministic seed support, and provider-native continuation evidence.
Reject lossy chat and stream shapes before they can be reinterpreted, preserve terminal usage and citation evidence, and make multi-candidate and continuation behavior explicit. Snapshot the request used for response lifting and strengthen regression tests around final-wire provenance, stable call identities, replay, and required output channels.
Surface prompt blocking reasons, preserve compatible streamed text deltas, and validate hosted-tool call state in opaque continuation history. Add fail-loud regression coverage for contradictory metadata, HTTP error status handling, and malformed or forged provider responses.
Treat each streamed candidate finishReason as terminal while continuing to allow response-level usage-only tail events. Add regression coverage and mutation verification for the terminal-state guard.
Add discriminating coverage for request rejection, opaque replay, malformed provider responses, streaming state, wire verifiers, prepared-request integrity, and final request evidence.\n\nExercise both high-level I/O boundaries and lower-level verifier contracts so guard removal or mutation fails loudly.
ethan-scitix
requested changes
Oct 14, 2026
ethan-scitix
left a comment
Collaborator
There was a problem hiding this comment.
Request changes. The design is solid and the tests are strong (unit + integration pass; ruff and ty are clean). Two accounting bugs put wrong numbers in reports without raising, and coverage is below the core/ gate.
Blocking
- Bug: wrong usage semantics (
google_genai.py:2039).candidatesTokenCountexcludes thoughts (totalTokenCount= prompt + toolUsePrompt + thoughts + candidates). ButUsageStats.reasoning_tokensmust be a breakdown ofoutput_tokens. With prompt=20, candidates=150, thoughts=3000, the profiler reports a 2000% reasoning share, and every thinking call trips the total-mismatch counter. Fix:output_tokens = candidates + thoughtsandinput_tokens = prompt + toolUsePrompt. - Bug: Google filter stops aren't counted as truncation (
meta.py:24). The set includescontent_filter, but not Google's equivalents:SAFETY,RECITATION,BLOCKLIST,PROHIBITED_CONTENT,SPII,IMAGE_SAFETY, … So a filtered Gemini rollout readsn_truncated = 0. Add them, or document why they're left out. - Coverage:
google_genai.pyis at 90%, below the 95% required forcore/. Most uncovered lines are the fail-loudraisebranches (e.g.:878–942,:1383–1447,:1743–1765).
Non-blocking
- Question:
CONTINUATIONis counted as truncation andcontinuationTokenis silently dropped. That's fine for scoring, but please note it ininfer.mdand add a comment in the code. - Nit:
cast(Any, group)andint(cast(int, minimum))indialect.pyaren't needed; ty passes without them. TheAssertionErroratdialect.py:~429is unreachable. - Nit: the body of commit
6f81a283contains a literal\n\n. - Follow-up: model-family limits are matched by regex on the model name, so gateway aliases skip them. Consider moving them into model profiles, and splitting the 3.8k-line module.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Type
Summary
google_genaidialect for the Gemini Developer API's nativegenerateContentprotocol, with audited request lowering and strict streaming/non-streaming response lifting.v1andv1betawire contracts, including version-specific operations, function schemas, final-request validation, and distinct runtime provenance.PreparedRequest, including an exact audit of requested hosted tools.MAX_TOKENSandCONTINUATIONas truncation while preserving partial output and provider-verbatim finish reasons.Related Issues
Refs #25
Test Plan
Automated
pdm run ruff check && pdm run ruff format --check)pdm run ty check sieval/)pdm run python -m pytest tests/unit tests/integration -q): 8420 passed, 84 skippedChecklist
Required (all PRs)
type(scope): description)AI-Generated Code - <model> (<provider>)in module docstringcore/