test(flash): build the turn-index runner without provider credentials - #91
Closed
basil-k-aji-dev wants to merge 1 commit into
Closed
basil-k-aji-dev wants to merge 1 commit into
basil-k-aji-dev wants to merge 1 commit into
Conversation
`tests/unit/agents/flash/test_turn_index_snapshot.py` failed 4 of its 5 tests without GOOGLE_API_KEY. `FlashRunner.__init__` constructs a `VisualStepSummarizer`, whose constructor resolves a real model client and raises `ValidationError: API key required for Gemini Developer API`. The `_runner` helper already discards the summarizer with `runner.summarizer = None`, but that runs after construction has raised, so it never takes effect. Patch the class for the construction itself. These tests assert index resolution across a turn and never summarize anything, so nothing they cover is mocked away. Closes google#31.
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #31.
Problem
tests/unit/agents/flash/test_turn_index_snapshot.pyfails 4 of its 5 tests withoutGOOGLE_API_KEY, contrary to the deterministic-suite contract inCONTRIBUTING.md:FlashRunner.__init__constructs aVisualStepSummarizer, whose constructor resolves a real model client. The_runnerhelper already intends to discard it:but that assignment runs after construction, so it never takes effect.
Change
Patch
VisualStepSummarizerfor the construction itself, in thewithblock that already patchesget_driver:Patching the whole class is safe here precisely because the helper was already throwing the instance away: these tests assert index resolution across a turn and never summarize anything, so nothing under test is mocked out.
Test-only; no production code is touched.
Relationship to #41
#41 covers the same root cause in four other Flash modules and I've sent a fix for it separately (#86). That one introduces a
stub_summarizer_model_factoriesfixture intests/unit/agents/conftest.py, which would also cover this file once merged.I kept this PR self-contained instead of depending on it, so the two can land in either order or independently. If #86 lands first and you'd rather this file used the shared fixture too, say so and I'll rebase it onto that.
Verification
mainuv run pytest -q tests/unit/agents/flash/test_turn_index_snapshot.pyuv run ruff check tests/uv run ruff format --checkon the touched fileRun with
GOOGLE_API_KEY,GEMINI_API_KEYandOPENAI_API_KEYall unset.make typecheckcould not be run in my environment — thepyrightwheel downloads a Node runtime on first use and my sandbox blocks it. The touched file is not inpyright-core.json.