fix: enforce BudgetGuard for pickled-iac / pickled-schema CLI drafters - #30
Merged
Merged
Conversation
The CLI helper bypassed pickled_core.llm.bootstrap.build_default_client and called build_client directly. As a side effect, _maybe_install_budget never ran, so a user-configured budget.max_cost_usd cap (in pickled.config.yaml or the PICKLED_MAX_COST_USD env var) was silently ignored by 'pickled-iac draft'. IaCDrafter retries the LLM up to 3 times per invocation, so a configured $0.50 cap could be overrun several-fold before the user noticed. Route the CLI helper through build_default_client (factory_env= PICKLED_IAC_LLM_FACTORY), matching pickled-bdd / pickled-data / pickled-diff and the MCP CLI of this same package. This also enables the disk-backed response cache when configured. Adds three regression tests pinning that the guard is installed from both yaml and the env override, and that the test-only LLM factory shortcut still works. Co-authored-by: Bartłomiej Rosa <bartrosa@users.noreply.github.com>
Same bug pattern as the prior pickled-iac commit: the CLI _build_llm_client called pickled_core.llm.factory.build_client directly, which skipped _maybe_install_budget. A configured budget.max_cost_usd cap (yaml or PICKLED_MAX_COST_USD) was silently ignored by 'pickled-schema draft'. OpenAPIDrafter retries up to 3 times per invocation, so the bypass had the same blast radius. Route through build_default_client (factory_env=PICKLED_SCHEMA_LLM_FACTORY) to match pickled-bdd / pickled-data / pickled-diff and this package's own MCP CLI. Disk cache is now also honored when configured. Adds three regression tests pinning guard installation and the test-only factory shortcut. Co-authored-by: Bartłomiej Rosa <bartrosa@users.noreply.github.com>
bartrosa
marked this pull request as ready for review
June 9, 2026 15:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug & impact
pickled-iac draftandpickled-schema draftsilently bypassed the user-configuredbudget.max_cost_usdcap (and disk cache).Both packages' CLI helper
_build_llm_clientconstructed the LLM client by callingpickled_core.llm.factory.build_clientdirectly, rather than going throughpickled_core.llm.bootstrap.build_default_client. Only the latter installs theBudgetGuardvia_maybe_install_budget.Concrete trigger scenario: a user adds to
pickled.config.yaml(or sets
PICKLED_MAX_COST_USD=0.50) expecting any LLM-driven drafter to abort once the cap is reached. For every other CLI / MCP entrypoint (pickled-bdd,pickled-data,pickled-diff, every MCP server) the guard is installed and enforced. Butpickled-iac draftandpickled-schema draftnever installed it, so the user's cap was a no-op. Both drafters call the LLM with up to 3 validation retries, meaning a configured $0.50 cap could be overrun several-fold per invocation before the user noticed — a real cost-safety hole, not a theoretical concern.Reproduction before the fix:
After the fix the same snippet prints a
<BudgetGuard …>instance with the configured cap.Root cause
_build_llm_clientinpickled-iac/cli.pyandpickled-schema/cli.pyreimplemented the factory env shortcut and provider resolution from scratch, callingbuild_client(provider, config=load_config())directly. That code path never invokes_maybe_install_budget(and never builds theLLMCache), so:budget.max_cost_usdfrom yaml is ignored.PICKLED_MAX_COST_USDenv override is ignored.Every sibling package (
pickled-bdd,pickled-data,pickled-diff) and both packages' own MCP CLIs go throughbuild_default_client(factory_env=...), which correctly installs the guard.Fix
Route the CLI helpers through
build_default_client(factory_env=PICKLED_{IAC,SCHEMA}_LLM_FACTORY). This keeps the test-only*_LLM_FACTORYshortcut working, restores cache wiring, and — most importantly — installs theBudgetGuardwhenever a cap is configured. Net effect on each CLI module is-18lines of bespoke bootstrapping replaced by one call.Validation
packages/pickled-iac/tests/test_cli_budget_bootstrap.pyandpackages/pickled-schema/tests/test_cli_budget_bootstrap.pywith three pinned cases each:Decimalvalue;PICKLED_MAX_COST_USDenv var overrides yaml and installs the guard;PICKLED_{IAC,SCHEMA}_LLM_FACTORYshortcut still returns a fakeCannedLLMClient(backward-compat).uv run pytest→ 458 passed, 5 skipped (up from 452 / 5; 6 new tests, 0 regressions).active_budget_guard()now returns a non-Noneguard withmax_cost_usd == Decimal("0.50").Two commits — one per affected package — each containing its fix + regression tests.