Skip to content

fix: enforce BudgetGuard for pickled-iac / pickled-schema CLI drafters - #30

Merged
bartrosa merged 2 commits into
mainfrom
cursor/critical-correctness-bugs-4aac
Jun 9, 2026
Merged

bartrosa merged 2 commits into
mainfrom
cursor/critical-correctness-bugs-4aac

Conversation

@cursor

@cursor cursor Bot commented Jun 5, 2026

Copy link
Copy Markdown

Bug & impact

pickled-iac draft and pickled-schema draft silently bypassed the user-configured budget.max_cost_usd cap (and disk cache).

Both packages' CLI helper _build_llm_client constructed the LLM client by calling pickled_core.llm.factory.build_client directly, rather than going through pickled_core.llm.bootstrap.build_default_client. Only the latter installs the BudgetGuard via _maybe_install_budget.

Concrete trigger scenario: a user adds to pickled.config.yaml

budget:
  max_cost_usd: "0.50"

(or sets PICKLED_MAX_COST_USD=0.50) expecting any LLM-driven drafter to abort once the cap is reached. For every other CLI / MCP entrypoint (pickled-bdd, pickled-data, pickled-diff, every MCP server) the guard is installed and enforced. But pickled-iac draft and pickled-schema draft never installed it, so the user's cap was a no-op. Both drafters call the LLM with up to 3 validation retries, meaning a configured $0.50 cap could be overrun several-fold per invocation before the user noticed — a real cost-safety hole, not a theoretical concern.

Reproduction before the fix:

import os, tempfile, pathlib
os.environ["TEST_API_KEY"] = "fake"
tmp = pathlib.Path(tempfile.mkdtemp())
(tmp / "pickled.config.yaml").write_text(
    'providers:\n  anthropic:\n    type: anthropic\n    default_model: c\n    api_key_env: TEST_API_KEY\n'
    'budget:\n  max_cost_usd: "0.50"\ncache:\n  mode: off\n'
)
os.chdir(tmp)
from pickled_core.llm.budget_context import active_budget_guard
from pickled_iac.cli import _build_llm_client
_build_llm_client()
print(active_budget_guard())  # → None  (BUG: cap silently ignored)

After the fix the same snippet prints a <BudgetGuard …> instance with the configured cap.

Root cause

_build_llm_client in pickled-iac/cli.py and pickled-schema/cli.py reimplemented the factory env shortcut and provider resolution from scratch, calling build_client(provider, config=load_config()) directly. That code path never invokes _maybe_install_budget (and never builds the LLMCache), so:

  • budget.max_cost_usd from yaml is ignored.
  • PICKLED_MAX_COST_USD env override is ignored.
  • The disk cache is never wired through to the client (extra cost on top).

Every sibling package (pickled-bdd, pickled-data, pickled-diff) and both packages' own MCP CLIs go through build_default_client(factory_env=...), which correctly installs the guard.

Fix

Route the CLI helpers through build_default_client(factory_env=PICKLED_{IAC,SCHEMA}_LLM_FACTORY). This keeps the test-only *_LLM_FACTORY shortcut working, restores cache wiring, and — most importantly — installs the BudgetGuard whenever a cap is configured. Net effect on each CLI module is -18 lines of bespoke bootstrapping replaced by one call.

Validation

  • Added packages/pickled-iac/tests/test_cli_budget_bootstrap.py and packages/pickled-schema/tests/test_cli_budget_bootstrap.py with three pinned cases each:
    1. yaml-configured cap installs the guard with the right Decimal value;
    2. PICKLED_MAX_COST_USD env var overrides yaml and installs the guard;
    3. PICKLED_{IAC,SCHEMA}_LLM_FACTORY shortcut still returns a fake CannedLLMClient (backward-compat).
  • Full workspace test suite: uv run pytest → 458 passed, 5 skipped (up from 452 / 5; 6 new tests, 0 regressions).
  • Manually re-ran the reproduction snippet above to confirm active_budget_guard() now returns a non-None guard with max_cost_usd == Decimal("0.50").

Two commits — one per affected package — each containing its fix + regression tests.

Open in Web View Automation 

cursoragent and others added 2 commits June 5, 2026 22:14
The CLI helper bypassed pickled_core.llm.bootstrap.build_default_client
and called build_client directly. As a side effect, _maybe_install_budget
never ran, so a user-configured budget.max_cost_usd cap (in
pickled.config.yaml or the PICKLED_MAX_COST_USD env var) was silently
ignored by 'pickled-iac draft'. IaCDrafter retries the LLM up to 3 times
per invocation, so a configured $0.50 cap could be overrun several-fold
before the user noticed.

Route the CLI helper through build_default_client (factory_env=
PICKLED_IAC_LLM_FACTORY), matching pickled-bdd / pickled-data /
pickled-diff and the MCP CLI of this same package. This also enables the
disk-backed response cache when configured.

Adds three regression tests pinning that the guard is installed from
both yaml and the env override, and that the test-only LLM factory
shortcut still works.

Co-authored-by: Bartłomiej Rosa <bartrosa@users.noreply.github.com>
Same bug pattern as the prior pickled-iac commit: the CLI _build_llm_client
called pickled_core.llm.factory.build_client directly, which skipped
_maybe_install_budget. A configured budget.max_cost_usd cap (yaml or
PICKLED_MAX_COST_USD) was silently ignored by 'pickled-schema draft'.
OpenAPIDrafter retries up to 3 times per invocation, so the bypass had
the same blast radius.

Route through build_default_client (factory_env=PICKLED_SCHEMA_LLM_FACTORY)
to match pickled-bdd / pickled-data / pickled-diff and this package's own
MCP CLI. Disk cache is now also honored when configured.

Adds three regression tests pinning guard installation and the test-only
factory shortcut.

Co-authored-by: Bartłomiej Rosa <bartrosa@users.noreply.github.com>
@bartrosa
bartrosa marked this pull request as ready for review June 9, 2026 15:15
@bartrosa
bartrosa merged commit b9b6b92 into main Jun 9, 2026
0 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants