feat: Capture Sonar search-context fees for exact pricing (MET-13)

[sam-bretz]

Summary

Part of MET-13 (exact Sonar pricing). Perplexity bills Sonar a per-request fee that depends on the search context size. Until now the catalog marked Sonar uncaptured_fees, so every Sonar call was a partial, lower-bound price.

• CatalogSnapshot.cost(...) and price_deployment(...) take search_context_size (low / medium / high, trimmed and case-folded).
• New price rule search_context_fee_per_request: {low, medium, high} (USD per request). With a valid size, the fee is added on top of token charges and the call is priced.
• With no valid size, the size is never inferred (not even Perplexity's API default of low). No fee is added and the call stays partial with reason search_context_size_unknown. If the size is valid but that tier has no fee in the table, the call is partial with search_context_fee_unavailable.
• The loader rejects a malformed rule (non-mapping, unknown tier, negative or non-numeric fee).
• perplexity/sonar on perplexity-api: uncaptured_fees is replaced by {low: 0.005, medium: 0.008, high: 0.012}.
• OSS server project_row passes the row's search_context_size through to pricing. The schema is unchanged.
• metergraph-core 0.2.24, catalog version 2026-09-16. docs/prices.md documents the rule and both reasons.

Price source

https://docs.perplexity.ai/docs/getting-started/pricing, checked 2026-09-16. Under "Request Pricing by Search Context Size", per 1000 requests: Sonar $5 / $8 / $12 (Sonar Pro and Sonar Reasoning Pro $6 / $10 / $14, which are not in the catalog yet). The page lists Low as the default. Response usage.search_context_size reports the size that was applied, per https://docs.perplexity.ai/api-reference/chat-completions-post.

The effective window of the existing Sonar entry is kept. Only its rule changes, because the token rates are the same.

How it was tested

uv venv .venv && uv pip install -e "core[dev]" -e "server[dev]"
.venv/bin/pytest core/tests server/tests -q          -> 296 passed, 8 skipped (Postgres-backed tests skip without a DSN)
.venv/bin/python core/tests/package/verify_artifacts.py         -> OK metergraph_core-0.2.24 sdist + wheel
.venv/bin/python server/tests/package/verify_server_artifacts.py -> OK

New tests: one per tier (tokens plus fee, priced, no reasons); missing or invalid size is partial + search_context_size_unknown; case and whitespace normalization; fee is independent of batch; a model without the rule ignores the size; price_deployment applies the fee; a tier missing from the table is partial, never free; loader rejects invalid rule shapes; server projection is priced with a size and partial without.

Acceptance criteria

• Per-request search-context fee by tier in the catalog, with a cited source
• Fee applied only when the context size is known
• Unknown size stays partial / lower bound, and no tier is inferred
• Tests for the supported tiers, missing metadata and final cost calculation
• Release metergraph-core 0.2.24 (needed before the metergraph-internal PR can take the pin)

Not done

• Sonar Pro, Sonar Reasoning Pro and Sonar Deep Research are not in the catalog. This PR does not add them.
• Pro Search (web_search_options.search_type = pro) fees for Sonar Pro are not modelled.
• The OSS server prices with the size but does not store it in a column.

Related PRs (MET-13)

Listed on the Linear issue; see the cross-link comment below.

Linear: https://linear.app/metergraph/issue/MET-13/capture-sonar-search-context-fees-for-exact-pricing

🤖 Generated with Claude Code

[sam-bretz]

Related PRs (MET-13)

• Core pricing and catalog: feat: Capture Sonar search-context fees for exact pricing (MET-13) #49 (release metergraph-core 0.2.24 first)
• Ingest, persistence and reporting: https://github.com/PioneerSquareLabs/metergraph-internal/pull/522 (blocked on the core release and lock refresh)
• SDKs (Python and TypeScript capture): feat: Capture Sonar search-context fees for exact pricing (MET-13) metergraphsdk#61
• Relay (Portkey export): branch sam/met-13-portkey-search-context-size on VasiliyRad/metergraphrelay; PR not opened yet: VasiliyRad/metergraphrelay@main...sam/met-13-portkey-search-context-size

Shared contract: native row field search_context_size, OTLP attribute metergraph.search_context_size, values low / medium / high, never inferred when absent.

Linear: https://linear.app/metergraph/issue/MET-13/capture-sonar-search-context-fees-for-exact-pricing

[sam-bretz]

Codex review

codex-run.sh review --base origin/main on 5f68ddc. Both findings were checked against the cited lines, confirmed, and fixed in f3c7b3d.

Confirmed and fixed

• [P2] LoadedCatalog.price did not accept search_context_size (core/src/metergraph_core/loader.py, price). This was true: the public load_catalog().price(...) wrapper forwards to price_deployment with a fixed keyword list, so passing the size raised TypeError. It now accepts and forwards the keyword. New test test_loaded_catalog_price_forwards_search_context_size: medium gives priced 2.00800000; no size gives partial 2.00000000 ("search_context_size_unknown",).
• [P1] Server kept metergraph-core>=0.2,<0.3 (server/src/metergraph_server/ingest.py:197). This was true: with core 0.2.23 installed, every ingest row would raise TypeError on the new keyword. server/pyproject.toml now requires metergraph-core>=0.2.24,<0.3, and the artifact release gate's EXPECTED_CORE_SPECIFIERS is updated to match.

Also fixed before the review (self-review, commit 5f68ddc): a tier missing from a fee table silently added no fee and still reported priced. It now reports partial with search_context_fee_unavailable (test_search_context_tier_missing_from_fee_table_is_partial_not_free).

After the fixes: .venv/bin/pytest core/tests server/tests -q → 297 passed, 8 skipped. verify_artifacts.py and verify_server_artifacts.py → OK (core 0.2.24).

Rejected: none.

[sam-bretz]

Prove-it report

Verdict

Proven. With a size, a Sonar call is priced exactly (tokens plus the tier fee) through the real OSS server ingest into PostgreSQL. Without a valid size it stays partial and no tier is inferred. The same checks against origin/main show every Sonar row as partial, and the new tests fail there.

Claim	Evidence	Status
1. Tier fee added per request: low 0.005, medium 0.008, high 0.012	E2E: OSS server /v1/ingest into Postgres 17, plus unit tests	✅ Proven
2. Missing or invalid size stays partial at the token-only cost (never inferred)	E2E rows sonar-absent and sonar-invalid, plus unit tests	✅ Proven
3. Size is normalized (" Medium ", "HIGH")	E2E rows sonar-medium and sonar-high	✅ Proven
4. price_deployment and LoadedCatalog.price apply the same fee	Unit tests	✅ Proven
5. A tier missing from a fee table is partial, not free	Unit test (synthetic catalog)	✅ Proven
6. Malformed rule rejected at load	Unit tests	✅ Proven
7. Neighbouring pricing unchanged (other models, uncaptured_fees rule, artifacts)	Full core and server suites, artifact gates	✅ Proven

Evidence

§1-3 E2E: OSS server ingest into Postgres

Command: .venv/bin/python prove/oss_server_e2e.py postgresql://postgres:***@127.0.0.1:55413/mg_oss_prove (disposable postgres:17 container). The script posts 5 Sonar rows (provider: perplexity, 1M input + 1M output tokens) through TestClient(create_app()), then reads calls. Exit 0.

POST /v1/ingest 202 {'accepted': 5, 'ignored': 0}
('sonar-absent', 'partial', Decimal('2.00000000'))
('sonar-high', 'priced', Decimal('2.01200000'))
('sonar-invalid', 'partial', Decimal('2.00000000'))
('sonar-low', 'priced', Decimal('2.00500000'))
('sonar-medium', 'priced', Decimal('2.00800000'))

Negative control: the same script with PYTHONPATH set to an origin/main (eea0fc3) checkout of core/src and server/src:

('sonar-absent', 'partial', Decimal('2.00000000'))
('sonar-high', 'partial', Decimal('2.00000000'))
('sonar-invalid', 'partial', Decimal('2.00000000'))
('sonar-low', 'partial', Decimal('2.00000000'))
('sonar-medium', 'partial', Decimal('2.00000000'))

§4-7 Test suites

.venv/bin/pytest core/tests server/tests -q gives 297 passed, 8 skipped (the 8 skips are test_api.py, which needs MG_TEST_DATABASE_URL; the Postgres path is covered by the E2E above). core/tests/package/verify_artifacts.py reports OK: metergraph_core-0.2.24.tar.gz, metergraph_core-0.2.24-py3-none-any.whl verified. server/tests/package/verify_server_artifacts.py reports OK ... declares core and prices through metergraph_core-0.2.24.

Negative control: head's core/tests/test_catalog.py run against origin/main core source (-k "search_context or sonar") gives 13 failed, 93 deselected, for example FAILED test_price_deployment_applies_search_context_fee.

Tests used

• test_sonar_search_context_fee_is_added_per_request[low|medium|high] (core/tests/test_catalog.py, new, unit): the shipped Sonar entry at 1M/1M tokens costs 2.00 plus the fee, is priced, and has no reasons. Fails on base (unknown keyword, and Sonar is uncaptured_fees).
• test_sonar_without_valid_search_context_size_is_a_lower_bound (new, unit): None, "", "max", 3 give partial, 2.00, ("search_context_size_unknown",).
• test_sonar_search_context_size_normalizes_case_and_whitespace, test_search_context_fee_is_independent_of_batch, test_model_without_search_context_fee_rule_ignores_size, test_price_deployment_applies_search_context_fee (new, unit).
• test_search_context_tier_missing_from_fee_table_is_partial_not_free (new, unit): a table with only low gives high as partial with search_context_fee_unavailable.
• test_loaded_catalog_price_forwards_search_context_size (core/tests/test_loader.py, new, unit): the public wrapper gives priced 2.008 with a size and partial 2.00 without.
• test_search_context_fee_rule_rejects_invalid_shapes_or_values (core/tests/test_loader.py, new, unit): CatalogError on a non-mapping, an unknown tier, or a negative or non-numeric fee.
• test_projection_prices_sonar_search_context_size (server/tests/test_projection.py, new, unit): project_row gives priced 2.012 for high and partial 2.00 without a size.
• oss_server_e2e.py (scratch script, E2E, not committed): above.

Not covered

• test_api.py (Postgres API suite) was not run with MG_TEST_DATABASE_URL. The ingest path it would cover is exercised by the E2E script.
• Price correctness depends on Perplexity's published page (checked 2026-09-16). No billed invoice was compared.
• Sonar Pro, Sonar Reasoning Pro and Pro Search fees are not in the catalog.

[VasiliyRad]

[P2] The declared version is behind main. This PR bumps 0.2.23 → 0.2.24, but main is already at 0.2.28:

core/pyproject.toml on main:                    version = "0.2.28"
core/tests/package/verify_artifacts.py on main: EXPECTED_VERSION = "0.2.28"

Three releases landed since this branch was cut — 0.2.26 (Gemini cache-read correction), 0.2.27 (core-v0.2.27, the model spellings real traffic sends), and 0.2.28 on main but not yet tagged (DeepSeek's current rates and per-hour pricing).

Two consequences:

1. The release gate will fail. verify_artifacts.py asserts the built wheel's version equals EXPECTED_VERSION, so both need to move together, and the number has to be ahead of 0.2.28 rather than behind it.
2. data/prices.yaml will conflict. 0.2.26 through 0.2.28 all changed that file — 0.2.27 alone added 267 alias rows across 66 models. Rebasing before the version bump will be easier than after.

Worth knowing while rebasing: 0.2.28 added an off_peak_discount price rule, so the rules map now carries a second entry of its own shape. If the Sonar fee rule composes with other rules rather than replacing them, that is the one to check it against.