feat(models,analytics): GPT-6 and Claude 5.5 families, prompts tuned for both

[laithalsaadoon]

Summary

• The registry moves to GPT-6 (Luna, 6.1 Sol, Astra) and Claude 5.5 (Haiku, Sonnet, Opus), with list pricing, the long-context price steps, and a measured chars-per-token ratio per family.
• The anthropic family is runnable through a new AnthropicBedrockProvider (output_config.format on invoke_model), selected with ATIF_SQL_LLM_FAMILY=anthropic. The default family stays openai.
• The four classifier prompts are rewritten against the current OpenAI and Anthropic prompting guides: one provider-neutral text, content first in id-fenced tags, request last, JSON examples, and a think-first line on the three judgment prompts.
• Tool-result bodies stay out of the transcripts sent to Bedrock by default. Each result renders as a size stub, tool calls keep their arguments, and ATIF_SQL_SESSION_TEXT_TOOL_RESULT_BODIES=true restores the bodies.

Evidence

• mise run check passes: 2661 tests passed, 1 skipped, 1 xfailed.
• scripts/dev/smoke_llm_models.py returns a parsed structured output from all six model ids in us-east-1.
• An A/B eval over a 19-session, 63-message sample from a local corpus compared the old and new prompts across both families (about 650 classifier calls, 0 refusals, 0 invented turn ids). Opus 5.5 and GPT-6 Astra judged every emitted row and every disputed label. The sample is small, so treat the precision figures as direction, not rates.

Pipeline	Arm	Cost	Result
friction	old prompt, GPT-5.6 Luna (baseline)	$0.039	10 of 15 disputed labels match both judges
friction	new prompt, GPT-6 Luna	$0.017	13 of 15
friction	new prompt, Haiku 5.5	$0.025	9 of 15 (old prompt: 3 of 15)
classify	old prompt, GPT-5.6 Luna (baseline)	$0.077	5 of 7
classify	new prompt, GPT-6 Luna	$0.039	5 of 7
conflicts	old prompt, GPT-5.6 Sol (baseline)	$1.79	7 rows, 2 valid, 3 invalid, 2 contested
conflicts	new prompt, GPT-6 Astra	$4.32	4 rows, 2 valid, 0 invalid, 2 contested
conflicts	new prompt, GPT-6.1 Sol	$0.86	3 rows, 2 valid, 0 invalid, 1 contested
conflicts	new prompt, Opus 5.5	$2.53	3 rows, 2 valid, 0 invalid, 1 contested
perceived	old prompt, GPT-5.6 Terra (baseline)	$0.90	6 rows, 1 valid, 3 invalid, 2 contested
perceived	new prompt, GPT-6.1 Sol	$0.91	6 rows, 2 valid, 1 invalid, 3 contested
perceived	new prompt, Sonnet 5.5	$1.33	2 rows, 1 valid, 1 invalid

The perceived anchoring rule changed after the first judge pass: the two judges disagreed on 7 of 9 rows, mostly over which user turn an acknowledged_mistake belongs to, because rule 2 and its own example pointed at different turns. With the rule fixed, they agree on 8 of 11.

Tool results

Across three local corpora (537 interactive sessions sampled), tool results were 88-91% of rendered transcript characters, and 210 of the 537 sessions hit the 800K-character cap, so the model never saw how they ended. In the 212-session Claude Code corpus the mean rendered session drops from about 1.09M characters to 127K with stubs.

The same 19-session eval, re-run with stubs in place of result bodies:

Pipeline	Model	Input tokens, full → stub	Result
classify	GPT-6 Luna	366K → 146K	17 of 19 labels match the full run; a repeat of the identical full run matched 15 of 19
classify	Haiku 5.5	551K → 226K	18-19 of 19 match; the repeat matched 18 of 19
conflicts	GPT-6.1 Sol	405K → 185K	the same 3 rows as the full run, once the prompt says what a stub line is (2 rows without that line)
conflicts	Opus 5.5	614K → 289K	2 rows, both judged valid (full run: 2 valid, 1 contested)
perceived	GPT-6.1 Sol	419K → 199K	5 of the full run's 6 rows; the dropped row was the one judged invalid
perceived	Sonnet 5.5	634K → 308K	3 rows: the full run's 2 plus 1 new row judged valid

The sample was drawn from sessions of 8K-120K characters, so it understates the token cut on real corpora and cannot show the truncation benefit.

Notes

• Claude 5.5 spends about 1.5x GPT-6's input tokens on the same transcript (2.2 vs 3.3 characters per token), so the same pipeline costs more on Claude than the list prices suggest. The dry-run plans now use each family's ratio.
• RefusalError stays terminal. In the eval, one judge call on Opus 5.5 returned stop_reason: refusal on a security-themed user message, so an anthropic run should expect a few refusal rows on coding transcripts that discuss security work.
• No size assignment changes in this PR. On this sample, GPT-6.1 Sol matched Astra on conflicts at about a fifth of the cost, which makes ATIF_SQL_LLM_SIZE_CONFLICTS=medium worth a larger trial.