Skip to content

Tell the model how to look at a result it has not seen before - #36

Merged
CNSeniorious000 merged 1 commit into
mainfrom
ask-for-the-shape
Aug 31, 2026
Merged

CNSeniorious000 merged 1 commit into
mainfrom
ask-for-the-shape

Conversation

@CNSeniorious000

@CNSeniorious000 CNSeniorious000 commented Aug 31, 2026

Copy link
Copy Markdown
Owner

filter and aggregate in code and surface just the conclusion assumes the model knows which fields it wants. The first time through an unfamiliar tool it does not, and ending the cell on the whole value is then the rational move — you cannot project fields you have never seen. The block offers nothing cheaper, so that is what happens.

-…so filter and aggregate in code and surface just the conclusion.
+…so filter and aggregate in code and surface just the conclusion. Don't know the shape yet? Ask for the SHAPE, not the payload — `r.keys()`, `len(r["results"])`, `r["results"][0]` — then extract on the next line.

Two lines instead of one, and the first costs a few hundred characters instead of the whole answer.

Measured — 4/4 tasks, 3 arms, all complete

Carried with #27, #30 and #35 as one arm. gpt-5.6-terra, band 2K–20K, dump rate with median ratio in brackets.

task baseline #27+#30 + this and #35
private_lending_33d 80.5% (1.006) 50.0% (0.613) 13.3% (0.225)
hsa_medical_bill_liquidity_guard_30d 72.7% (1.180) 29.2% (0.039) 24.5% (0.013)
broadcast_exam_posture_breathing_32d 9.1% (0.302) 38.2% (0.479) 3.6% (0.022)
food_safety_dispute_33d 44.7% (0.345) 100.0% (1.018) 12.5% (0.069)
pooled 47.3% 43.7% (p = 0.51 vs baseline) 15.1% (z = 5.98)

4/4 tasks improved, against 2/4 for #27 + #30.

Why this clause specifically

The slice-bound histogram in #34. Under #27 + #30 the model stopped echoing raw results and started printing [:12000] slices — median bound 12000, 22 of 23 above 2000 — which is how broadcast_exam went from 9.1% to 38.2% and food_safety from 44.7% to 100%. With this clause in the arm, slice bounds collapse to a median of 300 on one task and 30 on another, nothing above 5000: macaron-v1-venti's regime (median 120, max 1200).

Giving the model a cheap first look is what removes the reason to print the payload. Telling it not to print the payload, without that, just changes which statement does the printing.

Note

Attribution between this and #35 is not separated — measured together, and the arm does not carry #34.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @CNSeniorious000, you've used your own review budget of 250,000 diff characters for the last 7 days.

You can request another review in 4 days and 17 hours by commenting @sourcery-ai review. Upgrade to get a review now.

@CNSeniorious000
CNSeniorious000 merged commit 7132137 into main Aug 31, 2026
15 checks passed
@CNSeniorious000
CNSeniorious000 deleted the ask-for-the-shape branch August 31, 2026 04:43
Repository owner deleted a comment from chatgpt-codex-connector Bot Sep 1, 2026
Repository owner deleted a comment from coderabbitai Bot Sep 1, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant