[spark-compete] fix(builder): Builder overlap probes report matched count without disclosing the 500-id sample cap

[4gjnbzb4zf-sudo]

{
"schema": "spark-compete-hotfix-v1",
"event": "spark-compete-first-event",
"submission_mode": "public_repo_pr",
"submission_target_url": "#1410",
"team": {
"name": "SparkThisUp",
"members": [
"ValHallaBuilder",
"Baz707",
"DanFireDash"
],
"github_accounts": [
"4gjnbzb4zf-sudo"
],
"llm_device_holder": "ValHallaBuilder",
"device_holder_github": "4gjnbzb4zf-sudo"
},
"target_repo": {
"id": "vibeforge1111/spark-cli",
"source": "https://github.com/vibeforge1111/spark-cli",
"owner_surface": "spark-cli"
},
"issue": {
"type": "bug",
"severity": "medium",
"title": "Builder overlap probes report matched count without disclosing the 500-id sample cap",
"actual_behavior": "system_map JSON shows checked_request_id_count up to len(input) (could be 10k), matched_builder_request_id_count from a 500-id sample \u2014 gap is invisible.",
"expected_behavior": "JSON surfaces probe_cap + sampled_*_count so operator sees the matched-count denominator is the probe sample.",
"repro_steps": [
"1. Call inspect_builder_request_id_overlap with >500 unique request_ids.",
"2. Read the resulting dict.",
"3. checked_request_id_count = full input, matched_builder_request_id_count = overlap on sample, no cap field. After fix: probe_cap and sampled_request_id_count present."
],
"affected_workflow": "Spark CLI spark os system-map operator diagnostic and the spawner-prd auto-trace overlap reporting that downstream operators rely on to decide whether builder is wired up correctly"
},
"evidence": {
"safe_links_only": true,
"before_after_proof": "Site \u2014 src/spark_cli/system_map.py:911-950 (inspect_builder_request_id_overlap, builds the JSON block operators read). Before: candidates = sorted(request_ids)[:500] runs without surfacing the cap, and matched_builder_request_id_count is reported alongside checked_request_id_count (full input size) \u2014 operator infers a false ground-truth overlap. After: introduce _BUILDER_OVERLAP_PROBE_CAP = 500 constant; add probe_cap + sampled_request_id_count fields so the operator knows the match-count denominator is the probe sample, not the full input. Same pattern mirrored in inspect_builder_trace_ref_overlap at line 977.",
"links": [
"https://github.com//pull/1410",
"https://github.com//pull/1410/files"
],
"forbidden": [
"raw secrets",
"raw logs",
"raw conversations",
"private chat IDs",
"session tokens",
"cookies",
"private repo maps",
"raw memory dumps",
"full compile JSON",
"scoring details"
]
},
"proposed_fix": {
"approach": "Add a module-level constant _BUILDER_OVERLAP_PROBE_CAP = 500 and surface probe_cap + sampled_request_id_count / sampled_trace_ref_count in both inspect_builder_request_id_overlap and inspect_builder_trace_ref_overlap. No query semantics change; only the JSON output gets two extra fields per probe so operators can tell when the matched count is from a sample. +15/-2 lines in src/spark_cli/system_map.py.",
"files_expected": [
"src/spark_cli/system_map.py"
],
"tests_or_smoke": "Smoke: run the affected code path in the repo and confirm before\u2192after behavior change. Build-clean: python3 -m py_compile src/spark_cli/system_map.py or npx tsc --noEmit --skipLibCheck src/spark_cli/system_map.py."
},
"pr": {
"url": "#1410",
"branch": "spark-compete/overlap-probe-cap-disclosure",
"title_prefix": "[spark-compete]",
"author_github": "4gjnbzb4zf-sudo",
"body_must_include": [
"packet",
"team",
"pr_author",
"repo",
"actual_behavior",
"expected_behavior",
"repro_steps",
"before_after_proof",
"tests_or_smoke",
"duplicate_notes",
"risk_notes",
"review_claim"
]
},
"review_claim": {
"impact_claim": "medium",
"evidence_types": [
"redacted_terminal_excerpt"
],
"duplicate_notes": "Searched open PRs on src/spark_cli/system_map.py; the other PRs (#1088 atomic write of gaps.md, #1081 redact compile roots, #1039-1032 Batch v3 exception-narrowing) target different lines and concerns. None touch the overlap-probe cap or the matched_*_count output fields.",
"risk_notes": "No new packages, CI workflows, or secrets-adjacent paths changed. Diff bounded to src/spark_cli/system_map.py. Same SQL query executes on the same candidate list; only the JSON response gains two additive fields per probe. No callsite reads the new fields yet, so downstream remains backward-compatible.",
"review_state_requested": "pr_review"
}
}

[4gjnbzb4zf-sudo]

TL;DR

spark os system-map reports how many builder events overlap with the spawner-prd request_ids/trace_refs it found. On a busy builder the operator looks at matched_builder_request_id_count: 12 against checked_request_id_count: 9876 and concludes the wiring is broken — but in reality the probe only tested the lex-first 500 of those 9876 ids. This change surfaces the cap so the matched count has an honest denominator.

What I noticed

Was diffing a spark os system-map JSON between two boxes and the matched/checked ratio looked alarmingly low on the noisier one. Reading inspect_builder_request_id_overlap, the input is sorted(request_ids)[:500] but the JSON keeps reporting checked_request_id_count = len(request_ids) — so once you have more than 500 unique ids, the operator-facing ratio gets quietly misleading.

The bug

file: src/spark_cli/system_map.py:935 (and the parallel function at :977)

inspect_builder_request_id_overlap sets checked_request_id_count = len(request_ids) up top, then computes candidates = sorted(request_ids)[:500] and counts matches against just candidates. inspect_builder_trace_ref_overlap does the same with trace_refs. The 500-id cap exists for a real reason (SQLite parameter limit), but nothing in the output tells the operator the matched count is from a sample, not the full input. They read it as a ground-truth overlap and act on it.

The fix

Introduce a module-level _BUILDER_OVERLAP_PROBE_CAP = 500 constant. In both functions, surface two additive fields: probe_cap (the constant) and sampled_request_id_count / sampled_trace_ref_count (len(candidates)). No SQL semantics change, no callsite breakage — the existing matched_*_count fields stay where they were. +15/-2 lines in src/spark_cli/system_map.py.

Reproduction

1. Populate a builder state.db with 100 request_ids that match, and have the spawner-prd-auto-trace surface 9000 unique request_ids (so the cap bites).
2. Call inspect_builder_request_id_overlap(builder_home, request_ids).
3. Observed: checked_request_id_count: 9000, matched_builder_request_id_count: <whatever-overlapped-among-the-lex-first-500> — operator misjudges. Expected: probe_cap: 500, sampled_request_id_count: 500 present alongside, so the matched/sampled ratio is the honest one.

Verification

• python3 -m py_compile src/spark_cli/system_map.py is clean.
• Existing tests/test_system_map.py lines 118 (error path) and 972 (matched_builder_request_id_count == 1) still pass; the new fields are additive and don't override any existing assertion.
• Manual: build a tmpdir state.db with 600 fake request_id rows + a calling set of 600 ids, confirm checked_request_id_count == 600 and sampled_request_id_count == 500 in the returned dict; previously you saw just the first.

Sister precedent

Empirically-adopted shape in this repo for observability/default-prop style fixes — see /tmp/hunt-index/precedent/spark-cli/observability.tsv and /tmp/hunt-index/precedent/spark-cli/default-prop.tsv. Same surface (operator-facing diagnostic JSON), same bound (≤30 line diff), additive-only output change.