F4 follow-ups: correct false provenance claims and two broken documented commands

[ErikaOnFire]

Five small follow-ups to the F4/F5 work merged in #98, all found by re-reading and actually running what #98 documented. Docs, one test, no behaviour change, no content-hash impact.

Targets erika/humanebench-cli. #99 and #100 are merged and nothing else is open against this base.

1. 8cd6795 — the slot comment asserted false provenance

The doc comment on the two slot constants said they "are the Go template markers from the upstream evaluator" and that keeping them verbatim is what makes the embedded prompt byte-identical to its source. Both halves are wrong, in exactly the direction F4 was about:

• Upstream spells the slots the Python str.format way — {user_prompt} / {message_content} at evaluator/humanebench_evaluator.py:395,401. The {{.UserPrompt}} spelling is this CLI's own canonicalisation.
• The slots are the only non-verbatim part of the prompt. Byte-identity holds after undoing this substitution, so "keeping them verbatim" is backwards as an explanation.

2. 199b27b — the documented bring-your-own-source command does not work

cli/README.md told converter authors to run humanebench ingest --stdin --source yourtool. That fails: adapters/mod.rs:41 rejects it with unknown source. Found by building the release binary and ingesting a synthetic normalized transcript.

The doc conflated the --source flag (an adapter allowlist: claude-code, codex, chatgpt, claude-app, hermes, normalized) with the source field inside each record, which is the free-form tag that really is "surfaced, never parsed". Corrected to --source normalized and spelled out the distinction.

3. 711275d — humanebench schema shipped the same two errors

The built-in schema help carried the identical mistakes at main.rs:894,917, so every converter author reading it got a command that fails. Same fix, verified against the rebuilt binary's actual output.

4. 18e0c29 — guard the split marker the rollup depends on

rubric/README.md records that re-deriving the prompt must preserve the literal "Now, evaluate the following", because rubric_preamble() splits on it — and that function silently falls back to returning the whole rubric when the marker is absent. Marker loss did surface before, but only as an opaque !contains("{{.UserPrompt}}") assertion naming no cause.

Now asserted directly: the marker appears exactly once, and the preamble is a proper prefix. Mutation-verified — renaming the marker in the embedded rubric fails this test with it must appear exactly once; the rubric was restored clean afterwards.

5. 367cc4f — a citation this branch itself invalidated

Commit 1 grew a doc comment by six lines, moving PRINCIPLES from judge/mod.rs:134–143 to 140–149, which stale-dated a citation in rubric/README.md. Caught by auditing all 25 line-number citations in the F4 write-up against the tree; the other 24 resolve.

Verification, and one gap

cargo test — 147 passed / 0 failed (146 + the new guard).

cargo clippy --all-targets — 0 warnings, 0 errors, verified on a forced cargo clean -p humanebench recompile rather than a cached run.

(An earlier revision of this description said clippy had not run on commits 3–5, because a local permission rule was refusing it. That refusal was transient; clippy has since run clean across the whole branch, and this supersedes that caveat.)

I also could not smoke-test the F5 caveat in a rendered report: report requires cached scores and scoring needs a live judge API key. The unit test at report/mod.rs:979 covers that text.

🤖 Generated with Claude Code

https://claude.ai/code/session_01T5Heg7NozQmgkz5Mgu52Sb