fix(grade): read assertions from the live evals.json (#295)

[slowdini]

Closes #295.

The problem

grade read the eval definition from the copy the iteration froze
(iteration-N/.skills/<skill>/evals/evals.json), so assertions authored after the
run — the workflow eval-magic docs judging prescribes — were never graded. The
result was Judge tasks: 0, with no error and nothing saying the file being graded
was not the file just edited.

Freezing is right for the treatment: what the agent loaded must not change after
the dispatch it explains. Assertions are not the treatment. They are the measuring
instrument, and the documented explore-first loop authors them from the run's own
paired evidence.

The change

grade splits the file. assertions and skill_should_trigger come from the live
<skill>/evals/evals.json, matched per eval id; prompt, files, turns,
codebase, guard, and runs stay as the run captured them. Resolution lives in a
new src/pipeline/grade/instrument.rs. command_check.setup_files resolve from
whichever tree supplied the assertion that names them, so a check added after the run
finds its held-out file.

Every invocation names the file it measured against:

Assertions: /path/to/skill/evals/evals.json
  refreshed — differs from the run-time copy for 1 eval(s): e1

Each grading.json records the same under assertion_source (path, digest,
refreshed). An unreadable live file falls back to the run-time copy with a warning; an
invalid one fails rather than grading around a broken edit. Eval ids present in one
file and not the other are named.

Two follow-on silences also closed: cached judge verdicts and command-check results
are keyed by assertion id, so an assertion edited in place is reported by name with
the --overwrite remedy rather than silently reused.

Before / after

Same iteration, one 5-sample llm_judge added after the run, no hand-copying into the
workspace:

$ eval-magic grade --iteration 1          # before the assertion is written
Assertions: …/iteration-1/.skills/demo/evals/evals.json (unchanged since the run)
Judge tasks: 1 (1 skill-invocation meta-judge(s))

$ eval-magic grade --iteration 1          # after
Assertions: …/skills/demo/evals/evals.json
  refreshed — differs from the run-time copy for 1 eval(s): e1
Judge tasks: 11 (1 skill-invocation meta-judge(s))

Previously the second run also printed Judge tasks: 1.

Schema updates

• grading.schema.json gains optional assertion_source.
• command-check.schema.json gains optional definition_digest.

Both are absent in older artifacts and read as making no claim, so existing iterations
still validate and aggregate reads them unchanged.

Documentation

• grade --help states the split and where it is recorded.
• eval-magic docs judging — new "Which evals.json grade reads" section, plus the step
  the explore-first loop was missing: which file to edit, and what re-reads it.
• eval-magic docs isolation — the copy freezes the treatment, not the assertions, so
  the two guides no longer contradict each other.
• RUNBOOK step 2 previously ended at compare with no route back into grading; golden
  fixtures re-blessed.

Verification

cargo fmt --check                        # clean
cargo clippy --all-targets -- -D warnings  # clean
cargo test                               # 923 + 213 + 201 passing, 0 failed

Plus a manual end-to-end run of the ticket's reproduction against the built binary,
covering the refresh, the recorded assertion_source, the stale-rubric warning, the
missing-live-tree fallback, and the invalid-live-file failure (exit 1).

One existing test, tests/run/skill_source.rs::grading_reads_the_eval_definitions_the_run_copied,
encoded the superseded contract by making the live file unreadable. It now asserts the
narrower one: the eval definitions still come from the copy, assertions are the
deliberate exception, and both sides of an eval-set mismatch are reported.

🤖 Generated with Claude Code

https://claude.ai/code/session_01MrfVg8vNeeZN7upRmWerYC