fix(scheduler): key runner teardown on actual assignment, not dispatch intent

[luthermonson]

The bug

ephemerd registers one just-in-time runner per queued job and keys the runner's lifecycle to that job: the completed-webhook handler destroys job.dispatched. But GitHub's scheduler treats registered JIT runners as fungible — a runner picks up ANY queued job matching its labels. Every multi-job workflow run queues several same-label jobs at once, so runner R dispatched "for" job A routinely gets assigned job B. When job A later completes (on a different runner), the handler destroys R — killing job B mid-flight. The orphaned job fails at exactly ~10m01s (GitHub's runner-communication-lost timeout) with no uploaded logs (BlobNotFound). This has been misdiagnosed as "dead runners" for two days and killed dozens of CI jobs.

Production evidence (ephemerd.log, 2026-07-10)

12:19:18 webhook event action=queued job_id=86444761266
12:19:18 webhook event action=queued job_id=86444761252
12:19:21 registered repo-level JIT runner name=ephemerd-github-ephpm-kind_cope
12:19:22 Linux runner dispatched job_id=86444761266 name=ephemerd-github-ephpm-kind_cope
12:19:26 webhook event action=in_progress job_id=86444761266
12:19:31 webhook event action=in_progress job_id=86444761252   <-- GitHub gave kind_cope job ...252
12:20:16 webhook event action=completed job_id=86444761266 (success, on another runner)
12:20:16 job completed, destroying runner environment job_id=86444761266
12:20:16 WARN dispatched runner killed (timeout or shutdown) job_id=86444761266
12:20:16 WARN failed to destroy dispatched runner error="job \"ephemerd-github-ephpm-kind_cope\" not found"
12:29:32 webhook event action=completed job_id=86444761252     <-- 12:19:31 + 10m01s: comm-lost timeout

Job 86444761266's completion destroyed kind_cope, which was mid-flight on job 86444761252; that job died 10m01s after its in_progress.

The fix

1. Capture runner_name from workflow_job events into providers.JobEvent.RunnerName (GitHub populates it on in_progress/completed). New optional interface providers.RunnerNameReporter marks providers that report it (GitHub only, today).
2. Runner-name ledger (Scheduler.runners) alongside the job-keyed running map. On in_progress, the event's runner is bound to the event's job; when the binding differs from the dispatch intent an INFO line fires — this log line is the ongoing observability for the race (the old logs above have no runner names on in_progress; now they do, including in the raw webhook-event log line).
3. completed destroys the runner NAMED IN THE EVENT (when ephemerd owns it), found via the ledger under whichever dispatch-intent entry holds it. Runner bookkeeping stays filed under its intent key — no re-keying, so the existing per-runner Wait-goroutine cleanup, drain, and destroyAll are untouched. If the event carries no runner_name (job cancelled before assignment, or a provider that doesn't report names), it falls back to the old dispatch-intent destroy — but only when that runner was never observed bound to a different job.
4. Orphan sweep ([runner.orphan_sweep], enabled by default, grace = "10m", same config shape as [runner.claim_retry] from fix(scheduler): claim retry queue with jittered backoff + fresh rate gauges #104): a dispatched runner never seen in an in_progress event within the grace window is destroyed and explicitly deregistered (JIT runners that never ran a job don't auto-remove). This replaces the implicit cleanup the old job-keyed destroy provided for stolen/cancelled intents. It runs only in webhook mode (polling has no in_progress events, so "never observed bound" carries no information) and only for runners from assignment-reporting providers — Forgejo/Gitea/GitLab/Woodpecker and polling-mode behavior are exactly unchanged.

macOS VM / native runner paths

They share the fungibility problem: macOS VM and native macOS runners are also JIT-registered per job with shared labels (self-hosted, macos, arch). The fix is uniform — teardown resolves the owning entry by runner name first, then uses that entry's own mechanic (macosVM.Stop() / nativeRunner.Stop() / dispatch destroy / runtime destroy), so all four provisioning paths are covered by the same resolution logic.

Design decisions / deviations

• The ledger is a side map rather than re-keying running on in_progress: re-keying invalidates the job keys captured by the per-runner Wait goroutines and breaks down when a reassignment event arrives before the displaced job's own provisioning finishes. Filing runners under intent and resolving teardown by name keeps every existing consumer of running (dedup, drain, destroyAll, healthz, vm-ssh) working unchanged.
• A completed event naming a runner we don't own (peer daemon / GitHub-hosted) now destroys nothing; our intent runner for that job stays up to take another queued job, exit on its own, or be swept.
• Drive-by fixes: the jobs-active gauge now decrements on natural-exit cleanup (it leaked permanently in polling mode); local-path runningJob now records its provider like the other three paths (its cleanup already called rj.provider.ReleaseJob behind a nil check that never fired); removed an accidentally duplicated claim_retry block in config.example.toml.

Composes with #95 (zombie-job skip — untouched, still keyed on provisioning attempts) and #104 (claim retry queue — the in_progress retry-drop moved into handleInProgress intact).

Tests

New pkg/scheduler/reassignment_test.go (fake dispatch gRPC server, matching the existing scheduler test style):

• TestHandleCompleted_ReassignedRunner_ProductionScenario — dispatch R for A and S for B; in_progress(B, R); completed(A, S) must NOT destroy R; completed(B, R) destroys R; bookkeeping/ledger fully drained at the end.
• TestHandleCompleted_NoReassignment_HappyPath — unchanged common case.
• TestHandleCompleted_CancelledBeforeAssignment — empty runner_name fallback still tears down the intent runner.
• TestHandleCompleted_EmptyRunnerName_IntentRunnerBoundElsewhere — fallback declines when the intent runner is bound to another job.
• TestHandleCompleted_ForeignRunner — foreign runner name destroys nothing.
• TestHandleInProgress_UnknownOrEmptyRunner — no-op paths.
• TestSweepOrphanRunners_GraceWindow — sweep matrix (orphan destroyed + deregistered; bound / fresh / unobservable exempt).
• TestSweepOrphanRunners_DisabledOrPolling — sweep inert when disabled or polling.

Plus TestConvertEvent_RunnerName / TestReportsRunnerNames (github provider) and TestOrphanSweepEnabled / TestOrphanSweepGrace (config).

Verified locally (CGO_ENABLED=0): go build ./..., go vet ./..., go test -tags containers_image_openpgp -count=1 on scheduler/config/providers/github/cmd (all pass; scheduler suite 60+ tests), golangci-lint run on touched packages — 0 issues.