[irvineoy]

Magpie recipes launch benchmarks but do not define a fixed Arena optimization task. This adds all 50 imported recipes as individual, selectable schema-v2 task directories under tasks/e2e/magpie/, backed by shared SGLang, vLLM and ATOM adapters. The catalog includes 45 text, three multimodal and two AgentX recipes. Future workloads reuse the same preparation and evaluation flow rather than adding another model-specific evaluator.

Each task directory contains its own config.yaml, original upstream.yaml, pinned recipe descriptor, instructions, license, initial compute replacement manifest and preparation entrypoint. Normal task discovery and task selectors remain unchanged. A run config supplies an explicit immutable binding in serving_runtime.bindings; the host prepares a content-addressed, self-contained runtime package outside the checkout and mounts only selected packages read-only at their normal task paths before workspace creation. Binding files stay on the host. Missing bindings, edited declarations and changed executable cache files fail before agent execution. Source-tree bytecode is excluded from preparation, while unexpected bytecode in a prepared executable package is rejected.

python3 -m src.magpie_catalog export-tasks --check checks all committed declarations against the pinned catalog. Export preserves existing edits unless an explicit reviewed --update is requested. Unexpected task directories are reported and retained. Existing fully bound serving tasks and single-kernel task execution retain their existing interfaces.

Agents can change compute throughout the model, add kernels and fuse operations. There is no enumerated operator inventory or requirement to write an oracle for each fusion. A maintainer-owned binding fixes model/image/client bytes, inputs, precision, TP/EP/PP, engine flags, environment and measurement policy before search. Configuration and observation APIs remain protected.

Each action reconstructs the task in a fresh evaluator container. Generic model checks, runtime fingerprints, per-rank execution and replacement observations accompany the measured result. The existing separate candidate_verifier reviews frozen source and execution evidence for preserved semantics, generality and benchmark integrity. Failed or inconclusive review rejects acceptance and cannot waive numerical failures. Review-detected wrappers or unchanged computation receive no positive optimization credit. Profiling is recommended without requiring a particular tool or report.

Other changes:

• Teacher-force each sampled diagnostic and ordinary continuation on a fresh uninjected production reference. Check every returned decode probability, sequence identity and finite-value constraint under bounds frozen before search.
• Capture native timer and phase source from hash-pinned prepared client archives. Include initial idle and terminal drain time in trace throughput, and retain bounded container-cleanup evidence.
• Prepare native text payload identities and realized token lengths before server startup; reconcile them with the timed native plan and per-request work counts. This handles native decode/re-encode length variation without relaxing counts or changing HTTP timing.
• Require prepared, hash-pinned client environments and offline assets; scoring never installs dependencies. Replay accounting rejects cancelled/skipped records and reconciles aggregate counters with individual requests.
• Preserve typed backend configuration and separate direct execution from compiled traces. Capture original compute source before candidate imports for independent review.
• Allow an explicitly bound FP32 token-only reference for native-token-only engines; never switch reference policy after a failed check. Explicitly select evaluator executables rather than inheriting an image's server entrypoint.

Validation used MI355X (gfx950), immutable images and frozen bindings.

The updated AgentX path passed all 12 checks in a framework-finalized report: compilation, required model correctness, performance, independent semantic review and harness/benchmark integrity. This used the original DeepSeek-V4-Pro-0813 FP4 SGLang workload, TP8 and concurrency 32, with the model, precision, engine flags, client, corpus and one-hour replay preserved.

• The original-only calibration was frozen before the separate validation run. Correctness checked 36 diagnostic and 36 ordinary generation cases, including long contexts and a complete concurrent trace batch. Diagnostic decode probabilities now use the independent score for each actual token and prefix. All six existing production bounds remained unchanged.
• The complete replay finished 3,844 requests with zero errors and 3,853,813 output tokens. Its native observation window, including drain, was 3,640.002 seconds; baseline throughput was 1,058.739 output tokens/s. This qualifies the original baseline; it does not report a later optimized candidate or completed final candidate A/B pairs.
• All raw per-request work and native timing counters were rechecked against the frozen turn manifest. Ten native timing/phase source files and the client bootstrap were captured. The full model-comparison data were recomputed locally, and eight offline corruption controls on captured GPU output were rejected.
• The finalized report SHA-256 is f2259e16a7ddc8746d2d4fd69061d370d2862323dbc9a474eac9ad19c96eea96. Actual GPU actions exited zero; all eight GPUs and action containers passed cleanup before the finite job ended.
• The earlier GPU-qualified tree integrated main at ac45f1209abd82401f644ff1ba323e1961fb2d01. Its full Linux non-GPU suite passed: 19,040 tests, 156 skips and 18 passing subtests. The 396 focused Mac checks and performance-helper audit also passed. Serving comparison and replay helpers remain byte-identical to the GPU-qualified source; the main integration was checked separately, and published branch history is preserved.

Earlier representative integration coverage, retained from the previously pushed revision:

Representative task	Result
Generated vLLM small-model fixture	Finalized PASS, all 12 checks
Generated ATOM small-model fixture	Finalized PASS, all 12 checks, including 8,000 native requests
Catalog Qwen3-Next-80B FP8, SGLang TP4	11 checks PASS; quality-floor calibration WARN, not waived
Generated SGLang small-model fixture	11 checks PASS; finite correctness coverage WARN accepted with the justification below
Existing single-kernel RMSNorm task	Finalized PASS in the integration campaign

Numerical probes cover representative requests rather than every timed streaming response and all long-run state. The maintainer accepts generic sampling plus mandatory independent candidate review for this scope. Production-model comparisons are not an FP32 oracle or universal proof for every future fusion. A later submitted candidate still needs its own full numerical checks, source/execution review and paired measurements. Multimodal GPU qualification was not run, and catalog integration does not qualify every model/image/hardware binding.

The earlier 80B binding warning remains separate and unwaived: observed quality scores of 59/64 and 60/64 are close to its fixed 0.9 floor (58/64), so normal variation could cause a false rejection. No such failure was observed in that run. This change does not declare that binding fully qualified or change its quality threshold. Its calibration/stability remains a binding-level follow-up.

Earlier GPU controls rejected wrong computation and a numerically valid runtime-setting mutation. Review-detected delegation to the original implementation received no optimization credit. Valid supplied rewrites passed numerical checks and independent review despite being slower; the compiled vLLM control completed three fresh A/B pairs and trace/direct-model observation checks (0.9929×). These were integration controls, not optimization-agent results.

The directory/preparation revision passed 411 focused Linux checks, make check-perf-helpers and source compilation. A full Linux suite at the preceding migration checkpoint passed 19,065 tests, with 154 skips and 18 passing subtests; the final cache-transient guard was covered by the subsequent focused run. After integrating main, 681 focused checks passed, followed by the helper audit and source compilation; the CPU container and temporary source directory were cleaned up. CPU checks used Python 3.12 and a pinned CPU-only container. A real Docker control-plane smoke test exercised normal task discovery, selected read-only nested mounts, writable candidate workspaces and unchanged-workspace resume; service and container cleanup passed.

Before the latest main integration, the prepared AgentX runtime package had the same 18 files and identical file bytes as the GPU-qualified package above. The final tree integrates main at 443ee96687784ed82c43798f0009e89697b66d58 and preserves 17 of those 18 files byte-for-byte. The remaining shared measurement module adds the main branch's cold-cache kernel policy and helper; all 11 existing measurement functions are unchanged. Integration checks cover the combined serving and cold-cache paths, and the real Docker mount/workspace/resume smoke test was repeated after integration. This revision reuses the earlier serving GPU evidence; it did not repeat GPU inference or measure a new performance gain. All 50 recipe declarations were checked, but not every binding was GPU-qualified.

GPU checks used Docker-backed make slurm-run CONFIG=<prepared-validator-config> SLURM_GPU_COUNT=8. Runtime bindings are prepared with python3 -m src.magpie_catalog bind; ordinary task selection now prepares the selected package automatically, and materialize remains available for exporting standalone packages to a new directory.