Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 3 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# Imported recipe hashes cover their original formatting as well as their data.
# Preserve those bytes; keep ordinary whitespace checks on all maintained code.
tasks/e2e/magpie/**/upstream.yaml whitespace=-blank-at-eol,-blank-at-eof
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The platform provides:
- **Multiple agent integrations**: Run Cursor Agent, Claude Code, Codex, DeepSeek Harness, GEAK-based agents, or custom agents through a shared interface.
- **Real GPU task environments**: Work with HIP, Triton, FlyDSL, PyTorch-to-kernel conversion, instruction-to-kernel generation, and image-backed kernel optimization tasks.
- **Isolated and reproducible execution**: Give every task its own timestamped workspace and preserve logs, modified sources, and structured results.
- **Fixed-workload serving tasks**: Evaluate kernel changes through model throughput with pinned settings and fresh paired measurements; see [the e2e guide](docs/how-to/e2e-kernel-tasks.md).
- **Fixed-workload serving tasks**: Evaluate model-wide kernel changes and fusion with pinned settings, fresh paired measurements and independent candidate review. Each imported [Magpie workload](tasks/e2e/magpie/README.md) has its own selectable task directory; shared adapters prepare its immutable runtime before agent execution. See [the e2e guide](docs/how-to/e2e-kernel-tasks.md).
- **Centralized evaluation**: Measure compilation, correctness, and GPU performance independently of the optimizing agent.
- **Multi-GPU scheduling**: Start one isolated Docker worker per GPU and dynamically claim tasks from a shared queue.
- **Slurm/Spur login-node workflow**: Allocate one or eight MI355X GPUs, then launch the same Docker runtime on the assigned compute node.
Expand Down
17 changes: 16 additions & 1 deletion agents/task_validator/validation_prompt_v2.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,22 @@ def build_v2_validation_prompt(*, task_id: str, task_config: dict, workspace: st
processing measured by the protected client with serving_wall_clock, not device
kernel duration. Review fixed token/request counts, settings, pinned runtime and
weights, clean candidate injection with per-rank actual-call evidence, independent
operator and model correctness, and warmups outside the measurement window.
model correctness, any task-declared operator checks, and warmups outside the
measurement window. Model-wide tasks may use a replacement manifest with no
initial patches: this is an implemented baseline backed by the pinned production
image, not missing kernel source. Inspect the manifest loader, pinned dependencies,
model-class observation and actual execution evidence. Later submissions may change
any allowed compute boundary or fuse kernels; do not require an exhaustive operator
inventory or a hand-written oracle for every possible fusion. Generic numerical
checks must still detect wrong model computation. The independent candidate
verifier separately reviews each submitted compute change for semantic validity,
generality and reward hacking; it cannot waive a numerical failure.
A declared production-model oracle is a fresh, uninjected pinned server. Initial
baseline calibration intentionally compares original production runs to check
normal numerical variation. It does not prove the production model mathematically
correct or qualify a later candidate. For a submitted bundle, verify that the
oracle remains independent of that bundle and checks the actual candidate's
outputs/continuations; do not require enumerating every operator in the model.
CUDA graph replay review is not applicable to this client timer; leave
replay_validation_valid null. This does not waive model or operator correctness,
workload symmetry, state, or timing-boundary checks. Declared read-only runtime
Expand Down
12 changes: 12 additions & 0 deletions docs/how-to/add-task.md
Original file line number Diff line number Diff line change
Expand Up @@ -808,6 +808,18 @@ See [e2e task setup, budget and evidence requirements](e2e-kernel-tasks.md).
Imported benchmark YAML without a task `config.yaml` is catalog data and is not
an executable or qualified task.

A serving recipe may additionally declare a relative `measurement.recipe`
descriptor. It still has its own schema-v2 task directory, implementation
boundary and protected original workload. Before starting the agent, the
host-owned Docker runner combines it with the run config's explicit
`serving_runtime.bindings` entry and prepares a complete immutable package through
the shared adapter. The prepared package replaces the recipe directory only in
the container view, read-only; it does not modify committed tasks. Its evaluator
and inputs are task-local and self-contained. An unresolved recipe is never a
passing task or an optimization-time setup action. The runtime lock and recipe
descriptor must be outside editable paths. Task qualification applies to the
exact prepared model/image/hardware binding, not merely the recipe declaration.

## Migration

### Current runtime versus the selected contract
Expand Down
Loading
Loading