feat(aorlta): two worked examples (HIP kernels, Q&A), before/after eval, VALIDATION.md

aorlta/README.md

@@ -128,6 +128,8 @@ flowchart TB
 
 ## Status
 
+What is verified, with the commands to reproduce it, and what is not yet shown: [VALIDATION.md](VALIDATION.md).
+
 On stand-ins the loop runs end to end on one node. The last stand-in training run (`configs/standin.yaml`'s settings,
 3 iterations of 16 prompts × 16 samples) trained every iteration, landed every weight update on all 4 engines, and
 passed the gap check every step. From its iteration summaries (the command is under "Run outputs"):
@@ -147,8 +149,9 @@ Not there yet:
 - No trained model worth sharing: only 3-iteration stand-in runs. The stand-in prompts show a kernel's original
   disassembly, with no bug planted, and ask for a HIP source, so they exercise the loop, not the task.
 - The stand-in checks are cheap, so the times above say little about a run with heavier checks.
-- One prompt format, the stand-in one, and one node layout are wired; [docs/ADAPTING.md](docs/ADAPTING.md) says how
-  to add a prompt source, a reward function or a check.
+- One node layout is wired. A task of your own (prompts, answer parsing, checks and reward) is a plugin, with no
+  edit to the package: [docs/ADAPTING.md](docs/ADAPTING.md) has the contract, and [docs/TUTORIAL.md](docs/TUTORIAL.md)
+  walks through `examples/hip-kernels/`, a kernel-writing task whose reward runs the kernel and checks its output.
 
 ## Layout
 
@@ -170,7 +173,8 @@ and `configs/dev-standin.yaml` (one trainer rank and the CPU checks of `standin-
 compile and waitcheck failures, so a candidate that passes them stays unscored); `jobs/aorlta-run.sbatch` (a run),
 `jobs/aorlta-reward-standin.sbatch` (a test of the stand-in GPU checks) and two scripts the run job calls; `docker/`
 the trainer image; `tests/` the CPU suite; `tools/merge_base_adapter.py`, which merges a PEFT adapter into its base
-checkpoint; `examples/standin-pool/`, a synthetic prompt pool; `docs/ADAPTING.md`. Code comments cite sections and
+checkpoint; `examples/standin-pool/`, a synthetic prompt pool; `examples/hip-kernels/` with `configs/hipk-score.yaml`
+and `configs/hipk-train.yaml`, the worked example of a task as a plugin; `docs/ADAPTING.md` and `docs/TUTORIAL.md`. Code comments cite sections and
 labels (Section 6.x, C1-C9, D7, ...) of our internal design notes, which are not included.
 
 ## Requirements

aorlta/configs/arith-score.yaml

@@ -0,0 +1,38 @@
+# Score a model on the 64 held-out multiplications, 4 samples each (orchestrator.mode: score): the before/after evaluation of VALIDATION.md.
+# One GPU: one TokenSpeed engine on GPU 0 and, for training, one LoRA rank beside it; the answer check runs on the CPU.
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir (and
+# model.adapter_dir to score a trained adapter) and data.prompt_pools[0].path, which is this checkout's
+# aorlta/examples/qa/arith/heldout.
+plugins: [examples/qa/qa_task.py]
+run:
+  run_id: arith-score
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/qa/arith/heldout
+      source: standin
+      task: qa
+rollout:
+  engines: [0]
+  K: 4
+  P: 16
+  max_repair_turns: 1
+  thinking: "off"
+  temperature: 0.7
+  max_new_tokens: 512
+  compile_adapter: none
+  parse_adapter: qa.parse
+  feedback_template: qa.feedback
+  # a question with one short answer can get K identical replies: that is not a broken sampler here (the engines'
+  # start-up self-test checks that), so only stop when most groups are
+  identical_groups_stop: 0.75
+reward:
+  profile: qa-match
+  reward_fn: qa.v1
+orchestrator:
+  mode: score
+  run_seed: 20261113

aorlta/configs/arith-train.yaml

@@ -0,0 +1,42 @@
+# The validation run (VALIDATION.md): train on 512 generated 3-digit multiplications, 16 per iteration x 8 samples, 20 iterations.
+# One GPU: one TokenSpeed engine on GPU 0 and, for training, one LoRA rank beside it; the answer check runs on the CPU.
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir (and
+# model.adapter_dir to score a trained adapter) and data.prompt_pools[0].path, which is this checkout's
+# aorlta/examples/qa/arith/train.
+plugins: [examples/qa/qa_task.py]
+run:
+  run_id: arith-train
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/qa/arith/train
+      source: standin
+      task: qa
+rollout:
+  engines: [0]
+  K: 8
+  P: 16
+  max_repair_turns: 1
+  thinking: "off"
+  temperature: 1.0
+  max_new_tokens: 512
+  compile_adapter: none
+  parse_adapter: qa.parse
+  feedback_template: qa.feedback
+  # a question with one short answer can get K identical replies: that is not a broken sampler here (the engines'
+  # start-up self-test checks that), so only stop when most groups are
+  identical_groups_stop: 0.75
+reward:
+  profile: qa-match
+  reward_fn: qa.v1
+orchestrator:
+  iterations: 20
+  run_seed: 20261113
+trainer:
+  ranks: 1
+  micro_batch_tokens: 20480
+  lr: 2.0e-5

aorlta/configs/arith4-score.yaml

@@ -0,0 +1,38 @@
+# Score a model on the 64 held-out multiplications, 4 samples each (orchestrator.mode: score): the before/after evaluation of VALIDATION.md.
+# One GPU: one TokenSpeed engine on GPU 0 and, for training, one LoRA rank beside it; the answer check runs on the CPU.
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir (and
+# model.adapter_dir to score a trained adapter) and data.prompt_pools[0].path, which is this checkout's
+# aorlta/examples/qa/arith4/heldout.
+plugins: [examples/qa/qa_task.py]
+run:
+  run_id: arith4-score
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/qa/arith4/heldout
+      source: standin
+      task: qa
+rollout:
+  engines: [0]
+  K: 4
+  P: 16
+  max_repair_turns: 1
+  thinking: "off"
+  temperature: 0.7
+  max_new_tokens: 1024
+  compile_adapter: none
+  parse_adapter: qa.parse
+  feedback_template: qa.feedback
+  # a question with one short answer can get K identical replies: that is not a broken sampler here (the engines'
+  # start-up self-test checks that), so only stop when most groups are
+  identical_groups_stop: 0.75
+reward:
+  profile: qa-match
+  reward_fn: qa.v1
+orchestrator:
+  mode: score
+  run_seed: 20261113

aorlta/configs/arith4-train.yaml

@@ -0,0 +1,42 @@
+# The validation run (VALIDATION.md): train on 512 generated 4-digit multiplications, 16 per iteration x 8 samples, 40 iterations.
+# One GPU: one TokenSpeed engine on GPU 0 and, for training, one LoRA rank beside it; the answer check runs on the CPU.
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir (and
+# model.adapter_dir to score a trained adapter) and data.prompt_pools[0].path, which is this checkout's
+# aorlta/examples/qa/arith4/train.
+plugins: [examples/qa/qa_task.py]
+run:
+  run_id: arith4-train
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/qa/arith4/train
+      source: standin
+      task: qa
+rollout:
+  engines: [0]
+  K: 8
+  P: 16
+  max_repair_turns: 1
+  thinking: "off"
+  temperature: 1.0
+  max_new_tokens: 1024
+  compile_adapter: none
+  parse_adapter: qa.parse
+  feedback_template: qa.feedback
+  # a question with one short answer can get K identical replies: that is not a broken sampler here (the engines'
+  # start-up self-test checks that), so only stop when most groups are
+  identical_groups_stop: 0.75
+reward:
+  profile: qa-match
+  reward_fn: qa.v1
+orchestrator:
+  iterations: 40
+  run_seed: 20261113
+trainer:
+  ranks: 1
+  micro_batch_tokens: 20480
+  lr: 2.0e-5

aorlta/configs/hipk-score.yaml

@@ -0,0 +1,39 @@
+# Score the public Qwen3.5-4B on the example task of docs/TUTORIAL.md (examples/hip-kernels): 8 kernels to write,
+# 4 samples each, 2 passes, up to 2 repair turns. 4 engines on GPUs 0-3; the kernels run on GPUs 4-7 (hipk-gpu.yaml).
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir and
+# data.prompt_pools[0].path, which is this checkout's aorlta/examples/hip-kernels.
+plugins: [examples/hip-kernels/hip_kernels.py]
+run:
+  run_id: hipk-score
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/hip-kernels
+      source: standin
+      task: hipk
+rollout:
+  engines: [0, 1, 2, 3]
+  K: 4
+  P: 8
+  max_repair_turns: 2
+  thinking: "off"
+  temperature: 0.7
+  max_new_tokens: 2048
+  compile_adapter: standin.hipcc
+  parse_adapter: hipk.parse
+  feedback_template: hipk.feedback
+  # a short kernel can have one obvious answer: 8 identical answers to one prompt is not a broken sampler here
+  # (the engines' self-test checks that), and with 8 prompts one such group is 12.5% of them
+  identical_groups_stop: 0.5
+reward:
+  profile: hipk-gpu
+  reward_fn: hipk.v1
+orchestrator:
+  mode: score
+  run_seed: 20261112
+  score:
+    passes: 2

aorlta/configs/hipk-train.yaml

@@ -0,0 +1,41 @@
+# Train on the example task of docs/TUTORIAL.md (examples/hip-kernels): 3 iterations of the 8 kernels x 8 samples,
+# one LoRA rank beside engine 0, the kernels run on GPUs 4-7 (hipk-gpu.yaml). A starting point to copy, not a result:
+# 8 prompts are far too few to learn from.
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir and
+# data.prompt_pools[0].path, which is this checkout's aorlta/examples/hip-kernels.
+plugins: [examples/hip-kernels/hip_kernels.py]
+run:
+  run_id: hipk-train
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/hip-kernels
+      source: standin
+      task: hipk
+rollout:
+  engines: [0, 1, 2, 3]
+  K: 8
+  P: 8
+  max_repair_turns: 2
+  thinking: "off"
+  temperature: 0.7
+  max_new_tokens: 2048
+  compile_adapter: standin.hipcc
+  parse_adapter: hipk.parse
+  feedback_template: hipk.feedback
+  # a short kernel can have one obvious answer: 8 identical answers to one prompt is not a broken sampler here
+  # (the engines' self-test checks that), and with 8 prompts one such group is 12.5% of them
+  identical_groups_stop: 0.5
+reward:
+  profile: hipk-gpu
+  reward_fn: hipk.v1
+orchestrator:
+  iterations: 3
+  run_seed: 20261112
+trainer:
+  ranks: 1
+  micro_batch_tokens: 20480

aorlta/configs/qa-score.yaml

@@ -0,0 +1,38 @@
+# Score a model on the Q&A example's 8 held-out questions, 4 samples each (orchestrator.mode: score).
+# One GPU: one TokenSpeed engine on GPU 0 and, for training, one LoRA rank beside it; the answer check runs on the CPU.
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir (and
+# model.adapter_dir to score a trained adapter) and data.prompt_pools[0].path, which is this checkout's
+# aorlta/examples/qa/gpu-facts/heldout.
+plugins: [examples/qa/qa_task.py]
+run:
+  run_id: qa-score
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/qa/gpu-facts/heldout
+      source: standin
+      task: qa
+rollout:
+  engines: [0]
+  K: 4
+  P: 8
+  max_repair_turns: 1
+  thinking: "off"
+  temperature: 0.7
+  max_new_tokens: 512
+  compile_adapter: none
+  parse_adapter: qa.parse
+  feedback_template: qa.feedback
+  # a question with one short answer can get K identical replies: that is not a broken sampler here (the engines'
+  # start-up self-test checks that), so only stop when most groups are
+  identical_groups_stop: 0.75
+reward:
+  profile: qa-match
+  reward_fn: qa.v1
+orchestrator:
+  mode: score
+  run_seed: 20261113

aorlta/configs/qa-train.yaml

@@ -0,0 +1,42 @@
+# Train on the Q&A example of docs/TUTORIAL.md: 16 GPU-programming questions, 8 samples each, 3 iterations.
+# One GPU: one TokenSpeed engine on GPU 0 and, for training, one LoRA rank beside it; the answer check runs on the CPU.
+# Site-specific paths, to edit here or to set in a site file (AORLTA_SITE): run.root, model.base_dir (and
+# model.adapter_dir to score a trained adapter) and data.prompt_pools[0].path, which is this checkout's
+# aorlta/examples/qa/gpu-facts/train.
+plugins: [examples/qa/qa_task.py]
+run:
+  run_id: qa-train
+  source: standin
+  root: /scratch/aorlta/runs
+model:
+  base_dir: /data/hf/hub/models--Qwen--Qwen3.5-4B/snapshots/851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a
+  source: standin
+data:
+  prompt_pools:
+    - path: /opt/aorlta/examples/qa/gpu-facts/train
+      source: standin
+      task: qa
+rollout:
+  engines: [0]
+  K: 8
+  P: 8
+  max_repair_turns: 1
+  thinking: "off"
+  temperature: 1.0
+  max_new_tokens: 512
+  compile_adapter: none
+  parse_adapter: qa.parse
+  feedback_template: qa.feedback
+  # a question with one short answer can get K identical replies: that is not a broken sampler here (the engines'
+  # start-up self-test checks that), so only stop when most groups are
+  identical_groups_stop: 0.75
+reward:
+  profile: qa-match
+  reward_fn: qa.v1
+orchestrator:
+  iterations: 3
+  run_seed: 20261113
+trainer:
+  ranks: 1
+  micro_batch_tokens: 20480
+  lr: 2.0e-5