docs(aorlta): getting-started guide for new users by vivekkhandelwal1 · Pull Request #610 · ROCm/aorta

vivekkhandelwal1 · 2026-11-12T17:30:34Z

The team is going to try AORLTA on their own tasks, so I walked through the README and ADAPTING.md from a fresh clone, the way a new user would. This is the docs half of what came out of that.

docs/GETTING-STARTED.md: the steps in order. What a run needs, and what is untested (fewer than 8 GPUs, MI300X/MI355X, models outside the Qwen3.5 family). The CPU tests the way CI runs them. The site file and the job's environment. A score run before a training run, with the times measured on main. Reading the outputs, the settings people change, and what each job exit code and common run stop means.
README: don't run the suite with -n auto on a many-core box (it runs out of memory). Only Qwen3.5 works today. The stand-in dataset isn't published and the example pool is too small to run on. --components fake doesn't read the configured pool or checks.

Also:

the guide covers running on 1, 2, 4 or 8 GPUs, with memory per GPU measured on MI350X (estimates are labelled), and how to share a node;
how to switch models, and what limits that today (Qwen3.5 family only);
where checkpoints and adapters go;
a "Getting help" section in the README and an AORLTA issue template (label aorlta, which now exists);
a pointer from the top-level aorta README, marked early and evolving.

Docs and an issue template only, no code change. Some lines refer to files in #611–#613 (the Q&A configs, the export tool), so this is best merged after them.

How I checked it: on a fresh clone of main I followed the steps and the CPU suite passed in all three parts. I also ran a score run and a 3-iteration stand-in training run on MI350X; the times in the guide come from those runs.

…r needs first docs/GETTING-STARTED.md puts the steps in order: what a run needs (and what is untested: fewer GPUs, other GPUs, other models), the CPU tests as CI runs them, the site file and the job's environment, a score run before a training run, measured times, reading the outputs, the settings people change, and what each of the job's exit codes and the common run stops mean. The README now says the test suite must not run with -n auto on a many-core machine, that only the Qwen3.5 family works as it stands, that the stand-in dataset is not published and the example pool is too small to run, and that the fake components do not read the configured pool or checks.

Copilot

🟡 Changes recommended

The getting-started guide misstates checkout requirements, scoring versus training submissions, the scored-fraction denominator, and where to find failed-iteration diagnostics, leaving readers with misleading setup and troubleshooting guidance.

4 open issues

What changed in this PR

This documentation-only PR adds an onboarding path for AORLTA, from CPU checks through GPU scoring and training, and clarifies current limitations.

Changes:

Adds a getting-started guide covering prerequisites, setup, outputs, settings, and troubleshooting.
Replaces unrestricted pytest parallelism with bounded test commands.
Clarifies model support, dataset availability, and what fake-component runs validate.

File	Description
`aorlta/README.md`	Links the guide and clarifies testing commands and prerequisites.
`aorlta/docs/GETTING-STARTED.md`	Adds step-by-step setup, run, and troubleshooting guidance.

_{Review effort: Balanced}

…utputs

Copilot

🔵 Needs a closer look

The troubleshooting guidance points to a summary.json that is not written for the stopped iteration instead of the failure diagnostics saved in run_result.json.

4 open issues

_{Review effort: Balanced}

…help, and a pointer from the aorta README GETTING-STARTED.md: how to place engines, trainer ranks and checks on 1, 2, 4 or 8 GPUs of one node, measured peak memory per GPU (labelled estimates beside them), how to share a node, how to switch models and what limits that today, and where checkpoints and adapters go. The README gains a Getting help section (open an issue with the new AORLTA template, label aorlta; docs feedback as docs: issues). The top-level README points to aorlta/ as early and evolving.

Copilot

🟡 Changes recommended

The getting-started guide references unshipped configs and an unsupported GPU setting, reports an unreproducible VRAM measurement, and promises an unimplemented checkpoint-to-PEFT export workflow.

7 open issues

_{Review effort: Balanced}

…nd of the GPU layout Co-authored-by: Cursor <cursoragent@cursor.com>

…t scrolling Co-authored-by: Cursor <cursoragent@cursor.com>

Copilot

🟡 Changes recommended

The documentation overstates when parse repairs occur and presents the AORTA checkout as optional despite unconditional preflight imports, potentially causing real jobs to fail during setup.

8 open issues

_{Review effort: Balanced}

…e or with main - Every GPU run needs the aorta checkout (AORLTA_AORTA_ROOT): the import pre-flight loads two of its backends whatever the profile, and exits 99 without it. Its own row, and out of the stand-in-only environment line. - The score run and the training run are two submissions; give the training command. - min_scored_fraction is over the candidates handed to the reward service, not every chain generated (also in the README's correctness table). - A scored_fraction_below_threshold stop writes no summary for the failing iteration: read stop.detail.policy and stop.detail.unsettled in run_result.json. - Name the pull requests that add the worked examples (#613), plugins (#611) and the adapter exporter (#612) where a passage depends on them; drop hipk.run.gpus as if it were a config key; a checkpoint is not a PEFT adapter.

Copilot

🔵 Needs a closer look

The setup guide incorrectly presents the launcher's required aorta checkout as optional, causing documented custom and Q&A Slurm jobs to exit 99 when that checkout is absent.

8 open issues

_{Review effort: Balanced}

amd-vivekag

Review summary

Verdict: fix-before-ship

The guide is the right shape: score before train, exit codes, and the GPU layout. A few of the steps a new user would follow do not match the code, and the one-GPU section points at configs this branch does not contain.

Docs only. I checked the claims against the loop, the stand-in parser, and the run job's import preflight.

Findings: 0 High · 5 Medium · 0 Low

Copilot's note that a real job still needs an aorta checkout is right, and so is the stand-in repair edge. The score-vs-train command and the missing summary on a scored-fraction stop are the same kind of mismatch.

amd-vivekag · 2026-11-13T08:06:14Z

+    subgraph rollout["2-3 generate: TokenSpeed engines at policy v(n-1), K chains per prompt"]
+        turn["one request per chain and turn, with its own seed"] --> parse{"parse the reply<br/>rollout.parse_adapter"}
+        parse -- "an answer" --> compile{"compile check<br/>rollout.compile_adapter (none: skipped)"}
+        parse -- "rejected, a turn left" --> feedback["repair turn<br/>rollout.feedback_template"]


[Medium] The stand-in parser never takes this repair edge.

StandinParse returns ParseOutcome("rejected") with no message, and the chain ends no_code_block when rejection is missing, even if a turn is left. A repair turn happens only when the parser fills in rejection (the HIP and Q&A plugins do).

Fix: draw the stand-in edge as "no code block → end", and keep "rejected, a turn left → repair" for a parser that sets rejection. Copilot flagged this too.

Fixed in b5f1ee7: a reply rejected with no reason (the stand-in parser's missing code block) now ends the chain in the diagram, and the repair edge is drawn only for a parser that sets rejection.