[vivekkhandelwal1]

A rewards catalog and a timing reward.

• docs/REWARDS.md: every check adapter and reward function AORLTA ships (parse, compile, reference match, normalised match, timing), what each needs, and its states and reasons. It also says how to write your own while keeping the service's rules: a reward in [-1.55, 1.0], parts base, broke_healthy, length, and an unsettled check that is never clean.
• aorlta.reward.timing: judges a candidate's per-launch times against a reference's, taken interleaved on one GPU. faster only past the noise floor (the larger of a minimum and the two relative MADs), not_faster only past it the other way. Noisy, within-noise, too-few or invalid samples are unscored, never clean. A settled verdict carries a measurement record.
• rewardfn.speedup: grades a clean reward by the measured speedup, 0.75 to 1.0 when faster by the ratio and below 0.5 when measured and not.
• The HIP example gets an optional hipk-speed profile with a ref.speedup leg (hipk.time). It uses the same design as ref.native in feat(aorlta): two worked examples (HIP kernels, Q&A), before/after eval, VALIDATION.md #613: the candidate and the reference are each compiled alone into a device code object, and a harness built from the task alone loads both, checks both outputs, times them interleaved, and writes its result to a record file. None of the candidate's host code runs, and stdout is never read.

Checked on an MI350X node with hip_kernels.py timecheck:

• each reference against itself is not_faster at about 1.00 (vector_add, transpose, block_sum);
• a candidate whose static initializer prints a fake 10x timing line is measured as itself (not_faster, 1.00);
• a deliberately slow candidate is too_slow_to_measure within the default budget, and not_faster at 0.001 with a larger one;
• a tiled transpose is 1.07x the naive reference: not_faster at the default ratio of 1.1, and clean at ratio 1.0, in two runs.

Stacked on #613. Draft.

[Copilot]

🟡 Changes recommended

Incomplete cache identities allow stale timing and reward reuse after code changes, while unresolved relative candidate paths cause valid inputs to fail compilation.

2 open issues

• Include reference and timing-code hashes in checker identity · New
• Resolve candidate paths before running hipcc · New

What changed in this PR

Adds timing-based rewards to AORLTA’s check-and-reward pipeline, extending the HIP-kernel example with an optional speedup profile.

Changes:

• Documents available checks, reward functions, and composition rules.
• Adds noise-aware timing judgments and speedup scoring, leaving undecidable measurements unscored.
• Adds an isolated HIP timing harness, scoring configuration, and test coverage.

File	Description
aorlta/​tests/​reward/​test_timing.py	Tests timing statistics, measurement records, and reward invariants.
aorlta/​tests/​examples/​test_hip_kernels.py	Tests timing outcomes, harness generation, and profile integration.
aorlta/​src/​aorlta/​reward/​timing.py	Implements timing judgments and speedup grades.
aorlta/​src/​aorlta/​reward/​rewardfn/​speedup.py	Wraps correctness rewards with speedup scoring.
aorlta/​src/​aorlta/​reward/​adapters/​base.py	Allows measurement records on clean legs.
aorlta/​README.md	Links the rewards catalog.
aorlta/​examples/​hip-kernels/​hipk-speed.yaml	Defines the optional timing profile and GPU allocation.
aorlta/​examples/​hip-kernels/​hip_kernels.py	Adds the timing harness, adapter, reward registration, and CLI check.
aorlta/​docs/​REWARDS.md	Catalogs checks, scoring rules, and extension contracts.
aorlta/​docs/​ADAPTING.md	Links reward-extension guidance to the catalog.
aorlta/​configs/​hipk-speed-score.yaml	Configures a HIP-kernel scoring run with timing rewards.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Critical issues allow timed launches to skip transpose work and cached speedups to persist across GPU changes, making performance rewards unreliable.

2 open issues

• Reset buffers before every measured launch · New
• Include GPU identity in timing cache keys · New

2 resolved since last review

• Include reference and timing-code hashes in checker identity
• Resolve candidate paths before running hipcc

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Timed launches are never verified, allowing a stateful kernel to pass separate correctness checks while skipping measured work and reporting a false speedup.

No open issues 🎉

1 previously missed issue

In code that hasn't changed since last review

Verify outputs after every timed launch

aorlta/​examples/​hip-kernels/​hip_kernels.py:656

Timed launches are never verified, so a kernel with a persistent device counter can pass the separate fresh(0) checks while skipping measured work and earning a clean speedup. With the default 20 launches, computing only on calls congruent to 1 modulo 21 passes every correctness check; buffer resets do not clear device globals. Verify each measured launch's outputs and guards after its stop event and before the next reset, propagate mismatches as unsettled, and add a stateful-kernel regression.

2 resolved since last review

• Include GPU identity in timing cache keys
• Reset buffers before every measured launch

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Timed launches lack output validation and retain device-global state, allowing stateful kernels to skip work while passing freshness checks and reporting invalid speedups.

1 open issue

• Timed launches never validate outputs, allowing stateful skipped-work candidates · New

_{Review effort: Balanced}