feat(hyvla): Hy-Embodied-0.5-VLA Orin SM87 — INT8 W8A8 + fused kernels

[DXICM]

Summary

Adds Jetson Orin SM87 port of Hy-Embodied-0.5-VLA using INT8 W8A8 quantization. Depends on #167 (Thor port with shared kernels).

Orin-specific additions:

• frontends/torch/hyvla_orin.py — HyVLATorchFrontendOrin (subclasses Thor frontend, overrides quantization and GEMM dispatch for SM87)
• models/hyvla/pipeline_orin.py — Orin compute path (INT8 GEMM via cutlass_sm80_int8_rowwise, BF16 fallback)
• _PIPELINE_MAP: ("hyvla","torch","rtx_sm87") + _SM87_ALLOWED entry
• tests/test_orin_hyvla05_dispatch.py — dispatch resolution smoke test (pure pytest, no GPU/checkpoint needed)

Shared kernels reused from #167 (compiled for SM87 via FLASHRT_HAVE_HYVLA_ORIN):

• hyvla_fused_thor.cu — fused RoPE+QK-Norm+KV-write megakernel (plain CUDA, no SM110-only instructions)
• hyvla_vit_fuse.cu — ViT residual-add + LayerNorm

Performance (real image, Orin SM87):

Metric	Value
E2E predict	277.5 ms (~11.3× vs reference eager)
Action cosine vs reference	≥ 0.999

Quantization approach: INT8 W8A8 with per-row dynamic scales. Orin SM87 has no native FP4/FP8 tensor cores, so INT8 integer GEMMs (hardware I2F dequant) are the optimal low-precision tier.

What is not changed

• No Thor-path code modified (additive Orin frontend + pipeline only)
• Shared kernel math unchanged; SM87 compilation gate is additive in CMake

Test plan

• python -m pytest tests/test_orin_hyvla05_dispatch.py -q
• cmake -B build -S . -DGPU_ARCH=87 && cmake --build build -j$(nproc) --target flash_rt_kernels
• python -c "from flash_rt.hardware import resolve_pipeline_class; print(resolve_pipeline_class('hyvla','torch','rtx_sm87'))"
• E2E validation with real checkpoint on Orin hardware

[LiangSu8899]

Thank you for following the Thor work with a separate HyVLA Orin implementation. Keeping the Orin frontend and pipeline as an additive layer over #167 is the right general dependency shape, and the Orin-only diff does not modify the Thor runtime implementation. Codex reviewed the current head against FlashRT's long-term maintenance standards. The following items should be addressed before merge.

Required changes

1. Merge and correct feat(hyvla): Hy-Embodied-0.5-VLA Thor SM110 — dynamic FP8 + fused megakernels #167 first, then rebase this PR

   This is a stacked PR and currently includes the full feat(hyvla): Hy-Embodied-0.5-VLA Thor SM110 — dynamic FP8 + fused megakernels #167 commit. Please first complete feat(hyvla): Hy-Embodied-0.5-VLA Thor SM110 — dynamic FP8 + fused megakernels #167's model-level build isolation, kernel contract checks, routing fix, and validation. After feat(hyvla): Hy-Embodied-0.5-VLA Thor SM110 — dynamic FP8 + fused megakernels #167 is merged, rebase this PR onto the new main so the review and rollback surface contains only the Orin additions.

2. Keep the SM87 shared kernels behind the HyVLA model option

   The shared fused RoPE/QK-Norm/KV-write and ViT fusion sources are currently compiled into every SM87 flash_rt_kernels build. They should require both FLASHRT_ENABLE_HYVLA=ON and GPU_ARCH=87, with matching symbol guards. An unrelated SM87 model build should remain unchanged when HyVLA is disabled.

3. Add an SM87 fail-fast check

   HyVLATorchFrontendOrin subclasses the Thor frontend but neither constructor validates the actual CUDA capability. The hardware="rtx_sm87" argument is currently descriptive rather than enforced. Please reject non-SM87 devices before checkpoint loading, weight quantization, and large CUDA allocation, with an optional development override only if it is explicit and documented.

4. Make the precision defaults and evidence internally consistent

   The source comments label use_int8_vlm_ffn and use_int8_exp as experimental and not validated at cosine >= 0.999, while both are part of the documented default path and the document reports that this default passes the gate. Please establish one authoritative contract: update the comments if the default path is validated, and commit the exact fixed-noise precision test that proves it. Features documented as failing the gate (use_int8_vlm, use_int8_vit) should remain clearly opt-in and must never be enabled indirectly by the standard API.

5. Commit the tests referenced by the documentation

   The Orin document references test_orin_hyvla05_e2e_check.py, test_orin_hyvla05_graphsafe.py, and test_orin_hyvla05_stageprof.py, but none of these files are present. The only committed test is a nine-line dispatch check. Please add at least:

   • fixed-noise BF16 and default-INT8 reference cosine gates
   • eager-versus-graph and repeated-replay stability
   • fused-versus-unfused attention-prep and ViT fusion checks
   • hardware fail-fast and missing-symbol behavior
   • input-shape boundaries inherited from the Thor frontend

   Benchmark/profiling scripts can remain outside pytest, but correctness and graph-safety gates need to be versioned with the implementation.

6. Validate inherited public inputs and clean the patch

   The Orin override inherits the same assertion-based prompt check and accepts oversized state vectors or arbitrary noise shapes before graph execution. Please convert these to explicit contract checks. git diff --check also currently reports an extra blank line at EOF in flash_rt/frontends/torch/hyvla_orin.py; please clean that before merge.

Pre-merge checklist

• feat(hyvla): Hy-Embodied-0.5-VLA Thor SM110 — dynamic FP8 + fused megakernels #167 is corrected and merged; this PR is rebased to contain only the Orin delta
• HyVLA OFF leaves the default SM87 build and symbol surface unchanged
• HyVLA ON + SM87 build, import, and expected-symbol checks pass
• Non-SM87 devices fail before checkpoint loading or quantization
• Default INT8 flags, comments, docs, and public API routing describe the same validated tier
• Below-gate INT8 experiments remain explicit opt-ins
• Fixed-noise cosine, graph/eager equivalence, and replay-stability tests are committed and pass on Orin
• Fused/unfused kernel reference tests and missing-symbol behavior are covered
• Prompt, image, state, and noise boundary tests pass
• git diff --check, Python compilation, dispatch tests, SM87 build, and real-checkpoint smoke pass

Please refer to the repository's PR Review Checklist and Adding a New Model guides.

This is a Codex-assisted maintainability review. The Orin separation is directionally good; the build isolation inherited from #167, hardware enforcement, precision contract, and missing reproducibility gates need to be completed before merge.

[DXICM]

Thanks for the review. The Orin delta has been corrected and rebased onto the fixed #167 head (1eea7f1); once #167 merges, this PR will be rebased onto main so the review surface contains only the Orin additions. Item-by-item:

1. Stacking — done as above; the branch now carries #167's corrected commits plus one Orin fix commit.

2. SM87 shared kernels behind the HyVLA option — inherited from the corrected #167: FLASHRT_ENABLE_HYVLA AND GPU_ARCH STREQUAL "87" gates the shared fused RoPE/QK-Norm/KV-write and ViT fusion sources with matching pybind guards. Production Orin matrix re-verified: OFF build exports no HyVLA symbols; ON build exports exactly hyvla_rope_qknorm_kvwrite_bf16 / hyvla_vit_add_layer_norm_bf16, and Thor-only symbols stay absent.

3. SM87 fail-fast — HyVLATorchFrontendOrin now sets _REQUIRED_CAPABILITY = (8, 7) / _ARCH_NAME = "Jetson Orin SM87" and reuses the inherited _require_arch() gate, which runs before checkpoint loading, quantization, and CUDA allocation. The dev override is the same documented FLASHRT_HYVLA_FORCE_ARCH env var. Covered by tests/test_orin_hyvla05_arch_gate.py (mocked CUDA, no device needed): rejects no-CUDA and non-SM87 capabilities, accepts SM87, honors the env override, and confirms FP4 is rejected before any CUDA work.

4. Precision contract consistency — one authoritative contract now: use_int8_vlm_ffn and use_int8_exp are labeled the validated default tier (cosine >= 0.999), matching the doc and the standard API routing; use_int8_vlm and use_int8_vit are labeled below-gate opt-ins and are never enabled by the default path. The default tier was re-validated on production Orin: INT8 vs local BF16 action cosine 0.999162 (>= 0.999 gate), MAE 6.39e-3.

5. Committed tests — the gates the doc references are now versioned:

• tests/test_orin_hyvla05_e2e_check.py — fixed-noise BF16-vs-default-INT8 cosine gate (>= 0.999), default-tier fusion-flag checks, eager determinism, oversized-state and wrong-noise-size boundaries (checkpoint-gated via FLASHRT_HYVLA_CHECKPOINT, sequential BF16-then-INT8 loads to respect Orin unified memory);
• tests/test_orin_hyvla05_graphsafe.py — graph-vs-eager (>= 0.9999), replay stability (bitwise), fused-vs-unfused attention-prep and ViT add+LayerNorm checks;
• tests/test_orin_hyvla05_arch_gate.py — hardware fail-fast and FP4 rejection, checkpoint-free.
  The doc's file table and Verification section now point only at committed files; stage-profiling remains outside pytest and is no longer referenced as an in-tree script.

6. Inherited contracts and patch hygiene — the Orin predict_actions now uses a stable RuntimeError for the missing-prompt state (deterministic under python -O), rejects oversized state vectors, and validates noise element count before graph lookup. The EOF blank line in hyvla_orin.py is removed; git diff --check is clean.

Production Orin re-test of this branch is queued (pull + FLASHRT_HYVLA_CHECKPOINT=/mnt/models/Hy-Embodied-0.5-VLA-RoboTwin pytest run); results will be reported here.