feat(hyvla): Hy-Embodied-0.5-VLA Thor SM110 — dynamic FP8 + fused megakernels

[DXICM]

Summary

Adds Hy-Embodied-0.5-VLA (HunYuan MoT dual-tower + flow matching) to FlashRT on Jetson Thor SM110. Orin SM87 port lands in a follow-up PR based on this branch.

New kernels (csrc/kernels/, SM110-gated via FLASHRT_HAVE_HYVLA_THOR):

• hyvla_fused_thor.cu — fused RoPE + QK-Norm + KV-write megakernel (replaces 11 launches per attention block; also SM87-portable)
• hyvla_vit_fuse.cu — ViT residual-add + LayerNorm fusion (also SM87-portable)
• hyvla_quant_fp8_thor.cu — single-CTA dynamic per-tensor FP8 quantize (graph-safe, 1 launch vs 4)
• hyvla_ffn_fp8_thor.cu — FFN gate/up+SiLU and down+residual fused GEMM kernels

Architecture (per docs/adding_new_model.md rules 1-4):

• models/hyvla/pipeline_thor.py — Thor compute path
• frontends/torch/hyvla_thor.py — HyVLATorchFrontendThor
• frontends/torch/_hyvla_thor_spec.py — declarative weight spec
• _PIPELINE_MAP: ("hyvla","torch","thor") one-to-one
• load_model(config="hyvla") dispatches through the standard VLA path
• use_fp4=True on Thor routes to NVFP4 FFN tier

Performance (real image, Thor SM110):

Metric	Value
E2E predict	159.6 ms (~5.8× vs reference eager ~930 ms)
Action cosine vs reference	0.999706

CMake/binding compliance:

• All kernels gated behind if(GPU_ARCH STREQUAL "110") + FLASHRT_HAVE_HYVLA_THOR
• Every m.def uses matching #ifdef guard; no unconditional binding references gated symbols
• Shared kernels (fused_thor, vit_fuse) also compile for SM87 under FLASHRT_HAVE_HYVLA_ORIN

What is not changed

• No existing kernel math, signatures, or bindings modified
• flash_rt_kernels symbol surface is additive only
• executors/torch_weights.py adds model-agnostic ToBf16 (mirrors existing ToFp16/ToFp32)

Test plan

• cmake -B build -S . -DGPU_ARCH=110 && cmake --build build -j$(nproc) --target flash_rt_kernels
• python -c "from flash_rt import flash_rt_kernels; print(flash_rt_kernels.__file__)"
• python -c "from flash_rt.hardware import resolve_pipeline_class; print(resolve_pipeline_class('hyvla','torch','thor'))"
• E2E validation with real checkpoint on Thor hardware

[LiangSu8899]

Thank you for contributing the native Hy-Embodied-0.5-VLA Thor path. The model/frontend/pipeline split follows the repository's intended structure, the kernel names are model-scoped, and the matching CMake definitions and pybind guards are a good start. Codex performed a strict review against FlashRT's long-term maintenance rules. The following items should be addressed before merge.

Required changes

1. Add a model-level build option, not only an architecture gate

   The new HyVLA sources are currently compiled into the common flash_rt_kernels module for every SM110 build, and two of them are also compiled for every SM87 build. Architecture gating prevents unsupported code generation, but it does not provide model-selected compilation: users building unrelated models on Thor or Orin still pay the compile, link, binary-size, and symbol-surface cost.

   Please add an option disabled by default, for example:

   option(FLASHRT_ENABLE_HYVLA "Build HyVLA model kernels" OFF)

   Gate the sources, compile definitions, and pybind symbols together. The expected matrix is HyVLA OFF on all architectures, HyVLA ON + SM110 for the Thor kernel set, and HyVLA ON + SM87 for only the shared portable kernels.

2. Fix the public load_model(..., use_fp4=True) route

   In the HyVLA FP4 branch, pipe_cls is changed to HyVLATorchFrontendThor, but use_fp4 is then set to False. The generic kwarg construction does not pass use_fp4=True to this frontend, so the advertised HyVLA FP4 route actually constructs the default non-FP4 frontend.

   Please preserve a dedicated HyVLA FP4 routing flag and pass it explicitly to the selected frontend. Add a routing test that inspects or mocks the constructed frontend and proves that use_fp4=True reaches it. The standard load_model path should also have an explicit, tested way to select the validated use_fused production tier if that is part of the supported API.

3. Fail early on unsupported hardware and avoid implicit remote-code execution

   HyVLATorchFrontendThor does not validate SM110 before loading weights and allocating CUDA state. Please check torch.cuda.get_device_capability() at the beginning of initialization and report a clear error on unsupported devices.

   The tokenizer currently uses trust_remote_code=True. That allows checkpoint-provided Python code to execute during model loading and conflicts with the stated self-contained runtime boundary. Please use the standard tokenizer path without remote code. If custom remote code is genuinely required, make it an explicit, documented opt-in rather than the default.

4. Validate the public fused-kernel contracts before launch

   The pybind APIs accept raw pointers and unconstrained dimensions, while the implementations rely on production-shape assumptions. At minimum, validate the supported HyVLA contract before launch:

   • fused RoPE/QK-Norm/KV-write: positive S, nq, nkv, S_tot; supported hd (currently 128); valid off and off + S <= S_tot; positive eps and kv_rep
   • ViT add + LayerNorm: positive rows, supported/even dimension, positive eps
   • FFN kernels: positive dimensions, required K alignment, and N/Nout divisibility by the 32-column tile

   Unsupported inputs must raise a Python exception rather than launch invalid grids or silently leave output columns unwritten.

5. Commit the validation that the documentation claims

   docs/hyvla05_thor_sm110.md references multiple precision, graph-safety, and stage tests under tests/test_thor_hyvla05_*, but none of those files are present in this PR. The PR itself adds no test file, and every test-plan item is currently unchecked. Performance and cosine numbers are useful evidence, but they need a reproducible in-tree gate.

6. Tighten the frontend input contract

   predict_actions() uses an assert for the missing-prompt state and does not reject state vectors larger than max_state_dim or invalid noise shapes. Please replace the assertion with a stable exception and validate state/noise/camera shapes before graph lookup or copy. Public API errors should be deterministic in both normal and python -O execution.

Pre-merge checklist

• FLASHRT_ENABLE_HYVLA=OFF leaves the default SM87/SM110 build and symbol surface unchanged
• HyVLA ON + SM110 build/import/symbol checks pass
• HyVLA ON + SM87 exposes only the intended shared portable symbols
• load_model(..., config="hyvla", use_fp4=True) demonstrably enables the FP4 frontend path
• Thor rejects non-SM110 devices before checkpoint loading
• Tokenizer loading does not execute remote checkpoint code by default
• Fused kernels have host-side contract validation and numerical reference tests
• Prompt, state, noise, image-shape, eager/graph, and replay-stability tests are committed
• Claimed fixed-noise action cosine and graph-safety gates are reproducible from files in this repository
• git diff --check, Python compilation, default build, and HyVLA smoke tests pass

Please refer to the repository's PR Review Checklist and Adding a New Model guides.

This is a Codex-assisted maintainability review. The architecture and optimization direction are reasonable; the model-selected build boundary, public routing correctness, safe input contracts, and reproducible validation need to be closed before this becomes a supported path on main.

[DXICM]

Thanks for the review. All six required changes are addressed, and re-validated on a production Jetson Orin (sm_87, CUDA 12.2, torch 2.3.0).

1. Model-level build option — option(FLASHRT_ENABLE_HYVLA "Build HyVLA model kernels" OFF) added; sources, compile definitions, and pybind symbols are gated together. Matrix verified on SM87:

• OFF: default build/symbol surface unchanged (no HyVLA symbols).
• ON + SM87: only the shared portable kernels export (hyvla_rope_qknorm_kvwrite_bf16, hyvla_vit_add_layer_norm_bf16 = True; Thor-only hyvla_quant_fp8_dyn_bf16 / hyvla_ffn_gu_silu_bf16 = False).
• ON + SM110: full Thor kernel set (build-gated; SM110 device run queued for the Thor hardware).

2. load_model(..., use_fp4=True) route — a dedicated HyVLA FP4 routing flag now carries through flash_rt/api.py and explicitly passes use_fp4=True to the selected frontend. tests/test_hyvla_fp4_routing.py mocks the constructed frontend and proves:

• use_fp4=True reaches the frontend;
• the validated FP8 production tier selects use_fused=True;
• the default route does not enable FP4;
• SM87 FP4 degrades to the INT8 path with an explicit warning instead of silently claiming FP4.
  Production: 3 passed, 1 skipped.

3. Fail-fast + no implicit remote code — HyVLATorchFrontendThor validates torch.cuda.get_device_capability() at the top of initialization (before checkpoint loading and CUDA allocation) and raises a clear RuntimeError on non-SM110 devices; dev override is the documented, HyVLA-specific FLASHRT_HYVLA_FORCE_ARCH env var. The tokenizer loads without trust_remote_code. Covered by tests/test_hyvla_arch_gate.py (4 passed).

4. Fused-kernel contracts — host-side validation added before launch for all five fused kernels (RoPE/QK-Norm/KV-write, ViT add+LayerNorm, dynamic FP8 quant, FFN gu/silu, FFN dn/res): positive dims, supported hd == 128, valid off + S <= S_tot, positive eps/kv_rep, required K alignment, and 32-column tile divisibility. Invalid inputs raise Python exceptions instead of launching invalid grids. Covered by tests/test_hyvla_kernel_contracts.py (5 passed, 3 skipped on SM87 where the Thor-only symbols are correctly absent).

5. Committed validation — the PR now ships test_hyvla_thor_dispatch.py (3 passed), test_hyvla_arch_gate.py (4 passed), test_hyvla_fp4_routing.py (3 passed / 1 skipped), test_hyvla_kernel_contracts.py (5 passed / 3 skipped), and test_hyvla_thor_graphsafe.py. docs/hyvla05_thor_sm110.md was updated to reference only committed files.

6. Frontend input contract — the missing-prompt assert is now a stable RuntimeError (deterministic under python -O); state/noise shapes are validated before graph lookup and copy, with oversized state vectors rejected.

Combined production run: 43 passed / 0 failed / 5 skipped (all skips are intended hardware gates). Remaining SM110-device checklist items (Thor build/import/symbol checks, fixed-noise cosine and graph-safety gates on real Thor hardware) are queued for the Thor device and will be reported here.