Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -162,10 +162,21 @@ def vocab_ce(
)


def vocab_ce_loss(
hidden: torch.Tensor,
weight: torch.Tensor,
labels: torch.Tensor,
z_loss_weight: float = 0.0,
ignore_index: int = -100,
) -> torch.Tensor:
"""Documented public alias of :func:`vocab_ce` (README/CARD API)."""
return vocab_ce(hidden, weight, labels, z_loss_weight, ignore_index)


def backend_marker(x: torch.Tensor) -> torch.Tensor:
if ops is None:
return x
return ops._flashrt_training_package_marker(x)


__all__ = ["vocab_ce", "reference_vocab_ce", "backend_marker"]
__all__ = ["vocab_ce", "vocab_ce_loss", "reference_vocab_ce", "backend_marker"]
25 changes: 25 additions & 0 deletions fp8-cross-attention-blackwell/VALIDATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,28 @@ both sides of every 128-token boundary are release blockers.

Installed-artifact correctness and `torch.compile(fullgraph=True)` remain
mandatory after the HF Jobs build. Source validation does not replace them.

## Installed artifact on NVIDIA Thor (SM110)

The installed-artifact gate (missing when this card was written) is now closed
on Thor `sm_110a`. Build notes:

- Requires CUTLASS **4.4.0** (the package declares `cutlass_4_4`); 4.5.x moved
`SM100_MMA_F8F6F4_SS` to a class template and breaks `csrc/fmha77`.
- `cutlass/util/packed_stride.hpp` is vendored by `fp4-gemm` and must be on the
include path (same local-build workaround as `fp8-gemm`).
- Compiled with `-gencode arch=compute_110a` and the FMHA sm100 kernel runs
natively on Thor (no fallback required, unlike `fused-mlp-megakernels`).

Command:

```bash
python fp8-cross-attention-blackwell/tests/test_fp8_cross_attention_blackwell.py \
--backend installed --artifact <installed-dir> --mode full
```

Result: 9/9 numeric rows, invalid-head rejection, CUDA Graph replay, and
`torch.compile(fullgraph=True)` all passed on SM110. Worst row across the
matrix (`B1,Sq786,Sk7984,Hq28,Hkv4,D128`): `max=0.00025749`, `p99=0.00012207`,
`mean=0.00003710`, `cosine=0.99978602` — comfortably inside the
`max<=0.004 / cosine>=0.9995` gate. Other rows held `cosine>=0.999778`.
Original file line number Diff line number Diff line change
Expand Up @@ -221,8 +221,8 @@ def main() -> None:

if not torch.cuda.is_available():
raise SystemExit("CUDA is required")
major, _minor = torch.cuda.get_device_capability(0)
if major < 12:
major, minor = torch.cuda.get_device_capability(0)
if major not in (10, 11, 12) or (major == 10 and minor not in (0, 3)):
raise SystemExit("sageattention2-blackwell requires Blackwell-class CUDA capability")

torch.manual_seed(2026)
Expand Down