Skip to content

qsmla: adapt Shared-B TransB usage to pto-spec #257 (compile fix for TileOP-API >= 748ac69) - #141

Closed
CYR-Firework wants to merge 2 commits into
PTO-ISA:mainfrom
CYR-Firework:qsmla-transb-257-adaptation
Closed

CYR-Firework wants to merge 2 commits into
PTO-ISA:mainfrom
CYR-Firework:qsmla-transb-257-adaptation

Conversation

@CYR-Firework

Copy link
Copy Markdown
Contributor

Summary

TileOP-API #104 (01d62c4) redefined the cooperative Shared-B TransB control as a physical-storage-shape declaration per pto-spec #257 (ASL BundleMatrixSharedBPrimarySchemaLegal):

TransB physical Shared source
0 [N, K] (K contiguous)
1 [K, N] (N contiguous)

The QSMLA kernel still used the legacy math-transpose-flag reading, so since TileOP-API 748ac69 it fails the compile-time effective-K / cooperative-D shape static_asserts (template_asm.hpp:4219/4232) — QSMLA is outside the microbenchmark guard coverage, so this was not caught by the daily reports.

This PR adapts the 5 call sites in quant_sparse_flash_mla.hpp. Both K and V are staged as SharedMatrixRight<kvdtype, kTk, kTd> (one KV token per row, D contiguous), so:

  • QK (Q @ K^T): the physical [kTk, kTd] layout IS the [N, K] form for the K^T operand → drop transpose_b() (TransB=0) — 4 sites
  • PV (P @ V): the same physical layout is the [K, N] form for the V operand (N contiguous) → add transpose_b() (TransB=1) — 1 site

Note the transposition itself does not disappear — under the new contract it is carried implicitly by the physical-layout declaration (TransB=0 ⇒ the engine transposes on read), which is why the numeric results below are bitwise identical.

Verification

Environment: TileOP-API 64ef554 + SuperScalarModel fix/gfrun-shared-tmatmul-layout-257 @ 6a60fb5 (the model-side TransB adaptation de3ea07f) + LLVM 10abd6e+, baseline tag ops-20260914 + this patch.

gfrun 4-PE (-s softcore.multiThreadNum=4, HIF8 in / BF16 out, atol=rtol=2e-2 vs CPU golden):

Case Result max_abs
swa_small 32768/32768 (100%) 0.004920
csa_small 32768/32768 (100%) 0.004677
ori_sparse_small 32768/32768 (100%) 0.009071

max_abs is bitwise identical across every baseline before and after the #257 contract change (ops-20260908 → ops-20260914), confirming the adaptation direction.

Compile: 0 errors / 0 warnings for all three IMPLs (swa_tadd_4pe / csa_tadd_4pe / ori_sparse_tadd_4pe, QSMLA_DTYPE=HIF8, embedded inputs).

References

TileOP-API PTO-ISA#104 (01d62c4) redefined the cooperative Shared-B TransB
control as a physical-storage-shape declaration per pto-spec #257
(ASL BundleMatrixSharedBPrimarySchemaLegal):
  TransB=0 -> physical [N, K] (K contiguous)
  TransB=1 -> physical [K, N] (N contiguous)

QSMLA stages both K and V as SharedMatrixRight<kvdtype, kTk, kTd>
(one KV token per row, D contiguous), so:
- QK (Q @ K^T): the physical [kTk, kTd] layout IS the [N, K] form
  for the K^T operand -> drop transpose_b() (TransB=0)
- PV (P @ V): the same physical layout is the [K, N] form for the V
  operand (N contiguous) -> add transpose_b() (TransB=1)

Without this adaptation the kernel fails the compile-time effective-K
and cooperative-D shape static_asserts against TileOP-API >= 748ac69.

Verified with SuperScalarModel fix/gfrun-shared-tmatmul-layout-257
@ 6a60fb5 (the model-side TransB fix, de3ea07f), TileOP-API 64ef554,
LLVM 10abd6e+: gfrun 4-PE on swa_small / csa_small / ori_sparse_small
all 32768/32768 (100%) with max_abs bitwise identical to the
pre-PTO-ISA#104 baselines.

Refs: PTO-ISA/pto-spec#255, pto-spec#257, LinxISA/Linx-TileOP-API#103,
LinxISA/Linx-TileOP-API#104, LinxISA/Linx-TileOP-API#122
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.

Same convention as mega_moe_sim.cpp: write the pass value to the
test-finisher address after BENCHEND so gfsim --dump-memory can
complete (requires SuperScalarModel#665 finisher observation fix).
Numerical verification remains gfrun R2=0.

Not part of PR#141 (which only contains the TransB adaptation);
this is a separate local commit for gfsim performance collection.
@CYR-Firework

Copy link
Copy Markdown
Contributor Author

Superseded by #151, which consolidates this work (commit-for-commit, rebased on current main) together with the single-PE CSA kernel. Closing in favor of #151.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant