[executorch][cuda] Optimize short-query INT5 matvec kernels

[Gasoonjia]

Stack from ghstack (oldest at bottom):

• -> [executorch][cuda] Optimize short-query INT5 matvec kernels #21458
• [executorch][cuda] Optimize short-query INT6 matvec kernels #21457
• [executorch][cuda] Optimize short-query INT4 matvec kernels #21456

Add specialized M=1 and M=4 short-query paths for the planar INT5 W5A8 matvec operator. These are the latency-sensitive single-row and four-row shapes. M=2/M=3 retain the generic fallback path; this change does not claim those shapes as optimized.

The M=1 implementation shares safe gs32 decode/accumulate logic through a compile-time ROWS device template. M=4 remains explicit because KernelAgent experiments with the generalized template increased register pressure and regressed latency. The final kernels were generated and tuned with KernelAgent.

A100 kernel microbenchmarks versus the previous generic path:

• M1: 1.09971 -> 0.70441 ms (1.56x; no regression)
• M4: 1.96445 -> 1.12976 ms (1.74x)
• M2/M3: no specialized dispatch; generic fallback remains unchanged

Correctness tests cover M=1,2,3,4 with three randomized seeds; ptxas reports zero spills.

Differential Revision: D114032323

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21458

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unclassified Failures

As of commit 0d04c58 with merge base e47f373 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t e26d1110344c8d8e920806d5d19d63abbbe8874d236cbe6bda7d40bd1e6e06d1 /exec failed with exit code 1
• pull / unittest-editable / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 735db75923d40a69bd3b722b10fadbeab1777cf0cb8fd4166a8f3a57fcac55c6 /exec failed with exit code 1

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Test CUDA Builds / export-model-cuda-artifact (SocialLocalMobile, Qwen3.5-35B-A3B-HQQ-INT4, quantized-int4-tile-packed) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  [runner-container-hooks] FATAL: received SIGINT
• Test CUDA Builds / unittest-cuda-runtime / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  RuntimeError: Command docker exec -t f386519535e0a03237c721743d713db327544e6d12bed7f0e523e61f51bb7a53 /exec failed with exit code 2

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.