Refactor quantization into a shared extension/llm/export/quant package (#21044)

[metascroy]

Summary:

Consolidates the quantization/packing logic that was previously duplicated
across per-model directories into a single reusable package under
extension/llm/export/quant, and introduces a model-free checkpoint loader.

Key changes:

• New extension/llm/export/quant package exposing a stable API:
  • convert.py: Convert, to_exportable/to_default, fuse_along_output, identity
  • quantize.py: quantize_model/quantize_weight/quantize_stream + dequantize
  • recipe.py: QuantConfig / QuantRecipe / QuantRule
• New extension/llm/export/load.py: model-free (iter_checkpoint) and
  model-aware (load_checkpoint) readers that stream GGUF / safetensors weights,
  remap names to FQNs, and wrap quantized subclasses in portable Exportable*
  form. Backend-specific layout (CUDA coalesced int4, MLX gather buffers) is a
  separate terminal pass.
• Extended int4.py / gguf.py to plug into the shared conversion path.
• Removed the old per-model examples/models/gemma4_31b/quant/ packers
  (pack.py, pack_cuda.py, pack_mlx.py) and moved packing into
  examples/models/gemma4_31b/cuda_packers.py, wiring gemma4_31b and
  qwen3_5_moe onto the shared package.
• Added unit tests for the new package and the gemma4_31b packers/pipelines.

Reviewed By: Gasoonjia

Differential Revision: D112867629

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/21044

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Unrelated MacOS MPS test jobs time out (30 min) due to stale cpp_extension lock file poisoning self-hosted runners

❌ 1 New Failure, 1 Unrelated Failure, 3 Unclassified Failures

As of commit d18198d with merge base 564e655 ():

NEW FAILURE - The following job has failed:

• pull / unittest-buck / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 9f8dc06889511006f90212d76f2fcfb07b0e55fd6f8c8bbd065f4bf79d14d183 /exec failed with exit code 3

UNCLASSIFIED FAILURES - DrCI could not classify the following jobs because the workflow did not run on the merge base. The failures may be pre-existing on trunk or introduced by this PR:

• Test CUDA Builds / export-model-cuda-artifact (facebook, dinov2-small-imagenet1k-1-layer, non-quantized) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  [OSDC] Step script exited with code 1. This is a script/workflow error, not an infrastructure issue. Check the step logs above for the actual failure.
• Test CUDA Builds / export-model-cuda-artifact (nvidia, diar_streaming_sortformer_4spk-v2, non-quantized) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  [OSDC] Step script exited with code 1. This is a script/workflow error, not an infrastructure issue. Check the step logs above for the actual failure.
• Test CUDA Builds / export-model-cuda-artifact (nvidia, parakeet-tdt, quantized-int4-tile-packed) / linux-job (gh) (this job did not run on the merge base, so DrCI cannot tell whether the failure is pre-existing)
  [OSDC] Step script exited with code 1. This is a script/workflow error, not an infrastructure issue. Check the step logs above for the actual failure.

BROKEN TRUNK - The following job failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest-buck / macos / macos-job (gh) (trunk failure)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[meta-codesync]

@metascroy has exported this pull request. If you are a Meta employee, you can view the originating Diff in D112867629.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[JakeStevens]

if the cuda packers are shared (at least across gemma4_31b and
qwen3_5_moe), why not also have this in something like extension.llm.cuda_packers?