Add Cosmos3 FP8 quantization cookbook and serving examples - #298
Conversation
New cookbooks/cosmos3/quantization/ ships a self-contained ModelOpt static-scale FP8 recipe: - src/: the Cosmos3 DiT model + full recipe (load, calibrate on real denoising, export a drop-in diffusers checkpoint). - notebooks/: quantize Nano/Super, Super Text-to-Image, and Super Image-to-Video (base + 4-step distilled). - Reproduces the shipped FP8 checkpoints bit-for-bit. Also add a "Quantized Checkpoints" section to the vLLM-Omni generator notebooks (audiovisual, action, transfer) and the vLLM reasoner notebook: serving the FP8 build is just --revision fp8. Signed-off-by: Wojciech Kutak <wkutak@nvidia.com>
wkutak
changed the title
|
@wkutak , who is the tester validate the inference function and user guide in this PR? The person should be reviewer of this PR as well, approve once tested working with log test result in google doc. |
Signed-off-by: Wojciech Kutak <wkutak@nvidia.com>
|
@KrzysztofTomala - please review and approve this PR once you completed testing. |
|
vLLM cookbook The model download and server startup works fine. I had to remove from the video example as the requests were failing, but the issue is happening also on the non-FP8 model. |
|
vLLM-Omni cookbooks |
|
Quantization cookbooks Everything works, there are a couple of warning, but nothing crucial. |
KrzysztofTomala
left a comment
KrzysztofTomala
left a comment
There was a problem hiding this comment.
Overall everything works as expected, all of the serious problems are not FP8 related
…d99f1254459ef3a4f1bd4263' Signed-off-by: Wojciech Kutak <wkutak@nvidia.com>
… README, small fixes Signed-off-by: Wojciech Kutak <wkutak@nvidia.com>
|
The transfer example issue @KrzysztofTomala mentioned will be fixed here: #299 |
|
@wkutak Can you please fix the merge conflict? |
Signed-off-by: Wojciech Kutak <wkutak@nvidia.com>
|
Merge conflicts fixed, @rickzw please review the PR |
4 participants
New cookbooks/cosmos3/quantization/ ships a self-contained ModelOpt static-scale FP8 recipe:
Also add a "Quantized Checkpoints" section to the vLLM-Omni generator notebooks (audiovisual, action, transfer) and the vLLM reasoner notebook: serving the FP8 build is just --revision fp8.