Skip to content

[AMD/ROCM] ATOM DS R1 FP8 MTP 3 tokens support #984

Open
seungrokj wants to merge 2 commits intomainfrom
srok/atom_dsr1fp8_mtp3
Open

[AMD/ROCM] ATOM DS R1 FP8 MTP 3 tokens support #984
seungrokj wants to merge 2 commits intomainfrom
srok/atom_dsr1fp8_mtp3

Conversation

@seungrokj
Copy link
Copy Markdown
Collaborator

Hi @functionstackx @cquil11

This PR include this config

  • dsr1-fp8-mi355x-atom-mtp

Now atom supports mtp 3 tokens.

Regards,
Seungrok

cc. @ChuanLi1101 @andyluo7 @chunfangamd

Signed-off-by: seungrokj <seungrok.jung@amd.com>
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

1 similar comment
@github-actions
Copy link
Copy Markdown
Contributor

Thanks for the contribution! For vLLM & SGLang, please ensure that your recipes is similar to the official vLLM recipes and/or the SGLang cookbook

If it is not, please create a PR first before we can merge your PR into the master branch. Let's ensure that the documentation is first class such that the entire ML community can benefit from your hard work! Thank you

@seungrokj seungrokj changed the title [AMD/ROCM] ATOM DS R1 MTP 3 tokens support [AMD/ROCM] ATOM DS R1 FP8 MTP 3 tokens support Mar 31, 2026
@functionstackx
Copy link
Copy Markdown
Contributor

Thanks! @seungrokj can u update atom recipes for this?

@seungrokj
Copy link
Copy Markdown
Collaborator Author

Thanks! @seungrokj can u update atom recipes for this?

yes it's updated here https://github.com/ROCm/ATOM/blob/main/recipes/DeepSeek-R1.md#fp8-with-mtp-speculative-decoding-recommended

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

2 participants