[1/N] Add optional NCCL m2n weight sync backend#159
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces an experimental NCCL Xfer weight synchronization backend (UpdateWeightFromNcclXfer) for non-colocated rollout engines, complete with command-line options, validation, layout analysis, and unit tests. The review feedback highlights two important improvements: caching the fallback reason during initialization to prevent a performance bottleneck from traversing model parameters on every weight update, and defensively handling quantization_config to avoid potential AttributeError crashes when it is passed as a configuration object rather than a dictionary.
Add the non-colocate backend switch and an explicit NCCL Xfer fallback wrapper so the experimental path can be enabled without changing the stable broadcast default. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: princepride <wangzhipeng628@gmail.com>
Keep the MVP PR focused by dropping standalone argument parser tests for the opt-in backend flag. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: princepride <wangzhipeng628@gmail.com>
Avoid repeated model scans during weight updates and handle object-style quantization configs when deciding broadcast fallback. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: princepride <wangzhipeng628@gmail.com>
Add the explicit zip strictness required by Ruff and include formatter-only cleanup so the branch passes local hooks. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: princepride <wangzhipeng628@gmail.com>
37f21e8 to
0e9795d
Compare
|
Superseded by #335, re-opened from the fork branch |
Summary
This PR is the first step toward the NCCL m2n rollout weight sync proposal in #52.
It adds an opt-in non-colocated weight sync backend:
The existing gather-broadcast path remains the default. Colocated rollout engines continue using the existing CUDA IPC path.
Changes
UpdateWeightFromNcclm2nas an optional non-colocated updater.Stack
This is
[1/N].Follow-up PRs will cover the native
ncclm2nReshardWithWindowruntime path, vLLM worker receive integration, staging/window management, and performance comparison against the legacy gather-broadcast path.Test Plan