Skip to content

Keep running PD services on device alerts and recover after idle - #37

Draft
huangxinV587 wants to merge 3 commits into
mainfrom
branch/pd-recovery-vllm-r1
Draft

huangxinV587 wants to merge 3 commits into
mainfrom
branch/pd-recovery-vllm-r1

Conversation

@huangxinV587

@huangxinV587 huangxinV587 commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

运行中的 vLLM P/D 服务在设备健康或故障事件变化时继续运行,并记录变化;只有复核确认其他进程占用 NPU 时,看护才停止固定容器。设备状态变化与外部占卡同时发生时,仍优先识别占卡冲突。

容器退出后的两小时空闲恢复仍要求设备状态和故障事件与验收基线一致。SSH 登录提示支持读取本地消息文件,不将部署现场的提示文字写入仓库。

验证:python -m unittest discover -s tests -v(245 项,10 项按环境跳过);新增测试覆盖健康变化、故障事件变化、冲突复核和故障状态保留。未宣称完成真实设备的两小时恢复观察。高影响看护行为仍需独立复核,PR 保持草稿。

@huangxinV587 huangxinV587 changed the title Recover pinned vLLM PD containers after an idle window Keep running PD services on device alerts and recover after idle Nov 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant