Skip to content

SGLang multi-node: auto-negotiated dist_init port can collide with pre-allocated engine ports #7

Description

@xy200303

In _prepare_sgl_engines (angelspec/inference/factory.py), Step 2 picks a free port for dist_init_addr starting from the default start_port=10000:

https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L288-L295

ip, port = ray.get(
    [head_engine.get_node_ip.remote(), head_engine.find_free_port.remote()],
    timeout=120,
)

find_free_port only probes ports; it never reserves them. Step 2.5 then restarts its sequential allocation at the literal next_start = 10000 without accounting for the port just handed out:

https://github.com/Tencent/AngelSpec/blob/main/angelspec/inference/factory.py#L297-L309

next_start = 10000
for i in range(num_engines):
    port = ray.get(engines[i].find_free_port.remote(start_port=next_start, consecutive=2), ...)
    pre_allocated_ports[i] = port
    next_start = port + 2

On a fresh node, Step 2 returns 10000 and Step 2.5 immediately hands engine 0 (which is the replica-0 head on the same node) ports 10000/10001. SglEngine.init then sets port=10000, nccl_port=10001, and dist_init_addr="<same-ip>:10000" — the head node is told to bind its server port and its dist-init TCPStore on the same port. The Step 2.5 comment says "allocate sequentially so engines on the same node never collide", but the Step 2 allocation is invisible to that sequencer.

The vLLM path avoids this by reusing the head's pre-allocated port as the dist-init port rather than drawing a second untracked port. Possible fixes: allocate the dist-init port from the same sequential allocator (e.g. run Step 2 after Step 2.5 using next_start, or advance next_start past the negotiated dist-init port). Only affects sglang_nnodes > 1 with auto-negotiated addr.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions