Slime bridge on sglang-router yields zero trainable tokens (prompt_token_ids and meta_info stripped by router); how was §4.1 run?

[zcaicaros]

Hi Polar team, thanks for the paper and the open reference examples.
I hit an issue reproducing §4.1 GRPO training with the polar_bridge
plus slime combination, and the empirical evidence points at
sglang-router silently stripping the extended response fields that
SGLangEngine.normalize_response relies on. I want to check whether
you saw the same thing internally and, if not, what your actual
setup differed on.

My setup

• Polar (this repo) commit: the latest stable branch
• Slime: 0.3.1 (from slime-main at HEAD)
• sglang: 0.5.15.post1
• sglang-router: swept 0.1.0 through 0.3.2 (see bisection below)
• Model: Qwen2.5-7B-Instruct (local HF snapshot)
• Hardware: 4 x L20X 140 GB, single node
• Config: adapted from examples/swegym_slime_grpo/run.sh for a
  HumanEval+MBPP task set; the polar-specific wiring is byte-for-byte
  the same
  (--rollout-function-path slime_bridge.rollout.generate_rollout_polar_async,
  default SGLANG_ROUTER_BASE_URL=http://<host>:9000, i.e. slime's
  managed sglang-router).

Symptom

Every rollout group is dropped by the slime bridge:

RolloutManager: Dropping Polar group N because of zero trainable tokens
adapter.py:71  Session ...: no usable trace (traces=K, max_tokens=60000)

Session-side everything looks healthy (claude_code CLI ran multi-turn
tool loops, gateway logs show 200 OK on /v1/messages, session
COMPLETED, trace count > 0, response messages populated). Every
resulting Trace has prompt_ids=[] and response_ids=[].

Root cause (empirically pinned)

Slime's sglang_router.launch_router deserializes upstream sglang
responses into strict Rust structs modeled on OpenAI's Chat
Completions schema (ChatCompletionResponse, ChatChoice). sglang's
extension fields prompt_token_ids (top of choice) and meta_info
(carrying output_token_logprobs etc.) are not in that schema, so
Rust serde silently drops them during the deserialize + reserialize
round trip.

Polar's SGLangEngine.normalize_response reconstructs both
trace.prompt_ids and trace.response_ids from those two fields.
Once the router strips them, _canonicalize_prompt_token_ids and
_canonicalize_response_token_ids produce nothing, and all three
fallbacks in record_utils.py::_extract_response_tokens return
None (there is no token_id in logprobs.content[] on either side
of the router, so the OpenAI-standard logprobs fallback also fails).

Empirical probe (5 minutes to reproduce)

# 1. Launch sglang engine on port 30000
python3 -m sglang.launch_server --model-path <snapshot> \
    --served-model-name Qwen2.5-7B-Instruct \
    --host 127.0.0.1 --port 30000 --tp 1

# 2. Launch sglang-router on port 30080 pointing at the engine
python3 -m sglang_router.launch_router \
    --host 127.0.0.1 --port 30080 \
    --worker-urls http://127.0.0.1:30000

# 3. Same request body sent to both endpoints
BODY='{"model":"Qwen2.5-7B-Instruct",
"messages":[{"role":"user","content":"Say hi in 5 words"}],
"max_tokens":12,"logprobs":true,"top_logprobs":0,
"return_prompt_token_ids":true,"return_meta_info":true}'

curl -s http://127.0.0.1:30000/v1/chat/completions \
    -H 'Content-Type: application/json' -d "$BODY" \
    | python3 -c "import json,sys;d=json.load(sys.stdin);print(sorted(d['choices'][0]))"

curl -s http://127.0.0.1:30080/v1/chat/completions \
    -H 'Content-Type: application/json' -d "$BODY" \
    | python3 -c "import json,sys;d=json.load(sys.stdin);print(sorted(d['choices'][0]))"

Output:

direct: ['finish_reason', 'index', 'logprobs', 'matched_stop', 'message', 'meta_info', 'prompt_token_ids']
router: ['finish_reason', 'index', 'logprobs', 'matched_stop', 'message']

Response body size: direct 2079 bytes, router 1191 bytes (43% drop).

Version bisection (sglang-router)

I swept every reachable version to find when the strip was
introduced:

version	usable?	strips prompt_token_ids and meta_info?
0.1.0 through 0.1.4	✅	no (passthrough)
0.1.5	✅	yes (regression introduced here)
0.1.9	✅	yes
0.2.0, 0.2.2, 0.2.3	❌ worker-registration path changed; /v1/models not routable with the standard launch args	n/a
0.2.4	✅	yes
0.3.0	✅	yes
0.3.1	✅	yes
0.3.2 (current pypi latest)	✅	yes

Strip has been present since 0.1.5, well before slime==0.3.1's
sglang-router>=0.3.0 constraint could be satisfied. Any run that
respects slime's own dependency floor will hit it.

Workaround we shipped

Bypass the router: point SGLANG_ROUTER_BASE_URL at slime's rollout
engine on :15000 directly rather than at slime's router on :9000
(with ROLLOUT_NUM_GPUS=1 ROLLOUT_NUM_GPUS_PER_ENGINE=1, the engine
port is predictable via
_allocate_rollout_engine_addr_and_ports_normal). Trades away
router load balancing and cache awareness at multi-engine scale, but
recovers prompt_ids and response_ids and lets GRPO advance. Nine
consecutive training steps clean, rollout_success_rate=1.0 every
batch.

Questions

1. What sglang-router version was in use for the §4.1 SWE-Gym GRPO
   experiments? examples/swegym_slime_grpo/run.sh defaults
   SGLANG_ROUTER_BASE_URL to :9000 (through the router). If the
   run was on 0.1.4 or older, that would explain why you saw nonzero
   rewards where we saw all-drop.
2. Was SGLANG_ROUTER_BASE_URL overridden in your actual run to
   point directly at the engine (i.e. was the committed default not
   what you actually used)? Our current fix is effectively this
   override.
3. Or is there a third path we are missing? e.g. a different
   SGLangEngine implementation that reads tokens from somewhere
   other than prompt_token_ids and
   meta_info.output_token_logprobs, a slime-side hook that captures
   token IDs before the response passes through the router, or a
   variant of polar_bridge.rollout not in this repo.
4. Do you plan to add an upstream fix or a workaround in
   SGLangEngine? Two options seem reasonable:
   • Router side (upstream sgl-project/sglang): add
     #[serde(flatten)] extra: HashMap<String, serde_json::Value> on
     ChatCompletionResponse, ChatChoice, ChatMessage, Delta,
     Usage, LogProbs so unknown fields survive the deserialize +
     reserialize round trip. Or a --forward-extra-fields flag with
     the same effect. Cleanest fix but requires an upstream release.
   • Polar side (this repo): either document the "point at engine
     directly" workaround in the reference example, or make
     SGLangEngine.prepare_request request a backend flag that emits
     token_id inside logprobs.content[] entries (sglang does not
     natively populate logprobs.content[i].token_id; I verified
     this empirically), so that the existing
     _paired_tokens_from_logprobs_content fallback becomes viable
     without meta_info.

Happy to test any patch or config. Repro scripts and full response
dumps available on request. Thanks for reading.