[gspark-etri]

Background

B1 ATOM PD submits the next remote prefill only after its decode lane becomes free. Producer compute is therefore serialized with current-request decode. This draft lets one queued request prefill while that lane decodes, keeping its result reserved until owner-thread KV import completes.

Changes

• Opt-in AtomBatcher(remote_prefetch=True) and serve_batched.py --pd-prefetch-next; restricted to single-lane ATOM remote-only PD, with invalid CLI routes rejected before model loading.
• Reuse the existing source slot after current KV import. No transport protocol change or additional shared-memory slot.
• Hold the next request's result until import. Client cancellation does not cancel the producer future or release a still-written buffer; the owner drains orphaned work before reuse.
• Keep cache admission on the owner after the lane is free. Early prefill sends the full prompt because the final cache state is not yet known. Default behavior and cost routing remain unchanged.

Verification

Code3e16c33, based on273952a. CPU fake Rebellions SDK with real tiny-LM TorchPrefillEngine KV and independent greedy outputs; not hardware performance evidence.

• RED: missing remote_prefetch API; invalid CLI route reached model loading before the validation was added.
• Focused AtomBatcher/factory suite:35passed,4.39s.
• Full suite from opt_prime/: CUDA_VISIBLE_DEVICES='' OMP_NUM_THREADS=2 MKL_NUM_THREADS=2 PYTHONPATH=. /home/gspark/venvs/pt213/bin/python -m pytest -q: 511passed,34skipped,253subtests,150.28s; one existing unfused flex-attention warning.
• Tests cover next producer submission before first decode finishes, exact outputs, bounded reservation, success/cancellation/producer failure, single-block and prefix/host-cache configurations, orphan draining, and safe request-ID/source-slot reuse. The full run includes the additional orphan non-idle assertion added after the focused run.
• Diff and scoped sensitive scan clean. Evidence and exact commands: opt_prime/examples/pd/http_matrix/experiments/2026-10-24-atom-prefetch253.md, logs in evidence253/.

Compatibility and limits

B1 remote-only and opt-in; no default promotion. Early requests cannot skip transferring a cached prefix. No within-request layer/chunk export is added (separate#47). Actual KONI RNGD-to-ATOM validation now passes: 48 timed batches, 8 fixed-output groups, 4 normal-EOS groups, cancellation/reuse, producer-exit isolation and new-request/readiness503. Same warm engine, three alternating off/on repeats: c4 throughput gains +40.87% (input128/output32), +25.75% (128/128), +31.57% (8192/32), +23.48% (8192/128); c1 -0.19% to -0.03%. One RNGD + one ATOM, width8192 research producer, B1 decoder, cache off. Completed measured producer host-call intervals overlap other-request decode in36/60 enabled calls versus0/60 disabled; this is not device-kernel tracing. Qwen3-4B now also passes48timed batches,8fixed-output/4EOS groups and the same live cancellation/producer-exit gates. Its c4 median gains are +25.70% (128/32), +41.00% (128/128), +21.20% (8192/32), +33.89% (8192/128); c1 remains near neutral. Qwen3-0.6B also passes48timed batches,8fixed-output/4EOS groups and cancellation/producer-exit checks. Its c4 median gains are +14.61% (128/32), +48.79% (128/128), +9.40% (8192/32), +37.74% (8192/128); c1 -1.20% to +0.05%. Gemma3-1B also passes48timed batches,8fixed-output/4EOS groups and cancellation/producer-exit checks. Its c4 median gains are +14.12% (128/32), +52.71% (128/128), +12.57% (8192/32), +47.87% (8192/128); c1 -0.78% to +1.36%. Allfour models192timed batches now complete. This ATOM-path validation does not resolve separate GPU numerical issue#256; no universal speedup claim. Reproducer atom_pipeline.py and raw evidence253/atom253-koni-a.json are included.

Related Issue

Refs#253; does not close it. Stacked on#250. Keep draft until hardware validation and review; maintainer merges.

[gspark-etri]

Landed on optimus-rm via #285. Closing; the branch is archived locally (experiment records and evidence included).