You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Allow cache-off single-lane ATOM PD with one KV block #236
The validated KONI-8B ATOM artifact uses a static B1 cache with one KV block. AtomBatcher rejects any prefill worker unless the artifact has more blocks than lanes, so standalone KV handoff cannot become live production PD with this artifact.
Evidence
#227 validates KONI native/imported decode through6K/8K and64slot readbacks using the single-block checker. AtomBatcher currently raises prefix caching / host tier / PD need kv_blocks > batch_size; AtomKVBlocks reserves a scratch block for idle lanes. A single active lane cannot coexist with another idle decoding lane, and cache-off requests release their block rather than retain prefixes.
Scope
Allow fresh full-prefix remote PD for static B1/one-block artifacts only when prefix and host caches are disabled. Preserve scratch requirements for multi-lane batching and unsupported cache combinations. Test request reuse, imported/native parity, cancellation/error recovery and no block leaks. Validate real KONI live PD through #234's worker and HTTP harness.
Done when
The existing one-block KONI artifact serves live remote prefill requests through AtomBatcher with correct block ownership and recovery; unsupported cache/multi-lane cases remain rejected. No hardware batch or cache restoration claim based on B1.
Activity
ATOM: hybrid KV tables and single-block PD (#207, #231, #236) (#280)
gspark-etri commentedon Oct 30, 2026
Fixed by #280, squash-merged into optimus-rm.