Skip to content

Allow cache-off single-lane ATOM PD with one KV block #236

Closed

Description

@gspark-etri

Problem

The validated KONI-8B ATOM artifact uses a static B1 cache with one KV block. AtomBatcher rejects any prefill worker unless the artifact has more blocks than lanes, so standalone KV handoff cannot become live production PD with this artifact.

Evidence

#227 validates KONI native/imported decode through6K/8K and64slot readbacks using the single-block checker. AtomBatcher currently raises prefix caching / host tier / PD need kv_blocks > batch_size; AtomKVBlocks reserves a scratch block for idle lanes. A single active lane cannot coexist with another idle decoding lane, and cache-off requests release their block rather than retain prefixes.

Scope

Allow fresh full-prefix remote PD for static B1/one-block artifacts only when prefix and host caches are disabled. Preserve scratch requirements for multi-lane batching and unsupported cache combinations. Test request reuse, imported/native parity, cancellation/error recovery and no block leaks. Validate real KONI live PD through #234's worker and HTTP harness.

Done when

The existing one-block KONI artifact serves live remote prefill requests through AtomBatcher with correct block ownership and recovery; unsupported cache/multi-lane cases remain rejected. No hardware batch or cache restoration claim based on B1.

Owner

Codex; maintainer review. Related #201/#231/#234.

Activity

  1. gspark-etri commented on Oct 30, 2026

    @gspark-etri
    CollaboratorAuthor

    Fixed by #280, squash-merged into optimus-rm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions