Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 79 additions & 2 deletions PROTOCOL.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,8 @@ limited to `rm_core.workload.SLO_KEYS`.
- SHOULD name the KV transport to use between prefill and decode in `kv_transport`.

The placement controller runs this policy over `AcceleratorProfile`s and live metrics, and writes
its decisions where the llm-d EPP scorer and serving pods read them (see ai-rm docs).
its decisions to the placement document (§5.4), where the llm-d EPP scorer, serving pods and
lmd-top read them.

### 5.3 Workload runners

Expand All @@ -108,6 +109,75 @@ callable that accepts a `Workload` plus keyword configuration and starts the wor
configuration keys MUST be documented in the component's README. The runner MUST select its KV
transport by name (`kv_transport=`), never by hard-coded import. (Formalized in v0.2.)

### 5.4 Placement document (`airm-placement`, document version 0.1)

The hand-off between the placement controller (writer) and its readers is one JSON document.
`rm_core.placement` implements it (`validate(doc)`, `PlacementDocument`, `build`, `parse`,
`configmap`); `k8s/airm-placement.schema.json` is the JSON Schema for the structure. The document
version is the `protocol` field and follows the protocol version (§1).

**Location.** ConfigMap `airm-placement` in the namespace of the llm-d InferencePools it describes
(the llm-d namespace, e.g. `llm-serving`), key `decisions.json`, value a UTF-8 JSON object. Writers
MUST NOT put decisions under other keys. The ConfigMap SHOULD carry the annotation
`airm.ai-computing.io/protocol: "<version>"` and the label `app.kubernetes.io/part-of: ai-rm`.

```json
{
"protocol": "0.1",
"generation": 3,
"updatedAt": "2026-10-19T00:00:00Z",
"placement": "topology",
"decisions": [
{"pool": "llama8b-pd-pool",
"workload": {"kind": "prefill", "model": "meta-llama/Llama-3.1-8B-Instruct",
"slo": {"tpot_p90_ms": 100},
"labels": {"airm.ai-computing.io/accelerator": "furiosa.RNGD"}},
"acceleratorIds": ["<node>/furiosa:0"], "kvTransport": "shm", "reason": "highest peak"},
{"pool": "llama8b-pd-pool",
"workload": {"kind": "decode", "model": "meta-llama/Llama-3.1-8B-Instruct",
"slo": {"tpot_p90_ms": 100},
"labels": {"airm.ai-computing.io/accelerator": "furiosa.RNGD"}},
"acceleratorIds": ["<node>/furiosa:1"], "kvTransport": "shm", "reason": "most memory"}
]
}
```

| Field | Rule |
|---|---|
| `protocol` | REQUIRED. `"MAJOR.MINOR"`, the protocol version the writer implements |
| `generation` | REQUIRED. Integer ≥ 0; `0` only for the empty document |
| `updatedAt` | REQUIRED. RFC 3339 UTC (`YYYY-MM-DDTHH:MM:SSZ`) time of the last change; `null` only when `generation` is 0 |
| `placement` | Name of the `airm.placements` policy that produced the decisions. REQUIRED when `generation` > 0 |
| `decisions` | REQUIRED. List, empty in the empty document |
| `decisions[].pool` | REQUIRED. InferencePool name (= the EPP's `--pool-name`) |
| `decisions[].workload` | REQUIRED. `Workload` in camelCase: `kind` and `model` REQUIRED; `slo` (keys from `SLO_KEYS`), `labels` (strings), `precision`, `batchHint` OPTIONAL |
| `decisions[].workload.kind` | The role. A prefill/decode pool has one `prefill` and one `decode` decision; an aggregated pool (any role) has one `serve` decision |
| `decisions[].workload.labels["airm.ai-computing.io/accelerator"]` | OPTIONAL. Card type the role was restricted to, `<vendor>.<model>` as in the node labels (§3.1). When set, every card in `acceleratorIds` MUST be of that vendor |
| `decisions[].acceleratorIds` | REQUIRED. Card references: `Accelerator.id`s `<node>/<vendor>:<index>` (= `AcceleratorProfile.spec.id`), no duplicates. The node of a card is the part before `/` |
| `decisions[].kvTransport` | `airm.kv_transports` name for the prefill→decode hand-off; `""` or absent if none |
| `decisions[].reason` | Human-readable. REQUIRED (non-empty) when `acceleratorIds` is empty, which means nothing fits |

The empty document, valid before the first placement, is
`{"protocol": "0.1", "generation": 0, "updatedAt": null, "decisions": []}`.

Writers:
- MUST write only documents that `rm_core.placement.validate` accepts.
- MUST write at most one decision per (`pool`, `workload.kind`).
- MUST increase `generation` by at least one whenever `placement` or `decisions` change, and MUST
NOT decrease it. A writer SHOULD NOT rewrite a document whose `placement` and `decisions` are
unchanged; if it does, it MUST keep the same `generation`.
- MUST NOT put addresses or credentials in the document (§7): cards are referenced by id only and
resolved through `AcceleratorProfile`s.

Readers:
- MUST ignore fields they do not know, at every level, and MUST accept a newer minor `protocol`
with the same major version.
- MUST NOT apply a document whose `generation` is not greater than the last one they applied
(`rm_core.placement.is_newer`); equal generations have equal content.
- MUST behave as if the ConfigMap did not exist when it is absent, when `decisions` is empty, or
when it has no decision for their pool.
- SHOULD treat a document that fails validation as absent and report it, not crash.

## 6. Metrics

- Components MUST report the quantities listed in `rm_core.metrics.METRICS` under exactly those
Expand All @@ -130,12 +200,19 @@ keys. Site values come from the environment (`cluster.local.env`), never from co
## 8. Conformance

`python -m rm_core.conformance [group [name ...]]` loads every registered implementation (or the
named ones) and runs the checks for its group. A component is conformant when its adapters pass,
named ones) and runs the checks for its group. For `airm.placements` the checks include that the
policy's decisions make a valid placement document (§5.4). A component is conformant when its adapters pass,
its tests pass, and `check_sensitive.py` reports clean. Each component's CI SHOULD run the checks
against its own entry points.

## Change log

- 0.1.2 (2026-10-19, rm-core patch; protocol stays 0.1): §5.4 makes the `airm-placement` document
normative as already written by rm-controller and hetero-llm-d and read by lmd-top (no field
changed). `rm_core.placement` (typed model, stdlib validator, `build` with the generation rule),
`k8s/airm-placement.schema.json`, and the `airm.placements` conformance check now also validates
the document made from a policy's decisions.

- 0.1.1 (2026-10-19, rm-core patch; protocol stays 0.1): `AcceleratorProfile` keeps `extra`
(CRD field added) and the provider check now verifies the profile round-trip; `NoHardware` moved to
`rm_core.errors` so `python -m rm_core.conformance` reports SKIP correctly; §6 migration rule.
Expand Down
6 changes: 5 additions & 1 deletion README.ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,13 +193,15 @@ src/rm_core/
├── accelerator.py # L1: Accelerator, AcceleratorProvider; 벤더·메모리·정밀도·인터커넥트 열거형
├── memory.py # L2: MemoryTier, KVLayout, TransferPlan/Handle/Result, KVTransport
├── workload.py # L3–L4: Workload, PlacementDecision, Placement; 워크로드 종류, SLO 키
├── placement.py # L4: airm-placement 문서(decisions.json): 모델, validate(), build()
├── metrics.py # METRICS, TAG_KEYS, Measurement, prometheus_name
├── k8s.py # AcceleratorProfile 객체와 노드 레이블; from_profile()
├── registry.py # entry-point 그룹: available(), load(), groups()
├── reference.py # sim / loopback / greedy (하드웨어 불필요)
├── conformance.py # python -m rm_core.conformance
└── errors.py # ContractError, TransportError, NoHardware
k8s/acceleratorprofile.crd.yaml # AcceleratorProfile CRD (airm.ai-computing.io/v1alpha1)
k8s/airm-placement.schema.json # airm-placement 문서의 JSON Schema (PROTOCOL §5.4)
```

노드 데몬은 장치마다 클러스터 범위 `AcceleratorProfile`을 게시하고, 요약 노드 레이블
Expand All @@ -220,6 +222,8 @@ CI는 Python 3.9–3.12에서 테스트를 돌린 뒤 `python -m rm_core.conform

- [x] 프로토콜 v0.1: 계약, `AcceleratorProfile` CRD, 참조 구현, 적합성 검사
- [x] 0.1.1: `AcceleratorProfile`이 `extra` 유지, `python -m` 실행 시 `NoHardware`를 SKIP으로 보고, 메트릭 이전 규칙
- [x] 0.1.2 (미릴리스): `airm-placement` 문서 규범화(§5.4), `rm_core.placement` 검증기, JSON Schema,
배치 적합성 검사가 문서까지 검증
- [ ] 프로토콜 v0.2: 전송 계획에 `MemoryRegion`(owner, fd/offset, size)을 넣어, 장치 간 전송이 실제 버퍼를 묶고
하드웨어 fixture로 적합성 검사를 통과할 수 있게 함
- [ ] 프로토콜 v0.2: 워크로드 runner 인터페이스(`airm.workloads`) 정식화
Expand All @@ -230,7 +234,7 @@ CI는 Python 3.9–3.12에서 테스트를 돌린 뒤 `python -m rm_core.conform

| | |
|---|---|
| **명세** | [PROTOCOL.md](PROTOCOL.md) · [AcceleratorProfile CRD](k8s/acceleratorprofile.crd.yaml) |
| **명세** | [PROTOCOL.md](PROTOCOL.md) · [AcceleratorProfile CRD](k8s/acceleratorprofile.crd.yaml) · [배치 스키마](k8s/airm-placement.schema.json) |
| **ai-rm** | [Umbrella](https://github.com/ai-computing/ai-rm) · [인터페이스](https://github.com/ai-computing/ai-rm/blob/master/docs/interfaces.md) · [통합 안내](https://github.com/ai-computing/ai-rm/blob/master/docs/integration-guide.md) |

## 📄 라이선스
Expand Down
6 changes: 5 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,13 +195,15 @@ src/rm_core/
├── accelerator.py # L1: Accelerator, AcceleratorProvider; vendor, memory, precision, interconnect enums
├── memory.py # L2: MemoryTier, KVLayout, TransferPlan/Handle/Result, KVTransport
├── workload.py # L3–L4: Workload, PlacementDecision, Placement; workload kinds, SLO keys
├── placement.py # L4: airm-placement document (decisions.json): model, validate(), build()
├── metrics.py # METRICS, TAG_KEYS, Measurement, prometheus_name
├── k8s.py # AcceleratorProfile objects and node labels; from_profile()
├── registry.py # entry-point groups: available(), load(), groups()
├── reference.py # sim / loopback / greedy (no hardware)
├── conformance.py # python -m rm_core.conformance
└── errors.py # ContractError, TransportError, NoHardware
k8s/acceleratorprofile.crd.yaml # AcceleratorProfile CRD (airm.ai-computing.io/v1alpha1)
k8s/airm-placement.schema.json # JSON Schema of the airm-placement document (PROTOCOL §5.4)
```

A node daemon publishes each device as a cluster-scoped `AcceleratorProfile` plus summary node labels
Expand All @@ -222,6 +224,8 @@ CI runs the tests on Python 3.9–3.12, then `python -m rm_core.conformance` and

- [x] Protocol v0.1: contracts, `AcceleratorProfile` CRD, reference implementations, conformance checks
- [x] 0.1.1: `AcceleratorProfile` keeps `extra`, `NoHardware` reported as SKIP under `python -m`, metric migration rule
- [x] 0.1.2 (unreleased): `airm-placement` document normative (§5.4), `rm_core.placement` validator, JSON Schema,
placement conformance validates the document
- [ ] Protocol v0.2: `MemoryRegion` (owner, fd/offset, size) in transfer plans, so device-to-device transports
can bind live buffers and pass conformance with a hardware fixture
- [ ] Protocol v0.2: formalized workload-runner interface (`airm.workloads`)
Expand All @@ -232,7 +236,7 @@ Change log: [PROTOCOL.md § Change log](PROTOCOL.md#change-log).

| | |
|---|---|
| **Specification** | [PROTOCOL.md](PROTOCOL.md) · [AcceleratorProfile CRD](k8s/acceleratorprofile.crd.yaml) |
| **Specification** | [PROTOCOL.md](PROTOCOL.md) · [AcceleratorProfile CRD](k8s/acceleratorprofile.crd.yaml) · [placement schema](k8s/airm-placement.schema.json) |
| **ai-rm** | [Umbrella](https://github.com/ai-computing/ai-rm) · [Interfaces](https://github.com/ai-computing/ai-rm/blob/master/docs/interfaces.md) · [Integration guide](https://github.com/ai-computing/ai-rm/blob/master/docs/integration-guide.md) |

## 📄 License
Expand Down
64 changes: 64 additions & 0 deletions k8s/airm-placement.schema.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://github.com/ai-computing/rm-core/blob/main/k8s/airm-placement.schema.json",
"title": "airm-placement decisions.json (ai-rm protocol 0.1, PROTOCOL.md section 5.4)",
"description": "Value of key decisions.json in ConfigMap airm-placement. Unknown properties are allowed and ignored by readers. Rules a schema cannot express (generation > 0 when there are decisions, updatedAt and placement set when generation > 0, one decision per pool and workload.kind, card vendor matching the airm.ai-computing.io/accelerator label) are checked by rm_core.placement.validate.",
"type": "object",
"required": ["protocol", "generation", "updatedAt", "decisions"],
"properties": {
"protocol": {"type": "string", "pattern": "^0\\.\\d+$"},
"generation": {"type": "integer", "minimum": 0},
"updatedAt": {
"oneOf": [
{"type": "null"},
{"type": "string", "pattern": "^\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(\\.\\d+)?Z$"}
]
},
"placement": {"type": "string", "minLength": 1},
"decisions": {"type": "array", "items": {"$ref": "#/$defs/decision"}}
},
"$defs": {
"decision": {
"type": "object",
"required": ["pool", "workload", "acceleratorIds"],
"properties": {
"pool": {"type": "string", "minLength": 1},
"workload": {"$ref": "#/$defs/workload"},
"acceleratorIds": {
"type": "array",
"uniqueItems": true,
"items": {"type": "string", "pattern": "^[a-z0-9][a-z0-9._-]*/[a-z0-9]+:\\d+$"}
},
"kvTransport": {"type": "string"},
"reason": {"type": "string"}
},
"if": {"properties": {"acceleratorIds": {"maxItems": 0}}},
"then": {"required": ["reason"], "properties": {"reason": {"minLength": 1}}}
},
"workload": {
"type": "object",
"required": ["kind", "model"],
"properties": {
"kind": {"enum": ["prefill", "decode", "serve", "train", "embed", "search"]},
"model": {"type": "string", "minLength": 1},
"slo": {
"type": "object",
"propertyNames": {"enum": ["ttft_p90_ms", "ttft_p99_ms", "tpot_p90_ms", "tpot_p99_ms", "e2e_p90_ms", "tok_s_min"]},
"additionalProperties": {"type": "number"}
},
"labels": {
"type": "object",
"additionalProperties": {"type": "string"},
"properties": {
"airm.ai-computing.io/accelerator": {
"type": "string",
"pattern": "^(nvidia|furiosa|rebellions|cpu|amd|intel|other)\\.[A-Za-z0-9][A-Za-z0-9._-]*$"
}
}
},
"precision": {"enum": ["fp32", "tf32", "bf16", "fp16", "fp8", "int8", "int4"]},
"batchHint": {"oneOf": [{"type": "null"}, {"type": "integer", "minimum": 1}]}
}
}
}
}
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "rm-core"
version = "0.1.1"
version = "0.1.2"
description = "Contracts that connect ai-rm components: accelerators, memory tiers, KV transport, placement, metrics"
readme = "README.md"
requires-python = ">=3.9"
Expand Down
2 changes: 1 addition & 1 deletion src/rm_core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from .metrics import METRICS, Measurement, prometheus_name
from .workload import Placement, PlacementDecision, Workload

__version__ = "0.1.1"
__version__ = "0.1.2"
PROTOCOL_VERSION = "0.1"

__all__ = [
Expand Down
13 changes: 12 additions & 1 deletion src/rm_core/conformance.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@
"""
from __future__ import annotations

import json
import sys
from typing import Callable, Dict, List, Optional, Tuple

from . import placement as placement_doc
from . import registry
from .accelerator import Accelerator, AcceleratorProvider
from .errors import ContractError, TransportError
Expand Down Expand Up @@ -80,14 +82,23 @@ def check_placement(placement, pool: Optional[List[Accelerator]] = None) -> None
assert isinstance(placement, Placement), "missing name/place()"
pool = pool or SimAcceleratorProvider("conf-node").discover()
ids = {a.id for a in pool}
entries = []
for kind in ("prefill", "decode"):
w = Workload(kind=kind, model="conformance-model", slo={"tpot_p90_ms": 100})
d1, d2 = placement.place(w, pool), placement.place(w, pool)
assert isinstance(d1, PlacementDecision), "place() must return a PlacementDecision"
assert set(d1.accelerator_ids) <= ids, f"placed outside the pool: {d1.accelerator_ids}"
assert d1 == d2, "place() must be deterministic for the same inputs"
empty = placement.place(Workload(kind="decode", model="m"), [])
entries.append(placement_doc.Decision.from_placement("conf-pool", w, d1))
w = Workload(kind="decode", model="m")
empty = placement.place(w, [])
assert empty.accelerator_ids == [] and empty.reason, "empty pool -> empty decision with a reason"
entries.append(placement_doc.Decision.from_placement("conf-empty-pool", w, empty))
# §5.4: the decisions must make a valid airm-placement document.
doc = placement_doc.PlacementDocument(1, "2026-01-01T00:00:00Z", placement.name, entries)
errors = placement_doc.validate(json.loads(doc.to_json()))
assert not errors, f"decisions do not make a valid airm-placement document: {errors}"
assert placement_doc.PlacementDocument.from_dict(doc.to_dict()) == doc, "document round-trip changed it"


def check_measurements(measurements: List[Measurement]) -> None:
Expand Down
Loading
Loading