Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
2800e52
feat(placement): add the measured-only planner core and its cost report
krisztian-gajdar Nov 5, 2026
46d5f33
fix(placement): keep current lanes and price every lane of the plan
krisztian-gajdar Nov 5, 2026
e803126
feat(placement): emit sie-cluster Helm values from a plan
krisztian-gajdar Nov 5, 2026
8274fa2
fix(placement): realize only the planned lanes in the Helm values
krisztian-gajdar Nov 5, 2026
aab529c
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 5, 2026
e682080
fix(placement): price come-alive levers by planning again
krisztian-gajdar Nov 5, 2026
c59d8c5
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 5, 2026
1c3d8c4
feat(helm): add lane envelopes and an opt-in placement applier
krisztian-gajdar Nov 5, 2026
f819b72
fix(placement): price the next cheaper option by planning again
krisztian-gajdar Nov 5, 2026
f7560ed
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 5, 2026
2c7cd63
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 5, 2026
2592c21
fix(helm): suggest only single-token Secrets to the placement applier
krisztian-gajdar Nov 5, 2026
884584b
feat(placement): keep unmeasured current placements and select lazy r…
krisztian-gajdar Nov 5, 2026
c469b25
docs(placement): rewrap the lazy-resident selection step
krisztian-gajdar Nov 5, 2026
e01b3ef
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 5, 2026
c23ed6e
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 5, 2026
195cd2e
fix(placement): decide lazy hosts against the plan being written
krisztian-gajdar Nov 5, 2026
37bf3d7
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 5, 2026
2144dd2
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 5, 2026
14ea897
chore: merge origin/main
krisztian-gajdar Nov 5, 2026
116befa
feat(placement): plan with GPU availability as measured evidence
krisztian-gajdar Nov 5, 2026
1af0c35
chore: merge origin/main
krisztian-gajdar Nov 5, 2026
c936cad
feat(placement): pack several models onto one GPU and price shared la…
krisztian-gajdar Nov 5, 2026
f6f8908
fix(placement): rank every cold attempt and hold lanes to the ceiling…
krisztian-gajdar Nov 5, 2026
091bb89
chore: merge feat/sie-placement-availability
krisztian-gajdar Nov 5, 2026
62481f7
fix(placement): hold joined lanes to their memory and ask the right r…
krisztian-gajdar Nov 5, 2026
1d4809b
fix(placement): hold lanes to the shared quota at maxContainers
krisztian-gajdar Nov 5, 2026
f553e14
chore: merge feat/sie-placement-availability
krisztian-gajdar Nov 5, 2026
e8d0f5e
test(placement): hold shared lanes to the ceilings at maxContainers
krisztian-gajdar Nov 5, 2026
c848c52
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 5, 2026
f653fb3
fix(placement): refuse lanes and warmths the chart cannot realize
krisztian-gajdar Nov 5, 2026
68d0ae2
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 5, 2026
85313d8
fix(placement): put each pinning lane on a named queue pool of its own
krisztian-gajdar Nov 5, 2026
a1f267a
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 5, 2026
33fafeb
test(placement): emit a planner estate with several pinned lanes as H…
krisztian-gajdar Nov 5, 2026
1dce2ec
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 5, 2026
9e65454
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 5, 2026
0c7c09c
refactor(placement): share the Helm emitter's plan refusals with the …
krisztian-gajdar Nov 5, 2026
4799e51
fix(placement): charge the shared quota at envelope maxima
krisztian-gajdar Nov 6, 2026
f8a1a58
chore: merge feat/sie-placement-schemas
krisztian-gajdar Nov 6, 2026
a249ef7
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 6, 2026
bc48ae5
chore: merge feat/sie-placement-availability
krisztian-gajdar Nov 6, 2026
2601bb6
test(placement): hold a shared lane to the shared quota at its envelo…
krisztian-gajdar Nov 6, 2026
efebe60
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 6, 2026
ff19c8f
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 6, 2026
4e2f88c
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
3a8d764
chore: merge feat/sie-placement-schemas
krisztian-gajdar Nov 6, 2026
b18b9e3
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 6, 2026
78fe4e3
fix(placement): check capacity counts as integers in documents built …
krisztian-gajdar Nov 6, 2026
2ec35fa
chore: merge feat/sie-placement-availability
krisztian-gajdar Nov 6, 2026
b8ff7d8
fix(placement): check memory sizes as integers in documents built in …
krisztian-gajdar Nov 6, 2026
ba68a30
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 6, 2026
19a3e2b
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
8265d4a
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 6, 2026
cb0ed0a
docs(placement): say a derived queue pool is the slug of the lane name
krisztian-gajdar Nov 6, 2026
5c33cc2
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
6f34a33
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 6, 2026
828b709
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 6, 2026
9c4fa43
chore: merge feat/sie-placement-schemas
krisztian-gajdar Nov 6, 2026
9560da6
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 6, 2026
36a7f43
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
4446ce1
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 6, 2026
7cf9626
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 6, 2026
84844ba
chore: merge feat/sie-placement-availability
krisztian-gajdar Nov 6, 2026
8d137c2
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
21b21d2
docs(placement): list the fallback-floor and queue-pool emit conditions
krisztian-gajdar Nov 6, 2026
1a23347
docs(helm): state where applier failures show and how to rotate its a…
krisztian-gajdar Nov 6, 2026
1e781a0
fix(placement): group plan lanes into chart lanes without regard to case
krisztian-gajdar Nov 6, 2026
52e4261
chore: merge feat/sie-placement-planner
krisztian-gajdar Nov 6, 2026
ed5c652
chore: merge feat/sie-placement-availability
krisztian-gajdar Nov 6, 2026
8676b86
fix(placement): compare kept queue pools without regard to case
krisztian-gajdar Nov 6, 2026
fd76a42
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 6, 2026
cefdf84
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
5f3d1a6
docs(placement): say when the planner requests footprints and what a …
krisztian-gajdar Nov 6, 2026
5042e8e
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 6, 2026
4abd76f
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
c35ea60
fix(placement): size a shared lane from its residents' demand together
krisztian-gajdar Nov 6, 2026
3c658ec
chore: merge feat/sie-placement-packing
krisztian-gajdar Nov 6, 2026
3f5855b
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
3523c0b
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 6, 2026
469bd42
docs(helm): say a stale admin token blocks only the applier's pool wr…
krisztian-gajdar Nov 6, 2026
0c6c00d
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 6, 2026
028fd4b
fix(helm): compare credential Secret keys as the manifests name them
krisztian-gajdar Nov 6, 2026
d3adf0b
fix(helm): accept only Kubernetes pull policies for the placement app…
krisztian-gajdar Nov 6, 2026
6f83ed8
fix(helm): mount the applier's admin token only when the gateway may …
krisztian-gajdar Nov 6, 2026
578b43e
chore: merge feat/sie-placement-keda-applier
krisztian-gajdar Nov 6, 2026
70138e3
docs(helm): say the applier reports a gateway error reply by its stat…
krisztian-gajdar Nov 6, 2026
f860df6
docs(placement): pass the planner's plan to helm_values and scope ena…
krisztian-gajdar Nov 6, 2026
c58111a
chore: merge feat/sie-placement-helm-values
krisztian-gajdar Nov 6, 2026
14214ce
docs(helm): say what the chart checks in the applier plan and what th…
krisztian-gajdar Nov 6, 2026
d5f8a5b
style(ci): format the Helm render tests
krisztian-gajdar Nov 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
176 changes: 172 additions & 4 deletions deploy/helm/sie-cluster/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -490,6 +490,168 @@ metrics, the bundled collector Prometheus exporter, and its ServiceMonitor.
The chart therefore cannot render collector-scoped application alerts without
a scrape path.

### Lane envelopes and the placement applier

A worker lane may declare an `envelope`, the range of warm floors that the
placement applier may hold it at:

```yaml
workers:
pools:
l4:
enabled: true
bundles:
default:
minReplicas: 1
maxReplicas: 4
envelope:
floorMin: 1
floorMax: 2
```

The render fails unless `floorMin` and `floorMax` are non-negative integers,
`floorMin` equals `minReplicas`, and `floorMax` lies between `floorMin` and
`maxReplicas`. A lane returns to `minReplicas` whenever nothing raises its
floor, and KEDA never runs more than `maxReplicas`. With autoscaling enabled, a
lane with `minReplicas: 0` takes a `floorMax` of 0 or at least its fallback
floor, `min(maxReplicas, autoscaling.fallbackReplicas)`, because KEDA runs at
least that many workers once a floor activates the lane. The envelope renders
nothing by itself: the lane's StatefulSet and ScaledObject are the same with or
without it.

`placementApplier.enabled=true` renders one Deployment that runs
`sie-placement tune` from the `sie-placement` package, whose README describes
the command, its audit rows and its refusals, against a placement plan:

```bash
helm upgrade <RELEASE> oci://ghcr.io/superlinked/charts/sie-cluster \
--namespace <NAMESPACE> \
-f <VALUES_FILE> \
--set placementApplier.enabled=true \
--set placementApplier.image.repository=<IMAGE> \
--set placementApplier.image.tag=<TAG> \
--set-json "placementApplier.plan=$(cat plan.json)"
```

Every 60 seconds the applier reads each plan lane's series from the Prometheus
that KEDA queries, scoped to this release's collector target, reads the lane's
floor-carrier pool from the gateway, decides one step inside the lane's
envelope, and prints one JSON audit row per lane to its log. A floor carrier is
an API logical pool named `placement-floor-<queuePool>-<machineProfile>-<bundle>`.
The gateway keeps each lane at the largest warm floor among its pools and
reports it to the lane's warm-floor KEDA trigger, so a carrier raises a lane
above its chart floor and never lowers the floor another pool sets. The
ScaledObject triggers are the same with or without the applier.

- **Shadow mode** (`placementApplier.apply: false`, the default) decides and
records and writes nothing. Run it first to see how often each lane reads as
under pressure.
- **Apply mode** (`placementApplier.apply: true`) upserts each lane's carrier on
every tick, which renews the carrier's lease and sets its floor. When the
applier stops, each carrier lapses 1200 seconds after its last write by
default, the gateway removes it, and the lane returns to the largest floor
its chart and its other pools set.

The render fails unless:

- `autoscaling.enabled=true`.
- `placementApplier.image.repository` and `placementApplier.image.tag` name an
image that installs `sie-placement` with its `keda` extra and can run as user
and group 65532. The chart publishes no such image.
`placementApplier.image.pullPolicy` is `Always`, `IfNotPresent` or `Never`.
- `placementApplier.plan` is a map whose `kind` is `placement-plan` and whose
`lanes` list holds at least one lane. Each of its lanes names an enabled
worker lane by its `queuePool`, `cell.machineProfile` and `bundle`, compared
as the chart canonicalizes lane tokens, and that worker lane declares an
`envelope` equal to the plan lane's. A plan therefore cannot move a floor
outside a range that the chart values bound. The values overlay that
`helm_values` in `sie-placement` emits from a plan sets each lane's
`minReplicas` and `maxReplicas` but no `envelope`, so copy each plan lane's
`envelope.floorMin` and `envelope.floorMax` into the worker lane it names.
The chart checks only these fields. The applier parses the whole plan when
it starts and refuses one that is not a valid placement-plan document, an
unsupported `schemaVersion` included.
- With gateway token auth, or a gateway auth mode known only at run time,
`placementApplier.clientToken.existingSecretName` names a Secret key that
holds one gateway client token, which the applier presents on pool reads.
The applier presents the whole value, so the key must not list several
comma-separated tokens, as an operator-managed `gateway.auth.tokenSecretName`
does during a [rotation](#gateway-token). The Secret name and key, after
trimming surrounding whitespace, are a Kubernetes Secret name and data key
that hold no other credential the chart wires.
- With `placementApplier.apply: true` and gateway token auth, or a mode known
only at run time, `gateway.auth.adminTokenSecretName` is set.
- `gateway.extraEnv` sets `SIE_GATEWAY_POOL_MAX_MINIMUM_WORKER_COUNT` and
`SIE_GATEWAY_MAX_POOLS`, if at all, as literal values. The chart passes an
integer value of either to the applier as `--max-pool-floor` or
`--max-pools`, so the applier refuses the plans the gateway would.

The applier itself refuses a plan whose `floorMax` exceeds the gateway's
per-pool floor limit, 4 by default, or whose lanes need more carriers than the
gateway admits, 64 by default. A refusal, an unreadable token file or an
invalid flag ends the container, which then shows as `CrashLoopBackOff`. A
failed Prometheus query or pool call fails only its tick. The applier prints a
failed query to standard error, and the audit rows show each series it could
not read as `null`. The audit row's `error` records a failed pool call.

#### Applier permissions

| Credential | Held | Authorizes |
| --- | --- | --- |
| Gateway admin token | Only with `placementApplier.apply: true` and a gateway that may authenticate, with token auth or a mode known only at run time, from the Secret `gateway.auth.adminTokenSecretName` (key `gateway.auth.adminTokenSecretKey`) | `POST`, `PUT` and `DELETE` under `/v1/pools`, `/v1/admin` and `/v1/configs`: every gateway admin route, not only floor carriers |
| Gateway client token | When `placementApplier.clientToken.existingSecretName` is set | Every gateway route that is not an admin route, inference included; the applier uses it for `GET /v1/pools/{name}` |
| Kubernetes API | Never | Nothing: the applier has its own ServiceAccount, mounts no ServiceAccount token, and has no Role or RoleBinding |

The chart never creates either token or renders its value. Each is mounted
read-only as a file of mode `0440` owned by group 65532, under
`/etc/sie-placement/`, and passed by path with `--admin-token-file` or
`--api-key-file`. The applier sends the admin token only on pool writes to the
in-cluster gateway Service, never the client token on writes, and never takes a
token on its command line or prints or logs one: a rejected token file is named
by its path, a gateway call that gets an error reply by its HTTP status alone,
so no text the server returns reaches the log, a failed connection by the SDK's
connection error message, and any other transport failure by its exception
type. Anyone who can read the applier's log sees lane telemetry and floors, not
credentials. Anyone who can create pods in the namespace with that Secret can
still read it, so the Secret read limits under [Who holds which
token](#who-holds-which-token) apply.

The applier reads both token files once, when it starts. After you rotate the
gateway client token, restart it as step 2 of
[rotating the gateway token](#gateway-token) says. The gateway also reads its
admin token once, when it starts, and accepts one at a time, so after you
rotate the gateway admin token, restart the gateway and then the applier:

```bash
kubectl rollout restart deployment/<FULLNAME>-gateway -n <NAMESPACE>
kubectl rollout restart deployment/<FULLNAME>-placement-applier -n <NAMESPACE>
```

The cutover is complete once both run with the new token. Pool reads use the
client token and pool writes the admin token, so a stale client token makes the
gateway refuse the applier's reads, and a tick that cannot read a carrier does
not write it, while a stale admin token makes it refuse only the writes. Either
way the pod keeps running: each tick's audit rows record the failures, and in
apply mode each carrier lapses 1200 seconds after its last write and the lane
returns to the largest floor its chart and its other pools set.

The Deployment runs one replica with the `Recreate` strategy, so at most one
applier writes carriers, also during a rollout. Its pod runs as user and group
65532 with `runAsNonRoot`, the `RuntimeDefault` seccomp profile, a read-only
root filesystem, no privilege escalation and every capability dropped. A
change to `placementApplier.plan` restarts it.

#### Applier network access

The applier accepts no connections. It opens connections to the gateway
Service (`gateway.service.port`, 8080 by default), to the Prometheus query API
(`autoscaling.prometheusAddress`, or the bundled Prometheus Service), and to
cluster DNS. The chart renders no NetworkPolicy for it. Under a default-deny
egress policy, allow those destinations for pods labelled
`app.kubernetes.io/component: placement-applier` and
`app.kubernetes.io/instance: <RELEASE>`. A policy that restricts ingress to the
gateway or to Prometheus must admit the same pods.

### Worker-sidecar Telemetry

When `workers.common.workerSidecar.enabled=true`, each worker Pod includes a
Expand Down Expand Up @@ -1412,16 +1574,20 @@ default `SIE_ADMIN_TOKEN`), which the chart passes to the gateway as
configured`). The chart does not generate this token, and it must be separate
from the sie-config tokens: the render fails when it names the Secret and key
of the sie-config admin token, which would give every gateway pod catalog
write access, or of the read token, which every worker sidecar holds.
write access, or of the read token, which every worker sidecar holds. The
placement applier mounts the same Secret only when `placementApplier.apply` is
true.

### Who holds which token

- **sie-config admin token:** sie-config and the admin tooling you give it to.
- **sie-config read token:** sie-config, every gateway pod, and every worker
sidecar. It can read the whole catalog, including every model config, but
cannot write.
- **Gateway admin token (when set):** every gateway pod and the tooling that
manages pools.
- **Gateway admin token (when set):** every gateway pod, the placement applier
when `placementApplier.apply` is true (see
[Lane envelopes and the placement applier](#lane-envelopes-and-the-placement-applier)),
and the tooling that manages pools.
- **Gateway client token (when set or generated):** API clients, every gateway
pod, and every worker sidecar, which presents it when it calls the gateway.
See [Securing the cluster](#securing-the-cluster).
Expand Down Expand Up @@ -1718,7 +1884,9 @@ without a cutover, use an operator-managed Secret:
`helm upgrade` (with the Secret's name in `gateway.auth.tokenSecretName`)
and restart the gateway and the worker StatefulSets. The gateway accepts
both tokens, and the worker sidecars present the first one.
2. Move every client, including the MCP edge's key, to the new token.
2. Move every client, including the MCP edge's key and the placement
applier's `placementApplier.clientToken` key, to the new token, and restart
the placement applier, which reads its token at start.
3. Remove the old token from the Secret and restart the gateway.

To leave a generated token, copy it into your own Secret as the old token in
Expand Down
Loading
Loading