You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
engines tab: engine card shows all-zero metrics for a busy llama-swap model; unreported reads as zero #140
With a llama-swap engine wired up, the engine card shows DECODE 0.0 t/s, PREFILL 0.0 t/s, "in 0 tok, out 0 tok, queue 0", and a model row with the name and no values — even while the model is actively serving.
The llama-swap reader (internal/engines/engines.go, llamaSwap) reads llama-swap's /running, then each ready model's proxy /metrics with the llamacpp parser.
llama-swap starts llama-server without --metrics, so the proxy answers 501 "This server does not support metrics endpoint. Start it with --metrics" and out.Metrics stays zero.
rates() computes decode/prefill from those counters, so they read 0, and the card renders (eng.decode_tps || 0).toFixed(1) — an unreported field is drawn as a measured zero.
Live state while this was captured: slot s0 for qwen-3.8-27b processing with 65,950 tokens in its KV window, and the node's own request log computing decode traffic for the same engine (the basic tab's ticket counter showed 14.5 t/s) — while the engine card shows 0.0 t/s and zero token totals.
The model row has the same shape: /running carries no per-model VRAM and the reader sets no ReportedVRAM for llama-swap models, so the row shows name and state with no values.
Suggestion:
the card should render n/a (the util.js na() helper already does this on the same tab, e.g. the leaderboard's decode median) for fields the engine does not report, instead of 0.0/0;
when the engine's counters are absent, the node's own request log already has the tps — falling back to it would keep the card honest for engines whose /metrics is off;
on the llama-swap side, starting its llama-server with --metrics would give the reader real counters (a llama-swap default, not a berth change).
Thanks for tracing the whole chain. Fixed in #146: engines carry metrics_reported, and the card shows n/a for decode, prefill and token totals when it is false. Starting llama-swap's llama-server with --metrics gives real numbers, as you suggested. Falling back to the node's own request log is a good idea, and I've left it as a follow-up.
Activity
T-Crypt commentedon Oct 22, 2026
Thanks for tracing the whole chain. Fixed in #146: engines carry
metrics_reported, and the card shows n/a for decode, prefill and token totals when it is false. Starting llama-swap'sllama-serverwith--metricsgives real numbers, as you suggested. Falling back to the node's own request log is a good idea, and I've left it as a follow-up.docs(engines): llama-swap counters need --metrics on llama-server