Skip to content

engines tab: engine card shows all-zero metrics for a busy llama-swap model; unreported reads as zero #140

Closed

Description

@azrael-229

With a llama-swap engine wired up, the engine card shows DECODE 0.0 t/s, PREFILL 0.0 t/s, "in 0 tok, out 0 tok, queue 0", and a model row with the name and no values — even while the model is actively serving.

The chain (main @ 050866f):

  1. The llama-swap reader (internal/engines/engines.go, llamaSwap) reads llama-swap's /running, then each ready model's proxy /metrics with the llamacpp parser.
  2. llama-swap starts llama-server without --metrics, so the proxy answers 501 "This server does not support metrics endpoint. Start it with --metrics" and out.Metrics stays zero.
  3. rates() computes decode/prefill from those counters, so they read 0, and the card renders (eng.decode_tps || 0).toFixed(1) — an unreported field is drawn as a measured zero.

Live state while this was captured: slot s0 for qwen-3.8-27b processing with 65,950 tokens in its KV window, and the node's own request log computing decode traffic for the same engine (the basic tab's ticket counter showed 14.5 t/s) — while the engine card shows 0.0 t/s and zero token totals.

The model row has the same shape: /running carries no per-model VRAM and the reader sets no ReportedVRAM for llama-swap models, so the row shows name and state with no values.

Suggestion:

  • the card should render n/a (the util.js na() helper already does this on the same tab, e.g. the leaderboard's decode median) for fields the engine does not report, instead of 0.0/0;
  • when the engine's counters are absent, the node's own request log already has the tps — falling back to it would keep the card honest for engines whose /metrics is off;
  • on the llama-swap side, starting its llama-server with --metrics would give the reader real counters (a llama-swap default, not a berth change).

Activity

  1. T-Crypt commented on Oct 22, 2026

    @T-Crypt
    Owner

    Thanks for tracing the whole chain. Fixed in #146: engines carry metrics_reported, and the card shows n/a for decode, prefill and token totals when it is false. Starting llama-swap's llama-server with --metrics gives real numbers, as you suggested. Falling back to the node's own request log is a good idea, and I've left it as a follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions