Skip to content

[fork/hydra-538] llama-engine readiness sentinels for event-driven supervision #79

Description

@ddvnguyen

Tracks the C++ side of ddvnguyen/hydra_vortex#538.

Context

Hydra Head's event-driven readiness (hydra_vortex#538) matches llama-engine's
stdout against lifecycle sentinels. The head config (global.yaml) lists
server is listening on, router server is listening on, model loaded
all emitted by tools/server/server.cpp (the classic llama-server).

But the llama-engine target (tools/llama-engine/CMakeLists.txt) links
only llama-engine.cpp + server-http.cpp + server-context — it never
compiles server.cpp. None of the three llama-engine startup paths print
any of those sentinels:

  • has_model (RTX 5060 Ti head): no "ready"/"listening" log after HTTP start
  • compute-only peer (RTX 3060): logs compute-only peer ready (doesn't match)
  • head-bootstrap (no model, waits for CONFIGURE T3): logs the start of
    bootstrap, not readiness; true readiness is after T3's deferred load

So /status would stay 503 forever on both RTX nodes. The P100 VM test in
hydra_vortex#538 passed only because node-p100.yaml points at the classic
llama-server binary, where the sentinels happen to exist.

Goal

Emit a single, well-defined readiness sentinel — hydra-engine ready — at
the true-readiness moment of each llama-engine startup path:

  1. has_model: after HTTP server start + is_ready
  2. compute-only peer: after RPC + HTTP ready (extend the existing line)
  3. head-bootstrap: after CONFIGURE T3's deferred first-load completes
    (in server-context.cpp apply_pending_hydra_config, is_first_load)

The head then matches this one sentinel for RTX nodes; P100 keeps matching
the classic server.cpp sentinels via node-p100.yaml.

Acceptance

  • llama-engine prints hydra-engine ready on all three paths
  • Builds for sm120 and sm60
  • Fork PR merged / SHA reachable on hydra-fork before parent bump

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    hydra-forkHydra fork-specific change

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions