Skip to content

[fork] hydra_config inline T3 + hydra_metrics response #45

Description

@ddvnguyen

Tracks the C++ side of the hydra_config / hydra_metrics implementation.

What

  • Parse optional hydra_config from chat completion request body
  • Compare model_path / split_mode / tensor_split against current engine state
  • Trigger inline T3 reload if config differs (first-load or reload)
  • Emit hydra_metrics in every chat completion response (model_path, split_mode, tensor_split, GPU memory, t3_reloaded, t3_reload_ms)
  • Handle empty-start: engine starts without model, loads on first hydra_config
  • split_mode / tensor_split added to server_context_meta

Files changed

  • tools/server/server-context.cpp — hydra_config parsing, first-load path, T3 rebuild, metrics
  • tools/server/server-context.h — meta fields, friend struct
  • tools/server/server-task.cpp — emit hydra_metrics in response
  • tools/server/server-task.h — hydra_metrics field
  • tools/server/server.cpp — empty model path handling
  • src/llama-hydra.cpp — split_mode/override_tensor staging without ctx

Hydra parent issue

ddvnguyen/hydra_vortex#TBD

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    hydra-forkHydra fork-specific change

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions