Tracks the C++ side of the hydra_config / hydra_metrics implementation.
What
- Parse optional
hydra_config from chat completion request body
- Compare model_path / split_mode / tensor_split against current engine state
- Trigger inline T3 reload if config differs (first-load or reload)
- Emit
hydra_metrics in every chat completion response (model_path, split_mode, tensor_split, GPU memory, t3_reloaded, t3_reload_ms)
- Handle empty-start: engine starts without model, loads on first hydra_config
- split_mode / tensor_split added to server_context_meta
Files changed
tools/server/server-context.cpp — hydra_config parsing, first-load path, T3 rebuild, metrics
tools/server/server-context.h — meta fields, friend struct
tools/server/server-task.cpp — emit hydra_metrics in response
tools/server/server-task.h — hydra_metrics field
tools/server/server.cpp — empty model path handling
src/llama-hydra.cpp — split_mode/override_tensor staging without ctx
Hydra parent issue
ddvnguyen/hydra_vortex#TBD
Tracks the C++ side of the hydra_config / hydra_metrics implementation.
What
hydra_configfrom chat completion request bodyhydra_metricsin every chat completion response (model_path, split_mode, tensor_split, GPU memory, t3_reloaded, t3_reload_ms)Files changed
tools/server/server-context.cpp— hydra_config parsing, first-load path, T3 rebuild, metricstools/server/server-context.h— meta fields, friend structtools/server/server-task.cpp— emit hydra_metrics in responsetools/server/server-task.h— hydra_metrics fieldtools/server/server.cpp— empty model path handlingsrc/llama-hydra.cpp— split_mode/override_tensor staging without ctxHydra parent issue
ddvnguyen/hydra_vortex#TBD