llama-bench: add kv-offload/workspace sweep flags - #70
Draft
Piggidragon wants to merge 1 commit into
Draft
Conversation
kv-gpu-layers, kv-cpu-pinned, recurrent-state-offload, phase-aware-workspace, and live-context-workspace exist as CLI/server context params (common/arg.cpp) but were never ported to llama-bench's own arg parser, so they could not be swept like the other KV-offload flags (no-kv-offload, no-op-offload). Wire them into cmd_params, cmd_params_instance, and test the same way no_kv_offload/no_op_offload already are: sweepable <0|1> lists for the four bools, a sweepable int list for kv-gpu-layers, applied on llama_context_params in to_llama_cparams(). They only affect context params, not model params, so equal_mparams() is unchanged. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012ghb3E6Q59LzHoLebeGzEu
github-actions
Bot
added
documentation
Piggidragon
marked this pull request as draft
September 5, 2026 21:28
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
--kv-gpu-layers,--kv-cpu-pinned,--recurrent-state-offload,--phase-aware-workspace, and--live-context-workspaceexist as CLI/server context params (common/arg.cpp) but were never wired intollama-bench's own arg parser (it doesn't usecommon_params_parse, so new flags need a manual port).cmd_params/cmd_params_instance/testthe same way the existingno_kv_offload/no_op_offloadflags are handled: sweepable<0|1>lists for the four booleans (-kvcp,-rso,-paw,-lcw), a sweepable int list forkv-gpu-layers(-kvgl), applied onllama_context_paramsinto_llama_cparams().equal_mparams()is unchanged and model reuse between test instances is unaffected.tools/llama-bench/README.md.Test plan
cmake --build --target llama-benchsucceedsllama-bench --helpshows the new flags with correct defaultsllama-bench -m <missing> -kvgl 4,8 -kvcp 1 -paw 0,1 -o csvreaches model-load (parsing succeeds) and the CSV header includes the new columns in the right place-o mdonly shows columns for values that were actually swept/changed from default (matches existing convention forno_kv_offloadetc.)🤖 Generated with Claude Code
https://claude.ai/code/session_012ghb3E6Q59LzHoLebeGzEu