docs: add docs/fork/ feature guide + README/AGENTS pointers - #61
Closed
Piggidragon wants to merge 4 commits into
Closed
docs: add docs/fork/ feature guide + README/AGENTS pointers#61Piggidragon wants to merge 4 commits into
Piggidragon wants to merge 4 commits into
Conversation
Collect the user-facing additions this branch carries over upstream master: the host-resident KV cache flags (--kv-cpu-pinned, --recurrent-state-offload, --kv-gpu-layers), the compute workspace flags, the speculative draft-context flags, and the optimizations that run with no flag. Includes the open PR GenerelSchwerz#57 (--attn-split, -sm tensor + -nkvo fix) and PR GenerelSchwerz#39 (--kv-pipeline-depth / --kv-pipeline-budget, llama-bench switches), marked as such. Assisted-by: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_013Bs926e6SdsxnkrqqKq5HP
From the moe-cache-drafting branch: --moe-expert-cache-size, --moe-expert-cache-l2-pinned-mb, --experimental-logs, the automatic grouped decode / prefetch / bias residency behaviour, and the layer-split-only and speculative interactions. Marked as coming from an unmerged branch, like the PR GenerelSchwerz#57 and GenerelSchwerz#39 material. Assisted-by: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_013Bs926e6SdsxnkrqqKq5HP
Short pointer to docs/llama-dev-guide.md instead of spelling the fork's additions out in the README. Keeps the rest of the README as upstream's. Assisted-by: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_013Bs926e6SdsxnkrqqKq5HP
Give fork-specific docs their own directory so they do not mix with the upstream docs/ tree. The guide becomes docs/fork/README.md, so browsing to docs/fork/ shows it. Update the README pointer and add a note to AGENTS.md that docs/fork/ is the home for fork docs and the guide tracks user-facing flag changes. Assisted-by: Claude Sonnet 5 Claude-Session: https://claude.ai/code/session_013Bs926e6SdsxnkrqqKq5HP
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Establishes
docs/fork/as the home for docs specific to this maintained fork,with the first page a user-facing guide to everything the branch adds over
upstream
master.Files
docs/fork/README.md- the guide (browsing todocs/fork/shows it).README.md- a 6-line note at the top linking todocs/fork/.llama/dev'sREADME is otherwise byte-identical to upstream; nothing fork-specific was in
it to remove.
AGENTS.md- one line under "Useful Resources":docs/fork/is where forkdocs go, upstream docs stay in
docs/, and the guide tracks user-facing flagchanges.
What the guide covers
LLAMA_ARG_*env var:
--kv-cpu-pinned,--recurrent-state-offload,--kv-gpu-layers,--phase-aware-workspace,--live-context-workspace,--spec-draft-ubatch-size,--spec-draft-kv-gpu-layers,--spec-mtp-rs-planes.quantization-preserving host KV copy, shared target/MTP workspaces.
llama-benchadditions and the extra environment variables.Material from branches not yet on
llama/dev, marked inline in the guide:-as/--attn-split, the-sm tensor+--no-kv-offloadcorrectness fix, the
--kv-gpu-layersslowest-link-first ordering.--kv-pipeline-depth/--kv-pipeline-budget, thellama-bench-kvcp/-rso/-kvpd/-kvpbswitches,GGML_SCHED_TRANSPORT_DEBUG.moe-cache-drafting- the CUDA MoE expert cache:--moe-expert-cache-size,--moe-expert-cache-l2-pinned-mb,--experimental-logs, plus the automatic grouped-decode / prefetch / biasresidency behaviour and the layer-split-only and speculative caveats.
Notes for reviewers
docs/multi-gpu.md, ASCII only perAGENTS.md. All internal links checked.#39(docs/kv-transport-pipelining.md,docs/repro/) andmoe-cache-drafting(README feature section, commit eb7f74e) carry their owndocs. When they rebase/land, those should move into
docs/fork/and theREADME section should collapse to the pointer added here, so there is a single
source.
moe-cache-draftingare unmerged; the flag names anddefaults for that material match those branches as of now.
AI usage disclosure: drafted with Claude Code, see the commit trailers.
🤖 Generated with Claude Code
https://claude.ai/code/session_013Bs926e6SdsxnkrqqKq5HP