recommend(): host-RAM-aware pin decision#42
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the pure policy engine with the pinning axis.
recommend()now returnspin,pinned_set_gb, andhost_ram_gbalongside the precision/device recommendation.Policy: phase-swapped weights are page-locked host copies that transfer at DMA rate (25 GB/s measured), but pinned pages are unswappable — on small-RAM machines they turn memory pressure into OOM kills. So
pin = TRUEonly when available host RAM (LinuxMemAvailable; NA elsewhere) covers the model's estimated pinned set twice over;FALSEbelow that;FALSEon the cpu tier (nothing stages);TRUEwhen RAM is undetectable, since page-locking already fails soft per component and /proc-less platforms rarely pair with CUDA.Pinned-set estimates are coarse artifact-scale numbers per model+precision (quantized checkpoint + text-encoder host copy) — the decision needs the order of magnitude, not the byte.
Also corrects stale CLAUDE.md prose claiming auto-device configuration requires gpuctl:
auto_devices()andrecommend()are self-contained (nvidia-smi + /proc), with no gpuctl reference anywhere in code or DESCRIPTION.Tests: 26 new results in test_recommend.R (80 total green) — large-RAM pin, small-RAM refusal, cpu-tier FALSE, NA fail-soft, detection helper, unknown-tier fallback.