Repository navigation
cli: import and export models on an external drive, and work offline - #72
Open
DevelopmentCats wants to merge 5 commits into
Open
DevelopmentCats wants to merge 5 commits into
DevelopmentCats wants to merge 5 commits into
Conversation
Engines say how their server stays off the network and which variable sets its hub cache, so no code has to name an engine to do it.
Reads the cache layout all engines share, and copies complete models between caches with every file checked against its hash.
new, inspect and search answer from the cache when the Hub is not used or does not answer, instead of failing on the network.
import and export move models on a drive. A new profile made offline tells its server to stay off the network, and doctor shows which cache the server reads.
Explains moving models on a drive, which cache the server reads, and what a profile made offline sets.
| if !ok { | ||
| return nil, fmt.Errorf("%s: %w", repo, ErrOffline) | ||
| } | ||
| f, err := os.Open(filepath.Join(dir, filepath.FromSlash(name))) |
| return "", "", false | ||
| } | ||
| dir = filepath.Join(RepoDir(hub, repo), "snapshots", rev) | ||
| if st, err := os.Stat(dir); err != nil || !st.IsDir() { |
| // mainRevision reads refs/main, or names the one snapshot a repo has when it | ||
| // has no ref. | ||
| func mainRevision(repoDir string) (string, error) { | ||
| if b, err := os.ReadFile(filepath.Join(repoDir, "refs", "main")); err == nil { |
| return rev, nil | ||
| } | ||
| } | ||
| entries, err := os.ReadDir(filepath.Join(repoDir, "snapshots")) |
There was a problem hiding this comment.
🟡 Changes recommended
Predictable temporary-file paths can follow attacker-controlled symlinks and overwrite files outside the cache before hash verification.
11 open issues
Create temporary files exclusively to prevent symlink overwrites · New Resolve recipe revisions across all cached snapshots · New Only treat transport failures as Hub outages · New Return the file directly to ensure response bodies close descriptors · New Preserve cached base-model metadata in offline model results · New Reject incomplete repository scans before calculating model size · New Preserve per-file hashes when exporting and importing caches · New Reuse existing snapshots before copying files into blobs · New Use consistent environment expansion for cache paths and the server · New Report directory traversal errors instead of exporting partial models · New Validate that all advertised GGUF shards are present · New
What changed in this PR
This PR adds drive-based model transfer and offline cache access to llm-helper, connecting both to the existing engine definitions and shared profile-creation logic.
Changes:
- Adds import/export commands with hash checks, resumable copies, space checks, and recipe staging.
- Adds local-cache fallback for Hub readers and engine-specific offline settings for new profiles.
- Adds cache diagnostics, regression tests, and user documentation.
| File | Description |
|---|---|
| internal/web/models.go | Applies offline settings during panel profile creation. |
| internal/resolve/offline.go | Adds engine-declared offline settings to profile plans. |
| internal/resolve/offline_internal_test.go | Tests offline environment and argument handling. |
| internal/hfcache/space_other.go | Provides unsupported-platform filesystem fallbacks. |
| internal/hfcache/space_linux.go | Detects Linux free space and FAT filesystems. |
| internal/hfcache/space_darwin.go | Detects macOS free space and FAT filesystems. |
| internal/hfcache/scan.go | Scans caches, download directories, and model folders. |
| internal/hfcache/hfcache.go | Defines cache paths, identifiers, and offline detection. |
| internal/hfcache/hfcache_internal_test.go | Tests scanning, copying, integrity, and filesystem checks. |
| internal/hfcache/copy.go | Implements model copying, hashing, and capacity checks. |
| internal/hf/local.go | Adds local-cache readers and temporary Hub fallback. |
| internal/hf/local_internal_test.go | Tests offline reads and Hub recovery. |
| internal/hf/hf.go | Integrates cache fallback into Hub operations. |
| internal/hf/arch.go | Reads model configuration through the shared fallback. |
| internal/engine/registry.go | Extends the provider interface with cache and offline settings. |
| internal/engine/engine.go | Defines and validates engine offline metadata. |
| internal/engine/engine_test.go | Tests offline declarations and validation. |
| internal/control/hubcache.go | Shares cache resolution and offline profile setup. |
| internal/control/hubcache_internal_test.go | Tests cache selection and offline profile creation. |
| internal/cli/search.go | Identifies offline results and empty-cache matches. |
| internal/cli/root.go | Registers transfer commands and initializes cache access. |
| internal/cli/root_internal_test.go | Updates the expected command set. |
| internal/cli/recipelaunch.go | Adds drive-import guidance for missing recipe models. |
| internal/cli/profiles.go | Applies offline settings during CLI profile creation. |
| internal/cli/hfcache.go | Implements transfer commands and model selection. |
| internal/cli/hfcache_test.go | Tests transfers, recipe staging, and offline search. |
| internal/cli/engines_internal_test.go | Checks recipe import guidance. |
| internal/cli/doctor_internal_test.go | Tests cache reporting and shell mismatch warnings. |
| internal/cli/detectcmds.go | Reports cache location, space, and offline status. |
| docs/reference/cli/import.md | Adds the generated import reference. |
| docs/reference/cli/export.md | Adds the generated export reference. |
| docs/reference/cli.md | Lists the new transfer commands. |
| docs/providers.md | Documents engine cache and offline fields. |
| docs/offline.md | Explains drive transfers and offline operation. |
| docs/manifest.json | Routes the new documentation pages. |
| CHANGELOG.md | Records transfer, offline, and diagnostic features. |
| catalogue/engines/vllm.json | Declares vLLM’s offline environment setting. |
| catalogue/engines/README.md | Documents the new engine metadata fields. |
| catalogue/engines/mlx.json | Declares MLX’s offline environment setting. |
| catalogue/engines/llamacpp.json | Declares llama.cpp’s cache variable and probed offline flag. |
Review effort: Balanced
| if st, err := in.Stat(); err != nil || st.Size() != size { | ||
| return "", fmt.Errorf("%s changed size while being read", src) | ||
| } | ||
| out, err := os.OpenFile(tmp, os.O_CREATE|os.O_TRUNC|os.O_WRONLY, 0o644) |
Comment on lines
+289
to
+290
| case m.Revision != p.Revision: | ||
| return nil, fmt.Errorf("the recipe pins %s at %s, but the drive has %s", p.ID, p.Revision, m.Revision) |
Comment on lines
+641
to
+643
| if hub := canFallBack(ctx, err); hub != "" && page == 0 { | ||
| return localTree(hub, repo) | ||
| } |
| if err != nil { | ||
| return &http.Response{StatusCode: http.StatusNotFound, Body: http.NoBody}, nil | ||
| } | ||
| return &http.Response{StatusCode: http.StatusOK, Body: io.NopCloser(f), ContentLength: -1}, nil |
| if _, _, ok := hfcache.Snapshot(hub, repo); !ok { | ||
| return Model{}, fmt.Errorf("%s: %w", repo, ErrOffline) | ||
| } | ||
| return Model{ID: repo}, nil |
Comment on lines
+184
to
+188
| tmp := partialPath(repoDir, f.Path) | ||
| if _, err := copyFile(f.Abs, tmp, f.Etag, f.Size); err != nil { | ||
| return false, err | ||
| } | ||
| return true, os.Rename(tmp, dst) |
Comment on lines
+193
to
+194
| name := f.Etag | ||
| if name == "" || !sizeIs(filepath.Join(repoDir, "blobs", name), f.Size) { |
Comment on lines
+55
to
+56
| v = strings.ReplaceAll(v, "${HOME}", home) | ||
| return strings.ReplaceAll(v, "$HOME", home) |
Comment on lines
+221
to
+223
| if err != nil || d.IsDir() { | ||
| return nil | ||
| } |
Comment on lines
+313
to
+315
| for _, f := range m.Files { | ||
| if !strings.HasSuffix(f.Path, ".index.json") { | ||
| continue |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Lets someone move models to a machine on an external drive and serve them without the Hub, for offline or air-gapped machines.
Commands
llm-helper export <org/model>... <drive-path>copies models from the Hugging Face cache onto a drive. The drive holds real files with no symlinks, so exFAT works.llm-helper import <drive-path>copies them into the cache the server reads. It accepts a drive written byexport, a copy of a cache, anhf download --local-dirfolder (with--repo), or a plain model folder (with--repoand a commit from--revisionor git HEAD).import --recipe <id>stages the models a recipe pins.Files are verified against their hashes as they are copied. An interrupted copy resumes. Incomplete models, a drive that is too small, and FAT32 (4 GiB file limit) are refused. The result is the standard
blobs/+snapshots/+refs/mainlayout, sohuggingface_hubtreats it as its own cache.Offline mode
With
HF_HUB_OFFLINE=1inconfig.env, or once the Hub doesn't answer and the cache holds a model,new,inspectandsearchread the local cache. A new profile tells its server to stay off the network. How is engine data:offlineincatalogue/engines/*.json(HF_HUB_OFFLINE=1for vLLM and MLX;--offlinefor llama.cpp, added only if the installedllama-server --helplists it).hub_cache_envnames an engine's own cache variable (LLAMA_CACHE). No engine names in Go.doctorprints the cache the server reads and its free space, and warns when the shell'sHF_HOME/HF_HUB_CACHEdiffer from what the service sees.Design notes
config.env/secrets.env, which is what the service sources, not the login shell.refs/mainis written only when absent; a different existing ref is left alone with a note.trees/<commit>.jsonis copied when present, never synthesized.*.llm-helper.incompleteinblobs/, so anhfpartial is never touched.internal/control(HubCache,HubOffline,KeepOffline) and is used by both the CLI and the panel's new-profile path.Verification
gofmt,go vet ./...,go test -race ./..., gendocs, builds for linux/amd64, linux/arm64, darwin/arm64, prettier 3.9.6: all pass.huggingface_hub0.36.2 on scratch dirs withtrl-internal-testing/tiny-Qwen3ForCausalLM: cache →export→ drive (0 symlinks) →importinto a new cache (12 symlinks); a second import skipped all files;snapshot_download(local_files_only=True)andscan_cache_dirread the imported cache; an onlinesnapshot_downloadagainst it downloaded nothing.--local-dirimport works with--repoand fails clearly without it. Offlinesearch/inspect/doctoranswered from the cache.Not verified
--offlineon a real build (only a fake--helpin tests). Offline loading from the cache had upstream bugs around April 2026 (Misc. bug:--offlineflag broken ggml-org/llama.cpp#21302, #21364).newend to end, since it starts services; offline profile creation is tested throughKeepOfflineandprofile.Createper engine.Follow-ups
Panel import/export, an
--update-refflag, copying to several nodes in one command, serving directly off the drive, and accounting for a profile's ownHF_HOMEoverride.Note
This conflicts mechanically with #71 (both add fields/methods at the same place in
internal/engine/engine.go,registry.go,engine_test.go, and both add CHANGELOG entries). Whichever merges second gets rebased.