Skip to content

fix(lora): resolve adapters through configured caches - #833

Merged
svonava merged 2 commits into
mainfrom
fix/lora-cluster-cache-01a145dd
Oct 20, 2026
Merged

svonava merged 2 commits into
mainfrom
fix/lora-cluster-cache-01a145dd

Conversation

@svonava

@svonava svonava commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Declared runtime LoRA adapters bypassed SIE’s configured cluster cache and downloaded directly from the Hub. This breaks cache-only deployments even when the adapter is already present in their local or cluster cache.

Route the PEFT mixin, SGLang and CrossEncoder LoRA loaders through one shared source resolver after their existing declaration/admission checks. It honors local → cluster → permitted Hub lookup, requires the exact immutable revision when supplied, validates the two adapter files, and rejects cross-repository path redirects. Explicit local HF snapshot directories retain their normal links to the same repository’s blobs. A local directory combined with a Hub revision is refused rather than silently ignoring the pin.

Allowed Hub fallback remains limited to adapter_config.json and adapter_model.safetensors. Optional keyword-only cache arguments let LoRA require that pair and hydrate incomplete snapshots through the existing tiers without deleting cached data. Unpinned LoRA resolves the normal main reference. Ordinary callers retain their unfiltered download and existing cache-hit defaults. PEFT config loading is local-only after resolution. Scoring, selection and adapter lifecycle methods are unchanged.

Builds on merged #827 for cluster download containment. That prerequisite provides per-file atomic replacement; it does not make a whole snapshot transactional or defend against a hostile concurrent writer with the same local privileges. The shared GLiNER2/decision runtime integration is handled separately.

Validation: 519 focused CPU cases pass across the resolver, all three adapter loaders, and SDK cache/storage. These include Hub/network-denied cache controls, partial-cache recovery, exact-pin and repository-boundary negatives, real tiny PEFT/CrossEncoder lifecycle checks, and both localhost SDK routes. Ruff, formatting and scoped type checks pass. Independent source/adversarial review passed eight additional network-denied controls. Full current-head CI is pending. No GPU or performance claim.

The branch now includes current main to refresh review ancestry after the prerequisite merge. All ten resolver source/test files remain byte-identical to the reviewed implementation; only five already-landed main files were inherited. Eight focused startup/CI contract checks pass, and the 519-case resolver validation maps by identical source. Fresh current-head CI and review are pending.

Summary by CodeRabbit

  • New Features

    • Added required-file validation for cached models and optional file-pattern filtering for Hugging Face downloads.
    • Added consistent LoRA adapter resolution from local directories and configured caches, with controlled Hugging Face fallback.
    • Added support for loading pinned LoRA adapters offline when complete cached files are available.
  • Bug Fixes

    • Improved protection against unsafe paths, symlinks, repository aliases, incomplete downloads, and partial filesystem updates.
    • LoRA sources now require valid repositories, immutable revisions, and complete adapter files.
    • Improved cleanup and preservation of existing files when downloads fail.

@svonava

svonava commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

Please review the current head 3dde949. This resolves admitted LoRA sources through the configured cache policy and depends on #827 for download containment.

@svonava
svonava requested a review from a team as a code owner October 20, 2026 00:40
@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration
  • Configuration used: Organization UI
  • Review profile: CHILL
  • Plan: Team
  • Run ID: 12c62fa1-d488-428c-be0e-415612b94c21
📥 Commits

Reviewing files that changed from the base of the PR and between 0c34750 and 58a6c9c.

📒 Files selected for processing: 10
  • packages/sie_sdk/src/sie_sdk/cache.py
  • packages/sie_sdk/tests/test_cache.py
  • packages/sie_server/src/sie_server/adapters/_lora_source.py
  • packages/sie_server/src/sie_server/adapters/cross_encoder/_lora.py
  • packages/sie_server/src/sie_server/adapters/peft_lora_mixin.py
  • packages/sie_server/src/sie_server/adapters/sglang/_lora.py
  • packages/sie_server/tests/adapters/test_cross_encoder_lora.py
  • packages/sie_server/tests/adapters/test_lora.py
  • packages/sie_server/tests/adapters/test_lora_source.py
  • packages/sie_server/tests/adapters/test_sglang_lora.py

Limit details: You’ve used all 10 included reviews currently available. Your 16 included PR review attempts over the past 7 days set your current allowance at 10 reviews per hour.

📝 Walkthrough

The cache API validates required files and supports Hugging Face file filters. LoRA adapters use a shared resolver for local and cached Hub sources, with revision and artifact validation.

Changes: Cache-backed LoRA source resolution

Layer / File(s) Summary
Required-file cache checks and Hub filters
packages/sie_sdk/src/sie_sdk/cache.py, packages/sie_sdk/tests/test_cache.py
Cache checks validate required-file paths and require complete snapshots. ensure_model_cached propagates required files and optional Hub file filters.
LoRA source resolution
packages/sie_server/src/sie_server/adapters/_lora_source.py, packages/sie_server/tests/adapters/test_lora_source.py
The shared resolver validates local directories and cached Hub snapshots, including repository identity, immutable revisions, required artifacts, and contained links.
Adapter fetch integration
packages/sie_server/src/sie_server/adapters/cross_encoder/_lora.py, packages/sie_server/src/sie_server/adapters/peft_lora_mixin.py, packages/sie_server/src/sie_server/adapters/sglang/_lora.py, packages/sie_server/tests/adapters/*
The three adapter fetch paths delegate source resolution to the shared resolver. Tests cover offline cache loading and adapter lifecycle behavior.

Sequence Diagram(s)

sequenceDiagram
  participant Adapter as LoRA adapter
  participant Resolver as resolve_lora_source
  participant Cache as Configured cache tiers
  participant Hub as Hugging Face Hub
  Adapter->>Resolver: source and optional revision
  Resolver->>Cache: search for complete adapter files
  Cache-->>Resolver: cached snapshot or no complete match
  opt No complete cached snapshot and fallback is enabled
    Resolver->>Hub: request adapter files
    Hub-->>Resolver: downloaded snapshot
  end
  Resolver-->>Adapter: validated adapter directory
Loading

Suggested reviewers: krisztian-gajdar

Priority: ➖ Normal

Merge Risk: ⚪ Minimal · up to 58a6c

No identified issue needs correction before merge; normal validation remains appropriate.

Pre-merge checks: Passed 4 · Failed 1

Failed Failed checks: 1 warning

Check name Status Explanation Resolution
Docstring Coverage Warning Docstring coverage is 11.32% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 53 functions across 10 files. Write docstrings for the functions missing them to satisfy the coverage threshold.
Passed Passed checks: 4 passed
Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check Passed The title clearly and concisely describes the main change: routing LoRA adapter resolution through configured caches.
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
  • Fix all pre-merge checks with AI
✨ Finishing Touches: 💡 1
📝 Generate docstrings: 💡
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Usage-based review receipt

Note

This review was completed with usage-based billing: files reviewed beyond your plan's included limits are billed at $0.25/file. View usage-based billing.

Comment @coderabbitai help to get the list of available commands.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Oct 20, 2026
@svonava
svonava dismissed coderabbitai[bot]’s stale review October 20, 2026 00:44

The merge-base changed after approval.

@svonava

svonava commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

The prerequisite #827 is now merged. Approval 5579476211 on unchanged head 3dde949 was automatically dismissed with “The merge-base changed after approval.” Please refresh the full review against the current main merge base. No source or dependency changes were made.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. Your current included review allowance is based on your included PR review attempts over the past 7 days. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 16 minutes.

coderabbitai[bot]
coderabbitai Bot previously approved these changes Oct 20, 2026
@svonava
svonava dismissed coderabbitai[bot]’s stale review October 20, 2026 00:50

The merge-base changed after approval.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. Your current included review allowance is based on your included PR review attempts over the past 7 days. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 9 minutes.

@svonava

svonava commented Oct 20, 2026

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

Please review exact head 58a6c9cd7c62c9854073bf39474463fafac0bc6d. This normal merge includes current main 0c347501aba5595e19a327a8630b25921a517400 after two prior approvals were automatically dismissed for merge-base changes. All ten resolver source/test files remain byte-identical to 3dde949125c4257c70bd0d1f30f6db7c092cc27e; the only inherited files are the five already-landed #825/#826 changes. Eight focused integration checks pass; the 519-case resolver validation maps by identical source.

@coderabbitai

coderabbitai Bot commented Oct 20, 2026 •

Copy link
Copy Markdown
✅ Action performed

Full review finished.


Your included review limit is currently reached under our Fair Usage Limits Policy. Your current included review allowance is based on your included PR review attempts over the past 7 days. This review may still proceed through usage-based billing if eligible. Your next included review will be available in 4 minutes.

@svonava
svonava merged commit e251cac into main Oct 20, 2026
46 checks passed
@svonava
svonava deleted the fix/lora-cluster-cache-01a145dd branch October 20, 2026 01:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant