Skip to content

Release to main: load local models on transformers 5 (#448) - #449

Merged
phanindra-max merged 2 commits into
mainfrom
dev
Sep 17, 2026
Merged

phanindra-max merged 2 commits into
mainfrom
dev

Conversation

@phanindra-max

Copy link
Copy Markdown
Member

Brings #448 from dev to main. It was reviewed and merged into dev in #448; this PR only promotes it.

Why this is urgent

On main, a fresh install cannot create a local-model extractor. That includes the CPU, no-API-key path that the documentation and the JOSS review thread point reviewers to:

LAiSERError: Could not initialize model 'Qwen/Qwen2.5-0.5B-Instruct': Qwen2ForCausalLM.__init__() got an unexpected keyword argument 'use_auth_token'

A fresh install now resolves transformers 5.17.0, which removed use_auth_token. The loader still passed it.

What reaches main

  • b015a10 fix(llm): pass token, not use_auth_token, when loading with transformers. The loader sends token= only when a token is given, so public models load the same way on every transformers version. Two tests guard against a regression.

Verification (from #448)

  • black, isort and flake8: clean. Tests (-m "not index"): 61 passed, 5 skipped (need API keys), 1 deselected.
  • In a fresh environment with transformers 5.17.0 and torch 2.14.0, the fixed code loaded Qwen/Qwen2.5-0.5B-Instruct on CPU with no API key and returned aligned skills. The same run on main fails with the error above.
  • The Quickstart example, run twice in separate processes in that environment, returned identical output that matches the documented example table.

After merge

  • Rerun the reviewer install command, pip install "git+https://github.com/LAiSER-Software/extract-module@main", in a fresh environment, and confirm the no-API-key example works.
  • Post the documentation and install update in the JOSS review thread, and add a note to [JOSS] Issues / suggestions documentation #438 mapping each point to its fix.

🤖 Generated with Claude Code

phanindra-max and others added 2 commits September 15, 2026 02:43
…formers

Installing from main into a fresh environment now resolves transformers
5.17.0 (CI resolves the same). transformers 5 no longer defines
`use_auth_token` anywhere, and from_pretrained forwards keywords it does not
recognise to the model's constructor. load_model_from_transformer passed
`use_auth_token=token` to both AutoTokenizer and AutoModelForCausalLM, so the
local-model path, including running on CPU with no API key, was exposed on
any current install.

CI did not catch it: the loader tests replace from_pretrained with fakes that
accept any keyword.

The loader now passes `token=` only when a token is given, and nothing when it
is not, so public models load the same way on every transformers version.
Two tests pin this: a given token reaches both from_pretrained calls as
`token` and `use_auth_token` is never sent, and no token keyword is sent
when the token is None or empty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
fix(llm): pass token, not use_auth_token, so local models load on transformers 5
Copilot AI lite review requested due to automatic review settings September 17, 2026 04:39

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@github-actions

Copy link
Copy Markdown

Code Hygiene

The following items are defined but not yet in active use. Routine cleanup — no impact on functionality.

  • laiser/llm_methods.py:87 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:45 — unused variable 'chat_format' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'exc' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'exc_type' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'tb' (100% confidence)
  • laiser/llm_models/llm_router.py:71 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:86 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:89 — unused variable 'args' (100% confidence)
  • laiser/services.py:132 — unused variable 'raw_description' (100% confidence)

@github-actions

Copy link
Copy Markdown

Complexity Analysis

Average: C — Moderate (13.75)

Hotspots tracked below. These are functional, well-exercised components — listed for visibility as the codebase scales.

Grade Name Type Score
F FAISSIndexManager.initialize_index Method 51
E SkillExtractionService.extract_and_align_core Method 36
E FAISSIndexManager.search_similar_skills Method 32
D ResponseParser._parse_skills_from_response Method 24
D FAISSIndexManager Class 22
C ResponseParser.parse_knowledge_task_response Method 20
C _BaseTaxonomyFAISSIndexManager.search_similar Method 18
C ResponseParser Class 17
C ResponseParser.parse_ksa_extraction_response Method 16
C ResponseParser.parse_skill_extraction_response Method 15
C openai_generate Function 14
C parse_output_vllm Function 13
C AlignmentService.align Method 11
C anthropic_generate Function 11
C LLMRouter._initialize_components Method 11
B SkillExtractionService._deduplicate Method 10
B load_model_from_vllm Function 10
B GeminiAPI.generate_with_config Method 10
B _BaseTaxonomyFAISSIndexManager.initialize_index Method 9
B ResponseParser.parse_ksa_details_response Method 8
B SkillExtractionService Class 8
B _BaseTaxonomyFAISSIndexManager Class 8
B llm_generate Function 8
B SkillExtractorRefactored._expand_allowed_sources Method 7
B get_completion_vllm Function 7
B llama_cpp_chat Function 7
B LLMRouter.generate Method 7
B AlignmentService Class 6
B SkillExtractionService._normalize_mixed_concept_rows Method 6
B fetch_model_output Function 6
B LlamaCppBackend.__init__ Method 6
B GeminiAPI Class 6

@phanindra-max
phanindra-max merged commit d4f541f into main Sep 17, 2026
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants