Skip to content

fix(llm): pass token, not use_auth_token, so local models load on transformers 5 - #448

Merged
phanindra-max merged 1 commit into
devfrom
fix/transformers-v5-token
Sep 17, 2026
Merged

phanindra-max merged 1 commit into
devfrom
fix/transformers-v5-token

Conversation

@phanindra-max

Copy link
Copy Markdown
Member

Problem

On current main, a fresh install cannot create a local-model extractor. That includes the CPU, no-API-key path the documentation and the JOSS review thread point reviewers to.

Reproduced by running the documented command in a new virtual environment:

pip install "git+https://github.com/LAiSER-Software/extract-module@main"

The install resolves to main at 86a2e32, with transformers 5.17.0, torch 2.14.0 and sentence-transformers 6.0.1. Creating the extractor then fails:

TypeError: Qwen2ForCausalLM.__init__() got an unexpected keyword argument 'use_auth_token'
laiser.exceptions.LAiSERError: Could not initialize model 'Qwen/Qwen2.5-0.5B-Instruct': Qwen2ForCausalLM.__init__() got an unexpected keyword argument 'use_auth_token'

load_model_from_transformer passed use_auth_token=token to both from_pretrained calls. transformers 5 no longer defines use_auth_token anywhere. Its from_pretrained forwards keywords it does not recognise to the model's constructor, which rejects them.

CI did not catch this. It resolves the same transformers 5.17.0, but the loader tests replace from_pretrained with fakes that accept any keyword. My earlier end-to-end runs used a local transformers 4.49, where use_auth_token was only deprecated.

Fix

b015a10 passes token= only when a token is given, and no token keyword when it is not, so public models load the same way on every transformers version.

Two new tests pin it down:

  • A given token reaches both from_pretrained calls as token, and use_auth_token is never sent.
  • No token keyword is sent when the token is None or empty.

Verification

  • black, isort and flake8: clean
  • Tests (-m "not index"): 61 passed, 5 skipped (need API keys), 1 deselected
  • Fresh virtual environment with transformers 5.17.0, torch 2.14.0 and sentence-transformers 6.0.1. I reinstalled LAiSER from b015a10 there, with dependencies unchanged, and ran the documented no-API-key example on CPU (Qwen/Qwen2.5-0.5B-Instruct, use_gpu=False). The extractor initialised, routed to the transformers backend, and returned 8 aligned skills. The same run against main (86a2e32) fails with the error above.

Follow-up

This needs to reach main before the no-API-key instructions work for a fresh install. The JOSS review thread and #438 point there, so both will be updated once it has.

🤖 Generated with Claude Code

…formers

Installing from main into a fresh environment now resolves transformers
5.17.0 (CI resolves the same). transformers 5 no longer defines
`use_auth_token` anywhere, and from_pretrained forwards keywords it does not
recognise to the model's constructor. load_model_from_transformer passed
`use_auth_token=token` to both AutoTokenizer and AutoModelForCausalLM, so the
local-model path, including running on CPU with no API key, was exposed on
any current install.

CI did not catch it: the loader tests replace from_pretrained with fakes that
accept any keyword.

The loader now passes `token=` only when a token is given, and nothing when it
is not, so public models load the same way on every transformers version.
Two tests pin this: a given token reaches both from_pretrained calls as
`token` and `use_auth_token` is never sent, and no token keyword is sent
when the token is None or empty.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 15, 2026 18:20
@github-actions

Copy link
Copy Markdown

Code Hygiene

The following items are defined but not yet in active use. Routine cleanup — no impact on functionality.

  • laiser/llm_methods.py:87 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:45 — unused variable 'chat_format' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'exc' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'exc_type' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'tb' (100% confidence)
  • laiser/llm_models/llm_router.py:71 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:86 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:89 — unused variable 'args' (100% confidence)
  • laiser/services.py:132 — unused variable 'raw_description' (100% confidence)

@github-actions

Copy link
Copy Markdown

Complexity Analysis

Average: C — Moderate (13.75)

Hotspots tracked below. These are functional, well-exercised components — listed for visibility as the codebase scales.

Grade Name Type Score
F FAISSIndexManager.initialize_index Method 51
E SkillExtractionService.extract_and_align_core Method 36
E FAISSIndexManager.search_similar_skills Method 32
D ResponseParser._parse_skills_from_response Method 24
D FAISSIndexManager Class 22
C ResponseParser.parse_knowledge_task_response Method 20
C _BaseTaxonomyFAISSIndexManager.search_similar Method 18
C ResponseParser Class 17
C ResponseParser.parse_ksa_extraction_response Method 16
C ResponseParser.parse_skill_extraction_response Method 15
C openai_generate Function 14
C parse_output_vllm Function 13
C AlignmentService.align Method 11
C anthropic_generate Function 11
C LLMRouter._initialize_components Method 11
B SkillExtractionService._deduplicate Method 10
B load_model_from_vllm Function 10
B GeminiAPI.generate_with_config Method 10
B _BaseTaxonomyFAISSIndexManager.initialize_index Method 9
B ResponseParser.parse_ksa_details_response Method 8
B SkillExtractionService Class 8
B _BaseTaxonomyFAISSIndexManager Class 8
B llm_generate Function 8
B SkillExtractorRefactored._expand_allowed_sources Method 7
B get_completion_vllm Function 7
B llama_cpp_chat Function 7
B LLMRouter.generate Method 7
B AlignmentService Class 6
B SkillExtractionService._normalize_mixed_concept_rows Method 6
B fetch_model_output Function 6
B LlamaCppBackend.__init__ Method 6
B GeminiAPI Class 6

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

@phanindra-max phanindra-max self-assigned this Sep 17, 2026
@phanindra-max
phanindra-max merged commit bc9cee2 into dev Sep 17, 2026
18 checks passed
phanindra-max added a commit that referenced this pull request Sep 17, 2026
Release to main: load local models on transformers 5 (#448)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants