Skip to content

docs: bring #438 documentation and CPU fixes to main - #443

Merged
phanindra-max merged 8 commits into
mainfrom
dev
Sep 15, 2026
Merged

phanindra-max merged 8 commits into
mainfrom
dev

Conversation

@phanindra-max

@phanindra-max phanindra-max commented Sep 14, 2026 •

Copy link
Copy Markdown
Member

Brings the #438 documentation work from dev to main. It was reviewed and merged into dev in #442. This PR promotes it, plus one fix made in response to this PR's own review (25c1314, below).

Once merged, the Docs workflow builds the site from main and deploys it to https://laiser-software.github.io/extract-module/. Pages is already enabled with GitHub Actions as its source.

What reaches main

  • 54ee128 fix(llm): run Hugging Face models on CPU without an API key. The CPU path never worked: 8-bit quantization was forced without CUDA, and generation was routed to vLLM with no engine. Verified end to end on CPU with no key.
  • 0330ad3 docs(api): correct extract_concepts threshold defaults and timing. The docstrings documented thresholds of 0.20/0.45/0.55; the code uses 0.60/0.50/0.50.
  • d948e6a docs: build the documentation site from this repository. MkDocs Material, API reference generated from docstrings, mkdocs build --strict on PRs, deploy on main.
  • c314289 docs(readme): say what LAiSER does, fix formatting, add keyless and Colab paths.
  • 1df8b1f docs(paper): correct the license and install command. The paper claimed MIT and a nonexistent laiser[cpu] extra.
  • bc965b9 fix: address review findings on docs: move documentation into this repo; fix keyless CPU extraction #442. A failed local model load now raises instead of silently skipping rows. Also: LLAMA_CPP_THREADS is cast to int, top_k is documented as per concept type, the syllabus column requirement is explained, and levels/batch_size are documented as no-ops.
  • 25c1314 fix(gpu): install 8-bit dependencies with laiser[gpu]; correct provider docs. Added in response to review of this PR. The gpu extra now includes bitsandbytes and accelerate: when vLLM cannot load a model, the transformers fallback loads it in 8-bit and needs both, so that fallback could not run from the documented install. The providers page now describes accurately when the fallback retries, which error a failed load raises, and that CPU users must pass a model_id, because the default checkpoint only runs on a GPU.

Full detail, verification and the item-by-item mapping to #438 are in #442.

After merge

🤖 Generated with Claude Code

phanindra-max and others added 7 commits September 13, 2026 23:08
Constructing SkillExtractorRefactored with a Hugging Face model id and
use_gpu=False could not produce any output. Reproduced with
Qwen/Qwen2.5-0.5B-Instruct and no provider key set:

    WARNING: Transformer model initialization failed: CUDA is required but
    not available for bitsandbytes.
    AttributeError: 'NoneType' object has no attribute 'generate'

Two independent bugs:

- load_model_from_transformer always requested 8-bit bitsandbytes
  quantization, which requires CUDA, so the model never loaded on CPU.
  Quantization now applies only when a CUDA device is available. On CPU a
  load failure raises the original error rather than falling back to a 7B
  AWQ checkpoint that cannot run on CPU either.
- LLMRouter.generate sent every non-hosted model to
  llm_generate_vllm(prompt, self.llm). A transformers model lives in
  self.model / self.tokenizer and self.llm is None; the existing
  llm_generate helper was never called. Transformers-loaded models now
  route there.

llm_generate also needed fixing to produce parseable output. It decoded the
prompt plus the completion, so the prompt's own example JSON could be matched
by the response parser; it now returns only the new tokens. It applies the
model's chat template when one exists, and uses MAX_NEW_TOKENS (1000) instead
of a hard-coded 100 that truncated JSON mid-list.

After the fix, on a laptop CPU with no key: load 7.9s, extraction 19s,
returning "Python programming" -> "Program in Python" (ukos, 0.70) and
"Docker" -> "Docker" (onet, 0.75).

Also:
- model_id="openai" fell through router init and was looked up as a Hugging
  Face model id; hosted providers now return before any local loading.
- A startup debug line read router.llama_llm, which is never set, so it
  always printed False.

tests/test_llm_router.py adds offline coverage with no weights downloaded:
routing to transformers vs vLLM, completion-only decoding, chat template use,
quantization only on CUDA, and raising instead of falling back on CPU.

Part of #438 (example that runs without an API key).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The API reference is now generated from these docstrings, and two of them
described behaviour the code does not have.

similarity_thresholds was documented as defaulting to
{"skill": 0.20, "knowledge": 0.45, "task": 0.55}, and similarity_threshold as
defaulting to 0.20. When neither is passed, extract_and_align_core starts from
DEFAULT_SIMILARITY_THRESHOLDS in config.py, which is
{"skill": 0.60, "knowledge": 0.50, "task": 0.50}; a scalar
similarity_threshold replaces all three, and similarity_thresholds overrides
the types it names. The docstrings in SkillExtractorRefactored and the
service now say so. Behaviour is unchanged.

timing was documented as printing per-stage timings. It is accepted and
passed through but never read, so it is now documented as having no effect.

The service docstring's list of output columns also omitted Source Url.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The documentation site was generated from a separate docs repository, which
drifted from the code and has been deleted. Documentation now lives in docs/
here and is reviewed with the changes it describes.

- MkDocs with Material, as before, plus mkdocstrings so the API reference is
  generated from docstrings rather than maintained by hand.
- Pages: home, installation, quickstart, usage, model providers, taxonomies,
  the existing examples, cookbook notebooks, and API reference. Every option
  of extract_concepts, every output column, and each provider's requirements
  and environment variables are documented against the current code.
- Internal notes already in docs/ (refactoring guide, vLLM batching,
  v0.5 architecture) stay in the repo but are excluded from the site.
- docs.yml runs `mkdocs build --strict` on PRs touching docs/, mkdocs.yml or
  laiser/, so broken links, missing anchors and unresolvable API references
  fail review. Merges to main deploy to GitHub Pages at
  laiser-software.github.io/extract-module/.
- docs/requirements.txt pins mkdocs<2: MkDocs 2.0 removes the plugin system
  mkdocstrings and Material rely on.

Part of #438.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…olab paths

Responds to the README points in #438.

- About: rewritten to describe the actual input and output (a table of
  documents in, one row per taxonomy match out) and to name the taxonomies
  (ESCO, O*NET, UK Skills Classification, Open Skills Network) instead of
  general claims about AI and insight.
- Architecture: each stage's description sat on an indented line under its
  numbered item, which GitHub renders run into the title. Stages are now
  "**Stage** - description" on one line. Common runtime options had the same
  defect and is now a table.
- Usage leads with an example that runs on CPU with no API key.
- New "Try it in Google Colab" section linking the three cookbook notebooks,
  which use the current API.
- Links to the documentation site, PyPI and the cookbook.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The summary stated the code is under the MIT License; LICENSE and
pyproject.toml are BSD-3-Clause. It also offered `pip install laiser[cpu]`,
but the package defines no cpu extra. Now `pip install laiser`, or
`laiser[gpu]` for GPU inference.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every point in Copilot's review was checked against the code; all held.

Code
- LLMRouter no longer swallows a local model load failure. The CPU branch
  caught the loader's exception and returned a router with no model; every
  generate() call then failed and extract_and_align_core skipped the row
  silently. Construction now raises LAiSERError, on CPU and on GPU when both
  vLLM and the transformers fallback fail. A vLLM failure alone still falls
  back to transformers.
- LLAMA_CPP_THREADS was read from the environment as a string and passed to
  llama.cpp's n_threads unchanged; it is now converted to int.

Docstrings (they generate the API reference)
- top_k caps matches per concept type per document, not per document: each
  type is aligned in its own call, so all three types can return 3x top_k.
- similarity_thresholds overrides similarity_threshold for the types it
  names.
- levels and batch_size are accepted and passed through but never read; both
  are now documented as having no effect, like timing.

Docs
- Syllabus input: extract_and_align_core reads only columns listed in
  text_columns, so description and learning_outcomes must both be listed.
  The page said they were read regardless.
- top_k corrected in README, index and usage.
- Each extracted phrase is matched to its single closest entry (the FAISS
  search uses top_k=1), so the quickstart's "one phrase can match several
  entries" was wrong.
- Usage now explains how per-type columns (Raw Skill, Taxonomy Knowledge,
  ...) are merged into Raw Concept / Taxonomy Concept, replaces the
  `extract_concepts(..., output_csv_path=...)` placeholder with a real call,
  and documents extract, levels and batch_size.
- providers.md dropped LAISER_GEMINI_TIMEOUT: GeminiAPI stores the value but
  never passes it to the client or the request.

Tests: four more in test_llm_router.py, through LLMRouter's constructor as
SkillExtractionService builds it: CPU load failure raises, GPU raises when
vLLM and transformers both fail, GPU falls back to transformers when only
vLLM fails, and hosted providers load no local weights.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs: move documentation into this repo; fix keyless CPU extraction
Copilot AI lite review requested due to automatic review settings September 14, 2026 04:10
@github-actions

Copy link
Copy Markdown

Code Hygiene

The following items are defined but not yet in active use. Routine cleanup — no impact on functionality.

  • laiser/llm_methods.py:87 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:30 — unused variable 'chat_format' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:69 — unused variable 'exc' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:69 — unused variable 'exc_type' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:69 — unused variable 'tb' (100% confidence)
  • laiser/llm_models/llm_router.py:70 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:85 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:88 — unused variable 'args' (100% confidence)
  • laiser/services.py:130 — unused variable 'raw_description' (100% confidence)

@github-actions

Copy link
Copy Markdown

Complexity Analysis

Average: C — Moderate (14.310344827586206)

Hotspots tracked below. These are functional, well-exercised components — listed for visibility as the codebase scales.

Grade Name Type Score
F FAISSIndexManager.initialize_index Method 51
E SkillExtractionService.extract_and_align_core Method 36
E FAISSIndexManager.search_similar_skills Method 32
D ResponseParser._parse_skills_from_response Method 24
D FAISSIndexManager Class 22
C ResponseParser.parse_knowledge_task_response Method 20
C _BaseTaxonomyFAISSIndexManager.search_similar Method 18
C ResponseParser Class 17
C ResponseParser.parse_ksa_extraction_response Method 16
C ResponseParser.parse_skill_extraction_response Method 15
C openai_generate Function 14
C parse_output_vllm Function 13
C AlignmentService.align Method 11
C LLMRouter._initialize_components Method 11
B SkillExtractionService._deduplicate Method 10
B load_model_from_vllm Function 10
B _BaseTaxonomyFAISSIndexManager.initialize_index Method 9
B anthropic_generate Function 9
B ResponseParser.parse_ksa_details_response Method 8
B SkillExtractionService Class 8
B _BaseTaxonomyFAISSIndexManager Class 8
B GeminiAPI.generate_with_config Method 8
B SkillExtractorRefactored._expand_allowed_sources Method 7
B get_completion_vllm Function 7
B LLMRouter.generate Method 7
B AlignmentService Class 6
B SkillExtractionService._normalize_mixed_concept_rows Method 6
B fetch_model_output Function 6
B LlamaCppBackend.__init__ Method 6

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The GPU fallback lacks required quantization dependencies, and provider fallback/error behavior is documented inaccurately.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Promotes the reviewed documentation site and CPU inference fixes from dev to main.

Changes:

  • Adds MkDocs documentation and GitHub Pages deployment.
  • Fixes CPU model loading, routing, decoding, and error handling.
  • Updates API guidance, README, taxonomy documentation, and paper metadata.
File summaries
File Description
tests/test_llm_router.py Adds offline routing and model-loading tests.
README.md Updates project overview, usage, and Colab links.
paper.md Corrects license and installation instructions.
mkdocs.yml Configures the documentation site and API reference.
laiser/skill_extractor_refactored.py Corrects public API documentation.
laiser/services.py Updates service documentation and logging.
laiser/llm_models/model_loader.py Adds CPU-safe model loading.
laiser/llm_models/llm_router.py Fixes provider routing and initialization errors.
laiser/llm_models/hugging_face_llm.py Fixes completion decoding and chat templates.
docs/usage.md Documents extraction options and outputs.
docs/taxonomies.md Documents taxonomy sources and aliases.
docs/requirements.txt Defines documentation dependencies.
docs/reference.md Adds generated API reference.
docs/quickstart.md Adds runnable quickstart examples.
docs/providers.md Documents supported model providers.
docs/installation.md Documents installation paths.
docs/index.md Adds the documentation homepage.
docs/cookbook.md Links cookbook notebooks.
.github/workflows/docs.yml Builds and deploys documentation.
Review details

Suppressed comments (3)

docs/providers.md:75

  • These statements do not match the public error contract: the loader only falls back for RepositoryNotFoundError, EntryNotFoundError, and OSError, while LLMRouter wraps CPU load failures in LAiSERError instead of re-raising the original exception type. Please narrow the documented fallback conditions and describe the wrapped error, or change the implementation to preserve the advertised contract.
If a GPU model fails to load, LAiSER falls back to `TheBloke/Mixtral-7B-Instruct-v0.1-AWQ`. On CPU there is
no fallback: a model that fails to load raises the original error.

docs/providers.md:68

  • This fallback requests BitsAndBytesConfig(load_in_8bit=True), which requires the separate bitsandbytes package, but pyproject.toml's gpu extra installs only vllm. When vLLM fails, a user following this page therefore hits a missing dependency instead of the documented transformers fallback. Add bitsandbytes to the GPU extra or document/install it as part of this path.
with 8-bit quantization if vLLM cannot load it. Install the GPU extra first:

docs/providers.md:47

  • The public model_id is optional and defaults to TheBloke/Mixtral-7B-Instruct-v0.1-AWQ, which is the AWQ checkpoint this PR notes cannot run on CPU. This makes the claim that any Hugging Face causal model can run with use_gpu=False misleading for the default constructor; please state that callers must choose a CPU-compatible checkpoint (as the example does), or change the default.
Any Hugging Face causal language model can run on CPU, with no API key:
  • Files reviewed: 19/19 changed files
  • Comments generated: 1
  • Review effort level: Lite

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread laiser/llm_models/model_loader.py Resolved
…er docs

Addresses Copilot's review of #443.

- The gpu extra installed only vllm. When vLLM cannot load a model, LAiSER loads it with transformers using BitsAndBytesConfig(load_in_8bit=True) and device_map="auto", and transformers raises ImportError without bitsandbytes and accelerate, so the documented fallback could never run from a documented install. Both are added, with the minimum versions transformers itself checks for (bitsandbytes 0.43.1, accelerate 0.26.0).

- providers.md said a GPU model that fails to load falls back to Mixtral, and that CPU load failures raise the original error. The loader only retries when the checkpoint is missing or unreadable (RepositoryNotFoundError, EntryNotFoundError, OSError), and since bc965b9 construction raises LAiSERError with the original exception as __cause__. The page now says both.

- providers.md said any Hugging Face causal model runs on CPU. Without a model_id the default is TheBloke/Mixtral-7B-Instruct-v0.1-AWQ, which is GPU-only, so the page now says to always pass a model_id on CPU. The default itself is unchanged to avoid altering behaviour for existing callers.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@phanindra-max
phanindra-max merged commit 976ad73 into main Sep 15, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[JOSS] Issues / suggestions documentation

2 participants