Repository navigation
docs: bring #438 documentation and CPU fixes to main - #443
Conversation
Constructing SkillExtractorRefactored with a Hugging Face model id and
use_gpu=False could not produce any output. Reproduced with
Qwen/Qwen2.5-0.5B-Instruct and no provider key set:
WARNING: Transformer model initialization failed: CUDA is required but
not available for bitsandbytes.
AttributeError: 'NoneType' object has no attribute 'generate'
Two independent bugs:
- load_model_from_transformer always requested 8-bit bitsandbytes
quantization, which requires CUDA, so the model never loaded on CPU.
Quantization now applies only when a CUDA device is available. On CPU a
load failure raises the original error rather than falling back to a 7B
AWQ checkpoint that cannot run on CPU either.
- LLMRouter.generate sent every non-hosted model to
llm_generate_vllm(prompt, self.llm). A transformers model lives in
self.model / self.tokenizer and self.llm is None; the existing
llm_generate helper was never called. Transformers-loaded models now
route there.
llm_generate also needed fixing to produce parseable output. It decoded the
prompt plus the completion, so the prompt's own example JSON could be matched
by the response parser; it now returns only the new tokens. It applies the
model's chat template when one exists, and uses MAX_NEW_TOKENS (1000) instead
of a hard-coded 100 that truncated JSON mid-list.
After the fix, on a laptop CPU with no key: load 7.9s, extraction 19s,
returning "Python programming" -> "Program in Python" (ukos, 0.70) and
"Docker" -> "Docker" (onet, 0.75).
Also:
- model_id="openai" fell through router init and was looked up as a Hugging
Face model id; hosted providers now return before any local loading.
- A startup debug line read router.llama_llm, which is never set, so it
always printed False.
tests/test_llm_router.py adds offline coverage with no weights downloaded:
routing to transformers vs vLLM, completion-only decoding, chat template use,
quantization only on CUDA, and raising instead of falling back on CPU.
Part of #438 (example that runs without an API key).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The API reference is now generated from these docstrings, and two of them
described behaviour the code does not have.
similarity_thresholds was documented as defaulting to
{"skill": 0.20, "knowledge": 0.45, "task": 0.55}, and similarity_threshold as
defaulting to 0.20. When neither is passed, extract_and_align_core starts from
DEFAULT_SIMILARITY_THRESHOLDS in config.py, which is
{"skill": 0.60, "knowledge": 0.50, "task": 0.50}; a scalar
similarity_threshold replaces all three, and similarity_thresholds overrides
the types it names. The docstrings in SkillExtractorRefactored and the
service now say so. Behaviour is unchanged.
timing was documented as printing per-stage timings. It is accepted and
passed through but never read, so it is now documented as having no effect.
The service docstring's list of output columns also omitted Source Url.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The documentation site was generated from a separate docs repository, which drifted from the code and has been deleted. Documentation now lives in docs/ here and is reviewed with the changes it describes. - MkDocs with Material, as before, plus mkdocstrings so the API reference is generated from docstrings rather than maintained by hand. - Pages: home, installation, quickstart, usage, model providers, taxonomies, the existing examples, cookbook notebooks, and API reference. Every option of extract_concepts, every output column, and each provider's requirements and environment variables are documented against the current code. - Internal notes already in docs/ (refactoring guide, vLLM batching, v0.5 architecture) stay in the repo but are excluded from the site. - docs.yml runs `mkdocs build --strict` on PRs touching docs/, mkdocs.yml or laiser/, so broken links, missing anchors and unresolvable API references fail review. Merges to main deploy to GitHub Pages at laiser-software.github.io/extract-module/. - docs/requirements.txt pins mkdocs<2: MkDocs 2.0 removes the plugin system mkdocstrings and Material rely on. Part of #438. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…olab paths Responds to the README points in #438. - About: rewritten to describe the actual input and output (a table of documents in, one row per taxonomy match out) and to name the taxonomies (ESCO, O*NET, UK Skills Classification, Open Skills Network) instead of general claims about AI and insight. - Architecture: each stage's description sat on an indented line under its numbered item, which GitHub renders run into the title. Stages are now "**Stage** - description" on one line. Common runtime options had the same defect and is now a table. - Usage leads with an example that runs on CPU with no API key. - New "Try it in Google Colab" section linking the three cookbook notebooks, which use the current API. - Links to the documentation site, PyPI and the cookbook. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The summary stated the code is under the MIT License; LICENSE and pyproject.toml are BSD-3-Clause. It also offered `pip install laiser[cpu]`, but the package defines no cpu extra. Now `pip install laiser`, or `laiser[gpu]` for GPU inference. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every point in Copilot's review was checked against the code; all held. Code - LLMRouter no longer swallows a local model load failure. The CPU branch caught the loader's exception and returned a router with no model; every generate() call then failed and extract_and_align_core skipped the row silently. Construction now raises LAiSERError, on CPU and on GPU when both vLLM and the transformers fallback fail. A vLLM failure alone still falls back to transformers. - LLAMA_CPP_THREADS was read from the environment as a string and passed to llama.cpp's n_threads unchanged; it is now converted to int. Docstrings (they generate the API reference) - top_k caps matches per concept type per document, not per document: each type is aligned in its own call, so all three types can return 3x top_k. - similarity_thresholds overrides similarity_threshold for the types it names. - levels and batch_size are accepted and passed through but never read; both are now documented as having no effect, like timing. Docs - Syllabus input: extract_and_align_core reads only columns listed in text_columns, so description and learning_outcomes must both be listed. The page said they were read regardless. - top_k corrected in README, index and usage. - Each extracted phrase is matched to its single closest entry (the FAISS search uses top_k=1), so the quickstart's "one phrase can match several entries" was wrong. - Usage now explains how per-type columns (Raw Skill, Taxonomy Knowledge, ...) are merged into Raw Concept / Taxonomy Concept, replaces the `extract_concepts(..., output_csv_path=...)` placeholder with a real call, and documents extract, levels and batch_size. - providers.md dropped LAISER_GEMINI_TIMEOUT: GeminiAPI stores the value but never passes it to the client or the request. Tests: four more in test_llm_router.py, through LLMRouter's constructor as SkillExtractionService builds it: CPU load failure raises, GPU raises when vLLM and transformers both fail, GPU falls back to transformers when only vLLM fails, and hosted providers load no local weights. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
docs: move documentation into this repo; fix keyless CPU extraction
|
Code Hygiene The following items are defined but not yet in active use. Routine cleanup — no impact on functionality.
|
|
Complexity Analysis Average: C — Moderate (14.310344827586206) Hotspots tracked below. These are functional, well-exercised components — listed for visibility as the codebase scales.
|
There was a problem hiding this comment.
🟡 Changes recommended
The GPU fallback lacks required quantization dependencies, and provider fallback/error behavior is documented inaccurately.
Get a fresh assessment by requesting another Copilot review.
Pull request overview
Promotes the reviewed documentation site and CPU inference fixes from dev to main.
Changes:
- Adds MkDocs documentation and GitHub Pages deployment.
- Fixes CPU model loading, routing, decoding, and error handling.
- Updates API guidance, README, taxonomy documentation, and paper metadata.
File summaries
| File | Description |
|---|---|
tests/test_llm_router.py |
Adds offline routing and model-loading tests. |
README.md |
Updates project overview, usage, and Colab links. |
paper.md |
Corrects license and installation instructions. |
mkdocs.yml |
Configures the documentation site and API reference. |
laiser/skill_extractor_refactored.py |
Corrects public API documentation. |
laiser/services.py |
Updates service documentation and logging. |
laiser/llm_models/model_loader.py |
Adds CPU-safe model loading. |
laiser/llm_models/llm_router.py |
Fixes provider routing and initialization errors. |
laiser/llm_models/hugging_face_llm.py |
Fixes completion decoding and chat templates. |
docs/usage.md |
Documents extraction options and outputs. |
docs/taxonomies.md |
Documents taxonomy sources and aliases. |
docs/requirements.txt |
Defines documentation dependencies. |
docs/reference.md |
Adds generated API reference. |
docs/quickstart.md |
Adds runnable quickstart examples. |
docs/providers.md |
Documents supported model providers. |
docs/installation.md |
Documents installation paths. |
docs/index.md |
Adds the documentation homepage. |
docs/cookbook.md |
Links cookbook notebooks. |
.github/workflows/docs.yml |
Builds and deploys documentation. |
Review details
Suppressed comments (3)
docs/providers.md:75
- These statements do not match the public error contract: the loader only falls back for
RepositoryNotFoundError,EntryNotFoundError, andOSError, whileLLMRouterwraps CPU load failures inLAiSERErrorinstead of re-raising the original exception type. Please narrow the documented fallback conditions and describe the wrapped error, or change the implementation to preserve the advertised contract.
If a GPU model fails to load, LAiSER falls back to `TheBloke/Mixtral-7B-Instruct-v0.1-AWQ`. On CPU there is
no fallback: a model that fails to load raises the original error.
docs/providers.md:68
- This fallback requests
BitsAndBytesConfig(load_in_8bit=True), which requires the separatebitsandbytespackage, butpyproject.toml'sgpuextra installs onlyvllm. When vLLM fails, a user following this page therefore hits a missing dependency instead of the documented transformers fallback. Addbitsandbytesto the GPU extra or document/install it as part of this path.
with 8-bit quantization if vLLM cannot load it. Install the GPU extra first:
docs/providers.md:47
- The public
model_idis optional and defaults toTheBloke/Mixtral-7B-Instruct-v0.1-AWQ, which is the AWQ checkpoint this PR notes cannot run on CPU. This makes the claim that any Hugging Face causal model can run withuse_gpu=Falsemisleading for the default constructor; please state that callers must choose a CPU-compatible checkpoint (as the example does), or change the default.
Any Hugging Face causal language model can run on CPU, with no API key:
- Files reviewed: 19/19 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…er docs Addresses Copilot's review of #443. - The gpu extra installed only vllm. When vLLM cannot load a model, LAiSER loads it with transformers using BitsAndBytesConfig(load_in_8bit=True) and device_map="auto", and transformers raises ImportError without bitsandbytes and accelerate, so the documented fallback could never run from a documented install. Both are added, with the minimum versions transformers itself checks for (bitsandbytes 0.43.1, accelerate 0.26.0). - providers.md said a GPU model that fails to load falls back to Mixtral, and that CPU load failures raise the original error. The loader only retries when the checkpoint is missing or unreadable (RepositoryNotFoundError, EntryNotFoundError, OSError), and since bc965b9 construction raises LAiSERError with the original exception as __cause__. The page now says both. - providers.md said any Hugging Face causal model runs on CPU. Without a model_id the default is TheBloke/Mixtral-7B-Instruct-v0.1-AWQ, which is GPU-only, so the page now says to always pass a model_id on CPU. The default itself is unchanged to avoid altering behaviour for existing callers. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Brings the #438 documentation work from
devtomain. It was reviewed and merged intodevin #442. This PR promotes it, plus one fix made in response to this PR's own review (25c1314, below).Once merged, the Docs workflow builds the site from
mainand deploys it to https://laiser-software.github.io/extract-module/. Pages is already enabled with GitHub Actions as its source.What reaches
main54ee128fix(llm): run Hugging Face models on CPU without an API key. The CPU path never worked: 8-bit quantization was forced without CUDA, and generation was routed to vLLM with no engine. Verified end to end on CPU with no key.0330ad3docs(api): correctextract_conceptsthreshold defaults and timing. The docstrings documented thresholds of 0.20/0.45/0.55; the code uses 0.60/0.50/0.50.d948e6adocs: build the documentation site from this repository. MkDocs Material, API reference generated from docstrings,mkdocs build --stricton PRs, deploy onmain.c314289docs(readme): say what LAiSER does, fix formatting, add keyless and Colab paths.1df8b1fdocs(paper): correct the license and install command. The paper claimed MIT and a nonexistentlaiser[cpu]extra.bc965b9fix: address review findings on docs: move documentation into this repo; fix keyless CPU extraction #442. A failed local model load now raises instead of silently skipping rows. Also:LLAMA_CPP_THREADSis cast to int,top_kis documented as per concept type, the syllabus column requirement is explained, andlevels/batch_sizeare documented as no-ops.25c1314fix(gpu): install 8-bit dependencies withlaiser[gpu]; correct provider docs. Added in response to review of this PR. Thegpuextra now includesbitsandbytesandaccelerate: when vLLM cannot load a model, the transformers fallback loads it in 8-bit and needs both, so that fallback could not run from the documented install. The providers page now describes accurately when the fallback retries, which error a failed load raises, and that CPU users must pass amodel_id, because the default checkpoint only runs on a GPU.Full detail, verification and the item-by-item mapping to #438 are in #442.
After merge
🤖 Generated with Claude Code