From 93bbb5557eca701a377324c1875dbcce5f3f692d Mon Sep 17 00:00:00 2001 From: phanindra-max Date: Sun, 8 Nov 2026 17:31:10 -0500 Subject: [PATCH 1/2] docs(paper): spell out NLP at first use The Statement of need rewrite used "NLP" before defining it. Define it there and use the abbreviation in State of the field, which keeps the paper within the JOSS word limit. Refs #424, #447 Co-Authored-By: Claude Opus 5.5 --- paper.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/paper.md b/paper.md index e26e602..fcbc971 100644 --- a/paper.md +++ b/paper.md @@ -35,11 +35,11 @@ LAiSER is open-source software that reads such documents and links each skill th # Statement of need -Students and job seekers acquire skills through university courses, online courses and previous jobs, while many employers define in-house taxonomies for skills. Across syllabi and job descriptions, skills are just keywords, and without additional context what a skill means is left to subjective inference. Existing approaches that implement classical NLP methods often miss domain context [@Boselli2018], and works that use transformers or LLMs leave gaps in reproducibility and source grounding [@Li2023SkillGPT; @Herandi2024SkillLLM; @Nguyen2024Rethinking]. Without grounding, researchers and curriculum designers can be misled by irrelevant matches, which could negatively affect the decisions those matches inform. LAiSER is an open-source research tool for skill researchers that acts as a translation layer between the skill language of universities and industry, grounding its results in national taxonomies, with the choice of LLM left to the researcher. Every match carries its taxonomy source, link and similarity score, so users can check it. We share it so that researchers and educators can focus on understanding skills and iterating on their findings instead of building the pipeline themselves. Target users include labor economists studying occupational transitions, curriculum designers auditing program alignment with industry needs, and credentialing organizations producing standards-aligned digital badges. +Students and job seekers acquire skills through university courses, online courses and previous jobs, while many employers define in-house taxonomies for skills. Across syllabi and job descriptions, skills are just keywords, and without additional context what a skill means is left to subjective inference. Existing approaches that implement classical natural language processing (NLP) methods often miss domain context [@Boselli2018], and works that use transformers or LLMs leave gaps in reproducibility and source grounding [@Li2023SkillGPT; @Herandi2024SkillLLM; @Nguyen2024Rethinking]. Without grounding, researchers and curriculum designers can be misled by irrelevant matches, which could negatively affect the decisions those matches inform. LAiSER is an open-source research tool for skill researchers that acts as a translation layer between the skill language of universities and industry, grounding its results in national taxonomies, with the choice of LLM left to the researcher. Every match carries its taxonomy source, link and similarity score, so users can check it. We share it so that researchers and educators can focus on understanding skills and iterating on their findings instead of building the pipeline themselves. Target users include labor economists studying occupational transitions, curriculum designers auditing program alignment with industry needs, and credentialing organizations producing standards-aligned digital badges. # State of the field -Several open-source tools address parts of the skill extraction problem. SkillNER [@SkillNER2022] uses rule-based matching against the EMSI (now Lightcast) skills database via spaCy, an open-source natural language processing library, offering fast lookups but limited ability to detect novel or context-dependent skills not present in its dictionary. Nesta's Skills Extractor Library [@Nesta2023] trains a spaCy named entity recognition (NER) model on labeled job advertisements and maps extracted phrases to ESCO or Lightcast taxonomies using sentence-transformer embeddings. The esco-skill-extractor, also published as ESCOX [@EscoExtractor2024; @Kavargyris2025ESCOX], embeds full input texts with a transformer and matches them against ESCO entries by cosine similarity. +Several open-source tools address parts of the skill extraction problem. SkillNER [@SkillNER2022] uses rule-based matching against the EMSI (now Lightcast) skills database via spaCy, an open-source NLP library, offering fast lookups but limited ability to detect novel or context-dependent skills not present in its dictionary. Nesta's Skills Extractor Library [@Nesta2023] trains a spaCy named entity recognition (NER) model on labeled job advertisements and maps extracted phrases to ESCO or Lightcast taxonomies using sentence-transformer embeddings. The esco-skill-extractor, also published as ESCOX [@EscoExtractor2024; @Kavargyris2025ESCOX], embeds full input texts with a transformer and matches them against ESCO entries by cosine similarity. LLMs have also been applied to skill extraction directly. SkillGPT [@Li2023SkillGPT] extracts skills from job descriptions with an open-source LLM and standardizes them to ESCO through vector similarity search. Skill-LLM [@Herandi2024SkillLLM] fine-tunes an LLM for skill extraction from job descriptions, and @Nguyen2024Rethinking evaluate in-context learning with LLMs on six skill extraction benchmarks, finding that LLMs trail supervised models overall but better handle syntactically complex skill mentions. From f332b29ffd9aba3a6d9ef7ead34776491f393fe0 Mon Sep 17 00:00:00 2001 From: phanindra-max Date: Sun, 8 Nov 2026 17:36:05 -0500 Subject: [PATCH 2/2] docs(paper): complete the AI disclosure and correct two details Commit history shows Claude Sonnet 4.6 and Claude Opus 5.5 also wrote code, so name them in the disclosure. Name the OpenAI Responses API, which is the API the backend uses and the one that accepts no seed. Add `laiser setup` to the install line, which has been required since #462. Refs #447 Co-Authored-By: Claude Opus 5.5 --- paper.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/paper.md b/paper.md index fcbc971..5bff625 100644 --- a/paper.md +++ b/paper.md @@ -53,9 +53,9 @@ We use LLMs because many skill keywords carry a domain context that can only be Matching uses exact search, primarily for reproducibility: as research tooling, LAiSER should not give completely different results from run to run, since that may jeopardize downstream work. Each extracted phrase is embedded with a sentence-transformer model [@Reimers2019] and compared with every entry in a Facebook AI Similarity Search (FAISS) index [@Johnson2019]. At about 27,000 skill entries, an exhaustive search is cheap next to the LLM call, and the same phrase always returns the same entry and score. The closest entry is kept only if its similarity clears a threshold (0.60 for skills by default), so a phrase with no good match, such as a bare "Airflow", is dropped rather than mismatched. -In its default configuration, LAiSER decodes at temperature zero and passes a fixed seed to every backend that accepts one, so that repeated runs with a local model should return the same output. These measures have limits. Hosted models may change under the same name, and providers do not guarantee identical output even at temperature zero; the OpenAI API also accepts no seed. Matching cannot recover a skill the model did not extract, and different models may extract different skills. The prompt asks for at most ten skills per document, a design choice to filter out noise, so long documents are summarized rather than covered exhaustively. The similarity thresholds were tuned against an internal hand-labeled sample that is not yet public. So that a batch keeps running, a document whose model call fails, or whose reply cannot be parsed, is skipped with a warning. All document types, including syllabi, use the same extraction prompt, because different prompts for different documents would give results that are hard to compare, and one prompt is easier to maintain. We therefore recommend that users record the model, temperature, seed and thresholds when reporting results, and spot-check a sample of the output. +In its default configuration, LAiSER decodes at temperature zero and passes a fixed seed to every backend that accepts one, so that repeated runs with a local model should return the same output. These measures have limits. Hosted models may change under the same name, and providers do not guarantee identical output even at temperature zero; the OpenAI Responses API also accepts no seed. Matching cannot recover a skill the model did not extract, and different models may extract different skills. The prompt asks for at most ten skills per document, a design choice to filter out noise, so long documents are summarized rather than covered exhaustively. The similarity thresholds were tuned against an internal hand-labeled sample that is not yet public. So that a batch keeps running, a document whose model call fails, or whose reply cannot be parsed, is skipped with a warning. All document types, including syllabi, use the same extraction prompt, because different prompts for different documents would give results that are hard to compare, and one prompt is easier to maintain. We therefore recommend that users record the model, temperature, seed and thresholds when reporting results, and spot-check a sample of the output. -The source code is available at [https://github.com/LAiSER-Software/extract-module](https://github.com/LAiSER-Software/extract-module) under the BSD 3-Clause License, and can be installed with `pip install laiser`, or `pip install laiser[gpu]` for GPU inference. +The source code is available at [https://github.com/LAiSER-Software/extract-module](https://github.com/LAiSER-Software/extract-module) under the BSD 3-Clause License, and can be installed with `pip install laiser`, or `pip install laiser[gpu]` for GPU inference, followed by `laiser setup`. # Research impact statement @@ -65,7 +65,7 @@ LAiSER received the Bronze Award in the AI in Education (Higher Education Instit # AI usage disclosure -Generative AI was used in developing LAiSER and preparing this paper. GitHub Copilot (completion, coding agent and Autofix) and Claude Code (Claude Opus 5) wrote and refactored code and tests, all checked by the test suite and reviewed by the authors. Gemini 2.5 Pro and 3.1 Pro, Claude 3.5 Sonnet, and Claude Opus 4, 5 and 5.5 drafted portions of the documentation, which project members reviewed and edited. For the paper, Copilot's coding agent shortened a November 2025 draft, Claude 3.5 Sonnet and Claude Opus 4 drafted the March 2026 version, and Claude Code (Claude Opus 5) co-authored the September 2026 revisions. In this revision, the authors answered structured questions for the Statement of need, State of the field and Software design; Claude Code (Claude Opus 5.5) assembled those answers into paragraphs, added technical details from the code, checked claims against the code and sources, and drafted the Software design limitations paragraph, which the authors revised. The authors made all design decisions, reviewed and validated all AI-assisted content, and take full responsibility for the paper. +Generative AI was used in developing LAiSER and preparing this paper. GitHub Copilot (completion, coding agent and Autofix) and Claude Code (Claude Sonnet 4.6, Opus 5 and 5.5) wrote and refactored code and tests, all checked by the test suite and reviewed by the authors. Gemini 2.5 Pro and 3.1 Pro, Claude 3.5 Sonnet, and Claude Opus 4, 5 and 5.5 drafted portions of the documentation, which project members reviewed and edited. For the paper, Copilot's coding agent shortened a November 2025 draft, Claude 3.5 Sonnet and Claude Opus 4 drafted the March 2026 version, and Claude Code (Claude Opus 5) co-authored the September 2026 revisions. In this revision, the authors answered structured questions for the Statement of need, State of the field and Software design; Claude Code (Claude Opus 5.5) assembled those answers into paragraphs, added technical details from the code, checked claims against the code and sources, and drafted the Software design limitations paragraph, which the authors revised. The authors made all design decisions, reviewed and validated all AI-assisted content, and take full responsibility for the paper. # Acknowledgments