From f332b29ffd9aba3a6d9ef7ead34776491f393fe0 Mon Sep 17 00:00:00 2001 From: phanindra-max Date: Sun, 8 Nov 2026 17:36:05 -0500 Subject: [PATCH] docs(paper): complete the AI disclosure and correct two details Commit history shows Claude Sonnet 4.6 and Claude Opus 5.5 also wrote code, so name them in the disclosure. Name the OpenAI Responses API, which is the API the backend uses and the one that accepts no seed. Add `laiser setup` to the install line, which has been required since #462. Refs #447 Co-Authored-By: Claude Opus 5.5 --- paper.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/paper.md b/paper.md index fcbc971..5bff625 100644 --- a/paper.md +++ b/paper.md @@ -53,9 +53,9 @@ We use LLMs because many skill keywords carry a domain context that can only be Matching uses exact search, primarily for reproducibility: as research tooling, LAiSER should not give completely different results from run to run, since that may jeopardize downstream work. Each extracted phrase is embedded with a sentence-transformer model [@Reimers2019] and compared with every entry in a Facebook AI Similarity Search (FAISS) index [@Johnson2019]. At about 27,000 skill entries, an exhaustive search is cheap next to the LLM call, and the same phrase always returns the same entry and score. The closest entry is kept only if its similarity clears a threshold (0.60 for skills by default), so a phrase with no good match, such as a bare "Airflow", is dropped rather than mismatched. -In its default configuration, LAiSER decodes at temperature zero and passes a fixed seed to every backend that accepts one, so that repeated runs with a local model should return the same output. These measures have limits. Hosted models may change under the same name, and providers do not guarantee identical output even at temperature zero; the OpenAI API also accepts no seed. Matching cannot recover a skill the model did not extract, and different models may extract different skills. The prompt asks for at most ten skills per document, a design choice to filter out noise, so long documents are summarized rather than covered exhaustively. The similarity thresholds were tuned against an internal hand-labeled sample that is not yet public. So that a batch keeps running, a document whose model call fails, or whose reply cannot be parsed, is skipped with a warning. All document types, including syllabi, use the same extraction prompt, because different prompts for different documents would give results that are hard to compare, and one prompt is easier to maintain. We therefore recommend that users record the model, temperature, seed and thresholds when reporting results, and spot-check a sample of the output. +In its default configuration, LAiSER decodes at temperature zero and passes a fixed seed to every backend that accepts one, so that repeated runs with a local model should return the same output. These measures have limits. Hosted models may change under the same name, and providers do not guarantee identical output even at temperature zero; the OpenAI Responses API also accepts no seed. Matching cannot recover a skill the model did not extract, and different models may extract different skills. The prompt asks for at most ten skills per document, a design choice to filter out noise, so long documents are summarized rather than covered exhaustively. The similarity thresholds were tuned against an internal hand-labeled sample that is not yet public. So that a batch keeps running, a document whose model call fails, or whose reply cannot be parsed, is skipped with a warning. All document types, including syllabi, use the same extraction prompt, because different prompts for different documents would give results that are hard to compare, and one prompt is easier to maintain. We therefore recommend that users record the model, temperature, seed and thresholds when reporting results, and spot-check a sample of the output. -The source code is available at [https://github.com/LAiSER-Software/extract-module](https://github.com/LAiSER-Software/extract-module) under the BSD 3-Clause License, and can be installed with `pip install laiser`, or `pip install laiser[gpu]` for GPU inference. +The source code is available at [https://github.com/LAiSER-Software/extract-module](https://github.com/LAiSER-Software/extract-module) under the BSD 3-Clause License, and can be installed with `pip install laiser`, or `pip install laiser[gpu]` for GPU inference, followed by `laiser setup`. # Research impact statement @@ -65,7 +65,7 @@ LAiSER received the Bronze Award in the AI in Education (Higher Education Instit # AI usage disclosure -Generative AI was used in developing LAiSER and preparing this paper. GitHub Copilot (completion, coding agent and Autofix) and Claude Code (Claude Opus 5) wrote and refactored code and tests, all checked by the test suite and reviewed by the authors. Gemini 2.5 Pro and 3.1 Pro, Claude 3.5 Sonnet, and Claude Opus 4, 5 and 5.5 drafted portions of the documentation, which project members reviewed and edited. For the paper, Copilot's coding agent shortened a November 2025 draft, Claude 3.5 Sonnet and Claude Opus 4 drafted the March 2026 version, and Claude Code (Claude Opus 5) co-authored the September 2026 revisions. In this revision, the authors answered structured questions for the Statement of need, State of the field and Software design; Claude Code (Claude Opus 5.5) assembled those answers into paragraphs, added technical details from the code, checked claims against the code and sources, and drafted the Software design limitations paragraph, which the authors revised. The authors made all design decisions, reviewed and validated all AI-assisted content, and take full responsibility for the paper. +Generative AI was used in developing LAiSER and preparing this paper. GitHub Copilot (completion, coding agent and Autofix) and Claude Code (Claude Sonnet 4.6, Opus 5 and 5.5) wrote and refactored code and tests, all checked by the test suite and reviewed by the authors. Gemini 2.5 Pro and 3.1 Pro, Claude 3.5 Sonnet, and Claude Opus 4, 5 and 5.5 drafted portions of the documentation, which project members reviewed and edited. For the paper, Copilot's coding agent shortened a November 2025 draft, Claude 3.5 Sonnet and Claude Opus 4 drafted the March 2026 version, and Claude Code (Claude Opus 5) co-authored the September 2026 revisions. In this revision, the authors answered structured questions for the Statement of need, State of the field and Software design; Claude Code (Claude Opus 5.5) assembled those answers into paragraphs, added technical details from the code, checked claims against the code and sources, and drafted the Software design limitations paragraph, which the authors revised. The authors made all design decisions, reviewed and validated all AI-assisted content, and take full responsibility for the paper. # Acknowledgments