Skip to content

docs(paper): rewrite remaining sections from JOSS paper review (#447) - #460

Merged
phanindra-max merged 12 commits into
devfrom
docs/447-paper-rewrite
Nov 7, 2026
Merged

phanindra-max merged 12 commits into
devfrom
docs/447-paper-rewrite

Conversation

@phanindra-max

Copy link
Copy Markdown
Member

Addresses the remaining items from @jansim's paper review in #447. Together with #450, this resolves the issue.

Paper (paper.md, paper.bib)

  • State of the field.

    • esco-skill-extractor and ESCOX turned out to be the same tool, so they are now described once and cited together.
    • The unsupported "lacks preprocessing and multi-model flexibility" clause is removed.
    • A new paragraph covers:
      • what LAiSER adds over the LLM-based tools: LLM extraction plus mapping to four taxonomies;
      • why LAiSER was built rather than extending Nesta's library (the Apache Airflow / HVAC example, checked against the bundled index);
      • that LAiSER has no published evaluation.
  • Software design. Rewritten from the authors' answers into four paragraphs:

    1. What happens to a document.
    2. Why an LLM rather than a trained NER model.
    3. Why matching uses exact search.
    4. What the reproducibility measures do and don't guarantee.

    API detail is kept out, per the JOSS guidelines.

  • Statement of need. Rewritten from the authors' answers on the problem, its cost and the response. Claims about existing work are now cited.

  • AI usage disclosure. Rewritten to the JOSS AI policy. It names each tool and model version, says what each was used for, and describes how this revision was produced.

  • Acknowledgments. Now states that the funders had no role.

  • Bibliography. Autor2013 is removed, since it is no longer cited.

Code

Syllabi were sent to a placeholder prompt (" Work in progress") that contained no document text. Every input type now uses the shared extraction prompt and reads every column in text_columns. Tests are in tests/test_syllabus_prompt.py, and docs/usage.md is updated.

Checks

  • Full test suite: 67 passed, 3 skipped.
  • Every citation key resolves, and every bib entry is cited.
  • Paper body: 1,748 words (JOSS limit 1,750).
  • The six JOSS-required section headings are unchanged.

Not in this PR

  • The vLLM fallback loads DEFAULT_VLLM_MODEL_ID silently when the requested model fails, and that ID may not exist on Hugging Face.
  • tests/test_index_process.py deletes and rebuilds the committed index files on every run.

🤖 Generated with Claude Code

dependabot Bot and others added 12 commits August 24, 2026 12:56
Updates the requirements on [python-dotenv](https://github.com/theskumar/python-dotenv) to permit the latest version.
- [Release notes](https://github.com/theskumar/python-dotenv/releases)
- [Changelog](https://github.com/theskumar/python-dotenv/blob/main/CHANGELOG.md)
- [Commits](theskumar/python-dotenv@v0.19.0...v1.2.3)

---
updated-dependencies:
- dependency-name: python-dotenv
  dependency-version: 1.2.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
…p/python-dotenv-gte-0.19.0-and-lt-2.0.0

Revert "chore(deps-dev): update python-dotenv requirement from <1.0.0,>=0.19.0 to >=0.19.0,<2.0.0"
fix(ci): repair release and PR workflows, run the real test battery
ci: publish release notes per tag, fix changelog template
docs: bring #438 documentation and CPU fixes to main
fix: deterministic decoding by default; paper updates for JOSS review #424
Release to main: load local models on transformers 5 (#448)
Release to main: JOSS paper review fixes (#450)
Release to main: Open Skills Network docs link (#457)
Paper
- Statement of need: rewritten from the authors' answers on the problem,
  its cost and the response; existing-work claims are now cited.
- State of the field: esco-skill-extractor and ESCOX are the same tool and
  are now described once; the unsupported "lacks preprocessing" clause is
  removed; a new paragraph states what LAiSER adds, why it was built rather
  than extending Nesta's library, and that it has no published evaluation.
- Software design: rewritten as four paragraphs (pipeline, LLM vs NER,
  exact search, reproducibility and its limits), checked against the code.
- AI usage disclosure: names each tool and model version, what it was used
  for, and how this revision was produced, per the JOSS AI policy.
- Acknowledgments: states that the funders had no role.
- paper.bib: remove Autor2013, no longer cited. Body is 1,748 words.

Code
- Syllabi used a placeholder prompt (" Work in progress") that contained no
  document text. Every input type now uses the shared extraction prompt and
  reads every column in text_columns. Tests added; docs/usage.md updated.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Copilot AI balanced review requested due to automatic review settings November 7, 2026 09:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown

Code Hygiene

The following items are defined but not yet in active use. Routine cleanup — no impact on functionality.

  • laiser/llm_methods.py:87 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:45 — unused variable 'chat_format' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'exc' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'exc_type' (100% confidence)
  • laiser/llm_models/llama_cpp_handler.py:86 — unused variable 'tb' (100% confidence)
  • laiser/llm_models/llm_router.py:71 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:86 — unused variable 'args' (100% confidence)
  • laiser/llm_models/llm_router.py:89 — unused variable 'args' (100% confidence)
  • laiser/services.py:127 — unused variable 'raw_description' (100% confidence)

@github-actions

github-actions Bot commented Nov 7, 2026

Copy link
Copy Markdown

Complexity Analysis

Average: C — Moderate (13.75)

Hotspots tracked below. These are functional, well-exercised components — listed for visibility as the codebase scales.

Grade Name Type Score
F FAISSIndexManager.initialize_index Method 51
E SkillExtractionService.extract_and_align_core Method 36
E FAISSIndexManager.search_similar_skills Method 32
D ResponseParser._parse_skills_from_response Method 24
D FAISSIndexManager Class 22
C ResponseParser.parse_knowledge_task_response Method 20
C _BaseTaxonomyFAISSIndexManager.search_similar Method 18
C ResponseParser Class 17
C ResponseParser.parse_ksa_extraction_response Method 16
C ResponseParser.parse_skill_extraction_response Method 15
C openai_generate Function 14
C parse_output_vllm Function 13
C AlignmentService.align Method 11
C LLMRouter._initialize_components Method 11
C anthropic_generate Function 11
B SkillExtractionService._deduplicate Method 10
B GeminiAPI.generate_with_config Method 10
B load_model_from_vllm Function 10
B _BaseTaxonomyFAISSIndexManager.initialize_index Method 9
B ResponseParser.parse_ksa_details_response Method 8
B SkillExtractionService Class 8
B _BaseTaxonomyFAISSIndexManager Class 8
B llm_generate Function 8
B SkillExtractorRefactored._expand_allowed_sources Method 7
B get_completion_vllm Function 7
B LLMRouter.generate Method 7
B llama_cpp_chat Function 7
B AlignmentService Class 6
B SkillExtractionService._normalize_mixed_concept_rows Method 6
B fetch_model_output Function 6
B GeminiAPI Class 6
B LlamaCppBackend.__init__ Method 6

@phanindra-max phanindra-max mentioned this pull request Nov 7, 2026
10 tasks done
@phanindra-max phanindra-max self-assigned this Nov 7, 2026
@phanindra-max
phanindra-max merged commit f99bc4f into dev Nov 7, 2026
19 checks passed
phanindra-max added a commit that referenced this pull request Nov 7, 2026
Release to main: JOSS paper rewrite and syllabus prompt fix (#460)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants