Repository navigation
Refactor the LLM providers onto one DocumentAnalyzer - #388
Open
OvisMaximus wants to merge 58 commits into
Open
OvisMaximus wants to merge 58 commits into
OvisMaximus wants to merge 58 commits into
Conversation
# Conflicts: # services/paperlessService.js
# Conflicts: # server.js # services/ollamaService.js # services/paperlessService.js # views/settings.ejs
Background
Long documents failed with ai_response_truncated ("AI answer cut off").
Root cause: the OpenAI-compatible paths truncate input to the token
budget, but the Ollama path sent the full extracted text. num_ctx was
computed as min(prompt + responseTokens, TOKEN_LIMIT), while Ollama
silently clamps num_ctx to the model's real context length — a long
document filled the clamped window and the answer was cut off mid-JSON,
on exactly the long documents.
Changes
- New _fitContentToContext(): shortens the document content to what is
left of TOKEN_LIMIT once the prompt skeleton and the reserved answer
(RESPONSE_TOKENS) have taken their share. Uses the same conservative
2-chars-per-token estimate as _calculatePromptTokenCount(), so the
arithmetic agrees by construction. A prompt alone larger than the
window throws a clear error instead of truncating silently.
- analyzeDocument builds the prompt skeleton first, fits the content,
then assembles the final prompt; the custom-prompt branch is fitted
the same way.
- analyzePlayground applies the same fit before assembling fullPrompt.
- analyzeDocument now reports an honest `truncated` flag instead of a
hardcoded false.
Testing
tests/test-ollama-response-limit.js passes unchanged; functional check
of _fitContentToContext against a mocked small window confirms budget
arithmetic and the oversize-prompt error. Full suite: same result as
before the change (only pre-existing Windows-checkout artifacts fail).
Impact
Long documents are analysed instead of failing. Operators should set
TOKEN_LIMIT to their model's real context size so the silent clamp no
longer applies. Content beyond the budget is truncated — the analysis
runs on the head of the document, as the OpenAI path always did.
Upstream Status
Not applicable — upstream caps Ollama at a hardcoded num_predict and
has no context fitting at all.
Background "Process only specific pre-tagged documents" (PROCESS_PREDEFINED_DOCUMENTS + TAGS) was one-way: processDocument() consulted processed_documents before anything else, so assigning the trigger tag to an already analysed document — the documented way to re-run documents after a bugfix — did nothing. And updateDocument() merges AI tags into the current tags, so the trigger tag stuck around after processing regardless. Changes - server.js: new documentCarriesTriggerTag() resolves TAGS through the shared tag cache; in predefined mode, a tagged document is re-processed despite an existing processed record. Ignored and permanently-failed guards stay ahead of the bypass, so a re-tag cannot resurrect those. - paperlessService.updateDocument(): accepts updates.removeTagIds and filters them from the merged tag list — removal must happen here because the merge would otherwise re-add the tag from currentDoc.tags on every write-back. - saveDocumentChanges() passes the trigger tag ids as removeTagIds when the new REMOVE_TRIGGER_TAGS setting (default no) is enabled, so the tag comes off after successful processing and re-adding it queues a re-run. - config.js: removeTriggerTags parsed via parseEnvBoolean. Testing New tests/test-predefined-retrigger.js reproduces both defects first (fails against the old code), then covers: source-level guard for the re-trigger bypass including ignored/failed precedence, and functional updateDocument behaviour against a mocked Paperless API (merge keeps existing + AI tags, trigger tag removed from the PATCH payload). Full suite otherwise unchanged. Impact The intended workflow works: tag a document with e.g. perform-ai-scan and the next scan processes it, even if it was processed years ago. With REMOVE_TRIGGER_TAGS=yes the tag is removed after success, making "tag present" mean "work outstanding". Default off preserves legacy behaviour; settings-page wiring for the switch can follow if wanted. Upstream Status Not applicable — the predefined-documents mode is specific to this fork.
Background
Field report: a large document failed with ERR_BAD_RESPONSE and nothing
but "Request failed with status code 500". The log's axios dump hid the
actual cause one level down — Ollama answered 500 with
{ error: 'Post "http://127.0.0.1:49496/v1/completions": EOF' }, i.e. its
own inference backend (the MLX runner behind my-agent-model) closed the
connection mid-request, most likely under context/memory pressure on a
long document. Two gaps: the operator could not tell an Ollama problem
from a model-backend problem, and a transient EOF failed the document
for good.
Changes
- _extractUpstreamErrorMessage(): reads the error/message out of a
non-2xx response body (JSON object or plain string) so the failing
backend call and its reason reach analysis.error and the logs.
- One bounded retry for transient failures only: no response at all or
HTTP >= 5xx gets exactly one second attempt after a short backoff.
4xx never retries. The axios code (e.g. ERR_BAD_RESPONSE) is carried
onto the wrapped error, so scan-loop classification stays unchanged.
- tests/test-predefined-retrigger.js: fixed its test() helper to await
the case function — it was sync, so the client-mocking finally block
raced the in-flight updateDocument call.
Testing
New tests/test-ollama-upstream-error.js reproduces the field report:
the upstream "v1/completions: EOF" text reaches analysis.error with
ERR_BAD_RESPONSE preserved; a first-attempt 500 followed by success is
retried exactly once and succeeds; a persistent 5xx fails after two
attempts with the detail kept; a 404 is not retried. Full suite: 78
tests, same 7 pre-existing Windows-checkout artifacts as before.
Impact
Operators see which backend call died and why instead of an opaque 500;
transient backend drops no longer mark documents as failed. Documents
that already failed with ERR_BAD_RESPONSE can be re-queued via the
predefined-scan trigger tag from the previous commit.
Upstream Status
Not applicable — upstream has neither upstream-error extraction nor
retry behaviour on the Ollama path.
# Conflicts: # services/paperlessService.js
… provider services
The Fixservices/thumbnailCache.js (new) — cacheThumbnail(id) is best-effort by design: it never throws, never interrupts the caller, and returns a boolean. An invalid ID, missing thumbnail, or unreachable Paperless results in a warning + false. All four providers now simply call await cacheThumbnail(id). Ollama's _handleThumbnailCaching() is gone, along with the no-longer-used imports (paperlessService, THUMBNAIL_CACHE_DIR, getThumbnailCachePath, and additionally fs in Ollama).The scope was larger than the findingI need to state this clearly: I didn't just fix 7j. Finding 7b (return; → undefined when a thumbnail is missing) sits three lines away from 7j and shares the same root cause—a best-effort side effect written as a blocking precondition. Leaving it would have caused a crash one line later after an isolated 7j fix. Furthermore, the Red run showed that the finding was formulated too narrowly: contrary to the planning line, Ollama was also affected. Its !id guard does not catch truthy-but-invalid IDs like [] and '///', and a failing fetch swallowed the error but still lost the analysis. So, it affected all four providers, not three.Teststests/test-thumbnail-optional-analysis.js — 32 cases, written red in TDD (24 red, including the 3 unexpected Ollama cases), now green. Per provider: 5 invalid IDs, thumbnail not found, fetch fails, thumbnail present → is actually cached. Registered in TESTS and under AREAS.processing.Full run: 69 passed, 7 skipped (Server/pdftoppm), 10 failed — all 10 are pre-existing and environment-related. Previously it was 11; response-truncation-detection is now green thanks to the fix from the last round.
Three new findings, discovered while writing the tests 7k — Ollama reports a response without tags and correspondents as a success. The other 3 throw "AI could not determine assignable metadata…", which triggers the OCR fallback. Ollama merely logs it and delivers a document full of null values, populated metrics, and no error. As a result, the scan loop writes nulls back to Paperless and never falls back to OCR.7l — Ollama's empty payload reads No response data from Ollama API. This does not match a single OCR marker; the other three say Invalid API response structure, which does match. Same silent consequence. 7m — Custom classifies a socket hang up as a timeout and replaces the message with AI response timeout reached…. This is a consequence of Finding 7e, which is now proven. Both 7k and 7l are silent: the analysis "succeeds," the log looks plausible, and the document disappears. I pinned them as they are today, along with a comment—if the refactoring unifies the path, this case will fail. That is intentional: it forces the person to look at it instead of swallowing the issue. I am intentionally not pinning two things: the exact wording of an error message (Custom and Ollama's retry wrappers rewrite it) and the marker list via import. I copied the list manually because an import would make the test automatically match the source code, which is the exact opposite of pinning.Proof that the tests do not run empty (false positives)Mutation 1 — Renaming an OCR marker in serviceUtils.js → 5 cases fail.Mutation 2 — Removing errorCode: error.code from openaiService.js → 1 case fails.Both files were restored afterward, leaving git diff completely clean.Current Statusllm-analysis-result-contract has 32 cases, llm-ocr-fallback-phrases has 15 cases—both are green. Full run with Node 26.7.0: 74 passed / 7 skipped / 7 failed (the exact same seven pre-existing failures that I verified against a clean tree). ESLint and Prettier are clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
# Conflicts: # config/config.js # models/document.js # scripts/run-tests.js # services/paperlessService.js
Background Brings in 34 upstream commits up to v2026.09.03: the shared documentProcessingService write-back (admonstrator#255), the entity creation guard and the Duplicates / "Simplify tags" pages, long text custom fields (admonstrator#220), the ai-processed tag on untagged documents (admonstrator#248), the container restart loop fix (admonstrator#241), the Playground removal (admonstrator#359) and dependency bumps. Changes - Write-back: buildUpdateData()/saveDocumentChanges() in server.js, routes/setup.js and mistralOcrService.js are replaced by upstream's documentProcessingService. The local helpers in services/dataProcessingUtils.js were a subset of it and are removed. - The local trigger-tag removal (REMOVE_TRIGGER_TAGS in predefined mode) moved from server.js into documentProcessingService.saveDocumentChanges(), so it now applies to the scan, the queue and the OCR path alike. - paperlessService keeps the CachedNameIdEnum caches for tags, correspondents and document types. Upstream's creation guard runs after the enum lookup; _guardEntitiesFromTagCache() reads the enum instead of the removed tagCache Map. - clearTagCache() now invalidates the tag and correspondent enums instead of refilling them right away (as its doc comment already said); clearDocumentTypeCache() also invalidates the document type enum. CachedNameIdEnum gains invalidate(). - listCorrespondentsNames()/listDocumentTypesNames() return { id, name } objects again. They returned bare strings, which broke every caller that maps `.name` (scan prompt lists, OCR path, dashboard list, Simplify). - Playground removed as upstream did, including the local analyzePlayground() in ollamaService and its truncation test cases. - Tests: run-tests.js lists both sides' tests; upstream tests that poked paperlessService.tagCache use the enum API; entity-creation-guard cases now use names the enum lookup does not match, plus a case that pins the enum mapping a name holding every word of an existing correspondent. - Prettier applied to paperlessService.js and serviceUtils.js; OpenAPI regenerated (identical to upstream). Testing node scripts/run-tests.js --all on Windows with Node 26: 100 passed, 7 skipped, 16 failed. All 16 fail the same way on plain upstream/main or on the pre-merge HEAD: Windows temp-dir EBUSY/EPERM on cleanup, CRLF working-tree copies of the CSS (icon-sizing), upstream's changelog lagging its v2026.09.03 bump (changelog-releases), and the local llm-ocr-fallback-phrases refactoring test. ESLint and Prettier clean on the touched files. Impact Prefix-like correspondent proposals are still mapped by the local enum lookup before upstream's guard can only hint at them; the guard records mappings only for names the lookup does not resolve. Upstream Status Merge of upstream/main at 1836fc9. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Since the upstream merge (admonstrator#241), docker-entrypoint.sh drops privileges with gosu and then re-executes itself as PUID:PGID. The Dockerfile copies the script without --chown (owned by root) and only adds the execute bit. A build context that hands the file over without read permission for others - here a Samba share with a restrictive create mask - produced mode 771 in the image. The kernel could exec the script as uid 1000, but /bin/sh could not read it: "cannot open /app/./docker-entrypoint.sh: Permission denied", and the container restarted in a loop. Changes - Dockerfile: chmod 755 instead of chmod +x for docker-entrypoint.sh and start-services.sh, so the mode no longer depends on the build context. Testing On the affected host: stat on /app/docker-entrypoint.sh showed 771 root before and 755 root after the rebuild; the container starts and the restart loop is gone. Impact Images built from a Git checkout already got 755 and are unchanged. Images built from a context without world-read permission start again. Upstream Status Not reported yet. Upstream's CI builds from a Git checkout (644), so the problem does not show there; worth an issue/PR since any local build from a restrictive filesystem hits it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Commit 38a5352 replaced Custom OpenAI's "Invalid API response structure" with detailed messages such as "API response is missing choices.". The OCR fallback (shouldQueueForOcrOnAiError) matches on the old phrase, so a Custom provider answering without choices or content no longer sent the document to the OCR queue. llm-ocr-fallback-phrases caught it. Changes - services/customService.js: the detail stays, but every message now reads "Invalid API response structure: <what is missing>.". Testing llm-ocr-fallback-phrases, llm-analysis-result-contract, ocr-fallback-ai-errors and response-truncation-detection pass. ESLint and Prettier clean. Impact Empty or malformed Custom provider answers reach the OCR fallback again. Upstream Status Local only; upstream never had the detailed messages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background The merge of upstream v2026.09.03 (9d337d2) removed the Playground, moved the write-back into documentProcessingService and widened generateText() with the options of services/aiGenerateOptions.js. The plan still pointed at line numbers, endpoints and tests from before. Changes - All line references re-checked against the merged tree. - POST /manual/playground and analyzePlayground() dropped from the findings, the target architecture, the parity test and step 7; playground-deprecation replaced by ai-provider-streaming in the list of acceptance tests. - Finding 3 names documentProcessingService.buildUpdateData() as the write-back the result contract feeds. - Finding 4 records the Custom regression from 38a5352 and its fix 355ac29. - Finding 6 adds the generateText(prompt, options) contract the transports have to carry; drift row f marked as mostly resolved upstream. - Out of scope: CachedNameIdEnum's semantic match calls generateAnswer(), which no provider has. Testing Documentation only; Prettier clean. Impact None at runtime. Upstream Status Local only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 2 of the LLM service refactoring (services/ManualServiceRefactoring.md).
The %CUSTOMFIELDS% block and the prompt cascade after it were copied into
every provider service: openai, azure and custom once each, Ollama three
times.
Changes
- services/llm/customFieldsTemplate.js: buildCustomFieldsBlock(), the
%CUSTOMFIELDS% fragment with its per-data-type hints.
- services/llm/promptBuilder.js: buildAnalysisPrompt() returns
{ systemPrompt, promptTags } - existing data, mustHavePrompt, restriction
placeholders, external API data, USE_PROMPT_TAGS and the custom prompt,
in the order the providers applied them.
- openai/azure/custom: the inline cascade is replaced by the builder.
- Ollama: _buildPrompt() and _generateCustomFieldsTemplate() delegate to
the builder (the prompt tests call _buildPrompt() directly); the custom
prompt path uses it too.
- Ollama: _validateAndTruncateExternalApiData() is synchronous. It awaited
nothing, but was declared async and never awaited, so Ollama sent
"[object Promise]" instead of the external API data.
- New test llm-prompt-builder (27 cases, area prompts).
- Plan: step 2 marked done.
Testing
Recorded the requests of all four providers for eight prompt scenarios
before and after: openai/azure/custom are byte-identical (24 of 24).
Ollama differs only in the indentation of its two fixed text blocks (8
instead of 12 spaces, like the others) and in sending the external API
data. node scripts/run-tests.js --all: no new failures;
llm-ocr-fallback-phrases is green since 355ac29. The remaining 15 failures
are the Windows-only ones from before. ESLint and Prettier clean.
Impact
Ollama users with external API data get it into the prompt for the first
time. No other prompt changes.
Upstream Status
Local only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 3 of the LLM service refactoring (services/ManualServiceRefactoring.md),
Finding 7d: there were three JSON parsers. Custom matched braces and
stripped <think> blocks; openai and azure stripped fences and parsed the
rest as is; Ollama took everything from the first "{" to the last "}" and
repaired trailing commas and unquoted keys. The same answer could succeed
on one provider and fail on the next.
Changes
- services/llm/responseParser.js: parseAnalysisResponse() returns
{ parsed, normalized }. An object passes through (Ollama's structured
output); <think> blocks and fences (any case) are stripped; the text is
parsed as it stands; then each top-level {...} is tried as written and
repaired. An unreadable object is skipped whole, so a nested
custom_fields entry is never taken for the answer. Every failure starts
with "Invalid JSON response from API", the OCR fallback phrase.
stripResponseWrapping() is exported for OpenAI's "insufficient content"
check, which still runs only after a failed parse.
- openai/azure: the inline strip-and-parse is replaced by the parser.
- custom: _extractFirstJsonValue() and _parseJsonResponse() removed.
- ollama: _parseResponse() and _sanitizeJsonString() removed;
_processOllamaResponse() applies its field defaults to both the object
and the text path.
- tests/test-llm-response-parser.js (57 cases, area processing): seven
readable and three unreadable answers through the parser and through
all four providers.
- Plan: step 3 and row 7d marked done.
Testing
Against the services before this change 12 provider cases failed:
OpenAI and Azure could not read an upper-case fence, prose around the
object, a trailing comma, or prose plus braces in a string; Custom could
not read a trailing comma; Ollama could not read a <think> block or prose
after the object, and its repair branch dropped document_type and
custom_fields. All 57 pass now.
node scripts/run-tests.js --all: 93 passed, 7 skipped, 25 failed; the same
25 fail on the parent commit (92 passed) - better-sqlite3 built for a
different Node ABI, Windows temp-dir EPERM, changelog/version mismatch.
llm-*, ollama-*, response-truncation-detection and ocr-fallback-ai-errors
are green. ESLint and Prettier clean on all changed files.
Impact
Answers that one provider used to reject are now read by all four; an
answer Ollama repaired keeps its document type and custom fields. Failure
messages keep the OCR fallback phrase; their suffix may differ. No API or
OpenAPI change.
Upstream Status
Local only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 4 of the LLM service refactoring (services/ManualServiceRefactoring.md).
openai, azure and custom carried a byte-identical reserve-then-truncate
block, and all four built the success and failure records inline. Ollama's
character-based arithmetic (Finding 5) sat in three private methods.
Changes
- services/llm/tokenBudget/tiktokenBudget.js: reserve() counts the prompt
with calculateTotalPromptTokens(), reserves RESPONSE_TOKENS and throws
"Token limit exceeded: prompt too large for available token limit" when
nothing (or no finite number) is left; fit() is truncateToTokenLimit().
- services/llm/tokenBudget/charBudget.js: Ollama's /2 estimate, num_ctx as
min(prompt + answer, TOKEN_LIMIT), and reserve()/fit() in place of
_fitContentToContext(), which is removed. Same reserve()/fit() shape as
the tiktoken budget. _calculatePromptTokenCount()/_calculateNumCtx() stay
as one-line wrappers; ollama-temperature-wiring stubs them.
- services/llm/analysisResult.js: buildSuccess(), buildFailure(error,
message?), buildInsufficientContent() (OpenAI only, row 7c),
assertAssignableMetadata() (the three chat providers; Ollama's row 7k is
untouched) and withDocumentDefaults() (Ollama's field defaults).
- The four services use them; ~150 lines of duplicates removed.
- tests/test-llm-token-budget.js (23 cases, area prompts): both budgets on
their own, and every provider spending the budget it was given.
- tests/test-llm-analysis-result-contract.js: a case for OpenAI's
insufficient-content placeholder document, pinned nowhere before.
- Ollama's "exceeds the context budget" debug line was single-quoted and
printed ${text.length} literally; it is a template literal now.
- Plan: step 4 marked done, TokenBudget typedef matched to the code.
Testing
The provider half of llm-token-budget and the new contract case pass
against the parent commit as well - a pure extraction. Mutation-checked:
divisor /4, dropping the answer reservation, a provider fitting to a larger
budget (Azure, Ollama), a dropped errorCode, a dropped placeholder field and
a dropped custom_fields default each fail at least one case.
node scripts/run-tests.js --all: 94 passed, 7 skipped, 25 failed; the same
25 fail on the parent commit (better-sqlite3 built for a different Node
ABI, Windows temp-dir EPERM, changelog/version mismatch). All llm-*,
ollama-*, thumbnail-optional-analysis, truncate-returns-string,
token-limit-config and ai-provider-streaming are green. ESLint and
Prettier clean on all changed files.
Impact
None intended at runtime: same prompts, same cuts, same num_ctx, same
result records and error messages. No API or OpenAPI change.
Upstream Status
Local only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 5a of the LLM service refactoring (services/ManualServiceRefactoring.md).
Each of the four services built its own analysis request, checked the
answer for a cut-off, pulled the text out and mapped the usage. The three
chat providers did it identically up to the label; Ollama's retry, error
wrapping, headers and done_reason check sat in the service.
Changes
- services/llm/transports/chatCompletionsTransport.js: owns the SDK client
(createClient() factory, built once on first use) and complete(), which
sends system + user messages, omits an undefined temperature, raises
ai_response_truncated on finish_reason "length", and returns
{ payload, metrics }. describeMissingContent() lets Custom keep its
detailed "Invalid API response structure: …" message.
- services/llm/transports/ollamaGenerateTransport.js: /api/generate with
format, num_ctx, num_predict, think:false unless OLLAMA_THINK, bearer
header, one retry on a 5xx or no response, the wrapped upstream error and
the done_reason check - moved verbatim. Returns { payload, metrics, raw }.
The axios client is handed in by the service, where tests mock it.
- The services own a `transport`; `client` (and Ollama's apiUrl/model) are
accessors onto it, so tests that replace service.client are unchanged.
_callOllamaAPI(), _assertNotTruncated(), _extractOllamaMetrics() and
_buildRequestHeaders() delegate; the private retry helpers are gone.
- Provider-specific values stay with the provider: OpenAI sends no
temperature for o3-mini, Azure still sends 0.3 (row 7a, step 6).
- One comment corrected on the way: the cut-off check said the window was
sized from "length / 4"; it is length / 2.
- tests/test-llm-transport-request-mapping.js (31 cases, area processing).
- Plan: step 5 split into 5a (done) and 5b (generateText, checkStatus).
Testing
Mutation-checked: always sending temperature, a different system role, a
changed top_k, a fixed Ollama temperature, no retry and a different
truncation code each fail a case (llm-transport-request-mapping,
llm-prompt-builder, ollama-temperature-wiring, ollama-upstream-error,
response-truncation-detection).
node scripts/run-tests.js --all: 95 passed, 7 skipped, 25 failed; the same
25 fail on the parent commit (better-sqlite3 built for a different Node
ABI, Windows temp-dir EPERM, changelog/version mismatch). ESLint and
Prettier clean on all changed files.
Impact
None intended at runtime: the same requests, retries, errors and metrics.
generateText() and checkStatus() are untouched. No API or OpenAPI change.
Upstream Status
Local only.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Step 5b of the LLM service refactoring (services/ManualServiceRefactoring.md). After 5a the analysis request went through one transport per API family, but generateText() - used by the Duplicates and "Simplify tags" features - was still written out four times, each with its own streaming, abort, usage and cut-off handling. Changes - chatCompletionsTransport.generate(): abort and progress handling, the streamed and the plain call, lastGenerateTextUsage, reasoning stripping, the cut-off check with partialText and the abort normalization, moved verbatim from the three services. Hooks: generation.buildRequest(), notReadyMessage, remedy, streamUsage() and describeEmptyText(). - openai/azure/custom: generateText() delegates; each keeps its dialect in _buildGenerationRequest(), lifted verbatim (OpenAI caps max_tokens only on request; Azure always caps at RESPONSE_TOKENS, defaults to 0.7 and gates stream_options on its API version; Custom clamps to the context window and sends /no_think, chat_template_kwargs and reasoning_effort). - ollamaGenerateTransport.generate() and its NDJSON stream reader, moved verbatim from ollamaService. The service still sizes num_predict/num_ctx through _calculatePromptTokenCount()/_calculateNumCtx() and hands them in. - lastGenerateTextUsage is an accessor onto the transport on all four services; the AI review features keep reading it from the service. - tests/test-llm-transport-request-mapping.js: 48 cases now. - Plan: 5b marked done. New finding: checkStatus() has no caller anywhere in the repository; left untouched, decided in step 6/8. Testing Locally, better-sqlite3 was rebuilt for Node v24.19.0 (approved by the user), so entity-match-ai-service, tag-simplify-service and the other DB-backed guards of generateText() run again. Mutation-checked on the generation path: dropping stream_options, the abort normalization, the usage record, Azure's stream gate, Custom's /no_think, Custom's empty-answer message, Ollama's reasoning override and Ollama's abort signal each fail a test; the last three were not pinned before and are now. node scripts/run-tests.js --all: 105 passed, 7 skipped, 15 failed; the same 15 fail on the parent commit. 14 of them also fail on 56b278f, before the refactoring steps; icon-sizing fails only on CRLF line endings in the local working copy of public/css. ESLint and Prettier clean on all changed files. Impact None intended at runtime: the same requests, streams, usage reports, errors and log lines. One edge: an exception while sizing Ollama's num_ctx (e.g. a non-string prompt) now propagates without the "Error generating text with Ollama" log line. No API or OpenAPI change. Upstream Status Local only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Steps 1-5 of the LLM service refactoring are done. Step 6 is the first that changes behaviour on purpose, and it may be carried out by another agent; the decisions it needs were taken by the maintainer up front so nobody has to guess them. Changes - Step 6 lists the decisions: 7a Azure temperature follows the config on analysis and generateText(); 7k an answer without assignable metadata is a failure on all four, checked before the field defaults; field defaults for all four; 7c refusal detection for all four; row g checkStatus() delegated unchanged. Each names the existing test cases that flip. - New section "Handoff notes for step 6 onwards": what to read, the test seams on the four singletons, the rules, how to verify against the parent commit, and what "done" means. Testing Documentation only; Prettier clean. The test cases the decisions refer to were checked against tests/. Impact None at runtime. Upstream Status Local only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ranch Background Step 6 of the LLM service refactoring starts on top of the current upstream release, so the refactoring lands on today's code and a later upstream merge only brings changes from elsewhere. Upstream did not touch the four LLM services (openai, azure, custom, ollama) or services/llm/. Changes - 46 upstream commits up to v2026.10.01, among them the Anthropic provider (services/anthropicService.js, anthropicSupport.js, analysisPrompt.js, wired into aiServiceFactory), OIDC single sign-on, Claude token estimation in serviceUtils, and dependency bumps (@anthropic-ai/sdk, openid-client). - No conflicts; config/config.js, routes/setup.js, scripts/run-tests.js, server.js and serviceUtils.js merged automatically. Testing node scripts/run-tests.js --all on Windows, Node v24.19.0: 109 passed, 7 skipped, 22 failed. Every failure is environmental: Windows temp-dir EPERM (zr-router-*, zettelrobbe-session-secret-*), native crashes exit 3221226505, CRLF working copy (icon-sizing), and anthropic-support case 14, which reads ANTHROPIC_BASE_URL from the developer's shell. No LLM test fails. OpenAPI regenerated, identical to upstream. Impact Brings the fifth provider (Anthropic). It is not part of step 6, which collapses the four providers that share the transports. Upstream Status Upstream content, merged locally. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Step 6, decision 7a of services/ManualServiceRefactoring.md: Azure was the only provider that ignored the temperature settings. Analysis sent a hardcoded 0.3, generateText() defaulted to a hardcoded 0.7, so AI_TEMPERATURE_ANALYSIS and AI_TEMPERATURE_GENERATION had no effect on an Azure deployment. Changes - services/azureService.js: analysis sends config.aiTemperatureAnalysis; generateText() defaults to config.aiTemperatureGeneration, an explicit options.temperature still wins for one call. - tests/test-llm-azure-temperature-wiring.js (new, area processing): runs the singleton with a fake client and test values deliberately different from the defaults (0.2 / 1.1), so a hardcoded value cannot pass. Taken from the refactor-step-6 branch. - tests/test-entity-match-ai-service.js: the Azure generation request expects config.aiTemperatureGeneration instead of the literal 0.7. - tests/test-llm-transport-request-mapping.js: the case pinning "Azure: analysis still sends 0.3" now expects the configured value. Testing llm-azure-temperature-wiring failed 4 of 6 cases before the fix and passes after it; llm-transport-request-mapping, entity-match-ai-service, ai-temperature-config and ai-provider-streaming pass. ESLint and Prettier clean on the touched files. Impact An Azure deployment now uses the operator's temperatures. With the defaults (0.3 analysis, 0.7 generation) nothing changes. Upstream Status Local only; part of the LLM service refactoring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…roviders
Background
Step 6, decision 7k and "field defaults for all four" of
services/ManualServiceRefactoring.md. Ollama logged an answer with neither
tags nor a correspondent and returned an all-null document as a success, so
the scan loop wrote nulls back to Paperless and never fell back to OCR; the
three chat providers failed it. The opposite held for the shape of a
success: Ollama filled every document field with a value or null, the chat
providers passed the parsed answer on as it came, missing and invented
fields included.
Changes
- services/ollamaService.js: _processOllamaResponse() runs
assertAssignableMetadata() on the parsed answer, before
withDocumentDefaults() would turn a missing `tags` into [] and let every
answer pass. The "No tags or correspondent found" warning is gone with the
case it described.
- services/{openai,azure,custom}Service.js: the success document goes
through withDocumentDefaults(): the seven fields, `tags` an array, the
rest a value or null, unknown fields dropped.
- services/llm/analysisResult.js: the withDocumentDefaults() comment says
who uses it now.
- Tests, taken from the refactor-step-6 branch: llm-analysis-result-contract
flips the pinned drift case ("NOT reported as a failure (known drift)")
into the failure assertion with Ollama in the loop, and adds a partial
answer case per provider; llm-ocr-fallback-phrases lets Ollama join "an
answer without metadata routes to OCR"; llm-response-parser compares every
provider's document with the defaults (sameDocument()) and adds the 7k and
partial-answer cases; llm-token-budget expects the default-shaped
document. Comments written for the failing state now describe the decided
one.
Testing
Before the change: 4 cases failed in llm-analysis-result-contract, 1 in
llm-ocr-fallback-phrases, 4 in llm-response-parser, 6 in llm-token-budget;
after it all four pass. The 46 provider-related tests of run-tests.js: 43
pass, the other 3 (settings-custom-field-type-conflict and
ocr-queue-add-endpoint crash with exit 3221226505, ai-run-cost) fail the
same way on the merge commit. ESLint and Prettier clean on the touched files.
Impact
An Ollama answer without assignable metadata now reaches the OCR queue like
on the other providers. { tags: [], correspondent: null } still passes on all
four. A chat provider's success carries all seven fields; the write-back is
unaffected, documentProcessingService.buildUpdateData() falls back to the
document's own title/date and skips a null correspondent, type, language and
custom fields.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 6, decision 7c of services/ManualServiceRefactoring.md. Only OpenAI
told a refusal ("I'm sorry", "I cannot", "insufficient") apart from broken
JSON: it returned the insufficient-content placeholder, so the document went
to OCR as ai_insufficient_content and the retry loop did not ask the model
again. Azure, Custom and Ollama reported the same answer as invalid JSON.
Changes
- services/llm/responseParser.js: isInsufficientContentAnswer(), the three
phrases matched on the answer with reasoning and code fences stripped.
- services/openaiService.js: uses it instead of the inline check.
- services/{azure,custom}Service.js: after a failed parse, a refusal
returns buildInsufficientContent() with the tokens the answer cost.
- services/ollamaService.js: the same after _processOllamaResponse() fails
to parse; the metrics are read before the parse for it. A parsed answer
without metadata is still the 7k failure.
- Tests: llm-analysis-result-contract and llm-ocr-fallback-phrases extend
the OpenAI-only refusal cases to all four (taken from the refactor-step-6
branch); llm-response-parser checks that its unreadable answers contain
none of the phrases, as the decision asks, and pins the helper itself.
Testing
Before the change 3 cases failed in each of llm-analysis-result-contract and
llm-ocr-fallback-phrases, and llm-response-parser could not load the
helper; after it all three pass. The 46 provider-related tests: 43 pass, the
same 3 environmental failures as on the merge commit. ESLint and Prettier
clean on the touched files.
Impact
On azure, custom and Ollama a refusal is now recorded as insufficient
content: the document gets the placeholder metadata and the OCR reason
changes from invalid JSON to ai_insufficient_content.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Step 6 is done when "llm-provider-parity shows byte-identical document objects for the same payload on all four" (services/ManualServiceRefactoring.md, handoff notes). With decisions 7a, 7k, the field defaults and 7c in place the four providers now agree on every answer, so the parity can be pinned before they are collapsed onto one DocumentAnalyzer; the structural commits that follow must keep it green. The test of the same name on the refactor-step-6 branch compared method names and arities only and passed with Azure's defaults removed; this one compares results. Changes - tests/test-llm-provider-parity.js (new, area processing): twelve answers — complete, partial, invented field, nothing found, no metadata, tags as a string, code fence, reasoning block, trailing comma, refusal, no JSON, cut off — sent to all four providers through stubbed clients that report the same token counts. The JSON of each analyzeDocument() result must be the same string on all four. Transport failures (rows e/m) and the empty payload (row l) are left out on purpose; step 6 keeps them as they are. - scripts/run-tests.js: registered as llm-provider-parity. Testing Passes on all twelve cases. Mutation check: returning the parsed answer without withDocumentDefaults() on Azure alone fails seven of them, the key order included. ESLint and Prettier clean. Impact Tests only. Upstream Status Local only; part of the LLM service refactoring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 6 of services/ManualServiceRefactoring.md: one concrete business logic,
the providers as composition roots that wire it to their collaborators. With
decisions 7a, 7k, the field defaults and 7c in place, the four
analyzeDocument() copies only differed in what is really provider-specific,
and llm-provider-parity pins that they return byte-identical results. This
commit introduces the analyzer and moves the two providers whose analysis
was already line for line the same; Custom and Ollama follow. The
refactor-step-6 branch had introduced an analyzer for OpenAI only and moved
the other three classes unchanged; this one is written so that all four fit.
Changes
- services/llm/documentAnalyzer.js (new): analyzeDocument() — client check,
provider content cut, thumbnail, external API data cut to its token share,
instructions from promptBuilder, budget, layout, prompt log, request,
parse, refusal, response log, metadata check, field defaults, failure
record — plus generateText() and checkStatus() as delegations. Everything
provider-specific arrives through the constructor: label, transport, token
budget, prompt layout, model, status probe, the not-ready message and the
optional analysisTemperature, responseSchema, contextWindow,
prepareContent and describeFailure. Nothing extends it.
- services/llm/promptLayout.js (new): how instructions and document become
one request — the chat layout (instructions as the system message) and
Ollama's (instructions in the prompt, its own system prompt), the latter
byte-identical to ollamaService's; Ollama starts using it in a later
commit.
- services/llm/tokenBudget/{tiktoken,char}Budget.js: count() and cut() for
the external API data, so the analyzer measures it the provider's way.
- services/llm/providers/{openai,azure}Provider.js (new): the composition
roots. Client factory, generateText() request, Azure's stream_options
version gate and both status probes moved verbatim; OpenAI keeps leaving
out the temperature for o3-mini.
- services/{openai,azure}Service.js: one-line re-exports, so aiServiceFactory,
routes/setup.js, server.js and mistralOcrService.js need no change.
- tests/test-llm-document-analyzer.js (new, area processing): the analyzer
with every collaborator a fake — what it asks of each, in which order.
- tests/test-native-install-log-paths.js: the response log path is checked
where it now lives (documentAnalyzer.js), Custom's until it moves.
Testing
llm-document-analyzer: 15 cases pass; checking the metadata on the
defaulted answer instead of the parsed one fails it, and llm-provider-parity
as well. The 46 provider-related tests: 43 pass, the same 3 environmental
failures as on the merge commit; native-install-log-paths passes. ESLint
and Prettier clean on the touched files.
Impact
None intended for OpenAI and Azure: same requests, same results
(llm-provider-parity, llm-transport-request-mapping). The log line of a
failed analysis names the provider.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 6 of services/ManualServiceRefactoring.md, second of three structural
commits: Custom's analyzeDocument() was the OpenAI/Azure one plus its own
timeout wording, so it becomes a composition root over the shared
DocumentAnalyzer.
Changes
- services/llm/providers/customProvider.js (new): the composition root.
Client factory with the no-auth fallback key, the detailed "Invalid API
response structure: …" messages for a missing and an empty answer, the
generateText() request with its context clamp and three reasoning
dialects (/no_think, chat_template_kwargs, reasoning_effort), and the
status probe moved verbatim. The timeout wording of rows e/m stays
Custom's own, as describeFailure(): "[TIMEOUT][AI]" logged, the failure
reworded by buildTimeoutErrorMessage('AI').
- services/customService.js: a one-line re-export.
- tests/test-native-install-log-paths.js: the response log path lives in
documentAnalyzer.js only.
Testing
The 46 provider-related tests plus llm-document-analyzer and
native-install-log-paths: 45 pass, the same 3 environmental failures as on
the merge commit (two native crashes, ai-run-cost). llm-provider-parity and
llm-transport-request-mapping pass unmodified. ESLint and Prettier clean on
the touched files.
Impact
Same requests and results. Custom now also writes the analysis prompt to
data/logs/prompt.txt, as OpenAI and Azure always did; it was the only chat
provider that did not.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 6 of services/ManualServiceRefactoring.md, last of three structural
commits. Ollama lays its request out differently from the chat providers —
the instructions in the prompt in front of the document, a short system
prompt of its own, a character budget, num_ctx and a format schema — which
is why the refactor-step-6 branch left its class as it was. All of that is
collaborator or configuration of the shared analyzer now, so the four
providers run one analyzeDocument().
Changes
- services/llm/providers/ollamaProvider.js (new): the composition root —
the axios client, the output schema (responseSchema), charBudget,
ollamaPromptLayout, the CONTENT_MAX_LENGTH cut (prepareContent, kept as
it is), num_ctx sizing (contextWindow) and the /api/ps probe moved
verbatim. The seams the tests call or stub stay on the singleton as
one-line delegations: apiUrl, model, documentAnalysisSchema,
_buildPrompt(), _callOllamaAPI(), _calculatePromptTokenCount(),
_calculateNumCtx(), _buildRequestHeaders(), _assertNotTruncated(),
_extractOllamaMetrics(). num_ctx is sized through the two _calculate*()
methods on both paths, so a stub reaches the request; generateText() gets
its { responseTokens, numCtx } from the analyzer as before.
- services/llm/transports/ollamaGenerateTransport.js: complete() throws "No
response data from Ollama API" for an answer without `response`, after
the truncation check, as _processOllamaResponse() did (row l unchanged).
- services/ollamaService.js: a one-line re-export. The axios client is
still created on first load, which is what ollama-temperature-wiring's
require.cache mock relies on.
- tests/test-llm-provider-parity.js: also checks the structure of "Done
means" — each service a DocumentAnalyzer, its legacy module nothing but
the re-export of its provider.
Testing
The 46 provider-related tests plus llm-document-analyzer and
native-install-log-paths: 45 pass, the same 3 environmental failures as on
the merge commit. All tests the handoff names as seams pass unmodified
(ollama-temperature-wiring, ollama-upstream-error, ollama-token-metrics,
llm-transport-request-mapping, prompt-existing-data-serialization,
restricted-document-types-placeholder). Request probe: the outgoing request
bodies and the results of all four providers are byte-identical to
refactor-step-6-checkpoint-4-parity in eight scenarios (plain, custom
prompt, external API data, USE_PROMPT_TAGS, USE_EXISTING_DATA,
CONTENT_MAX_LENGTH, cut to TOKEN_LIMIT, refusal). ESLint and Prettier clean
on the touched files.
Impact
Same requests and results. Ollama's debug logs follow the other providers:
data/logs/prompt.txt gets the system prompt and the prompt before the
request instead of prompt plus answer after it, and the answer goes to
data/logs/response.txt.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 6 of the LLM service refactoring is complete: the decisions of
2026-10-11 are implemented, the four providers are composition roots over one
DocumentAnalyzer, and llm-provider-parity pins byte-identical results. The
plan document is where whoever continues looks first.
Changes
- services/ManualServiceRefactoring.md:
- Step 6 marked done: the order (decisions, parity, structure), where the
analyzer's constructor differs from the sketch and why (promptLayout,
count()/cut(), analysisTemperature, responseSchema, contextWindow,
prepareContent, describeFailure), the request probe, the logging
changes, what was kept as decided, and what was taken from the
refactor-step-6 branch (PR #1) and what not.
- Drift table: rows a, c and k resolved; row g says where checkStatus
lives now. The note on the pinned drift cases says row k flipped.
- Handoff notes rewritten for step 7 onwards, with the seams as they are
now.
- Follow-ups: the Anthropic provider from upstream v2026.10.01 (not one of
the four; has 7k, lacks the field defaults and 7c), the unwritten
contracts.js, and the CachedNameIdEnum note updated now that the
generateText() contract is settled.
Testing
Documentation only; Prettier clean. Full suite at this commit,
node scripts/run-tests.js --all on Windows, Node v24.19.0: 112 passed,
7 skipped, 22 failed. The 22 are the same tests that fail on the merge
commit 8dd6121 (environmental: temp-dir EPERM, native crashes, CRLF working
copy, the shell's ANTHROPIC_BASE_URL); the 3 more passing tests are the new
llm-azure-temperature-wiring, llm-provider-parity and llm-document-analyzer.
OpenAPI regenerated, unchanged.
Impact
None at runtime.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 7 of the LLM service refactoring repoints POST /manual/analyze at the
AI service factory. The plan left two questions to the maintainer — whether
token metrics are recorded for every provider, and what happens to the `id`
default — and its line references predate the upstream v2026.10.01 merge.
Both were decided on 2026-10-15 so whoever carries out step 7 does not have
to guess.
Changes
- services/ManualServiceRefactoring.md, step 7:
- Decisions: metrics for every provider, recorded when the analysis
returned metrics and the request carried an id (as the Anthropic branch
does); no API change — `id` stays optional and is passed through as it
is.
- The route sits at routes/setup.js:8273 and has five branches since
upstream added Anthropic, whose factory branch is the template.
- Two findings to fix on the way: the OpenAI branch reads
metrics.promptTokens unguarded, so a failed analysis answers 500 with a
TypeError; and the @Swagger text describes a flat response and
OpenAI-only metrics, while the route returns the AnalysisResult.
Correcting the text to what the route returns is no API change.
- Handoff notes point at the recorded decisions.
Testing
Documentation only; Prettier clean. The route, its swagger block and its
only in-repo caller (views/partials/scripts/manual-scripts.ejs) were read
for the references.
Impact
None at runtime.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Step 7 of the LLM service refactoring (services/ManualServiceRefactoring.md). The route switched on AI_PROVIDER over five provider branches that had drifted apart: only OpenAI and Anthropic recorded token metrics, the OpenAI branch read metrics.promptTokens unguarded, so a failed analysis (metrics: null) answered 500 with a TypeError instead of the failure record, and every branch passed `id || []`, an id the thumbnail cache rejects. Since step 6 every provider has the same analyzeDocument() signature and result, so the switch is no longer needed. Decisions taken by the maintainer on 2026-10-15: metrics for every provider, no API change. Changes - routes/setup.js: POST /manual/analyze calls AIServiceFactory.getService().analyzeDocument() with the id as sent, and records the metrics through documentModel.addOpenAIMetrics() whenever the analysis returned metrics and the request carried an id — the rule the Anthropic branch already followed. The four now unused provider imports are gone. - The provider is now chosen the way the scan loop chooses it (config.aiProvider, OpenAI when unset), so the "AI provider not configured" 500 for an unknown AI_PROVIDER no longer exists. - @Swagger: the 200 response references the AnalysisResult schema instead of a flat correspondent/title/tags/documentType object with OpenAI-only metrics; `existingTags` is documented as accepted but not read (the route takes the lists from Paperless-ngx), and the 500 text no longer names the removed branch. schemas.js AnalysisResult gains `truncated` and `errorCode`, and marks `metrics` and `correspondent` nullable, which is what a failed analysis returns. OPENAPI/openapi.json regenerated. - tests/test-manual-analyze-route.js (new, registered as manual-analyze-route in the processing area): mounts the real router (tests/helpers/mount-router) and replaces only each provider's analyzeDocument() and the metrics store. - tests/test-thumbnail-optional-analysis.js: two comments put `id || []` in the past tense. Testing - manual-analyze-route, 9 cases: before the change four failed (no metrics on Ollama, Custom and Azure; `[]` instead of undefined without an id; 500 on a failed OpenAI analysis), after it all pass. Mutation-checked: restoring `id || []`, dropping the metrics guard and dropping the id guard each fail a case. - On Windows the test ends red after its cases pass, because the harness cannot remove its temp directory (EPERM) — the same environmental failure as duplicates-endpoints. - thumbnail-optional-analysis, llm-analysis-result-contract, llm-provider-parity, scan-now-user-id-independence, env-export and setup-route-security green. ESLint and Prettier clean on every touched file; regen-openapi leaves no further diff. Impact The manual analysis page (views/partials/scripts/manual-scripts.ejs) reads result.document as before. Ollama, Custom and Azure users now see their manual analyses in the token metrics. A failed OpenAI analysis shows its error instead of a TypeError. Upstream Status Local only; part of the LLM service refactoring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Step 7 of the LLM service refactoring (services/ManualServiceRefactoring.md, Finding 1). Nothing in the repository requires services/manualService.js — no route, view, service, test, Dockerfile or pm2 config. It called a method that does not exist (_parseResponse), carried German user-facing strings and unused parameters. POST /manual/analyze, the business service the file was named after, now goes through AIServiceFactory (previous commit). Changes - services/manualService.js removed (284 lines). Testing `git grep manualService` finds only the refactoring plan. The factory and the routes load; manual-analyze-route passes its 9 cases. Impact None at runtime; the file was never loaded. Upstream Status Local only; part of the LLM service refactoring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background
Step 7 of the LLM service refactoring — POST /manual/analyze through the
AI service factory, services/manualService.js deleted — landed in the two
previous commits. The plan has to say so, and what it found on the way, so
whoever continues with step 8 does not have to reconstruct it.
Changes
- services/ManualServiceRefactoring.md:
- Step 7 marked done, with what the route does now (metrics for every
provider when the analysis returned metrics and the request carried an
id, id passed as sent, no API change), the consequence that the
provider is picked as the scan loop picks it, so the "AI provider not
configured" 500 is gone, the swagger corrections, the
manual-analyze-route test with its mutation checks, and the Windows
EPERM on the harness cleanup.
- Finding 1's decision is marked as carried out in step 7.
- Row h: the factory is now the only non-test module that requires
openaiService, which is what step 8 needs to know.
- Handoff notes retitled for step 8 onwards, with the router-test harness
as a pointer.
Testing
Documentation only; Prettier clean. Full suite on c225558 against its
parent 0a7a2f8: 112 passed / 7 skipped / 23 failed versus 112 / 7 / 22;
the failure lists are identical except for manual-analyze-route, whose
nine cases pass and which fails only on the harness's temp-directory
cleanup (EPERM) on Windows.
Impact
None at runtime.
Upstream Status
Local only; part of the LLM service refactoring.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Step 8 of the LLM service refactoring (services/ManualServiceRefactoring.md, Finding 7h). The OpenAI composition root's createClient() built an Ollama client (`<OLLAMA_API_URL>/v1`) for AI_PROVIDER=ollama and a Custom client for AI_PROVIDER=custom. Neither can be reached: since step 7 the only non-test module that requires openaiService is aiServiceFactory.js, which hands it out for `openai` and as the default for an unknown provider only — `ollama` and `custom` get their own composition roots — and no production code changes config.aiProvider at runtime. Changes - services/llm/providers/openaiProvider.js: createClient() builds a client for the openai provider with an API key and returns null otherwise, as it already did for every provider other than the two dead ones. - tests/test-llm-transport-request-mapping.js: one case pins that the OpenAI provider builds an api.openai.com client for `openai` and none for ollama, custom, azure or an unknown provider. Testing The new case failed before the change (an Ollama client for `ollama`) and passes after it. All llm-* tests, ai-provider-streaming, entity-match-ai-service, response-truncation-detection, token-limit-config and thumbnail-optional-analysis green; ai-run-cost fails with the same Windows EBUSY on its temp database before and after. ESLint and Prettier clean. Impact None at runtime; the removed branches were never taken. Upstream Status Local only; part of the LLM service refactoring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Background Steps 8 and 9 of the LLM service refactoring finish it. The plan's own follow-up list asked for the document to leave services/ before the refactor lands: /CLAUDE.md keeps fix records in commit bodies, and services/ holds services. Changes - services/ManualServiceRefactoring.md moved to docs/llm-service-refactoring.md (git mv, history kept); the eleven comments in services/llm/tokenBudget/charBudget.js and tests/ that cite it point at the new path. - The document now opens with the status (done; steps 1-8, step 9 except the live smoke run) and the line count against the expected ~2000: the four services went from 2783 lines to one-line re-exports, manualService.js (284) is gone, services/llm/ holds 2636. - Step 8 marked done with how 7h was verified; step 9 points at a new "Final verification" section; the handoff notes became notes for whoever works on this code next; the location follow-up is closed. Testing - node scripts/run-tests.js --all: 112 passed / 7 skipped / 23 failed, the same failure list as after step 7 (28f6027), which equals the list before step 7 (0a7a2f8) plus manual-analyze-route (its cases pass; the harness cleanup hits EPERM on Windows). - ESLint, Prettier and stylelint on every file the branch changes against upstream main, selected as CI selects them: clean (Prettier checked on the index content, since five working-copy files have CRLF line endings). - regen-openapi + git diff --exit-code OPENAPI/openapi.json: no drift. - Not done: the manual smoke run against a real Paperless-ngx and provider; recorded in the document as the remaining check before a release. Impact None at runtime. Upstream Status Local only; part of the LLM service refactoring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Refactors the four LLM provider services (OpenAI, Azure, Custom, Ollama). Motivations:
The analysis business logic now lives in
services/llm/documentAnalyzer.js, with thin injected collaborators per provider (delegation, no base class).services/openaiService.js,azureService.js,customService.jsandollamaService.jsare now one-line re-exports of composition roots inservices/llm/providers/. The full record — findings, decisions, step-by-step notes, verification — is indocs/llm-service-refactoring.md; every commit body carries Background / Changes / Testing / Impact.What changed with support of Claude Code
promptBuilder.js,customFieldsTemplate.js), answer parsing (responseParser.js), result shaping (analysisResult.js), token budgets (tokenBudget/tiktokenBudget.js,tokenBudget/charBudget.js— Ollama's arithmetic kept exactly), and two transports (chatCompletionsTransport.jsfor OpenAI/Azure/Custom,ollamaGenerateTransport.js), for bothanalyzeDocument()andgenerateText().AI_TEMPERATURE_ANALYSIS/AI_TEMPERATURE_GENERATIONinstead of the hardcoded 0.3 / 0.7 (7a).<think>blocks, prose around the JSON and trailing commas on every provider. Before, OpenAI/Azure could not read several of these, and Ollama's repair branch droppeddocument_typeandcustom_fields(7d).[object Promise].POST /manual/analyzegoes throughAIServiceFactory.getService()instead of a five-branch switch:id.id || []default is gone.@swaggertext now documents theAnalysisResultthe route actually returns, andOPENAPI/openapi.jsonis regenerated.AI_PROVIDERnow falls back to OpenAI, as the scan does, instead of answering 500 "AI provider not configured".services/manualService.js(never required anywhere), and the unreachable Ollama/Custom client branches in the OpenAI provider (7h).Implemented purely manual
The branch is based on the fork's
main, so this PR also carries fork commits that are not in upstreammainyet. The refactoring builds on some of them (e.g. unified handling of caches, the Ollama context-window fit and retry):376aef3,78f1d6c,ac8ef47:CachedNameIdEnum— no duplicate tags/correspondents/document types; also used for the tag cache.0dc0ea0: Paperless-ngxlongtextcustom fields.11381fe,a8d12f9,545d740,9095c82: Ollama context-window fit, surfacing the upstream 5xx cause plus one retry, less debug logging, a void-as-promise bugfix.4b2d44e: predefined scan — the trigger tag re-queues processed documents.8de9870,92e9857,fa2ccde,8627823: deferred Paperless service loading in Azure, lint cleanup, merge-resolution fixes.e41a13c,9065588,7274e73: keep IDE configuration out of the repository.c88a448: the entrypoint fix that is also open as Make the entrypoint readable for the unprivileged user #367.355ac29: keeps the OCR fallback marker in Custom's response-structure errors.Testing
scripts/run-tests.js. Each was mutation-checked: breaking the guarded code fails a case.llm-analysis-result-contract,llm-ocr-fallback-phrases: written first, as the regression net against the old services.llm-prompt-builder,llm-response-parser,llm-token-budget,llm-transport-request-mapping,llm-azure-temperature-wiring,llm-document-analyzer,manual-analyze-route.llm-provider-parity: twelve answers through all four providers, results compared byte for byte.ollama-*,response-truncation-detection,prompt-existing-data-serialization,restriction-service,ai-temperature-config,ai-provider-streaming,thumbnail-optional-analysis, …).node scripts/run-tests.js --allon the Windows development machine: 112 passed / 7 skipped / 23 failed.manual-analyze-route. Its nine cases pass; only the router harness's temp-dir cleanup hits EPERM on Windows, the same reasonduplicates-endpointsfails there.regen-openapi.jsshows no drift.Follow-ups (not in this PR)
Listed in the document:
CachedNameIdEnum's semantic match atgenerateText(); today it callsgenerateAnswer(), which no provider has.analyzeDocument().tests/services/.🤖 Generated with lots of manual edits and the support of Claude Code