Compendiq Admin Guide

This guide covers installation, configuration, maintenance, and troubleshooting for Compendiq administrators.

System Requirements

Component	Minimum	Recommended
Docker	24.0+	Latest stable
Docker Compose	v2.20+	Latest stable
RAM	4 GB	8 GB+
Disk	10 GB	20 GB+ (depends on attachment cache size)
CPU	2 cores	4 cores

Additionally, the backend needs at least one OpenAI-compatible /v1 endpoint — local Ollama / LM Studio / vLLM, or a hosted API such as OpenAI or DeepSeek. Configure it under Settings → AI Models (see LLM providers); Ollama typically runs on the Docker host machine.

Installation

Docker Compose (Recommended)

Clone the repository:

git clone https://github.com/Compendiq/compendiq-ce.git
cd compendiq-ce

Create your environment file:
```
cp .env.example .env
```

Configure required secrets in .env:

# REQUIRED: Generate random 32+ character strings
JWT_SECRET=<random-32-char-string>
PAT_ENCRYPTION_KEY=<random-32-char-string>

# REQUIRED: strong infrastructure passwords — compose has no defaults
# and refuses to start when these are missing
POSTGRES_PASSWORD=<strong-password>
REDIS_PASSWORD=<strong-password>

Generate secure random strings:

# JWT_SECRET / PAT_ENCRYPTION_KEY
openssl rand -base64 48

# POSTGRES_PASSWORD / REDIS_PASSWORD — must be URL-safe because they are
# interpolated raw into POSTGRES_URL/REDIS_URL ('/', '+', '=' break them)
openssl rand -hex 24

Note: Docker Compose reads interpolation variables from docker/.env (the compose file's directory), not the repo-root .env. Copy your file there (cp .env docker/.env) or pass --env-file .env explicitly.

Start all services:
```
docker compose -f docker/docker-compose.yml up -d
```
This starts six containers:
- frontend -- nginx serving the React app (port 8081)
- backend -- Node.js Fastify server (port 3051, internal)
- postgres -- PostgreSQL 17 with pgvector
- redis -- Redis 8 Alpine with password auth and noeviction (required by BullMQ — eviction would silently drop queued jobs)
- mcp-docs -- documentation-search MCP server (port 3100, internal)
- searxng -- metasearch engine used by mcp-docs (port 8080, internal)
SearXNG receives mcp-docs requests directly on the Docker network. Its trusted-proxy list therefore defaults to loopback only; direct callers do not need X-Forwarded-For. If you deliberately route SearXNG through Traefik, set SEARXNG_TRUSTED_PROXIES to a comma-separated list containing only Traefik's source IPs or dedicated proxy-network CIDRs, for example 10.42.0.5/32,2001:db8:42::/64. Traefik emits X-Forwarded-For and X-Real-IP automatically, but SearXNG honours them only when the connection comes from one of those trusted addresses. Do not use 0.0.0.0/0, ::/0, or a blanket application Docker subnet: an untrusted caller could then forge its client IP. Invalid entries fail container startup instead of being ignored.

Tracker cleanup still works when all three upstream ClearURLs endpoints are unreachable: the derived image loads its pinned, licensed baseline of common tracking parameters. The remote lists remain primary when available; the fallback does not disable TLS verification or replace successful data.
Provide an LLM endpoint. Either pull local Ollama models on the host:
```
ollama pull bge-m3              # Common local embedding model
ollama pull qwen3.5            # Or any chat model
```
Or, after first login, add a hosted OpenAI / DeepSeek (or other /v1) provider under Settings → AI → AI Models → LLM providers — see LLM providers. Keep a real embeddings endpoint for the Embedding use case; a chat-only host is not enough for retrieval.
Access Compendiq at http://localhost:8081. Register the first user -- they automatically receive the admin role.

Verifying the Installation

Check that all containers are healthy:

docker compose -f docker/docker-compose.yml ps

All services should show (healthy) status. You can also hit the health endpoint:

curl http://localhost:8081/api/health

Configuration Reference

Required Secrets

Variable	Description
`JWT_SECRET`	JWT signing secret. Must be 32+ characters. Server refuses to start in production with default value.
`PAT_ENCRYPTION_KEY`	AES-256-GCM key for encrypting Confluence PATs. Must be 32+ characters. Server refuses to start in production with default value.

Database

Variable	Default	Description
`POSTGRES_URL`	`postgresql://kb_user:<password>@localhost:5432/kb_creator`	PostgreSQL connection string
`POSTGRES_TEST_URL`	`postgresql://kb_user:changeme-postgres@localhost:5433/kb_creator_test`	Test database connection (port 5433)
`POSTGRES_USER`	`kb_user`	PostgreSQL username (Docker Compose only)
`POSTGRES_PASSWORD`	--	Required PostgreSQL password — Docker Compose refuses to start without it (Docker Compose only)
`POSTGRES_DB`	`kb_creator`	PostgreSQL database name (Docker Compose only)

Redis

Variable	Default	Description
`REDIS_URL`	`redis://:<password>@localhost:6379`	Redis connection string
`REDIS_PASSWORD`	--	Required Redis AUTH password — Docker Compose refuses to start without it (Docker Compose only)

Server

Variable	Default	Description
`NODE_ENV`	`development`	Set to `production` for production deployments
`BACKEND_PORT`	`3051`	Backend server port
`FRONTEND_URL`	`http://localhost:5273`	CORS origin and the origin the post-login OIDC redirect is built from. Behind a reverse proxy with SSO, set this to your public origin (see OIDC / SSO).
`FRONTEND_PORT`	`5273`	Host port mapped to the frontend container (Docker Compose only)
`LOG_LEVEL`	`info`	Pino log level: `fatal`, `error`, `warn`, `info`, `debug`, `trace`
`ACCESS_TOKEN_EXPIRY`	`1h`	JWT access token lifetime (jose duration format: `30m`, `1h`, `2h`). Maximum `24h` — a longer value is clamped to `24h` at startup and a warning is logged, since the token lifetime is the worst-case window a deactivated or demoted account could retain API access. An invalid format (e.g. `banana`) still fails startup. Deactivation and role changes normally take effect within seconds (≤ 30s) via the per-user security check.

Immutable page baseline storage (foundation)

Issue #275 installs the inactive Community Edition foundation for immutable article baselines. It has no published admin or end-user UI, and baseline creation remains disabled until #276 completes collaboration, sync, purge, cascade, and cross-process writer enforcement. In a foundation-only deployment, GET /api/admin/page-baselines/activation reports creationEnabled: false, deploymentReady: false, and blocker protected_writer_enforcement_not_registered. Do not bypass that gate or describe the current release as covering every writer.

Retained media lives under ATTACHMENTS_DIR/page-baselines/ and is excluded from ordinary attachment sweeps and page-delete cleanup. Size limits are positive safe-integer environment values:

Variable	Default	Description
`PAGE_BASELINE_MAX_ATTACHMENTS`	`512`	Maximum referenced media objects in one baseline
`PAGE_BASELINE_MAX_BYTES`	`1073741824` (1 GiB)	Maximum retained media bytes in one baseline
`PAGE_BASELINE_MAX_RETAINED_BYTES`	`53687091200` (50 GiB)	Logical cap across published evidence and active reservations
`PAGE_BASELINE_MIN_FREE_BYTES`	`67108864` (64 MiB)	Filesystem free-space reserve left after an admitted copy

Capacity exhaustion refuses new preparation and never evicts published evidence. Lowering a limit does not prune existing baselines. For the exact manifest, media coverage, rollout gate, authority model, retention/backup rules, and conservative writer recovery procedure, see docs/runbooks/immutable-page-baselines.md.

Client inference (on-device editor model)

Optional WebGPU SLM for ghost text and Improve, plus Hunspell EN/DE spell lint. Admins download a transformers.js q4 checkpoint from Hugging Face on Settings → AI Models → Client inference (the server fetches Hub; the browser never does), or copy/upload onto /app/data/client-models — see docs/runbooks/client-inference.md. Dual opt-in, default off. The enable flag is blocked until ONNX is installed. Pre-download is per-browser on Settings → Editor.

LLM providers (primary)

Providers live in the llm_providers table and are configured in the admin UI (Settings → AI → AI Models → LLM providers), not by flipping an env toggle. Every row is a generic OpenAI-compatible /v1 endpoint (ADR-021): hosted OpenAI, DeepSeek, Groq, Mistral, OpenRouter, Together, Fireworks, Azure OpenAI, local Ollama / LM Studio / vLLM, or any other /v1 shim. There is no separate OpenAI SDK path and no DeepSeek-specific protocol.

Add a provider

Open Settings → AI → AI Models → LLM providers and click + Add.
Pick a preset (OpenAI, DeepSeek, …) to fill the base URL, bearer auth, and a suggested default model — or leave Custom blank for a local server (http://host.docker.internal:1234/v1 for LM Studio / vLLM in Docker). Typed values are not overwritten silently when you change preset.
Paste the API key into the password field (bearer). Keys are AES-256-GCM at rest; the UI only ever shows hasApiKey / keyPreview.
Set or edit the default model id, then Save.
On the provider row, press Test. That calls the existing admin health / listModels path (POST /admin/llm-providers/:id/test) and toasts connected + sample model count, or a sanitized failure (401 = bad key, unreachable / open breaker = cannot reach the host). Never paste a live key into docs or screenshots.

Recipe — hosted OpenAI

Field	Value
Preset	OpenAI
Base URL	`https://api.openai.com/v1`
Auth	Bearer + your OpenAI API key
Suggested model	`gpt-4.1-mini` (editable; type any id the account can call)

Save, press Test on the row, then under use-case assignments pin Chat (and optionally Summary / Quality / Auto-tag) to this provider. OpenAI also serves embeddings — assign Embedding only when you intend to index with an OpenAI embedding model, then press Start re-embed on that row (assigning alone does not rebuild the index; keep a local embedder if you are not cutting over).

Recipe — hosted DeepSeek

Field	Value
Preset	DeepSeek
Base URL	`https://api.deepseek.com/v1`
Auth	Bearer + your DeepSeek API key
Suggested model	`deepseek-chat` (editable; `deepseek-reasoner` works on the same `/v1`)

Save, Test, assign Chat (and other chat-shaped use cases as needed).

Hosted DeepSeek is a strict thinking host: Think never sends think or chat_template_kwargs to api.deepseek.com (those fields 400 there). Reasoning models that stream reasoning_content still return the visible answer on content; with Think off the reasoning channel is dropped.

Chat-only hosts must not cover embedding / rerank / image analysis

A hosted chat provider must not be assigned to:

Embedding — DeepSeek's hosted chat endpoint is not an embeddings index for Compendiq; pointing the use case at it fills the vector column with failures or garbage and stalls retrieval.
Rerank — never inherits (ADR-021). Unassigned means the rerank stage is off. It needs a Cohere/Jina-style /v1/rerank endpoint the chat host does not serve.
Image analysis — never inherits, and the assignment is the egress control (ADR-027 D3): unassigned means no image byte leaves the host. It needs a model that accepts an image part on /chat/completions, which a text-only chat host does not.

Chat / Summary / Quality / Auto-tag may inherit the default provider. Keep a separate local (or dedicated hosted) embedder for Embedding, and leave Rerank / Image analysis disabled unless you have a real endpoint for each.

Runtime knobs that stay as env vars

These are still read every request (or on a short cache) and are not provider rows:

Variable	Default	Description
`LLM_VERIFY_SSL`	`true`	Set to `false` to disable TLS verification for LLM connections (prefer `NODE_EXTRA_CA_CERTS` in production)
`LLM_STREAM_TIMEOUT_MS`	`300000`	Streaming request timeout in ms (5 min). Increase for very large articles.
`LLM_CACHE_TTL`	`3600`	Redis TTL in seconds for LLM response cache (1 hour)

Deprecated LLM env vars (bootstrap-only)

Bootstrap-only. The tables below are not the live configuration path. After migration 054 / ADR-021, providers and per-use-case models are rows in llm_providers / llm_usecase_assignments, edited under Settings → AI Models. Most of these vars seed a first row only on a fresh install when llm_providers is empty. Setting them on a migrated instance logs a deprecation notice and does not change runtime routing — with one named exception for OLLAMA_BASE_URL below. Do not revive LLM_PROVIDER=ollama|openai — that two-slot toggle is gone.

Variable	Seed / bootstrap role
`OLLAMA_BASE_URL`	Seeds an Ollama provider row on an empty table (`…` → `/v1` shim). On an existing install it also rewrites the default sentinel `http://localhost:11434/v1` when the env value differs — that sentinel rewrite is the only post-seed write.
`OPENAI_BASE_URL` / `OPENAI_API_KEY`	Seeds an OpenAI provider row (`https://api.openai.com/v1` + encrypted key) when the table is empty
`LLM_BEARER_TOKEN`	Deprecated notice only — not read as an Ollama auth seed (the Ollama bootstrap row is always `auth_type='none'`). Put a bearer key on the provider row in Settings.
`DEFAULT_LLM_MODEL`	Suggested `default_model` on the seeded row / worker fallbacks when no assignment exists
`QUALITY_MODEL` / `SUMMARY_MODEL`	Historical worker model seeds; prefer use-case assignments
`EMBEDDING_MODEL`	Fully inert since migration 054 — setting it only logs a notice. Assign Embedding in Settings.
`LLM_PROVIDER`	Removed — was the legacy `ollama` / `openai` toggle. Ignored if present.

Embedding

Variable	Default	Description
`EMBEDDING_MODEL`	`bge-m3`	Bootstrap-only / inert — see the deprecated table above. Assign the Embedding use case under Settings → AI Models.
`RAG_EF_SEARCH`	`100`	Deprecated (#1285) — the HNSW `ef_search` floor is now `admin_settings.rag_ef_search`, set in `Settings → AI Models → Retrieval`. Nothing seeds that row, so this variable is still read on an instance that has never saved the panel; from the first save on it is ignored on every successful read. A settings read that FAILS keeps whatever the server last resolved — or the row a save has just written, which is also what a read that succeeded on a pre-save snapshot keeps — rather than dropping the floor to 100 for a minute (#1512); both of those memories are per-process, so a process that has read neither — a fresh pod, a restart, a pod of the deployment that did not serve the write — and whose first settings read fails will read this variable again, and keep reading it (re-held each 60 s TTL) until one of its settings reads succeeds, and the panel can then re-offer its Keep button — whose press writes THIS value as the saved one, over anything the panel had saved before. Unset it once the panel is saved. Setting it logs a startup notice.

Background Workers

Variable	Default	Description
`QUALITY_CHECK_INTERVAL_MINUTES`	`60`	How often the quality analysis worker runs
`QUALITY_MODEL`	`DEFAULT_LLM_MODEL` then `qwen3:4b`	Bootstrap-only — prefer the Quality use-case assignment under Settings → AI Models
`SUMMARY_CHECK_INTERVAL_MINUTES`	`60`	How often the summary worker scans for pages
`SUMMARY_MODEL`	`DEFAULT_LLM_MODEL` then (disabled)	Bootstrap-only — prefer the Summary use-case assignment. Empty = disabled when no assignment exists.
`SYNC_INTERVAL_MIN`	`15`	Background sync scheduler polling interval (minutes)

Pages per batch for the quality and summary workers is not an environment variable: set it per worker under Settings → AI Models → Workers (default 5, range 1–100; admin_settings.quality_batch_size / summary_batch_size). Each scheduled run and each Run Now processes at most that many pages; the rest wait for the next run. The former QUALITY_BATCH_SIZE / SUMMARY_BATCH_SIZE variables are ignored.

Save applies to the next batch; edits made while saving remain unsaved until you save again. If settings cannot be read, the fields stay blank rather than claiming the default. Retry reloads settings without leaving the page. A failed refresh keeps the last loaded settings and any unsaved edits visible with a warning.

Run Now runs one batch, not the entire backlog. Quality and Summary each select up to their Pages per batch setting (five by default) and process them sequentially. More eligible pages wait for the next scheduled cycle (60 minutes by default) or another Run Now. Short-content skips can make a batch finish quickly.

Both scheduled and manual entrypoints share one lock per worker, renewed while inference runs; overlapping triggers do not process the same batch again. A failed article is counted as an error, not as processed. BullMQ records a batch containing errors as failed, including partial progress; successful processing counts may include deliberate skips, not just generated results. Per-page retries remain capped at three.

If a batch immediately fails, check the backend log and the Quality/Summary assignments under Settings → AI Models. Use the exact model identifier served by the provider's /v1/models endpoint, and confirm that a completion request succeeds from the backend's network. A responding server or a listed model alone does not prove inference works. In particular, LM Studio can answer HTTP 400 “No models loaded” for a stale model identifier. Correct the assignment/load the model before triggering another batch; retry-exhausted pages require a rescan, which also resets previously completed pages.

Background Job Queue (BullMQ)

Compendiq uses BullMQ (Redis-backed) for reliable background job processing. Six worker types run as BullMQ queues:

Queue	Purpose	Default Interval
`sync`	Confluence space synchronization	15 min
`quality`	Page quality analysis (LLM)	60 min
`summary`	Auto-summary generation (LLM)	60 min
`maintenance`	Expired token cleanup	24 hours
`maintenance`	Data retention cleanup	24 hours
`reembed-all`	Global re-embed of every non-folder page (issue #257)	On-demand

Configuration:

USE_BULLMQ=true (default) -- set to false to fall back to legacy setInterval workers
Redis maxmemory-policy must be noeviction (default in the provided Docker config)
Job history is stored in the job_history PostgreSQL table for observability; BullMQ queue metrics are not exposed via the API
REEMBED_WAIT_LOCKS_MS=600000 (default 10 min): how long the reembed-all worker waits for in-flight per-user embedding runs to finish before aborting the job. Visible in the admin UI via GET /api/admin/embedding/locks.
reembed_history_retention (admin setting, default 150): how many completed/failed reembed-all job records BullMQ keeps in Redis before sweeping the oldest. Range [10, 10000]. Takes effect on the next run.

LLM Request Queue

LLM requests are managed through a concurrency-controlled queue with backpressure:

Setting	Default	Description
`LLM_CONCURRENCY`	`4`	Max concurrent LLM requests
`LLM_MAX_QUEUE_DEPTH`	`50`	Max queued requests before rejecting
`LLM_STREAM_TIMEOUT_MS`	`300000`	Per-request timeout (5 min)

When the queue is full, new requests receive a 503 error with a message to retry later.

Per-User Concurrent SSE-Stream Cap (#268)

Streaming LLM endpoints (/api/llm/ask, generate, improve, summarize, analyze-quality, generate-diagram) intentionally bypass the queue above because streams are long-lived. Without a per-user bound, one user opening many tabs can saturate the upstream LLM.

A Redis counter (llm:streams:<userId>) tracks currently-open SSE streams per user. New opens beyond the cap are rejected with HTTP 429 and body:

{ "error": "too_many_concurrent_streams", "message": "…" }

Setting	Default	Range	Description
Settings → AI → AI Models → LLM providers → Runtime limits (primary)	`3`	`1`–`20`	Admin-configurable. Changes apply within 60 s in every worker; the local worker sees them immediately.
`LLM_MAX_CONCURRENT_STREAMS_PER_USER`	`3`	`1`–`20`	Deprecated bootstrap fallback — consulted only when the `admin_settings` row is absent.

Release semantics. The counter decrements on all four exit paths: normal completion, error, upstream timeout, and client disconnect. A 1-hour TTL self-heals leaked counters from crashed processes.

Lowering is graceful. Existing in-flight streams run to completion; only new opens see the lower cap.

No admin bypass. Admins share the same cap as regular users. This is intentional to prevent accidental concurrent-usage incidents from power users.

Interaction with LLM_STREAM_RATE_LIMIT. The rate limit caps requests per minute; this caps concurrently-open connections. Both can 429 — they are distinguished by the response body's error string.

Confluence

Variable	Default	Description
`CONFLUENCE_VERIFY_SSL`	`true`	Set to `false` to disable TLS verification for Confluence connections
`ATTACHMENTS_DIR`	`data/attachments`	Attachment cache directory. Set to an absolute path in production.

Notion import (#1459)

Users import selected Notion pages and databases from Library or New Page. This is not Confluence Sync and is not configured in Settings-only.

Auth: each user pastes an internal integration token. It is encrypted at rest with the same AES-256-GCM helpers as the Confluence PAT (user_settings.notion_integration_token). GET /api/notion/connection returns only { hasToken }. The token never appears on GET bodies, logs, or toasts.
Tree: GET /api/notion/tree uses Search and metadata lookups to recover accessible native, relation and block-host ancestor chains, without listing every page body. UUID spelling variants merge; equal titles do not. Cycles cannot hide the entire group. Wikis remain selectable even when Notion marks them inline. The picker groups descendants and batches selected IDs into requests of up to 200.
Database classification: isWiki is free from the schema — Notion adds a verification property to a wiki's row pages and nowhere else — and a wiki always recommends pages without sampling anything. Otherwise the tree samples at most NOTION_ROW_SAMPLE_SIZE (5) rows per database, with NOTION_ROW_SAMPLE_BUDGET (60) samples across the whole tree build, each probe reading one page of NOTION_ROW_PROBE_BLOCKS (8) blocks. A row counts as empty when it returns no blocks, or only blank text blocks (an empty paragraph, heading, callout, quote, list item, to-do, toggle or code block) and decorations (divider, table_of_contents, breadcrumb) — Notion leaves one behind on a row nobody opened, and a row template leaves a whole run of them. Visible text in any block, media, a child page, a block that still has children (a blank toggle wrapping prose is content, not an empty row), or a has_more page is body content; the probe reads eight blocks precisely so that a three-block template does not come back as has_more. Any sampled row with content → rowContent: 'some' and recommendedMode: 'pages'; otherwise any readable empty row → 'none' and 'table'; every sample failing → 'unknown' and 'pages'. A wiki, a zero-row database, or an exhausted budget lands on 'unknown' / 'pages' while spending zero requests. The sample is advisory only — the import re-checks every row before it flattens anything.
A database id is not a page id: GET /v1/pages/{id} answers a database id with a 400 validation_error ("doesn't support retrieving databases as pages"), not a 404. The client flags exactly that case as NotionError.objectTypeMismatch, and isNotionObjectMissing() treats it like 404/403, so the tree's missing-parent lookup and the import's classifier both retry it as getDatabase — this is what lets a wiki selected by its own id import as articles. An unflagged 400 (malformed id, bad cursor) is still a failure and still warn-logged.
Unsupported vs inline: data_source objects cannot be resolved by the pinned 2022-06-28 API and stay in Notion. A non-wiki inline database reads Imports inside its parent article: property-only rows become a table, while rows with content become articles. A wiki remains an article container, not a flattened table, even when inline.
Board layout: a database whose layout is Board (layout: board, format.board_*, or a board view from GET /v1/views on Notion-Version 2025-09-03) is not importable as a table or Kanban. The tree marks it reasonCode board_layout with NOTION_BOARD_REASON. An inline Board also marks its host page board_host. Row pages stay selectable; importing them creates a parent article titled as the containing Notion page and never writes a <table> for the board. Wiki databases are not classified as boards.
Run: POST /api/notion/import returns 202 and runs in the background; GET /api/notion/import/status is the result (idle / importing / complete / error). The POST used to wait for the whole paced Notion walk inside one HTTP request, which outruns the bundled nginx proxy_read_timeout 300 on a Knowledge Base and answered Gateway Time-out (HTTP 504) while the importer kept running. Closing or refreshing the tab does not cancel it. A second POST while the per-user lock is held is 409. The lock uses SET NX and a 10-minute safety TTL, renewed while the importer runs; a crashed job becomes idle once that TTL elapses (importing is not a 24h mutex). The importer plans pages and databases together, preserving the selected root's body and allocating parents before children. Actual embedded child pages and databases bring their owned descendants; ordinary links do not. An embedded database with no mode on the request and no body of its own folds into its host — property-only rows become the host's table with no extra database or row articles, article-bearing rows become the host's children — and row-only later batches recover that decision instead of creating duplicates. An explicit pages mode is honoured instead of folded: that database keeps its own article with its rows beneath it, hosted or not, and its rows are never flattened. Wiki databases and databases with bodies keep their own article.
Repairing an older import: reselect its root and enable Update existing pages with latest Notion content to replace imported bodies and rebuild their Child pages/table representation. This overwrites local edits; without it, existing bodies stay intact while resolvable parent relationships can be repaired. Previously created database-row articles are not automatically deleted. Review those separately rather than deleting by title: two equal titles can represent different Notion pages.
Out of scope: live two-way sync, OAuth, export ZIP, writing to Confluence.

Confluence API Rate Limiting

A token bucket rate limiter protects your Confluence Data Center instance from being overwhelmed during sync.

Default: 60 requests/minute
Configurable: via admin_settings table (key: confluence_rate_limit_rpm)
Behavior: When the rate limit is hit, requests are queued (not dropped). They resume automatically as tokens refill.
Applied to: All Confluence API calls including page fetches and attachment downloads

Security and Encryption

Variable	Default	Description
`PAT_ENCRYPTION_KEYS`	(none)	JSON array of versioned keys for rotation (see Encryption Key Rotation below)
`PAT_ENCRYPTION_KEY_V1`...`V10`	(none)	Numbered env vars for key rotation (alternative to JSON format)
`NODE_EXTRA_CA_CERTS`	(none)	PEM CA bundle file path for self-signed certificates

Email Notifications (SMTP)

Compendiq can send email notifications for key events. Configure SMTP in Settings → System → Integrations → Email / SMTP.

Configuration (env vars or admin UI):

Variable	Default	Description
`SMTP_HOST`	(empty)	SMTP server hostname
`SMTP_PORT`	`587`	SMTP port (587 for STARTTLS, 465 for TLS)
`SMTP_SECURE`	`false`	Use TLS/SSL
`SMTP_USER`	(empty)	SMTP username
`SMTP_PASS`	(empty)	SMTP password
`SMTP_FROM`	`noreply@compendiq.local`	Sender address
`SMTP_ENABLED`	`false`	Enable email notifications

Email types:

Sync completed / failed
Article comment notification
License expiry warning (Enterprise)

Settings configured via the admin UI are persisted in the admin_settings database table and take precedence over environment variables. Use the Send Test button to verify SMTP connectivity.

Enterprise

Variable	Default	Description
`COMPENDIQ_LICENSE_KEY`	(none)	License key for enterprise features (OIDC/SSO, advanced RBAC)

OIDC / SSO

OIDC provider settings (issuer, client ID/secret, group mappings) are configured entirely via the Admin UI (Settings → Governance → Access Control → SSO / OIDC) and stored in the database.

FRONTEND_URL must be your public origin behind a reverse proxy. OIDC login uses a two-hop callback:

The IdP redirects the browser back to the Redirect URI you configure in Settings → Governance → Access Control → SSO / OIDC — e.g. https://compendiq.example.com/api/auth/oidc/callback. This hits the backend.
The backend exchanges the code and then redirects the browser to the SPA callback route (/auth/oidc/callback?login_code=…). It builds that redirect from the backend's FRONTEND_URL env var — not from the Redirect URI.

These are two independent URLs. If FRONTEND_URL is left at its http://localhost:8081 / http://localhost:5273 default while the admin reaches the app through a public host, a correctly-configured Redirect URI still bounces the browser to localhost after the IdP round-trip, dead-ending login. Set:

FRONTEND_URL=https://compendiq.example.com

(matching the origin of your Redirect URI) and restart the backend. FRONTEND_URL also accepts a comma-separated list, which doubles as the CORS allowlist — so if you already configured multiple origins, append the public origin (…,https://compendiq.example.com) rather than overwriting the existing list. The Settings → Governance → Access Control → SSO / OIDC page warns inline when the Redirect URI origin diverges from the origin you loaded the app from, and surfaces the exact FRONTEND_URL value to set.

IP Allowlist (Enterprise, v0.4+)

The IP allowlist is configured entirely via the Admin UI (Settings → Governance → Access Control → IP allowlist) and persisted in the admin_settings table. It is an Enterprise-only feature — Community Edition builds ignore the configuration and register no routes. When enabled, a Fastify onRequest hook short-circuits requests from client IPs outside the configured CIDR ranges with a 403 ip_blocked response, before any auth / business logic runs.

Config fields (admin UI)

Field	Description
Enable toggle	Off by default. When off, all traffic is allowed; the trusted-proxy list still applies (see below).
Allowed CIDRs	IPv4 and IPv6 CIDRs whose client IPs may reach the API. Example: `10.0.0.0/8`, `2001:db8::/32`.
Trusted proxies	CIDRs of reverse-proxy hosts whose `X-Forwarded-For` header will be honoured when resolving the client IP. See the gotcha below.
Exempt paths	Read-only list: `/api/health`, `/api/admin/license`, `/api/auth/`. Always accessible so the health check, license-key rotation, and login flow cannot be locked out.

Lockout recovery

If an admin saves an allowlist that excludes their own IP, the UI's "save button disabled until your IP tests as allowed" guard normally catches it. If you do lock yourself out — for example because the upstream proxy changed its source range after save — recover via one of:

Use an exempt path. /api/admin/license and /api/auth/login bypass the allowlist. Authenticate from the exempt path, then use PUT /api/admin/ip-allowlist directly with curl / httpie to restore or widen the CIDR list.
Connect from an exempt network. The defaults trust loopback (127.0.0.1/32, ::1/128). Shell into a pod and curl http://localhost:3000/api/admin/ip-allowlist to edit locally.
Last resort — edit Postgres directly. UPDATE admin_settings SET setting_value = '…' WHERE setting_key = 'ip_allowlist'; — set enabled to false to disable the feature entirely, then reconfigure from the UI. Changes propagate cluster-wide via the cache-bus within ~1 second; no restart required.

Trusted proxies gotcha

If Compendiq sits behind a reverse proxy (nginx, Traefik, cloud load balancer) that injects X-Forwarded-For, you must add that proxy's source CIDR to the trusted-proxies list. Otherwise Compendiq sees every request as coming from the proxy itself — so either every request is blocked (proxy not in allowlist) or every request is allowed (proxy happens to be in allowlist, regardless of the real client IP).

The Test panel in the admin UI surfaces the resolved client IP for an arbitrary input; use it to verify the topology matches your expectations before enabling the allowlist.

IPv6 gotcha

Node's dual-stack sockets surface IPv4 peers as IPv4-mapped-IPv6 addresses (::ffff:1.2.3.4). The allowlist handles this automatically — an IPv4 CIDR will match an IPv4-mapped-IPv6 peer — so you do not need to add the ::ffff:…/96 mapped range to your IPv4 CIDRs. Conversely, a pure-IPv6 client that is not IPv4-mapped requires explicit IPv6 CIDR coverage.

Trust-proxy behaviour change (v0.4)

Before v0.4, Fastify was started with trustProxy: true (trust every peer's X-Forwarded-For). From v0.4, the server reads trusted-proxies from admin_settings at boot; when the row is absent or the feature is off, the default trusted list is loopback only (127.0.0.1/32, ::1/128). Deployments behind a non-loopback reverse proxy must populate trusted_proxies explicitly or req.ip will report the proxy's IP rather than the real client's. See the v0.4 CHANGELOG entry for the full behaviour-change note.

RAG Per-page ACL Enforcement (Enterprise, v0.4+)

RAG retrieval in CE already post-filters candidate chunks by the user's readable space set (ADR-022). Enterprise deployments additionally enforce per-page view restrictions: if a Confluence page has explicit read restrictions, users outside the allowed list won't see the page's chunks in any RAG response (chat, ask, search-backed AI flows). Feature flag: rag_permission_enforcement.

How it works

The Confluence sync calls /rest/api/content/{pageId}/restriction for each page (and walks ancestors — Confluence view restrictions are inherited) and writes the effective read list into access_control_entries as source='confluence' rows with synced_at=<run start>.
A stale-ACE sweep at the end of the sync removes any source='confluence' row that wasn't refreshed this run — so restrictions removed in Confluence, or users de-listed from a restriction, propagate on the next sync.
The RAG hybrid retrieval (hybridSearch) fetches max(rag_fetch_width, topK, ceil(topK × 1.5)) candidates per leg (see Retrieval fetch width below). After the RRF merge, a second post-filter calls userCanAccessPage(userId, pageId) per candidate — in order-preserving batches, stopping once topK candidates have passed — and drops blocked pages. Rank order is preserved.
source='local' (admin-created) ACEs are never touched by sync. Manual overrides survive.

Sync cadence and leak-window behaviour

Restriction changes in Confluence take effect on Compendiq's next successful sync of the affected page — same cadence as content changes (SYNC_INTERVAL_MIN, default 15 min). Within that window, RAG may return stale visibility. This is the trade-off for synchronous sync mode in v0.4; an async-backfill mode (content first, ACLs after) is a future v0.5 option that shrinks the content-sync duration at the cost of a briefly larger leak window.

Sync duration impact

Enabling the feature adds at least one Confluence API call per synced page (restriction fetch), plus one per unique ancestor (cached across sibling pages within a sync run). At Confluence's 60 RPM default, a 1000-page space with moderate nesting typically adds 15-25 minutes to the first sync run. Subsequent runs are much faster because the ancestor cache hits and most pages' restrictions haven't changed. For deployments with large restricted spaces, expect the first post-enable sync to take notably longer than historical runs.

Unmapped Confluence users

If a Confluence page's read restriction names a user who has never logged into Compendiq (no matching users row — most often because the OIDC login flow hasn't provisioned them yet), the sync cannot persist an ACE for that user. The event is logged as an ACE_SYNC_SKIPPED_UNMAPPED_USER audit entry with { userKey, pageId }. Safe default: implicit deny — the missing user will not match any ACE, so they are denied access. Counter-intuitively this means first-time RAG access for the restricted user happens after they've logged in and the next sync has run. Admin-UI visibility of the skip counter is planned for v0.5.

Disabling the feature

Tier-downgrade (license change to a non-enterprise tier) disables the flag. Existing source='confluence' ACEs remain in the DB. userCanAccessPage() continues to honour them — safe over-restriction. To fully "undo" the sync, manually DELETE FROM access_control_entries WHERE source='confluence'; after disabling.

Verification

Integration regression guard: backend/src/domains/confluence/services/sync-service.integration.test.ts (sync path) and backend/src/domains/llm/services/rag-service.integration.test.ts (query path).
ADR-023 documents the full design and rationale.

Retrieval settings panel (`Settings → AI Models → Retrieval`)

Every retrieval setting below is edited from one admin panel (#1118, extended by #1114). They were DB-only until it shipped, and raw SQL still works — the panel writes the same admin_settings rows.

Setting	Key	Default	Range
Keyword index language	`fts_language`	`simple`	see allow-list below
Fetch width	`rag_fetch_width`	10	10–200
Index search depth (#1285)	`rag_ef_search`	100	1–1000
Rerank candidate pool	`rag_rerank_candidates`	30	10–100
Confidence gate, similarity basis	`rag_confidence_threshold`	0 (off)	0 – <1
Confidence gate, rerank basis	`rag_confidence_threshold_rerank`	0 (off)	0 – <1
Assembly budget	`rag_context_chars_per_page`	6000	0–24000
Exact-identifier pin	`rag_pin_identifiers`	on	on / off
MMR diversity narrow	`rag_mmr_enabled`	off	on / off
MMR λ	`rag_mmr_lambda`	0.7	0–1
Ranking prior weight	`rag_ranking_prior_weight`	0 (off)	0–0.05
Images per page (#1115 P2)	`rag_images_per_page_max`	20	1–200
Index external images (#1115 P2)	`rag_image_index_external`	on	on / off
Images shown to the model (#1115 P4)	`rag_answer_max_images`	2	0–8

The three image knobs are two parts of one feature. rag_images_per_page_max and rag_image_index_external bound what the intake worker takes off a page and hands to image analysis. rag_answer_max_images is the ANSWER side: how many of the matched pictures are attached to the chat request. There is no query-side image switch any more — #1618 stage 2 retired ADR-025's separate image leg, and a picture's derived description is retrieved by the ordinary semantic and keyword legs. Unassigning the image_analysis use case (Settings → AI Models → LLM providers) stops all new analysis. The whole feature — assigning the model, watching the queue drain, what a text-only chat model does — is "Image retrieval" below; operations are docs/runbooks/image-analysis.md.

Two things about the answer knob specifically. 0 is a legal value here — unlike rag_images_per_page_max, whose floor is 1 because a zero intake cap would reconcile the derived chunks away — and it is the honest off switch: analysis still runs, the derived text still ranks pages, and the pictures still appear as sources with their thumbnails. And a chat model that has not probed vision-capable never receives images whatever this says; the answer is then text-only with nothing on it saying so, which is why the panel states that beside the control. Fix a wrong verdict with Re-check on the chat row in Settings → AI Models, not with this number.

Index search depth (rag_ef_search) is a FLOOR, and one worth leaving alone. It is pgvector's hnsw.ef_search: how many candidates the HNSW graph walk visits before returning. Every probe runs at this value or at twice the rows it asks for, whichever is larger, capped at pgvector's own limit of 1000, and all three kNN probes in the product (retrieval's vector leg, page relationships and duplicate detection) read the same number. That second half is why it sits beside Fetch width as context, not as a companion knob: each probe already raises its own depth to twice the rows it fetches, so this floor only ever binds the probes that are narrower than half of it, and widening the fetch makes this setting matter less, not more — do not raise it because you raised the width, which costs scan time (below) and buys nothing. Raising it is not measured to buy recall: on the 2,560-dimension halfvec corpus the search is effectively exact from 40, recall@10 was 0.9995 at 100 and unchanged to 1000. What does move with it is scan time — 0.39 ms per probe at 100 against 1.74 ms at 1000 on that corpus — and nothing else: hnsw.ef_search is a query-time setting, so the index footprint the re-embed runbook tells you to watch is fixed by how the index was built and is identical at every value here. It was RAG_EF_SEARCH before #1285 — env-only, read once at startup — and that variable is now a bootstrap fallback consulted while no rag_ef_search row has been read. On an instance that is still running on it the panel says so above the control and offers a Keep button, because Save only sends values you changed and the number shown already matches what the server resolved — pressing it writes the row, and every successful read from then on ignores the variable (see the read-failure exception below). A value outside pgvector's [1, 1000] is ignored rather than clamped (the pre-#1285 reader accepted up to 10000), and the startup log says so by name.

A settings read that fails does not change the floor (#1512). The reader is cached for 60 seconds; when that cache expires and the admin_settings SELECT then fails — pool pressure, a statement timeout — it keeps the value and the provenance it last resolved. Saving the panel clears that cache, so in the window right after a save what it keeps is the row the save just wrote — and so does a read that SUCCEEDS in that window but whose snapshot predates the save, since nothing deletes that row and an absent-row read there is a raced snapshot rather than evidence; only a resolve holding neither — nothing resolved and no row written by that process since boot — keeps this variable. Both memories are per-process, so a pod that restarts with the variable still set, or a second pod that never served the write, still falls back to it if its own first settings read fails, and keeps falling back to it — re-held each TTL — until one of its reads succeeds. It used to fall to the constant 100, which dropped every kNN probe on that pod to 100 for a full minute and made Settings → AI Models → Retrieval report the depth as 100 with no environment note and no Keep button — losing the remedy exactly while the number was wrong. Saving the panel still takes effect immediately on the pod that served the write. Unset RAG_EF_SEARCH once the panel is saved. While that fallback stands the panel shows the environment note and its Keep button again, and pressing Keep writes the variable's number as the saved value — over the number an admin saved earlier, which that pod could not read. The note now says so where the button is ("if a value was saved before and this server could not read it, saving here replaces it"), but it cannot tell you the value it could not read, so the variable — unset once the panel is saved — is where that risk is actually removed.

Fuzzy title matching is fixed at similarity 0.3 and is deliberately not a setting (#1285). The typo-tolerant title lookup in search uses pg_trgm's % operator, which is what lets PostgreSQL use the idx_pages_title_trgm GIN index instead of scanning every title — and % compares against PostgreSQL's own pg_trgm.similarity_threshold GUC, whose default is 0.3. The constant in the code is pinned to it so the retained similarity() > 0.3 check stays exact. A deployment that genuinely needs a different threshold changes the database setting (ALTER DATABASE … SET pg_trgm.similarity_threshold) and the constant together; changing only one of them either loses the index or changes which rows match without saying so.

Three things worth knowing before you use it.

The panel writes only what you changed. A numeric or on/off knob you never touched keeps no admin_settings row at all — an absent row and an explicitly-default one read alike, but the assembly budget's fail-toward-last-known behaviour is written assuming no row exists until an operator sets one. fts_language is the exception, and deliberately so: migration 049 seeds it with simple on every instance before the first request, which is exactly what made the removed FTS_LANGUAGE environment variable unreachable (see below).

A save of a numeric or on/off knob takes effect within a minute. Each of them is read through a 60-second in-process cache. The server that handled your save drops its cache immediately; every other server in the deployment converges as its own TTL expires. (Before #1118 the cache was never dropped at all, so a saved value could take a full minute even on the server that wrote it.) The keyword index language is the exception: it is read uncached, and saving it rebuilds every page's tsv inside the request, so it is in effect the moment the save returns — and the save takes as long as re-indexing your corpus does (see below). If that rebuild fails, the language is rolled back with it, so the setting and the index cannot disagree.

The rerank stage is not on this panel. It is on if and only if a rerank use-case assignment exists (ADR-021 — rerank never inherits the default provider), which lives in Settings → AI Models → LLM providers. The Retrieval panel owns the candidate-pool size and reports whether the stage is live; setting the pool before assigning a provider is fine, it simply has nothing to size yet.

The panel refuses out-of-range values at the input instead of accepting a save the reader would silently discard — most importantly 1 for either confidence threshold, which the reader rejects outright.

Production benchmark

The Production benchmark section replays the most recent distinct questions from this deployment's search_analytics table against the current pages and embeddings. Each question is measured in both ordinary mode and deep-search mode, so admins can see latency, empty-result counts, top-result movement, result-set overlap and whether expansion actually ran on their own data.

This is an asynchronous, admin-only, read-only retrieval operation. It does not seed or change pages, embeddings or retrieval settings, and replayed questions are not added to search analytics. Since real questions have no ground-truth labels by default, the production report does not claim Recall or MRR. An explicitly labelled custom suite can be submitted to POST /api/admin/retrieval-benchmark when the team has expected page IDs; only those labels produce Recall/MRR.

The single run slot is shared. Only one run of this kind exists at a time, and the other holder is the shadow model comparison — Settings → AI Models → LLM providers → "Compare on real queries", which appears inside the shadow migration card once a #1116 candidate column is fully backfilled and scores that candidate against the live embedding model on the same real queries (agreement first, and a judged Recall/MRR/McNemar verdict once 20 side-by-side picks exist). Both spend the same LLM queue, so while either runs the other answers 409 "already running" — and the message names the run that is actually holding the slot, so a benchmark refused by a comparison says so rather than reporting a benchmark nobody started. The comparison's own lifecycle step is 3b in docs/runbooks/shadow-reembed.md.

Keyword index language (`fts_language`)

The PostgreSQL text-search configuration the keyword leg of hybrid search is built and queried with — that is, its stemming and stop-word rules. admin_settings key fts_language; default simple, which does no stemming at all. Allowed values: simple, english, german, french, spanish, italian, portuguese, dutch, swedish, norwegian, danish, finnish, hungarian, turkish, russian, arabic, romanian.

Two things separate it from the knobs below. Saving it re-indexes the whole corpus inside the request — every page's tsv is rebuilt with the new configuration, pages in the trash included, so a restored page cannot come back indexed in the previous language — rather than taking effect on the next cached read. And the value is not a range but a closed allow-list, because PostgreSQL has no bind-parameter form for a regconfig, so the name is interpolated into the search SQL.

The row and the rebuild are one transaction, with statement_timeout lifted for its duration (a PG_STATEMENT_TIMEOUT deployment would otherwise kill a corpus-wide UPDATE deterministically) and lock_timeout set to 30 seconds in its place — the rebuild's own work is unbounded, but no single lock wait exceeds 30 seconds (lock_timeout applies to every lock the statement waits on, not only the first), so a page row held by an in-flight edit makes the save fail and retry rather than hang on a database connection. So the save is slow on a large corpus, and if the database rejects it nothing changed: the language stays as it was and you can retry. The panel cannot tell you why it was rejected — the 503 it shows is a fixed message and the driver error stays off the wire — so the cause is in the server log; if that names a lock timeout, retry when editing has quietened. If the same request also carried other retrieval knobs, those were written before the rebuild started and are kept — the error message says so. It is not touched by Reset all to defaults in the panel — a bulk reset of the cheap knobs must not quietly re-index your corpus back to simple. Change it from its own control.

One failure mode is not a rollback. The rebuild is unbounded on the database side, but the request still passes through your reverse proxy, and the bundled frontend/nginx.conf closes an /api/ response after proxy_read_timeout 300 (five minutes). Closing that connection does not cancel the PostgreSQL statement, so a corpus whose rebuild outruns the proxy budget reports a gateway timeout in the browser while the transaction goes on to commit. The panel re-reads the server after any failed save for exactly this reason: believe the value the panel shows after the error, not the error. You do not have to remember that — after a failed save that carried the language, the panel says on screen which of the two happened, because the value it just re-read is the evidence: "the server now reports German" means the transaction committed despite the error, and "the server still reports Simple — the language was not changed" means it rolled back and Save is the retry. If you front Compendiq with your own proxy and expect a rebuild longer than its read timeout, raise that timeout before changing the language, or run the rebuild during a window where a five-minute request is acceptable. Re-saving the same language is always safe — it just repeats the work.

FTS_LANGUAGE is removed (#1114). It was documented as the way to set this and never worked on a migrated instance: migration 049 seeds the fts_language row before the first request, so the code path that read the environment variable was unreachable. If your content is not English and you "set the language" that way, your keyword leg is still running simple — open Settings → AI Models → Retrieval and set it there. A leftover FTS_LANGUAGE in the environment is reported as ignored in the startup log and can be deleted.

Retrieval fetch width (`rag_fetch_width`)

How many candidate rows each RAG retrieval leg (vector + full-text) pulls before fusion and ranking, decoupled from how many results the caller gets back (#1103). admin_settings key rag_fetch_width; default 10 (the legacy per-leg limit), clamped to [10, 200] — values below 10 or non-numeric values fall back to the default, values above 200 are capped. Read through a 60-second in-process cache, so a change takes effect within a minute (immediately in the process that handled the save — see the panel section above).

Do not raise this without a reranker. More candidates is NOT more recall under plain RRF fusion: measured on the retrieval eval fixture, width 30 dropped Recall@5 from 0.88 to 0.72 while Recall@10 improved — deep legs let mediocre pages that match both legs outrank the best single-leg hit, so the right answers are in the pool but drowned in the top 5. The cross-encoder reranker (#1104) is the stage that turns a wide pool into better answers, and its rollout raises the effective width deliberately. On EE deployments with rag_permission_enforcement, a raised width also multiplies the per-candidate ACL checks a single search can perform.

Context assembly budget (`rag_context_chars_per_page`)

How many characters of each source page's sibling chunks the AI assistant's context window may carry, per page (#1106). The window is anchored at the matching chunk and expands to neighbouring sections under this budget, so the model sees contiguous prose instead of one arbitrary mid-page section. admin_settings key rag_context_chars_per_page; default 6000, clamped to [0, 24000] — 0 disables assembly entirely, the kill switch for small local models where a larger prompt costs more than the added context buys. Roughly: the default admits ~4 typical sections per source page, and the chat path carries 5 source pages, so raising this multiplies prompt size accordingly (the budget includes the joiners between sections). At the default the per-page ceiling is unchanged — chunks were already capped at 6000 chars — so the default changes the TYPICAL prompt, not the worst case; the 24000 cap quadruples the worst case and there is no input-side context-window guard, so raising it is a deliberate capacity decision. Negative values clamp to 0 (off). Read through the same 60-second cache as the other retrieval knobs; Settings → AI Models → Retrieval is the write surface (#1118).

Knowledge-base context sanitization (always on)

All knowledge-base content entering the AI assistant's prompt — retrieved chunks or assembled windows, and the sub-page tree — passes through the prompt-injection sanitizer, independent of the assembly budget above (setting the budget to 0 does not disable this). Runbook-shaped page content (a line starting "System:", phrases like "act as a ...") can therefore surface as [FILTERED] in an answer. That is the sanitizer working on content that pattern-matches an injection, not data corruption — the page itself is untouched, and the detection lands in the audit log tagged source: kb_context/subpage_tree with contentOrigin: first_party_kb, so it is distinguishable from an injection attempt in the asking user's own input.

Retrieval-confidence refuse gate (two thresholds, one per basis)

When raised above the default 0, the AI assistant refuses to answer a knowledge-base question whose retrieval confidence falls below the threshold — an honest "not enough grounded context" with the closest sources attached, instead of a low-grounded answer (#1105). Two admin_settings keys, because the two confidence bases are unrelated scales and the basis can flip per request (a rerank outage falls back to cosine):

rag_confidence_threshold — gates the similarity basis (max cosine over a vector-led returned set; also used when the rerank stage did not score the full set). The embedding model moves this scale.
rag_confidence_threshold_rerank — gates the rerank basis (max cross-encoder relevance, used only when the #1104 stage scored every returned row). The provider and its normalisation move this scale — and for a raw-logit reranker (e.g. llama.cpp) the per-request sigmoid normalisation makes scores only loosely comparable across requests (the backend logs Rerank scores were raw logits whenever that regime applied). Prefer a calibrated reranker if you intend to gate on this basis; tune conservatively from your own observed distribution either way.

Pick each number from the line under the control, not from another deployment (#1284). Both bases are recorded per request on search_analytics (confidence, confidence_basis, surface — migration 098), and Settings → AI Models → Retrieval shows the observed distribution under each threshold: p50, p90 and the number of assistant questions the two were computed over, for the last 7 days. The gate refuses when the score is below the threshold, so a threshold set at p50 puts about half the questions measured on that basis below the bar and one set at p90 about nine in ten — the consequence a bare number never states. Read that as a ceiling on refusals, not the rate: the gate stands down entirely for a turn that is grounded some other way — an assembled sub-page tree, an attached document, web results, or an earlier answer in the same conversation — and those turns are recorded in the sample all the same, because the analytics row is written during retrieval, before the refusal decision. On a multi-turn assistant the observed refusal rate at p50 is well under half. The sample counts assistant questions only — /llm/ask, deep search included — because the gate is never consulted on a page search; and below 30 questions the line says outright that the figures are too few to tune against. Do not expect the two counts to add up to the number of questions asked: a question that belongs to neither basis — keyword-led results, image-only results, a pinned exact-identifier hit, or a search that returned nothing at all — is recorded on basis none and appears in neither distribution. On a thin corpus that last case is the largest part of the gap. Nothing is hidden from you: neither threshold can act on those questions, because neither scale carries a number for them (the honest refusals that consult no threshold at all — an empty knowledge base, an embedding outage, an image-only match — are unaffected by these knobs; the four refusal reasons are enumerated in docs/architecture/09-flow-rag-chat.md). If the readout says the distribution could not be read, that is a failed request, not an empty corpus — press Retry at the top of the section (never reload the page: that discards every unsaved change on the panel). If it says the distribution could not be re-read, the figures beside it are real and merely not current — they are the last ones the panel could fetch, and the same Retry refreshes them. Per-request verdicts are still logged and traced (rag.confidence / rag.confidence_basis) for inspecting one answer.

An empty result set belongs to no basis, and neither knob decides it — it is refused ungated, ahead of any threshold comparison (no_context, #1105), so raising or lowering either number changes nothing for it. What the knobs can leave uncovered is a whole basis: on a rerank deployment with only the similarity knob raised, steady-state (fully reranked) requests are gated by the rerank knob, which is still 0 — raise both knobs for full coverage.

Each is a plain decimal in [0, 1) — 0.35, not 1, 0,35 or 35% (rejected loudly in the logs, gate stays off); 0, an absent row, or an empty value disables that basis and leaves its confidence diagnostic-only in logs and traces (rag.confidence / rag.confidence_basis span attributes). Set both from Settings → AI Models → Retrieval (#1118), which refuses 1 at the input rather than letting the reader discard it. SQL still works if you prefer it: INSERT INTO admin_settings (setting_key, setting_value) VALUES ('rag_confidence_threshold', '0.35') ON CONFLICT (setting_key) DO UPDATE SET setting_value = EXCLUDED.setting_value; (60-second read cache — takes effect within a minute).

A threshold is tied to the model it was tuned on (#1114). Because the scales belong to the models, changing a model reinterprets a number you did not touch: 0.35 tuned on bge-m3 is a different gate on Qwen3-Embedding-4B. Saving a threshold through the Retrieval panel records the resolved provider + model beside it (rag_confidence_threshold_calibration / rag_confidence_threshold_rerank_calibration); the panel then compares that record against the live assignment and shows an amber notice on that control once they diverge, naming the old model, the live one and which scale moved. Nothing rewrites the threshold — a shadow swap, its rollback and a direct use-case re-assignment all log a warning naming both models and leave the number exactly as you set it. Clear the notice by re-tuning the threshold against the observed distribution shown under the control (#1284 — the old advice was to grep your own logs). That distribution is not filtered to the new model: search_analytics records the score and its basis but no provider or model, so for up to seven days after a model change the window still contains questions scored on the previous scale, and the readout says so while the notice above it stands. Re-tune once the window has turned over, or tune conservatively and revisit. You can also clear the notice by pressing Keep <value> on the notice itself, which records the number you already have against the live model. That button is the only way an untouched threshold gets re-recorded: the panel's Save sends only knobs you actually edited, so saving something else on this page can never quietly certify a threshold you did not look at.

A threshold set before this shipped has no record and shows a muted "calibration unknown" line instead — an absent record is not evidence that anything changed. A threshold you write with SQL lands in the same state, because only the panel's PUT records a pair. Both clear the same way: press Record <value> on that line, which writes the number already in the row against the live model. (Pressing Save will not do it — Save sends only knobs you edited, which is the same rule that stops an unrelated edit certifying a gate you never looked at.) Saving a threshold while its stage has no model assigned (the usual case for the rerank basis — ADR-021 leaves the stage off until a rerank provider is assigned) is recorded too, as "tuned against nothing": it stays quiet while nothing is assigned, and turns amber the moment you assign a reranker, because the number then gates on a relevance scale it was never measured against.

If the server cannot resolve the model behind a basis at the moment you save — the provider row unreadable, an enterprise LLM policy pointing at a deleted provider — the threshold is saved and the calibration is deliberately left exactly as it was, with Could not resolve the model behind a confidence threshold in the log. An unknown is never written down as "tuned against nothing". You are told, on screen and not only in the log: Keep / Record answers Could not resolve the live <basis> model — the calibration was left as it was instead of reporting success, and the notice above the control says the live model could not be resolved (with a link to the provider row) rather than claiming no model is assigned. Both of those causes persist until you fix the provider, so pressing the button again will not help — check the provider's API key and, on Enterprise, the LLM policy.

Never auto-refused, whatever the thresholds: questions with other grounding that actually materialised (an assembled sub-page tree, successfully fetched URLs, web results that came back, a prior substantive assistant turn — request flags alone do not count: the "include sub-pages" toggle is session-sticky, so as a flag it disabled refusals for a whole session even when access to the tree was denied; when requested grounding fails to materialise the refusal says so and names the remedy), keyword-only or keyword-led results (no measurable signal), and empty results whenever retrieval health could not be positively verified (embedding provider down, corpus unembedded, or the coverage probe itself failed — an outage is not "the knowledge base has nothing on this"). Only an empty set from verifiably healthy retrieval, or a measured-weak set, refuses.

Note the coverage bar carefully: "healthy" means ≥95% of the user's accessible pages are embedded, measured per user over their accessible spaces. Below that — a large space mid-import, but also the quiet steady state of a handful of pages that persistently fail to embed — the zero-result refusal stands down for exactly the users who can see the under-embedded content, while weak non-empty sets still refuse normally. If you set the knob at 93% coverage and test with a nonsense question, you will get an answer, not a refusal, and the feature is not broken: the RAG retrieval confidence log line carries healthCaveat (partial_embeddings) and embeddingCoverage so you can see the gate standing down and why. The right values are deployment-specific — start from your logged confidence values, not from a number someone else used.

AI inline completion (`Settings → AI Models` and `Personal → Editor`)

Inline completion displays a short, one-line continuation as ghost text in the article editor. It is opt-in at the deployment level: under Settings → AI Models → LLM providers, assign the Inline completion row to a dedicated provider and model. Leaving the row at Disabled (no inline suggestions) is the server-wide off switch; it never inherits the default provider.

Choose a small, fast model. Editor requests are frequent, bypass the general LLM queue, and are limited to 48 output tokens in full-suggestion mode or 8 in word mode. FIM-capable coder models (for example Qwen Coder, DeepSeek Coder, StarCoder, Codestral, or CodeGemma families) use their fill-in-the-middle path; other OpenAI-compatible models use chat completion. A provider shared with other use cases also shares its circuit breaker, so a separate provider row gives stronger failure isolation.

Each user controls the feature under Settings → Personal → Editor:

Show inline suggestions enables or disables ghost text for that user.
Suggestion delay chooses 300 ms, 500 ms, 800 ms, or manual-only requests.
Default mode chooses a focused one-word completion or a full one-line suggestion. Existing users keep the full-suggestion default.
Code blocks only suppresses suggestions in prose.

In the editor, Tab accepts the visible suggestion, Option+] on macOS or Ctrl+] on other platforms accepts one word from a full suggestion, Escape dismisses it, and Option+\ or Command+Shift+Space on macOS (Alt+\ elsewhere) requests one manually. Automatic suggestions are also suppressed during IME composition, in tables, and on coarse-pointer devices.

No prompt or completion content is written to the LLM audit log. Monitoring is limited to aggregate request and token counters in Redis under metrics:llm:inline_completion.

Image retrieval (`Settings → AI Models`)

Pictures in your pages — architecture diagrams, screenshots, flowcharts — are described by a generative vision model at ingestion into bounded, versioned text, and that text is embedded by the ordinary text embedder as provenance-marked chunks of the page it came from. There is one index, one query embed and no third retrieval leg: the semantic and keyword legs you already run retrieve a picture's description like any other chunk, and a text-only chat model can cite the picture (ADR-027).

The legacy image-embedding leg is retired (#1618 stage 2, migration 118). ADR-025's separate page_image_embeddings index, its image_embedding use case, the MRL truncation width, the Image leg toggle (rag_image_leg_enabled) and the image-only refusal are gone. If you are upgrading from an instance that had them, read docs/runbooks/image-embedding-retirement.md — it covers what the migration drops and how to roll it back.

Everything here is off until you assign the model. An instance that never assigns image_analysis gets exactly today's behaviour and sends no image byte anywhere: the assignment IS the egress control (ADR-027 D3). Operations — assigning, probing, running the worker, re-analyzing, changing the model — are docs/runbooks/image-analysis.md.

The feature spans three sub-tabs, one per question you would actually ask.

1. Can it run? — `LLM providers` → the Image analysis row

image_analysis is a use case like chat or embedding, with one difference: it never inherits the default provider (the rerank rule, ADR-021). Leaving it unassigned is how you turn the whole feature off.

Assign a provider + model and press Save use-case assignments. It needs a generative vision model on an ordinary POST {baseUrl}/chat/completions that accepts an image_url part — not an embeddings endpoint, and not a text-only chat host.
Analysis output ceiling (image_analysis_max_output_tokens, Settings → AI Models → Workers) bounds the description each image may produce, so one pathological picture cannot write an unbounded chunk into a page.
The chat row's vision verdict is a different question and is probed separately — it decides whether the answering model is shown the picture itself. Analysis runs without it.

2. Is it running? — `Embeddings` → the Image analysis card

Status line — the assigned pair, or Not assigned — assign Image analysis under Settings → AI Models → LLM providers.
Images analyzed and Pages pending (dirty/total, pages.image_analysis_dirty is the queue).
Last run — pages, analyzed, reused, removed, plus Skipped: by reason. Images are skipped and counted, never resized, and SVG and draw.io files are never analyzed at all.
Process now drains the current backlog. Re-analyze all marks every page and re-reads its images; an unchanged image under an unchanged model/prompt identity is reused, so a re-analysis is far cheaper than it sounds.
Changing the model or the prompt revision changes the recorded identity and re-queues the corpus; the derived chunks for a page are replaced only when its new description is valid, so text search never goes dark in between.

3. How should it behave? — `Retrieval` → the Image retrieval group

Control	Key	Default	What it does
Images per page	`rag_images_per_page_max`	20	How many of a page's images are analyzed. A cost bound: each one past it is a request.
Index external images	`rag_image_index_external`	on	Whether pictures Confluence pulled from an external URL are analyzed.
Images shown to the model	`rag_answer_max_images`	2	How many matched pictures are attached to the chat request. 0 is legal and is the honest off switch.

When image_analysis is unassigned the group shows a muted note and keeps its controls enabled — they are settings, not actions, and configuring the intake before assigning the model is a reasonable order to work in.

What a text-only chat model does

A chat model that has not probed vision-capable never receives images, whatever Images shown to the model says. The answer is then produced from text alone — but it is no longer produced from titles alone: the picture's derived description is an ordinary retrieved chunk, so the assistant can answer about the diagram and cite it. The answer is unqualified: nothing on it, in the sources or in the screen-reader announcement says the picture itself was withheld. That is deliberate (ADR-025 D8) — on such a deployment the notice would appear on every answer, which is how a notice stops being read — and the fact is stated exactly once, under that control.

The matched pictures appear as sources with their thumbnails, and the citation opens the page the picture is on — the thumbnail itself is decorative, not a link to the file. To find out whether your model can see images, look at the chat row under Settings → AI Models → LLM providers: it shows the verdict, and Re-check there is how a wrong one is corrected.

Sync conflict resolution (Enterprise, v0.4+)

Confluence is the system of record for synced pages, but Compendiq supports local edits — the in-app editor, AI improve / generate / apply-improvement write-backs, and draft publishing all stamp pages.local_modified_at to mark the row as having unpublished local changes (CE #305). The next inbound sync pulls a Confluence-side change for that same page, and the question becomes: which side wins? The sync conflict policy answers that question per-page.

Three policies (Settings → Knowledge → Spaces & Sync → Conflict policy)

Policy	Behaviour	When to use
`Confluence wins` (default)	Apply the Confluence-side change; discard the local edit. Audit emits `SYNC_OVERWROTE_LOCAL_EDITS` whenever a local edit was actually present so you have a forensic record. No queue.	Compendiq is read-mostly — admins rarely edit pages locally, and Confluence remains the canonical source. The legacy v0.3 behaviour.
`Compendiq wins`	Skip the inbound write; preserve the local edit. Audit emits `SYNC_CONFLICT_DETECTED` with `resolution: 'kept_local'`. The local edit must be pushed back to Confluence manually (the editor's "Publish to Confluence" button) — otherwise the two systems drift further apart on every sync cycle.	Short-window edits where you definitely want the local copy to take precedence (e.g. an AI-improvement run is mid-publish and you don't want a remote overwrite to land on top of it). Use sparingly: drift compounds.
`Manual review`	Stash the inbound Confluence body in `pending_sync_versions`, flip `pages.conflict_pending = TRUE`, and surface the page in Settings → Knowledge → Spaces & Sync → Conflicts queue for admin review. The live row stays unchanged until the admin diff-reviews and picks. Audit emits `SYNC_CONFLICT_DETECTED` with `resolution: 'queued_for_review'`.	High-control workflows where every conflict needs explicit review (regulated documentation, contractual content). Trades sync latency for safety.

Important fall-through: the policy only kicks in when there are actual local edits (local_modified_at > last_synced). If the page has no local edits, both compendiq-wins and manual-review transparently fall through to confluence-wins — there's nothing to preserve, so the queue isn't padded with non-conflicts and the routine sync rate isn't degraded.

Resolution flow (manual-review)

Sync detects a conflict; the page lands in the conflicts list.
Admin opens Settings → Knowledge → Spaces & Sync → Conflicts queue, clicks Review on a row.
The dialog shows a side-by-side / unified diff of the local body against the queued Confluence body.
Admin picks one of three actions:
- Keep local — drop the queued Confluence version, keep the local row. Audit emits SYNC_CONFLICT_RESOLVED_LOCAL.
- Take Confluence — apply the queued version to the live row, clear local_modified_at. Audit emits SYNC_CONFLICT_RESOLVED_REMOTE.
- Cancel — close the dialog without resolving; the queue row remains.
The row clears from the conflicts list; if the page had multiple queued versions, the flag stays on until all are resolved.

Retention

pending_sync_versions rows are pruned by the daily retention worker after pending_sync_versions_retention_days (admin_settings key, default 90 days). Resolution deletes rows immediately, so the retention sweep only catches genuinely-abandoned conflicts. Override the default via the env fallback RETENTION_PENDING_SYNC_VERSIONS_DAYS (consulted only when the admin_settings row is absent). Clamp range: [7, 3650].

Audit trail

Five new event types under the existing closed AuditAction union — query them via the standard audit-log filters:

SYNC_POLICY_CHANGED — policy radio saved (metadata: { previous, next }).
SYNC_CONFLICT_DETECTED — emitted on every per-page conflict for the compendiq-wins and manual-review policies (metadata: { confluence_id, policy, resolution, sync_run_id }). Not emitted for the confluence-wins no-local-edits path — that's routine sync, not a conflict.
SYNC_OVERWROTE_LOCAL_EDITS — confluence-wins policy applied an inbound update on top of an actual local edit (metadata includes local_modified_at and last_synced so post-hoc analysis can compute the drift window).
SYNC_CONFLICT_RESOLVED_LOCAL / SYNC_CONFLICT_RESOLVED_REMOTE — admin clicked "Keep local" / "Take Confluence" in the resolve dialog.

Cluster-wide hot-reload

Saving a new policy publishes on the sync:conflict:policy:changed cache-bus channel; every other pod re-reads from admin_settings and switches to the new policy on its next sync run. The change does NOT retro-apply to a sync run already in flight — that run completes under the policy it started with. This is by design: midstream policy switches would create a hybrid trail of audit events that's hard to reason about.

Lost-update guard

The conflict-detection branch in sync-service.ts::syncPage runs the inbound-update UPDATE inside a transaction that opens with SELECT … FOR UPDATE. Two concurrent sync runs that touch the same page now serialise on the row lock — the second caller blocks until the first commits, then re-reads and either short-circuits (the first run already applied the same content) or queues a fresh pending_sync_versions row reflecting the post-merge truth. Without this lock, a second concurrent run could read the same local_modified_at snapshot, decide independently to apply the policy-driven UPDATE, and silently clobber the first run's writes.

AI Review Policy (Enterprise, v0.4+)

When the AI review workflow is enabled, AI-generated output is queued in ai_output_reviews rather than being written straight to the page draft. A reviewer must explicitly approve, reject, or edit-and-approve each row before the proposed content is applied. Configure the policy at Settings → AI → AI Safety → Review policy.

Three modes (per-action overridable)

Mode	Behaviour	When to use
`auto-publish`	Bypasses the queue entirely. AI output is applied to the draft as soon as the AI worker finishes. No audit gate, no human review.	Low-risk actions where you trust the model — typically `auto_tag` and `summary`. The fastest path; this is the legacy v0.3 behaviour.
`review-required`	Inserts a `pending` row in `ai_output_reviews`. The page draft is not updated until a reviewer approves or edit-and-approves. Audit emits `AI_REVIEW_SUBMITTED` on enqueue and `AI_REVIEW_APPROVED` / `AI_REVIEW_REJECTED` / `AI_REVIEW_EDIT_AND_APPROVED` on action.	Default for editable content actions (`improve`, `generate`, `apply_improvement`). The right balance for most teams.
`review-required-with-blocking-pii`	Same as `review-required`, plus: if the PII detector flagged the proposed content, the row is blocked from approval until the findings are cleared. Falls back transparently to plain `review-required` when the PII detector is disabled or absent (graceful degrade — `pii_findings_id` is nullable on the table).	Regulated content; high-stakes pages where personally identifiable data must not slip through unreviewed. Requires the `pii_detection` feature to be active for full effect.

Per-action configuration

The default_mode applies to every action that doesn't have an explicit override. Per-action overrides let you trust low-risk surfaces while gating riskier ones in the same deployment. Example: default_mode = 'review-required' plus an override of auto_tag = 'auto-publish' and summary = 'auto-publish' gives you human review on Improve / Generate / Apply-improvement while letting tag-suggestion and summary land directly. Set an action's mode select to Inherit default to remove its override.

The closed set of actions: improve, summary, generate, auto_tag, apply_improvement. Adding a new AI surface requires extending the AiReviewAction union in both ce/backend/src/core/services/ai-review-hook.ts and the per_action_overrides Zod schema in @compendiq/contracts/src/schemas/ai-review.ts.

Expiry semantics

A pending review that nobody acts on is auto-expired after expire_after_days days (default 30, range 1–365). The author is notified and the proposed content is discarded — there is no implicit auto-approval on expiry. The expire job is a daily BullMQ scheduler tick that runs UPDATE … SET status = 'expired' WHERE status = 'pending' AND now() > expires_at and aggregates author notifications into a single digest per author per run.

Rejected and expired rows linger in ai_output_reviews for forensic review for an additional 30 days (the cleanup retention window — adjust per the data-retention policy if you have one configured), then the daily retention sweep removes them. Approved and edit-and-approved rows live for the standard audit-retention window because they're the provenance trail for content that landed on a real page.

Reviewer-roles design

For v0.4 the reviewer access gate is admin-only, controlled by fastify.requireAdmin on every queue / detail / action route in the EE overlay. The original brief proposed a per-space scope (any user with editor on the page's space can review), but the matching getUserAccessibleSpaces helper isn't wired through yet — that lands in v0.5. Until then, treat the queue as an admin tool. A dedicated reviewer role (separate from admin) is also v0.5+ if customer demand surfaces.

Audit trail

Six new event types under the existing closed AuditAction union — query them via the standard audit-log filters:

AI_REVIEW_SUBMITTED — emitted by the EE overlay when an AI run is gated behind the review policy and a pending row is inserted (metadata: { actionType, pageId }).
AI_REVIEW_APPROVED — reviewer clicked Approve. The proposed content has been applied to pages.draft_body_*. Subsequent publish to Confluence is gated by the standard publish flow.
AI_REVIEW_REJECTED — reviewer clicked Reject. No page mutation. Optional notes field surfaces in the author's notification so they can re-run with better instructions.
AI_REVIEW_EDIT_AND_APPROVED — reviewer modified the proposed content before approving. Two audit rows are recorded — the original AI authorship plus the reviewer modification — preserving full provenance.
AI_REVIEW_EXPIRED — daily expire job swept a stale pending row. Aggregated per-author notifications follow.
AI_REVIEW_POLICY_CHANGED — admin saved a new policy via the policy tab (metadata: { previous, next }).

CE-only deployments

In CE-only deployments the routes 404 — the overlay isn't loaded, so there's nothing to enqueue against. The CE enqueueAiReview hook returns { mode: 'auto-publish' } and the AI surfaces fall through to their legacy direct-write behaviour. The admin tab surfaces a non-fatal "EE only" notice rather than crashing.

PII Detection (Enterprise, v0.4+)

When the PII detector is licensed and enabled, every AI inference response is scanned for personally identifiable information before it reaches the end user. Three layers run on every inference: regex (German + generic checksum-validated rules — IBAN, credit card, DE Tax ID, DE Personalausweis, DE RVNR), Transformers.js NER (Davlan/distilbert-base-multilingual-cased-ner-hrl, q8 ONNX, CPU-only, multilingual — covers PERSON / LOCATION / ORGANIZATION), and an optional asynchronous LLM-as-judge for ambiguous spans. Configure the policy at Settings → AI → AI Safety → PII detection.

Per-use-case actions

The five inference call sites (chat, improve, summary, generate, auto_tag) each carry an independent action mode so you can be strict on Improve / Generate while staying lightweight on Auto-tag:

Mode	Behaviour	When to use
`off`	Skip the scanner entirely. Output passes through with no findings recorded.	Auto-tag and other low-risk surfaces where the cost of every-call scanning isn't worth the false-positive noise.
`flag-only`	Run the scanner and persist findings, but pass content through unmodified. Findings appear in the LLM Audit page.	The default — gives you the audit trail without ever altering AI output.
`redact-and-publish`	Splice detected spans out of the output and replace each with `[REDACTED:CATEGORY]`. End users see the redacted version; findings persist for audit.	Customer-facing actions where leaks must be prevented but human review would be too slow.
`block-publication`	Reject the inference response with a 409 when findings are present. Integrates with the AI review queue (#120) — blocked outputs surface there for human review before reaching the page.	High-stakes, regulated content. Pair with the `review-required-with-blocking-pii` AI review policy mode.

Confidence threshold

NER findings below the configured threshold (0..1, default 0.7) are dropped before policy actions apply. Higher = fewer false positives, more false negatives. Regex findings always emit confidence: 1.0 and are unaffected.

Asynchronous LLM-as-judge

The optional second pass calls the configured LLM through the chosen LlmUsecase (defaults to quality — typically the cheap classification model). It runs on a low-priority BullMQ queue so it never blocks the user's request; findings are merged into the audit entry on completion. Three modes: off (default), flagged-only (only when regex+NER produced findings — minimises cost), always (every scan).

Categories to flag

Closed union of categories the scanner can emit. Uncheck a category to drop its findings before any action runs (regex still matches; the result is just discarded). Adding a new category requires extending the PII_CATEGORIES union in @compendiq/contracts/src/schemas/pii-policy.ts.

Cluster-wide hot reload

Saves publish on the new pii:policy:changed cache-bus channel; other pods invalidate their cached policy view within ~1 s. No restart required.

Audit trail

Every PUT emits PII_POLICY_CHANGED with the previous and next policy in metadata. Per-finding PII_DETECTED events are emitted by the inference call sites and queryable via the standard audit-log filters.

Air-gapped deployments

The EE Docker image bundles the NER model (/app/.cache/huggingface, ~67 MB) at build time so the runtime never reaches huggingface.co. Set PII_SCANNER_NO_REMOTE=1 to fail loud if the bundle is missing rather than silently re-downloading. Honours TRANSFORMERS_CACHE_DIR / HF_HOME for operators relocating the bundle.

CE-only deployments

In CE-only deployments the GET/PUT /api/admin/pii-policy routes 404 — the EE overlay isn't loaded. The admin tab surfaces a non-fatal "API not registered yet" notice rather than crashing, mirroring the IP allowlist / webhooks / AI review tabs.

Compliance Reports (Enterprise, v0.4+)

When the compliance_reports feature is licensed and enabled, admins can self-serve evidence packets for SOC 2 Type II and ISO 27001:2022 audits at Settings → Governance → Data & Compliance → Reports. Each report produces a ZIP containing a PDF cover sheet (with a SHA-256 integrity hash of the CSV body) and the CSV evidence body. Generation is recorded in the audit log under ADMIN_ACTION with action: 'generate_compliance_report' for chain-of-custody.

Customer model

The buyer is the customer's IT or security manager — not the auditor. The reports are designed to be generated by an admin and handed to a Big Four (or smaller) auditor as evidence, not consumed inside an audit firm's tooling. Output is intentionally portable (PDF + CSV) rather than tied to a specific GRC platform.

The seven reports

#	Report	id	SOC 2	ISO 27001	What it attests
1	User Access Report	`user_access`	CC6.2, CC6.6	A.5.15, A.5.18	Per-user identity, lifecycle (`created_at`, `last_login_at`, `deactivated_at`), current role + group + space-role assignments, and login activity in the reporting window.
2	Privileged Access Report	`admin_actions`	CC6.3	A.8.2	Every admin-role action in the reporting window — every privileged operation captured by the `ADMIN_ACTION` audit event.
3	Content Modification Report	`sync_data_flow`	—	A.8.15	Per-page create / update / delete / restore / move / reorder / draft-publish events plus the three bulk umbrella actions (tag, tag-replace, permission). One row per request for bulk events; per-page rollup lives in `metadata.bulkIds`.
4	Authentication & Session Report	`auth_session`	CC6.1, CC7.2, CC7.3	A.8.16	Logins (success + failure with `auth_method` and `failure_reason`), logouts, token refresh / family revocation, session create/revoke, password resets. MFA columns are reserved (MFA is v0.5+).
5	LLM Usage & Safety Attestation	`ai_usage`	CC6.7	A.8.15	Per-call attestation with the P0f safety flags (`prompt_hash`, `prompt_injection_detected`, `sanitized`). Plaintext prompts and responses are NEVER exported — only the SHA-256 fingerprint.
6	RBAC Change Log	`rbac_changes`	CC6.3	A.5.18	Every RBAC mutation: role grants/revokes, group lifecycle + membership, space access, page-level ACEs, page-permission inheritance toggles.
7	Data Retention Attestation	`data_retention`	—	A.8.15	Every retention-pruning sweep (`RETENTION_PRUNED` audit event) with the table touched, rows pruned, and retention window applied. Includes zero-row heartbeat sweeps so a clean window proves the cron ran.

Generating a report

Open Settings → Governance → Data & Compliance → Reports. The tab is admin-only and gated on a valid Enterprise license with the compliance_reports feature.
Pick a from / to window. The default is the last 30 days ending today (00:00 local). Inline validation rejects from >= to, to more than 24 hours in the future, and from more than 10 years in the past — the same bounds the backend orchestrator enforces.
Click Generate & download. The browser downloads a ZIP named compliance-<id>-YYYY-MM-DD.zip. The success toast surfaces the first 12 characters of the cover sheet's SHA-256 hash for at-a-glance verification.

Each card lists the SOC 2 / ISO 27001 control mapping for the report inline so the right evidence packet can be picked without consulting a separate mapping document.

Plaintext-safety contract for Report 5

Report 5 reads from the llm_audit_log table but its SQL SELECT lists 14 explicit columns — input_text, output_text, and input_messages (the columns EE deployments use to optionally store full prompts when admin_settings.llm_audit_full_text is enabled) are not projected. Defence-in-depth runs in three layers:

The report module's SELECT is explicit; no SELECT *.
The orchestrator's CSV writer enforces a FORBIDDEN_CSV_COLUMNS deny-list that includes password_hash, pat_encrypted, oidc_secret, webhook_secret, signing_key, jwt_secret, input_text, output_text, input_messages. Any column matching the list throws — and the thrown error message never echoes the offending cell value (no log leak).
Every report module declares its column set up front (via the ReportData.columns array); the writer asserts that no row introduces a column outside the declared set.

Auditors receive only the SHA-256 prompt_hash. To pivot from a hash back to the original call (e.g. during incident investigation), use the same hash on the LLM Audit page (Settings → AI → AI Safety → Audit log) — that page is gated separately by the llm_audit_trail feature and may surface plaintext if llm_audit_full_text is on.

PDF cover sheet integrity

Every cover sheet prints the SHA-256 of the CSV body bytes and the byte-length. The same hash is also returned in the X-Report-Sha256 response header for callers verifying the download programmatically. There is no PDF/A signing in v0.4 — the integrity model is "the hash is on the cover, the cover is in the ZIP, the ZIP is the evidence packet." Customers who require a digital signature (Adobe Sign, DocuSign) can sign the ZIP themselves; the v0.5 roadmap evaluates ETSI PAdES-LT signing of the cover.

Audit-log gap caveats

Each report's cover sheet prints any caveats discovered at generation time. Common cases:

Audit log is empty — the database has no audit rows at all. The report is empty for any window. Investigate whether audit-event capture is active.
Audit log earliest entry is after window start — the retention policy pruned older audit rows. The report covers only the visible portion of the window.
Pre-P0f rows in window (Report 5) — when migration 073 (introducing prompt_hash / prompt_injection_detected / sanitized) landed mid-window, rows from before the migration carry NULL prompt_hash and FALSE safety flags by default. The caveat tells auditors to read those as "not attested" rather than "no incidents".
No retention-pruning rows (Report 7) — the window contains no RETENTION_PRUNED audit events but the audit log itself is non-empty. Either the retention worker did not run, or it ran without emitting heartbeats. The cover flags this as a finding-class signal.
MFA not yet implemented (Report 4) — the columns for MFA_ENROLLED / MFA_DISABLED are reserved in the audit-action union but no emitter writes them yet (MFA ships in v0.5+). The caveat suppresses itself once the first real MFA row is observed.

Bulk audit-volume bound (Report 3)

BULK_PAGE_TAGGED, BULK_PAGE_TAGS_REPLACED, and BULK_PAGE_PERMISSION_CHANGED emit one umbrella row per request, NOT one row per affected page (per epic v0.4 §3.6 / R8). The per-page rollup lives in metadata.bulkIds + metadata.affectedCount. Auditors needing per-page granularity for a bulk event must dereference metadata.bulkIds. Page-permission ACE events (ACE_GRANTED / ACE_REVOKED / PAGE_INHERIT_PERMS_CHANGED) live in Report 6 (RBAC Change Log) and are deliberately not duplicated in Report 3.

v0.4 scope deferrals

Confluence sync-history attestation (originally part of Report 7's scope, P0e) — deferred to v0.5. The confluence_sync_history table does not yet exist in CE; the cover sheet documents this scope boundary so an auditor doesn't infer it as a coverage gap. Sync-related events (SYNC_STARTED, SYNC_COMPLETED, SYNC_CONFLICT_*) appear in the general audit_log and can be cross-referenced separately during audit prep.
Per-call PDF/A signing — deferred to v0.5 (see above).
Auditor-facing read-only role — the v0.4 access gate is admin-only. A separate auditor role that can generate reports without other admin powers is on the v0.5+ roadmap if customer demand surfaces.

Audit trail

Every generation emits ADMIN_ACTION with metadata { action: 'generate_compliance_report', reportId, rangeFrom, rangeTo, rowCount, csvBytes, pdfBytes, zipBytes, sha256 }. Row content is never logged — only the artefact's hash and byte sizes — so an admin running 100 reports does not bloat the audit log or leak content into operational logs.

CE-only deployments

In CE-only deployments the /api/admin/compliance-reports and /api/admin/compliance-reports/generate routes 404 — the EE overlay isn't loaded. The Settings → Governance → Data & Compliance → Reports tab is hidden by the requiresFeature: 'compliance_reports' filter on the tab list; for a defence-in-depth direct URL navigation, the tab itself surfaces an "EE license required" panel.

Monitoring (OpenTelemetry)

Variable	Default	Description
`OTEL_ENABLED`	`false`	Set to `true` to enable OpenTelemetry tracing
`OTEL_SERVICE_NAME`	`compendiq-backend`	Service name reported to the OTLP collector
`OTEL_EXPORTER_OTLP_ENDPOINT`	(none)	OTLP collector endpoint (e.g., `http://localhost:4318`)

When tracing is enabled, every outbound LLM call additionally emits a custom span (llm.chat, llm.stream_chat.dispatch, llm.embeddings, llm.list_models) tagged with the provider id and model, covering queue wait plus the upstream request. See docs/PERFORMANCE.md → "OpenTelemetry Tracing" for the full span reference.

Docker Compose Only

Variable	Default	Description
`HOST_CA_BUNDLE_PATH`	`/etc/pki/ca-trust/extracted/pem/tls-ca-bundle.pem`	Host path to CA bundle (bind-mounted into container)
`CONTAINER_CA_BUNDLE_PATH`	`/etc/ssl/certs/ca-certificates.crt`	Container path for the CA bundle

Upgrade Procedure

Breaking change: `FTS_LANGUAGE` is removed

The keyword-index language is now set only in Settings → AI Models → Retrieval. Delete FTS_LANGUAGE from your environment and docker/.env; a leftover value is reported as ignored in the startup log.

This one needs a look even if you never set the variable deliberately, because it never took effect: migration 049 seeds the fts_language row before the first request, so the code path that read the environment was unreachable. If your content is not English, your keyword leg has been running simple — no stemming, no stop words — whatever the variable said. Open the panel, set the language most of your content is written in, and Save; the save re-indexes the whole corpus inside the request, so read Keyword index language (fts_language) first if your corpus is large.

Breaking change: `POSTGRES_PASSWORD` / `REDIS_PASSWORD` are now required

docker/docker-compose.yml no longer ships the changeme-postgres / changeme-redis default passwords — docker compose up refuses to start until both variables are set (in docker/.env or exported in the environment).

Important: PostgreSQL bakes the password into the postgres-data volume when it is first initialized. Setting a new POSTGRES_PASSWORD on an existing install does not change the password inside the volume — the backend will fail to authenticate and crash-loop.

Pick one of the following when upgrading an existing install:

Option A — keep the existing credentials. Set the variables to the values your volumes/containers were initialized with. If you previously relied on the defaults, that is:
```
# docker/.env
POSTGRES_PASSWORD=changeme-postgres
REDIS_PASSWORD=changeme-redis
```

Option B — rotate properly (recommended). Change the live Postgres password first, then update the env and recreate:

# 1. Generate a URL-safe password (avoid base64: '/', '+', '=' break
#    the POSTGRES_URL/REDIS_URL interpolation)
openssl rand -hex 24

# 2. Change the password inside the running Postgres
docker compose -f docker/docker-compose.yml exec postgres \
  psql -U kb_user -d kb_creator \
  -c "ALTER USER kb_user WITH PASSWORD '<new-password>';"

# 3. Set POSTGRES_PASSWORD=<new-password> in docker/.env, then
docker compose -f docker/docker-compose.yml up -d

Redis needs no in-place step: requirepass comes from the container command line (no state in a volume), so setting a new REDIS_PASSWORD and running up -d re-creates it with the new password.

Docker Compose Upgrade

Pull the latest images:

docker compose -f docker/docker-compose.yml pull

Restart services:
```
docker compose -f docker/docker-compose.yml up -d
```
Database migrations run automatically on backend startup. No manual migration steps are required.
Verify the upgrade:
```
curl http://localhost:8081/api/health
```

From Source

Pull the latest changes:

git pull origin dev
npm install
npm run build

Restart the services. Migrations run automatically.

Immutable Page Baseline Foundation

The #275 foundation stores a canonical manifest, independent retained copies of referenced media, and append-only freeze/thaw evidence. CE provenance is a manual_assertion: optional reported signatories and references are caller-supplied assertions, not authenticated approval or a cryptographic signature. Signed governance, independently trusted keys, and evidence reports belong to #278; the shared UI belongs to #277. The #276 enforcement dependency must land first, so creation is deliberately off in this release.

Published evidence has no product TTL, ordinary-retention deletion, capacity eviction, or delete API. Thaw unlocks the live page but retains its old baseline. Evidence also survives page and actor deletion: live foreign keys may become null while original page identity and actor display snapshots remain. Old page history and media URLs do not become an evidence back door after deletion; they return not found. Deleted-source evidence is available only to an active system administrator through the baseline-ID admin routes.

Freeze preview/manual freeze requires current page access plus system-admin, space manage, or standalone-page ownership. Thaw is narrower: current page access plus system-admin or space manage; owner-only thaw is not allowed. Ordinary page history redacts reported signatory emails, while the system-admin evidence response may contain the stored caller-reported email. A UUID or digest never grants access.

Interrupted writers never expire by heartbeat or TTL. Recovery is active system-administrator work through /api/admin/page-write-recovery: quiescence must reach the process that owns the runtime ID, fencing accepts only an exact owner acknowledgment, server-proven no-start state, or independently verified local process termination, and outcome reconciliation uses trusted kind-specific server code. There is no caller-supplied proof or force-clear route. An unversioned remote write with an unknown result stays unresolved. Quiescence waits across successful multi-stage writer gaps through final settlement; it is not a timeout-based cancellation. Conditional Confluence recovery reads exact provider history and atomically publishes the matching local content and completion metadata before clearing the intent. It never reissues an unknown remote mutation. Recovery itself belongs to the process performing it: quiescence waits for its verification, repair and settlement. If that process dies, fence its new runtime ID before retrying. A failed provider read after an acknowledged page PUT keeps the acknowledgment and retries only observation/local publication, never PUT. Committed local changes and hard deletes also retain cache-publication work through Redis outages; the worker retries it after Redis returns.

The complete activation, storage, canonical digest, capacity, API, recovery, and backup procedure is docs/runbooks/immutable-page-baselines.md.

Attachment Storage & Orphan Sweep

ATTACHMENTS_DIR (default data/attachments) holds two stores that grow with use: the Confluence cache <confluence_id | page id>/<file> (synced attachments, pasted images — standalone pages key by numeric id, so the keyspace is shared with Confluence ids) and the local store local/<page id>/<file> (draw.io saves and relocated pages, with metadata rows in local_attachments). Sync and page deletes clean their own pages; everything else is covered by the observability card and the sweep shipped in #1349. A third entry, page-icons/<page id>/<sha>.<ext>, is a separate store for uploaded page marks: it is reserved by name, never walked and never swept. Those files are reconcilable — for icon_kind = 'image' the page row's icon_value is the sha that names the file — but they are deliberately out of scope for #1349, because they are the only copy of an uploaded mark and a wrong verdict there is unrecoverable rather than a re-fetch. Removal is therefore event-driven, and since #1349 every permanent delete performs it: unsetting or replacing an icon deletes the old file, a standalone hard delete or trash purge removes the icon directory beside that page's attachment directories — the icon unconditionally, because page-icons/<pk>/ is keyed by pages.id alone and no other page can own it, while the attachment directories follow the per-store rules stated further down — and a Confluence page that is hard-deleted (singly or in bulk), purged from the trash after its 30-day window, or removed with its space by Unsync does the same. "Unconditionally" is about ownership, not timing: each of those removals runs only once the row-deleting transaction has committed. If that local cleanup fails after the page is already gone in Confluence — the row survives, hidden and soft-deleted, for the 30-day trash purge to collect — the mark stays on disk with it and goes when the purge removes the row. A soft delete deliberately does not: a trashed page is restorable and its mark is its own content, not a re-fetchable cache.

Where: Settings → Knowledge → Spaces & Sync → Sync schedule, in the Attachment storage card (admin only; the wrapper opens on its Spaces tab, so the sub-tab hop is part of the path). The card shows per-store bytes / file / directory counts and the last sweep summary, read from a persisted record — the figures are as fresh as the last completed walk (they carry a "Measured … ago" date; a refused or failed run leaves them standing rather than clobbering them with its own partial view), and Dry run is how you refresh them.

How to run a sweep:

Press Dry run. It walks both stores, measures them, and lists orphan candidates without touching a single file. On a large corpus this takes minutes; the card polls until it finishes.
Review the candidates. The card's Show the N candidates disclosure lists them — store, key, filename, whole-directory vs single-file, and bytes — up to a bounded sample (the heading says so when the run found more than the sample holds). GET /api/admin/attachments/sweep returns the same sample if you would rather script the review.
Press Delete orphans and confirm. The live run re-walks and re-checks every candidate at delete time — it never trusts a stale dry-run list. When it finishes, the card's figures are the tree as it stands after the delete, so what it just removed no longer shows as a candidate.

What is deleted: only files that (a) sit in a directory whose key matches no page row at all — including soft-deleted/trashed pages and folders, which all count as owners — or (b) meet their store's per-file test. That test differs by store. In the Confluence cache a per-file candidate must be an image-like file referenced by no body text anywhere; a non-image cached attachment there is skipped on type before anything else is checked. In the local store there is no type filter: a per-file candidate is any file with no local_attachments row that is also referenced by no body text anywhere. In normal operation the row is what protects local files of every type: every local writer pairs the file with its row on its success path, so the two rules land in the same place. The exception is debris — bytes a Confluence-to-local move stages before the transaction that inserts their rows, whose compensating cleanup is best-effort — and collecting that debris, of any type, is precisely what this sweep is for. Either way the local test is a row check, not a type check. "Referenced by no body text anywhere" means the same thing in both stores: every page's body_html, draft and storage format (live and trashed), every retained version, every pending sync version, every template, every comment and every saved AI conversation (#1361 persists a matched image's URL per assistant turn) feed one global keep-set per store, because attachment URLs are copied verbatim between bodies. The keep-set outranks the directory verdict too: a pageless directory that still holds even one referenced filename is skipped whole and reported as keep-protected rather than deleted, so a referenced file is never removed at either level. A pageless directory that holds sub-folders or links is likewise never judged: attachment directories are flat by construction, so anything under a key-shaped name that is not a plain file — a nested tree, a symlink — is something the walk cannot measure, and the card reports it instead. Nothing younger than 24 hours is ever a candidate (paste and sync both write files before the referencing row exists); the card says how many candidates are waiting out that window, so a freshly-emptied store does not read as a clean one. Non-image attachments cached from Confluence (PDFs and other lazily fetched files) are never per-file candidates: the Confluence cache's type filter skips them before anything else is checked. That exemption is per-file only, and it belongs to one store. It does not override rule (a) — a directory whose key matches no page row is judged and removed whole, taking any non-image file inside it — and it does not extend to the local store, where a row-less file of any type is judged.

An image attached to a live Confluence page but embedded in no body is treated as cache and may be removed — the attachments macro and the article view fetch attachments lazily through GET /api/attachments/:pageId/:filename, which caches whatever filename it was asked for, and nothing in the corpus references those files. Removing one costs a re-fetch from Confluence the next time it is viewed, not the file: Confluence remains the copy of record. (A non-image attachment on that same path sits under that same live page, so the per-file type filter excludes it and a cached PDF is left alone.) Locally uploaded images are a different matter and are protected by their local_attachments row.

local_attachments rows whose file is missing on disk are counted, never deleted — a mis-mounted ATTACHMENTS_DIR must not wipe the metadata. Files the sweep deletes take their page_image_analyses rows with them and the owning pages are re-queued for image analysis.

A directory whose name is not a usable attachment key — tmp.12345/, 12345 (copy)/, anything a person or another tool left under ATTACHMENTS_DIR — is never opened, never judged and never removed. It is also not measured, so its bytes are missing from the per-store figures; the card says how many such directories the walk stepped over, so a store carrying a large one does not silently under-report its size.

The keep-set rule belongs to the sweep, not to the delete routes. A permanent page delete removes that page's directories directly (that is the leak the sweep does not have to converge), and that removal consults no keep-set: local/<pk>/ and page-icons/<pk>/ go unconditionally, and <pk>/ in the shared Confluence-style tree goes only when no page claims that confluence_id and the directory has aged past a 5-minute grace. So if you copy editor HTML containing an image out of one standalone page into another and then permanently delete the source page, the copy loses its picture. Nothing in the product duplicates pages, so this needs a deliberate copy/paste between two pages to reach; if you rely on that pattern, re-upload the image into the page that keeps it rather than pasting its URL.

Refusals: any run refuses when the attachments root is missing or unreadable, and a live run stands a store down when it has zero files on disk while the database still references it — the signature of an unmounted volume. That verdict is per store: with both stores anomalous the run refuses outright, and with only one the run completes, sweeps the sound store and names the skipped one in an amber note on the card (otherwise an instance whose re-fetchable Confluence cache is legitimately empty could never clear its local orphans, which are not re-fetchable). Nothing is ever deleted on the strength of a missing directory alone.

Bookkeeping: the sweep is single-flight (worker lock attachment-sweep), manual-only (no schedule), admin-rate-limited, and every run — dry runs included — emits a RETENTION_PRUNED audit event on attachments_orphan_sweep with counts by reason class, so the Data Retention Attestation report (Report 7) covers it. That report's table and rows pruned columns read the event's table / rows_pruned keys, which name the one table this sweep prunes rows from — page_image_analyses — and the row count it removed there. Files are not rows: what was deleted on disk is in the same event's files_pruned / directories_pruned / bytes_pruned, beside dry_run, which is what tells a heartbeat from a real prune. No retention window is reported because there is no retention policy here (Phase 3 was left out on purpose); the only time rule is the fixed 24-hour mtime grace. API: GET /api/admin/attachments/stats, GET|POST /api/admin/attachments/sweep (body { "dryRun": true|false }).

Backup Strategy

In-app encrypted backup (#1420)

Settings → Backup & Recovery (admin) streams a gzip+AES-256-GCM archive of pg_dump -Fc plus ATTACHMENTS_DIR. Before it enumerates attachment files, the exporter opens a read-only repeatable-read PostgreSQL transaction, exports its snapshot, and passes that snapshot to pg_dump. It keeps the exporting transaction open until pg_dump has exited, then closes and returns the pooled connection. Backup generation remains constant-memory: the backend does not buffer a complete dump, attachment, or archive, and it does not report success until pg_dump closes with exit code 0.

Immutable page baselines require the database records and ATTACHMENTS_DIR/page-baselines/ bytes as one recovery set. The in-app backup includes both because it walks the complete attachment root. A manual database-only dump or attachment-volume-only copy is not complete baseline evidence; stop application writers and capture both together. See the immutable baseline runbook for restore verification.

Download: the authenticated admin request creates a 256-bit Redis ticket with a 30-second TTL, then the browser performs native navigation to the returned same-origin URL. That redemption route is intentionally public because navigation cannot attach the admin access-token header; the random ticket is the bearer capability, is consumed atomically on the first GET, and returns 404 after use or expiry. The ticket contains no backup bytes, never places a passphrase in the URL, and the browser never buffers the archive in a Blob. Passphrases must be at least 12 characters and use PBKDF2-SHA256 (600,000 iterations). If BACKUP_ENCRYPTION_KEY is set (at least 32 bytes), downloads can omit the passphrase.
S3: only publicly reachable HTTP(S) S3-compatible endpoints are supported. Loopback, private, link-local/cloud-metadata, internal-hostname, and DNS-to-private destinations are rejected, even if another feature allowlisted the origin; the final path-style or virtual-host request URL is checked again immediately before network I/O. Path-style addressing remains available for public S3-compatible services. Access and secret keys are encrypted at rest with encryptPat, and one failed settings write rolls the complete update back. Test connection uses the saved configuration, so save S3 edits first.
Schedule: configure interval hours, keep-last-N, and delete-after-days. One cluster-wide lock (worker:lock:backup) prevents concurrent dumps. History refreshes while a run is active.

Enterprise KMS and immutable storage

The same CE frontend image includes these controls; they render only for a valid Enterprise license with enterprise_backup_dr. Backend support is still required. A server that omits an optional capability does not acquire it merely because the frontend is newer.

Envelope encryption: choose AWS KMS or Vault Transit, save the key identifier, then test the saved configuration. Unsaved settings cannot be tested or rotated. Credentials stay in the backend environment, not this form. AWS uses its default credential chain, including renewable workload/instance roles; profile and web-identity files need read-only container mounts. Vault uses VAULT_TOKEN, with VAULT_ADDR and optional VAULT_NAMESPACE. See .env.example for credential-chain inputs and the EE compose file for forwarding. KMS-only downloads, manual S3 runs, and schedules require neither BACKUP_ENCRYPTION_KEY nor a passphrase. A failed policy read or KMS request fails the backup rather than silently selecting legacy encryption.
Object Lock (WORM): save separately from S3 settings. Choose Compliance or Governance, 1–3650 days or 1–10 whole years, and optional legal hold. A year is 365 days; changing the unit to years visibly rounds up. Compliance retention cannot be shortened. Disabling the setting affects future uploads only: existing version retention and legal holds survive. Pruning reads each version's retention/hold metadata, skips protected versions, and deletes expired/unlocked versions by their exact version ID. It never sends Governance bypass and never creates a delete marker to hide a retained backup. Unreadable policy or protection metadata fails closed.

For version-aware pruning, grant s3:ListBucketVersions, s3:GetBucketObjectLockConfiguration, s3:GetObjectRetention, s3:GetObjectLegalHold, s3:GetObjectVersion, and s3:DeleteObjectVersion, in addition to upload/list permissions. Missing metadata-read permission is not treated as evidence that an object is unlocked. Multipart SHA-256 checksums are per-part and included in completion; a multipart checksum is not SHA-256 of the concatenated object bytes.

Enterprise replicas and disaster recovery

Enterprise supports a primary plus at most two additional destinations. Creation serializes the count check, and a run refuses older configurations over that cap before uploading. Branches share upstream chunks, write bounded slices, and honor backpressure; the cap bounds multipart-buffer amplification, not memory already allocated by the archive producer.

Every destination uses its own prefix. The per-run destination result retains the complete object key that was actually uploaded, so changing a prefix later does not rewrite history. A failed replica or a destination removed between configuration loading and history insertion does not cancel a healthy primary. Failed destinations remain failures; this does not imply independent retry scheduling.

Keep each recovery endpoint, bucket, region, complete object key, and credential access outside the database being protected. --destination-id is only a convenience while that database is available. To recover without it, supply BACKUP_RESTORE_S3_ENDPOINT, BACKUP_RESTORE_S3_BUCKET, optional region/prefix, and BACKUP_RESTORE_S3_ACCESS_KEY / BACKUP_RESTORE_S3_SECRET_KEY through the process environment, then run the EE restore CLI:

npx tsx backend/scripts/restore-backup.ts \
  --object-key 'replica-a/compendiq-backup-20260908T120000Z.enc' --dry-run

The key is complete, not relative to the configured prefix. This path does not query backup_destinations and works with an empty recovery database. Supply the archive's actual passphrase/master key, or AWS/Vault credentials for a KMS archive. Only remove --dry-run after stopping Fastify traffic and confirming the target database and attachment directory.

Offline restore (all editions)

Restore is an offline, standalone operation: stop Fastify traffic before a non-dry-run restore, and run backend/scripts/restore-backup.ts from a source checkout with postgresql17-client (pg_restore) installed. POSTGRES_URL must reach the target database. ATTACHMENTS_DIR must identify the real live attachment tree; its parent must be writable and on the filesystem that will hold the restored tree.

The CLI creates a mode-0700 .compendiq-restore-* directory next to ATTACHMENTS_DIR. Reserve free space there for the complete decrypted database.dump and attachment tree in addition to the existing live data (and account separately for the encrypted input file). Keeping staging beside the live tree is required for the rename-based attachment swap.

Prefer BACKUP_PASSPHRASE over the compatibility --passphrase flag. The following Bash example reads without echoing or recording the secret literal in shell history, exports it only for the restore process, validates the full archive, and then clears it:

read -r -s -p 'Backup passphrase: ' BACKUP_PASSPHRASE
printf '\n'
export BACKUP_PASSPHRASE
npx tsx backend/scripts/restore-backup.ts \
  --file compendiq-backup-YYYYMMDD.enc \
  --dry-run
unset BACKUP_PASSPHRASE

Remove --dry-run only after validation succeeds and an application outage is in place. Without a passphrase, the CLI uses BACKUP_ENCRYPTION_KEY. The legacy --passphrase form remains accepted but exposes the value in process listings and often shell history; do not use it for routine operations.

Before any live mutation, restore fully consumes AES-GCM authentication and validates safe/unique archive members, a strict closed manifest (bounded to 1 MiB), the exact SHA-256 checksum set, database size, the PAT_ENCRYPTION_KEY fingerprint, and that the archive migration is not newer than this binary. --force overrides only the PAT-key fingerprint mismatch; it never bypasses authentication, schema, path, member, checksum, size, or forward-migration checks. Validation failure and dry-run leave PostgreSQL and live attachments unchanged and remove staging.

Commit renames the previous attachment tree aside, installs the staged tree, and runs pg_restore --clean --if-exists --no-owner --no-acl --single-transaction, followed by the shipped database migrations. Before reporting success, it changes any backup_runs row restored with status running to failed, sets its finish time, and records Backup interrupted by restore; the live source row is never changed before restore. If pg_restore fails, the replacement attachment tree is removed and the previous tree is restored; the secure staging directory remains beside ATTACHMENTS_DIR for diagnosis and must be removed after recovery.

Migration-failure boundary: pg_restore has already committed its single transaction and the restored attachments remain live before migrations run. Arbitrary migrations cannot be rolled back safely, so a migration failure does not attempt a database or attachment rollback. The error reports both the retained old-attachment rollback directory and restore staging directory. Keep both while diagnosing/recovering the database; remove them manually only after the restored state is accepted or another restore has completed.

Manual PostgreSQL dump

Use pg_dump if you are not using the in-app exporter:

# Full backup
docker compose -f docker/docker-compose.yml exec -T postgres pg_dump -U kb_user kb_creator > backup_$(date +%Y%m%d_%H%M%S).sql

# Compressed backup
docker compose -f docker/docker-compose.yml exec -T postgres pg_dump -U kb_user -Fc kb_creator > backup_$(date +%Y%m%d_%H%M%S).dump

Restore from backup:

# From SQL dump
docker compose -f docker/docker-compose.yml exec -T postgres psql -U kb_user kb_creator < backup.sql

# From compressed dump
docker compose -f docker/docker-compose.yml exec -T postgres pg_restore -U kb_user -d kb_creator backup.dump

Redis

Redis is used as a cache layer. Data in Redis (page list caches, search result caches, LLM response caches) is transient and will be rebuilt automatically. Backing up Redis is optional.

If desired, Redis RDB snapshots can be triggered:

docker compose -f docker/docker-compose.yml exec redis redis-cli -a <redis-password> BGSAVE

Attachments

In-app backups already include ATTACHMENTS_DIR. To copy the volume alone:

docker run --rm -v compendiq_attachments:/data -v $(pwd):/backup alpine tar czf /backup/attachments_backup.tar.gz -C /data .

Backup Schedule Recommendation

Component	Frequency	Retention
Encrypted in-app backup	Daily (Settings → Backup & Recovery)	Keep last 7 / 30 days
PostgreSQL (manual `pg_dump`)	Daily if not using in-app backup	30 days
Attachments volume	Covered by in-app backup	4 weeks if copied separately
`.env` file	After every change	Keep current + previous

Monitoring

Health Endpoints

Compendiq provides Kubernetes-compatible health probes:

Endpoint	Purpose	Checks
`GET /api/health`	Public health status	PostgreSQL + Redis + LLM connectivity (coarse; no internal telemetry)
`GET /api/health/ready`	Readiness probe	PostgreSQL + Redis
`GET /api/health/live`	Liveness probe	Always returns 200 (process is alive)
`GET /api/health/start`	Startup probe	Startup complete + PostgreSQL + LLM availability
`GET /api/llm/circuit-breaker-status`	Per-provider circuit-breaker state	Returns a provider-keyed flat map: `{ [providerId]: { state, failureCount, nextRetryTime } }`. Requires auth.

Example response from GET /api/health:

{
  "status": "ok",
  "services": {
    "postgres": true,
    "redis": true,
    "llm": true
  },
  "llmProvider": "OpenAI Prod",
  "version": "1.0.0",
  "uptime": 3600.123
}

Status values: ok (all healthy), degraded (partial), error (all down).

/api/health is public (unauthenticated), so it deliberately omits internal operational telemetry. Per-provider circuit-breaker state is available to authenticated callers via GET /api/llm/circuit-breaker-status.

Docker Health Checks

The Docker Compose configuration includes built-in health checks for all services. Monitor container health with:

docker compose -f docker/docker-compose.yml ps
docker compose -f docker/docker-compose.yml logs -f backend

OpenTelemetry

Enable distributed tracing by setting:

OTEL_ENABLED=true
OTEL_EXPORTER_OTLP_ENDPOINT=http://your-collector:4318

This exports traces to any OTLP-compatible collector (Jaeger, Grafana Tempo, Datadog, etc.).

The backend auto-instruments HTTP (including incoming Fastify requests, as HTTP server spans), undici, pg and Redis. There is no Fastify-specific route instrumentation.

Stable semantic conventions (since the 0.221 OTel release)

The OpenTelemetry instrumentations now emit only the stable semantic conventions; there is no opt-out back to the old names. Collector rules, dashboards and alerts keyed on the old names stop matching after the upgrade:

Old	New
span attribute `db.system`	`db.system.name`
span attribute `db.statement`	`db.query.text`
span attribute `db.name`	`db.namespace`
span attribute `http.method`	`http.request.method`
span attribute `http.status_code`	`http.response.status_code`
span attributes `net.peer.name` / `net.peer.port`	`server.address` / `server.port`
metric `http.server.duration` (histogram, ms)	`http.server.request.duration` (histogram, s)
metric `http.client.duration` (histogram, ms)	`http.client.request.duration` (histogram, s)

The HTTP duration metrics changed unit as well as name: renaming a query is not enough — latency thresholds and panel units must move from milliseconds to seconds (a 500 ms alert threshold becomes 0.5), otherwise they are off by a factor of 1000. Database spans no longer carry db.user or db.connection_string (GHSA-qqmp-wf37-98f9).

Audit Logging

Compendiq logs user actions and system events. Query audit logs via the admin API: GET /api/admin/audit-log (admin-authenticated, supports pagination/filtering). Logs are stored in PostgreSQL and include:

User authentication events (login, logout, failed attempts)
Page operations (create, update, delete)
Admin actions (role changes, settings updates)
Sync operations

Managing Users

Admins manage the full user lifecycle from Settings → Governance → Access Control → Users. All mutations are audit-logged under ADMIN_ACTION events and rate-limited the same as other admin endpoints. Under the hood each action maps to one of the per-user admin endpoints (POST|PUT|DELETE /api/admin/users/...) so EE bulk tooling (#116) can compose the same primitives.

Creating a user

Go to Settings → Governance → Access Control → Users and click Add user.
Fill in username (required), optional email, optional display name, and pick the role (user or admin).
Choose one of two credential modes:
- Explicit password — type it into the form. The user signs in with that password on first login.
- Send invitation — leave the password blank and tick Send invitation email. The backend generates a random temporary password and tries to email it to the new user via the configured SMTP server (see SMTP_* env vars and the Settings → System → Integrations → Email / SMTP admin tab). When SMTP is not configured the API returns the temporary password in the response so the admin can hand it over out-of-band.
The new user is created immediately — there is no email-verification step in CE.

Editing a user

From the user's row in the table:

Display name, email, role can be patched via PUT /api/admin/users/:id. Changing an email to one that is already in use returns HTTP 409 (EMAIL_TAKEN).
Demoting the last active admin is rejected with HTTP 409 (LAST_ADMIN). Promote another admin first.
Username and password cannot be changed through this form. Password changes go through the self-service flow from the user's own profile; username is immutable once set.

Deactivating (soft) vs deleting (hard)

Deactivate is the recommended off-boarding path:

Sets deactivated_at, deactivated_by, and deactivated_reason on the user.
Revokes every refresh-token in the same transaction so any open sessions cannot mint new access tokens.
Blocks future logins at the auth layer with a generic "Account is deactivated — contact an administrator" message.
Preserves all history (audit_log, page authorship, comments, templates) untouched.
Is reversible via the Reactivate action, which clears the deactivation markers.
Rejected with LAST_ADMIN if the target is the only remaining active admin, and with SELF_FORBIDDEN if an admin tries to deactivate themselves (promote another admin first).

Delete is a hard delete — use it only when you need the row gone (accidental imports, GDPR erasure, fresh-install cleanup):

Removes the users row outright.
Cascades pages/comments/embeddings/settings that are tied to the user via ON DELETE CASCADE.
Nulls out historical pointers on rows that are worth keeping: audit_log.user_id, error_log.user_id, comments.resolved_by, pages.created_by_user_id, pages.owner_id, pages.verified_by, notifications.source_user_id, and so on. The records survive with NULL where the user used to be.
Reassigns any templates the user authored to the internal __system__ sentinel so the NOT NULL FK on templates.created_by stays valid.
Is refused if the target is the only active admin (LAST_ADMIN) or if the admin tries to delete themselves (SELF_FORBIDDEN).
Cannot be undone. There is no trash / recycle bin — back up PostgreSQL before running hard deletes in bulk.

Resetting a password

CE has no admin-side password reset — neither the Users admin UI nor the admin API can change an existing user's password. Users change their own password through the self-service flow on their profile page. If a user has lost their password, the recovery options are:

Set the password directly in PostgreSQL with a known bcrypt hash — the same procedure used when the last admin loses their password.
Delete and re-create the user via Settings → Governance → Access Control → Users with a new explicit password or Send invitation email (mind the hard-delete caveats above — history pointers are nulled and owned content cascades).

Self-service limits and the `system` user

An admin cannot deactivate, delete, or demote themselves via the admin API — deliberate, to avoid accidentally locking the install out of admin access.
The __system__ user (UUID 00000000-0000-0000-0000-000000000000) owns built-in templates. It does not count toward the "last active admin" check and cannot log in — it is an internal sentinel and must not be edited or deleted.

Bulk user operations (Enterprise, v0.4+)

Enterprise-only — this section applies only when the licence grants bulk_user_operations. The Settings → Governance → Access Control → Users page degrades to the per-user CRUD described above when the feature is unlicensed.

The Settings → Governance → Access Control → Users page surfaces two bulk affordances when the licence grants the feature:

Bulk import — header button. Opens a three-step CSV import dialog: pick a .csv file → preview parsed rows with field-level errors and duplicate detection → confirm and apply with the chosen mode.
Multi-select bulk actions — checkboxes on each row plus a select-all in the header. Once at least one row is checked the Bulk actions button appears, opening the action dialog (change role, deactivate with optional reason, reactivate, add to group, remove from group). The current admin's row never carries a checkbox so a single mistaken header click cannot bulk-deactivate the operator.

Both flows share the same audit semantics as the per-user CRUD: a wrapper BULK_USER_IMPORT event surrounds the per-row USER_CREATED / USER_UPDATED / USER_DEACTIVATED / USER_REACTIVATED / USER_HARD_DELETED audit entries so the compliance report can reconstruct exactly who triggered which change at the per-user grain.

CSV format

Column	Required	Notes
`username`	yes	1–50 chars; letters, digits, `.`, `_`, `-` only. Unique.
`email`	yes	RFC-5321 mailbox; max 254 chars. Lower-cased before comparison. Unique.
`displayName`	no	1–100 chars. Free text.
`role`	yes	`admin` or `user` (lower-case). Per-space role assignment lives in RBAC and is not writeable through this CSV.
`initialPassword`	no	When set, the imported user can log in immediately with this password. When blank, the server falls back to the invitation flow if `sendInvitation=true`.
`sendInvitation`	no (default `true`)	When true and SMTP is configured, the new user is emailed a temporary password with a forced reset prompt. When SMTP is not configured the temp password is shown in the import response so the admin can hand it over out-of-band.

A header row is required and must match the column names verbatim — the parser is column-name driven, not positional.

Example CSV

username,email,displayName,role,initialPassword,sendInvitation
alice,alice@corp.example,Alice Smith,user,,true
bob,bob@corp.example,Bob Jones,admin,ChangeMe!2026,false
charlie,charlie@corp.example,,user,,true
diana,diana@corp.example,Diana Prince,user,,true
eric,eric@corp.example,Eric Engineer,admin,,true

A starter template ships with the repository: docs/assets/bulk-users-template.csv. Save a copy and edit the rows to match your roster.

Import modes

The confirm step asks which mode to apply:

Create only — the safe default. The server errors out as soon as any row collides with an existing username or email. Use this for first-time onboarding batches where you are sure none of the rows overlap with the live roster.
Upsert — duplicates are updated in place (display name, role, optionally password); new rows are created. Useful for periodic full-roster syncs from an HRIS export.

Both modes apply atomically against the database — either every row in the batch lands or none of them do (the EE overlay wraps the apply in a single transaction). The preview step is read-only; nothing is written until you press the apply button.

Bulk-action catalogue

The multi-select action menu offers:

Action	Notes
Change role	Sets `role` for every selected user. Subject to the same `LAST_ADMIN` guard as the per-user route — demoting the only remaining admin is rejected with HTTP 409.
Deactivate (optional reason)	Soft-deactivates each selected user and revokes their refresh tokens. Reason is recorded against every per-row audit entry.
Reactivate	Reverses a previous deactivation. Refresh tokens are not restored — the user signs in fresh.
Add to group	Adds every selected user to the named RBAC group.
Remove from group	Removes every selected user from the named RBAC group.

The last-admin and self-action guards from the per-user CRUD apply unchanged: bulk requests that would demote/deactivate/delete the only remaining admin or the calling admin themselves are rejected at the EE overlay layer with HTTP 409 / 400 respectively.

CSV injection — operator hardening

Anyone authoring a bulk-user CSV that will later be opened in a spreadsheet should be aware that fields starting with =, +, -, or @ are interpreted as formulas by Excel, Google Sheets, and LibreOffice Calc. A maliciously crafted CSV can therefore execute formulas — including =HYPERLINK(...) data-exfiltration payloads — against an unsuspecting admin who opens the file to inspect it before re-uploading.

Mitigations on the operator side:

Never paste untrusted input directly into a displayName cell. If you receive a roster from an external system, sanitise it first (e.g. prefix any cell starting with = + - @ with a single quote).
Open the CSV in a plain-text editor first when in doubt. The Compendiq import dialog only ever reads the file as text — it does not execute formulas.
The export side of the bulk feature (when shipped) prefixes any cell starting with the four formula characters with a single quote per OWASP guidance.

Inert in CE-only deployments

The CE half of the bulk feature ships the contracts, the UI, and the audit-event names. The actual server implementation lives in the EE overlay and is loaded at runtime when the licence grants bulk_user_operations. In a CE-only deployment the routes do not exist and the UI is hidden by the licence gate; the affordance also degrades gracefully to a "requires Enterprise" message if the route ever 404s (for instance, between an EE licence-key install and the next overlay deployment).

Encryption Key Rotation

Compendiq supports zero-downtime rotation of the PAT encryption key:

Generate a new key:
```
openssl rand -base64 48
```

Add versioned keys to .env:

# Keep the old key as V1, add the new key as V2
PAT_ENCRYPTION_KEY_V1=old-key-at-least-32-characters!!!!!
PAT_ENCRYPTION_KEY_V2=new-key-at-least-32-characters!!!!!

Alternatively, use JSON format:

PAT_ENCRYPTION_KEYS='[{"version":2,"key":"new-key-..."},{"version":1,"key":"old-key-..."}]'

Restart the backend. New encryptions use the highest-version key. Decryption tries all keys, so existing PATs remain readable.
Re-encrypt existing PATs by calling the admin API: curl -X POST -H "Authorization: Bearer <admin-token>" http://localhost:8081/api/admin/rotate-encryption-key.
Remove the old key once all PATs have been re-encrypted.

Connections adoption review (#1314)

The four-week review starts at the deployment of the Connections panel, not at issue creation. Record that deployment time and the evaluated revision. Graph stays in primary navigation until an explicit post-release decision; no adoption result is assumed by this change.

The panel submits three authenticated event kinds through POST /api/pages/:id/connections/events: impression, connection_click, and graph_launch. An impression means a successful Connections result has rendered and the panel has entered the viewport. Empty successful panels are included; loading and failed reads are not. Ordinary re-renders and background refetches do not create another impression. The per-visit UUID ties clicks to the corresponding panel visit without storing article text.

Collection uses the existing audit_log: actions CONNECTION_IMPRESSION, CONNECTION_CLICK, and CONNECTION_GRAPH_LAUNCH, with resource type page_connections and the source article ID as resource_id. Metadata holds only event, visitId, and the click's group; the target is authorized at collection time but its ID is not retained. Article titles, bodies, labels, scores, and reasons are never analytics metadata. The ordinary audit IP/user-agent fields and configured retention policy still apply.

An article panel mount owns one visit UUID. The database deduplicates impressions by authenticated user, source article, and UUID, including duplicate delivery; clicks remain individual events. Leaving an article and returning creates a new visit. Match clicks to impressions on that same three-part key, not UUID alone, and retain audit records for at least the complete 28-day evaluation window.

For the first 28 days after deployment, evaluate:

Reach: recorded impressions and distinct participating users. Report the denominator explicitly: this is visible panel visits, not all article loads.
Read-next adoption: the share of impressed visits with at least one connection click, plus the linked / section / related breakdown.
Advanced exploration: the share of impressed visits launching the focused graph. Count visits, not raw repeat clicks, for the conversion rate.
Repeat use: users returning to Connections across separate days, with sample size and the deployment window shown alongside the rate.
Measurement health: collection failures, retention covering the entire window, and whether browser/network blocking could explain missing events. A missing event is not proof that a reader ignored the feature.

Choose what constitutes meaningful use with the product owner before interpreting the figures. Rates alone do not establish usefulness, and small samples are not evidence for removing navigation. If adoption is insufficient, the follow-up decision may remove Graph from primary navigation while retaining relationship data for recommendations, search, duplicate detection, and AI context. This PR does not make that decision and contains no fabricated adoption figures.

Troubleshooting

Backend fails to start with "JWT_SECRET must be at least 32 characters"

In production mode (NODE_ENV=production), the server enforces that JWT_SECRET and PAT_ENCRYPTION_KEY are not default values and are at least 32 characters long. Generate proper secrets with openssl rand -base64 48.

"Connection refused" to PostgreSQL or Redis

Ensure the infrastructure containers are running and healthy:

docker compose -f docker/docker-compose.yml ps

Check that the POSTGRES_URL and REDIS_URL in your .env match the container configuration. Inside Docker Compose, use service names (postgres, redis) instead of localhost.

LLM requests fail or time out

Open Settings → AI → AI Models → LLM providers. Confirm the provider row you assigned to Chat (or the failing use case) has the expected base URL, and press Test on that row.
For a local Ollama / LM Studio host: verify it is reachable from the backend container (curl the /v1/models URL; inside Docker use http://host.docker.internal:…). Env OLLAMA_BASE_URL seeds an empty table and, on an existing install, rewrites only the default sentinel http://localhost:11434/v1 when the env value differs — a non-sentinel saved URL is not overridden.
For hosted OpenAI / DeepSeek / similar: confirm the API key on the provider row (Edit → paste a new key) and that the model id is one the account can call. A 401 from Test is almost always a bad key.
For timeout issues with large articles, increase LLM_STREAM_TIMEOUT_MS.
Check circuit breaker status via GET /api/llm/circuit-breaker-status (authenticated, per-provider detail) — if a provider shows open, the circuit breaker has tripped due to repeated failures. It will reset automatically.

TLS certificate errors

For self-signed certificates on Confluence or LLM servers:

Set CONFLUENCE_VERIFY_SSL=false or LLM_VERIFY_SSL=false to disable verification (not recommended for production).
Or provide a CA bundle via NODE_EXTRA_CA_CERTS=/path/to/ca-bundle.pem.
In Docker, mount the host CA bundle using HOST_CA_BUNDLE_PATH and CONTAINER_CA_BUNDLE_PATH.

Embeddings are not being generated

Under Settings → AI Models → LLM providers, confirm the Embedding use case points at a provider that actually serves /v1/embeddings (local Ollama bge-m3, an OpenAI embedding model, etc.). Do not point Embedding at a chat-only host such as hosted DeepSeek. EMBEDDING_MODEL in .env is inert after migration 054.
For a local Ollama embedder: ollama pull bge-m3 (or the assigned model).
Changing the Embedding row does not go live via Save use-case assignments. Start the re-embed from that row (Start re-embed). Saving first would switch the live model before the index can store its vectors. Wipe current index rebuilds the live index only — it is not a way to apply an unsaved model change.
If the assigned model’s width does not match page_embeddings.embedding (for example Qwen3-Embedding-4B at 2560 vs a 1024 column), every page fails until that re-embed (or the assignment is pointed back at a matching model). See docs/runbooks/shadow-reembed.md.

Database migrations fail

Migrations run automatically on startup. If a migration fails:

Check the backend logs: docker compose -f docker/docker-compose.yml logs backend
Ensure the PostgreSQL user has sufficient privileges.
The pgvector extension must be available (the pgvector/pgvector:pg17 Docker image includes it).

High memory usage

Redis is configured with maxmemory 256mb and noeviction by default — BullMQ requires it, so a full instance rejects writes rather than evicting. That means Redis running out of memory stops job enqueue (sync, re-embed, summary, quality), not just caching. Watch used_memory against maxmemory in redis-cli INFO memory.
Lower Pages per batch for the quality and summary workers under Settings → AI Models → Workers to lower worker memory usage.
Consider increasing Docker container memory limits for the backend if processing large articles.

Users report "Image staging is temporarily unavailable … near its memory limit". That is the deliberate guard from issue #1183, not a fault: images attached to AI Generate/Improve are staged in Redis for 15 minutes (at most 5 MB each, one per user), and staging is refused once the write would take Redis above IMAGE_STAGING_MAX_REDIS_PERCENT (default 80) of maxmemory. The image feature degrades so that background processing keeps working. It clears on its own as staged entries expire. If it recurs, raise Redis --maxmemory in docker/docker-compose.yml (and REDIS_MEM_LIMIT above it) rather than raising the percentage — the remaining fifth is the headroom the queues write into.

Image analysis (vision) — #1615, ADR-027

The candidate that replaces the image-embedding leg. Settings → AI Models → LLM providers → Image analysis (vision).

The assignment is the egress control. No page image leaves the host until you save a provider here; the card names the saved provider that receives them. It never inherits the default or the chat provider.
Save is probe-gated. Saving runs the known-content vision probe on the pair the row would resolve to (your model, else the provider's default) and writes the row only on a true verdict. Four refusals, each leaving the previous assignment untouched: provider not found, no model resolves, the model refused the test image (text-only — pick a vision-capable model) and image support could not be confirmed (unreachable, auth, 429, open breaker — not a verdict about the model; fix the endpoint and save again, or Re-check).
Pause, not purge. Unassigning stops new analysis; descriptions already indexed and still valid stay searchable. The Index identity line shows the model identity the index was built under; if the provider's endpoint moved, an amber notice says analysis is paused and Re-check re-probes the new endpoint and, on true, adopts it — which re-analyzes every image (the toast states the count). GET /api/admin/llm-usecases/image_analysis/reanalysis-scope?providerId=…&model=… discloses that count before you commit.
Max output tokens (default 8,192, range 4,096–16,384) is the vision reply's ceiling. Lower it if the model's context refuses the default; raise it for scripts that tokenize below one character per token. Saving it fires no probe and re-analyzes nothing.
Inspection: GET /api/admin/pages/:id/image-analyses[?payload=1] (admin, page-visibility checked) lists a page's analysis rows with their failure class and validity.

Is it running? — the Image analysis card (#1618)

Settings → AI Models → Embeddings carries the processing half. The row above is can it run?; this card is is it running? — it sits beside the legacy Image index card for as long as both designs ship.

Counters. Analyzed is the images with a description the index can actually use. Stale is separate on purpose: analyzed on disk, but written under a model identity or a prompt/schema version that is no longer current, so no reader accepts it until the next run re-analyzes it. Pending, Failed, Given up (the attempt cap — only Retry failed moves those) and Skipped complete the work window.
Two backlogs that are not the same thing. "N pages analyzed, text embedding still pending" costs no vision call — the descriptions exist and the page is waiting for the ordinary text embedder. A partially analyzed corpus still owes vision calls. "N pages queued for an image re-read" is the third: those pages changed and their images have not been looked at yet.
Skip reasons are named. missing from the store is the one that is a gap rather than a decision — the page references an image the store does not hold, which is a broken sync, and it is what keeps an otherwise complete corpus reading partial. The rest (unsupported format, above the pixel bound, too large, external URL, past the per-page cap) are policy working as configured.
Last run reports the three steps and, when a batch stopped early, the reason and the endpoint's HTTP status. "Paused: no vision model is assigned" is not an error — the sweep and the reconcile still ran.
The three actions. Process now kicks one bounded batch (it reports already running rather than claiming a second one started). Retry failed gives every failed and given-up image a fresh attempt budget, due at once. Re-analyze all discards every stored description first and spends one vision call per image; it shows the exact count before it runs, and it is refused with a 409 while a corpus text re-embed or a #1116 shadow backfill holds the one-active-run slot.
A failed status read is not an unassigned leg. If the card says the status could not be read, the assignment and the stored analyses are untouched and all three actions stay available — retry the read, or check the backend logs.
Routes: GET /api/admin/embedding/image-analysis, and POST /api/admin/embedding/image-analysis/{process,retry-failed,reanalyze-all} (admin only).
Retiring the old leg: docs/runbooks/image-embedding-retirement.md has the destructive boundary, the backup set and the restore procedure. Nothing in it removes page content or attachment bytes.

FilesExpand file tree

ADMIN-GUIDE.md

Latest commit

History

ADMIN-GUIDE.md

File metadata and controls

Compendiq Admin Guide

System Requirements

Installation

Docker Compose (Recommended)

Verifying the Installation

Configuration Reference

Required Secrets

Database

Redis

Server

Immutable page baseline storage (foundation)

Client inference (on-device editor model)

LLM providers (primary)

Add a provider

Recipe — hosted OpenAI

Recipe — hosted DeepSeek

Chat-only hosts must not cover embedding / rerank / image analysis

Runtime knobs that stay as env vars

Deprecated LLM env vars (bootstrap-only)

Embedding

Background Workers

Background Job Queue (BullMQ)

LLM Request Queue

Per-User Concurrent SSE-Stream Cap (#268)

Confluence

Notion import (#1459)

Confluence API Rate Limiting

Security and Encryption

Email Notifications (SMTP)

Enterprise

OIDC / SSO

IP Allowlist (Enterprise, v0.4+)

RAG Per-page ACL Enforcement (Enterprise, v0.4+)

Retrieval settings panel (Settings → AI Models → Retrieval)

Production benchmark

Keyword index language (fts_language)

Retrieval fetch width (rag_fetch_width)

Context assembly budget (rag_context_chars_per_page)

Knowledge-base context sanitization (always on)

Retrieval-confidence refuse gate (two thresholds, one per basis)

AI inline completion (Settings → AI Models and Personal → Editor)

Image retrieval (Settings → AI Models)

1. Can it run? — LLM providers → the Image analysis row

2. Is it running? — Embeddings → the Image analysis card

3. How should it behave? — Retrieval → the Image retrieval group

What a text-only chat model does

Sync conflict resolution (Enterprise, v0.4+)

AI Review Policy (Enterprise, v0.4+)

PII Detection (Enterprise, v0.4+)

Compliance Reports (Enterprise, v0.4+)

Monitoring (OpenTelemetry)

Docker Compose Only

Upgrade Procedure

Breaking change: FTS_LANGUAGE is removed

Breaking change: POSTGRES_PASSWORD / REDIS_PASSWORD are now required

Docker Compose Upgrade

From Source

Immutable Page Baseline Foundation

Attachment Storage & Orphan Sweep

Backup Strategy

In-app encrypted backup (#1420)

Enterprise KMS and immutable storage

Enterprise replicas and disaster recovery

Offline restore (all editions)

Manual PostgreSQL dump

Redis

Attachments

Backup Schedule Recommendation

Monitoring

Health Endpoints

Docker Health Checks

OpenTelemetry

Stable semantic conventions (since the 0.221 OTel release)

Audit Logging

Retrieval settings panel (`Settings → AI Models → Retrieval`)

Keyword index language (`fts_language`)

Retrieval fetch width (`rag_fetch_width`)

Context assembly budget (`rag_context_chars_per_page`)

AI inline completion (`Settings → AI Models` and `Personal → Editor`)

Image retrieval (`Settings → AI Models`)

1. Can it run? — `LLM providers` → the Image analysis row

2. Is it running? — `Embeddings` → the Image analysis card

3. How should it behave? — `Retrieval` → the Image retrieval group

Breaking change: `FTS_LANGUAGE` is removed

Breaking change: `POSTGRES_PASSWORD` / `REDIS_PASSWORD` are now required

Self-service limits and the `system` user