From 30c73a6f22b61d612e903bb12e3264ca20b8cc8d Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 21:35:25 +0000 Subject: [PATCH 1/6] Document agent platform integration and repository evaluation Connect canonical workflows with verified examples, access boundaries and a paired evaluation procedure so platform builders can find and check the answers. Signed-off-by: zlareb1 --- docs/features/code-index/repositories.md | 61 +++++++- docs/getting-started/evaluate.md | 28 ++++ docs/guides/agent-platforms.md | 174 +++++++++++++++++++++++ docs/guides/repository-questions.md | 45 ++++++ docs/index.md | 2 + docs/use-cases/memory-linked-to-code.md | 49 ++++++- llms.txt | 1 + 7 files changed, 358 insertions(+), 2 deletions(-) create mode 100644 docs/guides/agent-platforms.md diff --git a/docs/features/code-index/repositories.md b/docs/features/code-index/repositories.md index f6fba29631..ed3a7965a3 100644 --- a/docs/features/code-index/repositories.md +++ b/docs/features/code-index/repositories.md @@ -217,6 +217,65 @@ that: A `code.repo.set_access` row wins over the configured classification and join policy and survives restarts. `code.refresh` (writer) and the admin ops always need a grant. +### Separate repositories for two teams + +Keep repositories with different readers in different code namespaces. For example, +index a billing repository as `code-billing` and an operations repository as +`code-operations`, both `classification: "restricted"` with `join_policy: "approval"`. +Use the [operator-configured entry format](../../operations/code-index-operations.md) +or the [self-service access controls](#self-service-repositories) for your deployment. +Bootstrap maintainer grants before restricting an existing namespace, following the +[restricted-namespace procedure](../../guides/tokens-and-grants.md#restricted-namespaces-classificationrestricted). + +Give a billing service a live viewer grant and a token explicitly naming only +`code-billing`. Its expected read behavior is: + +| Request | Expected scope outcome | +| ---------------------------------- | ------------------------------------------------------------------------------ | +| `code.search` in `code-billing` | Authorized; useful hits still depend on the index and query | +| `code.search` in `code-operations` | Denied; the billing grant and token do not cover it | +| `code.repos` | Can include request-only metadata for the other repository; inspect `readable` | + +Check the allowed request with a symbol you know is indexed, then repeat an equivalent +request against the other namespace. Over HTTP, a denied scoped code read returns +`AUTH_INSUFFICIENT`; a known object ID can instead read as missing to avoid revealing its +existence ([authorization diagnostics](../../operations/troubleshooting.md#an-id-you-know-exists-reads-as-missing)). +Test revocation separately from an empty search result. Repeat for each indexed branch: +namespace naming and maintainer inheritance do not give every reader access to every +branch. + +A same-organization repository with approval-based joining can appear in `code.repos` +as `readable: false`. That row exposes metadata for requesting access, with empty index +counts and watermarks. It does not authorize reading code. Do not treat a listing entry +as a successful access check. + +For example, with the billing service's token, replace the query with a known billing +symbol. Repeat with `namespace` set to `code-operations` and expect an authorization +failure, not an empty successful result: + +```sh +curl --fail-with-body -X POST "$LOOM_BASE_URL/v1/code.search" \ + -H "Authorization: Bearer $LOOM_API_KEY" \ + -H 'Content-Type: application/json' \ + --data '{"org":"example","namespace":"code-billing","query":"invoice authorization","top_k":3}' +``` + +The Git token or SSH identity that fetches source is separate from the Loom token and +grants used by readers. A chat or document connector login does not grant a native Loom +service repository access. Agent names and repository tags record attribution or +association; they are not access boundaries. + +### Check indexing isolation separately from read access + +Repository read grants do not sandbox indexing jobs. Self-service indexing requires +the sandbox unless an operator explicitly enables its restricted unsandboxed mode; +operator-configured repositories can run without it. To require isolation for every +repository job, set `LOOM_CODE_SANDBOX_REQUIRED=true` and satisfy the host requirements. +When the sandbox is unavailable, jobs are refused or skipped rather than silently run +outside it. The [sandbox runbook](../../operations/code-index-operations.md#repository-job-sandbox) +describes namespaces, mounts, egress, process limits, supported stacks and remaining +container/host responsibilities; Loom's per-process limits do not impose a job cgroup. + ## Failure modes | Symptom | Cause and fix | @@ -226,7 +285,7 @@ and survives restarts. `code.refresh` (writer) and the admin ops always need a g | Adding a repository answers `AUTH_INSUFFICIENT` | the message names `LOOM_CODE_SELF_SERVICE_PLATFORM_ADMIN_ONLY`: only platform admins add repositories and branches on this deployment, with a token of their own that is not scoped to namespaces | | A self-service card stays at `error` | read `stage_detail` in `code.repos`; a failed clone retries next pass (timeout `LOOM_CODE_BOOTSTRAP_CLONE_TIMEOUT_S`) | | A private self-service repository stops fetching | `LOOM_JWT_SECRET` was rotated; re-enter the token | -| An entry is missing from `code.repos` | check `loom.code.repos_config_errors`; `code.repos` also lists only what the caller may read | +| An entry is missing from `code.repos` | check `loom.code.repos_config_errors` and the caller's listing eligibility; request-only entries have `readable: false` and do not grant code access | | Revoking or lowering an admin answers busy | a revoke on a code namespace first re-reads `loom.code_repos`, so it sees the branches added here from another process, and answers busy while that read fails; retry once the read succeeds | | A restricted branch added here has no admin | its admins were all copies from branches added here, which the sync revokes; grant admin on one of the repository's operator entries (with none, the operator lists a branch of it in `LOOM_CODE_REPOS_JSON` first) | | A new maintainer lacks the other branches | the admin grant was made before this process read `loom.code_repos` (that process logs `not synced` the first time); the next admin grant or approval on a branch of the repository, or a branch added, copies it; or grant it again | diff --git a/docs/getting-started/evaluate.md b/docs/getting-started/evaluate.md index b8047d2da5..e5b309e3b7 100644 --- a/docs/getting-started/evaluate.md +++ b/docs/getting-started/evaluate.md @@ -155,6 +155,34 @@ agent tasks with ordinary repository tools only. Report task success first, then tokens, cost and wall time. Tool-schema overhead depends on the client, so record whether it defers MCP tool definitions. +#### Label your own repository before querying + +The built-in acceptance report is not a repository-neutral answer-quality test. Make a +task sheet for your own repository with these columns before collecting results: + +| Column | What to record | +| ---------------------- | ---------------------------------------------------------------------------------------------- | +| Task and frozen source | Question, repository commit, branch and task category | +| Expected evidence | Definition path and qualified symbol, accepted source ranges, or a reviewed caller/test set | +| Expected answer | Claims the answer must support and claims it must not make | +| Expected refusal | Unanswerable, wrong-repository and denied-access cases | +| Retrieval outcome | Returned identifiers, ranks, citations, freshness and completeness limits | +| Agent outcome | Actual calls and evidence used, answer or patch, objective test result, latency and total cost | + +This is a proposed evaluation record, not an input schema accepted by the supplied +harness. Call `code.symbol`, `code.callers`, `code.impact`, `code.search` or `code.ask` +for the corresponding labelled cases using the +[operation reference](../reference/operations/code.md). Read each result's uncertainty: +an incomplete graph should not be graded as an exhaustive caller set, and changed or +missing watermarks leave a freshness comparison unverified. Keep stale, failed and +permission-denied requests visible alongside successful retrieval. + +For task-level comparisons and a CTO decision scorecard, use the +[paired agent-platform evaluation](../guides/agent-platforms.md#give-a-cto-an-evaluation-they-can-check). +It includes application, hook and model-selected calls, zero-use tasks, citation checks, +objective tests, construction cost and uncertainty. That procedure supplies no measured +result until you run and report it on the chosen workload. + ### Keep the comparison fair - Use the same model, prompt, data snapshot and question order in paired arms. diff --git a/docs/guides/agent-platforms.md b/docs/guides/agent-platforms.md new file mode 100644 index 0000000000..354f6de621 --- /dev/null +++ b/docs/guides/agent-platforms.md @@ -0,0 +1,174 @@ +# Offer Loom in an agent platform or factory + +An agent platform or factory can offer Loom as an optional shared memory and code +retrieval service through MCP, HTTP, or the Python and TypeScript SDKs. Start with a +running [Loom deployment](../operations/deployment.md), credentials for the intended +namespaces, and an explicit choice of which application stages call it. + +## Choose what the integration owns + +Keep the framework's execution state and Loom's shared knowledge distinct. A framework +can retain its conversation, checkpoint and orchestration state while Loom holds facts +or documents that other agents and later sessions need. A native adapter can also put +the state its interface supports in Loom; each +[framework adapter](../features/integrations/frameworks.md) documents that boundary +and its optional dependencies. Installing an adapter does not make every framework +memory operation use Loom. + +Choose an interface for the application you deliver: + +| Integration | Application responsibility | Starting point | +| ------------------------ | ------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------- | +| MCP | Configure the endpoint, token, tool listing and model's tool loop | [MCP clients](../features/integrations/mcp-clients.md) | +| Native framework adapter | Install the matching dependency and bind the supported interface | [Frameworks and orchestrators](../features/integrations/frameworks.md) | +| Direct SDK or HTTP calls | Await recall, validation or recording at defined stages; handle errors and abstention | [Agent integration](agent-integration.md) | +| Model-selected SDK tools | Supply definitions, execute selected calls and return their results to the model | [Code tools](code-tools.md) and [SDK tool loops](../reference/python-sdk.md) | + +For a factory, place retrieval before planning or implementation when its result is +required. Let investigation tools answer additional questions during the run. Record +the outcome after the task's own checks. The existing +[invocation paths](agent-integration.md#who-invokes-loom) distinguish application calls, +plugin hooks and model choices; no particular factory controller is required. + +## Call Loom explicitly from an SDK + +Install the [Python SDK](../reference/python-sdk.md) and set `LOOM_BASE_URL` to the +deployment and `LOOM_API_KEY` to a token with read access to `team-context`. Replace +the example organization and namespace with scopes that token permits: + +```python +import asyncio +import os + +from loom import Agent + + +async def main() -> None: + async with Agent( + org="example", + namespace="team-context", + agent="triage-worker", + base_url=os.environ["LOOM_BASE_URL"], + api_key=os.environ["LOOM_API_KEY"], + ) as agent: + result = await agent.call( + "memory.search", + agent="*", + query="Which checks are required before a change?", + top_k=3, + ) + print(result) + + +asyncio.run(main()) +``` + +This application call runs without asking a model to select a tool. Inspect the returned +hits or `abstain` before adding evidence to the prompt; an abstention is an expected +outcome when no memory qualifies. Authentication or transport errors need their own +handling. `agent="*"` searches across writers within the token's permitted scope; +it does not widen namespace or user access. The [memory operation reference](../reference/operations/memory.md#memorysearch) +defines the response. Use `call_with_meta` when recording the request ID and timings +([request correlation](../features/integrations/index.md#joining-a-request-across-systems)). + +For code investigation, use the existing Python and TypeScript +[`loom_code_lookup` examples](code-tools.md). The application fixes the code namespace; +the model chooses `symbol`, `search`, `callers` or `impact`. The model's ordinary tool +loop must still dispatch the selected call. An application can instead call the +[code operations](../reference/operations/code.md) directly by their dotted names. + +## Diagnose agents that never call the tools + +Check a representative task from connection through use of evidence: + +1. Confirm that the client can discover the intended tool and that the token can read + its namespace. A successful connection alone does not test either condition. +2. Make one explicit, relevant call. Check for a usable result, abstention, denied access + or a provider error before changing the model's instructions. +3. Check whether the running client supplied tool definitions and executed selected + calls. A raw model request does not run coding-client lifecycle hooks. +4. Give the model a short policy saying when to use each tool and when existing evidence + is sufficient. Reuse the [coding-agent policy and diagnostics](../features/integrations/claude-code.md#claude-does-not-call-loom-tools). +5. Record application calls, hook retrieval and model-selected calls separately. Confirm + that the returned evidence reached the model. Count errors, unused results and + zero-call tasks as outcomes, then [evaluate answer quality](../getting-started/evaluate.md). + +If a stage must retrieve or validate something, enforce that dependency in application +code. Instructions guide discretionary tool use; they do not enforce a required gate. + +## Give a CTO an evaluation they can check + +Evaluate the infrastructure decision on the platform's own repositories and tasks. +The [benchmark coverage map](../benchmarks/coverage.md) separates published results from +harnesses without published measurements. Follow its links to the committed artifacts, +models, datasets, reproduction commands and limitations. A published retrieval score +does not establish a task-success or cost gain for a different agent platform. + +Use the existing [evaluation guide](../getting-started/evaluate.md) for indexing, +labelled retrieval cases and deployment smoke tests. Then write a paired protocol +before collecting results: + +1. Freeze repository commits, task inputs, models, prompts, test commands and execution + budgets. Include triage, design and implementation tasks, unanswerable questions, + changed code, permission-denied cases and tasks needing no shared context. Label + expected source ranges or caller sets without exposing reference answers to the agent. +2. Run a baseline using the platform's usual checkout, framework memory and local tools. + Run the treatment with the same inputs and access plus the intended Loom integration. + Keep reference fixes and held-out tests outside both agent checkouts. Use fresh + sessions; prevent one arm's writes from teaching the other arm evaluation answers. +3. Record each task's actual calls, request IDs, outcomes and evidence delivered to the + model. Separate explicit application calls, hooks and model-selected tools. A run with + no calls remains a non-use outcome. A failed or misconfigured request is an integration + failure; report it separately from a grounded retrieval miss. +4. Score task success with the same objective tests or a fixed human rubric. Check answer + claims and citations against the frozen source, and score retrieval separately from + synthesis. Review false answers, stale lessons, denied reads and appropriate abstention. +5. Measure end-to-end wall time and model tokens for both arms. Include tool-definition + overhead, failed requests, retries, Loom's provider work and validation time. Report + indexing, storage and operation costs separately from per-task cost. Mark unavailable + usage or prices as unknown; an absent cost field is not a zero-cost result. +6. Report task count, paired outcomes, category results, repeated-run variation and + uncertainty. Keep raw failures and non-use cases. Small samples and mixed outcomes + support a bounded decision or another experiment, not a general accuracy claim. + +Give the decision maker a short scorecard with links to the underlying task records: + +| Decision | Evidence to retain | +| ---------------------------------------- | ------------------------------------------------------------------------------------------------- | +| Are answers or completed tasks better? | Baseline and treatment outputs, objective test results, valid citations, misses and uncertainty | +| Did agents use the intended integration? | Per-task calls and evidence receipt, including non-use and unusable results | +| Is the benefit worth the cost? | Total task latency and model cost, construction cost, operating cost and expected workload volume | +| Can the team operate it? | Allowed and denied reads, provider failures, recovery, retention, erasure and upgrade checks | + +Set acceptance thresholds before running the protocol. No improvement follows merely +from installing Loom or completing this procedure. Real model and indexing providers +can incur charges; the protocol is a proposed evaluation, not a measured result. + +## Deliver a customer deployment + +A platform provider can integrate a customer-operated Loom service or package the +documented [container or Kubernetes deployment](../operations/deployment.md) with its +application. Agree who operates storage, model providers, backups, upgrades and +credential rotation. Test the same release and adapter dependencies that will ship; +main-branch documentation can describe changes absent from an installed release. Use +the [release history](https://github.com/ClickHouse/loom/releases) and +[supported versions](../operations/supported-versions.md) to choose the matching source. + +Give each service identity only the namespaces and roles it needs +([tokens and grants](tokens-and-grants.md)). Repository indexing credentials and Loom +read authorization are separate controls. A chat connector login does not grant a +native service token access to a repository. Test allowed reads, denied reads, revocation +and [repository-job isolation](../operations/code-index-operations.md#repository-job-sandbox) +before delivery. Set retention and +[erasure](../features/memory/retention-and-erasure.md) requirements for shared content. + +Make optional integration behavior explicit in the platform: whether a customer has +enabled it, which stages depend on it, and what users see when it is unavailable. Treat +required validation failures as failures; report omitted optional context rather than +silently claiming it was used. Evaluate the deployment with the customer's own tasks +and [published evidence](../benchmarks/coverage.md). + +These are technical delivery options. They do not establish a partner program, joint +delivery commitment, support agreement, SLA, pricing or commercial terms. Those terms, +including who owns customer support and escalation, require a separate agreement with +the parties involved. diff --git a/docs/guides/repository-questions.md b/docs/guides/repository-questions.md index 6c7eaddfc5..116b6d9fb4 100644 --- a/docs/guides/repository-questions.md +++ b/docs/guides/repository-questions.md @@ -4,6 +4,45 @@ Use `code.search` to retrieve source and documentation, and `code.ask` to produc cited answer from that evidence. Both can serve a reader who has no local checkout. The repository must be indexed in the code namespace passed to the request. +## Serve triage agents, design assistants and team chatbots + +Run the assistant against the indexed repository, with a Loom token that can read its +code namespace. The application supplies the repository scope; the user supplies the +question. Git fetch credentials used to build the index do not grant a chatbot's users +access to it. Follow the [repository access rules](../features/code-index/repositories.md#who-can-read) +and preserve each caller's authorized scope. + +| Assistant | Evidence to request | Result to check | +| ---------------- | ------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------------------- | +| Triage agent | `code.search` for the symptom, then `code.symbol` and `code.callers` for the relevant definition | Cited implementation and callers that explain the reported path | +| Design assistant | `code.impact` for a proposed symbol change and [linked team decisions](../use-cases/memory-linked-to-code.md) | Affected definitions, test mappings and lessons with their freshness state | +| Team chatbot | `code.ask` for a cited answer, or `code.search` for the application's own reader | Source ranges supporting each claim, or an explicit abstention | + +For example, replace the organization, namespace and question with those of your indexed +repository. Server-side synthesis needs a configured real LLM: + +```sh +curl --fail-with-body -X POST "$LOOM_BASE_URL/v1/code.ask" \ + -H "Authorization: Bearer $LOOM_API_KEY" \ + -H 'Content-Type: application/json' \ + --data '{"org":"example","namespace":"code-example","question":"Where are incoming requests authorized?","synthesize":true}' +``` + +A grounded answer returns `answer` and `citations`; an unanswerable question can return +`abstain` and `abstain_reason`. Inspect the completeness signals below before displaying +an answer. To retrieve evidence without server-side synthesis, pass `synthesize: false`; +`answer` is empty and `citations` holds the retrieved units. This does not disable any +embedding or reranking provider used by retrieval. The +[code operation reference](../reference/operations/code.md#codeask) gives the full contract. + +The answer concerns the indexed branch, not an uncommitted local checkout. Check +[freshness](../features/code-index/freshness.md), [language and compiler-tier support](../features/code-index/overview.md#language-support), +and returned truncation or lower-bound metadata. An empty caller list does not prove +there are no runtime callers, and an impact test mapping does not mean a test ran. +Use the [SDK intent table](code-tools.md#choose-evidence-by-intent) to expose bounded +lookups to a model, and [evaluate on your own repository](../getting-started/evaluate.md#code-intelligence) +before relying on its answers. + ## Calibrate the embedding model After indexing, run `code.calibrate` as an administrator for the indexed namespace. @@ -85,6 +124,12 @@ is a separate result. Keep both measures when evaluating a repository assistant. ## Using retrieval inside a coding agent +An existing checkout already provides local files and tools. Loom can add remote +cross-file relationships, cited retrieval and shared lessons that are not present in +that checkout. Whether these reduce work or improve answers depends on the task and +index coverage. Use local tools for uncommitted files and insufficient or unsupported +indexed results; check the indexed commit before treating evidence as current. + For a coding agent, also inspect whether the model requested tools and received useful results. A connected MCP server does not establish tool adoption. Hook-supplied evidence and model-selected lookups are separate paths; report them separately. diff --git a/docs/index.md b/docs/index.md index 8be60892b8..dfa85e1a23 100644 --- a/docs/index.md +++ b/docs/index.md @@ -110,6 +110,7 @@ client sees one the server no longer reads. ## Guides [Agent integration](guides/agent-integration.md) · +[Offer Loom in an agent platform or factory](guides/agent-platforms.md) · [Headless bots](guides/headless-bots.md) · [Tokens and grants](guides/tokens-and-grants.md) · [Trace and resume long runs](guides/trace-and-resume.md) · @@ -146,6 +147,7 @@ client sees one the server no longer reads. [Overview](features/code-index/overview.md) · [Repositories](features/code-index/repositories.md) · +[Repository Q&A for agents and chatbots](guides/repository-questions.md) · [Freshness](features/code-index/freshness.md) · [History and PRs](features/code-index/history-and-prs.md) · [Operations](features/code-index/operations.md) · diff --git a/docs/use-cases/memory-linked-to-code.md b/docs/use-cases/memory-linked-to-code.md index cc3da089b1..87b043d659 100644 --- a/docs/use-cases/memory-linked-to-code.md +++ b/docs/use-cases/memory-linked-to-code.md @@ -1,7 +1,8 @@ # Memory linked to code for coding agents Loom links coding agents' memories to the functions, classes and files they name, follows -them through refactors, and flags a memory when its code changes. By default, search leaves +them through refactors, and flags a memory when its code changes. Team decisions and notes +saved as memories use the same links. By default, search leaves out a memory whose code is all gone unless a person pinned, locked or verified it, and every read that serves a lesson says when its code changed or is gone. It runs over MCP, HTTP and the SDKs on the repositories a self-hosted Loom indexes (the @@ -82,6 +83,37 @@ writes at least 60 seconds old and passes run every 60 seconds, so once trunk is `code.symbol` on that definition returns the lesson under `lessons` within about two minutes. +### Save and retrieve a linked team decision + +Replace the repository tag, symbol and claim with a real, verified lesson from your +indexed repository. The token needs write access to `team-context` and read access to +`code-example`. This example saves a memory outside the code namespace, so the symbol +read explicitly names its memory namespace: + +```sh +curl --fail-with-body -X POST "$LOOM_BASE_URL/v1/memory.set" \ + -H "Authorization: Bearer $LOOM_API_KEY" \ + -H 'Content-Type: application/json' \ + --data '{"org":"example","namespace":"team-context","agent":"design-assistant","memory_key":"design/retry-tests","value":"Changes to `pkg.Reader.fetch_page` require retry tests.","tags":["repo:"]}' +``` + +Check the write receipt before claiming it was saved +([write outcomes](../guides/agent-integration.md#4-read-the-writes-answer)). After the +link worker has processed it, request the actual indexed definition: + +```sh +curl --fail-with-body -X POST "$LOOM_BASE_URL/v1/code.symbol" \ + -H "Authorization: Bearer $LOOM_API_KEY" \ + -H 'Content-Type: application/json' \ + --data '{"org":"example","namespace":"code-example","name":"pkg.Reader.fetch_page","include_lessons":true,"lesson_namespaces":["team-context"]}' +``` + +Look for the saved memory under `lessons` and inspect `code_state`. A saved memory is +not proof that a link resolved. An ambiguous name, pending indexing or link work, +permissions and response caps can prevent it appearing; use `memory.get` to inspect +the memory's own `code` links. A tag associates content with a repository; it grants +no read access. + ## Check a lesson after a code edit On the repository's indexed trunk, save a lesson such as `` `fetch_page()` retries on code 115 `` @@ -118,4 +150,19 @@ tool selection are not tested. For removal, held-memory exceptions and model-backed revision, see the [link policies](../concepts/memory.md#memories-linked-to-code). +To list review work for the example memory, call the read-only integrity report: + +```sh +curl --fail-with-body -X POST "$LOOM_BASE_URL/v1/namespace.validate" \ + -H "Authorization: Bearer $LOOM_API_KEY" \ + -H 'Content-Type: application/json' \ + --data '{"org":"example","namespace":"team-context","keys":["design/retry-tests"]}' +``` + +Inspect `code_links_stale` findings and any skipped checks. A report with unavailable +evidence is not a freshness proof. Review the current source and revise the same memory +key when the decision changes; +[history and provenance](../features/memory/history-and-provenance.md) retain the prior +versions. A code change alone does not authorize rewriting the decision. + Some ops on this page are marked experimental on the [status page](../reference/status.md). diff --git a/llms.txt b/llms.txt index 8c958c7a15..26f00cc810 100644 --- a/llms.txt +++ b/llms.txt @@ -88,6 +88,7 @@ Start with [the docs map](docs/index.md). To install Loom, a coding agent reads ## Guides - [Integrate Loom into an agent](docs/guides/agent-integration.md): This guide covers what an agent sends to Loom and how it reads the answers: the client surface, the identity, what to write, and how to read. +- [Offer Loom in an agent platform or factory](docs/guides/agent-platforms.md): An agent platform or factory can offer Loom as an optional shared memory and code retrieval service through MCP, HTTP, or the Python and TypeScript SDKs. - [Code tools for model integrations](docs/guides/code-tools.md): An indexed code namespace and a token with read access let an SDK tool loop investigate definitions, behavior and relationships with `loom_code_lookup`. - [Debug a request](docs/guides/debugging.md): Loom tags everything one request does with the same id, so tracing a request is a join. - [Customize the extraction pipeline](docs/guides/extraction-pipeline.md): `memory.set_from_messages` runs its transcript through an ordered list of steps that share one state. From 8bfd16c9a634627d0c7fa35fff9bceb9fded7048 Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 21:50:41 +0000 Subject: [PATCH 2/6] Clarify cached answers and repository listing metadata Document the existing cache and access metadata contracts in the API sources, generated references and agent examples. Signed-off-by: zlareb1 --- docs/guides/repository-questions.md | 6 ++++-- docs/operations/code-index-operations.md | 2 +- docs/reference/operations.md | 2 +- docs/reference/operations/code.md | 6 +++--- sdk/python/loom/ops.py | 4 ++-- server/code/api/models_repos.py | 20 +++++++++----------- server/code/api/models_search.py | 10 +++------- 7 files changed, 23 insertions(+), 27 deletions(-) diff --git a/docs/guides/repository-questions.md b/docs/guides/repository-questions.md index 116b6d9fb4..f6564b3237 100644 --- a/docs/guides/repository-questions.md +++ b/docs/guides/repository-questions.md @@ -30,8 +30,10 @@ curl --fail-with-body -X POST "$LOOM_BASE_URL/v1/code.ask" \ A grounded answer returns `answer` and `citations`; an unanswerable question can return `abstain` and `abstain_reason`. Inspect the completeness signals below before displaying -an answer. To retrieve evidence without server-side synthesis, pass `synthesize: false`; -`answer` is empty and `citations` holds the retrieved units. This does not disable any +an answer. For an uncached retrieval-only response, pass both `synthesize: false` and +`no_cache: true`; `answer` is empty and `citations` holds the retrieved units. With cache +lookup enabled, a matching cached synthesized answer can still be returned. These flags +do not disable any embedding or reranking provider used by retrieval. The [code operation reference](../reference/operations/code.md#codeask) gives the full contract. diff --git a/docs/operations/code-index-operations.md b/docs/operations/code-index-operations.md index f0f290dfd8..0b412bd833 100644 --- a/docs/operations/code-index-operations.md +++ b/docs/operations/code-index-operations.md @@ -779,7 +779,7 @@ The setting also resumes the git jobs of entries that members added before `LOOM_CODE_SELF_SERVICE_PLATFORM_ADMIN_ONLY` was set. They keep the remotes their adders named and are cloned and fetched the same way. Before turning it on, list the branches in the registry (the active rows of `loom.code_repos` whose `row_kind` is not `access`; -`code.repos` lists only what its caller may read) and have those a platform admin would +`code.repos` is caller-filtered and can include request-only metadata) and have those a platform admin would not add removed with `code.repo.remove`. `/v1/health` and `loom doctor` keep reporting `repo_jobs_unsandboxed` as a warning, with diff --git a/docs/reference/operations.md b/docs/reference/operations.md index da9c8cdd75..7a4fd37b7b 100644 --- a/docs/reference/operations.md +++ b/docs/reference/operations.md @@ -224,7 +224,7 @@ Fields for every `code` op: [operations/code.md](operations/code.md). | [`code.repo.remove`](operations/code.md#codereporemove) | Retire a repository index a person added. | admin | idempotent | yes | yes | | [`code.repo.set_access`](operations/code.md#codereposet_access) | A maintainer sets a repository's access — its classification (`standard` \| `restricted`) and join policy (`open` \| `approval` \| `closed`). | admin | idempotent | no | yes | | [`code.repo.set_source`](operations/code.md#codereposet_source) | A maintainer sets the trackers this repository reads (records of issues, PRs, tests and incidents registered with `tracker.source.add`), in order. | admin | idempotent | no | yes | -| [`code.repos`](operations/code.md#coderepos) | Lists the code namespaces (repo@branch entries) configured on this deployment that the caller is authorized to read, across orgs — each row carries its org. | viewer | read | yes | | +| [`code.repos`](operations/code.md#coderepos) | List readable repositories and permitted access-request metadata. | viewer | read | yes | | | [`code.review_brief`](operations/code.md#codereview_brief) | A merged PR's review brief: the point read of what the ingest built at merge — touched symbols with callers and fan-in, coverage verdicts, setting facts, regression history, blame of the deleted… | viewer | read | no | | | [`code.scip_ingest`](operations/code.md#codescip_ingest) | Accept one part of a SCIP index upload (compiler-tier symbols and references) for a batch. | writer | idempotent | no | yes | | [`code.scip_status`](operations/code.md#codescip_status) | Report the progress of a SCIP upload batch: parts received and apply status. | viewer | read | no | | diff --git a/docs/reference/operations/code.md b/docs/reference/operations/code.md index 97032eba01..d9c8887c36 100644 --- a/docs/reference/operations/code.md +++ b/docs/reference/operations/code.md @@ -22,9 +22,9 @@ Ask a QUESTION about this codebase and get a grounded answer with [file:line] ci | `user_id` | `str` | `'-'` | The end user the row belongs to; `'-'` for none. | | `consumer` | `str` | `''` | trace attribution: which integration is calling (clickgap\|claude-code\|…) | | `question` | `str` | required | The question to answer about the indexed code (grounded on a code.search of it). | -| `synthesize` | `bool` | `True` | False = retrieval-only (agent synthesizes); True = grounded server-side answer with citations, memoized across agents and invalidated when any cited file changes at merge. | +| `synthesize` | `bool` | `True` | False disables fresh synthesis; use no_cache=True to exclude cached answers. | | `rerank` | `str` | `''` | Reranker for the retrieval: "" = the deployment default (LOOM_CODE_SEARCH_RERANK_DEFAULT), "none" = off for this request, "builtin:openai" / "builtin:cohere", or the name of an endpoint in LOOM_RERANK_ENDPOINTS_JSON (not a URL). | -| `no_cache` | `bool` | `False` | Skip the cache lookup and synthesize fresh (the fresh answer still refreshes the cache entry). | +| `no_cache` | `bool` | `False` | Skip answer and pre-synthesis prompt-cache lookups; fresh synthesis, when enabled, still refreshes the answer cache. | | `token_budget` | `int \| None` | `None` | Retrieval context handed to synthesis, in tokens. | | Output field | Type | Meaning | @@ -1308,7 +1308,7 @@ A maintainer sets the trackers this repository reads (records of issues, PRs, te ## `code.repos` -Lists the code namespaces (repo@branch entries) configured on this deployment that the caller is authorized to read, across orgs — each row carries its org. +List readable repositories and permitted access-request metadata. `POST /v1/code.repos` · MCP `loom_code_repos` · role `viewer` · read diff --git a/sdk/python/loom/ops.py b/sdk/python/loom/ops.py index fd1913dc09..85b1b04eaf 100644 --- a/sdk/python/loom/ops.py +++ b/sdk/python/loom/ops.py @@ -7337,7 +7337,7 @@ class CodeRepoSetSourceOutput(TypedDict, total=False): class CodeReposInput(TypedDict, total=False): - "Lists the code namespaces (repo@branch entries) configured on this" + "List readable repositories and permitted access-request metadata." org: NotRequired[str] namespace: NotRequired[str] @@ -7350,7 +7350,7 @@ class CodeReposInput(TypedDict, total=False): class CodeReposBody(TypedDict, total=False): - "Lists the code namespaces (repo@branch entries) configured on this" + "List readable repositories and permitted access-request metadata." org: NotRequired[str] namespace: NotRequired[str] diff --git a/server/code/api/models_repos.py b/server/code/api/models_repos.py index 6ac41fc2d2..6c01e76d84 100644 --- a/server/code/api/models_repos.py +++ b/server/code/api/models_repos.py @@ -15,17 +15,17 @@ _GIT_PROXY_IDENTITY_OR_EMPTY = f"^(?:{GIT_PROXY_IDENTITY.pattern})?$" # ============================================================ -# code.repos — every configured repo the caller may read +# code.repos — readable repositories and permitted access metadata # ============================================================ class CodeReposInput(_CodeIdentity): - """Lists the code namespaces (repo@branch entries) configured on this - deployment that the caller is authorized to read, across orgs — each row - carries its org. Transport authz targets the default code namespace when - there is one to use; the handler then checks each entry separately, so - unreadable entries are omitted, never 403'd, and the dashboard renders an - empty state.""" + """List readable repositories and permitted access-request metadata. + + Each row names its organization. Rows with readable=False expose namespace + metadata without populated index counts or watermarks; other inaccessible + entries are omitted. + """ # A caller standing in one checkout (the plugin's code shunt, from the origin # remote) names it, and the answer is that repository's entries only — one status @@ -70,10 +70,8 @@ class CodeRepoSummary(ProjectedRow): indexed_at: str = "" # git_head watermark's updated_at last_refresh_at: str = "" counts: dict[str, int] = Field(default_factory=dict) - # False only for org-admin-shaped callers (flat ADMIN+) who hold no grant - # on the entry: they see it exists so they can adopt it (bootstrap of an - # auto-registered, ownerless code namespace). Everyone else never sees - # entries they cannot read. + # Whether the caller may read code; False rows expose permitted join/adoption + # metadata without populated index counts or watermarks. readable: bool = True # How the namespace is joined and where the caller stands, so the Code view can # render Join / Request access / Cancel without a second call. `my_status` is diff --git a/server/code/api/models_search.py b/server/code/api/models_search.py index 67d01dc2b8..0fd70fadd9 100644 --- a/server/code/api/models_search.py +++ b/server/code/api/models_search.py @@ -306,18 +306,14 @@ def _drop_empty_fanout(self, handler: SerializerFunctionWrapHandler) -> dict[str class CodeAskInput(_CodeIdentity): # The question to answer about the indexed code (grounded on a code.search of it). question: str = Field(min_length=1, max_length=2000) - # False = retrieval-only (agent synthesizes); True = grounded server-side - # answer with citations, memoized across agents and invalidated when any - # cited file changes at merge. + # False disables fresh synthesis; use no_cache=True to exclude cached answers. synthesize: bool = True # Reranker for the retrieval: "" = the deployment default # (LOOM_CODE_SEARCH_RERANK_DEFAULT), "none" = off for this request, "builtin:openai" / # "builtin:cohere", or the name of an endpoint in LOOM_RERANK_ENDPOINTS_JSON (not a URL). rerank: RerankName = "" - # Skip the cache lookup and synthesize fresh (the fresh answer still - # refreshes the cache entry). For eval and for agents that suspect a - # stale answer. Also skips the exact-prompt cache of the pre-synthesis - # LLM calls. + # Skip answer and pre-synthesis prompt-cache lookups; fresh synthesis, when + # enabled, still refreshes the answer cache. no_cache: bool = False # Retrieval context handed to synthesis, in tokens. None = the # deployment default (settings.code_ask_token_budget). From 4db12117df90fb8ad674f4f7a9435702b3780c82 Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 21:59:13 +0000 Subject: [PATCH 3/6] Align code response descriptions with retrieval and listing behavior Signed-off-by: zlareb1 --- docs/guides/tokens-and-grants.md | 5 ++++- docs/reference/operations/code.md | 6 +++--- server/code/api/models_repos.py | 4 ++-- server/code/api/models_search.py | 4 ++-- 4 files changed, 11 insertions(+), 8 deletions(-) diff --git a/docs/guides/tokens-and-grants.md b/docs/guides/tokens-and-grants.md index ee3cc8bb73..7a72eff81f 100644 --- a/docs/guides/tokens-and-grants.md +++ b/docs/guides/tokens-and-grants.md @@ -477,7 +477,10 @@ refusal is the generic "cannot access" message, so it does not reveal whether th exists. If the classification cannot be read, access is denied. - Search fan-out, context packs and org-wide rollups skip the restricted namespaces the - caller cannot read, and `code.repos` hides them. + caller cannot read. `code.repos` can return permitted join, access-request or adoption + metadata with `readable: false`; those rows omit populated index counts and watermarks + and do not authorize code reads. Other inaccessible entries are omitted. See + [repository listing and access](../features/code-index/repositories.md#who-can-read). - Each request admitted to the restricted namespace it names writes a `namespace.restricted_access` audit row before the answer (`LOOM_RESTRICTED_AUDIT_INLINE=false` writes it beside the request instead, diff --git a/docs/reference/operations/code.md b/docs/reference/operations/code.md index d9c8887c36..5820166433 100644 --- a/docs/reference/operations/code.md +++ b/docs/reference/operations/code.md @@ -29,12 +29,12 @@ Ask a QUESTION about this codebase and get a grounded answer with [file:line] ci | Output field | Type | Meaning | | --- | --- | --- | -| `answer` | `str` | The grounded answer text; "" on an abstain or when synthesize=false. | +| `answer` | `str` | The grounded answer text; empty on abstention or an uncached retrieval-only call. | | `partial` | `bool` | True when retrieval or provider limits prevent a complete response. | | `partial_reason` | `str` | Sanitized reason for an incomplete response; empty when complete. | | `note` | `str` | Caller-facing warning about missing retrieval or synthesis. | | `coverage` | `Literal['', 'full', 'partial', 'none']` | Synthesis self-assessment, not verified correctness; empty on cached answers. | -| `citations` | `list[dict[str, Any]]` | The units the answer used (or was shown when none is cited): {path, start_line, end_line, symbol, unit_id} each; the retrieved units when synthesize=false. | +| `citations` | `list[dict[str, Any]]` | The units the answer used (or was shown when none is cited): {path, start_line, end_line, symbol, unit_id} each; retrieved units on an uncached retrieval-only call. | | `source` | `Literal['cached', 'synthesized', 'abstained']` | cached = served from the answer cache \| synthesized = answered now (or retrieval-only when synthesize=false) \| abstained = no grounded answer (`abstain_reason` says why). | | `cache_similarity` | `float` | Cosine between this question and the served cached question; 0.0 unless source=cached. | | `cache` | `str` | What the answer cache did. hit (a cached answer whose key, identifiers and files all matched) \| miss \| stale (the nearest entry's files, prompt version or age invalidated it) \| guarded (the nearest cached question names different… | @@ -1325,7 +1325,7 @@ List readable repositories and permitted access-request metadata. | Output field | Type | Meaning | | --- | --- | --- | -| `repos` | `list[CodeRepoSummary]` | One row per entry the caller reads (plus unreadable rows an org admin may adopt), narrowed to the `repo` / `repos` match when either was given; [] when none. | +| `repos` | `list[CodeRepoSummary]` | Readable entries and permitted request-only or adoption metadata, narrowed to the `repo` / `repos` match when either was given; [] when none. | ## `code.review_brief` diff --git a/server/code/api/models_repos.py b/server/code/api/models_repos.py index 6c01e76d84..51abda2e6b 100644 --- a/server/code/api/models_repos.py +++ b/server/code/api/models_repos.py @@ -136,8 +136,8 @@ class CodeRepoSummary(ProjectedRow): class CodeReposOutput(BaseModel): - # One row per entry the caller reads (plus unreadable rows an org admin may adopt), - # narrowed to the `repo` / `repos` match when either was given; [] when none. + # Readable entries and permitted request-only or adoption metadata, narrowed to + # the `repo` / `repos` match when either was given; [] when none. repos: list[CodeRepoSummary] diff --git a/server/code/api/models_search.py b/server/code/api/models_search.py index 0fd70fadd9..9cbfedb13b 100644 --- a/server/code/api/models_search.py +++ b/server/code/api/models_search.py @@ -321,7 +321,7 @@ class CodeAskInput(_CodeIdentity): class CodeAskOutput(BaseModel): - # The grounded answer text; "" on an abstain or when synthesize=false. + # The grounded answer text; empty on abstention or an uncached retrieval-only call. answer: str = "" # True when retrieval or provider limits prevent a complete response. partial: bool = False @@ -332,7 +332,7 @@ class CodeAskOutput(BaseModel): # Synthesis self-assessment, not verified correctness; empty on cached answers. coverage: Literal["", "full", "partial", "none"] = "" # The units the answer used (or was shown when none is cited): {path, start_line, - # end_line, symbol, unit_id} each; the retrieved units when synthesize=false. + # end_line, symbol, unit_id} each; retrieved units on an uncached retrieval-only call. citations: list[dict[str, Any]] = Field(default_factory=list) # cached = served from the answer cache | synthesized = answered now (or retrieval-only # when synthesize=false) | abstained = no grounded answer (`abstain_reason` says why). From d6675819a9447ce97f5f21e1b72747c8b1a1cd7b Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 22:02:03 +0000 Subject: [PATCH 4/6] Clarify provider setup and failure diagnostics Signed-off-by: zlareb1 --- docs/architecture/llm-architecture.md | 55 ++++++++------- docs/getting-started/models-and-keys.md | 94 +++++++++++++++++++++---- 2 files changed, 110 insertions(+), 39 deletions(-) diff --git a/docs/architecture/llm-architecture.md b/docs/architecture/llm-architecture.md index 6d5e11fa70..75a59f7bbf 100644 --- a/docs/architecture/llm-architecture.md +++ b/docs/architecture/llm-architecture.md @@ -1,20 +1,21 @@ # LLM and embedding architecture Loom has two pluggable providers: an embedder (`server/embedding/`) that turns text into -vectors, and an LLM client (`server/llm/`) that completes prompts. Each is chosen once -per process from settings. Each falls back to a deterministic fake when its key is -missing, so a keyless install boots and answers. Operator setup is in +vectors, and an LLM client (`server/llm/`) that completes prompts. Their active +configuration comes from server settings. API-key providers can fall back to a +deterministic fake when their key is missing; native Bedrock uses deployment AWS +credentials and propagates credential failures. Operator setup is in [deployment](../operations/deployment.md); every setting is in the [configuration reference](../reference/configuration.md). ## Goals and non-goals -| Goals | Non-goals | -| --------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | -| Every correctness gate on the write path works without an LLM. | A general LLM gateway. Loom calls models for its own features only. | -| A keyless install boots and says clearly that it runs on fakes. | Provider-agnostic prompts. Prompts are tuned to a model and re-benched on a change. | -| Swapping the embedder does not silently miscalibrate recall. | Mixing vector spaces. Vectors from different models are never compared. | -| Every provider call is attributed to the op or worker pass that made it, with its cost. | Runtime prompt management. Prompts are code. | +| Goals | Non-goals | +| ------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | +| Every correctness gate on the write path works without an LLM. | A general LLM gateway. Loom calls models for its own features only. | +| A keyless install boots and says clearly that it runs on fakes. | Provider-agnostic prompts. Prompts are tuned to a model and re-benched on a change. | +| Swapping the embedder does not silently miscalibrate recall. | Mixing vector spaces. Vectors from different models are never compared. | +| Every provider call is attributed to its op or worker pass, with usage and cost when known. | Runtime prompt management. Prompts are code. | ## Embedders @@ -25,21 +26,25 @@ once per process: 1. A label resolves first. `prefix::` and `ctx[]:` are views over the served embedder and cannot be the served model. -2. `LOOM_EMBEDDING_PROVIDER` (`fake`, `local`, `openai`, `gemini`) when set; any other - value raises. +2. `LOOM_EMBEDDING_PROVIDER` (`fake`, `local`, `openai`, `gemini`, `compatible`, + `bedrock`) when set; any other value raises. 3. Otherwise inferred from `LOOM_EMBEDDING_MODEL`: `fake-ngram*` is fake, `local` or - `local:*` is local, `text-embedding-*` is OpenAI, `gemini*` is Gemini. Any other name - raises `unknown embedding model`. - -| Provider | Module | Model names | Needs | Notes | -| -------- | ---------------------------- | -------------------------------------------------------------------------- | ---------------------------------------- | ------------------------------------------------------------------------- | -| `fake` | `server/embedding/fake.py` | `fake-ngram-` | nothing | Hashed character trigrams, L2-normalised. Deterministic; the dev default. | -| `local` | `server/embedding/local.py` | `local` (`bge-small`), `local:bge-small`, `local:bge-base`, `local:minilm` | the `local-embedder` extra (`fastembed`) | CPU ONNX; 384, 768 and 384 dimensions. No network. | -| `openai` | `server/embedding/openai.py` | `text-embedding-3-small`, `text-embedding-3-large` | `OPENAI_API_KEY` | Requests `dimensions=LOOM_EMBEDDING_DIM`. | -| `gemini` | `server/embedding/gemini.py` | `gemini-embedding-001` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | Requests `outputDimensionality=LOOM_EMBEDDING_DIM` (up to 3072). | - -**Width.** The served width is `LOOM_EMBEDDING_DIM` (default 3072). OpenAI and Gemini are -asked for that width; choose a model that supports it. Local models emit their native width. Boot probes the real + `local:*` is local, `text-embedding-*` is OpenAI, `gemini*` is Gemini and `bedrock:*` + is Bedrock. Any other name raises `unknown embedding model`; a compatible endpoint + needs an explicit provider. + +| Provider | Module | Model names | Needs | Notes | +| ------------ | -------------------------------- | -------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------ | +| `fake` | `server/embedding/fake.py` | `fake-ngram-` | nothing | Hashed character trigrams, L2-normalised. Deterministic; the dev default. | +| `local` | `server/embedding/local.py` | `local` (`bge-small`), `local:bge-small`, `local:bge-base`, `local:minilm` | the `local-embedder` extra (`fastembed`) | CPU ONNX; 384, 768 and 384 dimensions. No network. | +| `openai` | `server/embedding/openai.py` | `text-embedding-3-small`, `text-embedding-3-large` | `OPENAI_API_KEY` | Requests `dimensions=LOOM_EMBEDDING_DIM`. | +| `gemini` | `server/embedding/gemini.py` | `gemini-embedding-001` | `GEMINI_API_KEY` or `GOOGLE_API_KEY` | Requests `outputDimensionality=LOOM_EMBEDDING_DIM` (up to 3072). | +| `compatible` | `server/embedding/compatible.py` | the endpoint's embedding model ID | `LOOM_EMBEDDING_BASE_URL`, `LOOM_EMBEDDING_API_KEY` or configured workload-token authentication | [OpenAI-compatible embeddings](../getting-started/models-and-keys.md#claude-for-the-llm); optional dimensions parameter. | +| `bedrock` | `server/embedding/bedrock.py` | Cohere Embed v4 model or inference-profile ID | deployment AWS credentials | [Native Bedrock](#native-bedrock); no API-key fallback. | + +**Width.** The served width is `LOOM_EMBEDDING_DIM` (default 3072, or 1536 for native +Bedrock when unset). OpenAI and Gemini are asked for that width; choose a model that +supports it. Local models emit their native width. Boot probes the real width (`probe_embedding_dim`); a mismatch shows as `embedding_dim_mismatch` on `/v1/health`. @@ -324,7 +329,9 @@ them. `LOOM_LLM_ANTHROPIC_EFFORT` sends a thinking effort to Claude models. (transport error, retryable status, timeout) or a rejected request (400, 413, 422); 401, 403 and 404 propagate. Each hand-over is logged and counted (`loom.llm.fallback`), and the result names the model that answered. Fallbacks do not chain. -- **Missing key.** The client is the fake (`_build_llm_client`), logged at WARNING. +- **Missing API key.** An OpenAI or Anthropic client without its key uses the fake + (`_build_llm_client`), logged at WARNING, unless configured workload-token + authentication supplies credentials. Native Bedrock credential failures propagate instead. - **One identity refused.** With identity headers set, an answer from a gateway host that `LOOM_INFERENCE_IDENTITY_REFUSALS_JSON` lists is the gateway refusing the call's identity or asking it to wait ([identity headers](#identity-headers)): never retried, never handed to the diff --git a/docs/getting-started/models-and-keys.md b/docs/getting-started/models-and-keys.md index 52104cd19c..5ac61f4d14 100644 --- a/docs/getting-started/models-and-keys.md +++ b/docs/getting-started/models-and-keys.md @@ -43,26 +43,30 @@ keyless providers a degraded-health condition. ## Choose a setup -| What your organization has | Loom LLM | Embeddings | -| ---------------------------------------------- | ------------------------- | ------------------------------------------------------ | -| OpenAI API access | An available OpenAI model | OpenAI; one API key can serve both roles. | -| Anthropic plus an approved embedding provider | An available Claude model | OpenAI, Gemini or a compatible endpoint, with its key. | -| Anthropic API access only | An available Claude model | Add an embedding provider and its API key. | -| No provider key; a model server on the machine | A local model (`local`) | The `local` embedder the image carries. | +| What your organization has | Loom LLM | Embeddings | +| ---------------------------------------------- | ---------------------------------------------- | ---------------------------------------------------------------- | +| OpenAI API access | An available OpenAI model | OpenAI; one API key can serve both roles. | +| Anthropic plus an approved embedding provider | An available Claude model | OpenAI, Gemini or a compatible endpoint, with its key. | +| Anthropic API access only | An available Claude model | Add an embedding provider and its API key. | +| AWS role with Bedrock model access | A permitted Bedrock model or inference profile | Cohere Embed v4 through Bedrock, or another configured embedder. | +| No provider key; a model server on the machine | A local model (`local`) | The `local` embedder the image carries. | **One OpenAI API key is enough for both roles; an Anthropic key is not required.** In Models & keys, choose one main LLM and leave per-job overrides blank. All enabled LLM jobs use that model, including verification and judging. Advanced overrides are optional and require another provider's key only when you explicitly select one. -Choose OpenAI, Google Gemini or an **OpenAI-compatible embedding endpoint**. OpenAI embeddings use +Choose OpenAI, Google Gemini, [Bedrock](#amazon-bedrock) or an +**OpenAI-compatible embedding endpoint**. OpenAI embeddings use an embedding model such as `text-embedding-3-large`, not a GPT chat model. Claude can handle Loom's LLM jobs, but requires a separate embedding provider and key. Anthropic does not provide its own embedding model; Voyage is not a built-in Loom provider. [Anthropic's embedding guidance](https://platform.claude.com/docs/en/build-with-claude/embeddings) -Loom's internal model jobs use provider API keys, not coding-agent CLI logins or -subscription credentials. Your external agent's choice of provider is independent. +Loom's internal model jobs can authenticate with provider API keys, gateway workload +tokens or deployment AWS credentials. Local models can run without provider credentials. +Coding-agent CLI logins and subscriptions do not configure Loom's internal providers; +your external agent's choice of provider is independent. ## Set the server configuration @@ -172,6 +176,37 @@ Changing the endpoint, model or width requires rebuilding existing embeddings. If your organization currently has only Anthropic API access, obtain access to an embedding provider before enabling semantic search. +### Amazon Bedrock + +Native Amazon Bedrock uses the server's AWS SDK credential chain, including deployment +roles, and needs permission to invoke the selected models or inference profiles. The +server image includes the AWS SDK; a source install needs its `aws` extra. Activate the +provider only after every server and worker supports it. AWS credentials stay in the +deployment; they are not entered as model-provider API keys in Models & keys. + +```dotenv +LOOM_LLM_PROVIDER=bedrock +LOOM_LLM_MODEL= +LOOM_BEDROCK_REGION=us-east-1 +LOOM_REQUIRE_REAL_PROVIDERS=true +``` + +Replace the model ID and region with ones your deployment can invoke; there is no +implicit Bedrock model. Keep the current embedder, or configure Cohere Embed v4 +separately, using an available model or inference profile, for example: + +```dotenv +LOOM_EMBEDDING_PROVIDER=bedrock +LOOM_EMBEDDING_MODEL=us.cohere.embed-v4:0 +LOOM_EMBEDDING_DIM=1536 +``` + +Changing the embedding provider requires rebuilding existing vectors. Missing or expired +AWS credentials fail rather than selecting a fake provider. Native Bedrock cannot be +used when an inference gateway is required. See [native Bedrock](../architecture/llm-architecture.md#native-bedrock) +for IAM resources, regional availability, supported dimensions and request limits, and +[per-job settings](#keep-every-llm-job-on-your-chosen-provider) for model overrides. + ### Keep every LLM job on your chosen provider Loom supports different models for different jobs, but you do not need accounts with @@ -307,17 +342,24 @@ setting names the same model. ### Through an inference gateway -A gateway that speaks the OpenAI or Anthropic API in front of the providers -(for budgets, cost attribution or one key for many services) takes Loom's -calls once its API roots point there; the model names stay what the gateway -serves: +An OpenAI-compatible proxy or inference gateway can serve Loom's chat-completions +requests. Select the provider and a model the gateway serves, as well as its API root +and key; a key alone does not change an explicitly configured fake provider: -```bash -LOOM_OPENAI_BASE_URL=https://gateway.example/v1 # chat and embeddings +```dotenv +LOOM_LLM_PROVIDER=openai +LOOM_LLM_MODEL= +LOOM_OPENAI_BASE_URL=https://gateway.example/v1 # chat and OpenAI embeddings LOOM_ANTHROPIC_BASE_URL=https://gateway.example # Messages: /v1/messages OPENAI_API_KEY= ``` +For an Anthropic-format gateway, select `LOOM_LLM_PROVIDER=anthropic`, its model ID +and `ANTHROPIC_API_KEY` instead. Configure embeddings separately using a supported +provider and its model; a chat endpoint does not imply embedding support. Check +[per-job overrides](#keep-every-llm-job-on-your-chosen-provider) too. Both API roots +above must use an allowed host when the gateway is required, even for an unused provider. + Off its default, the Anthropic key is also sent as `Authorization: Bearer`, the scheme gateways take. With a gateway key, require the gateway, so a wrong root can never send prompts and the key anywhere else. A token exchange requires it by itself. @@ -651,3 +693,25 @@ If dimensions change, check the existing schema's vector constraints before appl the new configuration. Preserve your data and follow [maintenance](../operations/maintenance.md#re-embed-and-re-extract) and [code-index operations](../operations/code-index-operations.md). + +### Diagnose silent model or provider failures + +Read the response body, not just the HTTP status: + +```bash +curl -sS "$LOOM_BASE_URL/v1/health" | jq '{status, fake_llm, fake_embedding, providers_degraded, providers, problems}' +``` + +The HTTP status checks ClickHouse availability; it does not prove model access. +`fake_llm` and `fake_embedding` expose configured fakes or missing-key fallbacks. +They can both be false while a real provider fails: inspect `providers_degraded`, each +provider's `failing` and `last_error`, and the fixes in `problems`. + +Provider outcomes are passive, process-local observations of actual calls; polling +health makes no model request. No recorded outcome is not proof of working inference. +A later recorded success clears that provider's failure. The `compatible` embedding +provider does not populate `providers`, so `providers_degraded: false` does not validate +that endpoint. Check its request errors and exercise retrieval. After checking configuration, +use the [end-to-end smoke](../operations/monitoring.md#smoke-test) and try a generated +answer to exercise both providers. The smoke makes model calls and can incur charges. +For a stored memory missing from search, also check the embedding backlog described there. From f93385b019dc0978daa33b08392e717348c7643c Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 22:08:54 +0000 Subject: [PATCH 5/6] Route the conflict judge in the single-provider recipe Signed-off-by: zlareb1 --- docs/getting-started/models-and-keys.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/getting-started/models-and-keys.md b/docs/getting-started/models-and-keys.md index 5ac61f4d14..9dee9989b2 100644 --- a/docs/getting-started/models-and-keys.md +++ b/docs/getting-started/models-and-keys.md @@ -215,6 +215,7 @@ normally provider-specific jobs use your selected model: ```dotenv LOOM_MEMORY_EXTRACT_VERIFY_MODELS=${LOOM_LLM_MODEL} +LOOM_CONFLICT_RESOLVER_MODEL=${LOOM_LLM_MODEL} LOOM_SESSION_BRIEF_LLM_MODEL= LOOM_CODE_BISECT_LLM_RERANK_MODEL= LOOM_TRACKER_DUPCHECK_RERANK_MODEL= @@ -226,6 +227,10 @@ The empty values above inherit the main LLM. This keeps the capture verifier ena an empty verifier list would disable it. Remove any other cross-provider model overrides you added previously, including fallback, synthesis and reranking models. +The conflict resolver can retire superseded memories. Validate a replacement judge +against your own conflicts before automatic resolution; set +`LOOM_MEMORY_CONFLICT_RESOLVER_ENABLED=false` while evaluating it. + These settings matter: some shipped advanced defaults name Claude or OpenAI models explicitly, and an override with no provider key can fall back to the fake client. Using one model across jobs is a convenience configuration, not the configuration behind From a6d8a1279a4a8fe4a4eae3b170610c662b52e0b5 Mon Sep 17 00:00:00 2001 From: zlareb1 Date: Fri, 20 Nov 2026 22:17:01 +0000 Subject: [PATCH 6/6] Describe the dashboard model selection boundary Signed-off-by: zlareb1 --- docs/getting-started/models-and-keys.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/docs/getting-started/models-and-keys.md b/docs/getting-started/models-and-keys.md index 9dee9989b2..423ffa982d 100644 --- a/docs/getting-started/models-and-keys.md +++ b/docs/getting-started/models-and-keys.md @@ -52,9 +52,10 @@ keyless providers a degraded-health condition. | No provider key; a model server on the machine | A local model (`local`) | The `local` embedder the image carries. | **One OpenAI API key is enough for both roles; an Anthropic key is not required.** -In Models & keys, choose one main LLM and leave per-job overrides blank. All enabled -LLM jobs use that model, including verification and judging. Advanced overrides are -optional and require another provider's key only when you explicitly select one. +In Models & keys, choose one main LLM and leave the displayed per-job overrides blank +to use it for those jobs, including capture verification and retrieval judging. +Background settings such as the conflict resolver are configured separately; use the +[per-job recipe](#keep-every-llm-job-on-your-chosen-provider) to route them explicitly. Choose OpenAI, Google Gemini, [Bedrock](#amazon-bedrock) or an **OpenAI-compatible embedding endpoint**. OpenAI embeddings use @@ -74,7 +75,8 @@ your external agent's choice of provider is independent. As a platform administrator, open **Settings → Models & keys** (also available during Setup). Choose your LLM and embedding provider, enter their model IDs and API keys, then click -**Test & apply**. Leave per-job overrides blank to use one LLM for every enabled job. +**Test & apply**. Blank per-job overrides apply the main LLM to the displayed jobs; +the [conflict resolver](#keep-every-llm-job-on-your-chosen-provider) keeps its separate setting. Saved dashboard settings override the environment and are encrypted in ClickHouse. **No Docker restart or Compose command is needed.** Key and LLM updates apply live; embedding changes start the rebuild described below. **Use environment