From 065d2fc1f1677d68153dbc35846d3b90d1b6e8a1 Mon Sep 17 00:00:00 2001 From: SohamRatnaparkhi Date: Wed, 23 Sep 2026 14:11:46 +0530 Subject: [PATCH 1/7] docs: unified is the only schema a reader sees; hide the knowledge and memory docs (PRO-1618) Knowledge and memory (split databases) are deprecated. Every page that documents them leaves the navigation, is marked noindex, and opens with a one-line deprecation warning; the files stay reachable by URL. - Navigation: the whole v1 version (the knowledge, memories and recall API), the v2 Split databases page and the Cookbooks tab (every cookbook is written against the split schema) are out of docs.json. Knowledge, Memories and App sources were already out of nav and get the same treatment. Bring your own graph is rewritten to the current contract and joins Concepts; Continuity assurance moves from v1 to Concepts so it stays visible. - One deprecation note, on Databases and collections: knowledge and memory are deprecated, unified is the way to go, and nothing is passed to get it. - Visible pages stop teaching the split schema: no `type`, no documents / app_knowledge / memories, no old /query fields, no build_string, no links to hidden pages. AGENTS.mdx, the plugin pages, the SDK page, Attributes and the context and database endpoint pages are rewritten to the current contract. Endpoint pages whose published spec still renders the corpus selector are unbound from OpenAPI and documented by hand. - /query contract: graph relations carry an optional `timestamp` (epoch seconds), `temporal_details` is omitted when empty, the graph is deduplicated across both lanes and not capped, forceful relations are followed only in thinking mode, and llm_prompt related facts no longer name the lane (`(relevance 0.81)`, no parenthetical when unranked). - Corrections found on the way: the ingest 202 result key is `results[].id`, and GET /context/inspect returns stored content only, so title and attributes come from POST /context/list with `ids`. Signed-off-by: SohamRatnaparkhi Co-Authored-By: Claude Opus 5.5 (1M context) --- AGENTS.mdx | 1693 +++++++++-------- api-reference/endpoint/add-memory.mdx | 3 + api-reference/endpoint/boolean-recall.mdx | 3 + api-reference/endpoint/create-tenant.mdx | 3 + api-reference/endpoint/delete-knowledge.mdx | 3 + api-reference/endpoint/delete-memory.mdx | 3 + api-reference/endpoint/delete-tenant.mdx | 3 + api-reference/endpoint/fetch-content.mdx | 3 + api-reference/endpoint/full-recall.mdx | 3 + api-reference/endpoint/graph-relations.mdx | 3 + api-reference/endpoint/infra-status.mdx | 3 + api-reference/endpoint/ingestion-overview.mdx | 3 + api-reference/endpoint/list-data.mdx | 3 + api-reference/endpoint/list-overview.mdx | 3 + .../endpoint/list-sub-tenant-ids.mdx | 3 + api-reference/endpoint/list-tenant-ids.mdx | 3 + api-reference/endpoint/memories-overview.mdx | 3 + api-reference/endpoint/monitor-tenant.mdx | 3 + api-reference/endpoint/recall-overview.mdx | 3 + api-reference/endpoint/recall-preferences.mdx | 3 + api-reference/endpoint/tenants-overview.mdx | 3 + api-reference/endpoint/upload-knowledge.mdx | 3 + api-reference/endpoint/verify-processing.mdx | 3 + api-reference/error-responses.mdx | 3 + api-reference/index.mdx | 3 + api-reference/sdks.mdx | 3 + .../v2/endpoint/connectors-overview.mdx | 2 +- api-reference/v2/endpoint/create-tenant.mdx | 23 +- .../v2/endpoint/delete-collection.mdx | 6 +- .../v2/endpoint/delete-connector-resource.mdx | 2 +- .../v2/endpoint/delete-connector.mdx | 2 +- api-reference/v2/endpoint/delete-source.mdx | 117 +- api-reference/v2/endpoint/delete-tenant.mdx | 6 +- api-reference/v2/endpoint/fetch-content.mdx | 122 +- api-reference/v2/endpoint/ingest-context.mdx | 933 ++------- api-reference/v2/endpoint/list-documents.mdx | 115 +- api-reference/v2/endpoint/list-tenants.mdx | 20 +- api-reference/v2/endpoint/query-overview.mdx | 14 +- api-reference/v2/endpoint/query.mdx | 137 +- .../v2/endpoint/source-relations.mdx | 41 +- api-reference/v2/endpoint/source-status.mdx | 23 +- .../v2/endpoint/sources-overview.mdx | 19 +- api-reference/v2/endpoint/subgraph.mdx | 44 +- api-reference/v2/endpoint/tenant-stats.mdx | 33 +- api-reference/v2/endpoint/tenant-status.mdx | 32 +- .../v2/endpoint/tenants-overview.mdx | 2 +- api-reference/v2/error-responses.mdx | 61 +- api-reference/v2/index.mdx | 21 +- api-reference/v2/sdks.mdx | 488 +++-- cookbooks/ai-chief-of-staff.mdx | 3 + cookbooks/ai-linkedin-recruiter.mdx | 3 + cookbooks/ai-onboarding-agent.mdx | 3 + cookbooks/ai-travel-planner.mdx | 3 + cookbooks/competitive-intelligence-agent.mdx | 3 + .../cookbook-01-build-cursor-for-docs.mdx | 3 + cookbooks/cookbook-04-build-notion-ai.mdx | 3 + .../cookbook-10-ai-financial-analyst.mdx | 3 + cookbooks/customer-support-agent.mdx | 3 + cookbooks/glean-clone.mdx | 3 + cookbooks/hydradb-cookbook-06.mdx | 3 + cookbooks/index.mdx | 3 + cookbooks/internal-search-perplexity.mdx | 3 + cookbooks/v2/ai-chief-of-staff.mdx | 3 + cookbooks/v2/ai-linkedin-recruiter.mdx | 3 + cookbooks/v2/ai-onboarding-agent.mdx | 3 + cookbooks/v2/ai-travel-planner.mdx | 3 + .../v2/competitive-intelligence-agent.mdx | 3 + .../v2/cookbook-01-build-cursor-for-docs.mdx | 3 + cookbooks/v2/cookbook-04-build-notion-ai.mdx | 3 + .../v2/cookbook-10-ai-financial-analyst.mdx | 3 + cookbooks/v2/customer-support-agent.mdx | 3 + cookbooks/v2/glean-clone.mdx | 3 + cookbooks/v2/hydradb-cookbook-06.mdx | 3 + cookbooks/v2/index.mdx | 3 + cookbooks/v2/internal-search-perplexity.mdx | 3 + docs.json | 163 +- essentials/api-results.mdx | 3 + essentials/app-sources.mdx | 3 + essentials/architecture.mdx | 3 + essentials/context-graphs.mdx | 3 + essentials/knowledge.mdx | 3 + essentials/memories.mdx | 3 + essentials/metadata.mdx | 3 + essentials/multi-tenant.mdx | 3 + essentials/recall.mdx | 3 + essentials/semantic-search.mdx | 3 + essentials/v2/access-control.mdx | 3 +- essentials/v2/api-results.mdx | 66 +- essentials/v2/app-sources.mdx | 3 + essentials/v2/architecture.mdx | 28 +- essentials/v2/attributes.mdx | 726 +++---- essentials/v2/bring-your-own-graph.mdx | 314 ++- essentials/v2/connectors.mdx | 13 +- essentials/v2/context-graphs.mdx | 23 +- essentials/v2/databases-and-collections.mdx | 31 +- essentials/v2/glossary.mdx | 8 - essentials/v2/ingest.mdx | 64 +- essentials/v2/knowledge.mdx | 3 + essentials/v2/memories.mdx | 3 + essentials/v2/query.mdx | 66 +- essentials/v2/semantic-search.mdx | 6 +- essentials/v2/split-databases.mdx | 3 + essentials/v2/webhooks.mdx | 8 +- essentials/webhooks.mdx | 3 + get-started/core-concepts.mdx | 3 + get-started/introduction.mdx | 3 + get-started/quickstart.mdx | 3 + get-started/v2/core-concepts.mdx | 4 +- get-started/v2/introduction.mdx | 6 +- get-started/v2/quickstart.mdx | 3 +- mintlify-hygiene.toml | 74 +- plugins/claude-code.mdx | 171 +- plugins/cli.mdx | 183 +- plugins/mcp.mdx | 160 +- plugins/openclaw.mdx | 112 +- snippets/legacy-line.mdx | 1 - 116 files changed, 2815 insertions(+), 3581 deletions(-) delete mode 100644 snippets/legacy-line.mdx diff --git a/AGENTS.mdx b/AGENTS.mdx index e95e603f..c0b7aab2 100644 --- a/AGENTS.mdx +++ b/AGENTS.mdx @@ -1,22 +1,23 @@ --- title: "HydraDB Agent Integration Guide" -description: "LLM-facing reference for building against the current HydraDB docs, API, SDKs, and cookbooks." +description: "LLM-facing reference for building against the current HydraDB API and SDKs." --- # HydraDB Agent Integration Guide -This document is a self-contained reference designed for AI coding agents. It covers everything needed to understand, install, configure, and integrate HydraDB into any project -- from zero prior knowledge to production-ready usage. +This document is a self-contained reference for AI coding agents. It covers everything needed to understand, install, configure and integrate HydraDB into a project, from zero prior knowledge to production use. + +HydraDB stores **context**: text and conversations that you send as items. It chunks, embeds and enriches every item, extracts entities and relations into a context graph, and answers questions over all of it through one query endpoint that returns ranked chunks, graph paths and a prompt-ready string. ### TL;DR: Critical endpoints -1. **`POST /databases`** (`client.databases.create()`): Provision an isolated database workspace. -2. **`GET /databases/status`** (`client.databases.status()`): Poll until `infra.ready_for_ingestion` is true. -3. **`POST /context/ingest`** (`client.context.ingest()`): Ingest documents, app records, or memories under `type="knowledge"` or `type="memory"`. -4. **`GET /context/status`** (`client.context.status()`): Poll status using `ids` until `indexing_status` is `completed` or `graph_creation`. -5. **`POST /query`** (`client.query()`): Retrieve context via `type: "knowledge"`, `"memory"`, or `"all"`. +1. **`POST /databases`** (`client.databases.create()`): Create an isolated database. A name is all it needs. +2. **`GET /databases/status`** (`client.databases.status()`): Poll until `data.infra.ready_for_ingestion` is `true`. +3. **`POST /context/ingest`** (`client.context.ingest()`): Send a `context` list of items. Each item is either a `text` or a `conversation`. +4. **`GET /context/status`** (`client.context.status()`): Poll with `ids` until `indexing_status` is `graph_creation` or `completed` (searchable), or `errored`. +5. **`POST /query`** (`client.query()`): Ask a question. The response `data` has exactly four keys: `chunks`, `graph`, `forceful_relations` and `llm_prompt`. Inject `llm_prompt` into your model call verbatim. 6. **`POST /feedback`** (`client.feedback.submit()`): Tell us when a query did not give you what you needed, and once the task is done, how the context held up. See [Sending feedback](#sending-feedback). - --- ## 1. Critical rules for LLMs @@ -32,7 +33,7 @@ This document is a self-contained reference designed for AI coding agents. It co ### Response envelope -Core raw HTTP responses (`/databases`, `/context/*`, and `/query`) are wrapped: +Core raw HTTP responses (`/databases`, `/context/*`, `/query` and `/feedback`) are wrapped: ```json { @@ -47,73 +48,64 @@ Core raw HTTP responses (`/databases`, `/context/*`, and `/query`) are wrapped: ``` - Parse core raw HTTP payloads from `data`. -- The SDKs return the full envelope object; read the payload from its `data` field (e.g. `response.data`). On failure they raise a typed exception rather than returning an envelope with `error` set. +- The SDKs return the full envelope object; read the payload from its `data` field (for example `response.data`). On failure they raise a typed exception rather than returning an envelope with `error` set. - Log `meta.request_id` for failed requests. -- `meta` may also carry an optional `deprecation` list (with a `Deprecation: true` response header) when a request uses a legacy `/tenants` route or a deprecated field (`tenant_id`/`sub_tenant_id`, or `sub_tenant_ids` on `/query`). Every entry has `deprecated`, `message`, and `deprecated_since`. `deprecated_field` and `preferred_field` are **optional**: the `sub_tenant_ids` notice on `/query` carries them, the `tenant_id`/`sub_tenant_id` notice does not. Read `message` for the migration, and do not key logic off the two optional fields. It is a non-breaking nudge. The status code is unchanged. Prefer the `/databases` routes and `database`/`collection`/`collections` fields to avoid it. Treat `meta` as an open object: ignore keys you do not recognize. +- `meta` may also carry an optional `deprecation` list (with a `Deprecation: true` response header) when a request uses a deprecated name: `tenant_id` / `sub_tenant_id`, or `sub_tenant_ids` on `/query`. Every entry has `deprecated`, `message` and `deprecated_since`; `deprecated_field` and `preferred_field` are optional. Read `message` for the migration and do not key logic off the optional fields. It is a non-breaking nudge and the status code is unchanged. Use `database`, `collection` and `collections` to avoid it. Treat `meta` as an open object: ignore keys you do not recognize. - Exception: webhook management endpoints (`/webhooks/indexing*`) return their documented response object directly, without the `{ success, data, error, meta }` envelope. ### Core endpoints -| Task | Use this endpoint / SDK method | +| Task | Endpoint · SDK method | |---|---| | Create database | `POST /databases` · `client.databases.create()` | | Check database readiness | `GET /databases/status` · `client.databases.status()` | | List databases | `GET /databases` · `client.databases.list()` | -| List collections | `GET /databases/collections` · TS `client.databases.collections()` / Python `client.databases.collections()` | +| List collections | `GET /databases/collections` · `client.databases.collections()` | | Database stats | `GET /databases/stats` · `client.databases.stats()` | -| Ingest documents, app sources, memories | `POST /context/ingest` · `client.context.ingest()` | +| Ingest text and conversations | `POST /context/ingest` · `client.context.ingest()` | | Check indexing | `GET /context/status` · `client.context.status()` | -| Search knowledge/memories/both | `POST /query` · `client.query()` | +| Search | `POST /query` · `client.query()` | | Report back on query results | `POST /feedback` · `client.feedback.submit()` | -| List sources/memories | `POST /context/list` · `client.context.list()` | -| Inspect source content | `GET /context/inspect` · `client.context.inspect()` | -| Delete sources/memories | `DELETE /context` · `client.context.delete()` | +| List items | `POST /context/list` · `client.context.list()` | +| Read an item's stored content | `GET /context/inspect` · `client.context.inspect()` | +| Delete items | `DELETE /context` · `client.context.delete()` | | Inspect graph relations | `GET /context/relations` · `client.context.relations()` | +| Walk an item's connected items | `GET /context/{id}/subgraph` · `client.context.subgraph()` | +| Edit an indexed item's attributes | `PATCH /context/{id}/metadata` · Python `client.context.update_source_metadata()` / TS `client.context.updateSourceMetadata()` | | Indexing webhooks | `/webhooks/indexing*` | ### Async lifecycle Two operations are asynchronous: -1. **Database creation**: after `POST /databases`, poll `GET /databases/status` until infrastructure is ready. -2. **Ingestion**: after `POST /context/ingest`, poll `GET /context/status` until the content is searchable or fully complete. +1. **Database creation**: after `POST /databases`, poll `GET /databases/status` until `data.infra.ready_for_ingestion` is `true`. +2. **Ingestion**: after `POST /context/ingest`, poll `GET /context/status` until each item is searchable or fully complete. Searchable status: -- `graph_creation`: searchable, graph may still be incomplete. +- `graph_creation`: searchable; the graph may still be incomplete. - `completed`: fully indexed and graphed. Failure status: -- Docs use `errored`; OpenAPI also exposes `failed` in one enum. Treat both as terminal failures. - -### Search parameter naming - -The canonical request field for `POST /query` is `type`: - -- `type: "knowledge"` -- `type: "memory"` -- `type: "all"` +- `errored` is the terminal failure. One OpenAPI enum also names `failed`; treat it the same way if you see it. -Use the same `type` enum across context lifecycle APIs and search. Search additionally accepts `"all"` to query knowledge and memories together. +### Names to use -### Metadata naming - -- `metadata`: schema-aligned, declared at database creation, fast filter path. Sent as a top-level request field on `/context/ingest`. -- `additional_metadata`: canonical free-form metadata; anything not present in the `metadata` schema. Also a top-level request field. -- `document_metadata`: on file uploads only, binds a source `id` to each uploaded file. It is not a metadata envelope - do not nest `metadata` or `additional_metadata` inside it. -- `metadata_filters` top-level keys match the `metadata` schema. -- `metadata_filters.additional_metadata` scopes free-form metadata. +- `database` (formerly `tenant_id`) is the hard isolation boundary. +- `collection` (formerly `sub_tenant_id`) is a logical partition inside a database. `/query` also takes `collections` to search several at once. +- `context_id` is your id for one item. The ingest response reports it as `results[].id`; query chunks carry it as `context_id`. +- `attributes` are declared, filterable fields; `custom_attributes` are free-form and not filterable. See [Attributes guide](#10-attributes-guide). ### Sending feedback -**If a query did not give you what you needed - nothing relevant came back, the top result was wrong, or you had to fall back to another source - please tell us with `POST /feedback`.** You are the only party that knows whether the retrieved context actually answered the question, and that judgement is what tunes retrieval. +**If a query did not give you what you needed (nothing relevant came back, the top result was wrong, or you had to fall back to another source), please tell us with `POST /feedback`.** You are the only party that knows whether the retrieved context actually answered the question, and that judgement is what tunes retrieval. -It is entirely optional, and there is no penalty for skipping it. Send it when you have something specific to report; a report on a query that disappointed you is worth far more than a stream of routine ones. If a query worked well and you want to say so, `rating: "positive"` is welcome too - it tells us what to preserve. +It is entirely optional, and there is no penalty for skipping it. Send it when you have something specific to report; a report on a query that disappointed you is worth far more than a stream of routine ones. If a query worked well and you want to say so, `rating: "positive"` is welcome too: it tells us what to preserve. -**Once you have finished the task, a short report on how the context actually performed is recommended.** The end of the task is when you know something you could not know at retrieval time: whether a chunk that looked plausible turned out to be stale, contradicted by another source, or never used at all. A single report then is worth more than one per query. +**Once you have finished the task, a short report on how the context actually performed is recommended.** The end of the task is when you know something you could not know at retrieval time: whether a chunk that looked plausible turned out to be stale, contradicted by another item, or never used at all. A single report then is worth more than one per query. -Give it context. We see only a `request_id` and whatever prose you send - nothing about what you were doing - so a report that stands on its own is far more useful than one that assumes the session can be reconstructed. Worth a sentence each: +Give it context. We see only a `request_id` and whatever prose you send, nothing about what you were doing, so a report that stands on its own is far more useful than one that assumes the session can be reconstructed. Worth a sentence each: - What the user was actually trying to do. - Which chunks carried the answer, and which you discarded and why. @@ -127,34 +119,34 @@ try: rating="positive", feedback=( "Task: user asked which refund window applies to EU orders. " - "Chunks 1 and 3 (Q3 handbook) carried the answer. Chunk 2 was the " + "Results [1] and [3] (Q3 handbook) carried the answer. Result [2] was the " "2023 policy and was discarded as superseded. No outside sources needed." ), database="acme_corp", ) except Exception: - pass # fire-and-forget, exactly as above + pass # fire-and-forget: a failed report must never reach the user ``` -At the end of a task `rating: "positive"` is as valuable as a negative one, because by then you know which part of the context did the work - and that is what tells us what to keep. +At the end of a task `rating: "positive"` is as valuable as a negative one, because by then you know which part of the context did the work, and that is what tells us what to keep. Rules, when you do send it: -- **Take `request_id` from the query's own response - never invent one.** It is in `meta.request_id`, and in the `X-Request-ID` response header. Send it back **verbatim**: an id you made up, or one from another query, correlates to nothing. +- **Take `request_id` from the query's own response. Never invent one.** It is in `meta.request_id`, and in the `X-Request-ID` response header. Send it back **verbatim**: an id you made up, or one from another query, correlates to nothing. - Set `source: "agent"`. It separates your reports from human ones, which fail in different ways. - Say **what** was wrong, not that something was wrong. "Returned the 2023 policy, current one is in the Q3 handbook" is actionable; "bad results" is not. -- Do not block the user on it. Feedback is fire-and-forget: if the **feedback call itself** fails, drop it and carry on - never retry in a loop, and never surface that error to the user. +- Do not block the user on it. Feedback is fire-and-forget: if the **feedback call itself** fails, drop it and carry on. Never retry in a loop, and never surface that error to the user. - Feedback never changes the query's result. It is a signal, not a correction, and reading it back is not possible. ```python -result = client.query(database="acme_corp", query="What is our refund policy?") +result = client.query(database="acme_corp", collection="support", query="What is our refund policy?") -# ... use result.data.chunks to answer the user ... +# ... answer the user from result.data.llm_prompt ... try: client.feedback.submit( request_id=result.meta.request_id, - feedback="Top chunk was the 2023 policy; the current one is in the Q3 handbook.", + feedback="Top result was the 2023 policy; the current one is in the Q3 handbook.", rating="negative", source="agent", database="acme_corp", @@ -163,11 +155,11 @@ except Exception: pass # fire-and-forget: a failed report must never reach the user ``` -Catch broadly. The point is that **nothing** escapes - narrowing to the SDK error type would still let a timeout or a DNS failure surface an error the user cannot act on, from a call they did not ask for. +Catch broadly. The point is that **nothing** escapes: narrowing to the SDK error type would still let a timeout or a DNS failure surface an error the user cannot act on, from a call they did not ask for. -**"Did not give me what I needed" is not the same as "errored".** A query that returned `200` with unhelpful results is exactly what this endpoint is for. A query that never returned - `4xx`/`5xx`, or the SDK raised - is not: handle the error and move on rather than reporting it. Feedback is a judgement about retrieval quality, and a query that produced no results has no retrieval to judge. Fix the request instead: a `404` means the database name is wrong, a `429` means back off, a `400` means the body was malformed. +**"Did not give me what I needed" is not the same as "errored".** A query that returned `200` with unhelpful results is exactly what this endpoint is for. A query that never returned (`4xx`/`5xx`, or the SDK raised) is not: handle the error and move on rather than reporting it. Feedback is a judgement about retrieval quality, and a query that produced no results has no retrieval to judge. Fix the request instead: a `404` means the database name is wrong, a `429` means back off, a `400` means the body was malformed. -**If you know the right answer, send it as `ground_truth`.** When you are running against a labelled set, or you know which document should have been returned, that is a far stronger signal than a comment - it can be scored without a human reading it. With `ground_truth` present, `feedback` prose is optional: +**If you know the right answer, send it as `ground_truth`.** When you are running against a labelled set, or you know which item should have been returned, that is a far stronger signal than a comment: it can be scored without a human reading it. With `ground_truth` present, `feedback` prose is optional: ```python try: @@ -175,8 +167,8 @@ try: request_id=result.meta.request_id, source="agent", ground_truth={ - "answer": expected_answer, # what a correct response looks like - "source_ids": ["policy_2024"], # sources that actually contain it + "answer": expected_answer, # what a correct response looks like + "source_ids": ["refund-policy"], # the context_ids that actually contain it }, ) except Exception: @@ -185,14 +177,13 @@ except Exception: Send `answer`, `source_ids`, or both. Do **not** guess: only send ground truth you actually have. A fabricated answer key is worse than none, because it is scored as if it were true. -Limit: 100 submissions per minute per organization - far above what reporting only the queries that fell short will ever reach. If you do hit `429`, honour `Retry-After` or simply skip that report - never spin. +Limit: 100 submissions per minute per organization, far above what reporting only the queries that fell short will ever reach. If you do hit `429`, honour `Retry-After` or simply skip that report. Never spin. ### Do not mix scopes accidentally -- `database` (formerly `tenant_id`) is the hard isolation boundary. -- `collection` (formerly `sub_tenant_id`) is a logical partition inside a database, typically user/workspace/team. -- Use the same `collection` on writes and reads. Data written under one collection should not be expected to appear from another. -- Do not use `metadata_filters` as a substitute for `collection`. +- Use the same `collection` on writes and reads. Data written under one collection does not appear in a query that names another. +- `context_id` lookups (`/context/status`, `/context/inspect`, `DELETE /context`) are scoped to a collection: send the collection you ingested into. +- Do not use an `attributes` filter as a substitute for `collection`. Partition by user, team or workspace with collections; filter inside a partition with attributes. --- @@ -200,100 +191,85 @@ Limit: 100 submissions per minute per organization - far above what reporting ### Databases and collections -A database is an isolated workspace. A collection partitions data inside a database. +A database is an isolated workspace. A collection partitions data inside a database. Everything ingested into a collection, text or conversation, from your code or from a connector, is searched together by one query. Recommended patterns: | Use case | Pattern | |---|---| -| B2B SaaS | one `database` per customer; `collection` for workspace/team/user | -| B2C app | one app/customer database; `collection = user_id` | -| Environment isolation | separate `database`s for prod/staging | -| Shared knowledge | database-wide/default collection or consistent shared scope | -| User memories | `collection = user_id` | +| B2B SaaS | one `database` per customer; a `collection` per workspace, team or user | +| B2C app | one database for the app; `collection = user_id` | +| Environment isolation | separate databases for prod and staging | +| Shared company context | a shared collection such as `company`, queried together with the user's collection through `collections` | +| Per-user context (preferences, conversation history) | `collection = user_id` | -### Knowledge +### Context items -Knowledge is shared context: documents, PDFs, Markdown, CSVs, Slack threads, Notion pages, Gmail threads, tickets, webpages, etc. +An item is one piece of context, and carries exactly one of: -- Ingest with `POST /context/ingest`, `type=knowledge`. -- Search with `POST /query`, `type: "knowledge"`. -- Use for content that should be reusable across users. -- Mutability is explicit: re-ingest with the same ID and `upsert: true`, or delete. +- `text`: a document, a note, a policy, an agent log line; anything you already have as a string. To ingest a file, extract its text first. +- `conversation`: a list of `{ role, content, name? }` turns, the same message list you already send to OpenAI or Anthropic. -### Memories +Every item can also carry a `context_id` (your id; reuse it to replace the item), a `title`, a `context_category` (`user_preference`, `business_knowledge`, `decision_trace`, or `auto` for no label; you set it, HydraDB never infers it), declared `attributes` and free-form `custom_attributes`, a `happened_at` date, `forceful_relations` to other items, an `acl`, and per-item `enrich` / `upsert` / `instructions`. See [Item fields](#item-fields). -Memories are user/workspace/session-scoped context: preferences, conversation history, behavioral signals, decisions, inferred traits. - -- Ingest with `POST /context/ingest`, `type=memory`. -- Search with `POST /query`, `type: "memory"` or `type: "all"`. -- Always pass the same `collection` used at ingestion. -- Use `infer: true` when the input is raw signal and HydraDB should extract the durable preference/fact. -- Use `infer: false` when the memory is already structured and should be stored verbatim. +With `enrich: true` (the default) HydraDB reads each item and extracts entities, relations and preferences into the context graph. The extracted statement comes back on each chunk as `enrichment`, separate from the chunk's verbatim `content`, and the item's category comes back as `enrichment_kind`. ### Query -`POST /query` is the single retrieval endpoint. - -It can search: - -- Knowledge only: `type: "knowledge"` -- Memories only: `type: "memory"` -- Both stores together: `type: "all"` +`POST /query` is the single retrieval endpoint. It searches every item in the collections you name. -It supports: +- Scope with `collection` (one) or `collections` (a list, or `{ "name": weight }` to rank one collection above another). +- `query_by: "hybrid"` (default) blends semantic and BM25 retrieval; `query_by: "text"` is BM25 keyword or phrase search. +- `mode: "auto"` (default) routes each query; `"fast"` is one low-latency pass; `"thinking"` expands the query, reranks, traverses the graph further and follows forceful relations. +- `attributes` filters on declared attributes with operators such as `$eq` and `$in`. -- `query_by: "hybrid"` for semantic + BM25 retrieval. -- `query_by: "text"` for BM25 keyword/phrase search. -- `mode: "fast"` for low latency. -- `mode: "thinking"` for query expansion, reranking, richer graph traversal, and forceful-relation expansion. +The response `data` is always the same four keys: `chunks`, `graph`, `forceful_relations` and `llm_prompt`. See [Query API](#8-query-api). ### Feedback -`POST /feedback` records how a query performed. It is the loop that closes retrieval quality, and it is open to agents whenever a query falls short - and, once a task is finished, for a short report on how the context held up - see [Sending feedback](#sending-feedback). +`POST /feedback` records how a query performed. See [Sending feedback](#sending-feedback) for when and how to send it. | Field | Required | Notes | |---|---|---| | `request_id` | yes | The UUID from that query's `meta.request_id`, verbatim | | `feedback` | yes* | What was right or wrong, up to 8000 chars. *Optional if `ground_truth` is sent | -| `rating` | no | `positive`, `negative`, `neutral`. Omitting it ≠ `neutral` | -| `source` | no | `user` (default) or `agent` - agents set `"agent"` | +| `rating` | no | `positive`, `negative`, `neutral`. Omitting it is not the same as `neutral` | +| `source` | no | `user` (default) or `agent`. Agents set `"agent"` | | `database` / `collection` | no | Optional scoping; `collection` requires `database` | -| `ground_truth` | no | `{ answer, source_ids }` - what the right answer was, when you know it. Makes `feedback` optional | -| `metadata` | no | Up to 20 string pairs of your own context (agent name, conversation id) | +| `ground_truth` | no | `{ answer, source_ids }`: the right answer and the `context_id`s that contain it, when you know them. Makes `feedback` optional | +| `metadata` | no | Feedback's own free-form context: up to 20 string pairs (agent name, conversation id) | -Each submission is its own record - sending a second report about the same query adds to it rather than replacing it, so refine as you learn more. +Each submission is its own record. Sending a second report about the same query adds to it rather than replacing it, so refine as you learn more. ### Context graph -HydraDB builds a graph of entity/relation triplets from ingested content. When `graph_context: true` (default), search can return: +HydraDB builds a graph of entities and relations from every enriched item, and from any graph you supply with `graph_payload`. With `graph_context: true` (the default), a query returns `graph[]`: paths through that graph that connect the question to the results. Each path has: -- `graph_context.query_paths` -- `graph_context.chunk_relations` -- `graph_context.chunk_id_to_group_ids` +- `origin`: `"query_path"` (grown from the entities in the query) or `"chunk_relation"` (the neighbourhood of a returned chunk). +- `triplets`: the chain of `source`, `relation`, `target` hops. Every hop's `relation.chunk_id` names the chunk it was extracted from. +- `path_summary`: one sentence describing the whole path. Never empty. -Graph context augments retrieval; `chunks` remain the primary search output. +Chunks remain the primary output; the graph explains how they connect. ### Forceful relations -At ingestion, sources can declare explicit relations: +At ingest, any item (text or conversation) can declare which other items it is linked to: ```json { - "relations": { - "ids": ["related-item-1", "related-item-2"], - "properties": { "reason": "same thread" } - } + "context_id": "refund-policy", + "text": "Refunds are processed within 5 business days.", + "forceful_relations": { "ids": ["refund-faq", "refund-escalations"], "properties": {} } } ``` -At search time, set or rely on default `query_forceful_relations: true` with `mode: "thinking"` to pull related chunks into `additional_context`. +At query time, with `follow_forceful_relations: true` (the default), a hit on `refund-policy` pulls chunks of `refund-faq` and `refund-escalations` into the response's `forceful_relations[]`, each with the `via` link that brought it in. Rules: -- Forceful relation expansion only takes effect in `mode: "thinking"`. -- Use `ids`. -- Relations are store-local: memory-to-memory or knowledge-to-knowledge. Cross-store relation lookups may not surface anything. +- Forceful relations are followed only in `thinking` mode (including when `mode: "auto"` routes a query to thinking). +- `ids` are `context_id`s. +- They are linked by the author, not ranked for the query, so do not read their `score` as relevance. --- @@ -302,7 +278,7 @@ Rules: ### Python SDK ```bash -pip install "hydradb-sdk>=2,<3" +pip install "hydradb-sdk>=2.1.5,<3" ``` ```python @@ -324,7 +300,7 @@ client = AsyncHydraDB(token=os.environ["HYDRA_DB_API_KEY"]) ### TypeScript SDK ```bash -npm install @hydradb/sdk@^2 +npm install @hydradb/sdk@^2.1.5 ``` ```ts @@ -337,15 +313,17 @@ const client = new HydraDBClient({ SDK naming: -- Python methods and fields: snake_case, e.g. `client.databases.collections()`, `database`, `collection`, `page_size`, `query_by`. -- TypeScript methods and fields: camelCase, e.g. `client.databases.collections()`, `database`, `collection`, `pageSize`, `queryBy`. -- Both SDKs return a `{ success, data, error, meta }` envelope; the payload is under `.data` (e.g. `response.data.infra`, `response.data.statuses`, `response.data.results`). -- Values that are passed as JSON strings (`memories`, `app_knowledge`, `document_metadata`) keep snake_case keys inside the stringified payload in both SDKs, since that is raw wire data. +- Python methods and fields: snake_case, for example `client.databases.collections()`, `max_results`, `query_by`, `result.data.llm_prompt`, `status.indexing_status`. +- TypeScript methods and fields: camelCase, for example `maxResults`, `queryBy`, `pageSize`, `result.data.llmPrompt`, `chunk.chunkId`, `chunk.contextId`, `chunk.enrichmentKind`, `path.pathSummary`, `result.data.forcefulRelations`, `status.indexingStatus`. +- Both SDKs return a `{ success, data, error, meta }` envelope; the payload is under `.data` (for example `response.data.infra`, `response.data.statuses`, `response.data.results`). +- `client.context.ingest()` sends a multipart form: the item list goes in the `items` form field as a JSON string. Keys inside each item stay snake_case in every language (`context_id`, `context_category`, `happened_at`), because that string is raw wire data. --- ## 4. Minimal end-to-end flow +Create a database, ingest a company policy into a shared collection and a customer conversation into that customer's collection, wait for both to index, then ask a question that needs both. + ### Python ```python @@ -355,64 +333,75 @@ import time from hydra_db import HydraDB client = HydraDB(token=os.environ["HYDRA_DB_API_KEY"]) -database = "my_first_database" +database = "acme" -# 1. Create database. +# 1. Create a database. Its name is all it needs. client.databases.create(database=database) # 2. Wait until the database can accept data. while True: - status = client.databases.status(database=database) - if status.data.infra.ready_for_ingestion: + infra = client.databases.status(database=database).data.infra + if infra.ready_for_ingestion: break time.sleep(5) -# 3. Ingest a user memory. -ingest = client.context.ingest( - type="memory", +# 3. Ingest a policy into the shared collection and a conversation into Alex's. +# The SDK sends the item list as a JSON string in the `items` form field. +client.context.ingest( + database=database, + collection="company", + items=json.dumps([{ + "context_id": "refund-policy", + "title": "Refund policy", + "text": "Refunds are processed within 5 business days.", + "context_category": "business_knowledge", + }]), +) +client.context.ingest( database=database, collection="user_alex", - memories=json.dumps([ - { - "id": "user_alex_pref_001", - "text": "User prefers detailed technical explanations and dark mode.", - "infer": True, - "user_name": "Alex", - } - ]), + items=json.dumps([{ + "context_id": "chat-alex-001", + "conversation": [ + {"role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex"}, + {"role": "assistant", "content": "Got it, short answers."}, + ], + "context_category": "user_preference", + "happened_at": "2026-09-01", + }]), ) -ingest_id = ingest.data.results[0].id - -# 4. Wait until searchable. -while True: - status = client.context.status( - database=database, - collection="user_alex", - ids=[ingest_id], - ).data.statuses[0] - - if status.indexing_status in ("graph_creation", "completed"): - break - if status.indexing_status in ("errored", "failed"): - # A wrong database/collection reports "errored" too, with FILE_NOT_FOUND. - # error_message is "" in that case, so branch on the code, not the text. - if status.error_code == "FILE_NOT_FOUND": - raise RuntimeError("No such id in this scope: check database/collection match ingestion") - raise RuntimeError(status.error_message or status.message or "Indexing failed") +# 4. Wait until both items are searchable. +pending = {"company": "refund-policy", "user_alex": "chat-alex-001"} +while pending: + for collection, context_id in list(pending.items()): + status = client.context.status( + database=database, collection=collection, ids=[context_id] + ).data.statuses[0] + if status.indexing_status in ("graph_creation", "completed"): + del pending[collection] + elif status.indexing_status in ("errored", "failed"): + # A wrong database/collection reports "errored" too, with FILE_NOT_FOUND. + # error_message is "" in that case, so branch on the code, not the text. + if status.error_code == "FILE_NOT_FOUND": + raise RuntimeError(f"{context_id} not found in {collection}: check the scope") + raise RuntimeError(status.error_message or status.message or "Indexing failed") time.sleep(2) -# 5. Search memories. +# 5. Query both collections, weighting Alex's own context higher. result = client.query( database=database, - collection="user_alex", - query="What does the user prefer?", - type="memory", - query_by="hybrid", - mode="thinking", + collections={"user_alex": 2, "company": 1}, + query="How should I explain our refund policy to this user?", ) -print(result.data.chunks) +for chunk in result.data.chunks: + print(chunk.score, chunk.context_id, chunk.content) +for path in result.data.graph: + print(path.origin, path.path_summary) + +# 6. The same context as one prompt-ready string, with citation labels. +print(result.data.llm_prompt) ``` ### TypeScript @@ -420,70 +409,163 @@ print(result.data.chunks) ```ts import { HydraDBClient } from "@hydradb/sdk"; -const client = new HydraDBClient({ - token: process.env.HYDRA_DB_API_KEY, -}); - -const database = "my_first_database"; +const client = new HydraDBClient({ token: process.env.HYDRA_DB_API_KEY }); +const database = "acme"; -await client.databases.create({ database: database }); +// 1. Create a database. Its name is all it needs. +await client.databases.create({ database }); +// 2. Wait until the database can accept data. while (true) { - const status = await client.databases.status({ database: database }); - if (status.data.infra.readyForIngestion) break; + const { data } = await client.databases.status({ database }); + if (data?.infra.readyForIngestion) break; await new Promise((resolve) => setTimeout(resolve, 5_000)); } -const ingest = await client.context.ingest({ - type: "memory", - database: database, +// 3. Ingest a policy into the shared collection and a conversation into Alex's. +// The SDK sends the item list as a JSON string in the `items` form field. +await client.context.ingest({ + database, + collection: "company", + items: JSON.stringify([{ + context_id: "refund-policy", + title: "Refund policy", + text: "Refunds are processed within 5 business days.", + context_category: "business_knowledge", + }]), +}); +await client.context.ingest({ + database, collection: "user_alex", - memories: JSON.stringify([ - { - id: "user_alex_pref_001", - text: "User prefers detailed technical explanations and dark mode.", - infer: true, - user_name: "Alex", - }, - ]), + items: JSON.stringify([{ + context_id: "chat-alex-001", + conversation: [ + { role: "user", content: "Keep answers short, I read on my phone.", name: "alex" }, + { role: "assistant", content: "Got it, short answers." }, + ], + context_category: "user_preference", + happened_at: "2026-09-01", + }]), }); -const ingestId = ingest.data.results[0].id; - -while (true) { - const status = (await client.context.status({ - database: database, - collection: "user_alex", - ids: [ingestId], - })).data.statuses[0]; - - if (["graph_creation", "completed"].includes(status.indexingStatus as string)) break; - if (["errored", "failed"].includes(status.indexingStatus as string)) { - // || rather than ??: errorMessage is "" (not null) on some failures, and ?? - // would pass the empty string through and throw an error with no message. - throw new Error(status.errorMessage || status.message || "Indexing failed"); +// 4. Wait until both items are searchable. +const pending = new Map([["company", "refund-policy"], ["user_alex", "chat-alex-001"]]); +while (pending.size > 0) { + for (const [collection, id] of pending) { + const status = (await client.context.status({ database, collection, ids: [id] })).data.statuses[0]; + if (["graph_creation", "completed"].includes(status.indexingStatus as string)) { + pending.delete(collection); + } else if (["errored", "failed"].includes(status.indexingStatus as string)) { + // A scope miss reports errored too, so name the cause rather than guessing. + if (status.errorCode === "FILE_NOT_FOUND") { + throw new Error(`${id} not found in ${collection}: check the scope`); + } + // Use ||, not ??: errorMessage is "" (not null) on some failures. + throw new Error(status.errorMessage || status.message || "Indexing failed"); + } } - await new Promise((resolve) => setTimeout(resolve, 2_000)); } +// 5. Query both collections, weighting Alex's own context higher. const result = await client.query({ - database: database, - collection: "user_alex", - query: "What does the user prefer?", - type: "memory", - queryBy: "hybrid", - mode: "thinking", + database, + collections: { user_alex: 2, company: 1 }, + query: "How should I explain our refund policy to this user?", }); -console.log(result.data.chunks); +console.log(result.data.chunks.map((c) => [c.score, c.contextId, c.content])); +console.log(result.data.graph.map((p) => p.pathSummary)); + +// 6. The same context as one prompt-ready string, with citation labels. +console.log(result.data.llmPrompt); +``` + +### cURL + +```bash +export HYDRA_DB_API_KEY="your_api_key" +DATABASE="acme" +API="https://api.hydradb.com" +AUTH=(-H "Authorization: Bearer $HYDRA_DB_API_KEY" -H "API-Version: 2") + +# 1. Create a database. +curl -s -X POST "$API/databases" "${AUTH[@]}" \ + -H "Content-Type: application/json" \ + -d "{\"database\":\"${DATABASE}\"}" + +# 2. Wait until the database can accept data. +until curl -s "$API/databases/status?database=${DATABASE}" "${AUTH[@]}" \ + | jq -e '.data.infra.ready_for_ingestion' > /dev/null; do + sleep 5 +done + +# 3. Ingest a policy into the shared collection and a conversation into Alex's. +curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ + -H "Content-Type: application/json" \ + -d "{ + \"database\": \"${DATABASE}\", + \"collection\": \"company\", + \"context\": [{ + \"context_id\": \"refund-policy\", + \"title\": \"Refund policy\", + \"text\": \"Refunds are processed within 5 business days.\", + \"context_category\": \"business_knowledge\" + }] + }" +curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ + -H "Content-Type: application/json" \ + -d "{ + \"database\": \"${DATABASE}\", + \"collection\": \"user_alex\", + \"context\": [{ + \"context_id\": \"chat-alex-001\", + \"conversation\": [ + { \"role\": \"user\", \"content\": \"Keep answers short, I read on my phone.\", \"name\": \"alex\" }, + { \"role\": \"assistant\", \"content\": \"Got it, short answers.\" } + ], + \"context_category\": \"user_preference\", + \"happened_at\": \"2026-09-01\" + }] + }" + +# 4. Wait until both items are searchable. +for pair in "company:refund-policy" "user_alex:chat-alex-001"; do + COLLECTION="${pair%%:*}"; ID="${pair#*:}" + while true; do + STATUS=$(curl -s -G "$API/context/status" "${AUTH[@]}" \ + --data-urlencode "database=${DATABASE}" \ + --data-urlencode "collection=${COLLECTION}" \ + --data-urlencode "ids=${ID}" \ + | jq -r '.data.statuses[0].indexing_status') + if [ "$STATUS" = "completed" ] || [ "$STATUS" = "graph_creation" ]; then break; fi + if [ "$STATUS" = "errored" ]; then echo "$ID errored"; exit 1; fi + sleep 2 + done +done + +# 5. Query both collections, weighting Alex's own context higher. +curl -s -X POST "$API/query" "${AUTH[@]}" \ + -H "Content-Type: application/json" \ + -d '{ + "database": "'"${DATABASE}"'", + "collections": { "user_alex": 2, "company": 1 }, + "query": "How should I explain our refund policy to this user?" + }' | jq '{chunks: [.data.chunks[]?.content], paths: [.data.graph[]?.path_summary], llm_prompt: .data.llm_prompt}' +``` + +The response has four keys. `chunks` are the matched pieces of your items, ranked, each with a `score` and its `context_id`. `graph` is the list of relation paths connecting them, each with a one-sentence `path_summary`. `forceful_relations` holds items linked at ingest (none here). `llm_prompt` is all of that as one markdown string with citation labels, ready to drop into your model call: + +```python +messages = [{"role": "system", "content": result.data.llm_prompt}, + {"role": "user", "content": "How should I explain our refund policy to this user?"}] ``` --- ## 5. Databases API -### Create database +### Create a database `POST /databases` · `client.databases.create()` @@ -505,188 +587,195 @@ console.log(result.data.chunks); Notes: -- Database creation is async. -- `database` should be stable; docs recommend lowercase letters, numbers, and underscores for portability. -- `database_metadata_schema` is effectively planned up front. If you need to change filterable metadata fields, expect to create a new database or re-ingest under a new schema. -- `POST /databases` may return `409 DATABASE_ALREADY_EXISTS` for duplicate database IDs and `403 FORBIDDEN` when the API key or plan cannot create more databases. +- `database` is the only required field. `database_metadata_schema` is optional and declares the filterable `attributes` (see [Attributes guide](#10-attributes-guide)). +- Database creation is asynchronous. +- `database` is a stable, case-sensitive id of up to 25 characters, immutable after creation. Lowercase letters, numbers and underscores are the most portable. +- Plan the schema up front. You can add fields later with `PATCH /databases/{database}/metadata-schema` (additive only: no deletes, no data-type or flag changes). +- `POST /databases` may return `409 DATABASE_ALREADY_EXISTS` for a duplicate name and `403 FORBIDDEN` when the API key or plan cannot create more databases. ### Check readiness `GET /databases/status?database=...` · `client.databases.status()` -Ready when infrastructure reports usable status. Current docs show `infra.ready_for_ingestion`; SDK examples also check: - -- `infra.scheduler_status` -- `infra.graph_status` -- `infra.vectorstore_status.knowledge` -- `infra.vectorstore_status.memories` - -Use SDK autocomplete/returned object shape for the exact field spelling. +Poll until `data.infra.ready_for_ingestion` is `true` (TypeScript: `data.infra.readyForIngestion`). The `infra` object also carries per-component flags (`scheduler_status`, `graph_status`, `vectorstore_status`); poll `ready_for_ingestion`, not the individual flags. ### Other database endpoints | Endpoint | Purpose | |---|---| -| `GET /databases` | list database IDs | -| `DELETE /databases?database=...` | delete a database and all data | +| `GET /databases` | list databases | +| `DELETE /databases?database=...` | delete a database and all its data | | `GET /databases/collections?database=...` | list active collections | -| `GET /databases/stats?database=...` | collection counts | +| `GET /databases/stats?database=...` | row counts | +| `PATCH /databases/{database}/metadata-schema` | add declared attribute fields | + +`GET /databases/stats` keeps two historical field names, `knowledge_collection.row_count` and `memory_collection.row_count`. Both count the same rows, so the two numbers are equal: read one, do not add them. --- -## 6. Source ingestion +## 6. Ingesting context -`POST /context/ingest` is the unified write endpoint for knowledge and memories. +`POST /context/ingest` · `client.context.ingest()` -### Common form fields +One endpoint takes every item, text or conversation, into any collection of a database. The raw HTTP body is JSON and the list is called `context`. -| Field | Description | -|---|---| -| `type` | `"knowledge"` or `"memory"`; default is `"knowledge"` | -| `database` | target database | -| `collection` | optional logical partition; default collection if omitted | -| `upsert` | default `true`; replaces existing sources with same ID | -| `documents` | binary uploads for knowledge | -| `metadata` | schema-aligned filterable fields; top-level `metadata_filters` match these | -| `additional_metadata` | free-form fields; anything you want as metadata that is not in the `metadata` schema | -| `document_metadata` | JSON-stringified array, one item per uploaded file; assigns a source `id` to that file | -| `app_knowledge` | JSON-stringified object/array of pre-extracted app sources | -| `memories` | JSON-stringified array of memory items | +```json +{ + "database": "acme", + "collection": "company", + "enrich": true, + "upsert": true, + "instructions": "Extract owners and due dates for every action item.", + "context": [ + { + "context_id": "refund-policy", + "title": "Refund policy", + "text": "Refunds are processed within 5 business days.", + "context_category": "business_knowledge", + "attributes": { "department": "support" }, + "custom_attributes": { "owner": "sam@acme.com" }, + "forceful_relations": { "ids": ["refund-faq"], "properties": {} } + }, + { + "context_id": "chat-alex-001", + "conversation": [ + { "role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex" }, + { "role": "assistant", "content": "Got it, short answers." } + ], + "context_category": "user_preference", + "happened_at": "2026-09-01", + "acl": ["user_email:alex@acme.com"], + "instructions": "Extract the customer's stated preferences. Ignore small talk." + } + ] +} +``` -`metadata` and `additional_metadata` are top-level request fields. They are not nested inside `document_metadata`. +### Request body -### Knowledge from files +| Field | Notes | +|---|---| +| `database` | Required. The database to write to. | +| `collection` | Optional. The collection to write to; the default collection when omitted. | +| `context` | The list of items, at most 100. | +| `enrich` | Request-level default for every item's `enrich`. Default `true`. | +| `upsert` | Request-level default for every item's `upsert`. Default `true`. | +| `instructions` | Request-level default for every item's `instructions`. Default empty. | +| `graph_payload` | Optional. A graph you built yourself, keyed by `context_id`. See [Bring your own graph](#bring-your-own-graph). | -Use for PDFs, DOCX, Markdown, CSV, TXT, and other files HydraDB should parse. +The request-level values apply to any item that does not set the field itself, so one call can enrich some items and store others as they are, or replace some items and append others. -```python -import json +### Item fields -with open("policy.pdf", "rb") as f: - result = client.context.ingest( - type="knowledge", - database="acme_corp", - documents=[("policy.pdf", f, "application/pdf")], - metadata={"department": "support", "status": "approved"}, - additional_metadata={"author": "Alice", "version": 2}, - document_metadata=json.dumps([{"id": "policy_v2"}]), - ) -``` +Each item carries exactly one of `text` or `conversation`. -`metadata` carries the schema-aligned fields declared on the database; top-level `metadata_filters` match these. Anything you want as metadata that is not present in the `metadata` schema goes in `additional_metadata`. +| Field | Notes | +|---|---| +| `context_id` | Your id for the item. Generated from `title` when omitted, so two untitled items without ids and with the same text collide. Must not contain commas. | +| `title` | Optional readable name, printed in `llm_prompt` and matchable with `titles` on `/query`. | +| `text` | Plain text or markdown. | +| `conversation` | A list of `{ role, content, name? }` turns. | +| `enrich` | Extract entities, relations and preferences from this item. Default: the request's `enrich`, else `true`. Set `false` to store the item only as searchable text. | +| `upsert` | Replace an existing item with the same `context_id`. Default: the request's `upsert`, else `true`. | +| `instructions` | Steer enrichment for this item. Default: the request's `instructions`. | +| `happened_at` | The date the item is about, `YYYY-MM-DD` only; a timestamp is a `400`. HydraDB records when it received the item separately. | +| `attributes` | Declared, filterable fields from the database's `database_metadata_schema`. | +| `custom_attributes` | Free-form fields. Not filterable. | +| `context_category` | `auto` (the default, no label), `user_preference`, `business_knowledge` or `decision_trace`. Validated strictly: a typo is a `400`, not a fallback to `auto`. | +| `forceful_relations` | `{ "ids": [...], "properties": {} }`: the `context_id`s this item is linked to. | +| `acl` | Principals allowed to retrieve the item: `user_email:a@x.com` (or a bare email), `group::`, `domain:acme.com`, `__public__`. Omit for unrestricted, `[]` for nobody. A malformed principal rejects the whole request with `400`. | +| `is_markdown` | Chunk `text` on its markdown structure instead of as flat prose. | +| `user_name` | The speaker for a text item. On a conversation, the per-turn `name` wins. | + +A key an item does not recognise is dropped without an error, so check spelling against this table. + +### Conversations + +- Roles are `user`, `assistant` and `system`. Any other role is a `400`; map roles such as `tool` or `human` before sending. +- `system` turns shape enrichment but are never stored as facts. A conversation of only `system` turns is a `400`. +- Consecutive turns with the same role are accepted and joined. +- Set `name` per turn when several people speak, so preferences are attributed to the right person. +- An empty list, or a turn with empty `content`, is a `400`. + +### Context categories + +| Category | For | +|---|---| +| `user_preference` | What a person likes, chose or asked for | +| `business_knowledge` | Documentation, policy, product and domain facts | +| `decision_trace` | What an agent or a team decided, and why | -`document_metadata` exists for one reason: when you upload a file there is otherwise no way to assign it a specific source ID. Use it to bind an `id` to each uploaded file. +The label is yours to set. Leave `context_category` out, or send `auto`, and the item is stored and enriched as general context with no category on its chunks. A preference or a decision is sent as text (or a conversation) with the matching label. -`document_metadata` item fields: +### IDs and replacement -| Field | Purpose | -|---|---| -| `id` | stable source ID / upsert key for the uploaded file | -| `relations` | explicit forceful relations, e.g. `{ "ids": [...] }` | +- `upsert: true` (the default) **replaces**: re-ingesting a `context_id` deletes everything derived from the previous version (its chunks and its graph contribution) before writing the new one. It does not merge. +- `upsert` is per item, with the request value as the default. +- Give repeated text either a `context_id` or a distinct `title`, or the second item replaces the first. -### Knowledge from app sources +### Bring your own graph -Use `app_knowledge` when your connector already extracted text from Slack, Gmail, Jira, Linear, Zendesk, Notion, Confluence, Salesforce, webpages, etc. +Skip extraction for an item and supply its entities and relations yourself with `graph_payload`, a map keyed by `context_id`: -```ts -await client.context.ingest({ - type: "knowledge", - database: "acme_corp", - appKnowledge: JSON.stringify([ - { - id: "slack_C123_1715012345_000100", - database: "acme_corp", - collection: "default", - title: "Pricing discussion - Slack #product", - type: "slack", - url: "https://slack.com/archives/C123/p1715012345000100", - timestamp: "2025-01-15T12:30:00Z", - content: { - text: "We agreed on three tiers: Starter, Pro, and Enterprise." - }, - tenant_metadata: { - channel: "product", - workspace: "acme" - }, - additional_metadata: { - author: "alice", - slack_ts: "1715012345.000100" +```json +{ + "database": "acme", + "context": [ + { "context_id": "billing-policy", "text": "Alice Carter owns the billing policy." } + ], + "graph_payload": { + "billing-policy": { + "entities": { + "alice": { "name": "Alice Carter", "type": "PERSON" }, + "billing": { "name": "Billing Policy", "type": "POLICY" } }, - relations: { - ids: ["slack_thread_root_001"] - } + "relations": [ + { "source": "alice", "target": "billing", "predicate": "OWNS", "context": "Alice Carter owns the billing policy." } + ] } - ]), -}); + } +} ``` -Recommended app-source fields: - -| Field | Why send it | -|---|---| -| `id` | stable HydraDB ID and upsert key | -| `title` | readable result/citation title | -| `type` | category such as `slack`, `gmail`, `jira`, `notion`, `webpage` | -| `content.text` / `content.markdown` | primary searchable content | -| `tenant_metadata` | fast filters such as channel, project, region, workspace | -| `additional_metadata` | provider IDs, author, status, assignee, timestamps | -| `url` | citation / navigation link | -| `timestamp` | freshness and chronology | -| `relations.ids` | thread, parent/child, linked issue, related source expansion | +Every key in `graph_payload` must equal the `context_id` of an item in the same request; a key that matches nothing is a `400`. A keyed item is still chunked and embedded, so it stays searchable. Entity and relation shapes and caps are on [Bring your own graph](/essentials/v2/bring-your-own-graph). -App-source object kinds in OpenAPI include: +### Limits and validation -- `email` -- `message` -- `ticket` -- `knowledge_base` -- `comment` -- `custom` +- At most **100 items** per request, **1 MiB** of text per item, **8 MiB** of text per request. Split larger batches. +- `attributes` are capped at **16 KiB** and `custom_attributes` at **1 KiB** per item, measured on the compact JSON encoding in UTF-8 bytes (keys and punctuation count). +- A validation error names the item it refers to as `context[N]`. +- Ingest takes text only. To ingest a PDF, DOCX or CMS export, extract its text in your application and send it as `text`, one item per document. For tools such as Slack, Notion or Google Drive, use a [connector](/essentials/v2/connectors): synced content lands in the same database and is queried together with your items. -Modern docs show both a generic `content` shape and a richer app shape with `kind`, `provider`, `external_id`, `fields`, `attachments`, `comments`, and `relations`. Use the richer shape when building app-aware search; at minimum provide stable IDs, content, source type, metadata, and relations. +### SDKs: the `items` form field -### Memories +The SDKs send a multipart form rather than a JSON body. The item array goes in the `items` form field as a JSON string, next to `database`, `collection`, `upsert` and `graph_payload`; the server runs the same validation on both entry points. Set `enrich` and `instructions` on each item. Python: `client.context.ingest(database=..., collection=..., items=json.dumps([...]))`. TypeScript: `await client.context.ingest({ database, collection, items: JSON.stringify([...]) })`. Keys inside each item stay snake_case in both. Full examples are in [Minimal end-to-end flow](#4-minimal-end-to-end-flow). -Use `type="memory"` and a JSON-stringified `memories` array. +### Response -```python -import json +`202 Accepted`: -client.context.ingest( - type="memory", - database="acme_corp", - collection="user_alex", - memories=json.dumps([ - { - "id": "pref_dark_mode", - "title": "Dark mode preference", - "text": "User prefers dark mode and concise answers.", - "infer": True, - "user_name": "Alex", - "tenant_metadata": json.dumps({"team": "engineering"}), - "additional_metadata": {"source": "onboarding"}, - "relations": {"ids": ["pref_concise"]} - } - ]), -) +```json +{ + "success": true, + "data": { + "success": true, + "message": "Context queued for ingestion successfully. ...", + "results": [ + { "id": "refund-policy", "title": "Refund policy", "status": "queued", "infer": true, "error": null, "error_code": null }, + { "id": "chat-alex-001", "title": null, "status": "queued", "infer": true, "error": null, "error_code": null } + ], + "success_count": 2, + "failed_count": 0 + }, + "error": null, + "meta": { "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d" } +} ``` -Memory item fields: - -| Field | Description | -|---|---| -| `id` | optional stable upsert key | -| `title` | label for listing/search display | -| `text` | raw text or Markdown; required unless `user_assistant_pairs` provided | -| `user_assistant_pairs` | dialogue pairs `{ user, assistant }`; useful for implied preferences | -| `is_markdown` | preserve Markdown structure during chunking | -| `infer` | extract durable preference/fact from raw signal | -| `custom_instructions` | guide inference when `infer: true` | -| `user_name` | helps inference | -| `expiry_time` | TTL in seconds | -| `tenant_metadata` | schema-aligned fields; for memory items, encode as a JSON string inside the JSON-stringified `memories` field | -| `additional_metadata` | free-form fields; keep as objects | -| `relations` | explicit memory-to-memory links | +- `results[].id` is the item's `context_id`: the one you sent, or the generated one. Pass it to `GET /context/status`. +- `results[].infer` mirrors the item's `enrich`. +- `results[].status` is `queued` or `failed`. A failed item carries `error` and `error_code`; the other items in the request are still queued. +- A `202` means queued, not searchable. Poll status before querying. --- @@ -698,50 +787,24 @@ Memory item fields: Parameters: -- `database` required -- `ids` required array -- `collection` - **required if you ingested into one.** The lookup is scoped: omitting `collection`, or sending the wrong one, returns `indexing_status: "errored"` with `error_code: "FILE_NOT_FOUND"` and `message: "ID not found"` for a source that exists and is fully searchable. That is a scope miss, not an indexing failure, and it is indistinguishable from one unless you read `error_code`. - -Status meanings from docs: +- `database`: required. +- `ids`: one or more `context_id`s (repeat the parameter, or comma-separate). +- `collection`: **required if you ingested into one.** The lookup is scoped: omitting `collection`, or sending the wrong one, returns `indexing_status: "errored"` with `error_code: "FILE_NOT_FOUND"` and `message: "ID not found"` for an item that exists and is fully searchable. That is a scope miss, not an indexing failure, and it is indistinguishable from one unless you read `error_code`. | Status | Searchable? | Meaning | |---|---:|---| | `queued` | No | accepted, not picked up | -| `processing` | No | parsing/chunking/embedding | +| `processing` | No | chunking, embedding, enriching | | `graph_creation` | Yes | chunks indexed; graph still building | | `completed` | Yes | fully indexed and graphed | -| `errored` / `failed` | No | terminal failure | +| `errored` | No | terminal failure | -Pattern: - -```ts -while (true) { - const status = (await client.context.status({ - database: "acme_corp", - collection: "support_docs", // must match the ingestion scope, or you get FILE_NOT_FOUND - ids: ["policy_v2"], - })).data.statuses[0]; - - if (["graph_creation", "completed"].includes(status.indexingStatus as string)) break; - if (["errored", "failed"].includes(status.indexingStatus as string)) { - // A scope miss reports errored too, so name the cause rather than guessing. - if (status.errorCode === "FILE_NOT_FOUND") { - throw new Error(`No such id in this scope: check database/collection match ingestion`); - } - // Use ||, not ??: errorMessage comes back as "" (not null) on some failures, - // and ?? passes an empty string straight through to an empty error. - throw new Error(status.errorMessage || status.message || "Indexing failed"); - } - await new Promise((r) => setTimeout(r, 2_000)); -} -``` +The polling loops in [Minimal end-to-end flow](#4-minimal-end-to-end-flow) handle every case, including the scope miss. On a failure, read `error_message || message`: `error_message` is `""` (not null) on some failures, so `??` would pass the empty string through. Wait for `completed` only when you need full graph traversal in the results. ### Webhooks instead of polling Use `/webhooks/indexing` to receive terminal indexing events. -Endpoints: - | Endpoint | Purpose | |---|---| | `GET /webhooks/indexing` | get current webhook registration | @@ -752,17 +815,15 @@ Endpoints: | `GET /webhooks/indexing/deliveries/{delivery_id}` | get one delivery | | `POST /webhooks/indexing/deliveries/{delivery_id}/retry` | retry delivery | -Supported event: +Supported event: `indexing.status_changed`. -- `indexing.status_changed` - -Payload shape: +Payload shape (`id` is the item's `context_id`): ```json { "event": "indexing.status_changed", "delivery_id": "", - "id": "", + "id": "", "database": "", "collection": "", "status": "completed", @@ -797,9 +858,9 @@ Managing the secret: Rules: -- Webhook URL must be public HTTPS; localhost/private networks are blocked. +- Webhook URL must be public HTTPS; localhost and private networks are blocked. - Store `delivery_id` to deduplicate retries. -- Verify `X-HydraDB-Signature` with a constant-time comparison. Use `verify_webhook_signature` (Python SDK) or `verifyWebhookSignature` (TypeScript SDK) from the SDK helpers module. +- Verify `X-HydraDB-Signature` with a constant-time comparison. Use `verify_webhook_signature` (Python, `hydra_db.helpers`) or `verifyWebhookSignature` (TypeScript SDK helpers). - Fail closed: reject the request when the signing secret is absent from the environment, rather than skipping verification. --- @@ -808,378 +869,512 @@ Rules: `POST /query` · `client.query()` -### Request fields that matter +### Request fields | Field | Values | Notes | |---|---|---| -| `database` | string | required | -| `collection` | string/null | required for per-user memory searches | -| `query` | string | required | -| `type` | `knowledge`, `memory`, `all` | store selector for knowledge, memories, or both | -| `query_by` | `hybrid`, `text` | default hybrid | -| `operator` | `or`, `and`, `phrase` | only for `query_by: "text"` | -| `mode` | `fast`, `thinking` | applies to hybrid; thinking improves quality but adds latency | -| `max_results` | integer/null | default around 10; max documented as 50 | -| `alpha` | `0.0`–`1.0` or `auto` | hybrid weight; `1.0` semantic, `0.0` BM25 | -| `recency_bias` | `0.0`–`1.0` | boost newer content | -| `graph_context` | boolean | default true; include graph slice | -| `query_forceful_relations` | boolean | only effective in `thinking` mode | -| `additional_context` | string | short factual hint; not a hard filter | -| `metadata_filters` | object | deterministic filters before/around ranking | -| `query_apps` | boolean | app-aware retrieval lane for app sources; still searches the full selected knowledge scope | +| `database` | string | Required. | +| `collection` | string | Search one collection; the default collection when neither this nor `collections` is sent. | +| `collections` | `string[]` or `{ [collection]: weight }` | Search several. A list weights them equally; an object ranks one above another (weights rank, they do not exclude). Maximum 100. | +| `query` | string | Required. The question or search terms. | +| `max_results` | integer | Default `10`, maximum `50`. Caps the merged result across collections. | +| `mode` | `auto`, `fast`, `thinking` | `auto` (default) routes each query. `thinking` expands the query, reranks, traverses the graph further and follows forceful relations; `fast` is one pass. | +| `query_by` | `hybrid`, `text` | `hybrid` (default) blends semantic and BM25; `text` is BM25 only. | +| `operator` | `or`, `and`, `phrase` | BM25 term matching; only for `query_by: "text"`. Default `or`. | +| `alpha` | `0.0` to `1.0`, or `"auto"` | Semantic versus keyword blend for `hybrid`. `1.0` is fully semantic, `0.0` fully BM25. Default `0.8`. | +| `recency_bias` | `0.0` to `1.0` | Boost newer content. `0` disables it. | +| `ids` | `string[]` | Restrict retrieval to these `context_id`s. | +| `titles` | `string[]` | Restrict retrieval to items with one of these exact titles (case-insensitive). | +| `attributes` | object | Filter on declared attributes with operators. See [Filtering with attributes](#filtering-with-attributes). | +| `acl` | `string[]` | Query on behalf of an identity: only items it may retrieve are returned. Omitted, empty or `["*"]` disables filtering. | +| `query_apps` | boolean | App-aware lane for connector content (exact ids, actors, threads), on top of normal retrieval. | +| `graph_context` | boolean | Default `true`: include `graph[]`. | +| `follow_forceful_relations` | boolean | Default `true`: pull declared forceful relations into `forceful_relations[]` (`thinking` mode only). | +| `temporal_reasoning` | boolean | Default `true`. Resolve time-based questions (current, as of, ranges); matched facts come back in `chunks[].temporal`. Never changes which chunks are returned. | +| `temporal_now` | ISO 8601 string | The time to treat as now, for example when replaying a past conversation. | +| `temporal_intent` | object | Override the temporal intent HydraDB would infer from the query. | ### Recommended configurations | Goal | Request shape | |---|---| -| Fast document RAG | `type: "knowledge"`, `query_by: "hybrid"`, `mode: "fast"`, `graph_context: false`, `max_results: 5-10` | -| Highest-quality document RAG | `type: "knowledge"`, `query_by: "hybrid"`, `mode: "thinking"`, `graph_context: true`, `alpha: "auto"` | -| Personalized grounded answer | `type: "all"`, include `collection`, `query_by: "hybrid"`, `mode: "thinking"` | -| User preferences only | `type: "memory"`, include `collection`, `query_by: "hybrid"` | -| Exact keyword / phrase | `type: "knowledge"`, `query_by: "text"`, `operator: "phrase"` | -| Recent operational updates | `query_by: "hybrid"`, `recency_bias: 0.2-0.4`, metadata filter to doc type/status | -| App search | `type: "knowledge"`, `query_by: "hybrid"`, `mode: "thinking"`, `query_apps: true`; still searches non-app knowledge too | - -### Search examples - -Knowledge RAG: - -```ts -const result = await client.query({ - database: "acme_corp", - query: "What is our refund policy?", - type: "knowledge", - queryBy: "hybrid", - mode: "thinking", - maxResults: 10, - graphContext: true, -}); -``` - -Personalized answer using knowledge and memories: - -```ts -const result = await client.query({ - database: "acme_corp", - collection: "user_alex", - query: "What is our refund policy, and how should I explain it to this user?", - type: "all", - queryBy: "hybrid", - mode: "thinking", - alpha: "auto", -}); -``` +| Fast RAG | `mode: "fast"`, `graph_context: false`, `max_results: 5` to `10` | +| Highest-quality RAG | `mode: "thinking"`, `alpha: "auto"` (graph on by default) | +| Personalized grounded answer | `collections: { "user_alex": 2, "company": 1 }`, `mode: "thinking"` | +| One person's context only | `collection: "user_alex"` | +| Exact keyword or phrase | `query_by: "text"`, `operator: "phrase"` | +| Error codes, SKUs, product names | `query_by: "hybrid"`, `alpha: 0.3` to `0.5` | +| Recent operational updates | `recency_bias: 0.2` to `0.4`, plus an `attributes` filter on status or doc type | +| Connector content (Slack, Jira, Gmail) | `query_apps: true`, `mode: "thinking"` | +| Follow linked items | `mode: "thinking"` (`follow_forceful_relations` is on by default) | +| A known item or document | `ids: [...]` or `titles: [...]` | + +### Examples Exact phrase: ```python result = client.query( database="acme_corp", + collection="support", query="ERROR_429 rate limit", - type="knowledge", query_by="text", operator="phrase", ) ``` -Metadata-scoped search: +Attribute-filtered search, on behalf of one user: ```json { "database": "acme_corp", - "query": "authentication control review", - "type": "knowledge", - "query_by": "hybrid", - "metadata_filters": { - "compliance_framework": "SOC2", - "additional_metadata": { - "author": "alice" - } - } + "collection": "company", + "query": "What is the refund window for enterprise customers?", + "attributes": { + "$and": [ + { "department": { "$eq": "support" } }, + { "region": { "$in": ["us", "eu"] } } + ] + }, + "acl": ["user_email:grace@acme.com"] } ``` ### Search response -The successful `data` object / SDK return is a `RetrievalResult`: +`data` is exactly `chunks`, `graph`, `forceful_relations` and `llm_prompt`: ```json { - "chunks": [ - { - "chunk_uuid": "...", - "id": "...", - "chunk_content": "...", - "source_title": "...", - "source_type": "...", - "source_upload_time": "...", - "source_last_updated_time": "...", - "relevancy_score": 0.91, - "metadata": {}, - "additional_metadata": {}, - "extra_context_ids": [] - } - ], - "sources": [ - { - "id": "...", - "title": "...", - "type": "...", - "url": "...", - "timestamp": "...", - "metadata": {}, - "additional_metadata": {}, - "app_kind": "message", - "app_provider": "slack", - "app_external_id": "..." - } - ], - "graph_context": { - "query_paths": [], - "chunk_relations": [], - "chunk_id_to_group_ids": {} + "success": true, + "data": { + "chunks": [ + { + "chunk_id": "ck_policy_3", + "context_id": "refund-policy", + "score": 0.91, + "content": "Refunds are processed within 30 days of purchase by the Finance Department.", + "enrichment": "Refund window is 30 days; Finance owns refund processing.", + "enrichment_kind": "business_knowledge", + "temporal": [ + { "content": "Refund policy effective_from June 2026. Start: 2026-06-01", "start_date": "2026-06-01", "end_date": null } + ] + }, + { + "chunk_id": "ck_chat_1", + "context_id": "chat-2026-07-29", + "score": 0.84, + "content": "user: Keep refund answers short please\nassistant: Got it.", + "enrichment": "User prefers short answers about refunds.", + "enrichment_kind": "user_preference" + } + ], + "graph": [ + { + "origin": "query_path", + "triplets": [ + { + "source": { "entity_id": "ent_refunds", "name": "Refund Processing" }, + "relation": { + "predicate": "managed by", + "context": "Refund processing is managed by the Finance Department.", + "relationship_id": "rel_managed_by", + "chunk_id": "ck_policy_3" + }, + "target": { "entity_id": "ent_finance", "name": "Finance Department" } + } + ], + "path_summary": "Refund processing is managed by the Finance Department." + }, + { + "origin": "chunk_relation", + "triplets": [ + { + "source": { "entity_id": "ent_user", "name": "User" }, + "relation": { "predicate": "prefers", "context": "The user prefers short answers about refunds.", "relationship_id": "rel_prefers", "chunk_id": "ck_chat_1" }, + "target": { "entity_id": "ent_short", "name": "short answers" } + } + ], + "path_summary": "The user prefers short answers about refunds." + } + ], + "forceful_relations": [ + { + "via": { "from": "refund-policy", "to": "refund-faq" }, + "chunk": { "chunk_id": "ck_faq_1", "context_id": "refund-faq", "score": 0, "content": "FAQ: refunds to a card take 5 to 7 business days to appear." } + } + ], + "llm_prompt": "# Query results\n\n**Query:** who owns refund processing?\n..." }, - "additional_context": {} + "error": null, + "meta": { "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", "api_version": "2.0.1", "latency_ms": 412.7, "database": "acme_corp", "collection": "support" } } ``` -Use `chunks` as the primary LLM context. Preserve server order; do not re-sort unless you have a deliberate reranking step. +`chunks[]`: the matched pieces of your items, ranked. Preserve the order; it is the server ranking. + +| Field | Meaning | +|---|---| +| `chunk_id` | The chunk's id. Referenced from `graph[].triplets[].relation.chunk_id`. | +| `context_id` | The item this chunk came from. | +| `score` | Relevance. Always present. | +| `content` | The chunk's own text, verbatim. Enrichment is never concatenated into it. | +| `enrichment` | A plain string: what enrichment extracted from this chunk (a preference, a fact). Omitted when there is none. | +| `enrichment_kind` | The item's `context_category` (`user_preference`, `business_knowledge` or `decision_trace`). Omitted when none was declared; present even when `enrichment` is omitted. | +| `temporal` | Present only when the query engaged temporal reasoning: `{ content, start_date, end_date }` entries, where `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` and either date may be `null`. | + +Chunks carry nothing about their source: no title, url, collection or attributes. `llm_prompt` prints the title, collection, last-updated date and url for the model. To read an item's stored content yourself, call `GET /context/inspect` with the chunk's `context_id`; to read its title and attributes, call `POST /context/list` with `ids: [context_id]`. + +`graph[]`: paths through the context graph, query paths first, then paths expanded from the returned chunks. The list is deduplicated across both lanes (a path both found is reported once, as a `query_path`) and is not capped. `[]` when `graph_context` is `false` or nothing connects. + +| Field | Meaning | +|---|---| +| `origin` | `"query_path"`: grown from the entities in the query. `"chunk_relation"`: the neighbourhood of a returned chunk. | +| `triplets[].source` / `triplets[].target` | `{ entity_id, name }`. | +| `triplets[].relation.predicate` | The relation, for example `managed by`. | +| `triplets[].relation.context` | The sentence the relation was extracted from. | +| `triplets[].relation.temporal_details` | When the relation holds, for example `since June`. Omitted when empty. | +| `triplets[].relation.timestamp` | Epoch seconds (a float) for the edge. Omitted when the edge has none. | +| `triplets[].relation.relationship_id` | The relation's id. | +| `triplets[].relation.chunk_id` | The chunk the relation was extracted from. | +| `path_summary` | One sentence summarizing the path. Never empty. | + +`forceful_relations[]`: chunks pulled in because an item declared `forceful_relations` at ingest. Followed only in `thinking` mode; `[]` when none were declared, the query ran in `fast` mode, or `follow_forceful_relations` is `false`. + +| Field | Meaning | +|---|---| +| `via.from` | The `context_id` whose declared relation pulled this chunk in. May be `""`. | +| `via.to` | The returned chunk's own `context_id`. | +| `chunk` | The same shape as an entry of `chunks[]`. | + +`llm_prompt`: a server-built markdown string of everything above, ready to inject into a model call. See the next section. + +`meta` carries `request_id`, `api_version`, `latency_ms`, `database` and `collection` (omitted on a `collections` fan-out), plus `deprecation` when the request used a deprecated name. --- ## 9. Turning search results into LLM prompts -Practical guidance from `essentials/v2/api-results.mdx`: +**Inject `llm_prompt` verbatim.** It is the whole context block, already ranked, deduplicated and labelled for citation. You do not build a string from the other three keys. -- Preserve the order of `chunks`; it is the server ranking. -- Use graph context only when it helps the question. -- Use `additional_context` to attach forcefully related chunks by `chunk_uuid` / `extra_context_ids`. -- For personalized answers, `type: "all"` is simplest: one result set already merges knowledge and memories. -- Do not pass raw JSON directly to the LLM if token budget matters; use the SDK formatting helper to produce a compact context string. -- Include a grounding instruction: answer only from provided context, and say when context is insufficient. +### What `llm_prompt` contains -Use the SDK helper from `essentials/v2/api-results.mdx`: +Sections, in order (a section with nothing in it is left out; when the query returns nothing at all, `llm_prompt` is `""`): -- Python: `build_string(result)` from `hydra_db.helpers` -- TypeScript: `buildString(result)` from `@hydradb/sdk` +| Section | Contents | +|---|---| +| `# Query results` | The query, a `**Found:**` line counting what follows, and the line telling the model to cite a result by its number. | +| `## Results` | One `### 1. title` block per chunk, in ranked order: relevance, collection, type, category, id and last-updated date, the chunk's `content`, then `**Enrichment:**`. | +| `## Forceful relations` | One `### R1. title` block per forceful-relation chunk, with `**Linked from:**` naming the item that pulled it in. | +| `## Related facts` | One line per graph path, such as `- [P1] **Refunds** -managed_by→ **Finance** (relevance 0.81) [1]`, with the `path_summary` indented under it unless it only restates the chain. A path without a reranked score has no parenthetical. | +| `## Temporal facts` | One line per dated fact the query engaged, with its resolved window, citing its result. | +| `## Sources` | Each item once: title, type, id, url and last-updated date. | -Python: +Citation labels: -```python -import os -from hydra_db import HydraDB -from hydra_db.helpers import build_string -from openai import OpenAI +| Label | Refers to | +|---|---| +| `[1]`, `[2]`, ... | Result `### 1.`, `### 2.`: that entry of `chunks[]`. Its `**Id:**` is the `context_id`. | +| `[R1]`, `[R2]`, ... | Forceful relation `### R1.`, `### R2.`: that entry of `forceful_relations[]`. | +| `[P1]`, `[P2]`, ... | A related fact: path 1, 2, ... of `graph[]`. | + +A related fact or a temporal fact ends with the labels of the results it was extracted from. The numbers in `## Sources` count items, not results, and are not citation labels. + +A trimmed example: + +```markdown +# Query results + +**Query:** who owns refund processing? +**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation +Cite a result by its number in brackets, e.g. [1]. + +## Results + +### 1. Refund policy +- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge +- **Id:** refund-policy · **Last updated:** 2026-07-02 + +Refunds are processed within 30 days of purchase by the Finance Department. + +**Enrichment:** Refund window is 30 days; Finance owns refund processing. + +--- + +### 2. Support chat with Priya +- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference +- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29 + +user: Keep refund answers short please +assistant: Got it. -hydra = HydraDB(token=os.environ["HYDRA_DB_API_KEY"]) -openai = OpenAI(api_key=os.environ["OPENAI_API_KEY"]) +**Enrichment:** User prefers short answers about refunds. -question = "How does authentication work?" +## Forceful relations -result = hydra.query( - database="your-database", - collection="your-collection", +Linked to a result by the author at ingest time (forceful_relations), not by relevance to this query. + +### R1. Refund FAQ +- **Linked from:** refund-policy · **Collection:** support +- **Id:** refund-faq + +FAQ: refunds to a card take 5 to 7 business days to appear. + +## Related facts + +- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1] + Refund processing is managed by the Finance Department. +- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2] + The user prefers short answers about refunds. + +## Sources + +1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02 +2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29 +3. **Refund FAQ** (id: refund-faq) +``` + +### Injecting it + +```python +question = "How should I explain our refund policy to this user?" + +result = client.query( + database="acme", + collections={"user_alex": 2, "company": 1}, query=question, - type="knowledge", - query_by="hybrid", max_results=5, ) -context = build_string(result) - messages = [ { "role": "system", - "content": "Answer using only the provided HydraDB context. If the answer is not in the context, say so. Cite source titles or URLs when available.", - }, - { - "role": "user", - "content": f"{context}\n\nQuestion: {question}", + "content": ( + "Answer using only the context below. Cite results by their bracketed labels. " + "If the answer is not in the context, say so.\n\n" + result.data.llm_prompt + ), }, + {"role": "user", "content": question}, ] ``` -TypeScript: - ```ts -import { HydraDBClient } from "@hydradb/sdk"; -import { buildString } from "@hydradb/sdk/helpers"; -import OpenAI from "openai"; - -const hydra = new HydraDBClient({ token: process.env.HYDRA_DB_API_KEY }); -const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY }); - -const question = "How does authentication work?"; - -const result = await hydra.query({ - database: "your-database", - collection: "your-collection", +const result = await client.query({ + database: "acme", + collections: { user_alex: 2, company: 1 }, query: question, - type: "knowledge", - queryBy: "hybrid", maxResults: 5, }); -const context = buildString(result); - const messages = [ - { - role: "system", - content: "Answer using only the provided HydraDB context. If the answer is not in the context, say so. Cite source titles or URLs when available.", - }, - { - role: "user", - content: `${context}\n\nQuestion: ${question}`, - }, + { role: "system", content: "Answer using only the context below. Cite results by their bracketed labels. If the answer is not in the context, say so.\n\n" + result.data.llmPrompt }, + { role: "user", content: question }, ]; ``` -Common mistakes: +Surface `llm_prompt` to the model as it is and let the model cite the labels. Map a cited `[1]` back to `chunks[0].context_id` (and `[R1]` to `forceful_relations[0].chunk.context_id`) when you need to link a citation to an item. + +### Structured output instead of a prompt + +When you render results yourself (a UI, a reranker, an eval), read `chunks[].content`, `chunks[].enrichment`, `chunks[].enrichment_kind`, `graph[].path_summary` and `forceful_relations[]`. To hang graph hops under the chunk they came from: + +- Build a `chunk_id` to `context_id` map from `chunks[]` and `forceful_relations[].chunk`. Do not parse the chunk id string. +- Group hops by `relation.chunk_id` and match against that map. +- A `chunk_relation` path always has a hop from a returned chunk (or a forceful-relation chunk): hang it under that chunk. Show `query_path` paths as their own group; a query-path hop whose `chunk_id` matches a returned chunk may also sit under that chunk. + +### Common mistakes | Mistake | Fix | |---|---| -| Passing raw retrieval JSON to the LLM | Use SDK `build_string(result)` / `buildString(result)` | -| Omitting `collection` for memories | Use the same scope as ingestion | -| Including too many chunks | Start with `max_results: 10`; reduce for tight windows | +| Building your own context string from the JSON | Inject `llm_prompt` verbatim | +| Looking for a title or url on a chunk | Read it from `llm_prompt`, or call `GET /context/inspect` / `POST /context/list` with the `context_id` | +| Querying one collection when the answer spans two | Send `collections`, with the person's collection weighted higher | +| Including too many chunks | Start with `max_results: 10`; reduce for tight context windows | | Re-sorting chunks client-side | Preserve HydraDB ranking | -| Setting `graph_context: false` then expecting graph fields | Leave default true or set true explicitly | -| Using `query_forceful_relations` in `fast` mode | Use `mode: "thinking"` | +| Expecting `graph` after setting `graph_context: false` | Leave it at the default `true` | +| Expecting `forceful_relations` in `mode: "fast"` | Use `mode: "thinking"` | --- -## 10. Metadata guide +## 10. Attributes guide + +Two kinds of structured fields travel with an item: + +| Field | Declared? | Filterable? | Use for | +|---|---|---|---| +| `attributes` | Yes, in the database's `database_metadata_schema` | Yes, with the `/query` `attributes` filter | Hard scopes you know before retrieval runs: department, region, status, plan | +| `custom_attributes` | No | No | Free-form bookkeeping: author, provider ids, version | + +Neither is returned on query chunks. A user, team or workspace partition is a `collection`, not an attribute. -### Database schema +### Declaring attributes -Declared at `POST /databases`: +Declared at `POST /databases` (or added later with `PATCH /databases/{database}/metadata-schema`): ```json { "database_metadata_schema": [ - { - "name": "department", - "data_type": "VARCHAR", - "enable_match": true, - "enable_dense_embedding": false, - "enable_sparse_embedding": false, - "max_length": 1024 - } + { "name": "department", "data_type": "VARCHAR", "enable_match": true }, + { "name": "priority", "data_type": "INT64", "enable_match": true }, + { "name": "summary_label", "data_type": "VARCHAR", "enable_dense_embedding": true, "enable_sparse_embedding": true } ] } ``` -Field options: - | Field | Purpose | |---|---| -| `name` | metadata key; must be valid and non-reserved | -| `data_type` | friendly names like `string`, `integer`, `float`, `boolean`, or backend types like `VARCHAR`, `INT*`, `JSON` | -| `enable_match` | allows exact matching in top-level `metadata_filters` | -| `enable_dense_embedding` | semantically embed `VARCHAR` field | -| `enable_sparse_embedding` | BM25 index `VARCHAR` field | -| `max_length` | max string length | -| `searchable` | shorthand for dense + sparse embedding in docs | -| `filterable` | shorthand for `enable_match` in docs | - -### Where metadata belongs - -| Need | Put it in | Filter with | -|---|---|---| -| Hot-path exact filters: department, region, environment, status | `tenant_metadata` | top-level `metadata_filters` | -| Free-form display/bookkeeping: author, provider IDs, version | `additional_metadata` | `metadata_filters.additional_metadata` | -| Provider app fields for UI/citations | `additional_metadata` or app `fields` | usually not filtered unless needed | -| User/workspace partition | `collection` | not metadata | +| `name` | Attribute key. Starts with a letter or `_`; letters, numbers and underscores only; not a reserved system name such as `chunk_id`. | +| `data_type` | `VARCHAR`, `BOOL`, `INT8`, `INT16`, `INT32`, `INT64`, `FLOAT`, `DOUBLE`, `JSON`, or the aliases `string`, `boolean`, `integer`, `float`, `object`. Default `VARCHAR`. Arrays are not supported. | +| `enable_match` | Fast exact-match path for a field you filter on often. | +| `enable_dense_embedding` | Semantic search over a `VARCHAR` field. | +| `enable_sparse_embedding` | BM25 search over a `VARCHAR` field. | +| `max_length` | Max `VARCHAR` length. Default `1024`, maximum `65535`; cannot be raised later. | + +Limits: up to 32 declared fields, and up to 6 embedding flags per database (`enable_dense_embedding` and `enable_sparse_embedding` each count as one). Field names are unique case-insensitively. + +### Attaching attributes at ingest + +```json +{ + "context_id": "refund-policy", + "text": "Refunds are processed within 5 business days.", + "attributes": { "department": "support", "region": "us" }, + "custom_attributes": { "owner": "sam@acme.com", "reviewed": "2026-08-01" } +} +``` + +To change an indexed item's attributes, re-ingest it with the same `context_id` (`upsert: true` replaces it), or merge new values in place with `PATCH /context/{id}/metadata`, whose body names the two maps `database_metadata` (declared attributes) and `additional_metadata` (custom attributes). + +### Filtering with attributes + +`attributes` on `POST /query` is an operator filter over declared attributes. It applies to chunks, forceful relations and graph paths alike. + +| Operator | Meaning | +|---|---| +| `$eq`, `$ne` | equal, not equal (a bare scalar value means `$eq`) | +| `$gt`, `$gte`, `$lt`, `$lte` | numeric or ordered comparison | +| `$in`, `$nin` | value is, or is not, in an array | +| `$exists` | the field is, or is not, set (`true` / `false`) | +| `$and`, `$or` | an array of sub-filters | +| `$not` | a sub-filter to negate | + +```json +{ + "attributes": { + "$or": [ + { "department": "support" }, + { "$and": [ { "priority": { "$gte": 7 } }, { "region": { "$in": ["us", "eu"] } } ] } + ] + } +} +``` Rules: -- Plan hot filter fields before first ingest; undeclared scope keys can be ignored. -- Use exact equality filters; range/contains/fuzzy matching should be in the query or downstream reranker. -- To change metadata on an indexed context item, re-ingest with the same `id` and `upsert: true`. -- Do not rename metadata keys without re-ingesting affected sources. +- There is no `$contains` and no fuzzy match. Put fuzzy concepts in `query`, or declare a `VARCHAR` field with `enable_dense_embedding` and include the concept in the query. +- `custom_attributes` cannot be filtered. Declare the field and send it in `attributes` instead. +- A field the database's schema does not declare is a `400`, as is an empty object (`{}`) anywhere in the filter. +- Each list holds at most 500 values, the whole filter is capped at 64 KiB, and nesting is capped at 10 levels. +- Filters are hard constraints, not hints: a valid filter that matches nothing returns an empty result rather than widening the search. +- Plan hot filter fields before the first ingest, and keep attribute names stable. --- ## 11. Browse, fetch, relations, delete -### List documents or memories +None of these calls takes anything beyond `database`, an optional `collection`, and the fields shown. Send the collection you ingested into. + +### List items `POST /context/list` · `client.context.list()` +Lists every item in the collection, text and conversation alike, in one paginated listing. + ```ts -const sources = await client.context.list({ - database: "acme_corp", - type: "knowledge", +const page = await client.context.list({ + database: "acme", + collection: "company", page: 1, pageSize: 50, filters: { - tenant_metadata: { department: "support" }, - additional_metadata: { author: "alice" }, - source_fields: { type: "slack" }, + source_fields: { title: "Refund policy" }, }, + includeFields: ["title", "timestamp"], }); ``` Parameters: -- `database` -- `collection` -- `type: "knowledge" | "memory"` -- `ids` -- `page`, `page_size` (1–100) -- `filters.tenant_metadata` -- `filters.additional_metadata` -- `filters.source_fields` -- `include_fields` for projection +- `database`, `collection` +- `ids`: only these `context_id`s (filters and paging still apply) +- `page` (1-indexed, default `1`), `page_size` (`1` to `100`, default `50`) +- `filters`: exact-match constraints, ANDed. `source_fields` matches built-in fields such as `title`, `url`, `timestamp` and, for connector content, `app_provider`, `app_kind`, `app_external_id`, `app_parent_id`. The list filter keeps its own wire names for the two attribute maps: `filters.metadata` matches declared `attributes` and `filters.additional_metadata` matches `custom_attributes`. +- `include_fields`: projection, for example `["title", "timestamp"]`. `content` and `url` are not projectable (a `400`); read them with inspect. +- `acl`: list as an identity; only items it may see are returned. + +Response `data`: `{ success, message, sources: [...], total, pagination }`. `sources` is the wire name for the listed items: one row per item with its `id`, title, timestamp and stored attributes, without content. `pagination` carries `page`, `page_size`, `total`, `total_pages`, `has_next` and `has_previous`. -### Fetch original content +### Fetch an item's stored content `GET /context/inspect` · `client.context.inspect()` ```python -file = client.context.inspect( - database="acme_corp", - id="policy_v2", +item = client.context.inspect( + database="acme", + collection="company", + id="refund-policy", mode="both", expiry_seconds=3600, ) ``` -Modes: - | Mode | Returns | |---|---| -| `content` | parsed text or base64 for binary files | -| `url` | presigned URL | +| `content` (default) | the stored content in `content` | +| `url` | a time-limited `presigned_url` | | `both` | content plus presigned URL | +`expiry_seconds` sets the URL lifetime (default `3600`). With `acl`, the item must be visible to that identity or the response is `404`. + ### Inspect graph relations `GET /context/relations` · `client.context.relations()` ```ts const relations = await client.context.relations({ - database: "acme_corp", - id: "policy_v2", - type: "knowledge", - limit: 5000, + database: "acme", + collection: "company", + id: "refund-policy", + limit: 500, }); ``` -Use it for graph debugging and provenance inspection. It supports pagination via `cursor`. +Returns `relations[]`, triplet groups with their evidence (predicate, the sentence it came from, `chunk_id`, confidence). Omit `id` for relations across the whole collection. Page with `cursor`: pass back `next_cursor` until it is `null`. Use it for graph debugging and provenance. + +### Walk connected items + +`GET /context/{id}/subgraph` · `client.context.subgraph()` (the SDKs call the query-string form, `GET /context/subgraph`, which also accepts an id containing `/`) + +Returns every item reachable from one item through item-level links (declared forceful relations, a shared thread, parent and child), breadth-first up to `depth` hops, with the relations among them. An unknown id returns an empty subgraph, not an error. -### Delete sources or memories +### Delete items `DELETE /context` · `client.context.delete()` ```ts -await client.context.delete({ - type: "knowledge", - database: "acme_corp", - ids: ["policy_v2"], +const res = await client.context.delete({ + database: "acme", + collection: "company", + ids: ["refund-policy", "chat-alex-001"], }); ``` -Use `type: "memory"` to delete memory IDs. +One call deletes each item and everything derived from it (chunks and graph contribution), whatever its shape. + +Response `data`: `results[]` (per id: `id`, `deleted`, `error`), `deleted_count` and `message`. Read `deleted_count` and each `results[].deleted`; the nested `data.success` is deprecated and is `true` even when nothing was removed. + +- `deleted_count: 0` means the ids matched nothing in that scope. Check `collection`. +- An item that is still indexing refuses the whole request: nothing is deleted. By default the response is still `200`, with `deleted_count: 0`. Send the header `X-HydraDB-Delete-Status: strict` to get honest codes instead: `404` when nothing matched, `409` while an item is still indexing (retry after `Retry-After`), `500` when a store failed (retryable). --- @@ -1189,12 +1384,12 @@ Use `type: "memory"` to delete memory IDs. | Code | Meaning | Retry? | |---|---|---| -| `400` | invalid parameters / malformed request | No | -| `401` | missing/invalid API key | No | +| `400` | invalid parameters or malformed request | No | +| `401` | missing or invalid API key | No | | `403` | authenticated but not permitted | No | -| `404` | database/context item/memory not found | No | -| `409` | conflict, e.g. existing database or context item ID | Usually no | -| `413` | upload too large | No | +| `404` | database or context item not found | No | +| `409` | conflict: existing database, or (strict delete) an item still indexing | Usually no; retry a strict-delete `409` after indexing finishes | +| `413` | request body too large | No | | `422` | semantic validation failure, including `TENANT_INFRA_NOT_READY` | Only for `TENANT_INFRA_NOT_READY`, after polling readiness | | `429` | rate limited | Yes with backoff | | `500` | internal error | Yes with backoff | @@ -1205,44 +1400,45 @@ Use `type: "memory"` to delete memory IDs. | Code | Meaning | |---|---| -| `INVALID_INPUT` | required, malformed, or conflicting parameter. This is the code for essentially every `400` - branch on it, not on a generic name | +| `INVALID_INPUT` | required, malformed or conflicting parameter. This is the code for essentially every `400`: branch on it, not on a generic name | | `VALIDATION_ERROR` | request shape valid but failed validation | -| `UNAUTHORIZED` | missing/invalid bearer token | -| `FORBIDDEN` | key lacks access | -| `DATABASE_ALREADY_EXISTS` | duplicate database ID | -| `DATABASE_NOT_FOUND` | database missing/not visible | -| `TENANT_INFRA_NOT_READY` | `422` - the database exists but its infrastructure is still provisioning. This is what querying before polling readiness returns | -| `FILE_NOT_FOUND` | id not found **in the selected scope** - usually a `collection` mismatch rather than a missing source | -| `SOURCE_PROCESSING` | the source is still indexing and cannot serve this request yet | +| `UNAUTHORIZED` | missing or invalid bearer token | +| `FORBIDDEN` | key lacks access, or a plan limit prevents the operation | +| `DATABASE_ALREADY_EXISTS` | duplicate database name | +| `DATABASE_NOT_FOUND` | database missing or not visible to this key | +| `TENANT_INFRA_NOT_READY` | `422`: the database exists but its infrastructure is still provisioning. This is what calling it before polling readiness returns | +| `FILE_NOT_FOUND` | id not found **in the selected scope**: usually a `collection` mismatch rather than a missing item | +| `SOURCE_PROCESSING` | the item is still indexing and cannot serve this request yet | | `NOT_FOUND` | requested resource does not exist | -| `PARSE_ERROR` | a document could not be parsed | -| `PROCESSING_FAILED` | indexing failed for a source | +| `PROCESSING_FAILED` | indexing failed for an item | | `RATE_LIMITED` | rate limit exceeded | | `INTERNAL_ERROR` | unexpected server error | -| `BACKEND_ERROR` | `502` - an upstream dependency returned an error | -| `SERVICE_UNAVAILABLE` | dependency unavailable / under load | +| `BACKEND_ERROR` | `502`: an upstream dependency returned an error | +| `SERVICE_UNAVAILABLE` | dependency unavailable or under load | + +### Ingest validation (`400`) -There is no `INVALID_PARAMETERS` and no `SOURCE_NOT_FOUND` - earlier revisions of this guide listed both, and no HydraDB response has ever carried either. Match on `INVALID_INPUT` and `FILE_NOT_FOUND` instead. +The whole request is rejected, and the message names the item as `context[N]`, when an item has both `text` and `conversation` (or neither), a conversation breaks the rules in [Conversations](#conversations), `context_category` or `happened_at` is malformed, an `acl` principal is malformed, a `graph_payload` key matches no item, or a size limit in [Limits and validation](#limits-and-validation) is exceeded. -Retry only `429`, `500`, `503`; use bounded exponential backoff with jitter. +Retry only `429`, `500`, `502` and `503`; use bounded exponential backoff with jitter. SDK errors: ```python from hydra_db.core.api_error import ApiError -RETRYABLE = (429, 500, 503) +RETRYABLE = (429, 500, 502, 503) try: result = client.query( database="acme_corp", + collection="support", query="pricing tiers", - type="knowledge", ) except ApiError as exc: # 500 and 503 have typed subclasses (InternalServerError, # ServiceUnavailableError); 429 does not, so branch on status_code to cover - # all three uniformly. Back off and retry. + # all of them uniformly. Back off and retry. if exc.status_code not in RETRYABLE: raise ``` @@ -1253,8 +1449,8 @@ import { HydraDBError } from "@hydradb/sdk"; try { const result = await client.query({ database: "acme_corp", + collection: "support", query: "pricing tiers", - type: "knowledge", }); } catch (error) { if (error instanceof HydraDBError) { @@ -1268,211 +1464,84 @@ try { --- -## 13. Cookbook patterns +## 13. Common patterns -The cookbooks show production patterns. Click the links below to read the raw Markdown guides with complete setup code and schemas: +Most production integrations share one shape: -| Cookbook Guide | Purpose & Retrieval Patterns | Link to Guide | -|---|---|---| -| **Cursor for Docs** | Ingest codebase, PRs, Slack; multi-hop reasoning | [Guide Link](https://docs.hydradb.com/cookbooks/v2/cookbook-01-build-cursor-for-docs.md) | -| **Build your own Glean** | Enterprise workplace search across SaaS platforms | [Guide Link](https://docs.hydradb.com/cookbooks/v2/glean-clone.md) | -| **Internal Search Perplexity** | Citations and answer synthesis over Confluence/Slack | [Guide Link](https://docs.hydradb.com/cookbooks/v2/internal-search-perplexity.md) | -| **Customer Support Agent** | Personalizing help with user/plan-specific memory | [Guide Link](https://docs.hydradb.com/cookbooks/v2/customer-support-agent.md) | -| **IT Support / Notion AI** | Relational reasoning across Notion and Slack threads | [Guide Link](https://docs.hydradb.com/cookbooks/v2/cookbook-04-build-notion-ai.md) | -| **Financial Analyst** | Trend analysis, temporal metadata, precise citations | [Guide Link](https://docs.hydradb.com/cookbooks/v2/cookbook-10-ai-financial-analyst.md) | -| **LinkedIn Recruiter** | Exact skill/experience matching and post-filtering | [Guide Link](https://docs.hydradb.com/cookbooks/v2/ai-linkedin-recruiter.md) | -| **AI Travel Planner** | Collaborative planning using personalized context | [Guide Link](https://docs.hydradb.com/cookbooks/v2/ai-travel-planner.md) | -| **AI Onboarding Agent** | Org charts, specifications, and meeting transcripts | [Guide Link](https://docs.hydradb.com/cookbooks/v2/ai-onboarding-agent.md) | -| **Competitive Intelligence** | Competitor announcements, job listings, pricing | [Guide Link](https://docs.hydradb.com/cookbooks/v2/competitive-intelligence-agent.md) | -| **Chief of Staff / Actions** | Function routing, task tracking, and planning | [Guide Link](https://docs.hydradb.com/cookbooks/v2/ai-chief-of-staff.md) | - -Common architecture across cookbooks: - -1. Create database and schema for hot filters. -2. Ingest shared knowledge/app sources with stable IDs. -3. Ingest user/session memories with `collection`. -4. Poll status or use webhooks. -5. Query `/query` with `type: "knowledge"`, `"memory"`, or `"all"`. -6. Format chunks, graph paths, and additional context into an LLM prompt. -7. Cite sources and handle missing context explicitly. +1. Create a database, and declare in `database_metadata_schema` the attributes you will filter on. +2. Ingest shared context (policies, docs, tickets, extracted file text) into a shared collection with stable `context_id`s and a `context_category` of `business_knowledge`. Sync SaaS tools with [connectors](/essentials/v2/connectors) instead of re-implementing them. +3. Ingest each user's conversations and stated preferences into their own collection (`collection = user_id`), labelling preferences `user_preference` and agent decisions `decision_trace`. +4. Link items that belong together (a ticket and its follow-ups, a policy and its FAQ) with `forceful_relations`. +5. Poll `GET /context/status` or register a webhook. +6. Query with `collections: { "": 2, "company": 1 }`, `mode: "thinking"` for quality or `"fast"` for latency, and an `attributes` filter for hard scopes. +7. Inject `llm_prompt` into the model call with a grounding instruction, and let the model cite `[1]`, `[R1]`, `[P1]`. +8. Say so explicitly when the context does not contain the answer, and report it with `POST /feedback`. + +Typical uses of that shape: a support agent that answers from policy while respecting each customer's stated preferences; workplace search over Slack, Notion and Drive with citations; an onboarding assistant over org charts, specs and meeting notes; an agent that records its own decisions as `decision_trace` items and consults them before acting again. --- ## 14. Common mistakes checklist - [ ] Forgetting `API-Version: 2` in raw HTTP calls. -- [ ] Parsing raw HTTP response from the top level instead of `data`. +- [ ] Parsing raw HTTP responses from the top level instead of `data`. - [ ] Assuming the SDK unwraps the envelope; it returns the full envelope, so read the payload from `data`. -- [ ] Searching immediately after database creation without polling readiness. -- [ ] Searching immediately after ingestion without polling `context.status` or using webhooks. -- [ ] Treating `processing`/`queued` content as searchable. -- [ ] Not handling both `errored` and `failed` as failure statuses. +- [ ] Ingesting immediately after database creation without polling readiness. +- [ ] Querying immediately after ingestion without polling `context.status` or using webhooks. +- [ ] Treating `queued` or `processing` content as searchable. +- [ ] Not handling `errored` (and `failed`) as terminal failures. - [ ] Omitting `collection` on `context.status` and reading the resulting `FILE_NOT_FOUND` as a genuine indexing failure. - [ ] Using `??` on `errorMessage`, which is `""` rather than null on some failures, so the fallback never fires. -- [ ] Omitting `collection` for user memories. +- [ ] Sending both `text` and `conversation` on one item. +- [ ] Misspelling `context_category` (`"business-knowledge"` is a `400`). +- [ ] Sending a full timestamp in `happened_at`. +- [ ] Reusing untitled text without a `context_id`, so the second item replaces the first. - [ ] Writing with one `collection` and reading with another. -- [ ] Using metadata filters for user partitioning instead of `collection`. -- [ ] Using undeclared `tenant_metadata` keys in `metadata_filters`. -- [ ] Putting hot filters in free-form metadata instead of database schema. -- [ ] Expecting `query_forceful_relations` to work in `mode: "fast"`. -- [ ] Setting `graph_context: false` and expecting graph fields. +- [ ] Using attributes for user partitioning instead of `collection`. +- [ ] Filtering on `custom_attributes`, or on an attribute the schema does not declare. +- [ ] Building a prompt string by hand instead of injecting `llm_prompt`. +- [ ] Expecting a title, url or attributes on a chunk. +- [ ] Expecting `forceful_relations` in `mode: "fast"`. +- [ ] Setting `graph_context: false` and expecting graph paths. - [ ] Re-sorting chunks before prompting without a deliberate reranker. -- [ ] Passing too many chunks to the LLM. - [ ] Using `operator` without `query_by: "text"`. - [ ] Forgetting to verify webhook signatures when a signing secret is configured. --- -## 15. SDK Method Reference +## 15. SDK method reference -### 15.1 TypeScript SDK +Method names are the same in both SDKs except where noted; Python takes snake_case keyword arguments, TypeScript takes one camelCase object. | Method | Endpoint | Purpose | |---|---|---| -| `client.context.ingest()` | `POST /context/ingest` | Ingest documents, app sources, or memories | -| `client.context.status()` | `GET /context/status` | Check processing status of ingested items | -| `client.context.inspect()` | `GET /context/inspect` | Inspect original source content or get a presigned URL | -| `client.context.list()` | `POST /context/list` | Browse knowledge or memories | -| `client.context.delete()` | `DELETE /context` | Delete sources or memories | -| `client.context.relations()` | `GET /context/relations` | Inspect graph relationships | -| `client.query()` | `POST /query` | Unified search over knowledge, memories, or both | -| `client.databases.create()` | `POST /databases` | Create a new isolated workspace | -| `client.databases.list()` | `GET /databases` | List all databases registered for the organization | +| `client.databases.create()` | `POST /databases` | Create an isolated database | +| `client.databases.status()` | `GET /databases/status` | Check provisioning readiness | +| `client.databases.list()` | `GET /databases` | List the organization's databases | | `client.databases.delete()` | `DELETE /databases` | Permanently remove a database | -| `client.databases.status()` | `GET /databases/status` | Check provisioning readiness of a database's infrastructure | | `client.databases.collections()` | `GET /databases/collections` | List active collections | -| `client.databases.stats()` | `GET /databases/stats` | Get database usage stats | - -### 15.2 Python SDK - -| Group | Method | Endpoint | Description | -|---|---|---|---| -| Context | `client.context.ingest()` | `POST /context/ingest` | Ingest documents, app sources, or memories | -| Context | `client.context.status()` | `GET /context/status` | Check processing status of ingested items | -| Context | `client.context.inspect()` | `GET /context/inspect` | Inspect original source content or get a presigned URL | -| Context | `client.context.list()` | `POST /context/list` | Browse knowledge or memories | -| Context | `client.context.delete()` | `DELETE /context` | Delete sources or memories | -| Context | `client.context.relations()` | `GET /context/relations` | Inspect graph relationships | -| Search | `client.query()` | `POST /query` | Unified search over knowledge, memories, or both | -| Databases | `client.databases.create()` | `POST /databases` | Create a new isolated workspace | -| Databases | `client.databases.list()` | `GET /databases` | List all databases registered for the organization | -| Databases | `client.databases.delete()` | `DELETE /databases` | Permanently remove a database | -| Databases | `client.databases.status()` | `GET /databases/status` | Check provisioning readiness of a database's infrastructure | -| Databases | `client.databases.collections()` | `GET /databases/collections` | List active collections | -| Databases | `client.databases.stats()` | `GET /databases/stats` | Get database usage stats | - -### 15.3 Minimal cURL example - -Use `type` on raw `/query` requests to select `knowledge`, `memory`, or `all`. The raw API response is wrapped; read payloads from `.data`. - -```bash -export HYDRA_DB_API_KEY="your_api_key" -DATABASE="agent_docs_demo" -COLLECTION="user_alex" - -# 1. Create a database. -curl -s -X POST 'https://api.hydradb.com/databases' \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" \ - -H "Content-Type: application/json" \ - -d "{\"database\":\"${DATABASE}\"}" - -# 2. Check readiness. Poll this until .data.infra.ready_for_ingestion is true. -curl -s "https://api.hydradb.com/databases/status?database=${DATABASE}" \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" | jq '.data.infra' - -# 3. Ingest a memory. -INGEST_ID=$(curl -s -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" \ - -F "type=memory" \ - -F "database=${DATABASE}" \ - -F "collection=${COLLECTION}" \ - -F 'memories=[{"id":"alex_pref_001","text":"Alex prefers concise technical answers.","infer":true,"user_name":"Alex"}]' \ - | jq -r '.data.results[0].id') - -# 4. Poll indexing. -curl -s -G 'https://api.hydradb.com/context/status' \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" \ - --data-urlencode "database=${DATABASE}" \ - --data-urlencode "collection=${COLLECTION}" \ - --data-urlencode "ids=${INGEST_ID}" \ - | jq '.data.statuses[0]' - -# 5. Search memories. -curl -s -X POST 'https://api.hydradb.com/query' \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" \ - -H "Content-Type: application/json" \ - -d "{\"database\":\"${DATABASE}\",\"collection\":\"${COLLECTION}\",\"query\":\"How should I answer Alex?\",\"type\":\"memory\",\"query_by\":\"hybrid\",\"mode\":\"thinking\"}" \ - | jq '.data.chunks' -``` - -### 15.4 Raw HTTP: ingest app-source knowledge - -Use this for connector output where your app already extracted the text. `app_knowledge` must be a JSON string in multipart form data. - -```bash -curl -s -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" \ - -F "type=knowledge" \ - -F "database=acme_corp" \ - -F 'app_knowledge=[ - { - "id": "slack_C123_1715012345_000100", - "database": "acme_corp", - "collection": "default", - "title": "Pricing discussion - Slack #product", - "type": "slack", - "url": "https://slack.com/archives/C123/p1715012345000100", - "timestamp": "2025-01-15T12:30:00Z", - "content": { - "text": "We agreed on three tiers: Starter at $29, Pro at $79, Enterprise at $199." - }, - "tenant_metadata": { - "channel": "product", - "workspace": "acme" - }, - "additional_metadata": { - "author": "alice", - "slack_ts": "1715012345.000100" - }, - "relations": { - "ids": ["slack_thread_root_001"], - "properties": { "relation": "same_thread" } - } - } - ]' -``` - -Search it with app-aware retrieval: - -```bash -curl -s -X POST 'https://api.hydradb.com/query' \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" \ - -H "Content-Type: application/json" \ - -d '{ - "database": "acme_corp", - "query": "What pricing tiers did product agree on?", - "type": "knowledge", - "query_by": "hybrid", - "mode": "thinking", - "query_apps": true, - "graph_context": true - }' | jq '.data.chunks' -``` +| `client.databases.stats()` | `GET /databases/stats` | Row counts | +| `client.databases.update_metadata_schema()` (TS `updateMetadataSchema()`) | `PATCH /databases/{database}/metadata-schema` | Add declared attribute fields | +| `client.context.ingest()` | `POST /context/ingest` | Ingest text and conversation items | +| `client.context.status()` | `GET /context/status` | Check indexing status | +| `client.query()` | `POST /query` | Search; returns `chunks`, `graph`, `forceful_relations`, `llm_prompt` | +| `client.feedback.submit()` | `POST /feedback` | Report how a query performed | +| `client.context.list()` | `POST /context/list` | List items | +| `client.context.inspect()` | `GET /context/inspect` | Read an item's stored content or a presigned URL | +| `client.context.relations()` | `GET /context/relations` | Inspect graph relations | +| `client.context.subgraph()` | `GET /context/subgraph` | Walk an item's connected items | +| `client.context.update_source_metadata()` (TS `updateSourceMetadata()`) | `PATCH /context/{id}/metadata` | Merge new attribute values into an indexed item | +| `client.context.delete()` | `DELETE /context` | Delete items | + +Helpers: `verify_webhook_signature` (Python, `hydra_db.helpers`) and `verifyWebhookSignature` (TypeScript) verify webhook signatures. --- ## 16. Navigation and support -- Docs root for v2: `get-started/v2`, `essentials/v2`, `api-reference/v2`, `cookbooks/v2` +- Docs for v2: [Introduction](/get-started/v2/introduction), [Quickstart](/get-started/v2/quickstart), [Ingest context](/essentials/v2/ingest), [Query](/essentials/v2/query), [Attributes](/essentials/v2/attributes), [API reference](/api-reference/v2) - API reference OpenAPI: `api-reference/v2/openapi.json` -- Python package: `hydradb-sdk>=2,<3` -- TypeScript package: `@hydradb/sdk@^2` -- Dashboard/API keys: `https://app.hydradb.com` -- Support email in docs: `founders@hydradb.com` +- Python package: `hydradb-sdk>=2.1.5,<3` +- TypeScript package: `@hydradb/sdk@^2.1.5` +- Dashboard and API keys: `https://app.hydradb.com` +- Support email: `founders@hydradb.com` diff --git a/api-reference/endpoint/add-memory.mdx b/api-reference/endpoint/add-memory.mdx index d480bdb4..d9e943c9 100644 --- a/api-reference/endpoint/add-memory.mdx +++ b/api-reference/endpoint/add-memory.mdx @@ -2,8 +2,11 @@ title: "Add Memory" description: "Ingest user memories – preferences, conversation history, or inline text." openapi: "POST /memories/add_memory" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it Use this endpoint to store user-specific context that personalizes recall: diff --git a/api-reference/endpoint/boolean-recall.mdx b/api-reference/endpoint/boolean-recall.mdx index 4d2147d0..7cfa13cc 100644 --- a/api-reference/endpoint/boolean-recall.mdx +++ b/api-reference/endpoint/boolean-recall.mdx @@ -2,8 +2,11 @@ title: "Boolean Recall" description: "Deterministic full-text search – exact term matching, no embeddings." openapi: "POST /recall/boolean_recall" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it `/recall/boolean_recall` runs a BM25-style full-text search. It's the right tool when: diff --git a/api-reference/endpoint/create-tenant.mdx b/api-reference/endpoint/create-tenant.mdx index 115dc03f..d2b7faba 100644 --- a/api-reference/endpoint/create-tenant.mdx +++ b/api-reference/endpoint/create-tenant.mdx @@ -2,8 +2,11 @@ title: "Create Tenant" description: "Create an isolated workspace for your data." openapi: "POST /tenants/create" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it Creating a new tenant is the first step before any ingestion or recall. A tenant is a fully isolated workspace – no tenant can read another tenant's data. diff --git a/api-reference/endpoint/delete-knowledge.mdx b/api-reference/endpoint/delete-knowledge.mdx index b8efe12b..fb258221 100644 --- a/api-reference/endpoint/delete-knowledge.mdx +++ b/api-reference/endpoint/delete-knowledge.mdx @@ -2,8 +2,11 @@ title: "Delete Knowledge" description: "Permanently delete one or more knowledge sources by ID." openapi: "POST /knowledge/delete_knowledge" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Cleanup** – remove outdated documents diff --git a/api-reference/endpoint/delete-memory.mdx b/api-reference/endpoint/delete-memory.mdx index 41afd1f9..3ecd2046 100644 --- a/api-reference/endpoint/delete-memory.mdx +++ b/api-reference/endpoint/delete-memory.mdx @@ -2,8 +2,11 @@ title: "Delete Memory" description: "Permanently delete a single memory by its ID." openapi: "DELETE /memories/delete_memory" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Right-to-deletion** – remove a specific user's memory upon request diff --git a/api-reference/endpoint/delete-tenant.mdx b/api-reference/endpoint/delete-tenant.mdx index 31f5b886..327ba020 100644 --- a/api-reference/endpoint/delete-tenant.mdx +++ b/api-reference/endpoint/delete-tenant.mdx @@ -2,8 +2,11 @@ title: "Delete Tenant" description: "Permanently delete a tenant and all its data." openapi: "DELETE /tenants/delete" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Offboarding** – a customer has churned and you need to remove their data diff --git a/api-reference/endpoint/fetch-content.mdx b/api-reference/endpoint/fetch-content.mdx index c025dfbc..f0c39342 100644 --- a/api-reference/endpoint/fetch-content.mdx +++ b/api-reference/endpoint/fetch-content.mdx @@ -2,8 +2,11 @@ title: "Fetch Content" description: "Retrieve the original content of an ingested source." openapi: "POST /fetch/content" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Download original files** – PDFs, DOCX, etc. that you uploaded diff --git a/api-reference/endpoint/full-recall.mdx b/api-reference/endpoint/full-recall.mdx index e5ff9ea0..d1d41839 100644 --- a/api-reference/endpoint/full-recall.mdx +++ b/api-reference/endpoint/full-recall.mdx @@ -2,8 +2,11 @@ title: "Full Recall" description: "Hybrid semantic + graph + metadata recall over knowledge sources." openapi: "POST /recall/full_recall" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it `/recall/full_recall` is the primary endpoint for retrieving knowledge (documents, app sources). It runs a multi-stage pipeline that combines: diff --git a/api-reference/endpoint/graph-relations.mdx b/api-reference/endpoint/graph-relations.mdx index b2cbb552..4551c203 100644 --- a/api-reference/endpoint/graph-relations.mdx +++ b/api-reference/endpoint/graph-relations.mdx @@ -2,8 +2,11 @@ title: "Graph Relations By Source ID" description: "Retrieve all entity relationships extracted from a single source." openapi: "GET /list/graph_relations_by_id" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Debugging graph construction** – verify HydraDB extracted the relationships you expected diff --git a/api-reference/endpoint/infra-status.mdx b/api-reference/endpoint/infra-status.mdx index 76406cd0..383fa356 100644 --- a/api-reference/endpoint/infra-status.mdx +++ b/api-reference/endpoint/infra-status.mdx @@ -2,8 +2,11 @@ title: "Check Infra Status" description: "Check whether a tenant's infrastructure is fully provisioned." openapi: "GET /tenants/infra/status" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **After tenant creation** – poll until provisioning completes diff --git a/api-reference/endpoint/ingestion-overview.mdx b/api-reference/endpoint/ingestion-overview.mdx index 029339f8..6c6cd835 100644 --- a/api-reference/endpoint/ingestion-overview.mdx +++ b/api-reference/endpoint/ingestion-overview.mdx @@ -1,8 +1,11 @@ --- title: "Ingestion - Overview" description: "Quick reference for ingestion endpoints, their lifecycle, and when to use which." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## Lifecycle ```mermaid diff --git a/api-reference/endpoint/list-data.mdx b/api-reference/endpoint/list-data.mdx index c38268ca..b10d1948 100644 --- a/api-reference/endpoint/list-data.mdx +++ b/api-reference/endpoint/list-data.mdx @@ -2,8 +2,11 @@ title: "List Data" description: "Paginated listing of stored knowledge or memories with optional filters." openapi: "POST /list/data" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Admin dashboards** – browse what's been ingested into a tenant diff --git a/api-reference/endpoint/list-overview.mdx b/api-reference/endpoint/list-overview.mdx index fab3e9bf..47609c34 100644 --- a/api-reference/endpoint/list-overview.mdx +++ b/api-reference/endpoint/list-overview.mdx @@ -1,8 +1,11 @@ --- title: "List – Overview" description: "Browse stored knowledge and memories, and inspect their graph relationships." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## Endpoint reference | Endpoint | Method | Purpose | diff --git a/api-reference/endpoint/list-sub-tenant-ids.mdx b/api-reference/endpoint/list-sub-tenant-ids.mdx index a7a70355..332cfe2a 100644 --- a/api-reference/endpoint/list-sub-tenant-ids.mdx +++ b/api-reference/endpoint/list-sub-tenant-ids.mdx @@ -2,8 +2,11 @@ title: "List Sub-Tenant IDs" description: "List all sub-tenants that have indexed data under a tenant." openapi: "GET /tenants/sub_tenant_ids" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Audit** – confirm which sub-tenants exist for compliance or usage reporting diff --git a/api-reference/endpoint/list-tenant-ids.mdx b/api-reference/endpoint/list-tenant-ids.mdx index 778471d3..26f06959 100644 --- a/api-reference/endpoint/list-tenant-ids.mdx +++ b/api-reference/endpoint/list-tenant-ids.mdx @@ -1,8 +1,11 @@ --- title: "List Tenant IDs" description: "List all tenants registered for your organization." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Org-level admin dashboards** – show all tenants belonging to an organization diff --git a/api-reference/endpoint/memories-overview.mdx b/api-reference/endpoint/memories-overview.mdx index d61ee57f..f0cd067b 100644 --- a/api-reference/endpoint/memories-overview.mdx +++ b/api-reference/endpoint/memories-overview.mdx @@ -1,8 +1,11 @@ --- title: "Memories – Overview" description: "Quick reference for all Memories endpoints, their lifecycle, and when to call each." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## Lifecycle ```mermaid diff --git a/api-reference/endpoint/monitor-tenant.mdx b/api-reference/endpoint/monitor-tenant.mdx index 69b3d40f..8ce6fb44 100644 --- a/api-reference/endpoint/monitor-tenant.mdx +++ b/api-reference/endpoint/monitor-tenant.mdx @@ -2,8 +2,11 @@ title: "Monitor Tenant" description: "Get usage statistics for a tenant – object counts." openapi: "GET /tenants/monitor" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it - **Verify ingestion** – confirm data landed after upload (expect `row_count` to grow) diff --git a/api-reference/endpoint/recall-overview.mdx b/api-reference/endpoint/recall-overview.mdx index 9acb0f64..82692e0c 100644 --- a/api-reference/endpoint/recall-overview.mdx +++ b/api-reference/endpoint/recall-overview.mdx @@ -1,8 +1,11 @@ --- title: "Recall – Overview" description: "Quick reference for all Recall endpoints, their differences, and when to call each." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## Which recall endpoint should I use? ```mermaid diff --git a/api-reference/endpoint/recall-preferences.mdx b/api-reference/endpoint/recall-preferences.mdx index be664678..34eab5c0 100644 --- a/api-reference/endpoint/recall-preferences.mdx +++ b/api-reference/endpoint/recall-preferences.mdx @@ -2,8 +2,11 @@ title: "Recall Preferences" description: "Hybrid recall over user memories – preferences, conversation history, inline notes." openapi: "POST /recall/recall_preferences" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it `/recall/recall_preferences` retrieves user-specific context that was ingested via [`POST /memories/add_memory`](/api-reference/endpoint/add-memory). Use it when your agent needs to: diff --git a/api-reference/endpoint/tenants-overview.mdx b/api-reference/endpoint/tenants-overview.mdx index 2961b1fa..dbc344f3 100644 --- a/api-reference/endpoint/tenants-overview.mdx +++ b/api-reference/endpoint/tenants-overview.mdx @@ -1,8 +1,11 @@ --- title: "Tenants – Overview" description: "Quick reference for all Tenants endpoints, their lifecycle, and when to call each." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## Lifecycle ```mermaid diff --git a/api-reference/endpoint/upload-knowledge.mdx b/api-reference/endpoint/upload-knowledge.mdx index 0f589ad1..17af78f8 100644 --- a/api-reference/endpoint/upload-knowledge.mdx +++ b/api-reference/endpoint/upload-knowledge.mdx @@ -2,8 +2,11 @@ title: "Upload Knowledge" description: "Ingest documents (files) and/or app-generated content (app knowledge) into a tenant." openapi: "POST /ingestion/upload_knowledge" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it Use this endpoint to add knowledge that your agents will recall later: diff --git a/api-reference/endpoint/verify-processing.mdx b/api-reference/endpoint/verify-processing.mdx index d1b499b5..53f1de4f 100644 --- a/api-reference/endpoint/verify-processing.mdx +++ b/api-reference/endpoint/verify-processing.mdx @@ -2,8 +2,11 @@ title: "Verify Processing" description: "Check the processing status of ingested content." openapi: "POST /ingestion/verify_processing" +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## When to use it Both `/ingestion/upload_knowledge` and `/memories/add_memory` are asynchronous. After ingesting, use this endpoint to check whether content is fully indexed and ready to be recalled. diff --git a/api-reference/error-responses.mdx b/api-reference/error-responses.mdx index d8013388..e4dbc58b 100644 --- a/api-reference/error-responses.mdx +++ b/api-reference/error-responses.mdx @@ -1,8 +1,11 @@ --- title: "Error Responses" description: "HTTP error codes, response shapes, and handling patterns across all HydraDB endpoints." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/api-reference/index.mdx b/api-reference/index.mdx index cbb14f02..45c084bb 100644 --- a/api-reference/index.mdx +++ b/api-reference/index.mdx @@ -1,8 +1,11 @@ --- title: "API Reference" description: "Complete reference for every HydraDB endpoint, organized by group." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## Quick links - **New to HydraDB?** Start with the [Quickstart](/get-started/quickstart) diff --git a/api-reference/sdks.mdx b/api-reference/sdks.mdx index 774490d9..c46061be 100644 --- a/api-reference/sdks.mdx +++ b/api-reference/sdks.mdx @@ -1,8 +1,11 @@ --- title: "SDKs – Node and Python" description: "Official TypeScript/Node.js and Python SDKs for the HydraDB platform." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/api-reference/v2/endpoint/connectors-overview.mdx b/api-reference/v2/endpoint/connectors-overview.mdx index 3e1cdfed..3d467322 100644 --- a/api-reference/v2/endpoint/connectors-overview.mdx +++ b/api-reference/v2/endpoint/connectors-overview.mdx @@ -3,7 +3,7 @@ title: "Connectors - Overview" description: "Quick reference for all connector endpoints, their lifecycle, and when to call each." --- -Connectors continuously sync external app data (Slack, GitHub, Linear, Notion, Gmail) into your knowledge store without manual ingestion. +Connectors continuously sync external app data (Slack, GitHub, Linear, Notion, Gmail) into your database without manual ingestion. ## Endpoint references diff --git a/api-reference/v2/endpoint/create-tenant.mdx b/api-reference/v2/endpoint/create-tenant.mdx index e3d03ad4..b36d508e 100644 --- a/api-reference/v2/endpoint/create-tenant.mdx +++ b/api-reference/v2/endpoint/create-tenant.mdx @@ -1,7 +1,6 @@ --- title: "Create Database" description: "Creates a space for storing context. " -openapi: "api-reference/v2/openapi.json POST /databases" --- import { Field } from "/snippets/field.jsx"; @@ -86,11 +85,29 @@ curl -X POST 'https://api.hydradb.com/databases' \ | Name | Description | | --- | --- | | | Account-scoped database identifier. Use a stable, case-sensitive ID up to 25 characters; prefer lowercase letters, numbers, and underscores for portability. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | -| | Defines database-level metadata fields. See the [Scoping using metadata](/essentials/v2/attributes#step-1a-declare-the-schema-at-database-creation) guide for detailed schema parameters. Formerly `tenant_metadata_schema`; the `tenant_metadata_schema` alias is still accepted (deprecated). (default=`null`) | +| | Defines database-level metadata fields, the declared attributes you can filter on. Each entry is a schema field (below). See [Declare the schema](/essentials/v2/attributes#2-declare-the-schema) for detailed schema parameters. Formerly `tenant_metadata_schema`; the `tenant_metadata_schema` alias is still accepted (deprecated). (default=`null`) | + +### Schema field + +| Name | Description | +| --- | --- | +| | Field name. Must start with a letter, contain only letters, numbers and underscores, and not be a reserved system name. Immutable after creation. | +| | `VARCHAR`, `BOOL`, `INT8`, `INT16`, `INT32`, `INT64`, `FLOAT`, `DOUBLE`, `JSON`, or the friendly aliases `string`, `integer`, `float`, `boolean`, `object`. `ARRAY` is rejected with `400`; for a multi-value field declare `VARCHAR` and store the values comma-joined. (default=`"VARCHAR"`) | +| | Maximum length for a `VARCHAR` field, up to `65535`. (default=`1024`) | +| | Enables exact-match filtering on this field. (default=`false`) | +| | Adds dense semantic search over a `VARCHAR` field. (default=`false`) | +| | Adds sparse (BM25) search over a `VARCHAR` field. (default=`false`) | ## Successful response -Always check if a database is ready before using it. Use [Database Status](/api-reference/v2/endpoint/tenant-status) to check. +Creation is asynchronous: the call returns as soon as provisioning starts. Always check if a database is ready before using it. Use [Database Status](/api-reference/v2/endpoint/tenant-status) to check. + +| Name | Description | +| --- | --- | +| | `accepted`: provisioning has started in the background. | +| | The database being created. | +| | Human-readable result message. | +| | Deprecated alias for `database`, carrying the same value. | diff --git a/api-reference/v2/endpoint/delete-collection.mdx b/api-reference/v2/endpoint/delete-collection.mdx index ab8121de..02267458 100644 --- a/api-reference/v2/endpoint/delete-collection.mdx +++ b/api-reference/v2/endpoint/delete-collection.mdx @@ -6,7 +6,7 @@ openapi: "api-reference/v2/openapi.json DELETE /databases/collections" import { Field } from "/snippets/field.jsx"; -This action is irreversible. Deleting a collection removes all knowledge, memories, embeddings, and graph data stored under that collection. The parent database and its other collections are not affected. There is no soft-delete and no recovery window. +This action is irreversible. Deleting a collection removes all context items, embeddings, and graph data stored under that collection. The parent database and its other collections are not affected. There is no soft-delete and no recovery window. ```python Python SDK @@ -85,7 +85,7 @@ You do not need one to reuse the name safely. Ingestion creates a missing collec ## Behavior notes -**Irreversible action.** Ingested documents, memories, embeddings, graph nodes, and storage objects for this collection are permanently removed. Other collections in the same database are not touched. There is no recovery window. +**Irreversible action.** Ingested context items, embeddings, graph nodes, and storage objects for this collection are permanently removed. Other collections in the same database are not touched. There is no recovery window. - **Async cleanup:** The endpoint returns immediately after accepting the request. Cleanup of vector stores, graphs, and storage objects runs in the background. @@ -103,7 +103,7 @@ Common codes: `400 VALIDATION_ERROR`, `404 DATABASE_NOT_FOUND`, `401 UNAUTHORIZE **Related Resources** - **Before this:** [List Collections](/api-reference/v2/endpoint/list-sub-tenants) - find the collection ID -- **Alternative:** [Delete Context](/api-reference/v2/endpoint/delete-source) - remove specific knowledge or memories without deleting the collection +- **Alternative:** [Delete Context](/api-reference/v2/endpoint/delete-source): remove specific context items without deleting the collection - **Larger scope:** [Delete Database](/api-reference/v2/endpoint/delete-tenant) - remove the entire database - **Read more:** [Concepts → Multi tenancy](/essentials/v2/databases-and-collections) diff --git a/api-reference/v2/endpoint/delete-connector-resource.mdx b/api-reference/v2/endpoint/delete-connector-resource.mdx index 673a1991..e9cb767d 100644 --- a/api-reference/v2/endpoint/delete-connector-resource.mdx +++ b/api-reference/v2/endpoint/delete-connector-resource.mdx @@ -4,7 +4,7 @@ description: "Remove a resource from a connector and stop syncing it." openapi: "api-reference/v2/openapi.json DELETE /connectors/{id}/resources/{resource_id}" --- -Removes a configured resource and stops syncing it. Objects already ingested into the knowledge store are not removed. +Removes a configured resource and stops syncing it. Objects already ingested into your database are not removed. diff --git a/api-reference/v2/endpoint/delete-connector.mdx b/api-reference/v2/endpoint/delete-connector.mdx index d714effb..65c1724e 100644 --- a/api-reference/v2/endpoint/delete-connector.mdx +++ b/api-reference/v2/endpoint/delete-connector.mdx @@ -4,7 +4,7 @@ description: "Permanently remove a connector and stop all associated syncs." openapi: "api-reference/v2/openapi.json DELETE /connectors/{id}" --- -Deletes the connector and all its configured resources. Synced objects already ingested into the knowledge store are not removed. +Deletes the connector and all its configured resources. Synced objects already ingested into your database are not removed. diff --git a/api-reference/v2/endpoint/delete-source.mdx b/api-reference/v2/endpoint/delete-source.mdx index 9f7827a8..bb4aeece 100644 --- a/api-reference/v2/endpoint/delete-source.mdx +++ b/api-reference/v2/endpoint/delete-source.mdx @@ -1,27 +1,16 @@ --- title: "Delete Context" -description: "Delete knowledge or memories by their IDs." -openapi: "api-reference/v2/openapi.json DELETE /context" +description: "Delete context items by their IDs." --- import { Field } from "/snippets/field.jsx"; -Specify the resource category with the `type` parameter: - -- `type=knowledge` _(default)_ - delete knowledge sources. -- `type=memory` - delete memories. - -Pass one or more IDs in `ids`. Send `database`, `collection`, `ids`, and `type` as top-level fields in the request body. - -### Knowledge deletion - -Use `type: "knowledge"` and pass knowledge `ids` in the `ids` array. Include the same `collection` you used when ingesting the knowledge; omitting it targets the default collection. +Pass one or more IDs in `ids` to delete those context items, whether you ingested them or a connector synced them. Send `database`, `collection`, and `ids` as top-level fields in the request body. Include the same `collection` you used when ingesting; omitting it targets the default collection. ```python Python SDK client.context.delete( - type="knowledge", database="acme_corp", collection="team_docs", ids=["policy_main", "runbook_deploy"], @@ -30,7 +19,6 @@ client.context.delete( ```typescript TypeScript SDK await client.context.delete({ - type: "knowledge", database: "acme_corp", collection: "team_docs", ids: ["policy_main", "runbook_deploy"], @@ -45,75 +33,34 @@ curl -X DELETE 'https://api.hydradb.com/context' \ -d '{ "database": "acme_corp", "collection": "team_docs", - "ids": ["policy_main", "runbook_deploy"], - "type": "knowledge" + "ids": ["policy_main", "runbook_deploy"] }' ``` - - -```json Success -{ - "success": true, - "data": { - "success": true, - "message": "Delete completed", - "results": [ - { "id": "policy_main", "deleted": true }, - { "id": "runbook_deploy", "deleted": true } - ], - "deleted_count": 2 - }, - "error": null, - "meta": { - "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", - "latency_ms": 12.3 - } -} -``` - - - -### Memory deletion - -Use `type: "memory"` and pass memory `ids` in the `ids` array. Include the same `collection` you used when ingesting the memories; omitting it targets the default collection. +## Request body - +| Name | Description | +| --- | --- | +| | Database to delete from. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | +| | Collection scope. If omitted, the default collection is used. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=`null`) | +| | IDs of the context items to delete. At least one. | -```python Python SDK -client.context.delete( - type="memory", - database="acme_corp", - collection="user_alex", - ids=["mem_user_alex_tone"], -) -``` +## Request headers -```typescript TypeScript SDK -await client.context.delete({ - type: "memory", - database: "acme_corp", - collection: "user_alex", - ids: ["mem_user_alex_tone"], -}); -``` +| Name | Description | +| --- | --- | +| | Selects the status behaviour for this request. `strict` opts in to honest `404` / `409` / `500` codes when the delete did not happen; `legacy` forces the unconditional `200`. Omitted, the server default applies (currently `legacy`). See [Status codes](#status-codes). | -```bash cURL -curl -X DELETE 'https://api.hydradb.com/context' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -H "Content-Type: application/json" \ - -d '{ - "database": "acme_corp", - "collection": "user_alex", - "ids": ["mem_user_alex_tone"], - "type": "memory" - }' -``` +## Response - +| Name | Description | +| --- | --- | +| | One entry per requested ID: `id`, `deleted` (whether this item was removed), and `error` (why not, present only when `deleted` is `false`). | +| | Number of items actually removed. `0` means nothing was deleted. | +| | Human-readable result message. | +| | Deprecated. `true` only when something was removed; read `deleted_count` and `results[]` instead. | @@ -122,12 +69,12 @@ curl -X DELETE 'https://api.hydradb.com/context' \ "success": true, "data": { "success": true, - "message": "Successfully deleted 1 memory source(s)", + "message": "Successfully deleted 2 source(s)", "results": [ - { "id": "mem_user_alex_tone", "deleted": true } + { "id": "policy_main", "deleted": true }, + { "id": "runbook_deploy", "deleted": true } ], - "deleted_count": 1, - "user_memory_deleted": 1 + "deleted_count": 2 }, "error": null, "meta": { @@ -145,13 +92,12 @@ curl -X DELETE 'https://api.hydradb.com/context' \ "message": "No sources were deleted", "results": [ { - "id": "mem_user_alex_tone", + "id": "policy_old", "deleted": false, - "error": "Memory not found or deletion failed" + "error": "Source not found or deletion failed" } ], - "deleted_count": 0, - "user_memory_deleted": 0 + "deleted_count": 0 }, "error": { "code": "NOT_FOUND", @@ -177,7 +123,7 @@ curl -X DELETE 'https://api.hydradb.com/context' \ -H "Authorization: Bearer " \ -H "API-Version: 2" \ -H "Content-Type: application/json" \ - -d '{ "database": "acme_corp", "ids": ["policy_main"], "type": "knowledge" }' + -d '{ "database": "acme_corp", "ids": ["policy_main"] }' ``` ### Opt in to honest status codes @@ -193,7 +139,7 @@ curl -X DELETE 'https://api.hydradb.com/context' \ -H "API-Version: 2" \ -H "X-HydraDB-Delete-Status: strict" \ -H "Content-Type: application/json" \ - -d '{ "database": "acme_corp", "ids": ["policy_main"], "type": "knowledge" }' + -d '{ "database": "acme_corp", "ids": ["policy_main"] }' ``` In strict mode: @@ -272,9 +218,8 @@ The header always wins. Without it, the server default applies. ## Some additional notes -- **Partial-success semantics:** For `type=knowledge`, each ID is reported independently in `results[]`. A failure on one ID does not stop the rest. For `type=memory`, the response reports an aggregate `user_memory_deleted` reflecting _all_ listed IDs. One exception: if any source in the request is still indexing, the whole request is refused and nothing is deleted - reported as `409` in strict mode, and as a `200` with `deleted_count: 0` by default. -- **Retrieval drops the source immediately:** Even before background cleanup finishes, deleted IDs disappear from `/query` and `/context/list` responses. -- **Mixed deletes need two calls:** To delete both knowledge and memory items, send two requests - one with `type=knowledge`, one with `type=memory`. +- **Partial-success semantics:** Each ID is reported independently in `results[]`, and `deleted_count` totals the items actually removed. An ID that matched nothing comes back with `deleted: false` and an `error`, and does not stop the rest. One exception: if any item in the request is still indexing, the whole request is refused and nothing is deleted. That is reported as `409` in strict mode, and as a `200` with `deleted_count: 0` by default. +- **Retrieval drops the item immediately:** Even before background cleanup finishes, deleted IDs disappear from `/query` and `/context/list` responses.
diff --git a/api-reference/v2/endpoint/delete-tenant.mdx b/api-reference/v2/endpoint/delete-tenant.mdx index 85dafa94..aadb8e57 100644 --- a/api-reference/v2/endpoint/delete-tenant.mdx +++ b/api-reference/v2/endpoint/delete-tenant.mdx @@ -6,7 +6,7 @@ openapi: "api-reference/v2/openapi.json DELETE /databases" import { Field } from "/snippets/field.jsx"; -This action is irreversible. Deleting a database removes all of its associated data, including ingested content, memories, metadata schema, vector indices, and graphs. There is no soft-delete and no recovery window. +This action is irreversible. Deleting a database removes all of its associated data, including all context items, embeddings, graph data, and the metadata schema. There is no soft-delete and no recovery window. The examples below use a placeholder name, `database_to_delete`. Replace it with the database you actually mean to destroy before running them - and check the name twice on a shared or team account, where you may not be the only one using it. @@ -77,7 +77,7 @@ After deletion completes, the same `database` can be used in a new `POST /databa ## Behavior notes -**Irreversible action.** Ingested documents, memories, embeddings, graph nodes, metadata schema, and storage objects are permanently removed. There is no recovery window - ensure you have a backup if the content matters. +**Irreversible action.** Ingested context items, embeddings, graph nodes, metadata schema, and storage objects are permanently removed. There is no recovery window, so ensure you have a backup if the content matters. - **Stop in-flight work first:** Stop all ingestion, polling, query, and background jobs targeting this database before deleting. Calls made after deregistration can fail with `DATABASE_NOT_FOUND` even while infrastructure cleanup is still running. @@ -95,6 +95,6 @@ Common codes: `404 DATABASE_NOT_FOUND`, `401 UNAUTHORIZED`, `422 VALIDATION_ERRO - **Before this:** [List Databases](/api-reference/v2/endpoint/list-tenants) - find the database ID - **Alternative:** [Delete Collection](/api-reference/v2/endpoint/delete-collection) - remove one collection without deleting the whole database -- **Alternative:** [Delete Context](/api-reference/v2/endpoint/delete-source) - remove specific knowledge or memories without deleting the whole database +- **Alternative:** [Delete Context](/api-reference/v2/endpoint/delete-source): remove specific context items without deleting the whole database - **Read more:** [Concepts → Multi-Tenant Support](/essentials/v2/databases-and-collections) diff --git a/api-reference/v2/endpoint/fetch-content.mdx b/api-reference/v2/endpoint/fetch-content.mdx index 8dcfa551..b7f2ed06 100644 --- a/api-reference/v2/endpoint/fetch-content.mdx +++ b/api-reference/v2/endpoint/fetch-content.mdx @@ -1,17 +1,17 @@ --- title: "Inspect Context" -description: "Inspect the content of a knowledge or memory source." +description: "Inspect the stored content of a context item." openapi: "api-reference/v2/openapi.json GET /context/inspect" --- import { Field } from "/snippets/field.jsx"; -Specify the `id` of the knowledge or memory you want to retrieve. +Specify the `id` of the context item you want to retrieve. The response carries the stored item, the enrichment the server wrote for it, and a download link, depending on `mode`. ```python Python SDK -file = client.context.inspect( +item = client.context.inspect( id="policy_main", database="acme_corp", mode="both", @@ -20,7 +20,7 @@ file = client.context.inspect( ``` ```typescript TypeScript SDK -const file = await client.context.inspect({ +const item = await client.context.inspect({ id: "policy_main", database: "acme_corp", mode: "both", @@ -45,7 +45,7 @@ curl -G 'https://api.hydradb.com/context/inspect' \ | Name | Description | | --- | --- | -| | ID of the source to fetch. | +| | ID of the context item to fetch. | | | Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | | Collection scope. If omitted, the default collection is used. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=`null`) | | | What to return. See [Fetch modes](#fetch-modes). (default=`"both"`) | @@ -55,18 +55,20 @@ curl -G 'https://api.hydradb.com/context/inspect' \ | Mode | Returns | Use when | | --- | --- | --- | -| `content` | The payload in `content_base64` (base64-encoded); `content` is `null` in this mode. Decode `content_base64` to recover the parsed text or the raw binary bytes. | You want to render the content in-app or feed it to another model. | -| `url` | Presigned URL (`presigned_url`) valid for `expiry_seconds`. | You want a client or a service to download the original file directly without proxying through your backend. | -| `both`_(default)_ | Text **and** presigned URL. | UI flows that show parsed text inline plus a "Download original" link. | +| `content` | The stored item in `content` (or, when it is not UTF-8 text, base64-encoded in `content_base64`), plus `inferred_content`. `presigned_url` is `null`. | You want to render the content in-app or feed it to another model. | +| `url` | A presigned URL (`presigned_url`) for the stored item, valid for `expiry_seconds`. `content`, `content_base64` and `inferred_content` are `null`. | You want a client or a service to download the item directly without proxying through your backend. | +| `both` _(default)_ | Everything `content` returns **and** the presigned URL. | UI flows that show the text inline plus a download link. | -Regardless of `mode`, the response also includes `inferred_content` when available - the model-derived text for the source (for memories, the inferred memory statement; for knowledge, derived/normalized content). It is `null` when the source has no inferred content. +`inferred_content` is the enrichment the server wrote for the item, or `null` when there is none (for example an item ingested with `enrich: false`, or one whose enrichment has not finished). It is returned in `content` and `both` modes; `url` mode leaves it `null`. -Use `mode=url` when you need the original file. Use `mode=content` when you only need extracted text for display, summarization, or prompting. +Use `mode=url` when a client should download the stored item directly. Use `mode=content` when you only need the text for display, summarization, or prompting. ### Mode examples +These examples inspect an item ingested as text. + ```json @@ -75,12 +77,12 @@ Use `mode=url` when you need the original file. Use `mode=content` when you only "data": { "success": true, "id": "policy_main", - "content": null, - "content_base64": "U2VjdGlvbiAxLiBBdXRoZW50aWNhdGlvbiBwb2xpY2llcy4uLg==", - "inferred_content": null, + "content": "Refund policy: customers can request a refund within 30 days of purchase.", + "content_base64": null, + "inferred_content": "Customers may request refunds up to 30 days after purchase.", "presigned_url": null, - "content_type": "application/pdf", - "size_bytes": 1842233, + "content_type": "text/plain; charset=utf-8", + "size_bytes": 73, "message": "File fetched successfully" }, "error": null, @@ -101,9 +103,9 @@ Use `mode=url` when you need the original file. Use `mode=content` when you only "content": null, "content_base64": null, "inferred_content": null, - "presigned_url": "https://storage.hydradb.com/.../policy_main.pdf?X-Amz-...", - "content_type": "application/pdf", - "size_bytes": 1842233, + "presigned_url": "https://storage.hydradb.com/.../policy_main?X-Amz-...", + "content_type": "text/plain; charset=utf-8", + "size_bytes": 73, "message": "File fetched successfully" }, "error": null, @@ -114,43 +116,20 @@ Use `mode=url` when you need the original file. Use `mode=content` when you only } ``` - + ```json { "success": true, "data": { "success": true, - "id": "diagram_png", - "content": null, - "content_base64": "iVBORw0KGgoAAAANSUhEUgAA...", - "inferred_content": null, - "presigned_url": "https://storage.hydradb.com/.../diagram.png?X-Amz-...", - "content_type": "image/png", - "size_bytes": 92844, - "message": "File fetched successfully" - }, - "error": null, - "meta": { - "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", - "latency_ms": 12.3 - } - } - ``` - - - ```json - { - "success": true, - "data": { - "success": true, - "id": "mem_user_alex_tone", - "content": "Prefers concise answers and dark mode.", + "id": "policy_main", + "content": "Refund policy: customers can request a refund within 30 days of purchase.", "content_base64": null, - "inferred_content": "User prefers concise answers and dark mode.", - "presigned_url": null, - "content_type": "text/plain", - "size_bytes": null, - "message": "Memory fetched successfully" + "inferred_content": "Customers may request refunds up to 30 days after purchase.", + "presigned_url": "https://storage.hydradb.com/.../policy_main?X-Amz-...", + "content_type": "text/plain; charset=utf-8", + "size_bytes": 73, + "message": "File fetched successfully" }, "error": null, "meta": { @@ -170,12 +149,12 @@ Use `mode=url` when you need the original file. Use `mode=content` when you only "data": { "success": true, "id": "policy_main", - "content": "Section 1. Authentication policies...", + "content": "Refund policy: customers can request a refund within 30 days of purchase.", "content_base64": null, - "inferred_content": null, - "presigned_url": "https://storage.hydradb.com/.../policy_main.pdf?X-Amz-...", - "content_type": "application/pdf", - "size_bytes": 1842233, + "inferred_content": "Customers may request refunds up to 30 days after purchase.", + "presigned_url": "https://storage.hydradb.com/.../policy_main?X-Amz-...", + "content_type": "text/plain; charset=utf-8", + "size_bytes": 73, "message": "File fetched successfully" }, "error": null, @@ -191,7 +170,7 @@ Use `mode=url` when you need the original file. Use `mode=content` when you only "success": false, "data": null, "error": { - "code": "SOURCE_NOT_FOUND", + "code": "NOT_FOUND", "message": "Source not found" }, "meta": { @@ -203,38 +182,15 @@ Use `mode=url` when you need the original file. Use `mode=content` when you only -```json For a binary file -{ - "success": true, - "data": { - "success": true, - "id": "diagram_png", - "content": null, - "content_base64": "iVBORw0KGgoAAAANSUhEUgAA...", - "inferred_content": null, - "presigned_url": "https://storage.hydradb.com/.../diagram.png?X-Amz-...", - "content_type": "image/png", - "size_bytes": 92844, - "message": "File fetched successfully" - }, - "error": null, - "meta": { - "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", - "latency_ms": 12.3 - } -} -``` - ## Behavior notes - **Text vs binary handling.** In `mode=both`, text-parseable sources (PDF, DOCX, MD, TXT) populate `content` with parsed text while non-text/binary files come back in `content_base64`; check both fields when handling unknown content types. In `mode=content`, the payload is always returned base64-encoded in `content_base64` and `content` is `null` - decode `content_base64` to recover the text or bytes. + **Text vs binary handling.** In `content` and `both` modes, `content` carries the stored item when it is valid UTF-8 text. When it is not, `content` is `null` and the bytes come back base64-encoded in `content_base64`. Check both fields when handling unknown content types. -- **`inferred_content`:** Alongside the raw `content`, the response carries `inferred_content` - the model-derived text for the source. For **memories** this is the inferred memory statement (e.g. `"User prefers concise answers and dark mode."`); for **knowledge** sources it is typically `null` unless derived content exists. It is returned for every `mode`. - -- **Recently ingested sources:** Fetching immediately after ingestion may return a record before the parsed text is ready. For reliable reads, use [Ingestion Status](/api-reference/v2/endpoint/source-status) first. -- **Presigned URL TTL:** The URL is valid only for `expiry_seconds`. Anyone with the URL can download the file during that window, so treat it as a short-lived secret. -- **Memory items:** Fetching a memory's `id` returns its raw text content. There are no presigned URLs for memory items - `mode=url` and `mode=both` return `presigned_url: null`. +- **Items ingested as text:** There is no separate original file. `content` is the text you sent (a `conversation` item is stored as JSON, so its `content` is a JSON document), `content_type` reports how it was stored, and in `url` and `both` modes `presigned_url` downloads that same stored item. +- **`inferred_content`:** The enrichment the server wrote for the item, or `null` when there is none. Only `content` and `both` modes return it. +- **Recently ingested items:** Fetching immediately after ingestion may return a record before enrichment is ready. For reliable reads, use [Ingestion Status](/api-reference/v2/endpoint/source-status) first. +- **Presigned URL TTL:** The URL is valid only for `expiry_seconds`. Anyone with the URL can download the item during that window, so treat it as a short-lived secret.
diff --git a/api-reference/v2/endpoint/ingest-context.mdx b/api-reference/v2/endpoint/ingest-context.mdx index b06b3064..14bbd270 100644 --- a/api-reference/v2/endpoint/ingest-context.mdx +++ b/api-reference/v2/endpoint/ingest-context.mdx @@ -1,14 +1,11 @@ --- title: "Ingest Context" -description: "Send text and conversations to a database as context items. Split databases keep documents, app_knowledge and memories." +description: "Send text and conversations to a database as context items." --- import { Field } from "/snippets/field.jsx"; -import LegacyLine from "/snippets/legacy-line.mdx"; -`POST /context/ingest` takes a list of **context items**, each a `text` or a `conversation`, and queues them for chunking, embedding, enrichment and graph extraction. That is the only ingest shape a unified database accepts. The guide is [Ingest context](/essentials/v2/ingest); this page is the field reference. - - +`POST /context/ingest` takes a list of **context items**, each a `text` or a `conversation`, and queues them for chunking, embedding, enrichment and graph extraction. The guide is [Ingest context](/essentials/v2/ingest); this page is the field reference. `database` and `collection` are the current field names (formerly `tenant_id` and `sub_tenant_id`). The old names remain accepted as deprecated aliases for full backward compatibility. @@ -23,7 +20,6 @@ import json result = client.context.ingest( database="acme_corp", collection="company", - upsert=True, items=json.dumps([ { "context_id": "refund-policy", @@ -46,7 +42,7 @@ result = client.context.ingest( ]), ) -print([r.source_id for r in result.data.results]) +print([r.id for r in result.data.results]) ``` ```typescript TypeScript SDK @@ -54,7 +50,6 @@ print([r.source_id for r in result.data.results]) const result = await client.context.ingest({ database: "acme_corp", collection: "company", - upsert: true, items: JSON.stringify([ { context_id: "refund-policy", @@ -77,7 +72,7 @@ const result = await client.context.ingest({ ]), }); -console.log(result.data.results.map((r) => r.sourceId)); +console.log(result.data?.results?.map((r) => r.id)); ``` ```bash cURL @@ -117,17 +112,19 @@ curl -X POST 'https://api.hydradb.com/context/ingest' \ ## Request body -Send `application/json`. The SDKs send `multipart/form-data` instead: the same array goes in the `items` form field, with `database`, `collection`, `upsert`, `enrich`, `instructions` and `graph_payload` as form fields. Both entry points run the same validation. +Send `application/json`. The SDKs send `multipart/form-data` instead: the same array goes in the `items` form field as a JSON string, and the request-level fields are form fields of the same name (`graph_payload` also as a JSON string). Both entry points run the same validation. Keys inside each item stay snake_case in every language. + +The SDK `ingest` methods take `database`, `collection`, `items`, `upsert` (the string `"true"` or `"false"`) and `graph_payload`. To set `enrich` or `instructions` through an SDK, set them on each item. | Name | Description | | --- | --- | | | Target database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | | Collection inside the database. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=the default collection) | -| | The items to ingest, at most 100. `items` and `contexts` are accepted as aliases; send `context`. In the multipart form the field is `items`, a JSON-encoded array. | +| | The items to ingest, at least 1 and at most 100. In the multipart form the field is `items`, a JSON-encoded array. | | | Request-level default for every item's `enrich`. (default=`true`) | | | Request-level default for every item's `upsert`. (default=`true`) | | | Request-level default for every item's `instructions`. (default=empty) | -| | Bring your own graph: a map of `context_id` to `{ entities, relations }` that replaces extraction for that item. Every key must match an item's `context_id` in the same request, otherwise `400`. See [Bring Your Own Graph](/essentials/v2/bring-your-own-graph). | +| | Bring your own graph: a map of `context_id` to `{ entities, relations }` that replaces graph extraction for that item. Every key must match the `context_id` of an item in the same request, otherwise `400`. See [Bring your own graph](#bring-your-own-graph) below. | ### Item fields @@ -135,864 +132,188 @@ Each item is exactly one of `text` or `conversation`. | Name | Description | | --- | --- | -| | Your id for the item; the upsert key. Generated from `title` when omitted. Must not contain a comma (`,`), which is the id separator on `/context/status?ids=`. | +| | Your id for the item; the upsert key. Generated when omitted. At most 100 bytes. Must not contain a comma (`,`), which is the id separator on `/context/status?ids=`, and must not start with `att_` or `cmt_` (reserved for connector ids). | | | Readable name. Searchable with `titles` on `/query`. | -| | Plain text. Send exactly one of `text` or `conversation`. `content` is accepted as an alias. | -| | Turns of `{ role, content, name? }`; roles are `user`, `assistant` and `system`. `system` turns shape enrichment but are never stored as facts. `messages` is accepted as an alias. | +| | Plain text. Send exactly one of `text` or `conversation`. | +| | Turns of `{ role, content, name? }`; roles are `user`, `assistant` and `system`. `system` turns shape enrichment but are never stored as facts. A conversation needs at least one `user` or `assistant` turn, and no turn may have empty `content`. | | | Extract entities, relations and preferences from this item into the graph; the output is stored separately and returned as `enrichment` on query. (default=the request's `enrich`, else `true`) | | | Replace an existing item with the same `context_id`, deleting its chunks and graph contribution first. (default=the request's `upsert`, else `true`) | -| | Steer enrichment for this item. `custom_instructions` is accepted as an alias. (default=the request's `instructions`) | +| | Steer enrichment for this item. (default=the request's `instructions`) | | | The date the item is about, `YYYY-MM-DD` only. A timestamp is a `400`. HydraDB records when it received the item separately. | -| | Declared, filterable fields; keys must be in `database_metadata_schema`. Filter with `attributes` on `/query`. | +| | Declared, filterable fields; keys must be in `database_metadata_schema`. Filter with `attributes` on `/query`. See [Attributes](/essentials/v2/attributes). | | | Free-form fields. Stored with the item; not filterable and not returned on query chunks. | -| | What kind of context this is. You set it; nothing infers it, and a misspelling is a `400`. Returned on query as `enrichment_kind` (omitted for `auto`). (default=`"auto"`) | -| | Relations you declare to other items: `{ "ids": ["", ...], "properties": {} }`. Followed on `/query` with `follow_forceful_relations` and returned in `forceful_relations[]`. `relations` is accepted as an alias for the field; `context_ids` or `source_ids` for the `ids` key. | -| | Principals allowed to retrieve the item: bare emails or `user_email:`, `group:`, `domain:` principals, or `__public__`. Omit for unrestricted; `[]` for nobody. A malformed principal rejects the whole request with `400`. | +| | What kind of context this is. You set it; nothing infers it, and a misspelling is a `400`. Returned on query as `enrichment_kind` (omitted for `auto`). See [Context categories](/essentials/v2/context-categories). (default=`"auto"`) | +| | Relations you declare to other items: `{ "ids": ["", ...], "properties": {} }`. Followed on `/query` with `follow_forceful_relations` and returned in `forceful_relations[]`. Each id follows the same rules as `context_id`. | +| | Principals allowed to retrieve the item: bare emails or `user_email:`, `group:`, `domain:` principals, or `__public__`. Omit for unrestricted; `[]` for nobody. A malformed principal rejects the whole request with `400`. See [Access control](/essentials/v2/access-control). | | | Chunk `text` on its markdown structure instead of as flat prose. (default=`false`) | | | The speaker for a text item. On a conversation each turn's `name` wins. (default=`"User"`) | ### Limits -- At most **100 items** per request, **1 MiB** of text per item, **8 MiB** of text per request. -- A validation error names the item as `context[N]`. - -### Refused on a unified database - -`type`, `documents` (file uploads), `app_knowledge`, `memories`, `evidence_kind`, `evidence_subject`, `expiry_time` and `retain_source` return `400 CORPUS_TYPE_UNSUPPORTED`. Extract text from files and send it as an item; structured app sources arrive through [connectors](/essentials/v2/connectors). Any other key an item does not recognise is dropped without an error. - -## Response - -`202 Accepted`, with one result per item: - -```json -{ - "success": true, - "data": { - "success": true, - "message": "Context queued for ingestion successfully. Ingestion is asynchronous: this 202 means the sources were accepted and queued, not indexed. Poll GET /context/status?database=&id= until each source's indexing_status reaches a terminal state (completed or errored) before querying. See https://docs.hydradb.com/api-reference/v2/endpoint/source-status for usage details. ", - "results": [ - { "source_id": "refund-policy", "title": "Refund policy", "status": "queued", "infer": true, "error": null, "error_code": null }, - { "source_id": "chat-alex-001", "title": null, "status": "queued", "infer": true, "error": null, "error_code": null } - ], - "success_count": 2, - "failed_count": 0 - }, - "error": null, - "meta": { "request_id": "bcd03673-174d-4a73-83e6-73bfcdc16061", "api_version": "2.0.1" } -} -``` - -| Field | Description | -| --- | --- | -| `message` | `Context queued for ingestion successfully` (`Context ingestion completed with some failures` when an item failed), followed by a reminder to poll status. | -| `results[].source_id` | The item's `context_id`, sent or generated. The result item keeps the name `source_id`; read it as the context id and pass it to [`GET /context/status`](/api-reference/v2/endpoint/source-status). | -| `results[].title` | The item's `title`, or `null`. | -| `results[].status` | `queued` or `failed`. A failed item does not stop the others. | -| `results[].infer` | Mirrors the item's `enrich`. | -| `results[].error`, `results[].error_code` | Why the item failed; `null` on success. | -| `success_count`, `failed_count` | Totals across `results`. | - - -**`202 Accepted` means queued, not indexed.** Ingestion is asynchronous. Poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) with the returned ids until each item reaches `completed` or `errored` (`graph_creation` is already searchable), or register a webhook for `indexing.status_changed` events (see [Webhooks](/essentials/v2/webhooks)). - +- At most **100 items** per request, **1 MiB** of text per item, **8 MiB** of text per request. Text is an item's `text`, or the `content` of every turn of its `conversation`; titles and attributes are not counted. +- `attributes` at most **16 KiB** and `custom_attributes` at most **1 KiB** per item, measured on their compact JSON encoding. +- The request is validated before anything is queued: one invalid item rejects the whole request with `400`, and the error names the item as `context[N]`. -
- - - **Related Resources** - - - **Always check** [ingestion status](/api-reference/v2/endpoint/source-status) to ensure context is ready to be retrieved - - [Query](/api-reference/v2/endpoint/query) once context is ready - - **Inspect:** [List Documents](/api-reference/v2/endpoint/list-documents) helps you fetch titles and descriptions of ingested context - - **Inspect:** [Fetch Content](/api-reference/v2/endpoint/fetch-content) returns the title, attributes and content behind a `context_id` - - **Cleanup:** [Delete Context](/api-reference/v2/endpoint/delete-source) - +### Text only -## Split databases +Every item is text or a conversation. To ingest a file, extract its text and send it as a `text` item (with `is_markdown: true` if the extracted text is markdown). Content from connected apps arrives through [connectors](/essentials/v2/connectors). An item key HydraDB does not recognise is ignored without an error, so check field names against the table above. -Everything below applies only to a database created with `type: "split"`. There, `type` picks the corpus and the payload is one of the older shapes: `documents` (files HydraDB parses) or `app_knowledge` (pre-extracted app sources) under `type=knowledge`, or `memories` under `type=memory`. Set `infer: true` on a memory to extract preferences from raw signals, or `infer: false` to store the text verbatim. Read more in [Knowledge](/essentials/v2/knowledge), [App sources](/essentials/v2/app-sources) and [Memories](/essentials/v2/memories); the mapping from each split field to its item field is on [Split databases and legacy fields](/essentials/v2/split-databases#3-split-ingest-fields). +## Bring your own graph -### Split request examples +`graph_payload` supplies the graph for an item yourself. HydraDB uses it instead of extracting a graph from that item; the item is still chunked and embedded, so it stays searchable. Each top-level key is the `context_id` of an item in the same request, so give keyed items an explicit `context_id`. See [Bring Your Own Graph](/essentials/v2/bring-your-own-graph) for the full guide. -```python Python SDK -import json - -with open("/path/to/policy.pdf", "rb") as policy: - knowledge_result = client.context.ingest( - # Knowledge requests can include documents, app_knowledge, or both. - type="knowledge", - database="acme_corp", - collection="team_docs", - upsert=True, - - # Use documents when HydraDB should parse PDFs, DOCX, CSV, Markdown, or text files. - documents=[("policy.pdf", policy, "application/pdf")], - document_metadata=json.dumps([ - { - "id": "policy_main", - "metadata": {"department": "legal"}, - "additional_metadata": {"source": "policy"}, - } - ]), - - # Use app_knowledge when your app already extracted the source text/metadata. - app_knowledge=json.dumps([ - { - "id": "slack_thread_001", - "database": "acme_corp", - "collection": "team_docs", - "title": "Pricing discussion", - "type": "slack", - "content": {"text": "We agreed on three tiers..."}, - "metadata": {"department": "product"}, - "additional_metadata": {"channel": "pricing"}, - } - ]), - ) - -text_memory_result = client.context.ingest( - type="memory", - database="acme_corp", - collection="user_alex", - memories=json.dumps([ - { - # Use text for raw notes or signals. - "text": "Prefers concise answers and dark mode.", - "infer": True, - "user_name": "Alex", - } - ]), -) - -conversation_memory_result = client.context.ingest( - type="memory", - database="acme_corp", - collection="user_alex", - memories=json.dumps([ - { - # Use user_assistant_pairs for conversation history instead of text. - "title": "Support conversation about refunds", - "user_assistant_pairs": [ - {"user": "Can I get a refund?", "assistant": "Refunds are available within 30 days."}, - ], - "infer": False, - } - ]), -) -``` - -```typescript TypeScript SDK -const knowledgeResult = await client.context.ingest({ - // Knowledge requests can include documents, app_knowledge, or both. - type: "knowledge", - database: "acme_corp", - collection: "team_docs", - upsert: true, - - // Use documents when HydraDB should parse PDFs, DOCX, CSV, Markdown, or text files. - documents: [ - { path: "/path/to/policy.pdf", filename: "policy.pdf", contentType: "application/pdf" }, - ], - documentMetadata: JSON.stringify([ - { - id: "policy_main", - metadata: { department: "legal" }, - additional_metadata: { source: "policy" }, - }, - ]), - - // Use app_knowledge when your app already extracted the source text/metadata. - appKnowledge: JSON.stringify([ - { - id: "slack_thread_001", - database: "acme_corp", - collection: "team_docs", - title: "Pricing discussion", - type: "slack", - content: { text: "We agreed on three tiers..." }, - metadata: { department: "product" }, - additional_metadata: { channel: "pricing" }, - }, - ]), -}); - -const textMemoryResult = await client.context.ingest({ - type: "memory", - database: "acme_corp", - collection: "user_alex", - memories: JSON.stringify([ - { - // Use text for raw notes or signals. - text: "Prefers concise answers and dark mode.", - infer: true, - user_name: "Alex", - }, - ]), -}); - -const conversationMemoryResult = await client.context.ingest({ - type: "memory", - database: "acme_corp", - collection: "user_alex", - memories: JSON.stringify([ - { - // Use user_assistant_pairs for conversation history instead of text. - title: "Support conversation about refunds", - user_assistant_pairs: [ - { user: "Can I get a refund?", assistant: "Refunds are available within 30 days." }, - ], - infer: false, - }, - ]), -}); -``` - ```bash cURL curl -X POST 'https://api.hydradb.com/context/ingest' \ -H "Authorization: Bearer " \ -H "API-Version: 2" \ - -F "type=knowledge" \ - -F "database=acme_corp" \ - -F "collection=team_docs" \ - -F "upsert=true" \ - -F "documents=@/path/to/policy.pdf" \ - -F 'document_metadata=[ - { - "id": "policy_main", - "metadata": { "department": "legal" }, - "additional_metadata": { "source": "policy" } - } - ]' \ - -F 'app_knowledge=[ - { - "id": "slack_thread_001", - "database": "acme_corp", - "collection": "team_docs", - "title": "Pricing discussion", - "type": "slack", - "content": { "text": "We agreed on three tiers..." }, - "metadata": { "department": "product" }, - "additional_metadata": { "channel": "pricing" } - } - ]' - -# OR: memory from raw text. -curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -F "type=memory" \ - -F "database=acme_corp" \ - -F "collection=user_alex" \ - -F 'memories=[ - { - "text": "Prefers concise answers and dark mode.", - "infer": true, - "user_name": "Alex" - } - ]' - -# OR: memory from conversation pairs. -curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -F "type=memory" \ - -F "database=acme_corp" \ - -F "collection=user_alex" \ - -F 'memories=[ - { - "title": "Support conversation about refunds", - "user_assistant_pairs": [ - { "user": "Can I get a refund?", "assistant": "Refunds are available within 30 days." } - ], - "infer": false + -H "Content-Type: application/json" \ + -d '{ + "database": "acme_corp", + "collection": "company", + "context": [ + { + "context_id": "billing-policy", + "title": "Billing policy", + "text": "Alice Carter owns the billing policy." + } + ], + "graph_payload": { + "billing-policy": { + "entities": { + "alice": { "name": "Alice Carter", "type": "PERSON", "namespace": "employees" }, + "billing": { "name": "Billing Policy", "type": "POLICY", "namespace": "policies" } + }, + "relations": [ + { "source": "alice", "target": "billing", "predicate": "OWNS", + "context": "Alice Carter owns the billing policy." } + ] + } } - ]' + }' ``` - - -### Upload in-memory text as a `.txt` file - -If you already have text in memory, create a file-like object and upload it through `documents` as a `text/plain` `.txt` file. - - ```python Python SDK -import io import json -text = """Q4 planning notes - -- Launch checklist is owned by Priya. -- Legal review is due by Friday. -""" - -# Create a file-like object in memory. No local .txt file is required. -txt_file = io.BytesIO(text.encode("utf-8")) - -response = client.context.ingest( - type="knowledge", +client.context.ingest( database="acme_corp", - collection="team_docs", - documents=[("meeting-notes.txt", txt_file, "text/plain")], - document_metadata=json.dumps([ - {"id": "meeting_notes_q4", "metadata": {"department": "product"}} + collection="company", + items=json.dumps([ + { + "context_id": "billing-policy", + "title": "Billing policy", + "text": "Alice Carter owns the billing policy.", + }, ]), + graph_payload=json.dumps({ + "billing-policy": { + "entities": { + "alice": {"name": "Alice Carter", "type": "PERSON", "namespace": "employees"}, + "billing": {"name": "Billing Policy", "type": "POLICY", "namespace": "policies"}, + }, + "relations": [ + {"source": "alice", "target": "billing", "predicate": "OWNS", + "context": "Alice Carter owns the billing policy."}, + ], + } + }), ) ``` ```typescript TypeScript SDK -const text = `Q4 planning notes - -- Launch checklist is owned by Priya. -- Legal review is due by Friday. -`; - -const response = await client.context.ingest({ - type: "knowledge", +await client.context.ingest({ database: "acme_corp", - collection: "team_docs", - documents: [ + collection: "company", + items: JSON.stringify([ { - filename: "meeting-notes.txt", - contentType: "text/plain", - data: Buffer.from(text, "utf-8"), + context_id: "billing-policy", + title: "Billing policy", + text: "Alice Carter owns the billing policy.", }, - ], - documentMetadata: JSON.stringify([ - { id: "meeting_notes_q4", metadata: { department: "product" } }, ]), + graphPayload: JSON.stringify({ + "billing-policy": { + entities: { + alice: { name: "Alice Carter", type: "PERSON", namespace: "employees" }, + billing: { name: "Billing Policy", type: "POLICY", namespace: "policies" }, + }, + relations: [ + { source: "alice", target: "billing", predicate: "OWNS", + context: "Alice Carter owns the billing policy." }, + ], + }, + }), }); ``` -```python API -import io -import json -import requests - -text = """Q4 planning notes - -- Launch checklist is owned by Priya. -- Legal review is due by Friday. -""" - -# Create a file-like object in memory. No local .txt file is required. -txt_file = io.BytesIO(text.encode("utf-8")) - -response = requests.post( - "https://api.hydradb.com/context/ingest", - headers={ - "Authorization": "Bearer ", - "API-Version": "2", - }, - data={ - "type": "knowledge", - "database": "acme_corp", - "collection": "team_docs", - "document_metadata": json.dumps([ - {"id": "meeting_notes_q4", "metadata": {"department": "product"}} - ]), - }, - files={ - "documents": ("meeting-notes.txt", txt_file, "text/plain"), - }, -) -response.raise_for_status() -``` -### Important form fields - -| Name | Description | -| --- | --- | -| | Use singular `"memory"` when writing memories. (default=`"knowledge"`) | -| | Target database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | -| | Logical partition inside the database. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=`""` - default collection) | -| | Replace existing sources with the same ID. Set to `false` to error on conflict. (default=`true`) | -| | Binary uploads, **knowledge only**. Required when `type=knowledge` and you want HydraDB to parse documents. Omit when ingesting memories. See [Supported file formats](#supported-file-formats). (default=`[]`) | -| | One entry per file in `documents`, in the same order. If omitted, documents index with inferred defaults such as filename/title. See the item shape below. | -| | Pre-extracted source objects (Slack, Notion, web pages, etc.), **knowledge only**. See the `app_knowledge` item shape below. | -| | Map of source id → your own entities + relations - replaces LLM graph extraction for each keyed source. Works for `type=knowledge` (key = a `document_metadata` id or `app_knowledge` item id) and `type=memory` (key = a memory `id`). See [Bring Your Own Graph](/essentials/v2/bring-your-own-graph) and the shape below. | -| | Memory items, **memory only**. Required and non-empty when `type=memory`. Use plural `memories` for the form field, even though `type` is singular `memory`. See the `memories` item shape below. | - - -1. **`id` must not contain a comma (`,`).** The comma is reserved as the id separator on [Ingestion Status](/api-reference/v2/endpoint/source-status) (`GET /context/status?ids=a,b`), so an `id` containing a comma cannot be looked up unambiguously. This applies to every `id` you supply - `document_metadata`, `app_knowledge`, and `memories` items. Ingesting an item whose `id` contains a comma is rejected with a `400`. - -2. **`202 Accepted` means queued, not indexed.** Ingestion is asynchronous. A successful response only confirms your sources were accepted, not that they are ready to query. Before querying, poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) with the returned IDs until each source reaches `completed` or `errored`. Alternatively, register a webhook for `indexing.status_changed` events (see [Webhooks](/essentials/v2/webhooks)). - - - -### Supported file formats - -Anything in this table can be uploaded through `documents` and HydraDB will read it. - -| Type | Formats | -| --- | --- | -| Documents | `.pdf` `.doc` `.docx` `.docm` `.dot` `.dotx` `.odt` `.ott` `.rtf` | -| Spreadsheets | `.xls` `.xlsx` `.xlsm` `.xlsb` `.ods` `.ots` `.csv` `.tsv` | -| Presentations | `.ppt` `.pptx` `.pptm` `.odp` `.otp` | -| Apple iWork | `.pages` `.numbers` `.key` | -| Images | `.png` `.jpg` `.jpeg` `.tif` `.tiff` `.webp` `.gif` `.bmp` | -| Plain text | `.txt` `.md` `.markdown` `.json` | - -What HydraDB extracts differs by type, and it is worth knowing which one you are uploading: - -| Type | What gets indexed | -| --- | --- | -| Documents and presentations | The text, in reading order. Slide titles and body text both come through. | -| Spreadsheets | Cell values rendered as a table, one per sheet. Formulas are indexed as their computed result, not the formula itself. | -| Images and scans | Text read out of the picture with OCR. A photo of a page and a scan of a page both work. | -| Plain text | The file as-is. | - -A scanned page with nothing readable on it, or a photo that happens to contain no text, will index as an empty document rather than fail. - -#### Formats we cannot read - -These are rejected the moment you upload them, before anything is queued. You get the answer in the upload response rather than minutes later. - -| Format | Upload this instead | +| Field | Description | | --- | --- | -| `.heic` `.heif` `.avif` | Export the photo as JPEG or PDF | -| `.svg` `.ico` | Export as PNG or JPEG | -| `.html` `.htm` `.xml` | Save the page as PDF | -| `.eml` `.msg` | Save the message as PDF, or send it as an app source | -| `.epub` `.wpd` | Convert to PDF or DOCX | -| `.zip` | Upload the files individually | +| | Top-level key: the `context_id` of an item in this request. Value is that item's graph. A key matching no item returns `400`. | +| | Non-empty map keyed by a caller-local handle (at most 256 characters) that `relations` reference; the handle is not stored. | +| | Entity name. At most 256 characters. | +| | Entity type (e.g. `PERSON`, `POLICY`). Stored as supplied. At most 256 characters. | +| | Logical grouping for the entity. Stored as supplied. At most 256 characters. | +| | Optional external id (email, URL, etc.), display only. At most 256 characters. | +| | Non-empty list of edges between entity handles. | +| | Handle of the source entity; must be a key in `entities`. | +| | Handle of the target entity; must be a key in `entities`. | +| | Relationship label, any plain string. At most 256 characters. | +| | Optional sentence supporting the edge. At most 2,000 characters. | +| | Optional timing info (e.g. "since 2021", "in Q3"). At most 256 characters. | -`.heic` is the format iPhones use for photos by default, so it is the one people hit most often without realizing. On iOS you can change this under **Settings > Camera > Formats > Most Compatible**, which makes the camera save JPEGs instead. Or export the photo as JPEG or PDF before uploading. +Caps per graph: at most 5,000 entities, 10,000 relations and 500 relations per entity. A graph over a cap, or one that fails any rule above, rejects the request with `400`. -#### How we decide the format - -The file extension is what counts. `report.pdf` is treated as a PDF because of the `.pdf`, not because of what is inside it. - -If the filename has no extension at all, the `Content-Type` you send with that part is used instead, so a file with no extension in its name, sent as `application/pdf`, is accepted. - - -Renaming a file does not convert it. A `.heic` photo renamed to `photo.pdf` passes this check, because the check reads the name, then fails later during parsing and comes back as `errored` on [`GET /context/status`](/api-reference/v2/endpoint/source-status). Upload files under their real extension. - - -#### Size limit - -Each file in `documents` can be up to **50 MB**. A larger file is rejected with `413` and a message naming the file, and no part of the request is processed. Split large documents, or upload them separately. - -### When a file is not supported - -A rejected file does not stop the rest of your upload. Each file is checked on its own, so a batch of ten with one bad file still indexes the other nine. +## Response -The request returns `202` as usual. The rejected file comes back with `status: "failed"` and `error_code: "E1002"`, and every other file is queued normally: +`202 Accepted`, with one result per item: ```json { "success": true, "data": { - "success": false, - "message": "Upload completed with some failures.", + "success": true, + "message": "Context queued for ingestion successfully. Ingestion is asynchronous: this 202 means the sources were accepted and queued, not indexed. Poll GET /context/status?database=&id= until each source's indexing_status reaches a terminal state (completed or errored) before querying. See https://docs.hydradb.com/api-reference/v2/endpoint/source-status for usage details. ", "results": [ - { "id": "6cc7bcd185b044cd", "filename": "report.pdf", "status": "queued", "error": null, "error_code": null }, - { "id": "f47411c736447126", "filename": "photo.heic", "status": "failed", "error": "This file format isn't supported. Please upload a PDF, Office document (Word, Excel, PowerPoint), image, CSV or text file.", "error_code": "E1002" }, - { "id": "d0c919bdec196a8d", "filename": "notes.txt", "status": "queued", "error": null, "error_code": null } + { "id": "refund-policy", "title": "Refund policy", "status": "queued", "infer": true, "error": null, "error_code": null }, + { "id": "chat-alex-001", "title": null, "status": "queued", "infer": true, "error": null, "error_code": null } ], "success_count": 2, - "failed_count": 1 + "failed_count": 0 }, "error": null, "meta": { "request_id": "bcd03673-174d-4a73-83e6-73bfcdc16061", "api_version": "2.0.1" } } ``` - -`results`, `success_count` and `failed_count` sit inside `data`, not at the top level, like every other v2 response. The outer `success: true` means the request was accepted; `data.success` is what tells you whether every file in it was queued. - +| Field | Description | +| --- | --- | +| `message` | `Context queued for ingestion successfully` (`Context ingestion completed with some failures` when an item failed), followed by a reminder to poll status. | +| `results[].id` | The item's `context_id`, sent or generated. Pass it to [`GET /context/status`](/api-reference/v2/endpoint/source-status). | +| `results[].title` | The item's `title`, or `null`. | +| `results[].status` | `queued` or `failed`. A failed item does not stop the others. | +| `results[].infer` | Mirrors the item's `enrich`. | +| `results[].error`, `results[].error_code` | Why the item failed; `null` on success. | +| `success_count`, `failed_count` | Totals across `results`. | -**Do not poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) for a file rejected with `E1002`.** The file never entered the pipeline, so it has no status record, and looking it up returns `FILE_NOT_FOUND` rather than the format error you were given. The upload response is the only place `E1002` appears. Read `error_code` on each item in `data.results` and act on it there. +**`202 Accepted` means queued, not indexed.** Ingestion is asynchronous. Poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) with the returned ids until each item reaches `completed` or `errored` (`graph_creation` is already searchable), or register a webhook for `indexing.status_changed` events (see [Webhooks](/essentials/v2/webhooks)). +
-### Common use-cases and their configurations - -#### Document metadata - -Per-document metadata (`id`, `metadata`, `additional_metadata`, `relations`) can be passed alongside each uploaded document to control indexing, filtering, and display. The key list is closed - an item carrying any other key is rejected with a `400` naming it, rather than being silently dropped. See the field reference below. - - - - - ```python Python SDK - import json - - with open("/path/to/policy.pdf", "rb") as f1, open("/path/to/runbook.pdf", "rb") as f2: - result = client.context.ingest( - type="knowledge", - database="acme_corp", - documents=[ - ("policy.pdf", f1, "application/pdf"), - ("runbook.pdf", f2, "application/pdf"), - ], - document_metadata=json.dumps([ - {"id": "policy_main", "metadata": {"department": "legal"}}, - {"id": "runbook_deploy", "metadata": {"department": "ops"}}, - ]), - ) - ``` - - ```typescript TypeScript SDK - const result = await client.context.ingest({ - type: "knowledge", - database: "acme_corp", - documents: [ - { path: "/path/to/policy.pdf", filename: "policy.pdf", contentType: "application/pdf" }, - { path: "/path/to/runbook.pdf", filename: "runbook.pdf", contentType: "application/pdf" }, - ], - documentMetadata: JSON.stringify([ - { id: "policy_main", metadata: { department: "legal" } }, - { id: "runbook_deploy", metadata: { department: "ops" } }, - ]), - }); - ``` - - ```bash cURL - curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -F "type=knowledge" \ - -F "database=acme_corp" \ - -F "documents=@/path/to/policy.pdf" \ - -F "documents=@/path/to/runbook.pdf" \ - -F 'document_metadata=[ - { "id": "policy_main", "metadata": { "department": "legal" } }, - { "id": "runbook_deploy", "metadata": { "department": "ops" } } - ]' - ``` - - - - | Field | Description | - | --- | --- | - | | Optional context ID. If set, becomes the `id` for this document (use your app's document ID for parity). Must not contain a comma (`,`) - it is reserved as the id separator on `/context/status?ids=`. | - | | Database-schema fields for filtering and search. Keys must be declared in `database_metadata_schema`. (default=`{}`) | - | | Free-form per-document fields for display or bookkeeping. To filter on these at search time, nest under `metadata_filters.additional_metadata`. (default=`{}`) | - | | Declare forceful relations to other sources. Shape: `{ "ids": ["...", "..."] }`. Surfaced via `additional_context` in `mode: "thinking"` search. | - - - Those four are the only keys accepted. Anything else - including `title`, `type`, `url` and `timestamp` - is rejected with a `400` naming the unsupported key, rather than being silently dropped. - - In particular, a document's **title is derived, not settable**. It defaults to the uploaded filename and is returned as `source_title` on query results and `title` on `/context/list`. It cannot be overridden at ingest, and [`PATCH /context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) only merges `additional_metadata` and `database_metadata`. Use `additional_metadata` for your own display fields, or ingest through `app_knowledge`, whose items carry an explicit `title`. - - - - - - - ```python Python SDK - import json - - client.context.ingest( - type="knowledge", - database="acme_corp", - collection="team_docs", - app_knowledge=json.dumps([ - { - "id": "slack_thread_001", - "database": "acme_corp", - "collection": "team_docs", - "title": "Pricing discussion", - "type": "slack", - "content": {"text": "We agreed on three tiers..."}, - "metadata": {"channel": "product"}, - } - ]), - ) - ``` - - ```typescript TypeScript SDK - await client.context.ingest({ - type: "knowledge", - database: "acme_corp", - collection: "team_docs", - appKnowledge: JSON.stringify([ - { - id: "slack_thread_001", - database: "acme_corp", - collection: "team_docs", - title: "Pricing discussion", - type: "slack", - content: { text: "We agreed on three tiers..." }, - metadata: { channel: "product" }, - }, - ]), - }); - ``` - - ```bash cURL - curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -F "type=knowledge" \ - -F "database=acme_corp" \ - -F "collection=team_docs" \ - -F 'app_knowledge=[ - { - "id": "slack_thread_001", - "database": "acme_corp", - "collection": "team_docs", - "title": "Pricing discussion", - "type": "slack", - "content": { "text": "We agreed on three tiers..." }, - "metadata": { "channel": "product" } - } - ]' - ``` - - - - | Field | Description | - | --- | --- | - | | Context ID. Treated as the upsert key. Send an empty string to have one generated upstream. Must not contain a comma (`,`) - it is reserved as the id separator on `/context/status?ids=`. | - | | Target database. Must match the form-level `database`. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | - | | Logical partition inside the database. Must match the form-level `collection`. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). | - | | Short title or subject shown in search results. | - | | Source category (`slack`, `notion`, `gmail`, `webpage`, etc.). Used for filtering and display. | - | | Optional long-form description. | - | | Canonical URL or reference link. | - | | ISO-8601 timestamp (creation or last-updated). | - | | Content payload. Use `{ "text": "..." }` for plain text. Required for app sources. | - | | Database-schema fields. (default=`{}`) | - | | Free-form per-document fields. (default=`{}`) | - | | Optional related attachments. (default=`[]`) | - | | Forceful relations, same shape as on `metadata`. | - - - - - - ```python Python SDK - import json - - client.context.ingest( - type="memory", - database="acme_corp", - collection="user_alex", - memories=json.dumps([ - { - "text": "Prefers dark mode and short answers.", - "infer": True, - "user_name": "Alex", - } - ]), - ) - ``` - - ```typescript TypeScript SDK - await client.context.ingest({ - type: "memory", - database: "acme_corp", - collection: "user_alex", - memories: JSON.stringify([ - { - text: "Prefers dark mode and short answers.", - infer: true, - user_name: "Alex", - }, - ]), - }); - ``` - - ```bash cURL - curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -F "type=memory" \ - -F "database=acme_corp" \ - -F "collection=user_alex" \ - -F 'memories=[ - { - "text": "Prefers dark mode and short answers.", - "infer": true, - "user_name": "Alex" - } - ]' - ``` - - - - | Field | Description | - | --- | --- | - | | Optional unique ID. Acts as the upsert key. Must not contain a comma (`,`) - it is reserved as the id separator on `/context/status?ids=`. (default=auto-generated) | - | | Short label for display in `/context/list` and search hits as `source.title`. (default=truncated `text`) | - | | Raw text or markdown content. Required unless `user_assistant_pairs` is provided. | - | | Conversation pairs `{ user, assistant }`. Required unless `text` is provided. | - | | Treat `text` as markdown for chunking. (default=`false`) | - | | When `true`, HydraDB extracts the underlying preference from raw signal. (default=`false`) | - | | Guides extraction when `infer: true`. Ignored when `infer: false`. | - | | The user's name. Feeds inference. (default=`"User"`) | - | | TTL in seconds. Memory stops surfacing after expiry. | - | | Database-schema fields as a **JSON-stringified** object (e.g. `"{\"department\":\"legal\"}"`). Unlike `metadata` and `app_knowledge`, memory items take this as a string, not an object. (default=`""`) | - | | Free-form per-document fields as a **JSON-stringified** object. Same string-vs-object difference as `metadata` above. (default=`""`) | - | | Forceful relations within the Memories store. Shape: `{ "ids": ["...", "..."] }`. | - - - - - - ```python Python SDK - import json - - client.context.ingest( - type="memory", - database="acme_corp", - collection="user_alex", - memories=json.dumps([ - { - "title": "Support conversation about refunds", - "user_assistant_pairs": [ - {"user": "Can I get a refund?", "assistant": "Refunds are available within 30 days."}, - ], - "infer": False, - } - ]), - ) - ``` - - ```typescript TypeScript SDK - await client.context.ingest({ - type: "memory", - database: "acme_corp", - collection: "user_alex", - memories: JSON.stringify([ - { - title: "Support conversation about refunds", - user_assistant_pairs: [ - { user: "Can I get a refund?", assistant: "Refunds are available within 30 days." }, - ], - infer: false, - }, - ]), - }); - ``` - - ```bash cURL - curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -F "type=memory" \ - -F "database=acme_corp" \ - -F "collection=user_alex" \ - -F 'memories=[ - { - "title": "Support conversation about refunds", - "user_assistant_pairs": [ - { "user": "Can I get a refund?", "assistant": "Refunds are available within 30 days." } - ], - "infer": false - } - ]' - ``` - - - - - - `graph_payload` is a **map of source id → graph** that **replaces LLM graph extraction** for each keyed source. For `type=knowledge`, the key is a `document_metadata` id or an `app_knowledge` item id; for `type=memory`, the key is a memory `id`. Keyed sources are still chunked and embedded, so they stay searchable. See [Bring Your Own Graph](/essentials/v2/bring-your-own-graph) for the full guide. - - - - ```bash cURL - curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer " \ - -H "API-Version: 2" \ - -F "type=knowledge" \ - -F "database=acme_corp" \ - -F "documents=@/path/to/policy.pdf" \ - -F 'document_metadata=[{ "id": "billing-policy-doc" }]' \ - -F 'graph_payload={ - "billing-policy-doc": { - "entities": { - "alice": { "name": "Alice Carter", "type": "PERSON", "namespace": "employees" }, - "billing": { "name": "Billing Policy", "type": "POLICY", "namespace": "policies" } - }, - "relations": [ - { "source": "alice", "target": "billing", "predicate": "OWNS", - "context": "Alice Carter owns the billing policy." } - ] - } - }' - ``` - - ```python Python SDK - import json - - with open("/path/to/policy.pdf", "rb") as f: - client.context.ingest( - type="knowledge", - database="acme_corp", - documents=[("policy.pdf", f, "application/pdf")], - document_metadata=json.dumps([{"id": "billing-policy-doc"}]), - graph_payload=json.dumps({ - "billing-policy-doc": { - "entities": { - "alice": {"name": "Alice Carter", "type": "PERSON", "namespace": "employees"}, - "billing": {"name": "Billing Policy", "type": "POLICY", "namespace": "policies"}, - }, - "relations": [ - {"source": "alice", "target": "billing", "predicate": "OWNS", - "context": "Alice Carter owns the billing policy."}, - ], - } - }), - ) - ``` - - ```typescript TypeScript SDK - await client.context.ingest({ - type: "knowledge", - database: "acme_corp", - documents: [{ path: "/path/to/policy.pdf", filename: "policy.pdf", contentType: "application/pdf" }], - documentMetadata: JSON.stringify([{ id: "billing-policy-doc" }]), - graphPayload: JSON.stringify({ - "billing-policy-doc": { - entities: { - alice: { name: "Alice Carter", type: "PERSON", namespace: "employees" }, - billing: { name: "Billing Policy", type: "POLICY", namespace: "policies" }, - }, - relations: [ - { source: "alice", target: "billing", predicate: "OWNS", - context: "Alice Carter owns the billing policy." }, - ], - }, - }), - }); - ``` - - - - | Field | Description | - | --- | --- | - | | Top-level key: a `document_metadata` id or `app_knowledge` item id for `type=knowledge`, or a memory `id` for `type=memory`. Value is that source's graph. A key matching no source returns `400`. | - | | Map keyed by a caller-local id; each value is an entity. The key is only a handle for `relations` to reference - it is not stored. | - | | Entity name. Normalized (lowercased) server-side so it matches at query time. ≤ 256 chars. | - | | Entity type (e.g. `PERSON`, `POLICY`). Stored as supplied. | - | | Logical grouping for the entity. Stored as supplied. | - | | Optional external id (email, URL, etc.) - display only. | - | | Edges referencing entity-map keys. | - | | Entity-map key of the source entity. | - | | Entity-map key of the target entity. | - | | Relationship label, any plain string. ≤ 256 chars. | - | | Optional sentence supporting the edge. ≤ 2,000 chars. | - | | Optional timing info (e.g. "since 2021", "in Q3"). | - - - **Per-source replace mode.** Each top-level key must match a `document_metadata` id or `app_knowledge` item id for `type=knowledge`, or a memory `id` for `type=memory`, in the same request; attach graphs to multiple sources at once. Extraction is skipped for keyed sources. Caps per graph: ≤ 5,000 entities, ≤ 10,000 relations, ≤ 500 relations per entity; over-cap returns `400`. Graphs survive re-ingest (re-upload or connector re-sync re-applies the stored graph). - - - -### Some important notes - -- **Async indexing.** `202 Accepted` means HydraDB queued the work, not that content is searchable. Poll [Ingestion Status](/api-reference/v2/endpoint/source-status) until `indexing_status` reaches `graph_creation` (searchable) or `completed` (graph-ready). -- **Multipart, not JSON.** This endpoint uses `multipart/form-data`. Stringify all JSON arrays (`metadata`, `app_knowledge`, `memories`) before placing them in the form field. -- **Declare hot schema fields upfront.** Put frequently filtered fields in `metadata`, define them in `database_metadata_schema` with `enable_match: true`, and use `additional_metadata` for free-form display/bookkeeping fields. Define filterable fields when creating the database via [Create Database](/api-reference/v2/endpoint/create-tenant). Additive schema updates exist, but data already ingested is not re-indexed for newly added dense/sparse metadata fields. -- **Memory vs knowledge.** Use `type: "memory"` for memory ingestion, listing, and deletion. Use `type: "all"` on `POST /query` when results should combine both. The multipart field name for memories is always `memories`. -- **Collection defaulting.** Omitting `collection` writes to the default collection. List available collections with [List Collections](/api-reference/v2/endpoint/list-sub-tenants). + + **Related Resources** + + - **Always check** [ingestion status](/api-reference/v2/endpoint/source-status) to ensure context is ready to be retrieved + - [Query](/api-reference/v2/endpoint/query) once context is ready + - **Inspect:** [List Context](/api-reference/v2/endpoint/list-documents) lists the items you ingested, with their titles + - **Inspect:** [Inspect Context](/api-reference/v2/endpoint/fetch-content) returns the stored content behind a `context_id`; [List Context](/api-reference/v2/endpoint/list-documents) with `ids` returns its title and attributes + - **Cleanup:** [Delete Context](/api-reference/v2/endpoint/delete-source) + - **Collections:** omitting `collection` writes to the default collection; list them with [List Collections](/api-reference/v2/endpoint/list-sub-tenants) + diff --git a/api-reference/v2/endpoint/list-documents.mdx b/api-reference/v2/endpoint/list-documents.mdx index 2a592ce8..19f0687d 100644 --- a/api-reference/v2/endpoint/list-documents.mdx +++ b/api-reference/v2/endpoint/list-documents.mdx @@ -1,45 +1,39 @@ --- title: "List Context" -description: "Browse over knowledge or memories with optional filters. Results are paginated. " -openapi: "api-reference/v2/openapi.json POST /context/list" +description: "Browse the context items in a database or collection with optional filters. Results are paginated. " --- import { Field } from "/snippets/field.jsx"; -Specify the category using the `type` parameter to filter and view ingested knowledge or user memories within a database or collection: - -- `type=knowledge` _(default)_ - knowledge sources (documents, app sources). -- `type=memory` - user memories. +List the context items in a database or collection: everything you ingested and everything your connectors synced, in one paginated listing. Each row carries an item's `id` and its metadata; fetch the full content of one item with [Inspect Context](/api-reference/v2/endpoint/fetch-content). Supports pagination, metadata filters, and field projection. For metadata design and query-time behavior, see [Scoping using metadata](/essentials/v2/attributes). ```python Python SDK -sources = client.context.list( +items = client.context.list( database="acme_corp", - type="knowledge", page=1, page_size=50, filters={ "metadata": {"department": "legal"}, "source_fields": {"type": "slack"}, }, - include_fields=["title", "type", "timestamp", "additional_metadata"], + include_fields=["title", "timestamp", "additional_metadata"], ) ``` ```typescript TypeScript SDK -const sources = await client.context.list({ +const items = await client.context.list({ database: "acme_corp", - type: "knowledge", page: 1, pageSize: 50, filters: { metadata: { department: "legal" }, source_fields: { type: "slack" }, }, - includeFields: ["title", "type", "timestamp", "additional_metadata"], + includeFields: ["title", "timestamp", "additional_metadata"], }); ``` @@ -50,14 +44,13 @@ curl -X POST 'https://api.hydradb.com/context/list' \ -H "Content-Type: application/json" \ -d '{ "database": "acme_corp", - "type": "knowledge", "page": 1, "page_size": 50, "filters": { "metadata": { "department": "legal" }, "source_fields": { "type": "slack" } }, - "include_fields": ["title", "type", "timestamp", "additional_metadata"] + "include_fields": ["title", "timestamp", "additional_metadata"] }' ``` @@ -69,16 +62,17 @@ curl -X POST 'https://api.hydradb.com/context/list' \ | --- | --- | | | Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | | Collection scope. If omitted, the default collection is used. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=`null`) | -| | Bucket to list. (default=`"knowledge"`) | -| | When provided and non-empty, only items with these IDs are returned (pagination \+ filters still apply). (default=`null`) | +| | When provided and non-empty, only items with these IDs are returned (pagination \+ filters still apply). At most `100` IDs. (default=`null`) | | | Page number (1-indexed). (default=`1`) | | | Items per page, from `1` to `100`. (default=`50`) | -| | Structured exact-match filters. See [Filters](#1-filters). (default=`null`) | -| | Field projection. Only the listed fields plus `id`, `database`, `collection` are populated. Only applies to `type=knowledge`. (default=`null` - all fields) | +| | Structured filters. See [Filters](#1-filters). (default=`null`) | +| | Field projection. Only the listed fields plus `id`, `database`, `collection` are populated. See [Including fields](#2-including-fields-for-convenient-data-objects). (default=`null`, meaning all fields) | +| | Nest each thread's comments and replies under their parent row as `comments`, newest first, instead of listing them as separate rows. (default=`false`) | +| | Principals to answer as: only items they may see are listed. Omit for no access scoping. See [Access control](/essentials/v2/access-control). | ### 1. Filters -- `filters` is a structured object with three optional categories. Filters are exact-match constraints i.e. filtered values are matched against stored values as exact values. There are no range, contains, or OR operators on this endpoint; run multiple calls and merge client-side for OR behavior. +- `filters` is a structured object with three optional categories. Filters are exact-match constraints i.e. filtered values are matched against stored values as exact values. The one exception is `source_fields.title`, which matches as a case-insensitive prefix. There are no range, contains, or OR operators on this endpoint; run multiple calls and merge client-side for OR behavior. - **AND/OR:** All filter pairs combine with a logical AND. To express OR semantics, run multiple calls and union them client-side. - `ids `**\+ filters:** When `ids` is non-empty, only those IDs are considered, but other `filters` still apply on top - useful for "show me items 1, 2, 3 that also belong to department=legal". @@ -95,8 +89,8 @@ curl -X POST 'https://api.hydradb.com/context/list' \ | Category | Matched against | Notes | | --- | --- | --- | | | Context item's schema-aligned `metadata` payload | Use for database metadata fields. `tenant_metadata` is accepted as a legacy alias. Keys must be declared in the database's `database_metadata_schema` with `enable_match: true`; undeclared keys are silently ignored. | -| | Context item's `additional_metadata` payload | Free-form per-document JSON. No schema declaration required. `document_metadata` is accepted as a legacy alias. | -| | Built-in source fields (`type`, `title`, `description`, `url`, `timestamp`) | Use for app-source categories or quick title lookups. | +| | Context item's `additional_metadata` payload | Free-form per-item JSON. No schema declaration required. `document_metadata` is accepted as a legacy alias. | +| | Built-in item fields: `type`, `title`, `description`, `url`, `timestamp`, and the connector fields `app_provider`, `app_kind`, `app_external_id`, `app_parent_id` | Use for connector categories or quick title lookups. `app_external_id` and `app_parent_id` are only unique per provider, so pair them with `app_provider`. | ### 2. Including Fields for convenient data objects @@ -105,10 +99,48 @@ When you don't need every field on every row, pass `include_fields` to keep resp Allowed values are `title`, `type`, `description`, `note`, `timestamp`, `metadata`, `additional_metadata`, and `relations`. Omit or pass `null` to return everything. - **Projectable vs. fetchable fields.** `content`, `url`, and `attachments` are **not** valid `include_fields` values - they are stripped from list responses, and requesting one returns `400`. Fetch them per-source via [Inspect Context](/api-reference/v2/endpoint/fetch-content). + **Projectable vs. fetchable fields.** `content`, `url`, and `attachments` are **not** valid `include_fields` values: they are stripped from list responses, and requesting one returns `400`. Fetch them per item via [Inspect Context](/api-reference/v2/endpoint/fetch-content). -`include_fields` only applies to `type=knowledge`. It is ignored for `type=memory`. +## Response + +`data` holds one page of the listing: + +| Name | Description | +| --- | --- | +| | The listed context items, one row per item (fields below). | +| | Total number of matching items across all pages. | +| | `page`, `page_size`, `total`, `total_pages`, `has_next`, `has_previous`. | +| | Human-readable result message. | +| | Deprecated. Always the same value as the envelope's top-level `success`; check the HTTP status instead. | + +Each row in `sources` carries: + +| Name | Description | +| --- | --- | +| | The item's ID. Always present. | +| | Database the item was listed from. Always present. | +| | Collection the item was listed from. Empty when it is in the database's default collection. Always present. | +| | Title of the item. | +| | Source kind of the item. | +| | Human-readable description. | +| | Free-form note attached to the item. | +| | RFC3339 timestamp associated with the item. | +| | Database metadata (declared fields) supplied at ingest. | +| | Free-form metadata supplied at ingest or by a connector. | +| | Relations attached to the item. Returned only when requested with `include_fields`. | +| | Connector the item came from (for example `slack` or `github`). Absent for items that did not come from a connector. | +| | Connector item category. | +| | Provider-assigned identifier for the item. | +| | Provider ID of the item's parent in a conversation (for example a Jira comment's issue key, or a Slack reply's thread root). | +| | Discussion grouping key shared by a thread root and its replies or comments. | +| | Connector-derived relations for the item. | +| | With `group_threads`, the item's comments and replies as full rows, newest first, capped per parent. | +| | With `group_threads`, `true` when `comments` hit the per-parent cap and more exist. | +| | Deprecated alias for `database`. | +| | Deprecated alias for `collection`. | + +Fields a row does not have, or that `include_fields` left out, are omitted. @@ -117,14 +149,13 @@ Allowed values are `title`, `type`, `description`, `note`, `timestamp`, `metadat "success": true, "data": { "success": true, - "message": "Sources retrieved successfully", + "message": "Successfully fetched sources", "sources": [ { "id": "policy_main", "database": "acme_corp", "collection": "team_docs", "title": "Compliance Policy", - "type": "pdf", "timestamp": "2026-05-12T08:14:00Z", "additional_metadata": { "author": "Compliance Team" } } @@ -147,36 +178,6 @@ Allowed values are `title`, `type`, `description`, `note`, `timestamp`, `metadat } ``` -```json Memories response -{ - "success": true, - "data": { - "success": true, - "user_memories": [ - { - "memory_id": "mem_user_alex_tone", - "memory_content": "Prefers concise answers and dark mode.", - "inferred_content": "User prefers concise answers and dark mode." - } - ], - "total": 1, - "pagination": { - "page": 1, - "page_size": 50, - "total": 1, - "total_pages": 1, - "has_next": false, - "has_previous": false - } - }, - "error": null, - "meta": { - "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", - "latency_ms": 12.3 - } -} -``` - ```json Failure { "success": false, @@ -194,8 +195,6 @@ Allowed values are `title`, `type`, `description`, `note`, `timestamp`, `metadat -When `type=memory`, `data` is a `ListUserMemoriesResponse` instead - same idea but with a `data.user_memories[]` array of memory items. - **Use the canonical v2 names.** Prefer `filters.metadata` and `filters.additional_metadata`. Legacy `filters.tenant_metadata` and `filters.document_metadata` are accepted for back-compat, with canonical keys winning on conflicts. @@ -206,7 +205,7 @@ When `type=memory`, `data` is a `ListUserMemoriesResponse` instead - same idea **Related Resources** - - **Fetch content:** [Fetch Content](/api-reference/v2/endpoint/fetch-content) + - **Fetch content:** [Inspect Context](/api-reference/v2/endpoint/fetch-content) - **Retrieve:** [Query](/api-reference/v2/endpoint/query) - **Delete:** [Delete Context](/api-reference/v2/endpoint/delete-source) diff --git a/api-reference/v2/endpoint/list-tenants.mdx b/api-reference/v2/endpoint/list-tenants.mdx index bfe7ea3f..b46fb68d 100644 --- a/api-reference/v2/endpoint/list-tenants.mdx +++ b/api-reference/v2/endpoint/list-tenants.mdx @@ -1,10 +1,11 @@ --- title: "List Databases" description: "List all databases created. " -openapi: "api-reference/v2/openapi.json GET /databases" --- -The response separates active or provisioning databases (in `data.databases`) from databases whose provisioning failed (in `data.failed_databases`). Use [Database Status](https://docs.hydradb.com/api-reference/v2/endpoint/tenant-status) to confirm readiness before ingestion. +import { Field } from "/snippets/field.jsx"; + +The response separates active or provisioning databases (in `data.databases`) from databases whose provisioning failed (in `data.failed_databases`). Use [Database Status](/api-reference/v2/endpoint/tenant-status) to confirm readiness before ingestion. This endpoint takes no parameters. @@ -32,7 +33,7 @@ curl -X GET 'https://api.hydradb.com/databases' \ "data": { "databases": ["acme_corp", "my_first_database"], "failed_databases": [], - "message": "Successfully retrieved database IDs" + "message": "Successfully retrieved tenant IDs" }, "error": null, "meta": { @@ -53,7 +54,7 @@ curl -X GET 'https://api.hydradb.com/databases' \ "error": "Provisioning failed. Re-create the database to retry." } ], - "message": "Successfully retrieved database IDs" + "message": "Successfully retrieved tenant IDs" }, "error": null, "meta": { @@ -80,6 +81,17 @@ curl -X GET 'https://api.hydradb.com/databases' \ +## Response + +| Name | Description | +| --- | --- | +| | Active or provisioning databases in your organization. | +| | One entry per database in `databases`, each carrying its `database` name. | +| | Databases whose provisioning failed. Each entry has `database` and `error` (why provisioning failed). Empty when none failed. | +| | Human-readable result message. | +| | Deprecated alias for `databases`. | +| | Deprecated alias for `failed_databases`; `null` when none failed. | + ## Retry notes - If provisioning failed for a database, `data.failed_databases` contains diagnostic entries as shown in the **Provisioning issue** tab. diff --git a/api-reference/v2/endpoint/query-overview.mdx b/api-reference/v2/endpoint/query-overview.mdx index aab96cff..5e70b0b1 100644 --- a/api-reference/v2/endpoint/query-overview.mdx +++ b/api-reference/v2/endpoint/query-overview.mdx @@ -5,7 +5,7 @@ description: "Quick reference for scoping, matching and retrieval modes, and wha import { Field } from "/snippets/field.jsx"; -Use this page to choose the right query shape before opening the full [Query](/api-reference/v2/endpoint/query) endpoint reference. A query has three decisions: where to look (`collection` or `collections`), how to match (`query_by`), and how much retrieval work to spend (`mode`). There is no corpus selector on a unified database: one database is one corpus. +Use this page to choose the right query shape before opening the full [Query](/api-reference/v2/endpoint/query) endpoint reference. A query has three decisions: where to look (`collection` or `collections`), how to match (`query_by`), and how much retrieval work to spend (`mode`). ```mermaid flowchart LR @@ -41,7 +41,7 @@ linkStyle default stroke:#64748b,stroke-width:2px; | | `0.0` to `1.0` or `"auto"` | Tune hybrid query. Lower values favor BM25 keywords; higher values favor semantic similarity. | | | object | Narrow candidates with operators (`$eq`, `$in`, `$gte`, `$and`, ...) on the fields declared in `database_metadata_schema`. | | | boolean | Include graph paths in `graph[]`. On by default; set `false` for chunk-only responses. | -| | boolean | Pull items linked with `forceful_relations` at ingest into `forceful_relations[]`. On by default. | +| | boolean | Pull items linked with `forceful_relations` at ingest into `forceful_relations[]`. On by default; followed only in `thinking` mode. | | | boolean | Adds app-aware retrieval for connector content while still querying the full selected scope. | @@ -158,17 +158,13 @@ Use text query when literal wording matters: legal clauses, SKUs, error codes, I | Key | Contents | | --- | --- | -| `chunks[]` | Ranked matches: `chunk_id`, `context_id`, `score`, `content`, optional `enrichment` (a string), `enrichment_kind` and `temporal`. No source details; call `GET /context/inspect` with the `context_id` for those. | -| `graph[]` | Paths through the context graph: `origin` (`query_path` or `chunk_relation`), `triplets[]` and a `path_summary`, which is never empty. Each hop's `relation.chunk_id` names the chunk it came from; a `chunk_relation` path is only returned when one of its hops came from a returned chunk or a `forceful_relations` chunk. | -| `forceful_relations[]` | Chunks linked with `forceful_relations` at ingest, each with the `via` that brought it in. | +| `chunks[]` | Ranked matches: `chunk_id`, `context_id`, `score`, `content`, optional `enrichment` (a string), `enrichment_kind` and `temporal`. No source details; call `POST /context/list` with the `context_id` in `ids` for those. | +| `graph[]` | Paths through the context graph, deduplicated across both lanes and not capped: `origin` (`query_path` or `chunk_relation`), `triplets[]` and a `path_summary`, which is never empty. Each hop's `relation.chunk_id` names the chunk it came from, and `relation.timestamp` (Unix epoch seconds) is present when the edge has one; a `chunk_relation` path is only returned when one of its hops came from a returned chunk or a `forceful_relations` chunk. | +| `forceful_relations[]` | Chunks linked with `forceful_relations` at ingest, each with the `via` that brought it in. Followed only in `thinking` mode. | | `llm_prompt` | A server-built markdown string, ready to inject into a model call: results cited `[1]`, forceful relations `[R1]`, related facts labelled `[P1]` in `graph[]` order with each path's relevance when it has one, then temporal facts and sources. | Inject `llm_prompt` for the model; preserve `chunks[]` order when you render results yourself. See [How to Use API Results](/essentials/v2/api-results). -## Split databases - -A database created with `type: "split"` adds one decision: `type` (`"knowledge"`, `"memory"` or `"all"`) picks the corpus, and the response is the older shape (`chunk_content`, `sources[]`, `graph_context`, `additional_context`). See the [Split databases](/api-reference/v2/endpoint/query#split-databases) section of the endpoint reference. - ## Related sections - [Query](/api-reference/v2/endpoint/query): full endpoint reference diff --git a/api-reference/v2/endpoint/query.mdx b/api-reference/v2/endpoint/query.mdx index 739b66c3..f2dec76b 100644 --- a/api-reference/v2/endpoint/query.mdx +++ b/api-reference/v2/endpoint/query.mdx @@ -4,18 +4,15 @@ description: "Retrieve ranked chunks, graph paths, forceful relations and a prom --- import { Field } from "/snippets/field.jsx"; -import LegacyLine from "/snippets/legacy-line.mdx"; The single retrieval endpoint. Use it any time you need to feed an LLM with grounded, personalized context, or fetch chunks ranked by relevance. -Two dimensions control behavior on a unified database: +Two dimensions control behavior: - **`query_by`** picks **how** to match: `"hybrid"` (semantic + BM25, the default) or `"text"` (BM25 only, pair with `operator`). - **`mode`** picks **how much** retrieval work to spend: `"fast"` (single pass, low latency), `"thinking"` (expands the query, reranks, follows declared relations) or `"auto"` (scores the query and routes to one of the two, defaulting to `"thinking"` when the signal is inconclusive; **the default when `mode` is omitted**). -There is no `type`: a unified database is one corpus. Scope with `collection` or `collections`. The guide is [Query](/essentials/v2/query); this page is the field reference. - - +Scope with `collection` or `collections`. The guide is [Query](/essentials/v2/query); this page is the field reference. `database` and `collection` are the current field names (formerly `tenant_id` and `sub_tenant_id`). The old names remain accepted as deprecated aliases for full backward compatibility. @@ -130,7 +127,7 @@ A list gives every collection equal normalized weight. An object treats values a ### Using the response -`data.llm_prompt` is the whole context block as one markdown document: `# Query results`, then `## Results` (cited `[1]`), `## Forceful relations` (`[R1]`), `## Related facts` (`[P1]`, one per path in `graph[]`), `## Temporal facts` and `## Sources`. The SDK `build_string` / `buildString` helpers return it verbatim on a unified database. Inject it directly: +`data.llm_prompt` is the whole context block as one markdown document: `# Query results`, then `## Results` (cited `[1]`), `## Forceful relations` (`[R1]`), `## Related facts` (`[P1]`, one per path in `graph[]`), `## Temporal facts` and `## Sources`. Inject it directly: ```python messages = [{"role": "system", "content": result.data.llm_prompt}, @@ -361,15 +358,11 @@ result = client.query( | | Principals to answer as: only items they may retrieve are returned. Omit, or send `[]` or `["*"]`, for no access scoping. See [Access control](/essentials/v2/access-control). | | | Hard filter on declared attributes with operators: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$and`, `$or`, `$not`, `$exists`. Applies to chunks, forceful relations and graph paths alike. See [Filters](#decision-matrix). | | | When `true`, includes graph paths in `graph[]`. Set to `false` when you only need ranked chunks; `graph` is then `[]`. Relations you supplied via [Bring Your Own Graph](/essentials/v2/bring-your-own-graph) appear identically to extracted ones. (default=`true`) **Under `mode: "auto"`, this value is overridden by the resolved mode.** | -| | Pull the items each hit declared with `forceful_relations` at ingest into `forceful_relations[]`. Set to `false` for `forceful_relations: []`. `query_forceful_relations` is the deprecated alias. (default=`true`) | +| | Pull the items each hit declared with `forceful_relations` at ingest into `forceful_relations[]`. Declared relations are followed only in `thinking` mode. Set to `false` for `forceful_relations: []`. `query_forceful_relations` is the deprecated alias. (default=`true`) | | | Resolve time-based questions (current, as of, ranges, upcoming) and return matched facts in `chunks[].temporal`. Never changes which chunks are returned. (default=`true`) | | | ISO 8601 time to treat as now for temporal reasoning. Set it when replaying past conversations. | | | Override the temporal intent HydraDB would infer from the query. | -| | Deprecated. The older filter language; still accepted on every database and ANDed with `attributes` when both are sent. See [Split databases](#split-databases). | - - -**Do not send `type` on a unified database.** Absent, `"all"` and `"unified"` are accepted and mean the one corpus. `"knowledge"` and `"memory"` return `400 CORPUS_TYPE_UNSUPPORTED`. Detect a database's layout once from `GET /databases` (`details[].type`) rather than branching on a request flag. - +| | The older filter language, ANDed with `attributes` when both are sent. Prefer `attributes`; use `metadata_filters` only to filter on connector fields under `additional_metadata`, which `attributes` does not reach (see [Connectors](/essentials/v2/connectors)). | **Tuning heuristics.** @@ -472,6 +465,7 @@ result = client.query( "relation": { "predicate": "managed by", "context": "Refund processing is managed by the Finance Department.", + "timestamp": 1782984600.0, "relationship_id": "rel_managed_by", "chunk_id": "ck_policy_3" }, @@ -520,7 +514,7 @@ result = client.query( } } ], - "llm_prompt": "# Query results\n\n**Query:** who owns refund processing?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (query path, relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (chunk relation, relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" + "llm_prompt": "# Query results\n\n**Query:** who owns refund processing?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" }, "error": null, "meta": { @@ -538,8 +532,8 @@ result = client.query( "success": false, "data": null, "error": { - "code": "INVALID_PARAMETERS", - "message": "query must not be empty" + "code": "INVALID_INPUT", + "message": "query cannot be empty" }, "meta": { "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", @@ -555,17 +549,17 @@ result = client.query( | Key | Contents | | --- | --- | -| `chunks[]` | Ranked matches: `chunk_id`, `context_id`, `score`, `content` (verbatim), `enrichment` (the extracted statement as a string, omitted when empty), `enrichment_kind` (the item's declared `context_category`: `user_preference`, `business_knowledge` or `decision_trace`; omitted when none was declared, present even without `enrichment`), `temporal[]` (only when temporal reasoning engaged; `{ content, start_date, end_date }`, dates may be `null`). | -| `graph[]` | Paths through the context graph, query paths first then chunk expansions, deduplicated: `origin`, `triplets[]` of `source` / `relation` / `target`, plus `path_summary`. `origin` is `"query_path"` (grown from the entities in the query) or `"chunk_relation"` (the neighbourhood of a returned chunk, only returned when one of its hops came from a returned chunk or a `forceful_relations` chunk). `path_summary` is never empty: when the server wrote no summary, it narrates the hops. Entities are `{ entity_id, name }`; relations are `{ predicate, context, temporal_details?, relationship_id, chunk_id }`. `[]` when `graph_context` is `false`. | -| `forceful_relations[]` | Chunks pulled in through `forceful_relations` declared at ingest: `via.from` (the context whose declaration pulled it in, may be `""`), `via.to` (the chunk's own `context_id`), `chunk` (same shape as `chunks[]`). `[]` when none or `follow_forceful_relations` is `false`. | -| `llm_prompt` | A server-built markdown string ready to inject into a model call: `# Query results`, then `## Results`, `## Forceful relations`, `## Related facts`, `## Temporal facts` and `## Sources`, each left out when empty. Results are cited `[1]` and forceful relations `[R1]`; related facts are labelled `[P1]`, `[P2]`, ... in `graph[]` order, carry the path's relevance after reranking when it has one, and end with the results they were extracted from. Sources print only web (`http` or `https`) links. `""` when nothing matched. The layout is on [Query](/essentials/v2/query#llm_prompt). | +| `chunks[]` | Ranked matches: `chunk_id`, `context_id`, `score`, `content` (verbatim), `enrichment` (the extracted statement as a plain string, omitted when there is none), `enrichment_kind` (the item's declared `context_category`: `user_preference`, `business_knowledge` or `decision_trace`; omitted when none was declared, present even without `enrichment`), `temporal[]` (only when the query engaged temporal reasoning; `{ content, start_date, end_date }`, where `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` and either date may be `null`). | +| `graph[]` | Paths through the context graph, query paths first then chunk expansions: `origin`, `triplets[]` of `source` / `relation` / `target`, plus `path_summary`. `origin` is `"query_path"` (grown from the entities in the query) or `"chunk_relation"` (the neighbourhood of a returned chunk, only returned when one of its hops came from a returned chunk or a `forceful_relations` chunk). The array is deduplicated across both lanes and is not capped. `path_summary` is never empty: when the server wrote no summary, it narrates the hops. Entities are `{ entity_id, name }`; relations are `{ predicate, context, temporal_details?, timestamp?, relationship_id, chunk_id }`, where `temporal_details` is omitted when empty and `timestamp` (Unix epoch seconds, a float) is omitted when the edge has none. `[]` when `graph_context` is `false`. | +| `forceful_relations[]` | Chunks pulled in through `forceful_relations` declared at ingest, followed only in `thinking` mode: `via.from` (the context whose declaration pulled it in, may be `""`), `via.to` (the chunk's own `context_id`), `chunk` (same shape as `chunks[]`). `[]` when none, when `follow_forceful_relations` is `false`, or when the query ran in `fast` mode. | +| `llm_prompt` | A server-built markdown string ready to inject into a model call: `# Query results`, then `## Results`, `## Forceful relations`, `## Related facts`, `## Temporal facts` and `## Sources`, each left out when empty. Results are cited `[1]` and forceful relations `[R1]`; related facts are labelled `[P1]`, `[P2]`, ... in `graph[]` order, as in `- [P1] **Refunds** -managed_by→ **Finance** (relevance 0.81) [1]`: the parenthetical is the path's relevance after reranking and is left out when the path has none, and the line ends with the results the path was extracted from. Sources print only web (`http` or `https`) links. `""` when nothing matched. The layout is on [Query](/essentials/v2/query#llm_prompt). | -To show a chunk's graph paths under that chunk, group hops by `triplets[].relation.chunk_id` and match it against `chunks[].chunk_id` (and `forceful_relations[].chunk.chunk_id`). This replaces the split response's `chunk_id_to_group_ids`. See [Attaching graph paths to chunks](/essentials/v2/query#attaching-graph-paths-to-chunks). +To show a chunk's graph paths under that chunk, group hops by `triplets[].relation.chunk_id` and match it against `chunks[].chunk_id` (and `forceful_relations[].chunk.chunk_id`). See [Attaching graph paths to chunks](/essentials/v2/query#attaching-graph-paths-to-chunks). -`meta` on a unified response carries `request_id`, `api_version`, `latency_ms`, `database` and `collection`, plus a `deprecation` list when the request used a deprecated name. It has no `tenant_id`, `sub_tenant_id` or `source_type`. `collection` is present when the query searched one collection (named, or the default); a `collections` fan-out omits it. +`meta` carries `request_id`, `api_version`, `latency_ms`, `database` and `collection`, plus a `deprecation` list when the request used a deprecated name. `collection` is present when the query searched one collection (named, or the default); a `collections` fan-out omits it. -**Chunks carry no source details.** No title, url, collection, timestamps or attributes. `llm_prompt` prints the title, url, collection and last-updated date for the model; call [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content) with `id=` to show them yourself. +**Chunks carry no source details.** No title, url, collection, timestamps or attributes. `llm_prompt` prints the title, url, collection and last-updated date for the model. To show an item's title, timestamp or attributes yourself, call [`POST /context/list`](/api-reference/v2/endpoint/list-documents) with `ids: [""]`; [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content) returns its stored content. ## Behavior notes @@ -580,106 +574,15 @@ To show a chunk's graph paths under that chunk, group hops by `triplets[].relati **Important Considerations & Common Mistakes** - **`mode: "auto"` overrides `graph_context`.** Whatever you send for `graph_context` is replaced to match the resolved mode: `true` if auto escalates to `thinking`, `false` if it resolves to `fast`. This also applies when `mode` is omitted. Set `graph_context` explicitly only when calling `"fast"` or `"thinking"` directly. -- **`type` is refused.** `"knowledge"` or `"memory"` on a unified database is a `400`. Scope with `collection` or `collections`. -- **Filter with `attributes`, on declared fields.** A key that is not in `database_metadata_schema`, or a value sent in `custom_attributes`, never matches. `metadata_filters` still works but is deprecated. -- **Parse by shape.** A response with `llm_prompt` and a `graph` array is this shape; one with `graph_context` or `chunk_content` came from a split database. Stored logs and split databases keep producing the old one. +- **Filter with `attributes`, on declared fields.** A key that is not in `database_metadata_schema`, or a value sent in `custom_attributes`, never matches. - **Common mistakes.** Check [Ingestion Status](/api-reference/v2/endpoint/source-status) for recently ingested items before querying. If you omit `collection` and `collections`, HydraDB queries the default collection; use [List Collections](/api-reference/v2/endpoint/list-sub-tenants) to discover available IDs. ## Errors -Common codes: `400 INVALID_PARAMETERS` (empty `query`), `400 CORPUS_TYPE_UNSUPPORTED` (`type: "knowledge"` or `"memory"` on a unified database), `404 DATABASE_NOT_FOUND`, `422 VALIDATION_ERROR`, `500 INTERNAL_ERROR`. See [Error Responses](/api-reference/v2/error-responses) for the full list. - -`400` also covers oversized filters: a `metadata_filters` list above 500 values, or a `metadata_filters` object above 64 KiB of compact JSON. The message names the offending key or reports the actual byte count. See [Filter size limits](/essentials/v2/attributes#filter-size-limits). - -## Split databases - -Everything above describes a unified database. A database created with `type: "split"` keeps the request and response it always had. The differences: - -### `type` selects the corpus - -| Value | Queries | Best for | -|---|---|---| -| `"knowledge"` *(default)* | Knowledge documents, files, and app sources | Document Q&A, RAG context. | -| `"memory"` | User memories | Personalization and user preferences. | -| `"all"` | Both, merged in one ranked result set | Personalized answers grounded in both shared and user-specific context. | - -```json -{ - "database": "legacy-app", - "collection": "user_alex", - "query": "What is our refund policy, and how should I explain it to this user?", - "type": "all", - "query_by": "hybrid", - "mode": "thinking", - "query_forceful_relations": true, - "metadata_filters": { - "department": "support", - "additional_metadata": { "source": "policy" } - }, - "additional_context": "User is asking from the billing help center." -} -``` - -- `query_forceful_relations` pulls author-declared related sources into the response's `additional_context`. It only takes effect when `mode` resolves to `"thinking"`; in `fast` mode the flag is silently ignored. -- `additional_context` on the request is a short factual hint to guide retrieval. It is different from the response `additional_context` map. -- `metadata_filters` is the split-era filter language. Top-level keys match schema-backed `metadata` and take an operator object (`equals`, `contains`, `contains_any`); free-form per-source fields go under `additional_metadata`, where a bare scalar is an exact match and a bare array matches any listed value. Each list holds at most 500 values and the whole object is capped at 64 KiB. The full semantics are on [Split databases and legacy fields](/essentials/v2/split-databases#metadata_filters). - -### The split response - -```json -{ - "success": true, - "data": { - "chunks": [ - { - "chunk_uuid": "policy_main_chunk_3", - "id": "policy_main", - "chunk_content": "Refunds are issued within 30 days...", - "source_type": "pdf", - "source_title": "Compliance Policy", - "source_upload_time": "2026-05-12T08:14:00Z", - "source_last_updated_time": "2026-05-12T08:14:00Z", - "relevancy_score": 0.91, - "extra_context_ids": ["pref-tone"], - "metadata": { "department": "legal" }, - "additional_metadata": { "author": "Legal Team" } - } - ], - "sources": [ - { "id": "policy_main", "title": "Compliance Policy", "type": "pdf", "timestamp": "2026-05-12T08:14:00Z" } - ], - "graph_context": { - "query_paths": [ - { - "triplets": [ - { - "source": { "name": "Compliance Policy", "type": "DOCUMENT", "entity_id": "entity_compliance_policy" }, - "relation": { "canonical_predicate": "GOVERNS", "context": "The compliance policy governs the refund timeline.", "timestamp": 1778573640.0, "chunk_id": "policy_main_chunk_3" }, - "target": { "name": "Refund Processing", "type": "PROCESS", "entity_id": "entity_refund_processing" } - } - ], - "relevancy_score": 0.89, - "combined_context": "The Compliance Policy governs the Refund Processing.", - "source_chunk_ids": ["policy_main_chunk_3"] - } - ], - "chunk_relations": [], - "chunk_id_to_group_ids": { "policy_main_chunk_3": ["p_0"] } - }, - "additional_context": { - "pref-tone": { "chunk_uuid": "pref-tone", "id": "mem_user_alex_tone", "chunk_content": "Prefers concise answers.", "source_type": "memory", "source_title": "User preferences" } - } - }, - "error": null, - "meta": { "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", "latency_ms": 12.3 } -} -``` - -- Relation `timestamp` in `graph_context` is a Unix epoch float (seconds). The dedicated [Context Relations](/api-reference/v2/endpoint/source-relations) endpoint returns the same field as an ISO-8601 string; normalize before comparing across endpoints. -- There is no `llm_prompt`. Format the split response for a model with the SDK's `build_string` / `buildString` helper; see [How to Use API Results](/essentials/v2/api-results#7-split-databases). +Common codes: `400 INVALID_INPUT` (empty `query`), `400 VALIDATION_ERROR` (a malformed `attributes` filter), `404 DATABASE_NOT_FOUND`, `500 INTERNAL_ERROR`. See [Error Responses](/api-reference/v2/error-responses) for the full list. -The field-by-field mapping between the two shapes is on [Split databases and legacy fields](/essentials/v2/split-databases#4-legacy-query-and-response-fields). +`400` also covers oversized filters: an `attributes` list above 500 values, or an `attributes` object above 64 KiB of compact JSON. The message names the offending key or reports the actual byte count. See [Attributes](/essentials/v2/attributes).
@@ -688,7 +591,7 @@ The field-by-field mapping between the two shapes is on [Split databases and leg - **Setup first:** [Ingest Context](/api-reference/v2/endpoint/ingest-context): content must be indexed - **Confirm indexing:** [Ingestion Status](/api-reference/v2/endpoint/source-status): wait for `completed` (or `graph_creation`) -- **Source details:** [Inspect Context](/api-reference/v2/endpoint/fetch-content): title, attributes and content for a `context_id` +- **Source details:** [List Context](/api-reference/v2/endpoint/list-documents) with `ids` for an item's title and attributes, [Inspect Context](/api-reference/v2/endpoint/fetch-content) for its stored content - **Graph follow-up:** [Context Relations](/api-reference/v2/endpoint/source-relations): inspect relationships in detail - **Concepts:** [Usage: Query](/essentials/v2/query) - **Concepts:** [Concepts: Context Graphs](/essentials/v2/context-graphs) diff --git a/api-reference/v2/endpoint/source-relations.mdx b/api-reference/v2/endpoint/source-relations.mdx index ca819275..6be2a556 100644 --- a/api-reference/v2/endpoint/source-relations.mdx +++ b/api-reference/v2/endpoint/source-relations.mdx @@ -1,6 +1,5 @@ --- title: "Inspecting Context Relations" -openapi: "api-reference/v2/openapi.json GET /context/relations" description: "See and explore relationships that create the brain for your AI. " --- @@ -8,7 +7,7 @@ import { Field } from "/snippets/field.jsx"; This endpoint queries entity-and-relationship triplets extracted from your ingested content. -Pass `id` to scope to a single ingested item, or omit it to return all relations in the collection. Set `type=memory` to inspect a memory's relations. Pagination handles large result sets. +Pass `id` to scope to a single ingested item, or omit it to return all relations in the collection. Pagination handles large result sets. @@ -45,10 +44,37 @@ curl -G 'https://api.hydradb.com/context/relations' \ | --- | --- | | | Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | | When provided, returns relations for that specific source. When omitted, returns all relations across the collection. (default=`null`) | -| | Bucket selector. Use `"memory"` when `id` belongs to a memory item. (default=`"knowledge"`) | | | Collection scope. If omitted, the default collection is used. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=`null`) | -| | Maximum relation groups to return. Range `1–10000`. (default=`5000`) | +| | Maximum relation groups to return. Range `1` to `10000`. (default=`5000`) | | | Opaque pagination cursor from a previous response's `next_cursor`. (default=`null`) | +| | Principals to answer as: only relations from items they may see are returned. Repeated (`acl=a&acl=b`) or comma-separated. Omit for no ACL scoping. | + +## Response + +| Name | Description | +| --- | --- | +| | Entity relations, grouped per entity pair. Each group has `source` and `target` entities, the `relations[]` evidence between them, and the `chunk_id` the group was found in. Counted against `limit` and paged with `cursor`. | +| | The structural graph around those relations: where entities appear, comments and attachments on items, who authored what, and links between items. Same shape as `relations`, so concatenate the two for one graph. Does not count against `limit` or move the cursor. | +| | `true` when `auxiliary_relations` was cut off by a size limit. Independent of `is_truncated`. | +| | `true` when more `relations` exist beyond this page. | +| | Cursor for the next page, or `null` when there are no more. | +| | Human-readable result message. | +| | Deprecated. Always the same value as the envelope's top-level `success`. | + +Each entity (`source`, `target`) carries `name`, `type`, `namespace`, `entity_id`, `identifier` (`null` when none) and `provider` (the connector its evidence came from, empty when none). Each entry in a group's `relations[]` carries: + +| Name | Description | +| --- | --- | +| | Normalized relation name. | +| | The relation as extracted from the text. | +| | The sentence or passage the relation was extracted from. | +| | Extraction confidence. | +| | When the relation held, if the text said. | +| | When the relation was recorded, as an ISO-8601 string. | +| | Stable ID of the relation. | +| | Chunk the relation was extracted from. | +| | Entity ID of the relation's source end. | +| | Entity ID of the relation's target end. | @@ -89,6 +115,8 @@ curl -G 'https://api.hydradb.com/context/relations' \ "chunk_id": "policy_main_chunk_3" } ], + "auxiliary_relations": [], + "auxiliary_truncated": false, "is_truncated": false, "next_cursor": null, "success": true, @@ -188,10 +216,9 @@ while True: - **Collection-wide queries:** Omitting `id` returns relations across the entire collection. This is useful for full-graph exports; pair with a small `limit` and paginate. -- **Knowledge vs memory:** If the `id` belongs to a memory, set `type=memory`; otherwise the endpoint searches the Knowledge graph. The two graphs are completely separate. - **Ordering:** Treat `data.relations[]` as ranked by relevance within the response. Preserve order for display or LLM context, but do not compare ordering across unrelated queries as an absolute signal. - **Graph completeness:** Source relations only fully populate once the source's `indexing_status` reaches `completed`. Items in `graph_creation` are searchable but their relations may still be in flight. -- **`timestamp` format differs by endpoint.** On this endpoint each relation's `timestamp` is an ISO-8601 string (e.g. `2026-05-12T08:14:00Z`). The same relations surfaced as passthrough on [Query](/api-reference/v2/endpoint/query) (in `graph_context`) and [List Documents](/api-reference/v2/endpoint/list-documents) carry `timestamp` as a Unix epoch float (seconds) instead. Normalize before comparing relation timestamps across endpoints. +- **`timestamp` format differs by endpoint.** On this endpoint each relation's `timestamp` is an ISO-8601 string (e.g. `2026-05-12T08:14:00Z`). On [Query](/api-reference/v2/endpoint/query), each `graph[].triplets[].relation` may carry `timestamp` as Unix epoch seconds (a float) instead. Normalize before comparing relation timestamps across the two endpoints.
@@ -199,6 +226,6 @@ while True: **Related Resources** - **Indexing status:** [Ingestion Status](/api-reference/v2/endpoint/source-status) - confirm the graph is complete - - **Query with graph context:** [Query](/api-reference/v2/endpoint/query) with `graph_context: true` + - **Query with graph paths:** [Query](/api-reference/v2/endpoint/query) returns graph paths in `graph[]`, controlled by the `graph_context` request flag - **Concepts:** [Concepts → Context Graphs](/essentials/v2/context-graphs) diff --git a/api-reference/v2/endpoint/source-status.mdx b/api-reference/v2/endpoint/source-status.mdx index 3d02557f..1cdc0d78 100644 --- a/api-reference/v2/endpoint/source-status.mdx +++ b/api-reference/v2/endpoint/source-status.mdx @@ -8,7 +8,7 @@ import { Field } from "/snippets/field.jsx"; Since ingestion is asynchronous, use this endpoint to determine when context is ready to be retrieved. -Pass one or more IDs in `ids` to retrieve status. Works for documents, app sources, and memories. When passing multiple IDs on the query string, use either repeated params (`?ids=policy_main&ids=runbook_deploy`) or a single comma-joined value (`?ids=policy_main,runbook_deploy`); both forms are equivalent and can be mixed. Surrounding whitespace is trimmed and empty entries are dropped. For more information, see the [Knowledge](/essentials/v2/knowledge) and [Memories](/essentials/v2/memories) guides. +Pass one or more IDs in `ids` to retrieve status. Works for every context item, whether you ingested it or a connector synced it. When passing multiple IDs on the query string, use either repeated params (`?ids=policy_main&ids=runbook_deploy`) or a single comma-joined value (`?ids=policy_main,runbook_deploy`); both forms are equivalent and can be mixed. Surrounding whitespace is trimmed and empty entries are dropped. For more information, see the [Ingest](/essentials/v2/ingest) guide. **Prefer webhooks over polling?** Register a webhook for `indexing.status_changed` events and HydraDB will `POST` to your endpoint when content reaches a terminal state (`completed` or `errored`). See [Webhooks](/essentials/v2/webhooks) for setup and receiver examples. @@ -47,7 +47,7 @@ curl -G 'https://api.hydradb.com/context/status' \ | Name | Description | | --- | --- | -| | One or more `id` values returned at ingestion. Accepts IDs for documents, app sources, or memories. Pass either repeated params (`ids=a&ids=b`) or a single comma-joined value (`ids=a,b`). Source IDs never contain commas (they are rejected at ingest), so the comma-joined form always splits unambiguously. | +| | One or more `id` values returned at ingestion. Accepts the ID of any context item, including connector items. Pass either repeated params (`ids=a&ids=b`) or a single comma-joined value (`ids=a,b`). Source IDs never contain commas (they are rejected at ingest), so the comma-joined form always splits unambiguously. | | | Database the items belong to. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | | Collection scope. If omitted, the default collection is used. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=`null`) | @@ -113,7 +113,7 @@ Each entry in `data.statuses` describes one requested `id`: | Field | Type | Description | | --- | --- | --- | -| `id` | string | The source, app-source, or memory ID you asked about (echoed back). | +| `id` | string | The context item ID you asked about (echoed back). | | `indexing_status` | string | One of the [status values](#status-values) below. `errored` is terminal. | | `error_code` | string | Machine-readable reason an entry is `errored`; **empty string (`""`) when the entry is not errored.** See [`error_code` values](#error-code-values). | | `error_message` | string | Human-readable explanation that accompanies a non-empty `error_code`; empty otherwise. | @@ -167,7 +167,7 @@ flowchart LR | --- | --- | --- | | `queued` | No | Accepted by the server, not yet picked up by a worker. | | `processing` | No | Content is being parsed, chunked, and embedded. | -| `graph_creation` | **Yes** | Indexed and retrievable; the knowledge graph is still being built. Already searchable via `/query`, but graph context may still be incomplete. | +| `graph_creation` | **Yes** | Indexed and retrievable; the graph is still being built. Already searchable via `/query`, but graph paths may still be incomplete. | | `completed` | Yes | Fully indexed and graphed. Ready for all retrieval modes. | | `errored` | No | Processing failed. Inspect `error_code` and `error_message`. | @@ -224,7 +224,7 @@ while (true) { ### Stop when graph processing is complete -Use this before graph-heavy operations such as `/context/relations` or when you require complete `graph_context`. +Use this before graph-heavy operations such as `/context/relations`, or when you need complete graph paths (`graph[]` on `POST /query`). ```python while True: @@ -245,21 +245,21 @@ while True: Typical processing time: -- **Memories** (text, markdown, conversation pairs): seconds -- **Small documents** (under 50 pages): 1–5 minutes -- **Large documents** (50\+ pages): 5–15 minutes +- **Short text and conversation items:** seconds +- **Long text items** (the extracted text of a document under 50 pages): 1 to 5 minutes +- **Very long text items** (50\+ pages of extracted text): 5 to 15 minutes ## Behavior notes - **`graph_creation` is searchable.** Items in this state are already retrievable via `/query`. Wait for `completed` only when you specifically need full graph traversal (`graph_context: true`). + **`graph_creation` is searchable.** Items in this state are already retrievable via `/query`. Wait for `completed` only when you specifically need full graph traversal (graph paths in `graph[]`, which the `graph_context` request flag turns on). - **Unknown IDs return as `errored`:** If you pass an ID that does not exist (e.g., a typo), HydraDB returns an entry with `indexing_status: "errored"` and `error_code: "FILE_NOT_FOUND"` rather than silently dropping it. Use `error_code` to distinguish this from a genuine ingestion failure - see [`error_code` values](#error-code-values). ## Errors -Common codes: `400 INVALID_PARAMETERS`, `404 DATABASE_NOT_FOUND`, `422 VALIDATION_ERROR`. See [Error Responses](/api-reference/v2/error-responses) for the full list. +Common codes: `400 INVALID_INPUT`, `404 DATABASE_NOT_FOUND`, `422 VALIDATION_ERROR`. See [Error Responses](/api-reference/v2/error-responses) for the full list.
@@ -270,6 +270,5 @@ Common codes: `400 INVALID_PARAMETERS`, `404 DATABASE_NOT_FOUND`, `422 VALIDATIO - **After completion:** [Query](/api-reference/v2/endpoint/query) - **After completion:** [Fetch Content](/api-reference/v2/endpoint/fetch-content) - **After completion:** [Context Relations](/api-reference/v2/endpoint/source-relations) - - **Read more:** [Usage → Knowledge](/essentials/v2/knowledge) - - **Read more:** [Usage → Memories](/essentials/v2/memories) + - **Read more:** [Usage → Ingest](/essentials/v2/ingest) diff --git a/api-reference/v2/endpoint/sources-overview.mdx b/api-reference/v2/endpoint/sources-overview.mdx index 2adf6c29..b02db7b1 100644 --- a/api-reference/v2/endpoint/sources-overview.mdx +++ b/api-reference/v2/endpoint/sources-overview.mdx @@ -10,12 +10,13 @@ description: "Quick reference for context management endpoints, their lifecycle, | Send text and conversations as context items | `POST /context/ingest` with `context[]` | | Poll indexing progress | `GET /context/status` | | Browse stored items | `POST /context/list` | -| Inspect an item's title, attributes and original content | `GET /context/inspect` | +| Read an item's stored content | `GET /context/inspect` | | Delete items | `DELETE /context` | | Inspect graph relations | `GET /context/relations` | | Walk everything connected to one item | `GET /context/{id}/subgraph` | +| Update an item's metadata without re-ingesting | `PATCH /context/{id}/metadata` | -On a split database, `POST /context/ingest` also takes `documents` and `app_knowledge` under `type=knowledge`, and `memories` under `type=memory`. See [Split databases](#split-databases). +Ingest is text only: to add a file, extract its text and send it as an item. Content from connected apps arrives through [connectors](/essentials/v2/connectors) rather than this endpoint. ## Lifecycle @@ -48,7 +49,7 @@ flowchart LR ## Core concepts - **Items**: everything you ingest is a piece of context, a `text` or a `conversation`, with an optional `context_category` (`user_preference`, `business_knowledge`, `decision_trace`) that you set. One database holds all of them; collections partition them per user, team or project. See [Ingest context](/essentials/v2/ingest). -- **IDs**: each item has a `context_id`, yours or generated. The ingest response reports it as `results[].source_id`. Use it for polling status, inspecting content, deleting, and inspecting relations. +- **IDs**: each item has a `context_id`, yours or generated. The ingest response reports it as `results[].id`. Use it for polling status, inspecting content, deleting, and inspecting relations. - **Attributes**: `attributes` are the declared, filterable fields from `database_metadata_schema`; `custom_attributes` are free-form and stored with the item. Filter queries with `attributes`. See [Attributes](/essentials/v2/attributes). - **Enrichment**: on by default (`enrich: true`). HydraDB extracts entities, relations and preferences from each item into the [context graph](/essentials/v2/context-graphs); the extracted text comes back on query as `enrichment`, separate from the item's own `content`. - **Declared relations**: any item can name the items it relates to with `forceful_relations`, so they surface together at query time in `forceful_relations[]`. @@ -76,18 +77,6 @@ Paired with declared attributes, you get deterministic control over how results } ``` -## Split databases - -A database created with `type: "split"` keeps two corpora and the older ingest shapes: - -| Task | Endpoint | -| :-- | :-- | -| Upload PDFs, DOCX, CSVs, and other documents HydraDB should parse | `/context/ingest` with `type=knowledge` and `documents` | -| Upload Slack messages, Notion pages, Gmail threads, or webpages with pre-extracted text | `/context/ingest` with `type=knowledge` and `app_knowledge` | -| Upload user preferences, conversation history, or inline notes | `/context/ingest` with `type=memory` and `memories` | - -There, `type` picks the bucket (`knowledge` for shared documents, `memory` for per-user context) and routes the write to the right store; within `type=knowledge` you pick the payload shape (`documents` or `app_knowledge`, or both in one request). Both corpora can be searched together with `type: "all"` on `POST /query`. See [Knowledge](/essentials/v2/knowledge), [Memories](/essentials/v2/memories) and [Split databases and legacy fields](/essentials/v2/split-databases). - ## Related sections - [Ingest Context](/api-reference/v2/endpoint/ingest-context): the field reference diff --git a/api-reference/v2/endpoint/subgraph.mdx b/api-reference/v2/endpoint/subgraph.mdx index e8c9a8d9..c72b44ca 100644 --- a/api-reference/v2/endpoint/subgraph.mdx +++ b/api-reference/v2/endpoint/subgraph.mdx @@ -1,6 +1,5 @@ --- title: "Connected Subgraph" -openapi: "api-reference/v2/openapi.json GET /context/{id}/subgraph" description: "Everything connected to one item: its thread, its replies, its parents and children, and the items it links to." --- @@ -48,9 +47,8 @@ hydradb --output json subgraph slack_C0BE77_1788320073 | jq '.sources[].source_i | --- | --- | | | Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | | Collection scope. If omitted, the default collection is used. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). (default=`null`) | -| | Which graph the `id` belongs to. The two graphs are completely separate. (default=`"knowledge"`) | -| | Maximum traversal depth in hops. Range `1–10`. (default=`5`) | -| | Maximum number of members returned. Range `1–1000`. When this clips the traversal, `is_truncated` is `true`. (default=`200`) | +| | Maximum traversal depth in hops. Range `1` to `10`. (default=`5`) | +| | Maximum number of members returned. Range `1` to `1000`. When this clips the traversal, `is_truncated` is `true`. (default=`200`) | | | Principals to answer as (document ACLs). The subgraph then contains only items those principals may see, filtered at every hop. Repeated (`acl=a&acl=b`) or comma-separated. Omit for no ACL scoping. | ## How items connect @@ -62,6 +60,37 @@ Every member except the start item records how the traversal found it: Traversal is breadth-first, so `depth` on each member is its distance from the start item. The start item itself is a member at depth `0`, with neither field set. +## Response + +| Name | Description | +| --- | --- | +| | The item the traversal started from (the `id` you passed). | +| | Every member of the subgraph, in the order it was reached, start item first (fields below). | +| | Item-level relations among the members, in the triplet shape of [Inspecting Context Relations](/api-reference/v2/endpoint/source-relations). | +| | The structural graph around the members: entities mentioned, comments, attachments, and who authored what. Same shape as `relations`. | +| | `true` when `auxiliary_relations` was cut off by a size limit. | +| | `true` when the traversal stopped before reaching every connected item: `max_sources` or a size limit was hit, or `depth` left items unexpanded. | +| | Deepest level at which a member was added. | +| | Human-readable result message. | +| | Whether the request succeeded. | + +Each member in `sources` carries: + +| Name | Description | +| --- | --- | +| | The member's item ID. | +| | Title of the item. | +| | Connector item category, for connector items. | +| | Connector the item came from (for example `slack`), for connector items. | +| | Provider-assigned identifier, for connector items. | +| | Thread the item belongs to, when it has one. | +| | Hops from the start item (`0` for the start item). | +| | `resolved` (ingested), `stub` (a relation points at it but it is not ingested yet) or `placeholder`. | +| | The `source_id` of the member this one was first reached from. Absent on the start item. | +| | How it was reached: a declared relation type, `same_thread`, `parent` or `child`. Absent on the start item. | + +Fields a member does not have are omitted. + ```json Success @@ -169,9 +198,9 @@ Traversal is breadth-first, so `depth` on each member is its distance from the s ## Reading the response - **`sources[]`** are the members, the start item included at `depth: 0`. Every `source_id` is an id you can pass to [Fetch Content](/api-reference/v2/endpoint/fetch-content) for the full document, or back to this endpoint to re-centre the subgraph on it. `discovered_via` on each member is another member's `source_id`, so the list is also a tree. -- **`relations[]`** are the item-level relations *among the members* (`relates_to`, `same_thread`, `parent_of`), in the same triplet shape as [Inspecting Context Relations](/api-reference/v2/endpoint/source-relations). Their endpoints are `SOURCE` entities whose `entity_id` is the item's id. +- **`relations[]`** are the item-level relations *among the members* (declared `relates_to` links, plus `same_thread` and `child_of`), in the same triplet shape as [Inspecting Context Relations](/api-reference/v2/endpoint/source-relations). Their endpoints are `SOURCE` entities whose `entity_id` is the item's id. - **`auxiliary_relations[]`** is the structural graph around the members: which person sent a message, which entities are mentioned in it, which comments and attachments hang off it. These are recorded from the item itself, not extracted from text, so their `context` is empty. -- **Not included:** the chunk-level entity relations that Query returns as `graph_context`. Those are a different read. +- **Not included:** the chunk-level entity relations that [Query](/api-reference/v2/endpoint/query) returns as graph paths in `graph[]`. Those are a different read. ## Some additional notes @@ -181,7 +210,6 @@ Traversal is breadth-first, so `depth` on each member is its distance from the s - **An item nothing links to** comes back as a one-member subgraph: itself, at depth `0`, with `max_depth_reached: 0`. That is a real answer ("this stands alone"), distinct from an unknown id, which has no members. - **Bounding the traversal.** Threads and hierarchies can be large. `depth` bounds how far the walk goes; `max_sources` bounds how many members it returns. When `max_sources` clips it, `is_truncated` is `true` and the members you have are the ones closest to the start item. `auxiliary_truncated` reports the same for the structural graph. -- **Knowledge vs memory.** If the `id` belongs to a memory, set `type=memory`; the two graphs never connect to each other. - **Completeness.** An item's links populate once its `indexing_status` reaches `completed`. Items still in `graph_creation` may appear with fewer connections than they will have. - **Cost.** One request fans out into a bounded series of graph reads, so it is rate-limited like a Query, not like a status poll. @@ -192,6 +220,6 @@ Traversal is breadth-first, so `depth` on each member is its distance from the s - **Entity relations:** [Inspecting Context Relations](/api-reference/v2/endpoint/source-relations) - the triplets extracted from text - **Full content of a member:** [Fetch Content](/api-reference/v2/endpoint/fetch-content) - - **Query with graph context:** [Query](/api-reference/v2/endpoint/query) with `graph_context: true` + - **Query with graph paths:** [Query](/api-reference/v2/endpoint/query) returns graph paths in `graph[]`, controlled by the `graph_context` request flag - **Concepts:** [Concepts → Context Graphs](/essentials/v2/context-graphs) diff --git a/api-reference/v2/endpoint/tenant-stats.mdx b/api-reference/v2/endpoint/tenant-stats.mdx index 0543ecde..41ee5e00 100644 --- a/api-reference/v2/endpoint/tenant-stats.mdx +++ b/api-reference/v2/endpoint/tenant-stats.mdx @@ -2,17 +2,18 @@ title: "Database Stats" description: "Retrieve usage statistics for a database." -openapi: "api-reference/v2/openapi.json GET /databases/stats" --- -Get detailed metrics including storage growth and object counts. +import { Field } from "/snippets/field.jsx"; -The response splits stats into two database-wide collections: `data.knowledge_collection` (Knowledge) and `data.memory_collection` (User memories), each with its own row count. Counts aggregate across all collections in the database. +Get the indexed row count for a database. Counts aggregate across all collections in the database. + +The count is reported under two field names, `data.knowledge_collection` and `data.memory_collection`. Both are historical names for the database's one collection, so the two `row_count` values are always equal. Read either one. ```python Python SDK -stats = client.databases.stats(database="your database id") +stats = client.databases.stats(database="my_first_database") ``` ```typescript TypeScript SDK @@ -29,6 +30,22 @@ curl -X GET 'https://api.hydradb.com/databases/stats?database=my_first_database' +## Query parameters + +| Name | Description | +| --- | --- | +| | Database to report on. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | + +## Response + +| Name | Description | +| --- | --- | +| | The database the stats describe. | +| | Number of indexed chunks in the database. | +| | The same count as `knowledge_collection.row_count`, under a historical field name. | +| | Human-readable result message. | +| | Deprecated alias for `database`, carrying the same value. | + ```json Success @@ -40,7 +57,7 @@ curl -X GET 'https://api.hydradb.com/databases/stats?database=my_first_database' "row_count": 42318 }, "memory_collection": { - "row_count": 287 + "row_count": 42318 }, "message": "Successfully retrieved tenant collection statistics" }, @@ -89,10 +106,10 @@ curl -X GET 'https://api.hydradb.com/databases/stats?database=my_first_database' ## Behavior notes - **`row_count` is chunks, not sources.** One ingested document typically becomes many chunks (e.g., a 30-page PDF can produce 100\+ rows). To count distinct sources, use [List Documents](/api-reference/v2/endpoint/list-documents) with `page_size=1` and read `total`. + **`row_count` is chunks, not context items.** One ingested item typically becomes several chunks (a long text item can produce 100\+ rows). To count distinct items, use [List Context](/api-reference/v2/endpoint/list-documents) with `page_size=1` and read `total`. -- **Both collections always exist:** Even if you've only ingested knowledge (or only memories), both collections are provisioned. Empty collections report `row_count: 0`. +- **Empty databases report zero:** A database with nothing ingested yet reports `row_count: 0` under both field names. - **Stats are eventually consistent:** Immediately after ingestion or deletion, counts may lag until background processing completes. Check [Ingestion Status](/api-reference/v2/endpoint/source-status) for recently ingested IDs before treating counts as final.
@@ -100,7 +117,7 @@ curl -X GET 'https://api.hydradb.com/databases/stats?database=my_first_database' **Related Resources** - - **List sources/memories:** [List Documents](/api-reference/v2/endpoint/list-documents) + - **List context items:** [List Context](/api-reference/v2/endpoint/list-documents) - **List collections:** [List Collections](/api-reference/v2/endpoint/list-sub-tenants) - **Check provisioning:** [Database Status](/api-reference/v2/endpoint/tenant-status) - **Read more:** [Concepts → Multi-Tenant Support](/essentials/v2/databases-and-collections) diff --git a/api-reference/v2/endpoint/tenant-status.mdx b/api-reference/v2/endpoint/tenant-status.mdx index ce1eb16c..f8ea9f52 100644 --- a/api-reference/v2/endpoint/tenant-status.mdx +++ b/api-reference/v2/endpoint/tenant-status.mdx @@ -1,18 +1,13 @@ --- title: "Database Status" description: "Check the readiness of a database's infrastructure." -openapi: "api-reference/v2/openapi.json GET /databases/status" --- -Database creation is asynchronous, check if your database is ready before executing ingestion or any queries. +import { Field } from "/snippets/field.jsx"; -The response includes `infra.ready_for_ingestion`, a convenience flag derived from the vectorstore fields below. For a single check, read that field. The underlying signals are: +Database creation is asynchronous, check if your database is ready before executing ingestion or any queries. -- `infra.ready_for_ingestion === true` (derived - true once both vectorstores are provisioned) -- `infra.scheduler_status === true` (the background indexing scheduler) -- `infra.graph_status === true` (the graph layer) -- `infra.vectorstore_status.knowledge === true` -- `infra.vectorstore_status.memories === true` +Poll this endpoint until `data.infra.ready_for_ingestion` is `true`. That one flag is the readiness signal: the server derives it from the individual infrastructure flags below, so read it rather than combining them yourself. @@ -41,6 +36,25 @@ curl -X GET 'https://api.hydradb.com/databases/status?database=my_first_database +## Query parameters + +| Name | Description | +| --- | --- | +| | Database to check. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | + +## Response + +| Name | Description | +| --- | --- | +| | The database the status describes. | +| | Organization that owns the database. | +| | **The flag to poll.** `true` once the database is fully provisioned and ready to accept ingestion and serve queries. | +| | `true` once database setup has finished. Stays `false` while the database is still being created. | +| | `true` when the graph layer is healthy for this database. | +| | Vector store readiness, reported under two historical field names, `knowledge` and `memories`. Both are `true` once the database is ready. | +| | Human-readable result message. | +| | Deprecated alias for `database`, carrying the same value. | + ```json Success @@ -88,7 +102,7 @@ curl -X GET 'https://api.hydradb.com/databases/status?database=my_first_database **Stale database IDs:** If `database` does not exist, the call returns `404 DATABASE_NOT_FOUND`. Always verify that the database was created successfully before polling. - **Common mistake:** `row_count` from [Database Stats](/api-reference/v2/endpoint/tenant-stats) counts individual chunks, not documents. For distinct source or memory counts, use [List Documents](/api-reference/v2/endpoint/list-documents) and read `pagination.total` from the response. + **Common mistake:** `row_count` from [Database Stats](/api-reference/v2/endpoint/tenant-stats) counts individual chunks, not context items. For a distinct item count, use [List Context](/api-reference/v2/endpoint/list-documents) and read `pagination.total` from the response.
diff --git a/api-reference/v2/endpoint/tenants-overview.mdx b/api-reference/v2/endpoint/tenants-overview.mdx index 81d9d111..18bef5e6 100644 --- a/api-reference/v2/endpoint/tenants-overview.mdx +++ b/api-reference/v2/endpoint/tenants-overview.mdx @@ -22,7 +22,7 @@ Databases are physically isolated spaces for storing context. In most integratio For a new database from scratch: 1. Create the database: `POST /databases` -2. Wait for provisioning: `GET /databases/status` until `scheduler_status`, `graph_status`, `vectorstore_status.knowledge`, and `vectorstore_status.memories` are all `true` +2. Wait for provisioning: `GET /databases/status` until `data.infra.ready_for_ingestion` is `true` 3. Ingest content: `POST /context/ingest` 4. Wait for indexing: `GET /context/status` until sources are searchable 5. Retrieve context: `POST /query` diff --git a/api-reference/v2/error-responses.mdx b/api-reference/v2/error-responses.mdx index 77e557d6..72fba9bd 100644 --- a/api-reference/v2/error-responses.mdx +++ b/api-reference/v2/error-responses.mdx @@ -44,8 +44,8 @@ Use `error.code` for branching and log `meta.request_id` for every failed reques | `400` | Invalid parameters or malformed request | No | | `401` | Missing, expired, or invalid API key | No | | `403` | Authenticated, but not permitted for the resource | No | -| `404` | Database, source, memory, or related resource was not found | No | -| `409` | Conflict, usually an existing database or context item ID | Usually no | +| `404` | Database, context item, or related resource was not found | No | +| `409` | Conflict, usually an existing database, or a strict-mode delete of an item that is still indexing | Usually no | | `422` | Well-formed request that failed validation | No | | `429` | Rate limit exceeded | Yes, with backoff | | `500` | Internal server error | Yes, with backoff | @@ -55,13 +55,15 @@ Use `error.code` for branching and log `meta.request_id` for every failed reques | Code | Typical status | Meaning | |---|---|---| -| `INVALID_PARAMETERS` | `400` | A required parameter is missing, malformed, or mutually incompatible with another parameter. | +| `INVALID_INPUT` | `400` | A required parameter is missing, malformed, or mutually incompatible with another parameter. | | `UNAUTHORIZED` | `401` | The `Authorization: Bearer ` header is missing or invalid. | | `FORBIDDEN` | `403` | The API key is valid but does not have access to the requested resource, or the account/plan limit prevents the operation. | | `DATABASE_ALREADY_EXISTS` | `409` | `POST /databases` received a `database` (formerly `tenant_id`) that is already in use. | | `DATABASE_NOT_FOUND` | `404` | The requested database does not exist or is not visible to the current API key. | -| `SOURCE_NOT_FOUND` | `404` | The requested source or memory ID does not exist in the selected database/collection. | +| `NOT_FOUND` | `404` | The requested context id does not exist in the selected database/collection. | +| `SOURCE_PROCESSING` | `409` | A strict-mode delete (`X-HydraDB-Delete-Status: strict`) named an item that is still indexing. Retry after ingestion completes; see the `Retry-After` header. | | `VALIDATION_ERROR` | `422` | The request shape was valid JSON/form data, but one or more fields failed semantic validation. | +| `TENANT_INFRA_NOT_READY` | `422` | The database exists but its infrastructure is still provisioning. Poll [`GET /databases/status`](/api-reference/v2/endpoint/tenant-status) until `data.infra.ready_for_ingestion` is `true`. | | `RATE_LIMITED` | `429` | The API key exceeded its current rate limit. | | `INTERNAL_ERROR` | `500` | HydraDB hit an unexpected server-side error. | | `SERVICE_UNAVAILABLE` | `503` | A dependency is temporarily unavailable or the service is under load. | @@ -82,13 +84,8 @@ Many storage- and capacity-related ingestion errors are **transient**: the pipel | Code | Meaning | Severity | |---|---|---| -| `E1002` | The file format is not supported. Returned in the [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context) response itself, per file, before anything is queued. User message: *"This file format isn't supported. Please upload a PDF, Office document (Word, Excel, PowerPoint), image, CSV or text file."* | **Terminal** (not retried) | | `E6001` | Vector-store storage/indexing error while persisting processed data. The pipeline retries automatically and it usually clears within minutes. User message: *"Failed to store the processed data. Please try again. If the issue persists, contact support@hydradb.com."* | **Transient** (retryable) | - -`E1002` is the one ingestion code that does **not** follow the polling advice above. The file is rejected at upload, so it never enters the pipeline and never gets a status record. Polling [`/context/status`](/api-reference/v2/endpoint/source-status) for it returns `FILE_NOT_FOUND`, not `E1002`. Read `error_code` on each item in the upload response instead. See [Supported file formats](/api-reference/v2/endpoint/ingest-context#supported-file-formats) for what is accepted, and note that one rejected file does not affect the other files in the same request. - - `E6001` is **transient**, not terminal. If you observe it on an in-flight item, keep polling [`/context/status`](/api-reference/v2/endpoint/source-status) - the item normally advances to `graph_creation` / `completed` on a subsequent retry with no action on your part. Only contact support if the item is still reported as `errored` after retries are exhausted. @@ -125,9 +122,8 @@ async function withRetry( const result = await withRetry(() => client.query({ - database: "my_first_database", - query: "What are the pricing tiers?", - type: "knowledge", + database: "my_first_database", + query: "What are the pricing tiers?", }) ); ``` @@ -152,7 +148,6 @@ def with_retry(operation, max_retries=3): result = with_retry(lambda: client.query( database="my_first_database", query="What are the pricing tiers?", - type="knowledge", )) ``` ```typescript fetch @@ -182,14 +177,17 @@ import { HydraDBError } from "@hydradb/sdk"; try { await client.context.ingest({ - type: "knowledge", database: "my_first_database", - documents: [{ path: "policy.pdf", filename: "policy.pdf" }], + collection: "support", + items: JSON.stringify([ + { context_id: "refund-policy", title: "Refund policy", text: "Refunds are processed within 5 business days." }, + ]), }); } catch (error) { if (error instanceof HydraDBError) { - const code = error.body?.error?.code; - const requestId = error.body?.meta?.request_id; + const body = error.body as { error?: { code?: string }; meta?: { request_id?: string } } | undefined; + const code = body?.error?.code; + const requestId = body?.meta?.request_id; if (code === "DATABASE_NOT_FOUND") { // Create or select a valid database before ingesting. @@ -205,13 +203,17 @@ try { } ``` ```python Python SDK +import json + from hydra_db.core.api_error import ApiError try: client.context.ingest( - type="knowledge", database="my_first_database", - documents=[("policy.pdf", open("policy.pdf", "rb"), "application/pdf")], + collection="support", + items=json.dumps([ + {"context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days."}, + ]), ) except ApiError as exc: # The machine-readable code lives in the response envelope, not on the @@ -243,25 +245,28 @@ Also send `API-Version: 2` on raw HTTP requests. The official SDKs set the versi ### Database not found after creation -Database creation is asynchronous. After `POST /databases`, poll [`GET /databases/status`](/api-reference/v2/endpoint/tenant-status) until `infra.scheduler_status`, `infra.graph_status`, `infra.vectorstore_status.knowledge`, and `infra.vectorstore_status.memories` are all `true`. +Database creation is asynchronous. After `POST /databases`, poll [`GET /databases/status`](/api-reference/v2/endpoint/tenant-status) until `data.infra.ready_for_ingestion` is `true`. ### Ingestion validation errors -Common causes: +`POST /context/ingest` validates every item before queuing any of them, and the error message names the failing item as `context[N]`. Common causes: -- `document_metadata` length does not match the `documents` array length. -- `app_knowledge`, `memories`, or `document_metadata` was sent as an object instead of a JSON-stringified multipart field. -- A memory item has neither `text` nor `user_assistant_pairs`. -- A typed `tenant_metadata` value does not match the database metadata schema. +- An item has neither `text` nor `conversation`, or has both. +- A conversation turn has a role other than `user`, `assistant` or `system`, or empty `content`; or the conversation has only `system` turns. +- `context_category` is not one of `auto`, `user_preference`, `business_knowledge` or `decision_trace`. +- `happened_at` is not a `YYYY-MM-DD` date. +- A `context_id` contains a comma. +- A `graph_payload` key matches no `context_id` in the same request. +- An `acl` entry is not a valid principal. +- The request exceeds the limits: 100 items, 1 MiB of text per item, 8 MiB of text per request. ### Empty query results Empty results are not always errors. Check these first: - Context status may still be `queued` or `processing`; poll [`GET /context/status`](/api-reference/v2/endpoint/source-status). -- `metadata_filters` may be too restrictive or may target the wrong metadata namespace. -- The query may be scoped to the wrong `database` or `collection` (formerly `tenant_id` / `sub_tenant_id`). -- The `type` value may exclude the collection you need. Use `type: "all"` when combining knowledge and memories in `POST /query`. +- An `attributes` filter may be too restrictive, or the items may not carry the attribute values you filter on. See [Attributes](/essentials/v2/attributes). +- The query may be scoped to the wrong `database` or `collection` (formerly `tenant_id` / `sub_tenant_id`). To search several collections at once, send `collections`. ## Related sections diff --git a/api-reference/v2/index.mdx b/api-reference/v2/index.mdx index d09775af..d5c93e21 100644 --- a/api-reference/v2/index.mdx +++ b/api-reference/v2/index.mdx @@ -27,7 +27,6 @@ description: "Single reference to all HydraDB endpoints" | `database` | Your isolated workspace for data, metadata schema, and query. | Send it on every API call so HydraDB knows which workspace to read or write. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | `collection` | Optional partition inside a database, often a user, team, account, or customer. | Use it when one database contains data for multiple users or customers. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). Read more about our [multi-tenant architecture](/essentials/v2/databases-and-collections) | | [Context items](/essentials/v2/ingest) | A `text` or a `conversation`, sent in the `context` list of `POST /context/ingest`, optionally labelled with a `context_category`. | Everything you ingest. Shared context goes in a shared collection; a person's preferences go in their own. | -| [Split databases](/essentials/v2/split-databases) | Databases created with `type: "split"` keep two corpora, knowledge and memory, selected with `type`. | Only for integrations that still use the older `documents`, `app_knowledge` and `memories` shapes. | | `database_metadata_schema` | Database-level fields you define up front so metadata can be filtered or queried consistently. | Use it for stable fields like department, customer, region, plan, category, or compliance label. | | `attributes` | Declared, filterable fields on an item, matching `database_metadata_schema`; `custom_attributes` are free-form. | Send them at ingest; filter with `attributes` on `/query`. | | `ids` | IDs returned by ingestion or visible from `/context/list`. | Use them when polling processing status, inspecting content, listing a specific subset, deleting sources, or inspecting relations. | @@ -94,21 +93,25 @@ SDK methods mirror the API: `client..()` maps to the correspondin | Endpoint | Method | SDK method | Purpose | Use when | |---|---|---|---|---| | [`/databases`](/api-reference/v2/endpoint/create-tenant) | `POST` | `databases.create` | Create a database | You are setting up a new isolated workspace and optional metadata schema. | -| [`/databases/{database}/metadata-schema`](/api-reference/v2/endpoint/update-metadata-schema) | `PATCH` | REST | Add metadata schema fields | You need to add filterable metadata fields after database creation. | +| [`/databases/{database}/metadata-schema`](/api-reference/v2/endpoint/update-metadata-schema) | `PATCH` | `databases.update_metadata_schema` | Add metadata schema fields | You need to add filterable metadata fields after database creation. | | [`/databases`](/api-reference/v2/endpoint/list-tenants) | `GET` | `databases.list` | List databases | You need to discover database IDs available to the current API key. | | [`/databases`](/api-reference/v2/endpoint/delete-tenant) | `DELETE` | `databases.delete` | Delete a database | You need to permanently remove a workspace and its data. | | [`/databases/status`](/api-reference/v2/endpoint/tenant-status) | `GET` | `databases.status` | Check provisioning readiness | You just created a database and need to wait before ingesting data. | -| [`/databases/collections`](/api-reference/v2/endpoint/list-sub-tenants) | `GET` | `databases.collections` / `databases.collections` | List active collections | You partition data by user, team, customer, or account and need to inspect those partitions. | +| [`/databases/collections`](/api-reference/v2/endpoint/list-sub-tenants) | `GET` | `databases.collections` | List active collections | You partition data by user, team, customer, or account and need to inspect those partitions. | +| [`/databases/collections`](/api-reference/v2/endpoint/delete-collection) | `DELETE` | `databases.delete_collection` | Delete a collection | You need to permanently remove one collection and its data. | | [`/databases/stats`](/api-reference/v2/endpoint/tenant-stats) | `GET` | `databases.stats` | Get usage statistics | You want to monitor object counts for a database. | | [`/context/ingest`](/api-reference/v2/endpoint/ingest-context) | `POST` | `context.ingest` | Ingest context items | You are sending text or conversations. | | [`/context/status`](/api-reference/v2/endpoint/source-status) | `GET` | `context.status` | Check processing status | You have IDs from ingestion and need to know when they are queryable. | -| [`/context/inspect`](/api-reference/v2/endpoint/fetch-content) | `GET` | `context.inspect` | Inspect an item's title, attributes and original content | Query chunks carry no source details; look them up here by `context_id`. | +| [`/context/inspect`](/api-reference/v2/endpoint/fetch-content) | `GET` | `context.inspect` | Read an item's stored content | Query chunks carry no source details; look up an item's title and attributes with `POST /context/list` and `ids`. | | [`/context/list`](/api-reference/v2/endpoint/list-documents) | `POST` | `context.list` | Browse items | You need pagination, filters, field projection, or a specific subset by `ids`. | -| [`/context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) | `PATCH` | REST | Update source metadata | You need to merge `metadata` or `additional_metadata` onto one existing source without re-ingesting. | +| [`/context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) | `PATCH` | `context.update_source_metadata` | Update an item's metadata | You need to merge `database_metadata` or `additional_metadata` onto one existing item without re-ingesting. | | [`/context`](/api-reference/v2/endpoint/delete-source) | `DELETE` | `context.delete` | Delete items | You need to remove one or more items by ID. | -| [`/context/relations`](/api-reference/v2/endpoint/source-relations) | `GET` | `context.relations` | Inspect entity relationships | You need graph relations for a source or collection. | +| [`/context/relations`](/api-reference/v2/endpoint/source-relations) | `GET` | `context.relations` | Inspect entity relationships | You need graph relations for an item or collection. | +| [`/context/{id}/subgraph`](/api-reference/v2/endpoint/subgraph) | `GET` | `context.subgraph` | Walk everything connected to one item | You need an item's thread, replies, parents, children and linked items. | | [`/query`](/api-reference/v2/endpoint/query) | `POST` | `query` | Retrieve context | You need ranked chunks, graph paths, declared relations and a prompt-ready `llm_prompt`, with `hybrid` or `text` matching across one or more collections. | +SDK method names are the Python names; TypeScript camelCases the multi-word ones (`update_metadata_schema` is `updateMetadataSchema`). Connector, webhook and feedback endpoints have their own pages: [Connectors](/api-reference/v2/endpoint/connectors-overview), [Register Webhook](/api-reference/v2/endpoint/register-webhook) and [Submit Feedback](/api-reference/v2/endpoint/submit-feedback). + ## Conventions **Authentication:** Every endpoint requires `Authorization: Bearer ` in the request header. Get your key at [app.hydradb.com](https://app.hydradb.com). @@ -153,9 +156,9 @@ Errors use the same envelope with `success: false`, `data: null`, and an `error` - **Pagination.** Listing endpoints (`/context/list`) return pagination fields for browsing large result sets. -- **Parameter casing.** The REST API uses snake_case (`database`). The TypeScript SDK accepts the same snake_case keys; method names are camelCase when generated for TypeScript. The Python SDK uses snake_case throughout. +- **Parameter casing.** The REST API uses snake_case (`max_results`). The Python SDK uses snake_case throughout; the TypeScript SDK uses camelCase for method names, parameters and response fields (`maxResults`, `llmPrompt`). Keys inside the JSON `items` string sent to `context.ingest` stay snake_case in every language. -- **Query modes.** `POST /query` supports `query_by: "hybrid"` or `"text"` and `mode: "auto"`, `"fast"` or `"thinking"`. There is no `type` on a unified database; a database created with `type: "split"` takes `type: "knowledge"`, `"memory"` or `"all"` on context and query calls. +- **Query modes.** `POST /query` supports `query_by: "hybrid"` or `"text"` and `mode: "auto"`, `"fast"` or `"thinking"`. **Status codes:** Successful responses return `200` (or `202` for async accepts). Errors follow standard HTTP semantics: @@ -181,8 +184,6 @@ Rate limits apply per API key. For production deployments, build retry logic wit ## Next steps -Existing v1 endpoints remain available under the v1 API Reference. - - **Build something:** [Quickstart](/get-started/v2/quickstart) walks through your first integration in five minutes - **Understand the model:** [Core Concepts](/get-started/v2/core-concepts) explains databases, items, categories, query and attributes - **Go deeper:** [Usage](/essentials/v2/query) covers each primitive in depth diff --git a/api-reference/v2/sdks.mdx b/api-reference/v2/sdks.mdx index a6c8f4ee..b609fc25 100644 --- a/api-reference/v2/sdks.mdx +++ b/api-reference/v2/sdks.mdx @@ -1,20 +1,18 @@ --- -title: "SDKs – Python and Node" +title: "SDKs for Python and Node" description: "Official Python and TypeScript/Node.js SDKs for the HydraDB API." --- -The SDKs wrap every endpoint in the [API Reference](/api-reference/v2) with typed methods and IDE autocomplete. They automatically set the `API-Version: 2` header on every request - you do not need to send it manually. +The SDKs wrap every endpoint in the [API Reference](/api-reference/v2) with typed methods and IDE autocomplete. They set the `API-Version: 2` header on every request, so you never send it yourself. ## Installation -The SDKs ship as major-version bumps of the same packages used for v1. Pin to `>=2.0.0` to use the latest method surface. - ```bash Python SDK -pip install "hydradb-sdk>=2,<3" +pip install "hydradb-sdk>=2.1.5,<3" ``` ```bash TypeScript / Node.js -npm install @hydradb/sdk@^2 +npm install @hydradb/sdk@^2.1.5 ``` @@ -43,51 +41,49 @@ const client = new HydraDBClient({ -**Python:** Both synchronous (`HydraDB`) and asynchronous (`AsyncHydraDB`) clients are available. They share an identical surface - choose based on your application's concurrency model. +**Python:** Both synchronous (`HydraDB`) and asynchronous (`AsyncHydraDB`) clients are available. They share an identical surface; choose based on your application's concurrency model. ## Versioning -The SDKs automatically include `API-Version: 2` on every outbound request. Server-side routing resolves to the matching routes; the response includes an `X-API-Version: 2` header echoing the resolved version. - -You can verify which version your client is using by inspecting the response headers in any SDK call. +The SDKs include `API-Version: 2` on every outbound request. The response carries an `X-API-Version: 2` header echoing the resolved version, so you can confirm which version a call used by inspecting the response headers. ## Naming conventions -The REST API uses **snake_case** for all request and response fields, and both SDKs preserve those field names for request/response objects. TypeScript only camelCases multi-word **method names**. +The REST API uses **snake_case** for every request and response field. The Python SDK keeps snake_case; the TypeScript SDK uses camelCase for method names, request parameters and response fields. -| | Method naming | Parameter field naming | Example | +| | Method names | Request and response fields | Example | |---|---|---|---| -| **Raw HTTP / cURL** | - | snake_case | `database`, `collection`, `query_by` | -| **Python SDK** | snake_case | snake_case | method: `query()`, params: `database`, `query_by` | -| **TypeScript SDK** | camelCase for multi-word methods | snake_case | method: `query()`, params: `database`, `query_by` | +| **Raw HTTP / cURL** | n/a | snake_case | `max_results`, `llm_prompt` | +| **Python SDK** | snake_case | snake_case | `client.query(max_results=8)`, `result.data.llm_prompt` | +| **TypeScript SDK** | camelCase | camelCase | `client.query({ maxResults: 8 })`, `result.data.llmPrompt` | -**Use snake_case request fields in TypeScript examples too** (e.g., `client.context.list({ database: "...", page_size: 50 })`). The SDK sends the same names on the wire, matching the OpenAPI schema. - -The Python SDK also uses snake_case throughout (e.g., `client.context.list(database="...", page_size=50)`). +**Item keys stay snake_case in every language.** `client.context.ingest` takes the item list as a JSON string in the `items` field, so the keys inside each item (`context_id`, `context_category`, `happened_at`, `custom_attributes`) are the wire names in TypeScript too. -`database` and `collection` are the current field names (formerly `tenant_id` and `sub_tenant_id`). The old names remain accepted as deprecated aliases for full backward compatibility. +`database` and `collection` are the current field names (formerly `tenant_id` and `sub_tenant_id`). The old names remain accepted as deprecated aliases. -Your IDE's autocomplete and type checking work directly off the API contract - if a field is optional in the API, it's optional in the SDK. +Your IDE's autocomplete and type checking work directly off the API contract: if a field is optional in the API, it's optional in the SDK. ## SDK method structure -SDK methods are grouped under three top-level namespaces - one per `/api-reference/v2` group: +SDK methods are grouped by the endpoint they call: | URL prefix | SDK group | Purpose | |---|---|---| -| `/context/*` and `/context` | `client.context` | Ingest, status, fetch, list, delete, graph relations | -| `/query` | `client.query` | Unified retrieval over knowledge, memories, or both | -| `/databases/*` and `/databases` | `client.databases` | Create, list, delete, status, collections, stats. Legacy `/tenants/*` paths remain as deprecated aliases. | - +| `/context/*` and `/context` | `client.context` | Ingest, status, inspect, list, delete, relations, subgraph, metadata updates | +| `/query` | `client.query` | Retrieval: ranked chunks, graph paths, forceful relations and a prompt-ready `llm_prompt` | +| `/databases/*` and `/databases` | `client.databases` | Create, list, delete, status, collections, stats, metadata schema | +| `/connectors/*` | `client.connectors` | Connect and sync external apps. See [Connectors](/api-reference/v2/endpoint/connectors-overview). | +| `/webhooks/indexing*` | `client.webhooks` | Indexing webhooks. See [Register Webhook](/api-reference/v2/endpoint/register-webhook). | +| `/feedback` | `client.feedback` | Report how a query performed. See [Submit Feedback](/api-reference/v2/endpoint/submit-feedback). | ## Method reference ### Context -`client.context.*` covers every flow around content lifecycle - document uploads, app sources, memories, polling, fetching, listing, deletion, and graph inspection. +`client.context.*` covers the lifecycle of context items: ingesting, polling, inspecting, listing, deleting and exploring the graph. | Method | Endpoint | |---|---| @@ -97,10 +93,12 @@ SDK methods are grouped under three top-level namespaces - one per `/api-refer | `client.context.list()` | `POST /context/list` | | `client.context.delete()` | `DELETE /context` | | `client.context.relations()` | `GET /context/relations` | +| `client.context.subgraph()` | `GET /context/{id}/subgraph` | +| `client.context.update_source_metadata()` | `PATCH /context/{id}/metadata` | ### Query -A single method covers all retrieval. Pick `type` (`"knowledge"`, `"memory"`, `"all"`) and `query_by` (`"hybrid"`, `"text"`) to control behavior. +One method covers all retrieval. `query_by` (`"hybrid"` or `"text"`) picks how text is matched, and `mode` (`"auto"`, `"fast"` or `"thinking"`) picks how much work the search does. | Method | Endpoint | |---|---| @@ -115,41 +113,17 @@ A single method covers all retrieval. Pick `type` (`"knowledge"`, `"memory"`, `" | `client.databases.delete()` | `DELETE /databases` | | `client.databases.status()` | `GET /databases/status` | | `client.databases.collections()` | `GET /databases/collections` | +| `client.databases.delete_collection()` | `DELETE /databases/collections` | | `client.databases.stats()` | `GET /databases/stats` | +| `client.databases.update_metadata_schema()` | `PATCH /databases/{database}/metadata-schema` | -The method-reference tables above use the Python (snake_case) method names. TypeScript keeps the same method names but camelCases any that are multi-word (for example, the connector method `list_resources()` in Python is `listResources()` in TypeScript). Request parameters stay snake_case in both SDKs. - -## Migrating from v1 - -The v2 SDKs are a deliberate consolidation. A few common v1 → v2 method swaps: - -| v1 | v2 | -|---|---| -| `client.upload.knowledge(...)` | `client.context.ingest(type="knowledge", ...)` | -| `client.upload.addMemory(...)` | `client.context.ingest(type="memory", ...)` | -| `client.upload.verifyProcessing(...)` | `client.context.status(...)` | -| `client.search.fullRecall(...)` | `client.query(type="knowledge", ...)` | -| `client.search.recallPreferences(...)` | `client.query(type="memory", ...)` | -| `client.search.booleanRecall(...)` | `client.query(query_by="text", ...)` | -| `client.fetch.listData(...)` | `client.context.list(...)` | -| `client.fetch.graphRelationsBySourceId(...)` | `client.context.relations(...)` | -| `client.fetch.content(...)` | `client.context.inspect(...)` | -| `client.data.delete(...)` | `client.context.delete(type="knowledge", ...)` | -| `client.upload.deleteMemory(...)` | `client.context.delete(type="memory", ...)` | -| `client.tenant.create(...)` | `client.databases.create(...)` | -| `client.tenant.getTenantIds(...)` | `client.databases.list(...)` | -| `client.tenant.deleteTenant(...)` | `client.databases.delete(...)` | -| `client.tenant.getInfraStatus(...)` | `client.databases.status(...)` | -| `client.tenant.getSubTenantIds(...)` | `client.databases.collections(...)` | -| `client.tenant.monitor(...)` | `client.databases.stats(...)` | - -The v1 SDK methods remain available on the `<2.0.0` releases of `hydradb-sdk` / `@hydradb/sdk` for as long as v1 routes are supported. +The tables use the Python method names. TypeScript camelCases the multi-word ones: `update_source_metadata()` is `updateSourceMetadata()`, `delete_collection()` is `deleteCollection()` and `update_metadata_schema()` is `updateMetadataSchema()`. ## Getting started ### Create a database -A database is an isolated workspace. Most organizations create one database total, with collections for users or teams. See [Multi-Tenant](/essentials/v2/databases-and-collections) for the full model. +A database is an isolated workspace. Most organizations create one database, with collections for users, teams or projects. See [Databases and collections](/essentials/v2/databases-and-collections) for the full model. ```python Python SDK @@ -180,295 +154,310 @@ const response = await client.databases.create({ ``` -Database creation is asynchronous. Poll [`databases.status`](/api-reference/v2/endpoint/tenant-status) until provisioning completes: +`database_metadata_schema` declares the fields you can later filter on with `attributes`. See [Attributes](/essentials/v2/attributes). + +Database creation is asynchronous. Poll [`databases.status`](/api-reference/v2/endpoint/tenant-status) until `ready_for_ingestion` is `true`: ```python Python SDK import time while True: - status = client.databases.status(database="my_first_database") - infra = status.data.infra - if ( - infra.scheduler_status - and infra.graph_status - and infra.vectorstore_status.knowledge - and infra.vectorstore_status.memories - ): + infra = client.databases.status(database="my_first_database").data.infra + if infra.ready_for_ingestion: break - time.sleep(2) + time.sleep(5) ``` ```typescript TypeScript SDK while (true) { - const status = await client.databases.status({ - database: "my_first_database", - }); - const { schedulerStatus, graphStatus, vectorstoreStatus } = status.data.infra; - if ( - schedulerStatus && - graphStatus && - vectorstoreStatus.knowledge && - vectorstoreStatus.memories - ) break; - await new Promise((r) => setTimeout(r, 2000)); + const { data } = await client.databases.status({ database: "my_first_database" }); + if (data?.infra?.readyForIngestion) break; + await new Promise((resolve) => setTimeout(resolve, 5_000)); } ``` -### Ingest knowledge +### Ingest context -Upload documents, app sources, or both in one call: +Everything you ingest is a context item: one `text` or one `conversation`, with optional fields such as `context_id`, `title`, `context_category`, `happened_at` and `attributes`. The SDK sends a multipart form and puts the item list, as a JSON string, in the `items` field. ```python Python SDK import json -with open("contract.pdf", "rb") as f1, open("policy.pdf", "rb") as f2: - result = client.context.ingest( - type="knowledge", - database="my_first_database", - documents=[ - ("contract.pdf", f1, "application/pdf"), - ("policy.pdf", f2, "application/pdf"), - ], - document_metadata=json.dumps([ - { - "id": "contract_q4", - "metadata": {"department": "legal"}, - "additional_metadata": {"author": "Alice"}, - }, - { - "id": "policy_2025", - "metadata": {"department": "legal"}, - "additional_metadata": {"author": "Bob"}, - }, - ]), - ) +result = client.context.ingest( + database="my_first_database", + collection="support", + items=json.dumps([ + { + "context_id": "refund-policy", + "title": "Refund policy", + "text": "Refunds are processed within 5 business days.", + "context_category": "business_knowledge", + "attributes": {"department": "support"}, + }, + { + "context_id": "chat-alex-001", + "conversation": [ + {"role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex"}, + {"role": "assistant", "content": "Got it, short answers."}, + ], + "context_category": "user_preference", + "happened_at": "2026-09-01", + }, + ]), +) + +print([r.id for r in result.data.results]) ``` ```typescript TypeScript SDK const result = await client.context.ingest({ - type: "knowledge", database: "my_first_database", - documents: [ - { path: "contract.pdf", filename: "contract.pdf", contentType: "application/pdf" }, - { path: "policy.pdf", filename: "policy.pdf", contentType: "application/pdf" }, - ], - documentMetadata: JSON.stringify([ + collection: "support", + items: JSON.stringify([ { - id: "contract_q4", - metadata: { department: "legal" }, - additional_metadata: { author: "Alice" }, + context_id: "refund-policy", + title: "Refund policy", + text: "Refunds are processed within 5 business days.", + context_category: "business_knowledge", + attributes: { department: "support" }, }, { - id: "policy_2025", - metadata: { department: "legal" }, - additional_metadata: { author: "Bob" }, + context_id: "chat-alex-001", + conversation: [ + { role: "user", content: "Keep answers short, I read on my phone.", name: "alex" }, + { role: "assistant", content: "Got it, short answers." }, + ], + context_category: "user_preference", + happened_at: "2026-09-01", }, ]), }); + +console.log(result.data?.results?.map((r) => r.id)); ``` -For app sources (Slack, Notion, Gmail, webpages), pass `app_knowledge` instead of `documents`. You can combine both in a single request. +The response is `202 Accepted` with one result per item; `results[].id` is the item's `context_id`. Up to 100 items fit in one call. Every item field, default and limit is on [Ingest Context](/api-reference/v2/endpoint/ingest-context). -### Add user memories +### Verify processing -Use the same `ingest` method with `type: "memory"`: +Ingestion is asynchronous. Poll `context.status` with the context ids until each `indexing_status` is `completed` (`graph_creation` is already searchable) or `errored`: ```python Python SDK -import json +import time -result = client.context.ingest( - type="memory", - database="my_first_database", - collection="user_alex", - memories=json.dumps([ - { - "text": "User prefers detailed technical explanations and dark mode", - "infer": True, - "user_name": "Alex", - "metadata": {"team": "engineering"}, - "additional_metadata": {"source": "onboarding"}, - } - ]), -) +ids = ["refund-policy", "chat-alex-001"] +while True: + statuses = client.context.status( + database="my_first_database", + collection="support", + ids=ids, + ).data.statuses + failed = [s for s in statuses if s.indexing_status == "errored"] + if failed: + raise RuntimeError([s.error_message for s in failed]) + if all(s.indexing_status in ("graph_creation", "completed") for s in statuses): + break + time.sleep(2) ``` ```typescript TypeScript SDK -const result = await client.context.ingest({ - type: "memory", - database: "my_first_database", - collection: "user_alex", - memories: JSON.stringify([ - { - text: "User prefers detailed technical explanations and dark mode", - infer: true, - user_name: "Alex", - metadata: { team: "engineering" }, - additional_metadata: { source: "onboarding" }, - }, - ]), -}); +const ids = ["refund-policy", "chat-alex-001"]; +while (true) { + const { data } = await client.context.status({ + database: "my_first_database", + collection: "support", + ids, + }); + const statuses = data?.statuses ?? []; + const failed = statuses.filter((s) => s.indexingStatus === "errored"); + if (failed.length > 0) throw new Error(failed.map((s) => s.errorMessage).join("; ")); + if (statuses.every((s) => s.indexingStatus === "graph_creation" || s.indexingStatus === "completed")) break; + await new Promise((resolve) => setTimeout(resolve, 2_000)); +} ``` - -**`infer` defaults to `false`.** Set `infer: true` for conversational content where you want HydraDB to extract implicit preferences. Use `infer: false` for content that should be stored verbatim. - +See [Ingestion Status](/api-reference/v2/endpoint/source-status) for the full status pipeline. -### Verify processing +### Query -Poll `context.status` with the IDs from the ingest response: +One call returns four keys: ranked `chunks`, the `graph` paths that connect them, the `forceful_relations` you declared at ingest, and `llm_prompt`, which is all of that as one markdown string with citation labels. Most integrations inject `llm_prompt` into the model call and are done. ```python Python SDK -status = client.context.status( +question = "How should I explain our refund policy to this user?" + +result = client.query( database="my_first_database", - ids=["contract_q4", "policy_2025"], + collection="support", + query=question, + mode="thinking", + max_results=8, ) + +for chunk in result.data.chunks: + print(chunk.score, chunk.context_id, chunk.content) +for path in result.data.graph: + print(path.path_summary) + +# Inject the prompt-ready context into your model call. +messages = [ + {"role": "system", "content": result.data.llm_prompt}, + {"role": "user", "content": question}, +] ``` ```typescript TypeScript SDK -const status = await client.context.status({ +const question = "How should I explain our refund policy to this user?"; + +const result = await client.query({ database: "my_first_database", - ids: ["contract_q4", "policy_2025"], + collection: "support", + query: question, + mode: "thinking", + maxResults: 8, }); -``` - -Wait until each `indexing_status` is `completed` (or `graph_creation` if you don't need full graph traversal). See [Context Overview](/api-reference/v2/endpoint/sources-overview) for the full status pipeline. +for (const chunk of result.data.chunks) { + console.log(chunk.score, chunk.contextId, chunk.content); +} +for (const path of result.data.graph) { + console.log(path.pathSummary); +} -### Query +// Inject the prompt-ready context into your model call. +const messages = [ + { role: "system", content: result.data.llmPrompt }, + { role: "user", content: question }, +]; +``` + -A single method covers every retrieval pattern - switch `type` and `query_by` to control behavior: +A few common variations on the same call: ```python Python SDK -# Knowledge over documents -knowledge = client.query( +# Search a user's collection and a shared one, ranking the user's higher. +personal = client.query( database="my_first_database", - query="What are the pricing tiers?", - type="knowledge", - query_by="hybrid", - mode="thinking", - graph_context=True, + collections={"user_alex": 2, "support": 1}, + query="How should I explain our refund policy to this user?", ) -# Personalized — searches knowledge AND user memories together -personalized = client.query( - database="my_first_database", - collection="user_alex", - query="What are the pricing tiers, and how should I explain them to this user?", - type="all", - query_by="hybrid", - mode="thinking", -) - -# Exact-phrase BM25 search +# Exact-phrase keyword search. exact = client.query( database="my_first_database", - query="ERROR_429 rate limit", - type="knowledge", + collection="support", + query="5 business days", query_by="text", operator="phrase", ) + +# Filter on a declared attribute. +filtered = client.query( + database="my_first_database", + collection="support", + query="refund window", + attributes={"department": {"$eq": "support"}}, +) ``` ```typescript TypeScript SDK -// Knowledge over documents -const knowledge = await client.query({ +// Search a user's collection and a shared one, ranking the user's higher. +const personal = await client.query({ database: "my_first_database", - query: "What are the pricing tiers?", - type: "knowledge", - queryBy: "hybrid", - mode: "thinking", - graphContext: true, -}); - -// Personalized — searches knowledge AND user memories together -const personalized = await client.query({ - database: "my_first_database", - collection: "user_alex", - query: "What are the pricing tiers, and how should I explain them to this user?", - type: "all", - queryBy: "hybrid", - mode: "thinking", + collections: { user_alex: 2, support: 1 }, + query: "How should I explain our refund policy to this user?", }); -// Exact-phrase BM25 search +// Exact-phrase keyword search. const exact = await client.query({ database: "my_first_database", - query: "ERROR_429 rate limit", - type: "knowledge", + collection: "support", + query: "5 business days", queryBy: "text", operator: "phrase", }); + +// Filter on a declared attribute. +const filtered = await client.query({ + database: "my_first_database", + collection: "support", + query: "refund window", + attributes: { department: { $eq: "support" } }, +}); ``` -For the full parameter reference, see [Query – Overview](/api-reference/v2/endpoint/query-overview). +Chunks carry no title or URL. To show where a chunk came from, pass its `context_id` to `context.inspect` (below). For every request and response field, see [Query](/essentials/v2/query) and the [Query reference](/api-reference/v2/endpoint/query); for injecting `llm_prompt` and mapping citations back, see [How to Use API Results](/essentials/v2/api-results). -### Browse, fetch, and inspect +### Browse, inspect, and delete ```python Python SDK -# List knowledge sources with pagination -sources = client.context.list( +# List the items in a collection, 50 per page. +listing = client.context.list( database="my_first_database", - type="knowledge", + collection="support", page=1, page_size=50, ) -# Inspect graph relationships for a specific source -relations = client.context.relations( +# Look up the title, attributes and content behind a context_id. +item = client.context.inspect( database="my_first_database", - id="doc_12345", + collection="support", + id="refund-policy", ) -# Inspect original source content or get a presigned URL -file = client.context.inspect( +# Graph relations extracted from one item. +relations = client.context.relations( database="my_first_database", - id="doc_12345", - mode="url", + collection="support", + id="refund-policy", ) -# Delete a knowledge source -client.context.delete( - type="knowledge", +# Delete items by context_id. +deleted = client.context.delete( database="my_first_database", - ids=["doc_12345"], + collection="support", + ids=["chat-alex-001"], ) ``` ```typescript TypeScript SDK -// List knowledge sources with pagination -const sources = await client.context.list({ +// List the items in a collection, 50 per page. +const listing = await client.context.list({ database: "my_first_database", - type: "knowledge", + collection: "support", page: 1, pageSize: 50, }); -// Inspect graph relationships for a specific source -const relations = await client.context.relations({ +// Look up the title, attributes and content behind a context_id. +const item = await client.context.inspect({ database: "my_first_database", - id: "doc_12345", + collection: "support", + id: "refund-policy", }); -// Inspect original source content or get a presigned URL -const file = await client.context.inspect({ +// Graph relations extracted from one item. +const relations = await client.context.relations({ database: "my_first_database", - id: "doc_12345", - mode: "url", + collection: "support", + id: "refund-policy", }); -// Delete a knowledge source -await client.context.delete({ - type: "knowledge", +// Delete items by context_id. +const deleted = await client.context.delete({ database: "my_first_database", - ids: ["doc_12345"], + collection: "support", + ids: ["chat-alex-001"], }); ``` +A delete reports each id in `results[]` with a `deleted_count`. An item that is still indexing cannot be deleted yet; see [Delete Context](/api-reference/v2/endpoint/delete-source). + ## Response envelope All responses are wrapped in a consistent envelope: @@ -498,15 +487,15 @@ Both SDKs are fully typed: - **Type checking** for request and response objects - **Inline documentation** for each parameter, sourced from the OpenAPI spec - **Compile-time validation** for required vs optional fields -- **Enum-typed values** for `type`, `query_by`, `operator` and `mode` +- **Enum-typed values** for `query_by`, `operator` and `mode` -The SDKs provide exact type parity with the API specification: +The SDKs are generated from the API specification: -- **Request parameters** - every field documented in the API reference is reflected in method signatures -- **Response objects** - return types match the JSON schema for each endpoint -- **Error types** - exception structures mirror error response formats -- **Nested objects** - complex parameters and responses keep their full structure +- **Request parameters:** fields documented in the API reference are reflected in method signatures +- **Response objects:** return types match the JSON schema for each endpoint +- **Error types:** exception structures mirror error response formats +- **Nested objects:** complex parameters and responses keep their full structure ## Error handling @@ -521,24 +510,17 @@ try: result = client.query( database="my_first_database", query="...", - type="knowledge", ) except NotFoundError: # Handle missing database / resource (HTTP 404) pass except ApiError as exc: if exc.status_code == 429: - # Rate limit — retry with backoff + # Rate limited: retry with backoff pass else: raise ``` - - -The Python SDK raises a typed exception per status from `hydra_db.errors`. Each subclasses `ApiError` and carries `status_code`, `headers`, and the parsed response `body`, from which you can read `body["error"]["code"]`. - -Not every status has its own class - `401` and `429` do not. Catch the base `ApiError` and branch on `status_code`, as above, whenever you need to handle those. - ```typescript TypeScript SDK import { HydraDBError } from "@hydradb/sdk"; @@ -546,16 +528,16 @@ try { const result = await client.query({ database: "my_first_database", query: "...", - type: "knowledge", }); } catch (error) { if (error instanceof HydraDBError) { - const code = error.body?.error?.code; + const body = error.body as { error?: { code?: string } } | undefined; + const code = body?.error?.code; if (code === "DATABASE_NOT_FOUND") { // Handle missing database } else if (error.statusCode === 429) { - // Rate limit — retry with backoff + // Rate limited: retry with backoff } else { throw error; } @@ -566,6 +548,12 @@ try { ``` + +The Python SDK raises a typed exception per status from `hydra_db.errors`. Each subclasses `ApiError` and carries `status_code`, `headers`, and the parsed response `body`, from which you can read `body["error"]["code"]`. + +Not every status has its own class: `401` and `429` do not. Catch the base `ApiError` and branch on `status_code`, as above, whenever you need to handle those. + + For the full list of error codes and retry patterns, see [Error Responses](/api-reference/v2/error-responses). ## IDE-driven discovery @@ -576,12 +564,12 @@ Whether you're using TypeScript, Python, VS Code, PyCharm, or any modern IDE, th 2. Open the parentheses → see all required and optional parameters 3. Press `Cmd+Space` (macOS) or `Ctrl+Space` (Windows/Linux) → get inline documentation -This works because the SDKs are fully typed with comprehensive parameter docs sourced from the OpenAPI spec. +This works because the SDKs are fully typed with parameter docs sourced from the OpenAPI spec. ## Related sections -- [API Reference](/api-reference/v2) - complete endpoint documentation -- [Error Responses](/api-reference/v2/error-responses) - HTTP codes, error codes, retry patterns -- [Quickstart](/get-started/v2/quickstart) - build your first integration in five minutes -- [Query](/essentials/v2/query) - conceptual overview of `POST /query` -- [Knowledge](/essentials/v2/knowledge) and [Memories](/essentials/v2/memories) - ingestion deep-dives +- [API Reference](/api-reference/v2): complete endpoint documentation +- [Error Responses](/api-reference/v2/error-responses): HTTP codes, error codes, retry patterns +- [Quickstart](/get-started/v2/quickstart): build your first integration in five minutes +- [Ingest context](/essentials/v2/ingest): every item field, conversations and enrichment +- [Query](/essentials/v2/query): every field of `POST /query` and its response diff --git a/cookbooks/ai-chief-of-staff.mdx b/cookbooks/ai-chief-of-staff.mdx index 47239201..6f4f89da 100644 --- a/cookbooks/ai-chief-of-staff.mdx +++ b/cookbooks/ai-chief-of-staff.mdx @@ -1,8 +1,11 @@ --- title: AI Chief of Staff description: “Quick-start guide to building an AI Chief of Staff with HydraDB using the TypeScript SDK. Register workspace functions as knowledge objects and let any agent ask 'What should I do?' to receive a structured, personalized execution plan. For the full production guide with Python, multi-step planning, security, and observability, see the complete AI Chief of Staff cookbook.” +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + >This page covers the core concepts and TypeScript patterns in 20 minutes. For the full production implementation - Python SDK, multi-step planning, policy engine, approval workflows, observability, and benchmarks - see the [complete AI Chief of Staff cookbook](/cookbooks/hydradb-cookbook-06). This guide walks you through the key building blocks of an **AI Chief of Staff** - an _AI version of n8n_ - powered by HydraDB. Instead of only _answering_ questions, this assistant can **_take actions_** across every app in your workspace by selecting and executing the correct function at the right time. diff --git a/cookbooks/ai-linkedin-recruiter.mdx b/cookbooks/ai-linkedin-recruiter.mdx index 190a27ba..4051f523 100644 --- a/cookbooks/ai-linkedin-recruiter.mdx +++ b/cookbooks/ai-linkedin-recruiter.mdx @@ -1,8 +1,11 @@ --- title: "AI LinkedIn: People search in Natural Language" description: "Learn how to build an intelligent recruiting platform that understands natural language queries like 'find me someone who has 5+ years of experience in machine learning and has worked at Apple before' using HydraDB's AI search capabilities." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide demonstrates how to build a revolutionary AI-powered hiring platform that transforms how recruiters and hiring managers discover candidates. Instead of traditional keyword searches, your platform will understand natural language queries and provide intelligent candidate matching using HydraDB's advanced AI capabilities. > **Note**: All code in this guide uses the official HydraDB TypeScript SDK (`@hydradb/sdk`). Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/ai-onboarding-agent.mdx b/cookbooks/ai-onboarding-agent.mdx index eb429a8b..24a1f00a 100644 --- a/cookbooks/ai-onboarding-agent.mdx +++ b/cookbooks/ai-onboarding-agent.mdx @@ -1,8 +1,11 @@ --- title: "AI Onboarding Agent" description: "Go from zero to a working onboarding agent in three phases. Upload decision logs, org charts, meeting notes, and product specs into HydraDB. New hires ask 'why did we choose Postgres?' or 'who owns the payments service?' and get answers from real company context - not generic LLM guesses. Every API call in this guide is real and verified." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building an **AI onboarding agent with full institutional memory** powered by HydraDB. Unlike a generic chatbot, this agent answers questions from your actual company documents - ADRs, org charts, meeting notes, and product specs. New hires get real context, not hallucinated guesses. > **Note**: All code in this guide uses the official HydraDB Python SDK. Install it with `pip install hydradb-sdk`. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/ai-travel-planner.mdx b/cookbooks/ai-travel-planner.mdx index e7430729..07e71401 100644 --- a/cookbooks/ai-travel-planner.mdx +++ b/cookbooks/ai-travel-planner.mdx @@ -1,8 +1,11 @@ --- title: "AI Travel Planner" description: "Learn how to build an intelligent travel planning platform that understands natural language queries and provides personalized recommendations using HydraDB's AI search and memory capabilities." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide demonstrates how to build a revolutionary AI-powered travel planning platform that transforms how travelers discover, plan, and book their trips. Instead of traditional keyword searches and manual browsing, your platform will understand natural language queries and provide intelligent, personalized travel recommendations using HydraDB's advanced AI capabilities. > **Note**: All code in this guide uses the official HydraDB TypeScript SDK. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/competitive-intelligence-agent.mdx b/cookbooks/competitive-intelligence-agent.mdx index 13d221c6..e1a8a53f 100644 --- a/cookbooks/competitive-intelligence-agent.mdx +++ b/cookbooks/competitive-intelligence-agent.mdx @@ -1,8 +1,11 @@ --- title: "AI Competitive Intelligence Agent" description: "Continuously ingest competitor press releases, job postings, customer reviews, and earnings transcripts into HydraDB. Answer 'What is Competitor X doing right now?' and 'How has their messaging shifted over the last 6 months?' with full temporal context. Every API call in this guide is real and verified." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building a **competitive intelligence agent with persistent temporal memory** powered by HydraDB. Unlike a static market research doc or a naive RAG pipeline, this agent continuously ingests competitor signals and answers both point-in-time questions ("What has Acme Corp announced about enterprise?") and trend questions ("How has their pricing messaging shifted since Q1?") - with full context across press releases, job postings, customer reviews, and earnings calls unified in one retrieval layer. > **Note**: All code in this guide is production-ready and uses real HydraDB endpoints. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/cookbook-01-build-cursor-for-docs.mdx b/cookbooks/cookbook-01-build-cursor-for-docs.mdx index 5e10854c..66befebc 100644 --- a/cookbooks/cookbook-01-build-cursor-for-docs.mdx +++ b/cookbooks/cookbook-01-build-cursor-for-docs.mdx @@ -1,8 +1,11 @@ --- title: "Cursor for Docs" description: "Go from zero to a production AI assistant that answers 'why was this built this way?' - in four phases. Start with one file and a real recall query. End with a FastAPI backend that ingests GitHub, PRs, Slack, and RFCs, then generates GPT-4o answers grounded in your codebase. Every endpoint in this guide is real and copy-paste ready." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Go from zero to a production AI assistant that answers "why was this built this way?" in four phases. Start with one file and a real recall query. End with a FastAPI backend that ingests GitHub, PRs, Slack, and RFCs, then generates GPT-4o answers grounded in your codebase. > **How this guide is structured.** Each phase ends with something that works. Phase 0 is a complete minimal system in under 10 minutes. Phases 1–3 are progressive upgrades. You never need to redo what came before. diff --git a/cookbooks/cookbook-04-build-notion-ai.mdx b/cookbooks/cookbook-04-build-notion-ai.mdx index 481e107a..2fbfd638 100644 --- a/cookbooks/cookbook-04-build-notion-ai.mdx +++ b/cookbooks/cookbook-04-build-notion-ai.mdx @@ -1,8 +1,11 @@ --- title: "Internal IT Support Agent" description: "Ingest your entire workspace - Notion, Confluence, Slack - into HydraDB and build a conversational interface that understands relationships between documents. Answer 'why did we decide X?' using HydraDB's context graph. Every endpoint in this guide is real and copy-paste ready." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/cookbooks/cookbook-10-ai-financial-analyst.mdx b/cookbooks/cookbook-10-ai-financial-analyst.mdx index e8cc11bd..6c664724 100644 --- a/cookbooks/cookbook-10-ai-financial-analyst.mdx +++ b/cookbooks/cookbook-10-ai-financial-analyst.mdx @@ -1,8 +1,11 @@ --- title: "AI Financial Analyst with Memory" description: "Upload earnings PDFs, internal metrics, and board memos into HydraDB. Ask trend questions across quarters, get temporally-aware answers, and surface the exact clause, figure, or narrative shift - not a generic summary." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/cookbooks/customer-support-agent.mdx b/cookbooks/customer-support-agent.mdx index 78b44e35..4bf32e32 100644 --- a/cookbooks/customer-support-agent.mdx +++ b/cookbooks/customer-support-agent.mdx @@ -1,8 +1,11 @@ --- title: "AI Customer Support Agent with Memory" description: "A support agent that never forgets. Ingest help docs, past ticket resolutions, and every conversation turn into HydraDB. Every response is personalized using per-user memory - the agent knows the customer's plan, their past issues, their preferences, and what already failed before it starts typing." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building a **customer support agent with persistent memory** powered by HydraDB. Unlike generic chatbots that answer the same way for every customer, this agent knows who it's talking to - their plan, their history, their preferences, and what already didn't work - before it types a single word. > **Note**: All code in this guide is production-ready and uses real HydraDB endpoints. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/glean-clone.mdx b/cookbooks/glean-clone.mdx index 3075dd62..e3d7af7e 100644 --- a/cookbooks/glean-clone.mdx +++ b/cookbooks/glean-clone.mdx @@ -1,8 +1,11 @@ --- title: "Build your own Glean with HydraDB" description: "Learn how to build a comprehensive workplace search and AI assistant platform using HydraDB APIs. This guide covers data ingestion, retrieval, and app-layer answer generation across multiple data sources." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide will walk you through building an extremely powerful workplace search and AI assistant platform that rivals Glean using HydraDB APIs. You'll learn how to create a unified retrieval experience across multiple data sources and generate answers in your application layer. > **Note**: All code in this guide uses the official HydraDB Python SDK. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/hydradb-cookbook-06.mdx b/cookbooks/hydradb-cookbook-06.mdx index 6bfc1439..5033bfa8 100644 --- a/cookbooks/hydradb-cookbook-06.mdx +++ b/cookbooks/hydradb-cookbook-06.mdx @@ -8,8 +8,11 @@ tags: - automation - multi-agent - cookbook +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/cookbooks/index.mdx b/cookbooks/index.mdx index e3d7452e..516d1e43 100644 --- a/cookbooks/index.mdx +++ b/cookbooks/index.mdx @@ -1,8 +1,11 @@ --- title: "Introduction" description: "Production-ready guides for building AI agents with HydraDB." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Step-by-step tutorials that go from zero to a working agent. Each cookbook uses real HydraDB endpoints, includes copy-paste code, and ends with something you can ship. diff --git a/cookbooks/internal-search-perplexity.mdx b/cookbooks/internal-search-perplexity.mdx index 57fa1d3a..ff2586bc 100644 --- a/cookbooks/internal-search-perplexity.mdx +++ b/cookbooks/internal-search-perplexity.mdx @@ -1,8 +1,11 @@ --- title: "Perplexity for Internal Knowledge" description: "Ingest Slack, Gmail, Confluence, GitHub, and Linear into one HydraDB tenant. Ask any question in natural language and get a cited, synthesized answer drawing from across all your company's knowledge - including 'what led to the decision to sunset Project X?' with full decision provenance." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/cookbooks/v2/ai-chief-of-staff.mdx b/cookbooks/v2/ai-chief-of-staff.mdx index dacefb1c..80967eda 100644 --- a/cookbooks/v2/ai-chief-of-staff.mdx +++ b/cookbooks/v2/ai-chief-of-staff.mdx @@ -1,8 +1,11 @@ --- title: AI Chief of Staff description: “Quick-start guide to building an AI Chief of Staff with HydraDB using the TypeScript SDK. Register workspace functions as knowledge objects and let any agent ask 'What should I do?' to receive a structured, personalized execution plan. For the full production guide with Python, multi-step planning, security, and observability, see the complete AI Chief of Staff cookbook.” +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + >This page covers the core concepts and TypeScript patterns in 20 minutes. For the full production implementation - Python SDK, multi-step planning, policy engine, approval workflows, observability, and benchmarks - see the [complete AI Chief of Staff cookbook](/cookbooks/hydradb-cookbook-06). This guide walks you through the key building blocks of an **AI Chief of Staff** - an _AI version of n8n_ - powered by HydraDB. Instead of only _answering_ questions, this assistant can **_take actions_** across every app in your workspace by selecting and executing the correct function at the right time. diff --git a/cookbooks/v2/ai-linkedin-recruiter.mdx b/cookbooks/v2/ai-linkedin-recruiter.mdx index aebf0be5..757a6d54 100644 --- a/cookbooks/v2/ai-linkedin-recruiter.mdx +++ b/cookbooks/v2/ai-linkedin-recruiter.mdx @@ -1,8 +1,11 @@ --- title: "AI LinkedIn: People search in Natural Language" description: "Learn how to build an intelligent recruiting platform that understands natural language queries like 'find me someone who has 5+ years of experience in machine learning and has worked at Apple before' using HydraDB's AI search capabilities." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide demonstrates how to build a revolutionary AI-powered hiring platform that transforms how recruiters and hiring managers discover candidates. Instead of traditional keyword searches, your platform will understand natural language queries and provide intelligent candidate matching using HydraDB's advanced AI capabilities. > **Note**: All code in this guide uses the official HydraDB TypeScript SDK (`@hydradb/sdk`). Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/v2/ai-onboarding-agent.mdx b/cookbooks/v2/ai-onboarding-agent.mdx index 39f6efc9..4048cbc2 100644 --- a/cookbooks/v2/ai-onboarding-agent.mdx +++ b/cookbooks/v2/ai-onboarding-agent.mdx @@ -1,8 +1,11 @@ --- title: "AI Onboarding Agent" description: "Go from zero to a working onboarding agent in three phases. Upload decision logs, org charts, meeting notes, and product specs into HydraDB. New hires ask 'why did we choose Postgres?' or 'who owns the payments service?' and get answers from real company context - not generic LLM guesses. Every API call in this guide is real and verified." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building an **AI onboarding agent with full institutional memory** powered by HydraDB. Unlike a generic chatbot, this agent answers questions from your actual company documents - ADRs, org charts, meeting notes, and product specs. New hires get real context, not hallucinated guesses. > **Note**: All code in this guide uses the official HydraDB Python SDK. Install it with `pip install hydradb-sdk`. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/v2/ai-travel-planner.mdx b/cookbooks/v2/ai-travel-planner.mdx index c63793eb..1325a559 100644 --- a/cookbooks/v2/ai-travel-planner.mdx +++ b/cookbooks/v2/ai-travel-planner.mdx @@ -1,8 +1,11 @@ --- title: "AI Travel Planner" description: "Learn how to build an intelligent travel planning platform that understands natural language queries and provides personalized recommendations using HydraDB's AI search and memory capabilities." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide demonstrates how to build a revolutionary AI-powered travel planning platform that transforms how travelers discover, plan, and book their trips. Instead of traditional keyword searches and manual browsing, your platform will understand natural language queries and provide intelligent, personalized travel recommendations using HydraDB's advanced AI capabilities. > **Note**: All code in this guide uses the official HydraDB TypeScript SDK. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/v2/competitive-intelligence-agent.mdx b/cookbooks/v2/competitive-intelligence-agent.mdx index 4b684e2c..4ff97228 100644 --- a/cookbooks/v2/competitive-intelligence-agent.mdx +++ b/cookbooks/v2/competitive-intelligence-agent.mdx @@ -1,8 +1,11 @@ --- title: "AI Competitive Intelligence Agent" description: "Continuously ingest competitor press releases, job postings, customer reviews, and earnings transcripts into HydraDB. Answer 'What is Competitor X doing right now?' and 'How has their messaging shifted over the last 6 months?' with full temporal context. Every API call in this guide is real and verified." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building a **competitive intelligence agent with persistent temporal memory** powered by HydraDB. Unlike a static market research doc or a naive RAG pipeline, this agent continuously ingests competitor signals and answers both point-in-time questions ("What has Acme Corp announced about enterprise?") and trend questions ("How has their pricing messaging shifted since Q1?") - with full context across press releases, job postings, customer reviews, and earnings calls unified in one retrieval layer. > **Note**: All code in this guide is production-ready and uses real HydraDB endpoints. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/v2/cookbook-01-build-cursor-for-docs.mdx b/cookbooks/v2/cookbook-01-build-cursor-for-docs.mdx index 142694d2..4b742ed4 100644 --- a/cookbooks/v2/cookbook-01-build-cursor-for-docs.mdx +++ b/cookbooks/v2/cookbook-01-build-cursor-for-docs.mdx @@ -1,8 +1,11 @@ --- title: "Cursor for Docs" description: "Go from zero to a production AI assistant that answers 'why was this built this way?' - in four phases. Start with one file and a real search query. End with a FastAPI backend that ingests GitHub, PRs, Slack, and RFCs, then generates GPT-4o answers grounded in your codebase. Every endpoint in this guide is real and copy-paste ready." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Go from zero to a production AI assistant that answers "why was this built this way?" in four phases. Start with one file and a real search query. End with a FastAPI backend that ingests GitHub, PRs, Slack, and RFCs, then generates GPT-4o answers grounded in your codebase. > **How this guide is structured.** Each phase ends with something that works. Phase 0 is a complete minimal system in under 10 minutes. Phases 1–3 are progressive upgrades. You never need to redo what came before. diff --git a/cookbooks/v2/cookbook-04-build-notion-ai.mdx b/cookbooks/v2/cookbook-04-build-notion-ai.mdx index a533f47a..151504a4 100644 --- a/cookbooks/v2/cookbook-04-build-notion-ai.mdx +++ b/cookbooks/v2/cookbook-04-build-notion-ai.mdx @@ -1,8 +1,11 @@ --- title: "Internal IT Support Agent" description: "Ingest your entire workspace - Notion, Confluence, Slack - into HydraDB and build a conversational interface that understands relationships between documents. Answer 'why did we decide X?' using HydraDB's context graph. Every endpoint in this guide is real and copy-paste ready." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Notion's built-in AI keyword-searches. It returns the document you asked about and stops. It can't tell you **why** a decision was made, who influenced it, or whether it's been superseded by something newer. HydraDB is different. It doesn't just store vectors - it builds a **living context graph**. Every memory is parsed, enriched, and connected to other memories. When your agent calls `POST /query`, it doesn't just get semantically similar chunks. It gets the most useful context for that exact query - weighted by recency, relevance, relationships, and historical usage patterns. diff --git a/cookbooks/v2/cookbook-10-ai-financial-analyst.mdx b/cookbooks/v2/cookbook-10-ai-financial-analyst.mdx index a38a549e..7e38b29a 100644 --- a/cookbooks/v2/cookbook-10-ai-financial-analyst.mdx +++ b/cookbooks/v2/cookbook-10-ai-financial-analyst.mdx @@ -1,8 +1,11 @@ --- title: "AI Financial Analyst with Memory" description: "Upload earnings PDFs, internal metrics, and board memos into HydraDB. Ask trend questions across quarters, get temporally-aware answers, and surface the exact clause, figure, or narrative shift - not a generic summary." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building a production-grade **AI Financial Analyst** powered by HydraDB. The agent ingests structured and unstructured financial data - earnings call transcripts, PDF filings, internal metric exports, and board memos - and answers questions that require reasoning across time: - _"How did our gross margin trend across the last four quarters?"_ diff --git a/cookbooks/v2/customer-support-agent.mdx b/cookbooks/v2/customer-support-agent.mdx index aa9b891b..355c61c0 100644 --- a/cookbooks/v2/customer-support-agent.mdx +++ b/cookbooks/v2/customer-support-agent.mdx @@ -1,8 +1,11 @@ --- title: "AI Customer Support Agent with Memory" description: "A support agent that never forgets. Ingest help docs, past ticket resolutions, and every conversation turn into HydraDB. Every response is personalized using per-user memory - the agent knows the customer's plan, their past issues, their preferences, and what already failed before it starts typing." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building a **customer support agent with persistent memory** powered by HydraDB. Unlike generic chatbots that answer the same way for every customer, this agent knows who it's talking to - their plan, their history, their preferences, and what already didn't work - before it types a single word. > **Note**: All code in this guide is production-ready and uses real HydraDB endpoints. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/v2/glean-clone.mdx b/cookbooks/v2/glean-clone.mdx index e5d0d01f..97740e70 100644 --- a/cookbooks/v2/glean-clone.mdx +++ b/cookbooks/v2/glean-clone.mdx @@ -1,8 +1,11 @@ --- title: "Build your own Glean with HydraDB" description: "Learn how to build a comprehensive workplace search and AI assistant platform using HydraDB APIs. This guide covers data ingestion, retrieval, and app-layer answer generation across multiple data sources." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide will walk you through building an extremely powerful workplace search and AI assistant platform that rivals Glean using HydraDB APIs. You'll learn how to create a unified retrieval experience across multiple data sources and generate answers in your application layer. > **Note**: All code in this guide uses the official HydraDB Python SDK. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/cookbooks/v2/hydradb-cookbook-06.mdx b/cookbooks/v2/hydradb-cookbook-06.mdx index 390c0196..d76f572e 100644 --- a/cookbooks/v2/hydradb-cookbook-06.mdx +++ b/cookbooks/v2/hydradb-cookbook-06.mdx @@ -8,8 +8,11 @@ tags: - automation - multi-agent - cookbook +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + # Build an AI Chief of Staff diff --git a/cookbooks/v2/index.mdx b/cookbooks/v2/index.mdx index a49ec182..c01db0ea 100644 --- a/cookbooks/v2/index.mdx +++ b/cookbooks/v2/index.mdx @@ -1,8 +1,11 @@ --- title: "Introduction" description: "Production-ready guides for building AI agents with HydraDB." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Step-by-step tutorials that go from zero to a working agent. Each cookbook uses real HydraDB endpoints, includes copy-paste code, and ends with something you can ship. diff --git a/cookbooks/v2/internal-search-perplexity.mdx b/cookbooks/v2/internal-search-perplexity.mdx index 7d040365..0006e272 100644 --- a/cookbooks/v2/internal-search-perplexity.mdx +++ b/cookbooks/v2/internal-search-perplexity.mdx @@ -1,8 +1,11 @@ --- title: "Perplexity for Internal Knowledge" description: "Ingest Slack, Gmail, Confluence, GitHub, and Linear into one HydraDB database. Ask any question in natural language and get a cited, synthesized answer drawing from across all your company's knowledge - including 'what led to the decision to sunset Project X?' with full decision provenance." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + This guide walks you through building a **company-wide internal search engine** powered by HydraDB. Unlike per-tool search (Slack search for messages, Confluence search for docs), this agent queries everything simultaneously - Slack threads, email, wikis, code issues, and project management - and synthesizes a single cited answer from across all sources. > **Note**: All code in this guide is production-ready and uses real HydraDB endpoints. Base URL: `https://api.hydradb.com`. Get your API key at [app.hydradb.com](https://app.hydradb.com). diff --git a/docs.json b/docs.json index c532d696..efe65d63 100644 --- a/docs.json +++ b/docs.json @@ -74,10 +74,11 @@ "essentials/v2/architecture", "essentials/v2/databases-and-collections", "essentials/v2/context-graphs", + "essentials/v2/bring-your-own-graph", "essentials/v2/graph-collections-byog", "essentials/v2/semantic-search", "essentials/v2/glossary", - "essentials/v2/split-databases" + "continuity-assurance" ] }, { @@ -97,30 +98,6 @@ } ] }, - { - "tab": "Cookbooks", - "groups": [ - { - "group": "Cookbooks", - "public": true, - "pages": [ - "cookbooks/v2/index", - "cookbooks/v2/glean-clone", - "cookbooks/v2/ai-chief-of-staff", - "cookbooks/v2/ai-linkedin-recruiter", - "cookbooks/v2/ai-travel-planner", - "cookbooks/v2/cookbook-01-build-cursor-for-docs", - "cookbooks/v2/customer-support-agent", - "cookbooks/v2/cookbook-04-build-notion-ai", - "cookbooks/v2/competitive-intelligence-agent", - "cookbooks/v2/hydradb-cookbook-06", - "cookbooks/v2/internal-search-perplexity", - "cookbooks/v2/ai-onboarding-agent", - "cookbooks/v2/cookbook-10-ai-financial-analyst" - ] - } - ] - }, { "tab": "API Reference", "groups": [ @@ -208,142 +185,6 @@ ] } ] - }, - { - "version": "v1", - "tabs": [ - { - "tab": "Guides", - "groups": [ - { - "group": "Get Started", - "public": true, - "pages": [ - "get-started/introduction", - "get-started/core-concepts", - "get-started/quickstart" - ] - }, - { - "group": "Essentials", - "public": true, - "pages": [ - "essentials/architecture", - "essentials/memories", - "essentials/knowledge", - "essentials/app-sources", - "essentials/recall", - "essentials/webhooks", - "essentials/multi-tenant", - "essentials/metadata", - "essentials/context-graphs", - "essentials/api-results", - "essentials/semantic-search", - "continuity-assurance" - ] - }, - { - "group": "Plugins", - "public": true, - "pages": [ - "plugins/claude-code", - "plugins/mcp", - "plugins/cli", - "plugins/openclaw" - ] - }, - { - "group": "For Agents", - "public": true, - "pages": ["AGENTS"] - } - ] - }, - { - "tab": "Cookbooks", - "groups": [ - { - "group": "Cookbooks", - "public": true, - "pages": [ - "cookbooks/index", - "cookbooks/glean-clone", - "cookbooks/ai-chief-of-staff", - "cookbooks/ai-linkedin-recruiter", - "cookbooks/ai-travel-planner", - "cookbooks/cookbook-01-build-cursor-for-docs", - "cookbooks/customer-support-agent", - "cookbooks/cookbook-04-build-notion-ai", - "cookbooks/competitive-intelligence-agent", - "cookbooks/hydradb-cookbook-06", - "cookbooks/internal-search-perplexity", - "cookbooks/ai-onboarding-agent", - "cookbooks/cookbook-10-ai-financial-analyst" - ] - } - ] - }, - { - "tab": "API Reference", - "groups": [ - { - "group": "API Documentation", - "public": true, - "pages": [ - "api-reference/index", - "api-reference/sdks", - "api-reference/error-responses" - ] - }, - { - "group": "Tenants", - "public": true, - "pages": [ - "api-reference/endpoint/tenants-overview", - "api-reference/endpoint/create-tenant", - "api-reference/endpoint/monitor-tenant", - "api-reference/endpoint/infra-status", - "api-reference/endpoint/list-sub-tenant-ids", - "api-reference/endpoint/list-tenant-ids", - "api-reference/endpoint/delete-tenant" - ] - }, - { - "group": "Knowledge & Memories", - "public": true, - "pages": [ - "api-reference/endpoint/ingestion-overview", - "api-reference/endpoint/memories-overview", - "api-reference/endpoint/upload-knowledge", - "api-reference/endpoint/add-memory", - "api-reference/endpoint/verify-processing", - "api-reference/endpoint/delete-memory", - "api-reference/endpoint/delete-knowledge" - ] - }, - { - "group": "Recall", - "public": true, - "pages": [ - "api-reference/endpoint/recall-overview", - "api-reference/endpoint/full-recall", - "api-reference/endpoint/recall-preferences", - "api-reference/endpoint/boolean-recall" - ] - }, - { - "group": "Managing Data", - "public": true, - "pages": [ - "api-reference/endpoint/list-overview", - "api-reference/endpoint/list-data", - "api-reference/endpoint/fetch-content", - "api-reference/endpoint/graph-relations" - ] - } - ] - } - ] } ] }, diff --git a/essentials/api-results.mdx b/essentials/api-results.mdx index 56d51cfb..9c9ce3eb 100644 --- a/essentials/api-results.mdx +++ b/essentials/api-results.mdx @@ -1,8 +1,11 @@ --- title: "How to Use API Results" description: "Turn a recall response into a grounded LLM answer." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + A recall response is structured JSON. Your model wants prose. This page shows how to bridge that gap - turning a `RetrievalResult` into a clean, grounded prompt for an LLM. The same pattern works for [`full_recall`](/api-reference/endpoint/full-recall), [`recall_preferences`](/api-reference/endpoint/recall-preferences), and combinations of the two. diff --git a/essentials/app-sources.mdx b/essentials/app-sources.mdx index f95d9c8d..005e73c8 100644 --- a/essentials/app-sources.mdx +++ b/essentials/app-sources.mdx @@ -1,10 +1,13 @@ --- title: "App Sources" description: "How to ingest Slack, Gmail, Jira, Notion, CRM, and other app data so Recall can use IDs, threads, actors, and relations correctly." +noindex: true --- import { Field } from "/snippets/field.jsx"; +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + App sources are pre-parsed records from business applications: emails, chat messages, tickets, wiki pages, CRM objects, comments, and attachments. Use app sources when you already know the item's text and structured fields. HydraDB uses that structure to build better search context than raw text alone. diff --git a/essentials/architecture.mdx b/essentials/architecture.mdx index 625db6f9..e8e77b8c 100644 --- a/essentials/architecture.mdx +++ b/essentials/architecture.mdx @@ -1,8 +1,11 @@ --- title: "Architecture" description: "How HydraDB moves content from ingestion to indexed recall, and where tenants, metadata, graph context, and retrieval fit together." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + HydraDB is context infrastructure for AI applications. From the outside, you call a small set of HTTP APIs. Inside, HydraDB coordinates tenant isolation, async ingestion, indexing, graph construction, and hybrid retrieval so your application can store context once and recall the right pieces later. --- diff --git a/essentials/context-graphs.mdx b/essentials/context-graphs.mdx index abc5d7e4..19dbd344 100644 --- a/essentials/context-graphs.mdx +++ b/essentials/context-graphs.mdx @@ -1,8 +1,11 @@ --- title: "Context Graphs" description: "How HydraDB models relationships between chunks using triplets to improve recall quality." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## 1. What it is A context graph is a structured map of relationships between stored pieces of context in your tenant. diff --git a/essentials/knowledge.mdx b/essentials/knowledge.mdx index 5baa1692..c1aa2034 100644 --- a/essentials/knowledge.mdx +++ b/essentials/knowledge.mdx @@ -1,8 +1,11 @@ --- title: "Knowledge" description: "What Knowledge is, how it differs from Memories, and how to ingest it correctly." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## 1. What it is Knowledge is shared, tenant-wide context that all users and agents in a tenant can recall - product documents, internal wikis, policy PDFs, Slack threads, Notion pages, CSVs, emails, and any other source content. diff --git a/essentials/memories.mdx b/essentials/memories.mdx index 2966c486..957f9781 100644 --- a/essentials/memories.mdx +++ b/essentials/memories.mdx @@ -1,8 +1,11 @@ --- title: "Memories" description: "What Memories are, how they differ from Knowledge, and how to write them correctly." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/essentials/metadata.mdx b/essentials/metadata.mdx index 44268094..51002e31 100644 --- a/essentials/metadata.mdx +++ b/essentials/metadata.mdx @@ -2,8 +2,11 @@ title: Metadata description: How HydraDB uses metadata fields to filter recall results deterministically. category: Essentials +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## 1. What it is Metadata is structured data attached to knowledge and memories. It lets you narrow recall to a known scope before semantic retrieval runs. diff --git a/essentials/multi-tenant.mdx b/essentials/multi-tenant.mdx index c6a798f7..d0b61c6a 100644 --- a/essentials/multi-tenant.mdx +++ b/essentials/multi-tenant.mdx @@ -1,8 +1,11 @@ --- title: "Multi-Tenant Support" description: "How HydraDB scopes data using tenants and sub-tenants, and how scoping affects writes and recall." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + {/* PRO-1276 — DEFERRED, NOT FIXED. diff --git a/essentials/recall.mdx b/essentials/recall.mdx index 71b2d7cd..aa04f737 100644 --- a/essentials/recall.mdx +++ b/essentials/recall.mdx @@ -1,8 +1,11 @@ --- title: "Recall" description: "How HydraDB retrieves the right context for each query - across Knowledge, Memories, and the context graph." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Recall turns stored context into the *right* context for a specific query. This page covers what recall is, the endpoints HydraDB exposes, the parameters that matter, and the patterns you'll reach for in production. For request/response schemas, see the API reference: [Full Recall](/api-reference/endpoint/full-recall), [Memory Recall](/api-reference/endpoint/recall-preferences), [Boolean Recall](/api-reference/endpoint/boolean-recall). diff --git a/essentials/semantic-search.mdx b/essentials/semantic-search.mdx index 8a09a758..4c65525a 100644 --- a/essentials/semantic-search.mdx +++ b/essentials/semantic-search.mdx @@ -1,8 +1,11 @@ --- title: "Semantic Search & Retrieval" description: "How semantic, lexical, graph, and metadata signals work together in HydraDB recall." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Semantic search is useful because it retrieves by meaning instead of exact wording. It is also incomplete on its own: production agents need exact matches, freshness, scope, tenant isolation, metadata filters, and graph relationships. HydraDB recall combines those signals so you can retrieve context that is useful, not just similar. --- diff --git a/essentials/v2/access-control.mdx b/essentials/v2/access-control.mdx index 3c57e57a..0f921754 100644 --- a/essentials/v2/access-control.mdx +++ b/essentials/v2/access-control.mdx @@ -185,7 +185,7 @@ Send group principals explicitly when you want them: `"acl": ["grace@acme.com", An entry that is neither an email nor a recognized principal is kept as-is and matches nothing but public content. A typo narrows results; it never widens them. If a caller sees less than you expect, check the principal spelling first. -Access control composes with, and is independent of, [metadata filters](/essentials/v2/attributes): filters express *what you are looking for*, ACLs express *what you are allowed to find*. A caller cannot widen their own visibility with a filter. +Access control composes with, and is independent of, [attribute filters](/essentials/v2/attributes): filters express *what you are looking for*, ACLs express *what you are allowed to find*. A caller cannot widen their own visibility with a filter. --- @@ -231,7 +231,6 @@ Check the principal forms on both sides. `group:slack:C0123` on the document onl - [Connectors](/essentials/v2/connectors) - syncing app data, and per-resource ACL rules - [Ingest context](/essentials/v2/ingest#10-restricting-an-item): the `acl` item field -- [App Sources](/essentials/v2/app-sources) - the split-database app source shape, which also carries `acl` - [Query](/essentials/v2/query) - the `acl` field alongside every other retrieval parameter - [Metadata](/essentials/v2/attributes) - filtering by attributes, a different question from permission - [Multi-Tenant Support](/essentials/v2/databases-and-collections) - databases and collections, the isolation boundary ACLs work inside diff --git a/essentials/v2/api-results.mdx b/essentials/v2/api-results.mdx index a7bff006..845f6d9a 100644 --- a/essentials/v2/api-results.mdx +++ b/essentials/v2/api-results.mdx @@ -3,11 +3,7 @@ title: "How to Use API Results" description: "Inject llm_prompt into your model call, read the four response keys when you need structure, and map citations back to context." --- -import LegacyLine from "/snippets/legacy-line.mdx"; - -[`POST /query`](/essentials/v2/query) on a unified database returns four keys: `chunks`, `graph`, `forceful_relations` and `llm_prompt`. The last one is the whole context block, already formatted for a model. This page shows how to use it, and what to read when you need structure instead of a string. - - +[`POST /query`](/essentials/v2/query) returns four keys: `chunks`, `graph`, `forceful_relations` and `llm_prompt`. The last one is the whole context block, already formatted for a model. This page shows how to use it, and what to read when you need structure instead of a string. --- @@ -82,7 +78,7 @@ curl -s -X POST 'https://api.hydradb.com/query' \ ``` -There is no client-side string building on a unified database. Do not concatenate `chunks[].content` yourself; the prompt already contains it, in ranked order, with labels the model can cite. The SDK `build_string` / `buildString` helpers return `llm_prompt` verbatim on a unified database. +There is no client-side string building. Do not concatenate `chunks[].content` yourself; the prompt already contains it, in ranked order, with labels the model can cite. --- @@ -130,9 +126,9 @@ FAQ: refunds to a card take 5 to 7 business days to appear. ## Related facts -- [P1] **Refund Processing** -managed by→ **Finance Department** (query path, relevance 0.81) [1] +- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1] Refund processing is managed by the Finance Department. -- [P2] **User** -prefers→ **short answers** (chunk relation, relevance 0.74) [2] +- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2] The user prefers short answers about refunds. ## Temporal facts @@ -151,11 +147,11 @@ FAQ: refunds to a card take 5 to 7 business days to appear. | `# Query results` | The query, a `**Found:**` line counting what follows, and the instruction to cite a result by its number | None | | `## Results` | `chunks[]`, in ranked order: a `### 1. title` heading, a line with relevance (`score`), collection, type and category (`enrichment_kind`), a line with the id (`context_id`) and last-updated date, the `content`, then `**Enrichment:**` (`enrichment`). Results are separated by `---`. | `[1]`, `[2]`, ... | | `## Forceful relations` | `forceful_relations[]`, the items the hits declared with `forceful_relations` at ingest: a guide line, then `### R1. title` blocks laid out like results, with `**Linked from:**` (`via.from`) in place of relevance | `[R1]`, `[R2]`, ... | -| `## Related facts` | `graph[]`, one line per path: its chain of hops (`**A** -pred→ **B**`), `query path` or `chunk relation` for its `origin` with its relevance after reranking when it has one (`(query path, relevance 0.81)`), and the results its hops were extracted from, with the `path_summary` indented under it unless it only narrates the chain | `[P1]`, `[P2]`, ... in `graph[]` order; each line also cites its results | +| `## Related facts` | `graph[]`, one line per path: its chain of hops (`**A** -pred→ **B**`), its relevance after reranking in parentheses when it has one (`(relevance 0.81)`; a path with no reranked score has no parenthetical), and the results its hops were extracted from, with the `path_summary` indented under it unless it only narrates the chain | `[P1]`, `[P2]`, ... in `graph[]` order; each line also cites its results | | `## Temporal facts` | The dated facts behind `chunks[].temporal`, with window, precision and status, then the evidence phrase after a `;` | None; each fact cites its result | | `## Sources` | Each context once, in order of first appearance: title, type, id, url (web links only, never a storage location such as `s3://...`) and last-updated date | None; the numbers count contexts, not results | -A section with nothing in it is left out, and a query that returns nothing gets an empty `llm_prompt`. Ask the model to cite the labels and you get answers you can trace: a `[1]` in the reply is result 1, whose `**Id:**` is `refund-policy`, which you can open with [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content); a `[P1]` is the first path in `graph[]`. How a multi-hop chain reads is on [Query](/essentials/v2/query#llm_prompt). +A section with nothing in it is left out, and a query that returns nothing gets an empty `llm_prompt`. Ask the model to cite the labels and you get answers you can trace: a `[1]` in the reply is result 1, whose `**Id:**` is `refund-policy`, which you can look up with [`POST /context/list`](/api-reference/v2/endpoint/list-documents) or open with [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content); a `[P1]` is the first path in `graph[]`. How a multi-hop chain reads is on [Query](/essentials/v2/query#llm_prompt). --- @@ -206,16 +202,18 @@ for (const rel of result.data.forcefulRelations) { ## 4. Showing source details -A chunk carries only `chunk_id`, `context_id`, `score`, `content`, `enrichment`, `enrichment_kind` and `temporal`. It has no title, url, collection, timestamps or attributes. `llm_prompt` prints the title, url, collection and last-updated date for the model; to show any of that in your own UI, call [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content) with the chunk's `context_id`: +A chunk carries only `chunk_id`, `context_id`, `score`, `content`, `enrichment`, `enrichment_kind` and `temporal`. It has no title, url, collection, timestamps or attributes. `llm_prompt` prints the title, url, collection and last-updated date for the model. To show an item's title, timestamp or attributes in your own UI, list it by its `context_id` with [`POST /context/list`](/api-reference/v2/endpoint/list-documents): ```bash -curl -G 'https://api.hydradb.com/context/inspect' \ +curl -X POST 'https://api.hydradb.com/context/list' \ -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ -H "API-Version: 2" \ - --data-urlencode "database=acme" \ - --data-urlencode "id=refund-policy" + -H "Content-Type: application/json" \ + -d '{ "database": "acme", "collection": "company", "ids": ["refund-policy"] }' ``` +The row carries `title`, `timestamp` and the item's attributes (under `metadata` and `additional_metadata`, the list response's names for `attributes` and `custom_attributes`). [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content) with the same `context_id` returns the item's stored content. + Fetch it lazily, when a citation is opened, rather than for every chunk on every query. --- @@ -236,48 +234,10 @@ Fetch it lazily, when a citation is opened, rather than for every chunk on every | --- | --- | --- | | Building your own context string | Duplicates what the server already did, without the labels | Inject `llm_prompt`. | | Passing the raw `data` object to the model | Wastes tokens on ids and scores | Inject `llm_prompt`. | -| Expecting a title or url on a chunk | Chunks carry no source details | Call `GET /context/inspect` with `context_id`. | +| Expecting a title or url on a chunk | Chunks carry no source details | Read them from `llm_prompt`, or call `POST /context/list` with `ids: [""]`. | | Concatenating `content` and `enrichment` | Enrichment is stored separately on purpose | Use `content` for what was said, `enrichment` for what was extracted. | | Re-sorting chunks client-side | Overrides HydraDB's ranking | Preserve the returned order. | | Missing a grounding instruction | The model invents answers when retrieval is thin | System prompt: answer only from the provided context. | -| Reading `graph_context` or `chunk_content` | Those are split-database fields | On a unified database read `graph[]` and `chunks[].content`. | - ---- - -## 7. Split databases - -A database created with `type: "split"` returns the old response (`chunks[].chunk_content`, `sources[]`, `graph_context`, `additional_context`) and has no `llm_prompt`. There, build the context string with the SDK's `build_string` / `buildString` helper, which accepts the full envelope or just `data`: - - -```python Python SDK -from hydra_db.helpers import build_string - -result = client.query( - database="legacy-app", - collection="user_123", - query="What is our refund policy?", - type="all", - mode="thinking", -) - -context = build_string(result) -``` -```typescript TypeScript SDK -import { buildString } from "@hydradb/sdk/helpers"; - -const result = await client.query({ - database: "legacy-app", - collection: "user_123", - query: "What is our refund policy?", - type: "all", - mode: "thinking", -}); - -const context = buildString(result); -``` - - -Detect which shape you are holding by its keys: `llm_prompt` and a `graph` array mean unified; `graph_context` or `chunk_content` mean split. See [Split databases and legacy fields](/essentials/v2/split-databases#4-legacy-query-and-response-fields) for the field-by-field mapping. --- diff --git a/essentials/v2/app-sources.mdx b/essentials/v2/app-sources.mdx index 101486f3..3f0830f3 100644 --- a/essentials/v2/app-sources.mdx +++ b/essentials/v2/app-sources.mdx @@ -1,10 +1,13 @@ --- title: "App Sources" description: "How to ingest Slack, Gmail, Jira, Notion, CRM, and other app data using the field-based app-source model." +noindex: true --- import { Field } from "/snippets/field.jsx"; +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + App sources are pre-parsed records from business applications: emails, chat messages, tickets, wiki pages, CRM objects, comments, and attachments. Use them when your connector already extracted the item's text and structured fields. App sources are ingested through [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context) with the `app_knowledge` multipart field. The app-source payload is **not** a generic top-level `content` object. It uses the same app-native model as v1: `kind`, `provider`, `external_id`, `fields`, metadata, attachments, comments, and typed relations. diff --git a/essentials/v2/architecture.mdx b/essentials/v2/architecture.mdx index bae8aec0..039cbc1d 100644 --- a/essentials/v2/architecture.mdx +++ b/essentials/v2/architecture.mdx @@ -14,7 +14,7 @@ HydraDB organizes its work into three logical planes. You interact only with the | Plane | What it handles | Endpoints | |---|---|---| | **Control** | API authentication, database lifecycle, provisioning, and status | [`/databases`](/api-reference/v2/endpoint/tenants-overview) family | -| **Ingestion** | File and app-source uploads, memory writes, parsing, chunking, embedding, and graph construction | [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context), [`GET /context/status`](/api-reference/v2/endpoint/source-status) | +| **Ingestion** | Context item writes, connector syncs, parsing, chunking, embedding, and graph construction | [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context), [`GET /context/status`](/api-reference/v2/endpoint/source-status) | | **Retrieval** | Hybrid query, metadata filtering, graph context, keyword bm25 search, and response shaping | [`POST /query`](/api-reference/v2/endpoint/query), [`GET /context/relations`](/api-reference/v2/endpoint/source-relations) | ```mermaid @@ -47,14 +47,12 @@ flowchart LR subgraph Retrieval[Retrieval Plane - sync] direction LR Query([Query Orchestrator]) - MemVec([Memories Vector Store]) - KnowVec([Knowledge Vector Store]) + Vec([Vector Store]) KeywordBM25([Keyword BM25 Index]) ContextGraph([Context Graph]) Sources([Source Store]) style Query fill:#1e293b,stroke:#334155,stroke-width:2px,color:#f8fafc,stroke-linecap:round - style MemVec fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc,stroke-linecap:round - style KnowVec fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc,stroke-linecap:round + style Vec fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc,stroke-linecap:round style KeywordBM25 fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc,stroke-linecap:round style ContextGraph fill:#FF571A,stroke:#FF571A,stroke-width:2px,color:#f8fafc,stroke-linecap:round style Sources fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc,stroke-linecap:round @@ -67,15 +65,13 @@ flowchart LR Queue --> Parser Parser --> Sources Parser --> Embed - Embed --> MemVec - Embed --> KnowVec + Embed --> Vec Embed --> KeywordBM25 Parser --> Graph Graph --> ContextGraph API --|POST /query|--> Query - Query --> MemVec - Query --> KnowVec + Query --> Vec Query --> KeywordBM25 Query --> ContextGraph Query --> Sources @@ -83,7 +79,7 @@ flowchart LR Two details to notice in the diagram: -- **One ingest endpoint, one corpus.** [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context) takes `context` items, text or conversations, and writes them to the database. Collections partition them per user, team or project. A database created with `type: "split"` keeps two stores instead, [Knowledge](/essentials/v2/knowledge) and [Memories](/essentials/v2/memories), selected with `type`. +- **One ingest endpoint, one database of context items.** [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context) takes `context` items, text or conversations, and writes them to the database. [Connectors](/essentials/v2/connectors) sync provider content into the same database. Collections partition it per user, team or project. - **One query endpoint, every retrieval method.** [`POST /query`](/api-reference/v2/endpoint/query) is the only retrieval entry point. `collections` decides where to look and `query_by` how to match; see [Query](/essentials/v2/query) for the full picture. --- @@ -94,7 +90,7 @@ Ingestion is asynchronous. A successful upload means HydraDB accepted the work a ```mermaid flowchart LR - Upload([Upload Knowledge or Memory]) + Upload([Ingest context items]) Queued([Queued]) Processing([Processing]) Graph([Graph Creation]) @@ -151,7 +147,7 @@ The right mapping depends on your product shape: | **B2C app** | One database for your application | One collection per end user | | **Internal tools** | One database per company or environment | One collection per department or project | -The deeper trade-offs - when to spin up a new database vs. a new collection, how scoping interacts with [metadata filters](/essentials/v2/attributes), and how this affects [Memories](/essentials/v2/memories) - are covered in [Multi-Tenant Support](/essentials/v2/databases-and-collections). +The deeper trade-offs (when to spin up a new database vs. a new collection, how scoping interacts with [attribute filters](/essentials/v2/attributes), and how collections keep each person's context apart from shared context) are covered in [Databases and collections](/essentials/v2/databases-and-collections). --- @@ -163,7 +159,7 @@ The deeper trade-offs - when to spin up a new database vs. a new collection, h 2. **Filter before ranking.** Apply `attributes` to narrow the candidate set (see [Attributes](/essentials/v2/attributes)). 3. **Retrieve.** Run hybrid retrieval over the semantic vector store and the keyword bm25 index, or BM25-only retrieval when `query_by: "text"`. 4. **Blend.** Use `alpha` to weight semantic vs. keyword bm25 contributions (`1.0` = pure semantic, `0.0` = pure BM25). -5. **Enrich.** When `graph_context: true`, traverse the [context graph](/essentials/v2/context-graphs) and attach related paths. When `mode: "thinking"`, expand the query, rerank, and pull in the relations items declared at ingest. +5. **Enrich.** With `graph_context` on (the default), traverse the [context graph](/essentials/v2/context-graphs) and attach related paths. When `mode: "thinking"`, expand the query, rerank, and pull in the relations items declared at ingest. 6. **Shape the response.** Return ranked `chunks`, graph paths in `graph`, declared links in `forceful_relations`, and `llm_prompt`, the same context as one prompt-ready string with citation labels. The response is *retrieved context*, not a final LLM answer. You inject `llm_prompt` into your own agent or model prompt; see [How to Use API Results](/essentials/v2/api-results). @@ -183,7 +179,7 @@ A short cheat sheet for the parameters you'll touch most often, and where each o | `custom_attributes` | [Ingest](/api-reference/v2/endpoint/ingest-context) | Free-form per-item fields. Stored with the item; not filterable. | | `attributes` | [Query](/api-reference/v2/endpoint/query) | Deterministic narrowing with operators (`$eq`, `$in`, `$gte`, `$and`, ...) on declared attributes. | | `alpha` | [Query](/api-reference/v2/endpoint/query) | Blends semantic vs. keyword bm25 scores in `query_by: "hybrid"`. | -| `graph_context` | [Query](/api-reference/v2/endpoint/query) | When `true`, attaches the relevant slice of the [context graph](/essentials/v2/context-graphs) to the response. | +| `graph_context` | [Query](/api-reference/v2/endpoint/query) | On by default: returns relation paths from the [context graph](/essentials/v2/context-graphs) in `graph[]`. | | `mode` | [Query](/api-reference/v2/endpoint/query) | `"fast"` for low-latency single-pass retrieval; `"thinking"` for multi-query expansion and reranking. | --- @@ -202,9 +198,9 @@ This order catches almost every "I uploaded but query returns nothing" debugging ## Related -- [Core Concepts](/get-started/core-concepts) - the five primitives: Memories, Knowledge, Query, Databases, Metadata +- [Core Concepts](/get-started/v2/core-concepts): the primitives, from databases and collections to the context graph - [Quickstart](/get-started/v2/quickstart) - build your first integration in five minutes -- [Knowledge](/essentials/v2/knowledge) and [Memories](/essentials/v2/memories) - the two content models +- [Ingest context](/essentials/v2/ingest): one database of context items, text or conversations - [Query](/essentials/v2/query) - deep dive on `POST /query` - [Multi-Tenant Support](/essentials/v2/databases-and-collections) - scoping patterns and pitfalls - [Context Graphs](/essentials/v2/context-graphs) - how the graph layer enriches retrieval diff --git a/essentials/v2/attributes.mdx b/essentials/v2/attributes.mdx index 750faa7c..3f456bd7 100644 --- a/essentials/v2/attributes.mdx +++ b/essentials/v2/attributes.mdx @@ -1,199 +1,64 @@ --- title: "Attributes" -description: "Declared, filterable attributes versus free-form custom attributes, how the database schema defines them, and how query filters run." +description: "Declare filterable attributes in the database schema, attach attributes and custom attributes to items at ingest, and filter queries with the attributes operator language." --- - -**Names on a unified database.** An item carries `attributes` (declared, filterable; keys from `database_metadata_schema`) and `custom_attributes` (free-form, never filterable). A query filters with `attributes`, an operator object (`$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$and`, `$or`, `$not`, `$exists`); see [Query](/essentials/v2/query#2-request). Neither is returned on query chunks; read them with `GET /context/inspect`. The `metadata`, `additional_metadata` and `metadata_filters` names used below are the split-database and deprecated spellings of the same three things: the schema rules, size caps and filter semantics are the same. - +Attributes are structured values you attach to each item you ingest. Use them when you already know a hard constraint before retrieval runs, such as `department=legal`, `region=us`, `status=published` or `priority >= 5`. -Metadata is structured data attached to every item you ingest. Use it when you already know a hard constraint before retrieval runs, such as `department=legal`, `region=us`, `status=published`, or `author=alice`. +An item carries two kinds: -HydraDB has two metadata layers: +| Kind | Sent on an item as | Declared in the schema | Filterable at query time | Stored cap per item | +| --- | --- | --- | --- | --- | +| Attributes | `attributes` | Yes, in `database_metadata_schema` | Yes, with `attributes` on `POST /query` | 16 KiB | +| Custom attributes | `custom_attributes` | No | Never | 1 KiB | -| Layer | Sent at ingest as | Query filter shape | Best for | -| --- | --- | --- | --- | -| Tenant metadata | `metadata` | Top-level keys in `metadata_filters` | Stable, high-cardinality fields you filter on often. Declare these in `database_metadata_schema`, usually with `enable_match: true`. | -| Additional metadata | `additional_metadata` | Nested under `metadata_filters.additional_metadata` | Free-form per-source fields for display, citations, debugging, external IDs, or occasional filters. | - -If a field will be scoped on **every** query, it belongs in `metadata` and the [database schema](/api-reference/v2/endpoint/create-tenant). If it's ad-hoc or unique to one document, use `additional_metadata` instead. +If a field scopes your queries, declare it in the [database schema](/api-reference/v2/endpoint/create-tenant) and send it in `attributes`. If it is there for display, citations, debugging or an external ID, send it in `custom_attributes`. ```json { - "metadata_filters": { - "department": "legal", - "additional_metadata": { - "author": "alice" - } - } + "context_id": "auth-controls-001", + "text": "Authentication controls are reviewed quarterly for SOC2.", + "attributes": { "department": "security", "priority": 7 }, + "custom_attributes": { "author": "alice" } } ``` ---- - -## 1. Choose the right metadata layer - -| I want to… | Put it in… | Filter with… | Notes | -| --- | --- | --- | --- | -| Scope most queries by a field like department, region, plan, customer, or status | `metadata` | `metadata_filters: { "department": "legal" }` | Declare the field in `database_metadata_schema`; set `enable_match: true` for the intended filter path. | -| Store source-specific fields like author, Slack timestamp, external ID, document version | `additional_metadata` | `metadata_filters: { "additional_metadata": { "author": "alice" } }` | No schema required. Better for occasional filters and display metadata. | -| Combine hard scoping with semantic search | Both | `metadata_filters` plus your natural-language `query` | Filters narrow candidates; ranking still uses the query. | -| Search semantically over a metadata text field | `metadata` with `enable_dense_embedding: true` | Put the desired concept in `query` | Only supported for `VARCHAR` fields. Do not put fuzzy concepts in `metadata_filters`. | -| Search by keyword over a metadata text field | `metadata` with `enable_sparse_embedding: true` | Use normal `/query` text/BM25 behavior | Only supported for `VARCHAR` fields. | -| Partition by user, workspace, or team | `collection` | Send `collection` on every request | Use metadata filters inside that partition, not as a replacement for it. | - -The `database` field was formerly `tenant_id` and `collection` was formerly `sub_tenant_id`; the old names still work as deprecated aliases. [Follow this for when to use `database` and `collection`](./databases-and-collections#2-when-to-use-each). - ---- - -## 2. How metadata filters run - -For user-provided `metadata_filters`, HydraDB uses a correctness-first pipeline: - -1. **Source prefilter.** Safe scalar tenant/additional metadata filters are first resolved to the matching `source_id`s. -2. **Scoped retrieval.** Vector/BM25 retrieval searches only those source IDs. -3. **Post-filter correctness net.** Hydrated chunks are checked again against the requested metadata. This also protects graph expansion and fallback/retry paths from leaking excluded sources. - -That means a valid filter with no matching sources returns an empty result set; HydraDB does not silently widen it into an unfiltered search. - -| Field | Type | Purpose | -| --- | --- | --- | -| `metadata` | object for knowledge; JSON-encoded string for memory items | Database-schema fields. Keys must match `database_metadata_schema`. The fast scoping path. | -| `additional_metadata` | object | Free-form per-document or per-memory fields. No schema. The flexible, slower path. | - -### Filter semantics - -| Behavior | Contract | -| --- | --- | -| Multiple keys | AND logic. Every provided key/value must match. | -| Database + additional metadata | AND logic across both layers. | -| Operators | Each `metadata` (top-level key) takes `{"equals": value}`, `{"contains": value}`, or `{"contains_any": [values]}`. Not available inside `additional_metadata` - use the bare forms there. `contains` and `contains_any` need a `VARCHAR` field; `equals` works on any type. See [Filter operators](#filter-operators). | -| Values | `equals` is exact equality against the **whole** stored value. | -| Lists | **ANY, not ALL.** A list matches a source holding **any one** of the listed values (OR/IN), so `tags: ["alpha", "beta"]` matches a source tagged `"alpha"` and a source tagged `"beta"`. Write it as `{"contains_any": [...]}` in `metadata` (top-level key), or as a bare list in `additional_metadata`. Adding values widens the result set; there is no ALL/AND operator inside a single key. Lists are supported on `VARCHAR` fields only - a list passed for a declared field of another type is rejected with `400 VALIDATION_ERROR`. | -| Range / fuzzy | Not supported in user `metadata_filters`. Put fuzzy concepts in `query`, use semantic metadata fields, or post-process client-side. | -| OR | Within one key, use `contains_any`. Across different keys, run multiple queries and union client-side. | -| Graph context | Respects metadata filters; graph paths from excluded sources are removed. | -| Size | Each list holds at most **500** values, and the whole `metadata_filters` object is capped at **64 KiB**. See [Filter size limits](#filter-size-limits). | - - - `metadata_filters` are hard constraints, not semantic hints. A filter like `{ "mood": "happy" }` requires an exact stored value; it does not expand to related values like "joyful" or "cheerful". To search metadata text semantically, declare a `VARCHAR` field with `enable_dense_embedding` and include the concept in the main `query`. - - -### Filter operators - -Each `metadata` (top-level) key in `metadata_filters` takes an operator object naming the comparison you want. - -| Operator | Value | Matches | -| --- | --- | --- | -| `equals` | a single value | sources whose field is **exactly** that value | -| `contains` | a single value | sources whose field **holds** that value, at any position in a multi-value field | -| `contains_any` | an array | sources holding **any one** of the listed values (OR/IN) | +A query then filters on the declared fields: ```json { - "metadata_filters": { - "department": { "equals": "legal" }, - "attendee_emails": { "contains": "b@company.com" }, - "tags": { "contains_any": ["alpha", "beta"] } - } -} -``` - -Given four sources with these `emails` values: - -| Source | `emails` | -| --- | --- | -| A | `a@company.com` | -| B | `a@company.com,b@company.com` | -| C | `c@company.com` | -| D | `x@company.com,a@company.com,z@company.com` | - -| Filter | Returns | -| --- | --- | -| `{"equals": "a@company.com"}` | A | -| `{"contains": "a@company.com"}` | A, B, D | -| `{"contains_any": ["a@company.com", "c@company.com"]}` | A, B, C, D | - -`contains` is position-independent: it matches whether the value is the only one, the first, or in the middle. - -Operators apply to `metadata` (top-level keys) only. Inside `additional_metadata`, use a bare scalar for an exact match or a bare array to match any listed value. - - - An operator used inside `additional_metadata` is **not** rejected. It is read as an exact-match filter against a stored object, so on a normal field it matches nothing and the request returns `200` with an empty result rather than an error. - - -A known operator given the wrong operand type, or several operators in one object, is rejected with `400 VALIDATION_ERROR` rather than silently returning no results. A **misspelled** operator is not: `{"contian": "x"}` is indistinguishable from a filter for a stored object with that key, so it is left alone and matches nothing. - - - **`contains`, `contains_any` and `equals` are reserved key names.** An object built only from them is read as an operator, so it can no longer be used to exact-match a stored object: - - | Filter value | Read as | - | --- | --- | - | `{"contains": "x"}` | the `contains` operator | - | `{"contains": "a", "equals": "b"}` | rejected with `400` - every key is an operator name | - | `{"contains": "a", "other": 1}` | an exact-match object filter, unchanged | - - This only affects a `JSON`-typed field storing an object whose keys are all drawn from those three words. If you need to match such an object, rename the nested key or the field. - - - - The bare forms still work and are unchanged, but are **deprecated** in favour of the operators, because the comparison they perform is inferred from the JSON shape rather than stated. A bare scalar behaves as `equals`, a bare array as `contains_any`, and a bare single-element array as `contains` - so `{"emails": "a@x"}` and `{"emails": ["a@x"]}` differ by one character and return different results. - - -### Storing multiple values in one field - -A declared schema field holds a single value. To store several values in one field, declare it as `VARCHAR` and join the values with commas: - -```json -{ - "metadata": { - "attendee_emails": "a@company.com,b@company.com" - } -} -``` - -Then filter for one member with `contains`: - -```json -{ - "metadata_filters": { - "attendee_emails": { "contains": "b@company.com" } + "database": "acme_corp", + "collection": "company", + "query": "How do access control reviews work?", + "attributes": { + "department": "security", + "priority": { "$gte": 5 } } } ``` -Points to know: - -- `data_type: "array"` is **not** accepted on a declared field. Declaring one is rejected with `400`. -- Sending an array **value** for a declared `VARCHAR` field is also rejected: `metadata field "attendee_emails" must be of type string, got array`. Join the values yourself. -- The comma is the separator, so a value that itself contains a comma will not match as expected. Use a field per value, or a different value format, if your values can contain commas. -- Size the field for the whole joined string. `max_length` defaults to `1024` and its maximum is `65535`, and it **cannot be raised after the field is created**, so declare it large enough up front. -- `equals` compares the entire joined string, so it is rarely what you want on a multi-value field. Use `contains`. - --- -## 3. Define tenant metadata schema +## 1. Choose where a field belongs -Most of the time the defaults are right. When they aren't, here's where to start: +| I want to... | Put it in... | Then... | +| --- | --- | --- | +| Scope most queries by a field like department, region, plan, customer or status | `attributes` | Declare the field in `database_metadata_schema` and filter with `"attributes": { "department": "legal" }`. | +| Filter by a number or a date range | `attributes` | Store a number, or a date string in one fixed format such as `YYYY-MM-DD`, and filter with `$gt`, `$gte`, `$lt`, `$lte`. | +| Keep source details like author, a Slack timestamp, an external ID or a document version | `custom_attributes` | No schema needed. Stored with the item, never filterable. | +| Combine a hard scope with semantic search | `attributes` | Send the filter plus your natural-language `query`. The filter narrows the candidates; ranking still uses the query. | +| Search semantically over a text attribute | `attributes`, on a `VARCHAR` field with `enable_dense_embedding: true` | Put the concept in `query`. Do not put fuzzy concepts in the filter. | +| Search by keyword over a text attribute | `attributes`, on a `VARCHAR` field with `enable_sparse_embedding: true` | Normal `/query` keyword (BM25) matching covers it. | +| Partition by user, workspace or team | `collection` | Send `collection` on every request, and filter with `attributes` inside that partition, not as a replacement for it. | +| Restrict who may retrieve an item | `acl` | An attribute filter is not a permission. See [Access control](/essentials/v2/access-control). | -- **Plan scoping fields before first ingest.** Schema is immutable, and undeclared scope keys are silently ignored at query time. If you'll scope on it more than once, declare it. -- **Pick `metadata` for hot paths, `additional_metadata` for cold ones.** Database-level scopes are pre-applied in the vector store; document-level scopes force a post-retrieval pass with over-fetch. -- **Know which comparisons exist.** `contains` and `contains_any` are supported on `metadata` (top-level) keys only, and need a `VARCHAR` field; everywhere else, and for every type, filtering is exact equality, where a scalar is an exact match and a list matches any one of its values. See [Filter operators](#filter-operators). Range and fuzzy matching are still not filter operators - put those in the query, query mode, or downstream reranking. -- **Keep keys stable.** Renaming a metadata key requires re-ingesting affected sources. Same goes for changing `enable_match` flags. -- **Ingested metadata is immutable.** Once a source is indexed, its `metadata` and `additional_metadata` values are locked. To change them, re-ingest the source with the new values (use `upsert: true` and the same `id`). -- **Don't substitute `metadata_filters` for `collection`.** `metadata_filters` scopes results *inside* a partition. For partitioning by user, team, or workspace, use [`collection`](/essentials/v2/databases-and-collections). +The `database` field was formerly `tenant_id` and `collection` was formerly `sub_tenant_id`; the old names still work as deprecated aliases. See [when to use `database` and `collection`](/essentials/v2/databases-and-collections#2-when-to-use-each). --- -## 4. Minimal working example - -Two phases: **set up metadata** (declare the schema, then attach values at ingest), then **scope at query**. +## 2. Declare the schema -### Step 1 - Create metadata - -The schema lives at the database level; values land on each source at ingest time. Both happen before any query. - -#### Step 1a: Declare the schema at database creation +The schema lives on the database; values land on each item at ingest. Declare the schema when you create the database, before any ingest. ```bash cURL @@ -209,6 +74,11 @@ curl -X POST 'https://api.hydradb.com/databases' \ "data_type": "VARCHAR", "enable_match": true }, + { + "name": "region", + "data_type": "VARCHAR", + "enable_match": true + }, { "name": "priority", "data_type": "INT64", @@ -228,6 +98,7 @@ client.databases.create( database="acme_corp", database_metadata_schema=[ {"name": "department", "data_type": "VARCHAR", "enable_match": True}, + {"name": "region", "data_type": "VARCHAR", "enable_match": True}, {"name": "priority", "data_type": "INT64", "enable_match": True}, { "name": "summary_label", @@ -243,6 +114,7 @@ await client.databases.create({ database: "acme_corp", databaseMetadataSchema: [ { name: "department", dataType: "VARCHAR", enableMatch: true }, + { name: "region", dataType: "VARCHAR", enableMatch: true }, { name: "priority", dataType: "INT64", enableMatch: true }, { name: "summary_label", @@ -259,238 +131,175 @@ await client.databases.create({ | Field | Type / values | Purpose | | --- | --- | --- | -| `name` | string | Metadata key. Must start with a letter or `_`, contain only letters/numbers/underscores, and not use reserved system names such as `source_id`, `chunk_id`, or `metadata`. | -| `data_type` | `VARCHAR`, `BOOL`, `INT8`, `INT16`, `INT32`, `INT64`, `FLOAT`, `DOUBLE`, `JSON` or friendly aliases `string`, `boolean`, `integer`, `float`, `object` | Defaults to `VARCHAR`. `ARRAY` is **not** supported and is rejected with `400`; for multi-value fields see [Storing multiple values in one field](#storing-multiple-values-in-one-field). | -| `max_length` | integer | Max length for `VARCHAR`. Default `1024`, maximum `65535`. This sizes one declared schema field, and is **not** the limit on how much metadata you can send per request - for that see [Size limits](#size-limits). | -| `enable_match` | boolean | Creates the intended fast exact-match/filtering path for this field. Use this for top-level `metadata_filters` keys. | -| `enable_dense_embedding` | boolean | Adds dense semantic search for a `VARCHAR` metadata field. | -| `enable_sparse_embedding` | boolean | Adds sparse/BM25 keyword search for a `VARCHAR` metadata field. | -| `filterable` | boolean | Backward-compatible shorthand for `enable_match: true`. Prefer `enable_match` in new docs/code. | -| `searchable` | boolean | Backward-compatible input accepted by the API, but do not rely on it to replace explicit `enable_dense_embedding` / `enable_sparse_embedding`. Set those flags directly. | +| `name` | string | The attribute key. Must start with a letter and contain only letters, numbers and underscores, at most 255 characters. Reserved system names such as `chunk_id`, `source_id`, `source_title` and `description` are rejected; the error lists every reserved name. | +| `data_type` | `VARCHAR`, `BOOL`, `INT8`, `INT16`, `INT32`, `INT64`, `FLOAT`, `DOUBLE`, `JSON`, or the friendly aliases `string`, `boolean`, `integer`, `float`, `object` | Defaults to `VARCHAR`. `array` is **not** supported and is rejected with `400`; see [No containment on multi-valued fields](#no-containment-on-multi-valued-fields). | +| `max_length` | integer | Maximum length of a `VARCHAR` value. Default `1024`, maximum `65535`. It sizes one field and cannot be raised later, so declare it large enough up front. It is **not** the per-item budget for all attributes; for that see [Size limits](#size-limits). | +| `enable_match` | boolean | Turns on keyword matching (a text analyzer) for the field. An `attributes` filter works on every declared field whether or not this is set. | +| `enable_dense_embedding` | boolean | Adds dense semantic search over a `VARCHAR` field. | +| `enable_sparse_embedding` | boolean | Adds sparse (BM25) keyword search over a `VARCHAR` field. | +| `filterable` | boolean | Backward-compatible shorthand for `enable_match: true`. Prefer `enable_match` in new code. | +| `searchable` | boolean | Accepted for backward compatibility, but it does not turn on either embedding. Set `enable_dense_embedding` / `enable_sparse_embedding` directly. | Limits and guardrails: -- Up to 32 custom tenant metadata fields. -- Up to **6 embedding-enabled** fields per database. `enable_dense_embedding` and `enable_sparse_embedding` each count as one, so a field with both set counts as two. `enable_match` does not count against this limit - only the embedding flags do. Exceeding it fails database creation with `400` before anything is provisioned. +- Up to **32** declared fields per database. +- Up to **6 embedding-enabled** fields per database. `enable_dense_embedding` and `enable_sparse_embedding` each count as one, so a field with both set counts as two. `enable_match` does not count. Exceeding it fails database creation with `400` before anything is provisioned. - Field names are unique case-insensitively. -- Dense/sparse embedding flags are only valid on `VARCHAR` fields. -- Runtime `metadata` values must match the declared type when the tenant has a schema. -- Unknown `metadata` keys are rejected on ingest/edit when a non-empty tenant schema exists. -- Each metadata layer has a byte budget per request - see [Size limits](#size-limits). - -### Size limits - -Every request that attaches metadata is checked against two caps, on ingest and on -metadata edit alike: +- Dense and sparse embedding flags are only valid on `VARCHAR` fields. +- A declared field cannot be renamed, retyped, resized or have its flags changed after it is created. -| Layer | Send as | Cap | -| --- | --- | --- | -| Database (tenant) metadata | `metadata` at ingest, `database_metadata` on edit | **16 KiB** (16,384 bytes) | -| Document metadata | `additional_metadata` | **1 KiB** (1,024 bytes) | - -Older spellings are still accepted, but **not uniformly** - which one works -depends on the endpoint: - -| Alias | On `/context/ingest` | On `PATCH /context/{id}/metadata` | -| --- | --- | --- | -| `tenant_metadata` | accepted | accepted (`database_metadata` wins if both are sent) | -| `document_metadata` | accepted | **rejected with `400`** - send `additional_metadata` | - -Where an alias is accepted it is held to exactly the same cap as the canonical -field. Use the canonical names above and this never comes up. - -The cap applies to the **whole map**, not to any one value, and it is measured on -the map's compact JSON encoding in UTF-8 bytes. Three consequences worth planning -around: - -- **Keys and punctuation count.** Quotes, colons, commas and braces are all part of - the payload you are billed for. -- **Bytes, not characters.** Accented Latin characters cost 2 bytes, most CJK - characters 3, and emoji 4. -- **Budget in bytes from the start.** A 950-character summary sounds comfortably - under a 1 KiB cap, but with two small sibling keys it serializes to 1,015 - bytes - 65 bytes of that is structure alone. Push the summary to 1,000 - characters and the request is rejected at 1,065 bytes. - -```json Document metadata: 1,015 bytes, just inside the 1 KiB cap -{"title":"Q3 Board Deck","author":"ada@example.com","summary":"<950 characters>"} -``` +### Add schema fields later -Exceeding either cap fails the whole request with `400` before anything is -ingested. The message names the offending field and reports both numbers, so you -can see exactly how far over you are: +Add fields to an existing database with [`PATCH /databases/{database}/metadata-schema`](/api-reference/v2/endpoint/update-metadata-schema): ```json { - "success": false, - "data": null, - "error": { - "code": "INVALID_INPUT", - "message": "additional_metadata is too large (1065 bytes when serialized; the maximum is 1024). Reduce the number or size of metadata fields." - } + "add_fields": [ + { "name": "status", "data_type": "VARCHAR", "enable_match": true } + ] } ``` -On [`PATCH /context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) -the same message is prefixed with `invalid metadata edit:`. - - - If a document needs more than 1 KiB of descriptive metadata, put the long text - in the document body where it gets chunked and embedded, and keep - `additional_metadata` for the short values you actually filter on. - +The update is additive only: -### Filter size limits +- Add new fields: yes. +- Delete fields: no. +- Change the type, `max_length` or flags of an existing field: no, that is a `409`. Resubmitting an existing field with its exact definition is a no-op, so you can safely send your whole field list. +- `enable_dense_embedding` and `enable_sparse_embedding` can only be set when the database is created. A new field carrying either flag is rejected with `400`. +- The 32-field cap counts existing and new fields together. -The caps above bound the metadata you **store**. `metadata_filters` on -[`/query`](/api-reference/v2/endpoint/query) has its own, separate pair - these -bound what you **send at query time** and are unrelated to how much metadata a -source carries: - -| Limit | Cap | -| --- | --- | -| Values in any one list | **500** | -| Whole `metadata_filters` object | **64 KiB** (65,536 bytes) | - -Measured the same way - compact JSON, UTF-8 bytes, field names and punctuation -counted - and the object total includes the nested `additional_metadata` dict. - -The object total is measured **after** operator objects are reduced to their -values, so `{"contains": "x"}` counts as `["x"]` and the operator keyword itself -costs nothing. Both spellings of the same filter cost the same, because the cap -bounds the expression sent to the vector store, which the spelling does not -change. - -Both exist because every value in a list is expanded into the filter expression -sent to the vector store. The per-list cap catches one runaway list; the object -cap catches many individually-legal lists adding up. Twenty lists of 500 values -are each within the element cap but total roughly 127 KiB, so the object cap is -what rejects them. - -Over either limit returns `400` before the query runs, naming the offending key -or the actual byte count: - -``` -metadata_filters.customer_id (got 743) must contain at most 500 values - -metadata_filters is too large (130251 bytes when serialized; the maximum is 65536). -Reduce the number or size of filter values. -``` - - - Needing far more than 500 values in one filter usually means the constraint - belongs in the data rather than the query - add a metadata field that groups - those values (a segment, tier, or cohort key) and filter on that instead. - - -### Add schema fields later - -You can add database metadata fields after database creation with [`PATCH /databases/{database}/metadata-schema`](/api-reference/v2/endpoint/update-metadata-schema). This is additive only: - -- add new fields: yes -- delete fields: no -- change type/flags of existing fields: no +`GET /databases/{database}/metadata-schema` returns the current `fields` in the same shape `add_fields` accepts. - Adding fields updates the stored database schema and its filter indexes. Data already ingested is not re-indexed for newly added dense/sparse metadata fields yet. If you need a new metadata field to participate in semantic/BM25 metadata search for existing data, create a new database/schema and re-ingest, or confirm the current platform migration path with support. + Items ingested before a field existed have no value for it, so a comparison on the new field does not match them. Re-ingest those items with the value to include them. --- -## 5. Attach metadata at ingest +## 3. Attach attributes at ingest -For knowledge ingestion, send `metadata` and `additional_metadata` on each `document_metadata[]` item or `app_knowledge[]` item. +Send `attributes` and `custom_attributes` on each item in `context` on [`POST /context/ingest`](/essentials/v2/ingest). The SDKs send the same item array, as a JSON string, in the `items` form field; keys inside each item stay snake_case in every language. ```bash cURL curl -X POST 'https://api.hydradb.com/context/ingest' \ -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ -H "API-Version: 2" \ - -F "type=knowledge" \ - -F "database=acme_corp" \ - -F 'app_knowledge=[ - { - "database": "acme_corp", - "collection": "default", - "id": "auth-controls-001", - "kind": "knowledge_base", - "fields": { + -H "Content-Type: application/json" \ + -d '{ + "database": "acme_corp", + "collection": "company", + "context": [ + { + "context_id": "auth-controls-001", "title": "Authentication controls", - "body": "Authentication controls are reviewed quarterly for SOC2." - }, - "metadata": { - "department": "security", - "priority": 7, - "summary_label": "quarterly access control review" - }, - "additional_metadata": { - "author": "alice", - "doc_version": 3 + "text": "Authentication controls are reviewed quarterly for SOC2.", + "attributes": { + "department": "security", + "region": "us", + "priority": 7, + "summary_label": "quarterly access control review" + }, + "custom_attributes": { + "author": "alice", + "doc_version": 3 + } } - } - ]' + ] + }' ``` ```python Python SDK import json client.context.ingest( - type="knowledge", database="acme_corp", - app_knowledge=json.dumps([ + collection="company", + items=json.dumps([ { - "database": "acme_corp", - "collection": "default", - "id": "auth-controls-001", - "kind": "knowledge_base", - "fields": { - "title": "Authentication controls", - "body": "Authentication controls are reviewed quarterly for SOC2.", - }, - "metadata": { + "context_id": "auth-controls-001", + "title": "Authentication controls", + "text": "Authentication controls are reviewed quarterly for SOC2.", + "attributes": { "department": "security", + "region": "us", "priority": 7, "summary_label": "quarterly access control review", }, - "additional_metadata": {"author": "alice", "doc_version": 3}, + "custom_attributes": {"author": "alice", "doc_version": 3}, } ]), ) ``` ```typescript TypeScript SDK await client.context.ingest({ - type: "knowledge", database: "acme_corp", - appKnowledge: JSON.stringify([ + collection: "company", + items: JSON.stringify([ { - database: "acme_corp", - collection: "default", - id: "auth-controls-001", - kind: "knowledge_base", - fields: { - title: "Authentication controls", - body: "Authentication controls are reviewed quarterly for SOC2.", - }, - metadata: { + context_id: "auth-controls-001", + title: "Authentication controls", + text: "Authentication controls are reviewed quarterly for SOC2.", + attributes: { department: "security", + region: "us", priority: 7, summary_label: "quarterly access control review", }, - additional_metadata: { author: "alice", doc_version: 3 }, + custom_attributes: { author: "alice", doc_version: 3 }, }, ]), }); ``` - - For `type=memory`, the `memories` multipart field is already JSON-stringified, and each memory item's `metadata` is currently validated as a JSON-encoded string. Keep `additional_metadata` as an object. For knowledge ingestion (`document_metadata` and `app_knowledge`), `metadata` is an object. - +Rules checked before anything is queued: + +- When the database has a schema, every `attributes` key must be declared in it, and every value must match the declared type: a string for `VARCHAR`, `true` or `false` for `BOOL`, a whole number for the integer types, a number for `FLOAT` and `DOUBLE`, an object for `JSON`. `null` is accepted for any declared field. An undeclared key or a wrong type rejects the request with `400`. +- `custom_attributes` take any keys, with no schema. +- In both maps, keys must not start with `_`, must not be a reserved system name, and must not contain control characters. A value may be a scalar, a list or an object, but not a list or object nested inside another. +- Errors name the item they refer to, such as `context[0]: ...`. + +**Attributes are set at ingest.** The `attributes` query filter runs against the values indexed with the item. To change them, re-ingest the item with `upsert: true` and the same `context_id`, which replaces the item. See [IDs and replacement](/essentials/v2/ingest#13-ids-and-replacement). + +### Size limits + +Every item is checked against two caps: + +| Map | Cap | +| --- | --- | +| `attributes` | **16 KiB** (16,384 bytes) | +| `custom_attributes` | **1 KiB** (1,024 bytes) | + +The same two caps apply to [`PATCH /context/{id}/metadata`](#5-edit-values-in-place). + +The cap applies to the **whole map**, not to any one value, and it is measured on the map's compact JSON encoding in UTF-8 bytes. Three consequences worth planning around: + +- **Keys and punctuation count.** Quotes, colons, commas and braces are all part of the payload that is measured. +- **Bytes, not characters.** Accented Latin characters cost 2 bytes, most CJK characters 3, and emoji 4. +- **Budget in bytes from the start.** A 950-character summary sounds comfortably under a 1 KiB cap, but with two small sibling keys it serializes to 1,014 bytes: 64 bytes of that is structure alone. Push the summary to 1,000 characters and the request is rejected at 1,064 bytes. + +```json custom_attributes: 1,014 bytes, just inside the 1 KiB cap +{"deck":"Q3 Board Deck","author":"ada@example.com","summary":"<950 characters>"} +``` + +Exceeding either cap fails the whole request with `400 INVALID_INPUT` before anything is ingested. The message names the item and the map, and reports both numbers, so you can see exactly how far over you are: + +```json +{ + "code": "INVALID_INPUT", + "message": "context[0]: custom_attributes is too large (1064 bytes when serialized; the maximum is 1024). Reduce the number or size of metadata fields." +} +``` + + + If an item needs more than 1 KiB of descriptive detail, put the long text in the item's `text`, where it gets chunked and embedded, and keep `custom_attributes` for short values. + --- -## 6. Query with metadata filters +## 4. Filter a query with `attributes` -Mix database-level (top-level) and document-level (nested) scopes in the same `metadata_filters` object: +`attributes` on [`POST /query`](/essentials/v2/query#2-request) is an operator object over the declared fields. Several fields in one object must all match. ```bash cURL @@ -500,72 +309,163 @@ curl -X POST 'https://api.hydradb.com/query' \ -H "Content-Type: application/json" \ -d '{ "database": "acme_corp", - "collection": "default", + "collection": "company", "query": "How do access control reviews work?", - "type": "knowledge", - "query_by": "hybrid", - "metadata_filters": { + "attributes": { "department": "security", - "priority": 7, - "additional_metadata": { - "author": "alice" - } + "region": { "$in": ["us", "eu"] }, + "priority": { "$gte": 5 } } }' ``` ```python Python SDK result = client.query( database="acme_corp", - collection="default", + collection="company", query="How do access control reviews work?", - type="knowledge", - query_by="hybrid", - metadata_filters={ + attributes={ "department": "security", - "priority": 7, - "additional_metadata": {"author": "alice"}, + "region": {"$in": ["us", "eu"]}, + "priority": {"$gte": 5}, }, ) ``` ```typescript TypeScript SDK const result = await client.query({ database: "acme_corp", - collection: "default", + collection: "company", query: "How do access control reviews work?", - type: "knowledge", - queryBy: "hybrid", - metadataFilters: { + attributes: { department: "security", - priority: 7, - additional_metadata: { author: "alice" }, + region: { $in: ["us", "eu"] }, + priority: { $gte: 5 }, }, }); ``` -Use the legacy alias only when maintaining older clients: +Query chunks do not carry attributes. The filter decides which items can appear; see [Query](/essentials/v2/query) for what the response contains. + +### Operators + +| Operator | Operand | Matches items whose value... | +| --- | --- | --- | +| a bare value | a value of the field's type | equals it. `{"department": "legal"}` is the same as `{"department": {"$eq": "legal"}}`. | +| `$eq` | a value of the field's type | equals it. | +| `$ne` | a value of the field's type | is present and differs from it. | +| `$gt`, `$gte`, `$lt`, `$lte` | a value of the field's type | is greater than, at least, less than, or at most the operand. Numbers compare numerically; `VARCHAR` values compare as strings, character by character. | +| `$in` | a non-empty array, at most 500 values | equals any one of the listed values. | +| `$nin` | a non-empty array, at most 500 values | is present and equals none of the listed values. | +| `$exists` | `true` or `false` | has a value (`true`), or has none (`false`). | +| `$and` | a non-empty array of filter objects | matches every one of them. | +| `$or` | a non-empty array of filter objects | matches at least one of them. | +| `$not` | one filter object | does not match it, and has a value for every field it names. | + +Operators combine and nest: ```json { - "metadata_filters": { - "document_metadata": { "author": "alice" } + "attributes": { + "$or": [ + { "department": "legal" }, + { + "$and": [ + { "department": "security" }, + { "priority": { "$gte": 8 } } + ] + } + ], + "$not": { "region": "cn" }, + "priority": { "$gte": 3, "$lte": 9 } } } ``` -If both aliases are present and both are objects, `additional_metadata` wins on conflicts. +### How the filter behaves + +| Behavior | Contract | +| --- | --- | +| Several fields in one object | AND. Every clause must match. | +| Several operators on one field | AND. `{"priority": {"$gte": 3, "$lte": 7}}` is a range. | +| Equality | Exact, against the **whole** stored value. Strings are case-sensitive. | +| Value types | Every operand must match the field's declared type: a string for `VARCHAR`, `true` or `false` for `BOOL`, a whole number for the integer types, a number for `FLOAT` and `DOUBLE`. `{"priority": "7"}` on an `INT64` field is a `400`, not an empty result. | +| `JSON` fields | Only `$exists` applies. Any other operator on a `JSON` field is a `400`. | +| Missing values | An item with no value for a field never matches a comparison on that field. `$ne`, `$nin` and `$not` exclude it too. To keep such items, say so: `{"$or": [{"region": {"$ne": "eu"}}, {"region": {"$exists": false}}]}`. | +| Field names | Must be declared in `database_metadata_schema`. An undeclared field is a `400` (`unknown attribute`), never silently ignored. A reserved system column is a `400`. On a database created without any schema, every field is compared as a string. | +| Custom attributes | Cannot be filtered. Naming the custom attributes namespace inside `attributes` is a `400`. | +| Empty pieces | An empty object, an empty operator object, or an empty `$and`, `$or`, `$in` or `$nin` array is a `400`, not a filter that matches everything. | +| Unknown operators | A `400`. There is no `$contains`, `$regex` or fuzzy operator. | +| Nesting | At most 10 levels deep through `$and`, `$or` and `$not`. | +| Graph and forceful relations | The filter applies to chunks, forceful relations and graph paths alike. A graph path that touches an item the filter excludes is removed. | +| No matches | A valid filter that matches nothing returns an empty result. HydraDB never drops or widens the filter to find something. | +| Size | At most 500 values in each `$in` or `$nin` list, and 64 KiB for the whole object. See [Filter size limits](#filter-size-limits). | + + + `attributes` are hard constraints, not semantic hints. `{"mood": "happy"}` requires that exact stored value; it does not expand to "joyful" or "cheerful". To search attribute text semantically, declare a `VARCHAR` field with `enable_dense_embedding` and put the concept in the main `query`. + + +### No containment on multi-valued fields + +`attributes` compares an item's single stored value. There is no containment operator, so a multi-valued field cannot be matched by "does this item's list include X": + +- A declared field cannot be an array: `data_type: "array"` is rejected with `400`. +- A list sent as the value of a `VARCHAR` attribute is rejected at ingest. +- A `JSON` attribute cannot be compared at all (only `$exists` applies). +- A string that joins several values, such as `"alpha,beta"`, is one value: `$eq` and `$in` match it only as the whole string. + +`$in` runs the other way round: it asks whether the item's one value is among the values you list. + +If you need to select items by one member of a set: + +- Give each member you filter on its own `BOOL` attribute, such as `"tag_billing": true`, and filter with `{"tag_billing": true}`. This counts against the 32-field limit, so it suits a small, known set. +- If the set is really "who may see this item", use `acl` instead. See [Access control](/essentials/v2/access-control). + +### Filter size limits + +The [size limits](#size-limits) above bound the values you **store**. `attributes` on `/query` has its own, separate pair, which bound what you **send** at query time: + +| Limit | Cap | +| --- | --- | +| Values in one `$in` or `$nin` list | **500** | +| The whole `attributes` object | **64 KiB** (65,536 bytes) | + +The object total is measured on its compact JSON encoding in UTF-8 bytes, with field names, operator names and punctuation all counted. The per-list cap catches one runaway list; the object cap catches many individually legal lists adding up. + +Over either limit returns `400` before the query runs: + +``` +$in for "customer_id" must contain at most 500 values (got 743) + +attributes is too large (130251 bytes when serialized; the maximum is 65536). Reduce the number or size of filter values. +``` + + + Needing far more than 500 values in one filter usually means the constraint belongs in the data rather than the query. Add an attribute that groups those values (a segment, tier or cohort key) and filter on that instead. + + +### Errors + +Every malformed filter is a `400` with code `VALIDATION_ERROR` and a message naming the problem, for example: + +``` +unknown attribute "regoin" +value for "priority" does not match its type INT64 +$in for "region" expects an array +unsupported operator "$contains" for attribute "tags" +attributes filter nests deeper than 10 levels +``` --- -## 7. Update metadata without re-ingesting +## 5. Edit values in place -Use [`PATCH /context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) when you know the source ID and need to update metadata in place. +[`PATCH /context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) edits the stored values of one item you know the `context_id` of. This endpoint's body names the two maps `database_metadata`, which edits the item's `attributes`, and `additional_metadata`, which edits its `custom_attributes`: ```json { "database": "acme_corp", - "collection": "default", - "tenant_metadata": { + "collection": "company", + "database_metadata": { "department": "legal" }, "additional_metadata": { @@ -576,79 +476,69 @@ Use [`PATCH /context/{id}/metadata`](/api-reference/v2/endpoint/update-source-me Behavior: -- The source must already exist. +- The item must already exist. - `collection` is required. -- At least one of `tenant_metadata`, `additional_metadata`, or `acl` is required. -- The update is a merge/upsert: sent keys are inserted or overwritten; omitted keys are preserved. -- `document_metadata` is not accepted on this endpoint; use `additional_metadata`. -- The same endpoint accepts `acl` to change who may retrieve the source. Unlike metadata, `acl` **replaces** rather than merges, and an `acl`-only body is a valid edit. See [Access Control](/essentials/v2/access-control). -- `enable_match`-only tenant metadata updates take effect for filters/listing. -- If an edited tenant metadata field has `enable_dense_embedding` or `enable_sparse_embedding`, HydraDB synchronously updates the search index and reports `vector_sync_required` / `vector_synced` in the response (the `milvus_sync_required` / `milvus_synced` aliases are still emitted, deprecated). +- At least one of `database_metadata`, `additional_metadata` or `acl` is required. +- The update is a merge: sent keys are inserted or overwritten; omitted keys are preserved. +- `database_metadata` is checked against the schema exactly as `attributes` are at ingest, and both maps are held to the same [16 KiB and 1 KiB caps](#size-limits). A rejected edit's message is prefixed with `invalid metadata edit:`. +- The same endpoint accepts `acl` to change who may retrieve the item. Unlike the two maps, `acl` **replaces** rather than merges, and an `acl`-only body is a valid edit. See [Access control](/essentials/v2/access-control). +- If an edited attribute has `enable_dense_embedding` or `enable_sparse_embedding`, HydraDB updates its search index synchronously and reports `vector_sync_required` / `vector_synced` in the response. A `null` for such a field is rejected. -For full document/content replacement, re-ingest with `upsert: true` and the same source `id`. Upsert replaces the source payload and metadata supplied by ingestion. + + An edit here is not guaranteed to change what the `attributes` query filter sees, because the filter runs against the values indexed at ingest. To change a value you filter on, re-ingest the item with `upsert: true` and the same `context_id`. + --- -## 8. Listing with metadata filters +## 6. Browse items by attribute -Use [`POST /context/list`](/api-reference/v2/endpoint/list-documents) when you want to browse or page sources rather than run semantic retrieval: - -```json -{ - "database": "acme_corp", - "collection": "default", - "type": "knowledge", - "filters": { - "metadata": { "department": "legal" }, - "additional_metadata": { "author": "alice" }, - "source_fields": { "type": "slack" } - } -} -``` - -`/context/list` also accepts legacy aliases `tenant_metadata` for `metadata` and `document_metadata` for `additional_metadata`. +To page through items rather than run retrieval, use [`POST /context/list`](/api-reference/v2/endpoint/list-documents). Its `filters` object matches stored values exactly; the endpoint page documents its keys, `include_fields` and paging. --- -## 9. Common mistakes +## 7. Common mistakes | Symptom | Cause | Fix | | --- | --- | --- | -| `metadata_filters` doesn't scope results | Key isn't declared in `database_metadata_schema`, or `enable_match` is `false` | Re-create the database with the field declared (`enable_match: true`), or move the field to `additional_metadata` and nest the scope under `additional_metadata`. | -| Scope on an `additional_metadata` key silently ignored | Scope passed as top-level key | Nest it: `metadata_filters: { additional_metadata: { author: "alice" } }`. | -| Schema field changes don't take effect | Schema is immutable after database creation | Create a new database with the corrected schema. There's no in-place schema migration. | -| Metadata values on a source don't update | Ingested metadata is immutable | Re-ingest the source with `upsert: true` and the same `id`. | -| Query returns 0 results after adding a scope | Over-scoping - combined constraints exclude everything | Start with the minimum hard constraints; add scopes one at a time and re-check counts. | -| Range / contains / fuzzy scope doesn't work | `metadata_filters` is equality-only | Move that constraint into the `query` text, use a different `query_by`, or apply it in your own post-scoping. | -| Fields used in `metadata` but missing from response objects | Field declared as embedding-only without `enable_match` | Embedding-enabled doesn't imply scoping-enabled. Add `enable_match: true` to use it in `metadata_filters`. | -| `additional_metadata` scope feels slow | It is - over-fetches ~3× to compensate for post-retrieval matching | Move the hot field into `metadata` and declare it in the schema. | -| Metadata edit returns 400 | Unknown key, wrong type, over-size payload (16 KiB database metadata / 1 KiB document metadata), too-deep nesting, reserved key, or missing `collection` | Check schema and [size limits](#size-limits); send only `tenant_metadata` / `additional_metadata`. | -| Dense/sparse metadata edit rejects `null` | Null would leave stale semantic metadata vectors | Set a non-null value or re-ingest with the desired metadata. | -| New schema field does not participate in semantic metadata search | Additive schema update does not re-index data already ingested yet | Create the desired schema before ingesting, or migrate/re-ingest into a database with the final schema. | +| Query returns `400 unknown attribute` | The field is not declared in `database_metadata_schema` | Declare it with `PATCH /databases/{database}/metadata-schema`, then re-ingest the items that should carry it. | +| Ingest returns `400` naming an undeclared field | An `attributes` key is not in the schema | Declare the field, or move it to `custom_attributes` if you never filter on it. | +| A filter on a custom attribute is rejected | `custom_attributes` are never filterable | Declare the field, send it in `attributes`, and re-ingest. | +| `400 value for "priority" does not match its type` | The operand's JSON type differs from the declared type, such as `"7"` for an `INT64` field | Send the declared type: `{"priority": 7}`. | +| `$in` does not find an item whose field holds several values | There is no containment | See [No containment on multi-valued fields](#no-containment-on-multi-valued-fields). | +| `$ne` or `$not` drops items that have no value for the field | Missing values never match a comparison | Add `{"field": {"$exists": false}}` under `$or`. | +| Query returns 0 results after adding a filter | Over-scoping: the combined constraints exclude everything, or the items predate the field | Start with one constraint, add the others one at a time, and check with `$exists`. | +| A value edited with `PATCH /context/{id}/metadata` still filters as the old value | The filter runs against the values indexed at ingest | Re-ingest the item with `upsert: true` and the same `context_id`. | +| A schema field cannot be changed | Declared fields are immutable | Add a new field, or create a new database with the corrected schema and re-ingest. | +| Adding a field with `enable_dense_embedding` or `enable_sparse_embedding` returns `400` | Embedding flags can only be set at database creation | Create a new database with the final schema and re-ingest. | +| Ingest or edit returns `400 ... is too large` | Over the 16 KiB `attributes` or 1 KiB `custom_attributes` cap | Trim the map; move long text into the item's `text`. See [Size limits](#size-limits). | +| An edit returns `400` | Unknown key, wrong type, over-size map, too-deep nesting, reserved key, or missing `collection` | Check the schema and the [size limits](#size-limits). | +| A dense or sparse attribute edit rejects `null` | A null would leave stale search vectors | Set a non-null value, or re-ingest with the desired value. | --- -## 10. Advanced patterns +## 8. Advanced patterns + +**Stacked scopes with collection partitioning.** Use `collection` for the partition (per user, per workspace), and use `attributes` to scope *inside* that partition. They are complementary, not interchangeable. See [Databases and collections](/essentials/v2/databases-and-collections). -**Stacked scopes with collection partitioning.** Use `collection` for the partition (per-user, per-workspace), and use `metadata_filters` to scope *inside* that partition. They're complementary, not interchangeable. See [Multi-Tenant](/essentials/v2/databases-and-collections). +**Published versus draft.** Declare a `status` field; tag every item with `"attributes": { "status": "draft" }` or `"published"`; pass `"attributes": { "status": "published" }` on user-facing queries. Work in progress stays out of customer answers automatically. -**Published vs draft.** Add a `status` field with `enable_match: true` to your schema; tag every source with `metadata.status = "draft" | "published"`; pass `metadata_filters: { status: "published" }` on user-facing queries. Keeps work-in-progress out of customer answers automatically. +**Multi-language corpora.** Declare a `language` field and route each query to the right language by passing `"attributes": { "language": "" }`. -**Multi-language corpora.** Add a `language` field with `enable_match: true`; route each query to the right language by passing `metadata_filters: { language: detect_language(query) }`. +**Date windows.** Declare a `published_on` field as `VARCHAR`, store every date in one fixed format such as `YYYY-MM-DD` so string order is date order, and filter with `{"published_on": {"$gte": "2026-01-01", "$lt": "2026-07-01"}}`. -**Schema-as-product.** Treat `database_metadata_schema` as part of your data contract - review it like a database migration. The cost of getting it wrong (immutability + re-ingest) is real; the cost of getting it right is one extra meeting. +**Schema as a contract.** Treat `database_metadata_schema` as part of your data contract and review it like a database migration. Getting it wrong costs a re-ingest, because declared fields are immutable; getting it right costs one extra review. --- ## Related -- [Knowledge](/essentials/v2/knowledge) - what attaches to knowledge sources -- [Memories](/essentials/v2/memories) - metadata fields on memory items -- [Multi-Tenant Support](/essentials/v2/databases-and-collections) - partitioning vs scoping -- [Query](/essentials/v2/query) - how `metadata_filters` interact with ranking -- [Create Database - API Reference](/api-reference/v2/endpoint/create-tenant) - full `database_metadata_schema` reference -- [Query - API Reference](/api-reference/v2/endpoint/query) - full `metadata_filters` reference -- [List Context](/api-reference/v2/endpoint/list-documents) - source browsing filters -- [Access Control](/essentials/v2/access-control) - restricting who may retrieve a document, which is not a metadata filter -- [Update Source Metadata](/api-reference/v2/endpoint/update-source-metadata) - point metadata edits -- [Update Metadata Schema](/api-reference/v2/endpoint/update-metadata-schema) - additive database schema changes +- [Ingest context](/essentials/v2/ingest): every item field, including `attributes` and `custom_attributes` +- [Query](/essentials/v2/query): how `attributes` sits alongside ranking, graph and forceful relations +- [Databases and collections](/essentials/v2/databases-and-collections): partitioning versus filtering +- [Access control](/essentials/v2/access-control): restricting who may retrieve an item, which is not an attribute filter +- [Create Database API reference](/api-reference/v2/endpoint/create-tenant): the full `database_metadata_schema` reference +- [Ingest API reference](/api-reference/v2/endpoint/ingest-context): the full item reference +- [Query API reference](/api-reference/v2/endpoint/query): the full `attributes` request reference +- [List Context](/api-reference/v2/endpoint/list-documents): browsing items with exact-match filters +- [Update Source Metadata](/api-reference/v2/endpoint/update-source-metadata): in-place value edits +- [Update Metadata Schema](/api-reference/v2/endpoint/update-metadata-schema): additive schema changes diff --git a/essentials/v2/bring-your-own-graph.mdx b/essentials/v2/bring-your-own-graph.mdx index cdd038d0..0245281f 100644 --- a/essentials/v2/bring-your-own-graph.mdx +++ b/essentials/v2/bring-your-own-graph.mdx @@ -1,13 +1,13 @@ --- title: "Bring Your Own Graph" -description: "Supply your own entities and relations for a document and skip LLM graph extraction." +description: "Supply your own entities and relations for a context item and skip LLM graph extraction." --- ## 1. What it is -Bring Your Own Graph (BYOG) lets you attach a `graph_payload` - your own entities and relations - to a **source** (a document, an `app_knowledge` source, or a memory) on [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context). For that source, HydraDB **uses your graph instead of running LLM extraction**. +Bring Your Own Graph (BYOG) lets you attach your own entities and relations to a context item on [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context), with the request-level `graph_payload` field. For that item, HydraDB **uses your graph instead of running LLM graph extraction**. -Your graph is stored in exactly the same shape extraction produces (`source → relation → target` triplets), so it answers queries identically - it shows up in the `graph_context` slice and its relations point back at the source's chunks. No query-side changes are needed. +Your graph is stored in the same shape extraction produces (`source → relation → target` triplets), so it answers queries the same way: its relations come back in `graph[]` on [`POST /query`](/essentials/v2/query) and point at the item's chunks. No query-side changes are needed. --- @@ -17,80 +17,87 @@ Use BYOG when you already know the relationships and want them used verbatim: - You maintain a curated knowledge graph, an ontology, or a database export and want those exact facts in HydraDB. - You need deterministic, reproducible relations rather than model-extracted ones. -- You want faster ingestion - a BYOG document skips the extraction LLM call entirely. +- You want faster ingestion: a BYOG item skips the graph-extraction LLM call entirely. Pick the right tool: -| You want… | Use | +| You want... | Use | | --- | --- | -| HydraDB to discover relationships for you | [Context Graphs](/essentials/v2/context-graphs) (auto-extraction, the default) | -| To declare links **between whole sources** | [Forceful relations](/essentials/v2/knowledge) (`relations` on a document/app source) | -| To supply the **full entity/relation graph for one source** | **Bring Your Own Graph** (this page) | +| HydraDB to discover relationships for you | [Context graphs](/essentials/v2/context-graphs) (auto-extraction, the default) | +| To declare links **between whole items** | `forceful_relations` on an item. See [Declared relations](/essentials/v2/ingest#11-declared-relations). | +| To supply the **full entity and relation graph for one item** | **Bring Your Own Graph** (this page) | --- ## 3. The `graph_payload` shape -`graph_payload` is a JSON string: a **map keyed by source id** - a document's `document_metadata` `id`, an `app_knowledge` item's `id`, or a memory's `id` - where each value is that source's graph (an `entities` map + a `relations` list). Attach graphs to several sources in one request by adding more keys. +`graph_payload` sits at the top level of the ingest request, next to `context`. It is a **map keyed by `context_id`**, where each value is that item's graph: an `entities` map and a `relations` list. Attach graphs to several items in one request by adding more keys. ```json { - "billing-policy-doc": { - "entities": { - "alice": { "name": "Alice Carter", "type": "PERSON", "namespace": "employees", "identifier": "alice@acme.com" }, - "billing": { "name": "Billing Policy", "type": "POLICY", "namespace": "policies" } - }, - "relations": [ - { - "source": "alice", - "target": "billing", - "predicate": "OWNS", - "context": "Alice Carter owns the billing policy.", - "temporal_details": "since 2021" - } - ] + "database": "acme_corp", + "context": [ + { "context_id": "billing-policy", "title": "Billing policy", "text": "Alice Carter owns the billing policy. ..." } + ], + "graph_payload": { + "billing-policy": { + "entities": { + "alice": { "name": "Alice Carter", "type": "PERSON", "namespace": "employees", "identifier": "alice@acme.com" }, + "billing": { "name": "Billing Policy", "type": "POLICY", "namespace": "policies" } + }, + "relations": [ + { + "source": "alice", + "target": "billing", + "predicate": "OWNS", + "context": "Alice Carter owns the billing policy.", + "temporal_details": "since 2021" + } + ] + } } } ``` -- **Top-level key** - the id of the source this graph belongs to (a document's `document_metadata` `id`, an `app_knowledge` item's `id`, or a memory's `id`). A key matching no source in the request is rejected with `400`. -- **`entities`** - a map keyed by a caller-local id. Each entity has a `name` (required), `type`, `namespace`, and optional `identifier` (an external id - display-only). The entity key is just a handle for relations to reference; it is not stored. -- **`relations`** - a list of edges. `source` and `target` are entity-map keys; `predicate` is any plain string; `context` and `temporal_details` are optional per relation. -- **No `chunk_id`** - you never supply or see chunk ids; HydraDB resolves them server-side when it links your relations to the source's chunks. +- **Top-level key:** the `context_id` of an item in the same request. Every key must match one; a key that matches nothing is a `400`, so a typo cannot silently drop a graph. An item needs an explicit `context_id` to receive a graph (an item whose id is generated cannot be targeted). +- **`entities`:** a map keyed by a caller-local id. Each entity has a `name` (required), a `type`, a `namespace`, and an optional `identifier` (an external id, display only). The entity key is only a handle for relations to reference; it is not stored. +- **`relations`:** a list of edges. `source` and `target` are keys of the `entities` map, and a key that is not declared there is a `400`. `predicate` is required and is any plain string. `context` and `temporal_details` are optional per relation. +- Both `entities` and `relations` must be non-empty. +- **No `chunk_id`:** you never supply chunk ids. HydraDB links your relations to the item's chunks server-side. - Entity names are **normalized (lowercased)** so they match at query time, just like extracted entities. Entities that no relation references are dropped. +In a JSON body, `graph_payload` is an object. The SDKs send a multipart form, where `graph_payload` is a JSON string next to the `items` field; see the [examples](#6-example-several-items-in-one-request). + --- ## 4. How it behaves -- **Replace mode.** A BYOG document's graph is your `graph_payload`; LLM extraction is skipped for it. The document is still chunked and embedded, so it stays fully vector-searchable. -- **Chunk linking.** Each relation is linked to the source's most relevant chunk(s), so `graph_context` results hydrate the right passages. Linking is permissive (see [Limitations](#8-limitations)). -- **Queryable like any graph.** Your relations appear in the `/query` `graph_context` slice (tagged `origin: "byog"` in metadata) and traverse exactly like extracted ones - see [Context Graphs](/essentials/v2/context-graphs). -- **Durable across re-ingest.** Your graph is persisted server-side, so it outlives a single upload. Re-ingesting the same source **without** a `graph_payload` - a connector re-sync, or just iterating on the document's content - re-applies your stored graph: HydraDB does **not** fall back to LLM extraction and does **not** error. Your facts are never silently lost. To change the graph, re-ingest **with** a new `graph_payload`; it replaces the stored copy (replace mode). +- **Replace mode.** A BYOG item's graph is your `graph_payload`; LLM graph extraction is skipped for it. The item is still chunked and embedded, so it stays fully searchable. +- **Chunk linking.** Each relation is linked to the item's most relevant chunk, so the relation's `chunk_id` in `graph[]` points at the right passage. Linking is permissive (see [Limitations](#7-limitations)). +- **Queryable like any graph.** Your relations come back in `graph[]` on `POST /query` and traverse exactly like extracted ones. See [Context graphs](/essentials/v2/context-graphs). +- **Durable across re-ingest.** Your graph is stored server-side with the item, so it outlives a single request. Re-ingesting the same `context_id` **without** a `graph_payload` (for example, to update its text) re-applies your stored graph: HydraDB does **not** fall back to LLM extraction and does **not** error. To change the graph, re-ingest **with** a new `graph_payload`; it replaces the stored copy. Deleting the item removes its stored graph too. --- ## 5. Limits -`graph_payload` is validated up front; oversized payloads are rejected with `400`. +`graph_payload` is validated before anything is queued; a graph over a limit is rejected with `400`. | Limit | Value | | --- | --- | -| Entities | ≤ 5,000 | -| Relations | ≤ 10,000 | +| Entities per item | ≤ 5,000 | +| Relations per item | ≤ 10,000 | | Relations per entity (degree) | ≤ 500 | -| `context` length | ≤ 2,000 chars | -| `name` / `predicate` length | ≤ 256 chars | +| Relation `context` length | ≤ 2,000 characters | +| Entity key, `name`, `type`, `namespace`, `identifier`, `predicate` and `temporal_details` length | ≤ 256 characters each | - -`graph_payload` is **per-source**: each top-level key must match the `id` of a source in the same request - a document's `document_metadata` `id`, an `app_knowledge` item's `id`, or (when `type=memory`) a memory's `id`. Attach graphs to multiple sources at once. A source must carry an explicit `id` to be targeted (that `id` is the map key). Works for both `type=knowledge` and `type=memory` - a single request is one or the other, so its `graph_payload` keys target only that type's sources. - +The request itself keeps the normal ingest limits: at most 100 items, 1 MiB of text per item and 8 MiB of text per request. See [Ingest context](/essentials/v2/ingest#limits-and-refused-fields). --- -## 6. Example: multiple sources in one request +## 6. Example: several items in one request -`graph_payload` is a map, so one request can carry graphs for several sources at once - here **two documents and one app_knowledge source**, each keyed by its own id. Then query, and each source's triples surface. +`graph_payload` is a map, so one request can carry graphs for several items at once. Here three items, each keyed by its own `context_id`. Then query, and each item's triplets surface. @@ -98,42 +105,45 @@ Pick the right tool: curl -X POST 'https://api.hydradb.com/context/ingest' \ -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ -H "API-Version: 2" \ - -F "type=knowledge" \ - -F "database=acme_corp" \ - -F "documents=@/path/to/billing-policy.pdf" \ - -F "documents=@/path/to/deploy-runbook.pdf" \ - -F 'document_metadata=[{ "id": "billing-policy-doc" }, { "id": "deploy-runbook-doc" }]' \ - -F 'app_knowledge=[{ "id": "slack-incident-42", "kind": "message", "provider": "slack", "external_id": "slack-incident-42", "fields": { "kind": "message", "body": "Platform team paged for the payments outage." } }]' \ - -F 'graph_payload={ - "billing-policy-doc": { - "entities": { - "alice": { "name": "Alice Carter", "type": "PERSON", "namespace": "employees" }, - "billing": { "name": "Billing Policy", "type": "POLICY", "namespace": "policies" } - }, - "relations": [ - { "source": "alice", "target": "billing", "predicate": "OWNS", - "context": "Alice Carter owns the billing policy.", "temporal_details": "since 2021" } - ] - }, - "deploy-runbook-doc": { - "entities": { - "team": { "name": "Platform Team", "type": "TEAM", "namespace": "teams" }, - "svc": { "name": "Payments Service", "type": "SERVICE", "namespace": "services" } + -H "Content-Type: application/json" \ + -d '{ + "database": "acme_corp", + "context": [ + { "context_id": "billing-policy", "title": "Billing policy", "text": "Alice Carter owns the billing policy. Invoices are issued on the first business day of each month." }, + { "context_id": "deploy-runbook", "title": "Deploy runbook", "text": "The platform team operates the payments service and approves every production deploy." }, + { "context_id": "incident-42", "title": "Payments outage", "text": "Platform team paged for the payments outage at 02:10 UTC." } + ], + "graph_payload": { + "billing-policy": { + "entities": { + "alice": { "name": "Alice Carter", "type": "PERSON", "namespace": "employees" }, + "billing": { "name": "Billing Policy", "type": "POLICY", "namespace": "policies" } + }, + "relations": [ + { "source": "alice", "target": "billing", "predicate": "OWNS", + "context": "Alice Carter owns the billing policy.", "temporal_details": "since 2021" } + ] }, - "relations": [ - { "source": "team", "target": "svc", "predicate": "OPERATES", - "context": "The platform team operates the payments service." } - ] - }, - "slack-incident-42": { - "entities": { - "team": { "name": "Platform Team", "type": "TEAM", "namespace": "teams" }, - "inc": { "name": "Payments Outage", "type": "INCIDENT", "namespace": "incidents" } + "deploy-runbook": { + "entities": { + "team": { "name": "Platform Team", "type": "TEAM", "namespace": "teams" }, + "svc": { "name": "Payments Service", "type": "SERVICE", "namespace": "services" } + }, + "relations": [ + { "source": "team", "target": "svc", "predicate": "OPERATES", + "context": "The platform team operates the payments service." } + ] }, - "relations": [ - { "source": "team", "target": "inc", "predicate": "RESPONDED_TO", - "context": "The platform team was paged for the payments outage." } - ] + "incident-42": { + "entities": { + "team": { "name": "Platform Team", "type": "TEAM", "namespace": "teams" }, + "inc": { "name": "Payments Outage", "type": "INCIDENT", "namespace": "incidents" } + }, + "relations": [ + { "source": "team", "target": "inc", "predicate": "RESPONDED_TO", + "context": "The platform team was paged for the payments outage." } + ] + } } }' ``` @@ -141,10 +151,19 @@ curl -X POST 'https://api.hydradb.com/context/ingest' \ ```python Python SDK import json +items = [ + {"context_id": "billing-policy", "title": "Billing policy", + "text": "Alice Carter owns the billing policy. Invoices are issued on the first business day of each month."}, + {"context_id": "deploy-runbook", "title": "Deploy runbook", + "text": "The platform team operates the payments service and approves every production deploy."}, + {"context_id": "incident-42", "title": "Payments outage", + "text": "Platform team paged for the payments outage at 02:10 UTC."}, +] + graphs = { - "billing-policy-doc": { + "billing-policy": { "entities": { - "alice": {"name": "Alice Carter", "type": "PERSON", "namespace": "employees"}, + "alice": {"name": "Alice Carter", "type": "PERSON", "namespace": "employees"}, "billing": {"name": "Billing Policy", "type": "POLICY", "namespace": "policies"}, }, "relations": [ @@ -152,7 +171,7 @@ graphs = { "context": "Alice Carter owns the billing policy.", "temporal_details": "since 2021"}, ], }, - "deploy-runbook-doc": { + "deploy-runbook": { "entities": { "team": {"name": "Platform Team", "type": "TEAM", "namespace": "teams"}, "svc": {"name": "Payments Service", "type": "SERVICE", "namespace": "services"}, @@ -162,7 +181,7 @@ graphs = { "context": "The platform team operates the payments service."}, ], }, - "slack-incident-42": { + "incident-42": { "entities": { "team": {"name": "Platform Team", "type": "TEAM", "namespace": "teams"}, "inc": {"name": "Payments Outage", "type": "INCIDENT", "namespace": "incidents"}, @@ -174,29 +193,27 @@ graphs = { }, } -with open("/path/to/billing-policy.pdf", "rb") as f1, open("/path/to/deploy-runbook.pdf", "rb") as f2: - client.context.ingest( - type="knowledge", - database="acme_corp", - documents=[ - ("billing-policy.pdf", f1, "application/pdf"), - ("deploy-runbook.pdf", f2, "application/pdf"), - ], - document_metadata=json.dumps([{"id": "billing-policy-doc"}, {"id": "deploy-runbook-doc"}]), - app_knowledge=json.dumps([ - {"id": "slack-incident-42", "kind": "message", "provider": "slack", - "external_id": "slack-incident-42", - "fields": {"kind": "message", "body": "Platform team paged for the payments outage."}}, - ]), - graph_payload=json.dumps(graphs), - ) +client.context.ingest( + database="acme_corp", + items=json.dumps(items), + graph_payload=json.dumps(graphs), +) ``` ```typescript TypeScript SDK +const items = [ + { context_id: "billing-policy", title: "Billing policy", + text: "Alice Carter owns the billing policy. Invoices are issued on the first business day of each month." }, + { context_id: "deploy-runbook", title: "Deploy runbook", + text: "The platform team operates the payments service and approves every production deploy." }, + { context_id: "incident-42", title: "Payments outage", + text: "Platform team paged for the payments outage at 02:10 UTC." }, +]; + const graphs = { - "billing-policy-doc": { + "billing-policy": { entities: { - alice: { name: "Alice Carter", type: "PERSON", namespace: "employees" }, + alice: { name: "Alice Carter", type: "PERSON", namespace: "employees" }, billing: { name: "Billing Policy", type: "POLICY", namespace: "policies" }, }, relations: [ @@ -204,7 +221,7 @@ const graphs = { context: "Alice Carter owns the billing policy.", temporal_details: "since 2021" }, ], }, - "deploy-runbook-doc": { + "deploy-runbook": { entities: { team: { name: "Platform Team", type: "TEAM", namespace: "teams" }, svc: { name: "Payments Service", type: "SERVICE", namespace: "services" }, @@ -214,7 +231,7 @@ const graphs = { context: "The platform team operates the payments service." }, ], }, - "slack-incident-42": { + "incident-42": { entities: { team: { name: "Platform Team", type: "TEAM", namespace: "teams" }, inc: { name: "Payments Outage", type: "INCIDENT", namespace: "incidents" }, @@ -227,24 +244,17 @@ const graphs = { }; await client.context.ingest({ - type: "knowledge", database: "acme_corp", - documents: [ - { path: "/path/to/billing-policy.pdf", filename: "billing-policy.pdf", contentType: "application/pdf" }, - { path: "/path/to/deploy-runbook.pdf", filename: "deploy-runbook.pdf", contentType: "application/pdf" }, - ], - documentMetadata: JSON.stringify([{ id: "billing-policy-doc" }, { id: "deploy-runbook-doc" }]), - appKnowledge: JSON.stringify([ - { id: "slack-incident-42", kind: "message", provider: "slack", external_id: "slack-incident-42", - fields: { kind: "message", body: "Platform team paged for the payments outage." } }, - ]), + items: JSON.stringify(items), graphPayload: JSON.stringify(graphs), }); ``` -Poll [Ingestion Status](/api-reference/v2/endpoint/source-status) until the source is ready, then query with `graph_context: true`: +Keys inside each item and each graph stay `snake_case` in every language (`context_id`, `temporal_details`); only the SDK's own arguments follow the language's casing. + +Poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) until each item reaches `completed` (or `graph_creation`), then query. `graph_context` is on by default: ```bash cURL curl -X POST 'https://api.hydradb.com/query' \ @@ -254,75 +264,51 @@ curl -X POST 'https://api.hydradb.com/query' \ -d '{ "database": "acme_corp", "query": "Who owns the billing policy?", - "type": "knowledge", - "query_by": "hybrid", - "mode": "thinking", - "graph_context": true + "mode": "thinking" }' ``` -Your triplet comes back in `graph_context` and traverses just like an extracted one - the `origin: "byog"` tag on the relation marks it as yours: +Your triplet comes back in `graph[]` and traverses just like an extracted one: ```json { - "graph_context": { - "query_paths": [ - { - "triplets": [ - { - "source": { "name": "alice carter", "type": "PERSON" }, - "relation": { "canonical_predicate": "OWNS", "context": "Alice Carter owns the billing policy.", "origin": "byog" }, - "target": { "name": "billing policy", "type": "POLICY" } - } - ] - } - ] - } -} -``` - ---- - -## 7. Memories - -Memories accept a `graph_payload` too - send `type=memory`, give each memory an `id`, and key the graph by that `id`. The graph shape is identical; relations link to the memory's chunks and surface in `/query` with `type=memory` and `graph_context: true`. - -```bash cURL -curl -X POST 'https://api.hydradb.com/context/ingest' \ - -H "Authorization: Bearer $HYDRA_DB_API_KEY" \ - -H "API-Version: 2" \ - -F "type=memory" \ - -F "database=acme_corp" \ - -F 'memories=[{ "id": "mem-oncall-1", "text": "Dana Kim owns the incident playbook and leads the on-call rotation." }]' \ - -F 'graph_payload={ - "mem-oncall-1": { - "entities": { - "dana": { "name": "Dana Kim", "type": "PERSON", "namespace": "employees" }, - "pb": { "name": "Incident Playbook", "type": "DOCUMENT", "namespace": "runbooks" } - }, - "relations": [ - { "source": "dana", "target": "pb", "predicate": "OWNS", - "context": "Dana Kim owns the incident playbook.", "temporal_details": "since 2024" } - ] + "graph": [ + { + "origin": "query_path", + "triplets": [ + { + "source": { "entity_id": "ent_alice", "name": "alice carter" }, + "relation": { + "predicate": "OWNS", + "context": "Alice Carter owns the billing policy.", + "temporal_details": "since 2021", + "relationship_id": "rel_owns", + "chunk_id": "ck_billing_1" + }, + "target": { "entity_id": "ent_billing", "name": "billing policy" } + } + ], + "path_summary": "Alice Carter owns the billing policy." } - }' + ] +} ``` -A memory must carry an explicit `id` to receive a graph (an id-less memory gets a server-generated id and can't be targeted). The `memories` form field stays plural even though `type` is the singular `memory`. +The same facts also appear under `## Related facts` in `llm_prompt`. See [Query](/essentials/v2/query) for the full response. --- -## 8. Limitations +## 7. Limitations -- **Replace, not augment.** A BYOG source has no LLM-extracted facts - only the graph you supply (plus normal chunk search). Augment mode is a future enhancement. -- **Permissive linking → possible false positives.** Every relation links to its best-matching chunk even if the match is weak; there is no reject floor yet. A linked relation is **sourced** (similar to a chunk), not necessarily **supported** (stated by the source). -- **Bulk, one-shot.** You supply the whole graph with the source. Per-triple add/update/delete is not yet available. +- **Replace, not augment.** A BYOG item has no LLM-extracted graph facts, only the graph you supply (plus normal chunk search). Augment mode is a future enhancement. +- **Permissive linking can produce false positives.** Every relation links to its best-matching chunk even if the match is weak; there is no reject floor yet. A linked relation is **sourced** (similar to a chunk), not necessarily **supported** (stated by the item). +- **Bulk, one-shot.** You supply the whole graph with the item. Per-triple add, update and delete are not yet available. --- ## Related -- [Context Graphs](/essentials/v2/context-graphs) - the auto-extracted graph BYOG replaces; HydraDB builds it for you, BYOG lets you supply it -- [Knowledge](/essentials/v2/knowledge) - documents, and forceful relations between sources -- [Ingest Context](/api-reference/v2/endpoint/ingest-context) - the `graph_payload` form field reference -- [Query](/essentials/v2/query) - how chunks and graph context are retrieved together +- [Context graphs](/essentials/v2/context-graphs): the auto-extracted graph BYOG replaces. HydraDB builds it for you; BYOG lets you supply it. +- [Ingest context](/essentials/v2/ingest): every item field, including `forceful_relations` for links between items +- [Ingest context API reference](/api-reference/v2/endpoint/ingest-context): the `graph_payload` field reference +- [Query](/essentials/v2/query): how chunks and `graph[]` are retrieved together diff --git a/essentials/v2/connectors.mdx b/essentials/v2/connectors.mdx index 2d677f1a..907a2455 100644 --- a/essentials/v2/connectors.mdx +++ b/essentials/v2/connectors.mdx @@ -1,9 +1,9 @@ --- title: "Connectors" -description: "How HydraDB connectors continuously sync external app data into the knowledge store." +description: "How HydraDB connectors continuously sync external app data into your database as searchable context." --- -Connectors bring external app data into HydraDB automatically. Instead of manually ingesting documents, you authenticate once, pick which resources to sync, and HydraDB continuously replicates provider content as searchable [app sources](/essentials/v2/app-sources) in your knowledge store. +Connectors bring external app data into HydraDB automatically. Instead of ingesting it yourself, you authenticate once, pick which resources to sync, and HydraDB continuously syncs provider content into your database as searchable context, queried alongside the items you [ingest](/essentials/v2/ingest). --- @@ -15,9 +15,9 @@ A connector runs three stages on every sync cycle: Discover → Configure → Sync ``` -1. **Discover** - the connector lists all available resources from the provider: Slack channels, GitHub repos, Linear teams, projects and Workspace Documents, Notion databases and pages, Gmail labels. Call `GET /connectors/:id/discover` to see what's available before committing. -2. **Configure** - you choose which resources to activate and set sync options. Call `POST /connectors/:id/configure` with the list of resource IDs to enable. -3. **Sync** - HydraDB fetches objects from each active resource since the last cursor and ingests them as app sources. Sync runs on a schedule (default: hourly) or on demand via `POST /connectors/:id/sync`. +1. **Discover**: the connector lists all available resources from the provider (Slack channels, GitHub repos, Linear teams, projects and Workspace Documents, Notion databases and pages, Gmail labels). Call `GET /connectors/:id/discover` to see what's available before committing. +2. **Configure**: you choose which resources to activate and set sync options. Call `POST /connectors/:id/configure` with the list of resource IDs to enable. +3. **Sync**: HydraDB fetches objects from each active resource since the last cursor and ingests them into your database as searchable context. Sync runs on a schedule (default: hourly) or on demand via `POST /connectors/:id/sync`. ### Authentication @@ -127,7 +127,7 @@ Linear documents (the docs you write inside Linear) do not belong to a single te } ``` -Each document is indexed as a `knowledge_base` app source. Its markdown body is searchable. Files uploaded into the document (Linear-hosted `uploads.linear.app` files) are downloaded, parsed, and indexed too. Plain links to external URLs are kept as metadata, not fetched. +Each document is indexed as a `knowledge_base` object. Its markdown body is searchable. Files uploaded into the document (Linear-hosted `uploads.linear.app` files) are downloaded, parsed, and indexed too. Plain links to external URLs are kept as metadata, not fetched. **From the dashboard:** tick "Workspace Documents" in the resource list, the same way you tick a team or project. @@ -350,7 +350,6 @@ You can also route different resources from the same connector into different su ## Related -- [App Sources](/essentials/v2/app-sources) - ingestion model connector objects use - [Metadata](/essentials/v2/attributes) - tenant metadata vs document metadata in depth - [Multi-Tenant](/essentials/v2/databases-and-collections) - routing resources to tenants and sub-tenants - [Query](/essentials/v2/query) - querying connector-synced data with `query_apps: true` diff --git a/essentials/v2/context-graphs.mdx b/essentials/v2/context-graphs.mdx index 68555ef8..65a9708d 100644 --- a/essentials/v2/context-graphs.mdx +++ b/essentials/v2/context-graphs.mdx @@ -3,8 +3,6 @@ title: "Context Graphs" description: "How HydraDB models relationships between the entities in your context as triplets, and returns them as paths on every query." --- -import LegacyLine from "/snippets/legacy-line.mdx"; - ## 1. What it is A context graph is a structured map of the entities in your database and the relationships between them, built from everything you ingest. @@ -13,8 +11,6 @@ It represents those relationships as **triplets**: `source`, `relation`, `target Context graphs augment retrieval. They do not replace it. - - --- ## 2. What it does @@ -47,7 +43,7 @@ Context graphs are hybrid: relationships are extracted at ingestion time and tra 1. HydraDB runs hybrid retrieval to find relevant chunks. 2. It traverses the graph from the query and from the retrieved chunks. -3. It returns the paths it found in `graph[]`: paths grown from the query first (`origin: "query_path"`), then paths expanded from the returned chunks (`origin: "chunk_relation"`), deduplicated. +3. It returns the paths it found in `graph[]`: paths grown from the query first (`origin: "query_path"`), then paths expanded from the returned chunks (`origin: "chunk_relation"`). The list is deduplicated across both lanes, so a path both found appears once, and it is not capped. When no relevant relationships are found, `graph` is `[]`. That is not an error; it is the absence of structure for that query. @@ -55,7 +51,7 @@ When no relevant relationships are found, `graph` is `[]`. That is not an error; ## 5. Key concepts -**Triplets.** The unit of the graph. `source` and `target` are entities, `{ entity_id, name }`. `relation` describes the connection: `predicate`, the sentence it was extracted from (`context`), when it holds (`temporal_details`, absent when unknown), its `relationship_id`, and the `chunk_id` of the chunk that is evidence for it. +**Triplets.** The unit of the graph. `source` and `target` are entities, `{ entity_id, name }`. `relation` describes the connection: `predicate`, the sentence it was extracted from (`context`), when it holds (`temporal_details`, omitted when empty), the edge's `timestamp` in Unix epoch seconds (a float, omitted when the edge has none), its `relationship_id`, and the `chunk_id` of the chunk that is evidence for it. Example: `Alex`, `prefers`, `short answers`, from chunk `ck_9f2`. @@ -63,7 +59,7 @@ Example: `Alex`, `prefers`, `short answers`, from chunk `ck_9f2`. **Evidence.** `relation.chunk_id` names the chunk every step of a path was extracted from. Group hops by it against `chunks[].chunk_id` to show a chunk's relations under that chunk: a `chunk_relation` path is only returned when one of its hops came from a returned chunk (or a `forceful_relations` chunk), and hangs under that chunk; a `query_path` hop may also sit under the chunk it came from. See [Attaching graph paths to chunks](/essentials/v2/query#attaching-graph-paths-to-chunks). -**Forceful relations.** Links between items rather than between entities, which you declare at ingest with `forceful_relations`. They come back in `forceful_relations[]`, not in `graph[]`. +**Forceful relations.** Links between items rather than between entities, which you declare at ingest with `forceful_relations`. A `thinking` query follows them, and they come back in `forceful_relations[]`, not in `graph[]`. **Connected subgraph.** The graph also holds relations between items themselves: a Slack reply and the message it answers, a page and the pages it links to, a comment and its ticket. Given one item's id, [Connected Subgraph](/api-reference/v2/endpoint/subgraph) walks those links breadth-first and returns everything reachable with the relations among them. Reach for it when one query result is not enough and you need what surrounds it; it is also what the dashboard's **Subgraph** button opens. @@ -133,6 +129,7 @@ The `graph` key of the response looks like: "relation": { "predicate": "governs", "context": "The billing policy governs how failed payments are retried.", + "timestamp": 1782984600.0, "relationship_id": "rel_41", "chunk_id": "ck_2aa" }, @@ -160,14 +157,14 @@ The `graph` key of the response looks like: ## 7. Using graph context in your prompt -You do not format the graph yourself. The `llm_prompt` returned by the same query already contains a `## Related facts` section with one line per path in `graph[]`: its label (`[P1]`, `[P2]`, ... in `graph[]` order, so an agent can cite a fact by it), its chain of hops, `query path` or `chunk relation` for its `origin` with the path's relevance after reranking when it has one, and the numbers of the results its hops were extracted from. The `path_summary` is indented on the line under it, unless it only narrates the hops the chain already shows. From the [example response on Query](/essentials/v2/query#1-one-call): +You do not format the graph yourself. The `llm_prompt` returned by the same query already contains a `## Related facts` section with one line per path in `graph[]`: its label (`[P1]`, `[P2]`, ... in `graph[]` order, so an agent can cite a fact by it), its chain of hops, the path's relevance after reranking in parentheses when it has one (a path with no reranked score has no parenthetical), and the numbers of the results its hops were extracted from. The line does not say which lane found the path; read `graph[].origin` for that. The `path_summary` is indented on the line under it, unless it only narrates the hops the chain already shows. From the [example response on Query](/essentials/v2/query#1-one-call): ```markdown ## Related facts -- [P1] **Refund Processing** -managed by→ **Finance Department** (query path, relevance 0.81) [1] +- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1] Refund processing is managed by the Finance Department. -- [P2] **User** -prefers→ **short answers** (chunk relation, relevance 0.74) [2] +- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2] The user prefers short answers about refunds. ``` @@ -199,12 +196,6 @@ Inject `llm_prompt` and the model can reason over the paths and cite them. See [ --- -## 9. Split databases - -A database created with `type: "split"` returns the graph under `graph_context` as `query_paths`, `chunk_relations` and `chunk_id_to_group_ids`, with `canonical_predicate` and `relevancy_score` on each path. The mapping to `graph[]` is on [Split databases and legacy fields](/essentials/v2/split-databases#4-legacy-query-and-response-fields). - ---- - ## Related - [Query](/essentials/v2/query): how chunks, paths and forceful relations are retrieved together diff --git a/essentials/v2/databases-and-collections.mdx b/essentials/v2/databases-and-collections.mdx index bc0ba1e2..fe1f9390 100644 --- a/essentials/v2/databases-and-collections.mdx +++ b/essentials/v2/databases-and-collections.mdx @@ -3,6 +3,10 @@ title: "Databases and collections" description: "How HydraDB scopes data using databases and collections, and how scoping affects writes and query." --- + + **Knowledge and memory (split databases) are deprecated.** Unified is the way to go, and you do not pass anything to get it: every database you create is unified. Send `context` items with [Ingest context](/essentials/v2/ingest) and read them back with [Query](/essentials/v2/query). + + The request fields are now named **`database`** (formerly `tenant_id`) and **`collection`** (formerly `sub_tenant_id`). The old names and the old `/tenants` routes remain fully supported as deprecated aliases, so existing integrations keep working without any changes -- however, we recommend using the latest conventions. See [Migrating from `tenant_id` and `sub_tenant_id`](#7-migrating-from-the-legacy-tenant-and-sub-tenant-fields) for the full compatibility contract. @@ -28,7 +32,7 @@ Use the same scoping values consistently across writes and reads. If you write d | Separate environments (`prod` vs `staging`) | A different `database` per environment | | Separate per-user state within one customer | One `database`, `collection = user_id` | | Separate per-workspace data within one customer | One `database`, `collection = workspace_id` | -| Store broadly shared Knowledge | Omit `collection` to use the database's default collection | +| Store broadly shared context | Omit `collection` to use the database's default collection | Two practical rules: @@ -50,13 +54,13 @@ database = "acme_app" collection = "user_123" ``` -Use this when each user has private Memories, preferences, or conversation history. +Use this when each user has private context, preferences, or conversation history. Typical flow: -- Write user Memories with `collection = user_id`. -- Query user Memories with the same `collection`. -- Keep shared Knowledge outside the user-specific scope. +- Write each user's items with `collection = user_id`. +- Query that user's context with the same `collection`. +- Keep shared context outside the user-specific scope. ### B2B SaaS @@ -69,9 +73,9 @@ collection = "workspace_42" Typical flow: -- Customer-level Knowledge uses the customer `database`. +- Customer-level shared context uses the customer `database`. - Workspace-specific data uses a workspace `collection`. -- User-specific Memories use a user-level `collection`. +- User-specific context uses a user-level `collection`. ### Shared context + user personalization @@ -85,7 +89,7 @@ For personalized answers grounded in shared context, put the shared context in a } ``` -The weights rank the person's own context above the shared context without excluding either. There is no `type` on a unified database; the collections are the only scope you choose. The returned `llm_prompt` already merges both into one prompt-ready string. +The weights rank the person's own context above the shared context without excluding either. The collections are the only scope you choose. The returned `llm_prompt` already merges both into one prompt-ready string. --- @@ -99,9 +103,9 @@ Use `collection` when the data belongs to a specific user, workspace, team, or o Examples: -- A memory about John's preferences should be written with John's `collection`. +- An item about John's preferences should be written with John's `collection`. - Workspace-specific runbooks should be written with that workspace's `collection`. -- Broadly shared Knowledge should use the same scope you plan to use when querying it. +- Broadly shared context should use the same scope you plan to use when querying it. ### Reads @@ -242,8 +246,8 @@ Use `database` for primary boundaries such as customers or environments. Use `co **Assuming query automatically searches every collection.** A query call uses the scope you provide. If your application needs data from multiple scopes, pass `collections` as a list or weighted object. -**Writing shared Knowledge under a user scope by accident.** -If broadly shared Knowledge is written with a user-specific `collection`, it may not appear where other users expect it. Choose the write scope based on where the content should be queried later. +**Writing shared context under a user scope by accident.** +If broadly shared context is written with a user-specific `collection`, it may not appear where other users expect it. Choose the write scope based on where the content should be queried later. **Using attribute filters as a substitute for collections.** Attribute filters narrow results inside a scope. They are not a replacement for choosing the right `database` and `collection`. @@ -312,8 +316,7 @@ If you send **both** a canonical field and its deprecated alias: ## Related -- [Memories](/essentials/v2/memories) - user-scoped context -- [Knowledge](/essentials/v2/knowledge) - shared document context +- [Ingest context](/essentials/v2/ingest): writing items into a collection - [Query](/essentials/v2/query) - how scoping is applied at query time - [How to Use API Results](/essentials/v2/api-results) - merging query results into a prompt - [Create Database](/api-reference/v2/endpoint/tenants-overview) - defining databases and their metadata schema diff --git a/essentials/v2/glossary.mdx b/essentials/v2/glossary.mdx index a898c174..3ff596e6 100644 --- a/essentials/v2/glossary.mdx +++ b/essentials/v2/glossary.mdx @@ -32,14 +32,6 @@ response's `forceful_relations[]`, each with the `via` link that brought it in, separately from the ranked `chunks` and the `graph` paths. See [Declared relations](/essentials/v2/ingest#11-declared-relations). -## Unified and split databases - -A database is `unified` (the default: one corpus, no `type` on any call, four-key -query response with `llm_prompt`) or `split` (created with `type: "split"`: a -knowledge corpus and a memory corpus, selected with `type`, with the older request and -response shapes). `GET /databases` reports `details[].type`. See -[Split databases and legacy fields](/essentials/v2/split-databases). - ## Deprecated aliases `database` and `collection` were previously called `tenant_id` and diff --git a/essentials/v2/ingest.mdx b/essentials/v2/ingest.mdx index 8b8bf932..2a4d9d3d 100644 --- a/essentials/v2/ingest.mdx +++ b/essentials/v2/ingest.mdx @@ -3,14 +3,8 @@ title: "Ingest context" description: "Send text and conversations to HydraDB as context items in one call, label what kind of context each one is, and confirm it is searchable." --- -import LegacyLine from "/snippets/legacy-line.mdx"; - Everything you put into HydraDB is a piece of **context**: a text, or a conversation. You send a list of them to one endpoint, [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context), and HydraDB chunks each one, embeds it, enriches it, extracts entities and relations into the [context graph](/essentials/v2/context-graphs), and makes it searchable through [`POST /query`](/essentials/v2/query). -This page describes ingest on a **unified** database, which is every database unless it was created with `type: "split"`. - - - --- ## 1. One call for text and conversations @@ -74,7 +68,7 @@ ingest = client.context.ingest( ]), ) -print([r.source_id for r in ingest.data.results]) +print([r.id for r in ingest.data.results]) ``` ```typescript TypeScript SDK const ingest = await client.context.ingest({ @@ -101,7 +95,7 @@ const ingest = await client.context.ingest({ ]), }); -console.log(ingest.data.results.map((r) => r.sourceId)); +console.log(ingest.data.results.map((r) => r.id)); ``` @@ -118,8 +112,8 @@ The response is `202 Accepted`: "success": true, "message": "Context queued for ingestion successfully. Ingestion is asynchronous: this 202 means the sources were accepted and queued, not indexed. Poll GET /context/status?database=&id= until each source's indexing_status reaches a terminal state (completed or errored) before querying. See https://docs.hydradb.com/api-reference/v2/endpoint/source-status for usage details. ", "results": [ - { "source_id": "refund-policy", "title": "Refund policy", "status": "queued", "infer": true, "error": null, "error_code": null }, - { "source_id": "chat-alex-001", "title": null, "status": "queued", "infer": true, "error": null, "error_code": null } + { "id": "refund-policy", "title": "Refund policy", "status": "queued", "infer": true, "error": null, "error_code": null }, + { "id": "chat-alex-001", "title": null, "status": "queued", "infer": true, "error": null, "error_code": null } ], "success_count": 2, "failed_count": 0 @@ -130,7 +124,7 @@ The response is `202 Accepted`: ``` - `message` starts with `Context queued for ingestion successfully` (`Context ingestion completed with some failures` when an item failed), followed by a reminder to poll status. -- `results[].source_id` is the item's `context_id`: the one you sent, or the generated one. The result item keeps the name `source_id`; read it as the context id and pass it to [`GET /context/status`](/api-reference/v2/endpoint/source-status). +- `results[].id` is the item's `context_id`: the one you sent, or the generated one. Pass it to [`GET /context/status`](/api-reference/v2/endpoint/source-status). - `results[].infer` mirrors the item's `enrich`. - `results[].status` is `queued` or `failed`. A failed item carries `error` and `error_code`; the other items in the request are still queued. @@ -166,12 +160,12 @@ Each item is exactly one of `text` or `conversation`. | `conversation` | A list of `{ role, content, name? }` turns; roles are `user`, `assistant` and `system`. Shape B. See [Conversation items](#5-conversation-items). | | `enrich` | Extract entities, relations and preferences from this item. Default: the request's `enrich`, else `true`. | | `upsert` | Replace an existing item with the same `context_id`. Default: the request's `upsert`, else `true`. | -| `instructions` | Steer enrichment for this item. Default: the request's `instructions`. `custom_instructions` is accepted as an alias; send `instructions`. | +| `instructions` | Steer enrichment for this item. Default: the request's `instructions`. | | `happened_at` | The date the item is about, `YYYY-MM-DD` only. A timestamp is a `400`. HydraDB records when it received the item separately. | | `attributes` | Declared, filterable fields from the database's `database_metadata_schema`. See [Attributes](/essentials/v2/attributes). | | `custom_attributes` | Free-form fields. Not filterable. | | `context_category` | Optional label: `auto` (the default), `user_preference`, `business_knowledge` or `decision_trace`. You set it; nothing infers it. Validated strictly, so a typo is a `400`. See [Context categories](/essentials/v2/context-categories). | -| `forceful_relations` | Relations you declare to other items: `{ "ids": ["chat-w1"], "properties": {} }`. `relations` is accepted as an alias for the field, and `context_ids` or `source_ids` for the `ids` key; send `forceful_relations` with `ids`. See [Declared relations](#11-declared-relations). | +| `forceful_relations` | Relations you declare to other items: `{ "ids": ["chat-w1"], "properties": {} }`, where `ids` are the `context_id`s of the related items. See [Declared relations](#11-declared-relations). | | `acl` | Principals allowed to retrieve the item, such as `user_email:a@x.com` or `domain:acme.com`. Omit for unrestricted, `[]` for nobody. A malformed principal is a `400`. See [Restricting an item](#10-restricting-an-item). | | `is_markdown` | Chunk `text` on its markdown structure instead of as flat prose. | | `user_name` | The speaker for a text item. On a conversation, the per-turn `name` wins. | @@ -180,8 +174,7 @@ Each item is exactly one of `text` or `conversation`. - At most **100 items** per request, **1 MiB** of text per item, and **8 MiB** of text per request. - A validation error names the item it refers to as `context[N]`. -- These split-era fields are refused with a `400` on a unified database: `type`, `documents` (file uploads), `app_knowledge`, `memories`, `evidence_kind`, `evidence_subject`, `expiry_time` and `retain_source`. -- Any other key an item does not recognise is dropped without an error, so check spelling against the table above. +- An unrecognised field is either refused with a `400` or dropped without an error, so check spelling against the tables above. --- @@ -277,7 +270,7 @@ The label is yours to set; HydraDB never infers or changes it. Leave `context_ca } ``` -`attributes` are the fields you declared in the database's `database_metadata_schema`, and you can filter on them at query time with `attributes` on [`POST /query`](/essentials/v2/query#2-request). `custom_attributes` are free-form: they are stored with the item and cannot be filtered. Neither is returned on query chunks; read them with [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content). See [Attributes](/essentials/v2/attributes). +`attributes` are the fields you declared in the database's `database_metadata_schema`, and you can filter on them at query time with `attributes` on [`POST /query`](/essentials/v2/query#2-request). `custom_attributes` are free-form: they are stored with the item and cannot be filtered. Neither is returned on query chunks; read them from the item's row in [`POST /context/list`](/api-reference/v2/endpoint/list-documents). See [Attributes](/essentials/v2/attributes). --- @@ -357,7 +350,7 @@ Every key in `graph_payload` must match the `context_id` of an item in the same ## 14. Verify processing -Ingestion is asynchronous. Poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) with the `source_id`s from the ingest response until each item reaches `completed` or `errored`. +Ingestion is asynchronous. Poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) with the `results[].id` values from the ingest response until each item reaches `completed` or `errored`. ```python Python SDK @@ -408,46 +401,23 @@ To be notified instead of polling, register a [webhook](/essentials/v2/webhooks) --- -## 15. Before and now - -If you built against the split shapes (`documents`, `app_knowledge`, `memories`), this is what changed per field and why. - -| Before | Now | Why | -| --- | --- | --- | -| `source_id` / `id` | `context_id` | An item is a piece of context, not a source. | -| `text` / `user_assistant_pairs` | `text` / `conversation` | One item is one context. `conversation` is `{ role, content, name? }`, the shape you already send to OpenAI or Anthropic. | -| `infer` (default `false`) | `enrich` (default `true`) | Enrichment improves results, so it is on unless you turn it off. The enriched output is stored separately and comes back as `enrichment`. | -| `custom_instructions` | `instructions` (item and request level) | Steers enrichment on every shape. | -| `observation_date` | `happened_at` | The event time you state. HydraDB records the time it received the item separately. | -| `metadata` / `additional_metadata` | `attributes` / `custom_attributes` | Declared-and-filterable versus free-form. | -| `upsert` (request only) | `upsert` (item; the request value is the default) | Replace some items and append others in one call. | -| `relations` (knowledge only) | `forceful_relations` (any item) | A conversation can declare its links too. | -| `acl` (app sources only) | `acl` (any item) | Any item can be restricted. | -| (none) | `context_category` | An optional label for what kind of context this is. | -| `type` | removed | The database is unified or split, decided when it is created. | -| `documents` (file uploads), `app_knowledge`, `evidence_kind`, `evidence_subject`, `expiry_time`, `retain_source` | removed | Extract text from files and send it as an item. Structured app sources come through [connectors](/essentials/v2/connectors). | - -Split databases keep the old fields unchanged; see [Split databases and legacy fields](/essentials/v2/split-databases). - ---- - -## 16. Other ways context arrives +## 15. Other ways context arrives [Connectors](/essentials/v2/connectors) sync Slack, Notion, Google Drive, GitHub and other tools into a database on a schedule. Synced context lands in the same database as your items and is queried together with them. --- -## 17. Common mistakes +## 16. Common mistakes An item carries exactly one of `text` or `conversation`. Sending both, or neither, is a `400`. Split them into two items. - -A unified database has one corpus and takes text only. `type`, `documents`, `app_knowledge`, `memories`, `evidence_kind`, `evidence_subject`, `expiry_time` and `retain_source` are refused with a `400`. Send `context` items instead. + +Ingest takes text only. Extract the text from the file in your application and send it as a `text` item. See [Turning files into items](#turning-files-into-items). - -`source_id`, `infer`, `custom_instructions`, `observation_date`, `metadata` and `additional_metadata` are split-era names. Some are accepted as aliases, the rest are dropped. Use the names in [Item fields](#3-item-fields). + +An unrecognised field is refused or dropped, never guessed at. Use the names in [Item fields](#3-item-fields). `context_category` is validated strictly. `"business-knowledge"` is a `400`, not a silent fallback to `auto`. Use `user_preference`, `business_knowledge`, `decision_trace` or `auto`. @@ -474,4 +444,4 @@ A `202` means queued. Poll status until `graph_creation` or `completed` before e - [Query](/essentials/v2/query) - [Attributes](/essentials/v2/attributes) - [Ingest context API reference](/api-reference/v2/endpoint/ingest-context) -- [Split databases and legacy fields](/essentials/v2/split-databases) +- [Bring your own graph](/essentials/v2/bring-your-own-graph) diff --git a/essentials/v2/knowledge.mdx b/essentials/v2/knowledge.mdx index 4b2e17ad..74ce6f2f 100644 --- a/essentials/v2/knowledge.mdx +++ b/essentials/v2/knowledge.mdx @@ -1,10 +1,13 @@ --- title: "Knowledge" description: "What Knowledge is, how it differs from Memories, and how to ingest it correctly." +noindex: true --- import { Field } from "/snippets/field.jsx"; +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## 1. What it is Knowledge is the shared, database-wide context that every user and agent in a database can query - product documents, internal wikis, policy PDFs, Slack threads, Notion pages, CSVs, emails, and anything else that ought to be reusable across the workspace. diff --git a/essentials/v2/memories.mdx b/essentials/v2/memories.mdx index ed5b6f97..2b399ddd 100644 --- a/essentials/v2/memories.mdx +++ b/essentials/v2/memories.mdx @@ -1,10 +1,13 @@ --- title: "Memories" description: "How to use Memories to personalize responses with user-specific context." +noindex: true --- import { Field } from "/snippets/field.jsx"; +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## Why Memories exist Most applications start every conversation from scratch. Memories let your agent carry useful user-specific context from one session to the next, so responses become more personal over time. diff --git a/essentials/v2/query.mdx b/essentials/v2/query.mdx index ac570b38..adf9220b 100644 --- a/essentials/v2/query.mdx +++ b/essentials/v2/query.mdx @@ -1,15 +1,11 @@ --- title: "Query" -description: "One call to POST /query returns ranked chunks, graph paths, forceful relations and a prompt-ready string. Every request and response field on a unified database." +description: "One call to POST /query returns ranked chunks, graph paths, forceful relations and a prompt-ready string. Every request and response field." --- -import LegacyLine from "/snippets/legacy-line.mdx"; - Query turns stored context into the *right* context for one question. One endpoint, [`POST /query`](/api-reference/v2/endpoint/query), searches everything in the collections you name: your text and conversation items, connector content, and the [context graph](/essentials/v2/context-graphs) built from all of it. Three signals drive relevance: dense-vector similarity, BM25 keyword matching and graph traversal. -On a unified database there is no corpus selector. You never send `type`, and the response is one shape with four keys. - - +The response is one shape with four keys. --- @@ -100,6 +96,7 @@ The response `data` is exactly these four keys, inside the usual envelope: "relation": { "predicate": "managed by", "context": "Refund processing is managed by the Finance Department.", + "timestamp": 1782984600.0, "relationship_id": "rel_managed_by", "chunk_id": "ck_policy_3" }, @@ -148,7 +145,7 @@ The response `data` is exactly these four keys, inside the usual envelope: } } ], - "llm_prompt": "# Query results\n\n**Query:** who owns refund processing?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (query path, relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (chunk relation, relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" + "llm_prompt": "# Query results\n\n**Query:** who owns refund processing?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" }, "error": null, "meta": { @@ -213,7 +210,7 @@ Most integrations only need `llm_prompt`: put it in the model call and you are d | Parameter | Type / values | Purpose | | --- | --- | --- | | `graph_context` | boolean | Default `true`. Include graph paths in `graph[]`. Set `false` for chunks only; `graph` is then `[]`. | -| `follow_forceful_relations` | boolean | Default `true`. Pull in the items each hit declared with `forceful_relations` at ingest, into `forceful_relations[]`. Set `false` for `forceful_relations: []`. `query_forceful_relations` is the deprecated alias. | +| `follow_forceful_relations` | boolean | Default `true`. Pull in the items each hit declared with `forceful_relations` at ingest, into `forceful_relations[]`. Declared relations are followed only in `thinking` mode. Set `false` for `forceful_relations: []`. `query_forceful_relations` is the deprecated alias. | ### Time @@ -223,18 +220,13 @@ Most integrations only need `llm_prompt`: put it in the model call and you are d | `temporal_now` | ISO 8601 string | The time to treat as now. Set it when replaying past conversations. | | `temporal_intent` | object | Override the temporal intent HydraDB would infer from the query. | -### Do not send - -- **`type`.** Absent, `"all"` and `"unified"` are accepted and mean the one corpus; `"knowledge"` and `"memory"` are a `400`. A client branches on the database's layout from `GET /databases` (`details[].type`), never on a request flag. -- **`metadata_filters`.** Still accepted, deprecated. Use `attributes`. - --- ## 3. Response -`data` is exactly `chunks`, `graph`, `forceful_relations` and `llm_prompt`. Nothing else: no `sources`, no `graph_context`, no `additional_context`, no `temporal_facts`. +`data` is exactly `chunks`, `graph`, `forceful_relations` and `llm_prompt`, and nothing else. -`meta` on the envelope carries `request_id`, `api_version`, `latency_ms`, `database` and `collection`, plus a `deprecation` list when the request used a deprecated name. It has no `tenant_id`, `sub_tenant_id` or `source_type`. `collection` is present when the query searched one collection (named, or the default); a `collections` fan-out omits it. +`meta` on the envelope carries `request_id`, `api_version`, `latency_ms`, `database` and `collection`, plus a `deprecation` list when the request used a deprecated name. `collection` is present when the query searched one collection (named, or the default); a `collections` fan-out omits it. ### `chunks[]` @@ -248,15 +240,15 @@ The matched pieces of your items, ranked. Preserve the order. | `content` | string | The chunk's own text, verbatim. Enrichment is not concatenated into it. | | `enrichment` | string | What enrichment extracted from this chunk: the extracted statement (a preference, a fact). Omitted when enrichment extracted nothing. | | `enrichment_kind` | string | The item's declared `context_category`: `user_preference`, `business_knowledge` or `decision_trace`. Omitted when none was declared (`auto`); present even when `enrichment` is omitted. | -| `temporal` | array | Present only when the query engaged temporal reasoning. Each entry is `{ content, start_date, end_date }`; dates may be `null`. | +| `temporal` | array | Present only when the query engaged temporal reasoning. Each entry is `{ content, start_date, end_date }`: `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` (only the dated sides are printed), and either date may be `null`. | -**Chunks carry nothing about their source.** No title, url, collection, timestamps or attributes. `llm_prompt` prints the title, collection, type, last-updated date and url for the model; to show any of that yourself, call [`GET /context/inspect?database=acme&id=`](/api-reference/v2/endpoint/fetch-content) with the chunk's `context_id`. +**Chunks carry nothing about their source.** No title, url, collection, timestamps or attributes. `llm_prompt` prints the title, collection, type, last-updated date and url for the model. To show an item's title, timestamp or attributes yourself, call [`POST /context/list`](/api-reference/v2/endpoint/list-documents) with `ids: [""]`; [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content) returns its stored content. ### `graph[]` -One flat array of paths through the [context graph](/essentials/v2/context-graphs): paths grown from the query first, then paths expanded from the returned chunks, deduplicated. A path both lanes found is reported once, as a `query_path`. `[]` when `graph_context` is `false` or nothing connects. +One flat array of paths through the [context graph](/essentials/v2/context-graphs): paths grown from the query first, then paths expanded from the returned chunks. The array is deduplicated across both lanes (a path both lanes found is reported once, as a `query_path`) and is not capped: every path that survives deduplication is returned. `[]` when `graph_context` is `false` or nothing connects. | Field | Type | Meaning | | --- | --- | --- | @@ -266,14 +258,15 @@ One flat array of paths through the [context graph](/essentials/v2/context-graph | `triplets[].target` | object | `{ entity_id, name }`. | | `triplets[].relation.predicate` | string | The relation, for example `subscribed to`. | | `triplets[].relation.context` | string | The sentence the relation was extracted from. | -| `triplets[].relation.temporal_details` | string | When the relation holds, for example `since June`. Absent when unknown. | +| `triplets[].relation.temporal_details` | string | When the relation holds, for example `since June`. Omitted when empty. | +| `triplets[].relation.timestamp` | number | The relation's timestamp in Unix epoch seconds, as a float (for example `1782984600.0`). Omitted when the edge has none. | | `triplets[].relation.relationship_id` | string | The relation's id. | | `triplets[].relation.chunk_id` | string | The chunk this relation was extracted from. Use it to attach the hop to a chunk, below. | | `path_summary` | string | One sentence summarizing the whole path. Never empty: when the server wrote no summary for a path, it narrates the hops, such as `Priya owns refund processing.` | ### Attaching graph paths to chunks -Every hop carries `relation.chunk_id`, the chunk its edge was extracted from. That is all you need to show a chunk's relations under the chunk; it replaces the split response's `chunk_id_to_group_ids`. +Every hop carries `relation.chunk_id`, the chunk its edge was extracted from. That is all you need to show a chunk's relations under the chunk. - **Group hops by `relation.chunk_id`** and match it against `chunks[].chunk_id` (and `forceful_relations[].chunk.chunk_id`). - **A `chunk_relation` path hangs under a chunk in the answer.** It is only returned when one of its hops came from a returned chunk (or a `forceful_relations` chunk); hang it under that chunk. @@ -289,10 +282,10 @@ context_of = {c.chunk_id: c.context_id for c in data.chunks} context_of.update({r.chunk.chunk_id: r.chunk.context_id for r in data.forceful_relations}) hops_under = {} # chunk_id -> hops extracted from that chunk -query_paths = [] +own_group = [] # query_path paths, shown as their own group for path in data.graph: if path.origin == "query_path": - query_paths.append(path) + own_group.append(path) for hop in path.triplets: if hop.relation.chunk_id in context_of: hops_under.setdefault(hop.relation.chunk_id, []).append(hop) @@ -304,9 +297,9 @@ for (const c of data.chunks) contextOf.set(c.chunkId, c.contextId); for (const r of data.forcefulRelations) contextOf.set(r.chunk.chunkId, r.chunk.contextId); const hopsUnder = new Map(); // chunkId -> hops extracted from that chunk -const queryPaths = []; +const ownGroup = []; // query_path paths, shown as their own group for (const path of data.graph) { - if (path.origin === "query_path") queryPaths.push(path); + if (path.origin === "query_path") ownGroup.push(path); for (const hop of path.triplets) { const id = hop.relation.chunkId; if (!contextOf.has(id)) continue; @@ -319,7 +312,7 @@ for (const path of data.graph) { ### `forceful_relations[]` -Chunks pulled in because the caller declared `forceful_relations` at ingest: the same name as the ingest field and the `follow_forceful_relations` switch. They are linked to a result by the author, not ranked for this query. `[]` when none were declared or `follow_forceful_relations` is `false`. +Chunks pulled in because the caller declared `forceful_relations` at ingest: the same name as the ingest field and the `follow_forceful_relations` switch. They are linked to a result by the author, not ranked for this query, and they are followed only in `thinking` mode. `[]` when none were declared, `follow_forceful_relations` is `false`, or the query ran in `fast` mode. | Field | Type | Meaning | | --- | --- | --- | @@ -350,11 +343,11 @@ The sections, in order: | `# Query results` | `**Query:**` (the query), a `**Found:**` line counting what follows, and the line telling the model to cite a result by its number. | | `## Results` | One block per entry of `chunks[]`, in ranked order, separated by `---`: a `### 1. title` heading; a line with `**Relevance:**` (the `score`), `**Collection:**`, `**Type:**` and `**Category:**` (the `enrichment_kind`); a line with `**Id:**` (the `context_id`) and `**Last updated:**`; the chunk's `content`; then `**Enrichment:**` with the `enrichment`. | | `## Forceful relations` | A guide line, then one `### R1. title` block per entry of `forceful_relations[]`, laid out like a result, with `**Linked from:**` (the `via.from` context, when it is not `""`) in place of `**Relevance:**`. | -| `## Related facts` | One line per path in `graph[]`, such as `- [P1] **A** -pred→ **B** (query path, relevance 0.81) [1]`: the path's label, its chain of hops, `query path` or `chunk relation` for its `origin` with the path's relevance after reranking (printed only here: `graph[]` carries no score), and the results its hops were extracted from. A path with no reranked score, such as a graph summary a `thinking` query builds, prints no relevance. The `path_summary` is indented on the line under it, unless it only narrates the hops the chain already shows. | +| `## Related facts` | One line per path in `graph[]`, such as `- [P1] **A** -pred→ **B** (relevance 0.81) [1]`: the path's label, its chain of hops, the path's relevance after reranking in parentheses (printed only here: `graph[]` carries no score), and the results its hops were extracted from. A path with no reranked score, such as a graph summary a `thinking` query builds, has no parenthetical at all: `- [P3] **A** -pred→ **B** [1]`. The line never says which lane found the path. The `path_summary` is indented on the line under it, unless it only narrates the hops the chain already shows. | | `## Temporal facts` | One line per dated fact the query engaged (the facts behind `chunks[].temporal`): subject, relation and object, then the resolved window, precision and status, with the evidence phrase set apart after a `;`, citing its result. | | `## Sources` | Each context once, in order of first appearance: title, type, id, url and last-updated date. Only web (`http` or `https`) links are printed; a storage location such as `s3://...` never is. | -`**Type:**` is what the item is: the connector's word for it (a Slack `message`, a Jira `ticket`) when a connector set one, otherwise its source type, such as `file`. It is never `memory` or `knowledge`: a unified database has no such split. A field with no value is left out of its line. +`**Type:**` is what the item is: the connector's word for it (a Slack `message`, a Jira `ticket`) when a connector set one, otherwise its source type, such as `file`. A field with no value is left out of its line. | Label | Refers to | | --- | --- | @@ -413,9 +406,9 @@ FAQ: refunds to a card take 5 to 7 business days to appear. ## Related facts -- [P1] **Refund Processing** -managed by→ **Finance Department** (query path, relevance 0.81) [1] +- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1] Refund processing is managed by the Finance Department. -- [P2] **User** -prefers→ **short answers** (chunk relation, relevance 0.74) [2] +- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2] The user prefers short answers about refunds. ## Temporal facts @@ -441,7 +434,7 @@ const messages = [{ role: "system", content: result.data.llmPrompt }, { role: "user", content: question }]; ``` -Surface it to your agent verbatim, and let the model cite the labels. The SDK `build_string` / `buildString` helpers return `llm_prompt` verbatim on a unified database. When you need structured output instead, read `chunks[].content`, `chunks[].enrichment`, `chunks[].enrichment_kind`, `graph[].path_summary` and `forceful_relations[]`. See [How to Use API Results](/essentials/v2/api-results). +Surface it to your agent verbatim, and let the model cite the labels. When you need structured output instead, read `chunks[].content`, `chunks[].enrichment`, `chunks[].enrichment_kind`, `graph[].path_summary` and `forceful_relations[]`. See [How to Use API Results](/essentials/v2/api-results). --- @@ -460,8 +453,6 @@ Most of the time the defaults are right. When they are not, here is where to sta ### Production checklist - **Set per-call timeouts.** Generous for `thinking` (3 to 5 s), tight for `fast` (500 ms or less). For `auto`, size the timeout for the `thinking` case. -- **Detect the layout once per database** (`GET /databases`, `details[].type`) and cache it. Unified: this page. Split: the old request and response, byte for byte. -- **Parse by shape.** A response with `llm_prompt` and a `graph` array is the unified shape; one with `graph_context` or `chunk_content` is the split shape. Stored logs and split databases keep producing the old one. - **Cache by request.** If the same query repeats inside a session, cache `data` keyed by `(database, collections, query, query_by, mode)`. Sort list values and object keys before building the key. --- @@ -470,23 +461,16 @@ Most of the time the defaults are right. When they are not, here is where to sta | Symptom | Cause | Fix | | --- | --- | --- | -| `400` mentioning `type` | `type: "knowledge"` or `"memory"` sent to a unified database | Drop `type`. Scope with `collection` or `collections`. | | `graph` is `[]` | `graph_context: false`, or `mode` resolved to `fast`, or nothing connects the results | Set `graph_context: true` with `mode: "thinking"`. An empty array is normal when there is nothing to return. | -| `forceful_relations` is `[]` | Nothing in the hits declared `forceful_relations`, or `follow_forceful_relations: false` | Declare relations at ingest; leave the flag on. | +| `forceful_relations` is `[]` | Nothing in the hits declared `forceful_relations`, `follow_forceful_relations: false`, or the query ran in `fast` mode | Declare relations at ingest, leave the flag on, and use `mode: "thinking"`. | | Recent items do not appear | Indexing not finished | Poll `GET /context/status?ids=...&database=...`; chunks are invisible until processing reaches at least `graph_creation`. | | `attributes` does not narrow results | The key is not declared in `database_metadata_schema`, or the value does not match | Declare the field and send it in `attributes` at ingest; filter with an operator such as `$eq`. `custom_attributes` are never filterable. | -| Chunk has no title or url | Chunks carry no source details by design | Call `GET /context/inspect` with the chunk's `context_id`. | +| Chunk has no title or url | Chunks carry no source details by design | Read them from `llm_prompt`, or call `POST /context/list` with `ids: [""]`. | | `operator: "phrase"` ignored | `query_by` is not `"text"` | `operator` only applies to BM25 text query. | | `graph_context` value ignored | `mode` is `auto` (or omitted) | `auto` overrides `graph_context` to match the pipeline it picks. Set `mode` explicitly. | --- -## 6. Split databases - -A database created with `type: "split"` keeps the old query contract exactly as it was: `type` selects `knowledge`, `memory` or `all`, `query_forceful_relations` fetches declared relations into `additional_context`, and the response carries `chunks[].chunk_content`, `sources[]`, `graph_context` and `additional_context`. None of that appears on a unified database, and the SDK `build_string` helper is only needed there. The full mapping from each old field to its unified counterpart is on [Split databases and legacy fields](/essentials/v2/split-databases#4-legacy-query-and-response-fields). - ---- - ## Related - [How to Use API Results](/essentials/v2/api-results): injecting `llm_prompt` and reading the four keys diff --git a/essentials/v2/semantic-search.mdx b/essentials/v2/semantic-search.mdx index b49ef8e9..1557d264 100644 --- a/essentials/v2/semantic-search.mdx +++ b/essentials/v2/semantic-search.mdx @@ -17,7 +17,7 @@ Semantic search is useful because it retrieves by meaning instead of exact wordi `POST /query` is the single retrieval endpoint. Two parameters decide what runs: -- **`collections`** (or `collection`) picks where to look: one collection, or several with weights. There is no corpus selector; a unified database is one corpus. +- **`collections`** (or `collection`) picks where to look: one collection, or several with weights. - **`query_by`** picks the retrieval method: `"hybrid"` (semantic + BM25, the default) or `"text"` (BM25 only, with `operator: "or" | "and" | "phrase"`). --- @@ -32,7 +32,7 @@ Pure vector search can miss important production constraints: - Different users can need different context for the same query. - Relationship questions need graph context, not only similar text chunks. -That is why HydraDB exposes semantic retrieval through `query_by: "hybrid"` inside the unified `/query` endpoint rather than as a separate pure-vector mode. +That is why HydraDB exposes semantic retrieval through `query_by: "hybrid"` inside the single `/query` endpoint rather than as a separate pure-vector mode. --- @@ -290,7 +290,7 @@ result = client.query( 1. Call `POST /query` with the right `collections` and `query_by` for the query. 2. Inject `llm_prompt` into your model call, with a grounding instruction. 3. Ask your LLM to answer using only that context and to cite the bracketed labels. -4. Read `chunks`, `graph` and `forceful_relations` when you need structure, and `GET /context/inspect` for a chunk's source details. +4. Read `chunks`, `graph` and `forceful_relations` when you need structure, and `POST /context/list` with the chunk's `context_id` in `ids` for its source details. See [How to Use API Results](/essentials/v2/api-results) for complete examples. diff --git a/essentials/v2/split-databases.mdx b/essentials/v2/split-databases.mdx index 7608470a..35da3bff 100644 --- a/essentials/v2/split-databases.mdx +++ b/essentials/v2/split-databases.mdx @@ -1,8 +1,11 @@ --- title: "Split databases and legacy fields" description: "How databases created as type split work, the knowledge, memory and all selector, and every older field name with its current replacement." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Every other page in these docs describes a **unified** database: you send `context` items and query without a corpus selector. This page covers the older model, for integrations that still use it: - databases created with `type: "split"`, including every database created before unified became the default diff --git a/essentials/v2/webhooks.mdx b/essentials/v2/webhooks.mdx index 5aa09583..53e336dd 100644 --- a/essentials/v2/webhooks.mdx +++ b/essentials/v2/webhooks.mdx @@ -3,7 +3,7 @@ title: "Webhooks" description: "Receive indexing status events when ingested content finishes processing." --- -Webhooks let your application receive an HTTP callback when HydraDB finishes processing a document or memory. +Webhooks let your application receive an HTTP callback when HydraDB finishes processing an ingested item. Use them when you want to: @@ -313,7 +313,7 @@ For failed indexing, the payload can include `error_code` and `error_message`: |---|---| | `event` | Event type. Currently `indexing.status_changed`. | | `delivery_id` | Stable ID for this event. Store it to deduplicate retries. | -| `id` | The document, memory, or app item ID you supplied during ingestion. | +| `id` | The item's `context_id`: the one you supplied at ingestion, or the generated one. For connector-synced content, the connector item's id. | | `database` | The name of the database you ingested into - the value you sent as `database` (or `tenant_id`) on the ingest request. Empty only for items ingested before this field existed. | | `collection` | Collection scope for the indexed item. | | `status` | Terminal indexing status. Usually `completed` or `errored`. | @@ -324,7 +324,7 @@ For failed indexing, the payload can include `error_code` and `error_message`: | `sub_tenant_id` | Deprecated alias for `collection`, carrying the same value. | -Older examples may refer to this document identifier as `doc_id`. New webhook payloads use `id`. +Older examples may refer to this identifier as `doc_id`. New webhook payloads use `id`. @@ -785,4 +785,4 @@ The overlap lives in your receiver, not in HydraDB. Each delivery carries a sing | Signatures started failing after a rotation | Rotation applies immediately. Confirm your receiver has the new secret deployed, and see [Zero-downtime key rotation](#zero-downtime-key-rotation) to avoid the gap next time. | | Event arrives more than once | This is expected during retries. Deduplicate with `delivery_id`. | | Event never arrives | Check the dashboard delivery history for `failed` or `permanently_failed`. | -| `id` is unexpected | It is the ID you supplied at ingestion, such as document ID, memory ID, or app item ID. | +| `id` is unexpected | It is the item's `context_id` (the one you supplied, or the generated one), or the connector item's id for synced content. | diff --git a/essentials/webhooks.mdx b/essentials/webhooks.mdx index 87d451f9..6ab9e3d6 100644 --- a/essentials/webhooks.mdx +++ b/essentials/webhooks.mdx @@ -1,8 +1,11 @@ --- title: "Webhooks" description: "Receive indexing status events when ingested content finishes processing." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + Webhooks let your application receive an HTTP callback when HydraDB finishes processing a document or memory. Use them when you want to: diff --git a/get-started/core-concepts.mdx b/get-started/core-concepts.mdx index c8dd3ca3..ebacc5e7 100644 --- a/get-started/core-concepts.mdx +++ b/get-started/core-concepts.mdx @@ -1,8 +1,11 @@ --- title: "Core Concepts" description: "A tour of the five primitives that make HydraDB work." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + > A short overview of each primitive, with links to deeper [Essentials](/essentials/architecture) pages. ## The five primitives diff --git a/get-started/introduction.mdx b/get-started/introduction.mdx index 19ca06c0..71708e7e 100644 --- a/get-started/introduction.mdx +++ b/get-started/introduction.mdx @@ -1,8 +1,11 @@ --- title: "Introduction" description: "The unified context substrate for AI. One brain for semantic knowledge, user memories, and episodic experiences." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + ## What HydraDB is HydraDB is a unified context substrate for AI agents. One, three primitives: diff --git a/get-started/quickstart.mdx b/get-started/quickstart.mdx index c3cde2ac..20d67cf2 100644 --- a/get-started/quickstart.mdx +++ b/get-started/quickstart.mdx @@ -1,8 +1,11 @@ --- title: "Quickstart" description: "Build your first working recall in five minutes." +noindex: true --- +This page is deprecated: it documents the knowledge and memory API, which unified databases replace. See [Ingest context](/essentials/v2/ingest) and [Query](/essentials/v2/query). + --- ## What you'll build diff --git a/get-started/v2/core-concepts.mdx b/get-started/v2/core-concepts.mdx index 4d99fd78..89cf828c 100644 --- a/get-started/v2/core-concepts.mdx +++ b/get-started/v2/core-concepts.mdx @@ -25,7 +25,7 @@ One database holds all the context your AI needs, and it holds three kinds: - **Business knowledge:** what your company knows - **Decision traces:** what your agents and teams decided, and why -You split the database into **collections**, typically one per person plus one or more shared ones. You ingest everything as **items**. You read it back with one **query** that can weigh a person's collection above the shared ones, so answers are grounded in company knowledge and personalized for the person asking. +You partition the database into **collections**, typically one per person plus one or more shared ones. You ingest everything as **items**. You read it back with one **query** that can weigh a person's collection above the shared ones, so answers are grounded in company knowledge and personalized for the person asking. --- @@ -88,7 +88,7 @@ Personalize by querying several collections with weights: } ``` -Tune it with `query_by` (`hybrid` or `text`), `mode` (`auto`, `fast` or `thinking`) and `graph_context`. There is no `type`: a unified database is one corpus. +Tune it with `query_by` (`hybrid` or `text`), `mode` (`auto`, `fast` or `thinking`) and `graph_context`. The response is four keys: `chunks` (ranked matches with `content`, `score` and `enrichment`), `graph` (relation paths, each with a `path_summary`), `forceful_relations` (items you linked at ingest) and `llm_prompt`, a server-built string with citation labels that you inject into your model call as is. diff --git a/get-started/v2/introduction.mdx b/get-started/v2/introduction.mdx index 1af7ff3f..4aaa3beb 100644 --- a/get-started/v2/introduction.mdx +++ b/get-started/v2/introduction.mdx @@ -35,7 +35,7 @@ HydraDB is designed for teams building scalable, stateful AI agents, whether you We give you primitives so that you can build your own context stores, memory layers, and workflows that require context for your AI. Think of us a graph-native context delivery mechanism for your agents. -The graph, the memory primitives, the retrieval pipeline, and the ranking knobs are yours to compose. Your context. Your opinions. +The graph, the context primitives, the retrieval pipeline, and the ranking knobs are yours to compose. Your context. Your opinions. --- @@ -58,8 +58,6 @@ View the full technical breakdown in our [benchmarks](https://benchmarks.hydradb - Internal knowledge assistants spanning Slack, Notion, Drive, and email - Consumer AI apps where every user gets a "second brain" that evolves over time -Find more use cases at [Cookbooks](/cookbooks/v2/index). - --- ## Get started @@ -72,4 +70,4 @@ For enterprise onboarding, contact [founders@hydradb.com](mailto:founders@hydrad ## For AI agents -For AI coding agents and IDE assistants, use the [HydraDB Agent Integration Guide](/AGENTS) and the [v2 OpenAPI spec](/api-reference/v2/openapi.json). Ingest `context` items, and query without a corpus selector; the query returns `llm_prompt`, ready to inject. \ No newline at end of file +For AI coding agents and IDE assistants, use the [HydraDB Agent Integration Guide](/AGENTS) and the [v2 OpenAPI spec](/api-reference/v2/openapi.json). Ingest `context` items and query them with `POST /query`; the query returns `llm_prompt`, ready to inject. \ No newline at end of file diff --git a/get-started/v2/quickstart.mdx b/get-started/v2/quickstart.mdx index ddf66f9c..ee368a08 100644 --- a/get-started/v2/quickstart.mdx +++ b/get-started/v2/quickstart.mdx @@ -273,7 +273,7 @@ flowchart LR style E fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc,stroke-linecap:round ``` -Steps 1 and 3 are **asynchronous**: HydraDB provisions infrastructure and indexes your content in the background, so each needs a short polling loop. Steps 2, 4 and 5 run in real time. The same loop scales as your corpus grows; nothing in the code changes between 10 items and 10,000. +Steps 1 and 3 are **asynchronous**: HydraDB provisions infrastructure and indexes your content in the background, so each needs a short polling loop. Steps 2, 4 and 5 run in real time. The same loop scales as your data grows; nothing in the code changes between 10 items and 10,000. --- @@ -288,6 +288,5 @@ Steps 1 and 3 are **asynchronous**: HydraDB provisions infrastructure and indexe | Scope data per user or workspace | [Databases and collections](/essentials/v2/databases-and-collections) | | Inject `llm_prompt` and map citations back | [How to Use API Results](/essentials/v2/api-results) | | See the full endpoint reference | [API Reference](/api-reference/v2) | -| Pick from real-world recipes | [Cookbooks](/cookbooks/v2/index) | Stuck? Reach out at [founders@hydradb.com](mailto:founders@hydradb.com). diff --git a/mintlify-hygiene.toml b/mintlify-hygiene.toml index 00bb1e03..bb6e1213 100644 --- a/mintlify-hygiene.toml +++ b/mintlify-hygiene.toml @@ -8,12 +8,78 @@ exclude = [ # Intentionally hidden from nav (private, reachable only by direct URL) per PR #141; # excluded so the nav_registration rule does not flag it. "essentials/v2/graph-collections-byog.mdx", - # Split-database pages (PRO-1618): still live, linked only from - # essentials/v2/split-databases, so they are out of nav on purpose. + # Deprecated knowledge and memory pages (PRO-1618): the whole v1 version, + # the v2 split-database pages and the cookbooks. Still live by URL, marked + # noindex and deprecated, and out of nav on purpose. + "api-reference/endpoint/add-memory.mdx", + "api-reference/endpoint/boolean-recall.mdx", + "api-reference/endpoint/create-tenant.mdx", + "api-reference/endpoint/delete-knowledge.mdx", + "api-reference/endpoint/delete-memory.mdx", + "api-reference/endpoint/delete-tenant.mdx", + "api-reference/endpoint/fetch-content.mdx", + "api-reference/endpoint/full-recall.mdx", + "api-reference/endpoint/graph-relations.mdx", + "api-reference/endpoint/infra-status.mdx", + "api-reference/endpoint/ingestion-overview.mdx", + "api-reference/endpoint/list-data.mdx", + "api-reference/endpoint/list-overview.mdx", + "api-reference/endpoint/list-sub-tenant-ids.mdx", + "api-reference/endpoint/list-tenant-ids.mdx", + "api-reference/endpoint/memories-overview.mdx", + "api-reference/endpoint/monitor-tenant.mdx", + "api-reference/endpoint/recall-overview.mdx", + "api-reference/endpoint/recall-preferences.mdx", + "api-reference/endpoint/tenants-overview.mdx", + "api-reference/endpoint/upload-knowledge.mdx", + "api-reference/endpoint/verify-processing.mdx", + "api-reference/error-responses.mdx", + "api-reference/index.mdx", + "api-reference/sdks.mdx", + "cookbooks/ai-chief-of-staff.mdx", + "cookbooks/ai-linkedin-recruiter.mdx", + "cookbooks/ai-onboarding-agent.mdx", + "cookbooks/ai-travel-planner.mdx", + "cookbooks/competitive-intelligence-agent.mdx", + "cookbooks/cookbook-01-build-cursor-for-docs.mdx", + "cookbooks/cookbook-04-build-notion-ai.mdx", + "cookbooks/cookbook-10-ai-financial-analyst.mdx", + "cookbooks/customer-support-agent.mdx", + "cookbooks/glean-clone.mdx", + "cookbooks/hydradb-cookbook-06.mdx", + "cookbooks/index.mdx", + "cookbooks/internal-search-perplexity.mdx", + "cookbooks/v2/ai-chief-of-staff.mdx", + "cookbooks/v2/ai-linkedin-recruiter.mdx", + "cookbooks/v2/ai-onboarding-agent.mdx", + "cookbooks/v2/ai-travel-planner.mdx", + "cookbooks/v2/competitive-intelligence-agent.mdx", + "cookbooks/v2/cookbook-01-build-cursor-for-docs.mdx", + "cookbooks/v2/cookbook-04-build-notion-ai.mdx", + "cookbooks/v2/cookbook-10-ai-financial-analyst.mdx", + "cookbooks/v2/customer-support-agent.mdx", + "cookbooks/v2/glean-clone.mdx", + "cookbooks/v2/hydradb-cookbook-06.mdx", + "cookbooks/v2/index.mdx", + "cookbooks/v2/internal-search-perplexity.mdx", + "essentials/api-results.mdx", + "essentials/app-sources.mdx", + "essentials/architecture.mdx", + "essentials/context-graphs.mdx", + "essentials/knowledge.mdx", + "essentials/memories.mdx", + "essentials/metadata.mdx", + "essentials/multi-tenant.mdx", + "essentials/recall.mdx", + "essentials/semantic-search.mdx", + "essentials/v2/app-sources.mdx", "essentials/v2/knowledge.mdx", "essentials/v2/memories.mdx", - "essentials/v2/app-sources.mdx", - "essentials/v2/bring-your-own-graph.mdx", + "essentials/v2/split-databases.mdx", + "essentials/webhooks.mdx", + "get-started/core-concepts.mdx", + "get-started/introduction.mdx", + "get-started/quickstart.mdx", ] mdx_parse_mode = "strict" diff --git a/plugins/claude-code.mdx b/plugins/claude-code.mdx index a1079d1f..eea0752e 100644 --- a/plugins/claude-code.mdx +++ b/plugins/claude-code.mdx @@ -25,27 +25,27 @@ description: "HydraDB plugin for Claude Code. Persistent memory and contextual a ```bash - echo 'export HYDRA_DB_API_KEY="your-api-key"' >> ~/.bashrc - echo 'export HYDRADB_TENANT_ID="your-tenant-id"' >> ~/.bashrc - echo 'export HYDRADB_SUB_TENANT_ID=""' >> ~/.bashrc + echo 'export HYDRADB_API_KEY="your-api-key"' >> ~/.bashrc + echo 'export HYDRADB_DATABASE="your-database"' >> ~/.bashrc + echo 'export HYDRADB_COLLECTION=""' >> ~/.bashrc source ~/.bashrc ``` ```bash - echo 'export HYDRA_DB_API_KEY="your-api-key"' >> ~/.zshrc - echo 'export HYDRADB_TENANT_ID="your-tenant-id"' >> ~/.zshrc - echo 'export HYDRADB_SUB_TENANT_ID=""' >> ~/.zshrc + echo 'export HYDRADB_API_KEY="your-api-key"' >> ~/.zshrc + echo 'export HYDRADB_DATABASE="your-database"' >> ~/.zshrc + echo 'export HYDRADB_COLLECTION=""' >> ~/.zshrc source ~/.zshrc ``` ```powershell - [System.Environment]::SetEnvironmentVariable("HYDRA_DB_API_KEY", "your-api-key", "User") - [System.Environment]::SetEnvironmentVariable("HYDRADB_TENANT_ID", "your-tenant-id", "User") - [System.Environment]::SetEnvironmentVariable("HYDRADB_SUB_TENANT_ID", "", "User") + [System.Environment]::SetEnvironmentVariable("HYDRADB_API_KEY", "your-api-key", "User") + [System.Environment]::SetEnvironmentVariable("HYDRADB_DATABASE", "your-database", "User") + [System.Environment]::SetEnvironmentVariable("HYDRADB_COLLECTION", "", "User") ``` @@ -55,14 +55,14 @@ description: "HydraDB plugin for Claude Code. Persistent memory and contextual a ```bash - echo $HYDRA_DB_API_KEY - echo $HYDRADB_TENANT_ID + echo $HYDRADB_API_KEY + echo $HYDRADB_DATABASE ``` ```powershell - echo $env:HYDRA_DB_API_KEY - echo $env:HYDRADB_TENANT_ID + echo $env:HYDRADB_API_KEY + echo $env:HYDRADB_DATABASE ``` @@ -71,20 +71,41 @@ description: "HydraDB plugin for Claude Code. Persistent memory and contextual a ```bash /hydradb:setup - /hydradb:status + /hydradb:doctor ``` - `HYDRADB_SUB_TENANT_ID` must be set explicitly. If you want the plugin to use its default sub-tenant, set it to an empty string (`""`) rather than omitting it entirely. + `HYDRADB_COLLECTION` must be set explicitly. If you want the plugin to use the database's default collection, set it to an empty string (`""`) rather than omitting it entirely. -**`HYDRADB_SUB_TENANT_ID` strategy:** + + The older `HYDRA_DB_API_KEY`, `HYDRADB_TENANT_ID` and `HYDRADB_SUB_TENANT_ID` spellings still work, but each prints a one-line deprecation warning. The canonical name wins when both are set. + + +**`HYDRADB_COLLECTION` strategy:** - Use a workspace-specific value (e.g., `claude-my-project`) for project-isolated memory. - Use a shared value to carry memory across all sessions and workspaces. -Once configured, the plugin runs in the background: it syncs workspace docs on session start, recalls relevant context before each prompt, and captures conversation memory after each response. +Once configured, the plugin runs in the background: it syncs workspace docs on session start, queries HydraDB before each prompt and injects what it finds, and captures the conversation after each response. + +--- + +## How it works + +The plugin talks to your database through two endpoints. + +- **Recall.** Before each prompt, it sends the prompt text to `POST /query`, with `mode` from `recallMode`, `graph_context` from `graphContext`, `follow_forceful_relations` from `followForcefulRelations`, and `max_results` set to `maxMemoryResults + maxKnowledgeResults` (10 by default). It injects the server-built `llm_prompt` verbatim, and whole, inside a `` block: ranked results, forceful relations, related facts from the context graph, temporal facts and sources, labelled `[1]`, `[R1]` and `[P1]` for citation. +- **Capture.** Conversations, notes and workspace docs are sent to `POST /context/ingest` as `context` items with enrichment on: + - `turn` capture sends each exchange as a `conversation` item, with your turns named after `userName` when it is set. + - `session-upsert` capture keeps one `text` item per session, holding the session transcript, and replaces it after each response. + - `/hydradb:ingest --note` sends the note as a `text` item. + - Workspace sync sends each matching file as a `text` item titled with its relative path. A changed file replaces its item, and a full sync removes the item of a deleted or excluded file. + + `memoryCustomInstructions` steers enrichment for conversations, sessions and notes, and `workspaceMemoryCustomInstructions` for workspace docs. Secret-looking content is redacted before anything leaves the workspace. + +See [Query](/essentials/v2/query) and [Ingest](/essentials/v2/ingest) for the full request and response shapes. --- @@ -93,11 +114,14 @@ Once configured, the plugin runs in the background: it syncs workspace docs on s | Command | Description | | ------- | ----------- | | `/hydradb:setup` | Interactive configuration wizard | -| `/hydradb:status` | Show current config and connection status | -| `/hydradb:search ` | Search HydraDB manually | -| `/hydradb:remember ` | Save a note to memory | -| `/hydradb:save-session` | Manually save the full session to HydraDB | -| `/hydradb:sync-workspace [--force]` | Force re-sync workspace docs | +| `/hydradb:doctor` | Show current config, sync state and connection status | +| `/hydradb:query ` | Query HydraDB manually | +| `/hydradb:ingest --note ` | Save a note to HydraDB | +| `/hydradb:ingest --session` | Manually save the full session to HydraDB | +| `/hydradb:ingest [--force]` | Sync workspace docs now (`--force` re-sends unchanged files) | +| `/hydradb:last-recall` | Show what the most recent automatic recall returned and injected | + +The earlier names `/hydradb:status`, `/hydradb:search`, `/hydradb:remember`, `/hydradb:save-session` and `/hydradb:sync-workspace` still work. --- @@ -108,35 +132,19 @@ Once configured, the plugin runs in the background: it syncs workspace docs on s | Value | Behavior | | ---------------- | ------------------------------------------------------------------------------------------- | | `session-upsert` | **(default)** Maintains one evolving session transcript, upserted after each response | -| `turn` | Saves each user/assistant exchange as an isolated memory | -| `both` | Saves isolated turns and a rolling session transcript | -| `off` | No automatic saves; manual saves still work via `/hydradb:save-session` | - -### `searchMode` - what is recalled at prompt time - -| Value | Behavior | -| ----------- | ---------------------------------------------------------------------- | -| `memory` | **(default)** Recalls from [Memories](/essentials/memories) only | -| `knowledge` | Recalls from [Knowledge](/essentials/knowledge) only | -| `both` | Recalls from both in parallel, then merges results | - -### `ingestionMode` - where workspace docs are synced - -| Value | Behavior | -| ----------- | ------------------------------------------------------------------------------------------------------ | -| `memory` | **(default)** Syncs workspace docs into [Memories](/essentials/memories) using `infer` | -| `knowledge` | Syncs workspace docs into the [Knowledge](/essentials/knowledge) base | -| `auto` | Prefers memory; falls back to knowledge for larger files. Pair with `searchMode: "both"` | +| `turn` | Saves each user/assistant exchange as its own conversation item | +| `both` | Saves individual turns and a rolling session transcript | +| `off` | No automatic saves; manual saves still work via `/hydradb:ingest --session` | ### `recallMode` - speed vs. depth | Value | Behavior | | ---------- | ---------------------------------------------- | | `fast` | **(default)** Lower latency, standard recall | -| `thinking` | Deeper reasoning-based recall via graph traversal | +| `thinking` | Deeper reasoning-based recall via graph traversal; also follows forceful relations | - By default, HydraDB syncs each conversation pair (user and assistant) that does not include the `ignoreMarker` (`hydra-ignore`). For manual-only capture, set `captureMode` to `off` and use `/hydradb:save-session` or `/hydradb:remember `. + By default, HydraDB syncs each conversation pair (user and assistant) that does not include the `ignoreMarker` (`hydra-ignore`). For manual-only capture, set `captureMode` to `off` and use `/hydradb:ingest --session` or `/hydradb:ingest --note `. --- @@ -147,33 +155,30 @@ Once configured, the plugin runs in the background: it syncs workspace docs on s | Variable | Env var | Description | | ------------- | ------------------------ | ---------------------------------------------- | -| `apiKey` | `HYDRA_DB_API_KEY` | Your HydraDB API key | -| `tenantId` | `HYDRADB_TENANT_ID` | Your HydraDB tenant ID | -| `subTenantId` | `HYDRADB_SUB_TENANT_ID` | Sub-tenant scope. Set to `""` for default | +| `apiKey` | `HYDRADB_API_KEY` | Your HydraDB API key | +| `tenantId` | `HYDRADB_DATABASE` | Your HydraDB database | +| `subTenantId` | `HYDRADB_COLLECTION` | Collection within the database. Set to `""` for the default collection | ### Core behavior -| Variable | Default | Description | -| -------------- | ---------------- | -------------------------------------------------------------------- | -| `autoRecall` | `true` | Inject HydraDB context before every prompt automatically | -| `autoIngest` | `true` | Sync workspace docs on session start and file edits automatically | -| `captureMode` | `session-upsert` | How conversations are saved (see [Modes](#modes)) | -| `searchMode` | `memory` | What to recall at prompt time (see [Modes](#modes)) | -| `ingestionMode`| `memory` | Where workspace docs are synced (see [Modes](#modes)) | -| `recallMode` | `fast` | Recall depth: `fast` or `thinking` | -| `graphContext` | `true` | Include entity paths and graph relations in recalled context | +| Variable | Default | Description | +| ------------------------- | ---------------- | -------------------------------------------------------------------- | +| `autoRecall` | `true` | Query HydraDB before every prompt and inject the result automatically | +| `autoIngest` | `true` | Sync workspace docs on session start and file edits automatically | +| `captureMode` | `session-upsert` | How conversations are saved (see [Modes](#modes)) | +| `recallMode` | `fast` | Recall depth: `fast` or `thinking`, sent as the query `mode` | +| `graphContext` | `true` | Include related facts from the context graph in recalled context | +| `followForcefulRelations` | `true` | Follow the relations declared at ingest, so recall also returns the linked items under Forceful relations. The server follows them in `thinking` mode. Env var: `HYDRADB_FOLLOW_FORCEFUL_RELATIONS` | ### Limits | Variable | Default | Description | | ------------------------- | -------------- | ------------------------------------------------------------------- | -| `maxContextChars` | `7000` | Max characters injected into the prompt from HydraDB | -| `maxMemoryResults` | `6` | Max memory chunks returned per recall | -| `maxKnowledgeResults` | `4` | Max knowledge chunks returned per recall | +| `maxMemoryResults` | `6` | Added to `maxKnowledgeResults` to set how many chunks each recall asks for (`max_results`, 10 by default) | +| `maxKnowledgeResults` | `4` | Added to `maxMemoryResults`, as above | | `maxFileSizeBytes` | `52428800` (50 MB) | Max file size for workspace sync | | `maxFilesPerSync` | `25` | Max files synced per pass | -| `maxMemoryCharsPerChunk` | `52428800` (50 MB) | Max chars per memory chunk | -| `maxMemoryChunksPerFile` | `1` | Max chunks per file | +| `maxMemoryCharsPerChunk` | `52428800` (50 MB) | Max characters per synced item; a longer file is split into numbered parts, each its own item | ### Timeouts @@ -198,11 +203,11 @@ Once configured, the plugin runs in the background: it syncs workspace docs on s | Variable | Default | Description | | ------------------------------------ | ------------------------ | --------------------------------------------------------- | -| `subTenantId` | - | Workspace-level namespace within your tenant | -| `userName` | - | Optional user name attached to memories | +| `subTenantId` | - | Workspace-level collection within your database | +| `userName` | - | Your name: the speaker name on your conversation turns, also stored with notes and session transcripts | | `apiBaseUrl` | `https://api.hydradb.com`| HydraDB API base URL | -| `memoryCustomInstructions` | - | Custom instructions for how conversation memories are extracted | -| `workspaceMemoryCustomInstructions` | - | Custom instructions for how workspace doc memories are extracted | +| `memoryCustomInstructions` | - | Instructions that steer enrichment of saved conversations, sessions and notes | +| `workspaceMemoryCustomInstructions` | - | Instructions that steer enrichment of synced workspace docs | | `debug` | `false` | Enable debug logging to `.hydradb-plugin-data/debug.log` | --- @@ -223,14 +228,14 @@ The plugin resolves config in this order (later sources override earlier ones): ## Suggested Modes -| Use case | `captureMode` | `searchMode` | `ingestionMode` | -| ----------------------------- | ---------------- | ------------ | --------------- | -| Default / everyday use | `session-upsert` | `memory` | `memory` | -| Cross-session continuity | `session-upsert` | `memory` | `memory` | -| Team shared knowledge base | `session-upsert` | `memory` | `memory` | -| Maximum recall coverage | `both` | `both` | `auto` | -| Recall only, no auto-save | `off` | `memory` | `memory` | -| Isolated turn snapshots | `turn` | `memory` | `memory` | +| Use case | `captureMode` | `recallMode` | +| ----------------------------- | ---------------- | ------------ | +| Default / everyday use | `session-upsert` | `fast` | +| Cross-session continuity | `session-upsert` | `fast` | +| Shared team context | `session-upsert` | `fast` | +| Maximum recall coverage | `both` | `thinking` | +| Recall only, no auto-save | `off` | `fast` | +| Isolated turn snapshots | `turn` | `fast` | --- @@ -239,26 +244,23 @@ The plugin resolves config in this order (later sources override earlier ones): ### Minimal (env vars only) ```bash -export HYDRA_DB_API_KEY="your-api-key" -export HYDRADB_TENANT_ID="your-tenant-id" -export HYDRADB_SUB_TENANT_ID="" +export HYDRADB_API_KEY="your-api-key" +export HYDRADB_DATABASE="your-database" +export HYDRADB_COLLECTION="" ``` ### Recommended workspace config (`.hydradb-plugin.json`) ```json { - "apiKey": "${HYDRA_DB_API_KEY}", - "tenantId": "${HYDRADB_TENANT_ID}", + "apiKey": "${HYDRADB_API_KEY}", + "tenantId": "${HYDRADB_DATABASE}", "subTenantId": "claude-my-workspace", "autoRecall": true, "autoIngest": true, "captureMode": "session-upsert", - "searchMode": "memory", - "ingestionMode": "memory", "recallMode": "fast", "graphContext": true, - "maxContextChars": 7000, "requestTimeoutMs": 15000, "writeTimeoutMs": 15000 } @@ -269,11 +271,10 @@ export HYDRADB_SUB_TENANT_ID="" ```json { "captureMode": "both", - "searchMode": "both", - "ingestionMode": "auto", - "maxContextChars": 10000, - "maxMemoryResults": 8, - "maxKnowledgeResults": 6 + "recallMode": "thinking", + "followForcefulRelations": true, + "maxMemoryResults": 12, + "maxKnowledgeResults": 8 } ``` diff --git a/plugins/cli.mdx b/plugins/cli.mdx index ea3e6fa9..1f0f8433 100644 --- a/plugins/cli.mdx +++ b/plugins/cli.mdx @@ -1,6 +1,6 @@ --- title: "HydraDB CLI" -description: "Agent-friendly command line interface for HydraDB. Query context, ingest knowledge and memories, and inspect stored data from the terminal." +description: "Agent-friendly command line interface for HydraDB. Query context, ingest text and conversations, and inspect stored data from the terminal." --- ## Quick Start @@ -26,8 +26,7 @@ description: "Agent-friendly command line interface for HydraDB. Query context, - Requires Python 3.10 or later, and `hydradb-cli` 0.2.0 for the commands and flags on - this page. On earlier versions, see [Deprecated aliases](#deprecated-aliases). + Requires Python 3.10 or later. @@ -165,78 +164,106 @@ has run. | Command | Description | |---|---| -| `hydradb query QUERY` | Retrieve knowledge or memories - the single retrieval entry point | -| `hydradb ingest` | Store a memory, knowledge text, or knowledge file(s) | -| `hydradb list` | List ingested sources and memories | -| `hydradb inspect SOURCE_ID` | Fetch a source's content by ID | -| `hydradb delete IDS...` | Delete memories or knowledge sources by ID | -| `hydradb relations SOURCE_ID` | Explore knowledge-graph relations for a source | -| `hydradb verify IDS...` | Check per-source ingestion status | +| `hydradb query QUERY` | Query the database: the single retrieval entry point | +| `hydradb ingest` | Ingest one text or conversation item | +| `hydradb list` | List stored items | +| `hydradb inspect ID` | Fetch an item's content by ID | +| `hydradb delete IDS...` | Delete items by ID | +| `hydradb relations ID` | Explore context-graph relations for an item | +| `hydradb verify IDS...` | Check per-item ingestion status | #### Ingesting -`ingest` stores a memory by default. Pass `--kind knowledge` for knowledge text, or -give it file paths - files are always knowledge sources. +`ingest` sends one item to `POST /context/ingest`: exactly one of `--text` (a note or a +document's text, or `-` for stdin) or `--conversation-file` (a conversation). The CLI +does not upload files: to ingest a document, extract its text and pass it with +`--text`, up to 1 MiB of text per item. ```bash -# Store a memory +# Store a note hydradb ingest --text "The user prefers dark mode and uses VS Code" --database my-db -# Store knowledge text -hydradb ingest --kind knowledge --text "Project uses PostgreSQL 15 with pgvector" --database my-db +# Store a document with an ID, a title, a date, a filterable attribute and a category +hydradb ingest --text "Refund policy: 30-day window." --context-id policy-1 \ + --title "Refund policy" --happened-at 2026-07-29 \ + --attributes '{"team": "support"}' --category business_knowledge --database my-db -# Upload one or more files -hydradb ingest ./docs/architecture.md ./docs/runbook.md --database my-db +# Store a conversation, declared related to the policy +hydradb ingest --conversation-file ./chat.json --context-id chat-w1 \ + --forceful-relation policy-1 --database my-db -# Read from stdin -cat notes.txt | hydradb ingest --database my-db +# Read the text from stdin +cat notes.txt | hydradb ingest --title "Meeting notes" --database my-db +``` + +`chat.json` is a JSON list of turns: + +```json +[ + { "role": "user", "content": "Keep answers short please", "name": "soham" }, + { "role": "assistant", "content": "Got it." } +] ``` | Option | Description | |---|---| -| `--kind` | `memory` (default) or `knowledge` | | `--text`, `-t` | Text to ingest. Use `-` to read from stdin | +| `--conversation-file` | Path to a JSON list of `{role, content, name?}` turns (roles `user`, `assistant`, `system`) | | `--title` | Optional title | -| `--source-id` | Client-assigned source identifier | -| `--user-name` | User name (memory only) | -| `--infer` / `--no-infer` | Extract insights and build the knowledge graph (default on) | -| `--markdown` | Treat text as markdown (memory only) | -| `--upsert` / `--no-upsert` | Update existing items with the same `source_id` (default on) | - - - `--text`, `--title`, `--source-id`, `--user-name`, `--markdown` and `--no-infer` do not - apply to file ingest. Passing them alongside file arguments is rejected rather than - silently ignored. - +| `--context-id` | Caller-assigned ID for the item (generated when omitted) | +| `--enrich` / `--no-enrich` | Extract facts and graph relations for the item (default on) | +| `--instructions` | Steer enrichment for this item | +| `--happened-at` | The date the item is about, `YYYY-MM-DD` | +| `--attributes` | Declared, filterable attributes as a JSON object (keys from the database's metadata schema) | +| `--custom-attributes` | Free-form attributes as a JSON object, not filterable | +| `--category` | `context_category` label: `auto` (default, no label), `user_preference`, `business_knowledge` or `decision_trace` | +| `--forceful-relation` | A context ID this item is declared related to; repeatable | +| `--acl` | A principal allowed to retrieve the item (for example `user_email:a@x.com` or `domain:acme.com`); repeatable | +| `--upsert` / `--no-upsert` | Replace an existing item with the same context ID (default on) | + +The command prints the queued item's context ID. Pass it to `hydradb verify` to watch +indexing; an item is searchable once it has finished. See +[Ingest](/essentials/v2/ingest) for every item field. #### Querying ```bash -# Search knowledge -hydradb query "How is authentication implemented?" --kind knowledge --database my-db +# Ask a question +hydradb query "How is authentication implemented?" --database my-db -# Search memories -hydradb query "What IDE does the user prefer?" --kind memory --database my-db +# Print the server-built prompt, ready to pipe into a model call +hydradb query "What IDE does the user prefer?" --llm --database my-db # Deterministic keyword search hydradb query "PostgreSQL migration" --operator and --database my-db + +# Only what matched the query, without items declared related at ingest +hydradb query "refund window" --no-follow-forceful-relations --database my-db ``` | Option | Description | |---|---| -| `--kind` | Corpus to query: `memory` or `knowledge`. Omit to search both | -| `--operator` | Keyword operator: `or`, `and`, or `phrase` | -| `--max-results`, `-n` | Maximum results, 1–50 (default `10`) | -| `--mode`, `-m` | Retrieval mode: `fast` or `thinking` | +| `--operator` | Keyword operator: `or`, `and`, or `phrase`. Switches the query to keyword retrieval | +| `--max-results`, `-n` | Maximum results, 1-50 (default `10`) | +| `--mode`, `-m` | Retrieval mode: `fast` or `thinking`. Forceful relations are followed in `thinking` mode | | `--alpha` | Hybrid search weight (`0.0` keyword → `1.0` semantic) | -| `--recency-bias` | Preference for newer content (`0.0`–`1.0`) | -| `--graph-context` / `--no-graph-context` | Include knowledge graph relations | +| `--recency-bias` | Preference for newer content (`0.0` to `1.0`) | +| `--graph-context` / `--no-graph-context` | Include context-graph paths (`graph`) in the answer | +| `--follow-forceful-relations` / `--no-follow-forceful-relations` | Also return items declared related at ingest (server default on) | +| `--llm` | Print the server-built `llm_prompt` verbatim on stdout, ready to inject into a model call. The request ID goes to stderr | +| `--acl` | A principal to answer as; repeatable. Results are limited to items whose access list admits one of them | | `--context` | Additional context to guide retrieval | | `--title` | Restrict the search to documents with this exact title, ignoring case. Repeatable | +The default output shows each result with its context ID, score, content and +enrichment, then the graph paths (paths grown from the query's entities, and the +relations of each returned chunk), then any forceful relations, and ends with the +query's request ID. `--output json` prints the response body as the server sent it. +See [Query](/essentials/v2/query) for every field. + #### Filtering by document title -Use `--title` when you know document names but not their source IDs. Repeat the flag +Use `--title` when you know document names but not their IDs. Repeat the flag for several titles; they are ORed. ```bash @@ -249,32 +276,34 @@ hydradb query "ownership" --title "Q3 Roadmap.md" --title "Launch Plan.md" --dat - **Surrounding whitespace is trimmed**, and repeated values are de-duplicated. Punctuation such as commas is part of the title, so quote it: `--title "Smith, John"`. -- **No match returns an empty result** rather than widening to the whole corpus. +- **No match returns an empty result** rather than widening to the whole collection. -Because matching ignores case, two documents whose names differ only by case - -`Report.md` and `report.md` - are treated as the same title and both are returned. +Because matching ignores case, two documents whose names differ only by case, +`Report.md` and `report.md`, are treated as the same title and both are returned. #### Inspecting and deleting ```bash # List what is stored -hydradb list --kind knowledge --database my-db +hydradb list --database my-db -# Read one source back -hydradb inspect 39e8872d-1c4a-4f0b-9c1e-7d2a5b8e4f10 --database my-db +# Read one item back +hydradb inspect policy-1 --database my-db # Check indexing progress -hydradb verify 39e8872d-1c4a-4f0b-9c1e-7d2a5b8e4f10 --database my-db +hydradb verify policy-1 --database my-db # Delete (prompts unless --yes) -hydradb delete 39e8872d-1c4a-4f0b-9c1e-7d2a5b8e4f10 --kind knowledge --database my-db --yes +hydradb delete policy-1 --database my-db --yes ``` -`list` accepts `--kind`, `--page` and `--page-size` (1–100). `inspect` accepts -`--mode content` (default), `url`, or `both`. `delete` defaults to `--kind knowledge`, -so pass `--kind memory` to remove a memory. +`list` lists every item in scope, text and conversations alike, and accepts `--page` +and `--page-size` (1 to 100). `inspect` accepts `--mode content` (default), `url`, or +`both`. `list`, `inspect` and `relations` also accept `--acl` to answer as specific +principals. Deleting an ID that does not exist exits non-zero rather than reporting +success. ### Database Management @@ -295,8 +324,8 @@ hydradb database delete my-db --yes ``` - `hydradb database delete` permanently removes the database and all associated memories, - knowledge, and graph data. This action cannot be undone. It prompts for confirmation + `hydradb database delete` permanently removes the database with all of its stored + context and graph data. This action cannot be undone. It prompts for confirmation unless you pass `--yes`. @@ -310,50 +339,34 @@ hydradb database delete my-db --yes | `hydradb config show` | Show current CLI configuration | | `hydradb config set KEY VALUE` | Set `api_key`, `database`, `collection`, or `base_url` | -### Deprecated aliases - -The command groups below still work and behave identically, but each prints a -deprecation warning to stderr naming its replacement. They will be removed in a future -major version. - -| Deprecated | Use instead | -|---|---| -| `memories add` | `ingest` | -| `memories list`, `fetch sources` | `list` | -| `memories delete`, `knowledge delete` | `delete` | -| `knowledge upload`, `knowledge upload-text` | `ingest` | -| `knowledge verify` | `verify` | -| `recall full`, `recall preferences`, `recall keyword` | `query --kind … --operator …` | -| `fetch content` | `inspect` | -| `fetch relations` | `relations` | -| `tenant …` | `database …` | -| `whoami` | `doctor` | -| `--tenant-id`, `--sub-tenant-id` | `--database`, `--collection` | - ## Scripting & Automation The CLI is designed for both interactive use and scripting. Use `--output json` to get machine-readable output that pipes cleanly into `jq`, Python, or other tools: ```bash -# List sources as JSON and pull out their IDs +# List items as JSON and pull out their IDs hydradb -o json list --database my-db | jq '.sources[].id' # Query and extract just the matched text -hydradb -o json query "user preferences" --kind memory --database my-db \ - | jq '.chunks[].chunk_content' +hydradb -o json query "user preferences" --database my-db \ + | jq '.chunks[].content' -# Batch upload — ingest accepts many files in one invocation -hydradb ingest ./docs/*.md --database my-db +# Hand the server-built prompt to a model +hydradb query "user preferences" --llm --database my-db | my-model-call -# Or drive it from find -find ./docs -name "*.md" -exec hydradb ingest --database my-db {} + +# Ingest each markdown file as one text item, keyed by its name so a re-run replaces it +for f in ./docs/*.md; do + name="$(basename "$f" .md)" + hydradb ingest --text - --title "$name" --context-id "doc-$name" --database my-db < "$f" +done ``` - `query` returns `chunks`, each with `chunk_content`, `source_title` and - `relevancy_score`. `list` returns `sources`. Both memories and knowledge appear in the - same `sources` array, distinguished by their `type`. + `query` returns the response body as the server sent it: `chunks` (each with + `chunk_id`, `context_id`, `score` and `content`, plus `enrichment`, + `enrichment_kind` and `temporal` when present), `graph`, `forceful_relations` and + `llm_prompt`. `list` returns the listed items under `sources`, each with its `id`. ## Source & Show Support diff --git a/plugins/mcp.mdx b/plugins/mcp.mdx index 7d46883b..bcceed8a 100644 --- a/plugins/mcp.mdx +++ b/plugins/mcp.mdx @@ -1,6 +1,6 @@ --- title: "HydraDB MCP" -description: "MCP (Model Context Protocol) server for [HydraDB](https://hydradb.com). Provides tools for storing, recalling, and managing memories with knowledge-graph enriched context." +description: "MCP (Model Context Protocol) server for [HydraDB](https://hydradb.com). Gives any MCP client tools to ingest context, query it with context-graph results, and manage what is stored." --- ## Quick Start @@ -258,9 +258,10 @@ approve it in the browser, done. No API key, no config file to edit. **Local MCP server only:** the canonical tool names and `HYDRADB_*` variables need **@hydradb/mcp 1.1.0 or later**. If your config pins - `@0.0.1` or `@1.0.0`, change it to `@^1.1.0` and restart. Nothing else - changes - every name you were using before still works, and the caret - keeps you on the 1.x line this page documents. + `@0.0.1` or `@1.0.0`, change it to `@^1.1.0` and restart. The caret + keeps you on the 1.x line this page documents. If your config still + calls the older `hydra_db_*` tool names, see + [Older tool names](#older-tool-names). @@ -439,34 +440,61 @@ same scope names the rest of the product uses. See the | Tool | What it does | | ---- | ------------ | -| `hydradb_query` | Search memories; returns matching chunks plus graph-enriched context - entity paths, chunk relations, extra context | -| `hydradb_ingest` | Store a note (`text`) or a conversation (`turns`); HydraDB extracts insights and builds the knowledge graph | -| `hydradb_list` | Browse stored memories or ingested knowledge sources | -| `hydradb_inspect` | Retrieve the original content of a source by ID | -| `hydradb_delete` | Remove a memory or knowledge source by ID | +| `hydradb_query` | Query the database; returns the server-built `llm_prompt` (ranked results, forceful relations, related facts from the context graph) plus the same answer as structured content | +| `hydradb_ingest` | Store a note or document (`text`) or a conversation (`turns`) as one context item; HydraDB enriches it and adds it to the context graph | +| `hydradb_list` | List what is stored in a collection, one page at a time | +| `hydradb_inspect` | Retrieve the original content of a stored item by ID | +| `hydradb_delete` | Remove stored items by ID | +| `hydradb_status` | Check whether ingested items have finished indexing | | `hydradb_list_collections` | List collections (sub-tenants) in a database | | `hydradb_delete_collection` | Permanently delete a collection and all of its data | | `hydradb_databases` | List the databases this connection can use, with the default marked. Available on OAuth connections | ### hydradb_query +Sends the question to `POST /query` and returns the answer described under +[What the query returns](#what-the-query-returns). + | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `query` | string | Yes | The search query to find relevant memories | -| `kind` | string | No | Which family to search: `memory`, `knowledge`, or `all` (default: `all`) | +| `query` | string | Yes | What you want to know, as a question or topic | | `max_results` | number | No | Chunks to return, 1-50 (default: `10`) | -| `mode` | string | No | `fast` for quick semantic search, `thinking` for deeper personalised recall with graph traversal (default: `thinking`) | -| `graph_context` | boolean | No | Include knowledge graph relations in the results (default: `true`) | -| `titles` | string[] | No | Restrict the search to documents whose **complete** title exactly matches any value, ignoring case | -| `metadata_filters` | object | No | Exact-match metadata filters as `{key: value}` | +| `mode` | string | No | `thinking` (default) runs graph traversal and follows forceful relations; `fast` is plain semantic search and quicker; `auto` lets HydraDB pick | +| `graph_context` | boolean | No | Include related facts from the context graph (`graph[]`) in the answer (default: `true`) | +| `follow_forceful_relations` | boolean | No | Also return items declared related at ingest (see `forceful_relations` on `hydradb_ingest`), listed under Forceful relations with `[R1]` labels (default: `true`). They are followed in `thinking` mode | +| `operator` | string | No | `or`, `and`, or `phrase`. Switches the query to keyword retrieval, which matches the literal words instead of running hybrid semantic search. Leave unset for normal searches | +| `source_ids` | string[] | No | Restrict the search to these item IDs (context IDs from earlier results or `hydradb_list`). No match returns an empty result | +| `titles` | string[] | No | Restrict the search to items whose **complete** title exactly matches any value, ignoring case | +| `recency_bias` | number | No | Favour recently updated items when ranking, 0-1 (default: `0`). Re-ranks only; it never excludes older items | +| `query_apps` | boolean | No | App-aware retrieval over connector content: exact IDs and actors, thread reconstruction, parent and child expansion (default: `false`) | +| `acl` | string[] | No | Principals to answer as: an email, a `domain:`, or a `group::`. Results are limited to items whose access list admits one of them. Omit to search everything the key can reach. See [Access control](/essentials/v2/access-control) | +| `collections` | string[] | No | Search several collections at once. Pass either this or `collection`, not both | | `database` | string | No | Database (tenant) scope override for this request | | `collection` | string | No | Collection (sub-tenant) scope override for this request | +#### What the query returns + +The tool result is the server-built `llm_prompt`, verbatim and whole: it is never +trimmed or truncated. It is markdown: `# Query results`, then `## Results`, +`## Forceful relations`, `## Related facts`, `## Temporal facts` and `## Sources`, +each only when there is something to show. Results are numbered `1`, `2`; forceful +relations (linked by the author at ingest, not matched by the query) `R1`, `R2`; +related facts (context-graph paths) `P1`, `P2`. The model cites them as `[1]`, +`[R1]` and `[P1]`. Each entry shows its `**Id:**`, the item's context ID, which +`hydradb_inspect` and `hydradb_delete` accept. The result ends with the query's +`request_id`. + +The same answer rides beside it as structured content, also whole: `chunks[]` +(`chunk_id`, `context_id`, `score`, `content`, `enrichment`, `enrichment_kind`, and +`temporal[]` when the query engaged temporal reasoning), `graph[]` (`origin`, +`triplets[]`, `path_summary`) and `forceful_relations[]` (`via`, plus a `chunk` of +the same shape). See [Query](/essentials/v2/query) for every field. + #### Filtering by document title -Use `titles` when you know document names but not their source IDs. The titles are -resolved to source IDs first, then the normal semantic or keyword query runs inside -those sources. +Use `titles` when you know document names but not their IDs. The titles are +resolved to item IDs first, then the normal semantic or keyword query runs inside +those items. ```json { @@ -479,40 +507,53 @@ those sources. - **Case-insensitive.** `q3 roadmap.md` matches `Q3 Roadmap.md`. - **Several titles are ORed.** Surrounding whitespace is trimmed; punctuation such as commas is part of the title. -- **No match returns an empty result** rather than widening to the whole corpus. +- **No match returns an empty result** rather than widening to the whole collection. -Because matching ignores case, two documents whose names differ only by case - -`Report.md` and `report.md` - are treated as the same title and both are returned. +Because matching ignores case, two documents whose names differ only by case, +`Report.md` and `report.md`, are treated as the same title and both are returned. ### hydradb_ingest -Provide exactly one of `text` or `turns`. Passing both is rejected. +Sends one item to `POST /context/ingest` under the `context` list: a text item for +`text`, or a conversation item (`role` and `content` turns) for `turns`. Provide +exactly one of `text` or `turns`. Passing both is rejected. | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `text` | string | No* | The information to store | +| `text` | string | No* | A note, fact, decision, or document body | | `turns` | array | No* | Conversation turns to ingest instead of `text`, each with a `user` and an `assistant` field | -| `kind` | string | No | What to write: `memory` (default) or `knowledge` | -| `title` | string | No | Title for the entry | -| `source_id` | string | No | Optional identifier for this entry | -| `infer` | boolean | No | Let HydraDB extract insights and build the knowledge graph (default: `true`) | -| `is_markdown` | boolean | No | Treat `text` as markdown (default: `false`) | -| `metadata` | object | No | Key/value metadata stored with memory, matchable later via `metadata_filters` | -| `observation_date` | string | No | Calendar date YYYY-MM-DD when the fact was true | -| `user_name` | string | No | Name of the user, used with `turns` for personalisation (default: `User`) | +| `title` | string | No | Title for the item, shown next to it in later results | +| `source_id` | string | No | The item's context ID (generated when omitted). Reusing one replaces what is stored under it | +| `overwrite` | boolean | No | Allow that replacement (default: `true`) | +| `infer` | boolean | No | Enrich the item: extract facts and add them to the context graph (default: `true`) | +| `instructions` | string | No | Steers what enrichment extracts from this item; replaces the server's default guidance for it | +| `is_markdown` | boolean | No | Chunk `text` on its markdown structure (default: `false`) | +| `user_name` | string | No | Name of the user, used as the speaker of the user turns in `turns` (default: `User`) | +| `happened_at` | string | No | Calendar date `YYYY-MM-DD` when the fact was true, as opposed to when it was stored | +| `attributes` | object | No | Declared, filterable key/value attributes (keys from the database's metadata schema). See [Attributes](/essentials/v2/attributes) | +| `custom_attributes` | object | No | Free-form key/value data stored with the item, not filterable | +| `context_category` | string | No | Your label for the item: `user_preference`, `business_knowledge`, `decision_trace`, or `auto` (default, no label). Nothing infers it. See [Context categories](/essentials/v2/context-categories) | +| `forceful_relations` | string[] | No | Context IDs this item is declared related to, such as the thread or document it belongs to. A later query that returns this item can pull them in under Forceful relations | +| `acl` | string[] | No | Principals that may read the item: an email, a `domain:`, or a `group::`. Omit it for an item anyone holding the key may read | | `database` | string | No | Database (tenant) scope override for this request | | `collection` | string | No | Collection (sub-tenant) scope override for this request | +Ingestion is asynchronous: an item is not searchable the instant it is saved. Use +`hydradb_status` with the returned ID to confirm. + ### hydradb_list +Lists what is stored in the collection: notes, documents, conversations and +connector content appear in one listing. + | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `kind` | string | Yes | Which family to list: `memory` or `knowledge` | | `ids` | array | No | Specific IDs to filter by | | `page` | number | No | Page number, 1-indexed (default: `1`) | | `page_size` | number | No | Items per page, 1-100 | +| `acl` | string[] | No | Principals to answer as; the listing shows only items their access list admits | | `database` | string | No | Database (tenant) scope override for this request | | `collection` | string | No | Collection (sub-tenant) scope override for this request | @@ -520,10 +561,11 @@ Provide exactly one of `text` or `turns`. Passing both is rejected. | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `id` | string | Yes | The source ID to fetch content for | +| `id` | string | Yes | The item ID to fetch content for | | `mode` | string | No | `content` for text, `url` for a presigned URL, `both` for both (default: `content`) | | `offset` | number | No | Character offset to start reading from | | `limit` | number | No | Maximum characters to return (max `20000`) | +| `acl` | string[] | No | Principals to answer as; an item their access list does not admit is not returned | | `database` | string | No | Database (tenant) scope override for this request | | `collection` | string | No | Collection (sub-tenant) scope override for this request | @@ -533,7 +575,6 @@ Provide exactly one of `text` or `turns`. Passing both is rejected. | --------- | ---- | -------- | ----------- | | `ids` | array | No* | IDs to delete | | `id` | string | No* | Single ID to delete (prefer `ids` when deleting multiple) | -| `kind` | string | No | Which family the ID belongs to: `memory` or `knowledge` (default: `memory`) | | `database` | string | No | Database (tenant) scope override for this request | | `collection` | string | No | Collection (sub-tenant) scope override for this request | @@ -545,7 +586,7 @@ Provide exactly one of `text` or `turns`. Passing both is rejected. ### hydradb_delete_collection -Permanently removes one collection and every memory, knowledge source, and graph node inside it. The parent database is left intact. +Permanently removes one collection and every item and graph node inside it. The parent database is left intact. | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | @@ -556,7 +597,7 @@ Permanently removes one collection and every memory, knowledge source, and graph | Parameter | Type | Required | Description | | --------- | ---- | -------- | ----------- | -| `ids` | array | Yes | The source IDs to check indexing status for | +| `ids` | array | Yes | The item IDs to check indexing status for | | `database` | string | No | Database (tenant) scope override for this request | | `collection` | string | No | Collection (sub-tenant) scope override for this request | @@ -585,42 +626,35 @@ HydraDB MCP also exposes property graph tools for querying and writing domain gr - A knowledge source cannot be deleted while it is still being indexed - the - server refuses with *"Source is still processing; retry deletion after - ingestion completes"*, and the tool passes that back rather than reporting a - deletion that did not happen. Retry once ingestion finishes. This applies to - freshly ingested sources only; a source is listable and inspectable before it - is deletable. + An item cannot be deleted while it is still being indexed. The server + refuses with *"Source is still processing; retry deletion after ingestion + completes"*, and the tool passes that back rather than reporting a deletion + that did not happen. Retry once `hydradb_status` shows the item has finished. + This applies to freshly ingested items only; an item is listable and + inspectable before it is deletable. -### Deprecated tool names - -Tool names live in your `mcp.json`, so none were removed. Every earlier name is -still registered and still works; each is marked deprecated in its description so -your agent prefers the canonical one. Use the canonical names in new integrations. +### Older tool names -| Deprecated | Canonical | -| ---------- | --------- | -| `hydra_db_search` | `hydradb_query` | -| `hydra_db_store` | `hydradb_ingest` | -| `hydra_db_ingest_conversation` | `hydradb_ingest` (pass `turns`) | -| `hydra_db_list_memories` | `hydradb_list` (`kind: memory`) | -| `hydra_db_list_sources` | `hydradb_list` (`kind: knowledge`) | -| `hydra_db_fetch_content` | `hydradb_inspect` | -| `hydra_db_delete_memory` | `hydradb_delete` | +The earlier `hydra_db_*` tool names are no longer registered by default. If an +existing `mcp.json` still calls them, move it to the `hydradb_*` names above. On a +local or self-hosted server, `HYDRADB_MCP_LEGACY_TOOLS=1` registers the old names +again in the meantime. --- ## How It Works -At recall time, `hydradb_query` queries HydraDB and returns graph-enriched context - -entity paths, chunk relations, and linked extra context - injected directly into the -model's prompt. +At query time, `hydradb_query` sends `POST /query` and hands the model the +server-built `llm_prompt` as-is: ranked results, forceful relations, related facts +from the context graph, temporal facts and sources, each labelled for citation. The +structured answer (`chunks`, `graph`, `forceful_relations`) rides beside it. -At capture time, `hydradb_ingest` sends content with `infer: true`, triggering -HydraDB's knowledge graph extraction pipeline automatically. The same tool takes a -document (`text`) or a conversation (`turns`), and groups related writes under -`source_id`. +At capture time, `hydradb_ingest` sends one item to `POST /context/ingest`: a text +item for a note or document, or a conversation item for `turns`. Enrichment is on +by default, so HydraDB extracts facts from the item and adds them to the context +graph. Reusing an item's ID replaces it, and `hydradb_status` reports when a new +item becomes searchable. --- diff --git a/plugins/openclaw.mdx b/plugins/openclaw.mdx index 867b8ff2..2305b706 100644 --- a/plugins/openclaw.mdx +++ b/plugins/openclaw.mdx @@ -24,10 +24,10 @@ description: "HydraDB plugin for OpenClaw. Automatically captures conversations, ```bash # Basic onboarding - openclaw hydra onboard + openclaw hydradb onboard # Advanced onboarding - openclaw hydra onboard --advanced + openclaw hydradb onboard --advanced ``` The wizard writes credentials to `plugins.entries.openclaw.config` inside your OpenClaw settings file. @@ -59,26 +59,30 @@ description: "HydraDB plugin for OpenClaw. Automatically captures conversations, If you prefer not to use the onboarding wizard, configure credentials directly. -Two required values: your **API key** and **Tenant ID**. +Two required values: your **API key** and your **database** (the `tenantId` key). **Environment variables (recommended for secrets):** ```bash - export HYDRA_OPENCLAW_API_KEY="your-api-key" - export HYDRA_OPENCLAW_TENANT_ID="your-tenant-id" + export HYDRADB_API_KEY="your-api-key" + export HYDRADB_DATABASE="your-database" ``` ```powershell - [System.Environment]::SetEnvironmentVariable("HYDRA_OPENCLAW_API_KEY", "your-api-key", "User") - [System.Environment]::SetEnvironmentVariable("HYDRA_OPENCLAW_TENANT_ID", "your-tenant-id", "User") + [System.Environment]::SetEnvironmentVariable("HYDRADB_API_KEY", "your-api-key", "User") + [System.Environment]::SetEnvironmentVariable("HYDRADB_DATABASE", "your-database", "User") ``` + + The older `HYDRA_OPENCLAW_API_KEY` and `HYDRA_OPENCLAW_TENANT_ID` variables still work, but each prints a one-time deprecation warning. The canonical name wins when both are set. + + **Or set them directly in your OpenClaw settings file:** @@ -97,8 +101,8 @@ Two required values: your **API key** and **Tenant ID**. "openclaw": { "enabled": true, "config": { - "apiKey": "${HYDRA_OPENCLAW_API_KEY}", - "tenantId": "${HYDRA_OPENCLAW_TENANT_ID}" + "apiKey": "${HYDRADB_API_KEY}", + "tenantId": "${HYDRADB_DATABASE}" } } } @@ -118,12 +122,12 @@ openclaw gateway restart | Key | Type | Default | Description | | -------------------- | --------- | ------------------------- | ------------------------------------------------------------------------------- | -| `subTenantId` | `string` | `"hydra-openclaw-plugin"` | Sub-tenant for data partitioning within your tenant | -| `autoRecall` | `boolean` | `true` | Inject relevant memories before every AI turn | -| `autoCapture` | `boolean` | `true` | Store conversation exchanges after every AI turn | -| `maxRecallResults` | `number` | `10` | Max memory chunks injected into context per turn | -| `recallMode` | `string` | `"fast"` | `"fast"` for low latency; `"thinking"` for deeper graph-traversal recall | -| `graphContext` | `boolean` | `true` | Include knowledge graph relations in recalled context | +| `subTenantId` | `string` | `"hydra-openclaw-plugin"` | Collection within your database (also read from `HYDRADB_COLLECTION`) | +| `autoRecall` | `boolean` | `true` | Query HydraDB before every AI turn and inject the result | +| `autoCapture` | `boolean` | `true` | Save the conversation after every AI turn | +| `maxRecallResults` | `number` | `10` | Max chunks the query returns per turn (1 to 50) | +| `recallMode` | `string` | `"fast"` | `"fast"` for low latency; `"thinking"` for deeper graph-traversal recall, which also follows forceful relations | +| `graphContext` | `boolean` | `true` | Include related facts from the context graph in recalled context | | `ignoreTerm` | `string` | `"hydra-ignore"` | Messages containing this string are excluded from recall and capture | | `debug` | `boolean` | `false` | Verbose debug logs | @@ -134,11 +138,11 @@ openclaw gateway restart ```mermaid flowchart LR A([User Turn]) --> B([Auto-Recall]) - B -->|POST /recall/recall_preferences| C([HydraDB]) - C -->|graph-enriched context| D([Injected into prompt]) + B -->|POST /query| C([HydraDB]) + C -->|llm_prompt| D([Injected into prompt]) D --> E([AI Response]) E --> F([Auto-Capture]) - F -->|POST /memories/add_memory| C + F -->|POST /context/ingest| C style A fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc style B fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc @@ -148,56 +152,72 @@ flowchart LR style F fill:#0f172a,stroke:#334155,stroke-width:2px,color:#f8fafc ``` -- **Auto-Recall** - Before every AI turn, queries HydraDB (`POST /recall/recall_preferences`) for relevant memories and injects graph-enriched context: entity paths, chunk relations, and linked extra context. -- **Auto-Capture** - After every AI turn, the last user/assistant exchange is sent to HydraDB (`POST /memories/add_memory`) with `infer: true` and `upsert: true`. The session ID is used as `source_id` so HydraDB groups exchanges per session and builds the knowledge graph automatically. +- **Auto-Recall**: Before every AI turn, sends the prompt to `POST /query` (`max_results` from `maxRecallResults`, `mode` from `recallMode`, `graph_context` from `graphContext`, and `follow_forceful_relations` on) and injects the server-built `llm_prompt` verbatim. See [Context Injection](#context-injection). +- **Auto-Capture**: After every AI turn, sends the conversation (the user and assistant turns OpenClaw hands the plugin) to `POST /context/ingest` as one `conversation` item of `role` and `content` turns, with enrichment on and `upsert: true`. The item's `context_id` is derived from the session ID, so each capture replaces the session's earlier item, and HydraDB adds what it extracts to the context graph. --- ## Slash Commands -| Command | Description | -| --------------------------- | -------------------------------------- | -| `/hydra-onboard` | Show current configuration status | -| `/hydra-remember ` | Save something to HydraDB memory | -| `/hydra-recall ` | Search memories with relevance scores | -| `/hydra-list` | List all stored user memories | -| `/hydra-delete ` | Delete a specific memory by its ID | -| `/hydra-get ` | Fetch the full content of a source | +The earlier `/hydra-*` names still work and print a one-time deprecation warning. + +| Command | Earlier name | Description | +| ------------------------------ | ----------------- | ---------------------------------------------- | +| `/hydradb-ingest ` | `/hydra-remember` | Save a note to HydraDB as a text item | +| `/hydradb-query ` | `/hydra-recall` | Query HydraDB and list the results with scores | +| `/hydradb-list` | `/hydra-list` | List everything stored in the collection | +| `/hydradb-delete ` | `/hydra-delete` | Delete one stored item by its ID | +| `/hydradb-inspect ` | `/hydra-get` | Fetch the full content of an item | +| `/hydra-onboard` | - | Show current configuration status | --- ## AI Tools -| Tool | Description | -| ---------------------- | -------------------------------------------------------------- | -| `hydra_store` | Save the recent conversation history to HydraDB as memory | -| `hydra_search` | Search HydraDB memories (returns graph-enriched context) | -| `hydra_list_memories` | List all stored user memories (IDs and summaries) | -| `hydra_get_content` | Fetch full content for a specific `source_id` | -| `hydra_delete_memory` | Delete a memory by `memory_id` - use only on explicit request | +The earlier `hydra_*` names still work and print a one-time deprecation warning. + +| Tool | Earlier name | Description | +| ------------------ | ---------------------- | -------------------------------------------------------------- | +| `hydradb_ingest` | `hydra_store` | Save the recent conversation (up to the last 10 turns) as a conversation item, or the given text when there is no conversation | +| `hydradb_query` | `hydra_search` | Query HydraDB; returns the server-built `llm_prompt` | +| `hydradb_list` | `hydra_list_memories` | List everything stored (IDs and summaries) | +| `hydradb_inspect` | `hydra_get_content` | Fetch the full content of an item by its ID (`source_id`) | +| `hydradb_delete` | `hydra_delete_memory` | Delete one stored item by its ID (`memory_id`); use only on explicit request | --- ## CLI Reference ```bash -openclaw hydra onboard # Interactive onboarding wizard -openclaw hydra onboard --advanced # Advanced onboarding wizard -openclaw hydra search # Search memories -openclaw hydra list # List all user memories -openclaw hydra delete # Delete a memory -openclaw hydra get # Fetch source content -openclaw hydra status # Show plugin configuration +openclaw hydradb onboard # Interactive onboarding wizard +openclaw hydradb onboard --advanced # Advanced onboarding wizard +openclaw hydradb query # Query HydraDB +openclaw hydradb ingest # Save a note +openclaw hydradb list # List everything stored +openclaw hydradb delete # Delete an item +openclaw hydradb inspect # Fetch an item's content +openclaw hydradb status # Show plugin configuration ``` +The earlier `openclaw hydra` root still works and prints a one-time deprecation warning. + --- ## Context Injection -Recalled context is injected inside `` tags containing: +Recalled context is injected inside `` tags. The body is the server-built `llm_prompt` from `POST /query`, verbatim and whole: it is never truncated, summarised or held to a character budget. It is markdown: + +- `# Query results`, with the query and a count of what was found +- `## Results`: one `### 1.` entry per result with its relevance and category, the content, and its enrichment when there is any +- `## Forceful relations`: `### R1.` entries, context the author linked to a result at ingest time rather than matched by the query +- `## Related facts`: facts from the context graph, labelled `[P1]` +- `## Temporal facts` and `## Sources` + +Each section appears only when there is something to show. The `hydradb_query` tool returns the same `llm_prompt`. `/hydradb-query` and `openclaw hydradb query` print the response's structured fields instead: each chunk's `context_id`, score, `content`, `enrichment` and temporal facts, then the `graph` path summaries, then the `forceful_relations`. See [Query](/essentials/v2/query) for every field. -- **Entity Paths** - Knowledge graph paths connecting entities relevant to the query -- **Context Chunks** - Retrieved memory chunks with source titles, graph relations, and linked extra context + + OpenClaw caps a single tool result at `agents.defaults.contextLimits.toolResultMaxChars` (16,000 characters by default), so a longer `hydradb_query` result is cut by OpenClaw itself; raise that setting if you need more. The injected context is not a tool result and is not subject to that cap. + --- @@ -208,7 +228,7 @@ Recalled context is injected inside `` tags containing: The plugin is enabled but credentials are missing. Run: ```bash - openclaw hydra onboard + openclaw hydradb onboard openclaw gateway restart ``` diff --git a/snippets/legacy-line.mdx b/snippets/legacy-line.mdx deleted file mode 100644 index 8f2b2a47..00000000 --- a/snippets/legacy-line.mdx +++ /dev/null @@ -1 +0,0 @@ -Using a database created as `type: "split"`, or older fields like `tenant_id`, `metadata_filters` or `memories`? See [Split databases and legacy fields](/essentials/v2/split-databases). From 801711d53d5519e7d1f77ab201ec5946c2abfaa3 Mon Sep 17 00:00:00 2001 From: SohamRatnaparkhi Date: Wed, 23 Sep 2026 14:19:15 +0530 Subject: [PATCH 2/7] docs: inspect returns content only; delete success and inspect mode match the server (PRO-1618) - SDKs page: look an item's title and attributes up with context.list and ids; context.inspect returns its stored content (domain/fetch/v2.go has no title or attribute fields). Same fix on Ingest Context's related links and two AGENTS.mdx lines. - Delete Context and AGENTS.mdx: the deprecated data.success mirrors deleted_count > 0 (service/source_delete.go), so it is false for a delete that removed nothing. - AGENTS.mdx: GET /context/inspect defaults to mode=both, as the Inspect Context page says (handler/context.go). Signed-off-by: SohamRatnaparkhi Co-Authored-By: Claude Opus 5.5 (1M context) --- AGENTS.mdx | 10 +++++----- api-reference/v2/endpoint/delete-source.mdx | 2 +- api-reference/v2/endpoint/ingest-context.mdx | 4 ++-- api-reference/v2/sdks.mdx | 20 +++++++++++++++++--- 4 files changed, 25 insertions(+), 11 deletions(-) diff --git a/AGENTS.mdx b/AGENTS.mdx index c0b7aab2..06b87608 100644 --- a/AGENTS.mdx +++ b/AGENTS.mdx @@ -1187,7 +1187,7 @@ When you render results yourself (a UI, a reranker, an eval), read `chunks[].con | Mistake | Fix | |---|---| | Building your own context string from the JSON | Inject `llm_prompt` verbatim | -| Looking for a title or url on a chunk | Read it from `llm_prompt`, or call `GET /context/inspect` / `POST /context/list` with the `context_id` | +| Looking for a title or url on a chunk | Read it from `llm_prompt`, or call `POST /context/list` with `ids: [""]` for the title and attributes | | Querying one collection when the answer spans two | Send `collections`, with the person's collection weighted higher | | Including too many chunks | Start with `max_results: 10`; reduce for tight context windows | | Re-sorting chunks client-side | Preserve HydraDB ranking | @@ -1309,7 +1309,7 @@ Parameters: - `ids`: only these `context_id`s (filters and paging still apply) - `page` (1-indexed, default `1`), `page_size` (`1` to `100`, default `50`) - `filters`: exact-match constraints, ANDed. `source_fields` matches built-in fields such as `title`, `url`, `timestamp` and, for connector content, `app_provider`, `app_kind`, `app_external_id`, `app_parent_id`. The list filter keeps its own wire names for the two attribute maps: `filters.metadata` matches declared `attributes` and `filters.additional_metadata` matches `custom_attributes`. -- `include_fields`: projection, for example `["title", "timestamp"]`. `content` and `url` are not projectable (a `400`); read them with inspect. +- `include_fields`: projection, for example `["title", "timestamp"]`. `content` and `url` are not projectable (a `400`); read an item's content with `GET /context/inspect`. - `acl`: list as an identity; only items it may see are returned. Response `data`: `{ success, message, sources: [...], total, pagination }`. `sources` is the wire name for the listed items: one row per item with its `id`, title, timestamp and stored attributes, without content. `pagination` carries `page`, `page_size`, `total`, `total_pages`, `has_next` and `has_previous`. @@ -1330,9 +1330,9 @@ item = client.context.inspect( | Mode | Returns | |---|---| -| `content` (default) | the stored content in `content` | +| `content` | the stored content in `content` | | `url` | a time-limited `presigned_url` | -| `both` | content plus presigned URL | +| `both` (default) | content plus presigned URL | `expiry_seconds` sets the URL lifetime (default `3600`). With `acl`, the item must be visible to that identity or the response is `404`. @@ -1371,7 +1371,7 @@ const res = await client.context.delete({ One call deletes each item and everything derived from it (chunks and graph contribution), whatever its shape. -Response `data`: `results[]` (per id: `id`, `deleted`, `error`), `deleted_count` and `message`. Read `deleted_count` and each `results[].deleted`; the nested `data.success` is deprecated and is `true` even when nothing was removed. +Response `data`: `results[]` (per id: `id`, `deleted`, `error`), `deleted_count` and `message`. Read `deleted_count` and each `results[].deleted`; the nested `data.success` is deprecated and only mirrors `deleted_count > 0`. - `deleted_count: 0` means the ids matched nothing in that scope. Check `collection`. - An item that is still indexing refuses the whole request: nothing is deleted. By default the response is still `200`, with `deleted_count: 0`. Send the header `X-HydraDB-Delete-Status: strict` to get honest codes instead: `404` when nothing matched, `409` while an item is still indexing (retry after `Retry-After`), `500` when a store failed (retryable). diff --git a/api-reference/v2/endpoint/delete-source.mdx b/api-reference/v2/endpoint/delete-source.mdx index bb4aeece..c1f38b7a 100644 --- a/api-reference/v2/endpoint/delete-source.mdx +++ b/api-reference/v2/endpoint/delete-source.mdx @@ -60,7 +60,7 @@ curl -X DELETE 'https://api.hydradb.com/context' \ | | One entry per requested ID: `id`, `deleted` (whether this item was removed), and `error` (why not, present only when `deleted` is `false`). | | | Number of items actually removed. `0` means nothing was deleted. | | | Human-readable result message. | -| | Deprecated. `true` only when something was removed; read `deleted_count` and `results[]` instead. | +| | Deprecated. Mirrors `deleted_count > 0`, so it is `false` for a delete that removed nothing even when the request itself returned `200`. Read `deleted_count` and `results[]` instead. | diff --git a/api-reference/v2/endpoint/ingest-context.mdx b/api-reference/v2/endpoint/ingest-context.mdx index 14bbd270..0a4b4b0f 100644 --- a/api-reference/v2/endpoint/ingest-context.mdx +++ b/api-reference/v2/endpoint/ingest-context.mdx @@ -312,8 +312,8 @@ Caps per graph: at most 5,000 entities, 10,000 relations and 500 relations per e - **Always check** [ingestion status](/api-reference/v2/endpoint/source-status) to ensure context is ready to be retrieved - [Query](/api-reference/v2/endpoint/query) once context is ready - - **Inspect:** [List Context](/api-reference/v2/endpoint/list-documents) lists the items you ingested, with their titles - - **Inspect:** [Inspect Context](/api-reference/v2/endpoint/fetch-content) returns the stored content behind a `context_id`; [List Context](/api-reference/v2/endpoint/list-documents) with `ids` returns its title and attributes + - **Browse:** [List Context](/api-reference/v2/endpoint/list-documents) lists the items you ingested with their titles and attributes; send `ids` to look up specific ones + - **Inspect:** [Inspect Context](/api-reference/v2/endpoint/fetch-content) returns the stored content behind a `context_id` - **Cleanup:** [Delete Context](/api-reference/v2/endpoint/delete-source) - **Collections:** omitting `collection` writes to the default collection; list them with [List Collections](/api-reference/v2/endpoint/list-sub-tenants) diff --git a/api-reference/v2/sdks.mdx b/api-reference/v2/sdks.mdx index b609fc25..bab38c12 100644 --- a/api-reference/v2/sdks.mdx +++ b/api-reference/v2/sdks.mdx @@ -389,7 +389,7 @@ const filtered = await client.query({ ``` -Chunks carry no title or URL. To show where a chunk came from, pass its `context_id` to `context.inspect` (below). For every request and response field, see [Query](/essentials/v2/query) and the [Query reference](/api-reference/v2/endpoint/query); for injecting `llm_prompt` and mapping citations back, see [How to Use API Results](/essentials/v2/api-results). +Chunks carry no title or URL; `llm_prompt` prints them for the model. To show an item's title and attributes yourself, list it by its `context_id` with `context.list` and `ids` (below); `context.inspect` returns its stored content. For every request and response field, see [Query](/essentials/v2/query) and the [Query reference](/api-reference/v2/endpoint/query); for injecting `llm_prompt` and mapping citations back, see [How to Use API Results](/essentials/v2/api-results). ### Browse, inspect, and delete @@ -403,7 +403,14 @@ listing = client.context.list( page_size=50, ) -# Look up the title, attributes and content behind a context_id. +# Look up the title and attributes behind a context_id. +row = client.context.list( + database="my_first_database", + collection="support", + ids=["refund-policy"], +) + +# Read the stored content behind a context_id. item = client.context.inspect( database="my_first_database", collection="support", @@ -433,7 +440,14 @@ const listing = await client.context.list({ pageSize: 50, }); -// Look up the title, attributes and content behind a context_id. +// Look up the title and attributes behind a context_id. +const row = await client.context.list({ + database: "my_first_database", + collection: "support", + ids: ["refund-policy"], +}); + +// Read the stored content behind a context_id. const item = await client.context.inspect({ database: "my_first_database", collection: "support", From 40cd0e251c33a33f35ac9db3bdddbb189c3b223b Mon Sep 17 00:00:00 2001 From: SohamRatnaparkhi Date: Wed, 23 Sep 2026 17:09:15 +0530 Subject: [PATCH 3/7] docs(query): the llm_prompt sections the server now prints (PRO-1618) The unified llm_prompt also carries answer data the four JSON keys have no place for (hydradb-application#1645): a Duration line under Temporal facts, Source facts, Profiles and Code search sections, an Interpreted line for alias and reference expansion, a Note line when a lookup degraded, and a Decision line under a path carrying a decision trace. It is "" only when the query found nothing at all. Signed-off-by: SohamRatnaparkhi Co-Authored-By: Claude Opus 5.5 (1M context) --- essentials/v2/query.mdx | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/essentials/v2/query.mdx b/essentials/v2/query.mdx index adf9220b..4a57d395 100644 --- a/essentials/v2/query.mdx +++ b/essentials/v2/query.mdx @@ -334,17 +334,20 @@ Chunks pulled in because the caller declared `forceful_relations` at ingest: the ### `llm_prompt` -A server-built markdown document, ready to inject into a model call. It renders the chunks, the forceful relations, the graph paths and the dated facts, with the source details the chunk objects leave out (title, collection, type, last-updated date, url). A section with nothing in it is left out. When the query returns no chunks, forceful relations or graph paths, `llm_prompt` is `""`. +A server-built markdown document, ready to inject into a model call. It renders the chunks, the forceful relations, the graph paths and the dated facts, with the source details the chunk objects leave out (title, collection, type, last-updated date, url). It also carries answer data the four JSON keys have no place for: a computed duration, app-native source facts, entity profiles and code-search answers. A section with nothing in it is left out, and `llm_prompt` is `""` only when the query found nothing at all. The sections, in order: | Section | Contents | | --- | --- | -| `# Query results` | `**Query:**` (the query), a `**Found:**` line counting what follows, and the line telling the model to cite a result by its number. | +| `# Query results` | `**Query:**` (the query); an `**Interpreted:**` line when the query was widened by an alias (a workspace nickname for a name) or a resolved reference; a `**Found:**` line counting what follows; a `**Note:**` line when a temporal, source or profile lookup was degraded or truncated, so a thin answer is not read as an absence; and, when there is a result, the line telling the model to cite it by its number. | | `## Results` | One block per entry of `chunks[]`, in ranked order, separated by `---`: a `### 1. title` heading; a line with `**Relevance:**` (the `score`), `**Collection:**`, `**Type:**` and `**Category:**` (the `enrichment_kind`); a line with `**Id:**` (the `context_id`) and `**Last updated:**`; the chunk's `content`; then `**Enrichment:**` with the `enrichment`. | | `## Forceful relations` | A guide line, then one `### R1. title` block per entry of `forceful_relations[]`, laid out like a result, with `**Linked from:**` (the `via.from` context, when it is not `""`) in place of `**Relevance:**`. | | `## Related facts` | One line per path in `graph[]`, such as `- [P1] **A** -pred→ **B** (relevance 0.81) [1]`: the path's label, its chain of hops, the path's relevance after reranking in parentheses (printed only here: `graph[]` carries no score), and the results its hops were extracted from. A path with no reranked score, such as a graph summary a `thinking` query builds, has no parenthetical at all: `- [P3] **A** -pred→ **B** [1]`. The line never says which lane found the path. The `path_summary` is indented on the line under it, unless it only narrates the hops the chain already shows. | -| `## Temporal facts` | One line per dated fact the query engaged (the facts behind `chunks[].temporal`): subject, relation and object, then the resolved window, precision and status, with the evidence phrase set apart after a `;`, citing its result. | +| `## Temporal facts` | For a "how long between" question, a `**Duration:**` line first: the computed days, whether they are approximate, and the two dated facts it was measured between. Then one line per dated fact the query engaged (the facts behind `chunks[].temporal`): subject, relation and object, then the resolved window, fact type, precision and status, with the evidence phrase set apart after a `;`, citing its result (or naming its source id when that fact's chunk is not a result). | +| `## Source facts` | App-native facts about the sources behind the results (who acted, in what role, where, in which thread, from which connector, when synced), citing their result. Prompt only: no JSON key carries them. | +| `## Profiles` | The entity profiles the query selected, one `### name` block each: headline, summary and the profile's statements. Prompt only. | +| `## Code search` | The repository code-search answer, one `### repository` block each, with its status. Prompt only. | | `## Sources` | Each context once, in order of first appearance: title, type, id, url and last-updated date. Only web (`http` or `https`) links are printed; a storage location such as `s3://...` never is. | `**Type:**` is what the item is: the connector's word for it (a Slack `message`, a Jira `ticket`) when a connector set one, otherwise its source type, such as `file`. A field with no value is left out of its line. @@ -353,7 +356,7 @@ The sections, in order: | --- | --- | | `[1]`, `[2]`, ... | Result `### 1.`, `### 2.`, ...: that entry of `chunks[]`. Its `**Id:**` is the `context_id` to pass to `GET /context/inspect`. | | `[R1]`, `[R2]`, ... | Forceful relation `### R1.`, `### R2.`, ...: that entry of `forceful_relations[]`. | -| `[P1]`, `[P2]`, ... | A related fact: path 1, 2, ... of `graph[]`, the same numbering the dashboard and CLI show next to each hop. An agent can cite a fact by its label. | +| `[P1]`, `[P2]`, ... | A related fact: path 1, 2, ... of `graph[]`, the same numbering the dashboard and CLI show next to each hop. An agent can cite a fact by its label. A path that carries a decision trace has an indented `**Decision:**` line under it with the decision, when it was made, who made it and the evidence. | A related fact or a temporal fact ends with the labels of the results it was extracted from (`[1]`, or `[R1]` for a forceful relation); a fact extracted from a chunk that is not in the response carries none. The numbers in `## Sources` count contexts, not results, and are not citation labels. From ca2ac195b80e55cd8be5b171aa210b614086577c Mon Sep 17 00:00:00 2001 From: SohamRatnaparkhi Date: Wed, 23 Sep 2026 17:13:54 +0530 Subject: [PATCH 4/7] docs: every llm_prompt description lists the new sections (PRO-1618) API results, the Query reference (both mentions), AGENTS.mdx and the MCP and OpenClaw plugin pages still listed the previous six sections. They now list the Duration line, Source facts, Profiles and Code search (prompt only), the Interpreted and Note lines and the Decision line, matching the Query guide. Signed-off-by: SohamRatnaparkhi Co-Authored-By: Claude Opus 5.5 (1M context) --- AGENTS.mdx | 7 +++++-- api-reference/v2/endpoint/query.mdx | 4 ++-- essentials/v2/api-results.mdx | 9 ++++++--- plugins/mcp.mdx | 4 ++-- plugins/openclaw.mdx | 4 +++- 5 files changed, 18 insertions(+), 10 deletions(-) diff --git a/AGENTS.mdx b/AGENTS.mdx index 06b87608..f2f5c7b5 100644 --- a/AGENTS.mdx +++ b/AGENTS.mdx @@ -1063,11 +1063,14 @@ Sections, in order (a section with nothing in it is left out; when the query ret | Section | Contents | |---|---| -| `# Query results` | The query, a `**Found:**` line counting what follows, and the line telling the model to cite a result by its number. | +| `# Query results` | The query, an `**Interpreted:**` line when an alias or resolved reference widened it, a `**Found:**` line counting what follows, a `**Note:**` line when a lookup degraded, and (when there is a result) the line telling the model to cite it by its number. | | `## Results` | One `### 1. title` block per chunk, in ranked order: relevance, collection, type, category, id and last-updated date, the chunk's `content`, then `**Enrichment:**`. | | `## Forceful relations` | One `### R1. title` block per forceful-relation chunk, with `**Linked from:**` naming the item that pulled it in. | | `## Related facts` | One line per graph path, such as `- [P1] **Refunds** -managed_by→ **Finance** (relevance 0.81) [1]`, with the `path_summary` indented under it unless it only restates the chain. A path without a reranked score has no parenthetical. | -| `## Temporal facts` | One line per dated fact the query engaged, with its resolved window, citing its result. | +| `## Temporal facts` | A `**Duration:**` line first for a "how long between" question, then one line per dated fact the query engaged, with its resolved window, citing its result. | +| `## Source facts` | App-native facts about the sources behind the results (who, role, where, thread, connector, synced). Prompt only. | +| `## Profiles` | The entity profiles the query selected. Prompt only. | +| `## Code search` | The repository code-search answer. Prompt only. | | `## Sources` | Each item once: title, type, id, url and last-updated date. | Citation labels: diff --git a/api-reference/v2/endpoint/query.mdx b/api-reference/v2/endpoint/query.mdx index f2dec76b..f0ebf08d 100644 --- a/api-reference/v2/endpoint/query.mdx +++ b/api-reference/v2/endpoint/query.mdx @@ -127,7 +127,7 @@ A list gives every collection equal normalized weight. An object treats values a ### Using the response -`data.llm_prompt` is the whole context block as one markdown document: `# Query results`, then `## Results` (cited `[1]`), `## Forceful relations` (`[R1]`), `## Related facts` (`[P1]`, one per path in `graph[]`), `## Temporal facts` and `## Sources`. Inject it directly: +`data.llm_prompt` is the whole context block as one markdown document: `# Query results`, then `## Results` (cited `[1]`), `## Forceful relations` (`[R1]`), `## Related facts` (`[P1]`, one per path in `graph[]`), `## Temporal facts`, the prompt-only `## Source facts`, `## Profiles` and `## Code search` when the query produced them, and `## Sources`. Inject it directly: ```python messages = [{"role": "system", "content": result.data.llm_prompt}, @@ -552,7 +552,7 @@ result = client.query( | `chunks[]` | Ranked matches: `chunk_id`, `context_id`, `score`, `content` (verbatim), `enrichment` (the extracted statement as a plain string, omitted when there is none), `enrichment_kind` (the item's declared `context_category`: `user_preference`, `business_knowledge` or `decision_trace`; omitted when none was declared, present even without `enrichment`), `temporal[]` (only when the query engaged temporal reasoning; `{ content, start_date, end_date }`, where `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` and either date may be `null`). | | `graph[]` | Paths through the context graph, query paths first then chunk expansions: `origin`, `triplets[]` of `source` / `relation` / `target`, plus `path_summary`. `origin` is `"query_path"` (grown from the entities in the query) or `"chunk_relation"` (the neighbourhood of a returned chunk, only returned when one of its hops came from a returned chunk or a `forceful_relations` chunk). The array is deduplicated across both lanes and is not capped. `path_summary` is never empty: when the server wrote no summary, it narrates the hops. Entities are `{ entity_id, name }`; relations are `{ predicate, context, temporal_details?, timestamp?, relationship_id, chunk_id }`, where `temporal_details` is omitted when empty and `timestamp` (Unix epoch seconds, a float) is omitted when the edge has none. `[]` when `graph_context` is `false`. | | `forceful_relations[]` | Chunks pulled in through `forceful_relations` declared at ingest, followed only in `thinking` mode: `via.from` (the context whose declaration pulled it in, may be `""`), `via.to` (the chunk's own `context_id`), `chunk` (same shape as `chunks[]`). `[]` when none, when `follow_forceful_relations` is `false`, or when the query ran in `fast` mode. | -| `llm_prompt` | A server-built markdown string ready to inject into a model call: `# Query results`, then `## Results`, `## Forceful relations`, `## Related facts`, `## Temporal facts` and `## Sources`, each left out when empty. Results are cited `[1]` and forceful relations `[R1]`; related facts are labelled `[P1]`, `[P2]`, ... in `graph[]` order, as in `- [P1] **Refunds** -managed_by→ **Finance** (relevance 0.81) [1]`: the parenthetical is the path's relevance after reranking and is left out when the path has none, and the line ends with the results the path was extracted from. Sources print only web (`http` or `https`) links. `""` when nothing matched. The layout is on [Query](/essentials/v2/query#llm_prompt). | +| `llm_prompt` | A server-built markdown string ready to inject into a model call: `# Query results`, then `## Results`, `## Forceful relations`, `## Related facts`, `## Temporal facts` (with a `**Duration:**` line for a "how long between" question), `## Source facts`, `## Profiles`, `## Code search` and `## Sources`, each left out when empty. Source facts, profiles, code-search answers and the duration are prompt only: no JSON key carries them. Results are cited `[1]` and forceful relations `[R1]`; related facts are labelled `[P1]`, `[P2]`, ... in `graph[]` order, as in `- [P1] **Refunds** -managed_by→ **Finance** (relevance 0.81) [1]`: the parenthetical is the path's relevance after reranking and is left out when the path has none, and the line ends with the results the path was extracted from. Sources print only web (`http` or `https`) links. `""` only when the query found nothing at all. The layout is on [Query](/essentials/v2/query#llm_prompt). | To show a chunk's graph paths under that chunk, group hops by `triplets[].relation.chunk_id` and match it against `chunks[].chunk_id` (and `forceful_relations[].chunk.chunk_id`). See [Attaching graph paths to chunks](/essentials/v2/query#attaching-graph-paths-to-chunks). diff --git a/essentials/v2/api-results.mdx b/essentials/v2/api-results.mdx index 845f6d9a..56690f7f 100644 --- a/essentials/v2/api-results.mdx +++ b/essentials/v2/api-results.mdx @@ -144,14 +144,17 @@ FAQ: refunds to a card take 5 to 7 business days to appear. | Section | Built from | Labels | | --- | --- | --- | -| `# Query results` | The query, a `**Found:**` line counting what follows, and the instruction to cite a result by its number | None | +| `# Query results` | The query; an `**Interpreted:**` line when an alias or a resolved reference widened it; a `**Found:**` line counting what follows; a `**Note:**` line when a temporal, source or profile lookup degraded or was truncated; and, when there is a result, the instruction to cite it by its number | None | | `## Results` | `chunks[]`, in ranked order: a `### 1. title` heading, a line with relevance (`score`), collection, type and category (`enrichment_kind`), a line with the id (`context_id`) and last-updated date, the `content`, then `**Enrichment:**` (`enrichment`). Results are separated by `---`. | `[1]`, `[2]`, ... | | `## Forceful relations` | `forceful_relations[]`, the items the hits declared with `forceful_relations` at ingest: a guide line, then `### R1. title` blocks laid out like results, with `**Linked from:**` (`via.from`) in place of relevance | `[R1]`, `[R2]`, ... | | `## Related facts` | `graph[]`, one line per path: its chain of hops (`**A** -pred→ **B**`), its relevance after reranking in parentheses when it has one (`(relevance 0.81)`; a path with no reranked score has no parenthetical), and the results its hops were extracted from, with the `path_summary` indented under it unless it only narrates the chain | `[P1]`, `[P2]`, ... in `graph[]` order; each line also cites its results | -| `## Temporal facts` | The dated facts behind `chunks[].temporal`, with window, precision and status, then the evidence phrase after a `;` | None; each fact cites its result | +| `## Temporal facts` | For a "how long between" question, a `**Duration:**` line first (the computed days, whether approximate, and the two dated facts). Then the dated facts behind `chunks[].temporal`, with window, fact type, precision and status, then the evidence phrase after a `;` | None; each fact cites its result, or names its source id when that chunk is not a result | +| `## Source facts` | App-native facts about the sources behind the results: who acted and in what role, where, which thread and connector, when synced. Prompt only: no JSON key carries them | None; each fact cites its result | +| `## Profiles` | The entity profiles the query selected, one `### name` block each. Prompt only | None | +| `## Code search` | The repository code-search answer, one `### repository` block each, with its status. Prompt only | None | | `## Sources` | Each context once, in order of first appearance: title, type, id, url (web links only, never a storage location such as `s3://...`) and last-updated date | None; the numbers count contexts, not results | -A section with nothing in it is left out, and a query that returns nothing gets an empty `llm_prompt`. Ask the model to cite the labels and you get answers you can trace: a `[1]` in the reply is result 1, whose `**Id:**` is `refund-policy`, which you can look up with [`POST /context/list`](/api-reference/v2/endpoint/list-documents) or open with [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content); a `[P1]` is the first path in `graph[]`. How a multi-hop chain reads is on [Query](/essentials/v2/query#llm_prompt). +A section with nothing in it is left out, and only a query that finds nothing at all gets an empty `llm_prompt`. A path in `## Related facts` that carries a decision trace has an indented `**Decision:**` line under it. Ask the model to cite the labels and you get answers you can trace: a `[1]` in the reply is result 1, whose `**Id:**` is `refund-policy`, which you can look up with [`POST /context/list`](/api-reference/v2/endpoint/list-documents) or open with [`GET /context/inspect`](/api-reference/v2/endpoint/fetch-content); a `[P1]` is the first path in `graph[]`. How a multi-hop chain reads is on [Query](/essentials/v2/query#llm_prompt). --- diff --git a/plugins/mcp.mdx b/plugins/mcp.mdx index bcceed8a..09b42e62 100644 --- a/plugins/mcp.mdx +++ b/plugins/mcp.mdx @@ -476,8 +476,8 @@ Sends the question to `POST /query` and returns the answer described under The tool result is the server-built `llm_prompt`, verbatim and whole: it is never trimmed or truncated. It is markdown: `# Query results`, then `## Results`, -`## Forceful relations`, `## Related facts`, `## Temporal facts` and `## Sources`, -each only when there is something to show. Results are numbered `1`, `2`; forceful +`## Forceful relations`, `## Related facts`, `## Temporal facts`, `## Source facts`, +`## Profiles`, `## Code search` and `## Sources`, each only when there is something to show. Results are numbered `1`, `2`; forceful relations (linked by the author at ingest, not matched by the query) `R1`, `R2`; related facts (context-graph paths) `P1`, `P2`. The model cites them as `[1]`, `[R1]` and `[P1]`. Each entry shows its `**Id:**`, the item's context ID, which diff --git a/plugins/openclaw.mdx b/plugins/openclaw.mdx index 2305b706..5b068ec4 100644 --- a/plugins/openclaw.mdx +++ b/plugins/openclaw.mdx @@ -211,7 +211,9 @@ Recalled context is injected inside `` tags. The body is the serv - `## Results`: one `### 1.` entry per result with its relevance and category, the content, and its enrichment when there is any - `## Forceful relations`: `### R1.` entries, context the author linked to a result at ingest time rather than matched by the query - `## Related facts`: facts from the context graph, labelled `[P1]` -- `## Temporal facts` and `## Sources` +- `## Temporal facts` (a `**Duration:**` line first for a "how long between" question) +- `## Source facts`, `## Profiles` and `## Code search`, when the query produced them +- `## Sources` Each section appears only when there is something to show. The `hydradb_query` tool returns the same `llm_prompt`. `/hydradb-query` and `openclaw hydradb query` print the response's structured fields instead: each chunk's `context_id`, score, `content`, `enrichment` and temporal facts, then the `graph` path summaries, then the `forceful_relations`. See [Query](/essentials/v2/query) for every field. From d41def2711ef288f18a55be700043a81d0cd9575 Mon Sep 17 00:00:00 2001 From: SohamRatnaparkhi Date: Wed, 23 Sep 2026 17:20:55 +0530 Subject: [PATCH 5/7] docs(api-reference): leave the v2 spec as it is on main (PRO-1618) The spec ships separately, through the auto-generated spec PR. Every page this stack binds to it (27 visible pages, webhooks, connectors, feedback, context status and inspect, context metadata, collections and database deletes, the metadata schema) names an operation that main's spec already has, so the stack as a whole makes no change to api-reference/v2/openapi.json. Signed-off-by: SohamRatnaparkhi Co-Authored-By: Claude Opus 5.5 (1M context) --- api-reference/v2/openapi.json | 6462 +++++++++++---------------------- 1 file changed, 2028 insertions(+), 4434 deletions(-) diff --git a/api-reference/v2/openapi.json b/api-reference/v2/openapi.json index 4705e607..e319a5b9 100644 --- a/api-reference/v2/openapi.json +++ b/api-reference/v2/openapi.json @@ -4,7 +4,7 @@ "connectors.Resource": { "properties": { "acl": { - "description": "Principals allowed to retrieve objects synced from this resource. Omit for unrestricted. When the provider supports permission capture, the provider's own permissions take precedence.", + "description": "ACL is the customer-declared access-control list stamped onto every\nobject synced from this resource (PRO-1684; see internal/domain/acl).\nStored in caller-supplied form and normalized at transform time. nil\nmeans no ACL, documents stay unrestricted. Provider-derived ACLs\n(Phase 2) take precedence over this when the provider supports them.", "items": { "type": "string" }, @@ -12,20 +12,12 @@ "uniqueItems": false }, "acl_fingerprint": { - "description": "Identifies the access-control list last applied to this resource's indexed documents. Empty until one has been applied.", - "type": "string" - }, - "acl_warning": { - "description": "Why this resource's permissions could not be captured, in the provider's words. While set, the resource is readable by every caller. Clears on the next successful capture.", - "type": "string" - }, - "acl_warning_at": { - "description": "RFC3339 time `acl_warning` last changed. An unchanged warning is not rewritten each sync, so read it as \"open since\".", + "description": "ACLFingerprint is the stable identity of the ACL last APPLIED to this\nresource's already-indexed documents (PRO-1684). The sync compares the\nfreshly-resolved provider ACL against it: equal means nothing to do,\ndifferent means fan the new ACL out to existing documents. Empty means\nnothing has been applied yet (first capture-enabled sync).", "type": "string" }, "additional_metadata": { "additionalProperties": {}, - "description": "Custom attributes merged into every object synced from this resource. Fields set by the provider win on conflict.", + "description": "AdditionalMetadata is merged into the additional_metadata (document\nmetadata) layer of every object synced from this resource. User-supplied\nkeys are shallow-merged as the base; provider-generated fields are\napplied on top and always win on conflict.", "example": { "author": "ada", "doc_version": 3 @@ -33,25 +25,21 @@ "type": "object" }, "backfill_chunk_interval_seconds": { - "description": "Seconds between chunks of the historical backfill.", + "description": "BackfillChunkIntervalSeconds is the pacing interval persisted at configure\ntime so the scheduler can thread it into each chunk's workflow input.", "example": 86400, "type": "integer" }, - "backfill_floor": { - "description": "RFC3339 time the historical backfill works back to, fixed when the resource was configured. Empty on resources configured before this was recorded.", - "type": "string" - }, "backfill_next_chunk_at": { - "description": "RFC3339 time the next backfill chunk is due.", + "description": "BackfillNextChunkAt is the RFC3339 time the next chunk becomes due. The\nbackfill workflow processes one chunk then sets this to now+interval and\nexits; the connector scheduler starts the next chunk once it passes.", "type": "string" }, "backfill_oldest": { - "description": "RFC3339 time of the oldest period the historical backfill still has to fetch. Empty when the backfill is complete or not needed.", + "description": "BackfillOldest is an RFC3339 timestamp marking the oldest boundary remaining\nfor async historical backfill. Empty means backfill is complete or not needed.", "example": "2026-06-01T00:00:00Z", "type": "string" }, "backfill_status": { - "description": "Present while a historical backfill is in progress; removed when it completes.", + "description": "BackfillStatus gates the sparse ResourcesByBackfillNextChunkAt GSI: it is\nset to BackfillStatusActive while a historical backfill is in progress and\nremoved when it completes, so only actively-backfilling resources appear in\nthe scheduler's due query. Pacing between chunks is driven by that scheduler\n(see BackfillNextChunkAt), not by an in-workflow sleep.", "type": "string" }, "collection_override": { @@ -64,11 +52,11 @@ "type": "string" }, "custom_instructions": { - "description": "Ingestion instructions for objects synced from this resource. When set, they replace the connector's `custom_instructions`; empty inherits the connector's. Max 4000 characters; changes apply from the next sync cycle.", + "description": "CustomInstructions is optional free-text ingestion guidance scoped to\nthis resource. When set it replaces the connector-level\ncustom_instructions for documents synced from this resource; empty means\nthe resource inherits the connector's value. Max 4000 characters;\nchanges apply from the next sync cycle.", "type": "string" }, "database_override": { - "description": "Database that objects from this resource go to, overriding the connector's. Empty inherits the connector's database. `tenant_id_override` is its deprecated alias.", + "description": "DatabaseOverride/CollectionOverride are the canonical v2 names for the\ndeprecated tenant_id_override/sub_tenant_id_override wire fields. Empty\nmeans the resource inherits the connector's database/collection, exactly\nas the deprecated fields do. Not persisted (dynamodbav:\"-\"): mirrored from\nthe tenant_id_override/sub_tenant_id_override values at construction time.", "type": "string" }, "display_name": { @@ -86,25 +74,13 @@ }, "metadata": { "additionalProperties": {}, - "description": "Attributes merged into every object synced from this resource. Fields HydraDB sets (`connector_id`, `provider`) win on conflict.", + "description": "Metadata is merged into the tenant metadata layer of every object synced\nfrom this resource. User-supplied keys are shallow-merged as the base;\nsystem defaults (connector_id, provider) are applied on top so they\nalways win on conflict — user keys extend the map but cannot override\nsystem-set fields.", "example": { "department": "finance", "priority": 7 }, "type": "object" }, - "page_acl_warning": { - "description": "Set when individual pages inside this resource had restrictions that could not be read, so they were opened to every caller. Tracked separately from the resource's own access-control warning.", - "type": "string" - }, - "page_acl_warning_at": { - "description": "RFC3339 time `page_acl_warning` last changed.", - "type": "string" - }, - "page_acl_warning_run": { - "description": "Sync run that last saw a page opened this way. The warning clears after a full sync cycle in which no page was.", - "type": "string" - }, "provider_cursor": { "description": "Bookmark of the last synced position. Non-empty value confirms the first sync has run.", "example": "1699999999.000100", @@ -140,16 +116,16 @@ "x-deprecated": "true" }, "sync_blocked": { - "description": "Set when the provider keeps refusing this resource (for example a dropped table, or a channel the credential was never invited to). The resource stays listed but is no longer synced.", + "description": "SyncBlocked marks a resource the provider will go on refusing — a table\nthat was dropped, a channel this credential was never invited to.\n\nDeliberately not a Status value. Status gates ListConnectorResources,\nwhich is what GET /connectors/{id}/status reads, so expressing this as a\nstatus would hide the resource from the one endpoint that explains why it\nstopped. The resource stays active and visible; this only takes it out of\nwhat gets synced.", "example": true, "type": "boolean" }, "sync_blocked_at": { - "description": "RFC3339 time the resource stopped syncing.", + "description": "SyncBlockedAt is when the resource was stopped (RFC3339).", "type": "string" }, "sync_blocked_reason": { - "description": "The provider's explanation for why the resource stopped syncing.", + "description": "SyncBlockedReason is the provider's own explanation, carried forward from\nthe health that triggered the block so it survives the next sync\noverwriting that health.", "type": "string" }, "tenant_id_override": { @@ -199,12 +175,12 @@ "feedback.GroundTruth": { "properties": { "answer": { - "description": "The answer you expected: what a correct response would have produced from the retrieved context.", + "description": "Answer is the response the caller expected — the text a correct system\nwould have produced from the retrieved context.", "maxLength": 8000, "type": "string" }, "source_ids": { - "description": "IDs of the ingested sources that contain the answer, as returned in query results.", + "description": "SourceIDs are the ingested source IDs that actually contain the answer,\nas returned in query results and accepted by /context endpoints.", "example": [ "HydraDoc1234", "HydraDoc4567" @@ -245,55 +221,51 @@ ] }, "feedback.SubmitRequest": { - "allOf": [ + "anyOf": [ { - "anyOf": [ - { - "patternProperties": { - "^feedback$": { - "minLength": 1, - "pattern": "\\S" - } - }, - "required": [ - "feedback" - ] - }, - { - "patternProperties": { - "^ground_truth$": { - "anyOf": [ - { - "properties": { - "answer": { - "minLength": 1, - "pattern": "\\S" - } - }, - "required": [ - "answer" - ] - }, - { - "properties": { - "source_ids": { - "contains": { - "minLength": 1, - "pattern": "\\S" - } - } - }, - "required": [ - "source_ids" - ] + "properties": { + "feedback": { + "minLength": 1, + "pattern": "\\S" + } + }, + "required": [ + "feedback" + ] + }, + { + "properties": { + "ground_truth": { + "anyOf": [ + { + "properties": { + "answer": { + "minLength": 1, + "pattern": "\\S" + } + }, + "required": [ + "answer" + ] + }, + { + "properties": { + "source_ids": { + "contains": { + "minLength": 1, + "pattern": "\\S" + } } + }, + "required": [ + "source_ids" ] } - }, - "required": [ - "ground_truth" ] } + }, + "required": [ + "ground_truth" ] } ], @@ -486,10 +458,148 @@ }, "success": { "deprecated": true, - "description": "Deprecated. Always carries the same value as the envelope's top-level `success`. Check the HTTP status code (2xx is success) instead.", + "description": "Deprecated for API clients: to decide whether the request succeeded,\ncheck the HTTP status code — 2xx is success — or equivalently the\nenvelope's top-level `success`. This nested copy always carries the same\nvalue as that flag and never carries independent information. Still\nemitted unchanged for existing clients (PRO-1208).", + "example": true, + "type": "boolean", + "x-deprecated": "true" + } + }, + "type": "object" + }, + "github_com_hydradb_hydradb-application_internal_service.MetadataEditResult": { + "properties": { + "acl_drift_recorded": { + "description": "ACLDriftRecorded reports that a failed ACL mirror was durably recorded\nfor reconciliation. Always true when vector_acl_synced is true. False\nbeside acl_updated=true and vector_acl_synced=false is the one state\nthe operator must act on (the error log names the document).", + "example": true, + "type": "boolean" + }, + "acl_updated": { + "description": "ACLUpdated reports that this edit replaced the source's ACL (PRO-1684).", + "example": true, + "type": "boolean" + }, + "additional_metadata_keys": { + "description": "Additional metadata keys included in the update request.", + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": false + }, + "chunk_rows_matched": { + "description": "Number of MongoDB chunk rows matched by the source update.", + "example": 1, + "type": "integer" + }, + "chunk_rows_modified": { + "description": "Number of MongoDB chunk rows modified by the source update.", + "example": 1, + "type": "integer" + }, + "collection": { + "description": "Collection that contained the source. Canonical name; mirrors the deprecated `sub_tenant_id` alias.", + "example": "team_docs", + "type": "string" + }, + "database": { + "description": "Owning database. Canonical name; mirrors the deprecated `tenant_id` alias.", + "example": "acme_corp", + "type": "string" + }, + "database_metadata_keys": { + "description": "Database metadata keys included in the update request. Canonical name; `tenant_metadata_keys` is a deprecated alias.", + "example": [ + "department", + "priority" + ], + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": false + }, + "id": { + "description": "Unique identifier for this resource.", + "example": "HydraDoc1234", + "type": "string" + }, + "milvus_rows_synced": { + "deprecated": true, + "description": "deprecated: use vector_rows_synced", + "example": 1, + "type": "integer", + "x-deprecated": "true" + }, + "milvus_sync_required": { + "deprecated": true, + "description": "Deprecated: use vector_sync_required / vector_synced / vector_rows_synced.\nRetained as additive aliases for existing clients; carry the same values.", + "example": true, + "type": "boolean", + "x-deprecated": "true" + }, + "milvus_synced": { + "deprecated": true, + "description": "deprecated: use vector_synced", "example": true, "type": "boolean", "x-deprecated": "true" + }, + "partial_commit": { + "description": "PartialCommit reports that the edit committed in at least one database\nof a shared deployment but a later write in another failed; the\nidempotent retry converges the database that fell behind.", + "type": "string" + }, + "sub_tenant_id": { + "deprecated": true, + "description": "deprecated: use collection", + "example": "sub_tenant_4567", + "type": "string", + "x-deprecated": "true" + }, + "tenant_id": { + "deprecated": true, + "description": "deprecated: use database", + "example": "tenant_1234", + "type": "string", + "x-deprecated": "true" + }, + "tenant_metadata_keys": { + "deprecated": true, + "description": "deprecated: use database_metadata_keys", + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": false, + "x-deprecated": "true" + }, + "updated": { + "description": "Whether the source metadata was updated.", + "example": true, + "type": "boolean" + }, + "vector_acl_synced": { + "description": "VectorACLSynced reports that the ACL edit also reached the vector rows'\npushdown columns (PRO-1740). False with ACLUpdated true means the\ndocument's own-ACL projection is stale until its next re-index: still\nenforced correctly from Mongo, but invisible to the pushdown lane for\nany principal the edit ADDED. Always false for collections created\nbefore PRO-1740, which carry no pushdown columns.", + "example": true, + "type": "boolean" + }, + "vector_rows_synced": { + "description": "Number of chunk rows synced to the vector store when sync was required.", + "example": 1, + "type": "integer" + }, + "vector_sync_error": { + "description": "VectorSyncError explains a vector_synced=false when a sync was\nrequired: the authority (Mongo) committed, the vector metadata did\nnot follow; the idempotent retry converges it.", + "type": "string" + }, + "vector_sync_required": { + "description": "Vendor-neutral vector-sync signal (PRO-1185): the canonical field must not\nname the vector store. The milvus_* fields below are deprecated aliases kept\nfor backward compatibility (additive change, not a rename) and carry the same\nvalues; they are slated for removal in a future major version.", + "example": true, + "type": "boolean" + }, + "vector_synced": { + "description": "Whether the vector store metadata sync completed. Present when sync was required.", + "example": true, + "type": "boolean" } }, "type": "object" @@ -502,11 +612,11 @@ "type": "string" }, "hydration": { - "description": "Set on source nodes in `auxiliary_relations` only. `resolved`: the source is ingested. `stub`: a relation points at the source but it is not ingested yet. `placeholder`: known only by its provider id, awaiting ingestion.", + "description": "Hydration is set on Source nodes in AuxiliaryRelations only, and omitted\neverywhere else. RELATES_TO MERGEs its target by source_id, so a target\nthat has not been ingested yet still exists as a node — callers must be\nable to tell a real document from a forward reference to one.\n\n\tresolved — ingested; source_id and app_provider both present\n\tstub — MERGE-created target; source_id present, no app_provider\n\tplaceholder — source_id IS NULL, keyed by app_external_id, awaiting\n\t builder.py's reconciliation pass", "type": "string" }, "identifier": { - "description": "Entity identifier, or null.", + "description": "NO omitempty — serialize as null", "example": "Acme Corp", "type": "string" }, @@ -521,7 +631,7 @@ "type": "string" }, "provider": { - "description": "Connector app the entity's evidence came from (for example `slack` or `intercom`). Empty when the evidence was ingested directly rather than through a connector.", + "description": "Provider is the source app the entity's evidence chunk came from (e.g.\n\"slack\", \"google\", \"intercom\"), read from the owning Source node's\napp_provider. Empty string when the evidence has no app source (plain\ndocument / web ingest). Consumed by the dashboard to render a connector\nlogo inside the graph node.", "example": "slack", "type": "string" }, @@ -536,7 +646,7 @@ "graph.GraphRelationsResponse": { "properties": { "auxiliary_relations": { - "description": "Structural graph around the entity relations: where entities appear, comments and attachments on sources, who authored what, and links between sources. Same item shape as `relations`, so concatenate the two for one graph. Kept separate so it does not count against `limit` or move the cursor.", + "description": "AuxiliaryRelations carries the structural graph around the entity\nrelations: Entity-\u003eSource presence, Source-\u003eComment/Attachment,\nActor-\u003eSource/Comment, and Source-\u003eSource links. Same item shape as\nRelations, so a caller wanting one graph concatenates the two.\n\nDeliberately a SEPARATE array rather than merged into Relations:\ncapPreservingTies counts triplets against the caller's limit, and\ncomputeNextCursor keys on relation timestamps. Auxiliary edges carry\ncreated_at — a different clock — so merging them would both shrink the\nentity relations returned for a given limit and corrupt the cursor.", "example": [ { "chunk_id": "HydraEmbeddings123_0", @@ -581,7 +691,7 @@ "uniqueItems": false }, "auxiliary_truncated": { - "description": "True when the auxiliary graph was cut off by a size limit. Independent of `is_truncated`, which covers `relations` pagination only.", + "description": "AuxiliaryTruncated reports that an aggregate row ceiling clipped the\nauxiliary graph. Distinct from the per-triplet Truncated flag, which only\ncovers a single node exceeding its fan-out cap: a wide page can blow the\naggregate ceiling with every individual node still under its own cap, and\nwithout this the caller would receive a subset presented as complete.\n\nIndependent of IsTruncated, which describes Relations pagination only.", "example": true, "type": "boolean" }, @@ -596,7 +706,7 @@ "type": "string" }, "next_cursor": { - "description": "Cursor for the next page, or null when there are no more.", + "description": "NO omitempty", "example": 0.5, "type": "number" }, @@ -647,7 +757,7 @@ }, "success": { "deprecated": true, - "description": "Deprecated. Always carries the same value as the envelope's top-level `success`. Check the HTTP status code (2xx is success) instead.", + "description": "Deprecated for API clients: to decide whether the request succeeded,\ncheck the HTTP status code — 2xx is success — or equivalently the\nenvelope's top-level `success`. This nested copy always carries the same\nvalue and never carries independent information. Still emitted unchanged\nfor existing clients (PRO-1208).", "example": true, "type": "boolean", "x-deprecated": "true" @@ -663,7 +773,7 @@ "type": "string" }, "chunk_id": { - "description": "Chunk the relation was extracted from, or null.", + "description": "NO omitempty", "example": "HydraEmbeddings123_0", "type": "string" }, @@ -688,22 +798,22 @@ "type": "string" }, "source_entity_id": { - "description": "ID of the relation's source entity, or null.", + "description": "NO omitempty", "example": "entity_1a2b", "type": "string" }, "synthesized": { - "description": "True for a `present_in` relation derived from where an entity appears, rather than a stored relation. Its relation id is generated.", + "description": "Synthesized marks a triplet with no stored edge behind it. Only\n`present_in` sets it: that edge is derived by collapsing\nEntity-PRESENT_IN-\u003eChunk-HAS_CHUNK-\u003eSource, so its RelationshipID is a\ndeterministic synthetic id rather than a graph relationship id. Omitted\n(false) on every stored edge.", "example": true, "type": "boolean" }, "target_entity_id": { - "description": "ID of the relation's target entity, or null.", + "description": "NO omitempty", "example": "entity_3c4d", "type": "string" }, "temporal_details": { - "description": "Time information attached to the relation, or null.", + "description": "NO omitempty", "example": "since 2024", "type": "string" }, @@ -718,7 +828,7 @@ "graph.SourceSubgraphResponse": { "properties": { "auxiliary_relations": { - "description": "Structural graph around the member sources: where entities appear, comments and attachments on sources, and who authored what. Same item shape as `relations`.", + "description": "AuxiliaryRelations carries the structural graph around the member\nsources: Entity-\u003eSource presence, Source-\u003eComment/Attachment and\nActor-\u003eSource/Comment links. Same item shape as Relations, matching\nGraphRelationsResponse so the dashboard renderer works unchanged.", "example": [ { "chunk_id": "HydraEmbeddings123_0", @@ -763,17 +873,17 @@ "uniqueItems": false }, "auxiliary_truncated": { - "description": "True when the auxiliary graph was cut off by a size limit.", + "description": "AuxiliaryTruncated reports that a fetch ceiling clipped the auxiliary\ngraph, same contract as GraphRelationsResponse.AuxiliaryTruncated.", "example": true, "type": "boolean" }, "is_truncated": { - "description": "True when the traversal stopped before reaching every connected source: the source budget or a size limit was hit, or `depth` left sources unexpanded.", + "description": "IsTruncated reports that the traversal stopped before exhausting the\nconnected component: the source budget or an edge/expansion fetch cap\nwas hit, or the depth limit left an unexpanded frontier.", "example": false, "type": "boolean" }, "max_depth_reached": { - "description": "Deepest level at which a source was added.", + "description": "MaxDepthReached is the deepest BFS level that admitted a member.", "example": 1, "type": "integer" }, @@ -783,7 +893,7 @@ "type": "string" }, "relations": { - "description": "Relations between member sources: declared relations whose two ends are both members, plus `same_thread` and `child_of` links for sources connected through thread or parent fields (these are marked `synthesized`).", + "description": "Relations holds the Source-\u003eSource triplets: every RELATES_TO edge whose\nendpoints are both members, plus synthesized same_thread / child_of\nprovenance edges for members reached through a node property rather than\na stored edge (those carry Synthesized on their evidence).", "example": [ { "chunk_id": "HydraEmbeddings123_0", @@ -831,7 +941,7 @@ "type": "string" }, "sources": { - "description": "Every source in the subgraph, in the order it was reached, starting source first.", + "description": "Sources is every member of the subgraph in BFS discovery order, seed\nfirst.", "example": [ { "app_external_id": "C0123456789", @@ -874,7 +984,7 @@ "type": "string" }, "depth": { - "description": "Steps from the starting source (0 for the starting source).", + "description": "Depth is the BFS distance from the seed (0 for the seed itself).", "example": 1, "type": "integer" }, @@ -882,11 +992,11 @@ "type": "string" }, "discovered_via": { - "description": "The already-included source this one was first reached from, and how: a declared relation type (such as `reply_to` or `child_of`), `same_thread`, `parent` or `child`. Empty on the starting source.", + "description": "DiscoveredVia and DiscoveredRelation record the traversal provenance:\nwhich already-admitted source this member was first reached from, and\nthrough which mechanism — a RELATES_TO relation_type (reply_to,\nchild_of, ...), same_thread, parent or child. Empty on the seed.", "type": "string" }, "hydration": { - "description": "Same classification as an entity's `hydration`: `resolved`, `stub` (a relation points at it but it is not ingested yet) or `placeholder`.", + "description": "Hydration carries the same resolved/stub/placeholder classification\nEntity.Hydration documents: a RELATES_TO target may be a MERGE-created\nforward reference to a document that has not been ingested yet.", "type": "string" }, "source_id": { @@ -957,7 +1067,41 @@ } }, "truncated": { - "description": "True on an auxiliary relation when a node had more of these than could be returned, so \"no more comments\" and \"not all shown\" can be told apart. Omitted on entity relations, which page with `limit` and the cursor.", + "description": "Truncated is set on auxiliary triplets whose fan-out hit a per-node cap,\nso a caller can tell \"this source has no more comments\" from \"we stopped\ncounting\". Omitted (false) on entity relations, which are bounded by the\nrequest's own limit/cursor instead.", + "example": true, + "type": "boolean" + } + }, + "type": "object" + }, + "handler.Envelope-tenants_SubTenantDeleteResponse": { + "properties": { + "data": { + "$ref": "#/components/schemas/tenants.SubTenantDeleteResponse", + "example": { + "collection": "engineering", + "database": "acme_corp", + "message": "Collection deregistered. Background cleanup is in progress.", + "status": "deletion_scheduled" + } + }, + "error": { + "$ref": "#/components/schemas/handler.apiError", + "description": "Error message, empty string on success.", + "example": { + "code": "DATABASE_NOT_FOUND", + "message": "Database not found" + } + }, + "meta": { + "$ref": "#/components/schemas/handler.responseMeta", + "example": { + "latency_ms": 12.3, + "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d" + } + }, + "success": { + "description": "Whether the request succeeded.", "example": true, "type": "boolean" } @@ -1050,7 +1194,7 @@ "handler.Envelope-github_com_hydradb_hydradb-application_internal_service_MetadataEditResult": { "properties": { "data": { - "$ref": "#/components/schemas/service.MetadataEditResult", + "$ref": "#/components/schemas/github_com_hydradb_hydradb-application_internal_service.MetadataEditResult", "example": { "acl_drift_recorded": true, "acl_updated": true, @@ -1635,86 +1779,12 @@ }, "type": "object" }, - "handler.Envelope-search_EntityProfileView": { - "properties": { - "data": { - "$ref": "#/components/schemas/search.EntityProfileView", - "example": { - "entity_id": "entity_1a2b", - "entries": [ - { - "confidence": 0.92 - } - ], - "name": "general", - "pending_importance": 1, - "version": 1 - } - }, - "error": { - "$ref": "#/components/schemas/handler.apiError", - "description": "Error message, empty string on success.", - "example": { - "code": "DATABASE_NOT_FOUND", - "message": "Database not found" - } - }, - "meta": { - "$ref": "#/components/schemas/handler.responseMeta", - "example": { - "collection": "team_docs", - "database": "acme_corp", - "latency_ms": 12.3, - "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", - "source_type": "file", - "sub_tenant_id": "sub_tenant_4567", - "tenant_id": "tenant_1234" - } - }, - "success": { - "description": "Whether the request succeeded.", - "example": true, - "type": "boolean" - } - }, - "type": "object" - }, "handler.Envelope-search_V2RetrievalResult": { "properties": { "data": { "$ref": "#/components/schemas/search.V2RetrievalResult", "example": { "additional_context": "The user is a senior engineer onboarding to the platform.", - "app_search_fusion": { - "stats": { - "app_chunks": 1, - "app_has_exact_ids": true, - "app_lane_empty_text": true, - "consensus": 1, - "exact_candidates": 1, - "exact_promoted": 1, - "limit": 1, - "normal_chunks": 1, - "normal_displaced": 1, - "tail_added": 1, - "tail_candidates": 1 - }, - "stats_by_pass": [ - { - "app_chunks": 1, - "app_has_exact_ids": true, - "app_lane_empty_text": true, - "consensus": 1, - "exact_candidates": 1, - "exact_promoted": 1, - "limit": 1, - "normal_chunks": 1, - "normal_displaced": 1, - "tail_added": 1, - "tail_candidates": 1 - } - ] - }, "chunks": [ { "additional_metadata": { @@ -1742,103 +1812,13 @@ "sub_tenant_id": "sub_tenant_4567" } ], - "forceful_relations": { - "declared": [ - { - "chunk": { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "collection": "team_docs", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" - ], - "id": "HydraDoc1234", - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 - }, - "relevancy_score": 0.87, - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" - } - } - ], - "inferred": [ - { - "chunk": { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "collection": "team_docs", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" - ], - "id": "HydraDoc1234", - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 - }, - "relevancy_score": 0.87, - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" - } - } - ] - }, - "graph": { - "paths": [ - { - "chunk_ids": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", - "relevancy_score": 0.87, - "triplets": [ - { - "relation": { - "confidence": 0.92, - "predicate": "works_at" - }, - "source": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "target": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - } - } - ] - } - ] - }, - "graph_context": { - "chunk_id_to_group_ids": { - "HydraEmbeddings123_0": [ - "grp_1234" - ] - }, - "chunk_relations": [ + "graph_context": { + "chunk_id_to_group_ids": { + "HydraEmbeddings123_0": [ + "grp_1234" + ] + }, + "chunk_relations": [ { "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", "group_id": "grp_1234", @@ -1897,24 +1877,6 @@ } ] }, - "profile_context": { - "entity_id": "entity_1a2b", - "entries": [ - { - "confidence": 0.92 - } - ], - "name": "general", - "version": 1 - }, - "profile_filter": { - "applied": true, - "degraded": true, - "entity_id": "entity_1a2b", - "found": true, - "selected_entries": 1, - "version": 1 - }, "source_facts": [ { "app_kind": "slack", @@ -2090,48 +2052,6 @@ }, "message": "Success", "org_id": "org_1a2b3c", - "tenant_id": "tenant_1234", - "type": "split" - } - }, - "error": { - "$ref": "#/components/schemas/handler.apiError", - "description": "Error message, empty string on success.", - "example": { - "code": "DATABASE_NOT_FOUND", - "message": "Database not found" - } - }, - "meta": { - "$ref": "#/components/schemas/handler.responseMeta", - "example": { - "collection": "team_docs", - "database": "acme_corp", - "latency_ms": 12.3, - "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", - "source_type": "file", - "sub_tenant_id": "sub_tenant_4567", - "tenant_id": "tenant_1234" - } - }, - "success": { - "description": "Whether the request succeeded.", - "example": true, - "type": "boolean" - } - }, - "type": "object" - }, - "handler.Envelope-tenants_SubTenantDeleteResponse": { - "properties": { - "data": { - "$ref": "#/components/schemas/tenants.SubTenantDeleteResponse", - "example": { - "collection": "team_docs", - "database": "acme_corp", - "message": "Success", - "status": "completed", - "sub_tenant_id": "sub_tenant_4567", "tenant_id": "tenant_1234" } }, @@ -2294,12 +2214,6 @@ "acme_corp", "research_kb" ], - "details": [ - { - "database": "acme_corp", - "type": "split" - } - ], "failed_databases": [ { "database": "acme_corp", @@ -2396,46 +2310,6 @@ }, "type": "object" }, - "handler.Envelope-tenants_TenantRenameResponse": { - "properties": { - "data": { - "$ref": "#/components/schemas/tenants.TenantRenameResponse", - "example": { - "connector_reassignment": "complete", - "database": "acme_corp", - "message": "Success", - "status": "completed", - "tenant_id": "tenant_1234" - } - }, - "error": { - "$ref": "#/components/schemas/handler.apiError", - "description": "Error message, empty string on success.", - "example": { - "code": "DATABASE_NOT_FOUND", - "message": "Database not found" - } - }, - "meta": { - "$ref": "#/components/schemas/handler.responseMeta", - "example": { - "collection": "team_docs", - "database": "acme_corp", - "latency_ms": 12.3, - "request_id": "9d13aef4-02f4-4e73-8c62-4c2601d04f9d", - "source_type": "file", - "sub_tenant_id": "sub_tenant_4567", - "tenant_id": "tenant_1234" - } - }, - "success": { - "description": "Whether the request succeeded.", - "example": true, - "type": "boolean" - } - }, - "type": "object" - }, "handler.Envelope-tenants_TenantStatsResponse": { "properties": { "data": { @@ -2840,7 +2714,7 @@ }, "success": { "deprecated": true, - "description": "Deprecated. Always false on errors. Read the HTTP status code, `error.code` and `error.message` instead, and `meta.request_id` when reporting a problem.", + "description": "Deprecated for API clients: always false on this path, so it carries no\ninformation. To detect a failure read the HTTP status code; for what\nwent wrong read the envelope's error.code and error.message, and\nmeta.request_id when reporting it. The whole `detail` object is\ndeprecated legacy — tagging the field individually so SDK users see it\non the property, not just the container (PRO-1208).", "example": true, "type": "boolean", "x-deprecated": "true" @@ -2991,7 +2865,7 @@ "type": "string" }, "rank": { - "description": "Display order, lowest first. Null means unranked.", + "description": "Rank is the dashboard display order (lower first); null means unranked.", "example": 1, "type": "integer" }, @@ -2999,7 +2873,7 @@ "type": "string" }, "rbac_support": { - "description": "Whether document-level permission capture is available for this provider.", + "description": "RBACSupport reports whether document-level ACL capture (PRO-1684) is\nenabled for this provider (the acl_supported control-plane flag), and\nRBACDescription says in one sentence WHAT is captured, so the dashboard\ncan explain the capability instead of showing a bare boolean.", "example": true, "type": "boolean" }, @@ -3008,7 +2882,7 @@ "type": "boolean" }, "webhook_support": { - "description": "Whether this provider is fed by an inbound webhook rather than polled.", + "description": "WebhookSupport marks a provider fed by an inbound webhook. The dashboard\nneeds it to pick the credential form: it otherwise reads moveit_support=false\nas \"classic\", and renders the single-token form instead of the provider's\ndeclared credential schema.", "example": true, "type": "boolean" } @@ -3053,7 +2927,7 @@ "uniqueItems": false }, "table_configs": { - "description": "Per-table replication settings, for connectors that take them (BigQuery). Applied before the first sync. Rejected for connectors that do not support them.", + "description": "TableConfigs carries per-table replication settings for MOVEIT\nconnectors whose tap reads a `table_configs` credential input (bigquery).\nConfigure merges them into the stored credential bundle before the first\nsync, so the mode chosen at selection time governs every sync from the\nstart. Optional; rejected for non-MOVEIT engines.", "items": { "$ref": "#/components/schemas/handler.tableConfigEntry" }, @@ -3082,7 +2956,7 @@ "type": "string" }, "first_sync_at": { - "description": "RFC3339 time of the first sync: now if configuring started one, otherwise when the scheduled one runs.", + "description": "FirstSyncAt/Message state the timing expectation: whether the first\nsync is already running (configure triggers one) or when the scheduled\none runs, so clients stop inventing their own copy (PRO-1565).", "type": "string" }, "message": { @@ -3094,7 +2968,7 @@ "$ref": "#/components/schemas/handler.configureResponseMeta" }, "warnings": { - "description": "Resources that were saved but returned nothing when checked. They are valid (a table may be empty and fill later), so they are not rejected, but they are listed here so an empty result is not mistaken for a working one.", + "description": "Warnings names resources that were saved but produced nothing when\nprobed. They are valid — the user may know a table is empty and expect it\nto fill — so they are not rejected, but they are the case that is\notherwise indistinguishable from success at every layer, so they are\nnever saved silently either.", "items": { "type": "string" }, @@ -3119,11 +2993,11 @@ "handler.connectorAPIView": { "properties": { "acl_changes_cursor": { - "description": "Position in the provider's permission-change feed. Empty until the first sync sets a starting point.", + "description": "ACLChangesCursor is the provider permission-change feed's persisted\ncursor (PRO-1684; e.g. the Drive changes.list page token). Empty means\nuninitialized: the next cycle fetches a baseline and starts from now.\nAdvanced ONLY after every reported change was applied, so a failed\napply replays the same changes next cycle (at-least-once; the writes\nare idempotent full replacements).", "type": "string" }, "active_resource_count": { - "description": "Number of resources that are not disabled.", + "description": "ActiveResourceCount mirrors the number of non-disabled resource rows so\nlist responses can distinguish \"no resources configured yet\"\n(pending_setup) without a per-connector resources query.", "example": 1, "type": "integer" }, @@ -3143,21 +3017,21 @@ "type": "string" }, "custom_instructions": { - "description": "Ingestion instructions applied to this connector's documents. Max 4000 characters; changes apply from the next sync cycle.", + "description": "CustomInstructions is optional free-text guidance applied when this\nconnector's documents are ingested: it steers how content is interpreted\nand indexed. Max 4000 characters; changes apply from the next sync cycle.", "type": "string" }, "database": { - "description": "Database this connector syncs into. `tenant_id` is its deprecated alias.", + "description": "Database/Collection are the canonical v2 names for the deprecated\ntenant_id/sub_tenant_id wire fields. They mirror the same values so a v2\nclient sees the canonical names on responses while a legacy client keeps\nreading tenant_id/sub_tenant_id. Not persisted (dynamodbav:\"-\"): the store\nbuilds items from tenant_id/sub_tenant_id and mirrors these on load. They\nare populated at every construction point (toConnector, connectorFromItem)\nrather than via MarshalJSON so Temporal's JSON data converter round-trips\nConnector activity inputs without spuriously populating them.", "example": "acme_corp", "type": "string" }, "documents_dispatched": { - "description": "Running total of objects sent for ingestion across completed sync cycles. It counts activity, not indexed documents: updates count each time and deletes are not subtracted.", + "description": "DocumentsDispatched is the running total of objects handed to ingestion\nacross all completed cycles. It is dispatch *activity*, not an indexed\ncount: upserts count every time they change, deletes are never\nsubtracted, and an activity retry can double-count. Suitable as an\nis-data-moving signal, never as \"N documents indexed\".", "example": 1, "type": "integer" }, "first_data_dispatched_at": { - "description": "RFC3339 time of the first sync cycle that sent any objects for ingestion.", + "description": "FirstDataDispatchedAt is set once, by the first completed cycle that\ndispatched more than zero objects. Its presence is what proves the\npipeline end to end; after it is set, an empty cycle is \"nothing changed\nat the source\", not \"still ingesting\".", "type": "string" }, "last_attempted_sync_at": { @@ -3176,7 +3050,7 @@ "type": "string" }, "lifecycle": { - "description": "What the connector is doing now, and the status to read: `reconnect`, `syncing`, `pending_setup`, `ingesting` or `active`. `status` is kept for compatibility only, and `sync_status` only says whether a sync is running.", + "description": "Lifecycle is the derived what-is-it-doing-now field and the one status\nclients should read (PRO-1565): reconnect | syncing | pending_setup |\ningesting | active. The embedded `status` field is a scheduler-internal\nconstant (\"active\" always) kept only for compatibility, and `sync_status`\nis the narrower mid-cycle indicator. Computed at the HTTP boundary from\nthe connector's stored facts, never persisted, so it cannot disagree\nwith them.", "type": "string" }, "name": { @@ -3199,18 +3073,6 @@ "example": "2026-07-02T18:00:00Z", "type": "string" }, - "paused": { - "description": "True when the connector was paused by its owner. Only an explicit resume lifts it. Resources keep their sync position, so resuming continues where each one stopped.", - "example": true, - "type": "boolean" - }, - "paused_at": { - "type": "string" - }, - "paused_next_sync_at": { - "description": "The next sync time the pause replaced.", - "type": "string" - }, "provider": { "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", "example": "slack", @@ -3222,7 +3084,7 @@ "type": "string" }, "resources_pending_first_sync": { - "description": "Number of active resources that have not completed their first sync.", + "description": "ResourcesPendingFirstSync counts active resources whose provider_cursor\nis still empty — resources that have never been successfully pulled.\nMOVEIT commits provider_cursor after every successful pull (even a\nzero-row one), so this self-clears one cycle after each resource first\nsyncs. Recomputed by the MOVEIT sync workflow each cycle and by the\nresource-mutating handlers, so a resource added to a long-active\nconnector re-enters the ingesting state.", "example": 1, "type": "integer" }, @@ -3238,7 +3100,7 @@ "x-deprecated": "true" }, "sync_blocked": { - "description": "True when the connector stopped syncing after a failure no retry can fix, such as rejected credentials. Updating the credentials or configuration clears it.", + "description": "NeedsReauth is set by MOVEIT's OAuth refresh sweep when the provider has\nrejected the connector's refresh token (`invalid_grant` — expired, revoked,\nor, for a provider with single-use tokens, already spent).\n\nIt is deliberately distinct from LastError, which records a *sync* failure.\nThis is the one failure class no amount of retrying resolves: the stored\ngrant is gone and only the tenant can mint a new one. Surfacing it as its\nown field is what lets a client show \"reconnect\" instead of a generic\n\"sync failed\", and the sweep clears it automatically on the next successful\nrotation, so a client can trust the absence of the flag as much as its\npresence.\n\nOnly ever set on OAuth-bundle connectors. A connector authenticated with a\nstatic token or with client credentials (X posts: see the `client_id` /\n`client_secret` inputs on tap-twitter) has no refresh token and therefore\ncannot reach this state at all — which is the reason to prefer that shape\nwhere a provider offers it.\nSyncBlocked marks a connector stopped by a terminal failure — one no\nretry can fix. The scheduler skips it and next_sync_at is parked a\ncentury out; only a credential or config update clears it. Distinct from\nNeedsReauth, which is the OAuth sweep's own narrower signal: this covers\nany provider rejection of the stored credentials, including static keys\nthat have no refresh token to sweep.", "example": true, "type": "boolean" }, @@ -3249,16 +3111,16 @@ "type": "string" }, "sync_cycles_completed": { - "description": "Number of sync cycles completed successfully.", + "description": "SyncCyclesCompleted counts successfully completed sync cycles. Bounded\nuse only: it lets DeriveLifecycle stop reporting \"ingesting\" after a few\nclean-but-empty cycles on a source that genuinely has nothing to pull.", "example": 1, "type": "integer" }, "sync_engine": { - "description": "Sync engine that runs this connector. Set by HydraDB; clients do not need to act on it.", + "description": "SyncEngine is \"classic\" (default, empty treated as classic) or \"moveit\".\nSee the SyncEngine* constants; the scheduler branches on it.", "type": "string" }, "sync_interval_seconds": { - "description": "How often incremental syncs run, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", + "description": "How frequently the scheduler triggers incremental syncs, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", "example": 3600, "type": "integer" }, @@ -3308,7 +3170,7 @@ "type": "string" }, "collection": { - "description": "Default collection for synced objects. Deprecated alias: `sub_tenant_id`.", + "description": "Default collection partition for synced objects. Deprecated alias: `sub_tenant_id`.", "example": "team_docs", "type": "string" }, @@ -3321,11 +3183,11 @@ "type": "object" }, "custom_instructions": { - "description": "Ingestion instructions for this connector's documents. Max 4000 characters; can be changed later with PATCH.", + "description": "CustomInstructions optionally steers how this connector's synced\ndocuments are ingested and indexed. Max 4000 characters; editable later\nvia PATCH.", "type": "string" }, "database": { - "description": "Database to sync into. `tenant_id` is accepted as a deprecated alias.", + "description": "Database/Collection are the canonical v2 names; TenantID/SubTenantID are\ntheir deprecated aliases, reconciled by the TenantAliases middleware before\nbinding so TenantID is always populated. Neither is marked binding:required\n(mirroring TenantCreateRequest): a caller may send either spelling, and the\ntenant scope is validated downstream by resolveTenant. Requiring tenant_id\nhere would force the generated SDK to demand the deprecated field.", "example": "acme_corp", "type": "string" }, @@ -3352,7 +3214,7 @@ "x-deprecated": "true" }, "sync_interval_seconds": { - "description": "How often incremental syncs run, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", + "description": "How frequently the scheduler triggers incremental syncs, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", "example": 3600, "type": "integer" }, @@ -3372,11 +3234,11 @@ "handler.connectorCreateResponse": { "properties": { "acl_changes_cursor": { - "description": "Position in the provider's permission-change feed. Empty until the first sync sets a starting point.", + "description": "ACLChangesCursor is the provider permission-change feed's persisted\ncursor (PRO-1684; e.g. the Drive changes.list page token). Empty means\nuninitialized: the next cycle fetches a baseline and starts from now.\nAdvanced ONLY after every reported change was applied, so a failed\napply replays the same changes next cycle (at-least-once; the writes\nare idempotent full replacements).", "type": "string" }, "active_resource_count": { - "description": "Number of resources that are not disabled.", + "description": "ActiveResourceCount mirrors the number of non-disabled resource rows so\nlist responses can distinguish \"no resources configured yet\"\n(pending_setup) without a per-connector resources query.", "example": 1, "type": "integer" }, @@ -3396,25 +3258,25 @@ "type": "string" }, "custom_instructions": { - "description": "Ingestion instructions applied to this connector's documents. Max 4000 characters; changes apply from the next sync cycle.", + "description": "CustomInstructions is optional free-text guidance applied when this\nconnector's documents are ingested: it steers how content is interpreted\nand indexed. Max 4000 characters; changes apply from the next sync cycle.", "type": "string" }, "database": { - "description": "Database this connector syncs into. `tenant_id` is its deprecated alias.", + "description": "Database/Collection are the canonical v2 names for the deprecated\ntenant_id/sub_tenant_id wire fields. They mirror the same values so a v2\nclient sees the canonical names on responses while a legacy client keeps\nreading tenant_id/sub_tenant_id. Not persisted (dynamodbav:\"-\"): the store\nbuilds items from tenant_id/sub_tenant_id and mirrors these on load. They\nare populated at every construction point (toConnector, connectorFromItem)\nrather than via MarshalJSON so Temporal's JSON data converter round-trips\nConnector activity inputs without spuriously populating them.", "example": "acme_corp", "type": "string" }, "documents_dispatched": { - "description": "Running total of objects sent for ingestion across completed sync cycles. It counts activity, not indexed documents: updates count each time and deletes are not subtracted.", + "description": "DocumentsDispatched is the running total of objects handed to ingestion\nacross all completed cycles. It is dispatch *activity*, not an indexed\ncount: upserts count every time they change, deletes are never\nsubtracted, and an activity retry can double-count. Suitable as an\nis-data-moving signal, never as \"N documents indexed\".", "example": 1, "type": "integer" }, "first_data_dispatched_at": { - "description": "RFC3339 time of the first sync cycle that sent any objects for ingestion.", + "description": "FirstDataDispatchedAt is set once, by the first completed cycle that\ndispatched more than zero objects. Its presence is what proves the\npipeline end to end; after it is set, an empty cycle is \"nothing changed\nat the source\", not \"still ingesting\".", "type": "string" }, "first_sync_at": { - "description": "RFC3339 time the first scheduled sync runs.", + "description": "FirstSyncAt is when the first scheduled sync runs (RFC3339).", "type": "string" }, "last_attempted_sync_at": { @@ -3433,11 +3295,11 @@ "type": "string" }, "lifecycle": { - "description": "What the connector is doing now, and the status to read: `reconnect`, `syncing`, `pending_setup`, `ingesting` or `active`. `status` is kept for compatibility only, and `sync_status` only says whether a sync is running.", + "description": "Lifecycle is the derived what-is-it-doing-now field and the one status\nclients should read (PRO-1565): reconnect | syncing | pending_setup |\ningesting | active. The embedded `status` field is a scheduler-internal\nconstant (\"active\" always) kept only for compatibility, and `sync_status`\nis the narrower mid-cycle indicator. Computed at the HTTP boundary from\nthe connector's stored facts, never persisted, so it cannot disagree\nwith them.", "type": "string" }, "message": { - "description": "What to expect next, safe to show to the user as is.", + "description": "Message is a human-readable expectation, safe to show verbatim.", "example": "Success", "type": "string" }, @@ -3461,18 +3323,6 @@ "example": "2026-07-02T18:00:00Z", "type": "string" }, - "paused": { - "description": "True when the connector was paused by its owner. Only an explicit resume lifts it. Resources keep their sync position, so resuming continues where each one stopped.", - "example": true, - "type": "boolean" - }, - "paused_at": { - "type": "string" - }, - "paused_next_sync_at": { - "description": "The next sync time the pause replaced.", - "type": "string" - }, "provider": { "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", "example": "slack", @@ -3484,7 +3334,7 @@ "type": "string" }, "resources_pending_first_sync": { - "description": "Number of active resources that have not completed their first sync.", + "description": "ResourcesPendingFirstSync counts active resources whose provider_cursor\nis still empty — resources that have never been successfully pulled.\nMOVEIT commits provider_cursor after every successful pull (even a\nzero-row one), so this self-clears one cycle after each resource first\nsyncs. Recomputed by the MOVEIT sync workflow each cycle and by the\nresource-mutating handlers, so a resource added to a long-active\nconnector re-enters the ingesting state.", "example": 1, "type": "integer" }, @@ -3500,7 +3350,7 @@ "x-deprecated": "true" }, "sync_blocked": { - "description": "True when the connector stopped syncing after a failure no retry can fix, such as rejected credentials. Updating the credentials or configuration clears it.", + "description": "NeedsReauth is set by MOVEIT's OAuth refresh sweep when the provider has\nrejected the connector's refresh token (`invalid_grant` — expired, revoked,\nor, for a provider with single-use tokens, already spent).\n\nIt is deliberately distinct from LastError, which records a *sync* failure.\nThis is the one failure class no amount of retrying resolves: the stored\ngrant is gone and only the tenant can mint a new one. Surfacing it as its\nown field is what lets a client show \"reconnect\" instead of a generic\n\"sync failed\", and the sweep clears it automatically on the next successful\nrotation, so a client can trust the absence of the flag as much as its\npresence.\n\nOnly ever set on OAuth-bundle connectors. A connector authenticated with a\nstatic token or with client credentials (X posts: see the `client_id` /\n`client_secret` inputs on tap-twitter) has no refresh token and therefore\ncannot reach this state at all — which is the reason to prefer that shape\nwhere a provider offers it.\nSyncBlocked marks a connector stopped by a terminal failure — one no\nretry can fix. The scheduler skips it and next_sync_at is parked a\ncentury out; only a credential or config update clears it. Distinct from\nNeedsReauth, which is the OAuth sweep's own narrower signal: this covers\nany provider rejection of the stored credentials, including static keys\nthat have no refresh token to sweep.", "example": true, "type": "boolean" }, @@ -3511,16 +3361,16 @@ "type": "string" }, "sync_cycles_completed": { - "description": "Number of sync cycles completed successfully.", + "description": "SyncCyclesCompleted counts successfully completed sync cycles. Bounded\nuse only: it lets DeriveLifecycle stop reporting \"ingesting\" after a few\nclean-but-empty cycles on a source that genuinely has nothing to pull.", "example": 1, "type": "integer" }, "sync_engine": { - "description": "Sync engine that runs this connector. Set by HydraDB; clients do not need to act on it.", + "description": "SyncEngine is \"classic\" (default, empty treated as classic) or \"moveit\".\nSee the SyncEngine* constants; the scheduler branches on it.", "type": "string" }, "sync_interval_seconds": { - "description": "How often incremental syncs run, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", + "description": "How frequently the scheduler triggers incremental syncs, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", "example": 3600, "type": "integer" }, @@ -3553,39 +3403,8 @@ }, "type": "object" }, - "handler.connectorLimitView": { - "description": "Present when the workspace has used its plan's connector allowance (Free: 3). Creating another connector returns 402; connectors already created keep syncing. Sent only with health rollups.", - "properties": { - "count": { - "description": "Total number of items returned.", - "example": 12, - "type": "integer" - }, - "limit": { - "example": 1, - "type": "integer" - }, - "message": { - "description": "Human-readable result message.", - "example": "Success", - "type": "string" - }, - "plan": { - "type": "string" - } - }, - "type": "object" - }, "handler.connectorListResponse": { "properties": { - "connector_limit": { - "$ref": "#/components/schemas/handler.connectorLimitView", - "example": { - "count": 12, - "limit": 1, - "message": "Success" - } - }, "connectors": { "example": [ { @@ -3601,7 +3420,6 @@ "name": "general", "needs_reauth": true, "next_sync_at": "2026-07-02T18:00:00Z", - "paused": true, "provider": "slack", "provider_account_scope": "T12345ACME", "resources_pending_first_sync": 1, @@ -3624,47 +3442,16 @@ "additionalProperties": { "type": "string" }, - "description": "Map of connector id to health (`healthy`, `degraded`, `failed`, `checking` or `capped`), returned with `?include=health`. A connector missing from the map has unknown health and should not be shown as failed. `capped` means the workspace is at a plan cap; `plan_cap` says which.", + "description": "Health maps connector_id to its rollup (healthy | degraded | failed |\nchecking), present only when the caller asks for `?include=health`.\nA connector missing from the map has an unknown rollup — its resources\ncould not be read — which clients must not render as a failure.", "type": "object" - }, - "plan_cap": { - "$ref": "#/components/schemas/handler.planCapView", - "example": { - "message": "Success" - } - } - }, - "type": "object" - }, - "handler.connectorPauseResponse": { - "properties": { - "connector_id": { - "description": "Connector this resource belongs to.", - "example": "conn_abc123", - "type": "string" - }, - "paused": { - "example": true, - "type": "boolean" - }, - "paused_at": { - "type": "string" } }, "type": "object" }, "handler.connectorResourceStatus": { "properties": { - "acl_warning": { - "description": "Why permission capture could not read this resource. While set, the resource is readable by every caller whatever `acl` they send. It is not a sync failure: content stays current. Fix it by granting the missing permission or setting an access rule.", - "type": "string" - }, - "acl_warning_at": { - "description": "RFC3339 time `acl_warning` last changed, not when it was last checked. Read it as \"open since\".", - "type": "string" - }, "action": { - "description": "What the user needs to do, when there is something they can do.", + "description": "Action is what the user must do, when there is something they can do.", "type": "string" }, "checked_at": { @@ -3676,54 +3463,46 @@ "type": "string" }, "http_status": { - "description": "The provider's HTTP status code, when it reported one.", + "description": "HTTPStatus is the provider's response code when one was reported.", "example": 1, "type": "integer" }, "last_row_count": { - "description": "Records fetched by the last sync.", + "description": "LastRowCount is the rows produced by the last sync.", "example": 1, "type": "integer" }, "message": { - "description": "The provider's own error message, when `status` is `failed`.", + "description": "Message is the provider's own words when Status is failed.", "example": "Success", "type": "string" }, - "page_acl_warning": { - "description": "Set when pages inside this resource had restrictions that could not be read and were opened to every caller. Separate from `acl_warning`, which covers the resource itself.", - "type": "string" - }, - "page_acl_warning_at": { - "description": "RFC3339 time `page_acl_warning` last changed.", - "type": "string" - }, "resource_id": { "description": "Resource identifier from the Discover endpoint.", "example": "C0123456789", "type": "string" }, "retryable": { - "description": "Set only on a failed resource: false when the user has to fix something (a 403, a 404, a misconfigured table), true when it may clear on its own.", + "description": "Retryable is set only for a failed resource: false for a provider\nrejection the user must fix (403, 404, a misconfigured table), true for\nsomething that may clear on its own.", "example": true, "type": "boolean" }, "status": { - "description": "One of `ok`, `empty`, `failed`, `checking` or `unknown`.", + "description": "Status is one of ok | empty | failed | checking | unknown.", "example": "completed", "type": "string" }, "sync_blocked": { - "description": "True when this resource has stopped syncing, rather than failing and retrying.", + "description": "SyncBlocked reports that this resource has stopped syncing. Distinct from\na failed status: a resource can fail a cycle and be retried, and the\ndifference between \"failing\" and \"given up on\" is the one a user needs to\nact on.", "example": true, "type": "boolean" }, "sync_blocked_at": { - "description": "RFC3339 time the resource stopped syncing.", + "description": "SyncBlockedAt is when it stopped (RFC3339).", "type": "string" }, "sync_blocked_reason": { - "description": "Why the resource stopped syncing.", + "description": "SyncBlockedReason is why it stopped, preserved from the failure that\nstopped it so it survives later syncs overwriting the health block.", "type": "string" } }, @@ -3769,14 +3548,14 @@ "type": "object" }, "handler.connectorStatusError": { - "description": "Error is the connector-level failure: a rejected credential, a blocked\nconnector, or a latest sync cycle that failed as a whole. Absent when the\ntrouble is confined to individual resources — those carry their own\nmessages below.", + "description": "Error is the connector-level failure, set when Status is failed. Absent\nwhen the trouble is confined to individual resources — those carry their\nown messages below.", "properties": { "action": { - "description": "What the user needs to do, when there is something they can do.", + "description": "Action is what the user must do, when there is something they can do.", "type": "string" }, "detected_at": { - "description": "RFC3339 time the failure was detected.", + "description": "DetectedAt is when the failure was observed (RFC3339).", "type": "string" }, "message": { @@ -3785,7 +3564,7 @@ "type": "string" }, "retryable": { - "description": "Whether waiting can fix this. False means the user has to act, for example by reconnecting a rejected credential.", + "description": "Retryable reports whether waiting can fix this. False means only the user\ncan: a rejected credential is not a transient error, and telling someone\nto retry a dead OAuth grant wastes their time.", "example": true, "type": "boolean" } @@ -3818,7 +3597,7 @@ "type": "string" }, "lifecycle": { - "description": "What the connector is doing now: `reconnect`, `syncing`, `pending_setup`, `ingesting` or `active`. Independent of `status`: a connector can be ingesting and healthy, or active and degraded.", + "description": "Lifecycle is the derived what-is-it-doing-now field (PRO-1565):\nreconnect | syncing | pending_setup | ingesting | active. Orthogonal to\nthe health rollup above — a connector can be ingesting and healthy, or\nactive and degraded.", "type": "string" }, "next_sync_at": { @@ -3826,19 +3605,13 @@ "example": "2026-07-02T18:00:00Z", "type": "string" }, - "plan_cap": { - "$ref": "#/components/schemas/handler.planCapView", - "example": { - "message": "Success" - } - }, "provider": { "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", "example": "slack", "type": "string" }, "resources": { - "description": "One entry per configured resource. An empty list when there are none.", + "description": "Resources is one entry per configured resource, always non-nil so it\nserialises as [] rather than null.", "example": [ { "display_name": "general", @@ -3858,12 +3631,12 @@ "uniqueItems": false }, "status": { - "description": "Health rollup, the worst of the credential and every resource: `healthy`, `degraded`, `failed` or `checking`. `capped` replaces it when the workspace is at a plan cap; `plan_cap` says which.", + "description": "Status is the rollup: healthy | degraded | failed | checking. It is the\nworst of the credential state and every resource state.", "example": "completed", "type": "string" }, "sync_status": { - "description": "`syncing` or `idle`. Independent of `status`.", + "description": "SyncStatus is the in-progress indicator (\"syncing\"/\"idle\"), orthogonal to\nStatus — a connector can be mid-sync and degraded at the same time.", "example": "idle", "type": "string" } @@ -3892,11 +3665,11 @@ "type": "object" }, "custom_instructions": { - "description": "Replaces the connector's ingestion instructions. Omit to leave them unchanged; send an empty string to clear them. Max 4000 characters; applies from the next sync cycle.", + "description": "CustomInstructions replaces the guidance applied when this connector's\ndocuments are ingested. Omitted leaves it unchanged; an explicit empty\nstring clears it. Max 4000 characters; applies from the next sync cycle.", "type": "string" }, "sync_interval_seconds": { - "description": "How often incremental syncs run, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", + "description": "How frequently the scheduler triggers incremental syncs, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", "example": 3600, "type": "integer" } @@ -3911,12 +3684,12 @@ "type": "string" }, "credentials_updated": { - "description": "True when this request replaced the stored credentials, which also clears any reconnect state.", + "description": "CredentialsUpdated reports that the stored credential bundle was\nre-written (and any needs-reauth flag cleared) by this request.", "example": true, "type": "boolean" }, "custom_instructions_updated": { - "description": "True when this request changed or cleared the ingestion instructions. Takes effect from the next sync cycle.", + "description": "CustomInstructionsUpdated reports that the steering text was rewritten\n(or cleared) by this request; it takes effect from the next sync cycle.", "example": true, "type": "boolean" }, @@ -3936,7 +3709,7 @@ "type": "string" }, "sync_interval_seconds": { - "description": "How often incremental syncs run, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", + "description": "How frequently the scheduler triggers incremental syncs, in seconds. Bounded per provider; send 0 or omit to use the provider default. Change it later with PATCH /connectors/{id}.", "example": 3600, "type": "integer" } @@ -3946,7 +3719,7 @@ "handler.contextMetadataUpdateRequest": { "properties": { "acl": { - "description": "Replaces the source's access-control list without re-ingesting. Send the complete new list, `[]` to make it private, `[\"__public__\"]` to open it to every identified caller, or `null` to revoke all access. Omit to leave it unchanged.", + "description": "ACL, when present, REPLACES the source's access-control list without\nre-ingestion (PRO-1684): pass the COMPLETE new allow-list (adding a\nthird user means sending all three), an empty list to make the source\nprivate, or [\"__public__\"] to open it to every identified caller. A\npointer so omitted (nil, ACL untouched) is distinguishable from an\nexplicit empty list (private).\nACL uses RawMessage so the handler can tell three wire states apart:\nabsent (leave the stored ACL untouched), explicit null (revoke to\nnobody, JSON-merge-patch semantics), and a list (replace). A plain\n*[]string cannot: encoding/json leaves the pointer nil for BOTH\nabsent and null, which silently ignored an explicit null revocation.", "items": { "type": "string" }, @@ -3968,7 +3741,7 @@ "type": "string" }, "database": { - "description": "Database containing the source. `tenant_id` is accepted as a deprecated alias.", + "description": "Database/Collection are the canonical v2 names; TenantID/SubTenantID are\ntheir deprecated aliases. The TenantAliases middleware reconciles them in\nthe request body before binding, so the handler reads TenantID/SubTenantID.", "example": "acme_corp", "type": "string" }, @@ -4153,64 +3926,6 @@ }, "type": "object" }, - "handler.instructionsResponse": { - "properties": { - "collections": { - "additionalProperties": { - "type": "string" - }, - "description": "Map of collection name to that collection's own instructions.", - "example": [ - "team_docs", - "engineering" - ], - "type": "object" - }, - "custom_instructions": { - "description": "Instructions applied to every document ingested into the database.", - "type": "string" - }, - "database": { - "description": "Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated).", - "example": "acme_corp", - "type": "string" - }, - "tenant_id": { - "deprecated": true, - "description": "Deprecated alias for `database`.", - "example": "acme_corp", - "type": "string", - "x-deprecated": "true" - } - }, - "type": "object" - }, - "handler.instructionsUpdateReq": { - "properties": { - "collections": { - "additionalProperties": { - "type": [ - "string", - "null" - ] - }, - "description": "Per-collection instructions to merge in: a collection with a value is set, a collection with an empty string or null is cleared, and a collection left out is unchanged.", - "example": { - "engineering": null, - "team_docs": "Summarise decisions and who owns them." - }, - "type": "object" - }, - "custom_instructions": { - "description": "Instructions for the whole database. Send an empty string to clear them.", - "type": [ - "string", - "null" - ] - } - }, - "type": "object" - }, "handler.metadataSchemaUpdateResponse": { "properties": { "added_fields": { @@ -4239,23 +3954,6 @@ }, "type": "object" }, - "handler.planCapView": { - "description": "Present when the workspace is at a plan cap. Syncs are skipped until the month resets or the plan changes.", - "properties": { - "message": { - "description": "Human-readable result message.", - "example": "Success", - "type": "string" - }, - "meter": { - "type": "string" - }, - "plan": { - "type": "string" - } - }, - "type": "object" - }, "handler.providerListResponse": { "properties": { "providers": { @@ -4283,7 +3981,7 @@ "handler.resourceCreateReq": { "properties": { "acl": { - "description": "Principals allowed to retrieve objects synced from this resource. Omit for unrestricted.", + "description": "ACL restricts every object synced from this resource to the listed\nprincipals (see resourceMapping.ACL). Omitted means unrestricted.", "items": { "type": "string" }, @@ -4303,7 +4001,7 @@ "type": "string" }, "custom_instructions": { - "description": "Ingestion instructions for objects synced from this resource. When set, they replace the connector's `custom_instructions`; empty inherits the connector's. Max 4000 characters; changes apply from the next sync cycle.", + "description": "CustomInstructions optionally steers how documents synced from this\nresource are ingested and indexed. When set it replaces the\nconnector-level custom_instructions for this resource; empty inherits\nthe connector's value. Max 4000 characters.", "type": "string" }, "database_override": { @@ -4357,7 +4055,7 @@ }, "tenant_id_override": { "deprecated": true, - "description": "Deprecated alias for `database_override`.", + "description": "DatabaseOverride/CollectionOverride are the canonical v2 names;\nTenantIDOverride/SubTenantIDOverride are their deprecated aliases.", "type": "string", "x-deprecated": "true" } @@ -4390,7 +4088,7 @@ "handler.resourceMapping": { "properties": { "acl": { - "description": "Principals allowed to retrieve objects synced from this resource: emails, or `user_email:`, `group:` or `domain:` principals. Omit for unrestricted; an empty list makes them private.", + "description": "ACL restricts every object synced from this resource to the listed\nprincipals (emails, or prefixed principals, see the query-side\nuser_email parameter). Omitted means unrestricted. An explicitly empty\nlist means private (visible only to unfiltered queries).", "items": { "type": "string" }, @@ -4399,7 +4097,7 @@ }, "additional_metadata": { "additionalProperties": {}, - "description": "Custom attributes merged into every object synced from this resource. Fields set by the provider win on conflict.", + "description": "AdditionalMetadata is merged into the additional_metadata layer of every\nobject synced from this resource. Provider-generated fields take precedence.", "example": { "author": "ada", "doc_version": 3 @@ -4407,22 +4105,22 @@ "type": "object" }, "collection": { - "description": "Collection that objects from this resource go to, overriding the connector's. Empty inherits the connector's collection. `sub_tenant_id` is accepted as a deprecated alias.", + "description": "Collection is the canonical v2 name for the per-resource sub-tenant\noverride: routes synced objects from this resource into a specific\ncollection. Empty means the resource inherits the connector collection.\nSubTenantID is the deprecated alias for this field, reconciled by\nConfigure before toResource runs.", "example": "team_docs", "type": "string" }, "custom_instructions": { - "description": "Ingestion instructions for objects synced from this resource. When set, they replace the connector's `custom_instructions`; empty inherits the connector's. Max 4000 characters; changes apply from the next sync cycle.", + "description": "CustomInstructions optionally steers how documents synced from this\nresource are ingested and indexed. When set it replaces the\nconnector-level custom_instructions for this resource; empty inherits\nthe connector's value. Max 4000 characters.", "type": "string" }, "database": { - "description": "Database that objects from this resource go to, overriding the connector's. Empty inherits the connector's database. `tenant_id` is accepted as a deprecated alias.", + "description": "Database is the canonical v2 name for the per-resource tenant override:\nroutes synced objects from this resource into a specific database.\nEmpty means the resource inherits the connector database. TenantID is the\ndeprecated alias for this field, reconciled by Configure before\ntoResource runs.", "example": "acme_corp", "type": "string" }, "metadata": { "additionalProperties": {}, - "description": "Attributes merged into every object synced from this resource. Fields HydraDB sets (`connector_id`, `provider`) win on conflict.", + "description": "Metadata is merged into the tenant metadata layer of every object synced\nfrom this resource. System fields (connector_id, provider) take precedence.", "example": { "department": "finance", "priority": 7 @@ -4446,13 +4144,13 @@ }, "sub_tenant_id": { "deprecated": true, - "description": "Deprecated alias for `collection`.", + "description": "Routes synced objects from this resource into a specific sub-tenant\npartition. Overrides the connector-level sub_tenant_id. Deprecated: use\ncollection.", "example": "sub_tenant_4567", "type": "string", "x-deprecated": "true" }, "sync_mode": { - "description": "Update strategy, for providers that support one (Attio objects and lists). `rescan` (default) re-reads everything each sync and picks up edits. `new_only` reads only the latest window: cheaper, but edits to older records are not picked up.", + "description": "SyncMode is the per-resource update strategy for taps that support one\n(today: attio objects/lists). \"rescan\" (default) re-reads the full set\nevery sync — the only way edits are seen on APIs with no updated_at.\n\"new_only\" bounds each scan to the sync window and stops paging at its\nfloor — cheap, and an explicit opt-in to not seeing edits until\nwebhooks land. Stored in the resource's filters and carried to the tap\non every window.", "enum": [ "rescan", "new_only" @@ -4461,7 +4159,7 @@ }, "tenant_id": { "deprecated": true, - "description": "Deprecated alias for `database`.", + "description": "Optional per-resource tenant override (for \"route specific resources to\ndifferent tenants\"). Empty means the resource inherits the connector\ntenant. Deprecated: use database.", "example": "tenant_1234", "type": "string", "x-deprecated": "true" @@ -4475,7 +4173,7 @@ "handler.responseMeta": { "properties": { "api_version": { - "description": "Version of the API that served the request; the same value as the spec's `info.version`.", + "description": "APIVersion echoes the version of the API that served the request (PRO-1209),\nsourced from reqmeta.APIVersion — the same value carried by OpenAPI\ninfo.version and /health — so a client always knows which API version\nproduced a response. Always present (no omitempty).", "type": "string" }, "collection": { @@ -4489,7 +4187,7 @@ "type": "string" }, "deprecation": { - "description": "Migration notices, present when the request used a legacy route or field. The status code is unaffected.", + "description": "Deprecation lists any migration nudges that apply to this request — the\ncaller used a legacy /tenants route, a legacy tenant_id/sub_tenant_id field,\nor the deprecated sub_tenant_ids selector. It is a non-breaking signal (the\nstatus code is unchanged); omitempty keeps it absent for fully-migrated\nrequests. A list so independent deprecations coexist without clobbering.", "items": { "$ref": "#/components/schemas/handler.deprecationNotice" }, @@ -4558,126 +4256,6 @@ }, "type": "object" }, - "handler.vaultCredentialEntry": { - "properties": { - "collection": { - "description": "Collection scope. Defaults to the default collection when omitted. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated).", - "example": "team_docs", - "type": "string" - }, - "connector_id": { - "description": "Connector this resource belongs to.", - "example": "conn_abc123", - "type": "string" - }, - "credential_id": { - "type": "string" - }, - "database": { - "description": "Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated).", - "example": "acme_corp", - "type": "string" - }, - "fields": { - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "label": { - "type": "string" - }, - "provider": { - "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", - "example": "slack", - "type": "string" - } - }, - "type": "object" - }, - "handler.vaultCredentialListResponse": { - "properties": { - "count": { - "description": "Total number of items returned.", - "example": 12, - "type": "integer" - }, - "credentials": { - "description": "Provider-specific credentials (typically `{\"api_token\": \"...\"}` or `{\"access_token\": \"...\"}`).", - "example": { - "api_token": "xoxb-..." - }, - "items": { - "$ref": "#/components/schemas/handler.vaultCredentialEntry" - }, - "type": "array", - "uniqueItems": false - }, - "unavailable_count": { - "example": 1, - "type": "integer" - } - }, - "type": "object" - }, - "handler.vaultCredentialRevealReq": { - "properties": { - "field": { - "type": "string" - } - }, - "required": [ - "field" - ], - "type": "object" - }, - "handler.vaultCredentialRevealResponse": { - "properties": { - "credential_id": { - "type": "string" - }, - "field": { - "type": "string" - }, - "provider": { - "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", - "example": "slack", - "type": "string" - }, - "value": {} - }, - "type": "object" - }, - "handler.vaultCredentialUpdateReq": { - "properties": { - "credentials": { - "additionalProperties": {}, - "description": "Provider-specific credentials (typically `{\"api_token\": \"...\"}` or `{\"access_token\": \"...\"}`).", - "example": { - "api_token": "xoxb-..." - }, - "type": "object" - } - }, - "required": [ - "credentials" - ], - "type": "object" - }, - "handler.vaultCredentialUpdateResponse": { - "properties": { - "credential_id": { - "type": "string" - }, - "updated": { - "description": "Whether the source metadata was updated.", - "example": true, - "type": "boolean" - } - }, - "type": "object" - }, "ingestion.SourceStatus": { "description": "Status is the item's initial lifecycle state. Both modes share this\nvocabulary — memory mode reuses the same values.", "enum": [ @@ -4750,7 +4328,7 @@ }, "success": { "deprecated": true, - "description": "Deprecated. Always mirrors the envelope's `success`. Whether the request was accepted is the HTTP status (202). Whether each item ingested is in `results[].status` and `results[].error`, then `GET /context/status`.", + "description": "Deprecated for API clients: whether the REQUEST was accepted is the HTTP\nstatus code (202) or equivalently the envelope's top-level `success`.\nWhether each SOURCE ingested is per-item — read results[].status and\nresults[].error, then poll GET /context/status, since a 202 only means\nqueued. This flag answers neither question independently: it always\nmirrors the envelope. Still emitted unchanged for existing clients\n(PRO-1208).", "example": true, "type": "boolean", "x-deprecated": "true" @@ -4766,36 +4344,36 @@ "ingestion.V2IngestResultItem": { "properties": { "error": { - "description": "Why this item failed, or null on success.", + "description": "Error is the failure message for this item, null on success. Both modes.", "example": "", "type": "string" }, "error_code": { - "description": "Machine-readable failure code, or null on success.", + "description": "ErrorCode is the machine-readable failure classification, null on success.\nBoth modes; always null on the memory path, which produces no per-item code.", "type": "string" }, "filename": { - "description": "The file's original name. Split databases with `type=knowledge` only.", + "description": "Filename is the original filename as submitted. type=knowledge only.", "example": "policy.pdf", "type": "string" }, "id": { - "description": "Identifier assigned to this item.", + "description": "ID is the source identifier assigned to this item. Both modes.", "example": "HydraDoc1234", "type": "string" }, "infer": { - "description": "Whether the memory was queued for enrichment. Split databases with `type=memory` only.", + "description": "Infer reports whether the memory was queued for inference. type=memory only.", "example": true, "type": "boolean" }, "relations_created": { - "description": "Graph relations created from this item's `relations`. Split databases with `type=knowledge` only.", + "description": "RelationsCreated is the number of graph relations extracted from this file.\ntype=knowledge only, and only for items that carried a `relations` payload.", "example": 5, "type": "integer" }, "relations_error": { - "description": "Why creating those relations failed, if it did. Split databases with `type=knowledge` only.", + "description": "RelationsError is the relation-extraction failure message, if any.\ntype=knowledge only.", "type": "string" }, "status": { @@ -4803,7 +4381,7 @@ "description": "Current lifecycle or processing state." }, "title": { - "description": "The memory's title. Split databases with `type=memory` only.", + "description": "Title is the memory's title. type=memory only.", "example": "Project Phoenix Overview", "type": "string" } @@ -4839,7 +4417,7 @@ }, "success": { "deprecated": true, - "description": "Deprecated. Always mirrors the envelope's `success`, even for a source that failed indexing. Read `indexing_status`, and `error_code` / `error_message`, for this source.", + "description": "Deprecated for API clients: this reads like a per-source outcome but is\na constant echo of the envelope's `success` — it is true even for a\nsource that failed indexing. For the state of THIS source read\nindexing_status (and error_code/error_message when it is errored); for\nwhether the request itself succeeded read the HTTP status code or the\nenvelope's top-level `success`. Still emitted unchanged for existing\nclients (PRO-1208).", "example": true, "type": "boolean", "x-deprecated": "true" @@ -4867,7 +4445,7 @@ }, "source_fields": { "additionalProperties": {}, - "description": "Filter on source fields: `title`, `type`, `description`, `url`, `timestamp`, and the app-source fields `app_provider`, `app_kind`, `app_external_id` and `app_parent_id`. External and parent ids are only unique per provider, so pair them with `app_provider`.", + "description": "SourceFields filters by well-known source fields: title, type,\ndescription, url, timestamp, and the app-source keys app_provider,\napp_kind, app_external_id, app_parent_id.\n\napp_external_id and app_parent_id are provider-scoped: a Jira issue key\nand a Linear id can collide, so pair either with app_provider in the\nsame filter to identify one object. Without it a match may span\nproviders that reuse the same external id.", "type": "object" } }, @@ -4876,7 +4454,7 @@ "list.V2ListContentRequest": { "properties": { "acl": { - "description": "Principals to answer as: only sources they may see are listed. Omit for no access scoping.", + "description": "ACL: see ListContentRequest.ACL (PRO-1684 document ACLs).", "items": { "type": "string" }, @@ -4889,7 +4467,7 @@ "type": "string" }, "database": { - "description": "Database to list from. `tenant_id` is accepted as a deprecated alias.", + "description": "Database/Collection are the canonical v2 names; TenantID/SubTenantID are\ntheir deprecated aliases (reconciled here in UnmarshalJSON and centrally by\nthe TenantAliases middleware).", "example": "acme_corp", "type": "string" }, @@ -4905,7 +4483,7 @@ } }, "group_threads": { - "description": "On split databases with `type=knowledge`, nest each thread's comments and replies under their parent as `comments`, newest first, instead of listing them as separate rows. Off by default.", + "description": "GroupThreads (type=knowledge only) folds each ticket's/thread root's\ndiscussion (comment and message app sources carrying an app_parent_id)\nunder the parent row as `comments`, newest first, instead of listing them\nas separate top-level rows. Off by default: the flat shape is the\nexisting contract.", "example": true, "type": "boolean" }, @@ -4959,12 +4537,10 @@ "x-deprecated": "true" }, "type": { - "description": "Which corpus to list. Split database: `knowledge` (default) or `memory`. Unified database: omit it to list every item in one page; `knowledge` and `memory` are rejected.", + "description": "Bucket to list: `knowledge` (default) or `memory`.", "enum": [ "knowledge", - "memory", - "unified", - "all" + "memory" ], "example": "knowledge", "type": "string" @@ -4991,7 +4567,7 @@ } }, "sources": { - "description": "The listed items (on a split database, when `type=knowledge`, the default).", + "description": "Sources carries the rows when type=knowledge (the default).", "example": [ { "additional_metadata": { @@ -5026,18 +4602,18 @@ }, "success": { "deprecated": true, - "description": "Deprecated. Always carries the same value as the envelope's top-level `success`. Check the HTTP status code (2xx is success) instead.", + "description": "Deprecated for API clients: to decide whether the request succeeded,\ncheck the HTTP status code — 2xx is success — or equivalently the\nenvelope's top-level `success`. This nested copy always carries the same\nvalue and never carries independent information. Still emitted unchanged\nfor existing clients (PRO-1208).", "example": true, "type": "boolean", "x-deprecated": "true" }, "total": { - "description": "Total number of matching items across all pages.", + "description": "Total is the total number of matching rows across all pages.", "example": 128, "type": "integer" }, "user_memories": { - "description": "Memories, when listing a split database with `type=memory`. Same shape as `sources`, keyed by `memory_id`.", + "description": "UserMemories carries the rows when type=memory. Same item shape as Sources\nexcept each row is keyed by memory_id rather than id.", "example": [ { "additional_metadata": { @@ -5077,7 +4653,7 @@ "properties": { "additional_metadata": { "additionalProperties": {}, - "description": "Custom attributes supplied at ingest.", + "description": "AdditionalMetadata is the caller-supplied per-document metadata (stored as\ndocument_metadata).", "example": { "author": "ada", "doc_version": 3 @@ -5095,11 +4671,11 @@ "type": "string" }, "app_parent_id": { - "description": "Provider id of this source's parent in a conversation (for example a Jira comment's issue key, or a Slack reply's thread root). `app_thread_id` is the thread it belongs to. Absent for sources without one.", + "description": "AppParentID is the provider external id of this source's conversational\nparent (a Jira comment carries its issue key, a Slack reply its thread\nroot), and AppThreadID the discussion grouping key. Mirrored from the\ningestion pipeline; absent for sources without a parent/thread.", "type": "string" }, "app_provider": { - "description": "Connector provider the source came from (for example `slack` or `github`). Null when the source did not come from a connector.", + "description": "App* carry connector provenance, mirrored onto the source document by the\ningestion pipeline. Null for sources that were not connector-ingested.", "example": "slack", "type": "string" }, @@ -5108,12 +4684,12 @@ "type": "string" }, "collection": { - "description": "Collection the item was listed from. Empty when it is in the database's default collection.", + "description": "Collection is the canonical name for the sub-scope this row was listed\nfrom. Empty string when the row lives in the database's default collection.", "example": "team_docs", "type": "string" }, "comments": { - "description": "With `group_threads`, the source's comments and replies as full rows, newest first, capped per parent (`comments_truncated` marks when more exist). Present on every row of a `group_threads` response.", + "description": "Comments is the group_threads discussion: the source's comment/message\nchildren as full sibling rows, newest first, capped per parent with\nCommentsTruncated marking an overflow. Present (possibly empty) on every\nrow of a group_threads response; absent otherwise.", "items": { "additionalProperties": {}, "type": "object" @@ -5126,7 +4702,7 @@ "type": "boolean" }, "database": { - "description": "Database the item was listed from.", + "description": "Database is the canonical name for the scope this row was listed from.", "example": "acme_corp", "type": "string" }, @@ -5136,13 +4712,13 @@ "type": "string" }, "memory_id": { - "description": "Memory identifier; the memory spelling of `id`.", + "description": "MemoryID is the memory identifier — the type=memory spelling of ID, and\npresent on exactly the same terms.", "example": "memory_1234", "type": "string" }, "metadata": { "additionalProperties": {}, - "description": "Attributes supplied at ingest.", + "description": "Metadata is the caller-supplied source metadata (stored as tenant_metadata).", "example": { "department": "finance", "priority": 7 @@ -5154,18 +4730,18 @@ "type": "string" }, "relations": { - "description": "Graph relations attached to the source, returned only when requested with `include_fields`." + "description": "Relations/AppRelations are passthrough graph subtrees, returned only when\nrequested via include_fields. Their internal source_id/source_ids keys are\nrenamed to id/ids on the way out; the rest of the subtree is unconstrained." }, "sub_tenant_id": { "deprecated": true, - "description": "Deprecated alias for `collection`.", + "description": "SubTenantID is the deprecated spelling of Collection, carrying an identical value.", "example": "sub_tenant_4567", "type": "string", "x-deprecated": "true" }, "tenant_id": { "deprecated": true, - "description": "Deprecated alias for `database`.", + "description": "TenantID is the deprecated spelling of Database, carrying an identical value.", "example": "tenant_1234", "type": "string", "x-deprecated": "true" @@ -5181,7 +4757,7 @@ "type": "string" }, "type": { - "description": "Kind of item, for example `knowledge` or `memory`.", + "description": "Type is the source kind, e.g. \"knowledge\" or \"memory\".", "example": "knowledge", "type": "string" } @@ -5199,7 +4775,7 @@ "properties": { "additional_metadata": { "additionalProperties": {}, - "description": "Custom attributes supplied at ingest.", + "description": "AdditionalMetadata is the caller-supplied per-document metadata (stored as\ndocument_metadata).", "example": { "author": "ada", "doc_version": 3 @@ -5217,11 +4793,11 @@ "type": "string" }, "app_parent_id": { - "description": "Provider id of this source's parent in a conversation (for example a Jira comment's issue key, or a Slack reply's thread root). `app_thread_id` is the thread it belongs to. Absent for sources without one.", + "description": "AppParentID is the provider external id of this source's conversational\nparent (a Jira comment carries its issue key, a Slack reply its thread\nroot), and AppThreadID the discussion grouping key. Mirrored from the\ningestion pipeline; absent for sources without a parent/thread.", "type": "string" }, "app_provider": { - "description": "Connector provider the source came from (for example `slack` or `github`). Null when the source did not come from a connector.", + "description": "App* carry connector provenance, mirrored onto the source document by the\ningestion pipeline. Null for sources that were not connector-ingested.", "example": "slack", "type": "string" }, @@ -5233,12 +4809,12 @@ "type": "string" }, "collection": { - "description": "Collection the item was listed from. Empty when it is in the database's default collection.", + "description": "Collection is the canonical name for the sub-scope this row was listed\nfrom. Empty string when the row lives in the database's default collection.", "example": "team_docs", "type": "string" }, "comments": { - "description": "With `group_threads`, the source's comments and replies as full rows, newest first, capped per parent (`comments_truncated` marks when more exist). Present on every row of a `group_threads` response.", + "description": "Comments is the group_threads discussion: the source's comment/message\nchildren as full sibling rows, newest first, capped per parent with\nCommentsTruncated marking an overflow. Present (possibly empty) on every\nrow of a group_threads response; absent otherwise.", "items": { "additionalProperties": {}, "type": "object" @@ -5252,7 +4828,7 @@ "type": "boolean" }, "database": { - "description": "Database the item was listed from.", + "description": "Database is the canonical name for the scope this row was listed from.", "example": "acme_corp", "type": "string" }, @@ -5262,13 +4838,13 @@ "type": "string" }, "id": { - "description": "Source identifier. Always present.", + "description": "ID is the source identifier. Always present: buildProjection pins\nsource.id as an identity field on every projection path.", "example": "HydraDoc1234", "type": "string" }, "metadata": { "additionalProperties": {}, - "description": "Attributes supplied at ingest.", + "description": "Metadata is the caller-supplied source metadata (stored as tenant_metadata).", "example": { "department": "finance", "priority": 7 @@ -5281,18 +4857,18 @@ "type": "string" }, "relations": { - "description": "Graph relations attached to the source, returned only when requested with `include_fields`." + "description": "Relations/AppRelations are passthrough graph subtrees, returned only when\nrequested via include_fields. Their internal source_id/source_ids keys are\nrenamed to id/ids on the way out; the rest of the subtree is unconstrained." }, "sub_tenant_id": { "deprecated": true, - "description": "Deprecated alias for `collection`.", + "description": "SubTenantID is the deprecated spelling of Collection, carrying an identical value.", "example": "sub_tenant_4567", "type": "string", "x-deprecated": "true" }, "tenant_id": { "deprecated": true, - "description": "Deprecated alias for `database`.", + "description": "TenantID is the deprecated spelling of Database, carrying an identical value.", "example": "tenant_1234", "type": "string", "x-deprecated": "true" @@ -5308,7 +4884,7 @@ "type": "string" }, "type": { - "description": "Kind of item, for example `knowledge` or `memory`.", + "description": "Type is the source kind, e.g. \"knowledge\" or \"memory\".", "example": "knowledge", "type": "string" } @@ -5322,559 +4898,513 @@ ], "type": "object" }, - "memories.ConversationTurn": { - "properties": { - "content": { - "description": "Extracted text content of the source document.", - "example": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "type": "string" - }, - "name": { - "description": "Human-readable label for this resource.", - "example": "general", - "type": "string" - }, - "role": { - "type": "string" - } - }, - "type": "object" - }, - "memories.IngestItem": { + "search.ChunkInspectResult": { "properties": { - "acl": { - "description": "Principals allowed to retrieve this item: bare emails, `user_email:`, `group:` or `domain:` principals, or `__public__`. Omit for unrestricted; an empty list makes it private, so only requests that send no `acl` see it. A malformed principal is a 400.", - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "attributes": { - "additionalProperties": {}, - "type": "object" - }, - "content": { - "description": "Extracted text content of the source document.", - "example": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "type": "string" - }, - "context_category": { - "description": "What kind of context this is: `auto` (default, HydraDB decides), `user_preference`, `business_knowledge` or `decision_trace`. A category you name is never relabelled.", - "enum": [ - "auto", - "user_preference", - "business_knowledge", - "decision_trace" - ], - "type": "string" - }, - "context_id": { - "type": "string" - }, - "conversation": { - "description": "A conversation as a list of `role` / `content` turns, the same shape used for OpenAI or Anthropic messages. `messages` is accepted as an alias.", + "chunks": { "example": [ { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "name": "general" + "additional_metadata": { + "author": "ada", + "doc_version": 3 + }, + "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", + "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", + "extra_context_ids": [ + "HydraEmbeddings123_2", + "HydraEmbeddings123_3" + ], + "layout": "text", + "metadata": { + "department": "finance", + "priority": 7 + }, + "relevancy_score": 0.87, + "source_id": "HydraDoc1234", + "source_last_updated_time": "2026-07-02T12:30:00Z", + "source_title": "Project Phoenix Overview", + "source_type": "file", + "source_upload_time": "2026-07-02T10:00:00Z", + "sub_tenant_id": "sub_tenant_4567" } ], "items": { - "$ref": "#/components/schemas/memories.ConversationTurn" + "$ref": "#/components/schemas/search.VectorStoreChunk" }, "type": "array", "uniqueItems": false }, - "custom_attributes": { - "additionalProperties": {}, - "type": "object" - }, - "custom_instructions": { - "type": "string" - }, - "enrich": { - "example": true, + "is_truncated": { + "description": "IsTruncated reports that the source has more chunks than the limit\nreturned, so the reader knows the text they see is a prefix of the\ndocument and not the whole of it.", + "example": false, "type": "boolean" }, - "happened_at": { + "message": { + "description": "Human-readable result message.", + "example": "Success", "type": "string" }, - "is_markdown": { - "description": "Chunk the text on its markdown structure instead of as flat prose.", - "example": true, - "type": "boolean" - }, - "messages": { - "example": [ - { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "name": "general" - } - ], + "missing_chunk_ids": { + "description": "MissingChunkIDs are ids the caller asked for that have no chunk row in\neither store. An expected, documented state rather than an error: on\nstaging 61% of one Slack collection's sources had graph relations but no\nchunk_data row at all (see attributedSourceID), and the vector store is\nnot guaranteed to still hold a re-ingested source's older chunk ids.\nAlways empty for a source-scoped read, which discovers ids rather than\nbeing handed them.", "items": { - "$ref": "#/components/schemas/memories.ConversationTurn" + "type": "string" }, "type": "array", "uniqueItems": false }, - "text": { - "description": "Plain text. `content` is accepted as an alias.", - "type": "string" - }, - "title": { - "description": "Title of the item. The id is generated from it when `context_id` is omitted, so two items with the same text and no title collide.", - "example": "Project Phoenix Overview", - "type": "string" - }, - "user_name": { - "description": "Speaker for a text item. On a conversation, each turn's `name` wins, and this only fills in when no turn names anyone. Empty is stored as `User`.", - "type": "string" + "success": { + "description": "Whether the request succeeded.", + "example": true, + "type": "boolean" } }, "type": "object" }, - "memories.UnifiedIngestRequest": { + "search.GraphContext": { + "description": "GraphContext is omitted entirely when graph_context is disabled on the\nrequest (pointer + omitempty), so the response carries no graph slice\ninstead of an empty-but-present object.", "properties": { - "collection": { - "description": "Collection scope. Defaults to the default collection when omitted. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated).", - "example": "team_docs", - "type": "string" + "chunk_id_to_group_ids": { + "additionalProperties": { + "items": { + "type": "string" + }, + "type": "array" + }, + "description": "Mapping from chunk ID to the relation group IDs it participates in.", + "example": { + "HydraEmbeddings123_0": [ + "grp_1234" + ] + }, + "type": "object" }, - "contexts": { + "chunk_relations": { + "description": "Scored relation paths relevant to the query, grouped by chunk.", "example": [ { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "conversation": [ - { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "name": "general" - } + "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", + "group_id": "grp_1234", + "relevancy_score": 0.87, + "source_chunk_ids": [ + "HydraEmbeddings123_0", + "HydraEmbeddings123_1" ], - "enrich": true, - "is_markdown": true, - "messages": [ + "triplets": [ { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "name": "general" + "relation": { + "confidence": 0.92, + "predicate": "works_at" + }, + "source": { + "entity_id": "entity_1a2b", + "name": "Ada", + "type": "person" + }, + "target": { + "entity_id": "entity_3c4d", + "name": "Acme Corp", + "type": "organization" + } } - ], - "title": "Project Phoenix Overview" + ] } ], "items": { - "$ref": "#/components/schemas/memories.IngestItem" + "$ref": "#/components/schemas/search.ScoredPathResponse" }, "type": "array", "uniqueItems": false }, - "database": { - "description": "Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated).", - "example": "acme_corp", - "type": "string" - }, - "items": { - "description": "Items to ingest. `contexts` is accepted as an alias.", + "query_paths": { + "description": "Scored relation paths ranked by relevance to the query.", "example": [ { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "conversation": [ - { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "name": "general" - } - ], - "enrich": true, - "is_markdown": true, - "messages": [ - { - "content": "# Q4 Report\n\nRevenue grew 23% quarter over quarter.", - "name": "general" - } + "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", + "group_id": "grp_1234", + "relevancy_score": 0.87, + "source_chunk_ids": [ + "HydraEmbeddings123_0", + "HydraEmbeddings123_1" ], - "title": "Project Phoenix Overview" + "triplets": [ + { + "relation": { + "confidence": 0.92, + "predicate": "works_at" + }, + "source": { + "entity_id": "entity_1a2b", + "name": "Ada", + "type": "person" + }, + "target": { + "entity_id": "entity_3c4d", + "name": "Acme Corp", + "type": "organization" + } + } + ] } ], "items": { - "$ref": "#/components/schemas/memories.IngestItem" + "$ref": "#/components/schemas/search.ScoredPathResponse" }, "type": "array", "uniqueItems": false - }, - "upsert": { - "example": "true", - "type": "boolean" } }, "type": "object" }, - "search.AliasExpansionNote": { - "properties": { - "alias": { - "type": "string" + "search.MetadataFilters": { + "additionalProperties": {}, + "description": "Filters results by source metadata. Top-level keys target tenant metadata (for example department, priority, active, or tags). Nested additional_metadata keys target document metadata. Separate keys are ANDed. A scalar value is an exact match; an array means match ANY one of the listed values (OR) - there is no ALL/AND operator within a single key. Arrays are supported on VARCHAR fields only: an array passed for a declared field of any other type is rejected with 400 VALIDATION_ERROR. Size limits: each list may hold at most 500 values, and the whole metadata_filters object is capped at 64 KiB measured on its compact JSON encoding in UTF-8 bytes (keys and punctuation count). Exceeding either returns 400 naming the offending key or the actual byte count.", + "example": { + "active": true, + "additional_metadata": { + "author": "ada" }, - "canonical": { - "type": "string" - } + "department": "finance", + "priority": 7, + "tags": [ + "alpha", + "beta" + ] }, "type": "object" }, - "search.AppSearchFusionDiagnostics": { - "description": "Diagnostics for how connector (app) results were blended into the chunks. Present only when `query_apps` was on, the request was not access-scoped, and blended chunks remained.", + "search.Operator": { + "enum": [ + "or", + "and", + "phrase" + ], + "type": "string", + "x-enum-varnames": [ + "OperatorOr", + "OperatorAnd", + "OperatorPhrase" + ] + }, + "search.PathTriplet": { "properties": { - "app_recipes": { - "additionalProperties": { - "type": "string" + "relation": { + "additionalProperties": {}, + "description": "Relation properties including predicate and confidence score.", + "example": { + "confidence": 0.92, + "predicate": "works_at" }, - "description": "Map of each returned `chunk_uuid` found by connector search to the search strategy that found it.", "type": "object" }, - "chunk_origins": { - "additionalProperties": { - "type": "string" + "source": { + "additionalProperties": {}, + "description": "Source entity of the relationship.", + "example": { + "entity_id": "entity_1a2b", + "name": "Ada", + "type": "person" }, - "description": "Map of each returned `chunk_uuid` to where it came from: `normal` (regular search), `both`, `exact_id` (an exact identifier match that was promoted) or `app_tail` (a connector result appended at the end).", "type": "object" }, - "stats": { - "$ref": "#/components/schemas/search.AppSearchFusionStats", - "description": "Counts for the first or only blending pass, taken before final filtering. They are not final response counts, and not totals across alias alternatives. The whole diagnostic block is omitted for ACL-scoped requests.", - "example": { - "app_chunks": 1, - "app_has_exact_ids": true, - "app_lane_empty_text": true, - "consensus": 1, - "exact_candidates": 1, - "exact_promoted": 1, - "limit": 1, - "normal_chunks": 1, - "normal_displaced": 1, - "tail_added": 1, - "tail_candidates": 1 - } - }, - "stats_by_pass": { - "description": "Counts for each blending pass when results combine several, in order (original before alternate). Counts may overlap and must not be treated as unique totals.", - "example": [ - { - "app_chunks": 1, - "app_has_exact_ids": true, - "app_lane_empty_text": true, - "consensus": 1, - "exact_candidates": 1, - "exact_promoted": 1, - "limit": 1, - "normal_chunks": 1, - "normal_displaced": 1, - "tail_added": 1, - "tail_candidates": 1 - } - ], - "items": { - "$ref": "#/components/schemas/search.AppSearchFusionStats" + "target": { + "additionalProperties": {}, + "description": "Target entity of the relationship.", + "example": { + "entity_id": "entity_3c4d", + "name": "Acme Corp", + "type": "organization" }, - "type": "array", - "uniqueItems": false + "type": "object" } }, "type": "object" }, - "search.AppSearchFusionStats": { - "description": "Counts for the first or only blending pass, taken before final filtering. Not final counts, and not a sum across alias alternatives.", + "search.QueryBy": { + "enum": [ + "hybrid", + "text" + ], + "type": "string", + "x-enum-varnames": [ + "QueryByHybrid", + "QueryByText" + ] + }, + "search.QueryRequest": { "properties": { - "app_chunks": { - "description": "Chunks the connector search returned.", - "example": 1, - "type": "integer" - }, - "app_has_exact_ids": { - "description": "Whether the connector search matched on exact identifiers.", - "example": true, - "type": "boolean" - }, - "app_lane_empty_text": { - "description": "True when the connector search returned no chunks.", - "example": true, - "type": "boolean" - }, - "consensus": { - "description": "Connector chunks the regular search had already found. They keep their regular-search position.", - "example": 1, - "type": "integer" - }, - "exact_candidates": { - "description": "Connector chunks matched by exact identifier.", - "example": 1, - "type": "integer" - }, - "exact_promoted": { - "description": "Exact-identifier connector chunks placed above the regular-search results.", - "example": 1, - "type": "integer" - }, - "limit": { - "description": "Chunk limit applied to the blended results.", - "example": 1, - "type": "integer" + "acl": { + "description": "ACL scopes retrieval to documents the given principals may access\n(PRO-1684 document ACLs): a document matches when its stored ACL is\nempty (unrestricted, pre-RBAC content and connectors without permission\nsupport), contains __public__, or intersects these principals. Entries\nare bare emails or prefixed principals (user_email:/group:/domain:).\nOmitted, empty, or [\"*\"] disables ACL filtering entirely, today's\nbehavior. Like IDs, the resulting clause survives the metadata\nzero-result retry. An entry that is not a known principal fails CLOSED:\nit matches only public and unrestricted documents, never restricted.", + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": false }, - "normal_chunks": { - "description": "Chunks the regular search returned.", - "example": 1, - "type": "integer" + "additional_context": { + "description": "Optional context string prepended to the query to improve retrieval relevance.", + "example": "The user is a senior engineer onboarding to the platform.", + "type": "string" }, - "normal_displaced": { - "description": "Regular-search chunks pushed past the result limit by promoted and appended connector chunks.", - "example": 1, - "type": "integer" + "alpha": { + "description": "Weighting balance between dense and sparse retrieval in hybrid mode. `\"auto\"` lets HydraDB choose; a number from 0 (full BM25) to 1 (full dense) sets it explicitly." }, - "tail_added": { - "description": "Connector-only chunks appended at the end of the results.", - "example": 1, - "type": "integer" + "collection": { + "description": "Collection scope. Defaults to the default collection when omitted. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated).", + "example": "team_docs", + "type": "string" }, - "tail_candidates": { - "description": "Connector-only chunks eligible to be appended.", - "example": 1, - "type": "integer" - } - }, - "type": "object" - }, - "search.ChunkInspectResult": { - "properties": { - "chunks": { + "collections": { + "description": "Preferred /query scope selector. Send either a list of collection IDs for equal normalized weighting, or an object mapping collection ID to a positive relative ranking weight with at most one decimal place. Do not send together with the deprecated sub_tenant_ids or sub_tenant_id.", "example": [ + "team_docs", + "engineering" + ], + "oneOf": [ { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" + "example": [ + "finance", + "legal" ], - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 + "items": { + "type": "string" }, - "relevancy_score": 0.87, - "source_id": "HydraDoc1234", - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" + "maxItems": 100, + "minItems": 1, + "type": "array" + }, + { + "additionalProperties": { + "exclusiveMinimum": 0, + "multipleOf": 0.1, + "type": "number" + }, + "example": { + "finance": 1.5, + "legal": 0.8 + }, + "maxProperties": 100, + "minProperties": 1, + "type": "object" } ], - "items": { - "$ref": "#/components/schemas/search.VectorStoreChunk" - }, - "type": "array", - "uniqueItems": false - }, - "is_truncated": { - "description": "True when the source has more chunks than were returned, so the text is only the start of the document.", - "example": false, - "type": "boolean" + "x-preferred": true }, - "message": { - "description": "Human-readable result message.", - "example": "Success", + "database": { + "description": "Database is the canonical v2 name for the tenant scope. TenantID is its\ndeprecated alias and remains fully accepted. The TenantAliases middleware\nreconciles the two before binding, so TenantID is always populated and the\nhandler reads it; Database/Collection are carried only for docs/OpenAPI.", + "example": "acme_corp", "type": "string" }, - "missing_chunk_ids": { - "description": "Requested chunk ids that were not found. This can happen normally, for example after a source is re-ingested. Always empty when reading by source.", - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "success": { - "description": "Whether the request succeeded.", + "graph_context": { + "description": "Whether to include graph context in the response. Defaults to true for /query when omitted.", "example": true, "type": "boolean" - } - }, - "type": "object" - }, - "search.EntityProfileView": { - "properties": { - "compiled_at": { - "type": "string" }, - "entity_id": { - "description": "Unique identifier for this entity in the graph.", - "example": "entity_1a2b", - "type": "string" + "graph_vector_prune": { + "description": "GraphVectorPrune switches the graph-connected-chunks lane from \"fetch\ngraph-selected chunks and let the fusion reranker sort them out\" to \"fetch\na wider graph-selected candidate pool, then rank that pool by Milvus vector\nsimilarity, fully replacing the final chunk list.\" Works in either fast or\nthinking mode. Default false preserves existing behavior. Also gated\nserver-side by a repo-level config flag (SearchService's\ngraphVectorPruneEnabled) — if that flag is off, this is forced to false\nregardless of what the request sets, so a deployment can disable the\nmechanism without any client-side change.", + "example": true, + "type": "boolean" }, - "entity_type": { - "type": "string" + "graph_vector_prune_spacy_entities": { + "description": "GraphVectorPruneSpacyEntities: when GraphVectorPrune is also set, swaps the\ngraph lane's entity-extraction source from the default LLM-based extractor\nto a local spaCy subprocess (faster, no network round trip, but a\nnarrower/mismatched entity vocabulary versus the graph's own LLM-extracted\nnode names). No-op if GraphVectorPrune is false (including when forced\nfalse by the server-level flag) or no spaCy extractor was configured at\nstartup.", + "example": true, + "type": "boolean" }, - "entries": { + "ids": { + "description": "IDs optionally scopes retrieval to specific source ids. The v2 wire field is\n`ids` (matching /context/list); empty means search the whole corpus. Applied\nas a Milvus `source_id in [...]` pre-filter that is preserved across the\nmetadata zero-result retry, so a source-scoped search that matches nothing\nreturns nothing rather than silently widening to the whole corpus.", "example": [ - { - "confidence": 0.92 - } + "HydraDoc1234", + "HydraDoc4567" ], "items": { - "$ref": "#/components/schemas/search.ProfileEntry" + "type": "string" }, "type": "array", "uniqueItems": false }, - "headline": { - "type": "string" + "max_results": { + "description": "Maximum number of chunks to return.", + "example": 10, + "type": "integer" }, - "name": { - "description": "Human-readable label for this resource.", - "example": "general", - "type": "string" + "metadata_filters": { + "$ref": "#/components/schemas/search.MetadataFilters" }, - "pending_importance": { - "example": 1, + "mode": { + "$ref": "#/components/schemas/search.RecallMode", + "example": "thinking" + }, + "num_related_chunks": { + "description": "Number of adjacent chunks to pull alongside each matched chunk for additional context.", + "example": 3, "type": "integer" }, - "perspective": { - "type": "string" + "operator": { + "$ref": "#/components/schemas/search.Operator", + "example": "and" }, - "subject": { + "query": { + "description": "Natural-language search query.", + "example": "Which mode does the user prefer?", "type": "string" }, - "summary": { - "type": "string" + "query_apps": { + "description": "Whether to include app-aware knowledge retrieval. Applies to knowledge hybrid queries.", + "example": true, + "type": "boolean" }, - "summary_cites": { - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false + "query_by": { + "$ref": "#/components/schemas/search.QueryBy", + "description": "Retrieval method to use for the query.", + "example": "hybrid" }, - "unknown": { - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false + "query_forceful_relations": { + "description": "Whether to force relation expansion for graph-aware query retrieval. Defaults to true when omitted.", + "example": true, + "type": "boolean" }, - "version": { - "example": 1, - "type": "integer" - } - }, - "type": "object" - }, - "search.ForcefulRelationEntry": { - "properties": { - "chunk": { - "$ref": "#/components/schemas/search.V2Chunk", - "example": { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "collection": "team_docs", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" - ], - "id": "HydraDoc1234", - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 + "recency_bias": { + "description": "Recency boost applied to ranking. 0 disables it; higher values favour more recent sources.", + "example": 0.2, + "type": "number" + }, + "sub_tenant_id": { + "deprecated": true, + "description": "Deprecated for /query (since 2.0.1). Use collection for a single scope or collections for multiple. Backwards-compatible and will be removed in a future version. Do not send together with a multi-scope selector.", + "example": "sub_tenant_4567", + "type": "string", + "x-deprecated-since": "2.0.1" + }, + "sub_tenant_ids": { + "deprecated": true, + "description": "Deprecated for /query (since 2.0.1). Use collections instead; it accepts the same list or weighted-object shape. Backwards-compatible and will be removed in a future version. Do not send together with collections.", + "example": [ + "sub_tenant_4567", + "sub_tenant_8901" + ], + "oneOf": [ + { + "example": [ + "finance", + "legal" + ], + "items": { + "type": "string" + }, + "maxItems": 100, + "minItems": 1, + "type": "array" }, - "relevancy_score": 0.87, - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" + { + "additionalProperties": { + "exclusiveMinimum": 0, + "multipleOf": 0.1, + "type": "number" + }, + "example": { + "finance": 1.5, + "legal": 0.8 + }, + "maxProperties": 100, + "minProperties": 1, + "type": "object" + } + ], + "x-deprecated": "true", + "x-deprecated-since": "2.0.1" + }, + "temporal_intent": { + "$ref": "#/components/schemas/search.TemporalIntentOverride", + "example": { + "duration_to_now": true, + "mode": "thinking" } }, - "via": { - "$ref": "#/components/schemas/search.RelationVia" + "temporal_now": { + "description": "TemporalNow optionally anchors \"now\" for temporal reasoning (ISO-8601).\nCallers replaying past conversations (or backfilling) must supply it or\nto-now durations and recency windows resolve against the server's wall\nclock (LongMemEval measured 0 exact to-now durations from this alone).", + "type": "string" + }, + "temporal_reasoning": { + "description": "TemporalReasoning activates the temporal read path: the query is classified\ninto a temporal mode (current/as-of/range/upcoming...), matching edge-level\ntemporal facts are resolved from the edge_temporal store and ride back on\nthe response (temporal_facts / temporal_duration / temporal_filter).\nCONTRACT: chunk ranking is NEVER altered — ON returns the same chunks as\nOFF; the layer is additive payload + computed answers only (rank shaping\nmeasured net-negative on BEAM/LongMemEval/TEMPO; see temporal_filters.go).\nOptional; ON by default — pass temporal_reasoning:false to disable.\nResolved by GetTemporalReasoningOrDefault (ownership rule).", + "example": true, + "type": "boolean" + }, + "tenant_id": { + "deprecated": true, + "description": "deprecated: use database", + "example": "tenant_1234", + "type": "string", + "x-deprecated": "true" + }, + "type": { + "$ref": "#/components/schemas/search.SourceType", + "description": "Corpus to query: knowledge, memory, or all." } }, "type": "object" }, - "search.ForcefulRelationsBucket": { - "description": "Relations the caller declared at ingest, each with the edge it came from. Always present.", + "search.RecallMode": { + "enum": [ + "fast", + "thinking", + "auto" + ], + "type": "string", + "x-enum-varnames": [ + "RecallModeFast", + "RecallModeThinking", + "RecallModeAuto" + ] + }, + "search.ScoredPathResponse": { "properties": { - "declared": { + "combined_context": { + "description": "Merged text from all chunk passages in this relation path.", + "example": "Acme Corp deploys HydraDB in production for context retrieval.", + "type": "string" + }, + "group_id": { + "description": "Unique identifier for this relation group.", + "example": "grp_1234", + "type": "string" + }, + "relevancy_score": { + "description": "Relevance score for this item against the query.", + "example": 0.87, + "type": "number" + }, + "source_chunk_ids": { + "description": "IDs of the chunks that contribute to this relation path.", "example": [ - { - "chunk": { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "collection": "team_docs", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" - ], - "id": "HydraDoc1234", - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 - }, - "relevancy_score": 0.87, - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" - } - } + "HydraEmbeddings123_0", + "HydraEmbeddings123_1" ], "items": { - "$ref": "#/components/schemas/search.ForcefulRelationEntry" + "type": "string" }, "type": "array", "uniqueItems": false }, - "inferred": { + "triplets": { + "description": "Knowledge-graph triplets that make up this relation path.", "example": [ { - "chunk": { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "collection": "team_docs", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" - ], - "id": "HydraDoc1234", - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 - }, - "relevancy_score": 0.87, - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" + "relation": { + "confidence": 0.92, + "predicate": "works_at" + }, + "source": { + "entity_id": "entity_1a2b", + "name": "Ada", + "type": "person" + }, + "target": { + "entity_id": "entity_3c4d", + "name": "Acme Corp", + "type": "organization" } } ], "items": { - "$ref": "#/components/schemas/search.ForcefulRelationEntry" + "$ref": "#/components/schemas/search.PathTriplet" }, "type": "array", "uniqueItems": false @@ -5882,728 +5412,267 @@ }, "type": "object" }, - "search.GraphContext": { - "deprecated": true, - "description": "Graph context for the query. Omitted when `graph_context` is false.", + "search.SourceFact": { "properties": { - "chunk_id_to_group_ids": { - "additionalProperties": { - "items": { - "type": "string" - }, - "type": "array" - }, - "description": "Mapping from chunk ID to the relation group IDs it participates in.", - "example": { - "HydraEmbeddings123_0": [ - "grp_1234" - ] - }, - "type": "object" + "actor": { + "type": "string" }, - "chunk_relations": { - "description": "Scored relation paths relevant to the query, grouped by chunk.", - "example": [ - { - "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", - "group_id": "grp_1234", - "relevancy_score": 0.87, - "source_chunk_ids": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "triplets": [ - { - "relation": { - "confidence": 0.92, - "predicate": "works_at" - }, - "source": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "target": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - } - } - ] - } - ], - "items": { - "$ref": "#/components/schemas/search.ScoredPathResponse" - }, - "type": "array", - "uniqueItems": false - }, - "query_paths": { - "description": "Scored relation paths ranked by relevance to the query.", - "example": [ - { - "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", - "group_id": "grp_1234", - "relevancy_score": 0.87, - "source_chunk_ids": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "triplets": [ - { - "relation": { - "confidence": 0.92, - "predicate": "works_at" - }, - "source": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "target": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - } - } - ] - } - ], - "items": { - "$ref": "#/components/schemas/search.ScoredPathResponse" - }, - "type": "array", - "uniqueItems": false - } - }, - "type": "object", - "x-deprecated": "true" - }, - "search.GraphPath": { - "properties": { - "chunk_ids": { - "example": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "combined_context": { - "description": "Merged text from all chunk passages in this relation path.", - "example": "Acme Corp deploys HydraDB in production for context retrieval.", + "actor_role": { "type": "string" }, - "relevancy_score": { - "description": "Relevance score for this item against the query.", - "example": 0.87, - "type": "number" - }, - "triplets": { - "description": "Knowledge-graph triplets that make up this relation path.", - "example": [ - { - "relation": { - "confidence": 0.92, - "predicate": "works_at" - }, - "source": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "target": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - } - } - ], - "items": { - "$ref": "#/components/schemas/search.PathTriplet" - }, - "type": "array", - "uniqueItems": false - } - }, - "type": "object" - }, - "search.GraphPlane": { - "description": "Graph paths for the query as one ordered list: paths reached from the query first, then paths expanded from retrieved chunks, each with the chunk ids it supports. Present whenever `graph_context` is on.", - "properties": { - "paths": { - "example": [ - { - "chunk_ids": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", - "relevancy_score": 0.87, - "triplets": [ - { - "relation": { - "confidence": 0.92, - "predicate": "works_at" - }, - "source": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "target": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - } - } - ] - } - ], - "items": { - "$ref": "#/components/schemas/search.GraphPath" - }, - "type": "array", - "uniqueItems": false - } - }, - "type": "object" - }, - "search.MetadataFilters": { - "additionalProperties": {}, - "description": "DEPRECATED: use `attributes`, which is an operator language pushed into the vector search rather than bare equality applied after it. `metadata_filters` keeps working, and is still the only way to filter on per-context custom_attributes, which `attributes` does not cover yet. Filters results by context metadata. Top-level keys target tenant metadata (for example department, priority, active, or tags). Nested additional_metadata keys target document metadata. Separate keys are ANDed. Each top-level key accepts an operator object naming the comparison: {\"contains\": value} matches sources whose field holds that value (multi-value fields are stored comma-joined, so this matches one member); {\"contains_any\": [values]} matches sources holding ANY one of the listed values; {\"equals\": value} matches sources whose field is exactly that value. The bare forms remain supported and unchanged but are deprecated in favour of the operators, because the comparison they perform is inferred from the JSON shape rather than stated: a bare scalar behaves as equals, a bare array as contains_any, and a bare single-element array as contains. Operators apply to top-level keys only; inside additional_metadata use the bare scalar or array forms. An operator used inside additional_metadata is NOT rejected - it is read as an exact-match filter against a stored object, so on a normal field it matches nothing and the request returns 200 with an empty result rather than an error. A known operator given the wrong operand type, or several operators in one object, is rejected with 400 VALIDATION_ERROR rather than silently matching nothing. A MISSPELLED operator is not: {\"contian\": \"x\"} is indistinguishable from a filter for a stored object with that key, so it is left alone and matches nothing. An object whose keys are not operator names is likewise treated as an exact-match filter against a stored object, unchanged. RESERVED NAMES: contains, contains_any and equals are reserved as the keys of a top-level filter object, so an object built only from them is read as an operator and is no longer available for exact object matching -- {\"f\": {\"contains\": \"x\"}} is read as the operator, and an object whose keys are ALL operator names is rejected with 400. A caller matching such an object in a JSON-typed field must rename the nested key or the field. Mixing an operator name with any other key ({\"contains\": \"a\", \"other\": 1}) is unaffected and still exact-matches. There is no ALL/AND operator within a single key. contains, contains_any and arrays are supported on VARCHAR fields only: any of them passed for a declared field of another type is rejected with 400 VALIDATION_ERROR. equals works on every declared type, so {\"priority\": {\"equals\": 7}} is valid on an INT64 field. Size limits: each list may hold at most 500 values, and the whole metadata_filters object is capped at 64 KiB measured on its compact JSON encoding in UTF-8 bytes AFTER operator objects are reduced to their values, so {\"contains\": \"x\"} is measured as [\"x\"] and the operator keyword itself costs nothing. The cap bounds the cost of the resulting vector-store expression, which the operator spelling does not change. Field names and punctuation count. Exceeding either returns 400 naming the offending key or the actual byte count.", - "example": { - "active": true, - "additional_metadata": { - "author": "ada" - }, - "department": "finance", - "priority": 7, - "tags": [ - "alpha", - "beta" - ] - }, - "type": "object" - }, - "search.Operator": { - "enum": [ - "or", - "and", - "phrase" - ], - "type": "string", - "x-enum-varnames": [ - "OperatorOr", - "OperatorAnd", - "OperatorPhrase" - ] - }, - "search.PathTriplet": { - "properties": { - "relation": { - "additionalProperties": {}, - "description": "Relation properties including predicate and confidence score.", - "example": { - "confidence": 0.92, - "predicate": "works_at" - }, - "type": "object" - }, - "source": { - "additionalProperties": {}, - "description": "Source entity of the relationship.", - "example": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "type": "object" + "app_kind": { + "description": "App integration category, populated for connector-synced sources.", + "example": "slack", + "type": "string" }, - "target": { - "additionalProperties": {}, - "description": "Target entity of the relationship.", - "example": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - }, - "type": "object" - } - }, - "type": "object" - }, - "search.ProfileContext": { - "description": "Profile of the entity named by `profile_subject`. Omitted when no profile was requested.", - "properties": { - "entity_id": { - "description": "Unique identifier for this entity in the graph.", - "example": "entity_1a2b", + "chunk_id": { + "description": "Chunk that provides evidence for this relation.", + "example": "HydraEmbeddings123_0", "type": "string" }, - "entries": { - "example": [ - { - "confidence": 0.92 - } - ], - "items": { - "$ref": "#/components/schemas/search.ProfileEntry" - }, - "type": "array", - "uniqueItems": false + "connector": { + "type": "string" }, - "headline": { + "container": { "type": "string" }, - "name": { - "description": "Human-readable label for this resource.", - "example": "general", + "provider": { + "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", + "example": "slack", "type": "string" }, - "perspective": { + "relation": { "type": "string" }, - "subject": { + "relationship_id": { + "description": "Unique identifier for this relationship instance.", + "example": "rel_1234", "type": "string" }, - "summary": { + "source_id": { + "example": "HydraDoc1234", "type": "string" }, - "version": { + "synced_at": { "example": 1, "type": "integer" + }, + "thread_id": { + "type": "string" } }, "type": "object" }, - "search.ProfileEntry": { + "search.SourceFilterInfo": { + "description": "SourceFilter reports what the source layer did for this request.", "properties": { - "confidence": { - "description": "Confidence score, from 0 to 1.", - "example": 0.92, - "type": "number" - }, - "facet": { + "actor_scope": { "type": "string" }, - "since": { - "type": "string" - }, - "slot": { - "type": "string" - }, - "state": { - "description": "stated | observed | inferred | record", - "type": "string" - }, - "statement_keys": { - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "text": { - "type": "string" - } - }, - "type": "object" - }, - "search.ProfileFilterInfo": { - "properties": { "applied": { "example": true, "type": "boolean" }, - "degraded": { - "example": true, - "type": "boolean" - }, - "entity_id": { - "description": "Unique identifier for this entity in the graph.", - "example": "entity_1a2b", + "container_scope": { "type": "string" }, - "found": { + "degraded": { "example": true, "type": "boolean" }, - "selected_entries": { + "matched_facts": { "example": 1, "type": "integer" }, - "subject": { + "mode": { + "example": "thinking", "type": "string" }, - "version": { - "example": 1, - "type": "integer" + "provider": { + "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", + "example": "slack", + "type": "string" + }, + "thread_scope": { + "example": true, + "type": "boolean" + }, + "truncated": { + "example": true, + "type": "boolean" } }, "type": "object" }, - "search.QueryBy": { - "enum": [ - "hybrid", - "text" - ], - "type": "string", - "x-enum-varnames": [ - "QueryByHybrid", - "QueryByText" - ] - }, - "search.QueryRequest": { + "search.SourceInfo": { "properties": { - "acl": { - "description": "Principals to answer as: only documents they may access are retrieved. A document matches when it is unrestricted, public, or shares a principal with this list. Entries are bare emails or `user_email:`, `group:` or `domain:` principals. Omit, or send `[]` or `[\"*\"]`, for no access scoping. An unrecognised entry matches only public and unrestricted documents.", - "items": { - "type": "string" + "additional_metadata": { + "additionalProperties": {}, + "description": "Per-document free-form metadata.", + "example": { + "author": "ada", + "doc_version": 3 }, - "type": "array", - "uniqueItems": false + "type": "object" }, - "additional_context": { - "description": "Optional context string prepended to the query to improve retrieval relevance.", - "example": "The user is a senior engineer onboarding to the platform.", + "app_external_id": { + "description": "Provider-assigned identifier for this source (e.g. Slack channel ID).", + "example": "C0123456789", "type": "string" }, - "alpha": { - "description": "Weighting balance between dense and sparse retrieval in hybrid mode. `\"auto\"` lets HydraDB choose; a number from 0 (full BM25) to 1 (full dense) sets it explicitly." + "app_kind": { + "description": "App-source fields (populated when the source comes from an app integration).\nDefault null on the wire when absent.", + "example": "slack", + "type": "string" }, - "attributes": { - "additionalProperties": {}, - "description": "Filter on the database's attributes with operators: `$eq`, `$ne`, `$gt`, `$gte`, `$lt`, `$lte`, `$in`, `$nin`, `$and`, `$or`, `$not` and `$exists`. Applies to chunks, related context and graph paths alike. Combined with `metadata_filters` (AND) if both are sent.", - "type": "object" + "app_provider": { + "description": "Provider name for app-sourced items (e.g. `slack`, `github`).", + "example": "slack", + "type": "string" }, "collection": { - "description": "Collection scope. Defaults to the default collection when omitted. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated).", + "description": "Collection this source belongs to. Canonical name; mirrors the deprecated `sub_tenant_id` alias.", "example": "team_docs", "type": "string" }, - "collections": { - "description": "Preferred /query scope selector. Send either a list of collection IDs for equal normalized weighting, or an object mapping collection ID to a positive relative ranking weight with at most one decimal place. Do not send together with the deprecated sub_tenant_ids or sub_tenant_id.", - "example": [ - "team_docs", - "engineering" - ], - "oneOf": [ - { - "example": [ - "finance", - "legal" - ], - "items": { - "type": "string" - }, - "maxItems": 100, - "minItems": 1, - "type": "array" - }, - { - "additionalProperties": { - "exclusiveMinimum": 0, - "multipleOf": 0.1, - "type": "number" - }, - "example": { - "finance": 1.5, - "legal": 0.8 - }, - "maxProperties": 100, - "minProperties": 1, - "type": "object" - } - ], - "x-preferred": true - }, - "database": { - "description": "Database to query. `tenant_id` is accepted as a deprecated alias.", - "example": "acme_corp", + "description": { + "description": "Human-readable description of the source.", + "example": "Internal overview of the Project Phoenix rollout.", "type": "string" }, - "graph_context": { - "description": "Whether to include graph context in the response. Defaults to true for /query when omitted.", - "example": true, - "type": "boolean" - }, - "graph_vector_prune": { - "description": "Rank a wider pool of graph-connected chunks by vector similarity instead of the default reranking. Default false. Has no effect where the feature is disabled.", - "example": true, - "type": "boolean" - }, - "graph_vector_prune_spacy_entities": { - "description": "With `graph_vector_prune`, extract entities with a faster local model instead of the default one. Has no effect when `graph_vector_prune` is off.", - "example": true, - "type": "boolean" + "id": { + "description": "Unique identifier for this resource.", + "example": "HydraDoc1234", + "type": "string" }, - "ids": { - "description": "Restrict retrieval to these source ids. Empty searches the whole database. A scoped search that matches nothing returns nothing.", - "example": [ - "HydraDoc1234", - "HydraDoc4567" - ], - "items": { - "type": "string" + "metadata": { + "additionalProperties": {}, + "description": "Pydantic aliases (see VectorStoreChunk). Source metadata defaults to {} on\nthe wire (Python default_factory=dict), unlike chunk metadata which is null.", + "example": { + "department": "finance", + "priority": 7 }, - "type": "array", - "uniqueItems": false - }, - "max_results": { - "description": "Maximum number of chunks to return.", - "example": 10, - "type": "integer" + "type": "object" }, - "metadata_filters": { - "$ref": "#/components/schemas/search.MetadataFilters", + "sub_tenant_id": { "deprecated": true, - "x-deprecated": true - }, - "mode": { - "$ref": "#/components/schemas/search.RecallMode", - "example": "thinking" - }, - "num_related_chunks": { - "description": "Number of adjacent chunks to pull alongside each matched chunk for additional context.", - "example": 3, - "type": "integer" - }, - "operator": { - "$ref": "#/components/schemas/search.Operator", - "example": "and" + "description": "deprecated: use collection", + "example": "sub_tenant_4567", + "type": "string", + "x-deprecated": "true" }, - "profile_entity_type": { - "description": "Entity type of `profile_subject`, with `profile_namespace` refining it. Defaults to `PERSON` in the `users` namespace.", + "timestamp": { + "description": "RFC3339 timestamp associated with this item.", + "example": "2026-07-02T10:00:00Z", "type": "string" }, - "profile_namespace": { + "title": { + "description": "Title or name of the source.", + "example": "Project Phoenix Overview", "type": "string" }, - "profile_subject": { - "description": "Entity whose profile to return alongside the results, as `profile_context` and `profile_filter`. Does not change ranking. May not be available on every deployment.", + "type": { + "description": "Source content category (e.g. `knowledge`, `memory`).", + "example": "knowledge", "type": "string" }, - "query": { - "description": "Natural-language search query.", - "example": "Which mode does the user prefer?", + "url": { + "description": "URL to the original source, if available.", + "example": "https://docs.hydradb.com/phoenix", "type": "string" - }, - "query_apps": { - "description": "Whether to include app-aware knowledge retrieval. Applies to knowledge hybrid queries. Defaults to true when omitted; pass false to search files only.", + } + }, + "type": "object" + }, + "search.SourceType": { + "description": "Source is the wire field `type` (Python QueryRequest.source has alias=\"type\").\nSourceLegacy accepts the pre-rename `source` key (Python populate_by_name=True\nkeeps the field name valid on input); resolveSourceAlias folds it into Source.", + "enum": [ + "knowledge", + "memory", + "all" + ], + "type": "string", + "x-enum-varnames": [ + "SourceKnowledge", + "SourceMemory", + "SourceAll" + ] + }, + "search.TemporalDuration": { + "description": "TemporalDuration is the computed event-duration answer, when resolved.", + "properties": { + "approximate": { + "description": "Approximate is set when either endpoint's granularity is coarser than a\nday (month/year brackets) — the day count is then a floor-to-floor\nestimate, not an exact span; consumers should not present it as exact.", "example": true, "type": "boolean" }, - "query_by": { - "$ref": "#/components/schemas/search.QueryBy", - "description": "Retrieval method to use for the query.", - "example": "hybrid" + "days": { + "example": 1, + "type": "integer" }, - "query_forceful_relations": { - "description": "Whether to force relation expansion for graph-aware query retrieval. Defaults to true when omitted.", - "example": true, - "type": "boolean" + "from": { + "$ref": "#/components/schemas/search.TemporalFact", + "example": { + "chunk_id": "HydraEmbeddings123_0", + "event_end": 1, + "event_start": 1, + "relationship_id": "rel_1234", + "source_id": "HydraDoc1234", + "status": "completed" + } }, - "recency_bias": { - "description": "Recency boost applied to ranking (0.0-1.0). Omit it to get the always-on default baseline of 0.40 (a bounded \u003c=40% swing on normalized relevance — it reorders within a relevance gap of up to 0.40 but never buries a more strongly relevant result); send 0 to disable recency entirely; higher values favour more recent sources more strongly.", - "example": 0.2, - "type": "number" + "from_date": { + "type": "string" }, - "sub_tenant_id": { - "deprecated": true, - "description": "Deprecated for /query (since 2.0.1). Use collection for a single scope or collections for multiple. Backwards-compatible and will be removed in a future version. Do not send together with a multi-scope selector.", - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated-since": "2.0.1" + "pairing_confidence": { + "description": "PairingConfidence is the normalized pair-scorer margin (0..1); low values\nmean the endpoints were weakly anchored to the question. Durations whose\nendpoints share no entity token with the question are suppressed\nentirely (P4: a wrong confident day count misleads answerers).", + "example": 0.5, + "type": "number" }, - "sub_tenant_ids": { - "deprecated": true, - "description": "Deprecated for /query (since 2.0.1). Use collections instead; it accepts the same list or weighted-object shape. Backwards-compatible and will be removed in a future version. Do not send together with collections.", - "example": [ - "sub_tenant_4567", - "sub_tenant_8901" - ], - "oneOf": [ - { - "example": [ - "finance", - "legal" - ], - "items": { - "type": "string" - }, - "maxItems": 100, - "minItems": 1, - "type": "array" - }, - { - "additionalProperties": { - "exclusiveMinimum": 0, - "multipleOf": 0.1, - "type": "number" - }, - "example": { - "finance": 1.5, - "legal": 0.8 - }, - "maxProperties": 100, - "minProperties": 1, - "type": "object" - } - ], - "x-deprecated": "true", - "x-deprecated-since": "2.0.1" - }, - "temporal_intent": { - "$ref": "#/components/schemas/search.TemporalIntentOverride", + "to": { + "$ref": "#/components/schemas/search.TemporalFact", "example": { - "duration_to_now": true, - "mode": "thinking" + "chunk_id": "HydraEmbeddings123_0", + "event_end": 1, + "event_start": 1, + "relationship_id": "rel_1234", + "source_id": "HydraDoc1234", + "status": "completed" } }, - "temporal_now": { - "description": "ISO 8601 time to treat as now for temporal reasoning. Set it when replaying past conversations, otherwise durations and recency resolve against the current time.", - "type": "string" - }, - "temporal_reasoning": { - "description": "Resolve time-based questions (current, as of, ranges, upcoming) and return matched facts in `temporal_facts`, `temporal_duration` and `temporal_filter`. Never changes which chunks are returned. On by default; send false to turn it off.", - "example": true, - "type": "boolean" - }, - "tenant_id": { - "deprecated": true, - "description": "deprecated: use database", - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - }, - "titles": { - "description": "Optional exact document-title filter. Values are matched case-insensitively and ORed, resolved to source IDs, then the normal query pipeline runs within that source scope. When ids is also supplied, the two filters are intersected.", - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "type": { - "$ref": "#/components/schemas/search.SourceType", - "description": "Corpus to query. Split database: knowledge (the default), memory, or all (both, merged). Unified database: omit it, or send unified (all is accepted and means the same); knowledge and memory are rejected there." - } - }, - "type": "object" - }, - "search.RecallMode": { - "enum": [ - "fast", - "thinking", - "auto" - ], - "type": "string", - "x-enum-varnames": [ - "RecallModeFast", - "RecallModeThinking", - "RecallModeAuto" - ] - }, - "search.RelationVia": { - "properties": { - "from": { - "type": "string" - }, - "to": { - "type": "string" - } - }, - "type": "object" - }, - "search.ScoredPathResponse": { - "properties": { - "combined_context": { - "description": "Merged text from all chunk passages in this relation path.", - "example": "Acme Corp deploys HydraDB in production for context retrieval.", - "type": "string" - }, - "group_id": { - "description": "Unique identifier for this relation group.", - "example": "grp_1234", + "to_date": { "type": "string" - }, - "relevancy_score": { - "description": "Relevance score for this item against the query.", - "example": 0.87, - "type": "number" - }, - "source_chunk_ids": { - "description": "IDs of the chunks that contribute to this relation path.", - "example": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "triplets": { - "description": "Knowledge-graph triplets that make up this relation path.", - "example": [ - { - "relation": { - "confidence": 0.92, - "predicate": "works_at" - }, - "source": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "target": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - } - } - ], - "items": { - "$ref": "#/components/schemas/search.PathTriplet" - }, - "type": "array", - "uniqueItems": false } }, "type": "object" }, - "search.SourceFact": { + "search.TemporalFact": { "properties": { - "actor": { + "chunk_id": { + "description": "Chunk that provides evidence for this relation.", + "example": "HydraEmbeddings123_0", "type": "string" }, - "actor_role": { + "date_precision": { + "description": "DatePrecision is the resolution of the resolved dates: \"day\", \"month\",\n\"year\" (coarser-than-day dates are floored to bracket starts).", "type": "string" }, - "app_kind": { - "description": "App integration category, populated for connector-synced sources.", - "example": "slack", - "type": "string" + "event_end": { + "example": 1, + "type": "integer" }, - "chunk_id": { - "description": "Chunk that provides evidence for this relation.", - "example": "HydraEmbeddings123_0", - "type": "string" + "event_start": { + "example": 1, + "type": "integer" }, - "connector": { + "evidence_phrase": { + "description": "EvidencePhrase is the verbatim source phrase the dates were resolved\nfrom (e.g. \"today\", \"two weeks ago\").", "type": "string" }, - "container": { + "fact_type": { "type": "string" }, - "provider": { - "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", - "example": "slack", + "object": { "type": "string" }, "relation": { @@ -6618,30 +5687,31 @@ "example": "HydraDoc1234", "type": "string" }, - "synced_at": { - "example": 1, - "type": "integer" + "status": { + "description": "Current lifecycle or processing state.", + "example": "completed", + "type": "string" }, - "thread_id": { + "subject": { "type": "string" } }, "type": "object" }, - "search.SourceFilterInfo": { - "description": "What source reasoning did for this request.", + "search.TemporalFilterInfo": { + "description": "TemporalFilter reports what the temporal layer did for this request.", "properties": { - "actor_scope": { - "type": "string" - }, "applied": { + "description": "Applied is true when the temporal layer engaged for a classified temporal\nquery — including when it matched zero dated facts; MatchedFacts carries the\nactual count. It is false only when the fact lookup degraded (Degraded).", "example": true, "type": "boolean" }, - "container_scope": { - "type": "string" + "chunk_scope": { + "example": 1, + "type": "integer" }, "degraded": { + "description": "Degraded is true when the fact lookup FAILED (as opposed to matching\nnothing) — callers must not read an empty payload as \"no temporal facts\nexist\" when this is set.", "example": true, "type": "boolean" }, @@ -6653,312 +5723,75 @@ "example": "thinking", "type": "string" }, - "provider": { - "description": "External provider being synced (e.g. `slack`, `github`, `linear`, `notion`, `gmail`).", - "example": "slack", + "promoted": { + "example": 1, + "type": "integer" + }, + "scope": { + "description": "Scope reports how the chunk scope was applied: \"soft\" (bounded ranking\npromotion) or \"\" (no scope). Hard scoping was removed after TEMPO.", "type": "string" }, - "thread_scope": { + "truncated": { "example": true, "type": "boolean" + } + }, + "type": "object" + }, + "search.TemporalIntentOverride": { + "description": "TemporalIntent (EXPERIMENTAL) lets the caller supply the classification\n(mode/window/phrases) directly, bypassing the regex classifier — for\nagents whose own LLM already understands the query, and for non-English\nqueries. Invalid overrides fall back to the classifier.", + "properties": { + "cutoff": { + "type": "string" }, - "truncated": { + "duration_to_now": { "example": true, "type": "boolean" + }, + "event_phrases": { + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": false + }, + "mode": { + "example": "thinking", + "type": "string" + }, + "window_end": { + "type": "string" + }, + "window_start": { + "type": "string" } }, "type": "object" }, - "search.SourceInfo": { + "search.V2Chunk": { "properties": { "additional_metadata": { "additionalProperties": {}, - "description": "Per-document free-form metadata.", + "description": "Pydantic aliases (see VectorStoreChunk): document_metadata→additional_metadata,\ntenant_metadata→metadata. FastAPI serializes by_alias, so the wire uses the aliases.", "example": { "author": "ada", "doc_version": 3 }, "type": "object" }, - "app_external_id": { - "description": "The source's ID in the connected app. Null when the source did not come from a connector.", - "example": "C0123456789", + "chunk_content": { + "description": "Text content of this chunk.", + "example": "HydraDB supports hybrid retrieval across knowledge and memories.", "type": "string" }, - "app_kind": { - "description": "Kind of object in the connected app (for example a message or an issue). Null when the source did not come from a connector.", - "example": "slack", + "chunk_uuid": { + "description": "Unique identifier for this individual chunk.", + "example": "a1b2c3d4-e5f6-7890-1234-567890abcdef", "type": "string" }, - "app_provider": { - "description": "Connector provider the source came from (for example `slack` or `github`). Null when the source did not come from a connector.", - "example": "slack", - "type": "string" - }, - "collection": { - "description": "Collection this source belongs to. Canonical name; mirrors the deprecated `sub_tenant_id` alias.", - "example": "team_docs", - "type": "string" - }, - "description": { - "description": "Human-readable description of the source.", - "example": "Internal overview of the Project Phoenix rollout.", - "type": "string" - }, - "id": { - "description": "Unique identifier for this resource.", - "example": "HydraDoc1234", - "type": "string" - }, - "metadata": { - "additionalProperties": {}, - "description": "The source's attributes. An empty object when it has none.", - "example": { - "department": "finance", - "priority": 7 - }, - "type": "object" - }, - "sub_tenant_id": { - "deprecated": true, - "description": "deprecated: use collection", - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated": "true" - }, - "timestamp": { - "description": "RFC3339 timestamp associated with this item.", - "example": "2026-07-02T10:00:00Z", - "type": "string" - }, - "title": { - "description": "Title or name of the source.", - "example": "Project Phoenix Overview", - "type": "string" - }, - "type": { - "description": "Source content category (e.g. `knowledge`, `memory`).", - "example": "knowledge", - "type": "string" - }, - "url": { - "description": "URL to the original source, if available.", - "example": "https://docs.hydradb.com/phoenix", - "type": "string" - } - }, - "type": "object" - }, - "search.SourceType": { - "description": "Which corpus to read, sent as `type`. `source` is accepted as a legacy name for the same field.", - "enum": [ - "knowledge", - "memory", - "all", - "unified" - ], - "type": "string", - "x-enum-varnames": [ - "SourceKnowledge", - "SourceMemory", - "SourceAll", - "SourceUnified" - ] - }, - "search.TemporalDuration": { - "description": "Duration between two events, when one could be resolved from the question.", - "properties": { - "approximate": { - "description": "True when either date is only known to the month or year, so the day count is an estimate. Do not present it as exact.", - "example": true, - "type": "boolean" - }, - "days": { - "example": 1, - "type": "integer" - }, - "from": { - "$ref": "#/components/schemas/search.TemporalFact", - "example": { - "chunk_id": "HydraEmbeddings123_0", - "event_end": 1, - "event_start": 1, - "relationship_id": "rel_1234", - "source_id": "HydraDoc1234", - "status": "completed" - } - }, - "from_date": { - "type": "string" - }, - "pairing_confidence": { - "description": "Confidence, from 0 to 1, that the two events match the question. Durations with no link to the question are left out.", - "example": 0.5, - "type": "number" - }, - "to": { - "$ref": "#/components/schemas/search.TemporalFact", - "example": { - "chunk_id": "HydraEmbeddings123_0", - "event_end": 1, - "event_start": 1, - "relationship_id": "rel_1234", - "source_id": "HydraDoc1234", - "status": "completed" - } - }, - "to_date": { - "type": "string" - } - }, - "type": "object" - }, - "search.TemporalFact": { - "properties": { - "chunk_id": { - "description": "Chunk that provides evidence for this relation.", - "example": "HydraEmbeddings123_0", - "type": "string" - }, - "date_precision": { - "description": "Precision of the resolved dates: `day`, `month` or `year`. Coarser dates start at the beginning of their period.", - "type": "string" - }, - "event_end": { - "example": 1, - "type": "integer" - }, - "event_start": { - "example": 1, - "type": "integer" - }, - "evidence_phrase": { - "description": "Phrase in the source the dates were resolved from (for example \"two weeks ago\").", - "type": "string" - }, - "fact_type": { - "type": "string" - }, - "object": { - "type": "string" - }, - "relation": { - "type": "string" - }, - "relationship_id": { - "description": "Unique identifier for this relationship instance.", - "example": "rel_1234", - "type": "string" - }, - "source_id": { - "example": "HydraDoc1234", - "type": "string" - }, - "status": { - "description": "Current lifecycle or processing state.", - "example": "completed", - "type": "string" - }, - "subject": { - "type": "string" - } - }, - "type": "object" - }, - "search.TemporalFilterInfo": { - "description": "What temporal reasoning did for this request.", - "properties": { - "applied": { - "description": "True when temporal reasoning ran for a time-based query, even if no dated facts matched (`matched_facts` has the count). False only when the lookup failed.", - "example": true, - "type": "boolean" - }, - "chunk_scope": { - "example": 1, - "type": "integer" - }, - "degraded": { - "description": "True when the temporal fact lookup failed, as opposed to matching nothing. An empty result does not mean no temporal facts exist when this is set.", - "example": true, - "type": "boolean" - }, - "matched_facts": { - "example": 1, - "type": "integer" - }, - "mode": { - "example": "thinking", - "type": "string" - }, - "promoted": { - "example": 1, - "type": "integer" - }, - "scope": { - "description": "How temporal matches affected ranking: `soft` (matching chunks are gently promoted) or empty (not applied).", - "type": "string" - }, - "truncated": { - "example": true, - "type": "boolean" - } - }, - "type": "object" - }, - "search.TemporalIntentOverride": { - "description": "Experimental. Supply the temporal classification (mode, window, phrases) yourself instead of letting HydraDB classify the query, for example for non-English queries. An invalid override falls back to automatic classification.", - "properties": { - "cutoff": { - "type": "string" - }, - "duration_to_now": { - "example": true, - "type": "boolean" - }, - "event_phrases": { - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "mode": { - "example": "thinking", - "type": "string" - }, - "window_end": { - "type": "string" - }, - "window_start": { - "type": "string" - } - }, - "type": "object" - }, - "search.V2Chunk": { - "properties": { - "additional_metadata": { - "additionalProperties": {}, - "description": "The chunk's custom attributes.", - "example": { - "author": "ada", - "doc_version": 3 - }, - "type": "object" - }, - "chunk_content": { - "description": "Text content of this chunk.", - "example": "HydraDB supports hybrid retrieval across knowledge and memories.", - "type": "string" - }, - "chunk_uuid": { - "description": "Unique identifier for this individual chunk.", - "example": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "type": "string" - }, - "collection": { - "description": "Collection this chunk belongs to. Canonical name; mirrors the deprecated `sub_tenant_id` alias.", - "example": "team_docs", + "collection": { + "description": "Collection this chunk belongs to. Canonical name; mirrors the deprecated `sub_tenant_id` alias.", + "example": "team_docs", "type": "string" }, "extra_context_ids": { @@ -7033,53 +5866,9 @@ "additionalProperties": { "$ref": "#/components/schemas/search.V2Chunk" }, - "deprecated": true, - "description": "deprecated: use forceful_relations", + "description": "Map of chunk ID to chunk content for sources declared as related by the author (query_forceful_relations).", "example": "The user is a senior engineer onboarding to the platform.", - "type": "object", - "x-deprecated": "true" - }, - "alias_expansions": { - "description": "Nicknames expanded to canonical names while answering.", - "items": { - "$ref": "#/components/schemas/search.AliasExpansionNote" - }, - "type": "array", - "uniqueItems": false - }, - "app_search_fusion": { - "$ref": "#/components/schemas/search.AppSearchFusionDiagnostics", - "description": "Diagnostics for how connector (app) results were blended into the chunks: where each returned chunk came from, and counts from the blending step. Omitted for access-scoped requests and when no blended chunks remain.", - "example": { - "stats": { - "app_chunks": 1, - "app_has_exact_ids": true, - "app_lane_empty_text": true, - "consensus": 1, - "exact_candidates": 1, - "exact_promoted": 1, - "limit": 1, - "normal_chunks": 1, - "normal_displaced": 1, - "tail_added": 1, - "tail_candidates": 1 - }, - "stats_by_pass": [ - { - "app_chunks": 1, - "app_has_exact_ids": true, - "app_lane_empty_text": true, - "consensus": 1, - "exact_candidates": 1, - "exact_promoted": 1, - "limit": 1, - "normal_chunks": 1, - "normal_displaced": 1, - "tail_added": 1, - "tail_candidates": 1 - } - ] - } + "type": "object" }, "chunks": { "description": "Retrieved and ranked chunks from the knowledge store or memories.", @@ -7116,102 +5905,6 @@ "type": "array", "uniqueItems": false }, - "forceful_relations": { - "$ref": "#/components/schemas/search.ForcefulRelationsBucket", - "example": { - "declared": [ - { - "chunk": { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "collection": "team_docs", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" - ], - "id": "HydraDoc1234", - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 - }, - "relevancy_score": 0.87, - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" - } - } - ], - "inferred": [ - { - "chunk": { - "additional_metadata": { - "author": "ada", - "doc_version": 3 - }, - "chunk_content": "HydraDB supports hybrid retrieval across knowledge and memories.", - "chunk_uuid": "a1b2c3d4-e5f6-7890-1234-567890abcdef", - "collection": "team_docs", - "extra_context_ids": [ - "HydraEmbeddings123_2", - "HydraEmbeddings123_3" - ], - "id": "HydraDoc1234", - "layout": "text", - "metadata": { - "department": "finance", - "priority": 7 - }, - "relevancy_score": 0.87, - "source_last_updated_time": "2026-07-02T12:30:00Z", - "source_title": "Project Phoenix Overview", - "source_type": "file", - "source_upload_time": "2026-07-02T10:00:00Z", - "sub_tenant_id": "sub_tenant_4567" - } - } - ] - } - }, - "graph": { - "$ref": "#/components/schemas/search.GraphPlane", - "example": { - "paths": [ - { - "chunk_ids": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "combined_context": "Acme Corp deploys HydraDB in production for context retrieval.", - "relevancy_score": 0.87, - "triplets": [ - { - "relation": { - "confidence": 0.92, - "predicate": "works_at" - }, - "source": { - "entity_id": "entity_1a2b", - "name": "Ada", - "type": "person" - }, - "target": { - "entity_id": "entity_3c4d", - "name": "Acme Corp", - "type": "organization" - } - } - ] - } - ] - } - }, "graph_context": { "$ref": "#/components/schemas/search.GraphContext", "example": { @@ -7280,32 +5973,8 @@ ] } }, - "profile_context": { - "$ref": "#/components/schemas/search.ProfileContext", - "example": { - "entity_id": "entity_1a2b", - "entries": [ - { - "confidence": 0.92 - } - ], - "name": "general", - "version": 1 - } - }, - "profile_filter": { - "$ref": "#/components/schemas/search.ProfileFilterInfo", - "example": { - "applied": true, - "degraded": true, - "entity_id": "entity_1a2b", - "found": true, - "selected_entries": 1, - "version": 1 - } - }, "source_facts": { - "description": "Matched source facts, when source reasoning ran.", + "description": "SourceFacts surface the matched app-native (edge_source) facts when\nsource_reasoning was active; omitted otherwise (PRO-1602).", "example": [ { "app_kind": "slack", @@ -7390,7 +6059,7 @@ } }, "temporal_facts": { - "description": "Matched temporal facts, when `temporal_reasoning` ran.", + "description": "TemporalFacts surface the matched edge-level temporal facts when\ntemporal_reasoning was requested; omitted otherwise.", "example": [ { "chunk_id": "HydraEmbeddings123_0", @@ -7426,7 +6095,7 @@ "properties": { "additional_metadata": { "additionalProperties": {}, - "description": "The chunk's custom attributes.", + "description": "Pydantic aliases: document_metadata→additional_metadata, tenant_metadata→metadata.\nFastAPI serializes responses by_alias, so the wire uses the alias names.", "example": { "author": "ada", "doc_version": 3 @@ -7504,144 +6173,6 @@ }, "type": "object" }, - "service.MetadataEditResult": { - "properties": { - "acl_drift_recorded": { - "description": "True when an access-control change that did not reach the search index was recorded so it gets applied later. Always true when `vector_acl_synced` is true.", - "example": true, - "type": "boolean" - }, - "acl_updated": { - "description": "True when this edit replaced the source's access-control list.", - "example": true, - "type": "boolean" - }, - "additional_metadata_keys": { - "description": "Additional metadata keys included in the update request.", - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "chunk_rows_matched": { - "description": "Number of stored chunks matched by the update.", - "example": 1, - "type": "integer" - }, - "chunk_rows_modified": { - "description": "Number of stored chunks changed by the update.", - "example": 1, - "type": "integer" - }, - "collection": { - "description": "Collection that contained the source. Canonical name; mirrors the deprecated `sub_tenant_id` alias.", - "example": "team_docs", - "type": "string" - }, - "database": { - "description": "Owning database. Canonical name; mirrors the deprecated `tenant_id` alias.", - "example": "acme_corp", - "type": "string" - }, - "database_metadata_keys": { - "description": "Database metadata keys included in the update request. Canonical name; `tenant_metadata_keys` is a deprecated alias.", - "example": [ - "department", - "priority" - ], - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "id": { - "description": "Unique identifier for this resource.", - "example": "HydraDoc1234", - "type": "string" - }, - "milvus_rows_synced": { - "deprecated": true, - "description": "deprecated: use vector_rows_synced", - "example": 1, - "type": "integer", - "x-deprecated": "true" - }, - "milvus_sync_required": { - "deprecated": true, - "description": "Deprecated: use vector_sync_required / vector_synced / vector_rows_synced.\nRetained as additive aliases for existing clients; carry the same values.", - "example": true, - "type": "boolean", - "x-deprecated": "true" - }, - "milvus_synced": { - "deprecated": true, - "description": "deprecated: use vector_synced", - "example": true, - "type": "boolean", - "x-deprecated": "true" - }, - "partial_commit": { - "description": "Present when the edit was only partly applied because a later write failed, carrying that failure's message. Retrying the same request completes it.", - "type": "string" - }, - "sub_tenant_id": { - "deprecated": true, - "description": "deprecated: use collection", - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated": "true" - }, - "tenant_id": { - "deprecated": true, - "description": "deprecated: use database", - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - }, - "tenant_metadata_keys": { - "deprecated": true, - "description": "deprecated: use database_metadata_keys", - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false, - "x-deprecated": "true" - }, - "updated": { - "description": "Whether the source metadata was updated.", - "example": true, - "type": "boolean" - }, - "vector_acl_synced": { - "description": "True when the access-control change also reached the search index. When false, removed principals still lose access, but principals the edit added may not find the source in queries until its next re-index. Always false for collections created before access control was added to the search index.", - "example": true, - "type": "boolean" - }, - "vector_rows_synced": { - "description": "Number of chunks synced to the vector store when sync was required.", - "example": 1, - "type": "integer" - }, - "vector_sync_error": { - "description": "Why the search index was not updated, when an update was required. The edit itself is saved; retrying completes it.", - "type": "string" - }, - "vector_sync_required": { - "description": "Whether the edit needed the search index updated. The `milvus_*` fields are deprecated aliases with the same values.", - "example": true, - "type": "boolean" - }, - "vector_synced": { - "description": "Whether the vector store metadata sync completed. Present when sync was required.", - "example": true, - "type": "boolean" - } - }, - "type": "object" - }, "sources.MemoryDeleteResponse": { "properties": { "deleted_count": { @@ -7671,7 +6202,7 @@ }, "success": { "deprecated": true, - "description": "Deprecated. True even when nothing was removed. Read `deleted_count` and each `results[].deleted` / `results[].error`, and the HTTP status for whether the request succeeded.", + "description": "Deprecated for API clients: whether the REQUEST succeeded is the HTTP\nstatus code, or equivalently the envelope's top-level `success`. Whether\nanything was actually removed is deleted_count (0 means the ids matched\nnothing) and per-id results[].deleted / results[].error. This flag is\ntrue even for a delete that removed nothing, so it cannot answer either\nquestion on its own. Still emitted unchanged for existing clients\n(PRO-1208).", "example": true, "type": "boolean", "x-deprecated": "true" @@ -7712,7 +6243,7 @@ "type": "string" }, "database": { - "description": "Database to delete from. `tenant_id` is accepted as a deprecated alias.", + "description": "Database/Collection are the canonical v2 names; TenantID/SubTenantID are\ntheir deprecated aliases, reconciled by the TenantAliases middleware before\nbinding so TenantID is always populated.", "example": "acme_corp", "type": "string" }, @@ -7743,12 +6274,10 @@ "x-deprecated": "true" }, "type": { - "description": "Which corpus to delete from. Split database: `knowledge` (default) or `memory`. Unified database: omit it; `knowledge` and `memory` are rejected.", + "description": "Bucket to delete from: `knowledge` (default) or `memory`.", "enum": [ "knowledge", - "memory", - "unified", - "all" + "memory" ], "example": "knowledge", "type": "string" @@ -7756,22 +6285,10 @@ }, "type": "object" }, - "storagelayout.Layout": { - "description": "Storage type of a database, fixed when it is created: `unified` (one corpus for every kind of context) or `split` (a legacy database with separate knowledge and memory corpora).", - "enum": [ - "split", - "unified" - ], - "type": "string", - "x-enum-varnames": [ - "LayoutSplit", - "LayoutUnified" - ] - }, "tenants.CollectionStats": { "properties": { "row_count": { - "description": "Total number of indexed chunks in this collection.", + "description": "Total number of indexed rows in this collection.", "example": 1280, "type": "integer" } @@ -7782,7 +6299,7 @@ "properties": { "data_type": { "$ref": "#/components/schemas/tenants.MilvusDataType", - "description": "Data type of this attribute field.", + "description": "Milvus data type for this metadata field.", "example": "VARCHAR" }, "enable_dense_embedding": { @@ -7813,25 +6330,6 @@ }, "type": "object" }, - "tenants.DatabaseDetail": { - "properties": { - "database": { - "description": "Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated).", - "example": "acme_corp", - "type": "string" - }, - "type": { - "description": "Storage type the database was created with: `split` (separate knowledge and memory corpora, selected with `type` on each call) or `unified` (one corpus).", - "enum": [ - "split", - "unified" - ], - "example": "split", - "type": "string" - } - }, - "type": "object" - }, "tenants.FailedTenant": { "properties": { "database": { @@ -7887,15 +6385,6 @@ "example": "tenant_1234", "type": "string", "x-deprecated": "true" - }, - "type": { - "description": "Storage type the database was created with: `split` or `unified`. Absent while the database is being deleted.", - "enum": [ - "split", - "unified" - ], - "example": "split", - "type": "string" } }, "type": "object" @@ -7908,12 +6397,12 @@ "type": "boolean" }, "ready_for_ingestion": { - "description": "Derived readiness flag: true only when scheduler_status (database setup finished), graph_status, and both vectorstore_status.knowledge and vectorstore_status.memories are true — i.e. the database is fully provisioned and ready to accept ingestion and serve queries. Database creation is asynchronous: collections may appear before provisioning completes, so poll GET /databases/status until this is true before ingesting or querying.", + "description": "Derived readiness flag: true only when scheduler_status (lifecycle provisioning finished), graph_status, and both vectorstore_status.knowledge and vectorstore_status.memories are true — i.e. the database is fully provisioned and ready to accept ingestion and serve queries. Database creation is asynchronous: collections may appear before provisioning completes, so poll GET /databases/status until this is true before ingesting or querying.", "example": true, "type": "boolean" }, "scheduler_status": { - "description": "Whether setup has finished for this database (creation_status is ready). False while the database is still being created, even if individual collections already exist.", + "description": "Whether lifecycle provisioning has finished for this database (creation_status is ready). False while the database is still being created, even if individual collections already exist.", "example": true, "type": "boolean" }, @@ -7928,7 +6417,6 @@ "type": "object" }, "tenants.MilvusDataType": { - "description": "Declared type of a database metadata schema field. ARRAY appears in this enum because schemas persisted before it was rejected still read back and rebuild, but it CANNOT be declared on a new or evolved field: both the create and the update-metadata-schema endpoints answer 400 for it. For a field holding several values, declare VARCHAR and store the values comma-joined, then filter one member with the contains operator.", "enum": [ "BOOL", "INT8", @@ -7955,63 +6443,26 @@ "DataTypeArray" ] }, - "tenants.SubTenantDeleteResponse": { + "tenants.SubTenantIdsResponse": { "properties": { - "collection": { - "description": "Collection scope. Defaults to the default collection when omitted. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated).", - "example": "team_docs", - "type": "string" - }, - "database": { - "description": "Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated).", - "example": "acme_corp", - "type": "string" + "collections": { + "description": "List of collection identifiers for this database.", + "example": [ + "team_docs", + "engineering" + ], + "items": { + "type": "string" + }, + "type": "array", + "uniqueItems": false }, "message": { "description": "Human-readable result message.", "example": "Success", "type": "string" }, - "status": { - "description": "Current lifecycle or processing state.", - "example": "completed", - "type": "string" - }, - "sub_tenant_id": { - "deprecated": true, - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated": "true" - }, - "tenant_id": { - "deprecated": true, - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - } - }, - "type": "object" - }, - "tenants.SubTenantIdsResponse": { - "properties": { - "collections": { - "description": "List of collection identifiers for this database.", - "example": [ - "team_docs", - "engineering" - ], - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "message": { - "description": "Human-readable result message.", - "example": "Success", - "type": "string" - }, - "sub_tenant_ids": { + "sub_tenant_ids": { "deprecated": true, "description": "Deprecated alias for `collections`.", "example": [ @@ -8057,7 +6508,7 @@ "tenants.TenantCreateRequest": { "properties": { "database": { - "description": "Account-scoped database identifier. Use a stable, case-sensitive ID up to 25 characters. Immutable after creation. `tenant_id` is accepted as a deprecated alias.", + "description": "Database is the canonical v2 name; TenantID is its deprecated alias and\nremains fully accepted. The TenantAliases middleware reconciles them before\nthis binds, so TenantID is always populated.", "example": "acme_corp", "type": "string" }, @@ -8105,9 +6556,43 @@ "type": "array", "uniqueItems": false, "x-deprecated": "true" + } + }, + "type": "object" + }, + "tenants.SubTenantDeleteResponse": { + "properties": { + "collection": { + "description": "Collection that was deleted. Formerly `sub_tenant_id`.", + "example": "engineering", + "type": "string" }, - "type": { - "$ref": "#/components/schemas/storagelayout.Layout" + "database": { + "description": "Owning database. Formerly `tenant_id`.", + "example": "acme_corp", + "type": "string" + }, + "message": { + "description": "Human-readable result message.", + "example": "Collection deregistered. Background cleanup is in progress.", + "type": "string" + }, + "status": { + "description": "Current lifecycle or processing state.", + "example": "deletion_scheduled", + "type": "string" + }, + "sub_tenant_id": { + "deprecated": true, + "example": "engineering", + "type": "string", + "x-deprecated": "true" + }, + "tenant_id": { + "deprecated": true, + "example": "acme_corp", + "type": "string", + "x-deprecated": "true" } }, "type": "object" @@ -8152,20 +6637,6 @@ "type": "array", "uniqueItems": false }, - "details": { - "description": "One entry per database with its storage type, so a client can tell a unified database from a split one.", - "example": [ - { - "database": "acme_corp", - "type": "split" - } - ], - "items": { - "$ref": "#/components/schemas/tenants.DatabaseDetail" - }, - "type": "array", - "uniqueItems": false - }, "failed_databases": { "description": "Databases that failed provisioning, with error details.", "example": [ @@ -8276,57 +6747,6 @@ }, "type": "object" }, - "tenants.TenantRenameRequest": { - "properties": { - "new_name": { - "description": "The database's new name. Same rules as a name at creation.", - "type": "string" - } - }, - "required": [ - "new_name" - ], - "type": "object" - }, - "tenants.TenantRenameResponse": { - "properties": { - "connector_reassignment": { - "description": "How moving connectors to the new name ended: `complete`, `queued` (finishing in the background) or `failed` (contact support). The rename itself succeeded in all three cases.", - "example": "complete", - "type": "string" - }, - "database": { - "description": "Owning database. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated).", - "example": "acme_corp", - "type": "string" - }, - "message": { - "description": "Human-readable result message.", - "example": "Success", - "type": "string" - }, - "old_database": { - "type": "string" - }, - "old_tenant_id": { - "deprecated": true, - "type": "string", - "x-deprecated": "true" - }, - "status": { - "description": "Current lifecycle or processing state.", - "example": "completed", - "type": "string" - }, - "tenant_id": { - "deprecated": true, - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - } - }, - "type": "object" - }, "tenants.TenantStatsResponse": { "properties": { "database": { @@ -8562,529 +6982,183 @@ "url": { "description": "Registered endpoint URL that receives webhook event deliveries.", "example": "https://docs.hydradb.com/phoenix", - "type": "string" - } - }, - "type": "object" - }, - "webhooks.WebhookRegisterRequest": { - "properties": { - "event_types": { - "description": "Event types to subscribe to (e.g. `[\"indexing.status_changed\"]`).", - "example": [ - "indexing.status_changed" - ], - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "generate_signing_secret": { - "description": "Generate a signing secret as part of this request, so registering and enabling signing are one atomic operation. The secret is returned once on the response and cannot be retrieved later. Mutually exclusive with `signing_secret`.", - "example": true, - "type": "boolean" - }, - "signing_secret": { - "description": "Secret used to sign webhook payloads. Deliveries carry `X-HydraDB-Signature: sha256=\u003chex\u003e`, the HMAC-SHA256 of the raw request body keyed by this secret. Minimum 16 characters when you supply your own; omit it and one is generated for you. On registration, omitting this field preserves any existing secret - to disable signing, call DELETE /webhooks/indexing/signing-secret.", - "example": "whsec_EXAMPLE_ONLY_THIS_IS_NOT_A_REAL_SIGNING_KEY", - "type": "string" - }, - "url": { - "description": "Endpoint URL to deliver webhook events to.", - "example": "https://docs.hydradb.com/phoenix", - "type": "string" - } - }, - "type": "object" - }, - "webhooks.WebhookRegisterResponse": { - "properties": { - "event_types": { - "description": "Event types to subscribe to (e.g. `[\"indexing.status_changed\"]`).", - "example": [ - "indexing.status_changed" - ], - "items": { - "type": "string" - }, - "type": "array", - "uniqueItems": false - }, - "message": { - "description": "Human-readable result message.", - "example": "Success", - "type": "string" - }, - "registered": { - "description": "Whether a webhook is registered for this API key.", - "example": true, - "type": "boolean" - }, - "signing_secret": { - "description": "Secret used to sign webhook payloads. Deliveries carry `X-HydraDB-Signature: sha256=\u003chex\u003e`, the HMAC-SHA256 of the raw request body keyed by this secret. Minimum 16 characters when you supply your own; omit it and one is generated for you. On registration, omitting this field preserves any existing secret - to disable signing, call DELETE /webhooks/indexing/signing-secret.", - "example": "whsec_EXAMPLE_ONLY_THIS_IS_NOT_A_REAL_SIGNING_KEY", - "type": "string" - }, - "signing_secret_configured": { - "description": "Whether a signing secret has been configured for payload verification.", - "example": true, - "type": "boolean" - }, - "url": { - "description": "Registered endpoint URL that receives webhook event deliveries.", - "example": "https://docs.hydradb.com/phoenix", - "type": "string" - } - }, - "type": "object" - }, - "webhooks.WebhookTestResponse": { - "properties": { - "delivered": { - "description": "Whether the test delivery was accepted by the endpoint.", - "example": true, - "type": "boolean" - }, - "message": { - "description": "Human-readable result message.", - "example": "Success", - "type": "string" - }, - "status_code": { - "description": "HTTP status code returned by the webhook endpoint.", - "example": 200, - "type": "integer" - } - }, - "type": "object" - } - }, - "securitySchemes": { - "BearerAuth": { - "bearerFormat": "API key", - "description": "API key sent as a Bearer token: \"Bearer prefix.secret\"", - "scheme": "bearer", - "type": "http" - } - } - }, - "externalDocs": { - "description": "", - "url": "" - }, - "info": { - "contact": { - "email": "support@hydradb.com", - "name": "HydraDB Support" - }, - "description": "HydraDB Application API — knowledge ingestion, search, and memory management.", - "license": { - "name": "Proprietary" - }, - "title": "HydraDB Application API", - "version": "0.1.0" - }, - "openapi": "3.1.0", - "paths": { - "/connector-catalog": { - "get": { - "description": "List every connector provider with its availability, maturity and category.", - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.connectorCatalogResponse" - } - } - }, - "description": "OK" - } - }, - "summary": "List the connector catalog", - "tags": [ - "connectors" - ], - "x-fern-sdk-method-name": "catalog" - } - }, - "/connector-discovery": { - "post": { - "description": "List a provider's resources directly from supplied credentials, before creating a connector. Passing cursor or limit opts into pagination (currently Notion only): the response then adds next_cursor and has_more, a page may hold fewer than limit resources, and clients must continue while has_more is true. Without either param the full resource list is returned.", - "parameters": [ - { - "description": "Opaque pagination cursor from a previous response's next_cursor", - "in": "query", - "name": "cursor", - "schema": { - "type": "string" - } - }, - { - "description": "Max resources per page, 1-100 (values above 100 are clamped)", - "in": "query", - "name": "limit", - "schema": { - "type": "integer" - } - } - ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.discoverPreviewReq" - } - } - }, - "description": "Provider and credentials", - "required": true - }, - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.discoverResponseBody" - } - } - }, - "description": "OK" - }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Request" - }, - "502": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Gateway" - } - }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Preview provider resources", - "tags": [ - "connectors" - ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "discover_preview" - } - }, - "/connectors": { - "get": { - "description": "List all connectors for the authenticated org, optionally filtered by provider.", - "parameters": [ - { - "description": "Filter by provider", - "in": "query", - "name": "provider", - "schema": { - "example": "slack", - "type": "string" - } - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.connectorListResponse" - } - } - }, - "description": "OK" - }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" - } - }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "List connectors", - "tags": [ - "connectors" - ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "list" - }, - "post": { - "description": "Create a connector for a provider and store its credentials.", - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.connectorCreateReq" - } - } - }, - "description": "Connector configuration", - "required": true - }, - "responses": { - "201": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.connectorCreateResponse" - } - } - }, - "description": "Created" - }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Request" - }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" - }, - "503": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Service Unavailable" + "type": "string" } }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Create a connector", - "tags": [ - "connectors" - ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "create" - } - }, - "/connectors/providers": { - "get": { - "description": "Without ?id: returns every supported connector with its availability, maturity and category (the connector catalog). With ?id=\u003cprovider\u003e: returns what that provider stores and how to use it — indexed_object_types (the streams that become searchable documents), searchable_fields (rendered into the indexed text), filterable_fields (each with the exact filter_key to pass in a query's metadata_filters), the credential_schema for connecting it, and setup_guide (present for providers whose configuration goes beyond the credential schema — e.g. bigquery's per-table cursor/change-history settings and the one-time ALTER statement they may require).", - "parameters": [ - { - "description": "Provider name (e.g. slack, gmail). Omit to list all.", - "in": "query", - "name": "id", - "schema": { - "example": "HydraDoc1234", + "type": "object" + }, + "webhooks.WebhookRegisterRequest": { + "properties": { + "event_types": { + "description": "Event types to subscribe to (e.g. `[\"indexing.status_changed\"]`).", + "example": [ + "indexing.status_changed" + ], + "items": { "type": "string" - } - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.providerListResponse" - } - } }, - "description": "OK" + "type": "array", + "uniqueItems": false }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" + "generate_signing_secret": { + "description": "Generate a signing secret as part of this request, so registering and enabling signing are one atomic operation. The secret is returned once on the response and cannot be retrieved later. Mutually exclusive with `signing_secret`.", + "example": true, + "type": "boolean" + }, + "signing_secret": { + "description": "Secret used to sign webhook payloads. Deliveries carry `X-HydraDB-Signature: sha256=\u003chex\u003e`, the HMAC-SHA256 of the raw request body keyed by this secret. Minimum 16 characters when you supply your own; omit it and one is generated for you. On registration, omitting this field preserves any existing secret - to disable signing, call DELETE /webhooks/indexing/signing-secret.", + "example": "whsec_EXAMPLE_ONLY_THIS_IS_NOT_A_REAL_SIGNING_KEY", + "type": "string" + }, + "url": { + "description": "Endpoint URL to deliver webhook events to.", + "example": "https://docs.hydradb.com/phoenix", + "type": "string" } }, - "summary": "List supported providers, or describe one in detail", - "tags": [ - "connectors" - ], - "x-fern-sdk-method-name": "listProviders" - } - }, - "/connectors/{id}": { - "delete": { - "description": "Delete a connector, its resources, and stored credentials.", - "parameters": [ - { - "description": "Connector ID", - "in": "path", - "name": "id", - "required": true, - "schema": { - "example": "HydraDoc1234", + "type": "object" + }, + "webhooks.WebhookRegisterResponse": { + "properties": { + "event_types": { + "description": "Event types to subscribe to (e.g. `[\"indexing.status_changed\"]`).", + "example": [ + "indexing.status_changed" + ], + "items": { "type": "string" - } - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.connectorDeleteResponse" - } - } }, - "description": "OK" + "type": "array", + "uniqueItems": false }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" + "message": { + "description": "Human-readable result message.", + "example": "Success", + "type": "string" }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" + "registered": { + "description": "Whether a webhook is registered for this API key.", + "example": true, + "type": "boolean" + }, + "signing_secret": { + "description": "Secret used to sign webhook payloads. Deliveries carry `X-HydraDB-Signature: sha256=\u003chex\u003e`, the HMAC-SHA256 of the raw request body keyed by this secret. Minimum 16 characters when you supply your own; omit it and one is generated for you. On registration, omitting this field preserves any existing secret - to disable signing, call DELETE /webhooks/indexing/signing-secret.", + "example": "whsec_EXAMPLE_ONLY_THIS_IS_NOT_A_REAL_SIGNING_KEY", + "type": "string" + }, + "signing_secret_configured": { + "description": "Whether a signing secret has been configured for payload verification.", + "example": true, + "type": "boolean" + }, + "url": { + "description": "Registered endpoint URL that receives webhook event deliveries.", + "example": "https://docs.hydradb.com/phoenix", + "type": "string" } }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Delete a connector", - "tags": [ - "connectors" - ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "delete" + "type": "object" }, - "get": { - "description": "Fetch a single connector by ID.", - "parameters": [ - { - "description": "Connector ID", - "in": "path", - "name": "id", - "required": true, - "schema": { - "example": "HydraDoc1234", - "type": "string" - } + "webhooks.WebhookTestResponse": { + "properties": { + "delivered": { + "description": "Whether the test delivery was accepted by the endpoint.", + "example": true, + "type": "boolean" + }, + "message": { + "description": "Human-readable result message.", + "example": "Success", + "type": "string" + }, + "status_code": { + "description": "HTTP status code returned by the webhook endpoint.", + "example": 200, + "type": "integer" } - ], + }, + "type": "object" + } + }, + "securitySchemes": { + "BearerAuth": { + "bearerFormat": "API key", + "description": "API key sent as a Bearer token: \"Bearer prefix.secret\"", + "scheme": "bearer", + "type": "http" + } + } + }, + "externalDocs": { + "description": "", + "url": "" + }, + "info": { + "contact": { + "email": "support@hydradb.com", + "name": "HydraDB Support" + }, + "description": "HydraDB Application API — knowledge ingestion, search, and memory management.", + "license": { + "name": "Proprietary" + }, + "title": "HydraDB Application API", + "version": "0.1.0" + }, + "openapi": "3.1.0", + "paths": { + "/connector-catalog": { + "get": { + "description": "List every provider in the supported_connectors control-plane table (availability, sync engine, maturity, category) for the dashboard connector catalog.", "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.connectorAPIView" + "$ref": "#/components/schemas/handler.connectorCatalogResponse" } } }, "description": "OK" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" - }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" } }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Get a connector", + "summary": "List the connector catalog", "tags": [ "connectors" ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "get" - }, - "patch": { - "description": "Update a connector's mutable settings. `sync_interval_seconds` sets how often incremental syncs run: the allowed range is provider-aware and returned in the response; values outside it are rejected rather than clamped; 0 resets to the provider default; changing it re-anchors the next sync so a shorter cadence takes effect immediately. `credentials` reconnects the connector in place: send the provider's full credential set (what create accepts); supplied keys replace their stored values, other stored keys survive, the bundle is re-validated against the provider's credential schema, and a pending needs-reauth flag is cleared — the connector keeps its id, resources, and sync cursors. `custom_instructions` replaces the free-text steering applied when this connector's documents are ingested (an explicit empty string clears it); the change applies from the next sync cycle and does not re-process already-ingested documents. When several fields are supplied together they are validated up front and applied atomically: an invalid value rejects the whole request with nothing changed.", + "x-fern-sdk-method-name": "catalog" + } + }, + "/connector-discovery": { + "post": { + "description": "List a provider's resources directly from supplied credentials, before creating a connector. Passing cursor or limit opts into pagination (currently Notion only): the response then adds next_cursor and has_more, a page may hold fewer than limit resources, and clients must continue while has_more is true. Without either param the full resource list is returned.", "parameters": [ { - "description": "Connector ID", - "in": "path", - "name": "id", - "required": true, + "description": "Opaque pagination cursor from a previous response's next_cursor", + "in": "query", + "name": "cursor", "schema": { - "example": "HydraDoc1234", "type": "string" } + }, + { + "description": "Max resources per page, 1-100 (values above 100 are clamped)", + "in": "query", + "name": "limit", + "schema": { + "type": "integer" + } } ], "requestBody": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.connectorUpdateReq" + "$ref": "#/components/schemas/handler.discoverPreviewReq" } } }, - "description": "Connector update request", + "description": "Provider and credentials", "required": true }, "responses": { @@ -9092,7 +7166,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.connectorUpdateResponse" + "$ref": "#/components/schemas/handler.discoverResponseBody" } } }, @@ -9108,17 +7182,7 @@ }, "description": "Bad Request" }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" - }, - "500": { + "502": { "content": { "application/json": { "schema": { @@ -9126,7 +7190,7 @@ } } }, - "description": "Internal Server Error" + "description": "Bad Gateway" } }, "security": [ @@ -9134,71 +7198,39 @@ "BearerAuth": [] } ], - "summary": "Update a connector", + "summary": "Preview provider resources", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "update" + "x-fern-sdk-method-name": "discover_preview" } }, - "/connectors/{id}/configure": { - "post": { - "description": "Save the selected resources for a connector and trigger initial sync/backfill.", + "/connectors": { + "get": { + "description": "List all connectors for the authenticated org, optionally filtered by provider.", "parameters": [ { - "description": "Connector ID", - "in": "path", - "name": "id", - "required": true, + "description": "Filter by provider", + "in": "query", + "name": "provider", "schema": { - "example": "HydraDoc1234", + "example": "slack", "type": "string" } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.configureReq" - } - } - }, - "description": "Resource selection and sync options", - "required": true - }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.configureResponse" + "$ref": "#/components/schemas/handler.connectorListResponse" } } }, "description": "OK" }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Request" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" - }, "500": { "content": { "application/json": { @@ -9215,48 +7247,36 @@ "BearerAuth": [] } ], - "summary": "Configure connector resources", + "summary": "List connectors", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "configure" - } - }, - "/connectors/{id}/credentials": { - "patch": { - "description": "Internal endpoint. It is not callable with a customer API key and always returns 403 for external callers. Persists a rotated refresh_token for OAuth-bundle connectors: providers that rotate the refresh token on each exchange invalidate the previously stored one, so the new token must be written back or the next sync fails with invalid_grant. Only refresh_token is merged onto the current stored credential bundle and re-encrypted under the connector's identity context; credentials are never returned. It has no automated caller and is disabled by default: until an operator enables it, every call returns 500.", - "parameters": [ - { - "description": "Connector ID", - "in": "path", - "name": "id", - "required": true, - "schema": { - "example": "HydraDoc1234", - "type": "string" - } - } - ], + "x-fern-sdk-method-name": "list" + }, + "post": { + "description": "Create a connector for a provider and store its credentials.", "requestBody": { "content": { "application/json": { "schema": { - "type": "object" + "$ref": "#/components/schemas/handler.connectorCreateReq" } } - } + }, + "description": "Connector configuration", + "required": true }, "responses": { - "200": { + "201": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.credentialsUpdateResponse" + "$ref": "#/components/schemas/handler.connectorCreateResponse" } } }, - "description": "OK" + "description": "Created" }, "400": { "content": { @@ -9268,26 +7288,6 @@ }, "description": "Bad Request" }, - "403": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Forbidden" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" - }, "500": { "content": { "application/json": { @@ -9309,41 +7309,31 @@ "description": "Service Unavailable" } }, - "summary": "Rotate a connector's stored OAuth refresh token (internal use only)", + "security": [ + { + "BearerAuth": [] + } + ], + "summary": "Create a connector", "tags": [ "connectors" - ] + ], + "x-fern-sdk-group-name": "connectors", + "x-fern-sdk-method-name": "create" } }, - "/connectors/{id}/discover": { + "/connectors/providers": { "get": { - "description": "List a connected provider's resources using the connector's stored credentials. Passing cursor or limit opts into pagination (currently Notion only): the response then adds next_cursor and has_more, a page may hold fewer than limit resources, and clients must continue while has_more is true. Without either param the full resource list is returned.", + "description": "Without ?id: returns every supported connector with its availability, maturity, category, and sync engine (the connector catalog). With ?id=\u003cprovider\u003e: returns what that provider stores and how to use it — indexed_object_types (the streams that become searchable documents), searchable_fields (rendered into the indexed text), filterable_fields (each with the exact filter_key to pass in a query's metadata_filters), the credential_schema for connecting it, and setup_guide (present for providers whose configuration goes beyond the credential schema — e.g. bigquery's per-table cursor/change-history settings and the one-time ALTER statement they may require).", "parameters": [ { - "description": "Connector ID", - "in": "path", + "description": "Provider name (e.g. slack, gmail). Omit to list all.", + "in": "query", "name": "id", - "required": true, "schema": { "example": "HydraDoc1234", "type": "string" } - }, - { - "description": "Opaque pagination cursor from a previous response's next_cursor", - "in": "query", - "name": "cursor", - "schema": { - "type": "string" - } - }, - { - "description": "Max resources per page, 1-100 (values above 100 are clamped)", - "in": "query", - "name": "limit", - "schema": { - "type": "integer" - } } ], "responses": { @@ -9351,22 +7341,12 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.discoverResponseBody" + "$ref": "#/components/schemas/handler.providerListResponse" } } }, "description": "OK" }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Request" - }, "404": { "content": { "application/json": { @@ -9376,54 +7356,18 @@ } }, "description": "Not Found" - }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" - }, - "502": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Gateway" - }, - "503": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Service Unavailable" } }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Discover connector resources", + "summary": "List supported providers, or describe one in detail", "tags": [ "connectors" ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "discover" + "x-fern-sdk-method-name": "listProviders" } }, - "/connectors/{id}/pause": { - "post": { - "description": "Stop scheduling syncs and backfills for a connector until it is resumed. A sync already running is allowed to finish. Cursors are preserved, so resuming continues from where each resource left off.", + "/connectors/{id}": { + "delete": { + "description": "Delete a connector, its resources, and stored credentials.", "parameters": [ { "description": "Connector ID", @@ -9441,7 +7385,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.connectorPauseResponse" + "$ref": "#/components/schemas/handler.connectorDeleteResponse" } } }, @@ -9455,17 +7399,7 @@ } } }, - "description": "Not Found" - }, - "409": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Conflict" + "description": "Not Found" }, "500": { "content": { @@ -9483,17 +7417,15 @@ "BearerAuth": [] } ], - "summary": "Pause a connector", + "summary": "Delete a connector", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "pause" - } - }, - "/connectors/{id}/resources": { + "x-fern-sdk-method-name": "delete" + }, "get": { - "description": "List the configured resources for a connector.", + "description": "Fetch a single connector by ID.", "parameters": [ { "description": "Connector ID", @@ -9511,7 +7443,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.connectorResourcesResponse" + "$ref": "#/components/schemas/handler.connectorAPIView" } } }, @@ -9543,15 +7475,15 @@ "BearerAuth": [] } ], - "summary": "List connector resources", + "summary": "Get a connector", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "list_resources" + "x-fern-sdk-method-name": "get" }, - "post": { - "description": "Add a resource mapping to a connector.", + "patch": { + "description": "Update a connector's mutable settings. `sync_interval_seconds` sets the cadence at which the scheduler starts incremental syncs: the allowed range is provider-aware and returned in the response; values outside it are rejected rather than clamped; 0 resets to the provider default; changing it re-anchors the next sync so a shorter cadence takes effect immediately. `credentials` reconnects the connector in place: send the provider's full credential set (what create accepts); supplied keys replace their stored values, other stored keys survive, the bundle is re-validated against the provider's credential schema, and a pending needs-reauth flag is cleared — the connector keeps its id, resources, and sync cursors. `custom_instructions` replaces the free-text steering applied when this connector's documents are ingested (an explicit empty string clears it); the change applies from the next sync cycle and does not re-process already-ingested documents. When several fields are supplied together they are validated up front and applied atomically: an invalid value rejects the whole request with nothing changed.", "parameters": [ { "description": "Connector ID", @@ -9568,23 +7500,23 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.resourceCreateReq" + "$ref": "#/components/schemas/handler.connectorUpdateReq" } } }, - "description": "Resource configuration", + "description": "Connector update request", "required": true }, "responses": { - "201": { + "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/connectors.Resource" + "$ref": "#/components/schemas/handler.connectorUpdateResponse" } } }, - "description": "Created" + "description": "OK" }, "400": { "content": { @@ -9622,17 +7554,17 @@ "BearerAuth": [] } ], - "summary": "Create a connector resource", + "summary": "Update a connector", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "create_resource" + "x-fern-sdk-method-name": "update" } }, - "/connectors/{id}/resources/{resource_id}": { - "delete": { - "description": "Remove a resource mapping from a connector.", + "/connectors/{id}/configure": { + "post": { + "description": "Save the selected resources for a connector and trigger initial sync/backfill.", "parameters": [ { "description": "Connector ID", @@ -9643,24 +7575,25 @@ "example": "HydraDoc1234", "type": "string" } - }, - { - "description": "Resource ID", - "in": "path", - "name": "resource_id", - "required": true, - "schema": { - "example": "C0123456789", - "type": "string" - } } ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.configureReq" + } + } + }, + "description": "Resource selection and sync options", + "required": true + }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.resourceDeleteResponse" + "$ref": "#/components/schemas/handler.configureResponse" } } }, @@ -9702,15 +7635,17 @@ "BearerAuth": [] } ], - "summary": "Delete a connector resource", + "summary": "Configure connector resources", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "delete_resource" - }, + "x-fern-sdk-method-name": "configure" + } + }, + "/connectors/{id}/credentials": { "patch": { - "description": "Updates per-resource settings. `acl` sets the access-control list for one connector resource: bare emails are treated as user principals, `__public__` opens the resource to everyone, and an explicit empty list makes it private. It applies immediately to every document already synced from the resource, with no re-sync. For providers with provider-derived ACL capture enabled, the provider's own ACL takes precedence again at the next sync; the rule is the standing fallback. `custom_instructions` sets the resource-level ingestion-instructions override (max 4000 characters): when set it replaces the connector-level custom_instructions for documents synced from this resource, an explicit empty string clears the override back to inheriting the connector's value, and changes apply from the next sync cycle. At least one field must be supplied; omitted fields are left unchanged.", + "description": "Internal endpoint. It is not callable with a customer API key and always returns 403 for external callers. Persists a rotated refresh_token for OAuth-bundle connectors: providers that rotate the refresh token on each exchange invalidate the previously stored one, so the new token must be written back or the next sync fails with invalid_grant. Only refresh_token is merged onto the current stored credential bundle and re-encrypted under the connector's identity context; credentials are never returned. It has no automated caller and is disabled by default: until an operator enables it, every call returns 500.", "parameters": [ { "description": "Connector ID", @@ -9721,16 +7656,6 @@ "example": "HydraDoc1234", "type": "string" } - }, - { - "description": "Resource ID", - "in": "path", - "name": "resource_id", - "required": true, - "schema": { - "example": "C0123456789", - "type": "string" - } } ], "requestBody": { @@ -9747,8 +7672,7 @@ "content": { "application/json": { "schema": { - "additionalProperties": {}, - "type": "object" + "$ref": "#/components/schemas/handler.credentialsUpdateResponse" } } }, @@ -9764,17 +7688,7 @@ }, "description": "Bad Request" }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" - }, - "500": { + "403": { "content": { "application/json": { "schema": { @@ -9782,47 +7696,7 @@ } } }, - "description": "Internal Server Error" - } - }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Update a resource's ACL rule or custom instructions", - "tags": [ - "connectors" - ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "update_resource_acl" - } - }, - "/connectors/{id}/resume": { - "post": { - "description": "Return a paused connector to the schedule and make it due immediately. Each resource continues from its committed cursor, so data created during the pause is collected on the next cycle rather than skipped.", - "parameters": [ - { - "description": "Connector ID", - "in": "path", - "name": "id", - "required": true, - "schema": { - "example": "HydraDoc1234", - "type": "string" - } - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.connectorPauseResponse" - } - } - }, - "description": "OK" + "description": "Forbidden" }, "404": { "content": { @@ -9834,7 +7708,7 @@ }, "description": "Not Found" }, - "409": { + "500": { "content": { "application/json": { "schema": { @@ -9842,9 +7716,9 @@ } } }, - "description": "Conflict" + "description": "Internal Server Error" }, - "500": { + "503": { "content": { "application/json": { "schema": { @@ -9852,25 +7726,18 @@ } } }, - "description": "Internal Server Error" + "description": "Service Unavailable" } }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Resume a paused connector", + "summary": "Rotate a connector's stored OAuth refresh token (internal use only)", "tags": [ "connectors" - ], - "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "resume" + ] } }, - "/connectors/{id}/status": { + "/connectors/{id}/discover": { "get": { - "description": "Report whether a connector is working, in one call: a rollup status (healthy, degraded, failed, checking) plus per-resource detail. `degraded` means the connector is scheduled but not fully working: at least one configured resource is failing, or the latest sync cycle failed after the resources reported (in which case `error` carries the failure and `retryable` says whether waiting can fix it). `failed` means only the user can fix it: a rejected credential, a blocked connector, or a terminal cycle failure.", + "description": "List a connected provider's resources using the connector's stored credentials. Passing cursor or limit opts into pagination (currently Notion only): the response then adds next_cursor and has_more, a page may hold fewer than limit resources, and clients must continue while has_more is true. Without either param the full resource list is returned.", "parameters": [ { "description": "Connector ID", @@ -9881,6 +7748,22 @@ "example": "HydraDoc1234", "type": "string" } + }, + { + "description": "Opaque pagination cursor from a previous response's next_cursor", + "in": "query", + "name": "cursor", + "schema": { + "type": "string" + } + }, + { + "description": "Max resources per page, 1-100 (values above 100 are clamped)", + "in": "query", + "name": "limit", + "schema": { + "type": "integer" + } } ], "responses": { @@ -9888,12 +7771,22 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.connectorStatusResponse" + "$ref": "#/components/schemas/handler.discoverResponseBody" } } }, "description": "OK" }, + "400": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Bad Request" + }, "404": { "content": { "application/json": { @@ -9913,6 +7806,16 @@ } }, "description": "Internal Server Error" + }, + "502": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Bad Gateway" } }, "security": [ @@ -9920,17 +7823,17 @@ "BearerAuth": [] } ], - "summary": "Get a connector's health", + "summary": "Discover connector resources", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "status" + "x-fern-sdk-method-name": "discover" } }, - "/connectors/{id}/sync": { - "post": { - "description": "Start a manual sync for a connector.", + "/connectors/{id}/resources": { + "get": { + "description": "List the configured resources for a connector.", "parameters": [ { "description": "Connector ID", @@ -9944,15 +7847,15 @@ } ], "responses": { - "202": { + "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.connectorSyncResponse" + "$ref": "#/components/schemas/handler.connectorResourcesResponse" } } }, - "description": "Accepted" + "description": "OK" }, "404": { "content": { @@ -9964,16 +7867,6 @@ }, "description": "Not Found" }, - "409": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Conflict" - }, "500": { "content": { "application/json": { @@ -9983,16 +7876,6 @@ } }, "description": "Internal Server Error" - }, - "503": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Service Unavailable" } }, "security": [ @@ -10000,27 +7883,23 @@ "BearerAuth": [] } ], - "summary": "Trigger a connector sync", + "summary": "List connector resources", "tags": [ "connectors" ], "x-fern-sdk-group-name": "connectors", - "x-fern-sdk-method-name": "sync" - } - }, - "/context": { - "delete": { - "description": "Delete one or more knowledge sources or memories by ID.\n\nBy default this endpoint answers 200 for every outcome, including a delete\nthat removed nothing — check `data.deleted_count` and `data.results` rather\nthan the status code.\n\nSend `X-HydraDB-Delete-Status: strict` to opt in to honest status codes: a\ndelete that did not happen then answers 404/409/500 and never 200. This is\nthe recommended mode for new integrations. On those failures the response\n`data` still carries the same `results` / `deleted_count` payload a 200\ncarries, so per-id outcomes stay readable either way.\n\nThe default is expected to become strict in a future release, at which\npoint `X-HydraDB-Delete-Status: legacy` keeps the unconditional 200 for a\ncaller that is not ready.", + "x-fern-sdk-method-name": "list_resources" + }, + "post": { + "description": "Add a resource mapping to a connector.", "parameters": [ { - "description": "Selects the status behaviour for this request. `strict` opts in to honest 404/409/500 codes when the delete did not happen; `legacy` forces the unconditional 200. Omitted, the server default applies — currently `legacy`.", - "in": "header", - "name": "X-HydraDB-Delete-Status", + "description": "Connector ID", + "in": "path", + "name": "id", + "required": true, "schema": { - "enum": [ - "strict", - "legacy" - ], + "example": "HydraDoc1234", "type": "string" } } @@ -10029,23 +7908,23 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/sources.V2SourceDeleteRequest" + "$ref": "#/components/schemas/handler.resourceCreateReq" } } }, - "description": "Delete request", + "description": "Resource configuration", "required": true }, "responses": { - "200": { + "201": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-sources_MemoryDeleteResponse" + "$ref": "#/components/schemas/connectors.Resource" } } }, - "description": "OK" + "description": "Created" }, "400": { "content": { @@ -10061,31 +7940,21 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.ErrorEnvelope-sources_MemoryDeleteResponse" - } - } - }, - "description": "Strict mode only. No source matched the given ids; `data` carries the same results/deleted_count payload a 200 carries" - }, - "409": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorEnvelope-sources_MemoryDeleteResponse" + "$ref": "#/components/schemas/handler.ErrorResponse" } } }, - "description": "Strict mode only. Source is still indexing; retry after ingestion completes (see Retry-After). `data` carries the same results/deleted_count payload a 200 carries" + "description": "Not Found" }, "500": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.ErrorEnvelope-sources_MemoryDeleteResponse" + "$ref": "#/components/schemas/handler.ErrorResponse" } } }, - "description": "Strict mode only. A store failed to delete the source; the delete is retryable. `data` carries the same results/deleted_count payload a 200 carries" + "description": "Internal Server Error" } }, "security": [ @@ -10093,118 +7962,37 @@ "BearerAuth": [] } ], - "summary": "Delete documents or memories", + "summary": "Create a connector resource", "tags": [ - "context" + "connectors" ], - "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "delete" + "x-fern-sdk-group-name": "connectors", + "x-fern-sdk-method-name": "create_resource" } }, - "/context/chunks": { - "get": { - "description": "Return the indexed chunk text for a source, or for a specific set of chunk ids (the ids a graph relation cites as its evidence). Chunks whose source the request's principals may not see are omitted, and so are chunks whose source cannot be determined.", + "/connectors/{id}/resources/{resource_id}": { + "delete": { + "description": "Remove a resource mapping from a connector.", "parameters": [ { - "description": "Database ID.", - "in": "query", - "name": "database", - "required": true, - "schema": { - "example": "acme_corp", - "type": "string" - } - }, - { - "description": "Collection ID within the database. Defaults to the database's default collection.", - "in": "query", - "name": "collection", - "schema": { - "example": "team_docs", - "type": "string" - } - }, - { - "description": "Deprecated alias for database", - "in": "query", - "name": "tenant_id", - "schema": { - "deprecated": true, - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - } - }, - { - "description": "Deprecated alias for collection", - "in": "query", - "name": "sub_tenant_id", - "schema": { - "deprecated": true, - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated": "true" - } - }, - { - "description": "Source ID whose chunks to return. Required unless chunk_ids is given.", - "in": "query", + "description": "Connector ID", + "in": "path", "name": "id", + "required": true, "schema": { "example": "HydraDoc1234", "type": "string" } }, { - "description": "Chunk IDs to return. Repeated (chunk_ids=a\u0026chunk_ids=b) or comma-separated. Takes precedence over id.", - "in": "query", - "name": "chunk_ids", - "schema": { - "example": [ - "HydraEmbeddings123_0", - "HydraEmbeddings123_1" - ], - "items": { - "type": "string" - }, - "type": "array" - }, - "style": "form" - }, - { - "description": "Corpus type. Split database: 'knowledge' (default), 'memory', or 'all'. Unified database: 'unified' (default); knowledge/memory are rejected. This read addresses one corpus, so 'all' answers from knowledge and meta.source_type reports which corpus answered.", - "in": "query", - "name": "type", + "description": "Resource ID", + "in": "path", + "name": "resource_id", + "required": true, "schema": { - "enum": [ - "knowledge", - "memory", - "unified", - "all" - ], + "example": "C0123456789", "type": "string" } - }, - { - "description": "Max chunks to return", - "in": "query", - "name": "limit", - "schema": { - "default": 50, - "type": "integer" - } - }, - { - "description": "Principals to answer as: only chunks whose source they may see are returned. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", - "in": "query", - "name": "acl", - "schema": { - "items": { - "type": "string" - }, - "type": "array" - }, - "style": "form" } ], "responses": { @@ -10212,7 +8000,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-search_ChunkInspectResult" + "$ref": "#/components/schemas/handler.resourceDeleteResponse" } } }, @@ -10227,6 +8015,26 @@ } }, "description": "Bad Request" + }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Not Found" + }, + "500": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Internal Server Error" } }, "security": [ @@ -10234,117 +8042,57 @@ "BearerAuth": [] } ], - "summary": "Get chunk text", + "summary": "Delete a connector resource", "tags": [ - "context" + "connectors" ], - "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "chunks" - } - }, - "/context/ingest": { - "post": { - "description": "Ingest content for a database. `items` is the unified shape (text or a conversation per item) and the only one a unified database accepts; a split database still takes the deprecated `documents`, `app_knowledge` and `memories` fields selected by `type`. The same `items` array may also be sent as an application/json body.", - "requestBody": { - "content": { - "multipart/form-data": { - "schema": { - "properties": { - "app_knowledge": { - "deprecated": true, - "description": "App-knowledge items as a JSON array (type=knowledge). Per item, `metadata` is capped at 16 KiB and `additional_metadata` at 1 KiB, measured on the compact JSON encoding of the whole map in UTF-8 bytes (keys and punctuation count). The deprecated `tenant_metadata` / `document_metadata` spellings are accepted here and held to the same caps. Over-cap returns 400 with the actual byte count. Each item may also carry `acl`, a list of principals (`user_email:\u003cemail\u003e`, a bare email, `group:\u003cprovider\u003e:\u003cid\u003e`, `domain:\u003cdomain\u003e`, or `__public__`) restricting who may retrieve it; omit it to leave the document unrestricted, and send an empty list to restrict it to nobody. A malformed principal rejects the whole request with 400. Items may also carry `evidence_kind`/`evidence_subject` provenance labels (see document_metadata); an unknown kind returns 400.", - "title": "app_knowledge", - "type": "string", - "x-deprecated": "true" - }, - "collection": { - "title": "collection", - "type": "string" - }, - "database": { - "title": "database", - "type": "string" - }, - "document_metadata": { - "deprecated": true, - "description": "Per-document metadata as a JSON array (type=knowledge). Per item, `metadata` is capped at 16 KiB and `additional_metadata` at 1 KiB. Both caps are measured on the compact JSON encoding of the whole map in UTF-8 bytes, so keys, quotes, commas and braces count toward the budget. Over-cap returns 400 with the actual byte count. Each item may also carry evidence labels (`evidence_kind`: one of assertion, record, said, done, third_party, inferred; `evidence_subject`: a stable handle for who the evidence is about, e.g. `user:kiran@acme.com`) declaring the content's provenance for entity understanding; an unknown kind returns 400.", - "title": "document_metadata", - "type": "string", - "x-deprecated": "true" - }, - "documents": { - "deprecated": true, - "items": { - "format": "binary", - "type": "string" - }, - "title": "documents", - "type": "array", - "x-deprecated": "true" - }, - "graph_payload": { - "title": "graph_payload", - "type": "string" - }, - "items": { - "description": "Items to ingest, as a JSON-encoded array. Each item is either `text` or a `conversation`. The only ingest shape a unified database accepts. The same array can also be sent as an application/json request body.", - "title": "items", - "type": "string" - }, - "memories": { - "deprecated": true, - "description": "Memory items as a JSON array (type=memory). Per item, `metadata` is capped at 16 KiB and `additional_metadata` at 1 KiB, measured on the compact JSON encoding of the whole map in UTF-8 bytes (keys and punctuation count). Over-cap returns 400 with the actual byte count. Items may also carry `evidence_kind`/`evidence_subject` provenance labels (see document_metadata); an unknown kind returns 400.", - "title": "memories", - "type": "string", - "x-deprecated": "true" - }, - "sub_tenant_id": { - "deprecated": true, - "title": "sub_tenant_id", - "type": "string", - "x-deprecated": "true" - }, - "tenant_id": { - "deprecated": true, - "title": "tenant_id", - "type": "string", - "x-deprecated": "true" - }, - "type": { - "enum": [ - "knowledge", - "memory", - "unified" - ], - "title": "type", - "type": "string" - }, - "upsert": { - "default": "true", - "title": "upsert", - "type": "string" - } - }, - "required": [ - "database" - ], + "x-fern-sdk-group-name": "connectors", + "x-fern-sdk-method-name": "delete_resource" + }, + "patch": { + "description": "Updates per-resource settings. `acl` sets the customer-declared ACL for one connector resource (PRO-1684): the rule is normalized (emails prefixed, __public__ dominates, explicit-empty becomes __private__), persisted on the resource, and applied to enforcement immediately via the resource's ACL row, every already-indexed document of the resource is governed by it on the next query, with no re-sync. For providers with provider-derived ACL capture enabled, the provider's own ACL takes precedence again at the next sync; the rule is the standing fallback. `custom_instructions` sets the resource-level ingestion-instructions override (max 4000 characters): when set it replaces the connector-level custom_instructions for documents synced from this resource, an explicit empty string clears the override back to inheriting the connector's value, and changes apply from the next sync cycle. At least one field must be supplied; omitted fields are left unchanged.", + "parameters": [ + { + "description": "Connector ID", + "in": "path", + "name": "id", + "required": true, + "schema": { + "example": "HydraDoc1234", + "type": "string" + } + }, + { + "description": "Resource ID", + "in": "path", + "name": "resource_id", + "required": true, + "schema": { + "example": "C0123456789", + "type": "string" + } + } + ], + "requestBody": { + "content": { + "application/json": { + "schema": { "type": "object" } } - }, - "description": "Unified items[] body: the application/json alternative to this form. | Corpus to write to. Split database: 'knowledge' (default) or 'memory'. Unified database: 'unified' (default); knowledge/memory are rejected. 'all' is refused here: an ingest must name the one corpus it writes to. | Items as a JSON array; each item carries `text` or `conversation` (role/content turns), optional `context_id`, `title`, `attributes`, `custom_attributes`, `happened_at`, `enrich`, `context_category` (auto|user_preference|business_knowledge|decision_trace, unified databases only), and `acl` (principals allowed to retrieve the item; omit for unrestricted). Required on a unified database; on a split database items land in the memory corpus. | Database ID. | Collection ID within the database. Defaults to the database's default collection. | Deprecated alias for database | Deprecated alias for collection | Upsert existing content (true/false/1/0) | Deprecated: knowledge files to ingest (repeatable; type=knowledge, split databases only) | Deprecated: per-document metadata as a JSON array (type=knowledge, split databases only). Per item: metadata \u003c= 16 KiB, additional_metadata \u003c= 1 KiB. | Deprecated: app-knowledge items as a JSON array (type=knowledge, split databases only). Per item: metadata \u003c= 16 KiB, additional_metadata \u003c= 1 KiB, optional acl principal list. | Deprecated: memory items as a JSON array (type=memory, split databases only); use items. Per item: metadata \u003c= 16 KiB, additional_metadata \u003c= 1 KiB. | Optional bring-your-own-graph payload as JSON", - "required": true + } }, "responses": { - "202": { + "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-ingestion_V2IngestResponse" + "additionalProperties": {}, + "type": "object" } } }, - "description": "Accepted" + "description": "OK" }, "400": { "content": { @@ -10356,17 +8104,7 @@ }, "description": "Bad Request" }, - "413": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Request Entity Too Large" - }, - "415": { + "404": { "content": { "application/json": { "schema": { @@ -10374,9 +8112,9 @@ } } }, - "description": "Body is neither multipart/form-data nor application/json" + "description": "Not Found" }, - "422": { + "500": { "content": { "application/json": { "schema": { @@ -10384,7 +8122,7 @@ } } }, - "description": "Unprocessable Entity" + "description": "Internal Server Error" } }, "security": [ @@ -10392,98 +8130,27 @@ "BearerAuth": [] } ], - "summary": "Ingest content", + "summary": "Update a resource's ACL rule or custom instructions", "tags": [ - "context" + "connectors" ], - "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "ingest" + "x-fern-sdk-group-name": "connectors", + "x-fern-sdk-method-name": "update_resource_acl" } }, - "/context/inspect": { + "/connectors/{id}/status": { "get": { - "description": "Fetch a previously ingested source's content, inferred content, and a downloadable URL.", + "description": "Report whether a connector is working, in one call: a rollup status (healthy, degraded, failed, checking) plus per-resource detail. `degraded` means the connector is syncing but at least one configured resource is failing — the state that is otherwise invisible, because a connector whose resources partly fail still reports an idle sync status and no error.", "parameters": [ { - "description": "Source ID", - "in": "query", + "description": "Connector ID", + "in": "path", "name": "id", "required": true, "schema": { "example": "HydraDoc1234", "type": "string" } - }, - { - "description": "Database ID.", - "in": "query", - "name": "database", - "required": true, - "schema": { - "example": "acme_corp", - "type": "string" - } - }, - { - "description": "Collection ID within the database. Defaults to the database's default collection.", - "in": "query", - "name": "collection", - "schema": { - "example": "team_docs", - "type": "string" - } - }, - { - "description": "Deprecated alias for database", - "in": "query", - "name": "tenant_id", - "schema": { - "deprecated": true, - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - } - }, - { - "description": "Deprecated alias for collection", - "in": "query", - "name": "sub_tenant_id", - "schema": { - "deprecated": true, - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated": "true" - } - }, - { - "description": "Presigned URL expiry in seconds", - "in": "query", - "name": "expiry_seconds", - "schema": { - "default": 3600, - "type": "integer" - } - }, - { - "description": "Fetch mode", - "in": "query", - "name": "mode", - "schema": { - "example": "thinking", - "type": "string" - } - }, - { - "description": "Principals to answer as: the source must be visible to them, or the response is 404. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", - "in": "query", - "name": "acl", - "schema": { - "items": { - "type": "string" - }, - "type": "array" - }, - "style": "form" } ], "responses": { @@ -10491,13 +8158,13 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-fetch_V2SourceFetchResponse" + "$ref": "#/components/schemas/handler.connectorStatusResponse" } } }, "description": "OK" }, - "400": { + "404": { "content": { "application/json": { "schema": { @@ -10505,9 +8172,9 @@ } } }, - "description": "Bad Request" + "description": "Not Found" }, - "404": { + "500": { "content": { "application/json": { "schema": { @@ -10515,7 +8182,7 @@ } } }, - "description": "Not Found" + "description": "Internal Server Error" } }, "security": [ @@ -10523,40 +8190,41 @@ "BearerAuth": [] } ], - "summary": "Fetch document content", + "summary": "Get a connector's health", "tags": [ - "context" + "connectors" ], - "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "inspect" + "x-fern-sdk-group-name": "connectors", + "x-fern-sdk-method-name": "status" } }, - "/context/list": { + "/connectors/{id}/sync": { "post": { - "description": "List items (id + metadata) for a database. Split database: knowledge sources (default) or memories, selected by `type`. Unified database: every item in one page (type=unified, the default).", - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/list.V2ListContentRequest" - } + "description": "Start a manual sync workflow for a connector.", + "parameters": [ + { + "description": "Connector ID", + "in": "path", + "name": "id", + "required": true, + "schema": { + "example": "HydraDoc1234", + "type": "string" } - }, - "description": "List request", - "required": true - }, + } + ], "responses": { - "200": { + "202": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-list_V2ListResponse" + "$ref": "#/components/schemas/handler.connectorSyncResponse" } } }, - "description": "OK" + "description": "Accepted" }, - "400": { + "404": { "content": { "application/json": { "schema": { @@ -10564,116 +8232,86 @@ } } }, - "description": "Bad Request" - } - }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "List documents", - "tags": [ - "context" - ], - "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "list" - } - }, - "/context/profile": { - "get": { - "description": "Return the compiled profile of one entity in one collection: identity headline, a cited summary, and the current admitted entries (each pointing at the statements behind it). Profiles are kept up to date as context is ingested; this endpoint returns the current profile and never triggers a rebuild.", - "parameters": [ - { - "description": "Database ID.", - "in": "query", - "name": "database", - "required": true, - "schema": { - "example": "acme_corp", - "type": "string" - } - }, - { - "description": "Collection ID within the database.", - "in": "query", - "name": "collection", - "required": true, - "schema": { - "example": "team_docs", - "type": "string" - } - }, - { - "description": "Deprecated alias for database", - "in": "query", - "name": "tenant_id", - "schema": { - "deprecated": true, - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - } - }, - { - "description": "Deprecated alias for collection", - "in": "query", - "name": "sub_tenant_id", - "schema": { - "deprecated": true, - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated": "true" - } + "description": "Not Found" }, - { - "description": "Entity whose profile to return (e.g. a person's name)", - "in": "query", - "name": "subject", - "required": true, - "schema": { - "type": "string" - } + "409": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Conflict" }, - { - "description": "Corpus type: 'knowledge' or 'memory'", - "in": "query", - "name": "type", - "schema": { - "default": "memory", - "enum": [ - "knowledge", - "memory" - ], - "type": "string" - } + "500": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Internal Server Error" }, + "503": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Service Unavailable" + } + }, + "security": [ { - "description": "Graph entity type of the subject", - "in": "query", - "name": "entity_type", - "schema": { - "default": "PERSON", - "type": "string" - } - }, + "BearerAuth": [] + } + ], + "summary": "Trigger a connector sync", + "tags": [ + "connectors" + ], + "x-fern-sdk-group-name": "connectors", + "x-fern-sdk-method-name": "sync" + } + }, + "/context": { + "delete": { + "description": "Delete one or more knowledge sources or memories by ID.\n\nBy default this endpoint answers 200 for every outcome, including a delete\nthat removed nothing — check `data.deleted_count` and `data.results` rather\nthan the status code.\n\nSend `X-HydraDB-Delete-Status: strict` to opt in to honest status codes: a\ndelete that did not happen then answers 404/409/500 and never 200. This is\nthe recommended mode for new integrations. On those failures the response\n`data` still carries the same `results` / `deleted_count` payload a 200\ncarries, so per-id outcomes stay readable either way.\n\nThe default is expected to become strict in a future release, at which\npoint `X-HydraDB-Delete-Status: legacy` keeps the unconditional 200 for a\ncaller that is not ready.", + "parameters": [ { - "description": "Graph namespace of the subject", - "in": "query", - "name": "namespace", + "description": "Selects the status behaviour for this request. `strict` opts in to honest 404/409/500 codes when the delete did not happen; `legacy` forces the unconditional 200. Omitted, the server default applies — currently `legacy`.", + "in": "header", + "name": "X-HydraDB-Delete-Status", "schema": { - "default": "users", - "example": "organization", + "enum": [ + "strict", + "legacy" + ], "type": "string" } } ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/sources.V2SourceDeleteRequest" + } + } + }, + "description": "Delete request", + "required": true + }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-search_EntityProfileView" + "$ref": "#/components/schemas/handler.Envelope-sources_MemoryDeleteResponse" } } }, @@ -10693,11 +8331,31 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" + "$ref": "#/components/schemas/handler.ErrorEnvelope-sources_MemoryDeleteResponse" + } + } + }, + "description": "Strict mode only. No source matched the given ids; `data` carries the same results/deleted_count payload a 200 carries" + }, + "409": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorEnvelope-sources_MemoryDeleteResponse" } } }, - "description": "No profile compiled for this subject yet, or the feature is not enabled" + "description": "Strict mode only. Source is still indexing; retry after ingestion completes (see Retry-After). `data` carries the same results/deleted_count payload a 200 carries" + }, + "500": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorEnvelope-sources_MemoryDeleteResponse" + } + } + }, + "description": "Strict mode only. A store failed to delete the source; the delete is retryable. `data` carries the same results/deleted_count payload a 200 carries" } }, "security": [ @@ -10705,20 +8363,20 @@ "BearerAuth": [] } ], - "summary": "Get entity profile", + "summary": "Delete documents or memories", "tags": [ "context" ], "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "profile" + "x-fern-sdk-method-name": "delete" } }, - "/context/relations": { + "/context/chunks": { "get": { - "description": "Return knowledge-graph relations for a database or a single source.", + "description": "Return the indexed chunk text for a source, or for a specific set of chunk ids (the ids a graph relation cites as its evidence). Chunk rows are read from the document store first and from the vector store for anything it does not hold. Chunks whose source the request's principals may not see are omitted, and so are chunks whose source cannot be established.", "parameters": [ { - "description": "Database ID.", + "description": "Database (canonical name for the tenant scope)", "in": "query", "name": "database", "required": true, @@ -10728,7 +8386,7 @@ } }, { - "description": "Collection ID within the database. Defaults to the database's default collection.", + "description": "Collection (canonical name for the sub-tenant scope)", "in": "query", "name": "collection", "schema": { @@ -10759,7 +8417,7 @@ } }, { - "description": "Source ID (omit for database-wide relations)", + "description": "Source ID whose chunks to return. Required unless chunk_ids is given.", "in": "query", "name": "id", "schema": { @@ -10768,38 +8426,44 @@ } }, { - "description": "Corpus type. Split database: 'knowledge' (default), 'memory', or 'all'. Unified database: 'unified' (default); knowledge/memory are rejected. This read addresses one corpus, so 'all' answers from knowledge and meta.source_type reports which corpus answered.", + "description": "Chunk IDs to return. Repeated (chunk_ids=a\u0026chunk_ids=b) or comma-separated. Takes precedence over id.", + "in": "query", + "name": "chunk_ids", + "schema": { + "example": [ + "HydraEmbeddings123_0", + "HydraEmbeddings123_1" + ], + "items": { + "type": "string" + }, + "type": "array" + }, + "style": "form" + }, + { + "description": "Corpus type: 'knowledge' or 'memory'", "in": "query", "name": "type", "schema": { "enum": [ "knowledge", - "memory", - "unified", - "all" + "memory" ], "type": "string" } }, { - "description": "Max relations to return", + "description": "Max chunks to return", "in": "query", "name": "limit", "schema": { - "default": 5000, + "default": 50, "type": "integer" } }, { - "description": "Pagination cursor", - "in": "query", - "name": "cursor", - "schema": { - "type": "number" - } - }, - { - "description": "Principals to answer as: only relations from sources they may see are returned. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", + "description": "Principals to answer as (PRO-1684 document ACLs): only chunks whose source they may see are returned. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", "in": "query", "name": "acl", "schema": { @@ -10816,7 +8480,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-graph_GraphRelationsResponse" + "$ref": "#/components/schemas/handler.Envelope-search_ChunkInspectResult" } } }, @@ -10838,96 +8502,133 @@ "BearerAuth": [] } ], - "summary": "Get graph relations", + "summary": "Get chunk text", "tags": [ "context" ], "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "relations" + "x-fern-sdk-method-name": "chunks" } }, - "/context/status": { - "get": { - "description": "Return the processing status for one or more source IDs.", - "parameters": [ - { - "description": "Single source ID", - "in": "query", - "name": "id", - "schema": { - "example": "HydraDoc1234", - "type": "string" - } - }, - { - "description": "One or more source IDs", - "in": "query", - "name": "ids", - "schema": { - "example": [ - "HydraDoc1234", - "HydraDoc4567" - ], - "items": { - "type": "string" - }, - "type": "array" + "/context/ingest": { + "post": { + "description": "Ingest knowledge documents or memories for a tenant.", + "requestBody": { + "content": { + "multipart/form-data": { + "schema": { + "properties": { + "app_knowledge": { + "description": "App-knowledge items as a JSON array (type=knowledge). Per item, `metadata` is capped at 16 KiB and `additional_metadata` at 1 KiB, measured on the compact JSON encoding of the whole map in UTF-8 bytes (keys and punctuation count). The deprecated `tenant_metadata` / `document_metadata` spellings are accepted here and held to the same caps. Over-cap returns 400 with the actual byte count. Each item may also carry `acl`, a list of principals (`user_email:\u003cemail\u003e`, a bare email, `group:\u003cprovider\u003e:\u003cid\u003e`, `domain:\u003cdomain\u003e`, or `__public__`) restricting who may retrieve it; omit it to leave the document unrestricted, and send an empty list to restrict it to nobody. A malformed principal rejects the whole request with 400.", + "title": "app_knowledge", + "type": "string" + }, + "collection": { + "title": "collection", + "type": "string" + }, + "database": { + "title": "database", + "type": "string" + }, + "document_metadata": { + "description": "Per-document metadata as a JSON array (type=knowledge). Per item, `metadata` is capped at 16 KiB and `additional_metadata` at 1 KiB. Both caps are measured on the compact JSON encoding of the whole map in UTF-8 bytes, so keys, quotes, commas and braces count toward the budget. Over-cap returns 400 with the actual byte count.", + "title": "document_metadata", + "type": "string" + }, + "documents": { + "format": "binary", + "title": "documents", + "type": "string" + }, + "graph_payload": { + "title": "graph_payload", + "type": "string" + }, + "memories": { + "description": "Memory items as a JSON array (type=memory). Per item, `metadata` is capped at 16 KiB and `additional_metadata` at 1 KiB, measured on the compact JSON encoding of the whole map in UTF-8 bytes (keys and punctuation count). Over-cap returns 400 with the actual byte count.", + "title": "memories", + "type": "string" + }, + "sub_tenant_id": { + "deprecated": true, + "title": "sub_tenant_id", + "type": "string", + "x-deprecated": "true" + }, + "tenant_id": { + "deprecated": true, + "title": "tenant_id", + "type": "string", + "x-deprecated": "true" + }, + "type": { + "default": "knowledge", + "enum": [ + "knowledge", + "memory" + ], + "title": "type", + "type": "string" + }, + "upsert": { + "default": "true", + "title": "upsert", + "type": "string" + } + }, + "required": [ + "database" + ], + "type": "object" + } } }, - { - "description": "Database ID.", - "in": "query", - "name": "database", - "required": true, - "schema": { - "example": "acme_corp", - "type": "string" - } + "description": "Content type: 'knowledge' or 'memory' | Database (canonical name for the tenant scope) | Collection (canonical name for the sub-tenant scope) | Deprecated alias for database | Deprecated alias for collection | Upsert existing content (true/false/1/0) | Knowledge files to ingest (repeatable; type=knowledge) | Per-document metadata as a JSON array (type=knowledge). Per item: metadata \u003c= 16 KiB, additional_metadata \u003c= 1 KiB. | App-knowledge items as a JSON array (type=knowledge). Per item: metadata \u003c= 16 KiB, additional_metadata \u003c= 1 KiB, optional acl principal list (PRO-1684). | Memory items as a JSON array (type=memory). Per item: metadata \u003c= 16 KiB, additional_metadata \u003c= 1 KiB. | Optional bring-your-own-graph payload as JSON", + "required": true + }, + "responses": { + "202": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.Envelope-ingestion_V2IngestResponse" + } + } + }, + "description": "Accepted" }, - { - "description": "Collection ID within the database. Defaults to the database's default collection.", - "in": "query", - "name": "collection", - "schema": { - "example": "team_docs", - "type": "string" - } + "400": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Bad Request" }, - { - "description": "Deprecated alias for database", - "in": "query", - "name": "tenant_id", - "schema": { - "deprecated": true, - "example": "tenant_1234", - "type": "string", - "x-deprecated": "true" - } + "413": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Request Entity Too Large" }, - { - "description": "Deprecated alias for collection", - "in": "query", - "name": "sub_tenant_id", - "schema": { - "deprecated": true, - "example": "sub_tenant_4567", - "type": "string", - "x-deprecated": "true" - } - } - ], - "responses": { - "200": { + "415": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-ingestion_V2BatchProcessingStatus" + "$ref": "#/components/schemas/handler.ErrorResponse" } } }, - "description": "OK" + "description": "Body is not multipart/form-data (e.g. a JSON body)" }, - "400": { + "422": { "content": { "application/json": { "schema": { @@ -10935,7 +8636,7 @@ } } }, - "description": "Bad Request" + "description": "Unprocessable Entity" } }, "security": [ @@ -10943,20 +8644,20 @@ "BearerAuth": [] } ], - "summary": "Check processing status", + "summary": "Ingest content", "tags": [ "context" ], "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "status" + "x-fern-sdk-method-name": "ingest" } }, - "/context/subgraph": { + "/context/inspect": { "get": { - "description": "Query-string form of GET /context/{id}/subgraph: the same parameters, the same response, and the same rules. It exists for an id that contains '/', which cannot be spelled as one path segment; generated SDKs call this form for every id. Return the connected subgraph of one ingested item: every item reachable from it through item-level relations (explicit `relates_to` links, a shared thread, parent/child hierarchy, traversed breadth-first up to `depth` hops), the relations among those members, and the structural graph around them (entities, comments, attachments, actors). Chunk-level entity relations are not included; use Inspecting Context Relations for those. An unknown id returns an empty subgraph, not an error.", + "description": "Fetch a previously ingested source's content, inferred content, and a downloadable URL.", "parameters": [ { - "description": "Item ID: the ingested item whose connected subgraph to return. This form takes any id, including one that contains '/'.", + "description": "Source ID", "in": "query", "name": "id", "required": true, @@ -10966,7 +8667,7 @@ } }, { - "description": "Database ID.", + "description": "Database (canonical name for the tenant scope)", "in": "query", "name": "database", "required": true, @@ -10976,7 +8677,7 @@ } }, { - "description": "Collection ID within the database. Defaults to the database's default collection.", + "description": "Collection (canonical name for the sub-tenant scope)", "in": "query", "name": "collection", "schema": { @@ -11007,43 +8708,25 @@ } }, { - "description": "Corpus type. Split database: 'knowledge' (default), 'memory', or 'all'. Unified database: 'unified' (default); knowledge/memory are rejected. This read addresses one corpus, so 'all' answers from knowledge and meta.source_type reports which corpus answered.", - "in": "query", - "name": "type", - "schema": { - "enum": [ - "knowledge", - "memory", - "unified", - "all" - ], - "type": "string" - } - }, - { - "description": "Max traversal depth in hops", + "description": "Presigned URL expiry in seconds", "in": "query", - "name": "depth", + "name": "expiry_seconds", "schema": { - "default": 5, - "maximum": 10, - "minimum": 1, + "default": 3600, "type": "integer" } }, { - "description": "Max members returned; `is_truncated` reports when this clipped the traversal", + "description": "Fetch mode", "in": "query", - "name": "max_sources", + "name": "mode", "schema": { - "default": 200, - "maximum": 1000, - "minimum": 1, - "type": "integer" + "example": "thinking", + "type": "string" } }, { - "description": "Principals to answer as (document ACLs): the subgraph contains only items they may see, filtered at every hop. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", + "description": "Principals to answer as (PRO-1684 document ACLs): the source must be visible to them, or the response is 404. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", "in": "query", "name": "acl", "schema": { @@ -11060,7 +8743,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-graph_SourceSubgraphResponse" + "$ref": "#/components/schemas/handler.Envelope-fetch_V2SourceFetchResponse" } } }, @@ -11075,6 +8758,16 @@ } }, "description": "Bad Request" + }, + "404": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Not Found" } }, "security": [ @@ -11082,38 +8775,26 @@ "BearerAuth": [] } ], - "summary": "Get connected subgraph", + "summary": "Fetch document content", "tags": [ "context" ], "x-fern-sdk-group-name": "context", - "x-fern-sdk-method-name": "subgraph" + "x-fern-sdk-method-name": "inspect" } }, - "/context/{id}/metadata": { - "patch": { - "description": "Merge/upsert database_metadata and additional_metadata for one source. collection is required.", - "parameters": [ - { - "description": "Source ID", - "in": "path", - "name": "id", - "required": true, - "schema": { - "example": "HydraDoc1234", - "type": "string" - } - } - ], + "/context/list": { + "post": { + "description": "List knowledge sources or memories (id + metadata) for a tenant.", "requestBody": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.contextMetadataUpdateRequest" + "$ref": "#/components/schemas/list.V2ListContentRequest" } } }, - "description": "Metadata update request", + "description": "List request", "required": true }, "responses": { @@ -11121,7 +8802,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-github_com_hydradb_hydradb-application_internal_service_MetadataEditResult" + "$ref": "#/components/schemas/handler.Envelope-list_V2ListResponse" } } }, @@ -11136,26 +8817,6 @@ } }, "description": "Bad Request" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" - }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" } }, "security": [ @@ -11163,29 +8824,20 @@ "BearerAuth": [] } ], - "summary": "Update source metadata", + "summary": "List documents", "tags": [ "context" ], - "x-fern-sdk-group-name": "context" + "x-fern-sdk-group-name": "context", + "x-fern-sdk-method-name": "list" } }, - "/context/{id}/subgraph": { + "/context/relations": { "get": { - "description": "Return the connected subgraph of one ingested item: every item reachable from it through item-level relations (explicit `relates_to` links, a shared thread, parent/child hierarchy, traversed breadth-first up to `depth` hops), the relations among those members, and the structural graph around them (entities, comments, attachments, actors). Chunk-level entity relations are not included; use Inspecting Context Relations for those. An unknown id returns an empty subgraph, not an error.", + "description": "Return knowledge-graph relations for a tenant or a single source.", "parameters": [ { - "description": "Item ID: the ingested item whose connected subgraph to return. URL-encode it. An id containing a literal '/' cannot be spelled as one path segment; address those with the query form, GET /context/subgraph?id=.", - "in": "path", - "name": "id", - "required": true, - "schema": { - "example": "HydraDoc1234", - "type": "string" - } - }, - { - "description": "Database ID.", + "description": "Database (canonical name for the tenant scope)", "in": "query", "name": "database", "required": true, @@ -11195,7 +8847,7 @@ } }, { - "description": "Collection ID within the database. Defaults to the database's default collection.", + "description": "Collection (canonical name for the sub-tenant scope)", "in": "query", "name": "collection", "schema": { @@ -11226,152 +8878,68 @@ } }, { - "description": "Corpus type. Split database: 'knowledge' (default), 'memory', or 'all'. Unified database: 'unified' (default); knowledge/memory are rejected. This read addresses one corpus, so 'all' answers from knowledge and meta.source_type reports which corpus answered.", + "description": "Source ID (omit for database-wide relations)", "in": "query", - "name": "type", + "name": "id", "schema": { - "enum": [ - "knowledge", - "memory", - "unified", - "all" - ], + "example": "HydraDoc1234", "type": "string" } }, { - "description": "Max traversal depth in hops", + "description": "Corpus type: 'knowledge' or 'memory'", "in": "query", - "name": "depth", + "name": "type", "schema": { - "default": 5, - "maximum": 10, - "minimum": 1, - "type": "integer" + "enum": [ + "knowledge", + "memory" + ], + "type": "string" } }, { - "description": "Max members returned; `is_truncated` reports when this clipped the traversal", + "description": "Max relations to return", "in": "query", - "name": "max_sources", + "name": "limit", "schema": { - "default": 200, - "maximum": 1000, - "minimum": 1, + "default": 5000, "type": "integer" } }, { - "description": "Principals to answer as (document ACLs): the subgraph contains only items they may see, filtered at every hop. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", + "description": "Pagination cursor", "in": "query", - "name": "acl", - "schema": { - "items": { - "type": "string" - }, - "type": "array" - }, - "style": "form" - } - ], - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.Envelope-graph_SourceSubgraphResponse" - } - } - }, - "description": "OK" - }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Request" - } - }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "Get connected subgraph", - "tags": [ - "context" - ], - "x-fern-ignore": true - } - }, - "/credential-vault": { - "get": { - "description": "List metadata and field names for credentials already used by connectors in the current workspace.", - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.vaultCredentialListResponse" - } - } - }, - "description": "OK" - } - }, - "security": [ - { - "BearerAuth": [] - } - ], - "summary": "List connector credentials", - "tags": [ - "connectors" - ] - } - }, - "/credential-vault/{id}": { - "patch": { - "description": "Update fields on the credential currently used by a connector. Owner-only human action.", - "parameters": [ - { - "description": "Connector ID", - "in": "path", - "name": "id", - "required": true, - "schema": { - "example": "HydraDoc1234", - "type": "string" - } - } - ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.vaultCredentialUpdateReq" - } + "name": "cursor", + "schema": { + "type": "number" } }, - "description": "Credential fields", - "required": true - }, + { + "description": "Principals to answer as (PRO-1684 document ACLs): only relations attributable to sources they may see are returned. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", + "in": "query", + "name": "acl", + "schema": { + "items": { + "type": "string" + }, + "type": "array" + }, + "style": "form" + } + ], "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.vaultCredentialUpdateResponse" + "$ref": "#/components/schemas/handler.Envelope-graph_GraphRelationsResponse" } } }, "description": "OK" }, - "404": { + "400": { "content": { "application/json": { "schema": { @@ -11379,7 +8947,7 @@ } } }, - "description": "Not Found" + "description": "Bad Request" } }, "security": [ @@ -11387,50 +8955,96 @@ "BearerAuth": [] } ], - "summary": "Update connector credentials", + "summary": "Get graph relations", "tags": [ - "connectors" - ] + "context" + ], + "x-fern-sdk-group-name": "context", + "x-fern-sdk-method-name": "relations" } }, - "/credential-vault/{id}/reveal": { - "post": { - "description": "Reveal one field from the credential currently used by a connector in the workspace. The response must never be cached.", + "/context/status": { + "get": { + "description": "Return the processing status for one or more source IDs.", "parameters": [ { - "description": "Connector ID", - "in": "path", + "description": "Single source ID", + "in": "query", "name": "id", - "required": true, "schema": { "example": "HydraDoc1234", "type": "string" } - } - ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.vaultCredentialRevealReq" - } + }, + { + "description": "One or more source IDs", + "in": "query", + "name": "ids", + "schema": { + "example": [ + "HydraDoc1234", + "HydraDoc4567" + ], + "items": { + "type": "string" + }, + "type": "array" } }, - "description": "Credential field", - "required": true - }, + { + "description": "Database (canonical name for the tenant scope)", + "in": "query", + "name": "database", + "required": true, + "schema": { + "example": "acme_corp", + "type": "string" + } + }, + { + "description": "Collection (canonical name for the sub-tenant scope)", + "in": "query", + "name": "collection", + "schema": { + "example": "team_docs", + "type": "string" + } + }, + { + "description": "Deprecated alias for database", + "in": "query", + "name": "tenant_id", + "schema": { + "deprecated": true, + "example": "tenant_1234", + "type": "string", + "x-deprecated": "true" + } + }, + { + "description": "Deprecated alias for collection", + "in": "query", + "name": "sub_tenant_id", + "schema": { + "deprecated": true, + "example": "sub_tenant_4567", + "type": "string", + "x-deprecated": "true" + } + } + ], "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.vaultCredentialRevealResponse" + "$ref": "#/components/schemas/handler.Envelope-ingestion_V2BatchProcessingStatus" } } }, "description": "OK" }, - "404": { + "400": { "content": { "application/json": { "schema": { @@ -11438,7 +9052,7 @@ } } }, - "description": "Not Found" + "description": "Bad Request" } }, "security": [ @@ -11446,33 +9060,46 @@ "BearerAuth": [] } ], - "summary": "Reveal a connector credential value", + "summary": "Check processing status", "tags": [ - "connectors" - ] + "context" + ], + "x-fern-sdk-group-name": "context", + "x-fern-sdk-method-name": "status" } }, - "/databases": { - "delete": { - "description": "Delete a database and all associated data", + "/context/{id}/metadata": { + "patch": { + "description": "Merge/upsert database_metadata and additional_metadata for one source. collection is required.", "parameters": [ { - "description": "Database identifier to delete", - "in": "query", - "name": "database", + "description": "Source ID", + "in": "path", + "name": "id", "required": true, "schema": { - "example": "acme_corp", + "example": "HydraDoc1234", "type": "string" } } ], + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.contextMetadataUpdateRequest" + } + } + }, + "description": "Metadata update request", + "required": true + }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_TenantDeleteResponse" + "$ref": "#/components/schemas/handler.Envelope-github_com_hydradb_hydradb-application_internal_service_MetadataEditResult" } } }, @@ -11514,68 +9141,122 @@ "BearerAuth": [] } ], - "summary": "Delete a database", + "summary": "Update source metadata", "tags": [ - "database-management" + "context" ], - "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "delete" - }, + "x-fern-sdk-group-name": "context" + } + }, + "/context/{id}/subgraph": { "get": { - "description": "List all databases for the authenticated user", - "responses": { - "200": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_TenantIdsResponse" - } - } - }, - "description": "OK" + "description": "Return the connected subgraph of one ingested item: every item reachable from it through item-level relations (explicit `relates_to` links, a shared thread, parent/child hierarchy, traversed breadth-first up to `depth` hops), the relations among those members, and the structural graph around them (entities, comments, attachments, actors). Chunk-level entity relations are not included; use Inspecting Context Relations for those. An unknown id returns an empty subgraph, not an error.", + "parameters": [ + { + "description": "Item ID: the ingested item whose connected subgraph to return. URL-encode it. An id containing a literal '/' cannot be spelled as one path segment; address those with the query form, GET /context/subgraph?id=.", + "in": "path", + "name": "id", + "required": true, + "schema": { + "example": "HydraDoc1234", + "type": "string" + } + }, + { + "description": "Database (canonical name for the tenant scope)", + "in": "query", + "name": "database", + "required": true, + "schema": { + "example": "acme_corp", + "type": "string" + } + }, + { + "description": "Collection (canonical name for the sub-tenant scope)", + "in": "query", + "name": "collection", + "schema": { + "example": "team_docs", + "type": "string" + } + }, + { + "description": "Deprecated alias for database", + "in": "query", + "name": "tenant_id", + "schema": { + "deprecated": true, + "example": "tenant_1234", + "type": "string", + "x-deprecated": "true" + } + }, + { + "description": "Deprecated alias for collection", + "in": "query", + "name": "sub_tenant_id", + "schema": { + "deprecated": true, + "example": "sub_tenant_4567", + "type": "string", + "x-deprecated": "true" + } + }, + { + "description": "Corpus type: 'knowledge' or 'memory'", + "in": "query", + "name": "type", + "schema": { + "default": "knowledge", + "enum": [ + "knowledge", + "memory" + ], + "type": "string" + } + }, + { + "description": "Max traversal depth in hops", + "in": "query", + "name": "depth", + "schema": { + "default": 5, + "maximum": 10, + "minimum": 1, + "type": "integer" + } + }, + { + "description": "Max members returned; `is_truncated` reports when this clipped the traversal", + "in": "query", + "name": "max_sources", + "schema": { + "default": 200, + "maximum": 1000, + "minimum": 1, + "type": "integer" + } }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" - } - }, - "security": [ { - "BearerAuth": [] + "description": "Principals to answer as (document ACLs): the subgraph contains only items they may see, filtered at every hop. Repeated (acl=a\u0026acl=b) or comma-separated. Omit for no ACL scoping.", + "in": "query", + "name": "acl", + "schema": { + "items": { + "type": "string" + }, + "type": "array" + }, + "style": "form" } ], - "summary": "List databases", - "tags": [ - "database-management" - ], - "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "list" - }, - "post": { - "description": "Create a new database with optional custom metadata schema", - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/tenants.TenantCreateRequest" - } - } - }, - "description": "Database creation request", - "required": true - }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_TenantCreateAcceptedResponse" + "$ref": "#/components/schemas/handler.Envelope-graph_SourceSubgraphResponse" } } }, @@ -11590,36 +9271,6 @@ } }, "description": "Bad Request" - }, - "403": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Forbidden" - }, - "409": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Conflict" - }, - "500": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Internal Server Error" } }, "security": [ @@ -11627,20 +9278,20 @@ "BearerAuth": [] } ], - "summary": "Create a database", + "summary": "Get connected subgraph", "tags": [ - "database-management" + "context" ], - "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "create" + "x-fern-sdk-group-name": "context", + "x-fern-sdk-method-name": "subgraph" } }, - "/databases/collections": { + "/databases": { "delete": { - "description": "Permanently remove one collection and all of its data from a database. The database itself is left intact.", + "description": "Delete a database and all associated data", "parameters": [ { - "description": "Database identifier", + "description": "Database identifier to delete", "in": "query", "name": "database", "required": true, @@ -11648,16 +9299,6 @@ "example": "acme_corp", "type": "string" } - }, - { - "description": "Collection identifier", - "in": "query", - "name": "collection", - "required": true, - "schema": { - "example": "team_docs", - "type": "string" - } } ], "responses": { @@ -11665,7 +9306,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_SubTenantDeleteResponse" + "$ref": "#/components/schemas/handler.Envelope-tenants_TenantDeleteResponse" } } }, @@ -11707,58 +9348,26 @@ "BearerAuth": [] } ], - "summary": "Delete a collection", + "summary": "Delete a database", "tags": [ "database-management" ], "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "deleteCollection" + "x-fern-sdk-method-name": "delete" }, "get": { - "description": "List all collections for a given database", - "parameters": [ - { - "description": "Database identifier", - "in": "query", - "name": "database", - "required": true, - "schema": { - "example": "acme_corp", - "type": "string" - } - } - ], + "description": "List all databases for the authenticated user", "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_SubTenantIdsResponse" + "$ref": "#/components/schemas/handler.Envelope-tenants_TenantIdsResponse" } } }, "description": "OK" }, - "400": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Bad Request" - }, - "404": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Not Found" - }, "500": { "content": { "application/json": { @@ -11775,35 +9384,32 @@ "BearerAuth": [] } ], - "summary": "List collections", + "summary": "List databases", "tags": [ "database-management" ], "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "collections" - } - }, - "/databases/stats": { - "get": { - "description": "Get collection statistics for a database", - "parameters": [ - { - "description": "Database identifier", - "in": "query", - "name": "database", - "required": true, - "schema": { - "example": "acme_corp", - "type": "string" + "x-fern-sdk-method-name": "list" + }, + "post": { + "description": "Create a new database with optional custom metadata schema", + "requestBody": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/tenants.TenantCreateRequest" + } } - } - ], + }, + "description": "Database creation request", + "required": true + }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_TenantStatsResponse" + "$ref": "#/components/schemas/handler.Envelope-tenants_TenantCreateAcceptedResponse" } } }, @@ -11819,7 +9425,7 @@ }, "description": "Bad Request" }, - "404": { + "403": { "content": { "application/json": { "schema": { @@ -11827,7 +9433,17 @@ } } }, - "description": "Not Found" + "description": "Forbidden" + }, + "409": { + "content": { + "application/json": { + "schema": { + "$ref": "#/components/schemas/handler.ErrorResponse" + } + } + }, + "description": "Conflict" }, "500": { "content": { @@ -11845,17 +9461,17 @@ "BearerAuth": [] } ], - "summary": "Get database statistics", + "summary": "Create a database", "tags": [ "database-management" ], "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "stats" + "x-fern-sdk-method-name": "create" } }, - "/databases/status": { + "/databases/collections": { "get": { - "description": "Check the infrastructure provisioning status for a database", + "description": "List all collections for a given database", "parameters": [ { "description": "Database identifier", @@ -11873,7 +9489,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_InfraStatusResponseV2" + "$ref": "#/components/schemas/handler.Envelope-tenants_SubTenantIdsResponse" } } }, @@ -11915,46 +9531,43 @@ "BearerAuth": [] } ], - "summary": "Get infrastructure status", + "summary": "List collections", "tags": [ "database-management" ], "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "status" - } - }, - "/databases/{database}": { - "patch": { - "description": "Rename a database in place. The internal identity (and therefore all indexed data, graphs and documents) is unchanged — only the caller-facing name moves, atomically. Connectors syncing into the database are repointed at the new name. The old name stops resolving immediately, so callers must switch to the new name in the same rollout.", + "x-fern-sdk-method-name": "collections" + }, + "delete": { + "description": "Permanently remove one collection and all of its data from a database. The database itself is left intact and its other collections are untouched. `database` and `collection` are both required. The API still accepts the deprecated `tenant_id` and `sub_tenant_id` aliases in their place, but generated clients should send the canonical names.", "parameters": [ { - "description": "Current database identifier", - "in": "path", + "description": "Database identifier. The API also accepts the deprecated `tenant_id` alias in its place; this operation models only the canonical name, as every other operation in this spec does.", + "in": "query", "name": "database", "required": true, "schema": { "example": "acme_corp", "type": "string" } + }, + { + "description": "Collection identifier. Unlike the read endpoints this does not default to the database's own collection, because a delete has no safe default. The API also accepts the deprecated `sub_tenant_id` alias in its place; this operation models only the canonical name.", + "in": "query", + "name": "collection", + "required": true, + "schema": { + "example": "engineering", + "type": "string" + } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/tenants.TenantRenameRequest" - } - } - }, - "description": "New database name", - "required": true - }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.Envelope-tenants_TenantRenameResponse" + "$ref": "#/components/schemas/handler.Envelope-tenants_SubTenantDeleteResponse" } } }, @@ -11980,16 +9593,6 @@ }, "description": "Not Found" }, - "409": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.ErrorResponse" - } - } - }, - "description": "Conflict" - }, "500": { "content": { "application/json": { @@ -12006,21 +9609,21 @@ "BearerAuth": [] } ], - "summary": "Rename a database", + "summary": "Delete a collection", "tags": [ "database-management" ], "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "rename" + "x-fern-sdk-method-name": "deleteCollection" } }, - "/databases/{database}/instructions": { + "/databases/stats": { "get": { - "description": "Read the custom ingestion instructions configured for a database and for its collections.", + "description": "Get collection statistics for a database", "parameters": [ { "description": "Database identifier", - "in": "path", + "in": "query", "name": "database", "required": true, "schema": { @@ -12034,7 +9637,7 @@ "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.instructionsResponse" + "$ref": "#/components/schemas/handler.Envelope-tenants_TenantStatsResponse" } } }, @@ -12076,19 +9679,21 @@ "BearerAuth": [] } ], - "summary": "Get ingestion instructions", + "summary": "Get database statistics", "tags": [ "database-management" ], "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "get_instructions" - }, - "patch": { - "description": "Set or clear the custom ingestion instructions for a database and its collections. Database instructions apply to every document; a collection's instructions apply on top of them. Both stack with any connector- or resource-level instructions rather than replacing them. Applies from the next ingestion; already-indexed data is not reprocessed.", + "x-fern-sdk-method-name": "stats" + } + }, + "/databases/status": { + "get": { + "description": "Check the infrastructure provisioning status for a database", "parameters": [ { "description": "Database identifier", - "in": "path", + "in": "query", "name": "database", "required": true, "schema": { @@ -12097,23 +9702,12 @@ } } ], - "requestBody": { - "content": { - "application/json": { - "schema": { - "$ref": "#/components/schemas/handler.instructionsUpdateReq" - } - } - }, - "description": "Instructions to set or clear", - "required": true - }, "responses": { "200": { "content": { "application/json": { "schema": { - "$ref": "#/components/schemas/handler.instructionsResponse" + "$ref": "#/components/schemas/handler.Envelope-tenants_InfraStatusResponseV2" } } }, @@ -12155,12 +9749,12 @@ "BearerAuth": [] } ], - "summary": "Update ingestion instructions", + "summary": "Get infrastructure status", "tags": [ "database-management" ], "x-fern-sdk-group-name": "databases", - "x-fern-sdk-method-name": "update_instructions" + "x-fern-sdk-method-name": "status" } }, "/databases/{database}/metadata-schema": { @@ -12393,7 +9987,7 @@ }, "/query": { "post": { - "description": "Query a database, by hybrid or text search (`query_by`). Optionally narrow to one or more exact document titles with `titles`. Filter with `attributes`, an operator language applied during retrieval; `metadata_filters` is deprecated in favour of it and still works. On a split database `type` is knowledge (the default), memory, or all (both, merged). On a unified database there is one corpus: omit `type`, or send `unified` (`all` is accepted and means the same); knowledge and memory are rejected there. Scope the query with `collection` or `collections`; `sub_tenant_id` and `sub_tenant_ids` are deprecated, and `sub_tenant_ids` cannot be sent together with `collections`.", + "description": "Unified query endpoint that dispatches across type (knowledge/memory/all) and query_by (hybrid/text). Prefer sub_tenant_ids for sub-tenant scoping; legacy sub_tenant_id is deprecated for /query and cannot be sent together with sub_tenant_ids.", "requestBody": { "content": { "application/json": { From 006940a4a46c27ae7cb6d5f684b39249c3a55e4c Mon Sep 17 00:00:00 2001 From: SohamRatnaparkhi Date: Wed, 23 Sep 2026 17:23:41 +0530 Subject: [PATCH 6/7] docs: hide Context categories and stop promoting context_category (PRO-1618) Context categories is not promoted yet, so the page leaves the navigation, gets noindex and keeps no inbound links. It is not deprecated and carries no warning. On the visible pages context_category is no longer something to set: it is gone from the Ingest guide (section 7, the field row, the accordion and the examples, with the later sections renumbered and every inbound anchor repointed), Core concepts, Quickstart, Databases and collections, AGENTS.mdx, SDKs, the API reference index, Sources overview, Ingest Context, Error responses and the MCP and CLI plugin pages. enrichment_kind stays in the chunk field reference because it is part of the response contract, described as an optional label that is omitted when none was set, the same way on Query, the Query reference and AGENTS.mdx. Example responses no longer show it, and the example llm_prompt no longer prints a Category, matching example requests that set none. The example llm_prompt now echoes the example request's query. Forceful relations are followed only in thinking mode; Ingest, the Glossary, Ingest Context and AGENTS.mdx now say so, as Query already did. Databases and collections also names a shared collection such as company as the place for shared context (its own examples query one), and its rename note drops a double hyphen. Signed-off-by: SohamRatnaparkhi Co-Authored-By: Claude Opus 5.5 (1M context) --- AGENTS.mdx | 52 +++++------------- api-reference/v2/endpoint/ingest-context.mdx | 9 +-- api-reference/v2/endpoint/query.mdx | 8 +-- .../v2/endpoint/sources-overview.mdx | 4 +- api-reference/v2/error-responses.mdx | 1 - api-reference/v2/index.mdx | 2 +- api-reference/v2/sdks.mdx | 8 +-- docs.json | 1 - essentials/v2/access-control.mdx | 2 +- essentials/v2/api-results.mdx | 12 ++-- essentials/v2/attributes.mdx | 2 +- essentials/v2/bring-your-own-graph.mdx | 2 +- essentials/v2/context-categories.mdx | 1 + essentials/v2/context-graphs.mdx | 2 +- essentials/v2/databases-and-collections.mdx | 7 +-- essentials/v2/glossary.mdx | 4 +- essentials/v2/ingest.mdx | 55 +++++-------------- essentials/v2/query.mdx | 16 +++--- get-started/v2/core-concepts.mdx | 21 +------ get-started/v2/quickstart.mdx | 11 +--- mintlify-hygiene.toml | 3 + plugins/cli.mdx | 5 +- plugins/mcp.mdx | 1 - 23 files changed, 68 insertions(+), 161 deletions(-) diff --git a/AGENTS.mdx b/AGENTS.mdx index f2f5c7b5..ad4a8885 100644 --- a/AGENTS.mdx +++ b/AGENTS.mdx @@ -210,9 +210,9 @@ An item is one piece of context, and carries exactly one of: - `text`: a document, a note, a policy, an agent log line; anything you already have as a string. To ingest a file, extract its text first. - `conversation`: a list of `{ role, content, name? }` turns, the same message list you already send to OpenAI or Anthropic. -Every item can also carry a `context_id` (your id; reuse it to replace the item), a `title`, a `context_category` (`user_preference`, `business_knowledge`, `decision_trace`, or `auto` for no label; you set it, HydraDB never infers it), declared `attributes` and free-form `custom_attributes`, a `happened_at` date, `forceful_relations` to other items, an `acl`, and per-item `enrich` / `upsert` / `instructions`. See [Item fields](#item-fields). +Every item can also carry a `context_id` (your id; reuse it to replace the item), a `title`, declared `attributes` and free-form `custom_attributes`, a `happened_at` date, `forceful_relations` to other items, an `acl`, and per-item `enrich` / `upsert` / `instructions`. See [Item fields](#item-fields). -With `enrich: true` (the default) HydraDB reads each item and extracts entities, relations and preferences into the context graph. The extracted statement comes back on each chunk as `enrichment`, separate from the chunk's verbatim `content`, and the item's category comes back as `enrichment_kind`. +With `enrich: true` (the default) HydraDB reads each item and extracts entities, relations and preferences into the context graph. The extracted statement comes back on each chunk as `enrichment`, separate from the chunk's verbatim `content`. ### Query @@ -263,7 +263,7 @@ At ingest, any item (text or conversation) can declare which other items it is l } ``` -At query time, with `follow_forceful_relations: true` (the default), a hit on `refund-policy` pulls chunks of `refund-faq` and `refund-escalations` into the response's `forceful_relations[]`, each with the `via` link that brought it in. +At query time, in `thinking` mode and with `follow_forceful_relations: true` (the default), a hit on `refund-policy` pulls chunks of `refund-faq` and `refund-escalations` into the response's `forceful_relations[]`, each with the `via` link that brought it in. Rules: @@ -316,7 +316,7 @@ SDK naming: - Python methods and fields: snake_case, for example `client.databases.collections()`, `max_results`, `query_by`, `result.data.llm_prompt`, `status.indexing_status`. - TypeScript methods and fields: camelCase, for example `maxResults`, `queryBy`, `pageSize`, `result.data.llmPrompt`, `chunk.chunkId`, `chunk.contextId`, `chunk.enrichmentKind`, `path.pathSummary`, `result.data.forcefulRelations`, `status.indexingStatus`. - Both SDKs return a `{ success, data, error, meta }` envelope; the payload is under `.data` (for example `response.data.infra`, `response.data.statuses`, `response.data.results`). -- `client.context.ingest()` sends a multipart form: the item list goes in the `items` form field as a JSON string. Keys inside each item stay snake_case in every language (`context_id`, `context_category`, `happened_at`), because that string is raw wire data. +- `client.context.ingest()` sends a multipart form: the item list goes in the `items` form field as a JSON string. Keys inside each item stay snake_case in every language (`context_id`, `happened_at`, `custom_attributes`), because that string is raw wire data. --- @@ -354,7 +354,6 @@ client.context.ingest( "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", }]), ) client.context.ingest( @@ -366,7 +365,6 @@ client.context.ingest( {"role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex"}, {"role": "assistant", "content": "Got it, short answers."}, ], - "context_category": "user_preference", "happened_at": "2026-09-01", }]), ) @@ -431,7 +429,6 @@ await client.context.ingest({ context_id: "refund-policy", title: "Refund policy", text: "Refunds are processed within 5 business days.", - context_category: "business_knowledge", }]), }); await client.context.ingest({ @@ -443,7 +440,6 @@ await client.context.ingest({ { role: "user", content: "Keep answers short, I read on my phone.", name: "alex" }, { role: "assistant", content: "Got it, short answers." }, ], - context_category: "user_preference", happened_at: "2026-09-01", }]), }); @@ -509,8 +505,7 @@ curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ \"context\": [{ \"context_id\": \"refund-policy\", \"title\": \"Refund policy\", - \"text\": \"Refunds are processed within 5 business days.\", - \"context_category\": \"business_knowledge\" + \"text\": \"Refunds are processed within 5 business days.\" }] }" curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ @@ -524,7 +519,6 @@ curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ { \"role\": \"user\", \"content\": \"Keep answers short, I read on my phone.\", \"name\": \"alex\" }, { \"role\": \"assistant\", \"content\": \"Got it, short answers.\" } ], - \"context_category\": \"user_preference\", \"happened_at\": \"2026-09-01\" }] }" @@ -631,7 +625,6 @@ One endpoint takes every item, text or conversation, into any collection of a da "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", "attributes": { "department": "support" }, "custom_attributes": { "owner": "sam@acme.com" }, "forceful_relations": { "ids": ["refund-faq"], "properties": {} } @@ -642,7 +635,6 @@ One endpoint takes every item, text or conversation, into any collection of a da { "role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex" }, { "role": "assistant", "content": "Got it, short answers." } ], - "context_category": "user_preference", "happened_at": "2026-09-01", "acl": ["user_email:alex@acme.com"], "instructions": "Extract the customer's stated preferences. Ignore small talk." @@ -681,7 +673,6 @@ Each item carries exactly one of `text` or `conversation`. | `happened_at` | The date the item is about, `YYYY-MM-DD` only; a timestamp is a `400`. HydraDB records when it received the item separately. | | `attributes` | Declared, filterable fields from the database's `database_metadata_schema`. | | `custom_attributes` | Free-form fields. Not filterable. | -| `context_category` | `auto` (the default, no label), `user_preference`, `business_knowledge` or `decision_trace`. Validated strictly: a typo is a `400`, not a fallback to `auto`. | | `forceful_relations` | `{ "ids": [...], "properties": {} }`: the `context_id`s this item is linked to. | | `acl` | Principals allowed to retrieve the item: `user_email:a@x.com` (or a bare email), `group::`, `domain:acme.com`, `__public__`. Omit for unrestricted, `[]` for nobody. A malformed principal rejects the whole request with `400`. | | `is_markdown` | Chunk `text` on its markdown structure instead of as flat prose. | @@ -697,16 +688,6 @@ A key an item does not recognise is dropped without an error, so check spelling - Set `name` per turn when several people speak, so preferences are attributed to the right person. - An empty list, or a turn with empty `content`, is a `400`. -### Context categories - -| Category | For | -|---|---| -| `user_preference` | What a person likes, chose or asked for | -| `business_knowledge` | Documentation, policy, product and domain facts | -| `decision_trace` | What an agent or a team decided, and why | - -The label is yours to set. Leave `context_category` out, or send `auto`, and the item is stored and enriched as general context with no category on its chunks. A preference or a decision is sent as text (or a conversation) with the matching label. - ### IDs and replacement - `upsert: true` (the default) **replaces**: re-ingesting a `context_id` deletes everything derived from the previous version (its chunks and its graph contribution) before writing the new one. It does not merge. @@ -955,7 +936,6 @@ Attribute-filtered search, on behalf of one user: "score": 0.91, "content": "Refunds are processed within 30 days of purchase by the Finance Department.", "enrichment": "Refund window is 30 days; Finance owns refund processing.", - "enrichment_kind": "business_knowledge", "temporal": [ { "content": "Refund policy effective_from June 2026. Start: 2026-06-01", "start_date": "2026-06-01", "end_date": null } ] @@ -965,8 +945,7 @@ Attribute-filtered search, on behalf of one user: "context_id": "chat-2026-07-29", "score": 0.84, "content": "user: Keep refund answers short please\nassistant: Got it.", - "enrichment": "User prefers short answers about refunds.", - "enrichment_kind": "user_preference" + "enrichment": "User prefers short answers about refunds." } ], "graph": [ @@ -1020,7 +999,7 @@ Attribute-filtered search, on behalf of one user: | `score` | Relevance. Always present. | | `content` | The chunk's own text, verbatim. Enrichment is never concatenated into it. | | `enrichment` | A plain string: what enrichment extracted from this chunk (a preference, a fact). Omitted when there is none. | -| `enrichment_kind` | The item's `context_category` (`user_preference`, `business_knowledge` or `decision_trace`). Omitted when none was declared; present even when `enrichment` is omitted. | +| `enrichment_kind` | An optional label; omitted when none was set. | | `temporal` | Present only when the query engaged temporal reasoning: `{ content, start_date, end_date }` entries, where `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` and either date may be `null`. | Chunks carry nothing about their source: no title, url, collection or attributes. `llm_prompt` prints the title, collection, last-updated date and url for the model. To read an item's stored content yourself, call `GET /context/inspect` with the chunk's `context_id`; to read its title and attributes, call `POST /context/list` with `ids: [context_id]`. @@ -1064,7 +1043,7 @@ Sections, in order (a section with nothing in it is left out; when the query ret | Section | Contents | |---|---| | `# Query results` | The query, an `**Interpreted:**` line when an alias or resolved reference widened it, a `**Found:**` line counting what follows, a `**Note:**` line when a lookup degraded, and (when there is a result) the line telling the model to cite it by its number. | -| `## Results` | One `### 1. title` block per chunk, in ranked order: relevance, collection, type, category, id and last-updated date, the chunk's `content`, then `**Enrichment:**`. | +| `## Results` | One `### 1. title` block per chunk, in ranked order: relevance, collection, type, category (`enrichment_kind`, when set), id and last-updated date, the chunk's `content`, then `**Enrichment:**`. | | `## Forceful relations` | One `### R1. title` block per forceful-relation chunk, with `**Linked from:**` naming the item that pulled it in. | | `## Related facts` | One line per graph path, such as `- [P1] **Refunds** -managed_by→ **Finance** (relevance 0.81) [1]`, with the `path_summary` indented under it unless it only restates the chain. A path without a reranked score has no parenthetical. | | `## Temporal facts` | A `**Duration:**` line first for a "how long between" question, then one line per dated fact the query engaged, with its resolved window, citing its result. | @@ -1095,7 +1074,7 @@ Cite a result by its number in brackets, e.g. [1]. ## Results ### 1. Refund policy -- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge +- **Relevance:** 0.91 · **Collection:** support · **Type:** file - **Id:** refund-policy · **Last updated:** 2026-07-02 Refunds are processed within 30 days of purchase by the Finance Department. @@ -1105,7 +1084,7 @@ Refunds are processed within 30 days of purchase by the Finance Department. --- ### 2. Support chat with Priya -- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference +- **Relevance:** 0.84 · **Collection:** support · **Type:** message - **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29 user: Keep refund answers short please @@ -1179,7 +1158,7 @@ Surface `llm_prompt` to the model as it is and let the model cite the labels. Ma ### Structured output instead of a prompt -When you render results yourself (a UI, a reranker, an eval), read `chunks[].content`, `chunks[].enrichment`, `chunks[].enrichment_kind`, `graph[].path_summary` and `forceful_relations[]`. To hang graph hops under the chunk they came from: +When you render results yourself (a UI, a reranker, an eval), read `chunks[].content`, `chunks[].enrichment`, `graph[].path_summary` and `forceful_relations[]`. To hang graph hops under the chunk they came from: - Build a `chunk_id` to `context_id` map from `chunks[]` and `forceful_relations[].chunk`. Do not parse the chunk id string. - Group hops by `relation.chunk_id` and match against that map. @@ -1421,7 +1400,7 @@ Response `data`: `results[]` (per id: `id`, `deleted`, `error`), `deleted_count` ### Ingest validation (`400`) -The whole request is rejected, and the message names the item as `context[N]`, when an item has both `text` and `conversation` (or neither), a conversation breaks the rules in [Conversations](#conversations), `context_category` or `happened_at` is malformed, an `acl` principal is malformed, a `graph_payload` key matches no item, or a size limit in [Limits and validation](#limits-and-validation) is exceeded. +The whole request is rejected, and the message names the item as `context[N]`, when an item has both `text` and `conversation` (or neither), a conversation breaks the rules in [Conversations](#conversations), `happened_at` is malformed, an `acl` principal is malformed, a `graph_payload` key matches no item, or a size limit in [Limits and validation](#limits-and-validation) is exceeded. Retry only `429`, `500`, `502` and `503`; use bounded exponential backoff with jitter. @@ -1472,15 +1451,15 @@ try { Most production integrations share one shape: 1. Create a database, and declare in `database_metadata_schema` the attributes you will filter on. -2. Ingest shared context (policies, docs, tickets, extracted file text) into a shared collection with stable `context_id`s and a `context_category` of `business_knowledge`. Sync SaaS tools with [connectors](/essentials/v2/connectors) instead of re-implementing them. -3. Ingest each user's conversations and stated preferences into their own collection (`collection = user_id`), labelling preferences `user_preference` and agent decisions `decision_trace`. +2. Ingest shared context (policies, docs, tickets, extracted file text) into a shared collection with stable `context_id`s. Sync SaaS tools with [connectors](/essentials/v2/connectors) instead of re-implementing them. +3. Ingest each user's conversations and stated preferences into their own collection (`collection = user_id`). 4. Link items that belong together (a ticket and its follow-ups, a policy and its FAQ) with `forceful_relations`. 5. Poll `GET /context/status` or register a webhook. 6. Query with `collections: { "": 2, "company": 1 }`, `mode: "thinking"` for quality or `"fast"` for latency, and an `attributes` filter for hard scopes. 7. Inject `llm_prompt` into the model call with a grounding instruction, and let the model cite `[1]`, `[R1]`, `[P1]`. 8. Say so explicitly when the context does not contain the answer, and report it with `POST /feedback`. -Typical uses of that shape: a support agent that answers from policy while respecting each customer's stated preferences; workplace search over Slack, Notion and Drive with citations; an onboarding assistant over org charts, specs and meeting notes; an agent that records its own decisions as `decision_trace` items and consults them before acting again. +Typical uses of that shape: a support agent that answers from policy while respecting each customer's stated preferences; workplace search over Slack, Notion and Drive with citations; an onboarding assistant over org charts, specs and meeting notes; an agent that records its own decisions as items and consults them before acting again. --- @@ -1496,7 +1475,6 @@ Typical uses of that shape: a support agent that answers from policy while respe - [ ] Omitting `collection` on `context.status` and reading the resulting `FILE_NOT_FOUND` as a genuine indexing failure. - [ ] Using `??` on `errorMessage`, which is `""` rather than null on some failures, so the fallback never fires. - [ ] Sending both `text` and `conversation` on one item. -- [ ] Misspelling `context_category` (`"business-knowledge"` is a `400`). - [ ] Sending a full timestamp in `happened_at`. - [ ] Reusing untitled text without a `context_id`, so the second item replaces the first. - [ ] Writing with one `collection` and reading with another. diff --git a/api-reference/v2/endpoint/ingest-context.mdx b/api-reference/v2/endpoint/ingest-context.mdx index 0a4b4b0f..ed43f56a 100644 --- a/api-reference/v2/endpoint/ingest-context.mdx +++ b/api-reference/v2/endpoint/ingest-context.mdx @@ -25,7 +25,6 @@ result = client.context.ingest( "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", "attributes": {"department": "support"}, "custom_attributes": {"owner": "sam@acme.com"}, }, @@ -35,7 +34,6 @@ result = client.context.ingest( {"role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex"}, {"role": "assistant", "content": "Got it, short answers."}, ], - "context_category": "user_preference", "happened_at": "2026-09-01", "forceful_relations": {"ids": ["refund-policy"]}, }, @@ -55,7 +53,6 @@ const result = await client.context.ingest({ context_id: "refund-policy", title: "Refund policy", text: "Refunds are processed within 5 business days.", - context_category: "business_knowledge", attributes: { department: "support" }, custom_attributes: { owner: "sam@acme.com" }, }, @@ -65,7 +62,6 @@ const result = await client.context.ingest({ { role: "user", content: "Keep answers short, I read on my phone.", name: "alex" }, { role: "assistant", content: "Got it, short answers." }, ], - context_category: "user_preference", happened_at: "2026-09-01", forceful_relations: { ids: ["refund-policy"] }, }, @@ -90,7 +86,6 @@ curl -X POST 'https://api.hydradb.com/context/ingest' \ "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", "attributes": { "department": "support" }, "custom_attributes": { "owner": "sam@acme.com" } }, @@ -100,7 +95,6 @@ curl -X POST 'https://api.hydradb.com/context/ingest' \ { "role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex" }, { "role": "assistant", "content": "Got it, short answers." } ], - "context_category": "user_preference", "happened_at": "2026-09-01", "forceful_relations": { "ids": ["refund-policy"] } } @@ -142,8 +136,7 @@ Each item is exactly one of `text` or `conversation`. | | The date the item is about, `YYYY-MM-DD` only. A timestamp is a `400`. HydraDB records when it received the item separately. | | | Declared, filterable fields; keys must be in `database_metadata_schema`. Filter with `attributes` on `/query`. See [Attributes](/essentials/v2/attributes). | | | Free-form fields. Stored with the item; not filterable and not returned on query chunks. | -| | What kind of context this is. You set it; nothing infers it, and a misspelling is a `400`. Returned on query as `enrichment_kind` (omitted for `auto`). See [Context categories](/essentials/v2/context-categories). (default=`"auto"`) | -| | Relations you declare to other items: `{ "ids": ["", ...], "properties": {} }`. Followed on `/query` with `follow_forceful_relations` and returned in `forceful_relations[]`. Each id follows the same rules as `context_id`. | +| | Relations you declare to other items: `{ "ids": ["", ...], "properties": {} }`. Followed on `/query` in `thinking` mode with `follow_forceful_relations` and returned in `forceful_relations[]`. Each id follows the same rules as `context_id`. | | | Principals allowed to retrieve the item: bare emails or `user_email:`, `group:`, `domain:` principals, or `__public__`. Omit for unrestricted; `[]` for nobody. A malformed principal rejects the whole request with `400`. See [Access control](/essentials/v2/access-control). | | | Chunk `text` on its markdown structure instead of as flat prose. (default=`false`) | | | The speaker for a text item. On a conversation each turn's `name` wins. (default=`"User"`) | diff --git a/api-reference/v2/endpoint/query.mdx b/api-reference/v2/endpoint/query.mdx index f0ebf08d..316a3449 100644 --- a/api-reference/v2/endpoint/query.mdx +++ b/api-reference/v2/endpoint/query.mdx @@ -435,7 +435,6 @@ result = client.query( "score": 0.91, "content": "Refunds are processed within 30 days of purchase by the Finance Department.", "enrichment": "Refund window is 30 days; Finance owns refund processing.", - "enrichment_kind": "business_knowledge", "temporal": [ { "content": "Refund policy effective_from June 2026. Start: 2026-06-01", @@ -449,8 +448,7 @@ result = client.query( "context_id": "chat-2026-07-29", "score": 0.84, "content": "user: Keep refund answers short please\nassistant: Got it.", - "enrichment": "User prefers short answers about refunds.", - "enrichment_kind": "user_preference" + "enrichment": "User prefers short answers about refunds." } ], "graph": [ @@ -514,7 +512,7 @@ result = client.query( } } ], - "llm_prompt": "# Query results\n\n**Query:** who owns refund processing?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" + "llm_prompt": "# Query results\n\n**Query:** How are refunds processed, and how should I answer this user?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" }, "error": null, "meta": { @@ -549,7 +547,7 @@ result = client.query( | Key | Contents | | --- | --- | -| `chunks[]` | Ranked matches: `chunk_id`, `context_id`, `score`, `content` (verbatim), `enrichment` (the extracted statement as a plain string, omitted when there is none), `enrichment_kind` (the item's declared `context_category`: `user_preference`, `business_knowledge` or `decision_trace`; omitted when none was declared, present even without `enrichment`), `temporal[]` (only when the query engaged temporal reasoning; `{ content, start_date, end_date }`, where `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` and either date may be `null`). | +| `chunks[]` | Ranked matches: `chunk_id`, `context_id`, `score`, `content` (verbatim), `enrichment` (the extracted statement as a plain string, omitted when there is none), `enrichment_kind` (an optional label; omitted when none was set), `temporal[]` (only when the query engaged temporal reasoning; `{ content, start_date, end_date }`, where `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` and either date may be `null`). | | `graph[]` | Paths through the context graph, query paths first then chunk expansions: `origin`, `triplets[]` of `source` / `relation` / `target`, plus `path_summary`. `origin` is `"query_path"` (grown from the entities in the query) or `"chunk_relation"` (the neighbourhood of a returned chunk, only returned when one of its hops came from a returned chunk or a `forceful_relations` chunk). The array is deduplicated across both lanes and is not capped. `path_summary` is never empty: when the server wrote no summary, it narrates the hops. Entities are `{ entity_id, name }`; relations are `{ predicate, context, temporal_details?, timestamp?, relationship_id, chunk_id }`, where `temporal_details` is omitted when empty and `timestamp` (Unix epoch seconds, a float) is omitted when the edge has none. `[]` when `graph_context` is `false`. | | `forceful_relations[]` | Chunks pulled in through `forceful_relations` declared at ingest, followed only in `thinking` mode: `via.from` (the context whose declaration pulled it in, may be `""`), `via.to` (the chunk's own `context_id`), `chunk` (same shape as `chunks[]`). `[]` when none, when `follow_forceful_relations` is `false`, or when the query ran in `fast` mode. | | `llm_prompt` | A server-built markdown string ready to inject into a model call: `# Query results`, then `## Results`, `## Forceful relations`, `## Related facts`, `## Temporal facts` (with a `**Duration:**` line for a "how long between" question), `## Source facts`, `## Profiles`, `## Code search` and `## Sources`, each left out when empty. Source facts, profiles, code-search answers and the duration are prompt only: no JSON key carries them. Results are cited `[1]` and forceful relations `[R1]`; related facts are labelled `[P1]`, `[P2]`, ... in `graph[]` order, as in `- [P1] **Refunds** -managed_by→ **Finance** (relevance 0.81) [1]`: the parenthetical is the path's relevance after reranking and is left out when the path has none, and the line ends with the results the path was extracted from. Sources print only web (`http` or `https`) links. `""` only when the query found nothing at all. The layout is on [Query](/essentials/v2/query#llm_prompt). | diff --git a/api-reference/v2/endpoint/sources-overview.mdx b/api-reference/v2/endpoint/sources-overview.mdx index b02db7b1..55a1a6e1 100644 --- a/api-reference/v2/endpoint/sources-overview.mdx +++ b/api-reference/v2/endpoint/sources-overview.mdx @@ -48,7 +48,7 @@ flowchart LR ## Core concepts -- **Items**: everything you ingest is a piece of context, a `text` or a `conversation`, with an optional `context_category` (`user_preference`, `business_knowledge`, `decision_trace`) that you set. One database holds all of them; collections partition them per user, team or project. See [Ingest context](/essentials/v2/ingest). +- **Items**: everything you ingest is a piece of context, a `text` or a `conversation`. One database holds all of them; collections partition them per user, team or project. See [Ingest context](/essentials/v2/ingest). - **IDs**: each item has a `context_id`, yours or generated. The ingest response reports it as `results[].id`. Use it for polling status, inspecting content, deleting, and inspecting relations. - **Attributes**: `attributes` are the declared, filterable fields from `database_metadata_schema`; `custom_attributes` are free-form and stored with the item. Filter queries with `attributes`. See [Attributes](/essentials/v2/attributes). - **Enrichment**: on by default (`enrich: true`). HydraDB extracts entities, relations and preferences from each item into the [context graph](/essentials/v2/context-graphs); the extracted text comes back on query as `enrichment`, separate from the item's own `content`. @@ -86,7 +86,5 @@ Paired with declared attributes, you get deterministic control over how results Related Resources - - [Usage: Context categories](/essentials/v2/context-categories): preferences, knowledge and decisions - - [Usage: Attributes](/essentials/v2/attributes): declared versus free-form fields diff --git a/api-reference/v2/error-responses.mdx b/api-reference/v2/error-responses.mdx index 72fba9bd..3ffb62e1 100644 --- a/api-reference/v2/error-responses.mdx +++ b/api-reference/v2/error-responses.mdx @@ -253,7 +253,6 @@ Database creation is asynchronous. After `POST /databases`, poll [`GET /database - An item has neither `text` nor `conversation`, or has both. - A conversation turn has a role other than `user`, `assistant` or `system`, or empty `content`; or the conversation has only `system` turns. -- `context_category` is not one of `auto`, `user_preference`, `business_knowledge` or `decision_trace`. - `happened_at` is not a `YYYY-MM-DD` date. - A `context_id` contains a comma. - A `graph_payload` key matches no `context_id` in the same request. diff --git a/api-reference/v2/index.mdx b/api-reference/v2/index.mdx index d5c93e21..0d4c1508 100644 --- a/api-reference/v2/index.mdx +++ b/api-reference/v2/index.mdx @@ -26,7 +26,7 @@ description: "Single reference to all HydraDB endpoints" |---|---|---| | `database` | Your isolated workspace for data, metadata schema, and query. | Send it on every API call so HydraDB knows which workspace to read or write. Formerly `tenant_id`; the `tenant_id` alias is still accepted (deprecated). | | `collection` | Optional partition inside a database, often a user, team, account, or customer. | Use it when one database contains data for multiple users or customers. Formerly `sub_tenant_id`; the `sub_tenant_id` alias is still accepted (deprecated). Read more about our [multi-tenant architecture](/essentials/v2/databases-and-collections) | -| [Context items](/essentials/v2/ingest) | A `text` or a `conversation`, sent in the `context` list of `POST /context/ingest`, optionally labelled with a `context_category`. | Everything you ingest. Shared context goes in a shared collection; a person's preferences go in their own. | +| [Context items](/essentials/v2/ingest) | A `text` or a `conversation`, sent in the `context` list of `POST /context/ingest`. | Everything you ingest. Shared context goes in a shared collection; a person's preferences go in their own. | | `database_metadata_schema` | Database-level fields you define up front so metadata can be filtered or queried consistently. | Use it for stable fields like department, customer, region, plan, category, or compliance label. | | `attributes` | Declared, filterable fields on an item, matching `database_metadata_schema`; `custom_attributes` are free-form. | Send them at ingest; filter with `attributes` on `/query`. | | `ids` | IDs returned by ingestion or visible from `/context/list`. | Use them when polling processing status, inspecting content, listing a specific subset, deleting sources, or inspecting relations. | diff --git a/api-reference/v2/sdks.mdx b/api-reference/v2/sdks.mdx index bab38c12..b3e2239a 100644 --- a/api-reference/v2/sdks.mdx +++ b/api-reference/v2/sdks.mdx @@ -59,7 +59,7 @@ The REST API uses **snake_case** for every request and response field. The Pytho | **TypeScript SDK** | camelCase | camelCase | `client.query({ maxResults: 8 })`, `result.data.llmPrompt` | -**Item keys stay snake_case in every language.** `client.context.ingest` takes the item list as a JSON string in the `items` field, so the keys inside each item (`context_id`, `context_category`, `happened_at`, `custom_attributes`) are the wire names in TypeScript too. +**Item keys stay snake_case in every language.** `client.context.ingest` takes the item list as a JSON string in the `items` field, so the keys inside each item (`context_id`, `happened_at`, `custom_attributes`) are the wire names in TypeScript too. `database` and `collection` are the current field names (formerly `tenant_id` and `sub_tenant_id`). The old names remain accepted as deprecated aliases. @@ -179,7 +179,7 @@ while (true) { ### Ingest context -Everything you ingest is a context item: one `text` or one `conversation`, with optional fields such as `context_id`, `title`, `context_category`, `happened_at` and `attributes`. The SDK sends a multipart form and puts the item list, as a JSON string, in the `items` field. +Everything you ingest is a context item: one `text` or one `conversation`, with optional fields such as `context_id`, `title`, `happened_at` and `attributes`. The SDK sends a multipart form and puts the item list, as a JSON string, in the `items` field. ```python Python SDK @@ -193,7 +193,6 @@ result = client.context.ingest( "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", "attributes": {"department": "support"}, }, { @@ -202,7 +201,6 @@ result = client.context.ingest( {"role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex"}, {"role": "assistant", "content": "Got it, short answers."}, ], - "context_category": "user_preference", "happened_at": "2026-09-01", }, ]), @@ -219,7 +217,6 @@ const result = await client.context.ingest({ context_id: "refund-policy", title: "Refund policy", text: "Refunds are processed within 5 business days.", - context_category: "business_knowledge", attributes: { department: "support" }, }, { @@ -228,7 +225,6 @@ const result = await client.context.ingest({ { role: "user", content: "Keep answers short, I read on my phone.", name: "alex" }, { role: "assistant", content: "Got it, short answers." }, ], - context_category: "user_preference", happened_at: "2026-09-01", }, ]), diff --git a/docs.json b/docs.json index efe65d63..770ef687 100644 --- a/docs.json +++ b/docs.json @@ -58,7 +58,6 @@ "public": true, "pages": [ "essentials/v2/ingest", - "essentials/v2/context-categories", "essentials/v2/connectors", "essentials/v2/query", "essentials/v2/attributes", diff --git a/essentials/v2/access-control.mdx b/essentials/v2/access-control.mdx index 0f921754..bf0f9476 100644 --- a/essentials/v2/access-control.mdx +++ b/essentials/v2/access-control.mdx @@ -230,7 +230,7 @@ Check the principal forms on both sides. `group:slack:C0123` on the document onl ## Related - [Connectors](/essentials/v2/connectors) - syncing app data, and per-resource ACL rules -- [Ingest context](/essentials/v2/ingest#10-restricting-an-item): the `acl` item field +- [Ingest context](/essentials/v2/ingest#9-restricting-an-item): the `acl` item field - [Query](/essentials/v2/query) - the `acl` field alongside every other retrieval parameter - [Metadata](/essentials/v2/attributes) - filtering by attributes, a different question from permission - [Multi-Tenant Support](/essentials/v2/databases-and-collections) - databases and collections, the isolation boundary ACLs work inside diff --git a/essentials/v2/api-results.mdx b/essentials/v2/api-results.mdx index 56690f7f..f2565896 100644 --- a/essentials/v2/api-results.mdx +++ b/essentials/v2/api-results.mdx @@ -89,14 +89,14 @@ For the [example response on Query](/essentials/v2/query#1-one-call), `llm_promp ```markdown # Query results -**Query:** who owns refund processing? +**Query:** How are refunds processed, and how should I answer this user? **Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation Cite a result by its number in brackets, e.g. [1]. ## Results ### 1. Refund policy -- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge +- **Relevance:** 0.91 · **Collection:** support · **Type:** file - **Id:** refund-policy · **Last updated:** 2026-07-02 Refunds are processed within 30 days of purchase by the Finance Department. @@ -106,7 +106,7 @@ Refunds are processed within 30 days of purchase by the Finance Department. --- ### 2. Support chat with Priya -- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference +- **Relevance:** 0.84 · **Collection:** support · **Type:** message - **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29 user: Keep refund answers short please @@ -166,7 +166,6 @@ Render a UI, rerank, or apply your own rules from the three structured keys. The | --- | --- | | `chunks[].content` | The matched text, verbatim. | | `chunks[].enrichment` | What enrichment extracted from that chunk (a preference, a fact), as a string. | -| `chunks[].enrichment_kind` | The item's declared `context_category` (`user_preference`, `business_knowledge` or `decision_trace`); omitted when none was declared. | | `chunks[].score` | Relevance, for your own thresholds. | | `graph[].path_summary` | One sentence per graph path; `graph[].triplets` for the steps and `graph[].origin` for the lane that found it. To show a path under its chunk, see [Attaching graph paths to chunks](/essentials/v2/query#attaching-graph-paths-to-chunks). | | `forceful_relations[]` | Chunks linked at ingest with `forceful_relations`, each with the `via` link that brought it in. | @@ -177,8 +176,6 @@ for chunk in result.data.chunks: print(chunk.score, chunk.context_id, chunk.content) if chunk.enrichment: print(" enrichment:", chunk.enrichment) - if chunk.enrichment_kind: - print(" category:", chunk.enrichment_kind) for path in result.data.graph: print("path:", path.path_summary) @@ -190,7 +187,6 @@ for rel in result.data.forceful_relations: for (const chunk of result.data.chunks) { console.log(chunk.score, chunk.contextId, chunk.content); if (chunk.enrichment) console.log(" enrichment:", chunk.enrichment); - if (chunk.enrichmentKind) console.log(" category:", chunk.enrichmentKind); } for (const path of result.data.graph) console.log("path:", path.pathSummary); @@ -248,4 +244,4 @@ Fetch it lazily, when a citation is opened, rather than for every chunk on every - [Query](/essentials/v2/query): request parameters and the four response keys - [Context graphs](/essentials/v2/context-graphs): what `graph[]` contains -- [Ingest context](/essentials/v2/ingest): `context_category`, `enrich` and `forceful_relations` decide what comes back here +- [Ingest context](/essentials/v2/ingest): `enrich` and `forceful_relations` decide what comes back here diff --git a/essentials/v2/attributes.mdx b/essentials/v2/attributes.mdx index 3f456bd7..746126bd 100644 --- a/essentials/v2/attributes.mdx +++ b/essentials/v2/attributes.mdx @@ -259,7 +259,7 @@ Rules checked before anything is queued: - In both maps, keys must not start with `_`, must not be a reserved system name, and must not contain control characters. A value may be a scalar, a list or an object, but not a list or object nested inside another. - Errors name the item they refer to, such as `context[0]: ...`. -**Attributes are set at ingest.** The `attributes` query filter runs against the values indexed with the item. To change them, re-ingest the item with `upsert: true` and the same `context_id`, which replaces the item. See [IDs and replacement](/essentials/v2/ingest#13-ids-and-replacement). +**Attributes are set at ingest.** The `attributes` query filter runs against the values indexed with the item. To change them, re-ingest the item with `upsert: true` and the same `context_id`, which replaces the item. See [IDs and replacement](/essentials/v2/ingest#12-ids-and-replacement). ### Size limits diff --git a/essentials/v2/bring-your-own-graph.mdx b/essentials/v2/bring-your-own-graph.mdx index 0245281f..e54968fb 100644 --- a/essentials/v2/bring-your-own-graph.mdx +++ b/essentials/v2/bring-your-own-graph.mdx @@ -24,7 +24,7 @@ Pick the right tool: | You want... | Use | | --- | --- | | HydraDB to discover relationships for you | [Context graphs](/essentials/v2/context-graphs) (auto-extraction, the default) | -| To declare links **between whole items** | `forceful_relations` on an item. See [Declared relations](/essentials/v2/ingest#11-declared-relations). | +| To declare links **between whole items** | `forceful_relations` on an item. See [Declared relations](/essentials/v2/ingest#10-declared-relations). | | To supply the **full entity and relation graph for one item** | **Bring Your Own Graph** (this page) | --- diff --git a/essentials/v2/context-categories.mdx b/essentials/v2/context-categories.mdx index 3afdee7b..99606d26 100644 --- a/essentials/v2/context-categories.mdx +++ b/essentials/v2/context-categories.mdx @@ -1,6 +1,7 @@ --- title: "Context categories" description: "Label each item as a user preference, business knowledge or a decision trace. The label is yours to set; nothing infers it." +noindex: true --- A **context category** says what kind of context an item is. You set it per item with `context_category` on [`POST /context/ingest`](/essentials/v2/ingest). Every category lives in the same database and is searched by the same [query](/essentials/v2/query); the category describes the item, it does not decide where it is stored. diff --git a/essentials/v2/context-graphs.mdx b/essentials/v2/context-graphs.mdx index 65a9708d..b4ab5882 100644 --- a/essentials/v2/context-graphs.mdx +++ b/essentials/v2/context-graphs.mdx @@ -37,7 +37,7 @@ Skip them for direct factual lookups. Graph traversal adds response size and can Context graphs are hybrid: relationships are extracted at ingestion time and traversed at query time. -**At ingestion**, with `enrich: true` (the default), HydraDB extracts entities and relations from each item and stores them in the graph. An item can also declare its own links to other items with [`forceful_relations`](/essentials/v2/ingest#11-declared-relations), or skip extraction and supply its entities and relations with [`graph_payload`](/essentials/v2/ingest#12-bring-your-own-graph). +**At ingestion**, with `enrich: true` (the default), HydraDB extracts entities and relations from each item and stores them in the graph. An item can also declare its own links to other items with [`forceful_relations`](/essentials/v2/ingest#10-declared-relations), or skip extraction and supply its entities and relations with [`graph_payload`](/essentials/v2/ingest#11-bring-your-own-graph). **At query**, with `graph_context: true`: diff --git a/essentials/v2/databases-and-collections.mdx b/essentials/v2/databases-and-collections.mdx index fe1f9390..a22bc267 100644 --- a/essentials/v2/databases-and-collections.mdx +++ b/essentials/v2/databases-and-collections.mdx @@ -8,7 +8,7 @@ description: "How HydraDB scopes data using databases and collections, and how s - The request fields are now named **`database`** (formerly `tenant_id`) and **`collection`** (formerly `sub_tenant_id`). The old names and the old `/tenants` routes remain fully supported as deprecated aliases, so existing integrations keep working without any changes -- however, we recommend using the latest conventions. See [Migrating from `tenant_id` and `sub_tenant_id`](#7-migrating-from-the-legacy-tenant-and-sub-tenant-fields) for the full compatibility contract. + The request fields are now named **`database`** (formerly `tenant_id`) and **`collection`** (formerly `sub_tenant_id`). The old names and the old `/tenants` routes remain fully supported as deprecated aliases, so existing integrations keep working without any changes. We still recommend the new names. See [Migrating from `tenant_id` and `sub_tenant_id`](#7-migrating-from-the-legacy-tenant-and-sub-tenant-fields) for the full compatibility contract. ## 1. What it is @@ -32,7 +32,7 @@ Use the same scoping values consistently across writes and reads. If you write d | Separate environments (`prod` vs `staging`) | A different `database` per environment | | Separate per-user state within one customer | One `database`, `collection = user_id` | | Separate per-workspace data within one customer | One `database`, `collection = workspace_id` | -| Store broadly shared context | Omit `collection` to use the database's default collection | +| Store broadly shared context | A shared collection such as `company`, or omit `collection` to use the database's default collection | Two practical rules: @@ -146,7 +146,6 @@ curl -X POST 'https://api.hydradb.com/context/ingest' \ "context": [ { "text": "Prefers dark mode and short answers.", - "context_category": "user_preference", "user_name": "John" } ] @@ -181,7 +180,6 @@ await client.context.ingest({ items: JSON.stringify([ { text: "Prefers dark mode and short answers.", - context_category: "user_preference", user_name: "John", }, ]), @@ -212,7 +210,6 @@ client.context.ingest( items=json.dumps([ { "text": "Prefers dark mode and short answers.", - "context_category": "user_preference", "user_name": "John", } ]), diff --git a/essentials/v2/glossary.mdx b/essentials/v2/glossary.mdx index 3ff596e6..06a12e10 100644 --- a/essentials/v2/glossary.mdx +++ b/essentials/v2/glossary.mdx @@ -27,10 +27,10 @@ back from `POST /query` as `chunks`, each carrying the `context_id` it came from Links you declare between context items at ingest, with an item's `forceful_relations` field. When a query hits an item, HydraDB follows its declared links -(`follow_forceful_relations`, on by default) and returns the linked chunks in the +(`follow_forceful_relations`, on by default; `thinking` mode only) and returns the linked chunks in the response's `forceful_relations[]`, each with the `via` link that brought it in, separately from the ranked `chunks` and the `graph` paths. See -[Declared relations](/essentials/v2/ingest#11-declared-relations). +[Declared relations](/essentials/v2/ingest#10-declared-relations). ## Deprecated aliases diff --git a/essentials/v2/ingest.mdx b/essentials/v2/ingest.mdx index 2a4d9d3d..302af209 100644 --- a/essentials/v2/ingest.mdx +++ b/essentials/v2/ingest.mdx @@ -1,6 +1,6 @@ --- title: "Ingest context" -description: "Send text and conversations to HydraDB as context items in one call, label what kind of context each one is, and confirm it is searchable." +description: "Send text and conversations to HydraDB as context items in one call, and confirm they are searchable." --- Everything you put into HydraDB is a piece of **context**: a text, or a conversation. You send a list of them to one endpoint, [`POST /context/ingest`](/api-reference/v2/endpoint/ingest-context), and HydraDB chunks each one, embeds it, enriches it, extracts entities and relations into the [context graph](/essentials/v2/context-graphs), and makes it searchable through [`POST /query`](/essentials/v2/query). @@ -25,7 +25,6 @@ curl -X POST 'https://api.hydradb.com/context/ingest' \ "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", "attributes": { "department": "support" }, "custom_attributes": { "owner": "sam@acme.com" } }, @@ -35,7 +34,6 @@ curl -X POST 'https://api.hydradb.com/context/ingest' \ { "role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex" }, { "role": "assistant", "content": "Got it, short answers." } ], - "context_category": "user_preference", "happened_at": "2026-09-01" } ] @@ -52,7 +50,6 @@ ingest = client.context.ingest( "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", "attributes": {"department": "support"}, "custom_attributes": {"owner": "sam@acme.com"}, }, @@ -62,7 +59,6 @@ ingest = client.context.ingest( {"role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex"}, {"role": "assistant", "content": "Got it, short answers."}, ], - "context_category": "user_preference", "happened_at": "2026-09-01", }, ]), @@ -79,7 +75,6 @@ const ingest = await client.context.ingest({ context_id: "refund-policy", title: "Refund policy", text: "Refunds are processed within 5 business days.", - context_category: "business_knowledge", attributes: { department: "support" }, custom_attributes: { owner: "sam@acme.com" }, }, @@ -89,7 +84,6 @@ const ingest = await client.context.ingest({ { role: "user", content: "Keep answers short, I read on my phone.", name: "alex" }, { role: "assistant", content: "Got it, short answers." }, ], - context_category: "user_preference", happened_at: "2026-09-01", }, ]), @@ -128,7 +122,7 @@ The response is `202 Accepted`: - `results[].infer` mirrors the item's `enrich`. - `results[].status` is `queued` or `failed`. A failed item carries `error` and `error_code`; the other items in the request are still queued. -A `202` means the items were accepted and queued, not that they are searchable yet. See [Verify processing](#14-verify-processing). +A `202` means the items were accepted and queued, not that they are searchable yet. See [Verify processing](#13-verify-processing). --- @@ -142,7 +136,7 @@ A `202` means the items were accepted and queued, not that they are searchable y | `enrich` | Request-level default for every item's `enrich`. Default `true`. | | `upsert` | Request-level default for every item's `upsert`. Default `true`. | | `instructions` | Request-level default for every item's `instructions`. Default empty. | -| `graph_payload` | Optional. A graph you built yourself, keyed by `context_id`. See [Bring your own graph](#12-bring-your-own-graph). | +| `graph_payload` | Optional. A graph you built yourself, keyed by `context_id`. See [Bring your own graph](#11-bring-your-own-graph). | The three request-level defaults apply to any item that does not set the field itself, so one call can enrich some items and store others verbatim, or replace some items and append others. @@ -164,9 +158,8 @@ Each item is exactly one of `text` or `conversation`. | `happened_at` | The date the item is about, `YYYY-MM-DD` only. A timestamp is a `400`. HydraDB records when it received the item separately. | | `attributes` | Declared, filterable fields from the database's `database_metadata_schema`. See [Attributes](/essentials/v2/attributes). | | `custom_attributes` | Free-form fields. Not filterable. | -| `context_category` | Optional label: `auto` (the default), `user_preference`, `business_knowledge` or `decision_trace`. You set it; nothing infers it. Validated strictly, so a typo is a `400`. See [Context categories](/essentials/v2/context-categories). | -| `forceful_relations` | Relations you declare to other items: `{ "ids": ["chat-w1"], "properties": {} }`, where `ids` are the `context_id`s of the related items. See [Declared relations](#11-declared-relations). | -| `acl` | Principals allowed to retrieve the item, such as `user_email:a@x.com` or `domain:acme.com`. Omit for unrestricted, `[]` for nobody. A malformed principal is a `400`. See [Restricting an item](#10-restricting-an-item). | +| `forceful_relations` | Relations you declare to other items: `{ "ids": ["chat-w1"], "properties": {} }`, where `ids` are the `context_id`s of the related items. See [Declared relations](#10-declared-relations). | +| `acl` | Principals allowed to retrieve the item, such as `user_email:a@x.com` or `domain:acme.com`. Omit for unrestricted, `[]` for nobody. A malformed principal is a `400`. See [Restricting an item](#9-restricting-an-item). | | `is_markdown` | Chunk `text` on its markdown structure instead of as flat prose. | | `user_name` | The speaker for a text item. On a conversation, the per-turn `name` wins. | @@ -246,21 +239,7 @@ Use `instructions` to steer extraction. Set it on the request to apply it to eve --- -## 7. Context categories - -Each item can say what kind of context it is: - -| Category | For | -| --- | --- | -| `user_preference` | What a person likes, chose or asked for | -| `business_knowledge` | Documentation, policy, product and domain facts | -| `decision_trace` | What an agent or a team decided, and why | - -The label is yours to set; HydraDB never infers or changes it. Leave `context_category` out, or send `auto`, and the item is stored and enriched as general context with no category on its chunks. A preference or a decision is sent as text (or a conversation) with the matching label. See [Context categories](/essentials/v2/context-categories). - ---- - -## 8. Attributes and custom attributes +## 7. Attributes and custom attributes ```json { @@ -274,13 +253,13 @@ The label is yours to set; HydraDB never infers or changes it. Leave `context_ca --- -## 9. Time +## 8. Time `happened_at` is when the item is about: the meeting date, the decision date, the day a preference was stated. HydraDB records when it received the item separately. Set `happened_at` whenever it differs from ingest time, so recency and temporal reasoning at query time use the right date. --- -## 10. Restricting an item +## 9. Restricting an item ```json { @@ -294,7 +273,7 @@ Omit `acl` and the item is unrestricted. Send `[]` and nobody can retrieve it. A --- -## 11. Declared relations +## 10. Declared relations Any item, text or conversation, can declare which other items it relates to: @@ -306,11 +285,11 @@ Any item, text or conversation, can declare which other items it relates to: } ``` -`ids` are the `context_id`s of the related items. At query time, with `follow_forceful_relations` on (the default), a hit on one item pulls its declared relations into the response's `forceful_relations[]`, each with the `via` link that brought it in. See [Query](/essentials/v2/query#forceful_relations). +`ids` are the `context_id`s of the related items. At query time, in `thinking` mode and with `follow_forceful_relations` on (the default), a hit on one item pulls its declared relations into the response's `forceful_relations[]`, each with the `via` link that brought it in. See [Query](/essentials/v2/query#forceful_relations). --- -## 12. Bring your own graph +## 11. Bring your own graph Skip extraction for an item and supply its entities and relations yourself with `graph_payload`, a map keyed by `context_id`: @@ -338,7 +317,7 @@ Every key in `graph_payload` must match the `context_id` of an item in the same --- -## 13. IDs and replacement +## 12. IDs and replacement - `context_id` is yours. Reuse it to replace an item. - `upsert: true` (the default) **replaces**: re-ingesting a `context_id` deletes everything derived from the previous version (its chunks and its graph contribution) before writing the new one. It does not merge. @@ -348,7 +327,7 @@ Every key in `graph_payload` must match the `context_id` of an item in the same --- -## 14. Verify processing +## 13. Verify processing Ingestion is asynchronous. Poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) with the `results[].id` values from the ingest response until each item reaches `completed` or `errored`. @@ -401,13 +380,13 @@ To be notified instead of polling, register a [webhook](/essentials/v2/webhooks) --- -## 15. Other ways context arrives +## 14. Other ways context arrives [Connectors](/essentials/v2/connectors) sync Slack, Notion, Google Drive, GitHub and other tools into a database on a schedule. Synced context lands in the same database as your items and is queried together with them. --- -## 16. Common mistakes +## 15. Common mistakes @@ -419,9 +398,6 @@ Ingest takes text only. Extract the text from the file in your application and s An unrecognised field is refused or dropped, never guessed at. Use the names in [Item fields](#3-item-fields). - -`context_category` is validated strictly. `"business-knowledge"` is a `400`, not a silent fallback to `auto`. Use `user_preference`, `business_knowledge`, `decision_trace` or `auto`. - Only `user`, `assistant` and `system` are accepted. Map roles like `tool` or `human` before sending. @@ -440,7 +416,6 @@ A `202` means queued. Poll status until `graph_creation` or `completed` before e ## Related -- [Context categories](/essentials/v2/context-categories) - [Query](/essentials/v2/query) - [Attributes](/essentials/v2/attributes) - [Ingest context API reference](/api-reference/v2/endpoint/ingest-context) diff --git a/essentials/v2/query.mdx b/essentials/v2/query.mdx index 4a57d395..aed2dc1a 100644 --- a/essentials/v2/query.mdx +++ b/essentials/v2/query.mdx @@ -66,7 +66,6 @@ The response `data` is exactly these four keys, inside the usual envelope: "score": 0.91, "content": "Refunds are processed within 30 days of purchase by the Finance Department.", "enrichment": "Refund window is 30 days; Finance owns refund processing.", - "enrichment_kind": "business_knowledge", "temporal": [ { "content": "Refund policy effective_from June 2026. Start: 2026-06-01", @@ -80,8 +79,7 @@ The response `data` is exactly these four keys, inside the usual envelope: "context_id": "chat-2026-07-29", "score": 0.84, "content": "user: Keep refund answers short please\nassistant: Got it.", - "enrichment": "User prefers short answers about refunds.", - "enrichment_kind": "user_preference" + "enrichment": "User prefers short answers about refunds." } ], "graph": [ @@ -145,7 +143,7 @@ The response `data` is exactly these four keys, inside the usual envelope: } } ], - "llm_prompt": "# Query results\n\n**Query:** who owns refund processing?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" + "llm_prompt": "# Query results\n\n**Query:** How are refunds processed, and how should I answer this user?\n**Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation\nCite a result by its number in brackets, e.g. [1].\n\n## Results\n\n### 1. Refund policy\n- **Relevance:** 0.91 · **Collection:** support · **Type:** file\n- **Id:** refund-policy · **Last updated:** 2026-07-02\n\nRefunds are processed within 30 days of purchase by the Finance Department.\n\n**Enrichment:** Refund window is 30 days; Finance owns refund processing.\n\n---\n\n### 2. Support chat with Priya\n- **Relevance:** 0.84 · **Collection:** support · **Type:** message\n- **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29\n\nuser: Keep refund answers short please\nassistant: Got it.\n\n**Enrichment:** User prefers short answers about refunds.\n\n## Forceful relations\n\nLinked to a result by the author at ingest time (forceful_relations), not by relevance to this query.\n\n### R1. Refund FAQ\n- **Linked from:** refund-policy · **Collection:** support\n- **Id:** refund-faq\n\nFAQ: refunds to a card take 5 to 7 business days to appear.\n\n## Related facts\n\n- [P1] **Refund Processing** -managed by→ **Finance Department** (relevance 0.81) [1]\n Refund processing is managed by the Finance Department.\n- [P2] **User** -prefers→ **short answers** (relevance 0.74) [2]\n The user prefers short answers about refunds.\n\n## Temporal facts\n\n- **Refund policy** *effective_from* → **June 2026** (from 2026-06-01, precision: month, status: ongoing; evidence: \"from June\") [1]\n\n## Sources\n\n1. **Refund policy** (file, id: refund-policy) · https://docs.acme.com/refunds · updated 2026-07-02\n2. **Support chat with Priya** (message, id: chat-2026-07-29) · updated 2026-07-29\n3. **Refund FAQ** (id: refund-faq)" }, "error": null, "meta": { @@ -239,7 +237,7 @@ The matched pieces of your items, ranked. Preserve the order. | `score` | number | Relevance. Always present. | | `content` | string | The chunk's own text, verbatim. Enrichment is not concatenated into it. | | `enrichment` | string | What enrichment extracted from this chunk: the extracted statement (a preference, a fact). Omitted when enrichment extracted nothing. | -| `enrichment_kind` | string | The item's declared `context_category`: `user_preference`, `business_knowledge` or `decision_trace`. Omitted when none was declared (`auto`); present even when `enrichment` is omitted. | +| `enrichment_kind` | string | An optional label; omitted when none was set. | | `temporal` | array | Present only when the query engaged temporal reasoning. Each entry is `{ content, start_date, end_date }`: `content` reads `. Start: YYYY-MM-DD, End: YYYY-MM-DD` (only the dated sides are printed), and either date may be `null`. | @@ -372,14 +370,14 @@ The `llm_prompt` from the [example response](#1-one-call), unescaped: ```markdown # Query results -**Query:** who owns refund processing? +**Query:** How are refunds processed, and how should I answer this user? **Found:** 2 results across 2 sources · 2 related facts · 1 temporal fact · 1 forceful relation Cite a result by its number in brackets, e.g. [1]. ## Results ### 1. Refund policy -- **Relevance:** 0.91 · **Collection:** support · **Type:** file · **Category:** business_knowledge +- **Relevance:** 0.91 · **Collection:** support · **Type:** file - **Id:** refund-policy · **Last updated:** 2026-07-02 Refunds are processed within 30 days of purchase by the Finance Department. @@ -389,7 +387,7 @@ Refunds are processed within 30 days of purchase by the Finance Department. --- ### 2. Support chat with Priya -- **Relevance:** 0.84 · **Collection:** support · **Type:** message · **Category:** user_preference +- **Relevance:** 0.84 · **Collection:** support · **Type:** message - **Id:** chat-2026-07-29 · **Last updated:** 2026-07-29 user: Keep refund answers short please @@ -437,7 +435,7 @@ const messages = [{ role: "system", content: result.data.llmPrompt }, { role: "user", content: question }]; ``` -Surface it to your agent verbatim, and let the model cite the labels. When you need structured output instead, read `chunks[].content`, `chunks[].enrichment`, `chunks[].enrichment_kind`, `graph[].path_summary` and `forceful_relations[]`. See [How to Use API Results](/essentials/v2/api-results). +Surface it to your agent verbatim, and let the model cite the labels. When you need structured output instead, read `chunks[].content`, `chunks[].enrichment`, `graph[].path_summary` and `forceful_relations[]`. See [How to Use API Results](/essentials/v2/api-results). --- diff --git a/get-started/v2/core-concepts.mdx b/get-started/v2/core-concepts.mdx index 89cf828c..d70cdcd7 100644 --- a/get-started/v2/core-concepts.mdx +++ b/get-started/v2/core-concepts.mdx @@ -1,6 +1,6 @@ --- title: "Core Concepts" -description: "A tour of the primitives that make HydraDB: databases and collections, items, context categories, query, attributes, the context graph and access control." +description: "A tour of the primitives that make HydraDB: databases and collections, items, query, attributes, the context graph and access control." --- > A short overview of each primitive, with links to the page that covers it in depth. @@ -9,7 +9,6 @@ description: "A tour of the primitives that make HydraDB: databases and collecti | --- | --- | --- | | **Databases and collections** | Isolated databases, partitioned into collections per user, team or project | [Databases and collections](/essentials/v2/databases-and-collections) | | **Context (items)** | Text and conversations you ingest, one item at a time | [Ingest context](/essentials/v2/ingest) | -| **Context categories** | What kind of context an item is: a user preference, business knowledge or a decision trace | [Context categories](/essentials/v2/context-categories) | | **Query** | One endpoint that reads context back, personalized with weighted collections | [Query](/essentials/v2/query) | | **Attributes** | Declared fields you filter on, for deterministic retrieval | [Attributes](/essentials/v2/attributes) | | **Context graph** | Entities, relations and decisions extracted from everything you ingest | [Context graphs](/essentials/v2/context-graphs) | @@ -25,6 +24,8 @@ One database holds all the context your AI needs, and it holds three kinds: - **Business knowledge:** what your company knows - **Decision traces:** what your agents and teams decided, and why +These are kinds of content, not a setting you pass: you send all of them as text or conversations. + You partition the database into **collections**, typically one per person plus one or more shared ones. You ingest everything as **items**. You read it back with one **query** that can weigh a person's collection above the shared ones, so answers are grounded in company knowledge and personalized for the person asking. --- @@ -56,22 +57,6 @@ Read more: [Ingest context](/essentials/v2/ingest) --- -## Context categories - -Each item can carry a `context_category`: - -| Category | For | -| --- | --- | -| `user_preference` | Likes, choices and requests a person made. Usually conversations, stored in that person's collection. | -| `business_knowledge` | Documentation, policy, product and domain facts. Usually in a shared collection. | -| `decision_trace` | Agent runs, architecture decisions, postmortems, approvals. They show up in the graph as decisions connected to who made them. | - -The label is yours to set. Leave it out, or send `auto`, and the item carries no label; HydraDB never infers one. - -Read more: [Context categories](/essentials/v2/context-categories) - ---- - ## Query Query is how agents read from HydraDB. Storing data is easy; knowing _what_ to retrieve, _when_, and _why_ is the hard part. diff --git a/get-started/v2/quickstart.mdx b/get-started/v2/quickstart.mdx index ee368a08..a6f54021 100644 --- a/get-started/v2/quickstart.mdx +++ b/get-started/v2/quickstart.mdx @@ -41,7 +41,7 @@ curl 'https://api.hydradb.com/databases' \ ## Create, ingest, query -Create a database, ingest a company policy and a customer conversation in one call, wait for both to index, then ask a question that needs both. +Create a database, ingest a company policy into a shared collection and a customer conversation into that customer's own collection, wait for both to index, then ask a question that needs both. ```python Python SDK @@ -68,7 +68,6 @@ client.context.ingest( "context_id": "refund-policy", "title": "Refund policy", "text": "Refunds are processed within 5 business days.", - "context_category": "business_knowledge", }]), ) client.context.ingest( @@ -80,7 +79,6 @@ client.context.ingest( {"role": "user", "content": "Keep answers short, I read on my phone.", "name": "alex"}, {"role": "assistant", "content": "Got it, short answers."}, ], - "context_category": "user_preference", "happened_at": "2026-09-01", }]), ) @@ -133,7 +131,6 @@ await client.context.ingest({ context_id: "refund-policy", title: "Refund policy", text: "Refunds are processed within 5 business days.", - context_category: "business_knowledge", }]), }); await client.context.ingest({ @@ -145,7 +142,6 @@ await client.context.ingest({ { role: "user", content: "Keep answers short, I read on my phone.", name: "alex" }, { role: "assistant", content: "Got it, short answers." }, ], - context_category: "user_preference", happened_at: "2026-09-01", }]), }); @@ -200,8 +196,7 @@ curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ \"context\": [{ \"context_id\": \"refund-policy\", \"title\": \"Refund policy\", - \"text\": \"Refunds are processed within 5 business days.\", - \"context_category\": \"business_knowledge\" + \"text\": \"Refunds are processed within 5 business days.\" }] }" curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ @@ -215,7 +210,6 @@ curl -s -X POST "$API/context/ingest" "${AUTH[@]}" \ { \"role\": \"user\", \"content\": \"Keep answers short, I read on my phone.\", \"name\": \"alex\" }, { \"role\": \"assistant\", \"content\": \"Got it, short answers.\" } ], - \"context_category\": \"user_preference\", \"happened_at\": \"2026-09-01\" }] }" @@ -282,7 +276,6 @@ Steps 1 and 3 are **asynchronous**: HydraDB provisions infrastructure and indexe | If you want to... | Read... | |---|---| | See every item field, conversations and enrichment | [Ingest context](/essentials/v2/ingest) | -| Label preferences, knowledge and decisions | [Context categories](/essentials/v2/context-categories) | | Read every field of the query response | [Query](/essentials/v2/query) | | Filter on declared fields | [Attributes](/essentials/v2/attributes) | | Scope data per user or workspace | [Databases and collections](/essentials/v2/databases-and-collections) | diff --git a/mintlify-hygiene.toml b/mintlify-hygiene.toml index bb6e1213..a5ce1587 100644 --- a/mintlify-hygiene.toml +++ b/mintlify-hygiene.toml @@ -80,6 +80,9 @@ exclude = [ "get-started/core-concepts.mdx", "get-started/introduction.mdx", "get-started/quickstart.mdx", + # Not promoted yet (PRO-1618): live by URL and noindex, out of nav on + # purpose, with no inbound links from visible pages. Not deprecated. + "essentials/v2/context-categories.mdx", ] mdx_parse_mode = "strict" diff --git a/plugins/cli.mdx b/plugins/cli.mdx index 1f0f8433..9ed0d697 100644 --- a/plugins/cli.mdx +++ b/plugins/cli.mdx @@ -183,10 +183,10 @@ does not upload files: to ingest a document, extract its text and pass it with # Store a note hydradb ingest --text "The user prefers dark mode and uses VS Code" --database my-db -# Store a document with an ID, a title, a date, a filterable attribute and a category +# Store a document with an ID, a title, a date and a filterable attribute hydradb ingest --text "Refund policy: 30-day window." --context-id policy-1 \ --title "Refund policy" --happened-at 2026-07-29 \ - --attributes '{"team": "support"}' --category business_knowledge --database my-db + --attributes '{"team": "support"}' --database my-db # Store a conversation, declared related to the policy hydradb ingest --conversation-file ./chat.json --context-id chat-w1 \ @@ -216,7 +216,6 @@ cat notes.txt | hydradb ingest --title "Meeting notes" --database my-db | `--happened-at` | The date the item is about, `YYYY-MM-DD` | | `--attributes` | Declared, filterable attributes as a JSON object (keys from the database's metadata schema) | | `--custom-attributes` | Free-form attributes as a JSON object, not filterable | -| `--category` | `context_category` label: `auto` (default, no label), `user_preference`, `business_knowledge` or `decision_trace` | | `--forceful-relation` | A context ID this item is declared related to; repeatable | | `--acl` | A principal allowed to retrieve the item (for example `user_email:a@x.com` or `domain:acme.com`); repeatable | | `--upsert` / `--no-upsert` | Replace an existing item with the same context ID (default on) | diff --git a/plugins/mcp.mdx b/plugins/mcp.mdx index 09b42e62..109e1e1a 100644 --- a/plugins/mcp.mdx +++ b/plugins/mcp.mdx @@ -534,7 +534,6 @@ exactly one of `text` or `turns`. Passing both is rejected. | `happened_at` | string | No | Calendar date `YYYY-MM-DD` when the fact was true, as opposed to when it was stored | | `attributes` | object | No | Declared, filterable key/value attributes (keys from the database's metadata schema). See [Attributes](/essentials/v2/attributes) | | `custom_attributes` | object | No | Free-form key/value data stored with the item, not filterable | -| `context_category` | string | No | Your label for the item: `user_preference`, `business_knowledge`, `decision_trace`, or `auto` (default, no label). Nothing infers it. See [Context categories](/essentials/v2/context-categories) | | `forceful_relations` | string[] | No | Context IDs this item is declared related to, such as the thread or document it belongs to. A later query that returns this item can pull them in under Forceful relations | | `acl` | string[] | No | Principals that may read the item: an email, a `domain:`, or a `group::`. Omit it for an item anyone holding the key may read | | `database` | string | No | Database (tenant) scope override for this request | From 24f22d733b98a9270c05bbb732ca2bb888a2d02a Mon Sep 17 00:00:00 2001 From: SohamRatnaparkhi Date: Wed, 23 Sep 2026 17:27:43 +0530 Subject: [PATCH 7/7] docs: smooth the new-user path through Get started, Build and the API reference (PRO-1618) Sidebar: Get started reads Introduction, Quickstart, Core concepts, the order the Introduction's own steps give. Build now opens with Databases and collections (where data goes, and the one deprecation note), then Ingest, Query and How to Use API Results, ahead of Attributes, Connectors, Access control and Webhooks. The Cypher graph collections page keeps its title but gets its own sidebar label, so the sidebar no longer shows two "Bring Your Own Graph" entries; the ingest-time BYOG page points to it in its "pick the right tool" table. Contradictions between pages: - Ingest said an unknown item field is "refused or dropped"; the server ignores it without an error, as Ingest Context and AGENTS.mdx say. - Ingest's SDK note listed enrich and instructions as SDK form fields; Ingest Context says the SDK methods take them per item. Both now say so. - Architecture named the status error field message; it is error_message. Two names for one thing: Update Source Metadata and the API reference index now say that database_metadata and additional_metadata are this endpoint's names for an item's attributes and custom_attributes, and that the ingest names are not read there. The index's inspect row says what inspect is for instead of describing list. Signed-off-by: SohamRatnaparkhi Co-Authored-By: Claude Opus 5.5 (1M context) --- api-reference/v2/endpoint/update-source-metadata.mdx | 6 +++++- api-reference/v2/index.mdx | 8 ++++---- docs.json | 12 ++++++------ essentials/v2/architecture.mdx | 2 +- essentials/v2/bring-your-own-graph.mdx | 3 ++- essentials/v2/graph-collections-byog.mdx | 1 + essentials/v2/ingest.mdx | 8 ++++---- 7 files changed, 23 insertions(+), 17 deletions(-) diff --git a/api-reference/v2/endpoint/update-source-metadata.mdx b/api-reference/v2/endpoint/update-source-metadata.mdx index 00be8771..d74d46c5 100644 --- a/api-reference/v2/endpoint/update-source-metadata.mdx +++ b/api-reference/v2/endpoint/update-source-metadata.mdx @@ -8,6 +8,10 @@ import { Field } from "/snippets/field.jsx"; Use this endpoint when you know a source ID and need to update its metadata in place. It updates both the source row and indexed chunk metadata used by query/list filters. + + This endpoint uses older names for the fields you set at ingest: an item's `attributes` are `database_metadata` here, and its `custom_attributes` are `additional_metadata`. The source ID is the item's `context_id`. + + ```http PATCH /context/{id}/metadata ``` @@ -107,7 +111,7 @@ const response = await fetch("https://api.hydradb.com/context/policy_main/metada At least one of `database_metadata` or `additional_metadata` is required. - This edit endpoint uses `database_metadata` for schema-backed source metadata (deprecated alias: `tenant_metadata` - still accepted, but the canonical field wins if both are sent). The shorter `metadata` field used by ingestion/list examples is not accepted in this PATCH body. `document_metadata` is also not accepted; use `additional_metadata`. + This edit endpoint uses `database_metadata` for schema-backed source metadata (deprecated alias: `tenant_metadata` - still accepted, but the canonical field wins if both are sent). The ingest names `attributes` and `custom_attributes`, and the `metadata` name on list rows, are not read by this PATCH body. `document_metadata` is rejected; use `additional_metadata`. ## Behavior diff --git a/api-reference/v2/index.mdx b/api-reference/v2/index.mdx index 0d4c1508..af5de553 100644 --- a/api-reference/v2/index.mdx +++ b/api-reference/v2/index.mdx @@ -29,7 +29,7 @@ description: "Single reference to all HydraDB endpoints" | [Context items](/essentials/v2/ingest) | A `text` or a `conversation`, sent in the `context` list of `POST /context/ingest`. | Everything you ingest. Shared context goes in a shared collection; a person's preferences go in their own. | | `database_metadata_schema` | Database-level fields you define up front so metadata can be filtered or queried consistently. | Use it for stable fields like department, customer, region, plan, category, or compliance label. | | `attributes` | Declared, filterable fields on an item, matching `database_metadata_schema`; `custom_attributes` are free-form. | Send them at ingest; filter with `attributes` on `/query`. | -| `ids` | IDs returned by ingestion or visible from `/context/list`. | Use them when polling processing status, inspecting content, listing a specific subset, deleting sources, or inspecting relations. | +| `ids` | IDs returned by ingestion or visible from `/context/list`. | Use them when polling processing status, inspecting content, listing a specific subset, deleting items, or inspecting relations. | ## End-to-end lifecycle @@ -102,9 +102,9 @@ SDK methods mirror the API: `client..()` maps to the correspondin | [`/databases/stats`](/api-reference/v2/endpoint/tenant-stats) | `GET` | `databases.stats` | Get usage statistics | You want to monitor object counts for a database. | | [`/context/ingest`](/api-reference/v2/endpoint/ingest-context) | `POST` | `context.ingest` | Ingest context items | You are sending text or conversations. | | [`/context/status`](/api-reference/v2/endpoint/source-status) | `GET` | `context.status` | Check processing status | You have IDs from ingestion and need to know when they are queryable. | -| [`/context/inspect`](/api-reference/v2/endpoint/fetch-content) | `GET` | `context.inspect` | Read an item's stored content | Query chunks carry no source details; look up an item's title and attributes with `POST /context/list` and `ids`. | +| [`/context/inspect`](/api-reference/v2/endpoint/fetch-content) | `GET` | `context.inspect` | Read an item's stored content | You need the full stored content behind a `context_id`, such as the item a query chunk came from. For its title and attributes, use `POST /context/list` with `ids`. | | [`/context/list`](/api-reference/v2/endpoint/list-documents) | `POST` | `context.list` | Browse items | You need pagination, filters, field projection, or a specific subset by `ids`. | -| [`/context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) | `PATCH` | `context.update_source_metadata` | Update an item's metadata | You need to merge `database_metadata` or `additional_metadata` onto one existing item without re-ingesting. | +| [`/context/{id}/metadata`](/api-reference/v2/endpoint/update-source-metadata) | `PATCH` | `context.update_source_metadata` | Update an item's metadata | You need to change one existing item's attributes (`database_metadata`) or custom attributes (`additional_metadata`) without re-ingesting. | | [`/context`](/api-reference/v2/endpoint/delete-source) | `DELETE` | `context.delete` | Delete items | You need to remove one or more items by ID. | | [`/context/relations`](/api-reference/v2/endpoint/source-relations) | `GET` | `context.relations` | Inspect entity relationships | You need graph relations for an item or collection. | | [`/context/{id}/subgraph`](/api-reference/v2/endpoint/subgraph) | `GET` | `context.subgraph` | Walk everything connected to one item | You need an item's thread, replies, parents, children and linked items. | @@ -185,6 +185,6 @@ Rate limits apply per API key. For production deployments, build retry logic wit ## Next steps - **Build something:** [Quickstart](/get-started/v2/quickstart) walks through your first integration in five minutes -- **Understand the model:** [Core Concepts](/get-started/v2/core-concepts) explains databases, items, categories, query and attributes +- **Understand the model:** [Core Concepts](/get-started/v2/core-concepts) explains databases, items, query and attributes - **Go deeper:** [Usage](/essentials/v2/query) covers each primitive in depth - **Install an SDK:** [Python](https://pypi.org/project/hydradb-sdk/) · [TypeScript](https://www.npmjs.com/package/@hydradb/sdk) diff --git a/docs.json b/docs.json index 770ef687..a4f38175 100644 --- a/docs.json +++ b/docs.json @@ -49,20 +49,21 @@ "public": true, "pages": [ "get-started/v2/introduction", - "get-started/v2/core-concepts", - "get-started/v2/quickstart" + "get-started/v2/quickstart", + "get-started/v2/core-concepts" ] }, { "group": "Build", "public": true, "pages": [ + "essentials/v2/databases-and-collections", "essentials/v2/ingest", - "essentials/v2/connectors", "essentials/v2/query", + "essentials/v2/api-results", "essentials/v2/attributes", + "essentials/v2/connectors", "essentials/v2/access-control", - "essentials/v2/api-results", "essentials/v2/webhooks" ] }, @@ -71,11 +72,10 @@ "public": true, "pages": [ "essentials/v2/architecture", - "essentials/v2/databases-and-collections", "essentials/v2/context-graphs", + "essentials/v2/semantic-search", "essentials/v2/bring-your-own-graph", "essentials/v2/graph-collections-byog", - "essentials/v2/semantic-search", "essentials/v2/glossary", "continuity-assurance" ] diff --git a/essentials/v2/architecture.mdx b/essentials/v2/architecture.mdx index 039cbc1d..2b096cdf 100644 --- a/essentials/v2/architecture.mdx +++ b/essentials/v2/architecture.mdx @@ -115,7 +115,7 @@ flowchart LR Poll [`GET /context/status`](/api-reference/v2/endpoint/source-status) with the returned `id` to follow each item through the pipeline. Two practical notes: - **`graph_creation` is already queryable.** Chunks become retrievable as soon as embedding finishes; you only need to wait for `completed` when you specifically need full graph context (`graph_context: true` on query). -- **Failures surface with detail.** An `errored` status comes back with an `error_code` and `message` so you can distinguish parse failures from validation problems from infrastructure issues. +- **Failures surface with detail.** An `errored` status comes back with an `error_code` and `error_message` so you can distinguish parse failures from validation problems from infrastructure issues. The full status table and polling pattern lives at [Ingestion Status](/api-reference/v2/endpoint/source-status). diff --git a/essentials/v2/bring-your-own-graph.mdx b/essentials/v2/bring-your-own-graph.mdx index e54968fb..b0bc6bd0 100644 --- a/essentials/v2/bring-your-own-graph.mdx +++ b/essentials/v2/bring-your-own-graph.mdx @@ -26,6 +26,7 @@ Pick the right tool: | HydraDB to discover relationships for you | [Context graphs](/essentials/v2/context-graphs) (auto-extraction, the default) | | To declare links **between whole items** | `forceful_relations` on an item. See [Declared relations](/essentials/v2/ingest#10-declared-relations). | | To supply the **full entity and relation graph for one item** | **Bring Your Own Graph** (this page) | +| A standalone property graph you write and read with **Cypher**, separate from context items | [Cypher graph collections](/essentials/v2/graph-collections-byog) | --- @@ -91,7 +92,7 @@ In a JSON body, `graph_payload` is an object. The SDKs send a multipart form, wh | Relation `context` length | ≤ 2,000 characters | | Entity key, `name`, `type`, `namespace`, `identifier`, `predicate` and `temporal_details` length | ≤ 256 characters each | -The request itself keeps the normal ingest limits: at most 100 items, 1 MiB of text per item and 8 MiB of text per request. See [Ingest context](/essentials/v2/ingest#limits-and-refused-fields). +The request itself keeps the normal ingest limits: at most 100 items, 1 MiB of text per item and 8 MiB of text per request. See [Ingest context](/essentials/v2/ingest#limits-and-unrecognised-fields). --- diff --git a/essentials/v2/graph-collections-byog.mdx b/essentials/v2/graph-collections-byog.mdx index 1a7aaa3c..36140740 100644 --- a/essentials/v2/graph-collections-byog.mdx +++ b/essentials/v2/graph-collections-byog.mdx @@ -1,5 +1,6 @@ --- title: "Bring Your Own Graph (BYOG)" +sidebarTitle: "Cypher graph collections" description: "BYOG - full Cypher access to graph collections you own end-to-end." --- diff --git a/essentials/v2/ingest.mdx b/essentials/v2/ingest.mdx index 302af209..6099e31f 100644 --- a/essentials/v2/ingest.mdx +++ b/essentials/v2/ingest.mdx @@ -94,7 +94,7 @@ console.log(ingest.data.results.map((r) => r.id)); -**SDK users: the form field is still called `items`.** The SDKs send a multipart form rather than a JSON body, and the array goes in the `items` form field, which is why `items` is a JSON string there. The form also takes `database`, `collection`, `upsert`, `enrich`, `instructions` and `graph_payload` as fields. Both entry points run the same validation. Prefer the JSON body with `context` when you call the API directly. Keys inside each item stay `snake_case` in every language. +**SDK users: the form field is still called `items`.** The SDKs send a multipart form rather than a JSON body, and the array goes in the `items` form field, which is why `items` is a JSON string there. The SDK methods also take `database`, `collection`, `upsert` and `graph_payload`; to set `enrich` or `instructions` through an SDK, set them on each item. Both entry points run the same validation. Prefer the JSON body with `context` when you call the API directly. Keys inside each item stay `snake_case` in every language. The response is `202 Accepted`: @@ -163,11 +163,11 @@ Each item is exactly one of `text` or `conversation`. | `is_markdown` | Chunk `text` on its markdown structure instead of as flat prose. | | `user_name` | The speaker for a text item. On a conversation, the per-turn `name` wins. | -### Limits and refused fields +### Limits and unrecognised fields - At most **100 items** per request, **1 MiB** of text per item, and **8 MiB** of text per request. - A validation error names the item it refers to as `context[N]`. -- An unrecognised field is either refused with a `400` or dropped without an error, so check spelling against the tables above. +- An unrecognised field on an item is ignored without an error, so check spelling against the tables above. --- @@ -396,7 +396,7 @@ An item carries exactly one of `text` or `conversation`. Sending both, or neithe Ingest takes text only. Extract the text from the file in your application and send it as a `text` item. See [Turning files into items](#turning-files-into-items). -An unrecognised field is refused or dropped, never guessed at. Use the names in [Item fields](#3-item-fields). +An unrecognised field is ignored without an error, never guessed at. Use the names in [Item fields](#3-item-fields). Only `user`, `assistant` and `system` are accepted. Map roles like `tool` or `human` before sending.