feat(unified): follow the PRO-1618 unified API contract on unified databases

[SohamRatnaparkhi]

Summary

Brings the CLI onto the PRO-1618 unified API contract. A database is either split (knowledge + memory corpora, type on every call) or unified (one corpus). The CLI now reads the layout once per command from GET /databases details[].type and branches on that, never on a request flag.

Supersedes #30 (soham/pro-1618-unified), which conflicts with main. This branch starts from origin/main and ports the still-relevant parts of #30 (layout detection, databases.layouts()/layout(), --type split|unified on database create, the Type column on database list, file uploads refused on unified), then applies the contract. #30 is left open for the author to close.

Split databases: unchanged

Every existing request and rendering on a split database is byte-for-byte what it was: the SDK multipart ingest with type, the SDK query, the delete default of knowledge, the split query renderer against tests/golden/query.json. The only addition visible on a split database is one GET /databases probe per command, and a clear refusal of the new unified-only flags.

Unified databases

• hydradb query: raw JSON POST /query with no type; the four-key body (chunks, graph, forceful_relations, llm_prompt) is parsed and rendered as chunk panels (context_id, score, content, enrichment as a plain string with enrichment_kind, the declared category, beside it, temporal facts; either enrichment field can be absent, a kind with no enrichment is still shown, and the old {text, kind} object is not read), graph tables grouped by graph[].origin (query paths, whose hops cite the returned chunk they came from, apart from chunk relation paths, each listed under the chunk it hangs under by relation.chunk_id; a path with no known origin gets its own group) and a "Forceful relations" table (R1.., via.from -> via.to; each chunk has the same shape as a chunks[] item), hidden when the list is empty. relations (the old root key) is not read. New --llm prints llm_prompt, a markdown document (# Query results, ## Results, ## Forceful relations, ## Related facts, ## Temporal facts, ## Sources), verbatim on stdout (feedback hint on stderr) so it can be piped into a model call. --output json prints the body verbatim, nothing added; the request id is lifted from the envelope meta for hydradb feedback. --follow-forceful-relations/--no-follow-forceful-relations is forwarded. The renderer detects the shape by the body (llm_prompt, or graph/forceful_relations as arrays, vs chunk_content/graph_context; a split body carries graph and forceful_relations as objects), so a unified body that reaches the split path is still rendered as what it is. Only meta.request_id is read from a unified response's meta, which no longer carries tenant_id, sub_tenant_id or source_type.
• hydradb ingest: raw JSON POST /context/ingest with the context list and one item of exactly one --text or --conversation-file (JSON list of {role, content, name?}), plus --context-id, --title, --enrich/--no-enrich, --instructions, --happened-at (YYYY-MM-DD), --attributes and --custom-attributes (JSON objects), --category, repeatable --forceful-relation and --acl, --upsert/--no-upsert. Everything is validated locally and named by turn or flag before a round trip. Files, --kind, --user-name and --markdown are refused with a message. The 202's results[].source_id is rendered as the item's context id.
• list, delete, relations, subgraph, inspect: never send type; an explicit --kind is refused rather than silently dropped. The deprecated aliases that hard-code a kind (recall full, memories add, knowledge upload, ...) are refused on a unified database with the same message.
• database create --type split|unified, database list with a Type column.

The pinned SDK (hydradb-sdk>=2.1.4,<3) cannot be relied on for the unified values, so every unified request (and database create --type) goes over the wrapper's raw v2 path with the same headers, envelope unwrap and error translation as the SDK path. databases.list() stays on the SDK path: the SDK model already carries details[] with extra="allow".

One judgement call to flag: --context (additional_context) is still forwarded on a unified query when the user passes it. The contract's field list does not name it, but it is an existing v2 request field the server binds on the same request struct and nothing rejects it on unified.

Files

• src/hydradb_cli/hydra/client.py: LAYOUT_* constants, databases.create(layout=) over the raw path, databases.layouts()/layout() memoised probe, context.query_unified() returning (body, request_id), context.ingest_context(), _raw_post_with_meta().
• src/hydradb_cli/commands/_impl.py: layout helpers, shape-detecting renderer dispatch and the unified renderer, --llm output, do_ingest_unified + build_context_item + conversation/JSON/date validation, kind refusal on unified for list/delete/relations/subgraph, file refusal, database create --type, Type column.
• src/hydradb_cli/commands/canonical.py: --llm, --follow-forceful-relations, the unified ingest options and dispatch, delete --kind optional, database create --type, help text.
• tests/test_unified.py (new, 72 tests): layout probe, wrapper query/ingest bodies (exact JSON), CLI query rendering/JSON verbatim/--llm, split renderer unchanged against tests/golden/query.json, unified ingest items and refusals, split ingest unchanged, list/delete/relations/subgraph/inspect, database create/list, an end-to-end verbatim JSON run through the real wrapper, graph grouping by origin, no fallback to relations, enrichment as a string with enrichment_kind beside it (and no reading of the old object), and a pin that only meta.request_id is read on the unified path.
• tests/golden/query_unified.json (new): a real unified /query envelope rendered by the server's own handler test (hydradb-application PRO-1618): forceful_relations[], both graph[].origin values, enrichment with enrichment_kind, a start-only temporal fact and the markdown llm_prompt. The tests read its data body and hand the CLI their own request id.
• conformance/vectors.json: new ingest-unified-json vector (JSON body, context list, type forbidden). conformance/conftest.py: raw httpx calls routed to the recorder. conformance/test_conformance.py: unified dispatch. The master copy of the vectors that lives with the shared CONTRACT.md should gain the same vector.
• README.md, CHANGELOG.md: unified sections and the new options.

Verification

• pytest -q: 427 passed.

• ruff check . and ruff format --check .: clean.

• Added lines scanned for em dashes: none.

• Nothing is compacted. The readable view prints a unified chunk's content and enrichment, and a forceful chunk's content, whole (no 300/120 character trims). --llm prints llm_prompt verbatim and --output json is the body verbatim.

🤖 Generated with Claude Code

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge; no outstanding correctness, security, or repository-rule violations were identified.

Summary

The PR adds unified-database support while preserving the existing split-database behavior.

• Detects and memoizes database layout, propagating layout-discovery failures instead of guessing.
• Routes unified query, ingest, and database-creation operations through the raw v2 API with the required request shapes.
• Adds unified result rendering, LLM output, ingest validation, kind refusal, database type display, conformance coverage, documentation, and fixtures.
• The two previous findings are fixed: layout-probe failures now surface to the user, and partial unified-ingest failures exit non-zero.

_{Reviews (5) · Last reviewed commit: "fix(unified): show unified content and e..."}

[openhack-agent]

OpenHack Summary

Security review of feat(unified): follow the PRO-1618 unified API contract on unified databases. 10 changed files; 1 finding at or above the low reporting threshold.

0   0   1   0

Confidence Score: 3/5

Review the findings below before merging.

Security merge-readiness rubric: 1 = critical, 2 = high, 3 = medium, 4 = low, 5 = no reportable findings. This score reflects scan findings, not a guarantee of correctness or complete coverage.

Files Needing Attention: src/hydradb_cli/commands/_impl.py

Important Files Changed

• CHANGELOG.md (modified)
• README.md (modified)
• conformance/conftest.py (modified)
• conformance/test_conformance.py (modified)
• conformance/vectors.json (modified)
• src/hydradb_cli/commands/_impl.py (modified)
• src/hydradb_cli/commands/canonical.py (modified)
• src/hydradb_cli/hydra/client.py (modified)
• tests/golden/query_unified.json (added)
• tests/test_unified.py (added)

Prompt To Fix With AI

Review the findings for https://github.com/hydra-db/hydradb-cli/pull/38 at commit 0e2c33a8377ad0cf741bfbad49e5a28ac68c4be5. Verify each finding against the current code before fixing it. Preserve unrelated changes and run focused regression tests.

### Issue 1: [P3] Terminal escape-sequence injection through raw unified query prompt output
Vulnerability type: CWE-150: Improper Neutralization of Escape, Meta, or Control Sequences
src/hydradb_cli/commands/_impl.py:380

This pull request adds a raw-output mode for unified query results. In the `llm` branch, the CLI emits the server-built prompt verbatim with `typer.echo`. The prompt is assembled from content stored through the new unified ingest flow, so an authenticated contributor can store terminal escape sequences. A different user who runs `hydradb query ... --llm` interactively can receive those bytes in their terminal, where OSC/CSI sequences may alter display, create misleading hyperlinks or overwrite the clipboard in supporting emulators. The existing structured rendering protects API data through Rich text/table handling, but this new branch bypasses it. Piped output is not the demonstrated execution context; the issue requires interactive terminal output.

Recommendation: Preserve verbatim prompt bytes for non-interactive/piped `--llm` use, but sanitize terminal control sequences whenever output is directed to an interactive terminal. Strip CSI, OSC, and other escape sequences (including BEL and C1 variants), or render the prompt as non-markup plain text through a no-wrap terminal renderer. Add tests with OSC 8, OSC 52, CSI cursor-control, BEL, and C1 sequences for TTY output, while retaining a test that confirms the documented piped output behavior.

Last reviewed commit: 0e2c33a · View review on OpenHack

---

TIP: Mention @openhack-agent in a PR comment to request a review or ask a question. Use @openhack-agent fix all for every finding, or @openhack-agent fix unresolved threads for open review threads only.

[openhack-agent]

OpenHack reviewed this commit. See the OpenHack Summary for the confidence score and fix actions.

[openhack-agent]

OpenHack reviewed this commit. See the OpenHack Summary for the confidence score and fix actions.

[openhack-agent]

Terminal escape-sequence injection through raw unified query prompt output

Vulnerability type: CWE-150: Improper Neutralization of Escape, Meta, or Control Sequences

This pull request adds a raw-output mode for unified query results. In the llm branch, the CLI emits the server-built prompt verbatim with typer.echo. The prompt is assembled from content stored through the new unified ingest flow, so an authenticated contributor can store terminal escape sequences. A different user who runs hydradb query ... --llm interactively can receive those bytes in their terminal, where OSC/CSI sequences may alter display, create misleading hyperlinks or overwrite the clipboard in supporting emulators. The existing structured rendering protects API data through Rich text/table handling, but this new branch bypasses it. Piped output is not the demonstrated execution context; the issue requires interactive terminal output.

Location: src/hydradb_cli/commands/_impl.py:380

Recommendation:

Preserve verbatim prompt bytes for non-interactive/piped --llm use, but sanitize terminal control sequences whenever output is directed to an interactive terminal. Strip CSI, OSC, and other escape sequences (including BEL and C1 variants), or render the prompt as non-markup plain text through a no-wrap terminal renderer. Add tests with OSC 8, OSC 52, CSI cursor-control, BEL, and C1 sequences for TTY output, while retaining a test that confirms the documented piped output behavior.

Prompt To Fix With AI

Review the findings for https://github.com/hydra-db/hydradb-cli/pull/38 at commit 0e2c33a8377ad0cf741bfbad49e5a28ac68c4be5. Verify each finding against the current code before fixing it. Preserve unrelated changes and run focused regression tests.

### Issue 1: [P3] Terminal escape-sequence injection through raw unified query prompt output
Vulnerability type: CWE-150: Improper Neutralization of Escape, Meta, or Control Sequences
src/hydradb_cli/commands/_impl.py:380

This pull request adds a raw-output mode for unified query results. In the `llm` branch, the CLI emits the server-built prompt verbatim with `typer.echo`. The prompt is assembled from content stored through the new unified ingest flow, so an authenticated contributor can store terminal escape sequences. A different user who runs `hydradb query ... --llm` interactively can receive those bytes in their terminal, where OSC/CSI sequences may alter display, create misleading hyperlinks or overwrite the clipboard in supporting emulators. The existing structured rendering protects API data through Rich text/table handling, but this new branch bypasses it. Piped output is not the demonstrated execution context; the issue requires interactive terminal output.

Recommendation: Preserve verbatim prompt bytes for non-interactive/piped `--llm` use, but sanitize terminal control sequences whenever output is directed to an interactive terminal. Strip CSI, OSC, and other escape sequences (including BEL and C1 variants), or render the prompt as non-markup plain text through a no-wrap terminal renderer. Add tests with OSC 8, OSC 52, CSI cursor-control, BEL, and C1 sequences for TTY output, while retaining a test that confirms the documented piped output behavior.

---

TIP: Reply @openhack-agent or @openhack-agent fix this to fix this finding. To ask a question, mention @openhack-agent followed by your question.