feat(feedback): add hydradb feedback, and surface the request id it needs - #37
Merged
Merged
Conversation
…eeds
POST /feedback records whether a query's results were actually useful. It
correlates on ONE key -- the request_id from that query's response meta --
and nothing else about the original query is re-sent, so nothing has to be
trusted from the client.
That key was unreachable from the CLI. `_unwrap` returns `.data` and drops
`meta`, so a successful query discarded its own request id before any
caller saw it. A feedback command alone would therefore have shipped
unusable: you could run a query and then have nothing to give feedback ON.
So this is two changes:
1. `context.query` carries the request id into its payload. Additive --
/query's data has no `request_id` of its own -- so the documented
`--output json` shape gains a key and loses none, and piping it to the
feedback command works. The human output prints a copy-pasteable line,
including on an EMPTY result: a query that found nothing is the case
most worth reporting, and the one with no chunk ids to fall back on.
2. `hydradb feedback REQUEST_ID` itself, plus a _Feedback resource.
Hand-rolled rather than routed through sdk.feedback.submit (CONTRACT §2
rule 7). The SDK HAS the resource, but its generated model cannot express a
valid request: submit takes an undiscriminated union of {feedback} and
{ground_truth}, and ground_truth is itself a union of {answer} and
{source_ids}. Both unions dropped every field their branches share,
request_id included -- the correlation key the endpoint exists for is on
neither model.
It does work today, and I verified that against staging: the models are
extra="allow", so undeclared fields ride along to the wire. But that means
every field this endpoint needs travels as an accident of pydantic config,
at two levels of nesting, and sending an answer together with source ids
means picking one branch and smuggling the other past it. A generated model
that cannot name its own required field is equivalent to no model. When the
spec is fixed, this is a one-file change back to the SDK.
Details worth knowing:
- Ground-truth source ids are trimmed and de-duplicated, because they are
scored: the same document listed twice would weight one piece of evidence
as two. The 100-id cap is checked AFTER that, since the server also
de-duplicates first -- 150 ids collapsing to 80 is a request it accepts,
and refusing it locally would be worse than the round trip.
- A submission with neither prose nor ground truth is refused HERE. The
server refuses it too, but after a round trip.
- `--source` defaults to "user", not the "agent" the MCP client sends. A
person at a terminal is a user; MCP's caller is always a model. The two
populations are separated at write time, so the default has to follow who
is actually calling. `--source agent` is there for scripted runs.
- `recorded: false` is reported as "accepted but NOT durably stored".
Calling it success would tell an eval run it was captured when it was not.
11 tests, and verified end to end against staging: query -> request id
surfaced -> feedback recorded, prose and ground-truth paths both.
342 tests + 33 conformance pass, ruff clean.
Signed-off-by: gun29may <gunmayjhingran29@gmail.com>
Greptile SummaryThis PR adds a complete feedback workflow to the CLI and exposes each query’s request ID so users can correlate feedback with an existing query.
Confidence Score: 5/5The PR appears safe to merge; the previous findings are fully addressed and no new actionable defects were identified. Source validation now rejects unsupported values before network access, and the new command-level tests cover the previously untested public feedback surface. The additional validation and tests introduced since the previous review preserve the shared CLI execution, output, scope, and error contracts.
|
| Filename | Overview |
|---|---|
| src/hydradb_cli/hydra/client.py | Adds request-ID propagation and a normalized raw feedback resource with authentication, scope, validation, and response handling. |
| src/hydradb_cli/commands/_impl.py | Adds feedback validation, execution, result rendering, and query feedback hints; the previous source-validation concern is fixed. |
| src/hydradb_cli/commands/canonical.py | Defines the public feedback command options and maps them through the shared command implementation. |
| src/hydradb_cli/main.py | Registers the new top-level feedback command. |
| tests/test_cli_commands.py | Adds command-level coverage that resolves the previous testing finding and protects validation, mapping, output, and exit behavior. |
| tests/test_wrapper.py | Covers feedback wire payloads, request normalization, source-ID handling, defaults, limits, and query request-ID propagation. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart LR
User[CLI user] --> Query[hydradb query]
Query --> Context[HydraDB context client]
Context --> API[HydraDB API]
API --> Envelope[Query data plus meta.request_id]
Envelope --> Payload[Query payload with request_id]
Payload --> User
User --> Feedback[hydradb feedback REQUEST_ID]
Feedback --> Validation[Validate signal, rating, source, and scope]
Validation --> FeedbackClient[HydraDB feedback client]
FeedbackClient --> Post[POST /feedback]
Post --> Result[Recorded or not-durably-stored result]
Result --> User
Reviews (2): Last reviewed commit: "fix(feedback): validate --source, and re..." | Re-trigger Greptile
…the network Two Greptile P2s on #37, both correct. 1. --source forwarded any string. --rating was validated locally but its sibling was not, so `--source agnet` cost a round trip and a server rejection to learn what a local message could say immediately. 2. The command itself was untested. The wrapper tests call feedback.submit() directly, which left registration, option-to-wrapper mapping, both renderings and the exit code unproven -- and unlike the dashboard PR, this repo already has the harness for it (108 CliRunner invocations in test_cli_commands.py), so there was no reason not to use it. Eight command-level tests cover what the wrapper tests could not: the flags are registered, every option reaches the wrapper, an invalid rating or source is refused BEFORE the call goes out, `recorded: false` does not render as success, and --output json is the payload. Writing them surfaced a third thing, found by driving the real CLI: an empty submission reported ✗ Error: Connection error: feedback needs something to record: ... The wrapper raises HydraDBClientError(0, ...) for a local refusal, but status 0 is this codebase's marker for a TRANSPORT failure -- errors.py uses it only for connect/timeout/network -- and handle_api_error renders it as "Connection error:". So a local refusal blamed the network. The pre-existing guards avoid this by validating in the command layer (require_tenant_id), which is why `No database specified` reads cleanly; the feedback guards now do the same. The wrapper keeps its checks for anyone importing it as a library, but the command layer catches first. 350 tests + 33 conformance pass, ruff clean. Re-verified end to end against staging: every command, all four feedback shapes, and all three guards. Signed-off-by: gun29may <gunmayjhingran29@gmail.com>
abhishekt-hydra
approved these changes
Sep 11, 2026
gunmay-hydradb
pushed a commit
that referenced
this pull request
Sep 11, 2026
Two conflicts, both from #37 (feedback) landing on main first. 1. src/hydradb_cli/hydra/client.py — BOTH sides independently added a `_raw_post` helper: this branch for the titles field, #37 for /feedback. The implementations are identical below the signature, so the duplicate collapses to one. Kept main's: `json_body: Any` rather than `dict` (which still accepts this branch's call sites) and the docstring naming CONTRACT §2 rule 7. httpx sets Content-Type from `json=`, so dropping the explicit header changes nothing on the wire. 2. CHANGELOG.md — both sides appended under Unreleased. Kept both entries. Everything else merged cleanly, and both features are intact: `titles` on context.query and the _Feedback resource, each with their own tests. 354 tests + 33 conformance pass, ruff clean. Signed-off-by: gun29may <gunmayjhingran29@gmail.com>
gunmay-hydradb
pushed a commit
that referenced
this pull request
Sep 11, 2026
Minor, not patch: the command surface gains a capability. `hydradb query` accepts repeatable `--title` for exact multi-title filtering, and main has since taken `hydradb feedback` (#37) without a bump of its own, so this release carries both. Nothing was removed or renamed. Both places, because this package keeps the version twice -- pyproject.toml for packaging and __init__.py for `hydradb --version` -- and nothing in the tests or the build asserts they agree. They would drift silently, and the one users see is the one that is easiest to forget. Verified equal after the change, and `hydradb --version` reports 0.3.0. Signed-off-by: gun29may <gunmayjhingran29@gmail.com>
gunmay-hydradb
pushed a commit
that referenced
this pull request
Sep 11, 2026
Patch bump covering repeatable `--title` on hydradb query, and the `hydradb feedback` command (#37) which main took without a bump of its own. Nothing was removed or renamed, and no existing invocation changes. Both places, because this package keeps the version twice -- pyproject.toml for packaging and __init__.py for `hydradb --version` -- and nothing in the tests or the build asserts they agree. They would drift silently, and the one users see is the one that is easiest to forget. Verified equal after the change, and `hydradb --version` reports 0.2.1. Signed-off-by: gun29may <gunmayjhingran29@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
hydradb feedback REQUEST_ID— report whether a query's results were actually useful.It correlates on one key, the
request_idfrom that query's responsemeta. Nothing else about the original query is re-sent, so nothing has to be trusted from the client.This is two changes, because a feedback command alone would have shipped unusable
The request id was unreachable from the CLI.
_unwrapreturns.dataand dropsmeta, so a successful query discarded its own request id before any caller saw it — you could run a query and then have nothing to give feedback on.So
context.querynow carries the id into its payload. That is additive —/query's data has norequest_idof its own — so the documented--output jsonshape gains a key and loses none:The human output prints a copy-pasteable line, including on an empty result. A query that found nothing is the case most worth reporting, and the one with no chunk ids to fall back on.
It does work today, and I verified that against staging — the models are
extra="allow", so undeclared fields ride along to the wire. But that means every field this endpoint needs travels as an accident of pydantic config, at two levels of nesting, and sending an answer together with source ids means picking one branch and smuggling the other past it. A generated model that cannot name its own required field is equivalent to no model.When the spec is fixed so the branches keep their siblings, this is a one-file change back to the SDK.
Testing
11 new tests, including that
request_idactually reaches the wire and that an answer and source ids travel together — the two things the SDK model cannot do.Verified end to end against staging: query → request id surfaced → feedback recorded, both the prose and ground-truth paths, with real
feedback_ids returned.pytest— 342 passedpytest conformance— 33 passedmake lint— cleanRelated