Skip to content

feat: P1 usability findings from the demo evaluation (#232) - #234

Closed
vishr wants to merge 5 commits into
fix/p0-demo-findingsfrom
fix/p1-dashboard-usability
Closed

vishr wants to merge 5 commits into
fix/p0-demo-findingsfrom
fix/p1-dashboard-usability

Conversation

@vishr

@vishr vishr commented Sep 7, 2026 •

Copy link
Copy Markdown
Member

What changed

P1 items 9–13 from #232 — the places where the product knew something and did not act on it. Two commits.

Rows lead somewhere, links carry the view, removal asks first

  • Recent activity rows lit up on hover and did nothing when clicked. They open the same investigation a service-map node does, reachable by keyboard, labelled for a screen reader, and only clickable when an agent is there to answer.
  • The window and namespace lived only in the saved dashboard, so a shared link opened on whatever the recipient had chosen and two people discussed different numbers. Both are in the address bar now. A link that carries them wins for the visit; nothing is written back until the recipient changes a control themselves, and an unrecognised value is dropped rather than rejected so a stale link still opens.
  • Removing a view took effect on the click, and a dashboard has no undo. It asks first and says the card comes off for everyone.
  • A URL naming a dashboard that is not there says so and offers the default, instead of quietly rendering the default while the address bar pointed elsewhere.

Search finds services, and a generated dashboard admits its empty cards

  • Searching "checkout" answered "No matching chats / No matching dashboards" while checkout was live, traced and logged — search only knew what the user had already named. Matching services now appear under their own heading and open an investigation. The catalogue is fetched only once someone is searching, which a test pins.
  • dashboard_create and dashboard_update probe the filtered cards they were handed and name the ones with nothing to show. The generated demo dashboard shipped "Checkout Error Logs" and "Checkout Warnings", both empty on arrival and permanently so, and nothing in the tool result told the model — which described them to the user as working views. Probes are capped at eight per call, and a failed probe is silent: a dashboard that saved correctly is never reported as an error because a follow-up query failed.

Verification

  • just check
  • just test-race when auth, API, ingest, query, MCP, or agent paths changed
  • User-facing behavior and configuration docs are current
  • No credentials, private telemetry, host details, or enterprise-only source are included
  • API, migration, ingest, MCP/AG-UI, or release-contract changes are called out

New tests: activity rows opening an investigation, removal cancelled and confirmed, dashboard search-param parsing, a matching service opening an investigation, the catalogue not being fetched before a search, and the create tool both naming an empty card and staying quiet when the card has data.

MCP behaviour change to note: the dashboard_create and dashboard_update summaries can now carry a sentence about cards that will render empty. The structured output is unchanged.

Based on fix/p0-demo-findings — merge #233 first.

Part of #232 (P1 items 9, 10, 11, 12, 13).

…sks first

Three things the dashboard did not do for the person using it.

Recent activity rows lit up on hover and did nothing when clicked. They now
open the same investigation a service-map node does, reachable by keyboard and
labelled for a screen reader, and they only look clickable when an agent is
available to answer.

The window and namespace lived only in the saved dashboard, so a shared link
opened on whatever the recipient had chosen and the two people ended up
discussing different numbers. Both are in the address bar now; a link that
carries them wins for the visit, and nothing is written back until the
recipient changes a control themselves. An unrecognised value is dropped
rather than rejected, so a stale link still opens the dashboard.

Removing a view took effect on the click, and a dashboard has no undo. It asks
first, and says that the card comes off the dashboard for everyone.

A URL naming a dashboard that is not there now says so and offers the default,
instead of quietly rendering the default while the address bar still pointed
somewhere else.

Reported in #232 (P1 items 10, 12 and 13).

Claude-Session: https://claude.ai/code/session_017ovmTezqzvxVP8P9MGDQNk
…ty cards

Searching for "checkout" answered "No matching chats / No matching dashboards"
while checkout was live, traced and logged: the search only knew about things
the user had already named. Matching services now appear in the rail under
their own heading and open an investigation, and the catalogue is fetched only
once someone is actually searching.

dashboard_create and dashboard_update now probe the filtered cards they were
handed and name the ones with nothing to show. The generated demo dashboard
shipped "Checkout Error Logs" and "Checkout Warnings", both empty on arrival
and permanently so, and nothing in the tool result told the model — which went
on to describe them to the user as working views. Probes are capped, and a
probe that fails is silent: a dashboard that saved correctly is never reported
as an error because a follow-up query failed.

Reported in #232 (P1 items 9 and 11).

Claude-Session: https://claude.ai/code/session_017ovmTezqzvxVP8P9MGDQNk
…s one

Putting the namespace in the address bar made the URL authoritative for what is
shown, which also made the input unusable: every keystroke was overwritten by
the value in the URL, so a namespace that came in on a link could not be
cleared or changed. Typing now edits a draft that blur or Enter commits to both
the dashboard and the URL.

Found by driving a local build rather than by a test — the field looked right
and simply would not take input.

The empty-window hint is shortened to fit its tile, which was truncating it to
"Nothing reported in this wi…".

Reported in #232 (P1 item 12).

Claude-Session: https://claude.ai/code/session_017ovmTezqzvxVP8P9MGDQNk
…romptly

The check ran up to eight queries with no time limit while the caller waited on
a dashboard that had already been written. A log query over a busy window was
measured at twelve seconds during the evaluation, which would have made the
tool's answer slower than the work it describes.

The probes now share a three-second budget and stop once it is spent. Whatever
answered is reported; the rest go unmentioned, which is the same trade the
per-probe error path already makes.

Claude-Session: https://claude.ai/code/session_017ovmTezqzvxVP8P9MGDQNk
@vishr
vishr deleted the branch fix/p0-demo-findings September 7, 2026 03:07
@vishr vishr closed this Sep 7, 2026
@vishr
vishr deleted the fix/p1-dashboard-usability branch October 17, 2026 04:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant