Skip to content

feat(ai): hold MCP tools and chat to what an analyst may do - #2547

Merged
Makisuo merged 6 commits into
observer/enforcementfrom
observer/agents
Nov 13, 2026
Merged

Makisuo merged 6 commits into
observer/enforcementfrom
observer/agents

Conversation

@Makisuo

@Makisuo Makisuo commented Nov 12, 2026 •

Copy link
Copy Markdown
Collaborator

Third layer of the Analyst role, on top of #2546. MCP tools and the AI chat do not go through the HTTP auth middlewares, so this layer applies the same rule there: an analyst reads and builds dashboards, nothing else.

What changed

  • An analyst can call the seven dashboard tools (create_dashboard, update_dashboard, and add, update, remove, reorder and replace widgets). Every other MCP tool that changes data is refused with the same message as the API, and is not shown in their tools/list.
  • A read-only API key keeps its own rule: no tool that changes data, dashboards included. An analyst holding a read-only key gets the key's refusal.
  • The check reads the role, not a flag set where the tenant is assembled, so it holds for an API key, an MCP sign-in, a session and a chat turn alike.
  • A chat turn run for an analyst, which today means one from a linked chat connector, proposes dashboard changes for approval as it does for a member, and is offered no other approval-gated tool.
  • The in-app chat is an investigation's thread. An analyst reads it but is refused sending a follow-up or stopping a run (403): a follow-up can file a new diagnosis, and running investigations is a member's job. These routes are a raw router, so the HTTP gate below never covered them.

Verified

  • apps/ai: read-only-key.test.ts, agents.test.ts, the registry contract test, the server tests and the chat route tests, 44 passed, plus the tools/list test over the real MCP transport: an analyst's key lists create_dashboard and not create_alert_rule. An analyst gets 403 on a follow-up and on stop, and 200 on the thread's history. Cases: out of every tool in the catalog that changes data, an analyst is open to exactly the dashboard tools and Maple's feedback channel; with a read-only key, only the feedback channel; an analyst is refused a write tool by name with the role message; their chat turn proposes create_dashboard and is not offered any other mutating tool, while a member's turn still proposes all of them.
  • @maple/ai typecheck and the CI lint config on the changed files.

Not in this layer

  • On a chat connector that cannot say who clicked an approval, the approved tool still runs with the org's authority. That predates this work and is tracked separately.
  • Approving a proposal in the app (/internal/chat/apply) stays refused for an analyst. They cannot start an in-app turn, so the only proposals they could meet there are a member's.

Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.

An observer is refused every MCP tool that changes data and is not shown one in tools/list, the same as a read-only API key. The check reads the role, so it holds on every way a tenant reaches the registry: an API key, an MCP sign-in, a session and a chat turn.

A chat turn run for an observer is offered no approval-gated tool, since an observer could not approve the proposal. They can still ask questions: the read tools are unchanged.
@maple-review-bot

This comment has been minimized.

@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 17 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 4 included reviews currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 170ef471-2b8d-424a-927e-7fd5575745c9
Commits

Reviewing files that changed from the base of the PR and between f72b00b and ee977ef.

Files selected for processing: 10

  • apps/ai/src/chat/agents.test.ts
  • apps/ai/src/chat/profiles.ts
  • apps/ai/src/chat/run.ts
  • apps/ai/src/mcp/app.test.ts
  • apps/ai/src/mcp/server.ts
  • apps/ai/src/mcp/tools/__tests__/read-only-key.test.ts
  • apps/ai/src/mcp/tools/registry.ts
  • apps/ai/src/mcp/tools/types.ts
  • apps/ai/src/routes/chat-sessions.http.test.ts
  • apps/ai/src/routes/chat-sessions.http.ts
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

# Conflicts:
#	packages/domain/src/policy/roles.ts
The seven dashboard tools stay open to an analyst; every other tool that changes data is refused and hidden, and a chat turn proposes only dashboard changes to them.
The chat routes are a raw router, so the analyst gate never ran for them: an analyst could send a follow-up, which may file a new diagnosis, and stop a running pass. Both are refused now; reading the thread stays open. Also covers an analyst's tools/list over the real MCP transport.
@Makisuo Makisuo changed the title feat(ai): keep MCP tools and chat turns read-only for observers feat(ai): hold MCP tools and chat to what an analyst may do Nov 13, 2026
@maple-review-bot

maple-review-bot Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 10/10 · safe to merge. Every refusal reads the role from the resolved tenant and each branch has a test; the ai test shards were cancelled in CI, so those cases are unverified here.
no findings · tests covered · risk high

Adds a role-based analyst gate to MCP tools/list, tools/call and chat turns, and refuses analysts the investigation chat's POST routes. The open McpQueryError issues in registry.ts are in a query path this diff does not touch. Safe to merge.

  • refusesAnalyst/refusalFor refuse an analyst every mutating MCP tool but dashboards and feedback
  • tools/list hides those tools through a second visibility gate; read-only keys keep theirs
  • profileForRoles turns an analyst chat turn's ask rules into deny outside dashboards
  • resolveSession answers an analyst's POST /messages and /abort with a 403

Fixed since the last review

  • ✅ F1 · The observer's tools/list hiding has no test

Production impact

Open errors in the changed files (2)
Issue Service Occurrences File
@maple/mcp/errors/McpQueryError (7 issues) maple-ai 33 apps/ai/src/mcp/tools/registry.ts
@maple/mcp/errors/McpQueryError (3 issues) maple-chat 15 apps/ai/src/mcp/tools/registry.ts

After this merges, Maple checks whether they stop.

ee977ef · What was checked, coverage and telemetry details are on the Maple / review check. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

@Makisuo
Makisuo added this pull request to stack #2556 November 13, 2026 12:45
@Makisuo
Makisuo merged commit 8e1ad67 into main Nov 13, 2026
31 of 43 checks passed
@Makisuo
Makisuo deleted the observer/agents branch November 13, 2026 12:56
@maple-review-bot

Copy link
Copy Markdown

Maple: after this shipped

✅ Shipped clean. Nothing this change touches got worse in the hour after it deployed.

Deployed to maple-chat (production) at 2026-11-13 13:26 UTC as 83ec949, the first version after the merge (the merge commit itself never reported). Compared the 60 minutes before and after.

Errors in the changed files:

  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
  • ➖ @maple/mcp/errors/McpQueryError: 0/h → 0/h
Operation Calls/h Errors p95
tools/list 148 → 116 0.0% → 0.0% 0 → 0 ms
GET 0 → 0 0.0% → 0.0% 0 → 0 ms
GET 7 → 5 0.0% → 0.0% 734 → 500 ms

Maple looks once, an hour after the first deploy that carries the merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant