Skip to content

Amend the sensitivity model: source labelling is sound for hierarchies, unsound for opaque containers #394

Description

@KenTaniguchi-R

An amendment to the sensitivity/clearance design, from research on 2026-08-06. The model is largely right; three claims in it are not, and one of them was mine.

Verified against main @ 6dbf1c6.

1. Source labelling does not generalize past hierarchies

The design labels sources. FIDES, which it cites, labels content:

Trust and confidentiality become labels on content, propagated by middleware, checked deterministically before each tool call.

For a filesystem these coincide — a path is a usable proxy, and classifyPath's longest-prefix-wins gives real granularity via carve-outs. For an opaque endpoint they come apart completely: one label, heterogeneous contents, no structure to carve on. Either the label is false (internal when a secret is inside) or the source is useless (secret).

This is not our limitation. Authorization-First Retrieval (TrustNLP 2026) names it as unsolved:

mixed-content chunks (public and restricted material colocated in one indexable unit) cannot be tagged correctly under a single-label scheme regardless of process maturity

with retrieve-then-filter exposing unauthorized context in 86.1% of queries, and metadata-tag pre-filtering leaking 9.7% after a single policy-update cycle — "staleness is not an edge case. It is the steady state."

But whole-source labelling is still the right answer, for a reason worth writing into the spec. github/github-mcp-server hit this exact case and rejected per-item labels (pkg/ifc/ifc.go):

a tool result is delivered as one opaque payload… Once the items share a buffer in the agent's context they can be copied anywhere together, so the only sound bound for the whole result is the meet of every item's label. Per-item labels would only become load-bearing if the enforcement engine could partition a result and route individual items to different sinks.

Ours cannot — everything lands in one context window. Even Azure AI Search, the one system doing per-reference sensitivity labels, also returns a response-level "most restrictive across all references" and enforces on that.

So: keep whole-source labelling, and record that it is the converged answer rather than a shortcut. Then state plainly that an opaque container labelled below its most sensitive item is a false guarantee.

2. The three-level total order is a known-insufficient shape

public < internal < secret forces over-classification the moment something is sensitive to one audience and not another. Both prior designs that tried it moved away:

  • The MCP extensions draft killed sensitiveHint: low|medium|high because "sensitivity is set-theoretic (a card number and a medical record are both sensitive but to different readers), not a single scale."
  • Oracle Label Security is LEVEL : COMPARTMENTS : GROUPS, where dominance needs level ≥ and compartments ⊇ and group intersection — with non-comparable labels as a first-class outcome.

Not urgent, and adding compartments now would be premature. But the spec should stop presenting three levels as sufficient and record the known ceiling.

3. I proposed two fixes that the evidence rejects

Both are recorded here so they are not re-proposed:

Enumerate MCP servers at setup and ask once. This is per-tool allowlisting with a nicer UI, and the research note already rejected it: "Naming safe MCP servers does not converge. mcp: [openmemory] is exactly as dangerous as whatever is in that owner's openmemory." Renaming the entry to a label does not add granularity.

Add a call-time approval path (FIDES approval_on_violation). The evidence is strongly against it:

  • Anthropic's own telemetry: 93% approval rate on Claude Code permission prompts. Vista UAC 89–91%. Chrome SSL warnings 70%, half dismissed under 1.7s.
  • Microsoft's own FIDES docs recommend approval_on_violation=True for "interactive UX, dev/test" and hard block for "production, low-trust environment."
  • The dialog is itself an attack surface — "Lies-in-the-Loop" pads it so the dangerous part scrolls off.
  • FIDES's approval path shipped a replay bug (a granted call_id in a set never cleared).
  • "Oversight Has a Capacity" models realized safety as an inverted-U in escalation rate: escalating everything is strictly worse than escalating a middle amount.
  • As of Aug 2026, no mainstream agent product ships async approval for tool permissions — everyone blocks or removes the human.

What to build instead

From the same research, in order:

  1. Push denials out-of-band as information, not a decision. "Ken's agent asked X at 3am; policy said no." No approve button. Gives the owner the feedback loop they currently lack — today denials are logged locally and never surface — without a prompt to rubber-stamp.
  2. Time-boxed clearance grants. SpiceDB ships [expiration:...] as a first-class relationship primitive, clock-enforced server-side; they moved off caveat-based expiry because it required clients to supply now and did not garbage collect.
  3. Learned narrowing from the denial log. Wijesekera et al. (IEEE S&P 2017): predicting the user's own decision from their history reaches 96.8% accuracy, ~4x error reduction over ask-on-first-use, in a deployed 38-person field study.
  4. Answer-partially-then-defer rather than block — the caller gets what clearance permits now. Closer to Constrained-output tasks: declassification by capacity #387 than to approval.

_meta.ifc: real code, no spec — parse opportunistically, do not build on

  • Not in the MCP spec. Every revision through 2026-07-28 and current draft has only the four original tool hints. Zero merged SEPs. SEP-1913 was split to an Extensions Track in June 2026; sponsor unresponsive since, bot nagging through 2026-08-03.
  • But github/github-mcp-server emits labels today, behind an opt-in ifc_labels flag, and microsoft/agent-framework consumes them (Python, @experimental).
  • Three incompatible wire formats already exist: _meta.ifc, _meta["com.github.ifc/labels"], _meta["io.modelcontextprotocol/trust-annotations"].

Treat a present label as an opportunistic upgrade and absence as maximum sensitivity. Do not make the guarantee depend on it.

And do not auto-classify

REDACT (arXiv 2606.19881, Jun 2026) puts rule-based Presidio at 0.07 recall on HIGH-sensitivity categories — 93% false negatives. The structural argument matters more than the number: fail-closed auto-classification is sound exactly to the degree it is not classifying. If unlabelled means secret, a classifier can only ever downgrade, and every downgrade is an unproven declassification. Classic IFC (Denning, Myers/Liskov DLM, Jif) and every modern agent IFC system assign labels by provenance, never by inferring from content.

Gated on

Nothing technical. This is a spec amendment plus three decisions: the opaque-container position, whether to add compartments later, and which of the four "build instead" items is first. #392 (wiring) and #393 (the git-repo default) are the concrete work this frames.

Sourcing caveat

The MCP findings are traceable to primary sources (spec schemas, ifc.go, SEP threads) and were verified directly. One research agent in this batch fabricated a claim about having received sub-agent results and self-corrected; its remaining findings were separately confirmed, but anything from that batch should be re-checked before it becomes normative text.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:securityC — endpoint securitykind:decisionA decision to make, not code to writestatus:gatedBlocked on a stated precondition — do not start coding

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions