Skip to content

feat(kg): read-only MCP server over the exported graph - #520

Open
Sulstice wants to merge 46 commits into
devfrom
kg/mcp-server
Open

Sulstice wants to merge 46 commits into
devfrom
kg/mcp-server

Conversation

@Sulstice

@Sulstice Sulstice commented Nov 9, 2026 •

Copy link
Copy Markdown
Member

What

kg/mcp_server.py is a new MCP server, separate from supabase/functions/bbqs-mcp, which this PR does not touch. It loads the exported graph (one Turtle file) into memory and answers from that. No Supabase, no edge function, no sign-in, no key, no write path.

Part of #438. For that issue's first open point ("decide what directly means") this takes the exported graph, in a server of its own, as #434 laid out.

Also here: CQ20-CQ42 from #441 adopted into the questions the KG is scored on (see below), and three new Graph Inspector tabs, Publications, Meetings and Devices (see below).

Tools

All seven are reads and declare readOnlyHint. Every answer carries snapshot: which copy of the graph it came from, its date, and its age in days. An argument a tool does not take is refused rather than dropped (each inputSchema says additionalProperties: false), so a misspelt max_hops cannot quietly answer a different question.

Tool What it answers
describe_graph The map: node types, how full each property is, link types and their endpoints, which copy is served, and what the view leaves out. With a type: the schema's definitions and the values each property takes.
search_nodes Find nodes by text, or list them by type and exact property values.
get_node One node in full: every property, every link.
get_neighbors Everything within 1 to 4 links, nearest first, each with the link that reached it. It lists; it does not rank.
find_related What is most closely tied to a node, or to several at once (what they have in common), each type of node ranked on its own, up to four links away unless asked.
find_paths How two nodes are connected: shortest chains first, strongest first among them, then longer ones, a page at a time.
sparql_query SELECT / ASK over the graph as exported. No updates, CONSTRUCT, DESCRIBE, SERVICE or FROM; stopped after 10 s.

Running it

  • Locally, from a checkout: claude mcp add bbqs-kg -- uv run /path/to/bbqs-app/kg/mcp_server.py.
  • Without a checkout: the script names its own dependencies, so uv run https://raw.githubusercontent.com/brain-bbqs/bbqs-app/main/kg/mcp_server.py runs it from GitHub; it then reads the export the site publishes, or the one committed to main.
  • On a public URL (for a claude.ai connector): any host that runs Python from the repo, with python kg/mcp_server.py --http --host 0.0.0.0 --no-sparql --allow-host neuromcp.brain-bbqs.org. It listens on $PORT. --allow-host keeps the SDK's Host/Origin check on for the public name behind the host's proxy. There is no sign-in, so it should only ever serve the public export.

How it decides what is related

Put plainly: nodes are still listed by how many links away they are, as get_neighbors lists them. What is new is the order among nodes equally far away, and a number that says why.

That order comes from spreading activation, the account in psychology of how one idea brings up another. Three rules:

  1. Activation leaves the starting node one link at a time, so what is nearer is reached sooner.
  2. Each node it reaches splits what it received equally among the nodes it is linked to (the fan effect). A project with 5 neighbours passes on a fifth to each; a working group with 114 members passes on 1/114.
  3. Amounts that arrive together add up.

The number that comes out is strength: 1 for a direct link, otherwise the sum, over the shortest routes, of 1/neighbours for each node in between, multiplied along the route. It is worked in exact fractions, is the same from either end, and has nothing to tune. KG._wave is the rule.

Node (a species studied by two projects, with 5 and 2 neighbours) How activation gets there Strength
The two projects a direct link each 1
A person on both projects 1/5 through the first, 1/2 through the second 0.7
A person on the first project only 1/5 0.2
  • Each node type is ranked on its own, nearest first and then strongest. A working group collects activation from a hundred members and a paper from one project, so one list of both mostly sorts them by size.
  • Strength is a sum and can pass 1 (two working groups that share 57 members: 17.8). It compares nodes the same distance away.
  • Several starting nodes return only what every one of them reaches, each node placed by its worst tie, then its next worst; per_start gives each tie.
  • A row names the strongest route and its share of the strength. tied says when a list was cut among equals, which is common: 274 of 421 default lists of people.
  • find_paths gives every path its strength; its routes and strength are the count and sum of the shortest ones, the figures find_related shows for the pair. Both tools give the same answer from either end.

What it does not do, all said in kg/README.md: it reads only the first wave (the full theory lets activation settle, which was tried and found hidden ties no better); nearest-first is this server's rule, not the theory's; every link type counts the same; and it reads the data as recorded, so a record missing in the middle of a route makes a tie look stronger (87 of 210 linked investigators have no organization recorded).

How it was chosen: five natural processes were prototyped on the real graph (spreading activation left to settle, an animal foraging from a home base, slime-mould tubes, ant trails, a pulse on a spider's web), scored by three reviewers and attacked by a fourth. The web pulse scored highest. What is here is its arithmetic, told as the first wave of spreading activation, which is the theory that is about networks of concepts.

Benchmark

kg/bench_mcp_server.py starts the server the way a client does and asks it the 42 competency questions the KG is scored on (competency_questions.yaml) over MCP. Each is asked twice: as SPARQL, and without SPARQL through search_nodes, get_node and find_related. Both must return the rows the export gives when queried directly; the benchmark writes those reference rows itself, so a fault in how the server writes a value would show. It then measures the ranking and the speed, and writes the results block in the main README.md. On the export of 2026-11-05:

Result
Competency questions that return rows on this export 29 of 42
Those 29, asked as SPARQL through the server: same rows as the export queried directly 29 of 29
Those 29, asked without SPARQL through the graph tools: same rows again 29 of 29
The other 13 (7 need the full export, 5 find none in this export, 1 not modelled yet): no rows either way 13 of 13
Hide a real tie: the hidden end is among the first ten find_related lists after the start's direct ties (1,383 cases) 62% (65% looking up to 8 links)
The same for get_neighbors, which lists by distance alone 45% (47% walking through hubs)
Strengths equal to a slow second version of the rule, exactly 515 of 515 starting nodes
A find_related call over stdio (median; 95th percentile) about 6 ms; 9 ms

Competency questions: CQ20-CQ42 adopted (#441)

  • All 23 questions proposed in KG competency questions CQ20–CQ42 (+ SPARQL) for kg/competency_questions.yaml #441 move into competency_questions.yaml, so the gate, quality.py and the benchmark ask the same 42. competency_questions_extended.yaml is gone.
  • The six "which X has no Y" questions (CQ21, CQ27, CQ30, CQ33, CQ38, CQ42) carry reads: absence: they answer whether they list some X or none, so the gate counts them either way (recording every missing ORCID would otherwise empty CQ21 and fail the gate on an improvement), and the Inspector shows "none found" rather than an empty answer.
  • On the committed export the gate counts 31 answered (quality_baseline.json updated): 29 with rows, plus CQ27 and CQ42 with nothing missing. CQ39's empty answer is real: no funding call closes in its window.

What a hop is

  • A ProjectRole record is folded into one has_role link from the person to the project, carrying the role. It stays the only authority for a role on a project (Role identity has no single source of truth: investigators.role vs grant_investigators.role #283); consortium_role is shown as the free-text label it is.
  • An ORCID iD that two or more nodes carry becomes a node. That is the only authorship link the graph has today (23 shared iDs in the 2026-11-05 export, 3 of them a BBQS investigator's).
  • get_neighbors reaches a hub (over 30 links: today, the four working groups) but does not walk through it unless asked. Two links from a linked investigator is a median of 9 nodes this way, against 114.
  • find_related and find_paths walk through hubs: a hub passes on little, so what lies behind it comes after anything equally far that is reached by a more exclusive route. What limits them is distance (max_hops: 4 by default for find_related, 6 for find_paths, at most 8). find_paths also leaves out chains that go the long way round a direct link.
  • What the view cannot hold is mostly counted in describe_graph, not dropped silently: a link to a node missing from the export is an Unresolved node; roster records with an end missing (2 in the 2026-11-05 export) or with a field given twice are left out and counted, as are links to roster records, what a record says beyond its rule, nodes with two types and blank nodes (none in the export). Language tags and datatypes are dropped without a count.

Graph Inspector: Publications, Meetings and Devices tabs

  • Publications (every build, the public one too): each paper's tie to its grant, which is the produced link the exporter writes from project_publications (the NIH award credited on the PubMed record), and what the export lets us check about it. On the committed export: 88 of 89 are tied to a project (one to two projects) and 1 to none; 10 records share a title with another (preprint and published pairs, markup ignored); 21 carry author ORCIDs, but for 11 of those no one on the roster has an ORCID to compare (roster has no ORCIDs), and 1 of the other 10 has an author on the roster. The full text is in no export, because publication_fulltext is admin-only and its licences are still unknown (BBQS publications catalogue: papers and the project each is linked to #444). In the claude.ai artifact copy an admin can read it through their own BBQS-MCP connector (kg_query, run as them, so RLS decides); nothing read that way is stored in the page.
  • Questions: all 42, with "none found" for a question about what is missing.
  • Meetings (the artifact copy only): kg/meetings.py turns the working-group notes into one record per lesson, agreement, disagreement, decision, action and blocker, tied to the people the notes attribute it to (said), give it to (owns), or mention (named). Each name is resolved from the first name or nickname the notes use, from the meeting's own speakers first, then the group's, and linked to the Investigator node where one matches (both nodes, where the graph holds a person twice). From 80 meetings: 715 points, 576 ties, 2 names left unresolved and shown as such; 66 of 75 speakers link to an Investigator. The page says these are summaries, not transcripts: only words in quotation marks are quoted.
  • Devices (the artifact copy only): kg/devices.py reads the consortium device list (BBQS Device List, Who Said What, with References): 146 lines across 28 awards (117 devices and sensors, 29 methods), each with its evidence and tier. T1 is the team's Data Blitz slide; T2 the workshop summary, written from the transcripts, which names who spoke to a device but has not been checked against them; T3 a line of working-group notes; U a maker or model only the AI-assembled list gives, shown as AI-predicted, unverified and never as the device's model (44 lines). Every award links to its Project node, named speakers to their Investigator nodes (a name the source marks for checking stays unlinked), and makers and models to the catalog's nodes of the same name. A reviewer can flag a line to confirm or correct it.
  • The internal records stay internal. quality.py --internal embeds them only in a --fragment, refuses any path inside the repository and refuses to run on CI; so do kg/meetings.py and kg/devices.py. --html and --json (the committed copy, and what publish.yml serves) never get them. tests/guards/inspector-internal-data.test.mjs checks the template's placeholders, that the committed copy carries no records, that nothing under .github/ or in package.json passes --internal or runs either script (whole files, so continuation lines count), that no records are committed anywhere with no file exempt, and that no raw notes or device-list file is committed, by name; those names are git-ignored too.
  • Found on the way: seven people are held as two Investigator nodes each; for three of them the second copy has a doubled space in the name.

Outside kg/

  • README.md: a section on the server, with the commands to run it and the benchmark's results block.
  • kg.yml: one step installs kg/requirements-mcp.txt and runs kg/test_mcp_server.py. Adding the MCP SDK changes no version the harness pins (checked in a clean Python 3.10 environment with requirements-generate.txt).
  • publish.yml: when the live anon export succeeds, it is also copied to dist/kg/bbqs.ttl, so it is served at https://brain-bbqs.org/kg/bbqs.ttl once this reaches main. It is the file the Inspector page already embeds, so nothing new becomes public. This is the server's default source; until that URL exists the server serves the committed export and says so in every answer. The copy cannot fail the Inspector build or the deploy.
  • tests/guards/inspector-internal-data.test.mjs and .gitignore: the guard above, and the raw files' names.

How it was checked

  • python kg/test_mcp_server.py: 143 tests, on Python 3.10 and 3.13. A hand-built graph with known answers, strengths included; every argument of every tool shown to reach the graph and to change the answer; all 42 competency questions through the server, as SPARQL and through the graph tools, on the committed export, on a miniature in which every one has rows (counts worked out by hand), and on a graph of shapes an export could take and today's does not; the benchmark's own measure on a graph worked by hand, and its command line end to end; the public-URL options (an allowed host answered, another refused, over a real HTTP server); raw JSON-RPC against a real server process over stdio and over HTTP.
  • The 42 walks were also run against 780 random small exports, each question three ways. Every difference found was fixed, except what the view leaves out by design (above), which is documented and counted.
  • The ranking has checks of its own: the code gives exactly what a second, slower version of the process delivers with nothing left out; the paths between two nodes add up to the strength shown for them; the worked example in the tool descriptions is built and run; replies stay a size a client can take; and the hidden-tie measure.
  • Guards: every schema term the server or its descriptions name is declared in bbqs.owl.ttl, and every reply field is named in a description (both ways); the dependency pins agree in the places they are written; no call that writes to a graph appears outside the one that loads it.
  • Shown able to fail: the server broken 157 ways (82e4545), every break caught, with an unbroken copy passing first; the benchmark's walks broken 89 ways (3a91efe), 87 caught, and the 2 that were not are caught since 5afb97d.
  • Reviews: the ranking as first written (33c32ed) by four independent reviewers, each finding checked by a second (38 confirmed, none in the arithmetic; fixed in cfcef47). Then three more rounds, each finding checked by a second reviewer: 53 findings (fixed in 82e4545), 22 (3a91efe) and 8 (5afb97d); none refuted. A brute force agrees with both tools on 400 random small graphs and on the real export. The Inspector tabs, the meeting attributions and the leak guards: 29 findings from three reviewers, each checked by a skeptic (26 confirmed, 3 partly, none refuted), fixed in 68d34cf.
  • The Inspector tabs: browser runs of the built page with stand-in capabilities (db, user, mcp) through every tab, filters, the person and award views, saved meeting and device flags, a text read, 400 px wide and dark mode, with no console errors; the public copy shows no Meetings or Devices tab. Every check of the new guard was seen failing first. The refusals of quality.py, meetings.py and devices.py were tried with each path inside the repo, and on CI.
  • kg/ci_check.py --check-generated, npm run test:guards and npm run typecheck pass on this branch.
  • The Claude Code CLI connects to it over stdio (claude mcp add bbqs-kg -- uv run kg/mcp_server.py) and over HTTP; the MCP Inspector's CLI lists the seven tools and calls them; uv run of the script's GitHub URL loads the graph with no checkout.

Not in this PR

  • No hosted endpoint. The options to serve it on a public URL (--http, --allow-host, $PORT) are here and tested, but nothing here deploys it.
  • The meeting notes and the device list are not in the MCP server yet. They are internal, and this server has no sign-in, so they need an internal tier first. Nor is the device list in the database: the graph's own devices still come from device_models and the catalog.
  • The Publications tab's text read is unverified against the live connector. It follows supabase/functions/bbqs-mcp/index.ts (kg_query takes {path} and returns the rows as JSON text); the session that built it could not sign in to BBQS-MCP to make a real call.
  • Left on [KG]-MCP talks directly to the Knowledge Graph #438: field-level provenance (field_provenance is not in the anon export), usage logging, the /mcp-docs and /mcp-tutorial pages, and returning the competency questions themselves. Tiers and caller identity do not apply: the server has one tier and only serves an export.
  • Left on KG competency questions CQ20–CQ42 (+ SPARQL) for kg/competency_questions.yaml #441: CQ37 has no job-to-project link (it asks posted_by), and CQ39 keeps the fixed window the issue wrote rather than today plus six months.
  • History: commits 34c7c10 to 2f1a763 carried a few short example lines copied from the meeting notes in kg/meetings.py's docstring. The tip has invented examples instead; a squash merge keeps the old lines out of dev.
  • Spec entries in bbqs-agent/specs/ are not written; that repo was not reachable from the session that built this.
  • https://brain-bbqs.org was not reachable from the build sandbox, so the default source was exercised against a local HTTP server standing in for it.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ

Sulstice and others added 2 commits November 9, 2026 22:53
kg/mcp_server.py serves the graph the exporter writes to any MCP client, over
stdio or streamable HTTP. It is a separate server from
supabase/functions/bbqs-mcp and shares nothing with it: no database, no edge
function, no sign-in, no write path. It loads one Turtle file into memory and
answers from that. For the first open point of #438 ("decide what directly
means") this takes the exported graph, in a server of its own, as #434 laid
out.

Six tools, all declared read-only: describe_graph (the map; with a type, the
schema's definitions and the values each property takes), search_nodes,
get_node, get_neighbors, find_paths, and sparql_query (SELECT and ASK only, no
SERVICE or FROM, stopped after 10 s). Every answer says which copy of the
graph it came from and how old that copy is.

So that a hop means something:
- each ProjectRole record is folded into one has_role link carrying the role,
  still the only authority for a role on a project (#283);
- an ORCID iD that two or more nodes carry becomes a node, which is the only
  authorship link the graph has;
- get_neighbors reaches a hub (over 30 links: the working groups) but does not
  walk through it unless asked. Two links from a linked investigator is a
  median of 9 nodes this way, against 114;
- find_paths leaves out chains that go the long way round a direct link, and
  puts chains through hubs last.

Default source: the copy the site publishes (next commit), else the committed
export. A re-read that fails, or that lost most of the graph, keeps the graph
already loaded.

kg/test_mcp_server.py, 71 tests: a hand-built graph with known answers; every
competency question the baseline says is answered returns the same rows
through the server as from the graph directly; the walking tools against
SPARQL on the committed export; raw JSON-RPC against a real process over stdio
and HTTP. Guards: every schema term the server or its descriptions name is in
bbqs.owl.ttl, the dependency pins agree, no graph write outside the load. The
suite was shown able to fail: 28 deliberate breaks in a scratch copy, each
caught.

Part of #438

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
kg.yml: install kg/requirements-mcp.txt and run kg/test_mcp_server.py, under
the same condition as the rest of the gate. Adding the MCP SDK changes no
version the harness pins.

publish.yml: when the live anon export succeeds, also copy it to
dist/kg/bbqs.ttl, the default source of kg/mcp_server.py. It is the file the
Inspector page already embeds, so nothing new becomes public. The committed
fallback is not published, because the file's date is how a reader knows the
age of the data. The copy cannot fail the Inspector build or the deploy.

Part of #438

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
@Sulstice
Sulstice requested a review from nadernik as a code owner November 9, 2026 22:53
@github-actions github-actions Bot added the spec:C Danger layer — Spec Kit + sign-off required label Nov 9, 2026
@github-actions

github-actions Bot commented Nov 9, 2026 •

Copy link
Copy Markdown
Contributor

🤖 Triage: Class C → spec:C

Touches a danger layer. Requires Spec Kit artifacts (spec/plan/tasks/qa-itinerary), a Constitution Check, and human sign-off. Never auto-merges.

Blast radius: HIGH

  • ci-workflow: CI/CD workflow — this is the pipeline that ships everything else, including this gate.
  • shared-edge-code: Shared edge-function code (auth, CORS allow-list, grant-sync) — a change fans out to every function. A global header here once broke every function's CORS preflight (2026-08-07).
  • migration: Schema/RLS/view migration — applied MANUALLY to production and hard to reverse. RLS policy and CREATE OR REPLACE VIEW changes live here too.
  • triage-gate: The gate itself (classifier or guard suite). A change that could weaken the gate must never pass through the gate automatically.

Complexity: HIGH

  • 82 files changed (> 4).
  • 69983 lines changed (> 80).
  • 37 new files (> 2).

Files that set the class:

  • .github/workflows/kg.yml → ci-workflow (hard)
  • .github/workflows/publish.yml → ci-workflow (hard)
  • supabase/functions/_shared/identity-keys.ts → shared-edge-code (hard)
  • supabase/functions/_shared/kg-write.ts → shared-edge-code (hard)
  • supabase/functions/add-project-by-grant/index.ts → edge-function (soft)
  • supabase/functions/bbqs-mcp/index.ts → edge-function (soft)
  • supabase/functions/import-grant-publications/index.ts → edge-function (soft)
  • supabase/functions/nih-grants/index.ts → edge-function (soft)
  • supabase/functions/seed-consortium/index.ts → edge-function (soft)
  • supabase/functions/verify-affiliations/index.ts → edge-function (soft)
  • supabase/migrations/20261116150000_link_isolated_nodes.sql → migration (hard)
  • supabase/migrations/20261117160000_funding_call_expiration_dates.sql → migration (hard)
  • supabase/migrations/20261117170000_public_job_posters.sql → migration (hard)
  • supabase/migrations/20261117174500_revoke_writes_through_views.sql → migration (hard)
  • supabase/migrations/20261117181500_prevent_duplicates.sql → migration (hard)
  • supabase/migrations/20261117190000_hide_private_columns_from_anon.sql → migration (hard)
  • supabase/migrations/20261118100000_hotfix_anon_function_grants.sql → migration (hard)
  • supabase/migrations/20261118120000_privacy_round_two.sql → migration (hard)
  • supabase/migrations/20261118140000_role_vocabulary_collaborator.sql → migration (hard)
  • tests/guards/anon-private-columns.test.mjs → triage-gate (hard)
  • tests/guards/function-execute-grants.test.mjs → triage-gate (hard)
  • tests/guards/identity-normalization.test.mjs → triage-gate (hard)
  • tests/guards/inspector-internal-data.test.mjs → triage-gate (hard)
  • tests/guards/kg-identity-keys.test.mjs → triage-gate (hard)
  • tests/guards/kg-resource-links-parity.test.mjs → triage-gate (hard)
  • tests/guards/kg-role-vocabulary-parity.test.mjs → triage-gate (hard)
  • tests/guards/optional-columns.test.mjs → triage-gate (hard)
  • tests/guards/role-ui-vocabulary.test.mjs → triage-gate (hard)
  • tests/guards/view-write-privileges.test.mjs → triage-gate (hard)
  • tests/guards/write-paths-get-or-create.test.mjs → triage-gate (hard)

Deterministic classifier — computed from the diff, not from the issue text. Change the rules in scripts/triage/classify.mjs (itself a Class-C path).

@github-actions

github-actions Bot commented Nov 9, 2026

Copy link
Copy Markdown
Contributor

🛑 Class C — this touches a danger layer

Per the constitution (Development Workflow) danger-layer changes are spec-driven. Before this can merge, a maintainer must:

  1. Create the Spec Kit artifacts in bbqs-agent/specs/<NNN-feature>/ — spec.md, plan.md (with a Constitution Check), tasks.md, qa-itinerary.md.
  2. Point bbqs-agent/.specify/feature.json at that directory.
  3. Run the Find → Fix → Test → Verify-Live loop, incl. the sandbox QA workflow.
  4. Add the spec-signoff label to this PR to release the triage/spec-gate check.

This gate is here because a small-looking diff to .github/workflows/kg.yml reaches far beyond the file it edits.

Sulstice and others added 7 commits November 10, 2026 04:25
… find_paths

The MCP server walked by hop count alone and ranked equally short paths
by an ad hoc degree penalty. This replaces the ranking with a natural
process a person can state and check by hand: spreading activation, the
account of how one concept brings up another. Activation leaves a node
along every link, each node it reaches splits what it received equally
among its own links (the fan effect), and amounts that arrive together
add up. A node is read at the first wave to reach it, so

  strength = the sum, over the shortest routes, of the product of
             1/links over the nodes in between; 1 is a direct link

It is worked in exact fractions, is the same from either end, and has
nothing to tune. KG._wave is all of it.

- find_related (new): what is most closely tied to a node, or to several
  at once. Each node type is ranked on its own, nearest first and then
  strongest. Several starting nodes return what every one of them
  reaches, by the farthest start and then the weakest tie. A row names
  the strongest route and its share of the strength, `tied` says when a
  list was cut among equals, and avoid_edges leaves link types out.
- find_paths: equally short paths are ordered by the same strength, each
  path says its strength, `tie` gives the number and sum of the shortest
  paths (the figure find_related shows for the pair), and a tie is cut
  the same way from either end.
- get_neighbors is unchanged: it lists, it does not rank.

Five natural processes were prototyped on the real graph, scored by three
reviewers and attacked by a fourth before this one was taken. Within a
node type it puts the hidden end of a removed link in the first ten 65%
of the time, against 39% for distance alone; that measure is now a test.

Tests go from 71 to 90: strengths worked out by hand on the fixture; the
code against a second, slower version of the process with nothing left
out; paths that add up to the tie; the hidden-link measure. The server
was broken 62 ways in a scratch copy and every break turned the suite
red. kg/README.md explains the rule, works an example, and says what it
does not do.

Part of #438

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
Four reviewers, each checked by a second, went through the
spreading-activation change and confirmed 38 problems. None was in the
arithmetic: an independent listing of every shortest route reproduced
every strength exactly. They were around it: what a client is told, how
much it is sent, the flags on a path listing, gaps in the tests, and the
README's account of why the rule was chosen.

find_related
- Looks four links away unless asked (max_hops, at most 8, the same as
  find_paths) and counts per type what lies beyond. A default reply is
  about 7 KB, was 16; five starting nodes about 3 KB, was 78.
- Several starting nodes: a node is placed by its worst tie, then its
  next worst, and the row is headed by that tie. Rows that differ on any
  tie are no longer reported as equals.
- A row's route is id, label and `splits` per node; with several starting
  nodes it carries `ties` and find_paths has the routes. `carries` is now
  `share`; a list is capped at ten when no type is named.
- A list of starting nodes in one string is read however it is spelled,
  and one bad part is reported as that part.

find_paths
- Always searched from the same end and turned round, so both directions
  return the same list, cut in the same place.
- The shortest paths come strongest first from the wave itself, without
  listing them all and without the step budget. Longer ones follow.
- `more` only when more exist (it was missing when exactly `limit` were
  found, and wrongly set when the budget ran out); `partial` when the
  search among longer paths gave up; `offset` pages through them.
- `routes` and `strength` at the top replace `tie`.

Both
- Two nodes are one tie however many links record it, and a link from a
  node to itself leads nowhere. A person recorded twice on a grant is not
  twice as close to it.
- The route find_related shows is the path find_paths lists first.
- A strength is rounded from the exact fraction (1/64 is 0.01563) and a
  whole number is sent as one.

kg/README.md
- Opens with the plain statement: nodes are still listed nearest first,
  as get_neighbors lists them; the rule orders nodes equally far away.
- Says strength is a sum and can pass 1; that nearest first is the
  server's rule, not the theory's, and is what keeps the working groups
  in their place (172 of 210 down to 15); that reading only the first
  wave buys exact fractions, not a better ranking; and how the rule was
  chosen as it happened (the web pulse scored highest; this is its
  arithmetic, told as spreading activation).
- Quotes what the tools return: a hidden tie is in find_related's first
  ten 62% of the time, in get_neighbors' 45 to 47%.

Tests go from 90 to 103, and the worked example in the tool descriptions
is now built and run, so what a client is told is what the server does.
The server was broken 128 ways in a scratch copy and every break turned
the suite red. A separate brute force agrees with both tools on 400
random small graphs with repeated and self links, and on the export.

Part of #438

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
kg/bench_mcp_server.py starts the server the way a client does and asks
it every competency question twice over MCP: as SPARQL through
sparql_query, and without SPARQL through search_nodes, get_node and
find_related. Both must return the rows the export gives when it is
queried directly, and the command exits 1 if either does not. It then
measures the ranking (one real tie hidden: is its other end in the first
ten each tool returns?) and the speed and size of each tool's replies.

On the export of 2026-11-05:
- 16 of 19 questions have an answer (2 need the full export, 1 is not
  modelled yet), and all 16 come back the same both ways;
- a hidden tie is in find_related's first ten 62% of the time, and in
  get_neighbors' 45%;
- every strength equals the slow second version of the rule, on all 515
  starting nodes;
- a find_related call takes about 10 ms over stdio.

The main README gets a section on the server: the commands to run it,
and a results block the benchmark writes itself (--update README.md).
--show CQ04 prints one question's rows each way; --only questions
leaves the measurements out; --source takes another export.

Tests go from 103 to 107: the questions on the committed export, and on
a graph in miniature in which all nineteen have an answer, so that the
three walks the public export cannot exercise are held against rows
too.

Part of #438

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
 too

The benchmark asked the 19 questions the KG is scored on. #441 proposes 23
more (CQ20-CQ42). They are added as kg/competency_questions_extended.yaml,
which only the benchmark reads, so quality.py, the baseline and the gate's
"N of 19" stay as they are until #441 adopts them.

Each question is asked as SPARQL through sparql_query and without SPARQL
through the graph tools, and both are compared with the export queried
directly. On the export of 2026-11-05, 29 of the 42 return rows and agree
both ways. The other 13 return nothing both ways: 7 need the full export,
5 have nothing to list today, 1 is not modelled.

Asking every question of 780 random small exports found shapes today's
export does not have that led walks astray: an id nothing describes, a
property given twice, the same two nodes linked twice, two nodes with one
name, a name matched loosely. The walks handle them now, and a graph of
those shapes is in the tests, beside a miniature in which every question
has rows. One difference remains, and is documented: a roster record with
an end missing makes no link, so a walk cannot see it while SPARQL can.

Also, after review: the hidden-tie figure is described as measured (after
the start's direct ties), find_paths as listing shortest first, the hub
paragraph as it now behaves, and the questions the public export cannot
answer as seven.

Part of #438. Refs #441.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
Each of the 42 walks was broken on purpose in a scratch copy, one break
at a time (62 in all), to see whether the benchmark tests notice. Five
went unnoticed, each because no test graph had the shape that tells the
right walk from the wrong one:

- a project carrying the grant number only as its name, with a person on it
- a person on two projects, one of them without a grant number
- a paper whose name is not its title
- one project studying two species of one name
- two jobs of one title at one institution

The test graphs have them now, and all 62 breaks are caught, each by a
named test, with an unbroken copy passing first.

Part of #438.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…ference

The third review's 28 findings, checked by a fourth review, with fresh
reviews of the 42 walks and of the tests: 53 findings, 37 confirmed and
16 partly, none refuted. Seven were already fixed in the docs.

Server:
- An argument a tool does not take is refused. It was dropped, so a
  misspelt max_hops answered a different question. The tools are built
  from the SDK's public Tool class with a strict argument model, and
  each inputSchema says additionalProperties: false.
- A list argument that names nothing ("[", ",") is refused, not read as
  no filter. A deeply bracketed one no longer escapes as RecursionError.
- An ambiguous name says what it names however it is written.
- find_paths: the shortest paths get a step budget too, so a deep page
  cannot take minutes; a page cut short says partial. Two nodes linked
  both ways have each link drawn the way it points, and `also` in
  find_related carries direction and role.
- find_related: naming more than three types keeps the cap at 10 (it
  reached 72 kB); "within 1 link"; a node linked only to itself; with
  several starts, equals are equal on every tie.
- Reply fields say what they hold: reached is within, found is listed,
  ties is per_start. The descriptions name every field.
- describe_graph counts what the view leaves out: roster records with
  an end or a role given twice, nodes typed twice, blank nodes. Every id
  shown resolves back, whatever its shape or case.
- With --no-sparql, nothing the server says mentions SPARQL.

Benchmark:
- The reference cells are written out by the benchmark, not by the
  server, so a fault in how the server writes a value shows.
- Numbers compared to 12 significant digits (a sum's last digits depend
  on its order). Columns that hold a node or a value read either.
  Addresses are written as the server writes them. A question the file
  cannot answer, or the tools cannot walk in full, fails on its own.
  --access full for a full export.
- The measure, the speed summary, the report and the command line are
  tested, the measure on a graph whose every case is worked by hand.

Tests: 136, from 113. Each new one fails on a wrong version a reviewer
named. Every argument of every tool is shown to reach the graph and to
change the answer. A hang now fails the run after 15 minutes.

Part of #438. Refs #441.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
A fifth review, of the fourth round's fixes, found no regressions (9,400
random questions answered the same before and after, but for the intended
"1 link") and 22 problems in the fixes themselves, 19 confirmed and 3
partly.

- Every tool reads a name in quotes or brackets, and says what an
  ambiguous one names. Only find_related did.
- find_paths, after its budget runs out, says `more` only where more
  paths are known to exist.
- describe_graph refuses several types at once instead of picking one by
  hash order, and reads " " as no type.
- The notes count what they say: a roster record typed twice is not a
  node typed twice; a role's source given twice is named; links to
  records, and what a record says besides its rule, are counted.
- A NaN or a hexBinary value goes out as its text, not as a bare NaN or
  Python's repr. sparql_query's description says how cells are written.
- The descriptions say what equal means with several starting nodes
  (worst ties, then the next worst), that `also` is inside `link`, and
  what tied, note, avoided and a path's fields hold. A guard now checks
  both ways that every field a reply carries is named.

Benchmark: blank-node labels, and an IRI against the same text, compare
equal; whole numbers are compared to 12 digits as other numbers are; a
link and a value that read the same count once each; a question with
rows is answered whatever its terms; one the file refuses is set apart,
not shown as 0 rows. The test graph's amounts now agree only to 12
digits, so the rule is tested where it is used.

The README says what the SDK's Host and Origin check covers, and no
longer that the view counts everything it leaves out.

Tests: 140, from 136.

Part of #438. Refs #441.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
@Sulstice Sulstice self-assigned this Nov 12, 2026
@Sulstice Sulstice added this to the BBQS-KG-V1 milestone Nov 12, 2026
So the server can be reached at a URL such as neuromcp.brain-bbqs.org:

- --allow-host (or BBQS_KG_ALLOWED_HOSTS) names the public host names.
  The MCP SDK then keeps checking Host and Origin on /mcp for them and
  the local names, behind a proxy or a tunnel. Without it, a request
  forwarded for the public name is refused on 127.0.0.1, and nothing is
  checked on any other address.
- --port defaults to $PORT, as hosts set it.
- Run without the repository (uv run from its GitHub URL), it falls back
  to the export committed to main.

The last review of round 5 (8 findings, 7 confirmed, 1 partly):

- A name wrapped in quotes or brackets is unwrapped exactly once, as
  written: a comma or a quote inside stays, and [208980e9] is an id, not
  a number.
- sparql_query's description names truth values and how NaN and
  infinity are written; both writers spell them NaN, INF, -INF.
- A roster record's other types are counted among the triples the view
  does not use, and the README says the export does carry datatypes.
- A question the file refuses shows a dash throughout, and the report
  counts it. The IRI test is held by the walks, the blank node apart.

The benchmark walks for CQ16 and CQ18 now meet a repository and an
institution that are nodes; the two breaks of them that went unnoticed
are caught.

Tests: 143, from 140.

Part of #438. Refs #441.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
@Sulstice
Sulstice enabled auto-merge November 12, 2026 18:32
Sulstice and others added 12 commits November 16, 2026 15:14
Publications: each paper's tie to the grant it is credited to (the produced
link from project_publications), checked the ways the export allows: no
project, two or more projects, a title that appears twice, and whether an
author is on the project's roster by ORCID (21 of 89 papers carry author
ORCIDs). The text is in no export: publication_fulltext is admin-only and
its licences are unknown (#444). In the artifact viewer a curator reads it
through their own BBQS-MCP connector (kg_query, run as them under RLS);
nothing read that way is stored in the page.

Meetings, in the artifact copy only: kg/meetings.py turns the working-group
notes into one record per lesson, agreement, disagreement, decision and
action, tied to the people the notes attribute it to, give it to or name,
each resolved from the first name the notes use (meeting speakers first)
and linked to the Investigator node where one matches. The notes are
summaries, not transcripts, and the page says so. They are internal:
quality.py --internal embeds them in a --fragment written outside the repo
only, never in --html or --json, and refuses either path inside the repo.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
kg/browser/index.html and export/quality.json rebuilt from the committed
anon export with the new tabs: no meeting notes, INTERNAL null.

tests/guards/inspector-internal-data: the template holds both
placeholders in order; the committed copy carries no meeting records; no
workflow passes --internal or runs kg/meetings.py; no meeting records are
committed anywhere. Each was seen failing first (the old committed copy, a
stray notes file in kg/export, --internal added to publish.yml).

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
quality.py also runs competency_questions_extended.yaml, the 23 questions
proposed in #441, and the Questions tab shows them beside the 19 adopted
ones, each set labelled, with a filter for either. The gate's count is
unchanged: ci_check.py still scores report["competency"], the adopted file,
until #441 adopts the rest. On the committed export the 23 come out 13
answered, 2 none found, 3 empty, 5 needing a full export.

The six "which X has no Y" questions (CQ21, CQ27, CQ30, CQ33, CQ38, CQ42)
now say so with `reads: absence`: no rows means nothing is missing, which
is reported as "none found" rather than as an empty answer blamed on the
exporter.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
All 23 questions proposed in #441 move into competency_questions.yaml, so
the gate, quality.py and the benchmark ask the same 42.
competency_questions_extended.yaml is gone.

A "which X has no Y" question (reads: absence: CQ21, CQ27, CQ30, CQ33,
CQ38, CQ42) answers whether it lists some X or none, so the gate counts
it either way: otherwise recording every missing ORCID would empty CQ21
and fail the gate on an improvement. On the committed export the gate now
counts 31 answered (baseline updated): 29 with rows, plus CQ27 and CQ42
with nothing missing.

The benchmark reads the one file; its results block in README.md is
regenerated ("29 of 42"). Two tests follow the change: the miniature's 42
are all adopted, and a baseline question with reads: absence may have no
rows.

Still open on #441: CQ37 has no job-to-project link (it asks posted_by),
and CQ39 keeps the fixed window the issue wrote rather than today plus six
months.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
kg/devices.py reads the device list (BBQS Device List, Who Said What, with
References) into records: 146 lines across 28 awards (117 devices and
sensors, 29 methods), 273 evidence records and 38 statements, each with
its tier. T1 is the team's slide, T2 the workshop summary (written from
the transcripts; the speaker is named but not checked against them), T3
a line of working-group notes, U a maker or model only the AI-assembled
list gives. A U entry is kept as ai_predicted, shown as "AI-predicted,
unverified", and counts only where the list records it as a guess (44
lines); placeholders such as "Not specified in record" are not guesses.

quality.py --internal now takes the device list beside the meeting notes
(once each, by kind) and links it to the graph: every award to its
Project node (28 of 28), each full name in a "who spoke to it" line to
Investigator nodes (a name the source marks VERIFY or "likely" is left
unlinked), and a maker or model to the catalog node of the same name.
match_person is now shared by the meeting notes and the device list.

The Devices tab lists the lines by award, filtered by tier, kind,
category, award or person, with each record's source link; a reviewer
can flag a line to confirm or correct a model. Project and Investigator
cards link to their lines. The guard covers the device records and the
raw files by name, each seen failing first.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
Attribution (kg/meetings.py, quality.py):
- A person the graph holds twice links to both nodes: the same-name and
  same-first-and-last-name rules are unioned, so a full name links to the
  record with a middle name too (7 people, 2 records each).
- "Name:" at the start of any clause is "said", not only in actions, and a
  parenthesis of names with a speech word ("agreed", "proposed",
  "supportive") is too; a list of members after a group name ("NIH (...)
  and ...") is "named". Owner prefixes keep their names past "team" or an
  acronym, and "Name to <verb>" actions are owned.
- New ties: Alex, Sankar, and a surname only one speaker has; no tie from
  an eponym ("X's Rules"). Names resolve within the meeting's group before
  the whole notes: 3 more resolved, 2 left (both ambiguous).
- Blockers and attendee lists are read; any other list field is counted
  as unread. Points are keyed on where they sit, not their wording, and
  the Flags tab says when a flagged point's text or ties changed.

Page: a paper's text keeps its scroll on re-render; "Check again" and
"Reload text" read past the cache; a retry waits as long as it is told
(up to 60 s); a refused flag keeps its note and says why; an open flag box
shows the current flag and warns if it changed; a long source note wraps
at 400 px; flag fields are read as text whatever a viewer stored; lost
flag subscriptions show on their own tab; "turned off for this page" has
its own message; "Points someone said or owns"; the text is for admins.

Publications: a paper whose roster holds no ORCIDs is "roster has no
ORCIDs", not "no author on the roster" (11 of the 20); titles that differ
only by inline markup pair (CaBLAM).

Leaks: kg/meetings.py refuses notes inside the repo; the scripts and
quality.py --internal refuse to run on CI; the guard reads whole workflow
files (continuation lines, variables), .github/scripts and actions, and
package.json scripts, exempts no file (the markers are built from a
constant), and the raw files are git-ignored. Docstring examples are now
invented ones; earlier commits on this branch still hold a few lines
copied from the notes.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
… catalog field

Schema: Project gains uses_device_category (range DeviceCategory), the
link CQ14 asks for and the reason it was unmodeled. bbqs.owl.ttl and
bbqs.shapes.gen.ttl regenerated with the pinned linkml 1.11.1 (the only
new triples are the slot's).

Exporter: one uses_device_category link per project and category in the
project_device_usage view (anon-readable), found by project id, or by the
grant number in its core form; a row whose project or category has no
node is skipped, and the count is printed with the other edge counts.
It also writes what the catalog declares and the exporter left out: a
model's aliases, output signals, price tier, regulatory class and
product, manual and SDK links; a category's typical use cases and
schema.org type; a maker's country.

The committed export predates the link, so CQ14 now finds nothing rather
than being unmodeled; the next export answers it. Tried on stand-in rows
for every table the exporter reads (a repeated row, a prefixed grant
number, an unknown project and category). The benchmark block and its
tests follow: 6 questions find none in this export, none is not modelled.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
Sulstice and others added 23 commits November 16, 2026 21:43
…step in

The first Graph tab made you pick a mode and read a dense side panel.
This one follows the order people explore in:

- It opens on the map: every node, each kind gathered in its own region,
  the main network and the separate pieces named, and the 189 nodes
  nothing links to drawn hollow at the edge by kind.
- Click a dot or search (/) to step in: the view glides to the node and
  its neighbours move onto a ring around it, each link an arrow named by
  its predicate. Hovering a neighbour peeks at where it leads.
- Every step is history: back and forward buttons and arrow keys, a
  trail you can jump along, Esc for the map, and an inset map showing
  where you are.
- The card says what the node is, sums its links by kind and lists its
  neighbours as chips that light up the node when hovered.

Still self-contained: layout, animation, pan and zoom are in the page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
An independent review reproduced 29 defects in a browser; all fixed:

- A fast Esc then back no longer saves a half-zoomed view as the map;
  the map's camera is saved where it was heading, and only if moved.
- + and - mid-step zoom where the step is going instead of freezing it.
- The search results close when focus leaves the box, and its ARIA
  state (expanded, active option) follows the list; IME Enter is safe.
- A step clears the hover and tooltip, and focus dropped by a re-render
  goes to the drawing, where arrow keys and Enter work.
- A re-render (a flag update) or a resize that leaves the canvas size
  alone keeps the pan and zoom; a real resize refits only a fitted map.
- Ring labels are cut to the room before the canvas edge and the hint;
  link names and peek labels keep off ring labels and dots.
- The inset map hides where it would cover the ring; map labels sit
  above their group, clear of the hint, at full contrast; on a phone a
  label that does not fit is left out instead of overlapping.
- The layout runs in idle slices instead of one 350 ms block.
- Roles with no public holder are counted and listed on their project;
  hidden kinds are said on the card; a hover on a hidden node is none.
- Fonts arriving and a pixel-ratio change redraw the canvas.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…plicates

Acting on Tek's review of the Inspector (2026-11-17):

1. Link, don't copy. A project's study_species claim that resolves is
   now only the studies_species link; the name was also written as
   studies_species_name, a second copy of what the Species node holds.
   A claim with nothing to link to is kept as studies_species_unresolved,
   so the text exists only where no node does. Schema, exporter, the
   #7 shape (now: any unresolved claim fires; bbqs:readsAbsence keeps it
   counted as checking when there is none), CQ27 and its benchmark walk
   (which now also catch an unresolved claim on a project with other,
   linked species), fixtures, tests, roundtrip (a claim carried through
   a species' alias or common name) and the docs follow. OWL and SHACL
   regenerated. The committed export drops its 32 studies_species_name
   triples, all of which resolved: exactly what the new exporter writes
   from the same data (re-serialising the export is byte-identical).

2. "Consistency check" becomes SHACL validation wherever the shapes are
   meant: the Inspector, the shapes file (validation.shapes.ttl), the
   docstrings and the README, which notes that consistency in the
   logical sense is the separate OWL reasoner pass.

3. Duplicates are stopped, not cleaned. quality.py counts groups of
   same-class nodes sharing a folded name (people also by first and
   last name) or an identifier (ORCID, DOI, PMID, a URL that names the
   thing), and ci_check fails when the count rises above the baseline
   (14 today, all removed by the isolated-nodes SQL). Proven RED with a
   planted second DeepLabCut. The Inspector flags each node, says which
   one it duplicates, and filters by it. A preprint and its published
   version are named as such ("one work recorded twice") and counted
   apart, to be linked as versions rather than merged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…review)

Written for the Supabase SQL editor (vpexxhfpvghlejljwpvt), applied by
hand like every migration here. One transaction; set_actor and
set_source_class first, so every write lands in data_audit_log under
migration:link_isolated_nodes. It starts in dry-run mode: it does all the
work, rolls it back and shows a per-block report; changing one line to
'off' applies it.

33 guarded, idempotent blocks: delete the 14 orphan investigator spine
rows, the test record, the duplicate Lightning Pose and the superseded
forecast; merge Niegil Francis's two records and the acoel species;
affiliate 11 NIH staff and 19 members; set issuing organisations on the
funding calls and organisations on announcements and the workshop;
hide the EMBER test uploads (#506) and link EMBER's collections; plus
13 optional blocks, commented out, each with the curator question it
needs answered first.

Tested on a local replica of the production schema (every migration
replayed, #512's and #517's included) across six scenarios: dry run,
real run, second run (no change), drift, missing columns, a planted
error. 197 checks pass. Simulated on the export: 189 isolated nodes go
to about 70, and to about 47 once #512's exporter half merges.

Still under adversarial review; fixes from it follow in a later commit.
Do not apply before then.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…ation findings

Safety:
- people-03 merges Niegil's duplicate only if its addresses are a subset of the two proven ones
  and it holds no profile value the keeper lacks (letter case counts, except in free text). The
  optional Walker merge does the same.
- people-05/07/08 link a person only when the row's institution text starts with a spelling of
  the target and names no other institute or campus. NIH staff text must start with NIH or the
  institute.
- An organization is created only with its first member (people-06 removed).
- A dataset a curator already hid keeps its hide record.
- Only provenance claims that state the keeper's cells move. curation_audit_log is left as is.
- lock_timeout 5s.
- notices-01 decides per spine row.
- Every kg_* temp function an earlier run left in the session is dropped first (no 42725).

Evidence:
- The trained_on links, Psych-DS and Jongwoon Kim move to optional blocks with a curator
  question. Both people-09 lines may add a second organization.
- A device crosswalk row is added only where all the live evidence for the bucket is the
  project's own; a harvester bucket with no evidence is not mapped.
- The superseded forecast is never linked.
- Dates come from current_date.
- The header records the #512 overlap and a new developer item (project_device_usage counts
  neighbour evidence).

Replica suite: 270 checks in 9 scenarios pass; 8986039 fails the 36 new ones. Default run: 31
blocks plus 14 optional. Simulated effect: 189 -> 72-77 isolated with today's exporter.
Not applied to production; the admin runs it by hand, dry run first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…ated, 0 duplicates

The anon export of 2026-11-17, made after 20261116150000_link_isolated_nodes was applied in
production.
- Nodes 808 -> 758; isolated 189 -> 76.
- Possible duplicates 14 -> 0. 5 preprint + published pairs remain, to be linked as versions.
- CQ14 now has rows: 127 uses_device_category links. This is the first export by an exporter
  that reads project_device_usage.

Also in this commit:
- quality.json, the public Inspector page and the CI baseline are rebuilt from it.
- The MCP benchmark is rerun into README.md, and kg/README.md's latest-run table is updated.
- test_mcp_server: CQ14 is answered.
- test_mcp_server: naming every type in find_related gets the same size budget as naming none
  (60,000). With the device catalog joined to the projects, three more types are within reach
  of a person, and the largest reply is now 46,700. The cap per list is unchanged and still
  tested.

kg/export/roundtrip.json is not regenerated, because it reads the database.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
… the daily close job work)

Seven of the 13 calls have no expiration_date: the five PRIMED-AI RFAs, NSF 26-526 and the
Simons Empire AI RFA. So CQ39 finds nothing, and close-expired-funding-daily can never close
these calls: the Funding page lists the PRIMED-AI calls and the GRFP as open, although their
deadlines passed in October.

The script fills six dates from each call's notice, and only where the date is empty.
RFA-RM-27-015 is left out: no expiration date was found for it. The script does not touch
status: the existing daily job closes the expired calls.

Tested on the schema replica:
- The first run sets 5 dates, all audited under its own label, and keeps a date that was
  already set.
- A second run in the same session changes nothing.
- CQ39 then finds 4 calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
Supabase's default privileges grant ALL on every new view to anon and authenticated. A simple
view that runs with its owner's rights (as investigators_public, public_jobs,
feature_suggestions_public and investigator_directory do, on purpose) is auto-updatable. It
passes writes to its base table as the owner, which row-level security does not stop.

On a replica of this schema:
- An anonymous visitor rewrote a job through public_jobs.
- An anonymous visitor deleted a feature suggestion through feature_suggestions_public.
- A signed-in user renamed another person through investigators_public and through
  investigator_directory.

20261117174500 revokes INSERT, UPDATE, DELETE, TRUNCATE, TRIGGER and REFERENCES on every view
in public from anon, authenticated and PUBLIC. SELECT is untouched, and nothing in src/ or
supabase/functions writes through a view. On the replica all four writes are now refused and
every read still works. A second run changes nothing.

tests/guards/view-write-privileges holds every later migration that creates a view to revoke
the same rights. It is red on an open view and on a missing sweep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…e by anon

The four job questions were marked access: full and returned no rows. The exporter never read
job details (only the spine's name, description, external_url and part_of_org), and posted_by had
no anon source at all.

- Exporter (kg/bbqs_kg.py) reads the public_jobs view, which the job board already shows anyone,
  and writes job_type, location, department, expires_at (xsd:dateTime), application_url
  (xsd:anyURI) and is_active onto each row's Job spine node. A row with no Job node is skipped.
  Link, don't copy: the institution is the part_of_org link (the spine's organization_id, or the
  job row's own); job_institution text is written only for a job with no organization node.
  Contact fields are never read.
- New view public.public_job_posters (migration 20261117170000, apply by hand): job spine row ->
  poster's investigator spine row (jobs.posted_by = investigators.user_id), nothing else: no auth
  user id, email or contact field. Owner rights like investigators_public; SELECT only for anon and
  authenticated. The exporter writes posted_by only as a link from a Job node to an Investigator node.
  Tested on a replica clone: planted jobs with and without a matching investigator, anon reads
  only the two columns and cannot write, a second run changes nothing.
- fetch() keeps why a table or view gave no rows (HTTP status and PostgREST code). A missing or
  refused public_jobs / public_job_posters leaves those facts out with a note; the export goes on.
- competency_questions.yaml: CQ18 and CQ35 read the institution through part_of_org to the
  organization's name, and job_institution only for a job with no link (CQ35 groups by organization
  and name); CQ37 reads the poster's name through posted_by. All four access: anon.
- Kept in step: the bench walks, the MINIATURE and AWKWARD graphs and their row counts, the status
  test pinned to the committed export, quality_baseline.json (CQ18 and CQ35 now answer on the
  committed export, through part_of_org), the README benchmark rows, kg/README, roundtrip TABLES.
- LinkML: descriptions for job_institution, posted_by, part_of_org and the contact fields;
  bbqs.owl.ttl and bbqs.shapes.gen.ttl regenerated with gen-owl / gen-shacl (their text order is
  hash-seed dependent, so the diff is large; the triple change is the five descriptions).
- kg/test_exporter.py: export() against a PostgREST stand-in. RED on b10e648 (16 of 23 fail),
  GREEN now; wired into the KG workflow.

CQ36 and CQ37 still have no rows on the committed export (2026-11-17): it predates this exporter.
The test pinned to that export says so.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…nts to their papers (under review)

Answers the KG reviewer's point that "if something is in the KG already, you don't want to
duplicate it": the existing duplicates were cleaned by 20261116150000, but nothing stopped new
ones. One identity rule per key, written once (kg/identity_keys.json, 73 test vectors) and
applied the same way in SQL, TypeScript and Python. Guards hold the three equal.

Schema and KG:
- 10 LinkML unique_keys (owl:hasKey) and SHACL uniqueness shapes #21-#24 (exact match).
- revision_of (prov:wasRevisionOf), from a published paper to its preprint, with shape #25.
- The exporter reads resource_links.
- quality.py counts a linked preprint + published pair as resolved, and still flags an
  unlinked one. CI gates a rise in unlinked pairs.

Database (20261117181500_prevent_duplicates.sql, a dry run as shipped):
- Unique indexes on normalized ORCID, DOI, PMID, organization name, funding call FON and URL,
  and grant award. Each is created only if live rows don't collide; a collision is reported,
  not forced.
- revision_of is added to the resource_links rules, and kg_link_publication_versions() links
  the known pairs.
- Tested on the schema replica: 475 checks in 16 scenarios. They are red without the migration.

Write paths (edge functions, deployed one by one):
- Get-or-create in import-grant-publications, bbqs-mcp (add_resource, add_funding_opportunity
  with supersedes, create_grant, repoint_grant, refresh_grant_from_reporter), nih-grants,
  add-project-by-grant, seed-consortium and verify-affiliations.
- A person row is written before its spine row, so no new orphan.
- A unique-index violation (23505) is reported per row as "already in the KG".

Under review: an independent verification is running. Nothing is applied or deployed.
kg/export/quality.json and kg/browser/index.html are rebuilt after the review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…s; hide them column by column

An audit of every table and view the site's public key can read found private columns open to
SET ROLE anon (replica of this schema, a planted value per column): investigators_public.user_id
and pending_role; projects.last_edited_by (respondent and editor e-mails) and, inside
projects.metadata, questionnaire_submitted_by/_at/_response_id; the same three through
project_questionnaire_status (meant for curators); auth user ids in resources.created_by,
announcements.posted_by, entity_comments.user_id, harvester_relations.approved_by,
harvester_settings.updated_by and dandisets.hidden_by; dandisets.hidden_reason/hidden_at and raw
(EMBER's payload, contributor e-mails in it); feature_suggestions_public.submitter_name and
github_username (hidden by 20260515233153, put back by 20260813185734). Row-level security decides
rows; nothing decided columns.

Migration 20261117190000_hide_private_columns_from_anon (apply by hand, one transaction, re-runnable):
- Column grants: on those nine objects anon loses table-level SELECT and keeps SELECT on every column
  it could read before except the private ones. Signed-in users and service_role are untouched.
- project_questionnaire_status: anon loses SELECT; it reads the attribution from new columns.
- The questionnaire attribution moves out of the public metadata blob into three projects columns
  anon cannot read. A trigger (trg_aa_keep_questionnaire_attribution_private) moves the keys out of
  metadata on every write, so import_questionnaire_response, a hand-written import or a stale save
  cannot put them back; the first recorded submission wins, as before.
- Refuses to commit if anon could still read a private column, would lose or gain a public one, or
  if signed-in users' access would change. Reports per object what anon could read before and now.

PostgREST runs select=* as SELECT *, so every anon read of these objects now names its columns:
PrincipalInvestigators, InvestigatorSummary, GrantSummary, ResourceSummary, GenericSummary,
useResources, useProjectProfileEditor, Announcements, EmberDandisetsPanel; FeatureSuggestions and
EntityComments ask for the submitter / user_id only when signed in (visitors see "Submitted by: -");
e2e/data.spec.ts counts with select=id; kg/bbqs_kg.py fetch() uses PUBLIC_COLUMNS (roundtrip.py too).
The lists work before and after the migration.

Guard tests/guards/anon-private-columns (RED on the old tree: 14 readers): no select("*"), select()
or obj(*) on these objects in src/ or e2e/, no private column in a plain anon list, PUBLIC_COLUMNS for
every one the exporter reads, every listed column real, and no later migration that gives anon a
private column back or adds a column to one of them without saying whether anon may read it.
kg/test_exporter.py serves these objects as PostgREST does after the migration (18 tests fail with
the old exporter). Tested on replica clones: RED/GREEN, the site's and exporter's anon reads replayed
as SQL, signed-in reads, the import function and the trigger, a second run changing nothing, and
the live shapes types.ts implies (no dandisets.hidden*, a 16-column status view).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…in order, the KG gate checks node for node

Fixes the review findings of b7c6044 assigned to the KG and DB side: kg#1-#10, db_edge#2, #6, #7,
#8, #10 and the migration's part of #4. Nothing is applied or deployed: 20261117181500 still ships
as a dry run.

Identity rules (kg/identity_keys.json; SQL, TypeScript and Python held equal by 110 vectors, 97 of
them in the migration's self-test):
- kg#1: a funding call's URL is not an identity. funding_url leaves the LinkML keys (and its
  owl:hasKey); the url fold stays, for the write paths and duplicates(); the migration builds no
  uq_funding_opportunities_url_key and no kg_url_key.
- kg#4: DOI resolvers without a scheme (doi.org/, www.doi.org/, dx.doi.org/) and 'doi 10.x';
  'PMID 123'; 'www.x/a' and 'https://x/a' are one URL.
- kg#5: Greek letters are spelled out before the name and title rule (alpha- and beta-synuclein
  are two titles; the micro sign is mu), from one table in the JSON.
- db_edge#2: a bioRxiv/medRxiv DOI is told by its shape (10.1101/, an optional date, six or more
  digits), so Genome Res, Genes Dev and Learn Mem are no preprints. New rule "chronology"
  (kg_preprint_is_later, preprintIsLater, preprint_is_later): a preprint later than the paper, by
  year, else by PubMed id, is never paired.

Migration 20261117181500:
- The pairing rule skips a later preprint and says why. versions-01 links the 5 pairs a curator
  read and only them (new argument _both_listed); any other pair is listed as would_link (#6).
- Two publications on one node: give one its own spine row, then link; never merge (#7). The
  curator command names the curator (set_actor, curator_fill, that pair only), and the importer
  clause says its weekly schedule is commented out (#8).
- The header names every write path that meets the indexes, and says to check ../bbqs-agent first
  (#4). The Stony Brook pair is recorded as a known duplicate the name key cannot see (#10).
- A three-argument kg_link_publication_versions left by the draft is an error row: two overloads
  would make the importer's call ambiguous.
- Replica: 547 checks in 18 scenarios pass (pairs and draft3 are new). The new checks fail on
  b7c6044's migration, and 8 mutants of the new one are caught.

KG gate (kg/quality.py, kg/ci_check.py, the fixtures):
- kg#2: version_kind() resolves a same-title group only when its one published record links to
  each preprint. revision_links() reports a link from a preprint, or between two titles, and the
  gate fails when they exceed the baseline (0).
- kg#3: ci_check pins node for node what each shape, duplicates() and revision_links() must report
  on both fixtures. RED has cases for #25's four branches (now disjoint) and for version_kind, and
  IDENTIFIERS is pinned. 21 mutations fail the gate; the old gate passed 9 of the review's 10.
- kg#5: publications are grouped by title_key of bbqs:title, as the database pairs them. kg#4: an
  unreadable identifier is compared as written. kg#9: a shared URL counts only with a shared name;
  alone it is advisory.
- kg#6: tests/guards/kg-identity-keys parses each kg_unique_key call and compares index, table and
  expression exactly.
- kg#7, #8, #10: README and LinkML say what is planned (not applied), what the organization name key
  catches and misses, and that the ORCID owl:hasKey is declarative only.

kg/export/* and kg/browser/index.html are not rebuilt: no number on the committed export changes
(0 duplicates, 5 pairs to link, 0 links to check).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…duplicate refusal as what it is

Review findings on b7c6044, write paths and site (db_edge#1, #3, #4 edge/src part, #5, #9):

- add_resource (db_edge#1): only the name identifies a tool-like resource. A record that shares only
  a homepage or repository URL is never PATCHed: the new resource is inserted and the other one named
  in the note. same_as=<id> is the explicit way to update a record that has another name. The rule is
  decideResource in _shared/kg-write.ts; matchResources marks a URL-only hit same:false.
- add_funding_opportunity (db_edge#5): the FON is the call. A notice URL another call has is a hint
  in both branches (a landing page lists several calls), never a refusal; supersedes stays the way a
  final notice replaces its forecast. The PATCH has the 23505 handling the POST had
  (decideFundingCall).
- seed-consortium and matchPeople (db_edge#3): a name is a guess. A name match onto a record that
  carries another email, or a name key that finds several records, is ambiguous: nothing is written,
  and seed-consortium reports it for a curator. An email-less RePORTER twin is still reused, and noted.
  So no working_groups write from here fires trg_sync_member_groups for another person's address.
- A 23505 is said as what it is (db_edge#4): update_my_profile and MemberProfileEditor leave out an
  ORCID iD another record holds and save the rest (a null _orcid keeps the stored one; checked on the
  replica), and say to ask a curator to merge. create_grant and repoint_grant, nih-grants and
  add-project-by-grant name the award already in the KG. Publications (DOI), the add-funding dialog
  (FON), InvestigatorSummary's inline edit (ORCID; it said "try again" for every error) and the
  project profile editor (award) use src/lib/alreadyInKg.ts. Every other error is shown as it came.
- emailKey's comment (db_edge#9) says how it differs from investigators_email_unique_ci.

tests/guards/write-paths-get-or-create: 8 new tests (decideResource, decideFundingCall, matchPeople,
the src messages and saveOwnProfile, and the handlers' use of them). On 6b81261, 12 of its 27 tests
fail; 12 planted regressions are each caught.

Nothing is applied or deployed. Deploy per function: bbqs-mcp, seed-consortium, nih-grants,
add-project-by-grant (supabase functions deploy <name> --project-ref vpexxhfpvghlejljwpvt).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
… table

The 2026-11-18 export asked investigators_public for its public columns (PUBLIC_COLUMNS, since
the privacy change). The live view answered 400 42703 (no such column), so fetch() returned no
rows. The export lost every person's ORCID, roles, working groups, research areas and skills:
4,667 triples instead of 5,352.

fetch() now reads the column name from PostgREST's 42703 message, drops that column, says so,
and asks again. It also remembers the column, so a later read of the same object doesn't ask for
it either. Any other refusal still gives no rows, as before. The new exporter tests are red on
the old fetch() and green now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
… two curated links

The anon export made after 20261117160000 (funding call expiration dates) and 20261117170000
(public_job_posters) were applied.
- 758 nodes, 1,050 links; isolated 76 -> 73 (DLC Leaderboard, DeepLabCut and VCL Benchmark are
  joined by the two resource_links rows).
- CQ37 (who posted each job) and CQ39 (calls expiring in the window) now have rows: 36 of 42
  answered as the gate counts.
- CQ36 still has none: the live public_jobs view gave anon no rows, so no job carries a job_type.
- investigators_public has no `onboarded` column in production; the exporter dropped that one
  column and kept the rest (82b02ac).

One violation, recorded in the baseline until it is settled: ProjectRoleTokenShape (#4, issue
#283). A curator set one role on U24DA064429 to `collaborator` since the last export, and that
token is not in the role vocabulary. The Inspector shows it on the node.

quality.json, the public Inspector page, the CI baseline and the benchmark block in README.md
are rebuilt from this export; kg/README.md's latest-run table and the question-status test
follow it.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…s to any member's record

record_investigator_alias is SECURITY DEFINER, and its gate lets through any caller with no
auth.uid(). That is meant for the service role, but an anonymous API caller has no auth.uid()
either. No migration ever revoked the default EXECUTE from PUBLIC and anon. So the site's public
key could append an address to any member's secondary_emails, and globus-auth signs in an
address found there as that member. check_onboard_conflicts has the same gate and returns the
member's address by name. Six provenance maintenance functions (attach, detach, backfill) run
DDL or write claims with no check of the caller at all.

20261118100000 revokes EXECUTE on those eight functions from PUBLIC and anon, and grants it
back explicitly to the roles that call them: group-audit (service role) and the curators'
onboarding dialog (authenticated). It touches no function body, table or row.

Tested on a replica of the schema:
- Before: the anonymous alias call returned ok and changed the row.
- After: anon gets "permission denied" on both functions and may run none of the six.
- Curators, the service role, a member's profile save and the request-access check all behave
  as before.
- A second run changes nothing.

The full review of who may execute what is 20261118120000_privacy_round_two, in verification.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…or a URL never guesses a record

The last round's verification of 65b5b7a and 958f959 (both verifiers' new_problems, all fixed):

Pairing rule (20261117181500, kg/identity_keys.json, the three layers):
- Chronology: kg_preprint_is_later / preprintIsLater / preprint_is_later decide by PubMed id
  first; without two PubMed ids, by the posting date a bioRxiv/medRxiv DOI carries (doi_date,
  10.1101/ or 10.64898/), else the preprint's year, against the paper's year. A preprint's year
  is its latest version's (6 of 15 dated bioRxiv records in the export drift 1-2 years), so it
  only breaks a tie. New fifth argument (the preprint's DOI); 24 chronology vectors (were 12),
  109 SQL vectors in keys-02, with cases the PubMed id alone decides and the year alone would
  decide wrongly.
- kg_link_publication_versions tests an existing link (noop) and a reverse link before the
  order and the project credit, so a pair a curator linked by hand is reported as linked; a
  "later" row gives the SQL a curator runs to link the two by hand.
- links-02 refuses to run beside the year-first four-argument kg_preprint_is_later of the
  earlier draft (as it does the three-argument pairing function).
- The RESULT row says to resolve each group as its row says, not to merge every one.

KG gate (kg/quality.py, kg/ci_check.py, the fixtures):
- A same-title group whose only fault is a link from the preprint or a later preprint is
  "versions_to_check" (its own count, gated at 0, with advice to correct the link or value and
  never merge), no longer a duplicate to merge. The Inspector template shows it.
- RED fixtures pin the chronology wiring: PubMed id alone, the year that would decide wrongly,
  the DOI's date, the year alone, no year; GREEN a pair linked by hand against the PubMed order.
  12 mutants of the wiring (the verifiers' V1, V2, V9 among them) fail the gate.

Write paths (_shared/kg-write.ts, bbqs-mcp, seed-consortium, src):
- matchPeople: name-key records are ambiguous only when they carry two different emails; a
  curated record and its email-less RePORTER copy (20260811120000) is one person, the curated
  one reused, so nih-grants and add-project-by-grant link the PI again (pi@, #283).
- add_resource: a blank name is refused; a URL-only hit is never the same record; when one
  exists and no record has the name, nothing is written until same_as=<id> (update that one)
  or distinct=true (add it): no wrong overwrite, and no duplicate only SQL could remove.
- update_my_profile: through saveProfileLeavingOutHeldOrcid; only an ORCID 23505 is retried,
  and a failed retry still says the ORCID iD was left out.
- seed-consortium: the curator's worklist (needsCuratorDetails) is returned whole, errorDetails
  says how many it leaves out, and a name-only note says what was done, after it was done.
- The dead URL-identity code is gone (bbqs-mcp refusedBy's URL branch, the "url" key kind).
- tests/guards/write-paths-get-or-create checks every src insert, upsert and update of every
  column kg/identity_keys.json keys (investigators.orcid and email included), statement by
  statement; it found AdminUsers and AdminAccessRequests showing an email/name 23505 raw, now
  said through alreadyInKgMessage.

Docs: the migration header, kg/README.md and the LinkML revision_of description say what the
code does: member_self_update itself refuses the statement, its two callers save the rest; the
site first (main), then the six edge functions that import _shared/kg-write.ts (and through it
identity-keys.ts), then the script.

Evidence (replica, nothing applied or deployed): the DB suite passes 566 checks in 19
scenarios (the verifier's CaBLAM case among them: its preprint's year set to 2027, still linked);
on 958f959's migration 25 of them fail, and 6 migration mutants are caught. Guards 159/159 (15
of the 48 in the three changed files fail on 958f959's code; 13 guard mutants caught). KG gate: 12
mutants of the chronology wiring caught. Handler harness 25/25 (9 fail on 958f959).

Deploy (nothing is deployed): main for the site, then each of add-project-by-grant, bbqs-mcp,
import-grant-publications, nih-grants, seed-consortium, verify-affiliations with
supabase functions deploy <name> --project-ref vpexxhfpvghlejljwpvt; then the migration.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…he answers no page shows

The second privacy round, after 20261117190000. One migration,
supabase/migrations/20261118120000_privacy_round_two.sql (dry run as shipped):

1. Functions. On a replica with every migration applied, anon could run 100 of
   the 107 functions in schema public. record_investigator_alias let anyone add
   an address to a member's secondary_emails (its "a NULL auth.uid() is the
   service role" gate lets anon through), and globus-auth signs such an address
   in as that member; check_onboard_conflicts returned any member's e-mail
   address by name; the provenance maintenance functions created and dropped
   triggers; record_field_provenance and the vote counters wrote with no check.
   Every function now has a tier (on the replica: 10 anon, 58 authenticated,
   39 service). Before taking EXECUTE from a role, the run asks the live
   catalog what makes that role run the function (policies, views, index,
   default and CHECK expressions, invoker callers) and keeps it where something
   does, so an object only production has is not broken. The nine admin-only
   policies that named PUBLIC name authenticated (they are never true for
   anon), so has_role leaves anon.
2. projects.metadata is for signed-in users only. anon reads the 20 keys the
   public pages show from the new view project_metadata_public. GrantSummary
   and useMarrProjects read it through src/lib/projectPublicMetadata.ts;
   useProjectProfileEditor reads metadata signed in only and refuses to save
   answers it did not load; the KG exporter no longer asks for metadata (it
   writes nothing from it).
3. investigators_public.user_id and .pending_role are hidden from signed-in
   users too. No code read them from the view.

Guards: anon-private-columns reads both migrations' lists, checks embeds,
filters and the columns hidden from signed-in users, and checks the keys the
public pages read against the view; the new function-execute-grants holds every
later migration to say who may run each function it creates, and the decision
list to cover every function.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…p and privacy fix rounds

quality.py changed in 9600b2a (a preprint + published group whose only fault is a reversed link
or a preprint later than its paper is its own kind, "to check", never advice to merge). The
committed export is unchanged; quality.json and kg/browser/index.html are rebuilt from it.
Every gate passes on the merge of 9600b2a and 7fb8274: 165 guards, the KG gate, the MCP and
exporter tests, tsc and typecheck. Both rounds are still in independent verification.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…down

Production's investigators_public had no `onboarded` column on 2026-11-18 (20261008120000 is not
applied there). Since the privacy change the view is read by named columns, and PostgREST
refuses a select that names a missing column (400, 42703). So the People page and the person
summary on this branch would have failed for every visitor once deployed.

src/lib/optionalColumns.ts registers columns that only a hand-applied migration adds.
selectWithOptionalColumns asks for them, and when the database says one does not exist it asks
again without it. Both readers use it; a row without the field already falls back to
grants-or-working-groups.

Guards:
- tests/guards/optional-columns is red on the two readers as they were. It checks that no plain
  select names a registered column, that every helper call is literal and registered, and the
  helper's behaviour against a stand-in database (a mutant with no retry fails it).
- anon-private-columns now reads the helper's column lists too (red when a hidden column is
  slipped into one).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…d to the vocabulary

The roster editor and the person summary have always offered "Collaborator", but
normalize_grant_role never listed it, so the value was stored as an unknown token. The first
curator who chose it (U24DA064429, between the exports of 11-17 and 11-18) put a rule #4
violation into the graph that no data fix could clear. This is the same failure as #496
(`other`), one layer further out: the schema and the shape were held to the database function,
and the screens were held to nothing. The owner decided that collaborator is a valid role.

- 20261118140000 adds the token to normalize_grant_role. It replaces the function only if the
  live one is the August definition (or already this one); a hand-edited body stops it with
  nothing changed. It states who may run the function, and rewrites roster rows that hold the
  role in another spelling. Tested on the replica: a curator's "Collaborator" is now stored as
  the token, and a second run changes nothing.
- project_role_enum and shape #4 gain the token; OWL and SHACL are regenerated.
- tests/guards/role-ui-vocabulary is new: every value a role menu or the onboarding parser can
  write must be an input the function maps. It is red on the two screens before the migration.
- With the token valid, the committed export has 0 violations again. The baseline, quality.json
  and the public Inspector page are rebuilt.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
…t is on hold

The site reads the questionnaire answers a visitor may see from the view
project_metadata_public, which 20261118120000_privacy_round_two creates.
That migration is applied by hand and its review found problems, so it is
on hold; merging the site first would have shown every visitor empty MARR
and project details, because the view does not exist in production yet.

fetchPublicProjectMetadata now falls back when, and only when, the view is
missing (PGRST205 / 42P01): it reads projects.metadata, which anon may
still read in that state, and cuts it down to the keys the view carries.
A visitor sees the same content before the migration and after it, in
either deploy order. Any other refusal of the view shows nothing rather
than reading the table.

Guard (tests/guards/anon-private-columns), proven red on five mistakes:
- the fallback is the one read of projects.metadata visitor code may make,
  in that file, inside the view-missing branch;
- PUBLIC_METADATA_KEYS / PUBLIC_DEVICE_KEYS equal the view's key lists;
- behaviour: the fallback returns what the view would (public keys only,
  public device keys only, non-object devices as null), filters by grant
  number, and does not run for a permission error.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
… Python agree

The PR's `kg` check failed on 2026-11-19 with KeyError 'committed' in
test_the_committed_export_is_dated_by_its_commit_not_by_the_checkout.
Evidence: the runner's git is 2.56.1 and its Python 3.10; git 2.45 and
later print a UTC commit's strict ISO date (%cI) as 2026-11-18T16:34:20Z,
and Python 3.10's datetime.fromisoformat raises ValueError on the Z
(reproduced: "Invalid isoformat string"). _committed swallowed that and
returned None, so the server dated the export by the checkout's time,
which says when the repo was cloned, not how old the data is. It passed
locally only because the local git (2.43) prints +00:00.

_committed now asks git for %ct (seconds since the epoch), as the test
already did: one number, no format to disagree about.

Test (kg/test_mcp_server.py, TheCommitDate): a stand-in `git` on PATH
that answers as git 2.45+ does. RED on the old code under Python 3.10
(None != '2026-11-18T16:34:20Z', the CI's symptom), GREEN now; also
pins that a changed, untracked or commit-less file has no commit date.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pe69HFxTTGSnQirwPxqSVZ
@nadernik

Copy link
Copy Markdown
Collaborator

@Sulstice, holding this PR on a decision:

first, everything in Supabase connects to the KG in both directions, so operations like onboarding aren't disrupted. Then MCP query, read and write work on that graph. This gets specced properly before the second MCP server merges.

The spec is #522. Its phases:

  1. a table and column inventory, with a guard
  2. a lossless full-access export
  3. a KG → DB rebuild that matches production
  4. a scripted onboarding run that every write-path change must pass
  5. graph reads in bbqs-mcp

This PR's write-path changes, the unique indexes, create_grant and the edge-function rewrites, are what phase 4 tests. kg/mcp_server.py is weighed in phase 6.

Separately: 20261118140000_role_vocabulary_collaborator is live in production (normalize_grant_role now maps collaborator), but the file exists only on this branch. Please post on this PR when and by whom it was applied.

@nadernik

Copy link
Copy Markdown
Collaborator

@Sulstice, a note on the spec-signoff label. It's the reviewer's sign-off that the spec work for a danger-layer change is done, not a step for the author. On #512, #514, #517 and #520 it was added by the PR's author, before any spec entries existed. On #520 it then covered about 60 more files and 8 migrations pushed after it.

This check isn't a formality. The danger layers are migrations, edge functions, RLS and kg/, and a small diff there reaches far beyond the file it edits. In the last two weeks that showed up as:

  • an account-takeover hole in a function's grants
  • an open provenance writer
  • changes applied to production that no one else could trace

The spec and an independent sign-off are how those get caught before production, not after.

From #524 on, the gate counts the label only when a listed signer adds it, and any new push clears it. For danger-layer PRs, please request review, and I'll add the label once the spec entry is in. #522 is the spec for the graph work, so #520's parts can point there.

@nadernik

Copy link
Copy Markdown
Collaborator

@Sulstice, a follow-up to the hold above. Please close this PR and resubmit it as smaller PRs. At 71 files and 9 migrations it can't be reviewed or tested as one, and several of its parts are already live.

First, thank you for the security work. The record_investigator_alias hole was real in production. A function that adds a second e-mail to a member's record was callable without signing in, and sign-in accepts that second e-mail as the member. It is closed now, and the audit log shows no anonymous write to any investigator row. Round two also caught the open provenance writer and the vote counters. I've taken those three ahead as #523, because the rest of round two has to wait for the front end.

Since I last looked, the People page and the investigator panel fall back when onboarded is missing, the public setup uses --no-sparql, and the kg check passes.

Resubmit as five PRs, in this order:

  1. Security.
    • Record the hotfix you applied: date, who applied it, and its output.
    • Then the rest of round two's function grants.
    • The column changes (hide_private_columns_from_anon and round two's columns) come last, in this order: 20261008120000 → the front end and exporter read named columns → the column revokes.
  2. What's already live from this branch. link_isolated_nodes, funding_call_expiration_dates, public_job_posters and revoke_writes_through_views are all in production, and close_expired_funding_now is too, with no file anywhere. Please commit the missing file and post "applied by " for each one. public_job_posters reverses the earlier decision to hide who posted a job, so it needs sign-off.
  3. Duplicate detection. This is identity_keys, _shared/kg-write.ts, create_grant, prevent_duplicates, and the rewrites of bbqs-mcp, add-project-by-grant, nih-grants, seed-consortium, verify-affiliations and import-grant-publications.
  4. The graph MCP server, CQ20–42 and the benchmark, after Spec: connect every Supabase table to the KG in both directions, then KG reads and writes over MCP #522 phase 5.
    • The SPARQL deadline is checked inside triples(), so a single long evaluation step can still overrun it. In my test, single queries stalled the server for 16 to 20 seconds.
    • The uv run https://raw.githubusercontent.com/.../main/kg/mcp_server.py line runs whatever is on main at the time.
    • The description says this doesn't touch bbqs-mcp, but it changes 223 lines there.
  5. The Meetings and Devices tabs: on hold until we've decided about consent for attributing statements to named people.

Refs #522

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spec:C Danger layer — Spec Kit + sign-off required spec-signoff

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants