feat(knowledge): permission-aware knowledge bases behind a feature flag (#7385)

* feat(search): add Sim Search mode, connectors tab, and cited sources

- Build/Search mode switcher in the chat composer; Search mode swaps the
  suggested actions for Connect Sim Search rows that open the OAuth modal
- Search tab beside Integrations and Skills: a personally scoped catalog
  of knowledge-base connectors with connect-in-place rows and a
  connected-credential page without sharing or members
- <source> special tag contract: inline citation chips at the citation
  point plus a scrolling, right-faded sources strip under the reply
- chip `shape` variant so a fully round chip is chip chrome, not a class
  override
- keep the connector catalog out of the workflow editor's module graph
  and record the new routes in the boundary baseline

* fix(search): harden source parsing, availability, and provider matching

- validate <source> urls by parsing them and requiring a host, so a
  malformed value never renders a dead citation link
- gate Search-mode suggestions on deployment OAuth availability, sharing
  the predicate with the Search catalog
- match connected credentials across a service's additional provider ids
  (Salesforce sandbox) and count them in connector telemetry
- collect footer sources from the rendered text segments, covering a
  block-less message's fallback text and excluding subagent lanes
- use a distinctive citation-link sentinel and an absolute import for
  SuggestedActions
- teach the email tokens transcription test the composed chip geometry

* feat(knowledge): add document ACL schema and members-mode tables

Adds document.acl (sorted access tokens, fast default '{ws}') with a partial
GIN index and a token-shape CHECK, source_modified_at, the members-mode
columns and lease on knowledge_connector, a real FK on credential_id (left
NOT VALID until script migration 0011 has remapped legacy account ids), and
three tables: knowledge_connector_member, knowledge_document_observation,
knowledge_connector_member_sync_log.

The document contract stops passing unknown keys through so the new column
and storageKey never reach clients.

* feat(knowledge): add access scope, token, and predicate libraries

Pure building blocks for document-level access: the token vocabulary and
its single sorting/validation site, the subject token derived from a managed
credential row, the read-side predicate (scalar binds, literal ARRAY), the
per-principal scope resolver with a lazy memoising provider, the branded
system scope, and the members-mode sync limits. Nothing consumes them yet.

* feat(knowledge): enforce document access scope on every read path

Every document loader, chunk list, tag usage, search leg, metadata lookup,
and knowledge-base file serve now takes a required access scope and applies
the shared ACL predicate. Context resolvers attach a lazy, memoised scope
provider derived from the acting principal and load documents through it,
so a document the caller may not read is absent from the first read. Raw v1
routes and legacy helpers resolve the same scope; the outbox handler is the
one branded system-scope caller. Vector legs run inside a transaction that
enables pgvector's relaxed iterative scan (with a 42704 fallback) so a
narrow ACL still fills topK. Forked workspaces copy documents as '{ws}'.

Behaviour is identical today: every row carries the default '{ws}' token.

* refactor(knowledge): extract connector sync primitives from the sync engine

Splits the content sync engine into four modules without changing behaviour:
sync-lock (lease predicates, heartbeat, and a SyncRunLease the stages run
under), sync-persistence (storage and document-row writes), sync-primitives
(the pure decision functions plus the six orchestration stages: listing
pass, owned-corpus load, classification, hydrate/persist/dispatch, deletion
reconciliation, and the stuck-document sweep), and sync-engine (lock
acquisition, terminal state, and executeSync composed from the stages with
its original closures). The stages take the token, hydration, and lease as
inputs so a second engine can run the same guards under a different lease.

Tests keep every assertion; only import paths moved with the symbols.

* feat(knowledge): plumb credential groups into knowledge connectors

Registers Google Drive as a Credential Group provider (provider list,
adapter registry, and the Google managed-OAuth policy), marks the Drive and
Confluence connectors as permission-scoped listings with the config fields
that would cap them, and extends the Credential Group policy with a
knowledge_connector principal and a credential_group:OptionId condition so a
group can grant one connector the credentials collected under one option.

The policy codec gains one KnowledgeConnectorCredentialAccess statement per
option, decoded by SID; workflow-access edits now carry connector grants
through untouched. Knowledge-side helpers grant and revoke the binding under
the policy's revision CAS, mint a member token only when the policy names
the connector for the credential's option (audited per access), enumerate an
option's credentials in every state for membership reconciliation, and
validate a members-mode binding before anything is touched. Nothing calls
them yet.

* feat(knowledge): add the members-mode sync engine

Adds the engine that crawls a knowledge connector once per Credential Group
member with that member's own token and derives each document's ACL from who
observed it. A run takes the member lease (mutually exclusive with the
content engine's), finishes any pending access rewrite, mirrors the option's
credentials onto member rows (active or suspended, tokens rewritten, departed
members dropped), then claims due members one at a time until its budget
ends: each member's listing is admitted into a first-writer-wins union, every
listed document is hydrated once through its observers' tokens, observations
are recorded, removals follow only a full, complete, non-suspect listing, the
affected ACLs are rematerialised from the observation graph, and the document
lifecycle tombstones the unobserved, resurrects the re-observed, and purges
the long-unobserved under the lease. The run re-dispatches itself while
members remain due.

Around the engine: a member queue mirroring the content dispatch (token,
lease, lost-hand-off release), a Trigger.dev task on its own queue, a cron
scheduler that reclaims stale leases and lost dispatches, closes orphaned run
logs, sweeps members whose crawls stopped, and dispatches due connectors, and
the Docker and Helm cron entries. The content engine and its dispatch refuse
members-mode connectors, workspace-mode syncs restore the workspace ACL on
completion, document writes carry the ACL each mode needs, the hard-delete
guard can verify the member lease, and the Drive and Confluence connectors
report a scope the member cannot reach as an empty listing rather than an
error. Nothing enters members mode yet.

* feat(knowledge): let connectors enter members mode

Wires the members-mode engine to the connector surface. A connector can be
created in members mode (admin only) bound to a Credential Group option, or
switched between modes through a new admin-only access operation and route:
the switch takes the connector's content lease, rewrites the documents' ACLs
to the new mode's default in bounded batches (finishing under the member run
when the request budget runs out), grants or revokes the option's credentials
in the group's policy, flips the mode, and queues the first run of the new
engine. Members-mode source edits refuse listing caps and queue a member run,
a manual sync routes to the member queue, keep-documents deletion is refused,
and deletion revokes the grant. Contracts carry the access mode, the binding,
the member-sync state, the member summary, and member run logs; the member
scheduler and dispatch honour a paused connector.

* feat(knowledge): per-member access controls in connector settings

Add an Access section to the add and edit connector modals so a workspace
admin can sync a permission-scoped connector once per credential-group
member. The connector card shows the per-member badge, member sync status,
and a member run history in place of the content sync history.

* feat(knowledge): note member-scoped retrieval on the knowledge block

Knowledge bases report whether a live connector syncs per member, and the
knowledge block's selector explains that such documents depend on who
triggers the run.

* feat(knowledge): per-member change feeds for members-mode connectors

Connectors may open a change feed over a member's view of the source.
Google Drive reads changes.list, where a lost share arrives as a removal,
so a member's access is withdrawn without waiting for a full listing;
Confluence filters its CQL listing by lastModified for incremental
content refreshes. Each member stores where their feed resumes, and a
member with an open feed is relisted fully only weekly as a check.

* feat(knowledge): hybrid retrieval by default with a source-recency boost

Search fuses the full-text and semantic legs unless a caller asks for
vector-only, at every surface: internal, v1, v2, the knowledge block,
and Sim's own knowledge tools. Connector documents now record when the
source last changed them, and a recently modified document edges past
a stale one of similar relevance.

* feat(knowledge): gate permission-aware knowledge behind a flag and harden the stack

Add the knowledge-member-access feature flag (workspace allowlist and
platform admins via AppConfig; KNOWLEDGE_MEMBER_ACCESS off-AppConfig). It
gates members mode at creation and switching, the member engine, and the
hybrid-by-default retrieval with its recency boost; the connector modals
read it from the workspace host context. The knowledge block's retrieval
mode gains an Automatic option so the server's default applies.

An admin with no credential group for the connector's provider can create
one from the Access field; the group is named after the connector and
members are invited from Settings.

Review fixes: manual-only connectors no longer re-dispatch member runs
forever; ACL materialisation is idempotent so a run that died mid-way is
repaired; incomplete listings retry at once only when they can resume;
suspended members keep their observations; a scope the member cannot reach
closes their change feed; the workspace ACL restore runs inside the
completion lock and only for workspace-mode connectors; the mode switch
grants before rewriting and flips in one transaction, clearing listing
caps; minting checks group, option, and enrollment status; connector
document lists, bulk select-all operations, and tag usage stats apply the
access scope; the connector card reads the member engine's status; the
edit modal applies an access switch on its own instead of racing the
settings save.

* test(knowledge): cover retrieval default resolution behind the flag

* fix(knowledge): judge members mode by the workspace alone and fail closed when the flag is off

Members mode is gated by the workspace at creation, switching, in the
member engine, and in the workspace host context, so no gate can open a
mode another will refuse; the flag's admin clause now affects only the
retrieval default, and search passes the signed-in person rather than a
billing owner or key creator. Member tokens count only where the feature
is on, so turning it off hides member-scoped documents on the next read.
New and reactivated members are due at once so a drain re-dispatches
until everyone has been listed. Leaving members mode flips first under
the lease with the rewrite marked pending, then rewrites; a failed move
between options of one group restores the previous option. The edit modal
closes after a switch and cannot re-choose per-member where the flag is
off; connector document exclusion honours the caller's scope.

* feat(knowledge): provision per-member access in the background

Choosing per-member access no longer asks the admin for anything: the
server reuses the workspace's one credential group for the connector's
provider or creates one, invites every workspace member to connect, and
each member run invites people who joined since. A member's only step is
to connect: the knowledge base page shows a banner for every per-member
connector they have not connected, and a new enroll route mints their
enrollment link on demand so the invitation email is never needed. The
connector list reports the viewer's membership for that banner.

The permission-scoped listing CI test now compares OAuth provider ids,
which is what the runtime checks. The knowledge page module baseline is
re-recorded for the provisioning graph.

* fix(knowledge): make per-member access self-serve end to end

Choosing per-member access provisions a group named after the connector
and the first member run invites everyone in the workspace, under the
run's lease; the request itself sends nothing, so nobody gets two links.
A member connecting queues a member run for every connector on that
option, and the knowledge base page shows what each viewer must do —
connect, reconnect, verify their email, or nothing — opening enrollment
in a new tab and refreshing until they are connected.

Every gate reads one availability check (flag and Credential Groups).
When it is off, readers get no member token, the engine waits instead of
suspending anyone, and the field cannot re-choose per-member access.
Member tokens go only to current workspace members with a live group and
option. A connector that just entered members mode never tombstones its
documents before a member has listed, the stale sweep leaves paused and
manual connectors alone, a disabled member sync is re-enabled by
re-applying its binding, leaving members mode forces a full content
sync, and by-id bulk operations honour the caller's scope. Deleting a
per-member connector always takes its documents.

* feat(knowledge): list a workspace's per-member connectors for the viewer

One read returns every per-member connector in the workspace with where
the viewer stands on each, so a surface outside the knowledge base can
ask them to connect. The member sync status enum now has one home in
lib/knowledge/types.

* feat(search): show per-member knowledge connectors on the Search tab

The Search tab lists the knowledge bases whose connectors sync per member
under "Shared with you", with where the viewer stands on each and the same
one-click connect the knowledge base page offers. One hook opens
enrollment in a new tab and refreshes the surface until the viewer is
connected; the knowledge base banner uses it too.

* feat(knowledge): enable per-member access on every permission-scoped connector

Google Slides, Docs, Forms, Calendar, Gmail, Notion, Jira, Jira Service
Management, Box, Dropbox, monday, ClickUp, Asana, Salesforce, Linear,
DocuSign, and Zoom list only what the caller's own account can read, so
each now declares its listing caps and can sync per member. A shared
listing-scope error lets a connector report a folder, space, board, or
calendar the member cannot reach as a complete listing of nothing; Notion
and Asana report it from their typed errors, Dropbox from its 409, and
the Atlassian connectors also from an unreachable site. Google Docs and
Forms join the credential-group providers so their accounts can be
collected per person.

* refactor(knowledge): tighten the per-member access surface

- The credential-group policy canonicaliser carries knowledge-connector
  statements through instead of rejecting a third statement
- Google Docs and Box report an unreachable scope so a member's access is
  withdrawn rather than retried forever; Notion leaves members mode because
  its page access is granted to the shared integration bot
- Provisioning reuses a Credential Group only when other members-mode
  connectors already sync through it, never a group curated for something
  else; the fast dispatch on connect accepts the same statuses as the sweep
- Enrollment opens its tab inside the click so popup blockers cannot swallow
  it, and the membership poll stops after ten minutes
- Session-only routes use session auth; members mode stores the cap-stripped
  config; one liveness rule for members and managed bindings
- Drop dead code: the access barrel, system.ts and rank.ts folded into their
  siblings, unused ACL helpers, the unread change_cursor_at column, stale
  documentation, and the engine/provisioning/queue import cycle

* fix(knowledge): repair CI and apply cleanup passes

- Bump the chart version for the member-sync cron and secret
- Pass an access scope to getDocuments in the list-convention test and
  resolve a knowledge scope only for knowledge-base file reads
- Drop memoisation nothing observes, read the enrollment error from the
  mutation, hoist the empty connector list, use the default Cancel variant,
  let Badge own its gap, and keep the sidebar lit on the Search tab

* fix(knowledge): apply the audit swarm's findings

- Mutation responses failed contract validation because a viewer's
  membership was required; a mutation now answers with null
- Bulk enable/disable wrote to documents outside the caller's scope
- The staleness sweep tombstoned deferred and backing-off connectors;
  it now only touches connectors that completed a run in the window
- Members pace on the plain interval so the connector's jittered run
  finds them due; a member who alone exhausts the run budget backs off;
  an account that connects mid-run keeps the connector re-dispatching
- Gmail and Google Calendar kept a 500-item cap once the cap field was
  cleared; caps are now written as 0, which every connector reads as
  unlimited, on the update path as well
- Jira, ClickUp, Asana, Linear, and Dropbox classify an unreachable
  scope on the source's own error rather than a bare 404; an unmatched
  Atlassian domain is a configuration error, not this person's access
- A Credential Group or option serving a members-mode connector cannot
  be removed under it; the OAuth completion dispatch cannot fail the
  callback; the content queue refuses a connector that syncs per member
- The iterative vector scan runs only for a personal token set; the ACL
  check validates each element; the flag is read before the enrollment
  join; connector info is loaded only where it is shown
- Search tab: the empty state counts member rows, rows stack, one
  membership vocabulary shared with the knowledge base banner; an admin
  can rebind a per-member connector to any matching group

* feat(knowledge): search results a person can open, from Chat

- The composer's Search mode searches every knowledge base as the signed-in
  person and lists what they may read as result cards: source icon, title
  linking back to the document, knowledge base, updated date, and the
  matching passage with the query terms in bold; Summarize hands a document
  to the agent in Build mode
- The agent's knowledge tool returns each result's title, link, connector,
  and modified time and is told to cite with source tags carrying a snippet;
  a reply whose sources carry snippets ends with the same cards
- A session route for the search, bound to the shared search use case, so
  the browser reads through the same access predicate as everything else
- Search quality: hybrid legs over-fetch before fusion, the vector leg's
  iterative scan fills a limit past the default candidate pool, and the
  recency weight moves a fresh document a few places rather than the list
- A manual member sync makes every active member due, so Sync members now
  lists everyone instead of nobody

* fix(knowledge): land member observations and ACLs only under the lease

A run that stalled past the lease TTL and resumed after a replacement took
over could commit stale observations, membership rows, and document ACLs
over the replacement's. Every such write now runs in a transaction that
first proves the run still holds the connector's member lease, holding the
connector row's lock so the scheduler cannot reclaim it mid-transaction; a
run that lost it ends as superseded.

* fix(knowledge): disable a member sync only under the lease

A run whose binding was removed suspended members and rewrote ACLs before
proving it still held the lease; a run reclaimed meanwhile could suspend
the replacement's members and then fail. Suspension, the ACLs it changes,
and the disable now land in one lease-guarded transaction, and a reclaimed
run ends as superseded.

* fix(knowledge): record a member failure only under the lease

A listing that ran past the lease TTL and then failed wrote its backoff
over the replacement run's counters and schedule. The failure ladder now
lands under the same lease guard as every other member write.

* fix(knowledge): tombstone and resurrect only under the lease

The document lifecycle's tombstone and resurrection writes ran outside the
lease guard, so a run reclaimed after its ACL transaction could hide a
document the replacement restored or expose one it removed. They now run
in the same lease-guarded transaction as every other member write, and a
run whose lease is lost while it disables itself ends as superseded rather
than rejecting.

* fix(knowledge): prove the lease before a batch's document writes

Hydrating a batch can outlast the lease. processDocOps now takes the strict
lease probe before persisting anything from a batch, so a run replaced
during hydration cannot land stale content or queue processing over the
replacement's; it ends as superseded. Both engines share the guard.

* feat(knowledge): one click connects a Sim Search source and indexes it for the person

Connecting a source on the Search tab, or from the composer's Search-mode
suggestions, no longer creates a bare credential that indexes nothing. It
finds or creates the workspace's Sim Search knowledge base and a per-member
connector for the source, then enrolls the person; the OAuth completion
queues their member run, so indexing starts on its own and the row counts
their documents up as they land. Sources that need a site or space are set
up from a knowledge base instead. The credential-only detail page goes.

* feat(knowledge): the composer's Search mode shows every source and reads like a search

- Under the input, Search mode lists every Sim Search source as a chip with
  the person's own state: connected with a document count, indexing,
  reconnect, or one click to connect; sources that need a site link to
  Knowledge. The sampled four-row list goes, and with it the rows that
  offered sources the server had to refuse
- Results carry a header (how many documents, searched as you, and which
  source is still indexing), hover actions to copy the link or summarize,
  an Answer with Sim action for a prose answer, and source and recency
  filters once a list is long and mixed enough to need them
- An existing chat opens in Build; a new chat keeps the last mode. Results
  never join a transcript

* fix(knowledge): process connector-owned documents as the system

Members-mode documents are inserted hidden until the member sync
materializes who observed them, so the processor's own source-file read,
authorized as the actor with workspace scope, denied every document a
per-member connector created. Connector-owned rows now read their source
as the system; uploads keep the actor's authorization.

* feat(search): connect every personal source in place

The Sources strip lists only what a person can connect themselves and
asks for a source's required setup (a site, a space) in a small modal on
the first connect; sources an admin must set up as workspace connectors
no longer appear as dead chips. The indexing loader spins, the query
stays in the composer after a search, and the knowledge-base page drops
the per-member sync banner.

* feat(search): keep the results familiar to a search page

A result's second line names its source app rather than the knowledge
base, emptying the search box returns to the sources, and the arrow
keys walk the result links.

* feat(knowledge): sync Google Chat, Meet, Sheets, Bitbucket, and Airtable per member

Google Chat lists only the spaces the caller belongs to, Google Meet only
the conferences they organized, and Google Sheets, Bitbucket, and Airtable
one configured spreadsheet, repository, or table that a member either can
read in full or cannot reach at all, so each now declares its listing caps
and can sync per member. Sheets, Bitbucket, and Airtable report a 403 or
404 on their configured scope as a complete listing of nothing for that
member; Bitbucket reads a zero cap as unlimited instead of refusing it.
Google Chat, Meet, and Sheets join the Google managed OAuth providers, and
Bitbucket gains a managed policy that takes its subject from account_id
and its address from the confirmed primary email, requesting the email
scope that needs.

HubSpot stays a workspace connector: HubSpot documents that an OAuth token
reflects the app's scopes, not the authorizing user's record permissions.
Google Vault needs the Vault admin privilege and Trello authorizes through
its own OAuth 1.0a flow rather than a managed OAuth 2.0 connector.

* feat(knowledge): sync Microsoft connectors per member

Microsoft Teams, Outlook, OneDrive, SharePoint, and Microsoft Excel list
only what the caller's own account can read, so each now declares its
listing caps and can sync per member. Outlook reads a 0 conversation cap
as unlimited instead of the default. A team, channel, folder, site, or
workbook Graph answers 403 or 404 for reads as a complete listing of
nothing for that member, as does a channel or folder the member's own
listing does not show.

A Microsoft managed OAuth policy lets a Credential Group collect each
person's account: the id_token is verified against the identity
platform's keys and bound to the access token through the OIDC userinfo
subject, the person is recorded by oid and tid, and the email counts as
proven only through the claims Entra vouches for.

* fix(knowledge): close the connector access-switch and members-mode update races

- Leaving members mode dispatched the first workspace sync against a fresh
  clock read while the row held the flip's nextSyncAt, so the queue refused
  it as stale; the dispatch now asserts the instant the flip wrote.
- A workspace credential change kept the incremental watermark, so the new
  credential's corpus was never fully listed; it now drops lastSyncAt, makes
  the sync due, and refuses under a CAS while any sync owns the row.
- Both switch directions released the lease before revoking the previous
  group's grant, and a revoke drops the connector from every option of the
  group; the lease now outlives the revoke.
- The members-mode flip and the members-mode create lock the Credential
  Group row and re-check the option under it, the same lock the group's
  option removal and delete hold while they look for bound connectors.
- Members-mode updates wrote interval changes and resumes to nextSyncAt,
  which the member scheduler never reads; they now land on nextMemberSyncAt
  under the matching CAS, a running member run refuses every edit, a queued
  one refuses config edits, and a pause releases the queued entry.

* fix(knowledge): close member-sync races found in review

- sweep stale member observations under the connector share lock and the
  member row lock, re-checking mode and staleness so a run's fresh
  observations and a mode switch's ACLs are never overwritten
- gate the member queue CAS and lease on connector status and on the
  schedule the dispatch was made for; keep dispatching after one failure
- prove the run lease inside every connector document write transaction
- drop the per-member page cap on listing passes; a capped listing could
  never reach the documents behind page 200
- issue member invitations with reject-on-revoked so a concurrent
  revocation is never reactivated
- re-apply the caller's access at document delete and upsert-replace
- keep sourceModifiedAt/connectorType on provenance-bearing searches
- read Google Calendar's updatedTime as the source modified time

* fix(knowledge): resolve Sim Search review findings

- Sim Search connect: the first connect of a source requires a workspace
  admin and refuses everyone else with the way forward; per-member
  availability is checked before anything is created; creation runs under
  a per-workspace advisory lock with a re-check; the route forwards the
  source's setup fields
- Connector access switch: workspace mode requires credentialId in the
  contract and the credential must belong to the connector's provider
- Base64 hydration reads knowledge-base files as the run's principal
- Search surfaces gate on the workspace's per-member access flag; search
  results skip legacy personal bases, surface list errors, and report
  indexing across every per-member connector
- Enrollment hook: explicit copy for failed/disabled member sync, awaited
  connections are dropped once connected, each path clears the other's
  error; Search mode keeps its query in an existing chat and keeps its
  context chips diffable; citation labels are Markdown-escaped; an access
  switch also refetches per-document caches

* fix(connectors): withdraw member access on unreachable scopes and page unlimited listings

Gmail treated the members-mode cap of 0 as "capped after the first page",
so a member's sync never listed past 100 threads and could never withdraw
access. Box turned a subfolder a member cannot reach into a capped listing,
which suppressed removals forever; the members-mode crawl now marks its
sync context, and Box takes the skipped folder as absent from that member's
complete listing while a shared credential still caps. Google Calendar with
several calendars withdrew everything when one calendar answered 404; per
member it now skips only that calendar. JSM classifies 403 (a service desk
the caller may not view) alongside 404, and the Atlassian connectors also
treat a token that reaches only other sites as not on the site. Monday
reports every configured board coming back absent as the scope being
unavailable. Salesforce classifies 400 INVALID_TYPE and 403
INSUFFICIENT_ACCESS as the object being unreadable, and filters on
LastModifiedDate with supportsIncrementalSync so member passes stop
enumerating the whole org. The duplicated Graph listing error helper moves
into the shared connector utils.

* feat(search): make a search a shareable link

The composer's Search-mode query lives in the URL as q, so a search can
be bookmarked or sent; opening such a link restores the query and Search
mode.

* test(knowledge): pin member-access availability in the v1 search route test

* refactor(search): apply the cleanup passes to the Sim Search surface

Drop three memos over cheap derivations, make the search setter's
callers honest about their dependencies, ignore a whitespace-only query
from a link, give a dropdown setup field its hint, and drop the query
from the URL when the composer leaves Search.

* refactor(search): apply the simplify passes to the Sim Search surface

One availability gate for every members-mode refusal, the shared role
gate behind the admin-first message, setup fields precomputed on the
catalog, the connect flow owned by the enrollment hook and shared by the
strip and the Search page, the latest attempt derived instead of
cross-resets, a trailing adornment slot on the chip in place of a loader
wrapper, the processor's source access as one object with the
connector-row rule in one place, and the first connect's independent
lookups run together after its synchronous checks.

* fix(connectors): keep per-member listings complete across partial scopes and blank caps

Teams, SharePoint, and OneDrive skip one of several channels or a descendant
folder a member cannot reach instead of failing the whole listing, which the
members-mode crawl would read as the member reaching nothing. Gmail and
Outlook keep their default cap for a null, empty, or whitespace field rather
than lifting it, through a shared parseDefaultedUnlimitedSafeInteger. The
Confluence incremental CQL clause is fixed on the first page so a cursor is
never paired with a query that crossed a minute boundary.

* fix(knowledge): address review threads on search surface, lease-guarded dispatch, and selected-output principal

- Guard the processing-queue write and dispatch with the sync lease so a reclaimed run neither marks nor enqueues processing
- Pass the executing principal into the final-frame selected-output context
- Read JSM statusDate as the source modification time
- Keep a first-connected Sim Search source non-actionable until its membership row appears
- Render the Build/Search switcher only for the Home composer
- Keep every Search-mode submit out of the agent; attachment-only submits do nothing
- Skip the member-connector fetch and 'still indexing' note when per-member access is off, and scope it to the searched bases
- Queue the workspace member-connector list on a members-mode sync trigger
- Log a refused clipboard write on the source card instead of leaving it unhandled

* fix(connectors): validate Gmail's thread cap with the sync parser and reject invalid Date metadata

* fix(knowledge): prove the lease inside every ACL rewrite batch and harden source-modified-time parsing

* fix(knowledge): claim a member only under a proved lease and read email conversation dates

* feat(search): show the matching passage and the author on each result

A result's snippet is the window around the first query term, skipping
the header block an email carries, so the row shows why the document
matched; its meta line names the source, the person behind the document
from its author-like tag, and the date. The agent's citations carry the
author too.

* refactor(search): style the results and sources as one surface with the composer

Result rows take the chat surface's row rhythm with hairlines between
them, fade-clipped titles and meta lines, a proper icon button for Copy
link, a ghost Summarize matching Answer with Sim, actions revealed on
keyboard focus, and a linkless document rendered in the same row with
its author, date, and bolded passage. The source and date filters live
in the URL beside the query, cleared with it. The sources strip keeps
connected chips at full weight, and the member-connector query is gated
with an enabled option and cancelled before an optimistic queue write.

* fix(search): safe result links, script-aware term matching, and honest header stripping

A result links only to an http(s) URL; term matching judges word edges
by the surrounding characters instead of ASCII \b and strips quotes
from a phrase, and the same matcher bolds the passage; a chunk that is
nothing but fields keeps its content; the agent leaves unknown optional
citation fields out and keeps the tool's published result count; route
errors log the wrapped cause and Postgres code.

* fix(search): no cached member rows with access off, no double enrollment, code-point-safe matching

Surfaces consume member-connector rows only while the feature is on;
the Search page treats an awaited enrollment, including a first
connect, as non-actionable; a refused members-mode sync rolls back only
the member lists; result actions show on pointers without hover; route
error causes go through the redacting describer; the citation template
is valid JSON; term edges and snippet windows respect code points.

* fix(knowledge): create a Sim Search source without holding a transaction across the nested use cases

The first connect held an advisory-lock transaction while the nested
base and connector use cases queried the pool, which the transaction
tripwire now refuses, so every first connect of a new source failed.
Concurrent first connects are coalesced per workspace base and per
source within the process and re-check before creating instead.

* fix(search): admin-gated first connect on the surfaces, live indexing state, honest filters and loading

Route errors redact bound values from the outer message too; a source
nobody connected is offered only to an admin and others see why; a first
connect refreshes the base's connector list; member connectors poll
while a source is indexing for the viewer; a filter from a shared link
applies even when the controls would not appear; results kept from the
previous query show as loading.

* improvement(routes): log the deepest cause's stack outside production

* feat(home): add Ask mode, an agent turn grounded in the searched sources

* fix(home): keep the composer on the live search query, not a mount-time snapshot

* feat(home): make Ask turns answer from the searched sources with the knowledge tool alone

* improvement(chat): name the query in the knowledge search row and list a reply's sources densely

* fix(home): carry the Ask request mode through every send lane and refuse every non-server tool on an Ask turn

* improvement(chat): put a reply's sources behind a counted button in the action row, and keep Ask follow-ups to questions

* feat(chat): let the agent use the person's own Credential Group credentials, and keep integrations on Ask turns for questions knowledge cannot answer

* fix(knowledge): treat a reclaimed lease during a member purge as superseded; tidy the chat reply sources after cleanup

* improvement(home): fold Ask into Search behind an Answer toggle

* fix(credentials): prove a Chat managed-credential use only from a real tool call, and list or mint only live group bindings

* improvement(home): name both choices in Sources mode with a Search / Assistant toggle

* improvement(home): make Assistant a peer of Build and Search in the mode switcher

* improvement(home): keep the composer mode in the URL and drop the mode store

* fix(home): let Search be chosen inside a chat now that the mode lives in the URL

* fix(chat): scope the managed-credential listing to the group's workspace and carry only chat params on the handoff

* fix(credentials): refuse a managed credential whose group or option is disabled on every mint, workflow runs included

* fix(home): clear the composer when a result is handed to the Assistant, follow the search query in a chat, restore a queued message's mode, and seed the base list

* fix(knowledge): budget the pending ACL rewrite per run, and converge a racing first connect on the oldest base and connector

* fix(home): ground an Assistant send from the query cache instead of growing the page graph with a prefetch

---------

Co-authored-by: Emir Karabeg <emirkarabeg@berkeley.edu>

Author: waleedlatif1

apps/docs/content/docs/integrations/knowledge.mdx

@@ -33,7 +33,7 @@ Integrate Knowledge into the workflow. Perform full CRUD operations on documents
 
 ### Knowledge Search
 
-Search for similar content in a knowledge base using vector similarity
+Search for similar content in a knowledge base by relevance
 
 #### Input
 
@@ -43,7 +43,7 @@ Search for similar content in a knowledge base using vector similarity
 | `query` | string | No | Search query text \(optional when using tag filters\) |
 | `topK` | number | No | Number of most similar results to return \(1-100\) |
 | `tagFilters` | array | No | Array of tag filters with tagName and tagValue properties |
-| `searchMode` | string | No | Retrieval mode: 'vector' \(default\) uses semantic similarity only, 'hybrid' also runs a full-text leg and fuses both |
+| `searchMode` | string | No | Retrieval mode: 'hybrid' fuses a full-text leg with semantic similarity, 'vector' uses semantic similarity only; omit for the workspace's default |
 | `rerankerEnabled` | boolean | No | Whether to apply Cohere reranking to vector search results |
 | `rerankerModel` | string | No | Cohere rerank model to use \(one of: rerank-v4.0-pro, rerank-v4.0-fast, rerank-v3.5\) |
 | `rerankerInputCount` | number | No | Number of vector results sent to the Cohere reranker \(1–100\). Defaults to topK × 4 capped at 100. |

apps/docs/openapi-v2-knowledge.json

@@ -6144,7 +6144,6 @@
           },
           "searchMode": {
             "description": "Retrieval strategy: vector is semantic-only, while hybrid also runs full-text search.",
-            "default": "vector",
             "anyOf": [
               {
                 "type": "string",

apps/sim/.env.example

@@ -203,6 +203,7 @@ CRON_SECRET=your_cron_secret # Use `openssl rand -hex 32` to generate. Authentic
 # FORKING_ENABLED=                                                # Workspace forks
 # CREDENTIAL_GROUPS=                                              # Enterprise managed OAuth collections
 # TABLE_ROW_TTL=                                                  # Table TTL columns and expired-row cleanup
+# KNOWLEDGE_MEMBER_ACCESS=                                        # Per-member knowledge connectors and hybrid-by-default retrieval
 # ORGANIZATIONS_ENABLED= / NEXT_PUBLIC_ORGANIZATIONS_ENABLED=     # Organizations only
 
 # Instance organization (Optional). Most enterprise features read their settings from the

apps/sim/app/api/files/authorization.ts

@@ -4,6 +4,12 @@ import { createLogger } from '@sim/logger'
 import { permissionSatisfies } from '@sim/platform-authz/workspace'
 import { and, eq, isNull } from 'drizzle-orm'
 import { NextResponse } from 'next/server'
+import { knowledgeAccessCondition } from '@/lib/knowledge/access/predicate'
+import {
+  resolveUserKnowledgeAccessScope,
+  WORKSPACE_ACCESS_SCOPE,
+} from '@/lib/knowledge/access/scope'
+import type { KnowledgeAccessScope, SystemAccessScope } from '@/lib/knowledge/access/types'
 import { getFileMetadata } from '@/lib/uploads'
 import type { StorageContext } from '@/lib/uploads/config'
 import type { StorageConfig } from '@/lib/uploads/core/storage-client'
@@ -141,7 +147,7 @@ export async function verifyFileAccess(
   customConfig?: StorageConfig,
   context?: StorageContext | 'general',
   isLocal?: boolean,
-  options?: { requireWrite?: boolean }
+  options?: { requireWrite?: boolean; knowledgeAccess?: KnowledgeFileAccess }
 ): Promise<boolean> {
   const requireWrite = options?.requireWrite ?? false
   try {
@@ -182,7 +188,7 @@ export async function verifyFileAccess(
 
     // 4. KB files: kb/filename
     if (inferredContext === 'knowledge-base') {
-      return await verifyKBFileAccess(cloudKey, userId, customConfig)
+      return await verifyKBFileAccess(cloudKey, userId, customConfig, options?.knowledgeAccess)
     }
 
     // 5. Chat files: chat/filename
@@ -485,7 +491,14 @@ async function verifyCopilotFileAccess(
  * signal only: it reflects whether the file is still part of a live KB, not who
  * owns it (ownership comes from the binding).
  */
-async function hasActiveKbDocumentForKey(cloudKey: string, workspaceId: string): Promise<boolean> {
+/** A reader once resolved: a person's or the workspace's tokens, or the system reading its own rows. */
+type ResolvedKnowledgeFileAccess = KnowledgeAccessScope | SystemAccessScope
+
+async function hasActiveKbDocumentForKey(
+  cloudKey: string,
+  workspaceId: string,
+  access: ResolvedKnowledgeFileAccess
+): Promise<boolean> {
   const rows = await db
     .select({ id: document.id })
     .from(document)
@@ -497,14 +510,35 @@ async function hasActiveKbDocumentForKey(cloudKey: string, workspaceId: string):
         eq(document.userExcluded, false),
         isNull(document.archivedAt),
         isNull(document.deletedAt),
-        isNull(knowledgeBase.deletedAt)
+        isNull(knowledgeBase.deletedAt),
+        knowledgeAccessCondition(access)
       )
     )
     .limit(1)
 
   return rows.length > 0
 }
 
+/**
+ * How a KB file read identifies the reader for document access. `'user'` is
+ * for a session-authenticated person; a resolved scope is for a caller that
+ * already holds one (an execution with a principal). The system scope is for
+ * a background job reading a connector-owned row it is processing, which in
+ * members mode is hidden until the sync materializes its readers. Anything
+ * else — an internal token, a tool running with the workflow owner's id —
+ * reads as the workspace, never as the person whose id it happens to carry.
+ */
+export type KnowledgeFileAccess = 'user' | ResolvedKnowledgeFileAccess
+
+async function resolveKnowledgeFileAccess(
+  knowledgeAccess: KnowledgeFileAccess | undefined,
+  userId: string,
+  workspaceId: string
+): Promise<ResolvedKnowledgeFileAccess> {
+  if (knowledgeAccess === 'user') return resolveUserKnowledgeAccessScope(userId, workspaceId)
+  return knowledgeAccess ?? WORKSPACE_ACCESS_SCOPE
+}
+
 /**
  * Verify access to KB files (`kb/<key>`).
  *
@@ -522,7 +556,8 @@ async function hasActiveKbDocumentForKey(cloudKey: string, workspaceId: string):
 async function verifyKBFileAccess(
   cloudKey: string,
   userId: string,
-  customConfig?: StorageConfig
+  customConfig?: StorageConfig,
+  knowledgeAccess?: KnowledgeFileAccess
 ): Promise<boolean> {
   try {
     const binding = await getFileMetadataByKey(cloudKey, 'knowledge-base', {
@@ -552,10 +587,12 @@ async function verifyKBFileAccess(
       return false
     }
 
-    if (!(await hasActiveKbDocumentForKey(cloudKey, binding.workspaceId))) {
-      logger.warn('KB file access denied: no active document references the file', {
+    const access = await resolveKnowledgeFileAccess(knowledgeAccess, userId, binding.workspaceId)
+    if (!(await hasActiveKbDocumentForKey(cloudKey, binding.workspaceId, access))) {
+      logger.warn('KB file access denied: no readable document references the file', {
         userId,
         cloudKey,
+        accessScopeKind: access.kind,
       })
       return false
     }

apps/sim/app/api/files/export/[id]/route.test.ts

@@ -25,7 +25,10 @@ function embeds(...ids: string[]) {
   mockExtractEmbeddedFileRefs.mockReturnValue({ keys: [], ids })
 }
 
-vi.mock('@/lib/auth/hybrid', () => ({ checkSessionOrInternalAuth: mockCheckAuth }))
+vi.mock('@/lib/auth/hybrid', () => ({
+  AuthType: { SESSION: 'session', API_KEY: 'api_key', INTERNAL_JWT: 'internal_jwt' },
+  checkSessionOrInternalAuth: mockCheckAuth,
+}))
 vi.mock('@/lib/uploads/server/metadata', () => ({
   getFileMetadataById: mockGetFileMetadataById,
 }))

apps/sim/app/api/files/export/[id]/route.ts

@@ -7,7 +7,7 @@ import type { NextRequest } from 'next/server'
 import { NextResponse } from 'next/server'
 import { fileExportContract } from '@/lib/api/contracts/storage-transfer'
 import { parseRequest } from '@/lib/api/server'
-import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
+import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
 import { MATERIALIZE_CONCURRENCY, mapWithConcurrency } from '@/lib/core/utils/concurrency'
 import { isPayloadSizeLimitError } from '@/lib/core/utils/stream-limits'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
@@ -81,7 +81,10 @@ export const GET = withRouteHandler(
       return NextResponse.json({ error: 'Not found' }, { status: 404 })
     }
 
-    const hasAccess = await verifyFileAccess(record.key, userId)
+    const knowledgeAccess = authResult.authType === AuthType.SESSION ? 'user' : undefined
+    const hasAccess = await verifyFileAccess(record.key, userId, undefined, undefined, undefined, {
+      knowledgeAccess,
+    })
     if (!hasAccess) {
       logger.warn('Unauthorized file export attempt', { id, userId })
       return NextResponse.json({ error: 'Forbidden' }, { status: 403 })
@@ -164,7 +167,13 @@ export const GET = withRouteHandler(
         try {
           const imgRecord = await getFileMetadataById(storedFileId(imageId))
           if (!imgRecord) return null
-          if (!(await verifyFileAccess(imgRecord.key, userId))) return null
+          if (
+            !(await verifyFileAccess(imgRecord.key, userId, undefined, undefined, undefined, {
+              knowledgeAccess,
+            }))
+          ) {
+            return null
+          }
           return { imageId, record: imgRecord, size: getWorkspaceFileSize(imgRecord) }
         } catch (error) {
           logger.warn('Failed to resolve asset for export', {

apps/sim/app/api/files/serve/[...path]/route.test.ts

@@ -397,7 +397,8 @@ describe('File Serve API Route', () => {
       'test-user-id',
       undefined,
       'mothership',
-      false
+      false,
+      { knowledgeAccess: undefined }
     )
     expect(storageServiceMockFns.mockDownloadFile).toHaveBeenCalledWith({
       key: 'workspace/test-workspace-id/1234567890-photo.png',

apps/sim/app/api/files/serve/[...path]/route.ts

@@ -8,7 +8,7 @@ import {
   concealCrossTenantResourceError,
   InternalUnauthenticatedError,
 } from '@/lib/api/server/routes'
-import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
+import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
 import { resolveServableDocBytes } from '@/lib/copilot/tools/server/files/doc-compile'
 import { DocCompileUserError } from '@/lib/copilot/tools/server/files/doc-compile-error'
 import { asOrchestrationError } from '@/lib/core/orchestration/types'
@@ -26,7 +26,7 @@ import { internalWorkspaceFileServeAuth } from '@/lib/workspace-files/api'
 import { readWorkspaceFileContentByKey } from '@/lib/workspace-files/application/read-workspace-file-content-by-key'
 import { isSimPageSource, SIM_PAGE_CONTENT_TYPE } from '@/lib/workspace-files/page-compile'
 import { renderSimPageDocumentWithAssets } from '@/lib/workspace-files/page-document.server'
-import { verifyFileAccess } from '@/app/api/files/authorization'
+import { type KnowledgeFileAccess, verifyFileAccess } from '@/app/api/files/authorization'
 import {
   createErrorResponse,
   createFileResponse,
@@ -271,12 +271,29 @@ export const GET = withRouteHandler(
 
       const userId = legacyAuthResult?.userId
       if (!userId) throw new Error('Authenticated file serve request is missing a user ID')
+      /** Only a session identifies a person; an internal token's user id reads as the workspace. */
+      const knowledgeAccess =
+        legacyAuthResult?.authType === AuthType.SESSION ? ('user' as const) : undefined
 
       if (isUsingCloudStorage()) {
-        return await handleCloudProxy(cloudKey, userId, options, request.signal, storageContext)
+        return await handleCloudProxy(
+          cloudKey,
+          userId,
+          options,
+          request.signal,
+          storageContext,
+          knowledgeAccess
+        )
       }
 
-      return await handleLocalFile(cloudKey, userId, options, request.signal, storageContext)
+      return await handleLocalFile(
+        cloudKey,
+        userId,
+        options,
+        request.signal,
+        storageContext,
+        knowledgeAccess
+      )
     } catch (error) {
       if (error instanceof InternalUnauthenticatedError) {
         logger.warn('Unauthorized file access attempt', { error: error.message })
@@ -359,7 +376,8 @@ async function handleLocalFile(
   userId: string,
   options: ServeOptions,
   signal: AbortSignal | undefined,
-  context: StorageContext
+  context: StorageContext,
+  knowledgeAccess: KnowledgeFileAccess | undefined
 ): Promise<NextResponse> {
   const ownerKey = `user:${userId}`
   try {
@@ -368,7 +386,8 @@ async function handleLocalFile(
       userId,
       undefined, // customConfig
       context,
-      true // isLocal
+      true, // isLocal
+      { knowledgeAccess }
     )
 
     if (!hasAccess) {
@@ -419,7 +438,8 @@ async function handleCloudProxy(
   userId: string,
   options: ServeOptions,
   signal: AbortSignal | undefined,
-  context: StorageContext
+  context: StorageContext,
+  knowledgeAccess: KnowledgeFileAccess | undefined
 ): Promise<NextResponse> {
   const ownerKey = `user:${userId}`
   try {
@@ -430,7 +450,8 @@ async function handleCloudProxy(
       userId,
       undefined, // customConfig
       context, // context
-      false // isLocal
+      false, // isLocal
+      { knowledgeAccess }
     )
 
     if (!hasAccess) {

apps/sim/app/api/files/view/[id]/route.ts

@@ -3,7 +3,7 @@ import type { NextRequest } from 'next/server'
 import { NextResponse } from 'next/server'
 import { fileViewContract } from '@/lib/api/contracts/storage-transfer'
 import { parseRequest } from '@/lib/api/server'
-import { checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
+import { AuthType, checkSessionOrInternalAuth } from '@/lib/auth/hybrid'
 import { withRouteHandler } from '@/lib/core/utils/with-route-handler'
 import { getServeStoragePrefix, type StorageContext } from '@/lib/uploads/config'
 import { getFileMetadataById } from '@/lib/uploads/server/metadata'
@@ -37,7 +37,9 @@ export const GET = withRouteHandler(
       record.key,
       authResult.userId,
       undefined,
-      record.context as StorageContext | 'general'
+      record.context as StorageContext | 'general',
+      undefined,
+      { knowledgeAccess: authResult.authType === AuthType.SESSION ? 'user' : undefined }
     )
     if (!hasAccess) {
       logger.warn('Unauthorized file view attempt', { id, userId: authResult.userId })

apps/sim/app/api/knowledge/[id]/connectors/[connectorId]/access/route.ts

@@ -0,0 +1,39 @@
+import { updateKnowledgeConnectorAccessContract } from '@/lib/api/contracts/knowledge'
+import {
+  defineInternalJsonRoute,
+  internalRateLimits,
+  internalSessionAuth,
+} from '@/lib/api/server/routes'
+import {
+  resolveInternalKnowledgeBillingAttribution,
+  toInternalKnowledgeConnector,
+} from '@/lib/knowledge/api/internal-route'
+import { internalKnowledgeErrorPolicies } from '@/lib/knowledge/api/route-policies'
+import { updateKnowledgeConnectorAccess } from '@/lib/knowledge/application/connector-access'
+import { knowledgeOperations } from '@/lib/knowledge/application/operations'
+
+export const PATCH = defineInternalJsonRoute({
+  contract: updateKnowledgeConnectorAccessContract,
+  auth: internalSessionAuth,
+  operation: knowledgeOperations.updateConnectorAccess,
+  rateLimit: internalRateLimits.none({
+    reason: 'A settings action an admin performs by hand; the switch itself is bounded',
+  }),
+  errorPolicy: internalKnowledgeErrorPolicies.connectors,
+  mapInput: ({ params, body }, { principal, request }) => ({
+    connectorId: params.connectorId,
+    knowledgeBaseId: params.id,
+    accessMode: body.accessMode,
+    credentialGroupId: body.credentialGroupId,
+    credentialGroupOptionId: body.credentialGroupOptionId,
+    credentialId: body.credentialId,
+    resolveBillingAttribution: (workspaceId: string) =>
+      resolveInternalKnowledgeBillingAttribution(request, principal, workspaceId),
+    source: 'ui' as const,
+  }),
+  useCase: updateKnowledgeConnectorAccess,
+  present: ({ connector }) => ({
+    success: true as const,
+    data: toInternalKnowledgeConnector(connector),
+  }),
+})