Commit d9cfd7c
improvement(mothership): v0.9 (#6815)
* checkpoint
* Checkpoint
* dot fixes
* Make async tool resume delivery recoverable
* Support split table tools and option recovery
* Harden VFS mutation handling
* feat(platform): platform subagent support — docs corpus VFS, search_docs, account context
Squash of the feat/platform-agent branch (sim side): mounts the Sim docs
corpus in the copilot VFS, wires search_docs and retires the legacy docs
search tools, and syncs the generated tool catalog and trace contracts for
the platform subagent.
* Align Copilot tools and resource handling
* Expand workflow log query support
* checkpoint
* Port desktop-improvements-0 desktop and browser-agent work
* fix(desktop): keep browser-agent input alive through live-SPA re-renders
* Harden workflow sanitization and Slack setup
* feat(desktop): coordinate clicks, caret insertion, and drag for the browser agent
* Revert subagent group eager auto-collapse
* fix(chat): keep sends FIFO across the streaming-to-idle drain gap
* Add the steering backend surface for mid-turn sends
* Sync generated contracts for async subagent orchestration
Pulls the mothership tool catalog (wait_agents / tail_agent / steer_agent /
interrupt_agent), trace spans (chat.async_subagent.*, chat.orchestrate.*), and
trace attributes (copilot.async_subagent.*) into the generated TS contracts.
* Add display titles for the async subagent orchestration tools
wait_agents / tail_agent / steer_agent / interrupt_agent get natural-language
running titles (naming the agent id being waited on, tailed, steered, or
stopped) and a Steering→Steered completed-verb rewrite.
* Show orchestrator-chosen subagent names on agent groups
A subagent_start whose payload data carries a name (the orchestrator's new
name trigger parameter) now labels the agent group with that mission name —
the agent-type icon stays. The name flows through the live stream path, the
turn model (AgentNode.displayName) and its serialize/rebuild round-trip, and
persisted transcripts (PersistedContentBlock.name), so reloads keep the label.
* Improve Copilot error handling and logging
* Backfill the subagent display name from the second start event
The dispatch-time subagent_start fires before the trigger args (and therefore
the name parameter) have streamed; the phase-3 start re-announces the lane with
the name. The block builder was dropping that duplicate wholesale, losing the
name on streaming providers — now it backfills subagentName onto the existing
block instead. (The home turn-model path already reconciled this case.)
* Support Slack bot connection flow
* Harden Copilot error and VFS handling
* Harden VFS resource operations
* Show 'Waiting for the first of N agents' for mode-any waits
The wait_agents title ignored the mode argument, so an any-mode wait over
three agents read 'Waiting for 3 agents' while the model narrated waiting for
the first — contradicting the transcript.
* Collapsed-by-default agent cards with live intent status lines
Subagents now narrate their work through <intent>3-5 words</intent> tags (a
fleet-wide prompt protocol on the mothership side). The turn model streams
each subagent's text through a split-safe tag parser: complete tags update the
agent's currentIntent and disappear from the prose, tags split across deltas
are carried until their close arrives, and a tag that never closes flushes
back as plain text.
The agent card renders as one line — display name (or agent label) plus the
latest intent, replaced inline as the agent shifts gears — and never
auto-expands; expanding to the full tool log is a deliberate click. Only an
outstanding permission prompt or a browser hand-back forces a group open.
Intents persist on the subagent block (and through the legacy persisted-
message paths) so reloads keep the last status, and a renamed reinvocation
now takes the latest name instead of pinning the first.
* Add the internal in-band tool execution route for live mothership turns
POST /api/copilot/tools/execute (INTERNAL_API_SECRET, Go→Sim) runs one
sim-server tool through the same server tool router the resume driver uses
and returns the result synchronously — no checkpoint. This is what lets
background (async) subagents write files/tables/knowledge, and lets the main
lane keep streaming (instead of checkpoint-pausing and killing every
background run) while async agents are live.
* Persist resource side effects for in-band tool execution
Files/tables created through the internal execute route now register on the
chat's resources exactly like the resume driver's executions — the route runs
the same handleResourceSideEffects pass (persistence only; an out-of-band
route has no live event sink, so mid-turn chip pushes are a follow-up).
* Extract intents from group text on every path, sync and async
The turn-model intent filter only fires for span-scoped subagent lanes, but
this surface also delivers subagent text through the legacy block path — so
<intent> tags flowed through unparsed and rendered as prose rows. Groups now
extract intents from their accumulated text at append time: the last complete
tag becomes the card's status line and every complete tag is stripped from
the rendered prose. Covers span-scoped, legacy, and persisted-reload paths
for both synchronous and background delegations.
* Fall back to the live tool title for the agent card status line
Persisted data proved tool-first subagents (grok search agents) emit zero
prose, so intent tags never stream no matter what the prompt says. The
collapsed card now always narrates: the agent's own <intent> tag when present,
else the latest tool's display title while the lane is live.
* Catch subagent <intent> tags in the server relay
The relay's subagent text handler now runs the split-safe intent extraction
as chunks stream: the latest complete tag is stamped onto the lane's persisted
subagent block (subagentIntent) and stripped from the stored prose, so live,
persisted, and replayed views all agree. Per-lane carry handles tags split
across chunks; a never-closing tag flushes back as plain text.
* Drop the tool-title fallback: the status line is the agent's intent
With the intent protocol now injected into every spawn's task message, agents
open with an <intent> tag; the card shows that narration or nothing.
* Replace intents with live tool-title status lines on agent cards
Intent parsing is fully removed (turn model, relay handler, persistence
fields, group extraction). The collapsed card's status is the latest tool
call in its RUNNING phrasing — never the completed rewrite, which stays in
the expanded log. Parallel tools show the most recently started still-running
title with a +N for concurrent siblings; between rounds the last title stays
frozen; a closed lane shows the bare name. Nested agent cards compute their
own status recursively from their own items.
* Keep the main Sim lane live-expanded; collapse only real subagent cards
The mothership group is the turn's own narration, not a delegation card —
collapsing it hid main-lane text and tools until manual expand, which read as
mis-ordered streaming while async subagents interleaved. It keeps the
original live-expand behavior and no status suffix.
* Persist subagent lane lifecycle blocks from the span handler
Lane-scoped span events route to the span handler, which only recorded trace
side effects — no subagent start block was ever persisted (verified: a
seven-agent run stored 104 blocks with zero starts). Grouping then fell back
to keying lane content by agent NAME, so a respawned agent of the same type
merged invisibly into the first one's card until it resolved. The handler now
persists the start block (spanId-keyed and deduped, carrying the display
name) and stamps endedAt on close, giving every invocation its own card.
* Name agents in orchestration titles; '+ n more' overflow format
wait/tail/steer/interrupt titles humanize the slugified agent ids back to
their display names ('Waiting for the first of Digest Workflow Build + 4
more'), and the agent card's parallel-tool suffix uses the same '+ n more'
format.
* Harden in-band tool execution and resources
* Route in-band execution through the comprehensive tool dispatcher
The internal execute route used the bare server-tool router, which rejects
VFS tools with 'Unknown server tool: read/glob/grep' — so nearly every
background agent's first discovery call failed (102 in-band calls in one run,
dozens rejected). It now uses the relay's executeTool dispatcher: registered
handlers (VFS, function execute) with permission checks and param
normalization, falling back to the app tool router — the same surface
foreground execution gets.
* Harden chat stream transition handling
* Harden VFS provenance and resource writes
* Standardize tool environment references
* Harden browser panel and chat cleanup
* Descriptive, user-language tool titles across the board
House rules applied everywhere: use every argument the call carries, never
name internal machinery, and never lead with Getting (the Got rewrite is
deleted so it cannot return).
- Deployments name the workflow: Deploying {workflow} as API/chat app/MCP tool
- Workflow reads name the part: Reading {workflow} meta/state/deployment/notes;
generic reads always name the file (Reading {leaf}), never bare Reading file
- Block runs name block and workflow: Running {block} in {workflow}, Running
from {block} in {workflow}, Running {workflow} until {block}, and
Enabling/Disabling {block} in {workflow}
- The six split-table tools get per-operation verbs (Adding column {name},
Updating rows, Wiring automation, Creating view {name}) instead of a wall
of Querying table
- The manage quartet drops X-action system-speak for gerunds
- get_* internal names become user language (Checking run settings, Tracing
block inputs, Reading the deployed version); web_fetch says Fetching
- Scheduled-task titles removed entirely (feature deleted from the Go catalog)
- New verb rewrites: Fetched, Traced, Wired, Configured, Looked, Rotated
* Deploying {workflow} as chat, not as chat app
* Loader gerunds; mv names both ends; mkdir names the folder
search_integration_tools -> Finding the right integration;
load_integration_tool -> Loading {integration} tools; load_skill ->
Loading skill {name}; run_enrichment -> Looking up {subject}. mv prefers the
model's phrasing, else reads 'Moving {files} to {destination}'; mkdir reads
'Creating folder {name}' from the path.
* Overflow counts read '+ n', dropping 'more'
* Unify workspace find and search
* Scale desktop title bar with page zoom
* Serialize account and organization truth into the copilot VFS
Workspace standing, membership, billing, org role, access-control
restrictions, published-block provenance, and fork topology were reachable
only through three parameterless tools (or not at all). They are ambient
read-only facts, so they belong in the VFS where they are greppable, cost no
tool round-trip, and every agent that can read gets them — the same move that
retired get_blocks_and_tools and list_user_workflows.
Adds account/{workspace,workspaces,members,billing}.json (always mounted) and
organization/{organization,access-control,custom-blocks,forks}.json (only when
the workspace is org-hosted). Every file projects an existing use case or util
after getOrMaterializeVFS's access assert — no new queries, no new
authorization. One relation per file, cross-referenced by id-and-name stub, so
overlapping facts cannot disagree. Volatile content (billing, access control,
forks) is lazy, so numbers are read-time fresh and unasked-for reads cost
nothing.
Projection follows the viewer: member emails are admin-only, fork detail
requires workspace admin on a forking-enabled org, and the whole organization/
namespace is absent for a personal workspace — which is itself the answer.
Retires get_account_billing, get_enterprise_context, and list_user_workspaces
along with their handlers; display titles stay for transcript replay.
* Fix insert_text refusing an editable field focused inside a frame
describeFocusedEditable descended shadow roots but not frames, while
activeElementReadback descends both. Focus inside a same-origin frame therefore
surfaced to the first as the FRAME element — not an input, not contentEditable,
not a canvas, no textbox role — so it fell through to 'not-editable' and
insert_text refused a field that press_key had just typed a character into.
Two functions answering 'what is focused' with different answers is the bug;
the descent loops now match exactly.
The refusal also names what actually held focus (tag, role, contenteditable).
A bare 'not-editable' gave the agent nothing to act on, so it guessed at the
cause — a real run spent twenty rounds on the wrong theory and had to be
stopped by the user.
* Keep retired browser takeover renderable in history
The tool is gone from the catalog, so its generated constant went with it and
every path that referenced it stopped compiling. Deleting those paths instead
would have silently downgraded every past transcript containing a takeover card
to a generic tool row, and dropped the no-timeout budget that an in-flight
takeover still needs while a rolling deploy finishes.
retired-tools.ts gives the literal a documented home that says what it is and
why it survives its tool.
* Follow the agent into a tab it opened to work in
browser_open_tab created the page with activate: false, so the agent worked in
a tab the user could not see while the panel sat on a page where nothing was
happening. The panel now follows a tab the agent deliberately opened.
Scoped to that tool only. A page spawning its own tab (popup, target=_blank) is
the site grabbing the view rather than the agent choosing a workspace, and
stays in the background as before — two existing tests pin that and caught the
first version of this change, which moved both.
A tab the user claimed still wins over both: the work starts in the background
instead of pulling the page out from under them mid-read.
* Make the browser tools agree with each other
An audit of the module found the frame-descent bug was one instance of a
pattern: six independent definitions of 'is this editable' and seven of 'what
is focused', disagreeing with each other. A tool refusing what its sibling
accepts on identical page state is invisible at runtime — the agent follows a
snapshot that says one thing into a tool that says another.
- browser_type now accepts role="textbox" like browser_insert_text does. The
snapshot advertises those elements as [textbox] with a ref, so refusing them
meant rejecting exactly what the outline told the model to type into. Both
the native and synthetic paths, and their descendant scans.
- pressKeyOnPage descends shadow roots and frames like every other focus
reader. It was dispatching synthetic keys at the shadow host or <iframe>
element, where they bubble but never reach the editor, while reporting
success — and contradicting the activeElement reported beside it.
- not-editable and ambiguous-editable name what was found: the element's tag
and role, and the candidate fields. Both had the data and discarded it, which
is what turns one blocked step into twenty rounds of guessing.
- obstructedAfterNavigation requires a dialog that ARRIVED with the
navigation. It compared against nothing, so every SPA route change under a
persistent role=dialog reported a successful click as obstructed. The test
that covered this asserted the false positive; it now pins both directions.
- browser_insert_text observes the top document when typing inside a frame,
like every other input tool. A submit that navigates the top page was
invisible to its frame-scoped observation.
* Let hover actually see what it mounted
Four independent defects made browser_hover blind to the most common thing a
hover produces — a row's action bar — so it reported no effect on a hover that
worked, and the agent fell back to clicking pixels off screenshots.
- The popup scan matched only role=tooltip/menu/listbox. Slack's message
shortcuts bar is a labelled toolbar/group, so it registered as nothing at
all. Added toolbar, menubar, labelled group, and [popover].
- The baseline was captured BEFORE prepareElementSurface scrolled the target
into view, so scrollChanged was always set by the tool's own probe. That
pinned every unproductive hover to 'background DOM churn' instead of the
honest 'nothing happened', and hid scrolling the hover really caused.
Re-baselined once the scroll settles and before the pointer moves.
- The MutationObserver attached only on the first observation, while the roots
list is rebuilt every call and grows as shadow roots mount. Components that
appeared later were never observed, so their DOM changes raised no revision.
Roots are now observed as they show up.
- observationTruncated was computed and never read, so a scan capped at 12k
nodes reported 'nothing appeared' with the same confidence as a complete
one — and portalled overlays live at the end of <body>, exactly what the cap
drops. Hover now says the page was too large to scan and to confirm visually.
* Stop the browser agent acting on the wrong element, and say why it refused
Four findings from the module audit, the first of which could silently do the
wrong thing rather than merely fail.
- A ref whose node is gone is re-adopted by structural resemblance, matching on
ORIGIN only so a pushState between snapshot and act does not kill every ref.
That leniency also let a ref to a row control in one view rebind to the
identical control in a view the app had since navigated to — acting on the
wrong message, signalled by nothing louder than recovered: true. Adoption now
requires the same path; a view swap reports the ref stale, and the caller
re-snapshots. Revalidating a still-connected node stays lenient, because that
is literally the node the model chose.
- A hit INSIDE the requested element is its own nested control, not an overlay.
Both produced 'covered by X — close or move the overlay', advice that cannot
be followed because there is nothing to close. Nested hits now say so and
point at retargeting.
- browser_click_at, browser_insert_text, and browser_drag listed targetChanged
in their effect formulas, but none passes an elementId, so no targetState is
ever captured and the term was always false — coverage that read as real.
Removed, with a test pinning the dependency.
- The seven effect formulas are deliberately NOT collapsed into one predicate:
drag must trust domChanged where others must not, hover must ignore field and
focus changes, click counts focus only for editables. Forcing one would make
each tool wrong differently. The differences are now documented in one place
next to the shared computation, so divergence is a declared policy rather
than an accident.
* Let edit_workflow configure block retries
* Updates
* Always focus the resource the agent is working on, and its browser tab
The resource panel had a carve-out: an already-open browser session declined
to replace another selection and only got an attention marker, so agent
browser work happened off-screen. The panel now follows the agent to whatever
it touches — browser included — and an event can still opt out explicitly.
The browser panel also follows the agent BETWEEN tabs: the store already
tracked automationTabId (and the strip marked it), but the visible tab never
changed. It now switches when the agent's target tab changes, so watching the
agent never means hunting for the tab it moved to. Keyed on the target
changing rather than on it being set, so a user who browses elsewhere
mid-run is only pulled along when the agent itself moves.
* Never paint a browser snapshot at stale geometry (the modal-open flash)
Opening a modal locks scroll, which removes the window scrollbar and reflows
the panel — so a capture taken before the lock describes a rect the panel no
longer occupies. The handshake painted that frame anyway and only then
retried, so the replacement landed visibly offset from the page it stands in
for: the flash. A capture is now checked against the host's live rect before
it is painted; a mismatched frame is skipped and re-captured at the settled
layout instead (modal retries go 2 -> 3 to absorb the extra settle).
* Name the workflow in deployment and workflow-scoped tool titles
'Checked deployment status' never said which workflow — nor did the deployed-
state read, run settings, block outputs/inputs, redeploy, promote, or the
global-variable write. These tools carry a workflowId (often defaulting to the
current workflow), so only the client can resolve a name: the enrichment layer
now resolves it for the whole workflow-scoped family and passes it as
workflowName, which every workflow title already reads.
Titles: Checking {workflow} deployment status, Reading deployed {workflow},
Checking {workflow} run settings, Reading {workflow} block outputs, Tracing
{workflow} block inputs, Redeploying {workflow}, Promoting {workflow} version
{n} to live, and 'Adding workflow variable {name} in {workflow}' — each
falling back to its unnamed form when no workflow resolves.
* Name the block that ran; never fall back to a raw block id
run_block and set_block_enabled carry only a blockId, so their titles would
have printed an opaque UUID ('Running 7f3a2b91-… in Invoice Sync'). The
enrichment layer now resolves blockId against the workflow store the same way
it already did for run_from_block's startBlockId, and the base titles no
longer accept an id as a name — an unresolved block reads 'Running block'
rather than a UUID.
* Add the missing Removing -> Removed rewrite
The table work introduced 'Removing automation'/'Removing enrichment' with no
past form, so those rows kept their present tense after completing.
* Name the target resource in the remaining tool titles
Table tools keep their operands nested under args and identify the table by
id, so their rows said 'Adding rows' with no hint where: enrichment now lifts
the nested args and resolves tableId against the cached workspace table list,
giving 'Adding rows to Runtimes', 'Adding column status in Runtimes',
'Reading views of Runtimes'.
Also: a block-schema read names the block instead of the file ('Loading
Slack', 'Loading Google Sheets tips'); browser type/insert show the text they
send, middle-ellipsized; downloads name the file; library-docs searches name
the library and query; knowledge-base searches include the query; generated
media names its output file; and diff_workflows, list_deployment_versions,
and publish_custom_block joined the workflow-name enrichment set.
* Bubble nested agents' tool calls into the parent's status line
The collapsed status only scanned a group's OWN tool items and skipped nested
agent groups, so a parent that had delegated froze on its last own tool while
its child did the actual work — the line described nothing that was running.
Status now walks the whole subtree: any tool at any depth counts, the most
recently started running one is shown, and the rest become the same '+ n'
overflow. With nothing running it falls back to the last tool at any depth,
so an idle parent still reflects where its subtree got to.
* Align nested tool call status rows
* Revert branch-local KB connector error-message edits
Restores apps/sim/connectors/ to staging state. Two copilot-focused commits
on this branch (3a9fc1c5a5, 24b24e5f8e) drove by nine KB source connectors
(airtable, confluence, discord, github, gitlab, google-drive,
microsoft-teams, notion, slack) to enrich credential-validation error
messages. The env-reference resolution machinery those errors supported is
kept; the product connector surface stays unchanged in this branch so the
staging promotion remains scoped to copilot work.
* Fix the failing audits: NUL escape and route-count ratchet
check:source-text: resource-vfs.ts used a raw NUL byte as its folder-index
key separator (comment and template literal), which makes git treat the file
as binary and hide it from review. Written as the '\u0000' escape — the
runtime string is identical.
check:api-validation:strict: baseline 1120 -> 1122 for the two routes added
on this branch since the last bump.
* Add Force Reload (Cmd+Shift+R) to desktop View menu
* Route Force Reload through focused-resource boundary; regen docs manifest
* Align the title-bar surface audit with the zoom rework
'Scale desktop title bar with page zoom' (833d2f126d) changed the CSS contract
in two ways its audit test still pinned the old shape of: the lane vars gained
a max() floor around the platform env() terms so page zoom cannot shrink the
lane below the OS-drawn lights, and the control square became fixed px — CSS px
already scale under zoom — leaving only the centering offset derived from the
lane height. The test required the bare env() prefix and calc() on all three
control vars, so it failed the commit that implemented its own regression
comment.
Pins now assert the env() term inside the clamp (still platform-derived, the
test's actual intent) and split the control vars: offset must stay computed,
size and icon are explicit constants.
* Budget the two structurally slow tests explicitly
sso-trust imports the whole Better Auth module graph (2.5s on an idle
machine) and events.attribution scans call sites across the repo. Under a
fully-parallel uncached run on a loaded machine both blow the default
timeout while passing in isolation and on CI — a verdict decided by machine
load, not by the code. 30s budgets make a loaded local run mean what it says.
* Carry the cross-service trace id in sim log lines
* Improve nested tool status presentation
* Refresh secret schemas and shared test mocks
* Open the deployed graph of org-published blocks, read-only, org-wide
A consuming workspace could see a published block's interface but never what
it does: the backing workflow lives in the publishing workspace, and other
workspaces are nameable, not readable. Publishing a block org-wide is the act
of sharing it, so the graph it executes is now readable from any org
workspace — the DEPLOYED graph, not the publishing workspace's live editor
state, so nothing in-progress leaks and what you read is what runs.
The namespace also adopts the root's index/detail split: custom-blocks.json
slims to names with a detail pointer, organization/custom-blocks/{type}.json
carries provenance plus the deployed graph (loaded lazily through the cached
loadDeployedWorkflowState; credential ids and env references inside it belong
to the publishing workspace and say so), and organization/README.md is the
namespace guide WORKSPACE.md is at the root — files, usage, the in-depth
block inventory, and forks.json documented only when actually mounted.
The block list moves to materialize time (same indexed query the components
pass already runs) because the README, the index, and the per-block key-view
entries all need it; only the graph stays lazy.
* Withhold the deployed graph from external collaborators
Workspace access and org membership are different grants: an external
collaborator can open the workspace and use the published block, so the names
index and the interface schema stay visible to them — but the deployed graph
is org implementation internals, and their detail files now simply do not
exist. isHostOrganizationMember is the viewer bit the host context already
resolves for exactly this distinction.
* Fill out the organization namespace: workspaces, permission groups, credential groups
Three more read-only files, all lazily loaded — the paths appear in the key
view so glob discovers them, but no query runs until a read — and all gated by
registration, so an unpermitted viewer's file simply does not exist:
- workspaces.json (org members): the org's full workspace map with the
viewer's access flag and fork parentage — account/workspaces.json only ever
showed what the viewer can reach. Inaccessible workspaces stay nameable,
not readable.
- permission-groups.json (org admins): every group with member count,
targeted workspaces, and the restrictions its config activates.
access-control.json remains the per-viewer binding. The queries are lifted
into lib/permission-groups/queries.ts because their only prior home was
inline drizzle in the route handlers, which the VFS cannot import.
- credential-groups.json (entitlement-gated): per-option configuration
readiness and enrollment progress — the two facts that decide whether a
credential_group workflow will do anything at runtime. The note teaches the
contract that bit the audit: an active group with zero completed
enrollments yields an empty loop, not an error. Enrollee emails are
workspace-admin-only, matching the settings page; counts come from the
first enrollment page and say so when truncated.
The README documents each file only when mounted for this viewer.
* Stop reporting a click that navigated as a failed click
The field case: 'Begin Assessment' submits a form. The navigation tears the
origin document down while the press completes, so everything after dispatch —
the CDP call's own completion, the synthetic dispatch's return value, the
postcondition reads — fails against a destroyed context, and a maximally
successful click was reported 'Failed clicking element'. The agent's own
follow-up investigation in the transcript diagnosed exactly this.
navigationRescue detects it at the driver level, where the navigation epoch
and URL survive the renderer teardown: when the page provably navigated since
dispatch began, a dispatch-path failure becomes a success carrying
navigatedDuringDispatch and a note explaining why no postconditions exist.
Applied to all four click dispatch paths (unframed CDP, framed native,
synthetic in-page, click_at).
The soft path had the same blindness: with the after-state unreadable,
urlChanged computed false and a navigating click reported 'no observable
change'. navigatedByDriver now folds into navigated/effectObserved, which also
keeps the new-dialog obstruction check meaningful on real navigations.
* Re-hide the browser under a modal after main loses the occlusion lease
The punch-through: a modal opens, the native view hides behind a painted
snapshot, and the renderer records applied: true. Then one heartbeat commit is
skipped — renderer jank past the 2.5s bounds-lease TTL is enough — and main
expires the lease, resetting panelOccluded on its side. The next heartbeat
finds the modal marker still present and calls setDesired(true), but the
lease's dedupe sees applied === desired and sends nothing; the bounds commit
that follows lays out an unoccluded native view above the open modal, and no
later event ever re-hides it. The comment on this branch already claimed it
'reasserts the lease' — the dedupe made that claim false exactly when the
lease had been lost.
While the occlusion marker is present, each heartbeat now drops the applied
belief (assumeRevealed) before setDesired, so the reassert is a real, forced,
idempotent hide IPC — one per second while a modal covers the browser — and
any main-side lease loss self-heals within a heartbeat.
* Make a stale ref name which of its five causes fired
A field run burned five snapshot->click cycles on a STATIC landing page, every
one refused with the same sentence — 'the page changed since the last
snapshot' — and the agent reasonably concluded the page was regenerating its
DOM. It was not; the resolver was refusing, and the message could not say why.
Five distinct conditions produced that one string: an id missing from the
registry, a connected node whose identity drifted, the view-changed adoption
gate, no confident replacement, and a replacement tie.
The resolver now stamps the reason (with the drifted node's current identity,
or the from->to paths for the view gate) and every stale producer carries it
into the driver message. Same pattern as not-editable: a refusal that names
its cause costs one round; an opaque one costs a loop and a wrong theory in
the bug report.
* Pulse throttling off when the browser view is revealed, so it actually paints
The blank-page report: a navigation completes while the view is hidden, the
page 'finishes loading', and the panel shows white until the user re-navigates
by hand. invalidate() on reveal was already there but recomposites the LAST
frame — and the last frame is blank, because background throttling suspended
the rAF the page's SPA paints its first frame from. The reveal now pulses
throttling off (forcing the renderer to produce a real frame), invalidates,
and hands the policy back to the session a second later through
reassertTabThrottling, which preserves the automation-tab exemption.
* Let the agent browser join meetings and finish passkey sign-ins
Camera and microphone: 'media' joins the agent partition's allowlist, but
every grant is gated on the macOS grant first — asked via
systemPreferences.askForMediaAccess so the system prompt appears on first use,
and answered from getMediaAccessStatus on checks — so System Settings stays
the real authority and a page can never hold a grant the OS refused. Granting
site permission without the OS grant produced the misleading NotReadableError
Google Meet showed. Packaging gains the camera entitlement and usage string
(macOS kills the process on prompt without one) and the mic string now covers
meetings.
Passkeys: WebAuthn itself is Chromium-native and nothing in our handlers
blocks it — USB security keys need no permission at all. The hybrid transport
(passkey on a nearby phone via QR) rides Bluetooth, which signed builds
silently lacked: the bluetooth entitlement and usage string enable it.
iCloud-Keychain platform passkeys remain outside what an entitlement here can
grant — Apple restricts that to approved browsers.
* Compile and preview Sim-styled pages
* Wait for the batched prepare intent instead of instantly failing apply_file_edit
The model batches prepare_file_edit and apply_file_edit into one round and
the Go loop runs same-round tools concurrently, so apply could reach the
executor before its prepare staged the intent. The instant no-intent error
cost a model retry round and flashed 'Failed creating …' on the shared file
row before the retry succeeded. The apply handler now polls briefly (10s
cap) for the intent; a truly missing prepare still errors at the deadline.
* Store page source, render docs-styled documents on view
The pdf model for agent pages: the .html file keeps the markdown-shaped
source (frontmatter + prose + sim: fences) and every surface renders the
docs-styled document on demand — preview panel, /api/files/serve, public
shares, and downloads all call the same pure compiler, now shared in
lib/workspace-files. The docs chrome is reproduced from the real fumadocs
source: the left sidebar's exact pill metrics, the clerk TOC with its
animated scroll indicator, divider-style tables; cards and stats left the
vocabulary. Table cells and kv values render inline markdown, and
sim:workflow/table/knowledge/file links resolve to real workspace routes,
bridged out of the sandboxed preview to the app router. Hand-written
imitations of rendered output are rejected at apply_file_edit with a steer
back to source, and a streaming page hides its source behind the live
rendered preview (batched ~2s) the way a generating pdf hides its script.
* Stagger the page rails so the resource panel keeps the docs sidebar
Rails were gated at 1100px of iframe width — the chat resource panel never
reaches that, so pages rendered single-column there. The rails now stagger
the way the docs do on a laptop: >=640px keeps the section sidebar (240px)
beside the content, >=1060px restores the full three-column frame with the
clerk TOC, and only a truly narrow pane collapses to one column.
* Fix in-page anchors escaping the preview; add the docs' toggle, code frames, pagination, and images
Clicking a TOC or section link (or pressing Enter in the section filter,
which clicks one) navigated the sandboxed frame off about:srcdoc in
Electron and landed on a cookie-less sign-in page — the shell now
intercepts every '#' anchor and scrolls directly. The page chrome gains
the docs' exact theme toggle (emcn Sun/Moon, 30px rounded-lg, top right),
framed code blocks with language label and copy button, and footer
previous/next cards from prev/next frontmatter. Workspace images
() compile to /api/files/view and the preview host
inlines them as blob: URLs so the cookie-less frame can render them;
sim:accordion joins the vocabulary as the faq component with title keys.
* Size the section sidebar to its content so it appears at panel widths
The left rail was a fixed 240/300px column, so it only earned its place
once the pane was wide. fit-content caps at the docs width but shrinks to
the longest section title (150px floor for the pills), and the two-column
tier now starts at 560px instead of 640.
* Let the clerk TOC join at 860px by sizing it to its content
Same move as the section sidebar: the TOC column fits its longest link
(capped at the docs' 268px, 150px floor), so the full three-column frame
starts at 860px of pane width instead of 1060.
* Open external links from pages in a new tab
The preview bootstrap cancelled every non-anchor click, so an external
link (the Sim docs, a vendor page) did nothing. External http(s) links now
compile with target=_blank rel=noopener for the standalone and share
surfaces, and the sandboxed preview bridges the click to the host, which
window.opens a new tab — same channel the workspace deep links use.
* Lock Sim pages to the rendered view via an internal record type
The record's contentType is stamped text/x-sim-page when apply_file_edit
detects page source — the file stays .html to the user (serving and
downloads still emit text/html), but every surface now knows what the file
holds before content loads. The viewer forces the rendered view for these
files at every moment: the first streamed chunk (whose frontmatter is
still partial) no longer flashes raw source, the gaps between an agent's
tool calls no longer flip back to raw HTML, and both toggle surfaces (the
Files toolbar and the resource-panel tabs) stop offering a code view for
them. Mid-stream compiles run lenient — a fence still being written is
malformed by definition, so its skip-notice callout is suppressed until
the stream settles.
* Honor the model's declared page type; default copilot .html to a page
An explicit contentType on create_empty_file always wins (the skill now
declares text/x-sim-page for pages, text/html for bespoke raw pages);
with no declaration a copilot-created .html defaults to the page type.
The first apply_file_edit still re-confirms from the actual content, and
the category map knows the internal mime explicitly instead of falling
through to the extension.
* Default undeclared .html back to plain text/html
A file is a Sim page only when the model declares it at creation or the
first written content proves it — never by extension alone.
* Match the docs' PageFooter for page navigation
The invented bordered cards with Previous/Next labels are replaced by the
docs' actual footer: the destination name with a 14px emcn chevron on a
flex-1 hover pill (rounded-lg, px-3 py-3, --surface-active), next
right-justified, and a spacer holding the empty half — verified against
apps/docs/components/docs-layout/page-footer.tsx.
* Scroll the rails invisibly, like the docs
The sticky TOC box is overflow-y auto, and the clerk track's absolutely
positioned SVGs could tip it a few pixels into overflow — Chromium then
painted a full scrollbar beside the rail. Rails now hide their scrollbar
chrome entirely (scrollbar-width none + webkit display none), matching
how the docs scroll their sidebar and TOC.
* Send sim:file links to the Files page, like a markdown link
A workspace-file link in a page now navigates exactly as one tagged in a
.md does: an in-app SPA push to /workspace/{ws}/files/{id} (the Files
page with the file open). The fullscreen /view route stays reserved for
the standalone surface; image refs keep the /api/files/view byte route.
* Inline workspace images after the page compiles, not before
The blob substitution ran on the raw source, where the compiled
/api/files/view src it looks for does not exist yet — so the sandboxed
cookie-less frame fetched every image itself and got 401s (broken image
icons). The substitution now runs on the built document, covering
compiled pages, legacy stored-compiled pages, and bespoke HTML alike.
* Highlight the section you are AT in the left rail, not the last one visible
The rail's current-section pick walked every heading visible in the
viewport and kept the last h2 — so clicking a section landed correctly
but highlighted whichever later section peeked in from below. Current is
now the last h2 at or above the top reading line (matching the 72px
scroll-padding a clicked anchor settles at), falling back to the first
visible section when everything is below the line.
* Stop the TOC jittering sideways as the highlight moves
Active TOC links step from weight 430 to 470, and the rail is a
fit-content column — every active-section change re-measured the longest
link and shifted the rail a pixel or two side to side. Each link now
carries a hidden zero-height ghost of itself at the active weight, so
the column always occupies its bold width and the highlight moves
without the layout moving.
* Absolutize links and images in served page documents
A downloaded page must behave like a downloaded .md whose links are
absolute: clicking a workflow reference opens Sim in the browser at that
workflow. The standalone renderer (plain download, fullscreen viewer,
shares) now compiles sim: links and workspace image refs against
getBaseUrl(); in-app surfaces keep relative paths and SPA navigation.
* Drop the eyebrow; add the docs' top page controls
The docs have no eyebrow line, so compiled pages no longer render one
(old sources still parse; the field is ignored). The title row gains the
docs' top controls: Copy page (copies the page text) and prev/next
chevrons wired to the same neighbors as the footer cards, disabled-dim
when a side is missing.
* Read kv keys and table first columns as labels, the docs way
kv keys dropped the blanket monospace — they render as the docs' row
labels (500, primary, sans), with backticks in the source opting a
code-like key (a path, an env var) into the inline-code chip; keys now
run through inline markdown to make that work. Table body first columns
pick up the same label treatment the docs tables show.
* Platform font and emcn chrome for pages
Pages live inside the app, and the platform — emcn, every workspace
surface — renders the system stack, not the docs' Inter webfont; Inter
made pages read as foreign next to the app around them. The face is now
the platform stack with weights on the platform scale (400/500/600), the
Inter delivery machinery (page-font.ts, the preview data-URI fetch, the
public woff2) is gone, and the section filter wears emcn ChipInput's
exact chrome (30px rounded-lg, --surface-5 fill flipping to --surface-4
in dark, --border, 14px, no focus ring). The docs' geometry — layout,
spacing, rails, tables, code frames — is unchanged.
* Color-only active state in the TOC — width can never move again
Two attempts at reserving the bold width (a hidden ghost, then freezing
measured rail widths) each traded one artifact for another: the ghost
did not stop the fit-content column re-measuring, and the freeze made a
long link wrap to two lines when it gained weight. The root cause was
letting the active state change a width-affecting property at all: the
active TOC link now shifts color only (muted to primary — the clerk
thumb already carries the emphasis), the search input is a plain text
field (the native search clear button is not emcn chrome), and the
ghost/freeze machinery is gone.
* Drop the Copy page control; keep the top chevrons
The title-row actions keep only the previous/next chevrons (rendered
when the page has neighbors); the Copy page button is gone.
* Set-level sidebar: docs-style groups with the current page expanded
Multi-page sets can now carry the whole set's sidebar. nav frontmatter
(groups of labelled page links, identical on every page of the set)
compiles into hidden set-nav markup whose sim: links resolve like any
other; the shell lifts it into the left rail as muted group labels over
page links, recognises the current page by title, and nests that page's
section list beneath it — the docs sidebar's exact shape. Pages without
nav keep the plain section list.
* Center the content column at the docs' measure
On wide panes the 1fr center cell stretched, so content hugged the left
rail with dead space before the TOC. The main column now caps at the
docs' ~760px measure and centers in its cell, matching how the docs
balance a wide viewport.
* Docs Steps, code-tab groups, and API method chips
Three docs components join the vocabulary: sim:steps renders the
numbered timeline (muted circle markers, hairline connector, title and
content per step); sim:tabs renders the docs' grouped code block (mono
tab chips, one pane at a time, the icon copy control targeting the
visible pane); and a METHOD prefix on a set-nav page entry renders the
API-reference chip — sidebar entries only, on the platform badge tokens
(blue and purple added to the mirrors and the live bridge).
Also: preview images switched from blob: to data: URIs — blob URLs are
origin-bound and the sandboxed frame's origin is opaque, so Chromium
refused to render them — and the page view-lock is sticky per file so a
patch stream cannot flash raw source.
* Downloaded pages carry their images
Absolute URLs made LINKS survive a download, but an embedded image
request from a downloaded file is cross-site and carries no session
cookie, so images 401ed outside the app. The standalone renderer now
inlines every workspace image the page references as a data: URI at
serve time — like a pdf carrying its images — capped at 8MB per image,
restricted to the page's own workspace, falling back to the URL
reference on any miss. Applies to serve, download, and public shares.
* Dead-center the content column between equal gutters
The rails are content-sized and unequal, so the old grid (fit-content /
1fr / fit-content) skewed the middle cell toward whichever rail was
narrower. The wide tier now uses the docs' geometry: a fixed 760px
content column centered between two equal flexible gutters, the sidebar
hugging the container's left edge and the TOC its right — rail widths
can no longer move the content.
* Defer title-bar history state out of currententrychange dispatch
The Navigation API fires currententrychange synchronously from the
history mutation that caused it, which can originate inside another
component's useInsertionEffect (style libraries navigating during
commit) — setState there trips React's 'useInsertionEffect must not
schedule updates'. The arrow-state sync now defers to a microtask,
flushing after the commit unwinds, with a disposal guard.
* Show the section sidebar only when there are sections to list
Fewer than two sections (and no set nav): the left rail and its filter
disappear and the reserved left gutter collapses — the content column
leads the container with the TOC trailing. Two or more sections, or a
multi-page set, keep the full centered docs frame.
* Execute the page shell in a DOM harness
jsdom runs the real shell against compiled pages and asserts the layout
decisions: both rails on a many-section page, only the left rail dropped
on a one-section page.
* Medium panes keep the TOC, not the sidebar
Between 560 and 860px the frame showed the section sidebar and hid the
clerk TOC — backwards by our own reasoning, since the sidebar is the
redundant list on a single page. The TOC now survives at medium widths
and the sidebar joins only on wide panes.
* Sidebar is for doc sets only; stagger the rails like the docs
The left rail now exists only for multi-page sets — on a lone page it
just repeated the TOC. A set opens its sidebar at 560px with the TOC
joining on wide panes (the docs stagger); a lone page waits until 700px
and then shows the TOC alone. Set-sidebar spacing tightens to the docs
values, with the current page's nested sections styled as small muted
entries behind a hairline instead of full chips.
* Center the lone-page content and TOC as a pair
On a page with no sidebar the content column stretched while the TOC
hugged the far edge, leaving a field of dead space between them. The
content now caps at reading width with the TOC directly beside it and
the pair centered in the pane, and the TOC waits until 800px so narrow
panes stay single-column a while longer.
* Equalize grid-template specificity across the rail tiers
The 560px tier selects .art-cols:not(.no-side-nav) at (0,2,0), so the
860px tier's bare .art-cols template at (0,1,0) could never win and a
set page on a wide pane kept the 2-column template — wrapping the TOC
to the next grid row, bottom-left. The wide template now carries the
same :not() guard, the 860 block's duplicate of the 800px no-side-nav
rules is gone, and a test pins every template rule to equal specificity
so a future tier can't silently lose the cascade again.
* extract_doc_assets: pull a reference deck's assets into the workspace
Sim-side handler for the new file-agent tool: given an uploaded .pptx
or .docx, unzip it (OOXML is a zip), parse theme1.xml into theme.json
(color scheme as hex, major/minor fonts, slide size from
presentation.xml) and write every ppt|word/media file into a
"<Name> assets" folder with original bytes and real content types.
Re-runs overwrite the set in place. Pure extractor unit-tested against
in-test-built packages; display label "Extracting assets from <file>".
* extract_doc_assets learns .pdf via the doc sandbox
PDFs have no zip structure or declared theme, so extraction runs in the
same vetted sandbox that compiles and renders documents: poppler's
pdfimages dumps every embedded image in its native format (masks
filtered via -list), pdfplumber contributes each image's placement
rects in page points plus the document's font names, and rendered pages
are sampled into an explicitly-inferred color palette. theme.json for a
pdf carries fonts, page size and count, the inferred palette, and a
per-asset placement map.
* Pages have one navigation rail; compile errors go to the agent
The left sidebar leaves the renderer and the DSL: the shell builds only
the content column and the clerk TOC (pair centered at 800px, one bare
.art-cols selector per tier so the cascade cannot invert), the filter
box goes with it, and nav frontmatter is tolerated but no longer
rendered — sidebar METHOD chips and the set-nav markup are gone.
Malformed sim: blocks no longer render a reader-facing "block was
skipped" card: the block is omitted and the failure is reported as a
diagnostic that apply_file_edit appends to its result, so the authoring
agent sees exactly which fence to fix. The lenient flag existed only to
suppress those cards mid-stream and is removed.
The steps timeline connector now derives its position from the marker
size, so it stays centered under the number circles.
* Sync tool catalog: extract_doc_assets accepts pdf
* Asset extraction yields the rebuild recipe, not just the parts
pptx: theme.json now maps every image to its slide-by-slide placements
(slide rels resolve rIds to media names; each pic frame's EMU offset and
extent convert to inches) plus the slide count.
pdf: a second layout.json is written — per page, the text blocks with
content, position, font, size, and fill color; the filled rects
(backgrounds and scrims); and rect-over-image overlay detection with
coverage, which is the "image opacity" effect decks fake with a tinted
rect. Stream alpha is unrecoverable, so overlays name the color and the
rendered page remains the reference for strength.
* Split shared-baseline text runs into separate blocks
Two text boxes sitting at the same height merged into one wide line;
a gap much wider than a space now starts a new block, so columns and
label/value pairs land as distinct entries in layout.json.
* Extract faithful document layout recipes
* Add .chart files: live interactive ECharts docs, static or table-backed
* Size charts by width-driven aspect, not panel height; separate title and legend
* Map table chart rows from storage column ids to display names
* Inject table rows as datasetIndex 0 so specs can transform; stagger array legends
* Give .chart files their own bar-chart icon
* Chart table sources gain groupBy/aggregate/pivot shaping; renderer-owned chrome
* sim:chart page fence: ECharts SSR to themed inline SVG; shared option builder
* sim:chart hydrated embeds: inline or .chart file refs, live table reads per serve
* Finish extensionless pages and document staging
* Render live charts without server hydration
* Cover active-theme page token overrides
* Keep artifact tokens synced with the app theme
* Use tabs for multi-page Sim docs
* Preserve dollar-prefixed tool credentials
* Build chart specs from validated fields
* Sync new integration docs into Copilot manifest
* Add in-document tabs to Sim pages
* Rebuild page TOC on tab changes
* Keep page tabs with docs chrome
* Stabilize tabbed page layout
* Regenerate docs manifest for staging's Modal docs page
* Share one divider between the bar and the chrome tab row
* Pin the keyless OCR path in the unreadable-document test
---------
Co-authored-by: Justin Blumencranz <96924014+j15z@users.noreply.github.com>
Co-authored-by: Vikhyath Mondreti <vikhyath@simstudio.ai>1 parent 6f34dd6 commit d9cfd7c
393 files changed
Lines changed: 30457 additions & 6226 deletions
File tree
- .github/workflows
- apps
- desktop
- build
- src
- main
- browser-agent
- browser-search
- terminal
- preload
- test
- docs/content/docs/en/platform
- sim
- app
- _shell
- _styles
- api
- copilot
- chat/steer
- tools/execute
- files
- public/[token]/content
- serve/[...path]
- mothership/execute
- table
- workflows/[id]/execute
- workspace/[workspaceId]
- components
- find-bar
- resource
- search-highlight
- workspace-chrome
- files
- components/file-viewer
- home
- components
- message-content
- components
- agent-group
- special-tags
- mothership-chat
- mothership-view
- components
- resource-content
- components/browser-session
- resource-tabs
- hooks
- stream
- integrations/components/connect-slack-bot-modal
- knowledge/[id]
- [documentId]
- components
- search-highlight
- settings/components/browser
- tables/[tableId]
- components/table-grid
- w/[workflowId]/utils
- components/icons
- ee/access-control
- components
- utils
- hooks
- queries
- utils
- lib
- api/contracts
- auth
- billing/core
- browser-agent
- charts
- copilot
- application
- async-runs
- auth
- chat
- docs
- generated
- request
- context
- go
- handlers
- lifecycle
- session
- tools
- resources
- tool-executor
- tools
- client
- handlers
- deployment
- management
- workflow
- registry
- server
- docs
- enrichment
- files
- image
- knowledge
- media
- other
- table
- workflow
- edit-workflow
- vfs
- core
- config
- utils
- desktop
- folders/application
- knowledge
- application
- documents
- orchestration
- logs
- mcp
- application
- organizations
- permission-groups
- platform-context/application
- pptx-renderer
- renderer
- utils
- table
- application
- views
- uploads
- contexts/workspace
- utils
- workflows
- application
- custom-blocks
- orchestration
- sanitization
- search-replace
- resources
- workspace-files
- workspaces
- tools
- triggers
- packages
- browser-protocol/src
- desktop-bridge/src
- logger
- src
- testing/src/mocks
- scripts
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
123 | 123 | | |
124 | 124 | | |
125 | 125 | | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
126 | 129 | | |
127 | 130 | | |
128 | 131 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
9 | 9 | | |
10 | 10 | | |
11 | 11 | | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
12 | 22 | | |
13 | 23 | | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
61 | | - | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
62 | 64 | | |
63 | 65 | | |
64 | 66 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
2 | 2 | | |
3 | 3 | | |
4 | 4 | | |
5 | | - | |
| 5 | + | |
6 | 6 | | |
| 7 | + | |
7 | 8 | | |
8 | 9 | | |
9 | 10 | | |
| |||
482 | 483 | | |
483 | 484 | | |
484 | 485 | | |
| 486 | + | |
| 487 | + | |
| 488 | + | |
| 489 | + | |
| 490 | + | |
| 491 | + | |
| 492 | + | |
| 493 | + | |
| 494 | + | |
| 495 | + | |
| 496 | + | |
| 497 | + | |
| 498 | + | |
| 499 | + | |
| 500 | + | |
| 501 | + | |
| 502 | + | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
| 517 | + | |
| 518 | + | |
| 519 | + | |
| 520 | + | |
| 521 | + | |
| 522 | + | |
| 523 | + | |
| 524 | + | |
| 525 | + | |
| 526 | + | |
| 527 | + | |
| 528 | + | |
| 529 | + | |
| 530 | + | |
| 531 | + | |
| 532 | + | |
| 533 | + | |
| 534 | + | |
| 535 | + | |
| 536 | + | |
| 537 | + | |
| 538 | + | |
| 539 | + | |
| 540 | + | |
| 541 | + | |
| 542 | + | |
| 543 | + | |
| 544 | + | |
| 545 | + | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
| 552 | + | |
| 553 | + | |
| 554 | + | |
| 555 | + | |
| 556 | + | |
| 557 | + | |
| 558 | + | |
| 559 | + | |
| 560 | + | |
| 561 | + | |
| 562 | + | |
| 563 | + | |
| 564 | + | |
| 565 | + | |
| 566 | + | |
| 567 | + | |
| 568 | + | |
| 569 | + | |
| 570 | + | |
| 571 | + | |
0 commit comments