Skip to content

feat(hub): publish a folder's AI agents with their project - #12490

Merged
rubenfiszel merged 14 commits into
mainfrom
tl/hub-publish-agents
Nov 20, 2026
Merged

rubenfiszel merged 14 commits into
mainfrom
tl/hub-publish-agents

Conversation

@tristantr

@tristantr tristantr commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A saved AI agent is an ai_agent resource, and the Hub bundle ships every resource as an empty stub. Publishing a folder built around an agent sent the agent's tool scripts and not the agent: the publish screen did not even list it.

What this changes

  • The publish screen lists a folder's agents as items of their own, and an item's dependency panel lists the agents it uses.
  • Bundle: an agent is bundled like the one-step flow a run makes of it, so the scripts its tools run come along and its $res: references (provider, tools' fixed inputs) are relocated by the same code as a flow's. An agent the bundle ships is not also turned into a stub. Its deploy identity (on_behalf_of) and sharing switches stay in the workspace.
  • Publish: agents are synced as a set after the items (POST /hub/agents).
  • Recording: an agent's demo is a conversation. "Record conversation" chats with the agent in a conversation of its own, one run per message (runAgent with a fresh memory_id), and saves the turns, each with its run as a FlowRecording (POST /hub/projects/{slug}/agent_recording).
  • Install: a project's agents are recreated after the resource stubs they point at; retargeting to another folder rewrites them.
  • Folders page: an "Agents" count per folder; agents no longer count as resources.
  • @windmill-labs/components: exports the read-only AI chat components (AIChatMessage, ToolGroupDisplay, ChatTypingIndicator, chatViewHost, readOnlyChatViewHost, toolGroups, shared), so the Hub draws a recorded agent conversation like the AI Sessions share page.
  • Backend: two proxy routes in hub_publish.rs; the folder usage query counts agents.

Needs the Hub side: windmill-labs/windmillhub#237. Against a Hub without it, a folder with no agent publishes as before. The Hub renders conversations with the components exported here, so it needs a package release that includes this PR.

Tested

  • Unit tests for the agent bundle: refs, rewrite, bundling with its script and without a duplicate stub, retargeting (4 new, 132 pass in workspaceSettings).
  • Locally, backend built from this branch against a local Hub with the paired branch: the publish screen lists 16 items for a folder with one agent and 15 scripts; "Create Hub draft" stores the agent with relocated tool paths, provider and inputs; reopening the screen rehydrates the agent row with its record button. A second sample folder (one agent, 2 flows, 6 scripts) published with 8 of 9 recordings saved.
  • The Hub page renders the conversation with the exported chat components (tested with the exports patched into the installed package).
  • svelte-check: no error in the changed files.

Not tested

  • A full recorded conversation end to end: the local workspace had no model key. The record drawer opens, a turn starts and its failure is shown.
  • Installing a project that ships an agent, through the import wizard.

Notes

  • The demo conversation shows up in the recorder's own chat list, since it is a real conversation with the agent.

🤖 Generated with Claude Code

A saved AI agent is an `ai_agent` resource, and the Hub bundle shipped every
resource as an empty stub: a project built around an agent reached the Hub with
the agent's tools and without the agent.

- The publish screen lists a folder's agents as items. An agent is bundled like
  the one-step flow a run makes of it, so the scripts its tools run come along
  and its `$res:` references (provider, tools' fixed inputs) are relocated by
  the same code as a flow's. Its deploy identity and sharing switches stay here.
- Agents are synced as a set after the items (`POST /hub/agents`).
- Its demo is a conversation: "Record conversation" chats with the agent in a
  conversation of its own, one run per message, and saves each turn with its
  run (`POST /hub/projects/{slug}/agent_recording`).
- Installing a project recreates its agents after the resource stubs they
  point at.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Deploying windmill with  Cloudflare Pages  Cloudflare Pages

Latest commit: 7d62eb4
Status: ✅  Deploy successful!
Preview URL: https://f3348d90.windmill.pages.dev
Branch Preview URL: https://tl-hub-publish-agents.windmill.pages.dev

View logs

tristantr and others added 4 commits November 17, 2026 15:21
The Folders table counted a saved agent among the resources, so a folder built
around an agent showed nothing that said so. Agents get their own column, and
the resources count is the credentials and other resources only.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Widening the shared layout's item kind broke the workspace deploy screen, which
passes that kind on. The layout already draws an agent as a resource row it is
told about (isAgent): the publish screen now uses that, and the layout is
unchanged.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Hub renders recorded agent conversations with the same components as the
AI Sessions share page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tristantr
tristantr marked this pull request as ready for review November 18, 2026 14:01
@claude

claude Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @tristantr's task in 2m 10s —— View job


cc @tristantr

Should address issues before merging: each recorded turn repeats the earlier turns' tool calls (readAgentTurn); nits: crypto.randomUUID() used in frontend code, raw <textarea> in AgentRecordSession.svelte.

Findings

P1: a recorded turn repeats the tool calls of earlier turns. deployToHubSession.svelte.ts:2263
sendAgentMessage runs the agent with a fixed memory_id. The worker loads that stored conversation into messages (ai_executor.rs:1827-1836) and builds AgentHistory::new(messages) from it (:1972). It then returns messages.result() (:3166), so the result's messages also contain every earlier turn. readAgentTurn collects every assistant.tool_calls in that list. So turn N's tool_calls include all the tools from turns 1..N-1. They show twice in the drawer and are saved that way in the AgentRecording the Hub replays. The fix is to read only the messages after the last user message (suggestion inline). Fix this →

P2: AGENTS.md says not to call crypto.randomUUID(). deployToHubSession.svelte.ts:1864
The rule is: "do not call crypto.randomUUID() in frontend code. Import randomUUID from $lib/utils/uuid instead." It also matters in practice: on a self-hosted instance served over plain http, crypto.randomUUID does not exist, so opening the record drawer throws.

P2: raw <textarea>. AgentRecordSession.svelte:95
frontend/AGENTS.md says: "use Windmill's design-system components — never raw HTML elements." Use TextInput in textarea mode instead.

Checked, no issue found

  • Backend: /hub/agents and /hub/projects/{slug}/agent_recording use the same HubPublishCtx proxy as the existing publish routes, so auth is unchanged. The two folder-usage counts use IS DISTINCT FROM 'ai_agent', so resources with no type are still counted, and the totals add up.
  • Bundle: an agent's on_behalf_of, run_on_behalf_of and slack_anyone are dropped before publish. Paths of agents the bundle ships are removed from resourcePaths, so they are not also stubbed. Retargeting rewrites agent paths and values.
  • Install: agents are created after the stubs they point at, with updateIfExists: false.
  • One point I could not check from here: agent_recording is keyed by the workspace path, but agents are published under newPath with source_path. Make sure the Hub (windmillhub#237) looks the agent up by source_path, or the recording will not attach to it.

AGENTS.md compliance

The two P2s above are the only violations I found. The new .sqlx files match the changed queries, and the OpenAPI change documents the new agents field.

Test coverage

  • Backend: the two routes only proxy to the Hub, and the change to the count query is small. A unit test is not really needed, but nothing exercises the routes against a Hub stub.
  • Frontend: the new pure logic in projectBundle.ts (extractAgentRefs, rewriteAgentValue, bundling, retargeting) is covered in projectBundle.test.ts. readAgentTurn is pure parsing and is where the P1 bug is, so a small test there (a two-turn result with memory) would be worth adding.

Manual verification still needed: With a model key, open "Record conversation" on an agent and send two messages, each of which calls a tool. The second turn should list only its own tool calls. Save it, and the Hub page should replay both turns. Then install the published project into another folder through the import wizard. The agent should be created with its tools pointing at the retargeted scripts and its provider pointing at the retargeted stub. Both of these paths are listed as "Not tested" in the PR description.

- readAgentTurn reads the messages after the turn's own user message: a run
  with memory returns the whole conversation, so turn N repeated the tools of
  turns 1..N-1. Covered by a two-turn test.
- randomUUID from $lib/utils/uuid instead of crypto.randomUUID, which is
  missing on instances served over plain http.
- The record drawer's message box uses TextInput instead of a raw textarea.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

Good to merge

The three findings from my prior pass are all resolved in f823ea87 ("Record only the current turn's tool calls; address review nits"), and a fresh pass over the full diff turned up no new issues.

  • The P1 (readAgentTurn replaying earlier turns' tool calls) is fixed correctly: messages is sliced after the last role === 'user' message, which is the current turn's own message under managed memory, and the two-turn regression test (readAgentTurn.test.ts) pins it. The answer still comes from output, which is the run's own result, so it is unaffected.
  • The crypto.randomUUID() P2 is fixed: randomUUID is imported from $lib/utils/uuid (which is Math.random-based and works over plain HTTP).
  • The raw <textarea> P2 is fixed: TextInput with underlyingInputEl="textarea", which is the component's supported textarea variant (bind:value, inputProps.onkeydown/disabled all pass through).

Considered and dismissed

  • #pushAgents reads resp.ids[a.newPath] from a response shape I can't see here — the ids map keyed by the published path is a new Hub-side contract (windmillhub#237). Not a bug in this repo; it is the explicit cross-repo dependency the PR body names. A Hub without the route 404s, and #pushAgents only counts that as a failure when agents were actually present, so the "publishes as before" fallback holds.
  • agent_recording posts the workspace path while agents are published under newPath + source_path — same as my prior note: the Hub must resolve the recording by source_path. Hub-side, listed under manual verification below rather than re-flagged.
  • agentAsFlow synthesizes { type: 'aiagent', tools, input_transforms: { provider: static } } and reuses extractFlowRefs/rewriteFlowValue — verified that allFlowModules (via getAllModules) expands the aiagent module's tools, and the provider's $res: sits in a static input transform, so extraction and rewrite hit the same code paths a flow's does. The four projectBundle.test.ts cases cover refs, rewrite, bundling-without-duplicate-stub, and retargeting.
  • asLayoutItem rewrites an agent to kind: 'resource' + agent: true — WorkspaceDeployLayout's isAgent prop routes it back to kind 'agent', titles it by summary, and keeps key: agent:<path> so selection/status/hubItemIds all still address the item. Verified against the layout's itemRow.
  • runAgent({ memoryId, requestBody: { user_message } }) — matches the backend run_agent (Query<RunJobQuery> with memory_id, JSON args) and the user_message arg used by the chat path; memory_key accepts any trimmed string, so the UUID is fine.
  • New backend routes — both use HubPublishCtx, which requires admin (require_admin) and a folder scope; ProjectSlug validates the slug on deserialization, so nothing unvalidated is interpolated into a Hub URL. Same surface as the existing publish routes.
  • Folder usage counts — resource_type = 'ai_agent' for agents and IS DISTINCT FROM 'ai_agent' for the rest: a resource with no type still counts as a resource, and the two totals add up. .sqlx cache entries match the two changed queries, and the OpenAPI change documents the new agents field.
  • Agent value scrubbing on publish — on_behalf_of, run_on_behalf_of, slack_anyone are dropped, mirroring what the backend's agent_turn strips before building the one-step flow.
  • TextInput / randomUUID / test import paths — readAgentTurn.test.ts imports ./deployToHubSession.svelte (the .svelte.ts resolution used by existing tests), and TextInput is the canonical design-system component.

Test coverage

  • Backend: the two routes are thin proxies over the existing HubPublishCtx auth and the count-query change is a one-line predicate swap; no unit test is warranted, and nothing exercises the proxies against a Hub stub (acceptable, as they only forward).
  • Frontend: the new pure logic is covered — extractAgentRefs/rewriteAgentValue/bundling/retargeting in projectBundle.test.ts (4 new), and the readAgentTurn turn-isolation regression in readAgentTurn.test.ts (2 new). The rest is Svelte components, which this codebase does not unit-test.
  • CI/docs/config: the package.json export-map additions have no automated test; noted for completeness.

Manual verification still needed

  • With a model key, open "Record conversation" on an agent and send two messages that each call a tool: the second turn must list only its own tool calls in the drawer, and the saved recording must replay both turns with distinct tool sets on the Hub page.
  • Install a project that ships an agent through the import wizard: the agent must be created after its resource stubs, retargeted into the destination folder with its $res: provider and tool inputs rewritten, and must not collide-overwrite an existing resource (updateIfExists: false).
  • Confirm against the paired Hub branch that POST /hub/projects/{slug}/agent_recording attaches the recording by source_path, since the workspace path it sends is not the published newPath.

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

cc @tristantr

Should address issues before merging: MCP resource dependencies omitted, agent-only variables omitted, background tool calls omitted; nit: deprecated Button sizing.

[P1] MCP resource dependencies omitted — frontend/src/lib/components/workspaceSettings/projectBundle.ts:469

Publishing an agent with an MCP tool omits that tool’s resource dependency. extractAgentRefs delegates to the flow walker, which explicitly filters out tool_type: 'mcp'; the rewriter skips those tools too. I reproduced a bundle containing the model stub while retaining $res:f/source/mcp in the published agent. Installing it elsewhere leaves the agent pointing at the publisher’s resource path, without a corresponding credential stub. The new agent exporter needs to extract and rewrite MCP resource_path fields.

[P1] Agent-only variables omitted — frontend/src/lib/components/workspaceSettings/projectBundle.ts:602

The new agent retargeting uses a map built through collectExportVarPaths, but that collector still visits only flows, apps and triggers. A variable used exclusively in an agent’s tool input therefore stays $var:f/proj/region when importing into mine. I reproduced this against the pinned source. varContainmentViolation then rejects the agent, so it is skipped during installation. Importing into the original folder also creates no placeholder for that variable. Include agent values in the shared variable collector.

[P1] Background tool calls omitted — frontend/src/lib/components/workspaceSettings/deployToHubSession.svelte.ts:2267

A background tool’s completion is emitted as a user message with agent_action: tool_call (backend/windmill-worker/src/ai/tools.rs:402). Consequently, selecting the last message whose role is user can select a tool completion rather than the user’s prompt. For a turn that starts a background tool and then answers after its completion, readAgentTurn drops the original call and saves an empty tool_calls list. I reproduced this with the worker’s message shape. Exclude these generated completion messages when identifying the turn boundary.

[P2] Deprecated Button sizing — frontend/src/lib/components/workspaceSettings/DeployToHub.svelte:914

Both new conversation-recording buttons pass size="xs". frontend/AGENTS.md explicitly says: “Never pass a @deprecated prop.” It also specifies unifiedSize for Button sizing. Use unifiedSize="xs" at lines 914 and 924.

Test coverage

The added frontend tests cover agent bundling/retargeting and ordinary current-turn parsing. Add small pure-logic regression cases for the three functional findings above. No new backend unit or proxy integration tests are included. The package-export and OpenAPI configuration changes do not require automated tests.

I ran read-only, in-memory checks against the pinned production TypeScript, reproducing the three functional findings and confirming ordinary two-turn parsing. I did not run Vitest, Rust tests or the application. The author reports publish/draft and renderer checks; full conversation recording and agent installation remain untested.

Before merge, publish an agent using an MCP resource and a variable-backed tool input, install it into a different folder, configure its dependencies and run it successfully.

Record two messages including a background tool call, then save and reopen the Hub demo. Each turn should retain exactly its own tool calls.

- extractAgentRefs and rewriteAgentValue read an MCP tool's resource_path,
  which the flow walker skips, so its resource is stubbed and relocated
  instead of left pointing at the publisher's path.
- collectExportVarPaths visits agents, so a variable only an agent uses is
  relocated on retarget and gets its placeholder on install.
- readAgentTurn ignores the user messages that carry a background tool's
  result (tagged with agent_action) when finding the turn's start, so a turn
  that waits on a background tool keeps its call.
- The record buttons use unifiedSize instead of the deprecated size.

Tests for each.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tristantr

Copy link
Copy Markdown
Contributor Author

Codex review addressed in b713d28:

  • MCP resource dependencies: extractAgentRefs / rewriteAgentValue now read an MCP tool's resource_path (bare or $res:), so it is stubbed and relocated.
  • Agent-only variables: collectExportVarPaths visits bundle.agents, so they are relocated on retarget and get a placeholder on install.
  • Background tool calls: the turn boundary skips user messages tagged with agent_action (a background tool's result).
  • Button sizing: unifiedSize="xs" on both record buttons.

A regression test for each (137 pass in workspaceSettings).

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

cc @tristantr

Should address issues before merging: nested MCP dependencies omitted, linked agent tools not relocated.

[P1] Nested MCP dependencies omitted — frontend/src/lib/components/workspaceSettings/projectBundle.ts:478

The added MCP handling visits only the agent’s immediate tools. A supported nested AI-agent tool can itself contain MCP tools, which the delegated flow walker also skips. Against the pinned source, bundling such an agent emitted only its model stub and retained $res:f/source/mcp inside the nested tool. Even supplying an explicit path map did not rewrite it. Publishing and installing this agent therefore leaves its nested MCP connection pointing at the publisher’s resource, without a corresponding credential stub. Traverse nested agent tool rosters during both extraction and rewriting.

[P1] Linked agent tools not relocated — frontend/src/lib/components/workspaceSettings/projectBundle.ts:489

Delegating to rewriteFlowValue leaves nested tools’ aiagent.agent references untouched; extractAgentRefs misses them too. I reproduced an export containing parent and child agents: retargeting created f/mine/parent and f/mine/child, but the parent’s linked tool still named f/source/child. The worker resolves that field when invoking the nested agent, so the installed parent calls the wrong path and fails when it is absent. Extract and rewrite these links, preserving the agent dependency when bundling.

The earlier top-level MCP, agent-only variable, background-call boundary and Button-sizing findings are addressed.

Test coverage

Frontend tests cover the earlier regressions, but miss both nested dependency cases above. I executed 65 existing projectBundle/readAgentTurn test cases through an in-memory Node assertion harness against the pinned source; all passed. Additional pinned-source checks reproduced both findings. This was not a Vitest run.

No new backend unit or proxy integration tests are included. The package-export and OpenAPI configuration changes need no separate unit tests. I did not run Rust tests or the application.

Before merge, publish an agent containing both a nested MCP tool and a linked saved-agent tool, install into another folder, configure the generated stubs and successfully invoke both tools.

Full conversation recording remains to be exercised with a model credential: record two messages including a background tool call, save, and reopen the Hub demo to confirm each turn retains its own calls.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

cc @tristantr

Mergeable, but should ideally address nits: silent agent-list error swallow in #loadWorkspace.

I re-reviewed the full diff against the prior rounds. The three Codex findings (MCP resource dependencies, agent-only variables, background tool calls) and the two earlier nits (crypto.randomUUID, raw <textarea>) are all correctly fixed in the latest commits, with regression tests pinning each. One new nit below.

Findings

P2 — silent swallow of a genuine agent-list failure, made harmful by the full-set sync — frontend/src/lib/components/workspaceSettings/deployToHubSession.svelte.ts:583

this.#listAllPages((p) =>
    ResourceService.listResource({ workspace, resourceType: 'ai_agent', ...p })
).catch(() => [])

The comment claims the catch is there so "a workspace without the type lists none rather than failing the whole load", but that rationale doesn't match reality: list_resources (backend/windmill-store/src/resources.rs:356) just adds resource_type = 'ai_agent' as a WHERE filter and returns an empty list with no error when none exist. So the .catch(() => []) never fires for the case it names — it only swallows genuine failures (500, network, auth). The other four lists in the same Promise.all have no such catch, so any of their failures rejects the whole load and shows Failed to load project items. An agent-list failure is instead silent: the publish screen simply omits the agents, and because #pushAgents is a full-set sync (agents: [] clears the Hub's agents), publishing in that state removes the folder's agents from the Hub without the user being told they were ever missing. Recommend removing the .catch(() => []) (a workspace without the type already lists none), so a real failure surfaces like the other lists do.

Considered and dismissed

  • readAgentTurn turn boundary — verified against backend/windmill-worker/src/ai/tools.rs:402: background completions are user messages carrying agent_action: Some(AgentAction::ToolCall {...}), so role === 'user' && !m.agent_action correctly skips them while still finding the current turn's own prompt. The two-turn and background-tool regression tests pin this, and answer still comes from output (the run's own result).
  • agentAsFlow → extractFlowRefs / rewriteFlowValue — confirmed getAllModules/getChildModuleBranches expand an aiagent module's tools, and rewriteFlowValue rewrites flowmodule tool script/flow paths, rawscript content $res:, and static input transforms. The provider's $res: rides in the input_transforms.provider static value, so extraction and rewrite hit the same code paths a flow's does. Tests pin script path, rawscript content, input transform and provider.
  • MCP resource_path handling — collectFlowNodes/getChildModuleBranches exclude non-flowmodule (MCP) tools, so the explicit mcpResourcePath extraction/rewrite is necessary and correct; bare and $res: forms are both covered by tests.
  • Agent-only variables — collectExportVarPaths/buildRetargetMap now visit bundle.agents, so retarget rewrites them and install creates placeholders; the retargetProjectExport agent branch applies rewriteVarRefsInValue(rewriteAgentValue(...)). Test pins $var:f/proj/region relocation.
  • Backend routes — publish_agents and publish_agent_recording go through HubPublishCtx, which enforces require_admin, the folder scope query, and ProjectSlug validation on deserialize; they mirror publish_triggers/publish_pipeline_recording exactly. No new attack surface.
  • Folder usage counts — resource_type = 'ai_agent' vs IS DISTINCT FROM 'ai_agent' are disjoint, a NULL-typed resource still counts as a resource, and the two totals add up. The .sqlx cache files match the two changed queries.
  • resourcePaths.delete(it.path) — prevents an agent the bundle ships from also becoming an empty stub at the same path; resourcePaths holds original paths and it.path is the original, so the delete matches. The bundle test asserts no ai_agent stub and the whole-agent item present.
  • crypto.randomUUID / button sizing / <textarea> — randomUUID from $lib/utils/uuid (Math.random-based, works over plain HTTP), unifiedSize="xs", and TextInput underlyingInputEl="textarea" are all in place.
  • importAgent — runs after the resource stubs with updateIfExists: false, and is guarded by extractAgentRefs + varContainmentViolation; a collision is reported as a skipped/failed item.
  • Cross-repo contracts — #pushAgents reading resp.ids[a.newPath] and agent_recording posting the workspace path (while agents publish under newPath + source_path) are Hub-side contracts named in the PR body (windmillhub#237); they are not bugs in this repo but are listed under manual verification below.

Test coverage

  • Backend: the two new routes are thin proxies over the existing HubPublishCtx auth, and the count-query change is a one-line predicate swap plus its .sqlx cache entry; no new unit or integration test is warranted for a proxy/forward, and none is present (acceptable).
  • Frontend: the new pure logic in projectBundle.ts (extractAgentRefs, rewriteAgentValue, bundling, retargeting, MCP and variable handling) and readAgentTurn (including the background-tool boundary) is covered by projectBundle.test.ts and readAgentTurn.test.ts. The Svelte components themselves are not unit-tested, which matches this codebase's convention.
  • Package exports / OpenAPI / config — the package.json export-map and openapi.yaml additions are config-only; no automated tests expected.

Manual verification still needed before merge: with a model key, open "Record conversation" on an agent and send two messages, at least one of which triggers a background tool; each turn should retain exactly its own tool calls, and saving should produce a Hub demo that replays per-turn. Publish an agent whose tools use an MCP resource and a $var:-backed input, install it into a different folder, and confirm the agent's provider/tools/inputs resolve against the new folder's stubs and placeholders (this path — install through the import wizard — is still listed by the author as untested). Finally, confirm on the Hub side that agent_recording attaches to the agent by source_path, since the recording is posted with the workspace path while the agent row is keyed by newPath.

- The flow walker reads, on every aiagent step or tool, the saved agent it
  links to (`agent`) and its MCP tools' `resource_path`, and the rewriter
  relocates both keeping their prefix. This covers agents nested as tools and
  flows with agent steps, not only an agent's own tools.
- A referenced `ai_agent` resource is pulled into the bundle as an agent,
  so a linked agent from another folder ships whole instead of as an empty
  stub.
- The agent list no longer swallows failures: agents sync as a full set, so
  publishing without them would have removed them from the Hub.

Tests for the nested MCP tool, the linked agent's retarget and the linked
agent from outside the folder.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tristantr

Copy link
Copy Markdown
Contributor Author

Latest Codex and Pi findings addressed in 3f76368:

  • Nested MCP dependencies / linked agent tools: the flow walker now reads, on every aiagent step or tool, the linked saved agent (agent) and its MCP tools' resource_path; the rewriter relocates both, keeping the $res: / bare form. This covers agents nested as tools, and flows with agent steps.
  • A referenced ai_agent resource is pulled into the bundle as an agent, so a linked agent from another folder ships whole rather than as an empty stub.
  • Agent-list failure: the .catch(() => []) is gone, so a failed list fails the load like the other lists instead of syncing an empty set.

Tests for each (140 pass in workspaceSettings).

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

cc @tristantr

Should address issues before merging: linked-tool input resources omitted, resource-backed brain fields omitted.

[P1] Linked-tool input resources omitted — frontend/src/lib/components/workspaceSettings/projectBundle.ts:514

A saved agent can contain a linked agent tool whose tool_inputs supplies static resource bindings. The delegated flow walker and rewriter ignore those bindings, although the worker applies them through overlay_tool_inputs. I reproduced a bundle retaining $res:u/admin/db with no corresponding stub and no unresolved reference. Retargeting also leaves $res:f/proj/db unchanged even when its existing stub moves to f/mine/db. The installed agent therefore uses the source resource path and fails when that resource is absent. Extract and rewrite static resource references inside tool_inputs.

[P1] Resource-backed brain fields omitted — frontend/src/lib/components/workspaceSettings/projectBundle.ts:506

agentAsFlow exposes only provider as a static input, dropping resource dependencies in other brain fields. The runtime supports these: config_to_draft in backend/windmill-api/src/ai_evals/run.rs wraps every brain field as a static transform for resource interpolation. With a valid text agent whose output_schema is $res:f/proj/answer_schema, extraction misses the schema resource and rewriting leaves its reference unchanged even with an explicit relocation map. Publishing and installing the agent consequently loses its schema dependency. Include the other brain fields in extraction and rewriting.

Test coverage

I executed 68 existing projectBundle and readAgentTurn cases through an in-memory Node assertion harness; all passed. This was not a Vitest run. Additional checks against production source reproduced both findings and confirmed the latest nested MCP and linked-agent fixes.

The author reports 140 passing workspaceSettings tests and local draft-publishing/rendering checks. Add focused pure-logic regression cases for the two omissions above; Svelte component tests are unnecessary. No new Rust unit or integration tests cover the proxy routes or folder-count query. Package-export and OpenAPI configuration changes need no separate unit tests. I did not run Rust tests or the application.

Before merge, publish an agent containing a linked tool with a resource-backed input override and a resource-backed output schema. Install into another folder, fill the generated stubs, and confirm both references resolve there during a successful run.

With a model credential, record two conversational turns including a background tool call, save the recording, and reopen the Hub demo. Each turn should retain its own answer and tool calls.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

cc @tristantr

Mergeable, but should ideally address nits: stale doc comments in projectBundle.ts (extractFlowRefs, collectExportVarPaths).

The four prior review rounds are fully addressed in the latest commits, and a fresh pass over the full diff turned up no remaining functional or security issues. One P2 nit remains.

Findings

P2 — doc comments not updated for the new agent/MCP extraction — frontend/src/lib/components/workspaceSettings/projectBundle.ts:68 and :586

Two function-level doc comments now describe less than the code does:

  • extractFlowRefs (line ~68) enumerates the ref kinds it returns (rawscript $res:, static-input $res:, script steps, flow steps) but not the aiagent branch added here: a linked saved agent (agent → resource) and an MCP tool's resource_path (resource).
  • collectExportVarPaths (line ~586) says "referenced by the export's flows, apps and triggers" but now also walks bundle.agents.

Both are the same class of stale comment the diff otherwise carefully updated elsewhere (FetchedItem.value, ItemKind, the agentAsFlow rationale). Since extractFlowRefs/rewriteFlowValue are documented as "the single definition of what a tool can reference," a reader relying on those comments would miss that aiagent links and MCP servers are handled there.

Considered and dismissed

  • enabled_tools naming an MCP server by resource path would not be relocated — dismissed. enabled_tools is flow-local (AGENT_FLOW_LOCAL_KEYS in agentResourceUtils.ts): it narrows one step's roster and is never saved into the ai_agent resource value, so it cannot appear in the shipped agent value.
  • Non-provider brain keys (system_prompt, etc.) holding $res: refs are not extracted/rewritten — dismissed. Among the keys actually saved on an ai_agent resource, only provider carries a resource ref; system_prompt is free text and not a documented $res: location, so wrapping only provider in agentAsFlow is complete for the resource-ref surface.
  • readAgentTurn turn boundary — verified against ai_types.rs/types.rs: background completions serialize as user messages with a top-level agent_action, and OpenAIMessage.agent_action is skip_serializing, so the Message { #[serde(flatten)] message, agent_action } shape the worker returns puts agent_action alongside role/tool_calls exactly as readAgentTurn reads it. Slicing after the last non-tagged user message is correct; the two regression tests pin both multi-turn memory and the background-call case.
  • Backend routes — publish_agents and publish_agent_recording both go through HubPublishCtx, which enforces require_admin and the folder scope, and ProjectSlug validates on deserialization, so nothing unvalidated is interpolated into a Hub URL. Same surface as the existing publish routes.
  • Folder usage counts — resource_type = 'ai_agent' vs IS DISTINCT FROM 'ai_agent' are disjoint; an untyped (NULL) resource still counts as a resource; the two .sqlx cache entries match the changed queries.
  • resourcePaths.delete(it.path) — prevents an agent the bundle ships whole from also becoming an empty stub; the typeOf memoization and the queued/newResources dedup correctly turn a referenced ai_agent resource into a shipped agent item without re-fetching.
  • asLayoutItem / isAgent / WorkspaceDeployLayout — verified: the layout has an isAgent prop that titles agents by summary and draws the agent icon; asLayoutItem preserves key, so selection/status/hubItemIds all keep addressing the item.
  • #pushAgents reading resp.ids[a.newPath], and agent_recording posting the workspace path while agents publish under newPath+source_path — both are the explicit cross-repo contracts with windmillhub#237 the PR body names; the fallback ("a folder with no agent publishes as before") holds because #pushAgents only counts a 404 as a failure when agents were present.
  • sendAgentMessage error display for max-iteration/partial failures — the failed result still surfaces (whole-error JSON stringified), so this is a cosmetic fallback rather than a swallowed error.
  • Test import ./deployToHubSession.svelte — matches the existing .svelte.ts resolution used by sibling tests (quickjsEval, persistedCache, etc.).

Test coverage

  • Backend: the two new routes are thin proxies over the existing HubPublishCtx auth path, and the count-query change is a one-line predicate swap with matching .sqlx entries. No new unit/integration test is warranted, and nothing exercises the proxies against a Hub stub (acceptable — they only forward).
  • Frontend: the pure logic added in projectBundle.ts (agent refs, MCP resource_path in bare/$res: forms, nested agent tools, linked agents, agent-only variables, bundling-without-duplicate-stub, retargeting) is pinned by the new projectBundle.test.ts cases, and readAgentTurn is pinned by readAgentTurn.test.ts. No Svelte component tests are expected.
  • CI/config: the package.json export/typesVersions additions and OpenAPI change need no automated tests.

Manual verification still needed: with a model key, open "Record conversation" on an agent, send two messages that each call a tool (including one background tool call), save, and reopen the Hub demo — each turn should retain exactly its own tool calls and the Hub page should replay the conversation via the exported components. Separately, install a project that ships an agent into a different folder and confirm its provider/tool $res: refs resolve against the generated stubs and it runs.

Bundling
- A saved agent is read unvalidated and its tools nest, so references are no
  longer looked up field by field. One walk over the whole value serves
  extraction, rewriting and the install check: whole-string $res: values,
  $res: tokens in rawscript content, and the shapes that name things by path
  ({type: script|flow, path}, {type: aiagent, agent}, {tool_type: mcp,
  resource_path}) wherever they sit. Covers tool_inputs, output_schema and
  any key added later.
- Inputs are read as the worker reads them: a static value is data (only
  whole-string refs count), a JavaScript expr is never resolved. Only on real
  input entries, so nothing can be hidden beside them from the install check.
- Prose holding a $res: token mid-sentence is no longer a reference.
- enabled_tools follows an MCP server to its new path.
- An agent named by a trigger is shipped whole like one an item references.

Recording
- An agent parked on an approval or a question is detected and its run
  cancelled with a clear message; closing the drawer or disposing the session
  cancels the run in flight.
- Turns no longer keep the whole run: it names who ran it and holds the agent
  as the workspace stores it. Structured-output calls are not shown as tools;
  web searches are.
- Send is disabled while saving; a failed message is handed back to retype; a
  save does not close another agent's drawer.
- The empty agent sync ignores only a 404.

Backend: agent_recording requires the recording key (null clears). OpenAPI:
agents is required in folder usage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tristantr

Copy link
Copy Markdown
Contributor Author

Addressed in f12a0c7, aiming to close the class rather than the next instance:

  • Linked-tool input resources / brain fields (Codex): agent references are no longer looked up field by field. One walk over the agent's whole value now serves extraction, rewriting and the install check: whole-string $res: values anywhere, $res: tokens in rawscript content, and the path-naming shapes ({type: script|flow, path}, {type: aiagent, agent}, {tool_type: mcp, resource_path}) at any depth. This covers tool_inputs, output_schema and keys added later. Inputs are read as the worker reads them (static value is data, JavaScript expr is never resolved), only on real input entries so nothing can be hidden from the install check beside them.
  • Also: prose with a mid-sentence $res: token is no longer a reference, enabled_tools follows a relocated MCP server, an agent named by a trigger ships whole.
  • Stale comments (Pi): updated.
  • Recording: parked agents (approval/question) are detected and cancelled; closing the drawer cancels the run; turns no longer embed the whole run (identity and unrelocated agent value); structured-output calls hidden, web searches shown; save/send races fixed; the empty agent sync only ignores a 404; agent_recording requires the recording key.

212 tests pass in workspaceSettings + importWizard.

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

Good to merge

The previous findings are addressed. I found no further confident functional, security, performance, or AGENTS.md issues in the final diff.

Test coverage

All 76 projectBundle and readAgentTurn cases passed through an in-memory Node assertion harness using production TypeScript and traversal helpers. This was not Vitest. git diff --check passed. I did not run Rust tests or the application.

Frontend regression tests cover dependency extraction, relocation, bundling, and turn parsing; component tests are unnecessary. No new Rust unit or integration tests cover the proxy routes or folder-count query. Package-export and OpenAPI configuration changes need no separate unit tests.

Before merge, publish an agent using nested MCP tools, linked agents, resource-backed inputs/schema, and variables. Install it into another folder, configure its stubs, and confirm successful execution through the relocated dependencies.

With a model credential, record two conversational turns including a background tool call, save, and reopen the Hub demo. Each turn should retain its own answer and calls. Also confirm closing the drawer cancels an active run and an approval/question produces the cancellation message.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

Good to merge

I re-reviewed the full diff (base 3cc8b207 → head f12a0c744) against the six prior review rounds. Every previously-flagged finding — the readAgentTurn turn-boundary P1, the crypto.randomUUID and raw-<textarea> nits, the Button size deprecation, the MCP/agent-only-variable/background-call P1s, the nested-MCP and linked-tool P1s, and the two doc-comment nits — is correctly resolved in the latest commits, with regression tests pinning each. A fresh pass found no new functional, security, or compliance issues.

Considered and dismissed

  • readAgentTurn turn boundary (deployToHubSession.svelte.ts:2299) — verified against backend/windmill-worker/src/ai_executor.rs:3166-3226 and backend/windmill-types/src/flow_status.rs:206-241: the result is AIAgentResult { output, messages }, where messages is Message { #[serde(flatten)] message, agent_action } (agent_action serialized at top level, #[serde(tag="type", rename_all="snake_case")]). Slicing after the last role === 'user' && !m.agent_action message isolates the current turn; background tool completions and web-search messages carry agent_action and are excluded/handled correctly. The three regression tests pin multi-turn memory, background calls, structured output and web search.
  • walkAgentConfig whole-value walk (projectBundle.ts:141) — the one-walk extraction/rewrite is consistent with the worker: transform_json_unchecked (workspaces.rs:3914) resolves only whole-string $res: values and recurses into objects/arrays, which is exactly what the walk does (RES_VALUE_RE for strings, recursion for objects). Static value is walked as data (whole-string refs only, no shape interpretation) and JavaScript expr is copied through unresolved — matching "inputs are read as the worker reads them." The flow path's token-scan of static string values is pre-existing over-broad behavior, not something this PR changes or contradicts.
  • Nested MCP / linked-agent tools in flows — allFlowModules (flowExplorer.ts → flowTree.ts:60-67) filters only flowmodule tools (tool_type === undefined | 'flowmodule'), so nested aiagent tools are flattened as modules and each is inspected; their MCP tools and tool_inputs are then collected. No nested case falls through.
  • Agent promotion / double-stub (buildProjectBundle) — resourcePaths.delete(it.path) runs after buildPathMap (so the agent path is still relocated via itemPaths) and before stub creation, so a shipped agent is not also stubbed. agentsAmong/promoted/queued correctly promote a trigger-named ai_agent resource to a whole agent item and dedupe by agent:<path>.
  • Agent value scrubbing (deployToHubSession.svelte.ts:914) — drops on_behalf_of, run_on_behalf_of, slack_anyone, mirroring agent_turn (agent_runs.rs:209-212). workspaces_export.rs only drops on_behalf_of, but that is a separate, untouched feature; the new code is the more complete of the two.
  • Backend routes (hub_publish.rs:490-549) — publish_agents/publish_agent_recording go through HubPublishCtx, which enforces require_admin + the folder scope, and ProjectSlug validates on deserialization, so nothing unvalidated is interpolated into a Hub URL. The AgentRecordingBody double-Option deserializer is correct: absent → 400, null → clear, object → set.
  • #pushAgents reading resp.ids[a.newPath], and agent_recording posting the workspace path while agents publish under newPath+source_path — both are the explicit cross-repo contract with windmillhub#237 the PR body names; the 404-only fallback for the empty sync preserves "a folder with no agent publishes as before."
  • Recording send/save races and cancel — guarded by agentPending/agentSaving and the #agentTok epoch, which closeAgentRecord, openAgentRecord, and dispose() all bump before #cancelAgentRun; a run_agent invocation matches the backend (memory_id + user_message arg), and runAgent never runs two turns with the same memory_id concurrently.
  • Folder usage counts (folders.rs:753-766) — resource_type = 'ai_agent' vs IS DISTINCT FROM 'ai_agent' are disjoint; a NULL-typed resource still counts as a resource; the two .sqlx cache entries match the changed queries and the OpenAPI change marks agents required.
  • UI plumbing — asLayoutItem keeps key: agent:<path> so selection/status/hubItemIds still address the item; WorkspaceDeployLayout has an isAgent prop routing the row back to kind agent; TextInput supports underlyingInputEl="textarea"/unifiedHeight/inputProps; itemUrl('agent', …) resolves to the existing /agents/get/[...path] route.

Test coverage

  • Backend (Rust) — the two new routes are thin proxies over the existing HubPublishCtx gate, and the folder-usage change is a one-line predicate swap with .sqlx caches regenerated to match. No new unit or proxy-integration test is warranted; nothing exercises the proxies against a Hub stub, which is acceptable since they only forward.
  • Frontend (Svelte/TS) — the new pure logic is well covered: projectBundle.test.ts (extraction/rewrite of MCP servers, linked agents, tool_inputs, brain fields, retargeting, no-duplicate-stub) and readAgentTurn.test.ts (multi-turn memory, background calls, structured output, web search). No Svelte component tests are expected per repo norms. The package-export and OpenAPI changes need no automated tests.

Manual verification still needed before merge: with a model credential, open "Record conversation" on an agent and send two messages (one using a background tool), save, and confirm the Hub page replays each turn with exactly its own tool calls; and install a project that ships an agent (ideally one with an MCP tool or a resource-backed input/schema) through the import wizard into a different folder, fill the generated stubs, and run it successfully. The PR body already flags both as untested, so these remain the only unexercised paths.

{turn.user_message}
</div>
</div>
{#if turn.tool_calls.length > 0}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request: draw the recorder with the same chat components as the Hub replay

The recorder draws the conversation with its own markup: bubbles, plus one chip per tool showing only its name. The Hub draws the saved recording with Windmill's read-only AI chat components (AIChatMessage, ToolGroupDisplay, readOnlyChatViewHost), which this PR exports for that purpose. That causes two problems:

  1. The person recording can't see what gets published. Each saved AgentTurn keeps every tool call's args and result, and on the Hub page a visitor can expand a tool call to read them. The recorder shows only the tool name. The component's doc comment says "the recorder sees the answers and the tools called before keeping it", but a tool result goes to the Hub without anyone having seen it. Tool results are the part most likely to carry data the publisher didn't mean to share, such as a customer record a lookup tool returned.
  2. The same conversation looks different on the two sides, so the recorder isn't a preview of the demo.

Suggestion: the components live in this repo, so AgentRecordSession can render them directly. The conversion from AgentTurn[] to DisplayMessage[] exists today only on the Hub side (AgentView.svelte in windmillhub). If it moves here, next to readAgentTurn, and gets exported, both sides share one mapping, and the recorder shows the same expandable tool calls a visitor will see.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ce668e3. The recorder now draws the conversation with a new recording/AgentRecordingTranscript.svelte (AIChatMessage, ToolGroupDisplay, readOnlyChatViewHost), so each tool call expands on its arguments and result before anything is saved. The AgentTurn[] to DisplayMessage[] mapping moved here as recording/agentRecording.ts; both are exported from the package. The Hub page (windmillhub#237) still has its own copy until the package release it is waiting on; it will switch to these two in the same bump.

@Guilhem-lm Guilhem-lm left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few more points from a review of the head commit. The first is a bug; the other three are cleanup requests.

// An agent is the one resource a project ships whole: its prompt, model and tools are the
// product, and it holds no credential, only `$res:` references to stubs created above.
// Never overwrites: a path collision is reported as a failed item, like a stub's.
function importAgent(workspace: string, a: ExportItem): Promise<unknown> {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Bug: install keeps the identity switches that publish strips.

On publish, fetchItem drops on_behalf_of, run_on_behalf_of and slack_anyone because they "belong to this workspace and never leave it". The Hub stores an agent's value whole (projects/[slug]/agents/+server.ts only runs the secret scan), and importAgent writes a.value as it comes. A project published through the Hub API, rather than through this screen, can therefore carry run_on_behalf_of: true and slack_anyone: true.

I checked the backend half locally: creating an ai_agent resource with {run_on_behalf_of: true, slack_anyone: true, on_behalf_of: "u/someone_else"} stores run_on_behalf_of: true, slack_anyone: true, on_behalf_of: "u/admin". resolve_agent_on_behalf_of drops the shipped identity and binds the agent to the installer, or to the folder's default when there is one. From then on, anyone who can read the agent runs its tools, inline code included, as the installer. With slack_anyone on, so can anyone in a Slack channel the agent is shared to.

Suggested fix: drop the same three keys in importAgent. Ideally one exported list would serve publish and install; the backend strips the same three keys in agent_runs.rs:211.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in ce668e3. importAgent now drops the three keys through withoutAgentWorkspaceKeys (in projectBundle.ts), the same helper publish uses, so there is one list. Thanks for checking the backend half.

}

/** The call a structured answer is given through: the answer itself, not a tool of the agent. */
const STRUCTURED_OUTPUT_CALL = /^structured_output(_\d+)?$/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Request: build readAgentTurn on parseAgentResult, and keep one copy of this regex.

  • STRUCTURED_OUTPUT_CALL is a verbatim copy of flows/conversations/flowChatViewHost.svelte.ts:255. If the worker changes how it names the structured-output call, one copy gets updated and the other doesn't.
  • readAgentTurn reads result.messages as any[]. $lib/components/aiAgentResult.ts already has parseAgentResult and the AgentMessage / AgentAction types for this result.
  • It's a pure function inside the 2000-line session module, so readAgentTurn.test.ts imports the whole session ($lib/gen, toast, stores) to test it.

Moving it next to parseAgentResult, built on top of it, with the regex exported once, would fix all three. The test would move with it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ce668e3. readAgentTurn lives in aiAgentResult.ts, built on parseAgentResult and the AgentMessage types; STRUCTURED_OUTPUT_CALL is exported from there and flowChatViewHost imports it. The tests moved into aiAgentResult.test.ts.

if (!v || typeof v !== 'object') continue
if (v.type === 'script' && typeof v.path === 'string') add('script', v.path)
if (v.type === 'flow' && typeof v.path === 'string') add('flow', v.path)
if (v.type === 'aiagent') {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Request: one place that recognises the agent shapes.

walkAgentConfig's doc says one walk serves both reading and rewriting "so the two cannot disagree". The flow path still recognises the same shapes by hand, twice: a linked agent, the MCP resource_path loop over v.tools, and enabled_tools. Here in extractFlowRefs, and again in rewriteFlowValue (around line 554). The two agree today, but this is the kind of read/rewrite pair the walk was meant to replace. Passing the step's agent part (agent, tools, tool_inputs) through collectAgentConfigRefs / rewriteAgentConfig would leave one definition of each shape.

Two smaller comment points in the same file:

  • The doc block at lines 109-132 runs to about 24 lines and sits on type AgentRefResolver rather than on walkAgentConfig. AGENTS.md asks for each invariant stated once, in at most 4 lines, where someone would break it.
  • extractAgentRefs says "see collectAgentConfigRefs", but collectAgentConfigRefs has no doc comment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ce668e3. extractFlowRefs and rewriteFlowValue now hand the step's agent part (agent, its MCP tools, tool_inputs, enabled_tools) to collectAgentConfigRefs / rewriteAgentConfig via agentStepConfig; module tools stay with the flow walk. The doc block is cut down and sits on walkAgentConfig, and extractAgentRefs points there.

Comment thread backend/windmill-api/src/hub_publish.rs Outdated
recording: Option<Option<serde_json::Value>>,
}

fn deserialize_explicit_recording<'de, D: Deserializer<'de>>(

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Request: make deserialize_explicit generic instead of copying it.

This copies deserialize_explicit (line 392) with only the type changed. A generic version serves both, and any later double-Option field:

fn deserialize_explicit<'de, T: Deserialize<'de>, D: Deserializer<'de>>(
    d: D,
) -> Result<Option<Option<T>>, D::Error> {
    Option::<T>::deserialize(d).map(Some)
}

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in ce668e3: deserialize_explicit is generic and serves both fields.

…agent walk

- importAgent drops on_behalf_of, run_on_behalf_of and slack_anyone, through
  the same helper as publish: an export written through the Hub API could
  otherwise bind the agent to whoever installs it.
- The recorder draws the conversation with AgentRecordingTranscript, the AI
  chat's read-only components, so tool arguments and results are seen before
  they are saved. The component and the turns-to-messages mapping are
  exported for the Hub page to use the same ones.
- readAgentTurn moves to aiAgentResult.ts on top of parseAgentResult;
  STRUCTURED_OUTPUT_CALL has one copy.
- The flow walker hands an aiagent step's agent part to walkAgentConfig
  instead of recognising the shapes itself; comments trimmed.
- deserialize_explicit is generic.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

Good to merge

The previous findings are addressed. I found no further confident bugs, security issues, performance regressions, or AGENTS.md violations at ce668e3.

Test coverage

All 77 projectBundle/readAgentTurn cases passed against pinned production source through an in-memory Node assertion harness, plus a shared transcript conversion smoke check. This was not Vitest. git diff --check passed. I did not run Rust tests or the application.

Frontend tests cover dependency extraction, relocation, bundling, identity-key removal, and turn parsing; component tests are unnecessary. No new Rust unit or proxy integration tests cover the backend changes. Package-export and OpenAPI configuration changes need no separate tests.

Manual verification remains: publish an agent with nested MCP tools, linked agents, resource-backed inputs/schema, and variables; install into another folder, configure its dependencies, and run it successfully.

With a model credential, record two turns including a background tool call, save, and reopen the Hub demo to verify answers, calls, arguments, and results. Confirm closing the drawer cancels an active run and an approval/question produces the cancellation message.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

Good to merge

I re-reviewed the full diff (base 3cc8b207 → head ce668e36), focusing on the final commit ce668e36 ("install strips identity keys, shared transcript, one agent walk") layered on top of the previously-cleared f12a0c744. All prior findings across the seven review rounds are resolved, and the final commit's refactor, security hardening, and new exported components hold up under close inspection.

Considered and dismissed

  • readAgentTurn now routes through the stricter parseAgentResult — the old version read result.output/result.messages loosely, so a result with an output key but a malformed envelope would previously yield text(output) while the new code falls back to text(whole result) with empty tool calls. Dismissed: the worker's AIAgentResult serializes exactly the five keys in ENVELOPE_KEYS (output, messages, wm_stream, reasoning, usage, optionals skipped — verified in backend/windmill-ai/src/types.rs:547), messages is always non-empty with string role, and readAgentTurn is only called on successful agent runs. The closed-signature design is parseAgentResult's own documented intent, not a regression.
  • agentStepConfig drops non-MCP tools from the walk — could script/flow tool paths inside an aiagent module's value.tools stop being rewritten? Dismissed: allFlowModules → getAllModules → getChildModuleBranches flattens value.tools.filter(isFlowModuleToolCandidate) (tool_type undefined/flowmodule) into modules handled by the main loop; only MCP tools (tool_type === 'mcp') are excluded from flattening, and those are exactly what agentStepConfig keeps (t?.value?.tool_type === 'mcp'). No path falls through.
  • walkAgentConfig now also walks an MCP tool entry's input_transforms (the old mcpResourcePath read only resource_path) — broader extraction/rewrite. Dismissed: the worker resolves $res: in static tool inputs via transform_json_unchecked, so whole-string refs there genuinely move with the bundle; only whole-string forms are matched, so nothing inert is relocated.
  • enabled_tools double-rewrite — fixEnabled relocates bare server names, then the generic input_transforms block at the bottom of rewriteFlowValue runs rewriteContent over them. Dismissed: the names are bare paths with no $res:/res:// token, so rewriteContent is a no-op and the second pass cannot double-apply the map.
  • withoutAgentWorkspaceKeys is a shallow spread — nested copies of the identity keys survive. Dismissed: on_behalf_of/run_on_behalf_of/slack_anyone are top-level keys on the ai_agent value (see AIAgentConfig in agentResourceUtils.ts and resolve_agent_on_behalf_of in resources.rs).
  • setChatViewHost at AgentRecordingTranscript init — no teardown. Dismissed: setChatViewHost is setContext, which is lifecycle-scoped to the component subtree and released on destroy; the component also sets untrustedTranscript: true so replayed agent output never renders executable markup.
  • Synthesized tool_call_id: ${i}-${j}`` in agentRecordingMessages — collision risk. Dismissed: unique within a single transcript, and `chatItemKey`/`groupToolRuns` key on it only within that array.
  • findLastIndex support — dismissed: already used in AIChatManager.svelte.ts:4847 and elsewhere in the codebase.
  • Generic deserialize_explicit — type inference from Option<Option<ProjectLogoInner>>/Option<Option<Value>> is correct; absent → None, null → Some(None), value → Some(Some(v)), preserving the documented double-Option semantics.
  • New package exports (AgentRecordingTranscript.svelte, agentRecording) — svelte-package generates the .d.ts and copies the .svelte; the pattern matches the existing FlowRecordingReplay.svelte export verbatim.

Test coverage

  • Backend: the only backend change is making a deserializer helper generic — a mechanical, behavior-preserving refactor; no new test warranted.
  • Frontend: the moved/added pure logic is covered — readAgentTurn's multi-turn, background-tool, structured-output and web-search cases in aiAgentResult.test.ts, and withoutAgentWorkspaceKeys in projectBundle.test.ts. The agentRecordingMessages mapping and the transcript component are display-only; Svelte components are not unit-tested in this codebase, so no component test is expected.
  • Config: the package-export changes need no automated test.

Manual verification still needed

Publish a folder containing an agent with a linked saved-agent tool, an MCP tool, a resource-backed input override and a $res: brain field, install it into a different folder, fill the generated stubs, and run it successfully — this exercises the whole-walk extraction/rewrite and the install-side identity scrubbing end to end. Separately, with a model credential, record a two-turn conversation that includes a background tool call, save it, and reopen the Hub demo to confirm each turn retains only its own tool calls and that closing the drawer cancels an in-flight run. Both paths are called out as untested in the PR body and remain the residual risk before merge.

The record drawer embeds FlowChat as the agent page's Chat tab does (new
conversation, no sidebar), so the recorder uses the same composer, transcript,
tool cards and approval/question handling. Saving reads each answered
message off its completed run with readAgentTurn; failed or unfinished runs
are left out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tristantr

Copy link
Copy Markdown
Contributor Author

/review

@claude

claude Bot commented Nov 18, 2026 •

Copy link
Copy Markdown
Contributor

Claude finished @tristantr's task in 2m 26s —— View job


cc @tristantr

Mergeable, but should ideally address nits: Save drops a still-running message without saying so, no memory-off warning in the recorder.

This pass focuses on 9a3ca7b ("Record an agent's demo in the agent's own chat"). Every earlier round was already resolved, and the earlier reviews cleared ce668e3.

Findings

P2: Save drops a message that is still being answered, without telling the recorder. deployToHubSession.svelte.ts:1898
The old drawer disabled Save while a turn was pending. Now sent = agentRuns.length counts every run that started, so Save stays enabled while the last message is still answering. getCompletedJob 404s for that run, so it is skipped and the save succeeds. Then DeployToHub.svelte:1138 closes the drawer. The Hub demo is missing its last turn and nothing says so. Either say how many messages were left out, or refuse to save while a run is still queued.

P2: no warning for an agent without managed memory. AgentRecordSession.svelte:42
The agent page hides this chat behind the agentChatGap(...).memory banner (AgentEditorHost.svelte:733-775), because such an agent answers each message without the earlier ones. The recorder embeds the same chat with no check. The recording then replays as one conversation even though each turn was answered on its own. it.value.input_transforms is available for the same agentChatGap check.

Checked in the new commit, no issue found

  • New conversation: FlowChatPool starts on its draft (selectedId undefined), and with hideSidebar there is no newChat/select entry point. So the demo cannot continue or mix in one of the recorder's real chats.
  • Same run as the agent page: runAgentTurn matches AgentEditorHost's view branch exactly: runAgent with memoryId: conversationId and {...inputs, user_message}. The path is agentChatPath(path), and the kind is the default deployed, as the agent page passes. A linked step ({type:'aiagent', agent: path}) is resolved by agentChatInputs.ts:290 for the model button.
  • Errors: a 409 or a failed runAgent throws before anything is pushed to agentRuns. {#key s.agentTarget.key} remounts the chat per agent. The #agentTok check keeps a late run from landing on another agent's recording.
  • Save: turns are read off the completed jobs in the order they were sent. Failed and cancelled turns are dropped, and Retry only exists for failed turns, so a retried message is not saved twice. Each turn keeps only user_message, readAgentTurn(result) and duration_ms, so the identity-free recording shape is unchanged. Approvals and questions are now answered in FlowChat, and a run that resumes and completes is saved normally.
  • Cleanup: the removed randomUUID, isSuspended and cancel code leave no dangling use. AgentRecordingTranscript is still exported for the Hub.

AGENTS.md compliance

I found no violation in the new commit. Comments state constraints, and the buttons use unifiedSize.

Test coverage

  • Frontend: this commit only rewires Svelte components and session state, and the pure logic it relies on (readAgentTurn) is still covered by aiAgentResult.test.ts. No new test is needed.
  • Backend: not touched by this commit.

Manual verification still needed: with a model key, open "Record conversation" on an agent and send two messages, one that calls a tool and one that triggers an approval. Answer the approval in the drawer. Each message should run in a new conversation, which later shows up in the agent page's chat list. Save once both have answered: the Hub page should replay both turns, each with only its own tool calls. Separately, send a message and press Save before it answers, and check what the recorder is told. As before, installing a project that ships an agent into another folder through the import wizard is still unexercised.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

cc @tristantr

Mergeable, but should ideally address nits: in-flight agent run is no longer cancelled when the record drawer closes.

The prior rounds are all resolved; the only new issue is in the final commit 9a3ca7bb ("Record an agent's demo in the agent's own chat"), which replaced the recorder's manual poll loop with FlowChat and dropped the explicit cancellation that earlier rounds had requested and f12a0c744 implemented.

Findings

P2 — closing the record drawer (or disposing the session) leaves an in-flight agent run running, orphaned from the recording — frontend/src/lib/components/workspaceSettings/deployToHubSession.svelte.ts:1858 (closeAgentRecord) and :303 (dispose)

closeAgentRecord and dispose now only bump #agentTok; the #cancelAgentRun() call (which invoked JobService.cancelQueuedJob) is gone. The run itself is started by FlowChat via runAgentTurn → JobService.runAgent with memoryId: conversationId. When the drawer closes, AgentRecordSession unmounts and FlowChat's $effect cleanup runs createdPool.destroy() → entry.chat.destroy(), which is #leaveConversation() in chat-sdk/src/chat.ts:634/1188: it aborts the client's stream-follow controller but never calls #api.cancelJob — that only happens in stop() (chat.ts:405). So the server-side job keeps running: it keeps calling real tools and spending tokens after the user has closed the recording UI.

It is also orphaned from the recording's point of view: openAgentRecord resets agentRuns = [] and the {#key s.agentTarget.key} remount creates a fresh FlowChat/conversation, so the in-flight run's result is never read back into saveAgentRecording (its job id is no longer tracked). If the agent parks on an approval/question and the user closes the drawer, the run sits suspended until the agent's own timeout with no recorder to answer it. Recommend restoring a cancel on close — e.g. keep the tracked job id from runAgentTurn and call cancelQueuedJob in closeAgentRecord/dispose (the agent page's chat can leave runs running because its conversation is resumable from the chat list; the recorder intentionally starts a fresh conversation each open, so a leaked run serves no one).

Considered and dismissed

  • Approval/question handling in the recorder — the synthetic aiagent module sets agent: path, so FlowChatInterface.mayRequestApproval (!!value.agent) stays true and the chat wires agentRequest → the user can answer approvals/questions live. This is the intended replacement for the old detect-and-cancel path, not a regression.
  • Attachments in the recorder — the synthetic module's input_transforms: {} means the paperclip likely isn't offered even for agents that take user_attachments. Dismissed as a minor feature gap, not a bug: the recorder is a minimal one-step stub by design, and this doesn't break anything.
  • sent={s.agentRuns.length} counts failed/in-flight messages while the footer says "answered messages" — cosmetic; saveAgentRecording already skips non-successful jobs and toasts "No answered message to save yet" when nothing is savable.
  • runAgentTurn/memoryId: conversationId and readAgentTurn(job.result) — mirror AgentEditorHost.runChatTurn exactly and were verified against AIAgentResult in prior rounds; no new concern.

Test coverage

  • Frontend: the pure logic (projectBundle/readAgentTurn) already has regression tests, which the prior rounds confirmed. The cancellation path is component lifecycle behavior, not covered by a test, and Svelte component tests are not expected here.
  • Backend: no backend change in this final commit; the two proxy routes and the count-query change were assessed in prior rounds (thin proxies over HubPublishCtx, .sqlx entries match).

Manual verification still needed: with a model credential, open "Record conversation" on an agent, send a message that runs for a while (or parks on an approval/question), then close the drawer before it completes and confirm the run is cancelled rather than continuing to completion in the background. Then complete a two-message recording end to end, save, and confirm the Hub demo replays each turn's own answer and tool calls.

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

cc @tristantr

Should address issues before merging: existing-resource setup breaks imported agents.

[P1] Existing-resource setup breaks imported agents — frontend/src/lib/components/workspaceSettings/projectInstall.ts:428

The new agent import leaves provider/tool references pointing at the project’s resource stubs, but setup’s “use an existing resource” operation never scans or rewrites agents. planRetarget visits only scripts, flows, apps and triggers. For an admin importing an agent with provider $res:f/mine/model, choosing an existing model credential reports success and deletes f/mine/model, while the agent retains that reference and can no longer run. I reproduced this against production source: no referrers or gaps, stubDeleted: true, and the agent’s provider unchanged. Include agents in reference discovery and rewriting before permitting stub deletion.

Test coverage

All 105 existing projectBundle/aiAgentResult cases passed through an in-memory Node harness using production source, plus a transcript conversion smoke check. This was not Vitest. The focused reproduction above and git diff --check passed. Frontend dependencies are absent; I did not run Rust tests or the application.

Frontend logic has regression coverage; add a case for reusing an imported agent’s credential. Component tests are unnecessary. The backend proxy/count changes have no new unit or integration tests. Package exports and OpenAPI configuration need no separate tests.

Before merge, import an agent project, reuse an existing provider/MCP resource during setup, and confirm the agent runs through the replacement. Also record two conversational turns, including a background tool call, then save and reopen the Hub demo to verify its answers and tool calls.

- "Use an existing resource" now sees saved agents: each ai_agent resource
  is read, those in the project have their resource references moved
  (provider, MCP servers, tool inputs) with the run identity preserved, and
  any reference left unmoved, outside the project or unreadable keeps the
  stub. Before, the stub could be deleted under an agent still reading it.
- Closing the record drawer or disposing the session cancels the run in
  flight, which the chat stops following but does not stop.
- Saving waits for the message being answered instead of leaving it out,
  and says how many failed messages were left out.
- The recorder warns when the agent keeps no managed memory.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@tristantr

Copy link
Copy Markdown
Contributor Author

Addressed in c3426ba:

  • Existing-resource setup breaks imported agents (Codex P1): planRetarget now reads every ai_agent resource. Agents in the project have their resource references moved (rewriteAgentResourceRefs: provider, MCP servers, tool inputs; never a script sharing the path) with preserve_on_behalf_of; an agent outside the project, unreadable, or still naming the path keeps the stub. Three regression tests in retargetDeployed.test.ts.
  • Run left running when the drawer closes (Pi P2): closing the drawer or disposing the session cancels the last run.
  • Save drops a message still being answered / no memory warning (Claude P2s): fixed, see the threads.

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

cc @tristantr

Should address issues before merging: resource reuse disables nested MCP tools; closing during run submission leaves the job running.

[P1] Resource reuse disables nested MCP tools — frontend/src/lib/components/workspaceSettings/projectBundle.ts:612

The new helper calls walkAgentConfig directly, bypassing the enabled_tools relocation in rewriteAgentConfig. I reproduced existing-resource setup changing a nested agent’s MCP resource_path to f/shared/mcp while leaving enabled_tools as ['f/proj/mcp']. The worker matches those names exactly, so it removes the server from the tool roster. Retaining the stub does not prevent this breakage. Relocate the MCP selection alongside its resource reference.

[P1] Closing during run submission leaves the job running — frontend/src/lib/components/workspaceSettings/deployToHubSession.svelte.ts:1880

Cancellation only sees IDs already in agentRuns. If the drawer closes while runAgent awaits its response, that list has no current ID; the changed token then prevents runAgentTurn from retaining the returned ID. I reproduced zero cancellation calls and an empty tracked-run list after resolving the delayed response. FlowChat destruction only stops following the job, so it continues executing tools. Cancel the returned job when its recording token has become stale.

Test coverage

All 128 existing projectBundle, aiAgentResult, and retargetDeployed cases passed through an in-memory Node assertion harness using production source; this was not Vitest. Focused checks reproduced both findings and confirmed cancellation works once the ID is known. git diff --check passed.

Frontend coverage misses the two cases above; component tests are unnecessary. The backend proxy/count changes have no new unit or integration tests. Package-export and OpenAPI configuration changes need no separate unit tests. Frontend dependencies and application servers were unavailable; I did not run Vitest, Rust tests, or the application.

Import an agent project into another folder, select an existing MCP resource during setup, and confirm a nested agent with an explicit tool selection can still invoke that server.

With a model credential, record two turns including a background tool call and an approval, then save and reopen the Hub demo. Also delay the run-start response and close the drawer before it returns; verify the resulting job is cancelled.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

Good to merge

I focused on the latest commit c3426baf4a ("Retarget imported agents onto an existing resource; recorder guards"), which closes out the Codex P1 (existing-resource setup breaks imported agents), my prior P2 (orphaned in-flight run on drawer close), and the two Claude P2s (save dropping an in-flight message, no managed-memory warning). All four are implemented correctly, and the new retarget path holds up under close inspection.

Findings

None.

Considered and dismissed

  • #cancelAgentRun still has a narrow race — runAgentTurn pushes the job id to agentRuns only after JobService.runAgent resolves, so closing the drawer inside that single HTTP round-trip leaves the job untracked and uncancelled. Dismissed: the window is one POST round-trip (~tens of ms) and the fix correctly cancels the last tracked run in every other case (including a run parked on approval/question, the original concern). Tracking the in-flight promise to close that sliver is not warranted.
  • rewriteAgentResourceRefs vs extractAgentRefs consistency — verified both delegate to the same walkAgentConfig, so every ref agentReadsResource detects is exactly what the rewriter moves; no divergence could strand a $res: reference behind a deleted stub.
  • AGENT_LIST_LIMIT = 1000 — verified list_resources goes through paginate, which defaults to DEFAULT_PER_PAGE = 1000 (windmill-common/src/utils.rs), so a full page is correctly treated as a truncation gap.
  • preserve_on_behalf_of honored on agent writes — verified update_resource reads ns.preserve_on_behalf_of and routes the value through resolve_agent_on_behalf_of, so rewriteAgent's { ...PRESERVE_DEPLOYED_IDENTITY, value: next } keeps the agent's run identity.
  • enabled_tools not rewritten by rewriteAgentResourceRefs — dismissed: enabled_tools is flow-local (AGENT_FLOW_LOCAL_KEYS in agentResourceUtils.ts) and never saved into the ai_agent resource value, so it cannot appear in the retargeted value.
  • agentNamesPathUnreachably first rewrites bare MCP/aiagent links to '' before the bare-path scan — this is necessary and correct, since bareResourcePath treats those bare spellings as movable resource refs (unlike a flow's runnable paths); a bare script/flow tool path sharing the stub's name is not moved and is correctly flagged unreachable, keeping the stub (pinned by the first new test).
  • agentForgets .catch(() => {}) and the forgets warning wording — a failed getResourceValue merely suppresses a non-critical warning, and the "turn on managed memory" hint doesn't distinguish manual (legacy) memory; both cosmetic, not bugs.
  • saveAgentRecording early return leaving agentSaving stuck — dismissed: the finally block resets agentSaving on every return path.
  • Mid-sentence $res: tokens in agent prose — dismissed: the worker resolves only whole-string values (transform_json_unchecked), and textHoldsBarePath strips tokens before the bare scan, so inert prose tokens are neither extracted nor mis-flagged.
  • N+1 getResourceValue reads for agents — dismissed: bounded by the ≤1000 listing cap and runs only on the explicit "use an existing resource" action.
  • RLS-filtered agent listing for non-admin callers — dismissed: seesWholeWorkspace already pushes the UNSEEN_BY_CALLER gap for non-admin/non-superadmin, so the stub is never deleted under an incomplete scan.

Test coverage

  • Frontend: the three new retargetDeployed.test.ts cases pin the agent scenarios (provider + MCP moved with preserve_on_behalf_of, stub dropped once the only reader moves, stub kept for an out-of-project reader), and the existing projectBundle/aiAgentResult suites still cover the shared walk/rewrite logic. The recorder guards (#cancelAgentRun, save-wait, memory warning) are Svelte session/component lifecycle behavior; component tests are not expected here.
  • Backend: not touched by this commit; the two proxy routes and the count-query change were assessed in prior rounds.
  • CI/config/OpenAPI/package-exports: no automated tests expected.

Manual verification still needed

With a model credential, open "Record conversation" on an agent with managed memory, send two messages (including one that calls a tool and one that parks on an approval), answer the approval, then save: the Hub demo should replay both turns with only their own tool calls, and closing the drawer mid-run should stop the run (no further tool calls/token spend). Separately, import a project that ships an agent using a provider and an MCP tool, choose "use an existing resource" during setup, and confirm the agent's references point at the replacement and a successful run. These paths remain unexercised end-to-end, as the PR body notes.

- rewriteAgentResourceRefs goes through rewriteAgentConfig, so the
  enabled_tools entry naming an MCP server follows it when an existing
  resource replaces the stub. The worker matches names exactly and dropped
  the server otherwise.
- A run whose id comes back after the record drawer closed is cancelled:
  it was not tracked yet when the close looked for a run to stop.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@tristantr

Copy link
Copy Markdown
Contributor Author

Both Codex P1s addressed in 3e08ea3:

  • Resource reuse disables nested MCP tools: rewriteAgentResourceRefs now goes through rewriteAgentConfig (resource refs only), so enabled_tools follows the MCP server. Regression test in retargetDeployed.test.ts.
  • Closing during run submission: runAgentTurn cancels the job it gets back when the recording token has gone stale.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

Good to merge

I reviewed the full diff (base 3cc8b207 → head 3e08ea3f1e), focusing on the final commit 3e08ea3f1e ("Move enabled_tools with a reused MCP resource; cancel a late run"), which closes out the two Codex P1s from the previous round. Both fixes are correct, and a fresh pass over the whole PR found no remaining functional, security, performance, or AGENTS.md issues.

The two latest fixes, verified

  • rewriteAgentResourceRefs → rewriteAgentConfig(value, map, 'resource') (projectBundle.ts:611) — the only filter restricts walkAgentConfig to resource refs while fixEnabled still runs unconditionally over the output, so an enabled_tools entry naming a relocated MCP server follows it. I confirmed against the worker (ai_executor.rs:616-618): tool_enabled_name matches enabled_tools entries against resource_path.trim_start_matches("$res:"), so bare-name entries must move with the server or the roster silently drops it. The new regression test (retargetDeployed.test.ts) pins both resource_path and enabled_tools relocation plus stubDeleted: true.

  • runAgentTurn cancels a late-returning job id (deployToHubSession.svelte.ts:1904) — when runAgent resolves after #agentTok has been bumped (drawer closed or session disposed), the returned id is cancelled instead of being tracked. #agentTok is bumped only in openAgentRecord, closeAgentRecord, and dispose — all three mean the recording context is gone, so cancelling is always the right action. The stale-token branch also still returns jobId, but the consuming FlowChat is destroyed at that point and the job is cancelled, so nothing follows a dead run.

Considered and dismissed

  • fixEnabled rewrites every enabled_tools entry matching a map key — could it repoint a non-MCP tool name? Dismissed: map keys are resource paths (f/.../u/...), and tool names are function identifiers; no plausible collision. This is also pre-existing behavior, only newly routed through the retarget path.
  • fixEnabled reaching agentNamesPathUnreachably changes the unreachable check — before, an enabled_tools entry naming the stub kept it flagged after the MCP resource_path moved; now it follows and the stub can be deleted. Dismissed: correct. The worker treats enabled_tools as a name filter against the (relocated) resource_path, so an entry that follows the server is no longer a dangling reference; an entry naming a path with no matching MCP tool is inert and never read the resource anyway.
  • Prefixed $res: spellings in enabled_tools are not rewritten by fixEnabled — map.get('$res:f/proj/mcp') misses. Dismissed: the worker compares against the bare path (trim_start_matches("$res:")), so a prefixed entry never matched even before relocation; this is a pre-existing worker-side mismatch, not introduced here.
  • #cancelAgentRun() defaults to agentRuns.at(-1) (last run only) — the comment documents that the chat sends one message at a time, so only the last can be running; a finished-run cancel is refused server-side and caught by .catch(() => {}).
  • openAgentRecord calls #cancelAgentRun() before resetting agentRuns = [] — ordering verified: it reads the previous recording's list before clearing, so a still-running turn from the prior recording is cancelled on reopen.
  • AgentRecordingTranscript renders replay output — dismissed: readOnlyChatViewHost sets untrustedTranscript: true (readOnlyChatViewHost.svelte.ts), so replayed agent output never renders executable markup; setChatViewHost is setContext and released on component destroy.
  • Backend publish_agents full-set sync clearing the Hub's agents on an empty list — dismissed: the silent .catch(() => []) was removed in an earlier round; a failed agent list now rejects the whole #loadWorkspace like the other four lists, so an empty set cannot be synced on a transient failure.
  • Folder usage counts (folders.rs) — resource_type = 'ai_agent' vs IS DISTINCT FROM 'ai_agent' are disjoint and complete; a NULL-typed resource still counts as a resource, and the two totals sum correctly. The .sqlx cache matches the two changed queries.
  • Folder table column alignment — header now has 11 columns (Name, Labels, Scripts, Flows, Apps, Agents, Schedules, Variables, Resources, Members, Actions) and colspan="11" on the skeleton row; FolderUsageInfo's order gained agents, matching the 7 usage cells. Consistent.

Test coverage

  • Frontend — the two latest fixes each have a dedicated regression test: nested-agent enabled_tools relocation (retargetDeployed.test.ts) and the existing projectBundle/aiAgentResult suites still cover the shared walk/rewrite and turn-parsing logic. The recorder guards (#cancelAgentRun, save-wait, memory warning) are Svelte session/component lifecycle behavior; component tests are not expected in this codebase.
  • Backend — the only changes are two thin proxy routes over HubPublishCtx (admin + folder scope enforced, ProjectSlug validated on deserialize) and a folder-count query with matching .sqlx entries. No new unit or integration tests are added, which is proportionate for pass-through routes.
  • Config/exports — the package-export additions and OpenAPI entries need no automated tests; the export paths were checked against the actual files.

Manual verification still needed: with a model credential, record two conversational turns including a background tool call and an approval/question, answer it in the drawer, save, and reopen the Hub demo to confirm each turn keeps only its own answer and tool calls. Separately, import an agent project into another folder, choose an existing MCP resource during setup, and confirm a nested agent with an explicit tool selection still invokes that server. Also delay the run-start response and close the drawer before it returns, verifying the resulting job is cancelled (the fix just added for that path). Installing a project that ships an agent through the import wizard remains unexercised.

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

Good to merge

Both findings from the previous Codex review are addressed. I found no new confident bugs, security issues, performance regressions, or AGENTS.md violations at 3e08ea3f1e.

Test coverage

All 72 projectBundle and 24 retargetDeployed cases passed against pinned production source through an in-memory Node assertion harness; this was not Vitest. Four focused recorder lifecycle checks also passed, covering late submission, known-run cancellation, agent switching, and missing targets. git diff --check passed.

The new regression covers nested MCP selection during resource reuse. Svelte component tests are unnecessary. The backend proxy/count changes have no new Rust unit or integration tests. Package exports and OpenAPI configuration need no separate tests. Frontend dependencies were unavailable; I did not run Vitest, Rust tests, or the application.

Manual verification remains: publish an agent project, import it into another folder, select an existing MCP resource during setup, and confirm a nested agent with an explicit tool selection can invoke that server.

With a model credential, record two turns including a background tool call and an approval, answer the approval, then save and reopen the Hub demo. Confirm both turns retain their answers and tool calls. Delay a run-start response and close the drawer before it returns; verify the resulting job is cancelled.

@github-actions

Copy link
Copy Markdown
Contributor

Pi Review

Good to merge

The head 7d62eb4120 is a clean merge of main into the previously-cleared 3e08ea3f1e. I re-verified the merge itself and did a fresh pass over the full diff; nothing new was introduced, and every prior finding remains resolved.

Considered and dismissed

  • Merge drift — the merge's second parent is the base fb13514e50, so git diff fb13514e50...7d62eb4120 is exactly the PR's own work. I checked whether any of the PR's 24 files were modified on main between the merge-base 3cc8b207 and fb13514e50: none were. The only PR-touched file the merge changed is frontend/package.json, and only version bumps (1.876.0 → 1.880.1, windmill-parser-wasm-csharp/ts → 1.880.1); the PR's new exports entries (AIChatMessage, ToolGroupDisplay, ChatTypingIndicator, chatViewHost, readOnlyChatViewHost, toolGroups, AgentRecordingTranscript, agentRecording) are all intact in the merged result. No conflict markers; git diff --check is clean.
  • Dependency drift — main did not touch any file the new code imports (flowExplorer.ts, the exported copilot chat components, recording/, aiAgentResult.ts, importWizard/), so the merged tree has no silent API break against the PR's code.
  • Backend routes (hub_publish.rs) — publish_agents / publish_agent_recording go through HubPublishCtx (require_admin + folder scope), ProjectSlug validates on deserialization, and the double-Option AgentRecordingBody deserializer preserves absent → 400 / null → clear / object → set semantics. No unvalidated input is interpolated into a Hub URL.
  • Folder usage counts (folders.rs) — the resource_type = 'ai_agent' count and the resource_type IS DISTINCT FROM 'ai_agent' resource count are disjoint; a NULL-typed resource still counts as a resource and the totals add up. The .sqlx cache changes (one new query entry, one re-hashed rename) correspond to the two changed queries.
  • walkAgentConfig / rewriteAgentConfig / rewriteAgentResourceRefs — single walk over the agent value matches the worker's transform_json_unchecked (whole-string $res: only, recursion into objects/arrays); fixEnabled relocates enabled_tools with a moved MCP server, and the resource-reuse path routes through rewriteAgentConfig(value, map, 'resource') so both resource_path and enabled_tools move together.
  • Recorder cancellation races — #agentTok is bumped only in openAgentRecord/closeAgentRecord/dispose, and runAgentTurn cancels a job id that arrives after the token has gone stale, so a run submitted while the drawer closes is cancelled rather than leaked.

Test coverage

  • Frontend — the pure logic is pinned by projectBundle.test.ts, aiAgentResult.test.ts, and retargetDeployed.test.ts (dependency extraction, relocation, bundling, turn parsing, resource-reuse retargeting). The recorder save/cancel/memory guards are Svelte session/component lifecycle behavior, for which component tests are not expected here.
  • Backend — the two proxy routes and the folder-count query have no new Rust unit or integration tests; they are thin proxies over HubPublishCtx and a scalar count query whose .sqlx cache matches. Package-export and OpenAPI changes need no separate tests.

Manual verification still outstanding (not blocking, and unchanged from the last round): with a model credential, record two conversational turns including a background tool call and an approval, save, and reopen the Hub demo to confirm each turn retains its own answer and tool calls; and import an agent project into another folder, select an existing MCP/provider resource during setup, and confirm the agent runs through the relocated references.

@github-actions

Copy link
Copy Markdown
Contributor

Codex Review

Good to merge

No new confident bugs, security issues, performance regressions, or AGENTS.md violations found at 7d62eb4120. The latest commit cleanly merges main; the feature implementation and regression tests are unchanged from the previously approved head. The package, API schema, and shared chat changes remain compatible.

Test coverage

Frontend pure-logic tests cover agent dependency extraction, relocation, identity stripping, resource reuse, and conversation parsing. Component tests are unnecessary. The focused Vitest command could not run because frontend dependencies are absent (vitest: not found).

git diff --check and package export/type-mapping checks passed. The backend proxy routes and folder-count changes have no new Rust unit or integration tests; I did not run Rust tests. Package-export and OpenAPI configuration changes need no separate automated tests. No application servers were running, so application behavior was not exercised.

Before merge, publish an agent project, import it into another folder, reuse an existing provider/MCP resource during setup, and confirm nested agents retain their selected MCP tools and execute successfully.

With a model credential, record two turns including a background tool call and an approval, answer the approval, then save and reopen the Hub demo. Confirm both turns retain their answers and calls. Also close the drawer while a run-start response is delayed and confirm the returned job is cancelled.

@rubenfiszel
rubenfiszel merged commit 4fd4b04 into main Nov 20, 2026
21 checks passed
@rubenfiszel
rubenfiszel deleted the tl/hub-publish-agents branch November 20, 2026 15:37
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants