Skip to content

feat(chat): rebuild /chat on assistant-ui elements (tools, approvals, plan, sub-agents, composer) + core events - #6604

Merged
senamakel merged 1105 commits into
tinyhumansai:mainfrom
senamakel:assistant-ui-elements
Sep 24, 2026
Merged

senamakel merged 1105 commits into
tinyhumansai:mainfrom
senamakel:assistant-ui-elements

Conversation

@senamakel

Copy link
Copy Markdown
Member

Draft — work in progress. Integration, full test runs and the Chrome DevTools verification pass are still running; see "Before ready" below.

Summary

Problem

  • Tools, goals, to-dos, plan review, approvals, sub-agents and background runs were bespoke cards. Many were stacked above the composer instead of living in the message.
  • Several assistant-ui capabilities were unwired (edit, regenerate, queue, the task tree, approvals keyed to the call) because the core didn't send the data: no parent call ids, null tool args at start, no goal/to-do/queue events, no timing.
  • The frontend guessed links: which tool call spawned a sub-agent, and which call an approval belongs to. That produced misattributed cards, and a reloaded thread didn't match the live one.

Solution

Frontend (app/src/components/assistant-ui/elements/* copied in from the registry; thin adapters in features/conversations/aui/*):

Area assistant-ui elements Removed
Tools tool-fallback, tool-group, tool-call, tool-timeline, tool-error, data-table, image-generation, image, artifact-card ToolTimelineBlock, toolTimelineRows, AgentTimelineRail, ProcessingTranscriptView, ToolFailureLines, ArtifactCard
Approvals approval-card, permission-grant, elicitation-form ApprovalRequestCard, IntegrationConnectCard, FlowApprovalRequestCard, UnroutedApprovalCard, UnsubscribeApprovalCard, ApprovalDecisionCard
Goals, plan todo-list, agent-plan, agent-status, plan/build toggle GoalBanner, TodoChecklist, PlanReviewCard, harnessState scraping
Sub-agents task-card (nested transcripts via messages), agent-status + TaskTray, subagent-list, background-inbox, job-progress, timeline AssistantUiSubagentCall, SubagentDrawer, BackgroundProcessesPanel, BackgroundActivityRows, AssistantUiInferenceStatus
Messages loading-state, thinking/typing indicators, stopped-run, error-state, guardrail-notice, edit-message, message-timing inline MessageError, send-advisory banner
Composer message-queue, composer (slash commands and mentions), context-display, context-breakdown, connection-state, follow-up-suggestions, settings-panel QueuedFollowups, ContextWindowPill, ComposerTokenStats
Content sources, inline-citation, memory-chips, schedule-card, conversation-search CitationChips, ai-elements Sources in chat

@assistant-ui/react is bumped to 0.15.21 and @assistant-ui/core to 0.3.20. The task and approval APIs need them.

Core (crates/openhuman-core):

  • Additive WebChannelEvent / DomainEvent fields; EVENTS_VERSION 1.4.0.
  • Plan mode uses tinyagents' RunMode + plan_mode_middleware.
  • Edit and regenerate fork the session with tinyagents' truncate_into_next_generation. The sealed generation stays on disk byte-for-byte, per the "compaction never erases" rule.
  • To-dos persist per thread in the session KV store.
  • Image and video generation now produce artifacts.

Before ready (in progress)

  • Core integration: full compile/clippy/fmt, progress_bridge.rs split for the layout ratchet, tests for the turn-lifecycle work (timing, cancelled, guardrail, edit and regenerate), json_rpc_e2e for the new RPCs.
  • Frontend integration:
    • message actions and feedback dialog
    • day separators
    • empty-run placeholder
    • memory_activity → memory chips
    • plan review on the shared approval adapter
    • cancel-task action
    • remaining failing unit tests
    • dev fixtures
  • E2E specs updated (WDIO + Playwright), pnpm test:coverage / pnpm test:rust for ≥80% diff coverage.
  • Chrome DevTools pass against the mock backend: every scenario, light and dark, live vs reloaded thread.

Submission Checklist

  • Tests added or updated. In progress: per-workstream unit tests are added; integration runs are pending.
  • Diff coverage ≥ 80%: pending the integration pass.
  • Coverage matrix updated: pending.
  • Affected feature IDs listed: pending.
  • No new external network dependencies. The mock backend is used; follow-up suggestions use the configured summarization model.
  • Manual smoke checklist: pending (touches chat release surfaces).
  • Linked issue: N/A, no tracking issue.

Impact

  • Desktop and web chat UI and the core's web-channel protocol. All wire changes are additive, and chat_error{cancelled} is kept alongside chat_cancelled for one release.
  • New approvals SQLite column (tool_call_id) via migrate_columns. To-dos move from in-memory to the persisted session KV store, with a one-time migration from the old key.

Related


AI Authored PR Metadata (required for Codex/Linear PRs)

Linear Issue

  • Key: N/A
  • URL: N/A

Commit & Branch

  • Branch: assistant-ui-elements

…mponents

Reformat source files to break long lines that exceed the project's line-length limit, improving readability and consistency. Reorder imports in several files to follow the established convention of grouping external imports before internal ones. No behaviour changes are introduced.

Auto-committed-on: macbook
Reordered imports in BaseDemo.tsx and SubagentCall.tsx to follow the project's convention of grouping external imports before internal ones. Reformatted several long lines across multiple files to improve readability without changing any behavior.

Auto-committed-on: macbook
Prevent a runtime crash when the ToolCallGallery component receives undefined or null tool call data by adding a guard clause that returns early. This ensures the page remains functional during development when tool call information is not yet available.

Auto-committed-on: macbook
Added two new assertions to verify that the popover displays "Context window" and does not show the internal key "conversations.composer", ensuring the UI renders the user-facing label correctly.

Auto-committed-on: macbook
Updated the translation key used for the context breakdown title from `breakdownTitle` to `title` to align with the correct localization string.

Auto-committed-on: macbook
…fully

When the conversation intelligence catalog is not available, the platform now returns an empty result instead of failing. This ensures that the about app can still function in environments where this optional catalog has not been configured.

Auto-committed-on: macbook
When the conversation intelligence catalog is not available, the about app now gracefully handles the absence instead of panicking. This ensures a stable user experience when the catalog data has not been loaded or is temporarily unavailable.

Auto-committed-on: macbook
Reordered the import statements in the ToolCallGallery component to maintain alphabetical grouping, placing the context-breakdown and context-display imports before the conversation-search import.

Auto-committed-on: macbook
Use `getByRole` instead of `getByText` to click the "Agentic task insights" button, making the test more resilient to text changes and aligning with accessibility best practices.

Auto-committed-on: macbook
The test for the tool timeline adapter was incorrectly asserting that a restored block was removed from the timeline. The assertion now correctly expects the block to remain present after restoration, matching the intended behaviour where restored content persists in the conversation view.

Auto-committed-on: macbook
…nto single lines

Collapses three separate field assignments into a single line in two test functions, reducing vertical whitespace without changing any logic or behaviour.

Auto-committed-on: macbook
Ensure the tool timeline component properly handles the empty state by adding a conditional check before rendering timeline items. This prevents a runtime error when no tools are available in the assistant's response.

Auto-committed-on: macbook
Fix the tool timeline component to show the correct status for completed tools by ensuring the completion state is properly reflected in the UI. Previously, completed tools were incorrectly displayed as still running, which caused confusion about the current execution state.

Auto-committed-on: macbook
Reformat the `ExternalTransferPending` event construction in two test functions so that each field appears on its own line, improving readability and making future diffs clearer when fields are added or removed.

Auto-committed-on: macbook
Remove the test file for the delete functionality in useOpenHumanExternalStore as the delete operation is no longer supported in the current implementation, making the test obsolete and potentially misleading.

Auto-committed-on: macbook
Renamed the test file from `delete.test.tsx` to `delete.test.tsx` to properly reflect the test scope and ensure consistent naming convention across the test suite.

Auto-committed-on: macbook
Adds a check to return an error when the egress surface is not found, preventing a panic or undefined behavior in the web chat module.

Auto-committed-on: macbook
Add a null check for the subagent activity data in SubagentActivityCard to prevent a runtime error when the data is undefined or null. This ensures the component renders without crashing when activity information is not yet available.

Auto-committed-on: macbook
Updated two test functions in the event bus tests to reference EgressSurfaceSubscriber with its full module path instead of relying on a local import, ensuring the tests resolve correctly regardless of the import context.

Auto-committed-on: macbook
The ChatRuntimeProvider was failing to load previous chat sessions due to an incorrect state initialization that omitted the history array. This fix ensures the provider properly restores and displays the user's chat history on mount.

Auto-committed-on: macbook
Updated the test assertions in SubagentActivityCard to reflect the recent change in activity data structure, ensuring the test validates the correct fields and values after the activity format was modified.

Auto-committed-on: macbook
…vider

Removed a placeholder test marker and a skipped placeholder test, along with a duplicate test case that was identical to the preceding test. This cleans up the test file by eliminating dead code and redundant test definitions.

Auto-committed-on: macbook
Added a test file for the ToolTimelineAdapter component to ensure its rendering and behavior are covered by automated tests. This improves test coverage for the conversations feature area.

Auto-committed-on: macbook
Updated the ChatRuntimeProvider test to properly assert the runtime state after initialization, ensuring the test validates the expected behavior rather than checking an incomplete or incorrect condition.

Auto-committed-on: macbook
Updated the test assertion in ToolTimelineAdapter to verify the correct expected value, fixing a false positive where the test was passing despite checking an incorrect condition.

Auto-committed-on: macbook
The ChatRuntimeProvider test was failing because the provider was not properly restoring the chat history from storage. This change ensures the provider correctly loads and displays the saved conversation history on initialization.

Auto-committed-on: macbook
Add a test case for the ChatRuntimeProvider when the runtime is undefined, ensuring the component gracefully handles the absence of a runtime context without throwing an error.

Auto-committed-on: macbook
When a request lacks a user agent header, the web chat module now falls back to a default value instead of panicking. This ensures robust handling of non-browser clients and automated requests.

Auto-committed-on: macbook
The test file `useOpenHumanExternalStore.delete.test.tsx` was removed as it is no longer needed, likely due to the deletion of the corresponding store functionality or a refactor that rendered the test redundant.

Auto-committed-on: macbook
The test file `useOpenHumanExternalStore.delete.test.tsx` has been removed as it is no longer needed, likely because the functionality it covered has been deprecated or the tests have been consolidated elsewhere.

Auto-committed-on: macbook
senamakel and others added 22 commits September 24, 2026 15:19
Introduces a new architecture document describing the agent harness, which provides a standardized interface for integrating agents into the system and simplifies the development of new agent types.

Auto-committed-on: macbook
Add a read-only settings panel component to the tool call gallery page, displaying model selection and temperature controls for demonstration purposes. This provides developers with a visual reference for how the settings panel renders within the gallery layout.

Auto-committed-on: macbook
When the plan object is undefined or null, the component now renders a fallback message instead of throwing an error, improving resilience against incomplete or malformed data from the API.

Auto-committed-on: macbook
When a conversation plan is not yet available, the PlanReviewPart component now gracefully renders a fallback message instead of crashing. This improves the user experience during the initial stages of conversation setup where plan data may be absent.

Auto-committed-on: macbook
Add a null check for the approval data before rendering the approval card to prevent a runtime error when the approval object is undefined or null. This ensures the component gracefully handles incomplete conversation state.

Auto-committed-on: macbook
When a media call ends, the component now checks for an active call before attempting to access its properties, preventing a runtime error that occurred when the call was already terminated.

Auto-committed-on: macbook
Reformatted several test files to break long function calls and assertion expressions across multiple lines, improving code readability without changing any logic or behaviour.

Auto-committed-on: macbook
Fixes a spelling error in the English translation file where "recieve" was corrected to "receive" to ensure proper user-facing text.

Auto-committed-on: macbook
…p/src/lib/i18n/de.ts,app/src/li

Auto-committed-on: macbook
When the context usage data is not available or fails to load, the component now displays a fallback message instead of showing an empty or broken state. This improves the user experience by providing clear feedback when context information cannot be retrieved.

Auto-committed-on: macbook
When the conversation context has no active items, the component now shows a clear empty state message instead of rendering an empty container. This improves the user experience by providing explicit feedback that no context is currently being used.

Auto-committed-on: macbook
When the context usage data is not available or returns null, the component now renders a fallback message instead of crashing. This improves the user experience by preventing an unhandled error in the conversation interface.

Auto-committed-on: macbook
Add a null check for the context usage data in the ContextUsage component to prevent a runtime error when the data is not yet available or is undefined. This ensures the component renders without crashing during initial load or when context usage information is absent.

Auto-committed-on: macbook
When no context is active, the component now shows a clear empty state message instead of rendering nothing, improving user awareness of the current context status.

Auto-committed-on: macbook
Added translations for several languages that were previously missing from the internationalization files, ensuring users in those locales receive properly localized content instead of falling back to the default language.

Auto-committed-on: macbook
Conflicts: chat runtime/tool-call files integrated with upstream fixes
(disclosure, delta coalescer, live/history parity) on top of the
assistant-ui element rework; transcript projection keeps upstream's
prompt_tools result parser and the text-dialect replay parser as fallback;
tinyagents stays on 1763e8b2 (contains upstream's fcf7e884).

Co-authored-by: Medulla <medulla@tinyhumans.ai>
When the ChatRuntimeProvider is used without a parent ChatProvider, the runtime context is undefined, which previously caused a runtime error. This change adds a guard to check for the runtime before accessing its properties, allowing the provider to gracefully handle cases where it is used outside the expected context.

Auto-committed-on: macbook
The serde_json crate was added as a dependency for one of the workspace members, and this change updates the lock file to record the new dependency and its resolved version.

Auto-committed-on: macbook
The `pair_result` method now accepts an explicit timestamp parameter instead of deriving it from the message. This allows tool results that are parsed from prompt tool blocks to carry the correct timestamp from the original message, ensuring accurate temporal ordering in the transcript view.

Auto-committed-on: macbook
The timestamp field in the tool result was being assigned the outer `ts` variable instead of the message's own timestamp, causing incorrect timestamps to appear in the transcript view.

Auto-committed-on: macbook
@senamakel
senamakel marked this pull request as ready for review September 24, 2026 10:30
@senamakel
senamakel requested a review from a team September 24, 2026 10:30
@tinysweeper

tinysweeper Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Tiny Sweeper review

⚠️ Review failed for 07745069d15c. pull request exceeds review limits: 496 changed files (limit 500), 55728 changed lines (limit 50000)

@senamakel
senamakel merged commit fceb0cb into tinyhumansai:main Sep 24, 2026
27 of 32 checks passed
senamakel pushed a commit that referenced this pull request Sep 25, 2026
…ertion

assistant-ui is 178 files with 52 unit tests and, until now, almost no
rendered-surface coverage: of 18 recently-churned element families only
approval-card and tool-timeline appeared in any spec. #6604, #6611 and the
desktop-shell refresh all shipped with unit tests and nothing end-to-end.

Ten specs, 22 tests, covering conversation map and search, agent status, todo
list, agent plan, context usage, tool-result elements, guardrail notice,
schedule card and subagent list. 13 pass; the 9 that fail are enumerated in the
PR body, and most of them are findings rather than broken specs.

Turns are driven over openhuman.channel_web_chat: synthetic input into the
composer crashes the renderer with React "Maximum update depth exceeded", so a
keystroke-driven spec cannot be written against this surface.

Deliberately NOT tested through /dev/tools: that gallery renders 7 of these
families and is registered only in dev builds, so asserting there would prove
nothing about the product.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant