Skip to content

agent_send_async: deliver callback results through channel bridge #891

@pbranchu

Description

@pbranchu

Problem

agent_send_async delivers delegation results via a hardcoded voice-only channel (ASYNC_RESULT_TX). When the originating conversation is on Chat, Email, or any other channel, the result is silently dropped:

WARN: Async result received but no active voice session

The agent tells the user "I've delegated the task" but the result never arrives.

Proposed Design

Core idea

When the async delegation completes, inject the result back through the channel bridge as a callback message. The bridge handles all context (channel system prompt, output format, routing) — the same path as any incoming message.

Callback message structure

The callback is a ChannelMessage where:

  • sender: ChannelUser { platform_id: "hand:{agent_id}", display_name: agent_id } — e.g. "workspace-hand". This is a plain string from the delegation input, not an enum. No maintenance needed as new hands are added.
  • content: ChannelContent::Text(result_text) — the raw delegation result.
  • channel/thread_id: stored from the originating message at delegation time.

The agent sees: "workspace-hand wrote: Found 47 unread emails, 12 are promotional..." — clearly a hand reporting back, not user speech.

One-turn system prompt injection

The bridge injects a per-turn channel system prompt (using the existing send_message_with_channel_prompt mechanism from #876) on the callback turn:

"The following message is a result from a task you previously delegated via agent_send_async. Present the findings to the user."

This is:

  • Automatic — no LLM prompt engineering needed per-agent
  • Standard — works for any agent, even without custom system prompts
  • Clean — uses existing infrastructure (Section 9.2 channel prompt injection)

Response routing

The callback dispatch differs slightly from normal dispatch_message:

  • Agent routing: skipped — target agent_id is known (stored at delegation time)
  • Response recipient: the original user (stored at delegation time), not the callback sender (the hand)
  • Channel adapter: looked up by stored channel type

Stored context at delegation time

Minimal — just enough to route the callback:

struct AsyncCallbackContext {
    agent_id: AgentId,          // Jeeves (the caller)
    channel_type: ChannelType,  // e.g. Custom("google_chat")
    reply_to: ChannelUser,      // original user (Philippe)
    thread_id: Option<String>,  // for threaded conversations
}

No stale context risk — the bridge re-applies all current overrides/prompts at dispatch time.

Data flow

1. User: "Clean my inbox"
   → Bridge dispatches to Jeeves
   
2. Jeeves calls agent_send_async(agent_id="workspace-hand", message="...")
   → Tool captures AsyncCallbackContext from current session
   → Tool returns "Task dispatched. callback_id: xxx"
   → Jeeves responds: "Right away, sir."
   
3. Background task: workspace-hand runs, produces result
   
4. Callback: bridge.inject_callback(context, result_text)
   → Constructs ChannelMessage with sender=workspace-hand
   → Injects callback system prompt for this turn
   → Calls send_message_with_channel_prompt(jeeves, message, callback_prompt)
   → Jeeves formats: "Your inbox has been tidied, sir. 12 promotional emails archived..."
   → Response sent to original user via originating channel adapter

Implementation

Changes needed

File Change
crates/openfang-channels/src/bridge.rs Add BridgeManager::inject_callback() method
crates/openfang-channels/src/bridge.rs Add ChannelBridgeHandle::inject_callback() trait method
crates/openfang-api/src/channel_bridge.rs Implement inject_callback on kernel's bridge handle
crates/openfang-runtime/src/tool_runner.rs Change tool_agent_send_async to store context and use inject_callback instead of ASYNC_RESULT_TX
crates/openfang-runtime/src/tool_runner.rs Remove ASYNC_RESULT_TX, ASYNC_RESULT_RX, take_async_result_receiver()
crates/openfang-runtime/src/kernel_handle.rs Add inject_callback() to KernelHandle trait
crates/openfang-channels/src/voice.rs Remove ASYNC_RESULT_RX consumer (voice uses same bridge path now)

How agent_send_async gets channel context

The tool runner needs to know which channel/sender the current conversation is on. Options:

  1. Pass through tool context: execute_tool() already receives caller_agent_id. Extend with channel metadata from the bridge dispatch.
  2. Query the kernel: add get_agent_channel_context(agent_id) that returns the last known channel/sender from the bridge's dispatch history.
  3. Store on agent session: when the bridge dispatches a message, store the channel context on the agent session. The tool runner reads it.

Option 3 is cleanest — the session already exists and is the natural place for "who is this agent talking to right now."

Test Plan

Unit tests

  • test_inject_callback_constructs_correct_message — verify sender is hand:{agent_id}, content is the result text, channel/thread match stored context
  • test_callback_system_prompt_injected — verify the one-turn system prompt is passed to send_message_with_channel_prompt
  • test_callback_response_routed_to_original_user — verify send_response targets the stored reply_to user, not the hand sender
  • test_async_context_stored_on_delegation — verify AsyncCallbackContext is populated from the current session's channel info
  • test_callback_with_no_active_session — verify graceful handling when the channel/user is no longer reachable

Integration tests

  • Chat delegation: send message on Chat → Jeeves delegates → workspace-hand completes → result appears in Chat
  • Voice delegation: same flow but on voice → result spoken via TTS
  • Cross-channel: delegate on Chat, verify result doesn't leak to Voice (and vice versa)
  • Thread preservation: delegate in a threaded Chat conversation → callback response appears in the same thread
  • Multiple concurrent delegations: two async tasks in flight → both results delivered correctly

Manual test scenarios

  • DM Jeeves on Chat: "Check my calendar for tomorrow" → delegation → result appears in same DM
  • Voice call: "What's in my inbox?" → delegation → Jeeves speaks the result
  • Chat thread: delegate from a thread → response in the same thread

PR Strategy

This builds on PR #797 (agent_send_async). Options:

  1. Amend Add agent_send_async with channel-agnostic callback delivery #797: add the callback delivery fix to the existing PR. This changes the PR scope but keeps it self-contained.
  2. Stack a new PR on Add agent_send_async with channel-agnostic callback delivery #797: create a follow-up PR that depends on Add agent_send_async with channel-agnostic callback delivery #797 being merged first.

Recommendation: amend #797 since the current implementation is broken for non-voice channels. The callback delivery is not a new feature — it's a fix to make agent_send_async actually work as described.

If #797 has already been reviewed, request a re-review after the update with a comment explaining the change.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions