Skip to content

RAG file watcher causes panic "send on closed channel" when re-indexing spans a message boundary #2266

@steilerDev

Description

@steilerDev

The agent crashes with a fatal panic ("send on closed channel") when the RAG file watcher triggers a re-index while a session is active.

Stack trace:

  goroutine 10537 [running]:
  github.com/docker/docker-agent/pkg/runtime.(*LocalRuntime).configureToolsetHandlers.chanSend.func2(...)
          github.com/docker/docker-agent/pkg/runtime/loop.go:579 +0x28
  github.com/docker/docker-agent/pkg/runtime.(*LocalRuntime).configureToolsetHandlers.ragEventForwarder.func3(...)
          github.com/docker/docker-agent/pkg/runtime/rag.go:27 +0x204
  github.com/docker/docker-agent/pkg/tools/builtin.(*RAGTool).forwardEvents(...)
          github.com/docker/docker-agent/pkg/tools/builtin/rag.go:95 +0x58
  created by github.com/docker/docker-agent/pkg/tools/builtin.(*RAGTool).Start in goroutine 10536
          github.com/docker/docker-agent/pkg/tools/builtin/rag.go:62 +0x8c
  ERROR: agent exited with code 2

Claudes analysis:

RunStream creates a new buffered events channel per message and wires it into the RAGTool via SetEventCallback. When the message completes, finalizeEventChannel closes that channel. However, RAGTool.forwardEvents is started once with an app-level context and runs for the lifetime of the agent — it holds a closure over the now-closed per-message channel and panics when it tries to forward the next RAG event onto it.

Version affected

docker-agent v1.38.0

How To Reproduce

  1. Configure an agent with a RAG source using semantic-embeddings pointed at a directory that is written to during a session (e.g. a knowledge base updated by a writer agent)
  2. Send a message that causes a file to be written to the RAG-indexed directory, triggering a re-index
  3. Send another message (or wait) while re-indexing is still in progress
  4. The agent crashes with panic: send on closed channel as the EventTypeUsage event emitted at the end of re-indexing is forwarded onto the previous message's now-closed events channel

Most reliably reproduced with semantic-embeddings because it makes an LLM call per chunk to generate semantic summaries — re-indexing can take tens of seconds, making it likely to span multiple message boundaries. bm25 and chunked-embeddings are not immune (they emit indexing lifecycle events through the same path) but re-index much faster so the window is small in practice.

Expectation

The agent should not crash. RAG re-indexing events that arrive after a message's events channel has closed should be dropped or safely forwarded to the next message's channel. A consistent approach would be to make chanSend non-blocking (matching the existing pattern in strategy/helpers.go:EmitEvent):

  func chanSend(ch chan Event) func(Event) {
      return func(e Event) {
          select {
          case ch <- e:
          default:
          }
      }
  }

OS and Terminal type

macOS

Metadata

Metadata

Assignees

No one assigned

    Labels

    kind/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions