Issues in Deep Researcher version 2.1.0

[pixelThreader]

Step-wise Reasoning Engine: Architectural Scan & Proposals

This document provides a comprehensive technical scan of the Deep-Researcher-V2 backend and presents architectural proposals to optimize the step-wise reasoning engine for local, resource-constrained systems running Ollama (with a strict 8k token context window).

---

1. Executive Summary

The deep researcher has a phenomenal foundation: it leverages crawl4ai for parallelized high-performance scraping, maps data cleanups elegantly to local databases, and attempts a structured step-wise execution.

However, running a deep research ReAct loop on local models (e.g., gemma4:e2b or gemma4:e4b) within an 8k token limit creates two massive bottlenecks:

1. Context Window Blowout: Scraped web pages are extremely long. A single search query returning markdown for 30–45 pages can exceed 500,000 tokens. Passing this raw output back into an 8k context model instantly destroys the model's history, causing Ollama to drop the system instructions or truncate crucial context.
2. State Amnesia (Isolated Step Execution): The agent has absolutely zero memory of what was discovered in previous steps. Each plan step is run in complete isolation with a fresh thread_id and has no access to a vector database or running synthesis, leading to repetitive searches and fragmented findings.

---

2. v2.0.1 Project Scan: Root Causes of Failure

🔍 Cause A: Raw Tool Output Context Blowouts

In MCP_Tools_Server/main.py and src/web/scraper.py, the web_search and read_webpages tools leverage crawl4ai to scrape full page markdown.

• The Flow:
  1. The agent calls web_search(query="...") or read_webpages(urls=[...]).
  2. The crawler extracts the entire raw markdown content of these pages.
  3. The MCP server dumps this full markdown directly as a raw JSON string to the agent:

     # MCP_Tools_Server/main.py
     return json.dumps({"results": results})

  4. LangGraph receives this massive payload and inserts it directly into the agent's message history as a ToolMessage (Observation).
• The Bottleneck:
  A single average webpage's markdown is 3,000 to 10,000 tokens. Scraped batches of 5–10 pages easily result in 30k–80k tokens.
  Since the model's context window is only 8k tokens, Ollama is forced to truncate. This leads to:
  • Loss of the System Message (the agent forgets its personality, tool rules, and target task).
  • Loss of the original user query (the agent gets stuck in infinite ReAct loops because it doesn't remember what it was researching).
  • Out-of-memory crashes or severe generation slowdown (extreme prompt evaluation latency).

---

🔍 Cause B: Total Step Isolation (Zero Inter-Step Memory)

In backend/main/src/research/layer2/orchestrator1.py, the orchestrator processes the plan steps sequentially:

• The Flow:

  for step in context.plan:
      ...
      step_sources = await _run_step(
          step=step,
          context=context,
          ...
          thread_id=f"{base_thread_id}_step_{step_idx}_{uuid.uuid4().hex[:8]}",
          ...
      )

• The Bottleneck:
  • Fresh Sessions: Each plan step is executed as a completely brand-new LangGraph ReAct agent session with a randomized thread_id and empty conversation history.
  • No Shared State: The system prompt builder _build_system_message maps only the current step description and suggested tools. It never injects summaries, findings, or key facts discovered in steps 0 through N-1.
  • Unused RAG during reasoning: While a vector database (ChromaDB collection research_{research_id}) is defined in rag.py, it is never queried during the reasoning phase. It is only populated in orchestrator2.py after all steps are completed, and rag_search is never added to the reasoning agent's tool set.
  • Result: The agent executing Step 3 has no memory of what Step 1 or Step 2 discovered. It performs duplicate searches and cannot synthesize complex connections.

---

🔍 Cause C: Local Model ReAct Instability

• The Bottleneck:
  Local models have significantly weaker instruction-following capabilities than frontier APIs. ReAct loops require strict JSON block formatting or function-calling schemas.
  When context limits are approached, local models lose their tracking accuracy. They begin to hallucinate tool names, skip arguments, fail to output the final answer, or loop endlessly calling the same search tool.

---

Core areas to focus on:

• The context history never exceeds 6,000 tokens (comfortably within Ollama's 8k local system limit).
• The agent has perfect tracking of all data scraped.
• The local model never crashes, slows down, or loops endlessly.