Epic: Parallel Execution of Agents in the Loop Phase (Parallel Groups)

[maddes8cht]

Description

Currently, agents in the loop phase (as well as in preparation and finalization) run sequentially one after another. Since each agent runs as an isolated opencode run subprocess, they are inherently parallelizable – they don't wait for each other, share no memory, and communicate exclusively through WorkflowState.

A parallel execution feature would significantly reduce the total duration of a workflow run, especially for independent agents (e.g., multiple audit agents checking different aspects in parallel).

Design Proposal: Parallel Groups

Instead of a global toggle (all agents parallel or all sequential), the concept of Parallel Groups is introduced. A loop phase consists of a list of Groups. Each group has a mode (sequential or parallel) and a list of agents.

{
  "loop_agents": [
    { "agents": ["amala"], "mode": "sequential" },
    { "agents": ["vera", "critic", "linter"], "mode": "parallel" },
    { "agents": ["consolidator"], "mode": "sequential" }
  ]
}

Execution order in this example:

1. Amala runs alone (sequential) and writes state
2. Vera, Critic, and Linter start simultaneously (parallel)
3. When all three finish, their state updates are merged
4. Consolidator runs alone (sequential) with the merged state

Backward Compatibility

Existing workflow JSONs with "loop_agents": ["amala", "vera"] are still supported – they are internally interpreted as a single sequential group.

Sub-Issues (ToDo)

• Configuration System (core/config.py) #1 Extend WorkflowConfig – loop_agents accepts additional list[dict] with agents + mode. Backward compatible. Validation.
• Core State Management (core/state.py, core/parser.py) #2 Parallel Execution in Engine – _execute_group() in core/engine.py, using concurrent.futures.ThreadPoolExecutor (since agents are I/O-bound subprocesses). Timeout per group.
• Agent Definition Loader (core/agent.py) #3 State Merging for Parallel Agents – Each parallel agent gets a copy of the state. After completion, results are merged. Conflict rule: Last write wins for scalar fields, payload is deeply merged.
• Subprocess Runner (core/runner.py) #4 Error Handling for Parallel Agents – When a parallel agent fails (agent_error:): Options: a) abort all running, b) wait for remaining. Configurable via on_parallel_error: "abort" | "continue".
• Execution Engine (core/engine.py) #5 End Condition after Parallel Groups – end_state_condition is checked only after the entire group completes (not after each individual agent).
• Main Entry Point (openloop.py) #6 GUI: Parallel/Sequential Toggle – In the slot listboxes, provide a toggle per entry (e.g., right-click menu or toggle button) for the mode. Visual indication of parallel agents (e.g., icon or color).
• Example Agents & Workflow #7 Integration Tests – Tests for sequential, parallel, and mixed groups, error cases, state merging, timeout.

Open Questions

Question	Discussion
Does max_loops count per iteration or per agent?	Per iteration (as before) – one parallel group counts as one step
Thread pool size	Configurable via max_parallel_agents (default: number of CPU cores)
State copy for parallel agents	Should the state be deep-copied (copy.deepcopy) or is a shallow copy sufficient? Deep copy is safer but more expensive for large payloads
Visibility of parallel output in GUI log	Each parallel agent needs its own log entry, possibly with thread ID or agent name prefix
Also for Prep/Finalization?	Technically yes, practically less relevant (only 1–2 agents there). Could be added later

Example Workflow

{
  "preparation_agents": ["proteus"],
  "loop_agents": [
    { "agents": ["amala"], "mode": "sequential" },
    { "agents": ["vera", "linter", "security_auditor"], "mode": "parallel" }
  ],
  "finalization_agents": ["consolidator"],
  "end_state_condition": "payload.get('score', 0) >= 90",
  "max_loops": 10,
  "max_parallel_agents": 4
}