[Phase 4] Worktree-Per-Task Isolation for Parallel Batch Execution

[frankbria]

Summary

When running parallel batch execution (cf work batch run --strategy parallel), give each task its own git worktree instead of having multiple agents modify files in the same working directory. Inspired by Gastown's polecat worktree model, but much simpler.

Motivation

Currently parallel batch execution runs multiple agents in the same workspace directory. This creates race conditions: two agents editing the same file, conflicting git add operations, and verification gates that can't tell which agent broke what.

Gastown solves this with per-agent worktrees — git's built-in mechanism for multiple working directories sharing a single .git object store. This is fast (no full clone), isolated (each worktree has its own branch), and mergeable (standard git merge/rebase).

Keep it simple: We don't need Gastown's persistent polecat pools, sandbox repair, or worktree lifecycle daemons. Just: create worktree before task, run agent in it, merge results after.

Design

For serial execution: no change

Single tasks continue working in the main workspace directory. No overhead.

For parallel execution: worktree per task

cf work batch run task-1 task-2 task-3 --strategy parallel

Creates:

<workspace>/
├── .git/                          # shared object store
├── .codeframe/
│   └── worktrees/
│       ├── task-1/                # git worktree on branch cf/task-1
│       ├── task-2/                # git worktree on branch cf/task-2
│       └── task-3/                # git worktree on branch cf/task-3
└── <normal working dir>           # main branch, untouched during parallel run

Lifecycle (per task in parallel batch)

1. git worktree add .codeframe/worktrees/<task-id> -b cf/<task-id>
2. Run agent with cwd set to the worktree path
3. Agent modifies files in isolation
4. Verification gates run inside the worktree
5. On success: merge worktree branch back to main (or create PR)
6. Cleanup: git worktree remove .codeframe/worktrees/<task-id>

Conflict handling

• If merge has conflicts: create a blocker with the conflict details
• Let the human resolve, or re-run the task on top of the merged result
• For simple conflicts: attempt automatic resolution (ours/theirs for non-overlapping changes)

Scope

New module: core/worktrees.py

class TaskWorktree:
    def create(self, workspace_path: Path, task_id: str, base_branch: str = "main") -> Path:
        """Create an isolated worktree for a task. Returns worktree path."""
    
    def merge_back(self, workspace_path: Path, task_id: str) -> MergeResult:
        """Merge worktree branch back to base. Returns success/conflict."""
    
    def cleanup(self, workspace_path: Path, task_id: str) -> None:
        """Remove worktree and delete branch."""

Changes to existing files

• core/conductor.py — in parallel strategy, create worktrees and pass worktree paths to agents
• core/runtime.py — accept optional working_directory override for agent execution
• cli/app.py — add --isolate flag to batch run (default: True for parallel, False for serial)

Acceptance Criteria

• Parallel batch tasks each get their own worktree
• Agent executes with cwd set to worktree path
• Verification gates run inside the worktree
• Successful tasks merge back to base branch
• Merge conflicts create blockers with context
• Worktrees cleaned up after merge (success or failure)
• Serial execution is unchanged (no worktrees)
• --no-isolate flag disables worktrees for parallel (old behavior)
• Integration tests: parallel batch with worktree isolation

Dependencies

• No hard prerequisites
• Benefits from [Phase 4] Agent Adapter Architecture: Delegate to Frontier Coding Agents #408 (agent adapter) — external agents also need correct cwd
• Benefits from [Phase 4] Workspace Lifecycle Hooks System #392 (workspace hooks) — before_task hook runs in worktree

Complexity Note

This is deliberately simpler than Gastown's model:

• No persistent polecat pools (worktrees created and destroyed per batch)
• No sandbox reuse across batches (fresh worktree each time)
• No daemon-managed lifecycle (CLI-driven)
• No three-layer identity/sandbox/session split (just a worktree)

[traycerai]

Plan

Observations

The parallel execution path lives in file:codeframe/core/conductor.py: _execute_parallel → _execute_group_parallel → _execute_task_subprocess. The subprocess call passes cwd=workspace.repo_path, which is the shared main working directory — the root of the race-condition problem. Git operations already use gitpython throughout file:codeframe/core/git.py and file:codeframe/git/workflow_manager.py, so worktree commands via repo.git.worktree(...) fit naturally. The BatchRun dataclass carries engine as a field alongside strategy; an isolate flag follows the same pattern.

Approach

Introduce a thin TaskWorktree class in a new file:codeframe/core/worktrees.py module, then wire it into _execute_task_subprocess (via an optional worktree_path argument), _execute_group_parallel/_execute_single_task, start_batch/BatchRun, and the CLI batch run command. Serial execution is untouched. The subprocess cwd becomes the worktree path when isolation is active.

---

Implementation Plan

1. Create file:codeframe/core/worktrees.py

Define a MergeResult dataclass and a TaskWorktree class with three methods:

MergeResult fields: task_id: str, success: bool, conflict_details: str, merge_commit: Optional[str].

TaskWorktree.create(workspace_path, task_id, base_branch)

• Determine the worktree root: <workspace_path>/.codeframe/worktrees/<task_id>/
• Ensure the parent directory exists.
• Run repo.git.worktree("add", str(worktree_path), "-b", f"cf/{task_id}") using a git.Repo opened at workspace_path.
• Return the Path to the created worktree.

TaskWorktree.merge_back(workspace_path, task_id, base_branch)

• Open git.Repo(workspace_path).
• Checkout base_branch on the main repo.
• Attempt repo.git.merge(f"cf/{task_id}", "--no-ff", "-m", f"Merge cf/{task_id} into {base_branch}").
• On success: return MergeResult(success=True, merge_commit=repo.head.commit.hexsha, ...).
• On git.GitCommandError containing "CONFLICT": abort (repo.git.merge("--abort")), capture conflict output, return MergeResult(success=False, conflict_details=...).

TaskWorktree.cleanup(workspace_path, task_id)

• Run repo.git.worktree("remove", str(worktree_path), "--force").
• Delete the task branch: repo.delete_head(f"cf/{task_id}", force=True).
• Guard with try/except and log warnings; do not re-raise — cleanup must never fail the caller.

---

2. Update BatchRun dataclass in file:codeframe/core/conductor.py

Add an isolate: bool = True field to BatchRun. Update _save_batch and _row_to_batch to persist/restore this flag (add an isolate column to batch_runs in _ensure_schema_upgrades in file:codeframe/core/workspace.py).

---

3. Update start_batch signature in file:codeframe/core/conductor.py

Add isolate: bool = True parameter. Pass it to the BatchRun constructor and forward it when calling _execute_parallel and _execute_serial. Isolation only activates when strategy in ("parallel", "auto") and isolate is True.

---

4. Update _execute_task_subprocess in file:codeframe/core/conductor.py

Add an optional worktree_path: Optional[Path] = None parameter.

subprocess.Popen(
    cmd,
    cwd=worktree_path or workspace.repo_path,   # <- key change
    ...
)

No other change to this function.

---

5. Update _execute_single_task in file:codeframe/core/conductor.py

Add worktree_path: Optional[Path] = None parameter and forward it to _execute_task_subprocess.

---

6. Update _execute_group_parallel in file:codeframe/core/conductor.py

Add isolate: bool = False parameter. Inside the execute_task closure:

sequenceDiagram
    participant EGP as _execute_group_parallel
    participant WT as TaskWorktree
    participant Sub as _execute_task_subprocess
    participant Blocker as blockers.create

    EGP->>WT: create(workspace_path, task_id, base_branch)
    WT-->>EGP: worktree_path
    EGP->>Sub: (workspace, task_id, batch_id, engine, worktree_path)
    Sub-->>EGP: result_status
    alt result_status == COMPLETED
        EGP->>WT: merge_back(workspace_path, task_id)
        alt MergeResult.success
            EGP->>WT: cleanup(workspace_path, task_id)
        else conflict
            EGP->>Blocker: create(workspace, task_id, conflict_details)
            EGP->>WT: cleanup(workspace_path, task_id)
        end
    else FAILED or BLOCKED
        EGP->>WT: cleanup(workspace_path, task_id)
    end

Loading

When isolate=False, skip all worktree operations and call _execute_task_subprocess as before.

Use a dedicated lock (a new _merge_lock = threading.Lock()) around merge_back calls so concurrent merges are serialised — concurrent git merges on the same main branch head are not safe.

---

7. Update _execute_parallel in file:codeframe/core/conductor.py

• Determine base_branch by calling get_current_branch(workspace) (already in file:codeframe/core/git.py).
• Forward isolate=batch.isolate to both _execute_single_task and _execute_group_parallel.
• Single-task groups in parallel strategy also get a worktree when isolate=True — route through the same helper to keep the logic in one place.

---

8. Update file:codeframe/core/workspace.py — schema migration

In _ensure_schema_upgrades, add a migration block for batch_runs that adds the isolate column if absent:

ALTER TABLE batch_runs ADD COLUMN isolate INTEGER NOT NULL DEFAULT 1

Also update the CREATE TABLE DDL for batch_runs in _init_database to include isolate INTEGER NOT NULL DEFAULT 1.

---

9. Update the CLI batch run command in file:codeframe/cli/app.py

Add two mutually exclusive flags:

Flag	Type	Default	Behaviour
--isolate / --no-isolate	bool	True	Enable/disable worktree isolation

Wire the resolved value through to conductor.start_batch(... isolate=isolate). Only display the isolation note when strategy is parallel or auto:

  Isolation: worktree-per-task (--no-isolate to disable)

---

10. Conflict blocker integration

When merge_back returns success=False, in _execute_group_parallel (after the merge attempt):

• Import and call blockers.create(workspace, task_id=task_id, question=f"Merge conflict for cf/{task_id}:\n{merge_result.conflict_details}").
• Set result_status = RunStatus.BLOCKED.value so the standard blocked-task path handles the rest (event emission, batch result tracking).

This reuses the existing blockers module and blocked-run flow with zero new infrastructure.

---

Summary of touched files

File	Change
codeframe/core/worktrees.py	New — MergeResult, TaskWorktree
codeframe/core/conductor.py	BatchRun.isolate field; start_batch param; _execute_task_subprocess worktree_path; _execute_single_task worktree_path; _execute_group_parallel worktree lifecycle + merge lock; _execute_parallel branch detection + isolation forwarding; _save_batch/_row_to_batch persistence
codeframe/core/workspace.py	DDL and migration for batch_runs.isolate column
codeframe/cli/app.py	--isolate/--no-isolate flag on batch run; forward to start_batch

Serial execution (_execute_serial) is not touched.

Import In IDE

🤖 Prompt for AI Agents

## Observations

The parallel execution path lives in `file:codeframe/core/conductor.py`: `_execute_parallel` → `_execute_group_parallel` → `_execute_task_subprocess`. The subprocess call passes `cwd=workspace.repo_path`, which is the shared main working directory — the root of the race-condition problem. Git operations already use `gitpython` throughout `file:codeframe/core/git.py` and `file:codeframe/git/workflow_manager.py`, so worktree commands via `repo.git.worktree(...)` fit naturally. The `BatchRun` dataclass carries `engine` as a field alongside `strategy`; an `isolate` flag follows the same pattern.

## Approach

Introduce a thin `TaskWorktree` class in a new `file:codeframe/core/worktrees.py` module, then wire it into `_execute_task_subprocess` (via an optional `worktree_path` argument), `_execute_group_parallel`/`_execute_single_task`, `start_batch`/`BatchRun`, and the CLI `batch run` command. Serial execution is untouched. The subprocess `cwd` becomes the worktree path when isolation is active.

---

## Implementation Plan

### 1. Create `file:codeframe/core/worktrees.py`

Define a `MergeResult` dataclass and a `TaskWorktree` class with three methods:

**`MergeResult`** fields: `task_id: str`, `success: bool`, `conflict_details: str`, `merge_commit: Optional[str]`.

**`TaskWorktree.create(workspace_path, task_id, base_branch)`**
- Determine the worktree root: `<workspace_path>/.codeframe/worktrees/<task_id>/`
- Ensure the parent directory exists.
- Run `repo.git.worktree("add", str(worktree_path), "-b", f"cf/{task_id}")` using a `git.Repo` opened at `workspace_path`.
- Return the `Path` to the created worktree.

**`TaskWorktree.merge_back(workspace_path, task_id, base_branch)`**
- Open `git.Repo(workspace_path)`.
- Checkout `base_branch` on the main repo.
- Attempt `repo.git.merge(f"cf/{task_id}", "--no-ff", "-m", f"Merge cf/{task_id} into {base_branch}")`.
- On success: return `MergeResult(success=True, merge_commit=repo.head.commit.hexsha, ...)`.
- On `git.GitCommandError` containing "CONFLICT": abort (`repo.git.merge("--abort")`), capture conflict output, return `MergeResult(success=False, conflict_details=...)`.

**`TaskWorktree.cleanup(workspace_path, task_id)`**
- Run `repo.git.worktree("remove", str(worktree_path), "--force")`.
- Delete the task branch: `repo.delete_head(f"cf/{task_id}", force=True)`.
- Guard with `try/except` and log warnings; do not re-raise — cleanup must never fail the caller.

---

### 2. Update `BatchRun` dataclass in `file:codeframe/core/conductor.py`

Add an `isolate: bool = True` field to `BatchRun`. Update `_save_batch` and `_row_to_batch` to persist/restore this flag (add an `isolate` column to `batch_runs` in `_ensure_schema_upgrades` in `file:codeframe/core/workspace.py`).

---

### 3. Update `start_batch` signature in `file:codeframe/core/conductor.py`

Add `isolate: bool = True` parameter. Pass it to the `BatchRun` constructor and forward it when calling `_execute_parallel` and `_execute_serial`. Isolation only activates when `strategy in ("parallel", "auto")` and `isolate is True`.

---

### 4. Update `_execute_task_subprocess` in `file:codeframe/core/conductor.py`

Add an optional `worktree_path: Optional[Path] = None` parameter.

```
subprocess.Popen(
    cmd,
    cwd=worktree_path or workspace.repo_path,   # <- key change
    ...
)
```

No other change to this function.

---

### 5. Update `_execute_single_task` in `file:codeframe/core/conductor.py`

Add `worktree_path: Optional[Path] = None` parameter and forward it to `_execute_task_subprocess`.

---

### 6. Update `_execute_group_parallel` in `file:codeframe/core/conductor.py`

Add `isolate: bool = False` parameter. Inside the `execute_task` closure:

```mermaid
sequenceDiagram
    participant EGP as _execute_group_parallel
    participant WT as TaskWorktree
    participant Sub as _execute_task_subprocess
    participant Blocker as blockers.create

    EGP->>WT: create(workspace_path, task_id, base_branch)
    WT-->>EGP: worktree_path
    EGP->>Sub: (workspace, task_id, batch_id, engine, worktree_path)
    Sub-->>EGP: result_status
    alt result_status == COMPLETED
        EGP->>WT: merge_back(workspace_path, task_id)
        alt MergeResult.success
            EGP->>WT: cleanup(workspace_path, task_id)
        else conflict
            EGP->>Blocker: create(workspace, task_id, conflict_details)
            EGP->>WT: cleanup(workspace_path, task_id)
        end
    else FAILED or BLOCKED
        EGP->>WT: cleanup(workspace_path, task_id)
    end
```

When `isolate=False`, skip all worktree operations and call `_execute_task_subprocess` as before.

Use a **dedicated lock** (a new `_merge_lock = threading.Lock()`) around `merge_back` calls so concurrent merges are serialised — concurrent git merges on the same main branch head are not safe.

---

### 7. Update `_execute_parallel` in `file:codeframe/core/conductor.py`

- Determine `base_branch` by calling `get_current_branch(workspace)` (already in `file:codeframe/core/git.py`).
- Forward `isolate=batch.isolate` to both `_execute_single_task` and `_execute_group_parallel`.
- Single-task groups in parallel strategy also get a worktree when `isolate=True` — route through the same helper to keep the logic in one place.

---

### 8. Update `file:codeframe/core/workspace.py` — schema migration

In `_ensure_schema_upgrades`, add a migration block for `batch_runs` that adds the `isolate` column if absent:

```sql
ALTER TABLE batch_runs ADD COLUMN isolate INTEGER NOT NULL DEFAULT 1
```

Also update the `CREATE TABLE` DDL for `batch_runs` in `_init_database` to include `isolate INTEGER NOT NULL DEFAULT 1`.

---

### 9. Update the CLI `batch run` command in `file:codeframe/cli/app.py`

Add two mutually exclusive flags:

| Flag | Type | Default | Behaviour |
|---|---|---|---|
| `--isolate / --no-isolate` | `bool` | `True` | Enable/disable worktree isolation |

Wire the resolved value through to `conductor.start_batch(... isolate=isolate)`. Only display the isolation note when `strategy` is `parallel` or `auto`:

```
  Isolation: worktree-per-task (--no-isolate to disable)
```

---

### 10. Conflict blocker integration

When `merge_back` returns `success=False`, in `_execute_group_parallel` (after the merge attempt):
- Import and call `blockers.create(workspace, task_id=task_id, question=f"Merge conflict for cf/{task_id}:\n{merge_result.conflict_details}")`.
- Set `result_status = RunStatus.BLOCKED.value` so the standard blocked-task path handles the rest (event emission, batch result tracking).

This reuses the existing `blockers` module and blocked-run flow with zero new infrastructure.

---

### Summary of touched files

| File | Change |
|---|---|
| `codeframe/core/worktrees.py` | **New** — `MergeResult`, `TaskWorktree` |
| `codeframe/core/conductor.py` | `BatchRun.isolate` field; `start_batch` param; `_execute_task_subprocess` `worktree_path`; `_execute_single_task` `worktree_path`; `_execute_group_parallel` worktree lifecycle + merge lock; `_execute_parallel` branch detection + isolation forwarding; `_save_batch`/`_row_to_batch` persistence |
| `codeframe/core/workspace.py` | DDL and migration for `batch_runs.isolate` column |
| `codeframe/cli/app.py` | `--isolate/--no-isolate` flag on `batch run`; forward to `start_batch` |

Serial execution (`_execute_serial`) is **not touched**.

---

Execution Information

Branch: main
Commit: 705b160

---

💡 Tips

Supported Commands (Inside Comments)

• Use @traycerai generate to iterate on the previous version of the implementation plan.

Supported Commands (Inside Description)

• Add @traycerai ignore anywhere in the ticket description to prevent this ticket from being processed.
• Add @traycerai branch:<branch-name> anywhere in the ticket description to specify the target branch for the implementation plan.

Community

• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[coderabbitai]

🔗 Related PRs

#246 - feat(agents): trigger multi-agent execution after task approval [merged]
#319 - feat(cli): add cf work follow for live execution streaming [merged]
#363 - feat: wire ReactAgent into runtime with --engine react CLI flag [merged]
#369 - test: add CLI integration tests for ReactAgent runtime parameters [merged]
#382 - fix: return non-zero exit codes from work retry and batch run [merged]

---

📝 Issue Planner

_{Check the box below or use the @coderabbitai plan command to generate an implementation plan and prompts that you can use with your favorite coding assistant.}

• Create Plan

---

🧪 Issue enrichment is currently in open beta.

You can configure auto-planning by selecting labels in the issue_enrichment configuration.

To disable automatic issue enrichment, add the following to your .coderabbit.yaml:

issue_enrichment:
  auto_enrich:
    enabled: false

💬 Have feedback or questions? Drop into our discord!