feat: add Aforge harness option

.env.example

@@ -15,6 +15,7 @@ AGENTFIELD_PUBLIC_URL=
 NODE_ID=pr-af
 
 # --- AI / harness config ---
+# `aforge` selects the unreleased aforge-v2 CLI through AgentField's harness.
 PR_AF_PROVIDER=opencode
 PR_AF_MODEL=openrouter/moonshotai/kimi-k2.5
 # Falls back to PR_AF_MODEL when unset
@@ -24,6 +25,7 @@ PR_AF_AI_MAX_RETRIES=3
 PR_AF_AI_INITIAL_BACKOFF_SECONDS=2.0
 PR_AF_AI_MAX_BACKOFF_SECONDS=8.0
 PR_AF_OPENCODE_BIN=opencode
+PR_AF_AFORGE_BIN=aforge
 # Optional provider-agnostic harness executable override (leave unset to use provider defaults)
 # PR_AF_HARNESS_BIN=
 PR_AF_OPENCODE_SERVER=

Dockerfile

@@ -14,7 +14,7 @@ COPY pyproject.toml README.md ./
 COPY src/ src/
 
 RUN pip install --no-cache-dir --prefix=/install \
-    "agentfield>=0.1.84" \
+    "agentfield @ git+https://github.com/Agent-Field/agentfield.git@60b0aa76df604575429af60fb0ccffc9aaca7745#subdirectory=sdk/python" \
     "hax-sdk>=0.2.4" \
     "pydantic>=2.0" \
     "httpx>=0.27" \

README.md

@@ -263,7 +263,16 @@ The key knobs (see `.env.example` for the full list):
 |-----------------------------|----------------------------------------------------------------|
 | `OPENROUTER_API_KEY`        | LLM provider key (OpenRouter) — required                       |
 | `GH_TOKEN`                  | GitHub token (`repo` scope) for reading PRs and posting reviews |
-| `PR_AF_PROVIDER`            | Harness provider (default `opencode`)                          |
+| `PR_AF_PROVIDER`            | Harness provider (default `opencode`; accepts `aforge`)        |
+| `PR_AF_AFORGE_BIN`          | Path to an aforge-v2 binary (default `aforge`)                 |
+| `PR_AF_HARNESS_BIN`         | Provider-agnostic executable override                          |
+
+For an unreleased aforge-v2 source benchmark, build `./cmd/aforge`, then run
+PR-AF with `PR_AF_PROVIDER=aforge` and either `PR_AF_AFORGE_BIN` (Python node)
+or `PR_AF_HARNESS_BIN` (maintained Go node) set to the absolute binary path.
+The draft pins the exact AgentField harness commit used for benchmarking;
+replace that pin with the released SDK before merging. The current Docker
+images do not bundle the unreleased Aforge binary.
 | `PR_AF_MODEL`               | Harness model (default `openrouter/moonshotai/kimi-k2.5`)      |
 | `PR_AF_MAX_COST_USD`        | Per-run cost ceiling in USD (default `2.0`)                    |
 | `PR_AF_MAX_DURATION_SECONDS`| Per-run wall-clock ceiling in seconds (default `3600`)         |

agentfield-package.yaml

@@ -49,8 +49,10 @@ user_environment:
       type: secret
       scope: global
     - name: PR_AF_PROVIDER
-      description: Coding-agent harness provider
+      description: Coding-agent harness provider (aforge | claude-code | codex | gemini | opencode)
       default: opencode
+    - name: PR_AF_AFORGE_BIN
+      description: Optional path to the aforge-v2 binary (defaults to aforge on PATH)
     - name: PR_AF_MODEL
       description: Model the harness uses
       default: openrouter/moonshotai/kimi-k2.5

benchmark/martian-code-review-bench/scripts/README.md

@@ -7,8 +7,8 @@ when present. Raw per-run transcripts and caches are written to a gitignored
 
 | script | what it does |
 |---|---|
-| `run_node.sh` | Launches the local PR-AF runner with the whole pipeline pinned to GLM-5.2 (`openrouter/z-ai/glm-5.2` for both `.harness()` and `.ai()`), registering with the AgentField control plane on `:8080`. |
-| `campaign.py` | Runs a blind `depth=deep` review for each problem in `../problems.json` (hardest-first, resumable, one review per repo at a time), LLM-judges findings against the goldens for **recall**, and writes `../scoreboard.{md,jsonl}` + `../results/<id>.json`. |
+| `run_node.sh` | Launches the local PR-AF runner with the whole pipeline pinned to GLM-5.2 (`openrouter/z-ai/glm-5.2` for both `.harness()` and `.ai()`), registering with the AgentField control plane on `:8080`. Defaults to the historical OpenCode provider; set `PR_AF_PROVIDER=aforge` for Aforge. |
+| `campaign.py` | Runs a blind `depth=deep` review for each problem in `../problems.json` (hardest-first, resumable, one review per repo at a time), LLM-judges findings against the goldens for **recall**, and writes isolated result sets. It also persists wall time, summed agent time, total tokens, cost, and usage coverage from the execution DAG. |
 | `ensemble.py` | Self-consistency escalation: for every baseline miss, run K extra independent passes, union the findings, re-judge. Run after `campaign.py` prints `[campaign] done`. |
 | `all_metrics.py` | Golden-only precision/recall/F1 on the posted-comment basis, ranked against every leaderboard tool from the cloned Martian dataset. |
 | `honest_compare.py` | Honest scoring (Framing C, see `../RESULTS.md`): credits real non-golden bugs, applied uniformly to PR-AF and the leaders (cubic-v2, cubic-dev). |
@@ -22,12 +22,29 @@ uv run python benchmark/martian-code-review-bench/scripts/campaign.py  # termina
 uv run python benchmark/martian-code-review-bench/scripts/ensemble.py  # optional: miss escalation
 ```
 
+Run the same GLM-5.2 workload through Aforge without touching the committed
+OpenCode baseline:
+
+```bash
+PR_AF_PROVIDER=aforge AFORGE_BIN=/path/to/aforge \
+  bash benchmark/martian-code-review-bench/scripts/run_node.sh
+
+PR_AF_PROVIDER=aforge CAMPAIGN_VARIANT=aforge-glm52 \
+  uv run python benchmark/martian-code-review-bench/scripts/campaign.py
+```
+
+The variant writes raw caches under `_glm52_bench/aforge-glm52/` and curated
+outputs under `benchmark/martian-code-review-bench/variants/aforge-glm52/`.
+Never leave `CAMPAIGN_VARIANT` empty for a comparison run: the empty value is
+reserved for the historical OpenCode files.
+
 `all_metrics.py` and `honest_compare.py` additionally need Martian's cloned offline
 dataset. Set `CRBENCH_RESULTS_DIR` to the offline `results/` directory, or set
 `CRBENCH_JUDGE_FILE` directly for `honest_compare.py`.
 
 ## Knobs (env)
 
-`CAMPAIGN_CONCURRENCY` (default 3) · `CAMPAIGN_DEPTH` (deep) · `CAMPAIGN_MAX_COST`
+`CAMPAIGN_VARIANT` (isolated result-set name) · `CAMPAIGN_CONCURRENCY` (default 3)
+· `CAMPAIGN_DEPTH` (deep) · `CAMPAIGN_MAX_COST`
 · `CAMPAIGN_MAX_DURATION` · `CAMPAIGN_LIMIT` (cap unsolved problems per invocation)
 · `CAMPAIGN_FORCE` (comma-ids to re-run) · `ENSEMBLE_PASSES` (default 2).