Benchmark: compare tutorial orchestration patterns on a fixed task set

[Koukyosyumei]

Summary

The tutorial ships several orchestration patterns (arena_score.py, ensemble_score.py, judge_panel_score.py, tournament.py, ...) but there is no side-by-side comparison showing what each pattern actually buys you on the same task. New users have to guess which pattern to reach for.

This task is to run a small, fixed benchmark task through at least two tutorial patterns and contribute the measured comparison as an example doc. It requires actually installing and using h5i + h5i-python end to end, which also makes it a great smoke test of our onboarding docs. No engine (Rust) knowledge needed.

What to do

1. Follow the README install steps (h5i engine + pip install h5i-orchestra) and get the Quickstart ensemble.py running once. If anything in the install or Quickstart is wrong or confusing, note it: fixing that is in scope and welcome.
2. Pick 2 or 3 small, self-contained benchmark tasks with an objective pass signal, for example:
   • "implement quicksort in Python with unit tests" (the Quickstart task),
   • "fix this off-by-one bug so the provided pytest file passes",
   • "add input validation to this function; tests included".
     Each task should live in a tiny scratch Git repo with a pytest suite the verifier can run.
3. Run each task through at least two patterns, e.g. examples/tutorial/arena_score.py (independent arena) vs examples/tutorial/ensemble_score.py (mutual review). If you only have one agent CLI installed, hire the same runtime under two seat names; the comparison between patterns is still valid.
4. For each (task, pattern) run, record:
   • the verdict (winning submission) and whether verification passed,
   • wall-clock per step (h5i team trace prints recorded duration_ms per step),
   • diff size of the winning submission,
   • total number of agent turns.
5. Contribute examples/benchmarks/README.md with the setup, the tasks, the results table, and a short "which pattern when" takeaway. Include the h5i and h5i-orchestra versions.

Acceptance criteria

• examples/benchmarks/README.md exists with a results table covering at least 2 tasks x 2 patterns.
• Results come from real runs (link or paste the h5i team trace output for at least one run).
• The benchmark tasks are included (or linked as a tiny fixture repo) so someone else can re-run the comparison.
• Any install/Quickstart friction you hit is either fixed in the same PR or filed as a separate issue.

Hints

• The default launcher="resident" starts agent sessions via tmux, so run this on a machine where tmux and at least one agent CLI (Claude Code or Codex) are installed and logged in.
• Agent runs cost real tokens. Keep the tasks tiny; the point is comparing patterns, not solving hard problems.
• h5i team trace --dot renders the recorded DAG if you want a figure for the doc.