fix(benchmarks): stop inverting --web/--no-web in the public runner

[Samurai007AK]

run_eval passed the parsed open-book flag straight into resolve_closed_book(), whose override parameter is the closed-book decision and is returned verbatim. Both CLI flags therefore did the opposite of their help text, and REACT_NO_WEB/SWARM_NO_WEB reached every worker inverted, so a run scored under the book policy nobody asked for.

Negate at the call site rather than in resolve_closed_book(): that function is polarity-correct on its own, and flipping it would invert the benchmark-default path, which is currently unaffected.

The translation now lives in _closed_book_for() so the polarity is pinnable by a test; it was previously buried mid-way through a long async function with no seam.

Closes #26

[dq-ai-dev]

Thanks for the fix—the polarity change is correct and addresses #26.
I think _closed_book_for() is unnecessary abstraction for a bug that occurs at this single call site. Could we keep the fix inline in run_eval()?

web = getattr(args, "web", None)
closed = resolve_closed_book(
    args.benchmark,
    None if web is None else not web,
)

We should still retain regression coverage. The test can call run_eval() with resolve_closed_book() monkeypatched as a spy, assert that it receives False, True, and None for --web, --no-web, and no override respectively, then raise a sentinel exception to stop before task execution. That would cover the original faulty handoff without introducing a helper solely as a test seam.

[Samurai007AK]

@dq-ai-dev
Both applied in 2f12692.

• Dropped _closed_book_for() — the negation is now inline in run_eval() exactly as you wrote it.
• Rewrote the regression test as a spy: it calls run_eval() with resolve_closed_book() monkeypatched to record the override it receives and raise a sentinel, stopping before task execution. It asserts [False, True, None] for --web, --no-web, and no flag. Confirmed it fails against the pre-fix handoff (assert [True, False, None] == [False, True, None]).

Validation:

• uv run ruff check frontier_agent/ apodex/ benchmarks/ workflows/ plugins/ deploy/ tools/ scripts/ — All checks passed
• uv run pyright benchmarks/public/runner/run_subprocess.py tests/test_benchmark_sandbox_profiles.py — 0 errors
• uv run pytest tests/test_benchmark_sandbox_profiles.py -q — the new test passes; the one failure in that module is the pre-existing test_render_dir_labels_nested_outputs Windows path-separator case, which fails identically on a clean tree.

[Samurai007AK]

Hi @dq-ai-dev! I’ve been exploring the FrontierAgent repo and I’m genuinely impressed by the quality and depth of the work here. The architecture, benchmarking setup, and overall engineering approach are really inspiring, and I’d love to contribute more actively to the project. I’m currently looking for ways to contribute beyond individual PRs and would love to learn more from the community. Is there a Slack, Discord, or any other channel where contributors and the team coordinate and discuss ongoing work?

I’d be really glad to get involved and contribute wherever I can. Thanks!