Repository navigation
Players survive a long broadcast; a five-model war task; sponsor banners on the stream - #18
Merged
Merged
Conversation
…ng model request restarts the session The harness took "GAME OVER" anywhere in a tool result as the end of the game. The env echoes the agent's plan in end_turn results, and a plan that said "this session ends after 40 turns or GAME OVER" made GPT-6 Sol's player stop at its first session boundary, at T40 of 270. A rival's message quoting the words would do the same. The game is over now only when the env's footer says so, or a line opens with GAME OVER at the turn limit. A Kimi K3 request through the proxy hung with its connection open, past Claude Code's own timeout, and the player sat silent for the rest of the game. The Claude player now ends a session whose model has sent nothing for 240 s while no tool call runs (end_turn may wait for the other seats as long as it needs), and a fresh session plays on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the env ends its turn (300 s) Live on the probe: a Kimi K3 request hung at 08:26:39 and the fresh session started at 08:31:15; at 240 s the env had already ended the turn after holding the other seats for 300 s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
astra-opus-kimi seats the three on a Tiny map with 3 AI civs (Monarch, restless barbarians) for 300 turns at a 55 s pace, 4 h 35 min unless a conquest ends it, with 30-turn sessions. The prompt is sol-vs-opus's for three: conquering the other two models wins, the score only breaks a tie. Probes for it (FD-3666): three agents on a Small map kept the peace for 75 turns, at 70 % or 80 % ocean and with a 70- or 270-turn limit; on a Tiny map Opus declared war at turn 56 and razed two American cities by turn 60. Astra costs about 6x Sol a turn; the estimate for the whole broadcast is about $170 with the casters ($220 if the war doubles the calls). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p, told to conquer GPT-6 Astra, GPT-6 Sol and GPT-5.6 Terra (Codex) and Opus 5.5 and Sonnet 5.5 (Claude Code) on a Tiny map with no civilization run by the game's own AI, for 300 turns at a 55 s pace. The prompt makes conquering the other four the goal; with four rivals to eliminate, the game is likelier to reach its limit (4 h 35 min or more) than to end early. Terra is GPT-5.6 Terra: the proxy has no GPT-6 Terra. Estimated about $200 with the casters, $270 if the war doubles the calls. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Astra costs 5x Sol a token and, with four rivals in every brief, about 0.15 a turn from the start (smoke run, T0-T10). Shorter sessions shrink the context each call re-reads, which brings the estimate's high case from about 303 to 270. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e goal, the turn limit is 750 The prompt now reads "Build your empire, and make conquering the other civilizations your ultimate goal, as the game ends only when you have done so." With no score tiebreak to play for, the turn limit goes to 750, about 11 hours at a 55 s pace, so the broadcast runs until one model has conquered the rest or until it is stopped (about 65 turns an hour; about $160 for four hours, estimated). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…al with AgentEnv Framework")
broadcast.banners (up to 8) fill a sponsor slot at the right end of the stream's bottom row, the side panel's width,
on screen in every shot: one at a time, crossfading every 30 s. A banner is text with logos inline where {name}
stands, shrunk to fit down to 13 px, then cut with an ellipsis, on the stream's dark colours or a light plate.
A logo is an https:// URL, a data: URI or an image file. The openciv3_match step fetches or reads each one, checks it
is a PNG, JPEG, WebP, GIF or SVG image of at most 512 KB, and gives the env a data: URI, so the game's settings hold
all the stream shows and a broadcast never waits on another host; no third-party logo is committed. The env takes
only data: URIs, links each logo in its live data (live/logo/<banner>/<name>) rather than sending its bytes on every
poll, and serves it from that route, an SVG without scripts.
five-way-war's slot reads "Powered by Modal with AgentEnv Framework", with Modal's GitHub avatar and AgentEnv's icon.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the smoke run a 78-character banner lost the end of its link at 13 px on one line. Now a banner shrinks to 16 px on one line, then takes two lines from 17 px down to 13 px, so any 100-character banner shows whole. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ines On CI's Chromium the two-logo banner fits one line at 17 px, at 21 px on a Mac and on the streamer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and the first answer is the voice In the five-model dress rehearsal 18 of the casters' 53 lines went out as captions only: the speech service answered 4 of 6 identical calls in about 1.7 s, one in 22.5 s and one not within a minute, past the 15 s cutoff. A call still out at 4 s, or one that failed, now gets a second request with the rest of the 15 s; the first answer wins, and a line neither voiced in 15 s is still captioned, as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…roadcast isn't cut off at its end Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Env Framework" Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale
added a commit
that referenced
this pull request
Oct 12, 2026
#19) Stacked on #18: merge that first. OpenCiv3's env becomes an [agentenv-game-env](https://github.com/earakely-scale/agentenv-game-env) game env. This tests how well game-env generalizes beyond Warcraft III: OpenCiv3 is turn-based, has many players and has human players. ## What changes - **The lobby says who plays.** `urn:game:lobby/v1` takes: - the game's settings: `seed`, `size`, `difficulty`, `barbarians`, `landform`, `ocean`, `turn_limit`, `human_turn_seconds`, `min_turn_seconds` and `broadcast`; - one player slot per civilization, played by an agent, a person or the game's AI. An agent's or person's slot takes a `civ`, and the next free one is used when it names none. The engine picks the AI's civs. Closing the lobby creates the game. - **Routing by address, not header.** An agent plays at `/players/<player_id>/mcp`, which `add_player_slot` registers with it. `X-OpenCiv3-Seat` is gone from the env and from the player agents. In a lobby game, the env's own `/mcp` plays no one. - **A person's token is the slot's secret.** It is passed through game-env 0.6.0's new `@player_slot_headers` hook. `add_player_slot` puts it in the page link's fragment, and the play page reads it from there. - **The match** (`urn:game:match/v1`) reports: - the turns; - each player's outcome, by the game's own rules (see below); - scores: score, cities, population and techs; - moves: turns a player ended itself, and turns the env ended for it. - **Outcome rules:** - A defeated civilization has lost. - A conquest, domination, culture or score victory goes to its civ, and every other player loses. - At the turn limit, a tie on top is no one's victory. - **Finishing and files.** `finish_match` plays a match out by ending every remaining player's turn. The match keeps its recording as files, by kind: `map_video`, `html_replay`, `last_turn` and `client_video`. Its spectator view is `/live`. - **Tasks.** - The ten multi-agent tasks and `human-vs-ai` use `open_lobby`, `add_player_slot`, `close_lobby`, `finish_match` (with `await_match` first in a person's game), `grade_match` and `save_match_files`. - `grade_match` uses `max_missed: 0.1`, the victor verifier's rule for an agent that sat out. - `human-vs-agents` grades the person in a second step, without the gate. - **Removed:** the `openciv3_match` and `openciv3_await_game` steps and the victor verifier. - **Kept on the default game:** `new-game` keeps the single-civilization default game. `smoke`, `play` and `full-game` stay on it, with their baseline verifiers and `save_env_recording`. - **Broadcast.** It stays OpenCiv3's own, now set through the lobby, and `agent-env openciv3 stream` is unchanged. Banner logos must now be `https://` URLs or `data:` URIs, which the env fetches as the lobby closes. - **Image.** The env image installs game-env from the archive that `pyproject.toml` pins; `setup` passes it in. ## Behaviour changes to know about - **Each agent gets its own grade:** 1 for a win, 0.5 for a draw, 0 for a loss. The victor verifier graded the match once. - **An AI civilization's score victory at the turn limit now means every agent lost.** The victor verifier ignored AI scores. This applies to `sol-vs-opus`, which has three AI civs. ## Verification - **Local tests:** 408 pass. The 93 failures and 22 errors are the same set as on #18; they need the real engine, which can't run on a Mac without .NET. Ruff is clean. - **Real engine, EC2 Linux, image built from this branch:** 493 pass, 20 skipped, none failed. That includes the multi-player seat and HTTP routing tests. - **Real runs** (env `openciv3-port`, Claude Haiku 4.5 players). In every run each agent played at its own slot address and the match files were kept. - **`three-agents-quick`, 10 turns:** each agent ended 10 of 10 turns itself, and both gates passed. At turn 10, two players tied on top, so no one won, and each agent got 0.5. - **`human-vs-ai`, 4 turns:** a scripted person played through the play page's API. - The link added by `add_player_slot` served the page. - The API refused calls without the token, and accepted them with it. - The person ended 4 turns, `await_match` returned at the end, and the full-game verifier graded it 0.42. - **`showmatch-quick`, 6 turns:** played with `agent-env openciv3 stream --record --offline`. The casters commented over the whole game, and the recording is a 32 MB video. Each agent got 0.5 for a three-way tie. - **`playtest/bots.py`, 3 seats, 5 turns:** 84 tool calls, none failed. ## Not in this pass Casters in game-env's broadcast, so Warcraft III gets casters back. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two player-harness bugs that could take an agent out of a long broadcast, a task for that broadcast, and sponsor banners on the stream. The bugs turned up in the probes for the 4-hour Astra vs Opus vs Kimi livestream (FD-3666).
A plan that says "GAME OVER" ended the game for that player.
end_turnresults. GPT-6 Sol wrote "Session ends after 40 end_turn calls or GAME OVER." into its plan.game_turntook "GAME OVER" anywhere in a result as the end of the game. So at its first session boundary Sol's player didn't start a new session; it stopped, at T40 of 270.[GAME OVER T…/…], or a line that opens with "GAME OVER" at the turn limit.A model request that never returns silenced the Claude player for the rest of the game.
tool_useagainst itstool_result, soend_turncan still wait as long as the other seats need.A task for a long broadcast:
five-way-war.Why these settings (FD-3666):
Sponsor banners on the stream (FD-3672).
broadcast.banners(up to 8) fill a sponsor slot at the right end of the stream's bottom row, the side panel's width.{name}stands, on the stream's dark colours or a light plate.https://URL, adata:URI or an image file.data:URI, so the stream never waits on another host.live/logo/<banner>/<name>) rather than sending its bytes on every poll, and serves SVGs with a no-script CSP.five-way-warreads "Powered by [Modal logo] Modal with [AgentEnv logo] AgentEnv Framework".--offline --record, $0.httpslogos on a light plate, text only, adata:URI logo, a local file logo, and a 99-character text.Casters: a slow voice gets a twin request.
five-way-war's sandboxes and prompts now last 10 h, so an 8-hour broadcast isn't cut off at its end.Tests:
test_the_game_turn_comes_from_the_last_footergains a plan, a message and a line-start "GAME OVER" mid-game.claudecan hang after a tool result, or make oneend_turnslow.test_a_model_that_goes_silent_ends_its_session_and_a_fresh_one_plays_on.test_a_long_tool_call_is_not_a_silent_model.pytest tests/agent tests/packagingpasses (156), andruff checkis clean. The agent images are rebuilt from this branch on the EC2 box, and a live probe is running on them.🤖 Generated with Claude Code