Skip to content

Players survive a long broadcast; a five-model war task; sponsor banners on the stream - #18

Merged
earakely-scale merged 14 commits into
mainfrom
fix/claude-player-silent-model
Oct 12, 2026
Merged

earakely-scale merged 14 commits into
mainfrom
fix/claude-player-silent-model

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Two player-harness bugs that could take an agent out of a long broadcast, a task for that broadcast, and sponsor banners on the stream. The bugs turned up in the probes for the 4-hour Astra vs Opus vs Kimi livestream (FD-3666).

A plan that says "GAME OVER" ended the game for that player.

  • The env echoes each agent's plan in its end_turn results. GPT-6 Sol wrote "Session ends after 40 end_turn calls or GAME OVER." into its plan.
  • game_turn took "GAME OVER" anywhere in a result as the end of the game. So at its first session boundary Sol's player didn't start a new session; it stopped, at T40 of 270.
  • A rival's message quoting the words would do the same to everyone at their next boundary.
  • Now only the env says the game is over: its footer [GAME OVER T…/…], or a line that opens with "GAME OVER" at the turn limit.

A model request that never returns silenced the Claude player for the rest of the game.

  • A Kimi K3 request (Bedrock, through the proxy) hung with its connection open, past Claude Code's own 10-minute timeout. Kimi made 3 calls in 40 turns.
  • The env ends a silent seat's turns, so the game went on without China.
  • The Claude player now ends a session whose model has sent nothing for 120 s while no tool call is running. A fresh session picks the game up. Live on the probe it fired: a Kimi request hung at 08:26:39, and a fresh session was playing at 08:31:15. At 120 s the fresh session is back before the env ends the seat's turn (300 s), which holds the other seats.
  • It counts each tool_use against its tool_result, so end_turn can still wait as long as the other seats need.

A task for a long broadcast: five-way-war.

  • Five models: GPT-6 Astra (America), GPT-6 Sol (England) and GPT-5.6 Terra (Persia) on Codex; Opus 5.5 (Rome) and Sonnet 5.5 (Greece) on Claude Code.
  • No civilization run by the game's own AI, on a Tiny map with restless barbarians.
  • A 55 s pace (about 65 turns an hour) and a turn limit of 750, about 11 hours away, so in practice only a conquest ends the game. The broadcast runs until then, or until it is stopped.
  • Fresh sessions every 30 turns, and every 20 for Astra, which costs 5× Sol per token.
  • The prompt: "Build your empire, and make conquering the other civilizations your ultimate goal, as the game ends only when you have done so."
  • Terra is GPT-5.6 Terra: the proxy has no GPT-6 Terra.

Why these settings (FD-3666):

  • On a Small map, three agents kept the peace for 75 turns, at 70% or 80% ocean and with a 70- or 270-turn limit. They signed border pacts and traded techs, and only barbarians fought.
  • On a Tiny map, Opus declared war on Sol at turn 56, took New York the same turn, and razed Washington and Boston by turn 60.
  • A 10-turn smoke run of this task played all five seats to T10 in 217 s, for $2.15.
  • The estimate is about $160 for four hours with the casters, $211 in the high case; $232–300 for five hours.

Sponsor banners on the stream (FD-3672).

  • broadcast.banners (up to 8) fill a sponsor slot at the right end of the stream's bottom row, the side panel's width.
    • It is on screen in every shot, including the full-size client view and the agents' panels.
    • Banners take turns every 30 s with a crossfade.
  • A banner is text with logos inline where {name} stands, on the stream's dark colours or a light plate.
    • It shrinks to fit one line down to 16 px, then wraps onto two lines, and only then ends in an ellipsis.
  • A logo is an https:// URL, a data: URI or an image file.
    • The match step fetches or reads each one, checks it is an image of at most 512 KB, and gives the env a data: URI, so the stream never waits on another host.
    • No third-party logo is committed.
  • The env links each logo in its live data (live/logo/<banner>/<name>) rather than sending its bytes on every poll, and serves SVGs with a no-script CSP.
  • five-way-war reads "Powered by [Modal logo] Modal with [AgentEnv logo] AgentEnv Framework".
  • Smoke on the EC2 box: a real match, the real streamer --offline --record, $0.
    • Five banners rendered at 1080p: two https logos on a light plate, text only, a data: URI logo, a local file logo, and a 99-character text.
    • They rotated and came back round.
    • The slot stayed up in the agents-panel shot.
    • The live poll stayed at 2.3 KB.

Casters: a slow voice gets a twin request.

  • In the five-model dress rehearsal, 18 of the casters' 53 lines went out as captions only. The speech service answered 4 of 6 identical calls in about 1.7 s, one in 22.5 s and one not within a minute.
  • A voice request that fails, or isn't back in 4 s, now gets a second request with the rest of the 15 s, and the first answer is the voice.
  • A/B on the box, 20 lines each: the old single request voiced 19 of 20, losing one at 15 s; the twin request voiced 20 of 20, the slowest in 5.5 s.
  • five-way-war's sandboxes and prompts now last 10 h, so an 8-hour broadcast isn't cut off at its end.

Tests:

  • test_the_game_turn_comes_from_the_last_footer gains a plan, a message and a line-start "GAME OVER" mid-game.
  • The fake claude can hang after a tool result, or make one end_turn slow.
  • New: test_a_model_that_goes_silent_ends_its_session_and_a_fresh_one_plays_on.
  • New: test_a_long_tool_call_is_not_a_silent_model.

pytest tests/agent tests/packaging passes (156), and ruff check is clean. The agent images are rebuilt from this branch on the EC2 box, and a live probe is running on them.

🤖 Generated with Claude Code

earakely-scale and others added 3 commits October 7, 2026 01:11
…ng model request restarts the session

The harness took "GAME OVER" anywhere in a tool result as the end of the game. The env echoes the agent's plan in
end_turn results, and a plan that said "this session ends after 40 turns or GAME OVER" made GPT-6 Sol's player stop
at its first session boundary, at T40 of 270. A rival's message quoting the words would do the same. The game is
over now only when the env's footer says so, or a line opens with GAME OVER at the turn limit.

A Kimi K3 request through the proxy hung with its connection open, past Claude Code's own timeout, and the player
sat silent for the rest of the game. The Claude player now ends a session whose model has sent nothing for 240 s
while no tool call runs (end_turn may wait for the other seats as long as it needs), and a fresh session plays on.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…fore the env ends its turn (300 s)

Live on the probe: a Kimi K3 request hung at 08:26:39 and the fresh session started at 08:31:15; at 240 s the env had
already ended the turn after holding the other seats for 300 s.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
astra-opus-kimi seats the three on a Tiny map with 3 AI civs (Monarch, restless barbarians) for 300 turns at a 55 s
pace, 4 h 35 min unless a conquest ends it, with 30-turn sessions. The prompt is sol-vs-opus's for three: conquering
the other two models wins, the score only breaks a tie.

Probes for it (FD-3666): three agents on a Small map kept the peace for 75 turns, at 70 % or 80 % ocean and with a
70- or 270-turn limit; on a Tiny map Opus declared war at turn 56 and razed two American cities by turn 60. Astra
costs about 6x Sol a turn; the estimate for the whole broadcast is about $170 with the casters ($220 if the war
doubles the calls).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale earakely-scale changed the title Players: a plan that says GAME OVER no longer ends the game; a hung model request restarts the session Players survive a long broadcast, and a four-hour Astra vs Opus vs Kimi task Oct 7, 2026
earakely-scale and others added 2 commits October 7, 2026 01:45
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…p, told to conquer

GPT-6 Astra, GPT-6 Sol and GPT-5.6 Terra (Codex) and Opus 5.5 and Sonnet 5.5 (Claude Code) on a Tiny map with no
civilization run by the game's own AI, for 300 turns at a 55 s pace. The prompt makes conquering the other four the
goal; with four rivals to eliminate, the game is likelier to reach its limit (4 h 35 min or more) than to end
early. Terra is GPT-5.6 Terra: the proxy has no GPT-6 Terra. Estimated about $200 with the casters, $270 if the war
doubles the calls.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale earakely-scale changed the title Players survive a long broadcast, and a four-hour Astra vs Opus vs Kimi task Players survive a long broadcast, and a five-model war task for it Oct 7, 2026
earakely-scale and others added 5 commits October 7, 2026 03:49
Astra costs 5x Sol a token and, with four rivals in every brief, about 0.15 a turn from the start (smoke run, T0-T10).
Shorter sessions shrink the context each call re-reads, which brings the estimate's high case from about 303 to 270.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e goal, the turn limit is 750

The prompt now reads "Build your empire, and make conquering the other civilizations your ultimate goal, as the game
ends only when you have done so." With no score tiebreak to play for, the turn limit goes to 750, about 11 hours at a
55 s pace, so the broadcast runs until one model has conquered the rest or until it is stopped (about 65 turns an
hour; about $160 for four hours, estimated).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…al with AgentEnv Framework")

broadcast.banners (up to 8) fill a sponsor slot at the right end of the stream's bottom row, the side panel's width,
on screen in every shot: one at a time, crossfading every 30 s. A banner is text with logos inline where {name}
stands, shrunk to fit down to 13 px, then cut with an ellipsis, on the stream's dark colours or a light plate.

A logo is an https:// URL, a data: URI or an image file. The openciv3_match step fetches or reads each one, checks it
is a PNG, JPEG, WebP, GIF or SVG image of at most 512 KB, and gives the env a data: URI, so the game's settings hold
all the stream shows and a broadcast never waits on another host; no third-party logo is committed. The env takes
only data: URIs, links each logo in its live data (live/logo/<banner>/<name>) rather than sending its bytes on every
poll, and serves it from that route, an SVG without scripts.

five-way-war's slot reads "Powered by Modal with AgentEnv Framework", with Modal's GitHub avatar and AgentEnv's icon.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In the smoke run a 78-character banner lost the end of its link at 13 px on one line. Now a banner shrinks to 16 px on
one line, then takes two lines from 17 px down to 13 px, so any 100-character banner shows whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ines

On CI's Chromium the two-logo banner fits one line at 17 px, at 21 px on a Mac and on the streamer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale earakely-scale changed the title Players survive a long broadcast, and a five-model war task for it Players survive a long broadcast; a five-model war task; sponsor banners on the stream Oct 7, 2026
earakely-scale and others added 4 commits October 7, 2026 08:09
… and the first answer is the voice

In the five-model dress rehearsal 18 of the casters' 53 lines went out as captions only: the speech service answered
4 of 6 identical calls in about 1.7 s, one in 22.5 s and one not within a minute, past the 15 s cutoff. A call still
out at 4 s, or one that failed, now gets a second request with the rest of the 15 s; the first answer wins, and a
line neither voiced in 15 s is still captioned, as before.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…roadcast isn't cut off at its end

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…Env Framework"

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit 35ad20f into main Oct 12, 2026
5 checks passed
@earakely-scale
earakely-scale deleted the fix/claude-player-silent-model branch October 12, 2026 01:28
earakely-scale added a commit that referenced this pull request Oct 12, 2026
#19)

Stacked on #18: merge that first.

OpenCiv3's env becomes an
[agentenv-game-env](https://github.com/earakely-scale/agentenv-game-env)
game env. This tests how well game-env generalizes beyond Warcraft III:
OpenCiv3 is turn-based, has many players and has human players.

## What changes

- **The lobby says who plays.** `urn:game:lobby/v1` takes:
- the game's settings: `seed`, `size`, `difficulty`, `barbarians`,
`landform`, `ocean`, `turn_limit`, `human_turn_seconds`,
`min_turn_seconds` and `broadcast`;
- one player slot per civilization, played by an agent, a person or the
game's AI.
  
An agent's or person's slot takes a `civ`, and the next free one is used
when it names none. The engine picks the AI's civs. Closing the lobby
creates the game.
- **Routing by address, not header.** An agent plays at
`/players/<player_id>/mcp`, which `add_player_slot` registers with it.
`X-OpenCiv3-Seat` is gone from the env and from the player agents. In a
lobby game, the env's own `/mcp` plays no one.
- **A person's token is the slot's secret.** It is passed through
game-env 0.6.0's new `@player_slot_headers` hook. `add_player_slot` puts
it in the page link's fragment, and the play page reads it from there.
- **The match** (`urn:game:match/v1`) reports:
  - the turns;
  - each player's outcome, by the game's own rules (see below);
  - scores: score, cities, population and techs;
  - moves: turns a player ended itself, and turns the env ended for it.
- **Outcome rules:**
  - A defeated civilization has lost.
- A conquest, domination, culture or score victory goes to its civ, and
every other player loses.
  - At the turn limit, a tie on top is no one's victory.
- **Finishing and files.** `finish_match` plays a match out by ending
every remaining player's turn. The match keeps its recording as files,
by kind: `map_video`, `html_replay`, `last_turn` and `client_video`. Its
spectator view is `/live`.
- **Tasks.**
- The ten multi-agent tasks and `human-vs-ai` use `open_lobby`,
`add_player_slot`, `close_lobby`, `finish_match` (with `await_match`
first in a person's game), `grade_match` and `save_match_files`.
- `grade_match` uses `max_missed: 0.1`, the victor verifier's rule for
an agent that sat out.
- `human-vs-agents` grades the person in a second step, without the
gate.
- **Removed:** the `openciv3_match` and `openciv3_await_game` steps and
the victor verifier.
- **Kept on the default game:** `new-game` keeps the single-civilization
default game. `smoke`, `play` and `full-game` stay on it, with their
baseline verifiers and `save_env_recording`.
- **Broadcast.** It stays OpenCiv3's own, now set through the lobby, and
`agent-env openciv3 stream` is unchanged. Banner logos must now be
`https://` URLs or `data:` URIs, which the env fetches as the lobby
closes.
- **Image.** The env image installs game-env from the archive that
`pyproject.toml` pins; `setup` passes it in.

## Behaviour changes to know about

- **Each agent gets its own grade:** 1 for a win, 0.5 for a draw, 0 for
a loss. The victor verifier graded the match once.
- **An AI civilization's score victory at the turn limit now means every
agent lost.** The victor verifier ignored AI scores. This applies to
`sol-vs-opus`, which has three AI civs.

## Verification

- **Local tests:** 408 pass. The 93 failures and 22 errors are the same
set as on #18; they need the real engine, which can't run on a Mac
without .NET. Ruff is clean.
- **Real engine, EC2 Linux, image built from this branch:** 493 pass, 20
skipped, none failed. That includes the multi-player seat and HTTP
routing tests.
- **Real runs** (env `openciv3-port`, Claude Haiku 4.5 players). In
every run each agent played at its own slot address and the match files
were kept.
- **`three-agents-quick`, 10 turns:** each agent ended 10 of 10 turns
itself, and both gates passed. At turn 10, two players tied on top, so
no one won, and each agent got 0.5.
- **`human-vs-ai`, 4 turns:** a scripted person played through the play
page's API.
    - The link added by `add_player_slot` served the page.
- The API refused calls without the token, and accepted them with it.
- The person ended 4 turns, `await_match` returned at the end, and the
full-game verifier graded it 0.42.
- **`showmatch-quick`, 6 turns:** played with `agent-env openciv3 stream
--record --offline`. The casters commented over the whole game, and the
recording is a 32 MB video. Each agent got 0.5 for a three-way tie.
- **`playtest/bots.py`, 3 seats, 5 turns:** 84 tool calls, none failed.

## Not in this pass

Casters in game-env's broadcast, so Warcraft III gets casters back.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant