Skip to content

Advice margin 5, tested on fresh seeds - #138

Merged
ptescher merged 1 commit into
masterfrom
lookahead-margin
Nov 15, 2026
Merged

ptescher merged 1 commit into
masterfrom
lookahead-margin

Conversation

@ptescher

Copy link
Copy Markdown
Owner

What

ADVICE_MARGIN goes from 2 to 5: the computer commander takes the lookahead's proposal only when it gains more than 5 points. The number was found by looking at seeds 1..=400 (plans/actors.md, step 4), so it was tested on seeds nobody had looked at.

Rule, written before the run

Adopt 5 only if, on seeds 10001..=10800 (800 seeds, both sides, 1600 games a row), a commander with margin 5 beats one with margin 2 head to head, paired seeds, with a 95 % lower bound (margin - 1.96 error) above 0. Do not pick another number from the fresh table; report another margin that looks better as a lead for another fresh test.

Fresh seeds (cargo run --release -p cognition_towers --example margin 800 10000)

Margin Against no advisor Won Lost Against margin 2 (lower bound) Won Lost
2 -0.004 ± 0.031 833 767 +0.000 ± 0.031 (-0.060) 799 799
3 -0.031 ± 0.031 814 786 +0.074 ± 0.030 (+0.015) 841 758
5 +0.159 ± 0.032 916 683 +0.245 ± 0.030 (+0.187) 945 653
7 +0.103 ± 0.033 884 716 +0.076 ± 0.030 (+0.017) 836 763
10 +0.172 ± 0.033 923 677 +0.064 ± 0.030 (+0.005) 829 770

Verdict: the rule passes (lower bound +0.187 > 0), so the margin is 5. Margin 10 is level with 5 against no advisor and was not tested against 5 head to head: a lead for another fresh test, not a change. (Row 3 and row 7 beat margin 2 head to head while their score against no advisor is not above margin 2's; that is noise of about 1 sigma each, not explained.)

Before and after

advice 400 (seeds 1..=400), margin 2 then margin 5:

Delay Before After
0 s +0.05 ± 0.04 (430/370) +0.19 ± 0.04 (468/332)
2 s -0.16 ± 0.04 +0.03 ± 0.04
5 s -0.27 ± 0.04 -0.09 ± 0.04
10 s -0.55 ± 0.04 -0.19 ± 0.04
30 s -0.94 ± 0.04 -0.84 ± 0.04

judge --game captain --champion timid --challenger default --seeds 24: before +0.384 (lower bound +0.048, promoted, 30 won 18 lost); after +0.453 (lower bound +0.215, promoted, 32 won 16 lost).

Changes

  • the constant, its doc comment, and every place that states it: expectations 05-commander, 18-advice, glossary, domains/recreation/README.md, plans/actors.md.
  • Expectations that used 5 or 1 point as "enough" or "little": now 8 and 4 points (written first; "gains little" failed with margin 2, as it should). Unit tests in commander.rs likewise.
  • new example margin [SEEDS] [OFFSET] [MARGIN...].

Checks

cargo test --release -p recreation_towers -p cognition_towers -p recreation_towers_ground, cargo test --release -p recreation_towers_server --bins, cargo run -p expectation -- check expectations (0 errors), cargo fmt --check pass. cargo clippy --workspace --all-targets: only vendor warnings (gpui, xattr), none in the repo's own code.

🤖 Generated with Claude Code

Rule written first: margin 5 beats margin 2 head to head on seeds
10001..=10800 with a 95 % lower bound above 0. Result +0.245 (lower +0.187).
Adds the margin example.

Co-Authored-By: Claude <noreply@anthropic.com>
@ptescher
ptescher merged commit 0896e8d into master Nov 15, 2026
2 checks passed
@ptescher
ptescher deleted the lookahead-margin branch November 15, 2026 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant