Repository navigation
Description
Where a round's target comes from
emrg/server/competition_prompt.md is the operational contract for a competition round. Two of the
readings it depends on are described there in a way that leaves out what happens to them after the
round takes them, and both omissions put the same thing in the wrong place: a number a round compares
itself against, treated as a fact about the world.
A rule is not fixed at publication. §2 tells a round to hold its conclusions about what the rules
allow as readings with a source — the platform's own wording, quoted; a reading this round took; or an
assumption written into a prompt, labelled as an assumption. What it does not say is that the wording
it quotes is the text as first written. Competition platforms correct themselves afterwards:
clarifications, exceptions, changed evaluation rules and the outcome of appeals arrive in the platform's
own updates and discussion threads, sometimes days after the page was published and sometimes in
response to exactly the reading a round is relying on. A round that read the page once therefore holds
a conclusion about a text that has since moved, and nothing in the file asks it to look again. The
failure is silent, which is what makes it worth fixing in the prompt: the conclusion still cites a real
quote from a real page, and the round has no way to notice that the quote is no longer the rule.
The top of a leaderboard is an entry, not a standard. §5 asks a round to answer "how likely is the
goal to be reached" with the gap to the payout line or to the rank that pays, and §2 defines the goal
for a prize-less competition as leaderboard standing or percentile. Both take the standing on the board
as given. But the highest number there may be an entry the platform's rules do not permit — a leaked
target, an external dataset the rules exclude, a submission later disqualified — and in that case a gap
computed against it reports work that no permitted entry had to do. Worse, the levers such a gap implies
are the levers the rules forbid, so the round is pushed to imitate a route it may not take. A round
reading only the current file has no reason to ask whether the leader was eligible, because nothing in
it says eligibility is a question.
What the fix must satisfy
Add both rules to emrg/server/competition_prompt.md, in §2 — the section that already owns "how a
conclusion about the rules is read", not a new section beside it:
- The gap's target is the best standing the rules allow. Where that differs from the top of the
board, the round says which of the two its gap is computed against and what accounts for the
difference. An entry that is out of reach because the rules forbid the route is not evidence that
the route is needed. - The rules are re-read each round, because they are amended after publication. The corrections
live in the platform's updates and discussion threads; a conclusion drawn from the rules says which
round last read them; and a correction found there is settled against the rules page rather than
replacing it — the page is the rule, the thread is where it was corrected.
Both must be pinned by tests in tests/test_competition_prompt.py, following that file's existing
pattern: asserted against the section that owns the subject rather than against the whole document, so
that deleting the section fails the test instead of leaving a substring match standing.
The pin has to work in both directions, and both were measured before this was filed: with each
rule deleted from the template, exactly the test that pins it fails and the rest stay green; with the
template as committed, the whole file passes.
Where this comes from
The wording in this issue is a paraphrase of rules that an installation of this agent had written
into its own tasks.yml supplement to this task's prompt — the channel an installation has when a
general rule it needs is missing from the template itself. The supplement is a private file on one
host and is not part of this repository, so nothing here quotes it. The two rules above are its
content, rewritten for this template: no path, no account, no credential, and no host-specific
threshold crosses over.
The supplement carries eighteen such rules across its conclusion-discipline and measurement-discipline
sections. Sixteen of them are already stated in competition_prompt.md under other words — including
the approximate-identity test (a candidate correlating at 0.98 or above whose delta is within a tenth
of the reference's magnitude is the same answer in different clothes), the local-ruler-versus-board-ruler
ratio, aggregation-dependent selection cost, and the instruction to exhaust the platform's free
readings before spending a slot. Those need no change and are not part of this issue. The two above are
the remainder.
Activity
how2how2how2-arch commentedon Oct 30, 2026
Handled by #2657