Skip to content

emrg: a competition round's target is a reading, and the rules it quotes are amended after publication #2656

Closed

Description

@how2how2how2-arch

Where a round's target comes from

emrg/server/competition_prompt.md is the operational contract for a competition round. Two of the
readings it depends on are described there in a way that leaves out what happens to them after the
round takes them, and both omissions put the same thing in the wrong place: a number a round compares
itself against, treated as a fact about the world.

A rule is not fixed at publication. §2 tells a round to hold its conclusions about what the rules
allow as readings with a source — the platform's own wording, quoted; a reading this round took; or an
assumption written into a prompt, labelled as an assumption. What it does not say is that the wording
it quotes is the text as first written. Competition platforms correct themselves afterwards:
clarifications, exceptions, changed evaluation rules and the outcome of appeals arrive in the platform's
own updates and discussion threads, sometimes days after the page was published and sometimes in
response to exactly the reading a round is relying on. A round that read the page once therefore holds
a conclusion about a text that has since moved, and nothing in the file asks it to look again. The
failure is silent, which is what makes it worth fixing in the prompt: the conclusion still cites a real
quote from a real page, and the round has no way to notice that the quote is no longer the rule.

The top of a leaderboard is an entry, not a standard. §5 asks a round to answer "how likely is the
goal to be reached" with the gap to the payout line or to the rank that pays, and §2 defines the goal
for a prize-less competition as leaderboard standing or percentile. Both take the standing on the board
as given. But the highest number there may be an entry the platform's rules do not permit — a leaked
target, an external dataset the rules exclude, a submission later disqualified — and in that case a gap
computed against it reports work that no permitted entry had to do. Worse, the levers such a gap implies
are the levers the rules forbid, so the round is pushed to imitate a route it may not take. A round
reading only the current file has no reason to ask whether the leader was eligible, because nothing in
it says eligibility is a question.

What the fix must satisfy

Add both rules to emrg/server/competition_prompt.md, in §2 — the section that already owns "how a
conclusion about the rules is read", not a new section beside it:

  1. The gap's target is the best standing the rules allow. Where that differs from the top of the
    board, the round says which of the two its gap is computed against and what accounts for the
    difference. An entry that is out of reach because the rules forbid the route is not evidence that
    the route is needed.
  2. The rules are re-read each round, because they are amended after publication. The corrections
    live in the platform's updates and discussion threads; a conclusion drawn from the rules says which
    round last read them; and a correction found there is settled against the rules page rather than
    replacing it — the page is the rule, the thread is where it was corrected.

Both must be pinned by tests in tests/test_competition_prompt.py, following that file's existing
pattern: asserted against the section that owns the subject rather than against the whole document, so
that deleting the section fails the test instead of leaving a substring match standing.

The pin has to work in both directions, and both were measured before this was filed: with each
rule deleted from the template, exactly the test that pins it fails and the rest stay green; with the
template as committed, the whole file passes.

Where this comes from

The wording in this issue is a paraphrase of rules that an installation of this agent had written
into its own tasks.yml supplement to this task's prompt — the channel an installation has when a
general rule it needs is missing from the template itself. The supplement is a private file on one
host and is not part of this repository, so nothing here quotes it. The two rules above are its
content, rewritten for this template: no path, no account, no credential, and no host-specific
threshold crosses over.

The supplement carries eighteen such rules across its conclusion-discipline and measurement-discipline
sections. Sixteen of them are already stated in competition_prompt.md under other words — including
the approximate-identity test (a candidate correlating at 0.98 or above whose delta is within a tenth
of the reference's magnitude is the same answer in different clothes), the local-ruler-versus-board-ruler
ratio, aggregation-dependent selection cost, and the instruction to exhaust the platform's free
readings before spending a slot. Those need no change and are not part of this issue. The two above are
the remainder.

Activity

  1. how2how2how2-arch commented on Oct 30, 2026

    @how2how2how2-arch
    CollaboratorAuthor

    Handled by #2657

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions