Batched sends in main: a loop that waits holds its messages, and a nested loop loses the outer loop's #3117

A loop in main() that sends a message and then waits does not deliver the message until the loop ends, or until 256 messages have been sent. The compiler opens a send batch around every while loop of a non-actor function that contains a send (scheduler_send_batch_start before the loop, scheduler_send_batch_flush after it). Batched sends leave the thread only at the flush or when the buffer fills, whatever the loop does in between.

message Tick { n: int }
actor Clock {
    state seen = 0
    receive { Tick(n) -> { seen = seen + 1 } }
}

main() {
    c = spawn(Clock())
    other = spawn(Clock())
    i = 0
    while i < 3 {
        c ! Tick { n: i }
        sleep(300)
        println("after send ${i}, handled so far: ${c.seen}")
        i = i + 1
    }
    wait_for_idle()
    println("handled at the end: ${c.seen}")
}

On 0.839.0:

after send 0, handled so far: 0
after send 1, handled so far: 0
after send 2, handled so far: 0
handled at the end: 3

Each Tick waits 300 ms, 600 ms, 900 ms for the loop to end. A heartbeat loop, a loop that reads a line and sends it, or a loop that sleeps between sends gets its messages in bursts of 256, or not at all if it never ends.

Two more effects of the same batch:

• Order. Only a message with one int field is batched (scheduler_send_batch_add). A message with a payload (aether_send_message), and an ask, go around the batch, so they can overtake earlier one-int messages from the same loop to the same actor.
• wait_for_idle() inside the loop does not see the batched messages: they are not sent yet, so nothing counts them as pending.

Until #3112, a program with one actor did not show this, since its sends ran the handler inline. With #3112 every program does.

Fix:

• The compiler opens a batch only around a loop that cannot wait between its sends. Its body calls nothing and asks nothing; a send, arithmetic, an assignment and a branch are fine. That is the counting and fork-join shape the batch exists for.
• Inside such a batch, every send joins it, the payload kind included.
• A send that does go around a pending batch (from a function the loop's code calls, through the runtime) flushes it first, and so does wait_for_idle().
• On a scheduler thread nothing is batched: its sends keep their own core's channel and overflow order.

Worse: a loop with sends inside another loses the outer loop's messages. The inner loop opens its own batch, and scheduler_send_batch_start empties the buffer, discarding whatever the outer loop had batched:

message Add { n: int }
actor Sum {
    state count = 0
    receive { Add(n) -> { count = count + 1 } }
}

main() {
    a = spawn(Sum())
    b = spawn(Sum())
    i = 0
    while i < 10 {
        a ! Add { n: 1 }
        j = 0
        while j < 2 {
            b ! Add { n: 1 }
            j = j + 1
        }
        i = i + 1
    }
    wait_for_idle()
    println("a got ${a.count} of 10, b got ${b.count} of 20")
}

On 0.839.0 it prints a got 0 of 10, b got 20 of 20: every message to a is lost, and its payload leaks.

Fix: opening a batch never drops what one already holds. Only a flush empties the buffer.