Repository navigation
Every open bug, batch 6: no bug or critical issue left open - #3101
Merged
Merged
Conversation
A closure slot given `null` holds the empty closure, `{ .fn = 0 }`. A call
through it, `let c: fn = null` then `c()`, a field or parameter given
`null`, an array element, or a `fn?` holding `null` (a present value, so
`match`'s `some` arm and `!` take it), jumped to address 0: an access
violation with no message.
Every closure call is emitted through `_aether_closure_fn(f.fn, where)`, a
static inline in the prelude that panics with `where` when the function is
missing. `where` is a literal built at the call: the category
`empty_closure_call`, then the call's file, line and column. The erased
dispatch (#2847) needs nothing of its own, since no branch matches a null
function and the plain call under it is the checked one.
The reference says what `null` in a `fn?` is, a present empty closure,
and that `none` is the absence of any closure, and lists the new category
under Panic categories.
Not covered: a typed C function pointer (`fn(int) -> int`, a `cfn`) is no
closure, and a call through a null one still jumps to 0.
Test: tests/regression/test_issue3091_empty_closure_call.ae covers a bare
`fn`, a field, a parameter called directly, through `call` and as a
closure type with a keep mode, an array element, a `fn?` some arm and
unwrap, each caught with its position, and `none` taking the none arm. It
crashes before (0xC0000005 on both C paths) and passes after.
tests/integration/panic_categories pins the category token.
Every regression test that uses a closure (337 files) builds and runs on
both C paths as before; the three that fail (test_fs_stat_kind_socket_fifo,
test_os_run_full, test_run_capture_status) fail the same way before, as
POSIX-only checks on Windows.
This was referenced Oct 22, 2026
…so order holds and a ring gathers on one core (#3090, #3093) Two scheduler defects in moving actors between cores, and one fix for both. #3093, ordering. A message one actor sends another goes either straight into the target's mailbox (a local send) or through a core's queues (a from_queue, or a sender's overflow buffer on its way to one). When the target was moved while a message of the second kind was on its way, the old core forwarded that message after it, and the sender's next message, sent to the new core directly, could be stepped first. A message re-queued because the mailbox was full went to the back of a queue and could be overtaken the same way. A test with eight senders streaming numbered messages to four targets saw 14,400 to 21,924 of 3,200,000 out of order on the runtime before (three runs). #3090, rings never gathered. A cross-core send set migrate_to on its target, and the target's core moved it after running the message, onto the core the sender had just left. So each actor in a chain took the place of the one before it, and a thread ring only shifted around the cores: 3.03 M msg/s, on 5.6 s of CPU. The in-flight protocol: - Every message routed through a core's queues is counted in its target's new `inflight` (ActorBase, AETHER_ACTOR_BASE_FIELDS) before the sender reads assigned_core, and uncounted when it reaches the mailbox. That covers send_through_core, the batch flush (one count per actor in the batch), send_buffer_flush, the I/O poller's events, re-queues on a full mailbox and the overflow paths. A sender that finds the AETHER_INFLIGHT_MOVING mark waits for the move to end, then reads the new core. - Every move between cores is made in move_actor, by the actor's own core, while it steps nothing, under both table locks and the step lock, and only after a CAS of `inflight` from 0 to MOVING. No message for the actor is then anywhere but its mailbox, which moves with it. Work stealing goes through it too: the idle core asks for an actor by migrate_to, and the busy core moves it. - Message-driven migration moves the actor before it runs the message that brought the request, so a chain's next actor follows the message. A busy actor is asked again after its batch. - Delivery takes a run: the messages of one actor that follow each other in a batch go into its mailbox in order and are uncounted with one decrement. A full mailbox is made room in by stepping the actor there, its owner (make_room), so no message gives up its place. A main-thread actor waits for main-thread mode to let it go first. - A local send to an actor with messages still in the queues goes behind them (send_local, send_through_core). overflow_flush delivers its own core's entries directly only while that core's own queue is empty. A message re-queued onto its own core goes behind that core's overflow (requeue_own). Measured, Windows i7-13700K, 1,000,000 messages, 1 warmup and 10 interleaved runs, M msg/s, median (range), CV; CPU s: base (#3087) this thread ring 3.03 (2.92-3.14) 2.1% 5.555 24.91 (24.77-25.06) 0.4% 0.078 fork-join 30.23 (25.78-33.01) 6.2% 0.289 39.69 (37.97-42.08) 3.4% 0.281 ping-pong 12.36 (12.08-12.56) 1.4% 12.56 (11.82-12.70) 2.1% counting 42.84 (42.28-43.07) 0.6% 42.90 (37.73-43.01) 3.8% skynet 1.31 (0.66-1.68) 37.4% 1.69 (1.11-2.14) 16.3% The ring gathers on one core in its first lap: 8.2x the rate on 1/71 of the CPU time. Fork-join pays nothing for the counting. A first version counted and uncounted each message alone and cost it 8% (28.25 against 30.83). Delivered as runs, with one decrement per run, a worker's batch is now stepped together, and fork-join is 31% faster. Ping-pong and counting are unchanged; counting's range is one slow run. The C test actors stepped more than one message per call. docs/c- embedding.md already says a step takes one: the scheduler counts every call as one message processed. Those actors passed only while every step found one message waiting. Delivered a run at a time, a mailbox holds several, a drained batch counted as one, and scheduler_wait never balanced: "Scheduler bidirectional ping-pong" hung in its shutdown with all 50 pings and pongs through. They take one message per call now: counter, order, pingpong, echo and burst in test_scheduler.c, the three in test_scheduler_stress.c (whose stress step also took the 101st message out of the mailbox without counting it), wsteal_step, and the bench_scheduler and bench_zerocopy actors. benchmarks/http's servers drain too, but never wait for quiescence. Test: tests/regression/test_actor_pair_fifo.ae, eight senders over the cores and four targets, 3,200,000 numbered messages, each target checking every sender's sequence. Before: 21,924, 14,400 and 15,485 out of order. After: 0 in three runs. A variant with a slow handler (4000 rounds, handler loop of 400), so mailboxes fill and make_room runs: 0 in three. Checked: make test (593) passes three times. The 35 actor, ask, scheduler and cooperative tests in tests/regression and tests/syntax pass. The ownership grid's actor cells (P11, P12K, P12D) are 155 OK, none worse than the baseline. The POSIX side compiles with -Wall -Wextra and no warning (MSYS2 gcc -fsyntax-only).
#3097) An extern without @c_import gets a C prototype the compiler writes from its Aether types (long is int64_t, int is int, float is double, bool is int). The C that defines the function is compiled on its own, so no C compiler sees the two side by side, and where they disagree the call is made with the wrong width. #3097 named the C `long` case (32 bits on Windows, 64 elsewhere). A check over the whole tree found more. tests/extern_widths/check.ae, in Aether, reads every .ae and .c file under std, contrib, runtime, tests, examples, tools, lsp and benchmarks: - For each C file, it takes every function defined at file scope, its result and argument types reduced to a width class. Comments, literals and preprocessor lines are skipped, braces counted, and macro calls before a definition dropped. - For each extern, it pairs it with the C definitions of the symbol it binds (@extern("sym") included): those in its own directory when that defines it, else the shared runtime, std and contrib. Every definition is checked, so each #ifdef branch counts. - It reports any slot whose widths differ: arity, a result Aether reads that C does not return, and any plain C `long` in a bound signature, since no Aether type has its width on every platform. - Not checked: externs marked @c_import (the C compiler checks those against the header), externs with no C here (libc, system and vendored libraries), and types it cannot size (typedefs, structs by value). A pointer-sized C integer counts as 64 bits, since every target that builds the bound C is 64-bit. `make check-extern-widths` builds it and runs a self-test first: over tests/extern_widths/fixture, where it must report exactly the fixture's 11 mismatches (one of each kind, including an @extern alias and an #ifdef branch) and none of its agreeing definitions, macro-made ones, statics, or definitions in comments and strings. Then it runs over the tree. `make ci` runs it after check-tests. The whole tree takes 0.6 s: 4034 externs, 4557 C definitions. It found 55 externs that disagreed with their C (88 counting each #ifdef branch). All are fixed: - A C `long`, cut to 32 bits on Windows: aether_observe, aether_observe_fields and aether_unobserve's token (the runtime's tokens are int64_t throughout, the C test too); aether_tracking_bytes (and its sequence numbers, printed with %lld); the Lua host's lua_vm_set_global_int, lua_vm_result_int, lua_arg_int, lua_push_int, lua_table_seti_int, lua_table_set_int and lua_vm_set_instruction_limit, stubs included. Lua integers are 64-bit, and these are int64_t now. aether_casper_pwd_uid returns int, as std.casper reads it. getrusage_max_rss_kb in eight tests/integration shims returns int, as their probes read it. - An Aether `int` for a C size_t or uint64_t: the upper half of a 32-bit argument is unspecified on x86-64 and AArch64, so the callee could read garbage, such as the rest of a 64-bit value the int was cut from. Now `long`: - string_new_with_length's length, in std.string, std.brotli, std.cryptography, std.encoding, std.fs, std.http, std.lzf, std.zlib, std.zstd, contrib.avcodec, contrib.sqlite and a test probe. Also in the compiler's prelude (codegen.c, two declarations, int64_t), with the casts at its uses (the take of a C string, the actor-message string copy in codegen_expr.c). All of these share one prototype in a program, so they change together. - std.alloc's sizes and std.arena's. arena.used and arena.size return `long`, and docs/stdlib-reference.md says so. - Every contrib host's *_run_sandboxed_with_map map token (aether x2, duktape, factor, go x2, js, lua, perl, python, racket, rhombus, ruby). - aether_set_caller's count in the caller_info test. - An int argument widens to a long parameter at a call, so no caller changes. - tinygo_call_dynamic's `char return_kind` is an int, the width the extern passes. Found on the way: the Lua host's lua_push_double tested |v| against (long)v, so a negative whole Double never matched and went to Lua as a string. It compares v now, and checks the range before converting, where a double past 2^63 was undefined. docs/c-interop.md: the type table gains `long` -> int64_t, and a note says widths must agree, that a C size_t or uint64_t is a `long`, and why a bound C function must not use plain `long`. Checked: - make check-extern-widths: the self-test reports its 11, the tree 0. - make test (593) passes, the observe tests with int64_t tokens included. - make test-ae passes (RESULT). - The POSIX side compiles with -Wall -Wextra and no warning (MSYS2 gcc -fsyntax-only), compiler included. - Not built here: contrib/host/lua, tinygo and avcodec (no Lua, libffi or FFmpeg on this machine).
…3094) `state id`, `state active`, `state mailbox` (any of the 21 names in AETHER_ACTOR_BASE_FIELDS) passed the checker, then failed in the C compiler: "duplicate member 'id'". The generated actor struct spliced the base's fields in beside the states, so a state of one of those names was a second member of it. The base is a member of its own now. The generated struct starts with `ActorBase _aether_base;` and the states follow under the names the program gave them. The scheduler still gets a pointer to an ActorBase: the member is first, at offset 0. That is now strictly conforming C, where a spliced prefix relied on a common initial sequence. The static assertion beside each actor checks offsetof(Name, _aether_base) == 0 instead of the last base field's offset. Everything the compiler emits on a generated actor's base goes through the member now. That covers the step function (mailbox, active, the receive timeout's timeout_ns and last_activity_ns), the spawn function (active, migrate_to, dead, auto_process, thread, destroy_state, the timeout) and the typed send (assigned_core). All of these are in codegen_actor.c; nothing else reaches the base of a generated actor except through an ActorBase pointer. A C embedder's own actor structs, built with the macro or with an `ActorBase` member, are unchanged. A state named `_aether_base` itself is the one name left: the checker refuses it with "a state cannot be named '_aether_base'" and a hint, so it never reaches the C compiler. docs/actor-concurrency.md says how a generated actor holds the base. docs/architecture.md and docs/scheduler-quick-reference.md list `inflight` among the base's fields (#3090 added it). Tests: - tests/regression/test_actor_state_base_field_names.ae: an actor with 13 states named like base fields, written by its handlers, read back from main and answered through `ask`. A second actor arms a receive timeout at spawn beside states named timeout_ns and last_activity_ns, which keep their own values. On the compiler before: "duplicate member" for each. - tests/integration/actor_state_base_member_reject: the checker refuses `state _aether_base` with its message, and no C is written. Found while testing: a receive timeout never fires in a program with two actors or more, because no core steps an armed actor with an empty mailbox. Filed as #3102, fixed in its own commit. Checked: make test (596) passes, std/actors' registry test passes, and the full make test-ae sweep (RESULT).
`receive { ... } after N -> { ... }` fired only in main-thread mode (a
program with one actor), where scheduler_wait polls actors with an armed
timeout and steps them. The generated step function tests the deadline,
so a timeout fires only when something steps the actor. On the cores:
- a core's scan skipped every actor with an empty mailbox and no move
pending;
- an actor's own thread stepped it only for a message;
- the multicore scheduler_wait did not wait for timeouts.
So the timeout of an actor that received nothing never fired, and
wait_for_idle() returned at once. tests/syntax/test_actor_timeout_multiple.ae
printed "PASS: both timeouts fired" with neither handler's line in its
output.
- The scan (scheduler_thread): an actor with an empty mailbox and an
armed timeout is stepped, under its step lock, once its deadline has
passed. The test is the generated step's own (now - last_activity_ns >=
timeout_ns), and both read the same monotonic clock. An actor with no
timeout costs one more load of a field beside the two the scan already
reads. A parked core wakes within its timed wait (32 ms at most), so a
timeout fires at most that late.
- The actor thread (aether_actor_thread): the same, before it spins or
parks; its park is a timed wait too.
- scheduler_wait: once the cores are quiet, an armed timeout keeps it
waiting, 1 ms at a time, because its arm may send more. It waits as
long as main-thread mode's wait does (100,000 rounds).
Tests:
- tests/runtime/test_scheduler.c, "Scheduler fires receive timeouts on a
core and an actor thread": two actors whose step tests a 20 ms deadline
as a generated one does, one stepped by its core and one by its own
thread, and neither sent anything. Each must fire within 10 s. Without
the fix neither fires (the test waits out its 10 s each).
- tests/syntax/test_actor_timeout_multiple.ae now checks that both
timeouts fired, after wait_for_idle(). It prints both handler lines and
passes.
- The other timeout tests (test_actor_timeout, _cancel, _variable,
test_receive_arm_heap_locals) and examples/actors/timeout-demo.ae run
as before.
docs/actor-concurrency.md, "Actor Timeouts", says how the timeout fires
with no message and that wait_for_idle() waits for it.
Found while testing #3094.
Checked: make test (597) passes, and the full make test-ae sweep
(RESULT).
…3029) Called where the C compiler cannot see the argument's origin (another translation unit, a function value, a library boundary), every index of a `T[]` parameter tested for the unbounded-view sentinel and then the bound. GCC cannot unswitch that pair, so the loop around it did not vectorize: 4.2x on an int sum and 6.4x on an f32 map at -O3 in the issue's table. The owner chose codegen versioning (option 2). A function whose body indexes slice parameters that it never binds again, never takes the address of, and holds no closure, is emitted twice (codegen_func.c, generate_function_definition): - `name__aether_view`, static, where an index of those parameters is aether_slice_view_at, the element's address with no test, and a store is aether_slice_view_at_w, which keeps the read-only check (runtime/aether_slice.h); - the function itself, whose entry tests once `if (xs.len == AETHER_SLICE_UNBOUNDED && ...) return name__aether_view(args)` and otherwise runs its own checked body. A view has no length to test, which is what the checked access does with one too, so both bodies give the same answers and the same panics. Per function, not per loop: a body's declarations, defers and labels are the function's own. Not versioned: - a slice bound again (`xs = xs[1..3]`), since its uses are then not all the argument as it arrived; - a slice whose address is taken (`&xs`): through the address it could be made bounded after the entry's test, and the view body would index it with no check; - a body holding a closure or a quote, which is a body of its own, emitted once. The second rendering of the same source reported every diagnostic twice: the AETHER_EXPLAIN_THREADED notes of tests/integration/threaded_dispatch came out 10 for 5 loops in the make test-ae sweep. Diagnostics can now be muted (aether_diagnostics_set_muted in aether_error.c: neither printed nor counted); codegen mutes the view body, and the function's own body, emitted after it, reports each once. The threaded-dispatch notes check the same flag. That test's VM functions take their program as a slice, so each loop is threaded in both bodies: 18 tables for 9 loops, as the test now says. Measured with the origin hidden (each kernel called through a function value chosen at run time, n = 4096, 200,000 reps, a view and a bounded slice over the same memory), 7 interleaved rounds in a quiet window, ps per element, medians, before -> after: | kernel | -O2 | -O3 | -O3 -march=native | |------------------|-----------|----------|-------------------| | int sum, view | 193 -> 191 | 194 -> 83 | 217 -> 71 | | f32 map, view | 291 -> 174 | 302 -> 53 | 217 -> 42 | | int sum, bounded | 194 -> 195 | 196 -> 193 | 216 -> 214 | | f32 map, bounded | 291 -> 289 | 291 -> 289 | 218 -> 216 | The f32 map at -O3 reaches 53 ps against the issue's 45 ps upper bound, and -O2 matches its 1.7x and 1x. The int sum accumulates in a `long`, so each element is widened before it is added, which halves the vector width against the issue's int accumulator: 83 ps, against 47. Bounded slices take the checked body, as before. Tests: - tests/compiler/test_codegen.c: the view clone and the entry test are emitted, and a slice bound again, addressed or captured is not versioned. - tests/regression/test_slice_view_versioning.ae: - sums and fills give the same answers through views and bounded slices; - a view beside a bounded slice takes the checked body; - a bounded slice still traps out of bounds inside a versioned function. - tests/integration/threaded_dispatch: 5 notes, 18 tables. docs/language-reference.md (Slices) says when a function is versioned and what it changes. Checked: make test (597) passes, as do the regression test, threaded_dispatch, lane_wide and the codegen tests.
`.len` is a value computed from the slice (#1286): a slice's length is fixed when the slice is made, and an array's is a constant. A write to it passed the checker and stopped at the C compiler on generated code, a store to a non-lvalue ("lvalue required"): xs.len = 2 ((int)aether_slice_len(xs)) = 2 xs.len += 1 ((int)aether_slice_len(xs)) = (... + 1) xs.len++ (((int)aether_slice_len(xs)))++ a.len = 3 ((4) = 3) (int[4] a) reject_length_write (typechecker.c) refuses each, by the checker, at the three places a write is checked: - the assignment statement; - `=` as an expression, which a compound assignment becomes; - the operand of `++` and `--`, prefix and postfix. The message names the slice ("cannot write to the length of 'xs': a slice's length is fixed when the slice is made, and an array's is a constant"). The hint points at `xs[lo..hi]` for fewer elements. A struct's own field named `len` is storage like any other and takes every form. Found testing #3029, whose versioning asks whether a function writes to its slice parameters. Test: tests/integration/length_write. It refuses `=`, a compound assignment, `++` postfix, `--` prefix and an array's `.len = 3`, each with the message and the hint, and builds and runs a struct with a `len` field that takes them all. On the old compiler every refused program compiled. docs/language-reference.md (Slices): `s.len` is read, never written. Checked: make test (597) passes.
cmp and diff come from diffutils, which an MSYS2 install can lack (the
Windows CI shell has neither). A test that called one there failed for
the missing tool and read as a wrong result. In the local make test-ae
sweep:
[FAIL] ae_add_linux_libc: wrong bytes for linux-x86_64
...: line 98: cmp: command not found
tests/lib/same_bytes.sh defines `same_bytes a b`. It uses cmp where
there is one and otherwise compares cksum's line (CRC and byte count;
coreutils, on every leg). A file it cannot read matches nothing. Every
comparison of that kind goes through it:
- ae_add_linux_libc: its five cmp calls;
- http_server_h2: the POST round trip;
- fmt_gate: the branch that names which invariant broke, which named the
wrong one without cmp;
- tests/differential/opt_diff_one.sh: its own cksum fallback is replaced
by the helper;
- tests/differential/run_differential.sh: diff -u decided whether two
outputs agree, so without diff every case read as diverged. diff now
only shows a difference, when there is one and diff is present.
Its self-check proved only that a difference is detected, which a tool
that cannot run also "detects". It now proves both directions, as
opt_diff_one's --self-check already did.
Checked: ae_add_linux_libc passes in the MSYS2 shell that has no cmp
(it failed there), fmt_gate passes, and the opt_diff self-check passes.
make test (597) passes.
…oot (#2488) The `make test-ae` list built and run with -fsanitize=undefined on Linux (ubuntu-22.04, GCC 11), with libaether, aetherc and ae instrumented too: CI run 39711071570, on 1a55d92. Six programs wrote reports, from five causes, and one test failed. Each is fixed where it lives: - std/lzf/lzf_c.c:159, a misaligned u16 load (test_lzf_roundtrip, test_issue2301_codec_slices). Our lzfP.h set STRICT_ALIGN to 0, against its own comment ("do it if unsure ... disable by default"), so the compressor compared two bytes as `*(u16 *)p` at any address. It is 1 now: the byte-wise compare reads the same two bytes, so the output is unchanged, and GCC fuses it into one load where the target allows. - std/mem/aether_mem.c:585 and 590, mem.get_u64_le and get_u64_be (crypto_ascon_aead, crypto_ascon_aead128). `(int64_t)b[7] << 56` overflowed a signed integer for a top byte of 128 or more. The bytes are assembled as uint64_t, with one conversion at the end. - std/http/middleware/aether_middleware.c:154, basic auth's base64 decoder (an HTTP server test). `val = (val << 6) | d` kept every bit in an int, which overflowed after six characters. It keeps only the bits not yet written, unsigned. The same shape was in std/http/server/h2's base64url decoder for h2c's HTTP2-Settings, which the run never reached (no libnghttp2 on that runner). It is fixed the same way. - tests/integration/os_spawn_to: its --crash child wrote through null to fault, and UBSan reported the write instead. The child now calls crash.c, which the test ships with @source: - RaiseException(EXCEPTION_ACCESS_VIOLATION) on Windows, so the child still ends with 0xC0000005 through the unhandled-exception path the test is there for; - raise(SIGSEGV) with its default action elsewhere. - tests/integration/lane_wide failed only in that build. Its check that no vmovaps moves a 16-byte-aligned heap slice element also matched moves to and from the stack frame (`vmovaps %ymm0, 32(%rsp)`), which the compiler aligns itself, and which the instrumented build spills. Moves through %rsp or %rbp no longer count. Found beside them: runtime/aether_shared_map.c read its four-byte header through a plain char. A byte of 128 or more is negative there, and shifting a negative value left is undefined. It reads unsigned now. Test: std/mem/test_mem_accessors.ae reads a u64 whose top byte is 0xc6, both orders. The other findings are covered by the tests that found them, and by the UBSan run itself, rerun on the final branch. Checked: - make test (597) passes; - lzf roundtrip, issue2301 codec slices, the four crypto_ascon tests, std/mem accessors, os_spawn_to (0xC0000005 from RaiseException on Windows) and lane_wide all pass.
A local Windows sweep (MSYS2, ulimit -v 2000000, NPROC=2) ran every test:
1849 markers, of which 1760 PASS, 2 FAIL and 87 SKIP. make still exited 0.
The recipe's shell stopped about 20 ms after the last shell test. It
printed neither "=== FAILURE DETAILS ===" nor the "Aether Tests:" tally,
and never ran its own `exit 1` or removed its temp dir. The tally code is
not what failed: run against that temp dir under the same cap, it prints
the right counts. What stopped the shell is still being found, by a
rerun under the same conditions.
Whatever stops that shell, the gate must not read as a pass:
- The sweep writes build/test-ae.verdict ("passed failed skipped total")
right after it prints its tally. It removes the old file in a recipe
line of its own before it starts.
- A last recipe line, a shell of its own after the sweep's
(tests/scripts/sweep_verdict.sh), passes only on a verdict that
exists, reads as four numbers, has no failure and adds up to a nonzero
total.
A sweep that stops early now fails with "the sweep ended without its
tally", and its other failures still exit 1 as before.
Checked: sweep_verdict.sh on a missing, a failing, an inconsistent, an
unreadable and an empty verdict (each exits 1 and says why), and on a
passing one (exits 0). The full capped sweep runs next with this in
place.
… declare theirs (#3079, #3086) Whether a call keeps what it is given (E0902 for a borrowed parameter, the @Scoped escape check) was decided by the name the call spells ("list.add", "map.put", "list_add" and the rest). So `collections.list_add(routes, box_closure(cb))` kept `cb` with no error, a program's own `list_add` that keeps nothing refused it, and a raw borrowed `ptr` put in a container was never checked (#3001's holders-only rule). The owner decided #3086: a call keeps an argument exactly when the parameter it reaches is declared to keep it (R3), and std declares its stores. std, at the C boundary only (R4: no `@retain` on an Aether function): - std.list's `add`, std.map's `put` and std.collections' `list_add` and `map_put` only forwarded to the C store and turned its int into "" or an error string. C does that now (`list_add`, `map_put` in aether_collections.c) and they are externs, with the same names, arguments and results. - The item of list_add, list_add_raw, list_set, map_put and map_put_raw is `@aether @retain ptr`. A key is copied, so it stays borrowed; set.add and set.try_add copy their item and keep nothing. - string.seq_cons keeps its head and its tail: both are `@retain`, bound to a new C entry, string_seq_cons_adopt, which takes over the references the call hands it. string_seq_cons keeps taking references of its own for C and for the compiler's sequence literals. An extern's `@keep` was no fit: it is an Aether function's `@keep` across a binary boundary, which takes a string only through string.keep and whose fresh arguments the caller does not free. - Codegen hands an `@retain` parameter a `*StringSeq` as it does a string or a closure (R4): a fresh one as it is, a place's retained (emit_seq_take). Its container rewrite still gives a counted value its reference, so nothing is retained twice. A store says where it keeps what it is given, by declaration (approved): `@holds` on an extern's `ptr` parameter names the argument that holds what its `@retain` parameters keep (std's list and map stores, their container), and `@fresh` after an extern's `-> ptr` says the call allocates memory nothing else holds (list_new, list_new_in, map_new). A keep through such a store outlives the caller only when the holder does: a local bound only from `@fresh` calls (or null) that goes nowhere a borrowed `ptr` would escape to (c_name_outlives' probe) ends with the function. A parameter, a field, a module var, or a list returned or stored keeps it past the call. jq's free_node (a worklist of its root) and liquid's layout chain (the caller's lists in local chains, handed to helpers that read and free them) are accepted as they are written. Checker: - A call typed against an extern records it on the node (`callee_extern`, set where typecheck_function_call resolves the symbol). call_callee_decl reads it, so a module's private extern (std.list's `list_add_raw`) is seen by its declaration. The artifact codec refuses an AST that has it, as it does arg_order, and its size tripwire moves to 152. - check_ptr_keeps_in's container branch is gone: call_keeps_arg, where an extern's `@retain` is KEEP, decides. A borrowed `ptr` put in a list or a map is E0902, as one stored in a module `var` is. - @Scoped asks call_arg_mode for KEEP: "it is passed to a parameter that keeps it". A @Scoped tail given to string.seq_cons escapes too. - A `fn?` parameter is borrowed as the closure it may hold, and a struct or sum parameter one of whose fields or variants holds a closure or a `ptr` (an optional of one included) is borrowed as a holder. - The name list (scoped_store_call's sinks) is gone. Bug found and fixed: a glob import that names a builtin (std.string's `free`) left the builtin's symbol in place but registered the alias, and the pass that refreshes aliases after inference gave that symbol the import's type and declaration. A module's merged body, where a bare `free` is the builtin, was checked against `string.free`; with the declaration read for what a call keeps, its `free(p)` was E0902 (tests/integration/glob_import_builtin_free). The refresh now touches only symbols the import bound, and a program's call of such a name is resolved through its alias, as codegen already rewrote it. Bug found and fixed: `ae fmt` read a name written after `@` as a value, so the `*` of a pointer type after an attribute was spaced as an operator: `tail: @retain *StringSeq` came out `@retain * StringSeq`. A name after `@` ends no value now (tests/integration/fmt_type_prefix). tests/regression/test_string_field_arg_released_once_in_loop.ae was written in the old spacing and is written tight now. Bug found and fixed: std.collections declared `map_put_string_owned(map, key: @Aether @retain string, value)`, while C copies the key, so a counted key handed the map a reference it never released. std.map declared the same function without `@retain`, so which one applied depended on import order. The key is borrowed in both now. Callers migrated (declare `@keep`, the value being kept where the caller reaches it): std.spec's _suite_new (fw, parent) and _it_end (it_rec); std.cbor's object_set, set, array_add, push, map_set and map_push; std.msgpack's array_add and map_set; std.resp's array_add and map_add; std.jsonpath's _push, _fork_single and _collect_descendants (value), and the evaluation helpers that thread a node into them (`current` of _eval_test, _eval_value_fn, _arg_to_value, _arg_nodelist_size, _eval_regex_fn: a node of the `root` its public `query` already keeps); contrib/jq's _collect_pattern_vars and _push_child; liquid's context_set_partial_cache; contrib/parsers/xml_expat's bind_start, bind_end and bind_text (cb); tests/syntax/test_closure_context's add_child. std.worker's `run` takes `done: @keep fn(@keep ptr)` (and the extern `aether_worker_run` the same): the job's result is the completion's, so `map`'s completion stores it with `| r: @keep ptr |`. A completion that only reads it fits as before. Docs: the reference's E0902 paragraphs say a std container keeps what it is given by its declaration, the `@retain` section lists the stores and a `*StringSeq`, and a "Migrating from 0.838" section gives the rewrite; memory-management.md's @Scoped list reads the declarations; docs/api regenerated. Tests: tests/integration/keeps_borrowed gains the qualified store, a program's own list_add, a `fn?` stored in a module var, a struct with a `fn?` and a sum holding a closure captured by an escaping closure, a borrowed `ptr` into list.add, list.add_raw, list.set and map.put (refused) and with `@keep` (accepted), the jq and liquid shapes of a local holder (accepted), and the same list returned, stored in a field or a module var, a parameter list and one a function hands back (refused). tests/integration/scoped_escape gains a @Scoped tail given to string.seq_cons. The qualified store, the own list_add, the `fn?`, struct and sum cases and the raw `ptr` refusals fail on 0.838; the local holders are accepted there too, and stay accepted. Checks: make test (593); the integration tests by area (181: keep, borrow, scoped, containers, lists, maps, sets, sequences, workers, json, jq, cbor, msgpack, resp, fmt, glob, imports, closures, callbacks, externs, spec, liquid, xml) and the .ae-only ones, all passing; std's list, map, collections and string tests and every container and sequence regression test on both C paths. Corpus and grid-cell sweep against 639d264 (every .ae file in std, contrib, examples and tests, and the 2419 ownership grid cells): no exit status changes; the only diagnostic changes are the two @Scoped messages above, once the worktree paths in them are made equal. Ownership grid shards 1-4 of 16 on both C paths: OK 606, NOCOMPILE 1 (K1-P17-T8S, as on the baseline), none worse or better than the baseline.
…s itself (#3080) A program's `list_add(xs: ptr, f: fn) -> int` failed in the C compiler ("initialization of 'int' from 'const char *'"). No C symbol `list_add` existed then: std.list's `add` and std.collections' `list_add` were Aether wrappers, and codegen rewrote a call into std's container store whenever its C name was `list_add`, `map_put`, `list_set` or one of their raw and owning forms (g_closure_store_entries and the list, map and set shape tests). The program's own function was compiled as `_aether_list_add_closure(...)`, and its own `map_put` given a string as std's owning store. #3086 made `list_add` and `map_put` the C functions std's externs bind, so they are in the std C symbol table, and a program function of one of those names gets a C name of its own (`ae_list_add`, #1366), as one named `list_set` already did. The rewrites key on the C function a std extern binds, which a program function can no longer be, so no gate is needed in codegen: the earlier draft of this fix (a name list shared with the checker) is gone with #3086. Test: tests/regression/test_issue3080_own_container_names.ae defines its own `list_add`, `map_put` and `list_set` and checks each runs as itself, on both C paths. It fails on 0.838 (the C compiler refuses the list_add and map_put calls).
…d the string length The extern width fix made them `long`; the generated reference still said `int`.
closure_var_bind records which closure literal a variable holds: one entry per variable, with a strdup'd scope and name. code_generator_release freed the closure registry beside it but never this table. So every compile of a program that binds a closure to a variable leaked the array and its strings. No test exercised that: #3029's codegen test (codegen_slice_param_not_versioned_when_rebound_or_captured, `f = || { return xs[0] }`) is the first codegen unit test to bind one. CI's ASan/LSan and Valgrind legs then reported it: - LSan: 384 bytes direct and 11 indirect, from aether_xrealloc <- closure_var_bind <- discover_closures_scoped <- discover_closures <- generate_program; - Valgrind: the same 395 bytes definitely lost. code_generator_release now frees each entry's scope and name, then the array, beside the closure registry. Checked: make test (598) passes. The leak checkers that found it run only on Linux and macOS, so the CI legs that reported it are the proof.
… keeps binding, measured (#3089, #3106) #3089: measured whether Linux should keep binding scheduler thread i to one CPU, as #3087 measured for Windows. Method: - a CI job on a 4-CPU ubuntu-24.04 runner (EPYC 7763, kernel 6.17), two runs, with the job on a branch of its own; - the cross-language suite's thread ring, fork-join and skynet, 2M messages, 10 interleaved runs each; - bound (as today) against unbound: the same binary, with pthread_setaffinity_np made a no-op by an LD_PRELOAD shim, which reported the calls it saw in both modes. M msg/s, medians, bound / unbound: | load | ring | fork-join | skynet | |-------------------------------|---------------|-------------|-------------| | quiet (run 1) | 14.00 / 16.07 | 3.79 / 4.13 | 0.11 / 0.11 | | quiet (run 2) | 21.36 / 21.45 | 4.34 / 4.61 | 0.13 / 0.28 | | a loop pinned to CPU 1 | 10.86 / 9.00 | 2.28 / 3.22 | 0.11 / 0.11 | | loops pinned to CPUs 1 and 2 | 10.79 / 7.39 | 1.98 / 1.77 | 0.13 / 0.06 | | two loops the kernel placed | 10.82 / 7.38 | 1.96 / 1.82 | 0.13 / 0.09 | | a loop pinned to CPU 0 | 10.72 / 11.81 | 2.90 / 4.26 | 0.13 / 0.13 | | a loop pinned to every CPU | 6.66 / 4.99 | 1.17 / 1.17 | 0.05 / 0.04 | | four loops the kernel placed | 6.73 / 6.05 | 1.19 / 1.21 | 0.04 / 0.03 | - Bound won 7 of the 12 loaded cases, by 1.1x to 2.4x, and held the ring at 10.8 with a range of 10.5 to 11.0 where unbound fell to 7.4. - Unbound won only where another program's thread was pinned to a scheduler thread's own CPU: CPU 0 under the ring, fork-join's CPU 1. It won there by 1.1x to 1.5x. - Quiet is a tie, within the run-to-run spread. The kernel moves movable load off a bound thread's CPU; Windows did not, which is why #3087 made it a hint there. So Linux keeps the hard binding. #3106, found preparing #3089: placement and count were the machine's. - Placement: thread i went to CPU i, a number from 0. Under `taskset -c 4-7` the threads bound themselves to CPUs 0..3, outside the CPUs the program was started on; setting a thread's own affinity needs no privilege. In a cpuset cgroup, a binding outside the set was refused. Now scheduler_start reads the CPUs the process may run on, in ascending order, before any thread places itself (read_process_cpus: the calling thread's affinity on Linux, the process mask on Windows). Thread i goes to the (i mod n)-th of them, bound on Linux and as its ideal processor on Windows. scheduler_core_cpu(i) reports it. A mask that cannot be read leaves the threads where the OS puts them. macOS has no CPU numbers to place by and keeps its affinity tag. - Count: cpu_recommend_cores took the machine's online count, capped at 16. A program in a 2-CPU container started 16 scheduler threads. It now takes no more than aether_cpu_available() (the affinity mask, and on Linux a cgroup CPU quota), as std.worker and the HTTP server already do, still capped at 16. Tests: - tests/runtime/test_scheduler.c, "Scheduler counts and places its threads by the process's CPUs" (Linux, Windows): - the process is limited to its highest allowed CPU; - cpu_recommend_cores() is then 1; - both scheduler threads are placed on that CPU: Linux reads each thread's binding with pthread_getaffinity_np, Windows its ideal processor; - the mask is restored. The old code put the threads on CPUs 0 and 1 (CPU_SET(core_id)) and counted the machine's CPUs. - tests/integration/cpu_core_count: each mocked machine count is now capped by the host's available CPUs. docs/runtime-optimizations.md gives the count, the placement and the Linux numbers, and docs/getting-started.md the Linux note. The experiment job (.github/workflows/experiment-pinning.yml) lived only on its own branch and did not earn a place: the question is answered, and a 4-CPU runner's numbers move run to run (the quiet ring was 14.0 in one run and 21.4 in the other). The branch is deleted. Checked: make test (598) passes; so do cpu_core_count, cpu_available (1-CPU and 2-CPU cpusets honoured) and the multi-actor timeout test.
`g(0) -> 5` beside `g(k) -> uint8 { return 7 }` was refused: "the clauses
of 'g' are written to return uint8 (line 2), and this clause returns int,
which uint8 does not hold". clause_set_join fitted an inferred clause to
the written type by their join, and the join of an int literal and a
uint8 is an int. A local bound in several branches takes the same literal
beside its written type (#3070, ast_number_holds).
clause_set_join now asks ast_number_holds first: an inferred clause whose
value the written type holds (a literal in its range, or a value that
joins to it) fits it, as a binding fits a local's written type. A literal
it does not hold (`300` or `-1` beside `-> uint8`) is refused as before.
The reference says a literal clause fits a written type that holds its
value.
Test: tests/integration/clause_set_result_join gains literal_beside_written
(runs to "5 7"; fails before) and literal_outside_written (300 and -1
beside uint8 stay refused at the first).
Checks: make test (597); the clause, literal and join integration tests
(16); corpus and grid-cell sweep against 639d264: no exit status or
diagnostic changes but #3086's.
…lic functions, not other modules' raw bindings (#3107) They called std.net's, std.fs's and std.http's `*_raw` C bindings directly, skipping what each public function adds: - The TLS client dials with tcp.connect and the TLS client and server read records with tcp.read_n; both close with tcp.close, and std.net is no longer imported. The adopt-or-dial step in connect_full assigns `sock` in a branch, as written; the fresh-name dance its comment blamed on an old unresolved-type bug is gone. - tls13_server loads its certificate and key with fs.read, and tls13_cert its trust stores with fs.read and fs.list_dir. fs.read opens in binary mode, so Windows no longer reads them through text mode, and a store that cannot be opened names the path and the OS reason. - The HTTP load balancer starts with http.server_start and sets TLS with http.server_set_tls. - tinyweb's integration and schema tests and its schema showcase send requests through std.http.client, which returns the status and the body (the one-line http.get turns a non-2xx into an error string). The test server starts with http.server_start; the comment that said only the raw call runs the accept loop was wrong. The TLS client test checks the error of its sha2.new, which it left unread (the one warning its build printed).
This was referenced Oct 22, 2026
… runs (#3103) contrib/jq's test_with_jsonpath.ae and test_after_jsonpath.ae failed to build from inside contrib/jq: "'free_query' is not exported from module 'parser'". Built from the repository root they worked. std.jsonpath's facade does `import parser` for std/jsonpath/parser.ae, and module_resolve_local_path searched the current directory (the lib path, src/, the project root) before the importing module's own directory, so from contrib/jq it found contrib/jq/parser.ae and handed that to std.jsonpath. #2255 had kept the two `parser` modules apart once each was found; this is the finding. A module's own imports are resolved beside it first (the registry records, around a module's import loop, that source_dir is that module's directory: source_is_module), then where an entry program's are. The entry program's imports keep the current directory, the project root, first. The source-directory probes are one helper, resolve_under_source_dir, used in both orders. Found with it: contrib/jq's _str_indices allocated its result before returning `null` for an empty needle, which leaked it, and it and is_keyword ended with a `return` no path reaches (W1002 on every build that imports jq); std.jsonpath's _parse_segments did too, after its `while 1 == 1` (W1002 on every build that imports std.jsonpath); and jq's two jsonpath tests, which the error had stopped before the checker reached them, called `json.json_free` (W1006), as did two comments. The corpus sweep's diagnostics then showed the same dead `return` in liquid's `if` and `case` renderers (after `while 1 == 1`) and in four examples (after a `match` every arm of which returns: match-advanced, match-expressions, sum-types, message-with-string-seq); each is removed, and no W1002 is left outside the tests that expect one. docs/module-system-design.md listed the search without the importing file's directory at all; it lists it, and says a module's own imports search there first. Test: tests/integration/module_import_beside_itself builds, in a directory with a parser.ae of its own, a program that imports that parser and std.jsonpath, and runs a query through each. It fails before ("'free_query' is not exported") and passes after. Checks: make test (598); the module, import, namespace and package integration tests (113); std.jsonpath's tests and its parse regression on both C paths; jq's two jsonpath tests run from contrib/jq; liquid's tests and the four examples on both C paths. Corpus and grid-cell sweep against #3082 (4707 files): the two jq jsonpath tests now build (1 to 0), and the only diagnostic changes are the W1002 warnings gone from jq and std.jsonpath and everything that imports them.
Each scheduler thread keeps a deferred-send buffer per target core in thread-local storage (tls_overflow). It grows by doubling under a burst and keeps its capacity, so the next burst does not pay for it again. Nothing freed those buffers. When a thread left its loop at scheduler shutdown, the arrays it had grown became unreachable. CI's macOS leaks gate found it on test_actor_pair_fifo (#3093's test, pairs of actors trading 3.2M messages): 18 leaks, 115 MB, every one `realloc in overflow_append` from send_through_core. Shown on Windows, with a counter printed at each thread's exit (not committed), three runs of that test: - before: 15 to 28 MB still held per core, in 2 to 6 buffers, on every core that took part; - after: 0 bytes on all 16 cores, every run. overflow_release, called by scheduler_thread after its final overflow_flush, frees each buffer's two arrays and zeroes it. An entry still waiting at that point means the final flush found its target's queue full while the cores were stopping. It is dropped and uncounted (inflight_leave), as a message for a released actor is (scheduler_drop_dead_message), and taken off g_overflow_total. Checked: make test (598) passes, as do test_actor_pair_fifo (three runs), test_actor_balanced_spawn, test_actor_timeout_multiple and sched_foreign_send_lifecycle. The macOS leaks gate is the regression test.
A slice index outside `0 ..< s.len` is a named panic; an index into a
fixed-size array `T[N]` was not checked at all. Codegen wrote a plain C
`a[i]`, so `a = [1, 2, 3]; i = 4; a[i] = 99` wrote into the stack frame and
`at(a, 5)` read whatever lay past the array, exit 0, and so did an array
field, an array parameter, an actor's array state and a `const` table.
Every index into a `T[N]` is checked now, where the one place codegen
writes an index (AST_ARRAY_ACCESS) emits it: `a[aether_array_index(i, N,
file, line)]`. The helper (runtime/aether_slice.h, beside the slice's)
returns the index, so the element stays an lvalue for a store, `+=`, `++`
and `--`, and the index is evaluated once; outside the range it raises the
slice's panic with the array's length ("array index 4 out of range for
length 3", after the source line), which `try` catches. A negative index
is out of range. A literal index the checker has seen in range is written
as it is.
The checker refuses an index known when the program is compiled that is
out of range: a literal, a `const`, or arithmetic over them, evaluated by
contract_eval_int64, the evaluator contracts and switch tables use. A name
counts as a constant only when the scope sees the `const` itself (the
symbol's is_const), so a local or a parameter named like one is a value
known when the program runs and is checked then.
The reference states the rule in Array Types and beside the slice rule.
Measured, gcc -O2 (`ae build`), best of 15 runs, before and after
alternating, every digest unchanged, 1 MiB per digest: blake2b +0.7%,
blake2s -0.1%, gost3411_2012-256 -1.4%, sha256 -0.3%, sha512 -0.2%,
sha3-256 -1.6%, md5 -0.1%, sha1 +1.3%, sm3 0.0%, whirlpool -1.1%,
ripemd160 -1.2%, tiger -1.4%, skein512 +0.8%. That is noise because the C
compiler removes every test: in the assembly of those modules (42 indexes:
blake2's SIGMA, IV_B and IV_S, gost's STREEBOG_T and STREEBOG_C, sha2's
K256 and K512, sha3's RHO and RC_K, sm3's T_SM3, whirlpool's WP_SBOX,
tiger's four S-boxes, skein's SKEIN_ROT and SKEIN_PERM) no array index
panic is left. Of the cross-language benchmarks, ping_pong, counting,
thread_ring and skynet generate the same C as before; fork_join's one
index (`workers[i % num_workers]`) is removed at -O3, and its two
binaries differ only in the PE header's timestamp and checksum. No case
was found where the test stays.
Test: tests/integration/array_index_checked, on both C paths: a store, a
read through a parameter, `+=`, `++`, `--` with a negative index, a
struct field through a pointer, a `const` table and an actor's state, each
out of range and caught by `try` with its line, the array beside the one
written unchanged, and every form in range; an uncaught one panics with
its message; a literal, a `const` expression and a negative literal out
of range are refused at compile time; a parameter named like a `const` is
checked when the program runs. The refusals compile, and the program runs
past every bad index, before.
Checks: make test (598); 91 integration tests over arrays, slices,
consts, actor state and every crypto module; the corpus and grid-cell
sweep against #3103 (4712 files): no exit status changes, so no program
in it has a constant index out of range, and the C changes in 180 files,
none of them a grid cell; every runnable test among those 180 run on
both C paths (55 programs) and the 43 integration directories that hold
the rest. The GPU backends' tests (contrib/d3d12, metal, vulkan) were
left out.
…an expression, any base (#3115) The parser took the length as one NUMBER token read with atoi, so `long[QWORDS]` beside `const QWORDS: int = 128` and `long[NW + 1]` were "Expected NUMBER, got IDENTIFIER", `byte[0x40]` was an array of length 0, and `int[0]` compiled as a slice. std wrote a number beside the constant it stood for. A length is now a literal in any base the language reads, a top-level `const` of the same file whose value is an integer constant expression (declared above or below, built from other constants), or `+ - * / % << >>` and parentheses over those. The file's integer constants are scanned once, on the first length that needs them. Every value stays within +-2^31, so no step overflows. A length below 1, an unknown or non-integer name and a division by zero are compile errors that say what a length may be. The two lookaheads that tell `Pair[N] ps` from an index expression skip the whole bracket group. Sub-parsers are freed through free_parser. Tests: tests/regression/test_issue3115_array_length_const.ae (locals, fields, parameters, arrays of structs, hex, shifts, a constant built from one declared after it), which the old parser refuses; and tests/integration/array_length_errors for the six errors.
Eleven tests took the Windows branch on `getenv("OS") == "Windows_NT"`,
which cmd.exe and the desktop set. A shell without it (an MSYS2 login shell
started with a clean environment, a CI step, a service) ran a Windows build
down the POSIX branch, and the test failed. They ask `os.platform()`, which
answers from the build.
`make check-tests` runs tests/scripts/check_platform_env.py, which fails on
a `getenv("OS")` outside a comment in tests, std or contrib, whichever
spelling calls it, with unit tests of its own (test_check_platform_env.py).
Reverting one of the eleven makes it fail, naming the line.
…only (#3121) http_park_resume was compiled in every build and calls http_pool_submit_conn, which aether_http_pool.c defines only with threads. Its one caller is under AETHER_HAS_THREADS, so at -O2 GCC dropped the unused static and its reference; at -O0 it kept both, and a no-threads stdlib's shared runtime did not link on Windows (an unresolved symbol in the Linux .so). It is compiled under the condition its caller is. Test: tests/integration/no_threads_refs compiles every libaether source (the Makefile's own lists and flags) at -O0 with and without AETHER_NO_THREADING and fails on a symbol a no-threads object needs that only the threaded objects define. Under GCC (ubuntu 24.04) on the old tree it fails naming http_pool_submit_conn; clang drops unused statics at any level, so on clang it passes either way.
…3120) aether_thread.h took its no-threads branch first on every platform, and on Windows that branch's pthread.h test matched nothing MinGW defines, so a -DAETHER_NO_THREADING build got the bare-metal `typedef int` thread, mutex and condition while a threaded build has the Win32-shaped HANDLE, CRITICAL_SECTION and CONDITION_VARIABLE. A cooperative libaether is built with the flag and a program built against it by `ae` is not, so the two disagreed: the four ActorBase fields after `thread` sat 4 bytes apart (the rest agreed only through padding), ActorReplySlot was 40 bytes against 80, and Scheduler differed from `thread` on. The Windows types are defined once, ahead of both branches; the no-threads branch uses them on Windows, and its static initializers are `{0}` there. Test: tests/integration/thread_types_layout builds layout_probe.c (sizes and offsets of ActorBase, ActorReplySlot and Scheduler, the shared structs that hold thread types) with and without the flag and compares. Under MinGW (x86_64-w64-mingw32-gcc 13-win32, the values read from the generated assembly) 15 of the 35 values differ with the old header, none with the new one; every libaether source still compiles for Windows both ways.
…names (#3083) The checker and codegen asked, once per name bound in a function's branches, questions that each walked the whole body: the bindings' join (hoist_join_type, hoist_numeric_join), the type a binding writes (hoist_written_type), the names a branch hoists and their first binding (hoist_if_branch_candidates, hoist_common_decl_names), and whether the function reads the name (codegen_local_unread). Codegen's is_var_declared and declared_var_type scanned every declared local, and the checker's unused-variable pass compared each reference with each local, marking a local rebound n times n times per read. - hoist.c answers from per-body and per-statement indexes, each built by one walk and kept, by node, until hoist_cache_reset (at the start of the checker, and after codegen's renames) or until the node is freed, so a node allocated later at its address is asked afresh; a binding's type, which the checker fills in as it goes, is read when asked. HoistNames replaces the 64-entry name arrays: the names a branch hoists are any number, found by name. - Codegen indexes its declared locals by name (the earliest live slot, cleared as scope exits truncate them) and, per function, the names it reads. - The unused-variable pass finds a reference's locals by name and marks a name's entries once. Before, with 64 names hoisted at most, a 65th bound in an arm and read after it was "Undefined variable". Generated mains (each name bound in both arms of an if and read after it), min of 5 interleaved runs against the previous commit: names before after 1000 0.28 s 0.06 s 3000 2.57 s 0.13 s 6000 16.17 s 0.25 s The C every test, example and std test compiles to is unchanged (1063 programs compared). Test: tests/integration/hoist_scaling (300 names out of one-armed ifs, int/long, null/string and written-type joins, a name rebound 300 times, one unused local warned); before: "Undefined variable 'a16'", the 65th hoisted name.
…fer (#3108) std.tcp's writes are one send(2) each and return its count, which can be short: every socket std.net opens has a 30 s send timeout, and a send that runs into it while the peer reads slowly returns the bytes that fit by then, as one a signal cuts short does. tls13_client and tls13_server sent each record with one write_n and took a short count for "failed to send record", with part of the record already at the peer. std.http's server had the same send under every plaintext response, WebSocket frame, SSE event and upgrade reply (conn_send): the rest of a response never went, and the client got it truncated. - tcp_send_all_raw sends until every byte is out, retrying an interrupted send and stopping at one that fails or times out having sent nothing; tcp.write_all(sock, data: byte[]) returns (data.len, "") or (sent, "send failed"). The TLS modules' record sends use it, and write_n's and write_slice's notes point to it. - conn_send sends until every byte is out, retrying an interrupted send, and returns the length, or -1 once a send fails. The server's TLS paths send through their own layers, unchanged. Tests: - tests/runtime/test_runtime_net.c (a 4 KiB send buffer and a 500 ms send timeout on a socketpair): one 256 KiB send to a peer that is not reading comes back short (4096 bytes on macOS); with a reader that pauses 750 ms after 64 KiB, tcp_send_all_raw delivers all 262144 bytes in order. Made to send once, it delivered 69632. - std/tcp/test_tcp_slices.ae: write_all over loopback, a 48 KiB payload read back in order, an empty slice, an unbounded one. - tests/runtime/test_http_server.c: the same socketpair and reader under http_server_drain_connection with a 256 KiB response: every byte arrives, in order. With one send(2), 69510 of 262144 body bytes did.
…aded (#3118) Every HTTP connection carries SO_RCVTIMEO set to the keep-alive idle timeout (30 s by default), which guards the wait for the next request. An upgraded WebSocket kept it, so the handler's blocking ws_recv timed out on a client quiet that long, ws_recv_exact took the failed recv for a close, and ws_recv returned -1: a chat tab in the background or a dashboard between updates was dropped. Once the 101 has gone out the server clears it (conn_set_recv_timeout, the setter conn_apply_recv_timeout now goes through, with the per-connection bookkeeping of #1719); ws_recv_timeout still bounds a wait a handler wants bounded. The client side never set one. Test: tests/integration/ws_idle_server (a 1 s idle timeout, a client quiet for 2.5 s between two messages, both echoed), in the sweep's prune list with the other client/server pairs. Before: the client's wait for the second echo returned -1, the server having closed the connection.
…ap loops that cannot wait (#3112, #3117) #3112. With one actor the runtime ran the handler inside the sender's `!` (main-thread mode) until a second actor was spawned: a handler that blocked on something its sender did next deadlocked the program, output order depended on the actor count, a C host had to poll to keep the actor running, and the payload pointed at the sender's stack (g_skip_free). Tests and examples spawned a do-nothing actor to get the real behaviour. - Every send goes through the scheduler from the first actor on. Main-thread mode is gone: main_thread_only, the inline enqueue and step paths and their locks, g_sync_step_actor, g_skip_free, the deferred release for inline steps, scheduler_ensure_threads_running, and the AETHER_INLINE / AETHER_NO_INLINE switches. - aether_scheduler_poll returns 0 with threads. - The cooperative runtime queues a send instead of stepping the actor: into the mailbox, and past its 32 messages into the actor's inbox (the unbounded FIFO the actor threads use), refilled by aether_scheduler_poll, which wait_for_idle and shutdown run. - A message with no fields goes as a one-int one does, with no payload to allocate. - The 47 do-nothing actors in tests/ and contrib/ are removed. Unit tests that let a second spawn start the cores call scheduler_start(), and test_scheduler's two tests lose the poller thread whose walk went with the mode. foreign_thread_send keeps its scheduler-mode half. #3117. The compiler opened a send batch around every `while` loop of a non-actor function that sent. With #3112 every program shows the effects: - a loop that waited held its messages until it ended; - a payload message went around the batch and overtook the one-int ones (test_closure_actor_state_message_release, the one failure of the full .ae sweep on #3112 alone); - an inner loop's batch_start emptied the outer one's buffer. - The compiler batches only a loop that cannot wait between its sends: no call, print, ask, actor-state read, `return`, labeled `break` or `continue` (which can leave for an enclosing loop, past the flush) or closure in it. An inner loop joins the outer loop's batch, and a payload send in it joins too (aether_send_message_batch). - At run time, opening a batch keeps what it holds. A foreign send around a pending batch flushes it first, and so does wait_for_idle. A scheduler thread batches nothing. Measured, max of 11 interleaved runs against the previous commit (M msg/s): counting (1 actor, 10M) 30.70 -> 20.95 the inline handler is gone; 16.72 before the field-less message lost its payload ping_pong (2 actors, 1M) 11.43 -> 11.35 fork_join (1M) 13.72 -> 14.00 Tests: - tests/regression/test_send_async_one_actor.ae: a lone actor's handler waits for main to go on after the send. Before: it saw 0 after 2 s. - tests/regression/test_send_batches.ae: a waiting loop delivers as it goes, a mixed loop keeps its order, nested loops lose nothing, a loop left by `break outer` delivers without a flush after it. Before: "a payload message overtook a one-int one"; batching the labeled-break loop, "kept 5 of 5 messages". All 610 C unit tests pass; the .ae sweep on #3112 alone passed 1844 of 1847 (the batching failure above and two failures that happen only on this Mac).
…ttp, on the server's port (#3109) tinyweb listened for WebSocket and SSE on ports of their own, through accept loops a program had to start beside tw_start, which blocks in the HTTP loop. Its frame reader took one read(2) for a whole payload (a message over what one read returns arrived cut), had no 64-bit length (no message of 64 KiB or more), took a ping for a message and answered none, and took an empty frame for a close. Its SSE writes returned nothing, so a handler streaming in a loop never learnt that its client had left, and chunked_start wanted a raw socket an endpoint does not have. with_timeout, with_keep_alive and on_stats were stored and never applied; with_backlog, with_ws_backlog and the four on_*error callbacks were stored and never read. std.http, which tinyweb's HTTP routes already ran on, has a conformance- tested WebSocket codec and an SSE stream on the server's own connection (TLS included). It gains what tinyweb needs and lacked: - http.response_begin_chunked(res), chunked_write and chunked_end: a body of unknown length on the connection, as response_upgrade_sse takes it over; refused on a response that has a body. - http.sse_comment(sse, text): a keep-alive line. - http.response_get_status(res). tinyweb: - web_socket and sse_endpoint routes are std.http's server_websocket and server_sse, on the HTTP port. A web_socket handler gets each message whole (binary ones zero bytes and all), `sender` is the std.http WebSocket, and `ctx` is a map the connection's messages share. - sse_send, sse_send_event and sse_comment return 0, or -1 once the client has gone. chunked_start(res, status) returns (stream, err). - tw_start applies with_timeout and with_keep_alive through server_set_keepalive. on_stats gets a record of each endpoint request (path, route, status, durations, the filters and their verdicts), its accessors typed. - Removed: its frame code, accept loops, web_server_with_ws, web_server_full, the backlog options, the error callbacks, ws_handshake.c and ws_client.ae (std.http's ws_connect is the client). The contrib build catalogue loses the tinyweb archive; the install test's always-built archive canary is host/factor's. - The examples and the liveview spike serve everything on one port; the spike keeps its count in `ctx`. The liveview roadmap's "structural blocker" (serving the page and the WebSocket at once) is cleared. Tests: - contrib/tinyweb/test_streams.ae (replaces test_websocket.ae): over loopback, a 70000-byte message echoed both ways, a binary message with zero bytes, a ping answered with its payload, `ctx` across two messages, a WebSocket quiet past a 1 s with_timeout, an idle HTTP connection closed after it, on_stats for a filtered and a stopped request, an SSE handler ending once its client leaves, a chunked body read back with std.http.client, a chunked start over a body refused. Without the server_websocket registration the six WebSocket cases fail; without the keep-alive call the idle close takes the 30 s default; without on_stats on the routes both stats cases fail; without server_sse the SSE case fails. - tests/regression/test_http_server_streams.ae: the chunked head (the handler's Content-Length dropped) and body on the wire, an empty write sending nothing, the refusal over a body, a status read back, and sse_comment's 0 and -1 for a line break. On the previous commit none of these functions exist.
…is upgraded (#3118) std.http's server keeps a quiet WebSocket open, and a send to a peer that has gone fails rather than raising SIGPIPE (#3118) Every HTTP connection carries SO_RCVTIMEO set to the keep-alive idle timeout (30 s by default), which guards the wait for the next request. An upgraded WebSocket kept it, so the handler's blocking ws_recv timed out on a client quiet that long, ws_recv_exact took the failed recv for a close, and ws_recv returned -1: a chat tab in the background or a dashboard between updates was dropped. Once the 101 has gone out the server clears it (conn_set_recv_timeout, the setter conn_apply_recv_timeout now goes through, with the per-connection bookkeeping of #1719); ws_recv_timeout still bounds a wait a handler wants bounded. The client side never set one. The client side had its own way to die there. The send the client and the server share made a plain send(2), and one on a connection the peer had closed raised SIGPIPE; a server ignores it from its start, but a process that only dials, as this test's client does, was killed (exit 141 on Linux; macOS got by on timing). conn_send passes MSG_NOSIGNAL where Linux has it, and http_ws_connect sets SO_NOSIGPIPE where that is the platform's way, so the send fails with -1 as ws_send_text documents. Test: tests/integration/ws_idle_server (a 1 s idle timeout, a client quiet for 2.5 s between two messages, both echoed, then three sends to the connection the handler closed, each refused), in the sweep's prune list with the other client/server pairs. Before: the client's wait for the second echo returned -1, the server having closed the connection; and on Linux the client died of SIGPIPE at the sends after the close.
…ADING (#3120) One Windows thread-type layout with and without AETHER_NO_THREADING (#3120) aether_thread.h took its no-threads branch first on every platform, and on Windows that branch's pthread.h test matched nothing MinGW defines, so a -DAETHER_NO_THREADING build got the bare-metal `typedef int` thread, mutex and condition while a threaded build has the Win32-shaped HANDLE, CRITICAL_SECTION and CONDITION_VARIABLE. A cooperative libaether is built with the flag and a program built against it by `ae` is not, so the two disagreed: the four ActorBase fields after `thread` sat 4 bytes apart (the rest agreed only through padding), ActorReplySlot was 40 bytes against 80, and Scheduler differed from `thread` on. The Windows types are defined once, ahead of both branches; the no-threads branch uses them on Windows, and its static initializers are `{0}` there. Test: tests/integration/thread_types_layout builds layout_probe.c (sizes and offsets of ActorBase, ActorReplySlot and Scheduler, the shared structs that hold thread types) with and without the flag and compares. Under MinGW (x86_64-w64-mingw32-gcc 13-win32, the values read from the generated assembly) 15 of the 35 values differ with the old header, none with the new one; every libaether source still compiles for Windows both ways. The test compared the two layouts with `cmp`, which the Windows CI shell lacks (#3110), so it failed there for the missing tool; it uses same_bytes. `make check-tests` now fails on a `cmp` or `diff` command in a shell test that does not ask `command -v` for it first (tests/scripts/check_shell_tools.py, with a self-test), so the rule #3110 set holds.
…h std.http, on the server's port (#3109)
nicolas-maman
marked this pull request as ready for review
October 22, 2026 21:36
…readed only (#3121) no_threads_refs read the threaded build's sources for both builds; a no-threads build compiles the cooperative scheduler in place of the threaded one, so each is now read from the Makefile with its own EXTRA_CFLAGS. And aether_http.c includes <errno.h> on every platform: it reads errno everywhere, and on Windows a no-threads build reached it only through the threaded half of aether_thread.h (ENOTCONN undeclared).
This was referenced Oct 24, 2026
This was referenced Oct 26, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Every open bug, batch 6, after #3085. The bar for this PR: when it merges, no bug and no critical issue is left open, and nothing is deferred, worked around or patched over. Each issue gets one commit with a test that fails before it, the docs it makes stale updated in the same commit, and a changelog fragment. Every memory and ownership fix follows the ownership model's R1 to R4. This stays a draft until every issue below is on this branch and a review round comes back clean.
Every open issue this PR resolves, and where it stands
nullfn, or afn?holding null) crashed with no messageempty_closure_call), catchable@keepwas decided by function names; the owner decided (#3086) that a borrowedptrput into a std container is E0902@retainat the C boundary (R4)parser(jq's and jsonpath's) did not builda[4]on anint[3]wrote past itT[N]index is checked, a constant one out of range is a compile error; gcc removes all 42 checks in the hash moduleslong[QWORDS]with a const did not parse,byte[0x40]was 0free,free(xs)of a slice called the module's function with the slice's pointerfreetakes a slice*_rawC bindings instead of their public functions (owner report)mod_pow2.7x faster, the hashes within noise or faster) and the 286 container calls are moved;list.atandmap.lookupadded;make check-raw-callsinmake cikeeps it at 0longwere defined with a 32-bit Windowslong; 88 width mismatches in allT[]whose origin is hidden paid the view test and never vectorized.lenof a slice or an array passed the checker and failed in Ccmpordiff, which an MSYS2 install can lacktests/lib/same_bytes.shmem.get_u64_*, base64 decoders, the shared map header, a defined crash, a test reading spills as heap loads); a UBSan rerun on the final branch closes itmake test-aecould exit 0 without its tallymainheld, lost and reordered messages!is asynchronous whatever the actor count; empty messages carry no payload; the batch never empties or holds a waiting loop's sendsarr.lengthpassed the checker and failed in Cae fmtrewrotemake([]fn, 2)asmake([] fn, 2)null_function_callpanic at the callfs[1]()was refused as "Undefined function '?'"forinit declared a new variable; the owner decided it assigns an existing oneexamples/did not teach Aether (owner request)vkCreateRenderPassAfter that comes one review round of the whole branch and a full local gate on the final head: build,
make test, the wholemake test-aesweep, the strict C11 sweep, check-docs and the ownership grid.Closes #3091
Closes #3090
Closes #3093
Closes #3079
Closes #3086
Closes #3080
Closes #3082
Closes #3103
Closes #3113
Closes #3115
Closes #3116
Closes #3107
Closes #3097
Closes #3094
Closes #3102
Closes #3029
Closes #3104
Closes #3110
Closes #3089
Closes #3106