Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions CHANGELIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ The headline of this release: `modules/dasLLAMA`, a complete LLM inference engin

#### dasLLAMA: Text-to-Speech (#3929, #3985, #3992, #3933)

daslang speaks: KittenTTS nano/mini and Kokoro-82M from a das-native front end (a G2P rail with a packed lexicon, a POS tagger, and the JIT's aarch64 vector-math rail underneath) (#3929); Pocket TTS as the third family - voice cloning, six languages, published Q8_0 files, the server and a tutorial (#3985); its small form - the K-quant lane, a 75 MB English file, storywish on one voice, the parrot example (#3992); the control page speaks, the front end speaks British, and a pack the worker cannot load degrades the boot instead of killing it (#3933).
daslang speaks: KittenTTS nano/mini and Kokoro-82M from a das-native front end (a G2P rail with a packed lexicon, a POS tagger, and the JIT's aarch64 vector-math rail underneath) (#3929); Pocket TTS as the third family - voice cloning, six languages, published Q8_0 files, the server and a tutorial (#3985); its small form - the K-quant lane, a 75 MB English file, storywish on one voice, the parrot example (#3992); the control page speaks, the front end speaks British, and a pack the worker cannot load degrades the boot instead of killing it (#3933). Portable Pocket voices (#4055) - a reusable prompt format with preparation, validation, registration and a serialization tutorial, so an encoder-free Pocket model speaks a compatible prepared voice; the same change stages a wasm release's declared external files into its bundle and fixes touch-only browsers (Pointer Lock guarded, tap coordinates queued before button events) - browser applications need a rebuild for the input fixes.

#### dasMetal: Native Metal Compute + the dasLLAMA Metal Tier (#3443, #3459, #3474, #3484, #3496, #3509, #3515, #3519, #3521, #3526, #3527, #3530, #3533, #3535, #3546, #3554, #3558, #3562, #3653, #3663, #3754, #3763)

Expand Down Expand Up @@ -201,6 +201,7 @@ A native glTF 2.0 loader with an OpenGL PBR renderer, viewer, and capstone tutor
- **Pure-das Dear ImGui renderer** (#3283) and its compile-time WebGL2 draw path + `get_running_platform_name` (#3294); strudel audio in the playground (#3272); embedded fonts (#3292); canvas-CSS-size/DPR + `glfw_toggle_fullscreen` externs for responsive, fullscreen-capable cards (#3323, #3326)
- **Static-link gates** (#3351, #3358) - `examples/fatman` links every external C++ module into one host with a CI lane (the ABI canary for daslang-vs-externals drift), and its web sibling builds the imgui family from source into the threaded wasm playground
- **The browser tier's K-quant dot, FMA, spinning workers, and a mixer off the host clock** (#4045) - `dot_k4q8` rides the `idot` lattice op (both nibbles into a signed `byte16`, one body, no target branch; 1.1x -> 1.7x real time for Pocket TTS on the parrot page, 1.90x on x64, bit-exact), `+relaxed-simd` joins the wasm feature string so a contracted multiply-add becomes `f32x4.relaxed_madd` instead of splitting (8728 contractions across the module), the job queue's spin window reads the clock once per stride and pauses with no JS in it (pool clock time 25 s -> 5 s per 15 s window), and a browser's workers take the same 30 ms spin window as every other target (1.7x -> 2.3x); dasAudio's browser mixer parks on a drain counter the worklet bumps instead of a sleep, primes a 20 ms ring before the device starts, and counts underruns (`sound_playback_underrun_frames`)
- **A native WebAssembly SIMD128 vecmath backend** (#4052) - `include/vecmath/dag_vecMath_wasm.h` replaces the SSE2 arm run through emscripten's compat layer, where a float-to-int conversion was a per-lane `lrint` loop (86 wasm instructions for `v_cvti_vec4i`, 170 for `v_cvt_roundi`, 91 for `v_floor`) and min, max, select, abs and mul+add each cost two to four ops; the new arm answers with the ISA's own instructions (saturating truncation, floor/nearest, pmin/pmax with the SSE NaN and signed-zero rule, bitselect), the web build and the dasImgui wasm archives drop `-msse2`, relaxed SIMD stays off by default (WebKit implements none of the opcodes), and the backend battery builds as node-runnable pairs the wasm lane runs. On the web `int(x)` of a float outside the 32-bit range now saturates (NaN gives 0) instead of INT_MIN - the NEON contract. Interpreter under node: the float-to-int/floor/round loop 26.4 -> 11.9 ms

- **std140 on WebGL2** (#3273, #3277, #3278) - a UBO pack binder off a shared layout walk (mat3 and composites), buffer release, and the glsl emitter indexing a runtime matrix/vector/array column as `base[index]`
- **The browser lifecycle** (#3900, #4000, #4001, #4010) - both browser drivers collect the heap between frames; the wasm job pool floors at four workers so a browser reporting two cores runs the examples on more than one; arcanoid on the web with the worker heap, the cross-target layouts, CRT scanlines by pixel pitch, audio surviving live reloads, and dasbind proxies for cache-served registrars
Expand Down Expand Up @@ -256,7 +257,7 @@ An MCP bridge for DAP debugging (#3937) - launch, breakpoints, stepping, scopes,

#### The Document System: `[arch]` Citations and `REVIEW.das` Gates (#3882, #3887, #3766, #3914, #3896)

`REVIEW.md` carries bans and duties that bind a diff, `ARCHITECTURE.md` carries present-tense statements, one grammar under `REVIEW_COMMON.md` / `ARCHITECTURE_COMMON.md` (#3882). `[arch(at="doc.md#anchor")]` makes code cite the section it implements and LINT026 keeps every anchor cited and every citation resolving (#3887; mandatory across the Vulkan tier in #3896; the C++ pointer form in #4047). A checklist's mechanical half becomes an executable `REVIEW.das` gate (#3766). SDK bundles ship generated excerpts of the cited sections so LINT026 stays armed in an installed tree (#3914).
`REVIEW.md` carries bans and duties that bind a diff, `ARCHITECTURE.md` carries present-tense statements, one grammar under `REVIEW_COMMON.md` / `ARCHITECTURE_COMMON.md` (#3882). `[arch(at="doc.md#anchor")]` makes code cite the section it implements and LINT026 keeps every anchor cited and every citation resolving (#3887; mandatory across the Vulkan tier in #3896; the C++ pointer form in #4047). A checklist's mechanical half becomes an executable `REVIEW.das` gate (#3766). SDK bundles ship generated excerpts of the cited sections so LINT026 stays armed in an installed tree (#3914). The mechanical auditors (tdd, review-md, style-hygiene, placement, dupe, harvester, janitor, spartan, analysis-bot) pin high effort, skip CLAUDE.md, and gather what they read in dependency order before judging - a 33-run replay showed 43% of steps reading one file at a time; grouping cut steps 39%, cost 25%, time 38% (#4056).

#### Telegram Dictation Bot + Cadmus (#3408, #3411, #3465, #3482, #3483, #3491, #3517, #3531)

Expand Down Expand Up @@ -323,6 +324,7 @@ An MCP bridge for DAP debugging (#3937) - launch, breakpoints, stepping, scopes,
- **The JIT closes on the AOT** (#3732, #3868, #3880, #3917, #3981, #3999) - `each()`, generator ranges, array push and indirect dispatch match; binds are NTTP and the bind-flavor gate reaches the module; a handled type iterates and indexes through its annotation; hot-path wins, `[jit]` in a C++ AOT host, unresolved AOT-object address globals; `--jit-debug` describes das locals and a das name is what a debugger breaks on; the runtime entry points get their own file
- **Sealed lookup tables** (#3973) - `Context` finds functions and globals by name and by mangled hash through perfect-hash tables; standalone exes and AOT contexts adopt emitter-sealed constant data
- **Embedded hosts enable the JIT through the C API** (#3889) - and the macOS in-memory JIT arm is fixed; **a statement after a terminator parks in its own dead block** (#3964)
- **The LLVM-AOT rail links** (#4046) - a `no_jit` dependency stops folding a zero into the semantic hash (`linkCppAot` skipped the hash pass the emitter took), an AOT-consuming run is stamp-free whether or not the JIT is on (the object emitter baked its own tune sidecar into the artifact), every function of an in-memory module carries the host cpu and feature string so the subtarget it is selected under matches the gates that emitted it (`test_lattice_idot` aborted on `AArch64ISD::SDOT`), and the emitter disposes the target triple LLVM hands it - 12836 green under `test_llvm_aot -jit --run`

#### Runtime, Tooling, and Hosting

Expand Down Expand Up @@ -377,6 +379,7 @@ An MCP bridge for DAP debugging (#3937) - launch, breakpoints, stepping, scopes,

### Bug Fixes

- **The driver forwards `-h` / `--help` placed after `--`** (#4057) - it answered them with its own banner wherever they sat, so no interpreted clargs tool could print its help; the release validation catches up with the tools since RC2 in the same change (the bundle smoke compiles and launches the DAP bridge, the static watchdog's `--help` and `--lsp` fronts, the LLVM-AOT driver and the LSP subtools; the release audit's rows carry an expected exit code, the telegram rows become the crash example's local native package, the compile cap is 180 s)
- **Side-effect inference misses that DCE'd real writes** - writes through forwarded `const?` args (#3313, fixes #3311) and through pointer-offset args (#3324, fixes #3321) are now recorded
- **`E(a=1)` on an enum silently folded to the zero enumerant** (#3427, fixes #3426)
- **Block variables hoisted correctly in blocks with `finally`** (#3367)
Expand Down Expand Up @@ -436,6 +439,7 @@ An MCP bridge for DAP debugging (#3937) - launch, breakpoints, stepping, scopes,
- **A hung sweep names the program that hung** (#3949) - dastest prints a timestamped `run N/M: <file>` before every program of a sequential sweep and `worker K starts: <files>` before every isolated batch, on stderr, flushed at once and printed under `--failures-only` too; the `run_tests_*` cmake targets carry `USES_TERMINAL`, so Ninja streams the sweep's output instead of holding it until the command ends - a lane cancelled mid-hang keeps its last line instead of losing everything the sweep printed. The first hang those lines named was `tests/debug_agent/test_callback_threadlock.das`: its hook waited on a thread that had yet to create its context, while the tick holds the agent registry every context creation needs - `DapiDebugAgent` carries the contract, and the test now proves the callback/pinvoke serialization from one thread with the new `try_invoke_in_context`, the non-blocking `invoke_in_context` that returns false, running nothing, while another thread holds the context lock
- **Bounded apt on ubuntu runners** (#3785) - every workflow apt site goes through `ci/apt_install.sh`: the degrading Azure mirror swapped for the public archive (mirrorlist included), capped fetches, retried update - the worst case becomes a loud failure in minutes instead of a silent hours-long hang
- **Gate rosters catch up with the tree** (#4047) - `MESHOPTIMIZER.LICENSE` joins the bundle smoke's license roster (the module is default-on and its MIT notice installed, but the gate that exists to notice a silently-skipped install rule did not list it); preflight's compile sweep stops trying `examples/dasLLAMA/library/main.das`, whose `dasllama_lib_c` only the opt-in `dasllama_lib_jit` target writes (#4045); `tests-cpp`'s `standalone_ctx` fixture pins its output directory with `$<1:...>` so `standalone_capi_dasbind` passes under Visual Studio and Xcode generators (#4045)
- **The nightly goes green, and a PR that closes a nightly-failure issue runs the nightly** (#4046) - the LLVM-AOT rail's five causes fixed (above), `run_tests_llvm_aot` walks the curated corpus through `dastest --test-list` instead of all of `tests/`, the standalone sweeps hand their ~900-path file list to cmake in a file (one 70 KB `-DFILES=` argument overran cmd.exe's line limit), the RelWithDebInfo lane's `test_worker_heap` ceiling widens for a free-list-off allocator (10248 KB against a 10240 KB cap), the tsan lane's two plain-global writes in `Module::Initialize` (the fusion hooks, the deferred loader) become a `once_flag` and an atomic; `pre_job` resolves a PR's closing issues and labels into one flag that arms the nightly-only jobs and steps on that PR, while the issue-filing job still gates on `schedule` alone

- **The nightly benchmark stand** (#3976) - one box runs every `[benchmark]` in the tree and plots each arm over commits; `benchctl`'s `run` and `report` verbs, a static viewer with no build step
- **The SDK payload** (#3563, #3571, #3857, #3914, #3963) - the 0.6.4 version bump; the install ships what the docs promise and the smoke test stops hiding a hole; every third-party license ships; generated `[arch]` excerpts; the generated binding tables `aot_builtin_ast.h` includes
Expand Down
21 changes: 16 additions & 5 deletions ci/smoke_test_bundle.sh
Original file line number Diff line number Diff line change
Expand Up @@ -68,20 +68,26 @@ cd "$BUNDLE"
# running it: nothing ever compiled it, so nothing noticed.
COMPILE_TESTS=(
"aot|utils/aot/main.das"
"aot-llvm|utils/aot/main_llvm_aot.das"
"benchctl|utils/benchctl/main.das"
"dap|utils/dap/main.das"
"das-fmt|utils/das-fmt/dasfmt.das"
"dascov|utils/dascov/main.das"
"dasllama-convert|utils/dasllama-convert/main.das"
"dasllama-server|utils/dasllama-server/main.das"
"dasllama-server-bench|utils/dasllama-server/server_bench.das"
"daspkg|utils/daspkg/main.das"
"detect-dupe|utils/detect-dupe/main.das"
"fix-lint-errors|utils/fix-lint-errors/main.das"
"gen1-to-gen2|utils/gen1-to-gen2/main.das"
"jobque-timeline|utils/jobque-timeline/main.das"
"lint|utils/lint/main.das"
"lsp-nav|utils/lsp/subtools/nav.das"
"lsp-validate|utils/lsp/subtools/validate.das"
"mcp|utils/mcp/main.das"
"mcp-cpp|utils/mcp/cpp_main.das"
"mcp-setup|utils/mcp/setup.das"
"watchdog-das|utils/watchdog/main.das"
# Not an entry point, but the library an adopting repo's REVIEW.das requires
# (REVIEW_COMMON.md contract) — a bundle where it does not compile breaks
# every external gate.
Expand All @@ -108,6 +114,7 @@ COMPILE_TESTS=(
# user's box.
SHIPPED_EXE_TESTS=(
"daslang-live|cpp"
"watchdog|cpp"
"benchctl|dasexe"
"dascov|dasexe"
"das-fmt|dasexe"
Expand All @@ -117,11 +124,6 @@ SHIPPED_EXE_TESTS=(
"lint|dasexe"
)

# Stdio launch test for the mcp JSON-RPC server (run from source via daslang —
# the exe is no longer bundled): the only safe "did it actually start" probe is
# to feed empty stdin and check for a clean exit. It prints "Starting daslang
# MCP server" then "stdin closed, shutting down" within ~1s.

PASS=0
FAIL=0
LOG="$(mktemp)"
Expand Down Expand Up @@ -217,8 +219,17 @@ done

echo
echo "Runtime launch:"
# A stdio JSON-RPC server (the MCP server and the DAP bridge run from source, the watchdog's
# LSP front in the static exe) is started on empty stdin: a clean exit is the only safe
# did-it-start probe. The watchdog's own help is captured, not piped, so its exit code counts.
run_check "mcp.das (empty stdin)" bash -c \
"'$DASLANG' utils/mcp/main.das < /dev/null"
run_check "dap.das (empty stdin)" bash -c \
"'$DASLANG' utils/dap/main.das < /dev/null"
run_check "watchdog --help" bash -c \
"out=\"\$('$BUNDLE/bin/watchdog${CPP_SUFFIX}' --help)\" && printf '%s' \"\$out\" | grep -q -- '--stable-seconds'"
run_check "watchdog --lsp (empty stdin)" bash -c \
"'$BUNDLE/bin/watchdog${CPP_SUFFIX}' --lsp < /dev/null"

# Two prebuilt tools past --help. dastest.exe must compile and run a shipped suite
# (isolated mode also spawns its own workers); lint.exe over daslib must resolve every
Expand Down
11 changes: 11 additions & 0 deletions tests/fio/_fixture_echo_args.das
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
options gen2

require daslib/clargs

//! one argument per line - tests/fio/popen_argv.das matches on these lines
[export]
def main() {
for (a in get_cli_arguments()) {
print("{a}\n")
}
}
14 changes: 14 additions & 0 deletions tests/fio/popen_argv.das
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,20 @@ def test_popen_argv_unknown_flag_nonzero_exit(t : T?) {
}
}

[test]
def test_help_after_separator_reaches_the_script(t : T?) {
t |> run("--help and -h after -- are script arguments, not the driver's") @(t : T?) {
let fixture = path_join(get_das_root(), "tests/fio/_fixture_echo_args.das")
var output : string
let rc = run_argv([das_exe(), fixture, "--", "--help", "-h"], output)
t |> equal(rc, 0)
t |> success(find(output, "--help\n") >= 0, "the script should see --help: {output}")
t |> success(find(output, "-h\n") >= 0, "the script should see -h: {output}")
Comment thread
borisbat marked this conversation as resolved.
t |> success(find(output, "daslang version") < 0,
"the driver must not answer a help flag placed after --: {output}")
}
}

[test]
def test_popen_argv_unix_dollar_paren_not_expanded(t : T?) {
t |> run("argv with $(...) is passed literally, not expanded by a shell") @(t : T?) {
Expand Down
20 changes: 15 additions & 5 deletions utils/REVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,17 @@ doc: `CLAUDE.md` (repo root).

A tool is a directory holding a program and the files only that program uses. A directory is
a tool when `utils/CMakeLists.txt` (beside this file) or the repo root's `CMakeLists.txt`
builds or ships a program in it, wherever that directory sits; a directory under `utils/` is
builds or ships a program in it, or when `utils/CMakeLists.txt` runs a `DAS_UTILS_TO_TEST`
suite against a program in it, wherever that directory sits; a directory under `utils/` is
also a tool when its `.das_package` declares a program with `release_main` (its own) or
`release_program` (a companion). A change under `common/` (beside this file) is a change to
every tool that requires it. An arm is one test case of a tool's suite - a `t |> run(...)` case
of a `[test]` function. An arm's load-bearing assertions are the ones that prove the change,
never a skip-path assertion.
`release_program` (a companion). A tool is shipped when `cmake --install` puts it in the
bundle: a `DAS_UTILS_SHIPPED_EXES` entry or a C++ executable target in `utils/CMakeLists.txt`
(beside this file), a C++ executable target in the repo root's `CMakeLists.txt`, or an entry
point an `install(FILES ...)` rule in the repo root's `CMakeLists.txt` copies as source. A
change under `common/` (beside this file) is a change to every tool that requires it. An arm
is one test case `dastest` runs - a `[test]` function, or one `t |> run(...)` case inside one.
An arm's load-bearing assertions are the ones that prove the change, never a skip-path
assertion.
A CI row is a workflow step whose command runs the arm, directly or through a process it
spawns. An assertion no CI row can run is one where either no CI row runs the arm, or the arm
returns or skips before the assertion. An arm that skips unless a host tool is present has
Expand Down Expand Up @@ -58,3 +63,8 @@ A run against an already-deployed artifact proves nothing about the diff under r
**A diff that adds or renames a key in a `watchdog.json` that belongs to a tool, wherever the
tool sits, names a `WatchdogConfig` field in `watchdog/watchdog.das`, in the same change** -
the supervisor refuses to start on an unknown key.

**A diff that adds a front to a shipped tool adds a `run_check` row starting that front to
`ci/smoke_test_bundle.sh` (repo root), in the same change.** A front is a command-line mode
that makes a program serve a different role or protocol. The bundle gate proves only the
fronts it starts.
2 changes: 1 addition & 1 deletion utils/daslang/main.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -1120,7 +1120,7 @@ int MAIN_FUNC_NAME ( int argc, char * argv[] ) {
dumpLeaks = true;
} else if ( cmd=="-no-dump-leaks" ) {
dumpLeaks = false;
} else if ( cmd=="h" || cmd=="-help" ) {
} else if ( !scriptArgs && (cmd=="h" || cmd=="-help") ) {
print_help();
return 0;
} else if ( !scriptArgs) {
Expand Down
Loading
Loading