Keep the language concise to write, simple to read and consistent to understand. A design rule is a plan on trial. Discuss it when applying it makes a useful expression harder to read, removes something expressible or requires another special case. APL familiarity and compatibility are not costs in that decision. Change callers with an API change instead of adding compatibility layers. Give each kind of thing the general path rather than a separate one of its own. A system name, for example, resolves to a function and compiles as a glyph does.
Speed and simplicity come before edge cases no real user meets. Follow IEEE, Rust and CPU behaviour where it gives a reasonable result. Precision follows the rules in the "Overview" of rustymath's DEV.md.
A form with a reasonable meaning gives a result, not an error. A result that would be a bug in some programs, with no error, is acceptable when it buys useful behaviour or speed. Negative positions count from the end in every selection, as in Python, even though x.[i-1] at i=0 then reads the last item. ⌈ beside NaN gives NaN or the other argument, whichever its kernel gives. Jeremy settled negative positions on 18 October 2026, after the question had come up repeatedly.
BPL serves two kinds of work. Default, dynamic kinds give the APL REPL experience that most users want most of the time, with exact integers that never wrap, exact rationals and tolerant comparison. Static kinds serve speed-sensitive work. Supporting both kinds costs complexity, not static speed. Large work runs in notebook cells and scripts, not in the REPL.
Typed programs and the JIT have to end in code that is both fairly simple and very fast. If they can't, the approach itself is in question. Under static kinds, programs on integers should fuse, compile and run array-wise, with the simplest implementation that allows. That calls for one integer regime under static kinds, in which an ordinary integer is a fixed integer of the int width. Integer overflow should wrap on every route, because a check on each item and an error path out of a fused or compiled loop cost speed and code. Every integer operation should compute at its type's width. Each width's kernels cost compile time, binary size and code. A kernel family is added only where its benefit outweighs that cost.
Big-data analysis runs keyed arrays of functions over keyed arrays of data. Keyed and named arrays should therefore fuse and compile as plain ones do. The semantics of missing items and fill can change if that makes this fast and reasonably simple.
Laziness between cells isn't worth optimizing, nor are chains that rebind a name so that earlier lines compute nothing. Pending arrays fuse large calls in code that no trace holds. A view, such as a window or a transpose, stays a view until a call needs its items in memory.
In the language, a bare number is a float: 3 is a float, and an integer is written 3ₓ. That rule is about the language, not the implementation. An internal route may run a whole float as an integer where the result can't change.
This file is internal guidance for BPL's developers. It makes no commitments.
BPL and rustymath care about two things: speed and simplicity. Before designing a mechanism, study how great software such as tinygrad and JAX solved the same problem. Their authors worked hard to get the foundations right. Let those designs guide ours. Duplicate code, special-case hacks and several paths for similar work are complexity. Find the one principled mechanism. Give each fact one home. The aim is something pure, correct and principled (Jeremy, 30 October 2026). AGENTS.md states the founding principles. The paragraphs on accuracy and on dynamic kinds below give their details.
LLMS: Read nbs/llms.txt in full before doing any work. BPL differs substantially from Dyalog APL in grouping, names, numbers and operators. Use BPL syntax, not APL idioms assumed to carry over. The links in nbs/llms.txt are relative to the published site. In this repository, read each linked page's source under nbs/, as in nbs/glyphs/at.qmd for glyphs/at.html.md and nbs/indexing.ipynb for indexing.html.md. The published site lacks unreleased changes.
Before writing BPL for tests or examples, read nbs/composition.qmd on composing functions.
Rustymath and basedpl have no outside users yet, so treat neither as published. Machinery is shared when it has two users: basedpl and rustymath's Python API. Changing rustymath to give them shared machinery is encouraged. Separate machinery in basedpl and rustymath for the same task is an error, unless Jeremy has asked for them to differ there.
These design notes in meta/ guide work still ahead:
jit-plan.md: the staged plan for the typed program, the JIT, batching and AD, and the GPU, with "Settled with Jeremy".graphreview.md: the review whose path the plan stages, with the program's requirements in section 5 and the milestones that the stages follow.stage0-design.md: the numeric rules, operation plans and the typed program, with Jeremy's answers in section 7. Section 6 names the parts ofgraph-ir.md,gpu.mdandrecorded.mdthat still apply.programs.md: AD's design basis, in "The bar", "Role models: JAX, tinygrad and Dex", "Semantics of∂", "Rules" and "Language changes under study".index-compare.md's "Decisions" andbinder.md's levels: index notation.index-levels-2026-10-26.mddesigns levels 2 and 3, which aren't built.linear.md: the analysis that carries over to deferred Jacobians. Jeremy replaced its notation on 17 October 2026, as "Language changes under study" inprograms.mddescribes.stage3-build.md's "Step 4: shared values as outputs": the plan for bringing outputs back.stage4-build.md's "Step 5 in outline", its choice 8 and "Not in stage 4": the stage's last step and what it leaves out.design-audit-summary-2026-10-26.md: the combined view of four audits of stages 1 to 4, whose order of work the indexing release's plan replaced. The fourdesign-audit-*notes beside it give the evidence.learned-2026-10-26.md: what 26 October 2026 established, where each record is, and what's planned.open.md: every open item.
For the other notes, Jeremy's rule applies (26 October 2026): "note that meta/ docs used to build old features, like PRD.md, are of historical interest only." closed.md archives decisions and finished work, for study when an old decision matters.
BPL and rustymath give up minor reproducibility and tiny accuracy promises for faster, simpler code (Jeremy, 26 and 30 October 2026). The "Accuracy" section of nbs/numbers.qmd states how accurate float results are: within 4 units in the last place of the correctly rounded result, apart from the exceptions that rustymath's accuracy page lists in one place, with some results exact. Last bits, NaN behaviour and the sign of zero aren't promised. The last bits of a float result can differ between thread counts, array sizes, platforms, builds, runs, routes and spellings. Each of these can change the order in which a fold such as +/ adds. A few ulps lost for real speed or a simpler implementation become a documented exception on rustymath's accuracy page. Undoing agents' habit of treating identical bits, or a tighter bound than the documented one, as a goal has cost this project many rewrites. Never spend speed, memory or code on agreement between results, on reproducibility, or on accuracy past the documented bounds, unless Jeremy agrees that a real, common use needs it:
- Don't make a fused, compiled or batched route follow another route's fold order, block structure or kernel to match its bits. Each route may regroup a fold for its own speed.
- Don't add a reference case, harness run or test that compares floats from two routes, thread counts or array sizes bit for bit, or that checks NaN behaviour or the sign of a zero. Compare floats within the accuracy that the Numbers page describes.
- Don't count identical bits as a benefit of a design, or different last bits as a regression.
These non-commitments don't license breaking anything. Keep the exact results that the Numbers page lists. Remove code that exists only for agreement between results, or for accuracy past the documented bounds, where its removal cuts lines or adds speed without substantive complexity and breaks nothing else.
Autodiff and fast code are written for static kinds, with branches as Agenda (⍚) or f@[p], not dfn guards on traced values ("Role models: JAX, tinygrad and Dex" in programs.md). Dynamic kinds serve interactive analysis. Code under them, and dfn guards on traced values, must run correctly. Speed there is welcome when it costs no complexity, but don't add code or designs to trace, fuse or compile them. They come second, not nowhere. The coverage matrix and the speed tests keep dynamic-kinds cases, so that their costs stay known and a new slowdown fails. Annotate a slow event that a type depending on values causes under dynamic kinds as allows, never gap. An allows line records a known cost. A gap line marks work to do.
cargo develop
cargo test
pytest -qUse cargo develop to rebuild and install the native extension and bpl after Rust changes. It uses the test profile shared with cargo test. Cargo tests compile missing code but don't install it. Run plain cargo test while you work; use one test binary for a focused check. From a kernel, fastws.cargo.run_tests(root, *args) runs cargo test and returns the build's compiler messages and each test binary's results. Run pytest -q at the end of a piece of work. Before you report a pytest failure in code you didn't touch, run cargo develop and rerun it. Give builds three minutes and test runs one minute; Cargo serializes builds in the same target directory.
Before a stage's closing review and before a release, run the full checks: what CI runs, and the builds for the targets that this Mac's daily builds leave out. Run cargo develop, cargo test, pytest -q, bpl-nb, nbdev-test, node tests/input.mjs and node tests/highlighting.mjs. Then run bpl-tests target_builds /tmp/<prefix>_wasm, followed by wasm_test(DIR) and wasm_test(DIR, speed=True) on that directory. target_builds builds basedpl's and rustymath's libraries for Linux x86-64 and the browser package at once. "Codegen and rebuilds" says how to find the function that holds up a build that stays on one core for minutes.
Build nothing else without discussion: no scratch crates, alternate target directories, --release, -p or other-target builds. The documented build diagnostics, browser work and x86-64 inspection below are exceptions. Don't use maturin develop; its configuration does not reuse the normal local build.
When another agent's unfinished edits stop the shared tree compiling, run cargo wait with the Bash tool's run_in_background, then build after it exits. cargo wait comes from fastws-cli, as cargo develop does. It runs cargo check on every target of the workspace each time a source file or manifest of a local package changes, rustymath's included. It exits once the tree compiles, or after 30 minutes with the remaining errors. cargo wait --path ~/aai-ws/rustymath waits for rustymath's own workspace, its Python extension included. Use it rather than writing a wait loop.
An already-running Python process retains its loaded extension. In a kernel, use from basedpl.worker import fresh; fresh(code) to check the installed bpl in a fresh worker instead of restarting the kernel. A BPL error raises BplError. fresh(code, build=BASE) runs another build, such as the baseline described below. A list of expressions runs each in its own fresh worker and gives one row for each, holding its output or its error. prefs= runs before the code in each worker, as in fresh(cases, prefs='•prefs ["kinds":"static"]'). Check one expression and read its result before building on it. fresh, time_cases and the failures that check_file and check_page give show each input's linear form (basedpl.syntax.linear), which shows with brackets and parentheses how BPL groups it. basedpl.bpltests.check_file and check_page check reference files and documentation through the worker without building Rust. With build=, they check another build without installing it.
Measure with BASEDPL_COSTS=1 cargo test --test speed or basedpl.bpltests.time_cases(cases). Both count costs in copies. A copy is the time to write as many floats as the data has items, each negated, into a second buffer. rustymath::timing::COPIES stores the copy time of each machine that runs the speed tests, by its CPU, the item's size and the number of items, at the slower end of what that copy normally takes, for copies whose source and destination fit in cache, 64 MiB together (CACHED_BYTES). A copy of 65,536 items timed beside each case varied more than 3 times on CI's runner, because it read its source from the L3 cache that other VMs share, while the functions' own times varied at most 1.18 times. Stored times keep that noise out of every cost. A larger copy runs at the speed of memory, which a shared runner varies by up to 2 times between runs, and the cases of that size vary with it. The test measures such a copy once in each run. It also measures a cached size with no entry once, and prints a line to add to COPIES. The browser build has entries of its own, for its copy under the JavaScript engine. It can't read the CPU's name, and tests/wasm-pkg.mjs passes it the name that Node reads. Each case's line in a log ends with its time for each item, which logged_costs keeps in each row, and the log has a copy line for each size of data, which it reads into its result's copies. A case whose footprint, its data and the most bytes it holds at once (rustymath::memory::peak_held), passes 1 MiB (rustymath::timing::CORE_CACHE_BYTES, one core's L2 cache on CI's runner) also measures a copy of its footprint beside it. Its line then ends with the footprint and that copy's time for each item, which logged_costs keeps in footprints. Data past L2 reaches the L3 cache and memory that the runner shares with other VMs. In runs 56 to 62, cases from about 2 MB up swung by up to 4 times between runs, a different few in each run, while cases of up to 1.5 MB held their times. Costs still divide by the copy of the data. The footprint copy shows whether a case's time follows it. Budgets are single-thread measurements. thread_scaling() checks multithreaded speed. It times a large product, a 512×512 SVD and Hermitian eigendecomposition, a stack of QR decompositions, a product of a convolution's windows, a copy of those windows and a large elementwise function on one thread and on t threads, the smaller of 4 and the machine's physical cores. Each case's serial fraction s, the share of its one-thread time that doesn't split across threads, gives its expected speedup S = 1/(s + (1 − s)/t), by Amdahl's law. s comes from first principles for our own kernels. On macOS, Accelerate computes the product, the SVD, the eigendecomposition and the product of windows, and their s comes from measurement. The product of windows fails where its rows no longer split across threads, and the copy of windows where a copy's items no longer split. Its s comes from measurement too, because memory bandwidth, not serial code, limits a copy's gain. A case fails below half its expected gain, 1 + (S − 1)/2, which is 2.5 on 4 cores and 1.5 on 2 for a case that splits completely. A case expected not to split, with s of 1, fails below 0.9. Accelerate's SVD and eigendecomposition are expected not to split, because its GESDD and SYEVD run on one thread whatever its threading setting. The manual Speed workflow runs it. On Linux, thread_scaling counts one core for each set of hardware threads that share a core, so a VM that doesn't report those sets counts each vCPU as a core. The Speed workflow's lscpu step shows what CI's VM reports. speed_costs() reads the cases in tests/speed/*.bpl. cases= selects expressions, wasm=DIR selects a browser build, build= selects another build, such as a kept build's id, native=True times rustymath's own linear algebra, which dev builds compile in, and prefs= runs BPL before each section's header, such as •prefs ["kinds":"static"]. logged_costs(log) reads CI output from its text or a file. cargo_costs(path) runs either project's Rust speed test, or with build=EXE a saved copy of that test's executable, and pytest_costs(path) runs rustymath's Python speed test.
Costs in copies can't show a per-call change of a few percent, because code layout alone moves a timing by more than that. instructions(cases) counts instructions, which load and layout don't change. Each case is an expression, such as k(⌈/⊢)⌸a, run once per step inside {case ⋄ ⍵}⍣N 0. That dfn's own call adds about 1,900 instructions a step. A case that names N makes its own calls instead, such as {⍵}⍣N 0, which makes one dfn call a step. Such a case must make one call per step: a map such as f¨N⍴0 calls a dfn once on all its cells, so it gives no per-call cost. It runs fresh single-threaded bpl processes at four values of N, three at each value (runs=), and keeps each value's lowest count. For each pair of successive values, it divides the difference in instructions retired by the difference in N, which cancels startup and setup=, and it reports the median of those three slopes. The median drops a one-off cost that starts once N passes some size, because such a cost falls between only one pair. ≢•ucs N⍴"ab" "cde" has one: it retires about 5×10⁶ more instructions from about 131,350 items on. Every value of N lies past the warm-up, from a rough count at 1,000 steps, because a case that allocates an array at each step retires extra instructions over its first steps while the allocator settles. Each process pays a different warm-up, up to about 5×10⁶ instructions, so successive values of N lie 5×10⁸ instructions apart, where that difference moves a slope by under 1%. counts= sets two or more values instead. It reads /usr/bin/time -l on macOS and perf stat on Linux. build= counts another build. A list of builds, as in build=[base, None], counts each at the same values of N and shows them side by side, with the change from the first build to the last. A case whose result stays pending, such as large windows or a transpose, counts making the pending copy, not the copy: 3↕v retires about 12,000 instructions a step. force=True computes each step's result in full, through an indexed write, for any result that is an array with a first item: 3↕v then counts the copy itself, about 9 million instructions a step. The test that picks those results adds about 6,000 instructions a step. A join computes at once and needs no force.
tests/instructions.txt holds the instructions that each case retires for each step, as counted on macOS on Apple silicon. Most cases make one call per step, from {⍵}⍣N 0 to fib. Three map a function over the same 64 cells each step, through Each, Rank's general path and a system function's own item map. instruction_budgets() counts each case through instructions and marks a case over when it retires more than 3% above its count, and stale when it retires more than 3% below it. tests/test_instructions.py fails on either mark, and skips on other platforms. The change that makes a call cheaper also lowers each count it moves. A new compiler version can inline differently and move the counts, and then they're set again from bpl-tests instruction_budgets. CI checks only the speed budgets.
profile(case) shows where a case's instructions go, by function, on macOS. It takes a case, and force=, as instructions does. A fresh process repeats the case on one thread while sample counts how often each function is at the top of the main thread's stack. A function's share of those samples, times the build's count from instructions, estimates its instructions per step. The estimate assumes every function retires instructions at the same rate. When a count moves, profile_diff(case, base) shows which functions moved it, from those estimates for both builds. A function that one build inlines and the other calls out of line appears under one build alone. Run them before reading machine code. base is a build, such as the one installed before a change, and build= sets the build to profile, the installed one by default.
call_profile(case, function) counts every instruction that one call of function retires, with the calls it makes, by function, on macOS on Apple silicon. lldb runs the case once per step in a fresh process, lets skip= calls of function run, then single-steps the next one to its return. function is a regex for a demangled name, such as binder::derived, and needs code of its own. base= names a build whose counts appear beside those of build. The allocator's functions, with the functions that they call, share one row, because what they retire depends on the heap's state. A second total leaves that row out. These counts are exact, where profile's come from time. profile_diff once put about 50 instructions of a value mark's rise on a conversion loop whose machine code hadn't changed, and single-stepping found them in derivative::converted. Use profile_diff to find the function whose call changed, then call_profile on that function to settle the change. It counts calls of up to 100,000 instructions, at about 450 a second.
bpl-tests NAME ARGUMENT... runs one of these functions from the shell and prints its result: check_file, check_page, time_cases, instructions, instruction_budgets, profile, profile_diff, call_profile, disasm, jit_disasm, speed_costs, thread_scaling, cargo_costs, pytest_costs, compare_costs, wasm_test or wasm_stack. For example, bpl-tests time_cases '+/v' --threads 4 times +/v on four threads. time_cases, instructions and speed_costs take cases as positional arguments. When a case starts with -, put the cases last, after --, as in bpl-tests speed_costs -- '-.×m' '+/v'. bpl-tests NAME -h lists a function's arguments. Called in a kernel, the same functions return Python objects, such as the rows that compare_costs reads.
On each push, the Test workflow logs every native cost in its test job and every wasm cost in its wasm job. The speed test that writes them times the test-profile build that the budgets limit. bpl --costs PATH times a .bpl file or a directory of them as the speed test does, and prints each cost with this machine's budget, then each case still over its budget after the retries, marked over, without failing on a budget. The manual Speed workflow runs it on a dist build, the profile releases use, on CI's runner, beside its wasm timing. The Release workflow runs it on each binary it builds, on GitHub's runners. Their logs show what releases cost, for comparison with the Test workflow's. logged_costs reads all three workflows' logs. When you review a run, look for costs over their budgets and for costs that changed by more than 20% since the run before. compare_costs(before, after) lists both, from two runs' logs or from the rows that speed_costs and the other timing functions give. It measures each change on times per item where both runs give them, so a change to COPIES doesn't show as one. Every log names its CPU in its copy lines. Study a slowdown until you know its cause only when it appears in more than one CI run, or when two or more similar cases slow by similar amounts. Treat any other slowdown as normal variation.
A budget is a case's expected cost times a multiplier: 2 under 50 copies, 1.5 from 50 to 500, and 1.3 over 500. The multiplier is largest for cheap cases, which vary most between runs. The expected cost starts from a first-principles analysis of what the case should cost. A later expected cost comes from a fuller understanding of the case than that analysis gives, such as a platform difference or a cost the analysis missed, and the case's comment records it. Every footer holds the rule budget, the expected cost times its multiplier. Where a case costs more than that allows, a line among its directives holds the column: ⍝⍝⍝ held linux 15.6 wasm 18.9: REASON for a gap in the code, naming each held column with the cost it is held at, and ⍝⍝⍝ busy linux 2.7: REASON for a case whose data passes CI's 1 MB L2, which a busy host slows in some runs. The speed tests apply the multiplier to the held cost, and a case's line gives both budgets, as in budget 32 held from 16. A gap's hold is stale once the case costs at most 0.9 times its rule budget, inside the 10% margin under which a review holds a column. Each run prints such a case, marked stale, whose held line can go. A busy host's hold is never stale. Where a case runs within 10% of its budget on a platform that checks it, a held line raises that column's budget by 10% as a margin against noise, with the date (Jeremy, 30 October 2026: "so we don't have flaky CI"). Its held cost is the one whose budget, rounded up as budget rounds it, comes out at least 10% higher: ⍝⍝⍝ held wasm 17.6 raises a rule budget of 32 to 36. Such a hold goes stale under the rule above, once the case costs at most 0.9 times its rule budget. rustymath::timing holds the multiplier (budget), reads these lines (hold) and applies them (Budget) for every speed test. Rustymath's budget files write them as # held ... and # busy ... lines above a case's line. A platform's cost that no run has measured there, such as an estimate for Linux or wasm, keeps twice the estimate until CI measures it. A held line marks such a cost with a ~ before it, as in ⍝⍝⍝ held linux ~107: REASON. Rustymath's tables still sit at twice their expected cost, the rule before 27 October 2026. A case over its budget needs study, which finds either a speed bug to fix or something the analysis missed. Avoid raising a budget to match a run without understanding why the case costs what it does. Each ⍝⍝: footer in a BPL speed file holds a native budget for macOS on Apple silicon, then one for Linux on CI's x86 runner, then, in basedpl's file, a wasm budget for Node on CI's x86 runner. A speed test fails on a budget only on the CPU that the budgets were set for, rustymath::timing::BUDGET_CPU: the development Mac's for macOS, and CI's runner's for Linux and wasm. On any other CPU, it prints every cost, then each case still over its budget after the retries, marked over, and fails on none. A cost in copies depends on a CPU's caches as well as its speed, as rustymath's DEV.md shows for the 9950X that joined CI's runner pool in October 2026. One expected cost serves every platform at first, until a specific difference on one platform is found. macOS running matrix products and decompositions on Accelerate, where CI runs rustymath's own kernels, is such a difference. A comment above each case, or above its group, gives the expected cost with its reason, and any platform's own cost with the reason for it. set_budgets(expected) applies the multiplier and writes the rule budgets for the cases that expected names, a dict from each case's code to its expected cost, leaving held and busy lines as they are. builds= limits the change to one platform's column, for a cost that differs there. estimate=True keeps twice the cost, for a platform's estimates. A case can appear in more than one section, such as copies and widths. sections= limits the change to the sections it names. path= selects a BPL speed file or directory, or one of Rustymath's table-format files, tests/speed.txt or tests/speed_python.txt.
tests/explain.rs's speed_cases_take_their_routes captures every speed case after warm-up, with its normal setup and size, forcing the result outside timing (timing::routes). It also captures each reference case that has route directives, once under ordinary preferences. Within cargo test, only this test checks route directives. check_file checks a file's route directives too, as this test does, on one thread. Put ⍝⍝⍝ route NAME above a case to require that fast route to execute. A permitted slow event never excuses a missing required route.
The checker rejects unlisted slow events: pending items computed outside a fused group (stored), a call that computes pending items (forced), a batched fallback (fallbacks), a write that leaves mixed storage (mixed), itemwise work (itemwise), or a stopped trace (stopped). Name the event as •explain does: the function for forced, the code for mixed, the route for itemwise, and the reason for the others. ⍝⍝⍝ allows RECORD "EVENT": principle permits work such as a matrix product reading its arguments whole. ⍝⍝⍝ gap RECORD "EVENT": what should happen names a fast-path gap. A stopped trace takes an allows line only where the stop is by design, as dynamic kinds' rule for types that depend on values makes it. Any other stop takes a gap line. A considered route that was not taken (declined) is informational. You can name its route and reason, such as fusion: an exact argument, in a gap or allows. Every annotation must match an observed event. Remove it when the event disappears. cargo test -- --nocapture prints the remaining gaps.
The route test also runs the coverage matrix (reference::coverage), which checks every pervasive form in the facts table against the executors. Each form gives cases on floats under dynamic kinds and under static kinds: the form between two other steps of one block stage, as in 2×*1+x, 2×(1+x)|y, 2×(1+x)|c for the constants 0, 1, 2, 0.5 and ¯1, and 2×c|1+x for 1 and 2; its fold, |/2×x; the product m+.|k; and the gradients of each. They run with fusion at any size and every stage compiled (settings::Compiles::Always). A mapping case must take the compiled route, a fold the fused fold route, and a gradient the trace route. A block stage that runs compiled parts takes compiled. Where its compile setting asks for compiled parts, it declines compiled once for each step that doesn't lower, named by its form in the facts table, as in compiled: residue. A call large enough to wait that no block stage computes declines fusion with its plan's reason (fusion::defers). The message for a missing required route lists the run's declines. tests/coverage.txt lists the gaps that the matrix accepts, one line each: the section, dynamic or static, a tab, the case's code, a tab, then a ⍝⍝⍝ gap line for work to do or a ⍝⍝⍝ allows line for a known cost. By design, dynamic kinds keep a call whose result's type its values decide out of block stages and traces. Cases that miss their route for that reason take allows lines that cite the rule. A case that it lists needn't take its route. As with an annotation on a reference case, a line whose event no longer appears fails, so a fix deletes its gap's lines. A line that names no case is an error too. A change that opens a gap fails with the case's code and its declines or slow events: add a line that names one of them, with what should happen. The test prints and skips the cases whose evaluation fails, such as ~ on floats.
Rust code in both repositories reports routes taken, declines and stopped traces through rustymath::events. basedpl records itemwise routes in •explain's own rows. With no capture, reporting costs one atomic load. The route test compares observed successes, declines and fixed itemwise routes with the IDs declared there, including cases for idioms and annotated references. The inventory includes warm-up observations. Route contracts use the warmed run alone. This checks event-name coverage, not every branch at an emission site. Study any unexpected event before adding an annotation, as you would a speed case over budget.
The speed tests and thread_scaling time each failing case again after the rest, after each pause in rustymath::timing::RETRY_PAUSES, of 0.1, 1 and 10 seconds, and fail only for a case that fails every time. The 10-second pause waits out a longer burst of load from other VMs on CI's runner. Before each retry, the speed tests and bpl --costs keep the thread busy for 0.1 s (rustymath::timing::warm). On this Mac, small cases timed straight after the 10-second pause without it cost up to 3 times as much, because an idle core runs slower until it has been busy for a while. Each retry prints the case's earlier and new costs, or speedups in thread_scaling, even in a passing run. logged_costs keeps a retried case's lowest cost. Study a case that needs a retry in most runs, as you would a slowdown.
For a change meant to make something faster, or a change to code that every call goes through, time the cases it affects before and after the change. cargo develop keeps each build it installs, read-only, under an id that hashes the build's inputs and files, and installs it by pointing one link at it (fastws.store). fastws.store.builds(root) lists the newest builds of the project at root: each one's id, when it was kept, who kept it, and the input files that changed since the build before, with * on the installed one. Before your change, note the installed build's id. Every timing tool takes build=: a prefix of a kept build's id, a path to a bpl, or None for the installed build. It resolves each build once, as it starts, so another agent's cargo develop never changes a run in progress. Time both builds in one call: time_cases(cases, build=[base, None]) in a kernel, where base is the id you noted, or bpl-tests time_cases CASE --build BASE NEW in the shell, with both ids. Each build runs in fresh workers that take turns in alternating order, rounds times, and the result holds each case's fastest time per item on each build, in picoseconds. compare_costs(*result) lists the cases that moved. Timing sessions seed their random generators at 0, and so do the processes that instructions, profile and call_profile start. Every process and every build then draws the same data. A build from before a change to rustymath::timing::COPIES divides its costs by a different copy. On this Mac, a fused case over a large input can cost up to about 15% more in one process than in another, depending on whether malloc starts the input on a 32 KB boundary, where every one of a block stage's blocks then ends. Time again with more rounds before you read a lone difference of that size as a regression. The speed test fails a case only past its budget. It misses any smaller slowdown. Other changes need only the speed test. Don't restore old code in the shared tree to get a baseline. pytest_costs and run_pytest take a rustymath build's id as build=, for a change to rustymath's Python API. On macOS, native timings of matrix products and decompositions mostly measure Accelerate. Rustymath's DEV.md says how to time rustymath's own kernels.
Compare static kinds with dynamic kinds, or programs with the interpreter, only on code written for static kinds, with explicit sizes on literals and data. Code written for dynamic kinds can give wrong results under static kinds, and its timings don't show what static code costs.
In a Claude Code session, every exhash edit is recorded in /tmp/exhash-journal/<session id> under the agent that made it. The kernel startup sets EXHASH_JOURNAL from the session id, and EXHASH_AUTHOR from the RUSTYGATE_AGENT that rustygate gives each subagent's kernel. A fork reports its own lines with from exhash import journal, then journal(os.environ['EXHASH_JOURNAL'], author=os.environ['EXHASH_AUTHOR']).stat(). In a kernel, exhash names a function that has no journal. Main reviews a fork's edits with .diff() on the journal for that fork's agent id. git can't separate one fork's uncommitted edits from another's.
Cooperating tools share the machine's timing lock, /tmp/aai-ws-timing.lock. Timings take it exclusively. Every rustc call takes it shared, through the rustc wrapper .cargo/rustc-lock that ws-sync writes into the workspace's Cargo config, so a build waits behind a timing however it was started. The reference and explain tests, pytest, check_file, check_page, bpl-nb, instructions, profile and wasm_test without speed also take it shared. A timing holds a second file, /tmp/aai-ws-timing.gate, from the start of its wait, and a shared hold passes that file first. No new build or test run then starts while a timing waits, and the timing runs once the holds already taken end. Rust code takes the lock through rustymath::timing, and Python code through fastws.timelock. When a kernel calls time_cases or native speed_costs, the bpl worker that the call starts takes the lock, not the kernel. Node can't lock a file. wasm_test and speed_costs(wasm=DIR) hold the lock in Python while Node runs. After five minutes a timing fails. A build or test run then goes ahead without the lock. After 30 seconds, an exclusive wait prints the processes with the lock file open. The list can include processes that are only waiting for the lock. A process that holds the lock shared must not start a build or a timing, which would wait behind a timing that waits for that hold until the five minutes end. Never take the lock yourself in a kernel process. Never coordinate timings by hand: the lock schedules builds and timings. A hold that outlasts its call blocks the timings that later calls launch. Native timings run on one thread and request user-interactive macOS QoS, which favours the performance cores. QoS is not CPU affinity.
Run cargo wasm during development only for browser work. Agents use cargo wasm --out DIR, then run the browser tests against that build with basedpl.bpltests.wasm_test(DIR) for tests/wasm.mjs and wasm_test(DIR, speed=True) for tests/wasm-speed.mjs. On this Mac, that speed test prints every cost and marks each case still over its wasm budget without failing, because those budgets were set for CI's runner. This avoids replacing a module another agent is using. For other edits under cfg(web) or host_web.rs, check compilation with cargo cs -p basedpl-wasm --target wasm32-unknown-unknown, which checks the browser wrapper crate together with basedpl's browser code. cargo cs alone compiles neither.
bpl-tests disasm PATTERN shows the machine code of the function whose demangled name matches the regex PATTERN, read from the installed bpl with objdump. --build BUILD reads another build instead. Where several functions match, it lists their names. A function that is always inlined has no code of its own. Read its callers instead.
The JIT's kernels exist only while a process runs. With BASEDPL_JIT_DISASM set, bpl prints each kernel it compiles to stderr: a line describing the kernel, then its machine code. bpl-tests jit_disasm CASE --setup SETUP --prefs PREFS runs a case that way, as in bpl-tests jit_disasm '+/ux>127ₓ₁' --setup 'ux←(¿(2*22)⍴256)ₓ₁' --prefs '•prefs ["kinds":"static"]'. A stage compiles on its first run only from COMPILES_AT items, about 3 million.
--x86 shows the x86-64 code that CI and Linux run. It first builds basedpl's library and the rustymath library it links for Linux x86-64, into /tmp/aai-ws-x86. Where an unfinished edit stops basedpl compiling, it reads rustymath's library alone, and its output says so first. After a change, that build takes minutes. Use it only to answer an x86-code question. The target is Linux because macOS builds leave out rustymath's native linear algebra kernels. Only the libraries build, because the Python extension crates can't be built for Linux on this Mac. Their release profile matches CI's test profile in its x86-64 flags and optimization settings. The build sets CC_x86_64_unknown_linux_gnu=clang for itself. Don't move that setting into .cargo/config.toml. CI's Linux release builds also read that file, in a manylinux container that has no clang. Don't use --emit asm. A build with it splits code into other codegen units and inlines differently.
| Crate | Responsibility |
|---|---|
basedpl |
Python-free interpreter and public Rust API. |
cli/ (basedpl-cli) |
bpl, REPL, Jupyter kernel and JSON-lines worker. |
py/ (basedpl-py) |
Unpublished PyO3 extension, basedpl._core. |
wasm/ (basedpl-wasm) |
Browser bindings. |
xtask/ |
cargo wasm, which builds the browser package. |
The default workspace members are core, CLI and Python; wasm/xtask build only when requested. Frontends use the core's public API. Core/CLI Rust tests live in tests/ and cli/tests/. Libraries set test = false to avoid separate cfg(test) builds. Because no build runs #[cfg(test)] code, libraries hold no inline tests or test-only counters. Any internal behaviour worth a test, such as whether an expression fuses, is worth knowing for every user too. Tests in tests/ check it through the public API, such as •explain, •storage or •memory. PyO3 is in its own crate, not an optional core feature. The core uses serde_json for the process protocol and data functions.
Rust 1.98 or later supplies algebraic float operations. test inherits release: opt-level 2, incremental compilation, thin LTO across each crate's codegen units, no debug information, overflow checks or debug assertions. A debug_assert! therefore never runs here. Write a check that matters as an always-on test. .cargo/config.toml adds LLVM's loop and SLP vectorizers as extra passes, in that order, and targets x86-64-v3 on x86. Target-specific rustflags repeat the passes because they replace build-wide flags. dist disables incremental compilation and strips symbols. Keep these settings aligned with rustymath's.
fastws's cargo develop builds profile test, installs the extension and stages bpl into the venv. py/ enables PyO3's extension-module and supplies its link arguments without libpython. Editable installs use fastws.build_backend; they install metadata and a path to python/ without compiling Rust. After a standalone uv sync or editable pip install, run cargo develop. ws-sync performs that rebuild after syncing. Wheels use maturin for the extension and cargo stage for the Python-free CLI. Linux staging runs inside the manylinux container.
Basedpl's crates pass on rustymath's default native feature, which rustymath's DEV.md describes. Each depends on the crate below it with default-features = false and makes native its own default. Release staging and wheel builds both pass --no-default-features. The released Mac builds therefore leave out rustymath's own linear algebra. The Python binding inherits the CLI's pure-Rust frontend dependencies to keep shared dependency features identical for release builds. On Apple targets, flate2 uses the system zlib. Other targets keep its Rust backend, miniz_oxide.
Use published dependency ranges with the lower bounds required by the APIs in use. Cargo.toml names a published rustymath range; the ~/aai-ws Cargo patch selects the sibling checkout locally. CI resolves published crates. Release rustymath changes before relying on them in basedpl CI. Don't edit fastws-generated Cargo patches, .git/fastws-cargo-key or workspace configuration, and don't commit the workspace-generated Cargo.lock. meta/ is ignored planning material, not contributor documentation. [tool.uv] cache-keys tracks packaging manifests, not Rust sources.
Test runs native and wasm jobs on pushes to main and pull requests. Native CI runs cargo test, editable installation, cargo develop, pytest, bpl-nb and nbdev-test. Python tests use the same test-profile build. The wasm job builds with cargo wasm --strip and runs both Node suites. Both jobs run on ubicloud-standard-4-ubuntu-2604, which is Ubuntu 26.04 on the ubicloud-standard-4 VM size that speed budgets are set for. They generate a lockfile before restoring Swatinem/rust-cache, making dependency changes part of the cache key. Speed is manually triggered. Its wasm job restores the Test workflow's wasm cache. Its native job builds the dist profile without one.
Release runs separately on v* tags on GitHub's runners. It builds dist wheels, sdist, standalone CLI artifacts and the Windows keyboard archive, then publishes Python artifacts and the GitHub release. Its npm job publishes the stripped browser package through trusted publishing and purges jsDelivr's package files to avoid stale unversioned URLs. Release neither runs tests nor waits for the Test workflow.
Local builds of the test profile are incremental, with thin LTO across each crate's codegen units (Cargo's default, lto = false). Incremental builds put #[inline] helpers and generic instances, such as iterator adapters, array::map and copy_from_slice, in codegen units of their own, where only LTO can inline them. Measured on 22 October 2026, thin LTO cut per-call instruction counts by 4 to 18%, to within about 3% of dist. A one-line edit in basedpl then rebuilt bpl in about 9 to 12 s, against 6 to 8 s without LTO, and an edit in rustymath took about 30 s either way. Thin LTO imports a function into another unit, where it can inline it, only below LLVM's import limit of 100 instructions. A long fused iterator chain can exceed that limit. rustymath's DEV.md lists what else stays out of line, under "Loops and codegen". Under LTO, rustc runs the extra passes from .cargo/config.toml before the link step, and the loop vectorizer only at that step. The configs list loop-vectorize before slp-vectorizer for that reason. Listed alone, SLP packed the lanes of short inner loops before the loop vectorizer could vectorize those loops across items. ∪c then ran 71% slower, in dist builds too. After an interrupted build, a link error on undefined .llvm.<hash> symbols means incremental compilation reused stale codegen units. cargo clean -p <crate> --profile test clears them.
Moving lines changes the code below them, because each panic site, such as a bounds check, holds its line. Every unit that holds that code then recompiles. Related source is split into child modules to limit those shifts. For example, the generic block runner has a file of its own, program/blocks.rs. When fusion's old runner sat in fusion/blocks.rs, a one-line edit to Graph::run recompiled one unit of about 140 KB in about 2.4 s, against about 9 s when the runner sat below it in the same file. Children use use super::* and expose items to parents and siblings with pub(super). Parents own public re-exports.
Generic loop instances can concentrate in the defining module's codegen unit. A closure passed to a generic adapter such as map makes an instance that can compile in another codegen unit. Every edit to the closure's function then recompiles that unit. When fusion's old Graph::run built a vector with map, an edit to it recompiled a 1.9 MB unit in about 8 s, and a plain loop avoided that. Mark loop helpers #[inline] when their instances need to compile with the caller's kernel table, including nested helpers. Kernel tables have separate modules: booleans, wholes, float_kernels and complex_kernels. at_widths! gives the concrete widths their own modules. Keep new tables separate and add kernel specializations only for measured gains.
Use cargo build --lib --bins --profile test --timings to inspect the normal build. Its report under target/cargo-timings/ separates frontend and codegen time. BASEDPL_UNITS=1 cargo test --test units -- --nocapture lists large units and their functions. BASEDPL_SIZES=pattern cargo test --test units -- --nocapture measures machine code matching a demangled-name regex. BASEDPL_SIZES sees only basedpl's own codegen units. BASEDPL_SIZES_DIFF=old:new cargo test --test units machine_code_growth -- --nocapture compares two binaries, such as copies of bpl from before and after a change, by generic function across every crate. BASEDPL_CALLS=pattern cargo test --test units -- --nocapture lists each call from a function whose demangled name matches to a function in another codegen unit, in basedpl's build and in the rustymath rlib it links. Those calls stayed out of line through thin LTO. rustymath's DEV.md says which calls do, under "Loops and codegen". After an edit, BASEDPL_RECOMPILED=pattern cargo test --test units -- --nocapture lists the units that the rebuild compiled again, each with its biggest functions whose demangled names match. These commands diagnose the current compilation arrangement; don't create scratch builds to approximate it.
A build that stays on one core for minutes is usually waiting on one function that an LLVM pass takes minutes over. LLVM's loop vectorizer chooses each loop's vector width and interleave count from the target's cost model. A loop that it vectorizes in seconds for arm64 can take it minutes for x86-64. On macOS, sample <pid> 5 -file /tmp/<prefix>_sample.txt on the busy rustc shows the pass, such as LoopVectorizePass unrolling a loop by its interleave factor (VPlanTransforms::unrollByUF). The nightly toolchain names the function. cargo +nightly rustc -p CRATE --lib --release --target x86_64-unknown-linux-gnu --target-dir /tmp/<prefix>_x86 --config 'build.rustc-wrapper=""' -- -Z llvm-time-trace writes LLVM's time for each pass on each function to CRATE-<hash>.llvm_timings.json under the target directory. With sccache on, a cached crate writes no trace. A crate with C code also needs CC_x86_64_unknown_linux_gnu=clang. rg -o '"dur":[0-9]{8,},"name":"[^"]*"(,"args":\{"detail":"[^"]*"\})?' FILE lists the passes that took 100 s or more, with each function's mangled name. One crate's trace can be gigabytes. cargo llvm-lines -p rustymath --lib --release --target x86_64-unknown-linux-gnu lists each function's lines of unoptimized LLVM IR, with its copies. It shows where generic instances multiply, but not what LLVM does after inlining. cargo wasm builds xtask for the host, rustymath included. On CI's x86-64 runner, the wasm job therefore waits on any stall in rustymath's x86-64 build. After a change to inlining or to loops, time an x86-64 build of the crate. rustymath's DEV.md, under "Build time and code size", gives the command and describes the stall found in October 2026.
| Files | Responsibility |
|---|---|
syntax.rs, name.rs, eval/binder.rs, eval/tree.rs |
Structural parse, interned names, runtime category binding, trains, dot access and subscripts, and dfn statements' compiled recordings. |
eval.rs, eval/session.rs, eval/bindings.rs, eval/operators.rs, eval/rewrite.rs, eval/assign.rs |
Shared calls, sessions/modules, binding storage, operators, fast combinations, scopes and assignments. |
execution.rs, error.rs |
Cancellation, deadlines, input/output and retained diagnostics. |
array.rs, array/storage.rs, array/text.rs |
Values, layouts, compact/mixed storage, gathering and source formatting. |
agreement.rs, keyed.rs, selection.rs |
Rustymath's broadcasting Agreement with basedpl's layouts and their renderer, reading paired items, named/keyed alignment, and selection/writeback. |
primitive.rs, primitive/ |
Primitive metadata, ranks/axes, inverses and array operations. Structural glyphs read their arguments, take each result's layout and Steps from a plan in rustymath's structure, and copy by it. |
number.rs, element.rs, pervasive.rs, fusion.rs, explain.rs |
Numeric domains, item operations, compact kernels, pending arrays with the programs that compute them, and the events that •explain records and the live nodes that •memory counts. |
derivative.rs, linear.rs |
∂: duals, tags, seeding, the recording of rustymath's partial derivatives and the rules for operators, and the record of linear calls that reverse mode transposes. |
transform.rs |
The core that running transformations share: one stack of levels, routing to the newest level among a call's arguments, and the read point for code that isn't a call. |
batch.rs |
Batched maps: the batch carrier, the try in each_pair that calls the operand of an Each, a Rank or an outer product once on batches of its cells while the batch pref is on, and the batch rules that routing hands calls to. The pref is on by default. tests/reference.rs runs every reference case with it off and on. bpl-tests check_page PAGE --unbatched checks a page with it off. |
plan.rs, program.rs, program/, jit.rs |
Typed programs, which pending work and ∂ run: plans that give a call's result type and layout, keys and axis names included, from its arguments' types and layouts, the trace level that turns ∂ of a function into a program of typed steps that holds the call and its derivative (program/trace.rs), kept on a dfn's definition or, for any other function, by its structure (program::Tacit), replay of the program on later calls whose arguments and free values it still fits (program/run.rs), and the units that replay its runs of elementwise steps: block stages (program/blocks.rs), which run a block of items at a time, on compiled kernels (jit.rs) for runs of steps that lower and on the interpreter's kernels for the other steps. blocks::plan finds the runs. program/contraction.rs reads the sources of folds of products and of copies, and program/layout.rs sets the maps' storage orders, for fusion and traces alike. program/verify.rs checks a program's contracts in reference runs. |
search.rs, number_theory.rs, polynomial.rs, distribution.rs |
Array-level adapters to rustymath. |
system.rs, data.rs |
System-function metadata, native functions, options and import rules. |
csv.rs, json.rs, xml.rs, image.rs, plot.rs, canvas.rs, display.rs |
Formats, rich values and rendering. |
protocol.rs, reference.rs, timing.rs |
Process encoding, semantic checks and measurements. |
host.rs, host_web.rs, js.rs |
Platform services and JavaScript values/functions. |
symbols.rs, inspection.rs |
Shared names, shortcuts, syntax/help metadata and inspection. |
cli/src/, py/src/lib.rs, wasm/src/lib.rs |
Frontend boundaries. |
User-facing language rules belong in nbs/syntax.qmd, evaluation.qmd, numbers.qmd, arrays.qmd, terms.qmd and the glyph/system pages. Define terms in terms.qmd before using them. The sections below record contributor contracts and ownership rather than duplicating the language reference.
The lexer/parser records byte spans and structural syntax without evaluating expressions. parse distinguishes complete input, incomplete input and errors. It reads numbers under the current settings, which are process-wide (settings.rs). Session::parse passes the parser the hanzi_aliases setting. syntax_tree supplies source tools with structural nodes and recovers invalid chunks as Error nodes. highlight shares the lexer.
symbols.rs owns APL, simplified and traditional Hanzi spellings. Its rows also hold each glyph's names: a glyph name for the character, which is also its page's file name in nbs/glyphs/, and a name for each valence, or one name where both valences do the same thing. The glyph reference, named entry and help read them there. Python's exported names read them, with each system function's name, through symbols::builtins. hanzi_aliases enables Hanzi glyph spellings, canonicalizing token decisions while retaining original source and spans. Other Hanzi remain name characters. _ quotes the next non-whitespace character into a name, including a glyph alias. The notation preference affects structured function/operator display, not stored source, serialization or structural APIs; help and editor metadata use the same spelling table.
The binder resolves runtime categories right to left. It owns unfinished strands, trains and bound arguments; shared function nodes are immutable. Grammatical reduction produces application requests instead of calling functions. An array that holds a function as an item has its own binder category, Callable (Category::bound), while Category::of, which assignment's grammar reads, keeps it Value. Binder::push works out each entity's role once, as it pushes it: Category::before resolves Callable against the role of the entity below, which is to its right, to Function where that entity is a subject or an absent ⍺, and to Value otherwise. Rule::get and the strengths don't change, and recordings check Callable as a category of its own. A function-only operand slot converts such an array to FunctionNode::Array in constant_operand, and Function::from_value makes the same conversion where a call by text needs a function. Primitive/operator rows own names, operand kinds, ranks, pervasion, axis behaviour and reduction identities. Change those rows instead of duplicating metadata in consumers. A non-extending primitive above its natural ranks runs through Rank; pervasive Each/Outer/Rank paths reuse whole-array kernels.
Dot names rewrite to Pick and dot brackets to selection. Numeric subscripts derive selection on arrays and ⍠ on functions from the bound operand. Writable subscripts normalize to selection calls at assignment. ⱼ, ₑ or ᵣ between two arrays then rewrites to the primitives it stands for, as xⱼy to x+y×0ⱼ1 (syntax::joined_term). Dot parentheses evaluate a namespace: T.(expr) and T⍎text bind each key on the first axis of T to its item or major cell as a local, resolve other names in the caller and keep assignments local. They share the scoped execution route and lexical lifetime checks. Between functions, dot remains inner product.
Bindings (eval/bindings.rs) holds every binding: the frames of running calls and scopes, each module's names, the current frame and the running module. Other code reads and writes bindings only through its methods. A lookup gives where a name is bound, as a Place, with its binding, and reads, writes and removals take a Place. Bindings::here gives the place where a plain assignment binds a name. Parsing interns each name as a Name (name.rs), a number that is the same in every session, and a name becomes text again only where text comes in or goes out. Each frame has slots on one stack that every frame shares: first one for each of the six names that a call binds itself, then one for each of a dfn's locals, the names before an ← in its statements (Definition::locals). A call allocates nothing for its names. A call with no left argument binds ⍺ to an absent argument, so lookup finds the call's own ⍺ and never an outer frame's. A name without a slot, such as one that ⍎ binds or an assignment in a scope binds, goes in its frame's side map. A scope's frame holds its record, and a lookup that the side map misses there reads a key through the record's own index of its keys (Keys::position), so a scope copies nothing and a key's text never becomes a Name. Each module keeps its names as entries that an index finds, and deleting a name empties its entry without moving the others. A compiled read of a name carries an address (Bindings::address): the slots on its lexical chain that can hold it, and its module's entry. It reads through Bindings::load, which looks the name up as an uncompiled read does where a frame on the chain binds names without slots or the running module isn't the recorded one. Lexical frames use non-owning parent indices. Closures see live bindings, not snapshots. A dfn's last read of a local, ⍵ or ⍺ takes the binding out of its slot (syntax::release), and leaves Binding::Released, which stops lookup there with a VALUE error. The binder hands a dfn call the arguments it owns (Argument), so ⍵ and ⍺ hold them without a copy. dispatch gives a dfn the arguments as its caller gave them, and so a composition or fork with an operand that can use them (Function::owns, set when the function is built); it lends them to any other composition or fork, and a compiled statement gives a group's train the arguments that it holds (operators::train_call). An operator hands its operand each value that it made and doesn't read again: ∘'s inner result, a fork's tine results, ⍣'s states, a fold's running result, and the cells and selections that it gathers. It gives its own arguments to its operand's last call, as ∘, ⊘, ↣, ↢, a unit ⍚, ⍣ with one count and a fork's left tine do. It lends them where it reads them after a call, as Under, Rank, Axis, stencils, Over, inner products and ⍣'s other forms do, and it lends any value that it reads again, such as the state that ⍣∞ compares and the batches that a batched try reruns after a stop. A train reads its arguments in its last part and in each fork's left tine, and the last of those reads takes them. Plain assignment is local; modified/selective assignment updates the nearest existing binding. Retain the resolved place and original array across modifier calls rather than looking the name up again. Returns and outer writes reject functions, including those nested in arrays/prototypes, whose frames would not survive. Tail calls retain dependencies from arguments and the called function. No collector is used; recheck this lifetime reasoning before permitting escaping lexical captures.
Module 0 owns session names. Each •load runs in a new module, without the caller's frame and with echo off, and returns its non-private names alphabetically. Modules remain for the session's lifetime. Recursive loading of a canonical path is a DOMAIN error. Paths starting ./ or ../ resolve against the source file; other paths use the working directory. Execute/load do not run the top-level preference header.
Dfns run statements in order and return the last statement's value. Final assignments are silent. An absent left argument propagates through the binder's absent-binding rules; default-left assignment runs only when needed. Guard handlers see bindings as they stood at failure; nothing rolls back. Wildcard ∞:: excludes interrupt, timeout and unsupported errors. Explicit named guards can catch them; catching an interrupt clears its flag.
The binder compiles a dfn statement on the statement's first evaluation (eval/tree.rs). Its recorder follows the binder's two stacks over registers and constants, and emits an instruction for each node that needs evaluating and for each reduction. A move between the stacks and the drop of an absent ⍺ compile to nothing, and a derive or a bind on constants compiles to a constant, so an operator bound to a constant operand, as in ⍤1, derives once. Later evaluations run the instructions: they make the same calls in the same order, checking the category of each name, group and call result against the recorded one. Each checked instruction keeps a template of the binder's state at that point: what its two stacks hold, its cursor, the leaf it had just resolved or the result it had just pushed, and the assignment it was making. Where a category differs, the binder carries on from that state (Binder::resumed). Registers hold the binder's own entities (binder::Entity), in an inline array of 8 (INLINE) or on the heap for a statement that needs more. The recording keeps its constants as entities, and an instruction borrows them, so a call reads a constant's array or function without copying it (array_of, function_of). The binder evaluates a statement every time when the statement selects. The parse tree depends on the evaluator, because each syntax::Statement holds its own recording. A change to a reduction that the binder records (a drop, a strand, an attach, a call, a derive, a bound right operand or a train) must change the recorder and the instructions too. strand, attached, split_call, derived, bound_right, train and tails in eval/binder.rs serve both, and the binder tells the recorder its cursor and whether a call is its last reduction. The _ => rc.fail() arm in Binder::reduce stops recording any statement with a reduction that the instructions don't cover. The binder then evaluates that statement correctly every time.
Each group or run in the statement is a level, whose instructions sit inline between an Enter and an Exit, so one loop runs the statement. The recorder writes a level's instructions straight into the statement's as the binder evaluates the level, keeping a mirror of the level's binder (tree::Walk) while it runs. A failed check resumes the binder of the level that holds the instruction. Where the group's value keeps its recorded category, the instructions after its Exit carry on. Otherwise the level that holds it resumes, and so on out to the statement. Dot access, subscripts, joiners and runs that start with an operator rewrite a statement before the binder walks it (Session::joined, Session::members), and the recording keeps the rewritten nodes. Whether a rewrite applies, and whether a run starts with an operator or ends in a dyadic one, depend on what names hold, which one reader reads without evaluating anything (Session::static_category): a name by its binding, a dot path by its field, and any other group by the operand grammar of assignment targets (assignment_operand). While a statement records, the reader keeps a fact for each name or dot path it read, with the name's address and the category it found (tree::Fact). A recording checks a level's facts again when the level starts, and the binder evaluates the statement or group from its own nodes where one differs. A run that ends in a dyadic operator is a level whose binder starts with the run's right operand on its stack, read where the level that holds it keeps it.
A group or run whose leaves are all literals, primitives, operator glyphs or such groups, and which makes no call, records as the array or function it gave, so a train like (+/÷≢) builds once. The recorder drops such a group's instructions by truncating the statement's back to where the group started (Recorder::truncate). A run that ends in a dyadic operator, whose right operand is constant, records as a constant too. A group whose value is a train with a computed tine exits with the train's tines (Term::Tines). A call of that group calls the tines in place (Op::CallTines, operators::train_call). The call builds the train and calls it instead in three cases: while a transformation or •explain runs, where two array tines are side by side, and where the last tine orders its argument (idioms::orders).
An assignment compiles to one instruction, which runs Session::assigned, the step after the target is found that the binder runs too: Session::assign, the one home of every target form, or a store of a plain name straight into the frame's slot for it, which Bindings::here found when the statement compiled. What a target's names hold decides where a target of more than one node starts (Session::assignment_start), so the instruction checks that start again, and the binder carries on with the assignment in progress where it moved. A default argument, an error guard's kinds and a dfn's last statement compile as other statements do.
Each item of a bracket literal is a level whose value goes into the literal at once, as Session::item_result converts it, and Session::bracket builds the literal for resolve and for the compiled Array instruction alike. A literal with a computed key that isn't constant stays with resolve. Each statement keeps the recording of its first evaluation (Statement::recording). A later call whose ⍺ is absent where that evaluation's held an argument, or the reverse, resumes the binder where the statement reads ⍺.
Tail calls reuse frames while preserving lexical dependencies. Handlers disable tail reuse. Evaluation depth is capped at 20,000 natively and at 244 in the browser, whose stack runs out first. Each call of a function other than a primitive counts toward it, and so do each statement of a dfn other than its last, each group, each run that ends in a dyadic operator and each bracket item, compiled or not. A compiled statement or level counts with Session::descend alone, without the interrupt check and stack growth of Session::nested. Calls of primitives other than ⍎ don't count, because a primitive never calls back into BPL. They skip the depth, capture and stack bookkeeping, and on numbers a form that applies a kernel to each pair of items runs that kernel at once (Primitive::scalar_call). A separate 20,000-frame limit bounds retained lexical frames. Late-bound name resolution stops at 128 levels. These are execution limits, not a sandbox against arbitrarily deep source, arrays or function graphs.
Value holds a number, character, function, operator or shared array. Compact storage holds a rustymath buffer::Buffer of Booleans, narrow integers, width-specific floats or complex numbers, or characters, all of one dtype, and Items borrows it as a buffer::View. BPL's promotion rules for those dtypes stay in basedpl, as Promotion in array/storage.rs. A rustymath buffer keeps the dtype it's given, and basedpl picks an ordinary integer array's width (array::ordinary_width): the narrowest that holds its values under dynamic kinds, and the int width under static kinds, where an array's storage follows its kind. Mixed storage holds Values and memoizes its prototype, its innermost lexical dependency and whether any item is a function. Other properties are computed when needed. An array holds its layout and its storage in one allocation. Its Shape holds ranks 0 to 2 inline, so a small array allocates no shape. Constructors take impl Into<Shape>: pass a slice, or an array such as [n], rather than building a Vec. A shape, key or name change on a shared array makes a view of the base array's storage and copies no items. A part of compact storage, such as one of Rank's cells or a partition (Value::part), is a view of a range of its base's items. A view keeps its whole base alive, as a NumPy view does. A part of mixed storage is a copy, because the memo answers for all of a mixed array's items. A large result of a fusible float call holds a pending call in place of its storage, and computes the storage on the first read (fusion.rs). Value::view and imported_buffer are the public width-preserving frontend boundary. They give and take rustymath's View and Buffer, which basedpl's API re-exports as basedpl::buffer.
Atoms, rank-0 arrays and singleton vectors are distinct. Function nodes retain handles, not lexical frames or Python objects. Functions match by their parts; dfns and generators match by identity. Operators can be values and can cross the native Python boundary. Values and function nodes are Send + Sync.
Numeric storage fills with zero and characters with blank. Mixed nonempty arrays derive their prototype from their first item on demand; empty arrays require one explicitly. Builders accept a value or a lazy prototype function and evaluate the function only for an empty result. Preserve function handles inside prototypes.
Layout wraps rustymath's layout::Layout, which holds the dimensions, position keys and axis names, and adds the renderer. Methods that change only keys or names keep the renderer, as select does. Methods that build a layout from axes drop it, and pairing two layouts by broadcasting keeps the renderer they agree on (Layout::paired). Rustymath's agreement does the pairing: it unites keys and broadcasts through index maps. Missing positions read each source's prototype. Agreement pairs equal axis names first. It pairs remaining axes positionally unless both have different names. Result assembly drops colliding axis names; public name attachment rejects duplicates. Contracted axes require equal key sets and retain the first order. Nonempty key lists with no named positions become unkeyed; empty record keys survive.
Gather owns copying and widening. Steps, from rustymath's gather, describes strides, ranges/fills, rotations and offset tables. Gather::walk copies compact storage through rustymath::gather::walk. Structural operations should use remap/gathering rather than reconstructing Values per item. Equal-shape cells copy in blocks. Items reads counts, positions and axes from compact storage without boxing every item. Shape, tally, positions and predicates have dedicated integer/Boolean builders.
A call chooses its fill through ⍠'s options record or a window specification's fill key (WindowSpec, in primitive/structural.rs). Primitive::fill_for validates one scalar item of the data's type (Value::fill_of); nested data takes any BPL value. Native fills travel with rustymath's mappings through pending copies, kernels and traces. copied_fill consumes fills of mixed data eagerly. Keyed pervasive calls attach native fills to their agreements without first copying the operands. Values retain the chosen fill; tangents and label ids read their own zero fill. Maps needing two different fills keep their copies separate.
Under with a function and selective assignment find their targets through labels. Each begins a label level (transform::Kind::Label) and runs the selector, or evaluates the target, on the root wrapped as a selection::Labelled value. A primitive whose row selects carries the labels through its own code. Planned forms and ⌷ compose a rustymath Mapping relative to the root, so a selection reads only the items it selects. Ordering forms select through Primitive::taken. Any other call strips the labels and runs on the values. Labels::read takes the labels of the selector's result. Labels that are still a mapping with no fill write back through that mapping, as a walk over the root (Targets::Walk). Labels::replacements turns any other labels into offsets, or into paths where the root has nested items. A strand of labelled items joins into one labelled array (Carrier::joins_at_once). An item that the selector makes rather than selects has the FOREIGN label. Writing to it is a DOMAIN ERROR. A result without labels at the level sends Under to the selector's inverse. A batched try refuses while a label level runs (Levels::admits_batch). Function-Under applies its selector to a left argument as a plain value. Positions-Under passes the left argument whole. Keyed values align to selected keys, and nested paths retain empty-cell structure.
Selective assignment removes the binding before writing. Arc::make_mut copies only when another holder needs the old array. Repeated targets must read values written by earlier occurrences. pervasive::scatter combines each target of + - × ÷ ⌈ ⌊ into compact numbers in place, in turn, at the kind that the call's plan computes in (plan::pervasive_on), through rustymath's gather::Writes, whose combine and try_combine read a selection's walk a run of offsets at a time, without listing it. Other pervasive modifiers group repeated targets into rounds of distinct positions. A target at a fill reads the fill, and its write is dropped. Append updates use Value::append when layouts permit it; rank/key/name changes fall back to ordinary catenation. Modified destructuring calls modifiers left to right; ordinary destructuring assigns right to left. An assignment's RHS runs once.
Each, Rank and outer products call their operand through each_pair in eval/operators.rs. It first tries one batched call (batch::tried), then calls the operand once for each cell and assembles the results through shared layout and prototype rules. A 0-cell preserves an atom or encloses an array item. For custom operands, empty Each, Rank and outer product calls substitute prototypes for empty arguments and use prototype mode. Explicit effects remain observable when a call runs. Catchable prototype-call failures produce prototype zero, but interrupt, timeout and unsupported errors propagate. Restore prototype mode on success and error. A stencil runs as Rank over ↕'s windows, and reads its specification through ↕'s WindowSpec (primitive/structural.rs).
Rank tries whole before each_pair. whole calls the operand once where neither argument has a frame. It then tries decomposed, for •decompose⍤2 on a stack of matrices, which factors the whole stack through rustymath's linalg::Stacked, as •decompose factors one matrix, and whole_cells for a primitive or a product. whole_cells holds Rank's routes for whole arguments, which batched calls share (batch::whole_call). In order, they are primitive::stacked, which runs ⌹ and -.× on stacks of inexact matrices through rustymath's linalg::Stacked, as rustymath's Python API runs them, and every f.g that rustymath has a product kernel for, on compact stacks of any item kind, through linalg::Product (an exact product that doesn't fit 64 bits keeps the call for each cell, which computes exactly), one padded call of a pervasive primitive (padded_call), primitive_cells for a primitive with one argument or with a left argument that every cell shares, and paired_cells for a primitive whose arguments' frames match or whose right argument every cell shares. An argument with no axes besides its cells' is the same in every cell. After whole_cells, whole tries pervasive_cells, for a pervasive primitive under nested Rank, and then, for a monadic operand, cell_rule, which also takes folds, atops and a bound left argument. primitive_cells runs a primitive on the whole argument along the cells' axes. A form that extends runs whole at a cell rank at least its own. Other forms follow the cells fact (CellRule) in their row of Primitive::info(). With it, ⊖ , ↑ ↓ ⌷ # and rotate-first call their axis forms, ⍉ sends each cell's axes after the frame's, ≢ and ⍴ give constants, a reshape that keeps a cell's items in order is a view, and ⊂ encloses each cell of rank 1 or more through its axis form, whose cells are views of the argument's items. Any other form with a left argument that every cell shares takes paired_cells. paired_cells joins each pair of cells for , and ⍪, and calls a form that extends once where both cells are at least its ranks. padded_call and paired_cells leave an argument with a named frame axis to the path through cells, which pairs frames by name (named_frame). Names on cell axes pair in their whole calls as calls on cells pair them. It first repeats an argument that every cell shares over the frame (agreement::spread). A pervasive primitive folds or scans a simple array along the cell's axis (folded_cells). f/∘, folds all the cell axes at once. An atop applies its inner function's rule, then Rank for its outer function. Each rule applies only where Rank's mix of the cells' results gives what the whole-array call gives. Other nested results, and folds by other functions, keep the call for each cell.
Axis nodes select/move axes and reuse Rank when the function has no direct axis meaning. Vector-axis reduction uses the general composed-ravel rule, preserving axis order and the whole seed. Single-axis reductions retain direct kernels. Layout-only changes share storage. Structural results retain renderers where their rank permits; scans and in-place changes copy complete layouts. Match ignores renderers.
Inverse nodes propagate fixed-argument sides through composition. f⁻¹←g and f∂←g replace the binding with a Ruled node, which holds the function with its hand-written inverse and derivative. Calling it calls the function. ⁻¹ and ∂ use the rules, and Under counts an attached inverse as keeping every item. Candidate-building primitive inverses validate by applying the primitive again. Inverse scans share forward scan's seed/axis validation. Agenda calls its selector once, then passes the original arguments to the selected branch. Computed operands for Rank, Axis, Stencil and Agenda use shared operand evaluation. Python constructs these nodes directly, not by generating source.
The form rewriter (eval/rewrite.rs) is the one home of fast combinations. Each rule is a row of BPL text, a pattern over ⍵ and ⍺, beside the hidden Special function that computes it, as ⍟1+⍵ beside Log1p. rules adds the selections, one for each order and pick. The rewriter reads a function and an expression as one tree of calls (Form), and replaces each subtree that a rule matches with a call of the rule's function. Function::new rewrites a function's monadic and dyadic forms once and keeps the results (rewritten). dispatch calls them in the function's place, while inverses, Under and display read the original node. Binder::reduced rewrites each list of nodes after joined and members_with. Every spelling, expressions included, takes the combination from its first call. A rule's functions must be glyphs and its constants literals: a name that holds 1 or ⍟ isn't recognised. combined runs the structural replacements (selections, ranks, Key's counts and folds, the determinant and the permanent) before any running level. Where an algorithm doesn't suit the arguments, or a running level carries one, combined calls the functions that the combination replaces, which the level then sees. calls_tines builds a train whose tines may_combine accepts. Function::new then rewrites it. Each combination has a row under "Fast combinations" on its glyph pages. tests/explain.rs checks through •explain that every documented spelling of a structural combination takes it. Reference cases in core.bpl check that each gives what its functions give in turn. The coverage matrix runs each rule whose function maps its argument (mapping_patterns). It respells any form's own case that a rule would capture (rewrites). A combination chooses an algorithm, which stays in rustymath or the kernel modules. A whole-array path that follows from an operator's own definition, such as Rank's for pervasive operands, belongs with the operator.
number.rs owns scalar kinds, promotion and exactness, and the suffix table that spells numbers (e/ₑ, j/ⱼ, r/ᵣ, ₓ and the width marks), which the lexer, Number::parse and display read. Integers and BigRational share an exact domain. Overflowing scalar integer arithmetic widens to exact rationals; whole rational results narrow back when possible. Booleans are a distinct kind. Arithmetic reads them as 0/1; logical operations preserve them. Match/search/grade compare their numeric value, while reference checks also distinguish representation and exactness.
Joining compact arrays widens Boolean -> integer -> float -> complex, and float widths join at the wider width. Item-by-item construction preserves exact/approximate kinds and stays mixed when needed; only approximate real/complex items widen together. A write into an existing array follows the items rule (Widening::Items), as a literal of the same items would: exact and approximate numbers stay apart, and so do Booleans and other numbers. A rearrangement or combination of mixed storage finishes in Gather::finish, which holds the items compact where one dtype holds them under the items rule, as a literal of the same items would be. Its scan compares each item's variant with the first item's, so a copy that stays mixed pays little. An in-place write into mixed storage keeps it mixed, because a scan on every write would make a loop of writes quadratic. Fresh computations choose storage from their result path. Results that an operation computes one cell at a time, as Each, Rank, Key, power histories, and the polynomial and number-theory functions do, assemble as items, through Value::assemble. Mix and catenation join arrays. Keep this distinction between whole-array joining and element assembly.
element::Converts is the one rule that converts a compact item to another numeric type. Size marks, •float, and programs' Convert steps in replays and block stages all use it. A float target reads the item as rustymath's Source reads it. An integer target takes a Boolean as 0 or 1 and wraps an integer to its width. It takes a float's floor within the tolerance of the float's width, then saturates at the target's range. Value::converted converts a plain value. derivative::converted converts a carried value through its carrier (Carrier::cast). On a pending array it adds the conversion as one more pending call (fusion::convert).
Bare numbers, empty literals and NaN use the parsed float width. Top-level evaluate_code first validates the whole syntax, runs leading literal-record •prefs statements, then parses the remainder at the new width. A syntax error prevents header execution. The default width also applies to imported numbers, random data and float results from exact operands. Time/date/metadata values use f64. Formatting marks non-default widths; serialization marks every float for width-independent readback.
Float operations compute at the wider operand width. Exact operands round to the width an operation computes at. f16 operations run on f32 copies. Numeric::halving decides when: for a call with an f16 operand, or any call while the default width is 16 bits. pervasive::planned_halves applies it to whole operations and Number::halves to scalar arithmetic and comparisons. fusion::defer declines those calls. Exact inputs convert once, to f32, and float outputs round back to f16. The scalar Number path rounds each f16 step. Complex parts have f32/f64 widths; f16 becoming complex uses f32 parts. Scalar Complex32 arithmetic computes at Complex64 and rounds, while kernels can compute at f32. These paths need not agree in their last bits.
Floats use IEEE NaN, infinity and signed zero. Comparison treats NaN as unequal; match/search treat it as one value, and grade puts it last. Exact arithmetic involving division by zero can produce those float values. Floor/ceiling retain an exact operand beside infinity. Beside NaN the language leaves their result open: NaN or the other argument, varying by platform. Their folds, scans, windows, products and repeated writes give NaN or the maximum or minimum of the items that aren't NaN, never another number, by the rule in rustymath's Folds doc. A loop that folds by them either passes its running result as the right argument, which rustymath::reduce::maximum keeps beside NaN, or folds through rustymath's Max and Min, as Dyad::extremum does. Monadic floor/ceiling/sign turn whole finite results into exact integers, including big integers; nonfinite results remain floats. Use Number::float(value, width) and explicit-width array construction outside number.rs instead of implicit f64 conversion.
Under •prefs ["kinds":"static"], a result's kind follows from its arguments' kinds alone, as meta/kinds.md describes. Numeric carries kinds and int. Until the float pref sets a width, Numeric::set gives new floats the width the kinds give, 32 bits under static kinds and 64 under dynamic kinds (Kinds::float_width). Whether the pref has set one lives in settings::float_set, outside Numeric: every call builds and copies a Numeric, and a Boolean field there cost about 14 instructions on each primitive call and 34 on each dfn call. A pervasive call on arrays plans before it computes. plan::pervasive builds a plan::Pervasive from the operands' kinds (plan::Kind: type, fixed width and weakness) and the settings, without reading items. It holds the settings with the fixed width that the operands give (Numeric::for_kinds), whether 16-bit floats compute in 32 bits, the kind that the call computes in (computes_in, which applies rustymath's DType::common after Numeric::kind), its kernel table (computes) and, under static kinds, every result's kind (Pervasive::result). Eager dispatch, fusion's deferral (Pervasive::fuses) and typed programs read the same plan. Products and folds plan the same way. plan::Product holds g's pervasive plan and the settings under which f folds its results (Product::folds). plan::Fold holds a fold's or scan's settings and its operands' kinds. Eager products, folds and scans read them, as typed programs do. Comparisons, ⌈ ⌊ and bitwise functions compute at their arguments' storage width instead, where their results are the same at every width (pervasive::first_kind). A comparison between integers and a whole float that stands for an integer computes on integers, through Pervasive::computes, as does a residue by such a float or a residue scan from one, whose results then convert to floats (plan::whole_float). Literals, values computed only from literals, and Python's scalar arguments are weak (Number::is_weak). Before a call, Primitive::weakened converts a weak number beside a strong one to the width that rustymath's DType::common_weak gives it, or to a fixed integer's width where it fits. The loops keep the failed results that keeps names: floats' NaNs, and the wrapped integers of objects that report WRAPS. The kernel tables choose float ⌊ ⌈ × and wrapping negation from run.numeric(). Stored::build_at keeps complex results complex. Primitive::number gives every pervasive result on numbers, as well as each step of the fold of numbers. Under static kinds, statically converts it to the kind that the call's plan gives. A static call on ordinary integers that an integer kernel takes runs that kernel at the int width instead, as a call on fixed integers runs it at theirs (Primitive::integer_kernel). Under dynamic kinds, a call on two numbers stays outside plans, because the type of its result depends on their values. Number::wrapped and Value::wrapped wrap an exact integer result to the integer width, the fixed width that Numeric::for_kinds finds among a call's operands, or the int width under static kinds. Under static kinds a strong ordinary integer is a fixed integer of the width it computes at, the int width joined with its storage width (plan::Kind::stored). Every static call on integer arrays then computes under a fixed width. Weak integers have no fixed width. NEP 50 decides theirs, as before. Display marks a fixed integer of the int width with plain ₓ (Mark::integer). A fixed kernel that fails where a result doesn't fit, as lcm and binomial coefficients can, gives the exact result wrapped (Primitive::fixed_number). statically uses them, and so do the functions that compute exactly, such as ⊥, ⌻ and the determinant. Where integers wrap, a function whose results are approximate, such as ⌹ or polar parts, computes integers as floats of the width that their division gives (plan::Kind::approximate). ∂ converts the integers that wrap in its argument to those floats before it runs its function (derivative::differentiate). Number::parse keeps complex literals and checks integer literals against the int width.
Settings live once for the whole process, in settings.rs, and are read from there, not passed through functions, except where a read is slow and on a hot path. Each setting that calls and scalar operations read is an atomic: the numeric settings, the hanzi aliases, the batch pref, the verifier flag, the fusion report flag, fusion's sizes, which settings::set_sizes sets, and the compile setting. The display settings sit behind one lock. •prefs writes them, the lexer writes a source's leading •prefs header as it reads it, and settings::reset restores the defaults, as ]clear and the test harnesses do. A module that •load runs changes the settings for the whole process. Nothing restores them when it returns. Rustymath holds the thread count, the one setting that it reads itself, and •prefs writes it there. Each call reads the store once where it starts. Pervasive calls, folds and products read it as they plan (Primitive::pervasive_mapped). A dfn call reads it as it looks for its program (program::call). So do Domain::of and the lexer. Values derived from the settings within a call stay explicit arguments. A call's plan holds the settings it computes under. A pending array computes under the plans that its calls made, whatever settings are in force when it computes. A pending conversion holds the settings that give its floor's tolerance (Step::Convert's numeric). A dfn's programs are found by the settings they were traced under (program::Key). Under other settings, the dfn traces a new program. The loop structs keep the settings because their closures run on rustymath's worker threads. Number's tolerance methods and Primitive::number take them because they run once per item. The parser's leaves take a width marker's width, and matches_within takes ⍣∞'s convergence tolerance. Imports build complex numbers with Number::complex_in_session: Python's _Session.array and the JSON protocol's request. JavaScript values hold no complex numbers.
Numeric::mathish_at(width) owns Mathish eligibility: the preference must be on, kinds dynamic and the calculation ordinary exact, f64 or complex128. Fixed-integer operations are excluded even when computing at f64. Kernels select their numerical objects before the loop. Mathish comparison tolerance is 1E-14. f32/f16 compare exactly. Fixed-point convergence uses four ulps at those widths and 1E-14 at f64, independently of Mathish. Index/count/Boolean/character-offset conversion uses near_integer with the fixed default tolerance. The numerical objects come from rustymath. Use independent inside/outside-tolerance tests with exact/mixed controls rather than language comparison as its own oracle.
search.rs selects compact exact or tolerant backends. Classification of 1,024 items or more first groups items that share one array, before it reads any (search::shared_classes). Imports make repeated texts share: keyed::Texts makes each distinct text once, for each •csv column, each •json document and each Python conversion, up to 65,536 distinct texts. Exact integer/Boolean/character storage uses keys; float searches use their width. Integer haystacks accept the whole-number shortcut only for exactly whole float needles. Tolerant indexing validates whole-key matches unless the haystack is exactly whole and falls back to buckets on mismatch. Classification intentionally chains tolerance through sorted runs, so it need not match every member to one representative. Other cells use exact hashes or first-number buckets with Value::matches confirmation, falling back to pairwise comparison where indexing is unsuitable. Grade is stable. Position buffers use u32 and become integer storage without another pass. ⍷ and monadic Key take each class's positions from search::class_positions, which keeps a class's own buffer for 32-bit positions and copies narrower ones once from the shared buffer. Dyadic Key selects rows through Groups::each slices. Key's group-fold idiom folds through search::numbered's class numbers. Where fold_groups declines, Key builds the positions from those numbers. Key with the operand ≢⊢, which ≢∘⊢ also builds, counts its classes through search::class_counts and builds no positions. Under Rank and in batched maps, ⍳, ∊ and ⍸ whose haystack differs from cell to cell search each cell's own haystack in one call on the whole arguments (primitive::searched_cells).
Domain and with_field! select working reals, Complex64 or BigRational for generic numeric algorithms. Compact storage selects the domain without scanning its items. number::read converts once; Field::array builds compact float/complex results without a Number per item. Each caller lists the domains it supports. Compact complex input does not take a real path merely because all imaginary parts are zero; results built through Value::complex can normalize to float storage.
Rustymath owns the mathematical algorithms. Basedpl owns pervasion, storage, numeric promotion, axes and language policy. Code whose inputs and outputs are only buffers, shapes and plain numbers belongs in rustymath, even when only basedpl calls it. Approximate matrix solve/pseudoinverse and decomposition use rustymath factorizations. On macOS, rustymath sends much of this work to Accelerate, as its DEV.md describes. No platform-specific adapter belongs here. The Cargo configuration, for Cargo's own builds, and pyproject.toml, for the wheels, both set the macOS deployment minimum to 15. Exact solve/pseudoinverse/determinant use rational field algorithms. Generic permanent and polynomial helpers also run at floating/complex field types. Singular/wide matrix division gives least-norm solutions. LU eligibility and SVD cutoffs are rustymath policy, not duplicated here.
Hash maps and sets come from rustymath::hashing, as rustymath's DEV.md describes. Use its HashMap, HashSet and Hashing rather than another hasher, so every lookup in both projects inlines its hash the same way.
•decompose/•decompose⁻¹ share domain reading and Factor dispatch, with options for kind, vectors and complete QR. Polynomial adapters choose compact evaluation or generic field operations; gradients differentiate coefficients before evaluating. Number-theory adapters supply cancellation checks and assemble result cells. Distribution constructors expose native density/CDF/quantile/sample functions. Compact continuous calls use slice objects. Discrete samples and finite discrete quantiles are exact integers.
•rand seed makes a ChaCha8 stream behind a mutex (distribution::Stream), which every copy of its record shares. ¿ and a distribution's sample without a generator draw from the session's stream, which •prefs ["seed":n] reseeds. Draws take a stream's numbers in the order they run. The algorithm is fixed, but range/distribution sampler changes can change draws; the reference sequence detects that. Sampling from a generator record uses its roll function to recover the same stream.
A dispatch selects a primitive before its loop, and a pervasive call's plan picks its kernel table (plan::Pervasive::computes). Concrete Kernels tables call rustymath objects: many for slices and one/checked for scalar Numbers. at_widths! writes f32/f64 tables once. Arithmetic, circle-code conventions and scalar policy remain in the numeric modules rather than a second evaluator.
Arguments of mixed item types run through Converting where the object runs item by item in rustymath's own loop (Results::INLINE), which converts each item as it reads it. Other objects use Chunked, which reads each argument through rustymath's Side, converting a run at a time before calling one homogeneous implementation. Sharing those kernels avoids a storage-type cross-product of machine code. Compile only operand pairings that can occur. Same-type arguments can reach rustymath in one call; other shapes expand reused chunks or use block mappings to avoid per-item division.
Custom elementwise closures use rustymath::pairs and its uninitialized append path. Function objects use their own slice forms. Keep per-item bodies inlinable across codegen units; passing a closure by reference can introduce an uninlined Fn shim. Pass it by value, or wrap an existing reference in a new closure. Hot loops use slices and fixed chunks rather than iterator helpers that introduce calls in the incremental build. Avoid captured mutable flags that live in memory and block vectorization.
A failed compact kernel leaves placeholder results and triggers a wider/exact/complex or scalar path for the whole call. Complex fallback is shared with Number policy. Overflow detection belongs to rustymath: the integer forms of its arithmetic objects, its checked reductions and its checked products. Cheap integer multiplication uses bounds to select an exact width without a per-item overflow check. Add no blanket NaN/signed-zero canonicalization pass; algorithm-specific failure checks and rustymath's SIMD kernels remain part of their algorithms.
fusion.rs defers the calls that a block stage can run (plan::Pervasive::fuses): Primitive::pervasive_mapped gives a pending array (Data::Node holding a fusion::Pending) in place of the result. A pending call holds a typed Program of its one step, whose plan is fixed when the call is made, and holds the arrays and atoms that it reads, each pending array in a value of its own (fusion::held). Pending arrays survive assignment and link to the pending arrays they read. Reading one forces it through ArrayData::storage, which gathers into its program the programs of the pending arrays that it reads in order (Pending::gathered) and runs the result through the units of its plan. program::planned plans each structure of program once, from a copy with inputs in place of its held arrays, and keeps the plan for later forces of the same chain on new arrays. Both builders of programs, fusion's and a dfn's trace, add each node through program::add, which gives an earlier node that makes the same value instead: a held atom of the same identity, the same held array, or the same step on the same values. A pending array that the program reads in order and that a name or another array also holds joins as an output (Program::outputs): the program computes it in the same pass, and a block stage stores its items as it stores the result. Each other pending array that the program reads but doesn't compute as a step is computed first, on its own (Pending::gathered): one that it can't read in order, and a copy that something else holds. A fold of a pending call's result that another array also holds stores the result as an output of the fold's pass, once for all its readers (fusion::fold). A pending copy folds the same way whether or not another array holds it. A block stage runs the program's elementwise steps block by block, in parts (blocks::Part): each run of steps that lower to machine operations giving their kernels' results exactly (blocks::lowered_op) through one compiled kernel (jit.rs), and the other steps through the same pervasive kernels (pervasive::node). A part whose kernel can't take a run's items, such as an input that a broadcast reads as one item, runs its steps on those kernels for that run. A stage compiles all its parts or none. Its compile policy (blocks::Blocks::compiles) compiles them where the same kernel compiled or ran before, or where the stage has at least COMPILES_AT items. settings::Compiles can compile every stage or none instead, for the reference harness, and for timings through BASEDPL_COMPILE. Timings can also set the fewest items of a call that waits, MIN_ITEMS, through BASEDPL_MIN_ITEMS. Readers that need only the shape, kind or prototype, such as Value::len and Value::environment, answer without forcing. Value::force forces a result, as timing does. Each step writes a block to a buffer that later steps reuse once its readers have run. A comparison that only later steps of a stage read writes 1 and 0 as floats there (Dyad::compared, rustymath's tolerant::Indicator). ⍚ and @[p] run cases that can't fail on every item and select their results by a Boolean mask (selected in eval/operators.rs). plan::select gives the select's type and the maps through which its mask and cases read. fusion::select defers the select where the mask is such a comparison, and otherwise runs it at once through rustymath's chosen. A stage of one step runs only where it reads through a copy, and records no fused row. Node-making code stays out of line from dispatch, because inlined code grows the frame that stays on the stack at every recursion level. The module doc owns eligibility rules and limits; don't copy its thresholds here. A structural copy of one source through rustymath Steps (remapped) whose result has at least MIN_ITEMS items gives a pending copy (fusion::copy) for any compact kind. Its map is a Mapping::Steps with one entry for each result axis. Broadcasting's maps use the same variant. Forcing composes a read of a copy with the copy's map, and moves a read with no repeat or fill under the pervasive calls it reaches (fusion::resolved). 2×⌽v+1 then reads v backwards in one program. A read that copies (rustymath's Mapping::copies) moves only under a call that reads one array. A call whose two array arguments copy arrays of one layout through one map runs on those arrays, and the map then copies its result once (fusion::copied_once). A program reads any other array through a Copy step, which a block stage reads through without making the copy. A step reads an array's items in order under another shape through its operand's program::Read, which holds that layout. A block stage shrinks its blocks where its buffers wouldn't fit in the cache. Rustymath's agreement::expanded reads a forward run in place, and copies anything else through gather::walk_plan. A Mapping::Steps keeps the plan that Steps::simplify makes on its first walk (agreement::Route), and every later block's read walks that plan. A copy that no call reads through is walked as remapped walks it. Fused and ordinary folds share rustymath's reduce::along/fold_into traversal. A block stage of one compiled part folds the last step's items itself, block by block, and hands reduce::fold_pieces each block's total (reduce::Part): sums, float products, and maxima and minima. A float fold keeps as many running results as rustymath's folds keep (reduce::running). Boolean folds, float folds by ⌈ and ⌊, and blocks that cross a lane go through the block that the last step writes. On the pending route, a fold of a copy of windows along its own axis slides over the copy's source instead (fusion::slide, through rustymath's structure::Slide and reduce::sliding), and a fold along two window axes folds both in one pass (fusion::fold_windows). A short fold that a stage folds as one step for each window position (blocks::short_window) runs as a block stage that reads the copy's source at each position through the copies' maps: in a dfn's program, and on the pending route where the copy reads another pending copy, such as padding, into a result whose last axis is longer than DIRECT (fusion::staged). Under ∂, fold_rule takes a maximum's or minimum's positions in such windows from the source (fusion::picks). Under a trace, a program step (Step::Picks) gives each lane's pick, through the function that gives the untraced mask (derivative::chosen). A traced scan, or picks from sliding windows, by ⌈ or ⌊ stops the trace.
Ordinary pervasive kernels also read pending copies through their maps. fusion::source snapshots the source and mapping without taking the pending work, and rustymath composes that mapping with the call's agreement. pervasive::Input keeps the snapshot alive while kernels read its storage through existing blocked readers. Folds by rustymath's reductions read a pending copy through fusion::source too (pervasive::mapped_fold). reduce::along_mapped folds a copy that reads a contiguous block of its source with its axes in another order, as Mapping::block finds it, in the block's own order. Transposes, reversals and windows that don't overlap read such blocks. Where the totals come out in another order, pervasive::fold makes them a copy through the tables that along_mapped gives, and the next fold reads that copy as a block again. Any other copy folds a run at a time where it reads runs of at least reduce::RUN items, with no copy, and copies a piece at a time otherwise. A copy that a name or another array holds, read in shorter runs (reduce::rereads_cheaply), folds instead from the copy that native_fold computes, which keeps its items for later reads (fusion::held_copy). A read of a few items from a pending array goes through fusion::narrow, which narrows the pending work to the positions read without consuming it. A chain of copies becomes a read of its source, for any item kind. An elementwise call, or the program that a fold left, becomes one over only those positions on its first nonempty partial read, regardless of sharing, when the read takes at most 1 in 128 of its items. Where maps don't compose, as a rotation's doesn't with an overtake when it carries source items across the end, such a read lists its items' offsets. The original array retains its work; the next actual read computes and caches its whole result. Copy unwrapping does not count separately, nor do empty or all-missing reads. fusion::item serves single items and picks, and Gather::walk serves small copies and gathers through fusion::walk_through. Checks of a whole input's values, such as bounds and finiteness, read only the items that its logical ArrayView selects. Unsupported compositions and dense readers retain the pending array's one cached compact result.
explain.rs holds the events that •explain reports. One check in Function::reported serves both •prefs ["fusion":$t] and •explain: a call that computes a pending argument writes the pref's line and records a forced event. Blocks::run records a fused group for each block stage of two or more steps, its fold included. Idiom::call records each fast combination taken, by its Idiom::route. linear::scatter_add, and the transpose of a gather that scatters a source's items, record each accumulation. Selection::write_into, where assignment and Under write, records each write that leaves a compact array mixed. •explain and Python rm.explain use the rustymath::events capture implementation. Each native module captures its worker threads through one process-wide sink. Captures can nest on one thread. The shared Route declaration supplies fast, decline and fixed itemwise IDs and their coverage inventory. Successful routes aggregate call and item counts. Optional declines remain separate from stopped traces, which record a slow fallback on every execution, including cached failures. Outside capture, a native event checks one atomic flag. Each record node and pending array holds an explain::Live guard, which counts it for the thread that made it until it is dropped, on any thread. •memory reports those counts, and Value::footprint walks a value's allocations for its bytes.
derivative.rs implements ∂. A dual is a tag, a primal and a tangent: a transform::Carrier, held in Value::Carried for a number and in Data::Carried for an array, whose primal and tangent can be pending. A record of linear calls is the other kind of carrier. Routing (transform::call) hands a call to the newest level among its arguments, as Value::newest finds it. It reads their outer layer, but a newer level inside an argument held in mixed storage wins, as an array of dual atoms beside an older dual shows. A call whose newest level is a ∂ call goes to derivative::call, which reads how to run it from the form's facts in Primitive::info() (primitive::Facts). It runs a pervasive form's partial derivatives as primitive calls, from the recipe of the function object that the kernel table chooses for the call (partials, pervasive::Dyad::differentiable, rustymath::derivative), applies its own rule to ⌻, and calls forms whose results carry no tangent on the primals. Structural glyphs, Under and selective assignment need no rules. Their own code moves each part of a carried array through the carrier's moves: Carrier::view, Carrier::at, Carrier::copied, and transform::written for writes, which a record answers through its ids. Gather copies a dual's primal and tangent together through derivative::Join. A glyph that copies its right argument through one plan, and a broadcast, copy a record tangent as one Op::Copy (primitive::copied, and Form::Moves's planned). It holds the plan's Steps as a rustymath Mapping (Mapping::walk), the map type that a pending copy and a program's Step::Copy also hold. Its transpose reads through linear::transposed, built from rustymath's Walk and Steps: the inverse map for a copy that reads each item at most once, and a correlation for windows that move by 1. A copy of a copy transposes in one read where the composed copy still transposes by reads (linear::composed). Its maps compose through rustymath's Mapping::through, as fusion composes a read of a pending copy and a trace composes a copy of a copy. The record keeps the chain, because recognising windows needs each copy's own shape. A sum along a window's axis transposes as a sliding sum (linear::slide), and a spread axis as a sum along it. Windows that move by more than 1 transpose by rustymath's signal::overlap_add, which adds each window back where it read (linear::strided_back). Other copies, and copies of several sources, keep ids and scatter. •explain lists each scatter as an accumulation. In reverse mode, the rule for a ⌈ or ⌊ fold is a sum of the tangent times a mask of each lane's chosen item. Its transpose reads. Mix of a dual whose primal is nested mixes the primal and the tangent apart. fold and scans with +, ×, ⌈ and ⌊, +.× and outer products have rules, and kernels that read numbers directly leave active arguments to their general paths. Folds by primitives, products of primitives, ⌹, ⊥ and stacks of matrices route through transform::call as operations (primitive::Operation), so their rules (derivative::solve, decode, product_dual, determinant and fold_dual) run on a &Context and call Operation::call for the products, folds and inverses that they need. Rank's routes for whole arguments send stacks that a level carries through Pairing::Matrices, so ⌹⍤2, +.×⍤2 and -.×⍤2 take their stacked rules in one call, with the batch pref on or off. Attached rules, system functions, and folds and products by functions that aren't primitives route through the same transform::call, as transform::Op's other variants. The rules for ⌹ and -.× keep the glyph's own result as the primal, so that a function computes the same values with or without ∂. Their tangents and transposes multiply by R, the inverse that numeric::matrix_inverse takes from one factorization through rustymath's Stacked::inverse, or by exact row reduction for exact matrices. regular_inverse gives UNSUPPORTED where rustymath's regularity test finds the matrix singular. With only B active, R is ⌹A, a pseudoinverse included. With one number in x, f∂ x runs f once with x seeded along it. With more, linearized pushes a record level just below its ∂ call's level, then runs f once with each part of x seeded with a record at that level, from linear.rs: a tangent whose items are never computed, which the same rules extend into a graph of linear calls. A call whose newest level is a record level records the linear map that its form's facts name (linear::call). reverse then reads the Jacobian from the record, by a transpose for each result item or a forward evaluation for each number of x, whichever are fewer. A function with a JVP rule that f∂←g attached calls the rule with ⍺ a dual of zeros whose tangent is the argument's, so in reverse mode the rule extends the record too (derivative::attached). A call of such a function with two arguments gives DOMAIN ERROR wherever ∂ needs a derivative through it (derivative::two_arguments). Code that reads a dual array's storage gets its items as dual atoms, which is correct at per-item speed. A rule packs an array that holds its level's duals, at any depth, into one dual before it splits the array (derivative::packed). A reader outside a call sees a value through transform.rs's read point. Value::observed, which counts, positions, ranks, selectors, ≡ and guards' predicates use, reads a dual's primal. The integer readers of Items read carried items through it (Value::observed_number), so a count or position that ∂ carries reads its primal. It returns a Result whose error, transform::Stop, names the level that owns a batched run. A stop is ErrorKind::Stop, which no guard catches, even one that names it, and which a handler that catches every error lets pass (ErrorKind::is_control). Display reads through Value::displayed, because it never runs inside a batched run. Value::computed, which kernels use through primitive::numeric, gives UNSUPPORTED for a dual atom. Writes to bindings outside a ∂ call are recorded and stripped when it returns.
program.rs and program/ hold the typed programs that pending work and ∂ run. ∂ of a dfn runs from one program that holds its whole call, its value and its derivative (derivative::differentiate). The first call with given argument types, layouts and settings traces the call (program/trace.rs): the dfn runs on traced values, which carry a type and a layout but no items, and each call on them plans its result (plan) and adds a step through program::add. A trace that meets an effect, a read of a value, or a call that no plan covers stops, and the call runs in the interpreter. Programs keeps each trace's outcome, a program or a stop, on a dfn's Definition, or on the structure of any other function (program::Tacit, which keeps STRUCTURES of them). An outcome applies to a later call with the same argument signature whose free values still hold what the trace read (Entry::reread), so free values that cycle replay. A signature whose free values change on every call stops tracing after CHANGES traces in a row, and a function keeps at most KEPT outcomes. A replay runs the program on the call's values (program::call). program/run.rs replays a program one step at a time, as the reference that compiled code matches. Fusion's programs and traced programs pass through the same code. For index notation and its gradients, a program also holds steps that select by positions it computes (Step::Gather), write or accumulate at the places that a map or a selector gives (Step::At), add a windows copy's cotangent back to its source (Step::OverlapAdd) and mark the item that a ⌈ or ⌊ fold picks in each lane (Step::Picks). Under index notation's level 2, rustymath's selection planning (structure::joined) reads an index array whose positions are affine through strides instead of a table of offsets (Table::affine). program/contraction.rs runs a + fold of a × whose operands read their sources through affine maps as one product (contracted), and a fold of a copy that repeats items as a fold of the copy's source (fold_of_copy). program/layout.rs stores a program's maps in the storage order that their operands and readers need (layout::ordered). blocks::plan finds the runs of elementwise steps that block stages compute. program/verify.rs checks each program's contracts before it runs, in every reference case and page (settings::verifies). •explain lists the programs that a call built or replayed and the traces that stopped (programs, declined).
batch.rs makes a map's batched call. batch::tried, which each_pair calls, begins a batched level for the run (transform::Kind::Batched), wraps each argument that has a frame in a Batch carrier at that level, calls the operand once, and ends the level. Each map first lays its arguments over its frame through its own agreement, the pairing that its calls on each cell read (batch::Arguments::laid and agreement::laid_cells). each_pair makes no batched call for one cell of atoms, which runs faster without one. Routing hands each call whose newest level is the run's to batch::call: a primitive, an operation that primitives name, such as a fold by a primitive or +.×, or an operation by a dfn. It leaves ⍕ and random draws to their own code. It runs any other system function once for each cell, on that cell's arguments, as the function runs once for each item of an argument that holds several, and joins the cells' results into one batch (batch::system, batch::each_cell). batch::call runs the call on the whole arguments by Rank's routes: eval::whole_cells through batch::whole_call, and eval::folded_cells for a fold. A primitive therefore batches wherever Rank runs it on whole arguments. A fold or scan that folded_cells doesn't cover, such as one by a dfn, runs the fold's own loop over the batch's items. Each stage is then one call, on item j of every cell (Batch::at). fold_array and primitive_fold join a result that holds stage batches into one batch (joined). Every fold and scan route decides its structural cases, such as a unit, empty lanes and the result's shape, through FoldPlan, and ends in folded, which alone gives the result its layout, fused folds included. A fold or scan by any function other than a primitive, over more than one lane, also tries a batched run of its own: operators::by_lanes moves the fold axis last and runs the derived fold over the lanes through batch::tried. operators::by_triples does the same for f.g where g isn't a primitive: one batched call of g on a frame of rows by contraction by columns, then the fold by f along the contraction, for each chunk of rows whose triples hold at most batch::CHUNK items (operators::by_rows, which products of primitives share). Each runs its usual loop from the start where its try gives None. A fold keeps its stops on its operand's Definition (Function::remembers). batch::tries holds the conditions for any try. A call that no route covers stops the run (transform::Stop). A call whose constants with axes, repeated over the frame, would hold more than 4 times the items of its arguments and more than batch::CHUNK items, 2²¹, stops the run with the cells of a chunk whose share holds at most CHUNK items (oversized, Stop::chunk). tried then runs the operand once on each chunk of that many cells (batch::chunked), each as a batched level on a frame of one axis, and joins the chunks' results as the run over cells assembles its cells' results. A chunk that stops for too many items again starts the chunks over with its own smaller chunk. Where one cell's share alone passes CHUNK, the map calls its operand once for each cell. Code that must run once for each cell asks the levels first (Levels::effect): output, input, ⍕ of a carried value, and the code in a system function that acts (Effect::Acts, through Context::acts), a guard or a call on prototypes that would catch an error (Effect::Catches), and a write to a binding outside the map's frames (Effect::Writes, through Levels::record_write). A read through Value::observed, as a count or a position makes, stops the run too. A guard whose predicate is a batch of the run splits the run instead (Session::guard through operators::guard). batch::truths gives each cell's Boolean, and each side runs once, as a guard case on its own cells (Levels::case), through the Cells that ⍚ uses. Each batched binding in the frames that the run started is read at the side's cells (Session::restricted, through batch::read_cells and operators::cells_at), and Cells::assembled joins the sides' results. During a case, the run's frame is the case's cells, a random draw draws for the case's cells, and a write to an enclosing frame stops the run. A map inside the run widens the run instead of beginning another, where the run is the innermost level (batch::Entry::Widen). The widening is a case with no definition, whose frame is the run's frame followed by the map's. The run's batches among the map's arguments lay over that frame (batch::laid_over), and a binding made before the widening lays over it when code reads it (Session::read_binding). A widened run that stops, or an Each whose results are arrays, falls back to calling the map's operand once for each of its cells inside the run. A guard of a dfn that already splits the cells in another frame stops the run as a recursive call (Cause::Recursion), and sides whose cells differ in layout or kind stop it as a call with no rule. Each stop carries its cause (transform::Cause), which says how long a dfn operand remembers it (Cause::kept). An effect, a write, an observing read or a recursive call lasts for every later call. A call that no route covers lasts while later calls repeat the arguments' kinds and cell shapes, and the memory guard lasts while they also repeat the frame's shape. A caught error never lasts, because whether an error arises depends on the cells' values. The dfn's Definition holds what it remembers (Definition::remember): ALWAYS, or a signature of the arguments (batch::signature), each holding the stop's cause (batch::holding). A later try that it covers falls back without calling the operand, with an "an earlier stop" event that names the cause. A stop's own fallbacks event names its cause (Cause::reason), and the glyph of a primitive whose call no rule covers (Stop::refusing). Code with no rule for batches that reads a batch's items gets item carriers, item i of every cell, and records an itemwise row, route "batched items". explain::itemwise is the one helper for routes that handle items one at a time, and the innermost call claims its counts (Function::reported), as forced does. A kernel that then reads a number meets the stop that Carrier::computed gives a batch. A missed reader therefore costs speed or a fallback, never a wrong result. A batched run can also start inside a ∂ call. The run's level is newer than the ∂ tag, so the batch rules take each call first, and their calls on whole arrays reach the derivative rules. A batch's whole can then carry a dual, whose kind batch checks read through Value::template_dtype. Stacks of matrices whose wholes carry a dual take the derivative rules through Pairing::Matrices (eval::whole_cells), because the stacked kernels read compact storage. A ∂ call inside the run differentiates with respect to batches of atoms or of arrays. derivative::numbers counts a cell's numbers from the batch's template, every cell shares each direction's seed, and the Jacobian joins into one batch (batch::joined). A move with no plan on a dual whose primal is a batch runs on the primal and the tangent apart (derivative::apart). Adjoints that a transpose adds into a batch add as one sum on its whole (batch::scatter_add). An attached rule runs on the whole batch at its argument's ∂ tag (batch::call), and batch::split_whole and batch::dual_of_wholes keep the newest level outermost. A roll draws once, on its argument laid over the frame (distribution::drawn_items), and deals and samples draw for each cell in turn (distribution::each_cell), all from one stream in cell order. Where each cell draws once, seeded draws match a run cell by cell, except under a guard that splits the cells, where each side draws for its own cells in turn. A try that falls back puts back the state of each generator it drew from (distribution::settle). The map's result must be a batch at its level, a strand of batches whose kinds join (joined), or an atom or compact array with no carrier, as every cell's result. Any other result falls back. A pervasive call stops where its result's kind differs from the kind that one route gives every item (pervasive::expected). Each cell then keeps the kind that a run cell by cell gives it. reference::check runs each reference case with the batch pref off, then on, then on under fusion's small sizes (fusion::Sizes::SMALL), and compares the storage kinds of the later runs' values with the first's.
Rustymath owns numeric folds, scans, adjacent pairs, grouped reductions and integer bounds. Basedpl selects kernels, tolerance, seeds, storage widths and exact fallbacks. Which reduction folds each primitive comes from rustymath's facts! table, through pervasive::folds and truth_fold, and so do the object by whose rule each takes characters, the comparison objects that with_comparison! makes, and the product kernel that each pair f.g names (pervasive::known_product). Folds by algebraic float operations can regroup. Contiguous subtraction/division folds use Alternating; quotients fall back to item-by-item folding if their products leave the normal range. Boolean scans compose truth-table maps; comparison scans compute both possible totals in bulk and use reduce::chosen. Seeds, axis lanes and scalar fallback preserve the language's accumulation direction. Generic reductions stay right-associated and preserve operand call order/effects. Monadic f.g is the exception: it memoizes each minor's expansion, and a cached minor does not repeat operand calls or their effects. Determinant/permanent fast paths use the appropriate rustymath backend.
The automatic Rayon scheduler serves rustymath elementwise many and pairs, broadcasts through agreement::blocks, folds through reduce::along, block stages and matrix products. Scans don't split. Decompositions split in the products of their blocked updates, the passes of their Hermitian and bidiagonal reductions, and their divide-and-conquer merges. rank sends dyadic +.× and ⌹, monadic ⌹ and -.×, and •decompose over the 2-cells of an unkeyed stack with compact storage to primitive::stacked and stacked_decompositions. They run each cell through the rustymath function it uses alone. A pervasive operand, or Rank nested down to one, runs as one call: each argument gets length-1 axes after each level's frame, so leading-axis agreement pairs frames with frames and cells with cells. An argument with a named frame axis goes through a batched map instead, whose frame carries no names, so names pair only between cells (batch::tried). Rustymath's parallel::stack splits whole cells across threads in rounds, with rustymath's interrupt check between rounds. A caller whose checks run far apart calls check_now, because Context::check reads the clock only on every 64th call. The scheduler adapts to call cost, uses RAYON_NUM_THREADS by default and accepts one override for the whole process through parallel::set_threads (0 default, 1 serial), which •prefs ["threads":n] writes directly. The timing harness sets one thread after each reset of the settings. Nested Rayon work does not split again; wasm is serial. A block stage splits in pieces of whole blocks: an elementwise stage through parallel::extended, and a folded one through reduce::fold_pieces. On one thread an elementwise stage appends straight to its result, because copying each block into a piece's slots cost up to 1.2 copies there. Small converted chunks limit the work available to the scheduler. Don't infer multicore execution from a large array alone.
A pair that rustymath has a product kernel for, such as +.×, ⌈.+, ∨.∧ or +.=, runs through linalg::Product::run, which Python's inner also takes (pervasive::known_product). Other native pairs use its bounded inner_with runner and reduce::right_fold, carrying the right tail across blocks. Basedpl supplies its primitive kernels, tolerance and exact-widening fallback. An integer product that doesn't fit 64 bits gives way to the exact path. pervasive::inner takes each operand's Orientation. Its float and complex products read a transposed operand in place, as reverse mode's product_transpose needs. A float +.× whose operand is a pending copy, or a view of one, hands rustymath's Inner::inner_mapped the copy's source and map (fusion::source, pervasive::mapped_product). A +.× whose operands are both pending conversions of compact u8, i16 or f16 arrays to a wider type reads those arrays. Rustymath's kernel converts each item as it packs it (fusion::widened, pervasive::widened_product). A dfn program leaves such a conversion pending for its product (Unit::Pending). Rustymath gives gemm an operand that its map reads at strides, such as a transpose, where it lies (Mapped::matrix). It reads any other copy a block of rows at a time, except in a product with one column, where the copy's rows are adjacent in storage, as for a vector or a single-channel image. It then adds each column of the copy where it lies (linalg::by_columns). Neither builds the copy. A copy that a name holds is read again for each product rather than cached. Reverse mode transposes a product whose operand is windows that move by 1 as one product: the windows of the zero-padded cotangent with the kernel reversed (linear::convolved_back, rustymath's signal::correlation). Structural gather/compress and interval index also reuse rustymath. Measure changes to generic or backend code in both projects. The callers' codegen and layout decide costs that a kernel measured alone misses.
eval_with/call_with install EvalOptions. Echo defaults to true; disabling it suppresses implicit display, not explicit output or display commands. Output is ordered Output { kind, data } MIME bundles. An optional sink receives events as produced; otherwise the evaluation retains them. Values, functions, operators, outputs and errors remain separate results.
Execution::begin puts each evaluation's interrupt handle, deadline and poll in execution::STOPS, a static that every thread reads. Its Running guard restores the stops of any evaluation it ran inside, such as the one that called ⍎. execution::stopped reads them, and the first evaluation installs it as rustymath's interrupt check (rustymath::interrupt). Rustymath's long computations, such as exact matrix functions, primes and stacks of matrices, then stop for basedpl's interrupts and deadlines, on its worker threads as well. A thread without a turn, such as the one Python's in runs on, also stops at an interrupt of the running evaluation.
Evaluation code reads the process's one session through session(), and takes a &Span where it reports errors. Checks at interpreter boundaries and in long loops call execution::check, which counts down a static before it reads clocks and flags or calls the poll hook. It takes no span and no session, and code without a turn, such as Python's in, calls it too. Individual BigInt operations and bounded native kernels are not preempted. The session's state sits in Cells and RefCells whose methods hand out owned values, so no caller holds a borrow of it while a dfn evaluates (meta/session.md). An optional Input supplies ⎕ and •nget "-"; without one they raise IO. Python uses cooperative cancellation. The process worker can kill an unresponsive process after a grace period, losing its state; requests are never replayed.
Session::call resolves and calls a function expression on existing arrays without temporary argument bindings. Retained functions use call_function_with. Function::call gives each error that a function's code raised without a span the call's span. The evaluator gives its own errors the spans of the nodes they report, and a statement gives any error still without a span its own. An error raised outside an evaluation, as by Python's indexing, has no span. Diagnostics display with Unicode width. Python translates spans to character offsets; process JSON keeps bytes. Native function/operator handles cross the in-process Python boundary, not process JSON.
python/basedpl/__init__.py owns the shared bpl workspace and Array/result/error wrappers. An Array whose value holds a function as an item is a FunctionArray, which Array._classify chooses in __init__ and after __setitem__, so callable() is false for every other array. fn aliases bpl.fn; builtins and magics share that workspace. Registering the magics gives it the REPL's display defaults, through Session::make_interactive. Plain calls print explicit output and return Array, Function or Operator values. Capturing calls return Result. Capture mode is positional-only because keywords bind BPL names. _Session.eval and .call are the PyO3 entry points; request is their internal Rust helper.
functions.py constructs native nodes and lazy exports from Rust metadata. Builtins win name collisions with workspace names; operation names choose valence/currying. __all__ omits Python-builtin collisions but leaves explicit imports available. Function calls with keywords use a keyed left argument. A single positional argument stays the right argument unchanged; zero or multiple positional arguments form a vector. Operator properties and _Pending assemble native nodes, never generated BPL source. printing.py reads node parts without evaluating or resolving names.
.fn() retains a late-bound expression. Calls resolve affected nodes against the calling session while sharing unaffected graph nodes. This preserves primitive fast paths, inverses and reduction identities. The binder still resolves ordinary names during execution. Cyclic resolution is a LIMIT error.
Arrays retain native values. .py and .np make copies; NumPy is optional and lazy. The native _Array buffer export lends compact items read-only. An unbuilt structural copy whose read is affine (Value::affine) lends its source's items with strides, so windows reach NumPy without being built. Numeric buffers import with one copy through rustymath_buffer::read_array. Value::imported_buffer preserves supported item widths. The shared reader normalizes signed bytes, wider unsigned integers and foreign byte order. Sibling snapshots carry keys and names. A NumPy uint64 array containing a value above int64's range imports item by item as exact integers. own() moves compact items into a writable NumPy array where nothing else holds them, holding Booleans as u8 and characters as i32, because NumPy can write any bytes. Exact integers/fractions cross through native integers, not decimal strings. String/object arrays convert per item. Python data converts in Rust: the session's array (import in py/src/lib.rs) walks nested lists through rustymath_buffer::walk, the walker rustymath's Python package uses, with a collector that nests a level whose items differ in shape. Lists follow the shared import rule, and dictionaries read as literals do. rustymath_buffer::labels is the one rule for a dict's keys and a frame's column labels, in both Python packages. A frame reads through rustymath_buffer::frame as a keyed vector of its columns, which .df turns back into a frame. None reads as NaN as an item. A bare None doesn't convert, so an operator given one returns NotImplemented. syntax.py wraps the native syntax tree with character spans and span-editing helpers.
Array(existing) creates an independent native handle. Assignment and .keys/.names setters replace the owner's handle without changing prior views or captured functions. Both Python packages read written selectors as structure::IndexPart tokens through rustymath/buffer/src/selection.rs, and plan them with structure::plan_parts, the structural planner that ⌷ also uses. structure::addressed_parts routes source groups and expands Rest against the applicable frame. On a BPL array, Python's own kinds (selection::native: slices, keys, integers and lists of keys) decode in the buffer crate, Boolean masks select where they hold true, and other selectors become BPL values that decode as ⌷ decodes them (Value::planned). Pick paths select alone, as ⌷ reads them, keeping the written parts until the ending cell's geometry is known. _Array.select and _Array.assign use carrier-aware structural selection and writeback. Python assignment uses leading-axis broadcasting. Language writeback retains its own agreement.
Each _Session request waits for its evaluator mutex without the GIL, then evaluates on the caller's thread with the GIL released. A separate interrupt handle supports interruption from another thread. Signal polling briefly takes the GIL. Python input and optional streaming-output callbacks also cross at explicit boundary adapters. No Python numeric objects enter core values; arrays can be dropped on any thread without an unsafe Send implementation.
protocol.rs owns JSON request/value/error/output encoding. Requests are objects with fields such as code, call, args and bindings. Exact integers use arbitrary-precision JSON numbers; fractions and complex values have tags. Output bytes use base64. The worker omits eligible keyed entries holding functions/operators and returns value: null for other non-exportable values, retaining display output. JSON data serialization is a separate contract and can reject such values.
cli/src/worker.rs processes JSON lines in order. A separate reader handles interrupt controls but never evaluates BPL. IDs are optional, and bad requests don't end the session. python/basedpl/worker.py owns process lifetime, deadlines and hard-kill fallback. Protocol serialization walks native values directly rather than building an intermediate JSON value tree for every reply.
CLI -e, files and REPL stream outputs to stdout. Failed writes interrupt execution; a closed pipe exits successfully. -e, files and interactive input provide StandardInput; program text read from stdin does not. The REPL parses for completeness without evaluating twice. Rustyline owns terminal editing/history/modes; only typed backtick sequences auto-expand. Paste/history/navigation cancel automatic expansion. Shared symbol metadata supplies completion/highlighting. Keep pseudo-terminal tests for Ctrl-C/Ctrl-D and paste behaviour.
The Jupyter adapter in cli/src/kernel.rs uses kernmini and a ThreadWorker. It keeps transport/interrupts responsive, maps explicit output to streams and implicit output to results, and routes notebook input through input_request. The wheel and local install use the same kernelspec. inspection.rs embeds the glyph/system pages for help and uses the same names as the REPL and Python. Change names and shortcuts in symbols.rs/shared layout data, not each frontend.
cargo wasm builds an incremental release-profile npm package with wasm-bindgen glue, language/input assets, grammar and canvas support. --out DIR leaves the docs' local playground assets alone; --strip removes function-name metadata for distribution. Keep wasm-bindgen glue and crate versions aligned through the workspace lock. Avoid fat LTO here because rebuild cost and enlarged stack frames conflict with browser development and recursion limits. Liftoff, V8's first compiler, gives each wasm local its own stack slot, whether or not the path that uses it runs. So a function that recursion passes through keeps its general path out of line, as Each, Rank and the outer product do (each_items). For the same reason, ErrorKind, which the Result in every frame holds, stays at 16 bytes in wasm32 (transform::Stop). To find a frame that grew, read the function names in the stack of the RangeError at an overflow, then compare those functions' wasm locals between builds. wasm-objdump -h DIR/basedpl_wasm_bg.wasm lists the module's section sizes, including name. twiggy diff matches functions by full name, including crate hashes that change between builds. To compare two builds by function, compare their twiggy top -n 1000000 -f json listings with the hashes removed.
Browser Session.run streams MIME events and returns a protocol error or null. eval transfers values through js.rs but discards output and cannot export BPL functions/operators; bind imports page values/functions. Typed arrays transfer without another page-worker copy. JavaScript functions live in a thread-local registry and core function nodes hold slots, preserving Send/Sync values. host_web.rs implements the browser's clock, synchronous I/O/fetch and RegExp behaviour. File/URL reads resolve against the configured base. Unsupported native filesystem/locale operations stay unavailable; browser •nput writes only stdout and •js is browser-only.
The playground and gallery load released assets from jsDelivr, or local nbs/playground/ assets on localhost. Shared bpl.js owns editors, worker startup and output. The authored ES modules and font in python/basedpl/browser/ need no evaluator: browser/bpl-editor.js exports grammar/configuration/editor options/font installation and explicit install(). Importing data never mounts the bar. Solveit, the playground and the gallery supply a normal-flow #bpl-toolbar slot; elsewhere the same component overlays the page without padding or height adjustment. One persisted collapse button keeps keyboard input active. install() reuses a connected controller; its dispose() closes it. Python magics call Solveit's window.loadBplInput() when present, otherwise import the same public module from jsDelivr. Restarting a trapped/stopped worker loses its state. worker.js, page.js and gallery.js own their respective protocols/UI; nbs/driver.js documents canvas drawing data. Gallery demos are .bpl sources with shared preferences, checked natively by language_examples. Browser recursion tests cover both V8 compilers and fail with LIMIT before stack exhaustion.
CI's two wasm jobs, in the Test and Speed workflows, take Node's version from .node-version. TurboFan's frame sizes vary between V8 versions by enough to decide whether a recursion test overflows. On 28 October 2026, one build's each shape needed 0.68 MB of stack to reach 171 levels under Node 24's TurboFan, against 0.62 MB under Node 26's. tests/wasm.mjs runs its depth checks with Node's 192 KB reserve plus 80% of the roughly 470 KB that a Chrome worker gives, 0.5575 MB in all. A build whose frames grow into the remaining 20% fails the test. CI's x86-64 V8 takes more stack per level than Apple silicon's. The browser's depth limit, MAX_CALL_DEPTH in src/eval.rs, allows for levels up to 1.48 times their size on Apple silicon. basedpl.bpltests.wasm_stack(DIR) measures the stack that each recursion shape takes per level and below its first level. The limit is set from those figures. In wasm, Binder::reduce and Bindings::enter stay out of line, to keep their locals out of the frames that stay on the stack at every level of recursion. To find a frame that grew, print each function's TurboFan frame size, in "stack slots", with node --no-liftoff --print-wasm-code-function-index=N. wasm_stack runs each trial in a fresh node process whose worker compiles the module, as the test's worker does. Code compiled once and shared across workers gets different frames.
system.rs owns the system table: one SystemFunction for each system function, holding its case-insensitive name, call, valence, extension, result prototype and inverse. Its extension (primitive::Extension) is the field that each primitive's form holds too. A system function takes its right argument whole, or takes items (primitive::Item) and maps over a right argument that holds several, with its left argument whole for each. A function that takes text also maps over each row of a character matrix, as ⍎ does. A function that acts asks the running levels where it acts (Context::acts), as output and input do: where it reads or writes files, the network, the clock or the terminal, or makes something new on each call, as •index and •rand do. Any other system function asks nothing, and a batched run calls it once for each cell. The parser resolves a known system name to its table entry (system::Entry) through Builtin::find, which finds a glyph's primitive too, and primitive::Builtin holds either. A system name is therefore a NodeKind::Function node, compiles to a constant and resolves to the function that one process-wide table holds for each builtin (Builtin::function), as a glyph does. An unknown system name stays a NodeKind::System node, which is a VALUE error when evaluated. Code identifies a table function by the Entry that its SystemFunction records, as in Entry::PREFS, never by its name. A function that a factory made, such as •element "div", has no entry, although it shares its factory's name. Native records use ordinary function nodes and calls. Empty native item mapping returns its declared prototype without calling the function. data::Options is the shared keyed-option decoder; formats don't invent separate option parsers.
optimize.rs implements ⇄ on rustymath::optimize. The left argument holds the options. When the method can use a derivative (none named, L-BFGS, Levenberg–Marquardt, CG or MINRES), one linearisation at the start (derivative::linearized) finds whether ∂ differentiates the function, and gives its value there. Goal evaluates the function at points in the start's layout for minimize. When the search runs L-BFGS, each value comes from a linearisation that gradient then transposes at the same point (linearized_at), so a value and its gradient run the function once. With "solve":$t, Residual is the Mapping that solve takes, or that solve_affine takes with "linear":$t as well. Where ∂ differentiates the function, Residual gives JVPs and vector–Jacobian products from one record at each point, or from the start's record for an affine function. At a later point where ∂ gives an UNSUPPORTED error, Residual::jvp declines, and rustymath takes forward differences for that Jacobian and later ones. A square linear system takes no products with the transpose, so it runs GMRES on A, not the normal equations. The function in option precondition gives the preconditioner.
CSV columns, JSON arrays and Python lists share data::imported: a non-fill finite float or any nonfinite float triggers float promotion when every number converts losslessly. Finite fills alone do not trigger it. Otherwise items keep their kinds. JSON objects and Python dicts read as literals do, through keyed::vector and Value::from_parts. Binary data uses exact bytes. Writes validate before opening and create exclusively unless overwrite is explicit. CSV/JSON/XML exports use •csv⁻¹, •json⁻¹ and •xml⁻¹.
regex.rs shares host regex state between match/replace native functions; hosts retain their own regex syntax. xml.rs represents keyed element trees and distinguishes XML escaping/validation from HTML raw script/style and void-element rules. SVG uses its own renderer. image.rs owns pictures, their options and animation timing, as plain pixel buffers. Its codec module decodes and encodes those buffers through ImageIO on Apple targets (image/imageio.rs) and through the image and png crates elsewhere (image/codecs.rs). Native/browser renderers use the MIME/byte formats their frontends support. plot.rs stores specs and draws SVG on demand; the renderer owns defaults/validation. Rendering options belong in their user pages and implementation comments.
display.rs owns plain/boxed/function display, elision and MIME rendering. Display settings affect implicit presentation, not returned values or ⍕; explicit output prints all items. Display and source text read the items, not their storage: Value::shown_dtype (array/text.rs) gives the dtype that compact storage of the items would take, by storage::items_dtype's rule for items, which the shared ₓ or width marker, plain display's choice between brackets and a strand, the bracketed grid that plain display and ⍕ share (display::table and display::bracketed), a keyed table's corner and the boxed marker all read. •mime calls a renderer with echo disabled and cancellation active. Failed implicit rendering falls back to text; interrupts/timeouts propagate. canvas.rs serializes drawing data and typed bytes; canvas.js and nbs/driver.js own browser playback. Keep formatting, serialization and rich output as separate contracts.
tests/core.rs checks public storage/layout/API behaviour, ownership, parser diagnostics and recovery. Use equiv_in! for session workflows and fails_in for errors; each expected expression runs in a fresh session after its code, under the settings that the code leaves. tests/core.rs never changes settings. The settings are one store for the whole process, so the checks that change them live in tests/settings.rs, which runs them in turn in one test, each from the defaults. Both binaries share the helpers in tests/common/mod.rs. Keep constructors for representation/prototype/buffer checks. Self-contained language cases belong in tests/reference/core.bpl. cli/tests/cli.rs and Python's pseudo-terminal tests exercise the installed process/editor boundary.
reference.rs serves Rust tests and worker requests. Every reference case uses a fresh session; documentation examples share a session within each BPL code block, which starts from the default settings. Each page starts with no kept plans or compiled kernels, as a new process does (program::forget_plans, jit::forget_kernels), so that an example that shows block counts or compiles gives the same output whether its page runs alone, as under check_page, or after other pages, as under language_examples. A reference file's leading •prefs line sets the settings of all its cases. An optional ⍝⍝ setup supplies shared definitions, initialized independently for each run and its expectation. A section's inline •prefs suffix runs before setup. Each case runs five times, with the batch pref off, then on, then on with fusion's small sizes, under which every block stage compiles its parts, then the same with no compiled parts, then on with its code as the body of a dfn that one session calls twice (twice), and each run starts from the defaults and the file's header. Every run checks each program before it runs (program::verify): each step reads values before its own, each read lays out as many items as its node holds, and each node's description is the one that the shared planners give its step on its operands. A program that breaks one of these panics with its listing. The run with no compiled parts takes place only where the run before it used one (blocks::compiled_runs). The last run checks that a dfn's second call, which runs the statements its first call recorded, gives the first call's value, storage kind, error and output. Code that can't be a dfn's body skips it. tests/reference.rs checks the cases in a child process of itself for each core, and Python's check_file in a worker for each core, because a process evaluates one case at a time. batched leaves out the run with the batch pref off, unfused the runs with the small sizes, and compiled the run with no compiled parts, for a case whose value says whether a stage compiled. Each expectation evaluates after its code, under the settings that the code leaves. A float matches its expectation within 1e-13, or within 8 units in the last place of its own width where that's greater, unless the case's header sets a tolerance (reference::same_atom). .bpl files define active coverage. Captured foreign expectations have no exactness; local expectations compare exactness and structural values independently. A local expectation also fixes where storage is mixed: the result must be held in mixed storage exactly where the expectation is, at each level of nesting (kind_difference), unless the case's header carries mixed. Pending gaps belong in inventory JSONL, not passing error tests. See tests/reference/README.md for format and review/activation workflow. python/basedpl/reference.py, bpltests.py and scripts/reference.py own import, capture, inventory and worker checks. lib/*.bpl are readable dfn examples; modules import other library files with private relative names.
tests/scaling.rs measures time/allocation growth and result-size allocation budgets. tests/speed.rs reads native/browser budgets from speed/*.bpl through timing.rs; the worker and Python measurement tools reuse it. Both tests register rustymath's memory guard and turn on its checks (memory::set_checks) in processes that run one case at a time, tests/reference.rs in its children: a reference case that leaves more bytes held in large allocations than it found, on each of two runs, fails as a leak, and so does a speed case whose timed runs leave more than its first run left. A free or reallocation that passes more bytes than its block holds aborts with a backtrace. tests/units.rs reads existing incremental object files and checks unit size without building. Python tests cover conversion/cancellation/frontend behaviour, not a second Rust semantic suite. Combine related checks into readable tests.
nbs/index.ipynb generates the homepage and README. Reference pages in nbs/glyphs/ and nbs/system/ describe forms/options/errors; concept pages explain them. sidebar.yml owns navigation. Edit source notebooks/pages, not generated outputs. magics.ipynb and dyalog.ipynb export Python modules; other Python modules are hand-written. Cargo owns versions. nbs/llms.txt is the documentation's index for LLMs: an entry for each page, with how BPL differs from Dyalog, an example and the page that defines it. src/inspection.rs includes it in the build, so ]help llms shows it after cargo develop, and the project's CLAUDE.md includes it in every session's instructions.
bpl-nb
bpl-nb nbs/index.ipynb --save
nbdev-readme
nbdev-test --save
nbdev-test nbs/dyalog.ipynb --flags dyalog
nbdev-previewQuarto renders saved outputs. bpl-nb --save updates notebook outputs without changing execution counts. Use it for BPL examples/figures and nbdev-test for Python notebook code. Rust language_examples checks BPL page/gallery examples; pytest checks Python page examples. Use BASEDPL_PAGE to focus page checks. Don't run a full documentation build for routine output updates. Before a release, bpl-tests prerelease runs the normal development build and nbdev's preparation, as "Release" describes.
cargo develop and ws-sync run basedpl.editors in a fresh Python process after installing the extension and before copying wheel data. It generates grammars/highlighters, editor templates, keyboard bundles and glyph metadata, including Rust-derived browser/language.js. JavaScript UI changes are authored modules read directly by Vite: no generator or npm refresh. cargo wasm copies the whole browser directory into the npm package and local playground. browser/layout.json owns composed input; captured keyboards.json supports generation without macOS. basedpl.keyboards.capture() refreshes native data when needed, and keyboards.write() generates regional macOS bundles/maps. Regenerate keyboard sources through the normal build, not one-off transformations. basedpl-install copies packaged fonts and macOS bundles from python/basedpl/assets to the user's directories. See editors/jetbrains/README.md and editors/windows/README.md for target-specific work. Rebuild the common icon only when its design changes.
nbs/bpl.xml, the APL/BQN grammars and vim rules share generated glyph classes. hpl.xml includes BPL and adds Hanzi aliases from the same symbol table. Use hpl fences for Hanzi highlighting, bpl for Hanzi names. For Hanzi notebooks, set kernelspec.language to hpl and keep kernelspec.name as bpl. These labels do not enable runtime Hanzi aliases. TextMate exports feed Monaco/Shiki. fonts.css owns the BPL/APL font choices; rerun affected magics after changing them. tests/highlighting.mjs, input.mjs, keylayout/Windows tests and glyph-page checks cover their respective generated interfaces. All artifact generation belongs in repository functions/scripts, not ad hoc kernel transformations.
Before approval, verify successful tests for the release commit and the relevant installed artifacts. Reserve isolated wheel installs for packaging-sensitive work or release preparation, using a small relevant subset of existing tests. ship-rs-build --profile dist builds the distributed configuration; bare ship-rs-build uses release for local checks.
Before each release, run bpl-tests prerelease from basedpl's root, in a shell rather than a kernel, while no one else is editing either repository. It runs both repositories' release checks and saves. First come the builds, tests and saves, in parallel, with each step starting once the steps whose output it reads have passed. In each repository these are cargo develop, pytest -q, nbdev-prepare, and cargo test --profile checked in a target directory of its own. basedpl's development build includes editor generation; its Node tests, notebooks and target_builds wait for that fresh data. Browser tests follow target_builds. Then the timings run one at a time, with nothing else running: cargo test in each repository with <NAME>_COSTS set, rustymath's Python speed test, thread_scaling and the browser speed test. Last, nbdev-clean runs in both repositories, even after a failure. The first failure ends the run, unless --keep-going is given. The summary gives each step's status, time and log. --dry-run lists the steps in their order without running them. Before it starts, the run stops if node_modules or the Node version that .node-version names is missing. It also notes when basedpl needs an unreleased rustymath version, because rustymath then ships first. The run never commits, pushes or tags.
The run's cargo test --profile checked uses the test profile with debug assertions on, which adds session()'s turn check and the standard library's checks in unsafe code. Those checks cost 12 to 55% per call, too much for the test profile that the budgets measure. The speed and scaling tests mark themselves ignored under debug assertions, so that run checks correctness only.
Confirm the clean tree and [workspace.package].version, then run ship-release without flags. Python gets that version through dynamic = ["version"]. Fastship pushes/tags the current version, leaves publication to the Release workflow and bumps Cargo without rebuilding. There is no changelog step. Never commit, push, tag or publish without approval.