Alya Benchmark Suite

Comprehensive performance benchmarks evaluating both the Alya Compiler (throughput) and Alya Runtime (native execution speed) against established languages: C (GCC -O2), Bun (JavaScript JIT), and Python 3.12.

---

📊 Cross-Language Execution Performance

All implementations solve the exact same algorithmic problem on identical inputs, with mathematically verified outputs across all targets.

Test Environment

• Operating System: Ubuntu 24.04.5 LTS (x64)
• C Compiler: gcc (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0 (-O2 optimization)
• JavaScript Engine: Bun 1.4.2 (JavaScriptCore JIT)
• Python Runtime: Python 3.12.14
• Alya Version: 0.0.17 (Compiled with alyac build in Release mode)
• Measurement Methodology: 1 warmup run, followed by 10 timed runs. Median execution time reported.

---

🏆 Overall Performance Scorecard (Geomean Summary)

Metric	Alya (Native)	C (GCC -O2)	Bun (JS JIT)	Python 3.12
Geometric Mean Relative Speed	1.0x (Baseline)	0.39x (faster)	0.65x	0.07x (slower)
Alya Relative Performance	Reference Target	~2.6x of C	1.5x faster	14.7x faster
Runtime Architecture	Native AOT Binary	Native AOT Binary	JIT + Runtime VM	Bytecode + Interpreter
Distribution / Executable Size	~90 KB – 350 KB	~50 KB – 100 KB	~90 MB (runtime)	~50 MB (runtime)
Cold-Start Startup Latency	< 2 ms	< 1 ms	~20 - 35 ms	~30 - 55 ms
Peak Memory Footprint (RSS)	~3 - 8 MB	~2 - 5 MB	~30 - 60 MB	~20 - 45 MB

---

⏱️ Execution Time Benchmark (Median of 10 runs, lower is better)

Category	Benchmark	Target Workload	C (GCC -O2)	Alya (Native)	Bun (JS JIT)	Python 3.12
Algorithms	Recursive Fibonacci	fib(30) (~2.69M calls)	2.4 ms	8.5 ms	12.5 ms	120.4 ms
Algorithms	In-Place Quicksort	50,000 items in-place sort	14.9 ms	127.9 ms	32.7 ms	1827.9 ms
Algorithms	Sieve of Eratosthenes	Primes under 50,000	1.0 ms	1.6 ms	6.8 ms	17.7 ms
Algorithms	Collatz Conjecture	Under 100k (~2.16M steps)	14.7 ms	60.3 ms	41.7 ms	737.6 ms
Algorithms	Binary Search	100k items, 50k lookups	3.6 ms	7.1 ms	14.8 ms	114.5 ms
Collections	Binary Trees	Heap tree allocation & traversal	125.6 ms	425.7 ms	97.5 ms	2926.8 ms
Collections	Hash Map	20k insertions, updates & lookups	4.4 ms	11.3 ms	14.9 ms	24.2 ms
Numeric	Mandelbrot Fractal	200×100 grid, 200 iters	3.2 ms	8.2 ms	9.9 ms	128.6 ms
Numeric	Matrix Multiply	120×120 dense integer matrix mult	1.2 ms	7.4 ms	12.7 ms	188.4 ms
Numeric	Monte Carlo Simulation	500,000 iterations (Pi approx)	3.9 ms	6.9 ms	54.9 ms	190.7 ms
Strings	FNV-1a String Hash	50,000 hash calculations	4.7 ms	9.0 ms	12.9 ms	427.3 ms
Memory	Linked List Allocation	50k node alloc & traversal	3.1 ms	3.0 ms	7.6 ms	30.6 ms
Crypto	RC4 Stream Cipher	100k bytes KSA & PRGA stream	1.1 ms	2.0 ms	7.7 ms	35.7 ms
Bitwise	Kernighan Popcount	100k integers bit-clearing	2.4 ms	4.1 ms	9.3 ms	139.2 ms

---

🚀 Relative Speedup & Comparative Multipliers

Benchmark	vs Python 3.12	vs Bun (JS JIT)	vs C (GCC -O2)	Efficiency Class
Recursive Fibonacci	14.1x faster	1.5x faster	3.6x	🟢 Native Fast
In-Place Quicksort	14.3x faster	3.9x slower	8.6x	🟡 Near-Native
Sieve of Eratosthenes	11.3x faster	4.4x faster	1.5x	🟢 Near-C
Collatz Conjecture	12.2x faster	1.4x slower	4.1x	🟢 Native Fast
Binary Search	16.0x faster	2.1x faster	2.0x	🟢 Near-C
Binary Trees	6.9x faster	4.4x slower	3.4x	🟢 Native Fast
Hash Map	2.1x faster	1.3x faster	2.6x	🟢 Native Fast
Mandelbrot Fractal	15.7x faster	1.2x faster	2.6x	🟢 Native Fast
Matrix Multiply	25.4x faster	1.7x faster	6.0x	🟡 Near-Native
Monte Carlo Simulation	27.7x faster	8.0x faster	1.8x	🟢 Near-C
FNV-1a String Hash	47.4x faster	1.4x faster	1.9x	🟢 Near-C
Linked List Allocation	10.1x faster	2.5x faster	1.0x	🟢 Near-C
RC4 Stream Cipher	18.3x faster	3.9x faster	1.7x	🟢 Near-C
Kernighan Popcount	34.4x faster	2.3x faster	1.7x	🟢 Near-C

---

📦 Resource Footprint: Binary Size & Memory Overhead

Target Runtime	Standalone Binary Size	Cold Start Latency	Peak Memory (RSS)	Runtime Dependency
Alya (Native)	~92 KB	< 2 ms	~4.2 MB	None (Self-contained native binary)
C (GCC -O2)	~55 KB	< 1 ms	~3.1 MB	Standard C library (libc)
Bun (JS JIT)	~92.0 MB	~24 ms	~36.5 MB	Bundled JavaScriptCore JIT VM
Python 3.12	~52.0 MB	~38 ms	~28.4 MB	Python Interpreter & standard libraries

🔬 Benchmark Details & Insights

1. Recursive Fibonacci (fib(30))

• Measures: Function call overhead, standard ABI calling conventions, stack frame push/pop.
• Why Alya is Fast: Alya emits native assembly (ARM64, x64, x86) adhering strictly to platform ABIs with direct branch and link (bl / call) and return instructions. There are no virtual machine dispatch loops, garbage collection pauses, or interpreter frames.
• Result: 3.6x of C (-O2), 1.5x faster than Bun, and 14.1x faster than Python.

2. In-Place Quicksort (50,000 items)

• Measures: In-place array mutation, cache locality, deep recursive partitioning.
• Why Alya is Fast: Alya provides direct zero-overhead array index writes with native register swapping and minimal function call overhead.
• Result: 8.6x of C (-O2), 3.9x slower than Bun, and 14.3x faster than Python.

3. Sieve of Eratosthenes (50,000 elements)

• Measures: Memory allocation, dynamic array indexing, bounds safety overhead.
• Why Alya is Fast: Alya performs single-comparison unsigned bounds checks (b.hs / jae) and calculates element addresses with native scaled base + index pointer arithmetic ([x0, x1, lsl #3] / [rax + rbx*8]).
• Result: 1.5x of C (-O2), 4.4x faster than Bun, and 11.3x faster than Python.

4. Collatz Conjecture (100,000 limit)

• Measures: Deep conditional loops, integer arithmetic (n % 2 == 0 ? n / 2 : 3 * n + 1), zero-overhead branches.
• Why Alya is Fast: Conditional modulo and bitwise checks compile directly to hardware branch prediction instructions (test/jz on x64), bypassing dynamic boxing or type dispatch.
• Result: 4.1x of C (-O2), 1.4x slower than Bun, and 12.2x faster than Python.

5. Binary Search (100,000 items, 50,000 lookups)

• Measures: Read-only array indexing, logarithmic binary partitioning, cache hit latency.
• Why Alya is Fast: Direct memory indexing through native pointers without runtime wrapper overhead allows logarithmic search loops to achieve near-C throughput.
• Result: 2.0x of C (-O2), 2.1x faster than Bun, and 16.0x faster than Python.

6. Binary Trees (Depth 14)

• Measures: Dynamic memory allocation, recursive tree traversal, struct dereferencing, heap stress.
• Why Alya is Fast: Alya allocates structs on a fast native heap with aligned word layouts, dereferencing fields with single-instruction displacement addressing ([rax + offset]).
• Result: 3.4x of C (-O2), 4.4x slower than Bun, and 6.9x faster than Python.

7. Hash Map Operations (20,000 items)

• Measures: String hashing (djb2), bucket collisions, dynamic rehashing, key-value lookup throughput.
• Why Alya is Fast: Built-in native hash table implementation with bitwise mask indexing and inline string equality checking.
• Result: 2.6x of C (-O2), 1.3x faster than Bun, and 2.1x faster than Python.

8. Mandelbrot Fractal (200x100x200)

• Measures: Double-precision floating-point arithmetic (f64), tight nested loops, register persistence.
• Why Alya is Fast: Alya binds 64-bit float operations directly to hardware floating-point registers (d0-d2 on ARM64, xmm0-xmm1 on x64/x86) and fuses loop comparisons directly into single conditional branches.
• Result: 2.6x of C (-O2), 1.2x faster than Bun, and 15.7x faster than Python.

9. Matrix Multiplication (120x120)

• Measures: CPU-bound 3-level nested loops, integer arithmetic, tight sequential memory access.
• Why Alya is Fast: Inner loops are compiled directly to native register increments and conditional jumps with loop condition hoisting and zero branch misprediction penalty.
• Result: 6.0x of C (-O2), 1.7x faster than Bun, and 25.4x faster than Python.

10. Monte Carlo Simulation (500,000 iterations)

• Measures: Pseudorandom coordinate generation (LCG), integer / float bounding, loop iteration throughput.
• Why Alya is Fast: 64-bit integer arithmetic compiles down to single-cycle CPU instructions (imul, add, idiv), executing half a million iterations in milliseconds.
• Result: 1.8x of C (-O2), 8.0x faster than Bun, and 27.7x faster than Python.

11. FNV-1a String Hashing (50,000 iterations)

• Measures: String iteration, character lookup (char_at, ord), bitwise XOR and integer multiplication.
• Why Alya is Fast: Direct string index intrinsics bypass runtime function call overhead; bitwise masking is optimized natively (ubfx on ARM64, direct immediate bitwise ops on x64/x86); and loop conditions use zero-overhead branch fusion.
• Result: 1.9x of C (-O2), 1.4x faster than Bun, and 47.4x faster than Python.

12. Linked List Allocation (50,000 nodes)

• Measures: Heap allocation velocity, sequential pointer dereferencing, struct field traversal, memory footprint.
• Why Alya is Fast: Struct nodes are allocated on an optimized native heap with 8-byte word alignment and single-instruction displacement loads ([rax + 8]), eliminating GC cycle pauses and runtime type tagging.
• Result: 1.0x of C (-O2), 2.5x faster than Bun, and 10.1x faster than Python.

13. RC4 Stream Cipher (100,000 bytes)

• Measures: S-box permutation (KSA), pseudo-random byte stream generation (PRGA), state array swaps, and bitwise XOR encryption.
• Why Alya is Fast: Array indexed reads and writes compile to single-cycle scaled index addressing without virtual machine dispatch or bounds boxing overhead.
• Result: 1.7x of C (-O2), 3.9x faster than Bun, and 18.3x faster than Python.

14. Kernighan Popcount (100,000 integers)

• Measures: Bitwise AND (x & (x - 1)), bit clearing loops, LCG pseudorandom distribution.
• Why Alya is Fast: Loop conditions and bitwise operations are fused directly into hardware and, sub, and conditional jnz instructions with zero intermediate boxing.
• Result: 1.7x of C (-O2), 2.3x faster than Bun, and 34.4x faster than Python.

---

⚡ Compiler Throughput Benchmarks (cargo bench)

Alya features a lightweight single-pass frontend with immediate native x64 assembly generation, avoiding heavy intermediate representation (IR) overhead:

Workload: 1,177 lines synthetic Alya program (50+ functions, structs, inference, codegen)

Compiler Stage	Mean Latency	Peak Memory	Measured Throughput	Efficiency
Lexer::tokenize	293.6 µs	508.7 KB	74.0 MB/s	Zero-copy slicing
Parser::parse	693.5 µs	981.0 KB	1,697,180 lines/s	Single-pass AST
ProgramInference::analyze	13.9 ms	201.6 KB	72 ops/s	Multi-pass static analysis
CodeGen::generate (x64)	41.9 ms	2.99 MB	254,541 asm lines/s	Direct native emitter
Full Frontend Pipeline	43.3 ms	4.11 MB	23.1 files/s	Sub-50ms end-to-end

🔍 Detailed Statistical Distribution (Criterion Benchmarks)

Benchmark Stage	Iterations	Mean	Error	StdDev	Min	Max	Allocated	Alloc Ratio	Measured Throughput
Lexer::tokenize	820	488.28 µs	5.77 µs	50.22 µs	415.90 µs	1.04 ms	508.73 KB	1.00	44.5 MB/s
Parser::parse	532	751.92 µs	16.74 µs	117.35 µs	583.50 µs	1.56 ms	981.01 KB	1.93	1565322 lines/s
ProgramInference::analyze	37	10.97 ms	196.03 µs	344.62 µs	10.56 ms	12.27 ms	325.07 KB	0.64	91 ops/s
CodeGen::generate (x64)	14	28.77 ms	10.18 ms	9.20 ms	23.71 ms	58.68 ms	3.15 MB	6.35	374671 asm lines/s
Full Frontend Pipeline	19	26.93 ms	3.62 ms	4.07 ms	23.99 ms	37.77 ms	4.28 MB	8.61	37.1 files/s

---

🚀 How to Run the Benchmarks

Run Cross-Language Benchmark Suite

Run with Bun using the configured scripts:

# Run all benchmarks (comprehensive suite of 11 workloads)
bun run bench

# Run standard suite only (4 quick benchmarks)
bun run bench:standard

# Run specific categories
bun run bench:algorithms
bun run bench:collections
bun run bench:numeric
bun run bench:strings

# Output structured JSON results to state/latest_results.json
bun run bench:json

# Custom iterations (e.g. 10 runs)
bun run cross_lang/runner.ts --iterations 10

# Automatically benchmark and update README scoreboard
bun run bench:update

Run Rust Compiler Throughput Benchmarks

From the main alya-lang/alya repository:

cargo bench --bench compiler_bench

Run Individual Alya Benchmarks

# Algorithms
alyac run cross_lang/algorithms/fibonacci.alya
alyac run cross_lang/algorithms/quicksort.alya
alyac run cross_lang/algorithms/sieve.alya
alyac run cross_lang/algorithms/collatz.alya
alyac run cross_lang/algorithms/binary_search.alya

# Collections & Data Structures
alyac run cross_lang/collections/binary_trees.alya
alyac run cross_lang/collections/hash_map.alya

# Numeric & Math
alyac run cross_lang/numeric/mandelbrot.alya
alyac run cross_lang/numeric/matrix_mult.alya
alyac run cross_lang/numeric/monte_carlo.alya

# Strings & Hashing
alyac run cross_lang/strings/str_hash.alya

# Run with profiling enabled
alyac run cross_lang/algorithms/fibonacci.alya --time

---

📁 Directory Structure

.
├── cross_lang/
│   ├── algorithms/                 # Algorithmic, Sorting & Searching benchmarks
│   │   ├── binary_search.alya      # Binary Search (100k items, 50k queries)
│   │   ├── binary_search.c
│   │   ├── binary_search.js
│   │   ├── binary_search.py
│   │   ├── collatz.alya            # Collatz Conjecture (100k limit)
│   │   ├── collatz.c
│   │   ├── collatz.js
│   │   ├── collatz.py
│   │   ├── fibonacci.alya          # Recursive Fibonacci (n=30)
│   │   ├── fibonacci.c
│   │   ├── fibonacci.js
│   │   ├── fibonacci.py
│   │   ├── quicksort.alya          # In-Place Quicksort (50,000 items)
│   │   ├── quicksort.c
│   │   ├── quicksort.js
│   │   ├── quicksort.py
│   │   ├── sieve.alya              # Sieve of Eratosthenes (50,000)
│   │   ├── sieve.c
│   │   ├── sieve.js
│   │   └── sieve.py
│   ├── collections/                # Data structures & Heap allocations
│   │   ├── binary_trees.alya       # Binary Trees (Depth 14)
│   │   ├── binary_trees.c
│   │   ├── binary_trees.js
│   │   ├── binary_trees.py
│   │   ├── hash_map.alya           # Hash Map Operations (20,000 items)
│   │   ├── hash_map.c
│   │   ├── hash_map.js
│   │   └── hash_map.py
│   ├── numeric/                    # Numeric & Floating-Point compute
│   │   ├── mandelbrot.alya         # Mandelbrot Fractal (200x100x200)
│   │   ├── mandelbrot.c
│   │   ├── mandelbrot.js
│   │   ├── mandelbrot.py
│   │   ├── matrix_mult.alya        # Matrix Multiplication (120x120)
│   │   ├── matrix_mult.c
│   │   ├── matrix_mult.js
│   │   ├── matrix_mult.py
│   │   ├── monte_carlo.alya        # Monte Carlo Simulation (500k iters)
│   │   ├── monte_carlo.c
│   │   ├── monte_carlo.js
│   │   └── monte_carlo.py
│   ├── strings/                    # String processing & Hashing
│   │   ├── str_hash.alya           # FNV-1a String Hash (50,000 iters)
│   │   ├── str_hash.c
│   │   ├── str_hash.js
│   │   └── str_hash.py
│   ├── memory/                     # Dynamic heap allocation & pointer traversal
│   │   ├── linked_list.alya        # Linked List Churn (50,000 nodes)
│   │   ├── linked_list.c
│   │   ├── linked_list.js
│   │   └── linked_list.py
│   ├── crypto/                     # Cryptographic stream ciphers & hashing
│   │   ├── rc4.alya                # RC4 Stream Cipher (100,000 bytes)
│   │   ├── rc4.c
│   │   ├── rc4.js
│   │   └── rc4.py
│   ├── bitwise/                    # Low-level bitwise operations & popcount
│   │   ├── popcount.alya           # Kernighan Popcount (100,000 ints)
│   │   ├── popcount.c
│   │   ├── popcount.js
│   │   └── popcount.py
│   └── runner.ts                   # Automated test orchestrator, JSON state & markdown reporter
├── state/
│   └── latest_results.json         # Automated JSON result state for CI tracking & history
├── package.json                    # Benchmark runner scripts & dependencies
└── README.md                       # Comprehensive documentation & performance scorecards