diff --git a/benchmarks/README.md b/benchmarks/README.md index d6dd3e5fc..3ee6b6f4b 100644 --- a/benchmarks/README.md +++ b/benchmarks/README.md @@ -6,11 +6,13 @@ latencies, process-throughput windows, phase timings, counters, memory/code sizes, checksums, build fingerprints, and host provenance. Current samples identify their timing protocol as -`protocol.name = "shared-js-fixed-warmup-v2"`. The JSON envelope remains -`jit-benchmark-v1`, but these timings are **not equivalent to historical v1 -measurements**. The legacy `jit-bench-report` gate reporter explicitly rejects -v2 samples as an incompatible timing protocol. Do not pass `--report` when -collecting v2 evidence or interpret that rejection as a runtime test failure. +`protocol.name = "shared-js-multibatch-v3"`. The JSON envelope remains +`jit-benchmark-v1`, but these timings are **not equivalent to historical v1 or +`shared-js-fixed-warmup-v2` measurements** (see +[Multi-batch timing](#multi-batch-timing-and-readiness-diagnostics)). The legacy +`jit-bench-report` gate reporter explicitly rejects v2 and v3 samples as an +incompatible timing protocol. Do not pass `--report` when collecting v2/v3 +evidence or interpret that rejection as a runtime test failure. Build and collect the complete five-mode raw matrix: @@ -18,7 +20,7 @@ Build and collect the complete five-mode raw matrix: cargo build --release --manifest-path benchmarks/Cargo.toml --bins ./target/release/jit-bench compare \ --modes interpreter,tier1,tier2,automatic,bun \ - --output benchmarks/results/comparison-v2.json + --output benchmarks/results/comparison-v3.json ``` Bun uses its default engine configuration: the runner no longer adds `--smol`. @@ -28,6 +30,13 @@ counters in comparison samples are JSON `null` (N/A). Raw worker output keeps zero placeholders in the legacy cumulative counter fields; these are not Bun execution-tier measurements. +Bun runs the shared wrapper from a temporary `.mjs` file, not with `bun -e`; +the file is deleted after the run and its SHA-256 is recorded in +`protocol.bun_wrapper_sha256` (`protocol.bun_launch = "file"`). The wrapper does +not embed the script path, so its hash only changes with the driver, the warmup +count, or `JIT_BENCH_TIMED_BATCHES`. `JIT_BENCH_BUN_LAUNCH=eval` reproduces the +historical `bun -e` launch as a **control only**; `paired.py` rejects it. + For an external paired baseline/candidate collector, each command below emits one worker JSON object. Build both runtime revisions with the exact same benchmark harness and scripts, and retain immutable binaries with their hashes: @@ -43,46 +52,75 @@ External collectors must check checksums, protocol identity, script/driver hashes, sampling counts, and native-counter consistency themselves, and record engine identities and host/build provenance. -## Fixed-warmup timing and readiness diagnostics +## Multi-batch timing and readiness diagnostics Every engine executes one first call, then exactly 64 warmup batches of ten -calls, then one measured batch of ten calls. The shared `batch-driver.js` uses -identical workload arguments and stores every result. Promise results complete -sequentially before the batch ends. Primitive checksums are generated after -timing, including during warmup, and combine all ten results. The -`protocol.warmup_batch_ns` array records the 64 consecutive warmup timings. - -`elapsed_ns` is the **fixed-warmup batch latency**, not a claim that compilation -has settled. `protocol.fixed_metrics_before` and `fixed_metrics_after` surround -that measured batch and exclude checksum execution. Derive native, Tier 2, -deopt, fallback, compilation, and installation deltas from these snapshots. +calls, then **K consecutive timed batches** of ten calls (`K = 16` by default; +`JIT_BENCH_TIMED_BATCHES` overrides it, and `paired.py` passes the same K to +every engine and variant). The shared `batch-driver.js` uses identical workload +arguments and stores every result. Promise results complete sequentially before +the batch ends. Primitive checksums are generated after timing, including during +warmup, and combine all ten results; every timed batch must reproduce the first +timed batch's checksum. QuickJS polls the JIT after every warmup and timed +batch, outside the timed region, exactly as during warmup. + +- `elapsed_ns` is the **upper median of the K timed batches** (the harness-wide + `ceil((n - 1) / 2)` quantile rule). It is a steady-batch latency after a + fixed warmup, not a claim that compilation has settled. +- `protocol.timed_batch_ns` stores all K raw timings in execution order; + `protocol.warmup_batch_ns` stores the 64 warmup timings. +- `protocol.fixed_batch_ns` is the first timed batch, i.e. the single batch that + `shared-js-fixed-warmup-v2` reported as `elapsed_ns`. It is a cold/jitter + **diagnostic only**. +- `protocol.outlier_batches` counts timed batches slower than + `protocol.outlier_ratio` (5) times the median. Outliers stay in + `timed_batch_ns` and are never silently dropped; `summarize_paired.py` reports + their total and the number of affected processes. + +Why v3 exists: under `bun -e`, Bun's batch at index 64 (the only batch v2 timed) +deterministically hit a one-off ~0.1-0.2 ms JSC stall, inflating Bun latency by +1.2x-32x on two-argument kernels. Running the same wrapper from a file removes +the stall, and the median of K batches no longer depends on one batch index. +For example, a noisy smoke of `numeric` on one host measured Bun at about +0.005 ms per ten calls with the file launch (fixed batch also about 0.005 ms), +while the `-e` control still showed a 0.13-0.19 ms fixed batch with 2-3 flagged +outliers and a median of about 0.005 ms. v2 and v3 numbers must not be mixed in +one comparison; `protocol_check.py` rejects mixed evidence. + +`protocol.fixed_metrics_before` and `fixed_metrics_after` surround the first +timed batch and exclude checksum execution, as in v2. +`protocol.timed_metrics_before` and `timed_metrics_after` surround the whole +timed window, **including** the untimed checksum and JIT-poll steps between +batches. Derive native, Tier 2, deopt, fallback, compilation, and installation +deltas from these snapshots; `paired.py` stores them as `fixed_deltas` and +`timed_deltas` and uses the timed window for `compilation_quiet_samples`. Counters aggregate the shared driver and workload; they do not identify the native tier of an individual workload function. Bun has no equivalent internal metrics and reports these snapshots as `null`; interpreter workers have no attached JIT and also leave them `null`. -QuickJS still times one outer Rust `Function::call` and, for asynchronous work, -its Promise completion bridge. Bun times the corresponding JS call and await. -Both timings include driver array/closure work. The protocol removes the old -per-workload Rust lookup/checksum/poll loop from the measured region, but does -not eliminate the outer host-call difference. The driver is part of the -measured workload; these are not isolated function-body timings. +QuickJS still times one outer Rust `Function::call` per batch and, for +asynchronous work, its Promise completion bridge. Bun times the corresponding JS +call and await. Both timings include driver array/closure work. The protocol +removes the old per-workload Rust lookup/checksum/poll loop from the measured +region, but does not eliminate the outer host-call difference. The driver is +part of the measured workload; these are not isolated function-body timings. -After collecting the fixed-window sample, QuickJS runs the existing bounded +After collecting the timed batches, QuickJS runs the existing bounded JIT readiness and settling checks, then another shared batch. Its timing is stored in `protocol.readiness_diagnostic_ns` and `phases.steady_state_ns`. Bun has no separately conditioned readiness diagnostic. Do not divide Bun's -fixed-window latency by QuickJS's post-readiness diagnostic and call that a +multi-batch latency by QuickJS's post-readiness diagnostic and call that a matched steady-state comparison. Legacy cumulative counters include first -execution, warmup, the fixed sample, and subsequent diagnostics; they cannot -prove native execution or settled compilation during the fixed window. +execution, warmup, the timed batches, and subsequent diagnostics; they cannot +prove native execution or settled compilation during the timed window. Forced Tier 2 probes can declare `globalThis.tier2ReadyInstalls` to require a known publication count in the readiness diagnostic. The direct-call probe requires four: two baseline and two optimizing artifacts. An unrelated blacklisted function cannot bypass that requirement. Explicit forced-Tier2 requirements reject unsettled diagnostic startup or new compilation during -the diagnostic batch. Those checks happen after the fixed-window sample and +the diagnostic batch. Those checks happen after the timed batches and do not make its warmup policy variable. Unsupported probes retain bounded fallback/readiness behavior. @@ -161,10 +199,12 @@ fallback-only result as proof that a JIT tier supports the feature. | `strings-regexp` | String construction, slicing, RegExp capture and replacement | | `arrays-typed` | Packed-array growth/traversal and Int32Array/Float64Array traffic | | `objects-polymorphic` | Allocation, property reads/writes, and four stable shapes | +| `property-polymorphic` | Allocation-free reads at two sites that each observe four stable shapes (one prototype-less); the bounded polymorphic inline cache | | `calls-recursion-closures` | Four-deep calls, bounded recursion, and mutable closure capture | | `json-codec` | Repeated nested JSON encoding and decoding | | `map-set-bigint` | Map and Set mutation/iteration plus bounded BigInt arithmetic | | `exceptions-promises-async` | Throw/catch, Promise jobs, async functions, and continuations | +| `methods-dynamic` | Prototype methods using `this`, `typeof`/`in`/`instanceof` dispatch, object destructuring, `**`, and compound element updates | Select the matrix with `JIT_BENCH_WORKLOADS` for smoke or publishable runs; the sample/warmup/window rules above remain unchanged. diff --git a/benchmarks/model.rs b/benchmarks/model.rs index e30953d03..9e111cbca 100644 --- a/benchmarks/model.rs +++ b/benchmarks/model.rs @@ -202,9 +202,43 @@ mod tests { assert_eq!(p99, 100); assert!(ci[0] <= median && median <= ci[1]); } + + #[test] + fn timed_batch_summary_uses_the_median_and_counts_but_keeps_spikes() { + // A single one-off stall (the old fixed-batch artifact) no longer + // determines the reported latency, but it stays visible. + let mut batches = vec![5_000u64; 16]; + batches[0] = 187_000; + let (median, outliers) = timed_batch_summary(&batches); + assert_eq!(median, 5_000); + assert_eq!(outliers, 1); + // Exactly 5x the median is not an outlier; above it is. + batches[0] = 25_000; + assert_eq!(timed_batch_summary(&batches).1, 0); + batches[0] = 25_001; + assert_eq!(timed_batch_summary(&batches).1, 1); + // Even K uses the harness-wide upper median. + assert_eq!(timed_batch_summary(&[1, 2, 3, 4]), (3, 0)); + assert_eq!(timed_batch_summary(&[]), (0, 0)); + } } +/// Legacy timing protocol: 64 warmup batches, then one timed batch. +#[allow(dead_code)] +pub const PROTOCOL_FIXED_WARMUP_V2: &str = "shared-js-fixed-warmup-v2"; +/// Current timing protocol: 64 warmup batches, then K consecutive timed +/// batches whose median becomes `elapsed_ns`. +#[allow(dead_code)] +pub const PROTOCOL_MULTIBATCH_V3: &str = "shared-js-multibatch-v3"; +/// A timed batch slower than this multiple of the timed-batch median is +/// counted as an outlier. Outliers are never discarded. +#[allow(dead_code)] +pub const OUTLIER_RATIO: u64 = 5; + /// Protocol identity separates these fixed-warmup timings from historical v1 timing. +/// +/// Fields after `readiness_diagnostic_ns` were added by +/// `shared-js-multibatch-v3`; they default to empty in v2 evidence. #[derive(Clone, Debug, Default, Deserialize, Serialize)] #[serde(deny_unknown_fields)] pub struct ProtocolEvidence { @@ -217,4 +251,45 @@ pub struct ProtocolEvidence { pub fixed_metrics_before: Option>, pub fixed_metrics_after: Option>, pub readiness_diagnostic_ns: Option, + /// Number of consecutive timed batches after the warmups (v3). + #[serde(default)] + pub timed_batches: u32, + /// Raw latency of every timed batch, in execution order (v3). + #[serde(default)] + pub timed_batch_ns: Vec, + /// Diagnostic only: the first timed batch, i.e. the single batch that + /// `shared-js-fixed-warmup-v2` reported as `elapsed_ns` (v3). + #[serde(default)] + pub fixed_batch_ns: Option, + /// Outlier rule: a timed batch above `outlier_ratio` times the median (v3). + #[serde(default)] + pub outlier_ratio: u64, + /// Number of timed batches that exceed the outlier rule (v3). + #[serde(default)] + pub outlier_batches: u32, + /// JIT metric snapshots around the whole timed window. They include the + /// untimed checksum and poll steps between timed batches (v3). + #[serde(default)] + pub timed_metrics_before: Option>, + #[serde(default)] + pub timed_metrics_after: Option>, + /// How Bun loaded its wrapper: `file` (v3 default) or `eval` (control) (v3). + #[serde(default)] + pub bun_launch: Option, + /// SHA-256 of the exact Bun wrapper module that was executed (v3). + #[serde(default)] + pub bun_wrapper_sha256: Option, +} + +/// Summary of K timed batches: (upper median, outlier count). +/// +/// The median uses the same upper-quantile rule as the rest of the harness. +#[allow(dead_code)] +pub fn timed_batch_summary(batches: &[u64]) -> (u64, u32) { + let mut sorted = batches.to_vec(); + sorted.sort_unstable(); + let median = quantile(&sorted, 0.5); + let threshold = median.saturating_mul(OUTLIER_RATIO); + let outliers = batches.iter().filter(|&&ns| ns > threshold).count(); + (median, u32::try_from(outliers).unwrap_or(u32::MAX)) } diff --git a/benchmarks/paired.py b/benchmarks/paired.py index 10d746aa5..9a89bdbb3 100644 --- a/benchmarks/paired.py +++ b/benchmarks/paired.py @@ -13,6 +13,7 @@ from pathlib import Path import subprocess import time +from protocol_check import PROTOCOL_V3, timed_batches_from_env, validate_protocol p = argparse.ArgumentParser() p.add_argument('--binaries', required=True, help='JSON map from variant to binary') @@ -25,6 +26,10 @@ args = p.parse_args() if args.pairs < 1: p.error('--pairs must be positive') +try: + timed_batches = timed_batches_from_env() +except ValueError as error: + p.error(str(error)) if args.cpu is not None and platform.system() != 'Linux': p.error('--cpu requires Linux taskset') import fcntl @@ -53,8 +58,8 @@ bun_version=subprocess.check_output([str(bun), '--version'],text=True).strip(), bun_sha256=hashlib.sha256(bun.read_bytes()).hexdigest(), bun_flags=[], rustc=subprocess.check_output(['rustc','-Vv'],text=True), - protocol='shared-js-fixed-warmup-v2', throughput_windows=0, - note='Latency-only paired candidate diagnostic. Fixed 10-call window; counters include shared driver. Readiness diagnostic is separately conditioned.') + protocol=PROTOCOL_V3, timed_batches=timed_batches, throughput_windows=0, + note='Latency-only paired candidate diagnostic. elapsed_ns is the median of the timed 10-call batches that follow 64 warmups; the first timed batch is the legacy fixed-batch diagnostic. Counters include shared driver. Readiness diagnostic is separately conditioned.') (out / 'metadata.json').write_text(json.dumps(metadata,indent=2)+'\n') for workload in args.workloads.split(','): script = Path(args.scripts) / (workload + '.js') @@ -68,20 +73,18 @@ if target.exists(): raise RuntimeError(f'Refusing to overwrite existing evidence: {target}') command = [bins[variant], 'worker', '--mode', mode, '--script', str(script)] + # Every engine and variant receives the same K explicitly. + worker_env = dict(os.environ, JIT_BENCH_TIMED_BATCHES=str(timed_batches)) if args.cpu is not None: command = ['taskset', '-c', args.cpu] + command started = time.monotonic() - result = subprocess.run(command, capture_output=True, text=True, timeout=180) + result = subprocess.run(command, capture_output=True, text=True, timeout=180, env=worker_env) if result.returncode: (out / 'failure.json').write_text(json.dumps(dict(command=command, stdout=result.stdout, stderr=result.stderr),indent=2)) raise RuntimeError(f'Worker failed: {command}: {result.stderr}') value = json.loads(result.stdout) protocol = value['protocol'] - assert protocol['name'] == metadata['protocol'] - assert protocol['script_sha256'] == metadata['scripts'][workload] - assert protocol['warmup_batches'] == len(protocol['warmup_batch_ns']) == 64 - assert protocol['calls_per_batch'] == 10 - assert value['elapsed_ns'] > 0 + validate_protocol(value, metadata, workload, mode) if mode != 'bun': assert value['native_exits'] == value['native_entries'] before, after = protocol['fixed_metrics_before'], protocol['fixed_metrics_after'] @@ -91,12 +94,20 @@ assert all(v >= 0 for v in value['fixed_deltas'].values()) assert value['fixed_deltas']['native_entries'] == value['fixed_deltas']['native_exits'] value['fixed_compilation_quiet'] = (after['pending_worker_jobs'] == before['pending_worker_jobs'] == 0 and after['pending_snapshot_bytes'] == before['pending_snapshot_bytes'] == 0 and value['fixed_deltas']['installed'] == 0 and value['fixed_deltas']['compile_ns'] == 0) + before, after = protocol.get('timed_metrics_before'), protocol.get('timed_metrics_after') + if before is not None: + # Spans every timed batch plus the untimed checksum/poll steps between them. + value['timed_deltas'] = {k:after[k]-before[k] for k in cumulative} + assert all(v >= 0 for v in value['timed_deltas'].values()) + assert value['timed_deltas']['native_entries'] == value['timed_deltas']['native_exits'] + value['timed_compilation_quiet'] = (after['pending_worker_jobs'] == before['pending_worker_jobs'] == 0 and after['pending_snapshot_bytes'] == before['pending_snapshot_bytes'] == 0 and value['timed_deltas']['installed'] == 0 and value['timed_deltas']['compile_ns'] == 0) value.update(variant=variant, requested_mode=mode, pair_index=pair, process_wall_seconds=time.monotonic()-started) target.write_text(json.dumps(value,indent=2)+'\n') print(f'{workload}: pair {pair} complete',flush=True) values = [json.loads(path.read_text()) for path in out.glob(f'{workload}-*.json')] assert len({v['checksum'] for v in values}) == 1, workload assert len({v['protocol']['driver_sha256'] for v in values}) == 1, workload + assert len({v['protocol'].get('bun_wrapper_sha256') for v in values if v['requested_mode'] == 'bun'}) <= 1, workload (out / 'metadata.json').write_text(json.dumps(metadata,indent=2)+'\n') for variant, binary in bins.items(): assert hashlib.sha256(Path(binary).read_bytes()).hexdigest() == metadata['binary_sha256'][variant] diff --git a/benchmarks/protocol_check.py b/benchmarks/protocol_check.py new file mode 100644 index 000000000..826621c79 --- /dev/null +++ b/benchmarks/protocol_check.py @@ -0,0 +1,67 @@ +"""Shared worker-protocol validation for paired.py and summarize_paired.py. + +`shared-js-fixed-warmup-v2` timed one batch after 64 warmups. It is kept +readable so historical evidence can still be summarized, but it must never be +mixed with `shared-js-multibatch-v3`, where `elapsed_ns` is the upper median +of `timed_batches` consecutive timed batches. +""" +import os + +PROTOCOL_V2 = 'shared-js-fixed-warmup-v2' +PROTOCOL_V3 = 'shared-js-multibatch-v3' +WARMUP_BATCHES = 64 +CALLS_PER_BATCH = 10 +OUTLIER_RATIO = 5 +DEFAULT_TIMED_BATCHES = 16 + + +def timed_batches_from_env(): + raw = os.environ.get('JIT_BENCH_TIMED_BATCHES') + if raw is None: + return DEFAULT_TIMED_BATCHES + if not raw.isdigit() or int(raw) <= 0: + raise ValueError('JIT_BENCH_TIMED_BATCHES must be a positive integer') + return int(raw) + + +def upper_median(values): + ordered = sorted(values) + return ordered[-(-(len(ordered) - 1) // 2)] if ordered else 0 + + +def timed_summary(batches): + median = upper_median(batches) + return median, sum(ns > median * OUTLIER_RATIO for ns in batches) + + +def validate_protocol(sample, metadata, workload, mode): + """Reject incompatible or internally inconsistent worker evidence.""" + protocol = sample['protocol'] + name = protocol['name'] + if name not in (PROTOCOL_V2, PROTOCOL_V3): + raise AssertionError(f'unsupported timing protocol {name!r}') + if name != metadata['protocol']: + raise AssertionError(f'protocol {name!r} mixed into {metadata["protocol"]!r} evidence') + assert protocol['script_sha256'] == metadata['scripts'][workload] + assert protocol['calls_per_batch'] == CALLS_PER_BATCH + assert protocol['warmup_batches'] == len(protocol['warmup_batch_ns']) == WARMUP_BATCHES + assert sample['elapsed_ns'] > 0 + if name == PROTOCOL_V2: + return + timed = protocol['timed_batch_ns'] + expected = metadata.get('timed_batches', DEFAULT_TIMED_BATCHES) + assert protocol['timed_batches'] == len(timed) == expected, (protocol['timed_batches'], len(timed), expected) + assert protocol['fixed_batch_ns'] == timed[0] + assert protocol['outlier_ratio'] == OUTLIER_RATIO + assert (sample['elapsed_ns'], protocol['outlier_batches']) == timed_summary(timed) + if mode == 'bun': + assert protocol['bun_launch'] == 'file', 'publishable Bun evidence must use the file wrapper' + assert len(protocol['bun_wrapper_sha256']) == 64 + + +if __name__ == '__main__': + # Self-test: python3 benchmarks/protocol_check.py + assert upper_median([1, 2, 3, 4]) == 3 and upper_median([1, 2, 3]) == 2 + assert timed_summary([5] * 15 + [187]) == (5, 1) + assert timed_summary([5] * 15 + [25]) == (5, 0) + print('protocol_check self-test ok') diff --git a/benchmarks/report.rs b/benchmarks/report.rs index 52d1550e3..f6361f549 100644 --- a/benchmarks/report.rs +++ b/benchmarks/report.rs @@ -28,14 +28,25 @@ fn real_main() -> Result { } fn validate(data: &BenchmarkFile) -> Result<(), String> { - if data + if let Some(protocol) = data .modes .iter() .flat_map(|m| &m.workloads) .flat_map(|w| &w.samples) - .any(|s| s.protocol.is_some()) + .find_map(|s| s.protocol.as_ref()) { - return Err("incompatible timing protocol: the legacy gate reporter cannot evaluate shared-js-fixed-warmup-v2; use fixed-window metrics and report readiness diagnostics separately".into()); + let name = if protocol.name.is_empty() { + "an unnamed protocol" + } else { + protocol.name.as_str() + }; + return Err(format!( + "incompatible timing protocol: the legacy gate reporter cannot evaluate {name} \ + (it only understands pre-v2 samples without protocol evidence; {} and {} \ + need a protocol-aware paired summary such as summarize_paired.py)", + model::PROTOCOL_FIXED_WARMUP_V2, + model::PROTOCOL_MULTIBATCH_V3, + )); } if data.schema != "jit-benchmark-v1" { return Err("unsupported schema".into()); @@ -709,6 +720,35 @@ mod tests { .contains("incompatible timing protocol")); } + #[test] + fn legacy_gate_reporter_explicitly_rejects_the_multibatch_protocol() { + for name in [ + model::PROTOCOL_FIXED_WARMUP_V2, + model::PROTOCOL_MULTIBATCH_V3, + ] { + let mut data = valid_file(); + data.modes[1].workloads[0].samples[3].protocol = Some(model::ProtocolEvidence { + name: name.into(), + timed_batches: 16, + ..Default::default() + }); + let error = validate(&data).unwrap_err(); + assert!(error.contains("incompatible timing protocol"), "{error}"); + assert!(error.contains(name), "{error}"); + } + } + + #[test] + fn v2_protocol_evidence_still_deserializes_without_v3_fields() { + let v2 = r#"{"name":"shared-js-fixed-warmup-v2","script_sha256":"a","driver_sha256":"b", + "warmup_batches":64,"calls_per_batch":10,"warmup_batch_ns":[1], + "fixed_metrics_before":null,"fixed_metrics_after":null,"readiness_diagnostic_ns":null}"#; + let parsed: model::ProtocolEvidence = serde_json::from_str(v2).unwrap(); + assert_eq!(parsed.name, model::PROTOCOL_FIXED_WARMUP_V2); + assert_eq!(parsed.timed_batches, 0); + assert!(parsed.timed_batch_ns.is_empty() && parsed.fixed_batch_ns.is_none()); + } + #[test] fn validator_rejects_dirty_or_stale_provenance() { let mut data = valid_file(); diff --git a/benchmarks/results/p4d-scalar-codegen-noisy-diagnostic-x86_64.json b/benchmarks/results/p4d-scalar-codegen-noisy-diagnostic-x86_64.json new file mode 100644 index 000000000..f746f18e3 --- /dev/null +++ b/benchmarks/results/p4d-scalar-codegen-noisy-diagnostic-x86_64.json @@ -0,0 +1,610 @@ +{ + "description": "Noisy diagnostic, not publishable. jit-bench worker --mode automatic, alternating baseline 82d3808 (/home/jason/work/quickjs-jit/target/release/jit-bench) and perf/p4d-scalar-codegen per run; value per run = median(protocol.warmup_batch_ns[-16:]) in ns per batch of 10 workload calls. Host i7-13700KF shared with ~16 concurrent agents; hybrid P/E cores make runs bimodal. fast_mode = median of runs <= 1.4x the per-engine minimum. checksums_match compares both engines against the interpreter-mode checksum of the new binary.", + "targets": { + "numeric": { + "base_ns": [ + 54494.5, + 29215.0, + 34412.5, + 29503.5, + 31949.0, + 32261.5, + 29444.5, + 55794.0, + 54762.0, + 29313.5, + 42845.5 + ], + "new_ns": [ + 49389.5, + 52401.5, + 37153.5, + 21503.5, + 23211.5, + 21416.5, + 21375.0, + 21390.5, + 21486.0, + 21359.0, + 21320.5 + ], + "median_speed": 1.5015126128641907, + "fast_mode_speed": 1.3784427780503188, + "checksums_match": true + }, + "scalar-loop": { + "base_ns": [ + 29187.0, + 29600.0, + 29272.5, + 55925.5, + 29431.5, + 29263.5, + 30753.0, + 29127.5, + 29207.0, + 29431.0, + 55535.0 + ], + "new_ns": [ + 20652.0, + 49473.5, + 21461.5, + 21291.5, + 21454.0, + 21383.0, + 21422.5, + 21447.5, + 21499.0, + 50205.5, + 21298.5 + ], + "median_speed": 1.3722345261685511, + "fast_mode_speed": 1.3664371571945384, + "checksums_match": true + }, + "scalar-control-flow": { + "base_ns": [ + 54308.0, + 54458.0, + 54627.0, + 54600.0, + 54353.0, + 103877.5, + 54548.5, + 54277.5, + 54939.0, + 55237.5, + 55044.0 + ], + "new_ns": [ + 43934.5, + 49981.0, + 43947.0, + 43737.0, + 43768.5, + 43919.5, + 43723.5, + 44319.5, + 43922.0, + 43736.0, + 43844.0 + ], + "median_speed": 1.2431835517253156, + "fast_mode_speed": 1.242597251790207, + "checksums_match": true + }, + "scalar-expressions": { + "base_ns": [ + 50451.0, + 50391.0, + 50464.5, + 50451.0, + 50562.5, + 50430.0, + 50672.5, + 50643.0, + 50685.5, + 50440.5, + 50604.5 + ], + "new_ns": [ + 45820.5, + 45495.5, + 45484.0, + 45681.0, + 45572.5, + 45641.5, + 45603.0, + 45589.5, + 45924.0, + 45425.5, + 77217.5 + ], + "median_speed": 1.1066048286296954, + "fast_mode_speed": 1.1067686487375608, + "checksums_match": true + }, + "quickjs-bitops": { + "base_ns": [ + 62113.5, + 111476.5, + 77017.0, + 103516.5, + 61908.0, + 61901.0, + 65440.5, + 62794.0, + 63674.0, + 101951.0, + 61894.0 + ], + "new_ns": [ + 63716.5, + 31625.5, + 64177.5, + 31730.0, + 59846.0, + 62208.5, + 31998.5, + 64153.0, + 61983.0, + 31866.0, + 31547.0 + ], + "median_speed": 1.063964174715102, + "fast_mode_speed": 1.9682871099905452, + "checksums_match": true + }, + "quickjs-int-arith": { + "base_ns": [ + 1378596.0, + 1363117.5, + 1323394.5, + 1337757.0, + 1330072.0, + 1355545.0, + 1327279.5, + 1328156.0, + 1357338.5, + 1370443.5, + 1330557.5 + ], + "new_ns": [ + 331377.5, + 336021.0, + 326964.0, + 327726.5, + 330250.5, + 333516.0, + 328931.5, + 328968.5, + 325912.0, + 327642.5, + 327632.5 + ], + "median_speed": 4.0669774709932005, + "fast_mode_speed": 4.0669774709932005, + "checksums_match": true + }, + "fibonacci-iterative": { + "base_ns": [ + 986966.0, + 997385.5, + 1008717.0, + 1003569.5, + 980623.0, + 995214.5, + 989059.0, + 991964.0, + 1002596.0, + 1003265.0, + 986143.5 + ], + "new_ns": [ + 743178.0, + 742836.0, + 744703.0, + 747322.0, + 746036.5, + 744285.0, + 751882.0, + 744536.5, + 744561.0, + 746252.5, + 743135.5 + ], + "median_speed": 1.3366460236300317, + "fast_mode_speed": 1.3366460236300317, + "checksums_match": true + }, + "host-compute": { + "base_ns": [ + 68686.5, + 71117.5, + 71365.0, + 71299.5, + 70783.0, + 128140.5, + 139753.5, + 72315.5, + 70552.0, + 68864.0, + 72185.0 + ], + "new_ns": [ + 51547.0, + 50883.5, + 51380.0, + 51000.5, + 51628.5, + 57548.0, + 50849.0, + 50939.5, + 50996.5, + 52080.5, + 52412.0 + ], + "median_speed": 1.3876897625535227, + "fast_mode_speed": 1.3841475282210978, + "checksums_match": true + }, + "float64-dense": { + "base_ns": [ + 128755.5, + 129131.5, + 130562.0, + 129489.5, + 128452.0, + 129830.5, + 128900.0, + 128758.5, + 129125.5, + 131718.5, + 128510.0 + ], + "new_ns": [ + 101013.0, + 101455.5, + 101705.0, + 101200.0, + 100684.5, + 100508.0, + 99297.5, + 101186.5, + 100890.5, + 101116.0, + 101120.5 + ], + "median_speed": 1.2770036393844693, + "fast_mode_speed": 1.2770036393844693, + "checksums_match": true + } + }, + "controls": { + "property-heavy": { + "base_ns": [ + 98624.5, + 98886.0, + 103049.5, + 98892.5, + 98562.0, + 99010.5, + 148450.5, + 98658.0, + 98688.5 + ], + "new_ns": [ + 150656.5, + 96908.0, + 96232.5, + 96172.0, + 96474.0, + 96609.0, + 96370.5, + 150852.0, + 96324.0 + ], + "median_speed": 1.025001554823061, + "fast_mode_speed": 1.0250776949377662, + "checksums_match": true + }, + "int32array-traversal": { + "base_ns": [ + 49468.5, + 54987.0, + 49512.0, + 49578.5, + 49753.5, + 49437.5, + 49803.0, + 49412.5, + 49450.5 + ], + "new_ns": [ + 37405.0, + 66709.0, + 37436.0, + 37661.0, + 37417.5, + 67173.0, + 37536.0, + 37246.5, + 37479.0 + ], + "median_speed": 1.3210597934843513, + "fast_mode_speed": 1.3225771984186345, + "checksums_match": true + }, + "packed-array-traversal": { + "base_ns": [ + 48088.5, + 48567.0, + 48250.0, + 48017.5, + 47968.0, + 48144.0, + 48286.5, + 110812.0, + 111423.0 + ], + "new_ns": [ + 39759.5, + 39900.0, + 40148.5, + 40145.5, + 39922.5, + 40087.5, + 77187.0, + 77181.0, + 39937.0 + ], + "median_speed": 1.2036170876208294, + "fast_mode_speed": 1.2054986603901143, + "checksums_match": true + }, + "call-heavy": { + "base_ns": [ + 59794.5, + 105266.5, + 59370.5, + 60694.5, + 59662.5, + 59807.0, + 59407.5, + 59835.5, + 59556.5 + ], + "new_ns": [ + 41770.0, + 41635.5, + 41732.0, + 42405.0, + 44070.5, + 41310.0, + 41300.0, + 41555.0, + 41408.0 + ], + "median_speed": 1.4361422343913246, + "fast_mode_speed": 1.4345570486724069, + "checksums_match": true + }, + "float64array-traversal": { + "base_ns": [ + 1096087.5, + 1109460.0, + 1090380.5, + 1092694.0, + 1067363.0, + 1102924.0, + 1122704.0, + 1090801.5, + 1096589.0 + ], + "new_ns": [ + 1027350.5, + 1063833.5, + 1038933.5, + 1057446.5, + 1539293.0, + 1100576.0, + 1063299.5, + 1046204.0, + 1762201.5 + ], + "median_speed": 1.030836090866214, + "fast_mode_speed": 1.0365418014055559, + "checksums_match": true + }, + "arrays-typed": { + "base_ns": [ + 2974803.5, + 2966092.0, + 3011055.0, + 2992880.0, + 2985622.5, + 2958535.0, + 2977477.5, + 2992062.5, + 2967241.5 + ], + "new_ns": [ + 2956533.5, + 2942832.0, + 2955104.5, + 2925640.0, + 2925449.5, + 2986125.5, + 2948524.5, + 2942493.5, + 2980814.5 + ], + "median_speed": 1.009819487679346, + "fast_mode_speed": 1.009819487679346, + "checksums_match": true + }, + "generic-call-entry": { + "base_ns": [ + 32314.5, + 32176.5, + 32586.0, + 32660.5, + 70743.0, + 75214.5, + 71521.5, + 32213.0, + 32318.0 + ], + "new_ns": [ + 62168.0, + 32154.5, + 61640.0, + 61980.0, + 62243.0, + 61283.5, + 30006.5, + 30219.5, + 29970.0 + ], + "median_speed": 0.5317255052338721, + "fast_mode_speed": 1.0731660744528941, + "checksums_match": true + }, + "generic-call-fallback": { + "base_ns": [ + 9521545.5, + 9378445.0, + 9127386.5, + 9122762.5, + 9218784.0, + 9240363.0, + 9079020.0, + 9476228.5, + 9156024.5 + ], + "new_ns": [ + 9196247.0, + 9122232.0, + 9113840.0, + 9036587.0, + 9041973.0, + 9045365.5, + 9163495.0, + 9040979.5, + 9141391.0 + ], + "median_speed": 1.011514795080888, + "fast_mode_speed": 1.011514795080888, + "checksums_match": true + }, + "objects-polymorphic": { + "base_ns": [ + 24034730.0, + 24430464.5, + 24493621.5, + 23874224.5, + 24172948.5, + 23833836.5, + 24169369.5, + 24356837.0, + 33140320.0 + ], + "new_ns": [ + 23614740.0, + 23748308.0, + 24633324.5, + 23783489.5, + 23636898.0, + 23705968.5, + 25485659.0, + 32314781.5, + 24646027.0 + ], + "median_speed": 1.0163751832967993, + "fast_mode_speed": 1.0163751832967993, + "checksums_match": true + }, + "mixed-quotes": { + "base_ns": [ + 2547404.5, + 2528437.0, + 2502408.5, + 4013325.5, + 2518825.0, + 2506579.0, + 2682975.5, + 2556969.5, + 2535039.0 + ], + "new_ns": [ + 2597051.5, + 2560822.5, + 2529385.5, + 2614593.0, + 2714129.0, + 2647784.0, + 2654474.5, + 2638569.0, + 2550716.0 + ], + "median_speed": 0.9695730846062848, + "fast_mode_speed": 0.96831055540958, + "checksums_match": true + }, + "fibonacci-recursive": { + "base_ns": [ + 10044544.0, + 9849620.5, + 9888162.0, + 12593757.0, + 9855305.0, + 9839368.0, + 12742754.5, + 9847309.5, + 9857063.0 + ], + "new_ns": [ + 10160929.5, + 9899014.0, + 9847764.0, + 9856289.5, + 9834067.5, + 9875858.5, + 9842308.5, + 9839606.0, + 9834596.0 + ], + "median_speed": 1.000944275268985, + "fast_mode_speed": 1.000944275268985, + "checksums_match": true + }, + "calls-closures": { + "base_ns": [ + 3313062.5, + 3310130.0, + 3319059.0, + 3423131.0, + 3547198.0, + 4475368.5, + 4997349.0, + 3315583.5, + 3335263.0 + ], + "new_ns": [ + 3312646.5, + 3708894.5, + 3362192.5, + 3360146.5, + 3399654.5, + 3311014.5, + 3525387.0, + 3484235.5, + 5269595.0 + ], + "median_speed": 0.981059398830087, + "fast_mode_speed": 0.9840982796564312, + "checksums_match": true + } + } +} \ No newline at end of file diff --git a/benchmarks/results/roadmap-steady-diagnostic-82d3808-x86_64.json b/benchmarks/results/roadmap-steady-diagnostic-82d3808-x86_64.json new file mode 100644 index 000000000..346416c11 --- /dev/null +++ b/benchmarks/results/roadmap-steady-diagnostic-82d3808-x86_64.json @@ -0,0 +1,422 @@ +{ + "adversarial": { + "interpreter": [ + 0.936002, + 0.933591 + ], + "automatic": [ + 0.952012, + 0.9524565 + ], + "bun": [ + 0.123311, + 0.0355765 + ] + }, + "arrays-typed": { + "interpreter": [ + 4.134539, + 4.137498 + ], + "automatic": [ + 2.969289, + 2.9720165 + ], + "bun": [ + 0.13876, + 0.0446725 + ] + }, + "call-heavy": { + "interpreter": [ + 1.2778, + 1.2788855 + ], + "automatic": [ + 0.059685, + 0.059414 + ], + "bun": [ + 0.006336, + 0.006371 + ] + }, + "calls-closures": { + "interpreter": [ + 3.347925, + 3.3355985 + ], + "automatic": [ + 3.324853, + 3.3101155 + ], + "bun": [ + 0.184617, + 0.0823825 + ] + }, + "calls-recursion-closures": { + "interpreter": [ + 6.428335, + 6.405529 + ], + "automatic": [ + 6.415237, + 6.406498 + ], + "bun": [ + 0.206022, + 0.1252145 + ] + }, + "collections": { + "interpreter": [ + 1.631005, + 1.6327025 + ], + "automatic": [ + 1.638505, + 1.644423 + ], + "bun": [ + 0.135146, + 0.045662 + ] + }, + "exceptions-promises-async": { + "interpreter": [ + 2.332285, + 2.331115 + ], + "automatic": [ + 3.67627, + 3.6856845 + ], + "bun": [ + 0.282677, + 0.198623 + ] + }, + "fibonacci-iterative": { + "interpreter": [ + 35.46101, + 35.5026675 + ], + "automatic": [ + 1.003959, + 0.9942645 + ], + "bun": [ + 0.941968, + 0.159185 + ] + }, + "fibonacci-recursive": { + "interpreter": [ + 9.794635, + 9.7966025 + ], + "automatic": [ + 9.829058, + 9.8253595 + ], + "bun": [ + 0.539598, + 0.274126 + ] + }, + "float64-dense": { + "interpreter": [ + 2.297226, + 2.2795935 + ], + "automatic": [ + 0.128599, + 0.1291535 + ], + "bun": [ + 0.147671, + 0.0656965 + ] + }, + "float64array-traversal": { + "interpreter": [ + 1.532843, + 1.536856 + ], + "automatic": [ + 1.122251, + 1.1065345 + ], + "bun": [ + 0.00822, + 0.0082595 + ] + }, + "generic-call-entry": { + "interpreter": [ + 0.961134, + 0.961305 + ], + "automatic": [ + 0.032348, + 0.0321885 + ], + "bun": [ + 0.00493, + 0.0049685 + ] + }, + "generic-call-fallback": { + "interpreter": [ + 1.277341, + 1.280864 + ], + "automatic": [ + 9.117751, + 9.1241035 + ], + "bun": [ + 0.008123, + 0.008128 + ] + }, + "host-compute": { + "interpreter": [ + 2.164716, + 2.1640915 + ], + "automatic": [ + 0.068316, + 0.0714985 + ], + "bun": [ + 0.23893, + 0.016875 + ] + }, + "int32array-traversal": { + "interpreter": [ + 1.027674, + 1.034962 + ], + "automatic": [ + 0.049606, + 0.0496065 + ], + "bun": [ + 0.005769, + 0.0057285 + ] + }, + "json-codec": { + "interpreter": [ + 76.982995, + 77.0292145 + ], + "automatic": [ + 79.027859, + 78.986806 + ], + "bun": [ + 7.698369, + 7.5309665 + ] + }, + "map-set-bigint": { + "interpreter": [ + 16.504971, + 16.4096075 + ], + "automatic": [ + 17.758799, + 17.5385175 + ], + "bun": [ + 0.966065, + 0.7951905 + ] + }, + "mixed-quotes": { + "interpreter": [ + 2.76319, + 2.7641745 + ], + "automatic": [ + 2.47999, + 2.4808655 + ], + "bun": [ + 0.145391, + 0.0597135 + ] + }, + "numeric": { + "interpreter": [ + 0.531989, + 0.538418 + ], + "automatic": [ + 0.029369, + 0.029281 + ], + "bun": [ + 0.130152, + 0.0045095 + ] + }, + "objects-polymorphic": { + "interpreter": [ + 6.182188, + 6.189503 + ], + "automatic": [ + 23.811075, + 23.82931 + ], + "bun": [ + 0.198658, + 0.106358 + ] + }, + "packed-array-traversal": { + "interpreter": [ + 0.823972, + 0.826235 + ], + "automatic": [ + 0.047952, + 0.048104 + ], + "bun": [ + 0.004985, + 0.0049295 + ] + }, + "property-heavy": { + "interpreter": [ + 1.080826, + 1.066427 + ], + "automatic": [ + 0.097924, + 0.0984835 + ], + "bun": [ + 0.007811, + 0.007854 + ] + }, + "quickjs-bitops": { + "interpreter": [ + 1.193064, + 1.1930495 + ], + "automatic": [ + 0.061564, + 0.061514 + ], + "bun": [ + 0.147257, + 0.014029 + ] + }, + "quickjs-fibonacci": { + "interpreter": [ + 0.946607, + 0.9467165 + ], + "automatic": [ + 0.351308, + 0.3657225 + ], + "bun": [ + 0.143341, + 0.004444 + ] + }, + "quickjs-int-arith": { + "interpreter": [ + 5.801455, + 5.79549 + ], + "automatic": [ + 1.325363, + 1.3430865 + ], + "bun": [ + 0.131311, + 0.0476325 + ] + }, + "scalar-control-flow": { + "interpreter": [ + 2.131625, + 2.133814 + ], + "automatic": [ + 0.05449, + 0.0544175 + ], + "bun": [ + 0.090601, + 0.0103175 + ] + }, + "scalar-expressions": { + "interpreter": [ + 2.159235, + 2.1598165 + ], + "automatic": [ + 0.050452, + 0.050305 + ], + "bun": [ + 0.090121, + 0.009258 + ] + }, + "scalar-loop": { + "interpreter": [ + 0.54558, + 0.537524 + ], + "automatic": [ + 0.0293, + 0.0291975 + ], + "bun": [ + 0.132566, + 0.0044785 + ] + }, + "strings-json": { + "interpreter": [ + 1.926643, + 1.9159105 + ], + "automatic": [ + 2.001462, + 2.000367 + ], + "bun": [ + 0.165444, + 0.069683 + ] + }, + "strings-regexp": { + "interpreter": [ + 19.291352, + 19.2642415 + ], + "automatic": [ + 19.921122, + 19.89564 + ], + "bun": [ + 1.294154, + 0.973643 + ] + } +} \ No newline at end of file diff --git a/benchmarks/run.rs b/benchmarks/run.rs index 20fe492ac..9e9ef307a 100644 --- a/benchmarks/run.rs +++ b/benchmarks/run.rs @@ -17,6 +17,7 @@ use std::{ const BATCH_DRIVER: &str = include_str!("batch-driver.js"); const FIXED_WARMUP_BATCHES: usize = 64; +const DEFAULT_TIMED_BATCHES: usize = 16; const DEFAULT_WARMUPS: usize = 5; const DEFAULT_SAMPLES: usize = 30; @@ -152,6 +153,13 @@ const WORKLOADS: &[Workload] = &[ designated: false, file: "collections.js", }, + Workload { + name: "for-of-array", + suite: "rquickjs-jit focused", + group: "compute", + designated: false, + file: "for-of-array.js", + }, Workload { name: "strings-json", suite: "rquickjs-jit", @@ -215,6 +223,13 @@ const WORKLOADS: &[Workload] = &[ designated: false, file: "float64array-traversal.js", }, + Workload { + name: "typed-convert-traversal", + suite: "rquickjs-jit array traversal diagnostics", + group: "arrays-typed", + designated: false, + file: "typed-convert-traversal.js", + }, Workload { name: "objects-polymorphic", suite: "rquickjs-jit matrix", @@ -222,6 +237,13 @@ const WORKLOADS: &[Workload] = &[ designated: false, file: "objects-polymorphic.js", }, + Workload { + name: "property-polymorphic", + suite: "rquickjs-jit property diagnostics", + group: "objects-polymorphic", + designated: false, + file: "property-polymorphic.js", + }, Workload { name: "calls-recursion-closures", suite: "rquickjs-jit matrix", @@ -250,6 +272,20 @@ const WORKLOADS: &[Workload] = &[ designated: false, file: "exceptions-promises-async.js", }, + Workload { + name: "methods-dynamic", + suite: "rquickjs-jit matrix", + group: "methods-dynamic", + designated: false, + file: "methods-dynamic.js", + }, + Workload { + name: "exceptions-sync", + suite: "rquickjs-jit matrix", + group: "exceptions-sync", + designated: false, + file: "exceptions-sync.js", + }, ]; #[derive(Clone, Debug, Deserialize, Serialize)] @@ -468,12 +504,15 @@ fn worker(mode: &str, script: &str) -> Result<(), String> { let start = Instant::now(); let _first_checksum = timed_batch_count(&context, 1)?; phases.first_eval_ns = ns(start.elapsed()); + let timed_batches = timed_batch_count_config()?; let mut protocol = model::ProtocolEvidence { - name: "shared-js-fixed-warmup-v2".into(), + name: model::PROTOCOL_MULTIBATCH_V3.into(), script_sha256: sha256_file(script), driver_sha256: sha256("batch-driver.js"), warmup_batches: FIXED_WARMUP_BATCHES as u32, calls_per_batch: 10, + timed_batches: timed_batches as u32, + outlier_ratio: model::OUTLIER_RATIO, ..Default::default() }; for _ in 0..FIXED_WARMUP_BATCHES { @@ -483,10 +522,29 @@ fn worker(mode: &str, script: &str) -> Result<(), String> { jit.poll(); } } - let (fixed_elapsed, fixed_checksum, before_fixed, after_fixed) = - timed_batch_recorded(&context, 10, jit.as_ref())?; - protocol.fixed_metrics_before = before_fixed; - protocol.fixed_metrics_after = after_fixed; + // K consecutive timed batches continue the warmup regime exactly: + // checksum and JIT polling stay outside each timed region. The first + // one is the legacy v2 fixed batch, kept as a diagnostic. + protocol.timed_metrics_before = jit.as_ref().map(|j| metric_snapshot(&j.metrics())); + let mut fixed_checksum = String::new(); + for index in 0..timed_batches { + let (elapsed, checksum, before, after) = timed_batch_recorded(&context, 10, jit.as_ref())?; + if index == 0 { + protocol.fixed_batch_ns = Some(elapsed); + protocol.fixed_metrics_before = before; + protocol.fixed_metrics_after = after; + fixed_checksum = checksum; + } else if checksum != fixed_checksum { + return Err(format!("checksum changed at timed batch {index}")); + } + protocol.timed_batch_ns.push(elapsed); + if let Some(jit) = &jit { + jit.poll(); + } + } + protocol.timed_metrics_after = jit.as_ref().map(|j| metric_snapshot(&j.metrics())); + let (timed_median, outliers) = model::timed_batch_summary(&protocol.timed_batch_ns); + protocol.outlier_batches = outliers; let threshold_start = Instant::now(); let threshold_deadline = threshold_start + threshold_timeout(mode); let mut install_poll = 0u64; @@ -584,7 +642,7 @@ fn worker(mode: &str, script: &str) -> Result<(), String> { let abi = AbiInfo::linked().map_err(err)?; phases.total_ns = ns(total.elapsed()); let result = WorkerResult { - elapsed_ns: fixed_elapsed, + elapsed_ns: timed_median, checksum: fixed_checksum, native_entries: metrics.native_entries, native_acquisitions: metrics.native_acquisitions, @@ -731,6 +789,10 @@ fn sampling_config() -> Result { }) } +fn timed_batch_count_config() -> Result { + positive_env("JIT_BENCH_TIMED_BATCHES", DEFAULT_TIMED_BATCHES) +} + fn positive_env(name: &str, default: usize) -> Result { match env::var(name) { Ok(value) => value @@ -953,12 +1015,13 @@ fn child(executable: &Path, mode: &str, script: &Path) -> Result Result { - let bun = env::var("JIT_BENCH_BUN").unwrap_or_else(|_| "bun".into()); - let wrapper = format!( +/// Bun wrapper module shared by the file and eval launches. It never embeds +/// the script path, so its SHA-256 identifies the exact timing protocol. +fn bun_wrapper(timed_batches: usize) -> Result { + Ok(format!( r#" import {{ readFileSync }} from 'node:fs'; -(0,eval)(readFileSync(process.argv[1],'utf8')); +(0,eval)(readFileSync(process.argv.at(-1),'utf8')); (0,eval)({driver}); let first=benchmarkBatch(1); if(first&&typeof first.then==='function') await first; benchmarkChecksum(); const warmup_batch_ns=[]; @@ -969,35 +1032,103 @@ async function measure() {{ return [elapsed,benchmarkChecksum()]; }} for(let i=0;i<{warmups};i++) warmup_batch_ns.push((await measure())[0]); -const [elapsed_ns,checksum]=await measure(); -console.log(JSON.stringify({{elapsed_ns,checksum,warmup_batch_ns}})); +const timed_batch_ns=[]; +let checksum; +for(let i=0;i<{timed_batches};i++) {{ + const [elapsed,batch_checksum]=await measure(); + if(i===0) checksum=batch_checksum; + else if(batch_checksum!==checksum) throw new Error('checksum changed at timed batch '+i); + timed_batch_ns.push(elapsed); +}} +console.log(JSON.stringify({{checksum,warmup_batch_ns,timed_batch_ns}})); "#, driver = serde_json::to_string(BATCH_DRIVER).map_err(err)?, - warmups = FIXED_WARMUP_BATCHES - ); + warmups = FIXED_WARMUP_BATCHES, + )) +} + +/// A uniquely named wrapper file removed when the Bun child finishes. +struct TempWrapper(PathBuf); +impl TempWrapper { + fn create(contents: &str) -> Result { + use std::sync::atomic::{AtomicU64, Ordering}; + static NEXT: AtomicU64 = AtomicU64::new(0); + let nonce = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map(|d| d.as_nanos()) + .unwrap_or(0); + let path = env::temp_dir().join(format!( + "jit-bench-bun-{}-{}-{nonce}.mjs", + std::process::id(), + NEXT.fetch_add(1, Ordering::Relaxed) + )); + fs::OpenOptions::new() + .write(true) + .create_new(true) + .open(&path) + .and_then(|mut file| std::io::Write::write_all(&mut file, contents.as_bytes())) + .map_err(|error| format!("cannot write Bun wrapper {}: {error}", path.display()))?; + Ok(Self(path)) + } +} +impl Drop for TempWrapper { + fn drop(&mut self) { + let _ = fs::remove_file(&self.0); + } +} + +/// `file` (default) runs the wrapper from a temporary `.mjs` file. `eval` +/// reproduces the historical `bun -e` launch and is only a control. +fn bun_launch() -> Result<&'static str, String> { + match env::var("JIT_BENCH_BUN_LAUNCH").as_deref() { + Err(env::VarError::NotPresent) | Ok("file") => Ok("file"), + Ok("eval") => Ok("eval"), + Ok(other) => Err(format!( + "JIT_BENCH_BUN_LAUNCH must be file or eval, got {other}" + )), + Err(error) => Err(format!("cannot read JIT_BENCH_BUN_LAUNCH: {error}")), + } +} + +fn bun_child(script: &Path) -> Result { + let bun = env::var("JIT_BENCH_BUN").unwrap_or_else(|_| "bun".into()); + let timed_batches = timed_batch_count_config()?; + let launch = bun_launch()?; + let wrapper = bun_wrapper(timed_batches)?; + let temp = TempWrapper::create(&wrapper)?; + let wrapper_sha256 = sha256_file(&temp.0.to_string_lossy()); let started = Instant::now(); - let output = Command::new(&bun) - .args(["-e", &wrapper]) - .arg(script) - .output() - .map_err(err)?; + let mut cmd = Command::new(&bun); + if launch == "eval" { + cmd.args(["-e", &wrapper]); + } else { + cmd.arg(&temp.0); + } + let output = cmd.arg(script).output().map_err(err)?; + drop(temp); if !output.status.success() { return Err(String::from_utf8_lossy(&output.stderr).into_owned()); } #[derive(Deserialize)] struct BunOut { - elapsed_ns: u64, checksum: String, warmup_batch_ns: Vec, + timed_batch_ns: Vec, } let out: BunOut = serde_json::from_slice(&output.stdout).map_err(err)?; + if out.warmup_batch_ns.len() != FIXED_WARMUP_BATCHES + || out.timed_batch_ns.len() != timed_batches + { + return Err("Bun wrapper returned an incomplete batch sequence".into()); + } + let (elapsed_ns, outlier_batches) = model::timed_batch_summary(&out.timed_batch_ns); let phases = PhaseTiming { - steady_state_ns: out.elapsed_ns, + steady_state_ns: elapsed_ns, total_ns: ns(started.elapsed()), ..Default::default() }; Ok(WorkerResult { - elapsed_ns: out.elapsed_ns, + elapsed_ns, checksum: out.checksum, native_entries: 0, native_acquisitions: 0, @@ -1021,12 +1152,19 @@ console.log(JSON.stringify({{elapsed_ns,checksum,warmup_batch_ns}})); active_ir_bytes: 0, automatic_ready: true, protocol: model::ProtocolEvidence { - name: "shared-js-fixed-warmup-v2".into(), + name: model::PROTOCOL_MULTIBATCH_V3.into(), script_sha256: sha256_file(&script.to_string_lossy()), driver_sha256: sha256("batch-driver.js"), warmup_batches: FIXED_WARMUP_BATCHES as u32, calls_per_batch: 10, warmup_batch_ns: out.warmup_batch_ns, + timed_batches: timed_batches as u32, + fixed_batch_ns: out.timed_batch_ns.first().copied(), + timed_batch_ns: out.timed_batch_ns, + outlier_ratio: model::OUTLIER_RATIO, + outlier_batches, + bun_launch: Some(launch.into()), + bun_wrapper_sha256: Some(wrapper_sha256), ..Default::default() }, phases, @@ -1285,6 +1423,21 @@ mod tests { assert!(validate_sample("tier2", &r, true).is_err()); } #[test] + fn bun_wrapper_is_script_independent_and_times_k_batches() { + let wrapper = bun_wrapper(16).unwrap(); + assert!(wrapper.contains("process.argv.at(-1)")); + assert!(wrapper.contains(&format!("i<{FIXED_WARMUP_BATCHES};"))); + assert!(wrapper.contains("i<16;")); + assert_eq!(wrapper, bun_wrapper(16).unwrap()); + assert_ne!(wrapper, bun_wrapper(8).unwrap()); + let temp = TempWrapper::create(&wrapper).unwrap(); + let path = temp.0.clone(); + assert_eq!(path.extension().and_then(|e| e.to_str()), Some("mjs")); + assert_eq!(fs::read_to_string(&path).unwrap(), wrapper); + drop(temp); + assert!(!path.exists(), "Bun wrapper must be removed after use"); + } + #[test] fn rotation_interleaves_order() { let m = vec!["a".into(), "b".into(), "c".into()]; assert_eq!(rotated(&m, 1).collect::>(), vec!["b", "c", "a"]); @@ -1412,15 +1565,37 @@ mod tests { "scalar-expressions.js", "scalar-control-flow.js", "exceptions-promises-async.js", + "exceptions-sync.js", "packed-array-traversal.js", "int32array-traversal.js", "float64array-traversal.js", + "typed-convert-traversal.js", ] { let path = Path::new(env!("CARGO_MANIFEST_DIR")) .join("scripts") .join(script); let result = bun_child(&path).unwrap(); validate_sample("bun", &result, true).unwrap(); + let protocol = &result.protocol; + assert_eq!(protocol.name, model::PROTOCOL_MULTIBATCH_V3); + assert_eq!(protocol.warmup_batch_ns.len(), FIXED_WARMUP_BATCHES); + assert_eq!( + protocol.timed_batches as usize, + protocol.timed_batch_ns.len() + ); + assert_eq!( + protocol.fixed_batch_ns, + protocol.timed_batch_ns.first().copied() + ); + assert_eq!( + (result.elapsed_ns, protocol.outlier_batches), + model::timed_batch_summary(&protocol.timed_batch_ns) + ); + assert_eq!(protocol.bun_launch.as_deref(), Some("file")); + assert_eq!( + protocol.bun_wrapper_sha256.as_ref().map(String::len), + Some(64) + ); let sample = evidence_for_mode("bun", 0, result); assert!(sample.native_entries.is_none() && sample.native_exits.is_none()); assert!(sample.tier1_entries.is_none() && sample.tier2_entries.is_none()); @@ -1467,6 +1642,8 @@ mod tests { ("json-codec", "json-codec"), ("map-set-bigint", "map-set-bigint"), ("exceptions-promises-async", "exceptions-promises-async"), + ("methods-dynamic", "methods-dynamic"), + ("exceptions-sync", "exceptions-sync"), ] { let workload = WORKLOADS.iter().find(|w| w.name == name).unwrap(); assert_eq!(workload.group, group); @@ -1499,6 +1676,11 @@ mod tests { "arrays-typed", "float64array-traversal.js", ), + ( + "typed-convert-traversal", + "arrays-typed", + "typed-convert-traversal.js", + ), ] { let workload = WORKLOADS .iter() @@ -1523,6 +1705,7 @@ mod tests { ("packed-array-traversal.js", "number:41803450c0000000"), ("int32array-traversal.js", "number:41803450c0000000"), ("float64array-traversal.js", "number:416034cdc0000000"), + ("typed-convert-traversal.js", "number:416034cdc0000000"), ] { let runtime = Runtime::new().unwrap(); let context = Context::full(&runtime).unwrap(); @@ -1615,6 +1798,17 @@ mod tests { } std::thread::sleep(Duration::from_micros(50)); } + // Forced Tier 2 publishes the caller before its leaf's direct entry, + // so its first artifact goes through the generic CALL bridge. The + // timed Tier-2 trial (eight optimized caller entries) compares that + // artifact with baseline samples whose leaf calls may already use + // callback-free granted entries, and can demote it once; the + // replacement then captures the direct edge. Let the trial decide + // before settling so the steady window measures the final artifact. + for _ in 0..16 { + invoke_workload(&context).unwrap(); + jit.poll(); + } let (before, settled) = settle_compilation(&context, &jit, Duration::from_secs(60)).unwrap(); assert!( diff --git a/benchmarks/schema/jit-benchmark-v1.json b/benchmarks/schema/jit-benchmark-v1.json index 45cbc108b..6129e33a1 100644 --- a/benchmarks/schema/jit-benchmark-v1.json +++ b/benchmarks/schema/jit-benchmark-v1.json @@ -190,7 +190,7 @@ "additionalProperties": false, "required": ["name", "script_sha256", "driver_sha256", "warmup_batches", "calls_per_batch", "warmup_batch_ns", "fixed_metrics_before", "fixed_metrics_after", "readiness_diagnostic_ns"], "properties": { - "name": { "const": "shared-js-fixed-warmup-v2" }, + "name": { "enum": ["shared-js-fixed-warmup-v2", "shared-js-multibatch-v3"] }, "script_sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" }, "driver_sha256": { "type": "string", "pattern": "^[0-9a-f]{64}$" }, "warmup_batches": { "const": 64 }, @@ -198,7 +198,25 @@ "warmup_batch_ns": { "type": "array", "minItems": 64, "maxItems": 64, "items": { "type": "integer", "minimum": 0 } }, "fixed_metrics_before": { "type": ["object", "null"], "additionalProperties": { "type": "integer", "minimum": 0 } }, "fixed_metrics_after": { "type": ["object", "null"], "additionalProperties": { "type": "integer", "minimum": 0 } }, - "readiness_diagnostic_ns": { "type": ["integer", "null"], "minimum": 0 } + "readiness_diagnostic_ns": { "type": ["integer", "null"], "minimum": 0 }, + "timed_batches": { "type": "integer", "minimum": 0 }, + "timed_batch_ns": { "type": "array", "items": { "type": "integer", "minimum": 0 } }, + "fixed_batch_ns": { "type": ["integer", "null"], "minimum": 0 }, + "outlier_ratio": { "type": "integer", "minimum": 0 }, + "outlier_batches": { "type": "integer", "minimum": 0 }, + "timed_metrics_before": { "type": ["object", "null"], "additionalProperties": { "type": "integer", "minimum": 0 } }, + "timed_metrics_after": { "type": ["object", "null"], "additionalProperties": { "type": "integer", "minimum": 0 } }, + "bun_launch": { "enum": ["file", "eval", null] }, + "bun_wrapper_sha256": { "type": ["string", "null"], "pattern": "^[0-9a-f]{64}$" } + }, + "if": { "properties": { "name": { "const": "shared-js-multibatch-v3" } } }, + "then": { + "required": ["timed_batches", "timed_batch_ns", "fixed_batch_ns", "outlier_ratio", "outlier_batches"], + "properties": { + "timed_batches": { "minimum": 1 }, + "timed_batch_ns": { "minItems": 1 }, + "outlier_ratio": { "const": 5 } + } } }, "pair_index": { "type": "integer" }, diff --git a/benchmarks/scripts/exceptions-sync.js b/benchmarks/scripts/exceptions-sync.js new file mode 100644 index 000000000..2027c368c --- /dev/null +++ b/benchmarks/scripts/exceptions-sync.js @@ -0,0 +1,28 @@ +// Synchronous exception regions: a primitive throw caught every 16th +// iteration, a finally block on every iteration, and a helper-raised +// TypeError (property read of null) on one of every 256 iterations. +function workload(iterations, seed) { + const table = [{ w: 1 }, { w: 2 }, null, { w: 3 }]; + let caught = 0; + let total = seed | 0; + let finals = 0; + let missing = 0; + for (let i = 0; i < iterations; i++) { + try { + if ((i & 15) === 0) throw i; + total = (total + i) | 0; + } catch (value) { + caught += value; + } finally { + finals++; + } + if ((i & 63) === 1) { + try { + total = (total + table[(i >> 6) & 3].w) | 0; + } catch (error) { + missing++; + } + } + } + return caught + ':' + total + ':' + finals + ':' + missing; +} diff --git a/benchmarks/scripts/for-of-array.js b/benchmarks/scripts/for-of-array.js new file mode 100644 index 000000000..a795ca3db --- /dev/null +++ b/benchmarks/scripts/for-of-array.js @@ -0,0 +1,11 @@ +function workload(iterations) { + let values = globalThis.forOfArrayValues; + if (!values || values.length !== iterations) { + values = []; + for (let i = 0; i < iterations; i++) values.push((i * 17) & 1023); + globalThis.forOfArrayValues = values; + } + let sum = 0; + for (const value of values) sum += value; + return String(sum); +} diff --git a/benchmarks/scripts/methods-dynamic.js b/benchmarks/scripts/methods-dynamic.js new file mode 100644 index 000000000..ea95134e3 --- /dev/null +++ b/benchmarks/scripts/methods-dynamic.js @@ -0,0 +1,24 @@ +function Vec(x, y) { this.x = x; this.y = y; } +Vec.prototype.dot = function (other) { return this.x * other.x + this.y * other.y; }; +Vec.prototype.scale = function (k) { return new Vec(this.x * k, this.y * k); }; +function describe(value) { + if (typeof value === 'undefined') return 0; + if (typeof value === 'function') return 1; + if (value instanceof Vec) return 2; + if (typeof value !== 'object') return 3; + if ('kind' in value) return 4 + value.kind; + return 7; +} +function tally(counts, key) { counts[key] += 1; } +function workload(iterations, seed) { + const values = [undefined, describe, new Vec(1, 2), { kind: 1 }, { other: 1 }, 'text', { kind: 2 }]; + const counts = [0, 0, 0, 0, 0, 0, 0, 0]; + let sum = 0; + for (let i = 0; i < iterations; i++) { + const v = new Vec(i & 7, seed + 1); + const { x, y } = v.scale(2); + sum += v.dot({ x, y }) + (x ** 2) % 5; + tally(counts, describe(values[i % values.length])); + } + return sum + ':' + counts.join(','); +} diff --git a/benchmarks/scripts/property-polymorphic.js b/benchmarks/scripts/property-polymorphic.js new file mode 100644 index 000000000..41d10184f --- /dev/null +++ b/benchmarks/scripts/property-polymorphic.js @@ -0,0 +1,25 @@ +// Focused polymorphic-property kernel: the `point.x`/`point.y` sites each +// observe four stable own-data shapes, one of them prototype-less. The +// receivers are a benchmark argument so allocation stays outside the measured +// function; selecting them by branch (not by array element) keeps the hot +// sites on the optimized tier's bounded polymorphic inline cache. +globalThis.workloadArgument = { + a: { x: 1, y: 10 }, + b: { y: 20, x: 2, z: 5 }, + c: { tag: 7, x: 3, y: 30 }, + d: Object.assign(Object.create(null), { y: 40, w: 9, x: 4 }), +}; + +function workload(iterations, seed, shapes) { + let sum = seed; + let point = shapes.a; + for (let i = 0; i < iterations; i++) { + const k = i & 3; + if (k === 0) point = shapes.a; + else if (k === 1) point = shapes.b; + else if (k === 2) point = shapes.c; + else point = shapes.d; + sum = sum + point.x + point.y; + } + return sum; +} diff --git a/benchmarks/scripts/typed-convert-traversal.js b/benchmarks/scripts/typed-convert-traversal.js new file mode 100644 index 000000000..14a4cbc57 --- /dev/null +++ b/benchmarks/scripts/typed-convert-traversal.js @@ -0,0 +1,23 @@ +// Construct and populate both stable views before the harness starts timing. +// Unlike float64array-traversal, both typed arrays reach the hot loop as +// arguments of one callee, and only the source's length bounds the loop: the +// destination is a second, simultaneously live typed receiver. +const typedConvertInts = new Int32Array(2000); +const typedConvertFloats = new Float64Array(2000); +for (let i = 0; i < typedConvertInts.length; i++) { + typedConvertInts[i] = (i * 17) & 0xffff; +} +globalThis.workloadArgument = typedConvertInts; + +function convertAndSum(ints, floats) { + let mixed = 0.0; + for (let i = 0; i < ints.length; i++) { + floats[i] = ints[i] * 0.25 + 0.5; + mixed += floats[i]; + } + return mixed; +} + +function workload(iterations, seed, ints) { + return convertAndSum(ints, typedConvertFloats) + seed; +} diff --git a/benchmarks/suites.lock b/benchmarks/suites.lock index 8a1f6cadb..4ae0ae758 100644 --- a/benchmarks/suites.lock +++ b/benchmarks/suites.lock @@ -186,3 +186,12 @@ subdirectory = "benchmarks/scripts/generic-call-fallback.js" license = "workspace" upstream_sha256 = "not-applicable-local-workload" adapted_sha256 = "af1c8b8d3eb97968c1924927bdb5744c1678a2c9d82dec6f8b3f772451f12596" + +[[suite]] +name = "rquickjs-jit-matrix-methods-dynamic" +repository = "local" +commit = "local" +subdirectory = "benchmarks/scripts/methods-dynamic.js" +license = "workspace" +upstream_sha256 = "not-applicable-local-workload" +adapted_sha256 = "f358b173509f6aaa9f4f74b7d39b1e23391af24222829507e7da8a6347a4734d" diff --git a/benchmarks/summarize_paired.py b/benchmarks/summarize_paired.py index 96792ef7b..0670ee978 100644 --- a/benchmarks/summarize_paired.py +++ b/benchmarks/summarize_paired.py @@ -5,6 +5,7 @@ from pathlib import Path import random import statistics +from protocol_check import validate_protocol p=argparse.ArgumentParser() p.add_argument('directory') @@ -32,11 +33,7 @@ def interval(values): sample=json.loads((root/f'{workload}-{i}-{variant}-{mode}.json').read_text()) assert (sample['pair_index'],sample['variant'],sample['requested_mode']) == (i,variant,mode) protocol=sample['protocol'] - assert protocol['name'] == metadata['protocol'] - assert protocol['script_sha256'] == metadata['scripts'][workload] - assert protocol['calls_per_batch'] == 10 - assert protocol['warmup_batches'] == len(protocol['warmup_batch_ns']) == 64 - assert sample['elapsed_ns'] > 0 + validate_protocol(sample, metadata, workload, mode) checksums.add(sample['checksum']) drivers.add(protocol['driver_sha256']) if len(checksums) != 1 or len(drivers) != 1: @@ -50,13 +47,21 @@ def interval(values): speed,ci=interval([a['elapsed_ns']/b['elapsed_ns'] for a,b in zip(data[reference],values)]) comparisons['/'.join(reference)]=dict(speed=speed,ci95=ci) deltas=[v.get('fixed_deltas') for v in values] + multibatch=values[0]['protocol'].get('timed_batches',0) > 0 rows.append(dict(workload=workload,variant=variant,mode=mode, + protocol=metadata['protocol'], median_ns=statistics.median(v['elapsed_ns'] for v in values), + # Diagnostics only: the legacy single fixed batch and spikes + # among the timed batches. Spikes are counted, never dropped. + fixed_batch_median_ns=(statistics.median(v['protocol']['fixed_batch_ns'] for v in values) if multibatch else None), + outlier_batches=(sum(v['protocol']['outlier_batches'] for v in values) if multibatch else None), + outlier_processes=(sum(v['protocol']['outlier_batches'] > 0 for v in values) if multibatch else None), comparisons=comparisons, - compilation_quiet_samples=sum(v.get('fixed_compilation_quiet',False) for v in values), + compilation_quiet_samples=sum(v.get('timed_compilation_quiet' if multibatch else 'fixed_compilation_quiet',False) for v in values), native_entry_range=([min(d['native_entries'] for d in deltas),max(d['native_entries'] for d in deltas)] if all(d is not None for d in deltas) else None))) (root/'summary.json').write_text(json.dumps(rows,indent=2)+'\n') for row in rows: old=row['comparisons'][next(iter(metadata['binaries']))+'/automatic'] bun=row['comparisons'][next(iter(metadata['binaries']))+'/bun'] - print(f"{row['workload']:20} {row['variant']:10}/{row['mode']:11} {row['median_ns']/1e6:9.4f} ms; baseline {old['speed']:.3f}x {old['ci95']}; Bun {bun['speed']:.4f}x; quiet {row['compilation_quiet_samples']}/{metadata['pairs']}; entries {row['native_entry_range']}") + spikes='' if row['outlier_batches'] is None else f"; outlier batches {row['outlier_batches']} in {row['outlier_processes']} processes" + print(f"{row['workload']:20} {row['variant']:10}/{row['mode']:11} {row['median_ns']/1e6:9.4f} ms; baseline {old['speed']:.3f}x {old['ci95']}; Bun {bun['speed']:.4f}x; quiet {row['compilation_quiet_samples']}/{metadata['pairs']}; entries {row['native_entry_range']}{spikes}") diff --git a/core/src/runtime/jit.rs b/core/src/runtime/jit.rs index 7d35f9a8b..8d4323a37 100644 --- a/core/src/runtime/jit.rs +++ b/core/src/runtime/jit.rs @@ -327,6 +327,23 @@ pub unsafe trait JitBackend: Send + 'static { 0 } + /// Grants callback-free executions to the idle handle QuickJS just cached + /// after a `DONE` exit (ABI 1.25). Leaving `grant.budget` zero keeps every + /// call on the full callback path. A nonzero budget must point + /// `grant.state` at backend-owned state that stays at a fixed address until + /// `runtime_detach`; its epoch follows the [`Self::entry_cache_epoch`] + /// rules. Granted executions skip call feedback, hot probes, the epoch + /// query and `native_enter`, carry `JS_JIT_FRAME_FAST_ENTRY`, and report + /// `native_exit` only for non-`DONE` exits, tagged with + /// `JS_JIT_EXIT_FAST_UNPAIRED`. This query must not reenter JavaScript. + fn entry_fast_grant( + &mut self, + _id: u64, + _generation: u64, + _grant: &mut qjs::JSJitFastEntryGrant, + ) { + } + /// Records the exact boundary immediately before a published native entry. fn native_enter(&mut self, _id: u64, _generation: u64, _pc: u32) {} @@ -429,6 +446,21 @@ unsafe extern "C" fn entry_cache_epoch(opaque: *mut c_void) -> u64 { state.backend.entry_cache_epoch() } +unsafe extern "C" fn entry_fast_grant( + opaque: *mut c_void, + id: u64, + generation: u64, + grant: *mut qjs::JSJitFastEntryGrant, +) { + if grant.is_null() { + return; + } + let state = unsafe { BackendState::from_opaque(opaque) }; + state + .backend + .entry_fast_grant(id, generation, unsafe { &mut *grant }); +} + unsafe extern "C" fn native_enter(opaque: *mut c_void, id: u64, generation: u64, pc: u32) { let state = unsafe { BackendState::from_opaque(opaque) }; state.backend.native_enter(id, generation, pc); @@ -479,6 +511,7 @@ static BACKEND_VTABLE: qjs::JSJitBackendVTable = qjs::JSJitBackendVTable { native_exit: Some(native_exit), record_feedback: Some(record_feedback), entry_cache_epoch: Some(entry_cache_epoch), + entry_fast_grant: Some(entry_fast_grant), }; /// Owns the token for one backend allocation stored by the raw runtime. diff --git a/docs/PERFORMANCE_NEXT.md b/docs/PERFORMANCE_NEXT.md index c894ec736..6314914ee 100644 --- a/docs/PERFORMANCE_NEXT.md +++ b/docs/PERFORMANCE_NEXT.md @@ -1,5 +1,11 @@ # 下一阶段性能目标：以 Bun 为差距参照 +> **2026-09-27 补充：** [对齐 V8/Bun 路线图](PERFORMANCE_ROADMAP_V8_BUN.md) +> 发现 Bun worker（`bun -e`）的唯一计时批次会稳定遇到一次 JSC 侧停顿。 +> 双参数场景的 Bun 延迟因此被放大 1.2x–32x，本文及 README 中"快于 +> Bun"的标量场景读数都不成立。在协议修正（路线图 P0）之前，不要再 +> 引用这些读数设定 Bun 目标。 + 更新：2026-09-10。下一阶段以 **quickjs-jit automatic 相对 Bun 默认配置的性能差距**选择优化点和定义阶段目标。旧版 JIT 是回归基线，QuickJS 解释器是判断原生执行是否有收益的基线；二者都不能代替 Bun 这一目标参照。 以下优先级最初以运行时 `07535b1` 制定。本轮调用/属性/数组实现与验证见 diff --git a/docs/PERFORMANCE_ROADMAP_V8_BUN.md b/docs/PERFORMANCE_ROADMAP_V8_BUN.md new file mode 100644 index 000000000..49a307a98 --- /dev/null +++ b/docs/PERFORMANCE_ROADMAP_V8_BUN.md @@ -0,0 +1,433 @@ +# 对齐 V8 / Bun 的 JIT 性能路线图（2026-09-27 分析） + +本文件接续 [M3](M3.md)、[下一阶段目标](PERFORMANCE_NEXT.md)、 +[边界优化记录](BUN_BOUNDARIES_20260909.md) 和 +[semantic SSA 进展](SEMANTIC_SSA_PROGRESS.md)，基于 `82d3808`（0.12.9） +做一次**差距归因**，并给出分阶段、可验收的规划。 + +本文件**不包含任何运行时优化**，README 矩阵保持不变。文中所有实验 +数值都是单机诊断（i7-13700KF，CPU 8，`powersave`，Bun 1.4.0，3–7 个 +进程取中位数），不是 AGENTS.md 要求的发布级配对证据，不能当作优化 +收益引用；每个阶段落地时仍需完整三引擎矩阵。 + +## 1. 结论摘要 + +1. **测量协议有系统偏差，当前矩阵高估了 quickjs-jit 相对 Bun 的位置。** + Bun worker 用 `bun -e` 执行，第 64 个批次（恰好是协议唯一计时的 + 那一批）会稳定出现一次 JSC 侧的一次性停顿（约 0.1–0.2 ms）。 + 所有无 `workloadArgument` 的双参数场景的 Bun 延迟都被放大 + 1.2x–32x。README 中 7 个"快于 Bun"的场景，按稳态批次口径**全部 + 慢于 Bun**（见第 2 节）。先修协议，否则后续所有"对齐 Bun"的 + 目标都建立在错误基线上。 +2. **按稳态口径，quickjs-jit 在全部 30 个场景都慢于 Bun。** 最好的 + 标量循环约为 Bun 的 0.16x–0.5x；调用、属性、数组约 0.1x；闭包、 + 递归、集合、异常、多态对象约 0.01x–0.05x；最差的 + `generic-call-fallback` 为 0.0009x，而且只有解释器的 0.14x。 +3. **最大的单点损耗是 JIT 自身的防御性开销，不是缺少优化。** + - `qjsjit_validate_helper_frame_state`：每次 helper 调用都做 + ~30 项身份/ABI 校验，外加 `qjsjit_valid_pc` 的**线性字节码扫描** + （回边之后从字节码开头重扫）。它在 `objects-polymorphic` 中占 + 63% 自身时间，在真实的 `generic-call-fallback` 中占 61%。 + 把 PC 校验变成 O(1) 的诊断实验分别带来 **1.71x** 和 **1.29x**， + 其余场景持平，checksum 一致。 + - 每次调用的簿记：`record_hot`/`acquire_entry`/`native_enter`/ + `native_exit` 四次跨 C/Rust 回调、两次 `Instant::now()`、哈希表 + 查找、`record_benefit`，以及每 32 次调用一次的全量 `maintenance`。 + `mixed-quotes` 的排序回调中 `maintenance` 占 37.5%。 +4. **覆盖面比 V8 窄一个数量级。** 约 255 个 opcode 中有 124 个被 + policy 拒绝，包括全部闭包变量访问、`push_this`、`throw`、`typeof`、 + `in`/`instanceof`、`for_of`/`for_in`、`fclosure`、try/catch、 + `arguments`。含其中任一 opcode 的函数整体停在解释器。V8 Sparkplug + 和 JSC Baseline 覆盖全部字节码。 +5. **没有通用的原生到原生调用约定。** 除了受限的 Int32/Bool 叶函数 + 直调和有界内联，所有调用都经过 + `JS_JitHelperCall → JS_Call → JS_CallInternal → 回调 → 原生入口`。 + 递归因此被 policy 禁止进入原生层；强制进入 Tier 2 后 + `fibonacci-recursive` 反而比解释器慢 7.8x。 +6. 在这些之后，才轮到 V8 TurboFan/Maglev、JSC DFG/FTL 级别的优化 + 编译能力：冗余 guard 与检查消除、LICM、load/store 转发、边界检查 + 消除、多态内联缓存、逃逸分析和分配下沉、通用内联。 + `property-heavy`、`call-heavy`、`int32array-traversal` 已经在原生层， + 但每次迭代仍比 Bun 慢约 10x，差距就在这里。 + +## 2. 测量协议缺陷与修正 + +`benchmarks/run.rs` 的 `bun_child` 用 `bun -e