diff --git a/experiments/2026-10-24-atom-batch-interim.md b/experiments/2026-10-24-atom-batch-interim.md new file mode 100644 index 00000000..57494140 --- /dev/null +++ b/experiments/2026-10-24-atom-batch-interim.md @@ -0,0 +1,291 @@ +# Interim ATOM batch serving evidence (#270) + +- Date: 2026-10-24 +- Status: incomplete experiment; no default change proposed +- Related: ai-computing/aicomp #201, #225, #265, #270, #271; RBLN-SW/optimum-rbln #827 + +## Method + +Four models: Qwen3-0.6B, Qwen3-4B, KONI-8B, Gemma3-1B. ATOM compiled B1/B4 with five KV blocks, context9216; KONI uses TP2, others TP1. Native ATOM, RNGD prefill with whole KV transfer, and RNGD prefill with layer-stream transfer. Input128/512/2048/8192, output32/128, concurrency16, caches off, one measured batch per condition. Each successful mode has8 warmups,128 full-token checks and128 timed SSE text checks. Loading and correctness checks are excluded from serving timing. EOS is deliberately ignored for the fixed-output workload; full-token and through-EOS comparisons are different criteria. + +SDK: rebel-compiler0.11.0, optimum-rbln0.11.0.post1, torch2.11.0+cpu, Transformers5.8.1; RNGD worker uses torch2.10.0+cpu, Transformers5.19.0, furiosa-torch2026.4.0; both interpreters are Python3.10.12. These package versions were queried separately during the corrected Gemma run, rather than assumed to be shared. The corrected-Gemma source manifest records104 on-disk Python/script files, with no missing requested files. This is not proof of every imported dependency or of historical in-memory code. Historical per-run dependency identity, harness packaging and driver inventory remain publication work. Experimental serving modules are on the PR266 dependency stack, not on current main. This document does not make this checkout a runnable serving implementation. + +## Original inventory (before the corrected Gemma follow-up) + +| Model | Compiled batch | Route | Audit | 8K/128 tok/s | +|---|---:|---|---|---:| +| qwen06b | 1 | native | within_route_pass | 50.3245 | +| qwen06b | 1 | whole | within_route_pass | 33.1562 | +| qwen06b | 1 | stream | within_route_pass | 43.1570 | +| qwen06b | 4 | native | within_route_pass | 63.6370 | +| qwen06b | 4 | whole | within_route_pass | 58.4628 | +| qwen06b | 4 | stream | within_route_pass | 45.4781 | +| qwen4b | 1 | native | within_route_pass | 12.2832 | +| qwen4b | 1 | whole | within_route_pass | 12.6169 | +| qwen4b | 1 | stream | within_route_pass | 14.5038 | +| qwen4b | 4 | native | audit_failed | 18.2422 | +| qwen4b | 4 | whole | not_collected | — | +| qwen4b | 4 | stream | not_collected | — | +| koni | 1 | native | within_route_pass | 15.0183 | +| koni | 1 | whole | within_route_pass | 13.8944 | +| koni | 1 | stream | within_route_pass | 17.2012 | +| koni | 4 | native | within_route_pass | 23.8487 | +| koni | 4 | whole | within_route_pass | 31.8286 | +| koni | 4 | stream | within_route_pass | 38.5542 | +| gemma | 1 | native | within_route_pass; INVALID causal baseline | 57.9914 | +| gemma | 1 | whole | within_route_pass | 40.3069 | +| gemma | 1 | stream | within_route_pass | 43.6196 | +| gemma | 4 | native | not_collected | — | +| gemma | 4 | whole | not_collected | — | +| gemma | 4 | stream | not_collected | — | + +The inventory JSON records source result hashes and results of rerunning the HTTP, geometry and lane audit against the local raw files. Within-route success does not establish native/PD parity, answer quality, device utilization, or statistical improvement. Failed native Qwen3-4B B4 blocks its whole/stream sequence; their absence is not a pass. Corrected Gemma B4 native has now passed 264 responses; B4 whole and stream are pending. Its separate corrected-artifact128-token causal test passed all4 lanes×52 slots, including prefix-changing positive controls for every slot. This does not substitute for the B4 HTTP matrix. + +## Findings that constrain promotion + +- Qwen3-4B B4: repeated same-prompt lane outputs diverge. Direct SDK replay reproduces lane-dependent BF16 argmax ties without custom AtomBatcher or KV transport. Do not relax the parity gate or attribute this to PD overhead. +- Qwen3-0.6B B4: stream is slower than whole for8K/128 (45.4781 vs58.4628tok/s); repeat and stage attribution are needed before any default change. +- Gemma original k5 native: suffix-only prompt changes alter prefix KV in50/52 slots. Old corrected k3 control preserves52/52; a fresh k5 export with the existing sliding/global causal correction also preserves52/52. Original and corrected k5 tests used different assigned ATOMs. Raw per-slot files and patch hashes are included. +- Corrected Gemma B1 native subsequently passed264 HTTP responses, matches old corrected k3 native full8/8, and previous whole PD full6/8 / through-EOS8/8. Corrected whole subsequently passed264 responses: native/whole full6/8 and through-EOS8/8 on the same ATOM and corrected artifact. Whole references remain full8/8 identical to prior whole. At8K/128, native57.5700 vs whole39.9662tok/s; median response completion18.8013 vs27.4025s. Whole startup837.6305s is excluded. Corrected stream also passed264 responses; whole/stream full8/8 and native/stream through-EOS8/8 (full6/8). Stream8K/128 throughput43.2203tok/s, median completion25.1978s: +8.1422% throughput over whole, but below native. One timed batch; corrected B1 controller and cleanup exited0. These completed follow-up comparisons are separate from the original inventory so the invalid baseline is not silently replaced. + +## Remaining work + +Complete the four-model B4 fault-isolation matrix and operational generalization; diagnose/fix Qwen3-4B B4 numerical differences; repeat Qwen3-0.6B B4 performance with producer/import/decode timing; perform batched cancellation/reuse checks; package exact commands/harness source and SDK/driver identities; obtain engine-owned compile/cache handling under#271. No completion or universal performance claim is made. + +Generated with Codex; evidence reviewed against local result files. + +## Native B1/B4 output comparability + +Direct source-result comparison: Qwen3-0.6B full6/8, Qwen3-4B4/8, KONI1/8, corrected Gemma4/8. At8K/128 only Qwen3-0.6B and corrected Gemma full outputs match across batch sizes. Qwen3-4B differs first at output index12 for both8K lengths, independently of its within-B4 repeated-output failure at other conditions. KONI8K/128 also differs. Do not present all four raw throughput changes as equivalent-output speedups. See atom270-four-model-native-batch-comparison.json. + +## Corrected Gemma B4 native follow-up + +B4 native passed 264 responses. Successful decode-call histogram is `[0, 664, 32, 32, 5008]`; this establishes actual four-lane calls, not device utilization. B1/B4 references match in 4/8 full sequences and 8/8 through EOS. At 8K input / 128 output the full tokens match: throughput is 57.5700 → 95.1626 tok/s, median completion 18.8013 → 14.7210 s. Estimated TPOT worsens from 11.2129 to 36.0703 ms. B1 used ATOM0 and B4 ATOM2, with one measured batch each, so this is not an isolated same-card batch-size experiment. + +## Causal regression reproduction + +The included diagnostic scripts require the experimental AtomRblnEngine/AtomBatcher from the PR #266 dependency stack, the installed SDK versions above, one assigned ATOM with a valid existing RSD mapping, and the cached model weights. The fixture must be a torch file containing `ids`: a 128-token Gemma prompt. The original fixture and full HTTP harness packaging remain publication work. Commands below run from the evidence directory in that environment; no shared SDK files are edited. + +```bash +# Existing uncorrected B1 artifact: expected causal failure; retain its JSON. +python container_bootstrap270.py check_gemma270_causality.py --cache-dir atom270-artifacts --kv-blocks 5 --exports prompt128.pt --output original-k5.json +# Fresh corrected B1 export, then reload in a separate process. +python container_bootstrap270.py build_gemma270_causal.py +python container_bootstrap270.py check_gemma270_causality.py --cache-dir atom270-artifacts-causal --kv-blocks 5 --exports prompt128.pt --output corrected-k5.json +# B4 export and all four lanes, including prefix-changing positive controls. +python container_bootstrap270.py build_gemma270_causal_b4.py +python container_bootstrap270.py check_gemma270_causality_b4.py --cache-dir atom270-artifacts-causal --kv-blocks 5 --exports prompt128.pt --output corrected-b4-k5.json +``` + +Export scripts reject existing artifact directories; diagnostics reject output overwrites and never compile missing models. These are research reproductions, not the engine-owned compatibility implementation requested in #271. + + +## Corrected Gemma B4 whole follow-up: correctness passes, occupancy fails + +All 264 responses passed within-route HTTP checks, but the full B4 audit failed: successful decode-call lane histogram `[0, 14313, 3132, 93, 0]` contains no four-lane calls. Native and whole ran on the same physical ATOM2 and corrected artifact. Their full reference sequences match in 6/8 conditions and through EOS in 8/8. At 8K/128, full tokens match; whole reaches 77.0898 tok/s, median first SSE 13.3368 s and completion 15.7735 s, versus native 95.1626 tok/s and completion 14.7210 s. This is one timed batch per condition, with an extra RNGD for whole; no statistical or equal-resource speedup claim is supported. + +The strict queue stopped before stream and cleaned up its Pod. The occupancy failure remains a failure; HTTP correctness does not validate B4 utilization. The separate host-timeline pair below supports producer-limited readiness. The same-Pod width ABBA below then tests a short-input correction. + +For hybrid Gemma B4, `_import_progress` deliberately defers device import until the producer completes: idle compiled lanes can otherwise overwrite local sliding-window KV. Host layer streaming therefore does not establish early ATOM import overlap. Removing this guard requires an ownership solution and cancellation/reuse validation, not just a performance change. See the two new evidence JSON files. + + +## Gemma B4 host timeline pair + +The diagnostic pair completed on the same physical ATOM2. Both routes passed 264 HTTP responses; all eight full reference sequences match. Neither produced a successful four-active-lane decode call: whole histogram `[0,13952,3299,102,0]`, stream `[0,13676,3452,92,0]`. These occupancy failures remain failures even though HTTP correctness passed. + +| Host observation | Whole | Stream | +|---|---:|---:| +| Backlogged KV completion gap, median | 1.5548 s | 1.5085 s | +| Decode call, median | 13.1467 ms | 12.8802 ms | +| Ready prefill left at decode entry | 0 / 17353 | 0 / 17220 | +| 8K/128 output throughput | 76.2890 tok/s | 77.8249 tok/s | +| 8K/128 median completion | 15.9551 s | 15.6698 s | + +The observations support producer-limited readiness, rather than a universal scheduler cap below B4. Completion gaps include host delivery and Python timing; they are not pure NPU compute. Queue states are snapshots, and timing sums overlap. The roughly 2% throughput difference is one timed batch per route, not a stable speedup. All-global synthetic CPU controls also admit four lanes when all four results are ready, but do not emulate Gemma sliding-window hardware. + +For whole mode, backlogged completion-gap medians remain about 1.55 s at every input length from 128 to 8192. The short-input-only width1024 follow-up and same-Pod ABBA below completed against pinned width8192 outputs. It does not replace the wide path for long inputs or establish heldout quality. Earlier Gemma strict-format quality tests failed in both widths; zero regressions there is not quality acceptance. + +Whole initialization took 840.58 s and subsequent stream initialization 18.40 s. This warm-state difference is not isolated. Do not assume every startup takes 14 minutes or attribute the difference to streaming alone. + +Raw HTTP results and host-timeline logs are registered as research-hub attachments, identified by `raw-evidence-manifest.json` (research event `578a57666040434b90f5b76a51871973`). Remote synchronization has not been independently checked. Restore those exact files into evidence270 and verify their hashes before recomputing host summaries (use a new output filename; overwrites are rejected): + +```bash +python analyze_timeline270.py atom270-timeline-gemma-b4-whole.server0.log --output whole-recomputed.json +python analyze_timeline270.py atom270-timeline-gemma-b4-stream.server0.log --output stream-recomputed.json +python selfcheck_timeline270.py +python selfcheck_analyze_timeline270.py +``` + +The probe entrypoint depends on the original HTTP harness and PR266 runtime stack; it is not a standalone server in this branch. Raw-result HTTP audits use the included audit_http265.audit_result with backend atom, expected_slots=4, expected_atom_devices=1. Neither raw timing evidence nor the synthetic tests prove device DMA overlap. + + +## Same-Pod Gemma B4 width ABBA: measured short-input improvement + +Four whole-transfer runs used width8192 → 1024 → 1024 → 8192 in one Pod on ATOM3. All 528 responses passed (16 warmups, 256 full-token checks, 256 timed SSE responses). Independent audit rechecked the physical binding, compiled geometry, identical prompts/reference outputs, timing arithmetic and server settings differing only in producer width. Each condition has two timed batches per width. + +| Input / output tokens | Width8192 tok/s | Width1024 tok/s | Ratio | Median completion, wide → narrow | +|---|---:|---:|---:|---:| +| 128 / 32 | 20.199 | 40.348 | 1.9975× | 13.661 → 6.958 s | +| 128 / 128 | 76.874 | 145.839 | 1.8971× | 15.004 → 8.398 s | +| 512 / 32 | 19.943 | 40.157 | 2.0136× | 13.824 → 6.960 s | +| 512 / 128 | 76.522 | 144.942 | 1.8941× | 15.119 → 8.496 s | + +Values are medians of the two measured batches per width. Backlogged KV completion medians in run order were 1.5651, 0.7661, 0.7695 and 1.5618 s. Successful decode calls reached at most two active lanes in the wide arms and three in the narrow arms. None reached four: the B4 occupancy criterion remains failed, despite the measured serving gain and exact outputs. + +This supports selecting a smaller producer shape for short inputs in this configuration. It does not establish a production default, arbitrary-prompt quality, long-input support, GPU/other-model improvement, confidence intervals or device DMA overlap. Inputs above1024 require a separate wider path; the experiments here used only128 and512. Existing strict Gemma output-format quality failures remain unresolved. Qwen3-4B B4 numerical divergence also remains unresolved and is not waived by this Gemma result. + +Per-run host summaries and the terminal queue are included. Raw results/logs remain in the separate research archive above, as required by the contribution guide. After restoring the hash-matched raw files, reproduce the full audit from evidence270 with a fresh output filename: + +```bash +python selfcheck_atom270_abba.py +python audit_atom270_abba.py --queue atom270-abba-gemma-outer.json --output abba-recomputed.json +``` + +The self-check uses small synthetic fixtures and rejects eight corruption types; it runs without archived data and is not another hardware measurement. The CPU readiness control requires the PR266 test/runtime stack and tests synthetic all-global KV, not Gemma sliding-window hardware. The four-model hardware cancellation/reuse/producer-death matrix is running separately; completed cases and their limits follow below. + + +## B4 fault isolation: all four hardware cases complete + +Gemma3-1B, Qwen3-0.6B, Qwen3-4B and KONI-8B each completed one direct-batcher control/cancel-reuse/producer-death sequence. Three survivor requests each produced512tokens exactly matching their same-lane controls after either fault. The reused fourth lane produced16tokens matching its control. Owners drained idle/nonpoisoned; all slots were returned. These are hardware fault-isolation results, not HTTP disconnect, semantic-quality or throughput acceptance. + +| Model | Control active-lane histogram (0–4) | Cancel/reuse histogram | Producer-death histogram | +|---|---|---|---| +| Gemma3-1B | [0,220,220,276,15] | [0,221,221,275,15] | [0,212,212,299,0] | +| Qwen3-0.6B | [0,168,168,328,15] | [0,158,158,338,15] | [0,167,167,344,0] | +| Qwen3-4B | [0,120,120,376,15] | [0,110,110,386,15] | [0,108,108,403,0] | +| KONI-8B (TP2) | [0,174,174,322,15] | [0,164,164,332,15] | [0,171,171,340,0] | + +The longer512-output workload achieved real four-active-lane calls in control/reuse. This does not erase the short-output occupancy failures above. In all four models, both injected faults occurred with host-pending victim KV and **zero victim device-import spans**. At this stage hardware safety after partial victim KV import was unproven; the follow-up below covers the three dense models. Gemma's multilane hybrid path deliberately defers import; its guard has not been removed. + +The terminal queue and raw arrays were independently audited: eight fault cases,12,288 survivor-token comparisons and64 replacement-token comparisons match; all four owned Pods were cleaned up successfully. The separate Qwen3-4B B4 lane-dependent numerical divergence remains unresolved despite the same-lane fault test passing. Its three survivor control lanes happened to agree for this fixture on the RNGD-prefill route; that does not waive independent native-prefill failures on other prompts. Compact audits and source driver are included; raw token arrays are archived separately with full hashes and research event IDs in fault-evidence-manifest.json. Remote archive synchronization has not been independently verified. The driver requires the PR266 runtime and original hardware harness; it is not standalone on main. + +CPU coverage additionally verifies **actual partial writes** using the real AtomBatcher with the existing fake SDK: three active survivors, fourth-lane cancellation/late completion or producer error, same-lane reuse, and no stale overwrite of replacement KV. Survivor24-token outputs equal a no-fault control. New cases2passed; with related batching/hybrid/stream checks53passed6.19s on Torch2.13/Transformers5.15. The initial new tests failed because the test assumed an admitted attribute exists before admission; the test now follows the runtime's getattr default. No runtime change was made. This is CPU evidence, not NPU hardware or full-suite acceptance. + +Run the published test against the recorded PR266 checkout (eaf7daa219a6cea68db4ec5fc08850edc848b271): + +```bash +PYTHONPATH="$PR266/opt_prime:$PR266/opt_prime/tests" "$TEST_PYTHON" -m pytest experiments/evidence270/test_atom270_dense_partial.py "$PR266/opt_prime/tests/test_atom_batching.py" "$PR266/opt_prime/tests/test_atom_hybrid.py" "$PR266/opt_prime/tests/test_atom_stream_progress.py" -q +``` + + +## Qwen3-0.6B long-input whole/stream ABBA (2026-10-25) + +Repeated whole/stream/stream/whole in one Pod on ATOM2 with width8192,B4/TP1/k5,concurrency16,cacheoff,inputs2048/8192 and outputs32/128. All528responses passed (16warmups,256full-token checks,256timed SSE responses). Independent audit rechecked full prompts/references, config equality except the stream flag, compiled geometry, actual device binding, raw hashes and timing arithmetic. The queue completed and the owned Pod was cleaned up. Two timed batches per route/condition; no confidence interval or arbitrary-prompt quality claim. + +| Input/output | Whole tok/s | Stream tok/s | Stream/whole | Median E2E whole/stream s | +|---|---:|---:|---:|---:| +|2048/32|23.579|30.152|1.2788|12.111/9.423| +|2048/128|93.515|113.257|1.2111|12.645/11.222| +|8192/32|20.441|16.196|0.7923|15.826/19.796| +|8192/128|53.113|53.354|1.0045|24.817/26.332| + +Values are medians of two batches. Streaming improves2K throughput21–28percent but reduces8K/32throughput20.8percent and increases completion latency25.1percent. Both stream samples are below both whole samples for8K/32. At8K/128throughput is effectively unchanged(+0.45percent), with6.1percent higher completion latency. The earlier single-run22percent8K/128throughput penalty did not reproduce here. This supports condition-specific investigation, not globally enabling/disabling streaming. + +Import host totals inABBAorder:76.672,95.628,98.060,78.305s, with132,3696,3696,132calls respectively. Successful4-active-lane calls:639,1143,1065,619. Import includes torch stacking, dtype conversion, staging assignment and SDK calls; these counters do not isolate network/DMA or establish a causal bottleneck. Extra component instrumentation is a separate follow-up with no result claimed here. Startup14.05–14.30s is excluded from steady-state metrics and does not independently establish compiler-cache hits. + +The first attempt's whole arm passed132responses but its controller rejected missing binding logs caused by the older bootstrap configuration; it cleaned up its Pod. Those data were preserved separately and excluded from this ABBA. All four accepted arms were rerun after fixing the bootstrap and adding a preallocation guard. No production engine/default changed. + +Compact audit: atom270-qwen06-regression-retry1-audit.json. Eight raw results/logs remain outside Git; qwen06-regression-evidence-manifest.json records full hashes and research event ID. Remote archive synchronization has not been independently verified. For each restored raw result, use the included audit_http265.audit_result with backend atom, mode whole/stream, lengths=(2048,8192), expected_slots=4 and expected_atom_devices=1; compare complete reference dictionaries across arms and recompute median summaries from batches. This evidence does not waive the remaining Qwen4 native numerics, Gemma quality or remaining hybrid partial-import limits. + + +## Dense B4 faults after actual victim import (2026-10-25) + +The earlier hardware triggers occurred before device import. The strict follow-up driver waits for a positive victim progress-span count after the real import call, with the producer still pending, three active survivors, and the victim not admitted. No runtime or SDK behavior is patched. A separate driver preserves the original test and evidence. Gemma multilane hybrid import remains deliberately deferred and is excluded from this trigger. + +All three dense models passed cancellation/reuse and producer-death on RNGD→ATOM, B4/k5/context9216/producer-width8192. Survivors use512-input/512-output tokens, victim8192/16, replacement512/16. KONI usesTP2; Qwen modelsTP1. Runtime source hashes are included in the compact audit; the PR266 runtime and the same ATOM/RNGD SDK environment documented above are required. + +| Model | Victim spans at cancel/death | Active survivor tokens at cancel | At producer death | +|---|---|---|---| +| Qwen3-0.6B | 1 / 1 | 206,109,11 | 192,94,8 | +| Qwen3-4B | 1 / 1 | 115,57,6 | 108,55,5 | +| KONI-8B TP2 | 1 / 1 | 178,91,7 | 182,90,6 | + +Independent raw-array audit confirms9216 survivor-token comparisons and48 replacement-token comparisons exact against same-lane controls. Both controllers exited0; all three owned Pods were cleaned up and the task namespace was empty. The audit rejects the previous zero-import result. The test proves isolation for these fault points and fixtures; it does not prove correctness for every import boundary, native-reference parity, semantic quality, or a throughput improvement. Qwen4 native lane-dependent numerics remain unresolved. Hardware import is observed through the real SDK import call returning and the engine progress counter; no separate physical DMA trace is claimed. + +Reproduce using the existing PR266 hardware harness, an already compiled B4/k5/context9216 artifact, and the allocated task devices. MODEL/FAMILY/TP are Qwen/Qwen3-0.6B/qwen/1, Qwen/Qwen3-4B/qwen/1 or KISTI-KONI/KONI-Llama3.1-8B-Instruct-20241024/koni/2. Supply site paths through local variables: + +```bash +"$ATOM_PYTHON" container_bootstrap270.py check_atom270_b4_imported_faults.py --model "$MODEL" --family "$FAMILY" --cache-dir "$CACHE_DIR" --tensor-parallel-size "$TP" --worker-python "$WORKER_PYTHON" --output "$RAW_RESULT" +python3 experiments/evidence270/audit_atom270_imported_faults.py "$RAW_RESULT" +``` + +Raw token arrays stay outside Git; imported-fault-evidence-manifest.json records hashes and the research event ID. Remote archive synchronization is unverified. The included driver requires the existing bootstrap, atom_server270_http helpers, streaming factory and task worker environment; it is not standalone on main. + + +## Same-card Qwen3-4B lane diagnosis (2026-10-25) + +The same-Pod normal/fixed-prefill/fixed-prefill/normal ABBA completed128 cases. Each arm checks all16 decode-lane/KV-block placements for free32-token and forced-prefix14-token outputs. All four arms have identical tokens and recorded logits; only lane0 matches the saved reference (4/16free cases per arm). At forced index13 lane0 selects369 and lanes1–3 select264. Four matching binding records remove the earlier cross-card confound. Child/copy/cleanup all0; no task Pods remain. + +The first attempt failed SDK INIT_INTERNAL when loading a second engine in one process; the retry uses fresh processes in the same Pod. Both failure and accepted retry were recorded. This result plus the earlier direct SDK replay narrows the reproduction to SDK/compiled decode behavior, without identifying the faulty operator or declaring either numerical output correct. The production runtime is unchanged and B4 native parity remains failed. + +See evidence270/QWEN4-LANE-REPRO.md for the minimal token fixture, measured diagnostic scripts, prerequisite artifact/SDK versions, exact commands, acceptance criteria and archive hashes. No new hardware run was needed to package this reproduction. + + +## Follow-up instrumentation changes numerical behavior (2026-10-25) + +Supported hidden-state output and first-layer module hooks narrow the first observed difference to attention output on their instrumented graphs; first-layer KV equality passes positive/restore controls. However, exposing raw attention internals makes the first-attention probes lane-equal and removes the512/index50 token split, while2048/index13 still diverges later. Thus these observations do not identify the original faulty operator or constitute a fix. A packed-output attempt failed DEVICE_GRAPH_CONVERSION and is retained as failed. + +See evidence270/NUMERICS-DIAGNOSTICS.md for the progression, exact source scripts, SDK prerequisites, replay commands, compact audits, archive hashes and acceptance criteria. No production change or new performance gain is claimed. + + +## All-layer attention-output diagnostic: lane parity passes, no performance promotion (2026-10-25) + +Appending all36 core-attention outputs to the decode graph, while retaining all37 hidden states, makes all292 recorded outputs lane-exact across the four forced-history probe positions. Full-vocabulary logits also match across lanes. This is an isolated diagnostic export using the same Qwen3-4B weights, TP1/B4/k5/context9216 and SDK0.11.0/optimum0.11.0.post1; it adds1,179,648 bytes of attention outputs per decode call. It does not identify the original compiler transformation or establish a production fix. + +The free-generation native HTTP follow-up uses unchanged PR266 runtime33d6cb08ec806f693db4ecfd7d630331a3bd1bb2f, one assigned ATOM, inputs128/512/2048/8192, outputs32/128, concurrency16, caches off, and one timed batch per condition after warmup. All8warmups,128token comparisons and128timed-SSE text comparisons pass against the candidate's own per-condition references. The active-lane histogram is [0,664,32,32,5008]. Child, copies and UID-checked cleanup all exit0. + +This removes the original within-B4 parity failure on these fixtures (36/128 failed token checks), but historical B1 references match only4/8conditions and B4 references2/8. Those baselines ran on different physical cards. At8192input/128output, candidate throughput is16.438tok/s, median first SSE60.791s and median E2E86.400s; historical B4 was18.242tok/s and77.600s E2E. Thus there is no measured performance improvement to promote. Read-only metrics polling occurred during this exploratory run; same-card repeated controls and semantic quality remain separate gates. + +The task-only export/replay sources and compact audits are in evidence270; raw artifact hashes and the research event ID are in allcore-evidence-manifest.json. Run the builder with the established task environment and assigned device, then the SDK replay; use the existing HTTP harness with the all-core cache directory. As with the earlier reproduction, scripts expect the existing local evidence filenames and helper modules and are not standalone on main. Re-audit the collected HTTP result alongside both historical native files and the terminal controller state with `python3 audit_atom270_allcore_http.py`. The first audit correctly rejected a still-running cleanup state and passed after the controller reached completion. No measured response or timing batch was retried. + + +## Frozen sequential and concurrent quality gates (2026-10-25) + +The Qwen3-4B all-core artifact passes the existing36 frozen arithmetic/passkey/Korean-table cases at short/2K/8K input with max128 output, greedy selection, thinking disabled and EOS respected. Sequential and concurrent runs each score36/36 for answers and strict JSON format; all36 token arrays in each run equal the historical native-ATOM B1 baseline. Every request ends with EOS151645 and finish_reason=stop. The explicit EOS set is [151643,151645]; the fixed-length performance entrypoint is unchanged. + +The concurrent run sends three cohorts of12 distinct questions with a thread barrier. All12 client intervals overlap in each cohort. Aggregate server occupancy [0,13,5,9,64] proves64decode steps with four active lanes; it is not a per-condition occupancy trace. Sequential occupancy is [0,306,0,0,0]. Both successful runs have child/server/copy/cleanup exit0. The first sequential setup attempt failed before any request because the frozen fixture referenced another host's model path; the retry changes only that model-location field and verifies every original token ID before serving. Failed setup evidence is retained locally. No quality answers were retried or filtered. + +The new concurrent client reuses the unchanged strict scorer. Its real localhost HTTP self-check requires12 simultaneous requests before responding, checks all36 preserved outputs and EOS policy, and verifies exclusive output creation. Run `python3 selfcheck_quality270_concurrent.py`. The quality server, client and independent auditors are in evidence270. To reproduce, start atom_server270_quality.py with the same trusted runtime helper paths and all-core cache as the prior native HTTP setup, then run `python3 heldout_quality270_concurrent.py --fixtures "$FROZEN_FIXTURES" --endpoint "$OWNED_ENDPOINT" --output "$RAW_RESULT"`. Fixtures come from the existing quality26336-case export; only map the model location to the installed Qwen/Qwen3-4B and verify input IDs. Raw hashes and research event identifiers are in allcore-quality-evidence-manifest.json. + +These small structured-answer gates do not establish general language quality, parity with all historical performance-matrix outputs, or a performance gain. The same-card ABBA cost control is running separately; no diagnostic artifact is promoted to a production default. + + +## Four-model evidence coverage re-audit (2026-10-25) + +Re-reading raw historical records across four models, GPU B4 and ATOM B1/B4, and native/whole/stream routes yields33 within-route HTTP passes,1 failure and2 missing results out of36 combinations. The failure is the original Qwen3-4B ATOM B4 native run (36/128 token checks fail); its dependent B4 whole/stream runs are absent. The later all-core native candidate is separate evidence and does not erase this failure. Gemma B4 stream evidence exists in the timeline-instrumented run, passes all8conditions, and reaches at most3active lanes; it is not a four-lane occupancy pass or an uninstrumented performance control. + +All576 historical frozen quality responses were rescored without postprocessing. Strict answer counts, in native-GPU/RNGD-GPU/native-ATOM/RNGD-ATOM order, are Qwen3-0.6B32/34/33/33, Qwen3-4B36/36/36/36, KONI31/31/31/31 and Gemma0/0/0/0, each out of36. Strict format counts are36 for Qwen,34 for KONI and0 for Gemma. These model/task quality limitations are distinct from transport/runtime failure. Native-to-PD full-token discrepancies are recorded separately from within-route acceptance. + +`evidence270/audit_four_model_current270.py` recomputes these gates and records SHA256 for all raw files in `four-model-current270-audit.json`; run it in the task evidence directory containing the named raw records and existing audit/scoring helpers. This inventory covers historical experimental runtimes and isolated exports, not current-main support. At the read-only GitHub check, PR206 and PR266 remain unmerged and issue271 is open. Remaining work is candidate Qwen4 B4 PD compatibility, Gemma occupancy/import restrictions and owner integration, with measured performance and unchanged quality gates. No complete four-model support claim. + + +## Same-card all-core ABBA completes: consistency costs throughput (2026-10-25) + +Original/all-core/all-core/original runs each complete66responses at512/8192input,128output,c16, using fresh processes in one Pod on the same physical ATOM3. Same PR266 runtime, TP1/B4/k5/context9216, SDK0.11.0/optimum0.11.0.post1, caches off. Every arm has identical occupancy [0,262,8,8,2020]. A1/A2 each fail12full-token and12timed-text comparisons; B1/B2 have zero mismatches. Same-variant warmup references match, cross-variant references differ at both lengths. Full controller/copy/cleanup exit0; individual arm exits remain1/0/0/1, preserving the parity failures. + +| Input/output | Original mean tok/s | All-core mean tok/s | Change | Original mean of median E2E (s) | All-core mean of median E2E (s) | +|---|---:|---:|---:|---:|---:| +|512/128|75.976|62.894|-17.22%|17.195|20.775| +|8192/128|18.253|16.489|-9.66%|77.575|86.071| + +Two timed batches per variant/condition, each after warmup and complete token checks. These are means of batch statistics, not pooled-request quantiles or confidence intervals. Read-only metrics polling occurred during the run and is disclosed. The repeatable within-batch consistency improvement incurs serving cost; the candidate is not a speedup or production default. Different cross-variant outputs preclude an equivalent-output performance claim. Pending candidate RNGD→ATOM compatibility is a separate experiment, not evidence of support yet. + +The independent auditor recomputes flags and timing summaries, validates metadata, geometry, all four bindings, occupancy, arm exit semantics and terminal cleanup. Its self-check retains the historical failure and rejects altered flags, tokens, throughput and a missing batch. Published source is audit_atom270_allcore_abba.py; run it beside the named raw results/logs and terminal controller records. The child runner documents exact ordering and CLI; site configs, owned-Pod allocation and shell environment remain local prerequisites. Full raw hashes and archive event are in allcore-abba-evidence-manifest.json. No measured arm was retried or discarded. + + +## All-core RNGD→ATOM whole/stream compatibility (2026-10-25) + +The isolated Qwen3-4B all-core decoder accepts RNGD KV through both existing routes. Each passes8conditions (inputs128/512/2048/8192,outputs32/128,c16),8warmups,128full-token checks and128timed-SSE text checks:528responses total. Stream is checked against pinned whole-mode references, and all8reference arrays match. Both runs share the same physical ATOM1 in one Pod with one RNGD allocation; B4/k5/context9216/TP1. Actual four-lane decode calls are251whole and504stream. Child/copies/UID-checked cleanup all exit0. Runtime remains PR26633d6cb08ec806f693db4ecfd7d630331a3bd1bb2f; the only config differences from original B4 runs are the isolated artifact directory and persistent compiler-cache wrapper. No transport/batcher implementation change. + +Historical all-core native references match4/8conditions, so this does not establish universal native/PD token equivalence. At8K/128, whole27.525tok/s and stream28.588tok/s; across all8conditions stream throughput is2.55–29.24% higher in this single pair. One timed batch per condition and read-only metrics polling are disclosed. This is exploratory same-device PD-route timing, not statistical speedup or a resource-equivalent native comparison. The graph's separate native ABBA cost remains9.66–17.22%; no default promotion. Frozen PD quality is a separately launched pending gate. + +The independent auditor verifies raw arrays/flags/timing, geometry, matching device bindings, full compiled-batch occupancy and terminal cleanup. It explicitly separates HTTP parity from occupancy: a check on the previous Gemma timeline run retains its max3lane limitation. Source, compact audit and raw archive manifest are in evidence270. Run audit_atom270_allcore_pd.py beside the named result/log files, terminal controller records and historical all-core native result; the child runner records exact whole/stream ordering and reference linkage. Site-specific server configs and owned allocation remain local prerequisites. No measured response was retried or discarded. + + +## All-core PD frozen quality completes (2026-10-25) + +Whole and layer-streamed RNGD→ATOM each pass36/36 strict answers and formats on the unchanged short/2K/8K fixture, c12 cohorts, max128 output with EOS respected. All72 responses stop on EOS; no answer is retried. Every full token array equals the historical PD and native all-core concurrent baseline, and whole/stream match36/36. Both servers, controller, copies and UID-checked cleanup exit0. + +Measured occupancy is [0,230,38,0,0] for whole and [0,233,23,9,0] for stream: at most two and three active lanes. This does not establish full-four-lane PD semantic quality. The separate fixed-length performance run observes four lanes, but has a different acceptance criterion. These frozen cases establish neither general model quality nor a performance gain or production default. + +Run `python3 audit_atom270_allcore_pd_quality.py` beside the named raw JSONL, logs, terminal states and existing frozen fixtures/baselines. The auditor recomputes strict scores, verifies case hashes and raw payload consistency, EOS, cohort overlap, geometry, binding, occupancy and cleanup. Child/pair runners document route ordering and setup; site allocation/configs remain local prerequisites. Raw hashes and research event ID are in allcore-pd-quality-evidence-manifest.json. diff --git a/experiments/evidence270/NUMERICS-DIAGNOSTICS.md b/experiments/evidence270/NUMERICS-DIAGNOSTICS.md new file mode 100644 index 00000000..7a9e9301 --- /dev/null +++ b/experiments/evidence270/NUMERICS-DIAGNOSTICS.md @@ -0,0 +1,41 @@ +# Qwen3-4B diagnostic progression and limits + +The native B4 parity failure remains unresolved. These isolated exports are diagnostic experiments, not serving fixes or recommended defaults. All use the SDK environment and PR266 prerequisites in QWEN4-LANE-REPRO.md. Shared SDK files and engine code were unchanged. + +| Experiment | Result | Limit | +|---|---|---| +| Supported output_hidden_states export | Both original token splits reproduce. At all four observed positions, embeddings are lane-equal; first difference at boundary1, after layer0. | Recorded logits differ from the original uninstrumented graph. | +| First-layer physical KV readback | Sixteen K/V snapshots show identical initialized ranges across all four blocks. Changing only block3 token0 changes1024 K and1024 V elements; restoring the prompt restores equality. | Sampled first-layer stored KV only. Does not identify a compute operator. | +| First-layer module hooks | Input normalization stays equal; first difference at attention output, including output projection. Prior instrumented logit records match. | Extra graph outputs may change optimization. MLP receives differing inputs, so no MLP-origin conclusion. | +| Packed attention-internal outputs | Export falls back to jittrace, then DEVICE_GRAPH_CONVERSION failure. | No runtime numerical result; cause of compiler rejection unidentified. | +| Raw attention-internal decode outputs | Compiles; all observed first-attention internals are lane-equal.2048/index13 split remains,512/index50 split disappears. Later retained boundary11 differs. | Instrumentation materially changes numerical behavior. Not a fix and not proof the original attention kernel is correct or incorrect. | + +Compile times: supported hidden153.837882s; module hooks154.000048s; raw attention152.587502s. All owned Pods were cleaned up, including the failed compile attempt. No throughput or semantic-quality gain is claimed. + +## Interpretation + +Both module-hook and raw-attention exports log two torch.export→jittrace fallbacks; the supported hidden-state export logs none. Fallback alone does not distinguish the two instrumented outcomes. Any further attribution needs matched compiler controls or a smaller isolated replay with the same numerical failure. Do not adjust greedy tie-breaking to hide a changed logit order. + +Raw attention diagnostic slots0..10 are embedding, Q/K/V projections, Q/K normalization, rotary Q/K, attention V input, core attention output and output projection. Slots11..36 retain the ordinary hidden-state boundaries. Raw native tensor shapes are preserved only in decode; prefill keeps the original fixed-size hidden-output interface. Layer-module diagnostic slots0..5 have separate labels in their result. Neither layout is a general serving hidden-state API. + +## Reproduction + +Use a fresh task directory, the compact fixture from QWEN4-LANE-REPRO.md, the existing bootstrap/validation helpers and the recorded SDK versions. Allocate only the owned ATOM. Builders create distinct atom270-artifacts-* cache roots and reject overwrites. Run each compiler and replay in separate Python processes: + +```bash +"$ATOM_PYTHON" build_atom270_hidden.py +"$ATOM_PYTHON" container_bootstrap270.py diagnose_atom270_hidden.py +"$ATOM_PYTHON" container_bootstrap270.py diagnose_atom270_firstkv_control.py +"$ATOM_PYTHON" build_atom270_layerprobe.py +"$ATOM_PYTHON" container_bootstrap270.py diagnose_atom270_layerprobe.py +"$ATOM_PYTHON" build_atom270_attentionraw.py +"$ATOM_PYTHON" container_bootstrap270.py diagnose_atom270_attentionraw.py +``` + +Compact audits and source hashes are included. Raw token/probe arrays and failed compile log remain outside Git, linked by full hashes and research event IDs in diagnostic-evidence-manifest.json. Remote archive synchronization is unverified. Packaging does not constitute a new hardware run, nor a full repository test suite. + +## Required next evidence + +1. Confirm the original numerical failure under a controlled tracing/compiler comparison without changing observation outputs. +2. If using isolated replay, preserve real inputs/KV and establish the original failure before changing an operator. +3. Validate any candidate on both fixtures, all lane placements, free generation and held-out quality, then matched four-model serving and fault tests. Keep B1 defaults and B4 parity failure until these checks pass. diff --git a/experiments/evidence270/QWEN4-LANE-REPRO.md b/experiments/evidence270/QWEN4-LANE-REPRO.md new file mode 100644 index 00000000..b966be9a --- /dev/null +++ b/experiments/evidence270/QWEN4-LANE-REPRO.md @@ -0,0 +1,38 @@ +# Qwen3-4B B4 lane-dependent logits + +This reproduction is for engine/SDK diagnosis, not a correctness waiver or a serving fix. + +## Observed failure + +Qwen/Qwen3-4B, ATOM B4/TP1, five KV blocks, context9216. At input2048/output index13, forced identical history gives lane0 token369 with logits(369,264)=(29.75,29.625); lanes1–3 choose264 with(29.625,29.625). KV block permutation and fixing prefill batch_idx0 do not remove the split. Same-Pod ABBA completes128 cases with identical tokens and recorded logits across arms. Four bootstrap records verify the same assigned physical ATOM. + +Earlier direct SDK replay bypasses AtomBatcher and HTTP entirely and reproduces this split with identical per-lane decode tokens/positions. It also reproduces the512-input/index50 split (4829 versus1033). Native SDK output is bfloat16. This narrows reproduction to the SDK/compiled decode path; the responsible operator and preferred numerical result remain unidentified. + +## Environment and prerequisites + +Measured runtime is the PR266 stack (eaf7daa219a6cea68db4ec5fc08850edc848b271), rebel-compiler0.11.0, optimum-rbln0.11.0.post1, torch2.11.0+cpu, Transformers5.8.1. Use the previously compiled Qwen3-4B B4/k5/context9216 artifact. The scripts load from an atom270-artifacts directory alongside themselves. Provide the existing container_bootstrap270.py and atom_server270_http.py validation helper through the task harness/PYTHONPATH. Allocate one ATOM; no RNGD is needed. Do not reuse an unrelated workload or regroup devices. This package is not standalone against main. + +The compact fixture contains only the exact synthetic prompt IDs and reference tokens consumed by these scripts. It is an exact subset of the preserved native HTTP result; the source hash is in samecard-evidence-manifest.json. Reference tokens are replay inputs, not an independent quality oracle. + +## Run + +Copy the source scripts and fixture to a fresh task directory with the prerequisites above. The measured scripts retain their original input filename: + +```bash +cp qwen4-lane-fixtures.json atom270-qwen4b-tp1-b4-native.json +python3 selfcheck_atom270_direct.py +"$ATOM_PYTHON" container_bootstrap270.py diagnose_atom270_direct.py +for arm in a1 b1 b2 a2; do + "$ATOM_PYTHON" container_bootstrap270.py diagnose_atom270_samecard_retry1.py "$arm" +done +``` + +Each arm must use a fresh Python process within the same allocated Pod. Re-loading engines in one Python process failed with SDK INIT_INTERNAL after the first arm; that failed attempt is preserved in the research record. All output files must be absent before starting. Scripts reject overwrites. Keep the allocation unchanged and retain all four ATOM270_BINDING records. + +## Next investigation and acceptance + +- Obtain an SDK-supported intermediate-output/profiling path to identify the first divergent decode operator, preserving identical KV and forced history. +- Evaluate any compiler/runtime candidate against both2048/index13 and512/index50 reproductions, all active lanes and KV placements, then free generation and held-out prompts. +- Re-run matched native/whole/stream serving and fault checks before a B4 promotion. Do not change greedy tie-breaking merely to hide differing logits. + +Same-card raw arrays are archived outside Git; compact audit and full hashes are included. Remote synchronization is unverified. B1 remains the existing default and B4 parity remains failed; no production engine change is included. diff --git a/experiments/evidence270/allcore-abba-evidence-manifest.json b/experiments/evidence270/allcore-abba-evidence-manifest.json new file mode 100644 index 00000000..6abf9f91 --- /dev/null +++ b/experiments/evidence270/allcore-abba-evidence-manifest.json @@ -0,0 +1,46 @@ +{ + "research_event_id": "f2ce561ebec14afba60d52baad18094e", + "archive": "Local raw evidence; result JSONs registered with research-hub; remote synchronization unverified", + "files": { + "atom270-qwen4b-allcore-abba-a1.json": { + "sha256": "0a5b574e4382e9a2402b6f9f0799f0627310481fc8de9389a7379f60f200e381", + "bytes": 251121 + }, + "atom270-qwen4b-allcore-abba-a1.server0.log": { + "sha256": "9b7366cd6fe02e8b1d0ab75690d46c287dc13590051ed0020a3d13d1b4418bd1", + "bytes": 233 + }, + "atom270-qwen4b-allcore-abba-b1.json": { + "sha256": "26bfb41d507f171cd073078166b3afb346ae935451c6ce16f3a4f4e6af9594c5", + "bytes": 252978 + }, + "atom270-qwen4b-allcore-abba-b1.server0.log": { + "sha256": "9b7366cd6fe02e8b1d0ab75690d46c287dc13590051ed0020a3d13d1b4418bd1", + "bytes": 233 + }, + "atom270-qwen4b-allcore-abba-b2.json": { + "sha256": "1750d0b6a8844b8968900528f01be11ffba0da36eb7df77bc8a66c5e9e692a07", + "bytes": 252987 + }, + "atom270-qwen4b-allcore-abba-b2.server0.log": { + "sha256": "9b7366cd6fe02e8b1d0ab75690d46c287dc13590051ed0020a3d13d1b4418bd1", + "bytes": 233 + }, + "atom270-qwen4b-allcore-abba-a2.json": { + "sha256": "a7729ef1958aadcd6b70662c15f116275bebaf570a5b55df81e5be9db58e3fbe", + "bytes": 251128 + }, + "atom270-qwen4b-allcore-abba-a2.server0.log": { + "sha256": "9b7366cd6fe02e8b1d0ab75690d46c287dc13590051ed0020a3d13d1b4418bd1", + "bytes": 233 + }, + "atom270-allcore-abba-outer.json": { + "sha256": "0b5e495fbac1334417819c223ba95372717e00a621744a4e55e2277083e2eb56", + "bytes": 2079 + }, + "atom270-allcore-abba-arms.json": { + "sha256": "0d923ac06cbc430b304b9bf3d103b67ccb08d45261e57c85f6deb3233102b9fd", + "bytes": 3001 + } + } +} diff --git a/experiments/evidence270/allcore-evidence-manifest.json b/experiments/evidence270/allcore-evidence-manifest.json new file mode 100644 index 00000000..9ce83153 --- /dev/null +++ b/experiments/evidence270/allcore-evidence-manifest.json @@ -0,0 +1,18 @@ +{ + "research_event_id": "5314580e2c8845538448674bd734003d", + "archive": "Local research-hub artifacts; remote synchronization unverified", + "files": { + "atom270-qwen4b-allcoreprobe-sdk.json": { + "bytes": 124334, + "sha256": "32f89a5887a7bf8b69561ba78c2245b2d91baf04585bc283b39f9dfae5ef555c" + }, + "atom270-qwen4b-allcore-http.json": { + "bytes": 510017, + "sha256": "c33843569e3e172f9088fca2d15d4ecb9a2758788e5d345cec704a5f998ffaae" + }, + "atom270-qwen4b-allcore-http.server0.log": { + "bytes": 235, + "sha256": "8557841300ed96466b41e5972a9ac5da6f5d25c426c946100a456eed77494bd8" + } + } +} diff --git a/experiments/evidence270/allcore-pd-evidence-manifest.json b/experiments/evidence270/allcore-pd-evidence-manifest.json new file mode 100644 index 00000000..717a4bcc --- /dev/null +++ b/experiments/evidence270/allcore-pd-evidence-manifest.json @@ -0,0 +1,30 @@ +{ + "research_event_id": "2bcac27c59d1486a8e7df59049a391c5", + "archive": "Local raw evidence; result JSONs registered with research-hub; remote sync unverified", + "files": { + "atom270-qwen4b-allcore-pd-whole.json": { + "sha256": "fa6d2950f9f6c95e74ac2525e91d8f15fe09560d4d69594cd81cae31eefb4e24", + "bytes": 512556 + }, + "atom270-qwen4b-allcore-pd-whole.server0.log": { + "sha256": "4e04e39584bf660cbeadbd22946f775b0c8ca6e7913af153d06257b38de4a077", + "bytes": 272 + }, + "atom270-qwen4b-allcore-pd-stream.json": { + "sha256": "07ca97efe2ed748d927174ea352e990988cdd8ccd0095aae160412ed90d1ab5d", + "bytes": 512604 + }, + "atom270-qwen4b-allcore-pd-stream.server0.log": { + "sha256": "c910abaffd603607d01ce0eae13cf96851f56cbab1351a7399087afb77c63114", + "bytes": 272 + }, + "atom270-allcore-pd-outer.json": { + "sha256": "703597217d53f0d358accfa77a3d52c19e29d59fa0549e5353be0ce33029010a", + "bytes": 2004 + }, + "atom270-allcore-pd-arms.json": { + "sha256": "69d6df206fe58d4b5135df98e40053b30063d515ff333c69ea8c4daa5d97a94f", + "bytes": 1709 + } + } +} diff --git a/experiments/evidence270/allcore-pd-quality-evidence-manifest.json b/experiments/evidence270/allcore-pd-quality-evidence-manifest.json new file mode 100644 index 00000000..77ecb66f --- /dev/null +++ b/experiments/evidence270/allcore-pd-quality-evidence-manifest.json @@ -0,0 +1,34 @@ +{ + "research_event_id": "658d1ea450bc436cb73ebc1a484a05bc", + "archive": "Local raw evidence; JSONL results registered with research-hub; remote sync unverified", + "files": { + "atom270-allcore-pd-quality-outer.json": { + "sha256": "babe893e654293a781155dc7cc320fd004d9efa5762949ad06d45af768c2daa5", + "bytes": 1983 + }, + "atom270-qwen4b-allcore-pd-quality-whole.jsonl": { + "sha256": "f868c1bf6d024a8284e362ea2a89b55ba4b71cf02513023d1ba55fd581565070", + "bytes": 30294 + }, + "atom270-qwen4b-allcore-pd-quality-whole.server0.log": { + "sha256": "db709634d7c48970710d625606d84e52386314f3adc8dbdf58c4ccad78826be4", + "bytes": 301 + }, + "atom270-qwen4b-allcore-pd-quality-whole-run.json": { + "sha256": "5ee76d392020991ca918337f76d5428dd88e323c778f45b16d23e877a3c2701c", + "bytes": 1372 + }, + "atom270-qwen4b-allcore-pd-quality-stream.jsonl": { + "sha256": "4016161cc6d72357ae8a76b911ccc39199d53093c9f63d46fe5323cb90be6c2b", + "bytes": 30305 + }, + "atom270-qwen4b-allcore-pd-quality-stream.server0.log": { + "sha256": "e97e8808d5261893538e315840ccbd20aeb061ea073dc6504c0d237df7467df1", + "bytes": 301 + }, + "atom270-qwen4b-allcore-pd-quality-stream-run.json": { + "sha256": "791ec7f4c26fe7f4024758d48bc33c3e6b21ba50d9d927ad905eba5eee79bc95", + "bytes": 1395 + } + } +} diff --git a/experiments/evidence270/allcore-quality-evidence-manifest.json b/experiments/evidence270/allcore-quality-evidence-manifest.json new file mode 100644 index 00000000..c21fb7f3 --- /dev/null +++ b/experiments/evidence270/allcore-quality-evidence-manifest.json @@ -0,0 +1,16 @@ +{ + "research_events": [ + "5ceb6f2bb0644a4da66914ce79bb1fc1" + ], + "archive": "Local research-hub artifacts; remote synchronization unverified", + "files": { + "atom270-qwen4b-allcore-quality-retry1.jsonl": { + "sha256": "d55ad59583febee514dfc507f932d579cf32d1435eb5fcff63eb89f4001c8626", + "bytes": 27463 + }, + "atom270-qwen4b-allcore-quality-c12.jsonl": { + "sha256": "97dc5c9bb163f455dc6a0a85733c84de6f45dbd23f79fff0535d02a4ffa2d7be", + "bytes": 30295 + } + } +} diff --git a/experiments/evidence270/analyze_timeline270.py b/experiments/evidence270/analyze_timeline270.py new file mode 100644 index 00000000..9cbe33b9 --- /dev/null +++ b/experiments/evidence270/analyze_timeline270.py @@ -0,0 +1,47 @@ +"""Summarize host observations without inferring device utilization or DMA overlap.""" +import argparse +from collections import Counter +import hashlib +import json +from pathlib import Path +import statistics + + +def distribution(values): + return dict(count=len(values),sum=sum(values),min=min(values) if values else None, + median=statistics.median(values) if values else None,max=max(values) if values else None) + + +def summarize(trace): + remote,decode=trace['remote'],trace['decode'] + for row in remote+decode: + if row['end']0 for r in decode), + calls_with_waiting=sum(r['waiting']>0 for r in decode), + calls_with_prefilling=sum(r['prefilling']>0 for r in decode), + host_call_s=distribution([r['end']-r['start'] for r in decode])), + limits='All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.') + + +if __name__=='__main__': + parser=argparse.ArgumentParser(__doc__) + parser.add_argument('server_log',type=Path) + parser.add_argument('--output',type=Path,required=True) + args=parser.parse_args() + raw=args.server_log.read_bytes() + rows=[json.loads(l.split(' ',1)[1]) for l in raw.decode().splitlines() if l.startswith('ATOM270_HOST_TIMELINE ')] + if len(rows)!=1:raise ValueError('Expected one completed host timeline') + out=summarize(rows[0]);out['source_sha256']=hashlib.sha256(raw).hexdigest() + with args.output.open('x') as f:json.dump(out,f,indent=2);f.write('\n') + print(json.dumps(out,indent=2)) diff --git a/experiments/evidence270/atom-batch-inventory.json b/experiments/evidence270/atom-batch-inventory.json new file mode 100644 index 00000000..fd685b36 --- /dev/null +++ b/experiments/evidence270/atom-batch-inventory.json @@ -0,0 +1,989 @@ +{ + "scope": "Interim ATOM B1/B4 inventory; within-route pass is not cross-route parity or model quality. Gemma original k5 native violates causality.", + "conditions": { + "lengths": [ + 128, + 512, + 2048, + 8192 + ], + "outputs": [ + 32, + 128 + ], + "concurrency": 16, + "repeats": 1, + "context": 9216, + "kv_blocks": 5, + "caches": false + }, + "cases": [ + { + "model": "qwen06b", + "batch": 1, + "mode": "native", + "source": "atom270-qwen06b-tp1-b1-native.json", + "source_sha256": "1375ae8c9eb389b2a14bd70420539d3c48276261bae0d61c55e8e46973ac7b6f", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 40.69590429693926, + "output_tok_s": 50.324474548020554, + "median_first_sse_s": 20.20046385045862, + "median_tpot_est_s": 0.01152950295661466, + "median_e2e_s": 21.6256192799774, + "all_text_equal": true + } + }, + { + "model": "qwen06b", + "batch": 1, + "mode": "whole", + "source": "atom270-qwen06b-tp1-b1-whole.json", + "source_sha256": "a1fa82e039e96027508ab0da5b714575632aee09ff2fdbc9ff24c772a5911314", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 61.76825423899572, + "output_tok_s": 33.156190428756695, + "median_first_sse_s": 31.377333267533686, + "median_tpot_est_s": 0.011457031642226136, + "median_e2e_s": 32.836034505977295, + "all_text_equal": true + } + }, + { + "model": "qwen06b", + "batch": 1, + "mode": "stream", + "source": "atom270-qwen06b-tp1-b1-stream.json", + "source_sha256": "b0b2211cb820c2fa94f67d2cdee39def66032dcbd806320c0ed4d1fec47dfc9e", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 47.45461846899707, + "output_tok_s": 43.15702172040418, + "median_first_sse_s": 23.975205504510086, + "median_tpot_est_s": 0.011326029267910076, + "median_e2e_s": 25.426549739495385, + "all_text_equal": true + } + }, + { + "model": "qwen06b", + "batch": 4, + "mode": "native", + "source": "atom270-qwen06b-tp1-b4-native.json", + "source_sha256": "c9f2a5e4f193b0db8a4d498d332443ceea21b02bf982c6fdb1fe1256d09f96e9", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 4, + "active_lane_histogram": [ + 0, + 664, + 32, + 32, + 5008 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 32.18251903599594, + "output_tok_s": 63.63703219469319, + "median_first_sse_s": 14.797579117992427, + "median_tpot_est_s": 0.05466472298799699, + "median_e2e_s": 21.75026416644687, + "all_text_equal": true + } + }, + { + "model": "qwen06b", + "batch": 4, + "mode": "whole", + "source": "atom270-qwen06b-tp1-b4-whole.json", + "source_sha256": "b789259526fca6a904ef23e1cfbc256f7a22fbc7444b4d9d8f261154fabbb2b6", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 4, + "active_lane_histogram": [ + 0, + 13291, + 2162, + 319, + 571 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 35.03081701300107, + "output_tok_s": 58.46281002352646, + "median_first_sse_s": 17.814400231465697, + "median_tpot_est_s": 0.03908897198481837, + "median_e2e_s": 22.792797370057087, + "all_text_equal": true + } + }, + { + "model": "qwen06b", + "batch": 4, + "mode": "stream", + "source": "atom270-qwen06b-tp1-b4-stream.json", + "source_sha256": "c415ce39681718661a61a9f64a9115849210235e9b3ce315e636a2cb58a960e4", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 4, + "active_lane_histogram": [ + 0, + 7313, + 4068, + 469, + 1000 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 45.0327116510598, + "output_tok_s": 45.478051951859364, + "median_first_sse_s": 22.801500510016922, + "median_tpot_est_s": 0.04172879728712288, + "median_e2e_s": 28.132468500523828, + "all_text_equal": true + } + }, + { + "model": "qwen4b", + "batch": 1, + "mode": "native", + "source": "atom270-qwen4b-tp1-b1-native.json", + "source_sha256": "eb11ac21e4a6f74f2a258aaa319cabfcc7eea0b3b2e30f6dd4f1208d5f5e9c5a", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "1": 0 + }, + "opened": [ + "/dev/rbln1" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 166.73132611101028, + "output_tok_s": 12.283234637241682, + "median_first_sse_s": 83.1327906565275, + "median_tpot_est_s": 0.042929202673395755, + "median_e2e_s": 88.58196617756039, + "all_text_equal": true + } + }, + { + "model": "qwen4b", + "batch": 1, + "mode": "whole", + "source": "atom270-qwen4b-tp1-b1-whole.json", + "source_sha256": "ca1d7fe3bc2b55461fd4d7ef01bbeae68265ca4a4af03ade5714faaea509cb5c", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "1": 0 + }, + "opened": [ + "/dev/rbln1" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 162.32253570703324, + "output_tok_s": 12.616855639172119, + "median_first_sse_s": 80.82660805253545, + "median_tpot_est_s": 0.04286877280348678, + "median_e2e_s": 86.27872413001023, + "all_text_equal": true + } + }, + { + "model": "qwen4b", + "batch": 1, + "mode": "stream", + "source": "atom270-qwen4b-tp1-b1-stream.json", + "source_sha256": "a079bf27d380ecb93e1c33d7093b187fed24d96ecaea7cdaede8d34acf3263f3", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "1": 0 + }, + "opened": [ + "/dev/rbln1" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 141.20428703003563, + "output_tok_s": 14.50380893580355, + "median_first_sse_s": 69.71645488747163, + "median_tpot_est_s": 0.04285237829954415, + "median_e2e_s": 75.16798075096449, + "all_text_equal": true + } + }, + { + "model": "qwen4b", + "batch": 4, + "mode": "native", + "source": "atom270-qwen4b-tp1-b4-native.json", + "source_sha256": "11d07b625862acc161aa95c34cd2d1087e9bd302477c01ff7e7f66677f9234f7", + "state": "audit_failed", + "error": "runtime or parity failure", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 112.26723379304167, + "output_tok_s": 18.242188132784786, + "median_first_sse_s": 54.54269025201211, + "median_tpot_est_s": 0.1815217392240453, + "median_e2e_s": 77.60018511646194, + "all_text_equal": true + } + }, + { + "model": "qwen4b", + "batch": 4, + "mode": "whole", + "source": "atom270-qwen4b-tp1-b4-whole.json", + "state": "not_collected" + }, + { + "model": "qwen4b", + "batch": 4, + "mode": "stream", + "source": "atom270-qwen4b-tp1-b4-stream.json", + "state": "not_collected" + }, + { + "model": "koni", + "batch": 1, + "mode": "native", + "source": "atom270-koni-tp2-b1-native.json", + "source_sha256": "111c24490d36d835efa369d11ba05ce9a63e7c53da89ca4d7d8a72119c688f39", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 2 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 2, + "physical_to_visible": { + "0": 0, + "1": 1 + }, + "opened": [ + "/dev/rbln0", + "/dev/rbln1" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 136.36672446795274, + "output_tok_s": 15.018326560166782, + "median_first_sse_s": 67.78715079504764, + "median_tpot_est_s": 0.036702179535581754, + "median_e2e_s": 72.43892477202462, + "all_text_equal": true + } + }, + { + "model": "koni", + "batch": 1, + "mode": "whole", + "source": "atom270-koni-tp2-b1-whole.json", + "source_sha256": "a21aa944d52a1f0d9527f75e58d6b6f5bbc60cc7ddbef815517069018ceafe4c", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 2 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 2, + "physical_to_visible": { + "0": 0, + "1": 1 + }, + "opened": [ + "/dev/rbln0", + "/dev/rbln1" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 147.397185879061, + "output_tok_s": 13.89443080467207, + "median_first_sse_s": 73.67303348955465, + "median_tpot_est_s": 0.03654847306329537, + "median_e2e_s": 78.30206911102869, + "all_text_equal": true + } + }, + { + "model": "koni", + "batch": 1, + "mode": "stream", + "source": "atom270-koni-tp2-b1-stream.json", + "source_sha256": "37cdbd2af69c954c9d9b485274a765dda72e3f413e0ebeab239c73d60b895fe7", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 2 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 2, + "physical_to_visible": { + "0": 0, + "1": 1 + }, + "opened": [ + "/dev/rbln0", + "/dev/rbln1" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 119.06118814903311, + "output_tok_s": 17.201239394960897, + "median_first_sse_s": 58.09352488449076, + "median_tpot_est_s": 0.036696354886076935, + "median_e2e_s": 62.75135855545523, + "all_text_equal": true + } + }, + { + "model": "koni", + "batch": 4, + "mode": "native", + "source": "atom270-koni-tp2-b4-native.json", + "source_sha256": "b865d242b87b6a948b81fa3bf97927b302db923c3834f013fbacb0b9de6c509c", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 2 + }, + "max_active_lanes": 4, + "active_lane_histogram": [ + 0, + 664, + 32, + 32, + 5008 + ], + "binding": { + "visible_count": 2, + "physical_to_visible": { + "2": 0, + "3": 1 + }, + "opened": [ + "/dev/rbln2", + "/dev/rbln3" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 85.87466673506424, + "output_tok_s": 23.848709728543998, + "median_first_sse_s": 41.86685951001709, + "median_tpot_est_s": 0.1383990001654817, + "median_e2e_s": 59.45633385749534, + "all_text_equal": true + } + }, + { + "model": "koni", + "batch": 4, + "mode": "whole", + "source": "atom270-koni-tp2-b4-whole.json", + "source_sha256": "6d71f099b7811884212c4a00dc9ccfd4a0de661bb491a305aecab693d08048bb", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 2 + }, + "max_active_lanes": 4, + "active_lane_histogram": [ + 0, + 8833, + 5395, + 319, + 69 + ], + "binding": { + "visible_count": 2, + "physical_to_visible": { + "2": 0, + "3": 1 + }, + "opened": [ + "/dev/rbln2", + "/dev/rbln3" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 64.3446362640243, + "output_tok_s": 31.828604821021518, + "median_first_sse_s": 33.22209676599596, + "median_tpot_est_s": 0.05832567162982254, + "median_e2e_s": 40.67323362902971, + "all_text_equal": true + } + }, + { + "model": "koni", + "batch": 4, + "mode": "stream", + "source": "atom270-koni-tp2-b4-stream.json", + "source_sha256": "dbc521450419643fd1dbe75c8ed4da5b4081656ae3071addab718b3a73cb2996", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 2 + }, + "max_active_lanes": 4, + "active_lane_histogram": [ + 0, + 6036, + 4743, + 1398, + 285 + ], + "binding": { + "visible_count": 2, + "physical_to_visible": { + "2": 0, + "3": 1 + }, + "opened": [ + "/dev/rbln2", + "/dev/rbln3" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 53.119983348064125, + "output_tok_s": 38.554228953361225, + "median_first_sse_s": 26.988731065008324, + "median_tpot_est_s": 0.06271455727524938, + "median_e2e_s": 35.03915593097918, + "all_text_equal": true + } + }, + { + "model": "gemma", + "batch": 1, + "mode": "native", + "source": "atom270-gemma-tp1-b1-native.json", + "source_sha256": "9a670a7d4c16fa6b0d1a0ccaf6b94a0b45179ff33d6ef5112955289eb5c6f365", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 35.315574429929256, + "output_tok_s": 57.991411241618096, + "median_first_sse_s": 17.38664855447132, + "median_tpot_est_s": 0.011152653319222163, + "median_e2e_s": 18.80122325598495, + "all_text_equal": true + }, + "causal_baseline_valid": false + }, + { + "model": "gemma", + "batch": 1, + "mode": "whole", + "source": "atom270-gemma-tp1-b1-whole.json", + "source_sha256": "e0f4e8e093c9ec353a46edc459104c56a63160afe8a9de444d219e6bbd23548e", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 50.81015266897157, + "output_tok_s": 40.30690506566142, + "median_first_sse_s": 25.73407808301272, + "median_tpot_est_s": 0.011059337752544886, + "median_e2e_s": 27.12871677405201, + "all_text_equal": true + } + }, + { + "model": "gemma", + "batch": 1, + "mode": "stream", + "source": "atom270-gemma-tp1-b1-stream.json", + "source_sha256": "9470ce9d2a5f43787bb950a1de6f7c3db6a1a111a4b0428076c1dd27e2256623", + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "state": "within_route_pass", + "long_case": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 46.951324986992404, + "output_tok_s": 43.61964226925197, + "median_first_sse_s": 23.561160824494436, + "median_tpot_est_s": 0.01107739847644753, + "median_e2e_s": 24.964550331467763, + "all_text_equal": true + } + }, + { + "model": "gemma", + "batch": 4, + "mode": "native", + "source": "atom270-gemma-tp1-b4-native.json", + "state": "not_collected" + }, + { + "model": "gemma", + "batch": 4, + "mode": "whole", + "source": "atom270-gemma-tp1-b4-whole.json", + "state": "not_collected" + }, + { + "model": "gemma", + "batch": 4, + "mode": "stream", + "source": "atom270-gemma-tp1-b4-stream.json", + "state": "not_collected" + } + ] +} diff --git a/experiments/evidence270/atom270-abba-gemma-audit.json b/experiments/evidence270/atom270-abba-gemma-audit.json new file mode 100644 index 00000000..1d0c4a31 --- /dev/null +++ b/experiments/evidence270/atom270-abba-gemma-audit.json @@ -0,0 +1,270 @@ +{ + "stage": "passed", + "conditions": 16, + "warmups": 16, + "correctness_responses": 256, + "timed_text_responses": 256, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "active_lane_histograms": [ + [ + 0, + 9554, + 437, + 0, + 0 + ], + [ + 0, + 2745, + 2124, + 1145, + 0 + ], + [ + 0, + 2736, + 1899, + 1298, + 0 + ], + [ + 0, + 9200, + 614, + 0, + 0 + ] + ], + "full_batch_observed": [ + false, + false, + false, + false + ], + "rows": [ + { + "input_tokens": 128, + "output_tokens": 32, + "samples": { + "wide": [ + { + "requests": 16, + "output_tokens": 512, + "wall_s": 25.426173542975448, + "output_tok_s": 20.136730331625206, + "median_first_sse_s": 13.316575774573721, + "median_tpot_est_s": 0.012865578337016726, + "median_e2e_s": 13.71464374504285, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 512, + "wall_s": 25.269685094011948, + "output_tok_s": 20.261431754894584, + "median_first_sse_s": 13.201504309487063, + "median_tpot_est_s": 0.013081875355583767, + "median_e2e_s": 13.606829051976092, + "all_text_equal": true + } + ], + "narrow": [ + { + "requests": 16, + "output_tokens": 512, + "wall_s": 12.63976663700305, + "output_tok_s": 40.50707696620874, + "median_first_sse_s": 6.520951300451998, + "median_tpot_est_s": 0.013170043693765277, + "median_e2e_s": 6.929753344971687, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 512, + "wall_s": 12.73982515197713, + "output_tok_s": 40.18893461191195, + "median_first_sse_s": 6.563573691993952, + "median_tpot_est_s": 0.013206368629565281, + "median_e2e_s": 6.985853927966673, + "all_text_equal": true + } + ] + }, + "throughput_ratio": 1.9975169020139072 + }, + { + "input_tokens": 128, + "output_tokens": 128, + "samples": { + "wide": [ + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.93169531400781, + "output_tok_s": 76.04422878402262, + "median_first_sse_s": 13.520118443469983, + "median_tpot_est_s": 0.013123653555475907, + "median_e2e_s": 15.170441881520674, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.356424776953645, + "output_tok_s": 77.70401400537429, + "median_first_sse_s": 13.114678405516315, + "median_tpot_est_s": 0.013347451244187578, + "median_e2e_s": 14.837463286006823, + "all_text_equal": true + } + ], + "narrow": [ + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 13.99720606196206, + "output_tok_s": 146.31491391453596, + "median_first_sse_s": 6.520276393042877, + "median_tpot_est_s": 0.014263195208435159, + "median_e2e_s": 8.315985388529953, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 14.088801890960895, + "output_tok_s": 145.36367363600715, + "median_first_sse_s": 6.564609895052854, + "median_tpot_est_s": 0.01496072162967542, + "median_e2e_s": 8.48026739899069, + "all_text_equal": true + } + ] + }, + "throughput_ratio": 1.897118186580396 + }, + { + "input_tokens": 512, + "output_tokens": 32, + "samples": { + "wide": [ + { + "requests": 16, + "output_tokens": 512, + "wall_s": 25.538719116011634, + "output_tok_s": 20.04799056969928, + "median_first_sse_s": 13.348799959989265, + "median_tpot_est_s": 0.01290669272683801, + "median_e2e_s": 13.74773466953775, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 512, + "wall_s": 25.809769368963316, + "output_tok_s": 19.83744963702344, + "median_first_sse_s": 13.491266344499309, + "median_tpot_est_s": 0.013100789210218336, + "median_e2e_s": 13.900058634986635, + "all_text_equal": true + } + ], + "narrow": [ + { + "requests": 16, + "output_tokens": 512, + "wall_s": 12.742855163989589, + "output_tok_s": 40.179378436857384, + "median_first_sse_s": 6.551535644510295, + "median_tpot_est_s": 0.012900817258689072, + "median_e2e_s": 6.950775286532007, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 512, + "wall_s": 12.757254777941853, + "output_tok_s": 40.134026396124206, + "median_first_sse_s": 6.572366861568298, + "median_tpot_est_s": 0.012895949273144885, + "median_e2e_s": 6.969756557024084, + "all_text_equal": true + } + ] + }, + "throughput_ratio": 2.013602066737243 + }, + { + "input_tokens": 512, + "output_tokens": 128, + "samples": { + "wide": [ + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.727096959017217, + "output_tok_s": 76.62635426288014, + "median_first_sse_s": 13.363714031525888, + "median_tpot_est_s": 0.013571272979996454, + "median_e2e_s": 15.107852793007623, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.800160386017524, + "output_tok_s": 76.41745312347105, + "median_first_sse_s": 13.399831712478772, + "median_tpot_est_s": 0.013756781511694398, + "median_e2e_s": 15.13004133402137, + "all_text_equal": true + } + ], + "narrow": [ + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 14.179545062943362, + "output_tok_s": 144.43340677778278, + "median_first_sse_s": 6.624991283053532, + "median_tpot_est_s": 0.015133046558838133, + "median_e2e_s": 8.550461189530324, + "all_text_equal": true + }, + { + "requests": 16, + "output_tokens": 2048, + "wall_s": 14.080374562065117, + "output_tok_s": 145.4506761146578, + "median_first_sse_s": 6.572921755490825, + "median_tpot_est_s": 0.014857141130079523, + "median_e2e_s": 8.441328348999377, + "all_text_equal": true + } + ] + }, + "throughput_ratio": 1.8941248773342598 + } + ], + "limits": "Two timed batches per width per condition, whole route, short inputs only. No confidence interval, heldout quality acceptance, other-model generalization or device overlap proof. Occupancy failures remain separate from ABBA correctness.", + "source_sha256": { + "atom270-abba-gemma-outer.json": "e61aef85b2806b7a17feb86ad0ad66b0a1621ef8933be485458dca1d5ace417c", + "atom270-abba-gemma-b4-r0-w8192.json": "40f2702e77e93e120dafb3a46d697575a6a2fbea4b5248562b420f1107492df3", + "atom270-abba-gemma-b4-r0-w8192.server0.log": "89bf71fea58f9d11673ef8b71999777a8927d8cc9ea95674eb10b8acafafb0f2", + "atom270-abba-gemma-b4-r1-w1024.json": "9cf1dc2a82ef3e15fb479f2b547b625828f6ab1454f3ae1da065d5e769d7b308", + "atom270-abba-gemma-b4-r1-w1024.server0.log": "7bc8be3957f24aba400130619dba6b5c347ac25ce5090f93aaede9fbc83bcbf1", + "atom270-abba-gemma-b4-r2-w1024.json": "2939601a01acc5a4885ec5cebe2176a040106e1bb4d0c81b02bb850054085ef0", + "atom270-abba-gemma-b4-r2-w1024.server0.log": "a04491e23500acada9a6bfd676d2950f38e69e5f5083498ffe935554af29189e", + "atom270-abba-gemma-b4-r3-w8192.json": "cb82aacf02bb2403d2302502b8810db2838e645946843ca602b9e43dbda85efe", + "atom270-abba-gemma-b4-r3-w8192.server0.log": "00d8c2924b7cd5338aadbacefd4028b35944852203b1e608035fad49975724d0" + } +} diff --git a/experiments/evidence270/atom270-abba-gemma-b4-r0-w8192-summary.json b/experiments/evidence270/atom270-abba-gemma-b4-r0-w8192-summary.json new file mode 100644 index 00000000..4ccd0cbb --- /dev/null +++ b/experiments/evidence270/atom270-abba-gemma-b4-r0-w8192-summary.json @@ -0,0 +1,48 @@ +{ + "remote": { + "completed": 132, + "errors": 0, + "submit_to_completion_s": { + "count": 132, + "sum": 630.5492389528081, + "min": 1.5467666470212862, + "median": 4.665469824976753, + "max": 6.3881797469221056 + }, + "completion_gap_s": { + "count": 131, + "sum": 215.70328289608005, + "min": 1.5291937199654058, + "median": 1.5664046419551596, + "max": 3.263799298903905 + }, + "backlogged_completion_gap_s": { + "count": 120, + "sum": 187.63299905497115, + "min": 1.5291937199654058, + "median": 1.565149066504091, + "max": 1.6199305519694462 + } + }, + "decode": { + "calls": 9991, + "errors": 0, + "lane_histogram": { + "1": 9554, + "2": 437 + }, + "max_lanes": 2, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 7291, + "calls_with_prefilling": 9043, + "host_call_s": { + "count": 9991, + "sum": 127.78043289273046, + "min": 0.01172367692925036, + "median": 0.01277005800511688, + "max": 0.018381373956799507 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "89bf71fea58f9d11673ef8b71999777a8927d8cc9ea95674eb10b8acafafb0f2" +} diff --git a/experiments/evidence270/atom270-abba-gemma-b4-r1-w1024-summary.json b/experiments/evidence270/atom270-abba-gemma-b4-r1-w1024-summary.json new file mode 100644 index 00000000..d9447eb5 --- /dev/null +++ b/experiments/evidence270/atom270-abba-gemma-b4-r1-w1024-summary.json @@ -0,0 +1,49 @@ +{ + "remote": { + "completed": 132, + "errors": 0, + "submit_to_completion_s": { + "count": 132, + "sum": 248.99051873781718, + "min": 0.7663520610658452, + "median": 1.5330410929745995, + "max": 3.053357904078439 + }, + "completion_gap_s": { + "count": 131, + "sum": 111.87569276697468, + "min": 0.7604349369648844, + "median": 0.7669553339947015, + "max": 2.583119563991204 + }, + "backlogged_completion_gap_s": { + "count": 120, + "sum": 92.14946020406205, + "min": 0.7604349369648844, + "median": 0.7661169784259982, + "max": 0.7799153890227899 + } + }, + "decode": { + "calls": 6014, + "errors": 0, + "lane_histogram": { + "1": 2745, + "2": 2124, + "3": 1145 + }, + "max_lanes": 3, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 4386, + "calls_with_prefilling": 5066, + "host_call_s": { + "count": 6014, + "sum": 81.02800242800731, + "min": 0.011781868059188128, + "median": 0.01315762847661972, + "max": 0.01989528303965926 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "7bc8be3957f24aba400130619dba6b5c347ac25ce5090f93aaede9fbc83bcbf1" +} diff --git a/experiments/evidence270/atom270-abba-gemma-b4-r2-w1024-summary.json b/experiments/evidence270/atom270-abba-gemma-b4-r2-w1024-summary.json new file mode 100644 index 00000000..5000ebe9 --- /dev/null +++ b/experiments/evidence270/atom270-abba-gemma-b4-r2-w1024-summary.json @@ -0,0 +1,49 @@ +{ + "remote": { + "completed": 132, + "errors": 0, + "submit_to_completion_s": { + "count": 132, + "sum": 248.11195615539327, + "min": 0.7649028070736676, + "median": 1.5408351540681906, + "max": 3.081106089055538 + }, + "completion_gap_s": { + "count": 131, + "sum": 112.41881947498769, + "min": 0.7639877240872011, + "median": 0.7695843919645995, + "max": 2.597001599962823 + }, + "backlogged_completion_gap_s": { + "count": 120, + "sum": 92.57003784691915, + "min": 0.7639877240872011, + "median": 0.7694909729762003, + "max": 0.879818647983484 + } + }, + "decode": { + "calls": 5933, + "errors": 0, + "lane_histogram": { + "1": 2736, + "2": 1899, + "3": 1298 + }, + "max_lanes": 3, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 4300, + "calls_with_prefilling": 4985, + "host_call_s": { + "count": 5933, + "sum": 81.32150790723972, + "min": 0.0119595070136711, + "median": 0.013659317046403885, + "max": 0.027728866902180016 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "a04491e23500acada9a6bfd676d2950f38e69e5f5083498ffe935554af29189e" +} diff --git a/experiments/evidence270/atom270-abba-gemma-b4-r3-w8192-summary.json b/experiments/evidence270/atom270-abba-gemma-b4-r3-w8192-summary.json new file mode 100644 index 00000000..b6fe8915 --- /dev/null +++ b/experiments/evidence270/atom270-abba-gemma-b4-r3-w8192-summary.json @@ -0,0 +1,48 @@ +{ + "remote": { + "completed": 132, + "errors": 0, + "submit_to_completion_s": { + "count": 132, + "sum": 627.803693210124, + "min": 1.55163858900778, + "median": 4.637933455465827, + "max": 6.4622178609715775 + }, + "completion_gap_s": { + "count": 131, + "sum": 215.96686176199, + "min": 1.5128796129720286, + "median": 1.5658038019901142, + "max": 3.336499619996175 + }, + "backlogged_completion_gap_s": { + "count": 120, + "sum": 187.56649833288975, + "min": 1.5128796129720286, + "median": 1.5618087735492736, + "max": 1.7115255339303985 + } + }, + "decode": { + "calls": 9814, + "errors": 0, + "lane_histogram": { + "1": 9200, + "2": 614 + }, + "max_lanes": 2, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 7107, + "calls_with_prefilling": 8866, + "host_call_s": { + "count": 9814, + "sum": 127.628489312483, + "min": 0.011876814067363739, + "median": 0.01297879102639854, + "max": 0.02203579794149846 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "00d8c2924b7cd5338aadbacefd4028b35944852203b1e608035fad49975724d0" +} diff --git a/experiments/evidence270/atom270-abba-gemma-outer.json b/experiments/evidence270/atom270-abba-gemma-outer.json new file mode 100644 index 00000000..ba7f43d7 --- /dev/null +++ b/experiments/evidence270/atom270-abba-gemma-outer.json @@ -0,0 +1,159 @@ +{ + "state": "complete", + "uid": "1813819b-478a-4ed1-a35a-6405d454e7cd", + "runs": [ + { + "mode": "whole", + "stem": "atom270-abba-gemma-b4-r0-w8192", + "width": 8192, + "exit_code": 0, + "http_audit": { + "stage": "passed", + "conditions": 4, + "warmups": 4, + "correctness_responses": 64, + "timed_text_responses": 64 + }, + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "occupancy_audit": { + "stage": "failed", + "histogram": [ + 0, + 9554, + 437, + 0, + 0 + ] + } + }, + { + "mode": "whole", + "stem": "atom270-abba-gemma-b4-r1-w1024", + "width": 1024, + "exit_code": 0, + "http_audit": { + "stage": "passed", + "conditions": 4, + "warmups": 4, + "correctness_responses": 64, + "timed_text_responses": 64 + }, + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "occupancy_audit": { + "stage": "failed", + "histogram": [ + 0, + 2745, + 2124, + 1145, + 0 + ] + } + }, + { + "mode": "whole", + "stem": "atom270-abba-gemma-b4-r2-w1024", + "width": 1024, + "exit_code": 0, + "http_audit": { + "stage": "passed", + "conditions": 4, + "warmups": 4, + "correctness_responses": 64, + "timed_text_responses": 64 + }, + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "occupancy_audit": { + "stage": "failed", + "histogram": [ + 0, + 2736, + 1899, + 1298, + 0 + ] + } + }, + { + "mode": "whole", + "stem": "atom270-abba-gemma-b4-r3-w8192", + "width": 8192, + "exit_code": 0, + "http_audit": { + "stage": "passed", + "conditions": 4, + "warmups": 4, + "correctness_responses": 64, + "timed_text_responses": 64 + }, + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "occupancy_audit": { + "stage": "failed", + "histogram": [ + 0, + 9200, + 614, + 0, + 0 + ] + } + } + ], + "cleanup_exit_code": 0 +} diff --git a/experiments/evidence270/atom270-allcore-abba-audit.json b/experiments/evidence270/atom270-allcore-abba-audit.json new file mode 100644 index 00000000..8ccd0779 --- /dev/null +++ b/experiments/evidence270/atom270-allcore-abba-audit.json @@ -0,0 +1,231 @@ +{ + "arms": { + "a1": { + "responses": 66, + "parity_passed": false, + "failed_token_checks": 12, + "failed_timed_text_checks": 12, + "sha256": "0a5b574e4382e9a2402b6f9f0799f0627310481fc8de9389a7379f60f200e381", + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "active_lane_histogram": [ + 0, + 262, + 8, + 8, + 2020 + ] + }, + "b1": { + "responses": 66, + "parity_passed": true, + "failed_token_checks": 0, + "failed_timed_text_checks": 0, + "sha256": "26bfb41d507f171cd073078166b3afb346ae935451c6ce16f3a4f4e6af9594c5", + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "active_lane_histogram": [ + 0, + 262, + 8, + 8, + 2020 + ] + }, + "b2": { + "responses": 66, + "parity_passed": true, + "failed_token_checks": 0, + "failed_timed_text_checks": 0, + "sha256": "1750d0b6a8844b8968900528f01be11ffba0da36eb7df77bc8a66c5e9e692a07", + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "active_lane_histogram": [ + 0, + 262, + 8, + 8, + 2020 + ] + }, + "a2": { + "responses": 66, + "parity_passed": false, + "failed_token_checks": 12, + "failed_timed_text_checks": 12, + "sha256": "a7729ef1958aadcd6b70662c15f116275bebaf570a5b55df81e5be9db58e3fbe", + "binding": { + "visible_count": 1, + "physical_to_visible": { + "3": 0 + }, + "opened": [ + "/dev/rbln3" + ] + }, + "active_lane_histogram": [ + 0, + 262, + 8, + 8, + 2020 + ] + } + }, + "comparisons": { + "512:128": { + "measurements": { + "a1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.94432804104872, + "output_tok_s": 76.00857578930695, + "median_first_sse_s": 10.726579087029677, + "median_tpot_est_s": 0.05077514991321872, + "median_e2e_s": 17.1865681380732, + "all_text_equal": false + }, + "b1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 32.61415444500744, + "output_tok_s": 62.79482129310597, + "median_first_sse_s": 12.985864013491664, + "median_tpot_est_s": 0.06157640918519157, + "median_e2e_s": 20.813204389531165, + "all_text_equal": true + }, + "b2": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 32.511290373047814, + "output_tok_s": 62.99350091923182, + "median_first_sse_s": 12.935025235987268, + "median_tpot_est_s": 0.06139598419710143, + "median_e2e_s": 20.736302551464178, + "all_text_equal": true + }, + "a2": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.9672546479851, + "output_tok_s": 75.94395598415204, + "median_first_sse_s": 10.731639130506665, + "median_tpot_est_s": 0.05090198532667187, + "median_e2e_s": 17.204384183511138, + "all_text_equal": false + } + }, + "output_tok_s": { + "original_mean_of_two": 75.9762658867295, + "allcore_mean_of_two": 62.894161106168895, + "allcore_relative_change_pct": -17.21867299988693 + }, + "median_first_sse_s": { + "original_mean_of_two": 10.72910910876817, + "allcore_mean_of_two": 12.960444624739466, + "allcore_relative_change_pct": 20.797025115046843 + }, + "median_e2e_s": { + "original_mean_of_two": 17.19547616079217, + "allcore_mean_of_two": 20.77475347049767, + "allcore_relative_change_pct": 20.815226494667826 + } + }, + "8192:128": { + "measurements": { + "a1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 112.11799769301433, + "output_tok_s": 18.266469631464027, + "median_first_sse_s": 54.498256245511584, + "median_tpot_est_s": 0.18131695643313023, + "median_e2e_s": 77.51373708300525, + "all_text_equal": true + }, + "b1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 124.24148527404759, + "output_tok_s": 16.484027017888526, + "median_first_sse_s": 60.4849502054858, + "median_tpot_est_s": 0.20110434198048055, + "median_e2e_s": 86.02603251353139, + "all_text_equal": true + }, + "b2": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 124.16852944297716, + "output_tok_s": 16.493712289155507, + "median_first_sse_s": 60.586589515034575, + "median_tpot_est_s": 0.20074741440178007, + "median_e2e_s": 86.11682887352072, + "all_text_equal": true + }, + "a2": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 112.28452868107706, + "output_tok_s": 18.239378336947524, + "median_first_sse_s": 54.54293855454307, + "median_tpot_est_s": 0.18158868359040659, + "median_e2e_s": 77.63563472602982, + "all_text_equal": true + } + }, + "output_tok_s": { + "original_mean_of_two": 18.252923984205776, + "allcore_mean_of_two": 16.488869653522016, + "allcore_relative_change_pct": -9.66450269671968 + }, + "median_first_sse_s": { + "original_mean_of_two": 54.52059740002733, + "allcore_mean_of_two": 60.53576986026019, + "allcore_relative_change_pct": 11.032844002237297 + }, + "median_e2e_s": { + "original_mean_of_two": 77.57468590451754, + "allcore_mean_of_two": 86.07143069352605, + "allcore_relative_change_pct": 10.95298638974407 + } + } + }, + "limits": "Same-card ABBA, two timed batches per variant/condition. Original parity failures remain failures; no equivalent-output speedup or general-quality claim.", + "reference_parity": { + "a1-a2": { + "512:128": true, + "8192:128": true + }, + "b1-b2": { + "512:128": true, + "8192:128": true + }, + "a1-b1": { + "512:128": false, + "8192:128": false + } + } +} diff --git a/experiments/evidence270/atom270-allcore-http-audit.json b/experiments/evidence270/atom270-allcore-http-audit.json new file mode 100644 index 00000000..b39b8105 --- /dev/null +++ b/experiments/evidence270/atom270-allcore-http-audit.json @@ -0,0 +1,207 @@ +{ + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "sha256": "c33843569e3e172f9088fca2d15d4ecb9a2758788e5d345cec704a5f998ffaae", + "comparisons": { + "b1": { + "128:32": { + "tokens_equal": false, + "first_difference": 7 + }, + "128:128": { + "tokens_equal": false, + "first_difference": 7 + }, + "512:32": { + "tokens_equal": true, + "first_difference": null + }, + "512:128": { + "tokens_equal": false, + "first_difference": 107 + }, + "2048:32": { + "tokens_equal": true, + "first_difference": null + }, + "2048:128": { + "tokens_equal": false, + "first_difference": 94 + }, + "8192:32": { + "tokens_equal": true, + "first_difference": null + }, + "8192:128": { + "tokens_equal": true, + "first_difference": null + } + }, + "b4": { + "128:32": { + "tokens_equal": false, + "first_difference": 7 + }, + "128:128": { + "tokens_equal": false, + "first_difference": 7 + }, + "512:32": { + "tokens_equal": true, + "first_difference": null + }, + "512:128": { + "tokens_equal": false, + "first_difference": 64 + }, + "2048:32": { + "tokens_equal": true, + "first_difference": null + }, + "2048:128": { + "tokens_equal": false, + "first_difference": 59 + }, + "8192:32": { + "tokens_equal": false, + "first_difference": 12 + }, + "8192:128": { + "tokens_equal": false, + "first_difference": 12 + } + } + }, + "measurements": [ + { + "input_tokens": 128, + "output_tokens_per_request": 32, + "requests": 16, + "output_tokens": 512, + "wall_s": 8.018060112954117, + "output_tok_s": 63.855844529377364, + "median_first_sse_s": 3.2187578039593063, + "median_tpot_est_s": 0.06104695664591066, + "median_e2e_s": 5.11025653447723, + "all_text_equal": true + }, + { + "input_tokens": 128, + "output_tokens_per_request": 128, + "requests": 16, + "output_tokens": 2048, + "wall_s": 28.876017716014758, + "output_tok_s": 70.9239071724274, + "median_first_sse_s": 11.027380754472688, + "median_tpot_est_s": 0.0559405932600764, + "median_e2e_s": 18.142516813473776, + "all_text_equal": true + }, + { + "input_tokens": 512, + "output_tokens_per_request": 32, + "requests": 16, + "output_tokens": 512, + "wall_s": 12.007294308044948, + "output_tok_s": 42.640747104612686, + "median_first_sse_s": 5.380131792451721, + "median_tpot_est_s": 0.08467051691766228, + "median_e2e_s": 8.009860208956525, + "all_text_equal": true + }, + { + "input_tokens": 512, + "output_tokens_per_request": 128, + "requests": 16, + "output_tokens": 2048, + "wall_s": 32.64148516498972, + "output_tok_s": 62.742243180669476, + "median_first_sse_s": 13.037358152447268, + "median_tpot_est_s": 0.0615226962717294, + "median_e2e_s": 20.855518963944633, + "all_text_equal": true + }, + { + "input_tokens": 2048, + "output_tokens_per_request": 32, + "requests": 16, + "output_tokens": 512, + "wall_s": 26.342920669005252, + "output_tok_s": 19.435961806710853, + "median_first_sse_s": 12.92043256550096, + "median_tpot_est_s": 0.17363946958099521, + "median_e2e_s": 18.30325612251181, + "all_text_equal": true + }, + { + "input_tokens": 2048, + "output_tokens_per_request": 128, + "requests": 16, + "output_tokens": 2048, + "wall_s": 47.36955108807888, + "output_tok_s": 43.23452413960925, + "median_first_sse_s": 20.766711130505428, + "median_tpot_est_s": 0.08371657433857277, + "median_e2e_s": 31.39914850803325, + "all_text_equal": true + }, + { + "input_tokens": 8192, + "output_tokens_per_request": 32, + "requests": 16, + "output_tokens": 512, + "wall_s": 98.85874976590276, + "output_tok_s": 5.1791065658064115, + "median_first_sse_s": 51.10930258699227, + "median_tpot_est_s": 0.617650188402378, + "median_e2e_s": 70.26619802654022, + "all_text_equal": true + }, + { + "input_tokens": 8192, + "output_tokens_per_request": 128, + "requests": 16, + "output_tokens": 2048, + "wall_s": 124.58599320193753, + "output_tok_s": 16.43844502391582, + "median_first_sse_s": 60.79142673499882, + "median_tpot_est_s": 0.20129819753154465, + "median_e2e_s": 86.40002300450578, + "all_text_equal": true + } + ], + "ATOM270_BINDING": [ + { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + } + ], + "ATOM270_COMPILED": [ + { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + } + ], + "ATOM270_ACTIVE_LANES": [ + [ + 0, + 664, + 32, + 32, + 5008 + ] + ], + "limits": "One timed batch per condition; historical references are from other cards. No stable performance gain or general semantic-quality claim." +} diff --git a/experiments/evidence270/atom270-allcore-pd-audit.json b/experiments/evidence270/atom270-allcore-pd-audit.json new file mode 100644 index 00000000..426f21ec --- /dev/null +++ b/experiments/evidence270/atom270-allcore-pd-audit.json @@ -0,0 +1,322 @@ +{ + "modes": { + "whole": { + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "1": 0 + }, + "opened": [ + "/dev/rbln1" + ] + }, + "active_lane_histogram": [ + 0, + 4491, + 4811, + 1913, + 251 + ], + "full_batch_observed": true, + "sha256": "fa6d2950f9f6c95e74ac2525e91d8f15fe09560d4d69594cd81cae31eefb4e24" + }, + "stream": { + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "1": 0 + }, + "opened": [ + "/dev/rbln1" + ] + }, + "active_lane_histogram": [ + 0, + 4317, + 1134, + 4085, + 504 + ], + "full_batch_observed": true, + "sha256": "07ca97efe2ed748d927174ea352e990988cdd8ccd0095aae160412ed90d1ab5d" + } + }, + "limits": "Within-route HTTP parity and whole/stream parity; occupancy reported independently. Single timed batches, not a matched native/PD speedup or general quality result.", + "whole_stream_references_equal": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "historical_native_reference_matches": { + "128:32": false, + "128:128": false, + "512:32": true, + "512:128": false, + "2048:32": true, + "2048:128": false, + "8192:32": true, + "8192:128": true + }, + "full_batch_observed_both_modes": true, + "timings": { + "whole": [ + { + "input": 128, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 50.049922555917874, + "output_tok_s": 10.229786058669164, + "median_first_sse_s": 25.758454302500468, + "median_tpot_est_s": 0.05179539109550176, + "median_e2e_s": 27.36027618148364, + "all_text_equal": true + } + }, + { + "input": 128, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 56.11296734504867, + "output_tok_s": 36.49780250269927, + "median_first_sse_s": 26.526507069997024, + "median_tpot_est_s": 0.05345094197257386, + "median_e2e_s": 33.31220236502122, + "all_text_equal": true + } + }, + { + "input": 512, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 51.271326894988306, + "output_tok_s": 9.986088346195059, + "median_first_sse_s": 26.151025644969195, + "median_tpot_est_s": 0.05170970569146917, + "median_e2e_s": 27.752684108447284, + "all_text_equal": true + } + }, + { + "input": 512, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 57.62661516305525, + "output_tok_s": 35.53913403043295, + "median_first_sse_s": 27.61384799046209, + "median_tpot_est_s": 0.054331971653260466, + "median_e2e_s": 34.482641289476305, + "all_text_equal": true + } + }, + { + "input": 2048, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 53.58345978904981, + "output_tok_s": 9.555187403270871, + "median_first_sse_s": 27.772490268049296, + "median_tpot_est_s": 0.05183650716020155, + "median_e2e_s": 29.37373871501768, + "all_text_equal": true + } + }, + { + "input": 2048, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 58.01774567400571, + "output_tok_s": 35.29954458257392, + "median_first_sse_s": 27.49760697502643, + "median_tpot_est_s": 0.057903295976392134, + "median_e2e_s": 34.92069034004817, + "all_text_equal": true + } + }, + { + "input": 8192, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 57.2509332629852, + "output_tok_s": 8.943085655007593, + "median_first_sse_s": 30.254718909505755, + "median_tpot_est_s": 0.132154549291778, + "median_e2e_s": 34.550595093052834, + "all_text_equal": true + } + }, + { + "input": 8192, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 74.40545622701757, + "output_tok_s": 27.524863146479106, + "median_first_sse_s": 37.36498544801725, + "median_tpot_est_s": 0.08214601906296966, + "median_e2e_s": 48.32147652748972, + "all_text_equal": true + } + } + ], + "stream": [ + { + "input": 128, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 38.72618484101258, + "output_tok_s": 13.221028668379734, + "median_first_sse_s": 19.677395230042748, + "median_tpot_est_s": 0.05387854157954515, + "median_e2e_s": 21.347630019008648, + "all_text_equal": true + } + }, + { + "input": 128, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 44.31525360001251, + "output_tok_s": 46.21433555328727, + "median_first_sse_s": 19.956942630466074, + "median_tpot_est_s": 0.05568287057864414, + "median_e2e_s": 27.037517234974075, + "all_text_equal": true + } + }, + { + "input": 512, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 39.76504473306704, + "output_tok_s": 12.875629926658702, + "median_first_sse_s": 20.17587930749869, + "median_tpot_est_s": 0.056682406549131675, + "median_e2e_s": 21.9510454685078, + "all_text_equal": true + } + }, + { + "input": 512, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 45.533834114903584, + "output_tok_s": 44.97754339843026, + "median_first_sse_s": 20.588218804448843, + "median_tpot_est_s": 0.058182282834387145, + "median_e2e_s": 27.94050809700275, + "all_text_equal": true + } + }, + { + "input": 2048, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 43.27228118001949, + "output_tok_s": 11.832054748165449, + "median_first_sse_s": 22.155220676970202, + "median_tpot_est_s": 0.06706949591922062, + "median_e2e_s": 24.21183859294979, + "all_text_equal": true + } + }, + { + "input": 2048, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 50.82957261195406, + "output_tok_s": 40.29150541231096, + "median_first_sse_s": 23.262191632995382, + "median_tpot_est_s": 0.06721498609032833, + "median_e2e_s": 31.697799462999683, + "all_text_equal": true + } + }, + { + "input": 8192, + "output_per_request": 32, + "summary": { + "requests": 16, + "output_tokens": 512, + "wall_s": 55.82834975596052, + "output_tok_s": 9.170967836915802, + "median_first_sse_s": 30.056129633041564, + "median_tpot_est_s": 0.1493107068235986, + "median_e2e_s": 34.67807955702301, + "all_text_equal": true + } + }, + { + "input": 8192, + "output_per_request": 128, + "summary": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 71.63892357901204, + "output_tok_s": 28.587810895025783, + "median_first_sse_s": 34.93417623353889, + "median_tpot_est_s": 0.09393472040180248, + "median_e2e_s": 47.123385227518156, + "all_text_equal": true + } + } + ] + } +} diff --git a/experiments/evidence270/atom270-allcore-pd-quality-audit.json b/experiments/evidence270/atom270-allcore-pd-quality-audit.json new file mode 100644 index 00000000..e67c41ab --- /dev/null +++ b/experiments/evidence270/atom270-allcore-pd-quality-audit.json @@ -0,0 +1,68 @@ +{ + "modes": { + "whole": { + "cases": 36, + "answer_correct": 36, + "format_correct": 36, + "eos_stopped": 36, + "length_limited": 0, + "active_lanes": [ + 0, + 230, + 38, + 0, + 0 + ], + "full_batch_observed": false, + "historical_pd_comparison": { + "cases": 36, + "correct_to_wrong": 0, + "wrong_to_correct": 0, + "token_equal": 36 + }, + "native_allcore_comparison": { + "cases": 36, + "correct_to_wrong": 0, + "wrong_to_correct": 0, + "token_equal": 36 + }, + "raw_sha256": "f868c1bf6d024a8284e362ea2a89b55ba4b71cf02513023d1ba55fd581565070" + }, + "stream": { + "cases": 36, + "answer_correct": 36, + "format_correct": 36, + "eos_stopped": 36, + "length_limited": 0, + "active_lanes": [ + 0, + 233, + 23, + 9, + 0 + ], + "full_batch_observed": false, + "historical_pd_comparison": { + "cases": 36, + "correct_to_wrong": 0, + "wrong_to_correct": 0, + "token_equal": 36 + }, + "native_allcore_comparison": { + "cases": 36, + "correct_to_wrong": 0, + "wrong_to_correct": 0, + "token_equal": 36 + }, + "raw_sha256": "4016161cc6d72357ae8a76b911ccc39199d53093c9f63d46fe5323cb90be6c2b" + } + }, + "limits": "Frozen36cases per route, short/2K/8K,c12. Not general quality or performance; actual occupancy is separate from submitted concurrency.", + "whole_stream_comparison": { + "cases": 36, + "correct_to_wrong": 0, + "wrong_to_correct": 0, + "token_equal": 36 + }, + "all_answers_and_formats_pass": true +} diff --git a/experiments/evidence270/atom270-allcore-quality-audit.json b/experiments/evidence270/atom270-allcore-quality-audit.json new file mode 100644 index 00000000..45007a15 --- /dev/null +++ b/experiments/evidence270/atom270-allcore-quality-audit.json @@ -0,0 +1,28 @@ +{ + "cases": 36, + "answer_correct": 36, + "format_correct": 36, + "baseline_comparison": { + "cases": 36, + "correct_to_wrong": 0, + "wrong_to_correct": 0, + "token_equal": 36 + }, + "eos_stopped": 36, + "active_lanes": [ + 0, + 306, + 0, + 0, + 0 + ], + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "raw_sha256": "d55ad59583febee514dfc507f932d579cf32d1435eb5fcff63eb89f4001c8626", + "cleanup_exit_code": 0, + "scope": "Sequential fixed36cases, short/2K/8K; no concurrent B4 quality, general quality or speedup claim." +} diff --git a/experiments/evidence270/atom270-allcore-quality-c12-audit.json b/experiments/evidence270/atom270-allcore-quality-c12-audit.json new file mode 100644 index 00000000..9cd26c30 --- /dev/null +++ b/experiments/evidence270/atom270-allcore-quality-c12-audit.json @@ -0,0 +1,28 @@ +{ + "cases": 36, + "answer_correct": 36, + "format_correct": 36, + "baseline_comparison": { + "cases": 36, + "correct_to_wrong": 0, + "wrong_to_correct": 0, + "token_equal": 36 + }, + "eos_stopped": 36, + "active_lanes": [ + 0, + 13, + 5, + 9, + 64 + ], + "geometry": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "raw_sha256": "97dc5c9bb163f455dc6a0a85733c84de6f45dbd23f79fff0535d02a4ffa2d7be", + "cleanup_exit_code": 0, + "scope": "Three concurrent12-request cohorts,36frozen cases,short/2K/8K; actual B4 occupancy. No general quality or speedup claim." +} diff --git a/experiments/evidence270/atom270-allcoreprobe-audit.json b/experiments/evidence270/atom270-allcoreprobe-audit.json new file mode 100644 index 00000000..9904edf9 --- /dev/null +++ b/experiments/evidence270/atom270-allcoreprobe-audit.json @@ -0,0 +1,58 @@ +{ + "complete": true, + "compile_seconds": 159.2537886829814, + "probes": [ + { + "input_tokens": 2048, + "index": 12, + "tokens": [ + 4588, + 4588, + 4588, + 4588 + ], + "all_hidden_and_attention_outputs_equal": true, + "full_vocabulary_logits_equal": true + }, + { + "input_tokens": 2048, + "index": 13, + "tokens": [ + 369, + 369, + 369, + 369 + ], + "all_hidden_and_attention_outputs_equal": true, + "full_vocabulary_logits_equal": true + }, + { + "input_tokens": 512, + "index": 49, + "tokens": [ + 807, + 807, + 807, + 807 + ], + "all_hidden_and_attention_outputs_equal": true, + "full_vocabulary_logits_equal": true + }, + { + "input_tokens": 512, + "index": 50, + "tokens": [ + 4829, + 4829, + 4829, + 4829 + ], + "all_hidden_and_attention_outputs_equal": true, + "full_vocabulary_logits_equal": true + } + ], + "checked_boundary_outputs": 292, + "raw_sha256": "32f89a5887a7bf8b69561ba78c2245b2d91baf04585bc283b39f9dfae5ef555c", + "scope": "Two forced-history fixtures, four probe positions; not free-generation/native B1/quality/performance acceptance.", + "cleanup_exit_code": 0 +} diff --git a/experiments/evidence270/atom270-attentionraw-audit.json b/experiments/evidence270/atom270-attentionraw-audit.json new file mode 100644 index 00000000..082da066 --- /dev/null +++ b/experiments/evidence270/atom270-attentionraw-audit.json @@ -0,0 +1,821 @@ +{ + "complete": true, + "compile_seconds": 152.58750150108244, + "raw_sha256": "95ea83ef9bac821055405ab9aaedd3bc044076b619889caa0590f6d5202d3856", + "scope": "Instrumentation changed numerical behavior; cannot attribute original failure to a particular operator. Later retained boundaries still differ.", + "probes": [ + { + "input_tokens": 2048, + "index": 12, + "tokens": [ + 4588, + 4588, + 4588, + 4588 + ], + "prior_tokens": [ + 4588, + 4588, + 4588, + 4588 + ], + "token_patterns_equal": true, + "prior_logit_records_equal": false, + "diagnostic_outputs": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "q_proj", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "k_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 3, + "label": "v_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 4, + "label": "q_norm", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 5, + "label": "k_norm", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 6, + "label": "rotary_q", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 7, + "label": "rotary_k", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 8, + "label": "attention_v", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 9, + "label": "attention_result", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 10, + "label": "o_proj", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + } + ], + "first_retained_differing_boundary": "original_boundary_11" + }, + { + "input_tokens": 2048, + "index": 13, + "tokens": [ + 369, + 264, + 264, + 264 + ], + "prior_tokens": [ + 369, + 264, + 264, + 264 + ], + "token_patterns_equal": true, + "prior_logit_records_equal": false, + "diagnostic_outputs": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "q_proj", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "k_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 3, + "label": "v_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 4, + "label": "q_norm", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 5, + "label": "k_norm", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 6, + "label": "rotary_q", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 7, + "label": "rotary_k", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 8, + "label": "attention_v", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 9, + "label": "attention_result", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 10, + "label": "o_proj", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + } + ], + "first_retained_differing_boundary": "original_boundary_11" + }, + { + "input_tokens": 512, + "index": 49, + "tokens": [ + 807, + 807, + 807, + 807 + ], + "prior_tokens": [ + 807, + 807, + 807, + 807 + ], + "token_patterns_equal": true, + "prior_logit_records_equal": false, + "diagnostic_outputs": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "q_proj", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "k_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 3, + "label": "v_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 4, + "label": "q_norm", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 5, + "label": "k_norm", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 6, + "label": "rotary_q", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 7, + "label": "rotary_k", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 8, + "label": "attention_v", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 9, + "label": "attention_result", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 10, + "label": "o_proj", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + } + ], + "first_retained_differing_boundary": "original_boundary_11" + }, + { + "input_tokens": 512, + "index": 50, + "tokens": [ + 4829, + 4829, + 4829, + 4829 + ], + "prior_tokens": [ + 4829, + 1033, + 1033, + 1033 + ], + "token_patterns_equal": false, + "prior_logit_records_equal": false, + "diagnostic_outputs": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "q_proj", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "k_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 3, + "label": "v_proj", + "shape": [ + 4, + 1, + 1024 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 4, + "label": "q_norm", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 5, + "label": "k_norm", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 6, + "label": "rotary_q", + "shape": [ + 4, + 32, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 7, + "label": "rotary_k", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 8, + "label": "attention_v", + "shape": [ + 4, + 8, + 1, + 128 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 9, + "label": "attention_result", + "shape": [ + 4, + 1, + 4096 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 10, + "label": "o_proj", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + } + ], + "first_retained_differing_boundary": "original_boundary_11" + } + ], + "jit_fallback_counts": { + "hidden": 0, + "layerprobe": 2, + "attentionraw": 2 + } +} diff --git a/experiments/evidence270/atom270-causal-gemma-b1-three-route-summary.json b/experiments/evidence270/atom270-causal-gemma-b1-three-route-summary.json new file mode 100644 index 00000000..6ed0bfd3 --- /dev/null +++ b/experiments/evidence270/atom270-causal-gemma-b1-three-route-summary.json @@ -0,0 +1,245 @@ +{ + "stage": "three_routes_audited", + "audits": { + "native": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "whole": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "stream": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + } + }, + "result_sha256": { + "native": "76b81b2728303015358810b1a76391f5bad2f63291ab4b8aad39c44ced7d19b9", + "whole": "22fc823a863bd5ea01a80e7cad1ab01ce8519a89f1ab504f2c2eee8fe74bc54b", + "stream": "2fa51d84d752b2d370dc73504194835cd282aced9ce980e37557b92ef7c4ada1" + }, + "eos_ids": [ + 1, + 106 + ], + "comparisons": { + "native_whole": { + "128:32": { + "full": false, + "through_eos": true + }, + "128:128": { + "full": false, + "through_eos": true + }, + "512:32": { + "full": true, + "through_eos": true + }, + "512:128": { + "full": true, + "through_eos": true + }, + "2048:32": { + "full": true, + "through_eos": true + }, + "2048:128": { + "full": true, + "through_eos": true + }, + "8192:32": { + "full": true, + "through_eos": true + }, + "8192:128": { + "full": true, + "through_eos": true + } + }, + "whole_stream": { + "128:32": { + "full": true, + "through_eos": true + }, + "128:128": { + "full": true, + "through_eos": true + }, + "512:32": { + "full": true, + "through_eos": true + }, + "512:128": { + "full": true, + "through_eos": true + }, + "2048:32": { + "full": true, + "through_eos": true + }, + "2048:128": { + "full": true, + "through_eos": true + }, + "8192:32": { + "full": true, + "through_eos": true + }, + "8192:128": { + "full": true, + "through_eos": true + } + }, + "native_stream": { + "128:32": { + "full": false, + "through_eos": true + }, + "128:128": { + "full": false, + "through_eos": true + }, + "512:32": { + "full": true, + "through_eos": true + }, + "512:128": { + "full": true, + "through_eos": true + }, + "2048:32": { + "full": true, + "through_eos": true + }, + "2048:128": { + "full": true, + "through_eos": true + }, + "8192:32": { + "full": true, + "through_eos": true + }, + "8192:128": { + "full": true, + "through_eos": true + } + } + }, + "metrics_8192_128": { + "native": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 35.574109970009886, + "output_tok_s": 57.56995752603592, + "median_first_sse_s": 17.3802980305627, + "median_tpot_est_s": 0.011212882113745656, + "median_e2e_s": 18.801305017026607, + "all_text_equal": true + }, + "whole": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 51.24334673094563, + "output_tok_s": 39.966164012531635, + "median_first_sse_s": 25.982786704960745, + "median_tpot_est_s": 0.01119109856718271, + "median_e2e_s": 27.402480554475915, + "all_text_equal": true + }, + "stream": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 47.38516508892644, + "output_tok_s": 43.220277826542855, + "median_first_sse_s": 23.757880949531682, + "median_tpot_est_s": 0.011299933598369477, + "median_e2e_s": 25.19775077648228, + "all_text_equal": true + } + }, + "stream_whole_pct": 8.142171995768411, + "limits": "One timed batch per condition, same corrected B1k5 artifact and ATOM. Native uses one ATOM; PD adds one RNGD. Through-EOS agreement is not full-sequence equality or semantic-quality evidence. Not statistical speedup evidence." +} diff --git a/experiments/evidence270/atom270-causal-gemma-b4-whole-occupancy-failure.json b/experiments/evidence270/atom270-causal-gemma-b4-whole-occupancy-failure.json new file mode 100644 index 00000000..606326e3 --- /dev/null +++ b/experiments/evidence270/atom270-causal-gemma-b4-whole-occupancy-failure.json @@ -0,0 +1,89 @@ +{ + "http_audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "batch_occupancy_audit": "FAILED: no successful decode call with four active lanes", + "active_lane_histogram": [ + 0, + 14313, + 3132, + 93, + 0 + ], + "comparisons": { + "128:32": { + "full": true, + "through_eos": true, + "first_difference": null + }, + "128:128": { + "full": true, + "through_eos": true, + "first_difference": null + }, + "512:32": { + "full": false, + "through_eos": true, + "first_difference": 8 + }, + "512:128": { + "full": false, + "through_eos": true, + "first_difference": 8 + }, + "2048:32": { + "full": true, + "through_eos": true, + "first_difference": null + }, + "2048:128": { + "full": true, + "through_eos": true, + "first_difference": null + }, + "8192:32": { + "full": true, + "through_eos": true, + "first_difference": null + }, + "8192:128": { + "full": true, + "through_eos": true, + "first_difference": null + } + }, + "metrics_8192_128": { + "native": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 21.52105758502148, + "output_tok_s": 95.16260954691163, + "median_first_sse_s": 10.135787407518364, + "median_tpot_est_s": 0.036070315672843675, + "median_e2e_s": 14.72101910848869, + "all_text_equal": true + }, + "whole": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.56643162900582, + "output_tok_s": 77.0897660852558, + "median_first_sse_s": 13.33676557202125, + "median_tpot_est_s": 0.017586364243986217, + "median_e2e_s": 15.773472197470255, + "all_text_equal": true + } + }, + "sources": { + "atom270-causal-gemma-tp1-b4-native.json": "d9b5752d5fefe4a26d95118699c7ee50407ab20fc7ea9ee1909fb0a1eec41e78", + "atom270-causal-gemma-tp1-b4-native.server0.log": "1b76982780c2e0e4a181b6140f428c447e5696ff599d74bc5f7db9326b5e65e2", + "atom270-causal-gemma-tp1-b4-whole.json": "0750435be2d4dc9c7ace05e59fc29a5362153b0018e9698755ef24fd1c78bbf5", + "atom270-causal-gemma-tp1-b4-whole.server0.log": "da25643b1628e610cb4c8d1f96e31a12a73b789dde51da872045a1fe616a461a" + }, + "stream": "not run: preceding occupancy audit failed", + "limits": "One timed batch; same physical ATOM for native/whole. Within-route correctness is not native parity or answer quality. No device DMA overlap proof." +} diff --git a/experiments/evidence270/atom270-causal-gemma-native-batch-summary.json b/experiments/evidence270/atom270-causal-gemma-native-batch-summary.json new file mode 100644 index 00000000..05728e15 --- /dev/null +++ b/experiments/evidence270/atom270-causal-gemma-native-batch-summary.json @@ -0,0 +1,127 @@ +{ + "audits": { + "1": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "4": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 4, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 4, + "active_lane_histogram": [ + 0, + 664, + 32, + 32, + 5008 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "2": 0 + }, + "opened": [ + "/dev/rbln2" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + } + }, + "comparisons": { + "128:32": { + "full": false, + "through_eos": true + }, + "128:128": { + "full": false, + "through_eos": true + }, + "512:32": { + "full": false, + "through_eos": true + }, + "512:128": { + "full": false, + "through_eos": true + }, + "2048:32": { + "full": true, + "through_eos": true + }, + "2048:128": { + "full": true, + "through_eos": true + }, + "8192:32": { + "full": true, + "through_eos": true + }, + "8192:128": { + "full": true, + "through_eos": true + } + }, + "metrics_8192_128": { + "1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 35.574109970009886, + "output_tok_s": 57.56995752603592, + "median_first_sse_s": 17.3802980305627, + "median_tpot_est_s": 0.011212882113745656, + "median_e2e_s": 18.801305017026607, + "all_text_equal": true + }, + "4": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 21.52105758502148, + "output_tok_s": 95.16260954691163, + "median_first_sse_s": 10.135787407518364, + "median_tpot_est_s": 0.036070315672843675, + "median_e2e_s": 14.72101910848869, + "all_text_equal": true + } + }, + "b4_vs_b1_ratio": 1.6529907895776106, + "limits": "Single measured batch per condition; compiled batch and physical card can differ (bindings retained). Full sequence parity is reported separately from through-EOS and within-route parity. No statistical claim." +} diff --git a/experiments/evidence270/atom270-causal-gemma-native-comparison.json b/experiments/evidence270/atom270-causal-gemma-native-comparison.json new file mode 100644 index 00000000..bf4a5e99 --- /dev/null +++ b/experiments/evidence270/atom270-causal-gemma-native-comparison.json @@ -0,0 +1,99 @@ +{ + "audit": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "source_sha256": "76b81b2728303015358810b1a76391f5bad2f63291ab4b8aad39c44ced7d19b9", + "eos_ids": [ + 1, + 106 + ], + "comparisons": { + "128:32": { + "old_native_full": true, + "previous_whole_full": false, + "previous_whole_through_eos": true, + "first_difference": 5 + }, + "128:128": { + "old_native_full": true, + "previous_whole_full": false, + "previous_whole_through_eos": true, + "first_difference": 5 + }, + "512:32": { + "old_native_full": true, + "previous_whole_full": true, + "previous_whole_through_eos": true, + "first_difference": null + }, + "512:128": { + "old_native_full": true, + "previous_whole_full": true, + "previous_whole_through_eos": true, + "first_difference": null + }, + "2048:32": { + "old_native_full": true, + "previous_whole_full": true, + "previous_whole_through_eos": true, + "first_difference": null + }, + "2048:128": { + "old_native_full": true, + "previous_whole_full": true, + "previous_whole_through_eos": true, + "first_difference": null + }, + "8192:32": { + "old_native_full": true, + "previous_whole_full": true, + "previous_whole_through_eos": true, + "first_difference": null + }, + "8192:128": { + "old_native_full": true, + "previous_whole_full": true, + "previous_whole_through_eos": true, + "first_difference": null + } + }, + "metrics_8192_128": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 35.574109970009886, + "output_tok_s": 57.56995752603592, + "median_first_sse_s": 17.3802980305627, + "median_tpot_est_s": 0.011212882113745656, + "median_e2e_s": 18.801305017026607, + "all_text_equal": true + }, + "limits": "Historical corrected k3 native and prior k5 PD references, same prompt IDs. Current corrected whole/stream still running. One timed batch; no statistical claim. Full128-token fixture continues after EOS; report strict and through-EOS separately." +} diff --git a/experiments/evidence270/atom270-causal-gemma-native-whole-summary.json b/experiments/evidence270/atom270-causal-gemma-native-whole-summary.json new file mode 100644 index 00000000..86a0ec51 --- /dev/null +++ b/experiments/evidence270/atom270-causal-gemma-native-whole-summary.json @@ -0,0 +1,147 @@ +{ + "audits": { + "native": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + }, + "whole": { + "stage": "audited", + "responses": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "compiled": { + "batch_size": 1, + "kv_blocks": 5, + "context": 9216, + "tensor_parallel_size": 1 + }, + "max_active_lanes": 1, + "active_lane_histogram": [ + 0, + 20856 + ], + "binding": { + "visible_count": 1, + "physical_to_visible": { + "0": 0 + }, + "opened": [ + "/dev/rbln0" + ] + }, + "limits": "Full HTTP token/text checks plus successful decode-call lane counts. Not device utilization, answer quality or statistical speedup evidence." + } + }, + "comparisons": { + "128:32": { + "full": false, + "through_eos": true, + "first_difference": 5, + "whole_unchanged_from_old": true + }, + "128:128": { + "full": false, + "through_eos": true, + "first_difference": 5, + "whole_unchanged_from_old": true + }, + "512:32": { + "full": true, + "through_eos": true, + "first_difference": null, + "whole_unchanged_from_old": true + }, + "512:128": { + "full": true, + "through_eos": true, + "first_difference": null, + "whole_unchanged_from_old": true + }, + "2048:32": { + "full": true, + "through_eos": true, + "first_difference": null, + "whole_unchanged_from_old": true + }, + "2048:128": { + "full": true, + "through_eos": true, + "first_difference": null, + "whole_unchanged_from_old": true + }, + "8192:32": { + "full": true, + "through_eos": true, + "first_difference": null, + "whole_unchanged_from_old": true + }, + "8192:128": { + "full": true, + "through_eos": true, + "first_difference": null, + "whole_unchanged_from_old": true + } + }, + "eos_ids": [ + 1, + 106 + ], + "metrics_8192_128": { + "native": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 35.574109970009886, + "output_tok_s": 57.56995752603592, + "median_first_sse_s": 17.3802980305627, + "median_tpot_est_s": 0.011212882113745656, + "median_e2e_s": 18.801305017026607, + "all_text_equal": true + }, + "whole": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 51.24334673094563, + "output_tok_s": 39.966164012531635, + "median_first_sse_s": 25.982786704960745, + "median_tpot_est_s": 0.01119109856718271, + "median_e2e_s": 27.402480554475915, + "all_text_equal": true + } + }, + "load_s": { + "native": 9.349328789045103, + "whole": 837.6305059780134 + }, + "limits": "One timed batch, same ATOM and compiled artifact; native/PD full6/8 and throughEOS8/8. No statistical or answer-quality claim; stream still running." +} diff --git a/experiments/evidence270/atom270-causal-sdk-manifest.json b/experiments/evidence270/atom270-causal-sdk-manifest.json new file mode 100644 index 00000000..e1d14082 --- /dev/null +++ b/experiments/evidence270/atom270-causal-sdk-manifest.json @@ -0,0 +1,24 @@ +{ + "atom": { + "python": "3.10.12", + "packages": { + "torch": "2.11.0+cpu", + "transformers": "5.8.1", + "rebel-compiler": "0.11.0", + "optimum-rbln": "0.11.0.post1", + "furiosa-torch": null, + "furiosa-llm": null + } + }, + "rngd": { + "python": "3.10.12", + "packages": { + "torch": "2.10.0+cpu", + "transformers": "5.19.0", + "rebel-compiler": null, + "optimum-rbln": null, + "furiosa-torch": "2026.4.0", + "furiosa-llm": "2026.4.0" + } + } +} diff --git a/experiments/evidence270/atom270-causal-source-manifest.json b/experiments/evidence270/atom270-causal-source-manifest.json new file mode 100644 index 00000000..02d8143f --- /dev/null +++ b/experiments/evidence270/atom270-causal-source-manifest.json @@ -0,0 +1,422 @@ +{ + "scope": "On-disk source snapshot during corrected Gemma B1 whole startup; not proof of every imported dependency or exact in-memory code.", + "files": { + "container_bootstrap270.py": { + "sha256": "fdde633301da000e59c6d7be26aefbef404bf300769d7f81e6ef285c051c62e9", + "bytes": 3151 + }, + "atom_server270_http.py": { + "sha256": "e34a3eeb79c1d0a9547a21b6ad7f55c31484641661616edb1cba9e590119d278", + "bytes": 7147 + }, + "benchmark270_http.py": { + "sha256": "6a4f2ea720ab35fb8478fd172244a4776824af6641c3d5fe4d1e446262767751", + "bytes": 315 + }, + "benchmark267_deadline.py": { + "sha256": "c8ebb077a608e3da70a9748aed19c03f22ae87355f0167a2fb1e68b88ec21d79", + "bytes": 15230 + }, + "llmd267_backlog.py": { + "sha256": "4db30f755a1a1ecfe2f895ae01ef9327b9c582577e94dfedb1ab131aa51a2bb4", + "bytes": 33092 + }, + "streaming_factories.py": { + "sha256": "77745350973cc54bf508e0c8b473efc442afef20b1c64f7c4edc179ea1471389", + "bytes": 3517 + }, + "gemma_prefill.py": { + "sha256": "35791a3d1a9156533da6e0d8ed0bfacf544f5ac76085be3925251a40774d0353", + "bytes": 3712 + }, + "gemma_parts.py": { + "sha256": "51209fcd511bafe311e7b990b92ec73499de72949b8b6dc83139cbd420554617", + "bytes": 5164 + }, + "furiosa_prefill_python.sh": { + "sha256": "317036f016a063d4495f8d81af9523b327a95a52bc7d1d740d664e5f197a0edc", + "bytes": 717 + }, + "run-http270-atom.sh": { + "sha256": "b03644ed32448479cda724dbaf106f4a8dbb79a363e11ecc0bb2cf0ade84cfbb", + "bytes": 913 + }, + "gemma_causal_compat.py": { + "sha256": "c4a48fa1cce28865405a3211559e0ff2556b893072b789e9543b96bd34907930", + "bytes": 766 + }, + "gemma_global_causal_compat.py": { + "sha256": "c4c51d76dec643a8b7643e3f127c0e067a7ebb735b38bf67cb73212e20db17cb", + "bytes": 961 + }, + "stream265-code/opt_prime/IR.py": { + "sha256": "861c59b8e6467c2d2846e4d49ea5c51917a65ed2628f5cb9b6eb95da121cf02f", + "bytes": 145910 + }, + "stream265-code/opt_prime/__init__.py": { + "sha256": "6a60757b7460af785a9ea6c10f54ae2c012c6d7effbf72b0ca55b9f32ec9a3f5", + "bytes": 1780 + }, + "stream265-code/opt_prime/accel/__init__.py": { + "sha256": "f470435c102151bbbc982209ac82fa0d18fd587646d493f9cbfe9f2059120ab8", + "bytes": 1197 + }, + "stream265-code/opt_prime/accel/engines/__init__.py": { + "sha256": "8a66570663cd43a05c6d8b6a61ff1c8d4d2a7aea9268a0a8a14c7cb3275f5e37", + "bytes": 1788 + }, + "stream265-code/opt_prime/accel/engines/base.py": { + "sha256": "ada8e3363f44f9c250386c4d231682c19a20bc7dfff37caf66bf015ab83c34d7", + "bytes": 3139 + }, + "stream265-code/opt_prime/accel/engines/optimus.py": { + "sha256": "acd06d624343c1ada74a45a88378ef6deefef2d4bd2e3a0e87b1a496ee5f1994", + "bytes": 1578 + }, + "stream265-code/opt_prime/accel/moe/__init__.py": { + "sha256": "777ec452dd7e039f2c5a43e1e2582caf172567bd2744c89d492f5cbb90b892a0", + "bytes": 3036 + }, + "stream265-code/opt_prime/accel/moe/api.py": { + "sha256": "0f86bc3dba8c04fef69193ea7a8336e9727e12f138d5c58664c04e9122e54713", + "bytes": 5932 + }, + "stream265-code/opt_prime/accel/moe/backends.py": { + "sha256": "95f523ae8823f351ce09fed89ae5055d2332921f2ecc32591645a3ca258baead", + "bytes": 4764 + }, + "stream265-code/opt_prime/accel/moe/batched.py": { + "sha256": "aa9b2ba8766e26698eded50e4b11a112effe85225d73f45289e6397217fde5dc", + "bytes": 3824 + }, + "stream265-code/opt_prime/accel/moe/config.py": { + "sha256": "9c769a40f9c311e3e5cb00054a047c745a50505818154a20de84f9f9c9847230", + "bytes": 5636 + }, + "stream265-code/opt_prime/accel/moe/layer.py": { + "sha256": "387ad5c16368eb9a07039852ed73301f133f7303d2f70340738fe9c88d90ce7f", + "bytes": 22264 + }, + "stream265-code/opt_prime/accel/moe/ops.py": { + "sha256": "7b4ca318dd0d564ad2d1f98e7074bd81613f5c40a03c06ca60ecadfe84873ac8", + "bytes": 3121 + }, + "stream265-code/opt_prime/accel/moe/placement.py": { + "sha256": "ea5ba8f6fa67942dbb343e508f210075f0a8682085c635fc7d02a8ce43c58268", + "bytes": 1602 + }, + "stream265-code/opt_prime/accel/moe/planner.py": { + "sha256": "71446a3057d3b74b1d5df9066f9d6b9b6c9dff3371df42283a86989b3cdc4aff", + "bytes": 6993 + }, + "stream265-code/opt_prime/accel/moe/profiler.py": { + "sha256": "4a8e178c93ee4111872e78aa814361d2083a860d913e008f4338171ec94e0d79", + "bytes": 2129 + }, + "stream265-code/opt_prime/accel/moe/train.py": { + "sha256": "c45799a29e5b6f31e1f243a98cf89fc131c3442de324781fbb47aa1199ae40b7", + "bytes": 2180 + }, + "stream265-code/opt_prime/accel/pd/__init__.py": { + "sha256": "5ddd216926c7a81cc30a115bf966af863d9f90ba8ced04f6920b07591acdfde6", + "bytes": 540 + }, + "stream265-code/opt_prime/accel/pd/batching.py": { + "sha256": "94454441abe16037f221b48d4a7f082b4738aa222765af33216dffa48b763755", + "bytes": 13671 + }, + "stream265-code/opt_prime/accel/pd/capabilities.py": { + "sha256": "d7f5aca335aae432e21b1afb37f87cce67842996e5b4fe0ddb02247e2698598d", + "bytes": 1558 + }, + "stream265-code/opt_prime/accel/pd/contracts.py": { + "sha256": "390ba741e1fe42cf04433917aa4b7320794f0eb43a51a74e45d54f814781f941", + "bytes": 7162 + }, + "stream265-code/opt_prime/accel/pd/hetero_p2p.py": { + "sha256": "371e6c21cf53b57c37119f20d4011806393618d403ab53a9e4cb7bb6fb7900a4", + "bytes": 1662 + }, + "stream265-code/opt_prime/accel/pd/koni.py": { + "sha256": "c1d255d80627c351ad23b6313713cd32831f73f55d69425256b4db0455bfa4fa", + "bytes": 6007 + }, + "stream265-code/opt_prime/accel/pd/kvstore.py": { + "sha256": "e3ae98f1f72d64f96b2b71bdef889db9e751a905488a2d151ec6fc262bb1cb81", + "bytes": 7511 + }, + "stream265-code/opt_prime/accel/pd/layer_progress.py": { + "sha256": "19c8853cd20b386dc11064d4be63fa696468c86a26b76ecf5c48cdf3905a700b", + "bytes": 2529 + }, + "stream265-code/opt_prime/accel/pd/npu/__init__.py": { + "sha256": "b39b3b5363fe22a699feee7298ca212b2be12618076a246b3d095438a2200450", + "bytes": 280 + }, + "stream265-code/opt_prime/accel/pd/npu/engines.py": { + "sha256": "15e6f5815a0f89ece93bf65894e128683ca132e948cfbc8d1f483e0c268ce9da", + "bytes": 5985 + }, + "stream265-code/opt_prime/accel/pd/npu/factory.py": { + "sha256": "918dda5f9e80fe5b0b39e0d8372d4dc811d0e3ffa8105ced93d88a27d6210d1e", + "bytes": 8761 + }, + "stream265-code/opt_prime/accel/pd/npu/model.py": { + "sha256": "973c0b105b3890f286b4bca160844432b42beb67587f8c7db7232be60ff2a384", + "bytes": 3770 + }, + "stream265-code/opt_prime/accel/pd/npu/provenance.py": { + "sha256": "6222654b7ccaf284458f69c17be1e4fe809b2ce70479d836f2aaac8362f30891", + "bytes": 1648 + }, + "stream265-code/opt_prime/accel/pd/npu/shared_kv.py": { + "sha256": "6b6a1fb1744f02ccd93f394ef7dd309ce506e7c148f6f61fa6ec8fddaa6e8905", + "bytes": 2106 + }, + "stream265-code/opt_prime/accel/pd/npu/worker.py": { + "sha256": "744fd7a11111454cf4757be04fa866169f3f33a0f10875158c004d8a94c12f22", + "bytes": 7523 + }, + "stream265-code/opt_prime/accel/pd/paged.py": { + "sha256": "c9cfa1d39b9795a53fa2387ee4e5ac1db76a85f769597d0b942ad480eceb43c6", + "bytes": 31631 + }, + "stream265-code/opt_prime/accel/pd/pd_batching.py": { + "sha256": "c827775c1d399bf724a9747c3424ee5ea69f93054493bfbb71d5f0cf08fff65b", + "bytes": 32215 + }, + "stream265-code/opt_prime/accel/pd/prefill_server.py": { + "sha256": "5af1169ff5c6bfac6ce265ea3edf7f6a031c207bda7ce7bcbc996d7133576b1d", + "bytes": 15515 + }, + "stream265-code/opt_prime/accel/pd/runtime.py": { + "sha256": "f61b3803941e95ab2dc3b2fac4356146323320f1363836d115f552a852b40404", + "bytes": 9093 + }, + "stream265-code/opt_prime/accel/pd/sampling.py": { + "sha256": "8ee1303875cc1a58660fda75f014bec6b007f6424ec044dcb28bf75e98b966d2", + "bytes": 1204 + }, + "stream265-code/opt_prime/accel/pd/torch_cache.py": { + "sha256": "af42f8219e84cdb13e1303a4c6ee51dd36bed59ddc84b168c3347cbe0fd61796", + "bytes": 3211 + }, + "stream265-code/opt_prime/accel/pd/torch_engine.py": { + "sha256": "5f884074d1b920f026549672625ff062681d8596b6b8a689597f411a220b9d0e", + "bytes": 7410 + }, + "stream265-code/opt_prime/accel/pd/torch_models.py": { + "sha256": "3fa209f9649a868602dd97b030801d9e32ad74a144d83317dce75b96e5d1250d", + "bytes": 6088 + }, + "stream265-code/opt_prime/accel/platforms/__init__.py": { + "sha256": "6d062e2bbdb2c50cb2618de0729b01d1500d62c61b20e3cd51a24198075cbf27", + "bytes": 2144 + }, + "stream265-code/opt_prime/accel/platforms/atom/__init__.py": { + "sha256": "82a797e83449ff1624e95bc1e14fc3080a8431515909b56ca107bf1b56e76ca5", + "bytes": 61 + }, + "stream265-code/opt_prime/accel/platforms/atom/batching.py": { + "sha256": "d98a51d75a36cd8c379a6220adb92528b3607cf80145b85a032270494a429580", + "bytes": 33586 + }, + "stream265-code/opt_prime/accel/platforms/atom/device.py": { + "sha256": "8d85a6e85b6be4e3ad16e2e73f28a973bf5574b81ab6b1096dea5114eac4b9c9", + "bytes": 4994 + }, + "stream265-code/opt_prime/accel/platforms/atom/engine.py": { + "sha256": "aff783fc796278ad63b62f9ba99a2026d8a2f0dd695cd29b42ff561eeb73c153", + "bytes": 7235 + }, + "stream265-code/opt_prime/accel/platforms/atom/experts.py": { + "sha256": "101508da2369881647127b43200a831f027e96bde0fcae701b7ad0bcb365185b", + "bytes": 3657 + }, + "stream265-code/opt_prime/accel/platforms/atom/pd.py": { + "sha256": "be78e27c105a5b6d1de288f2c205bf18bbc3804e130546b29e4f350fb063980d", + "bytes": 2773 + }, + "stream265-code/opt_prime/accel/platforms/atom/pd_native.py": { + "sha256": "662ebfdaa874988fda0b4f367b273dadd98174a4aab28409fd1e0fcc2bf3ec4a", + "bytes": 5623 + }, + "stream265-code/opt_prime/accel/platforms/atom/pd_runtime.py": { + "sha256": "0e25819a56a99305f4a49c941cfe1094d2e22060c310ff5e83904473661e8018", + "bytes": 6970 + }, + "stream265-code/opt_prime/accel/platforms/capabilities.py": { + "sha256": "22eddcc844ce871fea3cba3a8530c4823ffc8c94afaff19d1e4f8105ff417442", + "bytes": 2195 + }, + "stream265-code/opt_prime/accel/platforms/cpu/__init__.py": { + "sha256": "82a797e83449ff1624e95bc1e14fc3080a8431515909b56ca107bf1b56e76ca5", + "bytes": 61 + }, + "stream265-code/opt_prime/accel/platforms/cpu/device.py": { + "sha256": "cd7310a5116a8ee8f10b10723a8c80db80a4d472494a6ecd649d4173c7a4cb52", + "bytes": 773 + }, + "stream265-code/opt_prime/accel/platforms/cpu/experts.py": { + "sha256": "43eacc7867b7d25c871a8ea74421e4adf0d9abd3f2571c1d2123c2618d76c9d4", + "bytes": 7823 + }, + "stream265-code/opt_prime/accel/platforms/cpu/native.py": { + "sha256": "000c6b5eff9e1ef3dc1528076a36bcdbb34d7910ccbb3187402e502e90f15892", + "bytes": 3274 + }, + "stream265-code/opt_prime/accel/platforms/cuda/__init__.py": { + "sha256": "82a797e83449ff1624e95bc1e14fc3080a8431515909b56ca107bf1b56e76ca5", + "bytes": 61 + }, + "stream265-code/opt_prime/accel/platforms/cuda/device.py": { + "sha256": "a89aef3677b7b63a62c186084047b473c84bbaa604b48225482d28c3f935a7e8", + "bytes": 1081 + }, + "stream265-code/opt_prime/accel/platforms/cuda/experts.py": { + "sha256": "ec5d29b9d774372c34e3f63106f7d5f46e2cc248ba8c09376d204a8de3022a97", + "bytes": 6556 + }, + "stream265-code/opt_prime/accel/platforms/lease.py": { + "sha256": "9e3bb982bcae961afa0c344e182a38ad8e87feaf27d1e2d62ccab97f88f6e61d", + "bytes": 5585 + }, + "stream265-code/opt_prime/accel/platforms/npu_graph.py": { + "sha256": "f1c01661c99530adf140544cca4a58a7d049fb0228a7bd7dc16b7d1bc5ef5bd3", + "bytes": 13336 + }, + "stream265-code/opt_prime/accel/platforms/registry.py": { + "sha256": "7c826ed1ce79569f0995830912d38af8b7b9f905481843b60a2f9f7c5ef3e699", + "bytes": 7370 + }, + "stream265-code/opt_prime/accel/platforms/rngd/__init__.py": { + "sha256": "82a797e83449ff1624e95bc1e14fc3080a8431515909b56ca107bf1b56e76ca5", + "bytes": 61 + }, + "stream265-code/opt_prime/accel/platforms/rngd/batching.py": { + "sha256": "fe13939f4bf6df9176cf32ca5b639529a5fec7fa59ea137e8ee25db98f98beb8", + "bytes": 16675 + }, + "stream265-code/opt_prime/accel/platforms/rngd/device.py": { + "sha256": "b3ae2dddd357c73d67abfddbfb6808e21e8f4cc2bee053aae06895b43229dc2e", + "bytes": 1864 + }, + "stream265-code/opt_prime/accel/platforms/rngd/engine.py": { + "sha256": "8ba4aa0eea25c3c53f88a2dca80d6ac503d8a810cc78ff429b5da2d060f0e565", + "bytes": 5161 + }, + "stream265-code/opt_prime/accel/platforms/rngd/experts.py": { + "sha256": "39124a145695186c565f182b32e53e7d686429dce93b5f8917cf1f2b1b443a3e", + "bytes": 3288 + }, + "stream265-code/opt_prime/accel/platforms/rngd/experts_opt.py": { + "sha256": "7910762835ac904db36f4084870a4394cc772b01e86a17c4d547dcdb238a9aa3", + "bytes": 11991 + }, + "stream265-code/opt_prime/accel/platforms/rngd/pd.py": { + "sha256": "74a06401eb48c9bd850271c6b16074916cbe92893a83b3a8638397d68a9f90aa", + "bytes": 8470 + }, + "stream265-code/opt_prime/accel/platforms/rngd/pd_packing.py": { + "sha256": "c015a1d742decee36f212cbc7841e5da6eddd9f0f0a0732f33da0fd0ff170491", + "bytes": 1989 + }, + "stream265-code/opt_prime/accel/serve/__init__.py": { + "sha256": "4347f5d1f61d3614e069f35cfd29b0aa13673129fa057dc9a5290e07c6c825c9", + "bytes": 42 + }, + "stream265-code/opt_prime/accel/serve/batched.py": { + "sha256": "81fa8c78648064c38a8146f52f5207553cf91fb5de8d9048f2de3340f4a8b316", + "bytes": 12131 + }, + "stream265-code/opt_prime/accel/serve/kv_events.py": { + "sha256": "3c6cdf45301f6a87eda86762a3f08d181b693adb8b9101f216bc579eb37b4844", + "bytes": 3033 + }, + "stream265-code/opt_prime/accel/serve/llmd.py": { + "sha256": "e3449979b88d03edb6d5bf0d402a708e5d98054fdf203e22b480f3d7e6fd759a", + "bytes": 32412 + }, + "stream265-code/opt_prime/accel/serve/metrics.py": { + "sha256": "deddb63fd17c2575fc40b7b644ca456cb68115dacaead7e5cc6e8364e4e5b7f9", + "bytes": 4826 + }, + "stream265-code/opt_prime/accel/spec/__init__.py": { + "sha256": "c44cdaee8215145fdf7d009fabaff2da83ae6c161a7380e687c576d4f255fcaa", + "bytes": 556 + }, + "stream265-code/opt_prime/accel/spec/adapters.py": { + "sha256": "71d065ee4390c48e3c830251e07662a01ec6c5e36bfcd281037331cb2338f06e", + "bytes": 7366 + }, + "stream265-code/opt_prime/accel/spec/blocks.py": { + "sha256": "44c6ba9b1a5d3696a7b3a735b51415b1056b3e9f7d9ab144eed7a5d850ae5dfe", + "bytes": 6541 + }, + "stream265-code/opt_prime/accel/spec/config.py": { + "sha256": "8c7b554d892bba21dc44b32e67749375f8f7b2adf62a86ea58b662a4526feac5", + "bytes": 1796 + }, + "stream265-code/opt_prime/accel/spec/discover.py": { + "sha256": "e2e83492dda003f8c889b4319e6e56de04f542fc447611c9203d759c607cd6ed", + "bytes": 1967 + }, + "stream265-code/opt_prime/accel/spec/experts.py": { + "sha256": "9fd7d7511bb08c3dd597a664b77129c69553b25386cd1a2de940ae1e5207ea67", + "bytes": 1983 + }, + "stream265-code/opt_prime/accel/spec/hf.py": { + "sha256": "e8c31fec5521a324d27b7287bc8591b1043229a7c563d6bf16d5cbdd08ed8cd0", + "bytes": 11521 + }, + "stream265-code/opt_prime/accel/testing.py": { + "sha256": "a3958066d9e54e227f15607da3b76222b668565fafb1de3e49025f34218b3763", + "bytes": 4057 + }, + "stream265-code/opt_prime/accel/train/__init__.py": { + "sha256": "a4e7ed011f2ef61353bd9c4ee57f40981968071cc61f9a79b4bd56863910ddd3", + "bytes": 1047 + }, + "stream265-code/opt_prime/accel/train/backends.py": { + "sha256": "fa01536ef84caab9148d32fde97126cfa603bfd560916154b2b845de4977a530", + "bytes": 14738 + }, + "stream265-code/opt_prime/comm.py": { + "sha256": "2cb826d82e498281663f27cd9d1d562f90d5a125f030d23effc924d45c4860d0", + "bytes": 15100 + }, + "stream265-code/opt_prime/hf_ckpt.py": { + "sha256": "090dad24ab0330e8b7d9f8ce2c5d625658dfc6a9a3a794498a2fc6f92cfc8621", + "bytes": 10146 + }, + "stream265-code/opt_prime/inference.py": { + "sha256": "f8259ff9e44d36fbd3e0b887cf1893dc61f5d16b294f6c08dee536e31059f56e", + "bytes": 52917 + }, + "stream265-code/opt_prime/kv_cache.py": { + "sha256": "38dfa13f5e8e1391c7d466e18801332a0c6a85d1624ae92bd5cae49cc596bb3e", + "bytes": 15267 + }, + "stream265-code/opt_prime/lora.py": { + "sha256": "71b4122c9aa88d2d8023310db61f2a77cffa150c5e98bd51cf537fc7bb7ce18c", + "bytes": 11440 + }, + "stream265-code/opt_prime/mps_manager.py": { + "sha256": "0482e0bf13d7e048806944bf35e25f5c25904fbcd3bd4ca9f1e153c0e401b21d", + "bytes": 38444 + }, + "stream265-code/opt_prime/opti_pri.py": { + "sha256": "6d97eecd85664261978a0bd2b980157a0f9e5cb558cf7ffb42bc33e21f5ab6db", + "bytes": 56155 + }, + "stream265-code/opt_prime/schedule.py": { + "sha256": "5cffc60057b86a728068fa79ca3519ffa8c021a0ef607e2076e80cc5ff9022e4", + "bytes": 56779 + }, + "stream265-code/opt_prime/schedule_infer.py": { + "sha256": "d66b245add5e57b21badb570256bd5fdc5358c1bde1cabc7bc837559dc06dd21", + "bytes": 18989 + } + }, + "missing": [] +} diff --git a/experiments/evidence270/atom270-dense-b4-partial-cpu-audit.json b/experiments/evidence270/atom270-dense-b4-partial-cpu-audit.json new file mode 100644 index 00000000..79317aa6 --- /dev/null +++ b/experiments/evidence270/atom270-dense-b4-partial-cpu-audit.json @@ -0,0 +1,11 @@ +{ + "stage": "passed", + "passed": 53, + "elapsed_s": 6.19, + "new_cases": 2, + "commit": "eaf7daa219a6cea68db4ec5fc08850edc848b271", + "batching_sha256": "d98a51d75a36cd8c379a6220adb92528b3607cf80145b85a032270494a429580", + "test_sha256": "d71d813c7fc95f0f44600635f1d804a8e86e25197a1359f23747e3966a28dd78", + "scope": "CPU fake SDK: real AtomBatcher, B4/k5 dense cache, three survivors, fourth lane partially imported, cancellation/late completion or producer error, same lane reuse. Survivor outputs equal no-fault control; replacement destination unchanged by late old result. Includes existing batching/hybrid/stream tests. Not hardware, quality, or full repository suite.", + "initial_failure": "Both new cases failed on test access to missing admitted attribute; changed test to use existing runtime convention getattr(...,False). No runtime change." +} \ No newline at end of file diff --git a/experiments/evidence270/atom270-fault-b4-gemma-audit.json b/experiments/evidence270/atom270-fault-b4-gemma-audit.json new file mode 100644 index 00000000..f5bc3fd8 --- /dev/null +++ b/experiments/evidence270/atom270-fault-b4-gemma-audit.json @@ -0,0 +1,75 @@ +{ + "model": "unsloth/gemma-3-1b-it", + "stage": "audited", + "source_sha256": "83b17d8acf98989a1e1df0dbb263cdf9588b3aa56b1942c29024458985227821", + "survivors": 3, + "survivor_output_tokens": 512, + "replacement_output_tokens": 16, + "cases": { + "control": { + "case": "control", + "histogram": [ + 0, + 220, + 220, + 276, + 15 + ] + }, + "cancel-reuse": { + "case": "cancel-reuse", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 225, + 110, + 4 + ], + "progress_spans": 78, + "admitted": false + }, + "histogram": [ + 0, + 221, + 221, + 275, + 15 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "replacement_exact": true + }, + "producer-death": { + "case": "producer-death", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 215, + 103, + 3 + ], + "progress_spans": 78, + "admitted": false + }, + "histogram": [ + 0, + 212, + 212, + 299, + 0 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "victim_error": "RuntimeError('prefill subprocess response failed: prefill subprocess exited')" + } + }, + "producer_startup_s": 779.9282271610573, + "decoder_load_s": 4.666098878020421, + "limits": "One direct-batcher hardware control/cancel-reuse/producer-death sequence, same compiled B4 artifact. Not HTTP disconnect coverage, arbitrary workload quality, or compiler-cache reuse proof." +} diff --git a/experiments/evidence270/atom270-fault-b4-koni-audit.json b/experiments/evidence270/atom270-fault-b4-koni-audit.json new file mode 100644 index 00000000..cf738090 --- /dev/null +++ b/experiments/evidence270/atom270-fault-b4-koni-audit.json @@ -0,0 +1,75 @@ +{ + "model": "KISTI-KONI/KONI-Llama3.1-8B-Instruct-20241024", + "producer_startup_s": 409.7310900699813, + "decoder_load_s": 12.88959055999294, + "stage": "audited", + "source_sha256": "5374d6383f09c444c9044a52b9a746f52baf5bb297d8af44817098c9af5a5941", + "survivors": 3, + "survivor_output_tokens": 512, + "replacement_output_tokens": 16, + "cases": { + "control": { + "case": "control", + "histogram": [ + 0, + 174, + 174, + 322, + 15 + ] + }, + "cancel-reuse": { + "case": "cancel-reuse", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 168, + 84, + 4 + ], + "progress_spans": 96, + "admitted": false + }, + "histogram": [ + 0, + 164, + 164, + 332, + 15 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "replacement_exact": true + }, + "producer-death": { + "case": "producer-death", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 176, + 89, + 5 + ], + "progress_spans": 96, + "admitted": false + }, + "histogram": [ + 0, + 171, + 171, + 340, + 0 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "victim_error": "RuntimeError('prefill subprocess response failed: prefill subprocess exited')" + } + }, + "limits": "One direct-batcher same-lane TP2 fault sequence; not HTTP disconnect, semantic quality, performance/cache-reuse acceptance or hardware partial-victim-import coverage." +} \ No newline at end of file diff --git a/experiments/evidence270/atom270-fault-b4-qwen06b-audit.json b/experiments/evidence270/atom270-fault-b4-qwen06b-audit.json new file mode 100644 index 00000000..1d0401ae --- /dev/null +++ b/experiments/evidence270/atom270-fault-b4-qwen06b-audit.json @@ -0,0 +1,75 @@ +{ + "model": "Qwen/Qwen3-0.6B", + "producer_startup_s": 316.16443911299575, + "decoder_load_s": 1.9797176460269839, + "stage": "audited", + "source_sha256": "e61b9829ed90c84f08186b84ab42c5a6472ed8e5a55cb37e20af92b653f0a0e2", + "survivors": 3, + "survivor_output_tokens": 512, + "replacement_output_tokens": 16, + "cases": { + "control": { + "case": "control", + "histogram": [ + 0, + 168, + 168, + 328, + 15 + ] + }, + "cancel-reuse": { + "case": "cancel-reuse", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 167, + 77, + 9 + ], + "progress_spans": 84, + "admitted": false + }, + "histogram": [ + 0, + 158, + 158, + 338, + 15 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "replacement_exact": true + }, + "producer-death": { + "case": "producer-death", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 174, + 89, + 7 + ], + "progress_spans": 84, + "admitted": false + }, + "histogram": [ + 0, + 167, + 167, + 344, + 0 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "victim_error": "RuntimeError('prefill subprocess response failed: prefill subprocess exited')" + } + }, + "limits": "One direct-batcher fault sequence. Not HTTP disconnect, semantic quality, or performance/cache-reuse acceptance." +} \ No newline at end of file diff --git a/experiments/evidence270/atom270-fault-b4-qwen4b-audit.json b/experiments/evidence270/atom270-fault-b4-qwen4b-audit.json new file mode 100644 index 00000000..0f0ca9d8 --- /dev/null +++ b/experiments/evidence270/atom270-fault-b4-qwen4b-audit.json @@ -0,0 +1,76 @@ +{ + "model": "Qwen/Qwen3-4B", + "producer_startup_s": 822.2584600540577, + "decoder_load_s": 7.551017142017372, + "stage": "audited", + "source_sha256": "f73a020b719e01212d239bcf0ce7ea6b0ce7d69c0c08f6d7abb7604e77a92e96", + "survivors": 3, + "survivor_output_tokens": 512, + "replacement_output_tokens": 16, + "cases": { + "control": { + "case": "control", + "histogram": [ + 0, + 120, + 120, + 376, + 15 + ] + }, + "cancel-reuse": { + "case": "cancel-reuse", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 114, + 59, + 4 + ], + "progress_spans": 108, + "admitted": false + }, + "histogram": [ + 0, + 110, + 110, + 386, + 15 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "replacement_exact": true + }, + "producer-death": { + "case": "producer-death", + "trigger": { + "host_pending": true, + "victim_imported_spans": 0, + "slot": 3, + "active_tokens": [ + 112, + 57, + 4 + ], + "progress_spans": 108, + "admitted": false + }, + "histogram": [ + 0, + 108, + 108, + 403, + 0 + ], + "survivors_exact": true, + "idle": true, + "poisoned": false, + "victim_error": "RuntimeError('prefill subprocess response failed: prefill subprocess exited')" + } + }, + "control_survivor_lanes_equal": true, + "limits": "One direct-batcher same-lane fault sequence. Does not resolve independently reproduced native B4 lane-dependent numerical differences. Not HTTP disconnect, quality, performance/cache-reuse acceptance, or partial victim-import coverage." +} \ No newline at end of file diff --git a/experiments/evidence270/atom270-firstkv-control-audit.json b/experiments/evidence270/atom270-firstkv-control-audit.json new file mode 100644 index 00000000..d85c9408 --- /dev/null +++ b/experiments/evidence270/atom270-firstkv-control-audit.json @@ -0,0 +1,27 @@ +{ + "complete": true, + "raw_sha256": "f1e4e89bbdb7b44fccc64c4590bff328fad6dcdcf63bb59030142c3366287e19", + "rows": [ + { + "input_tokens": 2048, + "snapshot_count": 8, + "all_four_blocks_bit_exact": true, + "changed_block": 3, + "control_changed_elements_per_kv": 1024, + "control_first_changed_position": 0, + "restored_exact": true, + "prior_hidden_and_logit_probes_exact": true + }, + { + "input_tokens": 512, + "snapshot_count": 8, + "all_four_blocks_bit_exact": true, + "changed_block": 3, + "control_changed_elements_per_kv": 1024, + "control_first_changed_position": 0, + "restored_exact": true, + "prior_hidden_and_logit_probes_exact": true + } + ], + "scope": "First-layer stored KV at sampled timepoints only; does not identify attention or MLP operator." +} diff --git a/experiments/evidence270/atom270-four-model-b4-fault-audit.json b/experiments/evidence270/atom270-four-model-b4-fault-audit.json new file mode 100644 index 00000000..2740934c --- /dev/null +++ b/experiments/evidence270/atom270-four-model-b4-fault-audit.json @@ -0,0 +1,139 @@ +{ + "stage": "audited", + "models": 4, + "fault_cases": 8, + "survivor_token_comparisons": 12288, + "replacement_token_comparisons": 64, + "all_task_pods_cleanup_exit_zero": true, + "rows": [ + { + "model": "unsloth/gemma-3-1b-it", + "raw_sha256": "83b17d8acf98989a1e1df0dbb263cdf9588b3aa56b1942c29024458985227821", + "fault_cases": 2, + "survivor_token_comparisons": 3072, + "replacement_token_comparisons": 16, + "control_histogram": [ + 0, + 220, + 220, + 276, + 15 + ], + "cancel_histogram": [ + 0, + 221, + 221, + 275, + 15 + ], + "death_histogram": [ + 0, + 212, + 212, + 299, + 0 + ], + "victim_imported_spans": [ + 0, + 0 + ] + }, + { + "model": "Qwen/Qwen3-0.6B", + "raw_sha256": "e61b9829ed90c84f08186b84ab42c5a6472ed8e5a55cb37e20af92b653f0a0e2", + "fault_cases": 2, + "survivor_token_comparisons": 3072, + "replacement_token_comparisons": 16, + "control_histogram": [ + 0, + 168, + 168, + 328, + 15 + ], + "cancel_histogram": [ + 0, + 158, + 158, + 338, + 15 + ], + "death_histogram": [ + 0, + 167, + 167, + 344, + 0 + ], + "victim_imported_spans": [ + 0, + 0 + ] + }, + { + "model": "Qwen/Qwen3-4B", + "raw_sha256": "f73a020b719e01212d239bcf0ce7ea6b0ce7d69c0c08f6d7abb7604e77a92e96", + "fault_cases": 2, + "survivor_token_comparisons": 3072, + "replacement_token_comparisons": 16, + "control_histogram": [ + 0, + 120, + 120, + 376, + 15 + ], + "cancel_histogram": [ + 0, + 110, + 110, + 386, + 15 + ], + "death_histogram": [ + 0, + 108, + 108, + 403, + 0 + ], + "victim_imported_spans": [ + 0, + 0 + ] + }, + { + "model": "KISTI-KONI/KONI-Llama3.1-8B-Instruct-20241024", + "raw_sha256": "5374d6383f09c444c9044a52b9a746f52baf5bb297d8af44817098c9af5a5941", + "fault_cases": 2, + "survivor_token_comparisons": 3072, + "replacement_token_comparisons": 16, + "control_histogram": [ + 0, + 174, + 174, + 322, + 15 + ], + "cancel_histogram": [ + 0, + 164, + 164, + 332, + 15 + ], + "death_histogram": [ + 0, + 171, + 171, + 340, + 0 + ], + "victim_imported_spans": [ + 0, + 0 + ] + } + ], + "limits": "One control/cancel-reuse/producer-death sequence per model, same-lane full-output comparison. All victim triggers precede device import. Not native-PD parity, semantic quality, HTTP disconnect, hardware partial-import safety or speedup evidence." +} \ No newline at end of file diff --git a/experiments/evidence270/atom270-four-model-native-batch-comparison.json b/experiments/evidence270/atom270-four-model-native-batch-comparison.json new file mode 100644 index 00000000..9e9f3015 --- /dev/null +++ b/experiments/evidence270/atom270-four-model-native-batch-comparison.json @@ -0,0 +1,348 @@ +{ + "scope": "Four-model native B1/B4 cross-check. Corrected causal Gemma only. Raw timings are not equivalent-output speedup when full tokens differ; hardware bindings/card differences and one-repeat limitations remain in per-model reports.", + "models": [ + { + "model": "qwen06b", + "tp": 1, + "b4_within_route_parity": true, + "comparison": { + "128:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "128:128": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "512:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "512:128": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "2048:32": { + "full": false, + "first_difference": 1, + "token_pair": [ + 374, + 594 + ] + }, + "2048:128": { + "full": false, + "first_difference": 1, + "token_pair": [ + 374, + 594 + ] + }, + "8192:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "8192:128": { + "full": true, + "first_difference": null, + "token_pair": null + } + }, + "source_sha256": { + "1": "1375ae8c9eb389b2a14bd70420539d3c48276261bae0d61c55e8e46973ac7b6f", + "4": "c9f2a5e4f193b0db8a4d498d332443ceea21b02bf982c6fdb1fe1256d09f96e9" + }, + "metrics_8192_128": { + "1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 40.69590429693926, + "output_tok_s": 50.324474548020554, + "median_first_sse_s": 20.20046385045862, + "median_tpot_est_s": 0.01152950295661466, + "median_e2e_s": 21.6256192799774, + "all_text_equal": true + }, + "4": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 32.18251903599594, + "output_tok_s": 63.63703219469319, + "median_first_sse_s": 14.797579117992427, + "median_tpot_est_s": 0.05466472298799699, + "median_e2e_s": 21.75026416644687, + "all_text_equal": true + } + } + }, + { + "model": "qwen4b", + "tp": 1, + "b4_within_route_parity": false, + "comparison": { + "128:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "128:128": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "512:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "512:128": { + "full": false, + "first_difference": 64, + "token_pair": [ + 30339, + 911 + ] + }, + "2048:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "2048:128": { + "full": false, + "first_difference": 59, + "token_pair": [ + 8365, + 7196 + ] + }, + "8192:32": { + "full": false, + "first_difference": 12, + "token_pair": [ + 4588, + 374 + ] + }, + "8192:128": { + "full": false, + "first_difference": 12, + "token_pair": [ + 4588, + 374 + ] + } + }, + "source_sha256": { + "1": "eb11ac21e4a6f74f2a258aaa319cabfcc7eea0b3b2e30f6dd4f1208d5f5e9c5a", + "4": "11d07b625862acc161aa95c34cd2d1087e9bd302477c01ff7e7f66677f9234f7" + }, + "metrics_8192_128": { + "1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 166.73132611101028, + "output_tok_s": 12.283234637241682, + "median_first_sse_s": 83.1327906565275, + "median_tpot_est_s": 0.042929202673395755, + "median_e2e_s": 88.58196617756039, + "all_text_equal": true + }, + "4": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 112.26723379304167, + "output_tok_s": 18.242188132784786, + "median_first_sse_s": 54.54269025201211, + "median_tpot_est_s": 0.1815217392240453, + "median_e2e_s": 77.60018511646194, + "all_text_equal": true + } + } + }, + { + "model": "koni", + "tp": 2, + "b4_within_route_parity": true, + "comparison": { + "128:32": { + "full": false, + "first_difference": 11, + "token_pair": [ + 4495, + 11190 + ] + }, + "128:128": { + "full": false, + "first_difference": 11, + "token_pair": [ + 4495, + 11190 + ] + }, + "512:32": { + "full": false, + "first_difference": 8, + "token_pair": [ + 70869, + 9290 + ] + }, + "512:128": { + "full": false, + "first_difference": 8, + "token_pair": [ + 70869, + 9290 + ] + }, + "2048:32": { + "full": false, + "first_difference": 27, + "token_pair": [ + 9257, + 6366 + ] + }, + "2048:128": { + "full": false, + "first_difference": 27, + "token_pair": [ + 9257, + 6366 + ] + }, + "8192:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "8192:128": { + "full": false, + "first_difference": 37, + "token_pair": [ + 279, + 44564 + ] + } + }, + "source_sha256": { + "1": "111c24490d36d835efa369d11ba05ce9a63e7c53da89ca4d7d8a72119c688f39", + "4": "b865d242b87b6a948b81fa3bf97927b302db923c3834f013fbacb0b9de6c509c" + }, + "metrics_8192_128": { + "1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 136.36672446795274, + "output_tok_s": 15.018326560166782, + "median_first_sse_s": 67.78715079504764, + "median_tpot_est_s": 0.036702179535581754, + "median_e2e_s": 72.43892477202462, + "all_text_equal": true + }, + "4": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 85.87466673506424, + "output_tok_s": 23.848709728543998, + "median_first_sse_s": 41.86685951001709, + "median_tpot_est_s": 0.1383990001654817, + "median_e2e_s": 59.45633385749534, + "all_text_equal": true + } + } + }, + { + "model": "gemma", + "tp": 1, + "b4_within_route_parity": true, + "comparison": { + "128:32": { + "full": false, + "first_difference": 5, + "token_pair": [ + 107, + 130614 + ] + }, + "128:128": { + "full": false, + "first_difference": 5, + "token_pair": [ + 107, + 130614 + ] + }, + "512:32": { + "full": false, + "first_difference": 8, + "token_pair": [ + 42547, + 10525 + ] + }, + "512:128": { + "full": false, + "first_difference": 8, + "token_pair": [ + 42547, + 10525 + ] + }, + "2048:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "2048:128": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "8192:32": { + "full": true, + "first_difference": null, + "token_pair": null + }, + "8192:128": { + "full": true, + "first_difference": null, + "token_pair": null + } + }, + "source_sha256": { + "1": "76b81b2728303015358810b1a76391f5bad2f63291ab4b8aad39c44ced7d19b9", + "4": "d9b5752d5fefe4a26d95118699c7ee50407ab20fc7ea9ee1909fb0a1eec41e78" + }, + "metrics_8192_128": { + "1": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 35.574109970009886, + "output_tok_s": 57.56995752603592, + "median_first_sse_s": 17.3802980305627, + "median_tpot_est_s": 0.011212882113745656, + "median_e2e_s": 18.801305017026607, + "all_text_equal": true + }, + "4": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 21.52105758502148, + "output_tok_s": 95.16260954691163, + "median_first_sse_s": 10.135787407518364, + "median_tpot_est_s": 0.036070315672843675, + "median_e2e_s": 14.72101910848869, + "all_text_equal": true + } + } + } + ] +} diff --git a/experiments/evidence270/atom270-gemma-causal-export-provenance.json b/experiments/evidence270/atom270-gemma-causal-export-provenance.json new file mode 100644 index 00000000..dfad3aeb --- /dev/null +++ b/experiments/evidence270/atom270-gemma-causal-export-provenance.json @@ -0,0 +1,11 @@ +{ + "model": "unsloth/gemma-3-1b-it", + "batch": 1, + "kv_blocks": 5, + "context": 9216, + "correction": "text prefill causal; image prefill retains bidirectionality", + "patch_sha256": { + "gemma_causal_compat.py": "c4a48fa1cce28865405a3211559e0ff2556b893072b789e9543b96bd34907930", + "gemma_global_causal_compat.py": "c4c51d76dec643a8b7643e3f127c0e067a7ebb735b38bf67cb73212e20db17cb" + } +} diff --git a/experiments/evidence270/atom270-gemma-causality-b4-k5.json b/experiments/evidence270/atom270-gemma-causality-b4-k5.json new file mode 100644 index 00000000..1deaf4d8 --- /dev/null +++ b/experiments/evidence270/atom270-gemma-causality-b4-k5.json @@ -0,0 +1,1510 @@ +[ + { + "artifact": "unsloth--gemma-3-1b-it-tp1-s9216-b4-k5", + "lane": 0, + "kv_blocks": 5, + "input_tokens": 128, + "unchanged_prefix": 64, + "slots": [ + { + "slot": 0, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 1, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 2, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 3, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 4, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 5, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 6, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 7, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 8, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 9, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 10, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 11, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 12, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 13, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 14, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 15, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 16, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 17, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 18, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 19, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 20, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 21, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 22, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 23, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 24, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 25, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 26, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 27, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 28, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 29, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 30, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 31, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 32, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 33, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 34, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 35, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 36, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 37, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 38, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 39, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 40, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 41, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 42, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 43, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 44, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 45, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 46, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 47, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 48, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 49, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 50, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 51, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + } + ], + "prefix_exact": true, + "prefix_control_changed_slots": [ + 0, + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26, + 27, + 28, + 29, + 30, + 31, + 32, + 33, + 34, + 35, + 36, + 37, + 38, + 39, + 40, + 41, + 42, + 43, + 44, + 45, + 46, + 47, + 48, + 49, + 50, + 51 + ], + "prefix_control_passed": true + }, + { + "artifact": "unsloth--gemma-3-1b-it-tp1-s9216-b4-k5", + "lane": 1, + "kv_blocks": 5, + "input_tokens": 128, + "unchanged_prefix": 64, + "slots": [ + { + "slot": 0, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 1, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 2, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 3, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 4, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 5, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 6, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 7, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 8, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 9, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 10, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 11, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 12, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 13, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 14, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 15, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 16, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 17, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 18, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 19, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 20, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 21, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 22, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 23, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 24, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 25, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 26, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 27, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 28, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 29, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 30, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 31, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 32, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 33, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 34, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 35, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 36, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 37, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 38, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 39, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 40, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 41, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 42, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 43, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 44, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 45, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 46, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 47, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 48, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 49, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 50, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 51, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + } + ], + "prefix_exact": true, + "prefix_control_changed_slots": [ + 0, + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26, + 27, + 28, + 29, + 30, + 31, + 32, + 33, + 34, + 35, + 36, + 37, + 38, + 39, + 40, + 41, + 42, + 43, + 44, + 45, + 46, + 47, + 48, + 49, + 50, + 51 + ], + "prefix_control_passed": true + }, + { + "artifact": "unsloth--gemma-3-1b-it-tp1-s9216-b4-k5", + "lane": 2, + "kv_blocks": 5, + "input_tokens": 128, + "unchanged_prefix": 64, + "slots": [ + { + "slot": 0, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 1, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 2, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 3, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 4, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 5, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 6, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 7, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 8, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 9, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 10, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 11, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 12, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 13, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 14, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 15, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 16, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 17, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 18, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 19, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 20, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 21, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 22, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 23, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 24, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 25, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 26, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 27, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 28, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 29, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 30, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 31, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 32, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 33, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 34, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 35, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 36, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 37, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 38, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 39, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 40, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 41, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 42, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 43, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 44, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 45, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 46, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 47, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 48, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 49, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 50, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 51, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + } + ], + "prefix_exact": true, + "prefix_control_changed_slots": [ + 0, + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26, + 27, + 28, + 29, + 30, + 31, + 32, + 33, + 34, + 35, + 36, + 37, + 38, + 39, + 40, + 41, + 42, + 43, + 44, + 45, + 46, + 47, + 48, + 49, + 50, + 51 + ], + "prefix_control_passed": true + }, + { + "artifact": "unsloth--gemma-3-1b-it-tp1-s9216-b4-k5", + "lane": 3, + "kv_blocks": 5, + "input_tokens": 128, + "unchanged_prefix": 64, + "slots": [ + { + "slot": 0, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 1, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 2, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 3, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 4, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 5, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 6, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 7, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 8, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 9, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 10, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 11, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 12, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 13, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 14, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 15, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 16, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 17, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 18, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 19, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 20, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 21, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 22, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 23, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 24, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 25, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 26, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 27, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 28, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 29, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 30, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 31, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 32, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 33, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 34, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 35, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 36, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 37, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 38, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 39, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 40, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 41, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 42, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 43, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 44, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 45, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 46, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 47, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 48, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 49, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 50, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 51, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + } + ], + "prefix_exact": true, + "prefix_control_changed_slots": [ + 0, + 1, + 2, + 3, + 4, + 5, + 6, + 7, + 8, + 9, + 10, + 11, + 12, + 13, + 14, + 15, + 16, + 17, + 18, + 19, + 20, + 21, + 22, + 23, + 24, + 25, + 26, + 27, + 28, + 29, + 30, + 31, + 32, + 33, + 34, + 35, + 36, + 37, + 38, + 39, + 40, + 41, + 42, + 43, + 44, + 45, + 46, + 47, + 48, + 49, + 50, + 51 + ], + "prefix_control_passed": true + } +] diff --git a/experiments/evidence270/atom270-gemma-causality-fixed-k5.json b/experiments/evidence270/atom270-gemma-causality-fixed-k5.json new file mode 100644 index 00000000..45d0361d --- /dev/null +++ b/experiments/evidence270/atom270-gemma-causality-fixed-k5.json @@ -0,0 +1,323 @@ +[ + { + "artifact": "unsloth--gemma-3-1b-it-tp1-s9216-b1-k5", + "kv_blocks": 5, + "input_tokens": 128, + "unchanged_prefix": 64, + "slots": [ + { + "slot": 0, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 1, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 2, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 3, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 4, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 5, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 6, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 7, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 8, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 9, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 10, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 11, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 12, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 13, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 14, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 15, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 16, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 17, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 18, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 19, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 20, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 21, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 22, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 23, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 24, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 25, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 26, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 27, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 28, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 29, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 30, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 31, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 32, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 33, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 34, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 35, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 36, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 37, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 38, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 39, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 40, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 41, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 42, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 43, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 44, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 45, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 46, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 47, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 48, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 49, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 50, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 51, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + } + ], + "prefix_exact": true + } +] diff --git a/experiments/evidence270/atom270-gemma-causality-k3.json b/experiments/evidence270/atom270-gemma-causality-k3.json new file mode 100644 index 00000000..c569bc77 --- /dev/null +++ b/experiments/evidence270/atom270-gemma-causality-k3.json @@ -0,0 +1,323 @@ +[ + { + "artifact": "unsloth--gemma-3-1b-it-tp1-s9216-b1-k3", + "kv_blocks": 3, + "input_tokens": 128, + "unchanged_prefix": 64, + "slots": [ + { + "slot": 0, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 1, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 2, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 3, + "layer": 1, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 4, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 5, + "layer": 2, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 6, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 7, + "layer": 3, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 8, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 9, + "layer": 4, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 10, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 11, + "layer": 5, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 12, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 13, + "layer": 6, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 14, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 15, + "layer": 7, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 16, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 17, + "layer": 8, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 18, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 19, + "layer": 9, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 20, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 21, + "layer": 10, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 22, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 23, + "layer": 11, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 24, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 25, + "layer": 12, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 26, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 27, + "layer": 13, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 28, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 29, + "layer": 14, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 30, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 31, + "layer": 15, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 32, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 33, + "layer": 16, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 34, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 35, + "layer": 17, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 36, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 37, + "layer": 18, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 38, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 39, + "layer": 19, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 40, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 41, + "layer": 20, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 42, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 43, + "layer": 21, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 44, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 45, + "layer": 22, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 46, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 47, + "layer": 23, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 48, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 49, + "layer": 24, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 50, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 51, + "layer": 25, + "equal": true, + "changed_fraction": 0.0 + } + ], + "prefix_exact": true + } +] diff --git a/experiments/evidence270/atom270-gemma-causality-k5.json b/experiments/evidence270/atom270-gemma-causality-k5.json new file mode 100644 index 00000000..5bc99b8f --- /dev/null +++ b/experiments/evidence270/atom270-gemma-causality-k5.json @@ -0,0 +1,323 @@ +[ + { + "artifact": "unsloth--gemma-3-1b-it-tp1-s9216-b1-k5", + "kv_blocks": 5, + "input_tokens": 128, + "unchanged_prefix": 64, + "slots": [ + { + "slot": 0, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 1, + "layer": 0, + "equal": true, + "changed_fraction": 0.0 + }, + { + "slot": 2, + "layer": 1, + "equal": false, + "changed_fraction": 0.99652099609375 + }, + { + "slot": 3, + "layer": 1, + "equal": false, + "changed_fraction": 0.99786376953125 + }, + { + "slot": 4, + "layer": 2, + "equal": false, + "changed_fraction": 0.99615478515625 + }, + { + "slot": 5, + "layer": 2, + "equal": false, + "changed_fraction": 0.997314453125 + }, + { + "slot": 6, + "layer": 3, + "equal": false, + "changed_fraction": 0.9959716796875 + }, + { + "slot": 7, + "layer": 3, + "equal": false, + "changed_fraction": 0.99761962890625 + }, + { + "slot": 8, + "layer": 4, + "equal": false, + "changed_fraction": 0.9962158203125 + }, + { + "slot": 9, + "layer": 4, + "equal": false, + "changed_fraction": 0.9984130859375 + }, + { + "slot": 10, + "layer": 5, + "equal": false, + "changed_fraction": 0.99749755859375 + }, + { + "slot": 11, + "layer": 5, + "equal": false, + "changed_fraction": 0.997802734375 + }, + { + "slot": 12, + "layer": 6, + "equal": false, + "changed_fraction": 0.99755859375 + }, + { + "slot": 13, + "layer": 6, + "equal": false, + "changed_fraction": 0.99847412109375 + }, + { + "slot": 14, + "layer": 7, + "equal": false, + "changed_fraction": 0.9970703125 + }, + { + "slot": 15, + "layer": 7, + "equal": false, + "changed_fraction": 0.99822998046875 + }, + { + "slot": 16, + "layer": 8, + "equal": false, + "changed_fraction": 0.99810791015625 + }, + { + "slot": 17, + "layer": 8, + "equal": false, + "changed_fraction": 0.9989013671875 + }, + { + "slot": 18, + "layer": 9, + "equal": false, + "changed_fraction": 0.99798583984375 + }, + { + "slot": 19, + "layer": 9, + "equal": false, + "changed_fraction": 0.998779296875 + }, + { + "slot": 20, + "layer": 10, + "equal": false, + "changed_fraction": 0.99713134765625 + }, + { + "slot": 21, + "layer": 10, + "equal": false, + "changed_fraction": 0.9990234375 + }, + { + "slot": 22, + "layer": 11, + "equal": false, + "changed_fraction": 0.99847412109375 + }, + { + "slot": 23, + "layer": 11, + "equal": false, + "changed_fraction": 0.9991455078125 + }, + { + "slot": 24, + "layer": 12, + "equal": false, + "changed_fraction": 0.998046875 + }, + { + "slot": 25, + "layer": 12, + "equal": false, + "changed_fraction": 0.99908447265625 + }, + { + "slot": 26, + "layer": 13, + "equal": false, + "changed_fraction": 0.99822998046875 + }, + { + "slot": 27, + "layer": 13, + "equal": false, + "changed_fraction": 0.99920654296875 + }, + { + "slot": 28, + "layer": 14, + "equal": false, + "changed_fraction": 0.99835205078125 + }, + { + "slot": 29, + "layer": 14, + "equal": false, + "changed_fraction": 0.99884033203125 + }, + { + "slot": 30, + "layer": 15, + "equal": false, + "changed_fraction": 0.99871826171875 + }, + { + "slot": 31, + "layer": 15, + "equal": false, + "changed_fraction": 0.99908447265625 + }, + { + "slot": 32, + "layer": 16, + "equal": false, + "changed_fraction": 0.99859619140625 + }, + { + "slot": 33, + "layer": 16, + "equal": false, + "changed_fraction": 0.99932861328125 + }, + { + "slot": 34, + "layer": 17, + "equal": false, + "changed_fraction": 0.99871826171875 + }, + { + "slot": 35, + "layer": 17, + "equal": false, + "changed_fraction": 0.9984130859375 + }, + { + "slot": 36, + "layer": 18, + "equal": false, + "changed_fraction": 0.998291015625 + }, + { + "slot": 37, + "layer": 18, + "equal": false, + "changed_fraction": 0.99932861328125 + }, + { + "slot": 38, + "layer": 19, + "equal": false, + "changed_fraction": 0.99853515625 + }, + { + "slot": 39, + "layer": 19, + "equal": false, + "changed_fraction": 0.99932861328125 + }, + { + "slot": 40, + "layer": 20, + "equal": false, + "changed_fraction": 0.998291015625 + }, + { + "slot": 41, + "layer": 20, + "equal": false, + "changed_fraction": 0.99920654296875 + }, + { + "slot": 42, + "layer": 21, + "equal": false, + "changed_fraction": 0.99835205078125 + }, + { + "slot": 43, + "layer": 21, + "equal": false, + "changed_fraction": 0.99945068359375 + }, + { + "slot": 44, + "layer": 22, + "equal": false, + "changed_fraction": 0.9984130859375 + }, + { + "slot": 45, + "layer": 22, + "equal": false, + "changed_fraction": 0.9993896484375 + }, + { + "slot": 46, + "layer": 23, + "equal": false, + "changed_fraction": 0.9984130859375 + }, + { + "slot": 47, + "layer": 23, + "equal": false, + "changed_fraction": 0.99932861328125 + }, + { + "slot": 48, + "layer": 24, + "equal": false, + "changed_fraction": 0.99871826171875 + }, + { + "slot": 49, + "layer": 24, + "equal": false, + "changed_fraction": 0.9993896484375 + }, + { + "slot": 50, + "layer": 25, + "equal": false, + "changed_fraction": 0.99859619140625 + }, + { + "slot": 51, + "layer": 25, + "equal": false, + "changed_fraction": 0.9993896484375 + } + ], + "prefix_exact": false + } +] diff --git a/experiments/evidence270/atom270-hidden-audit.json b/experiments/evidence270/atom270-hidden-audit.json new file mode 100644 index 00000000..f2fea77c --- /dev/null +++ b/experiments/evidence270/atom270-hidden-audit.json @@ -0,0 +1,173 @@ +{ + "complete": true, + "compile_seconds": 153.83788209198974, + "scope": "Added outputs can change compilation; original target tokens reproduce, original full logit records differ. Not an operator attribution or quality result.", + "raw_sha256": "fbeaf63365c11bd62d024aa01f62b889c7b3eea4401fde7ccc96826334f0bb6b", + "probes": [ + { + "input_tokens": 2048, + "index": 12, + "tokens": [ + 4588, + 4588, + 4588, + 4588 + ], + "first_differing_boundary": { + "boundary": 1, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + "original_logit_records_equal": false, + "last_boundary": { + "boundary": 36, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.25, + 0.25, + 0.25 + ] + } + }, + { + "input_tokens": 2048, + "index": 13, + "tokens": [ + 369, + 264, + 264, + 264 + ], + "first_differing_boundary": { + "boundary": 1, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0078125, + 0.0078125, + 0.0078125 + ] + }, + "original_logit_records_equal": false, + "last_boundary": { + "boundary": 36, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.25, + 0.25, + 0.25 + ] + } + }, + { + "input_tokens": 512, + "index": 49, + "tokens": [ + 807, + 807, + 807, + 807 + ], + "first_differing_boundary": { + "boundary": 1, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + "original_logit_records_equal": false, + "last_boundary": { + "boundary": 36, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.5, + 0.5, + 0.5 + ] + } + }, + { + "input_tokens": 512, + "index": 50, + "tokens": [ + 4829, + 1033, + 1033, + 1033 + ], + "first_differing_boundary": { + "boundary": 1, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.001953125, + 0.001953125, + 0.001953125 + ] + }, + "original_logit_records_equal": false, + "last_boundary": { + "boundary": 36, + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.25, + 0.25, + 0.25 + ] + } + } + ], + "cleanup_exit_code": 0 +} diff --git a/experiments/evidence270/atom270-hybrid-b4-stream-import-limit.json b/experiments/evidence270/atom270-hybrid-b4-stream-import-limit.json new file mode 100644 index 00000000..a39b41c4 --- /dev/null +++ b/experiments/evidence270/atom270-hybrid-b4-stream-import-limit.json @@ -0,0 +1,8 @@ +{ + "source": "opt_prime/accel/platforms/atom/batching.py", + "sha256": "d98a51d75a36cd8c379a6220adb92528b3607cf80145b85a032270494a429580", + "finding": "_import_progress returns for hybrid_cache and lanes>1 unless final=True. _pd_step imports progress with final=True only after pending producer future completes.", + "reason": "Fixed-batch decode gives inactive lanes dummy token/position0 and local tables for all lanes; comments explicitly retain local-KV ownership until joining decode.", + "scope": "Gemma B4 hybrid cache: early ATOM layer import is deliberately deferred. Producer layer emission/host transport may still pipeline. This is source evidence, not measured device overlap. Qwen dense cache takes a different branch.", + "next": "Measure stage timing separately; any early hybrid import requires proven local-cache ownership across inactive decode lanes, with corruption/cancel/reuse regressions before promotion." +} diff --git a/experiments/evidence270/atom270-imported-fault-audit.json b/experiments/evidence270/atom270-imported-fault-audit.json new file mode 100644 index 00000000..fe89f75e --- /dev/null +++ b/experiments/evidence270/atom270-imported-fault-audit.json @@ -0,0 +1,176 @@ +[ + { + "model": "Qwen/Qwen3-0.6B", + "raw_file": "atom270-imported-fault-b4-qwen06b.json", + "raw_sha256": "e92ca8ba00a8610d4e68b940f325dd9da2588cd9c44143f348088f085a5750de", + "runtime_sha256": { + "accel/platforms/atom/batching.py": "d98a51d75a36cd8c379a6220adb92528b3607cf80145b85a032270494a429580", + "accel/pd/pd_batching.py": "c827775c1d399bf724a9747c3424ee5ea69f93054493bfbb71d5f0cf08fff65b", + "accel/pd/prefill_server.py": "5af1169ff5c6bfac6ce265ea3edf7f6a031c207bda7ce7bcbc996d7133576b1d" + }, + "survivor_token_comparisons": 3072, + "replacement_token_comparisons": 16, + "cases": [ + { + "case": "cancel-reuse", + "trigger": { + "host_pending": false, + "victim_imported_spans": 1, + "slot": 3, + "active_tokens": [ + 206, + 109, + 11 + ], + "progress_spans": 85, + "admitted": false + }, + "histogram": [ + 0, + 195, + 195, + 301, + 15 + ] + }, + { + "case": "producer-death", + "trigger": { + "host_pending": false, + "victim_imported_spans": 1, + "slot": 3, + "active_tokens": [ + 192, + 94, + 8 + ], + "progress_spans": 85, + "admitted": false + }, + "histogram": [ + 0, + 184, + 184, + 327, + 0 + ] + } + ] + }, + { + "model": "Qwen/Qwen3-4B", + "raw_file": "atom270-imported-fault-b4-qwen4b.json", + "raw_sha256": "a45e0ec49fa2a1e50554c3f8c41cb9f446164895607344ea8dc8be2bff4689b0", + "runtime_sha256": { + "accel/platforms/atom/batching.py": "d98a51d75a36cd8c379a6220adb92528b3607cf80145b85a032270494a429580", + "accel/pd/pd_batching.py": "c827775c1d399bf724a9747c3424ee5ea69f93054493bfbb71d5f0cf08fff65b", + "accel/pd/prefill_server.py": "5af1169ff5c6bfac6ce265ea3edf7f6a031c207bda7ce7bcbc996d7133576b1d" + }, + "survivor_token_comparisons": 3072, + "replacement_token_comparisons": 16, + "cases": [ + { + "case": "cancel-reuse", + "trigger": { + "host_pending": true, + "victim_imported_spans": 1, + "slot": 3, + "active_tokens": [ + 115, + 57, + 6 + ], + "progress_spans": 109, + "admitted": false + }, + "histogram": [ + 0, + 109, + 109, + 387, + 15 + ] + }, + { + "case": "producer-death", + "trigger": { + "host_pending": true, + "victim_imported_spans": 1, + "slot": 3, + "active_tokens": [ + 108, + 55, + 5 + ], + "progress_spans": 109, + "admitted": false + }, + "histogram": [ + 0, + 103, + 103, + 408, + 0 + ] + } + ] + }, + { + "model": "KISTI-KONI/KONI-Llama3.1-8B-Instruct-20241024", + "raw_file": "atom270-imported-fault-b4-koni.json", + "raw_sha256": "37787b68b525c50d62fe69868e61091a33b2af180e0345da9dcac715a8d3112d", + "runtime_sha256": { + "accel/platforms/atom/batching.py": "d98a51d75a36cd8c379a6220adb92528b3607cf80145b85a032270494a429580", + "accel/pd/pd_batching.py": "c827775c1d399bf724a9747c3424ee5ea69f93054493bfbb71d5f0cf08fff65b", + "accel/pd/prefill_server.py": "5af1169ff5c6bfac6ce265ea3edf7f6a031c207bda7ce7bcbc996d7133576b1d" + }, + "survivor_token_comparisons": 3072, + "replacement_token_comparisons": 16, + "cases": [ + { + "case": "cancel-reuse", + "trigger": { + "host_pending": false, + "victim_imported_spans": 1, + "slot": 3, + "active_tokens": [ + 178, + 91, + 7 + ], + "progress_spans": 97, + "admitted": false + }, + "histogram": [ + 0, + 171, + 171, + 325, + 15 + ] + }, + { + "case": "producer-death", + "trigger": { + "host_pending": true, + "victim_imported_spans": 1, + "slot": 3, + "active_tokens": [ + 182, + 90, + 6 + ], + "progress_spans": 97, + "admitted": false + }, + "histogram": [ + 0, + 176, + 176, + 335, + 0 + ] + } + ] + } +] diff --git a/experiments/evidence270/atom270-layerprobe-audit.json b/experiments/evidence270/atom270-layerprobe-audit.json new file mode 100644 index 00000000..44de7f77 --- /dev/null +++ b/experiments/evidence270/atom270-layerprobe-audit.json @@ -0,0 +1,420 @@ +{ + "complete": true, + "compile_seconds": 154.00004827498924, + "scope": "First observed difference is attention module output including output projection. Not a specific kernel diagnosis.", + "raw_sha256": "8ef2432c82c9d39f4fb801c783776e5891fd5577a8b039fb3bcf2a21e3c31685", + "probes": [ + { + "input_tokens": 2048, + "index": 12, + "observations": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "pre_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "attention_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.001953125, + 0.001953125, + 0.001953125 + ] + }, + { + "boundary": 3, + "label": "post_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.001953125, + 0.001953125, + 0.001953125 + ] + }, + { + "boundary": 4, + "label": "mlp_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + { + "boundary": 5, + "label": "layer0_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + } + ], + "prior_instrumented_logit_records_equal": true + }, + { + "input_tokens": 2048, + "index": 13, + "observations": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "pre_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "attention_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0009765625, + 0.0009765625, + 0.0009765625 + ] + }, + { + "boundary": 3, + "label": "post_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.001953125, + 0.001953125, + 0.001953125 + ] + }, + { + "boundary": 4, + "label": "mlp_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + { + "boundary": 5, + "label": "layer0_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0078125, + 0.0078125, + 0.0078125 + ] + } + ], + "prior_instrumented_logit_records_equal": true + }, + { + "input_tokens": 512, + "index": 49, + "observations": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "pre_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "attention_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0009765625, + 0.0009765625, + 0.0009765625 + ] + }, + { + "boundary": 3, + "label": "post_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + { + "boundary": 4, + "label": "mlp_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + { + "boundary": 5, + "label": "layer0_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + } + ], + "prior_instrumented_logit_records_equal": true + }, + { + "input_tokens": 512, + "index": 50, + "observations": [ + { + "boundary": 0, + "label": "embedding", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 1, + "label": "pre_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.0, + 0.0, + 0.0 + ] + }, + { + "boundary": 2, + "label": "attention_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + { + "boundary": 3, + "label": "post_attention_norm", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.001953125, + 0.001953125, + 0.001953125 + ] + }, + { + "boundary": 4, + "label": "mlp_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.00390625, + 0.00390625, + 0.00390625 + ] + }, + { + "boundary": 5, + "label": "layer0_output", + "shape": [ + 4, + 1, + 2560 + ], + "dtype": "torch.bfloat16", + "max_abs_from_lane0": [ + 0.0, + 0.001953125, + 0.001953125, + 0.001953125 + ] + } + ], + "prior_instrumented_logit_records_equal": true + } + ] +} diff --git a/experiments/evidence270/atom270-qwen06-regression-retry1-audit.json b/experiments/evidence270/atom270-qwen06-regression-retry1-audit.json new file mode 100644 index 00000000..ecb52f90 --- /dev/null +++ b/experiments/evidence270/atom270-qwen06-regression-retry1-audit.json @@ -0,0 +1,299 @@ +{ + "stage": "audited", + "responses": 528, + "conditions": 16, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "2": 0 + }, + "opened": [ + "/dev/rbln2" + ] + }, + "raw_sha256": { + "atom270-qwen06-regression-retry1-outer.json": "7d05860ae23b3e670061819e6eb5c6adcc6e4b4c861f7a86adf7b9216fc81b8f", + "atom270-qwen06-regression-retry1-plan.json": "1aedb3d064f7ac6dcdc662d7a5f799b8f6567d9c7ee0d408aa14f7c99a4b9d5d", + "atom270-qwen06-regression-retry1-r0-whole.json": "5fc169b062bd3181b0a6fcfeef06e7586b79b2d0e502c2d485b22a4e020ba940", + "atom270-qwen06-regression-retry1-r0-whole.server0.log": "c109d050bdd9023aec04eec148a045715aa0cd8ce56be9ecc18160146ba0bb9c", + "atom270-qwen06-regression-retry1-r1-stream.json": "3ff17acf90f08ffe318f60155a0c23a0e3f65f6b0533f81eded290040e8056b5", + "atom270-qwen06-regression-retry1-r1-stream.server0.log": "81173dc46ac2c9c6da6230801fce5895e0858437e29091d40eb3bc9bd4023d01", + "atom270-qwen06-regression-retry1-r2-stream.json": "0a53097c042a7c46bd55c12c428c99afb792ed579816dd727a239a0dda0adf12", + "atom270-qwen06-regression-retry1-r2-stream.server0.log": "34ab0eff903c0f7f3589d7864d473f56bd3bb7470d904b926cf5135ca15e8fb1", + "atom270-qwen06-regression-retry1-r3-whole.json": "7621b42a720e44bce355e342ea536578b51fff212a93bf679b21c50704ad7bd7", + "atom270-qwen06-regression-retry1-r3-whole.server0.log": "611ba99cae925155b06b967f018bea0d28a25b1ad26b0e983979b38fa0e3f736" + }, + "rows": [ + { + "input_tokens": 2048, + "output_tokens": 32, + "whole": { + "output_tok_s": 23.579197994317543, + "median_first_sse_s": 11.793499694002094, + "median_e2e_s": 12.110661507525947, + "median_tpot_est_s": 0.010216625571633959, + "tok_s_samples": [ + 24.394001336303095, + 22.76439465233199 + ] + }, + "stream": { + "output_tok_s": 30.15207202484281, + "median_first_sse_s": 8.865385784505634, + "median_e2e_s": 9.423303873481927, + "median_tpot_est_s": 0.01704575523411134, + "tok_s_samples": [ + 30.12762881891311, + 30.17651523077251 + ] + }, + "stream_to_whole_tok_s": 1.2787573195708053, + "stream_to_whole_e2e": 0.7780998476116263 + }, + { + "input_tokens": 2048, + "output_tokens": 128, + "whole": { + "output_tok_s": 93.51517110007593, + "median_first_sse_s": 11.028839435486589, + "median_e2e_s": 12.644602753745858, + "median_tpot_est_s": 0.012733867590629137, + "tok_s_samples": [ + 97.60810379362088, + 89.42223840653098 + ] + }, + "stream": { + "output_tok_s": 113.25741800605329, + "median_first_sse_s": 8.921731540991459, + "median_e2e_s": 11.22233559601591, + "median_tpot_est_s": 0.01881865048611456, + "tok_s_samples": [ + 112.6933770293819, + 113.82145898272465 + ] + }, + "stream_to_whole_tok_s": 1.2111127710481335, + "stream_to_whole_e2e": 0.8875198228502186 + }, + { + "input_tokens": 8192, + "output_tokens": 32, + "whole": { + "output_tok_s": 20.440929089934805, + "median_first_sse_s": 14.119025979511207, + "median_e2e_s": 15.826438275515102, + "median_tpot_est_s": 0.054870423208358846, + "tok_s_samples": [ + 21.44422258867457, + 19.43763559119504 + ] + }, + "stream": { + "output_tok_s": 16.196140983753573, + "median_first_sse_s": 17.364907934999792, + "median_e2e_s": 19.79581503351801, + "median_tpot_est_s": 0.07701502136431725, + "tok_s_samples": [ + 16.23596081322384, + 16.156321154283308 + ] + }, + "stream_to_whole_tok_s": 0.7923387881487548, + "stream_to_whole_e2e": 1.2508067000863918 + }, + { + "input_tokens": 8192, + "output_tokens": 128, + "whole": { + "output_tok_s": 53.11295352654426, + "median_first_sse_s": 19.572416286740918, + "median_e2e_s": 24.817380180757027, + "median_tpot_est_s": 0.039747054714528024, + "tok_s_samples": [ + 51.875024957976585, + 54.35088209511194 + ] + }, + "stream": { + "output_tok_s": 53.35442869912747, + "median_first_sse_s": 19.17754786973819, + "median_e2e_s": 26.33155713728047, + "median_tpot_est_s": 0.042779082874217486, + "tok_s_samples": [ + 52.7894928068765, + 53.91936459137844 + ] + }, + "stream_to_whole_tok_s": 1.004546445952446, + "stream_to_whole_e2e": 1.0610127638572224 + } + ], + "host_stages": [ + { + "remote_completion": { + "calls": 132, + "errors": 0, + "wall_s": 492.4705371998716 + }, + "import": { + "calls": 132, + "errors": 0, + "wall_s": 76.67210865276866 + }, + "decode/1": { + "calls": 4385, + "errors": 0, + "wall_s": 45.29988210671581 + }, + "decode/2": { + "calls": 1361, + "errors": 0, + "wall_s": 21.072782702278346 + }, + "decode/3": { + "calls": 255, + "errors": 0, + "wall_s": 5.946987624512985 + }, + "decode/4": { + "calls": 639, + "errors": 0, + "wall_s": 18.24688756687101 + } + }, + { + "import": { + "calls": 3696, + "errors": 0, + "wall_s": 95.6276512950426 + }, + "remote_completion": { + "calls": 132, + "errors": 0, + "wall_s": 382.94632272736635 + }, + "decode/1": { + "calls": 1698, + "errors": 0, + "wall_s": 18.066271065734327 + }, + "decode/2": { + "calls": 1359, + "errors": 0, + "wall_s": 15.615915696835145 + }, + "decode/3": { + "calls": 480, + "errors": 0, + "wall_s": 7.34489822620526 + }, + "decode/4": { + "calls": 1143, + "errors": 0, + "wall_s": 32.43782124703284 + } + }, + { + "import": { + "calls": 3696, + "errors": 0, + "wall_s": 98.05997085268609 + }, + "remote_completion": { + "calls": 132, + "errors": 0, + "wall_s": 379.95639754168224 + }, + "decode/1": { + "calls": 1645, + "errors": 0, + "wall_s": 17.683394415304065 + }, + "decode/2": { + "calls": 1420, + "errors": 0, + "wall_s": 16.931638494483195 + }, + "decode/3": { + "calls": 561, + "errors": 0, + "wall_s": 8.901394719374366 + }, + "decode/4": { + "calls": 1065, + "errors": 0, + "wall_s": 30.38444321951829 + } + }, + { + "remote_completion": { + "calls": 132, + "errors": 0, + "wall_s": 490.5047698447015 + }, + "import": { + "calls": 132, + "errors": 0, + "wall_s": 78.30497097945772 + }, + "decode/1": { + "calls": 4781, + "errors": 0, + "wall_s": 49.53998960473109 + }, + "decode/2": { + "calls": 1188, + "errors": 0, + "wall_s": 19.2377365489956 + }, + "decode/3": { + "calls": 265, + "errors": 0, + "wall_s": 6.2684115517186 + }, + "decode/4": { + "calls": 619, + "errors": 0, + "wall_s": 17.58593289880082 + } + } + ], + "histograms": [ + [ + 0, + 4385, + 1361, + 255, + 639 + ], + [ + 0, + 1698, + 1359, + 480, + 1143 + ], + [ + 0, + 1645, + 1420, + 561, + 1065 + ], + [ + 0, + 4781, + 1188, + 265, + 619 + ] + ], + "startup_s": [ + 14.30028600699734, + 14.053873052005656, + 14.30276256101206, + 14.101302650989965 + ], + "limits": "Two measured batches per route/condition, same Pod and exact fixed-input outputs. Not arbitrary prompt quality or confidence intervals. Host import includes conversion/staging/SDK, not pure DMA. No isolated cache-hit claim; startup excluded from throughput." +} \ No newline at end of file diff --git a/experiments/evidence270/atom270-ready-control.json b/experiments/evidence270/atom270-ready-control.json new file mode 100644 index 00000000..fc5716c4 --- /dev/null +++ b/experiments/evidence270/atom270-ready-control.json @@ -0,0 +1,11 @@ +{ + "test": "test_atom270_ready_control.py", + "sha256": "86e269488fda2b6262ca9ef3b6f371983b00b9f802d12136ad0d18718d9ada1f", + "passed": 2, + "elapsed_s": 3.04, + "all_ready_max_lanes": 4, + "serial_ready_max_lanes": 1, + "outputs_equal": true, + "scope": "CPU FakeBlocks, dense and hybrid-admission branches; synthetic all-global KV. Not Gemma sliding-window or ATOM hardware proof.", + "initial_failure": "Hybrid fixture decoder wrapper lacked dec_attn_mask; fixture corrected, production engine untouched." +} diff --git a/experiments/evidence270/atom270-samecard-retry1-audit.json b/experiments/evidence270/atom270-samecard-retry1-audit.json new file mode 100644 index 00000000..09d41799 --- /dev/null +++ b/experiments/evidence270/atom270-samecard-retry1-audit.json @@ -0,0 +1,61 @@ +{ + "complete": true, + "cases": 128, + "identical_rows_across_all_arms": true, + "free_reference_matches_per_arm": 4, + "free_cases_per_arm": 16, + "binding": { + "visible_count": 1, + "physical_to_visible": { + "2": 0 + }, + "opened": [ + "/dev/rbln2" + ] + }, + "forced_index13": [ + { + "lane": 0, + "token": 369, + "logit369": 29.75, + "logit264": 29.625 + }, + { + "lane": 1, + "token": 264, + "logit369": 29.625, + "logit264": 29.625 + }, + { + "lane": 2, + "token": 264, + "logit369": 29.625, + "logit264": 29.625 + }, + { + "lane": 3, + "token": 264, + "logit369": 29.625, + "logit264": 29.625 + } + ], + "raws": [ + { + "file": "atom270-qwen4b-samecard-retry1-a1.json", + "sha256": "0c7ef20fd373eb1993680aaab83d12c63ab0ed1d45098abca30332dfc6c9383f" + }, + { + "file": "atom270-qwen4b-samecard-retry1-b1-prefill0.json", + "sha256": "c8f9b4c29d4f583c9753785f67fc90fb35d0088abb1e8820c75972abada0ba77" + }, + { + "file": "atom270-qwen4b-samecard-retry1-b2-prefill0.json", + "sha256": "c8f9b4c29d4f583c9753785f67fc90fb35d0088abb1e8820c75972abada0ba77" + }, + { + "file": "atom270-qwen4b-samecard-retry1-a2.json", + "sha256": "0c7ef20fd373eb1993680aaab83d12c63ab0ed1d45098abca30332dfc6c9383f" + } + ], + "cleanup_exit_code": 0 +} diff --git a/experiments/evidence270/atom270-timeline-gemma-b4-stream-summary.json b/experiments/evidence270/atom270-timeline-gemma-b4-stream-summary.json new file mode 100644 index 00000000..b677e5fd --- /dev/null +++ b/experiments/evidence270/atom270-timeline-gemma-b4-stream-summary.json @@ -0,0 +1,49 @@ +{ + "remote": { + "completed": 264, + "errors": 0, + "submit_to_completion_s": { + "count": 264, + "sum": 1142.7042513388442, + "min": 1.510972999036312, + "median": 4.303183654556051, + "max": 6.169583000941202 + }, + "completion_gap_s": { + "count": 263, + "sum": 424.09189840906765, + "min": 1.4572016739984974, + "median": 1.5101190001005307, + "max": 3.5004626109730452 + }, + "backlogged_completion_gap_s": { + "count": 240, + "sum": 363.64755154878367, + "min": 1.4572016739984974, + "median": 1.5085299970232882, + "max": 1.793420379050076 + } + }, + "decode": { + "calls": 17220, + "errors": 0, + "lane_histogram": { + "1": 13676, + "2": 3452, + "3": 92 + }, + "max_lanes": 3, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 12314, + "calls_with_prefilling": 15324, + "host_call_s": { + "count": 17220, + "sum": 224.36575269058812, + "min": 0.011706677032634616, + "median": 0.012880158028565347, + "max": 0.018264282029122114 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "bebfb02bde6f7c764fd6d9acd792c1621d84ebc7bd486c8c84208c37ab552c8a" +} diff --git a/experiments/evidence270/atom270-timeline-gemma-b4-whole-comparison.json b/experiments/evidence270/atom270-timeline-gemma-b4-whole-comparison.json new file mode 100644 index 00000000..ced847ab --- /dev/null +++ b/experiments/evidence270/atom270-timeline-gemma-b4-whole-comparison.json @@ -0,0 +1,51 @@ +{ + "backlogged_gap_by_input": { + "128": { + "count": 60, + "median_s": 1.5538023289991543, + "min_s": 1.517794425948523, + "max_s": 1.6095911220181733 + }, + "512": { + "count": 60, + "median_s": 1.5557903224835172, + "min_s": 1.510746017913334, + "max_s": 1.5699333479860798 + }, + "2048": { + "count": 60, + "median_s": 1.5534528234857135, + "min_s": 1.510221569915302, + "max_s": 1.595483287004754 + }, + "8192": { + "count": 60, + "median_s": 1.5562832074938342, + "min_s": 1.5177083379821852, + "max_s": 1.6454418420325965 + } + }, + "load_s": 840.5813920759829, + "metrics_8192_128": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.845269416924566, + "output_tok_s": 76.28904624473022, + "median_first_sse_s": 13.461700428510085, + "median_tpot_est_s": 0.01794637393712352, + "median_e2e_s": 15.955143106519245, + "all_text_equal": true + }, + "baseline_metrics_8192_128": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.56643162900582, + "output_tok_s": 77.0897660852558, + "median_first_sse_s": 13.33676557202125, + "median_tpot_est_s": 0.017586364243986217, + "median_e2e_s": 15.773472197470255, + "all_text_equal": true + }, + "reference_match_conditions": 8, + "limits": "Not isolated probe overhead: sequential runs with possible host/card variation; one timed batch per condition. Gaps are host completion gaps with backlog, not pure NPU compute." +} diff --git a/experiments/evidence270/atom270-timeline-gemma-b4-whole-summary.json b/experiments/evidence270/atom270-timeline-gemma-b4-whole-summary.json new file mode 100644 index 00000000..d0578297 --- /dev/null +++ b/experiments/evidence270/atom270-timeline-gemma-b4-whole-summary.json @@ -0,0 +1,49 @@ +{ + "remote": { + "completed": 264, + "errors": 0, + "submit_to_completion_s": { + "count": 264, + "sum": 1170.3865798402112, + "min": 1.5152255450375378, + "median": 4.479313701973297, + "max": 6.292836716049351 + }, + "completion_gap_s": { + "count": 263, + "sum": 434.28598437004257, + "min": 1.510221569915302, + "median": 1.5562208250630647, + "max": 3.503013251000084 + }, + "backlogged_completion_gap_s": { + "count": 240, + "sum": 373.0109350649873, + "min": 1.510221569915302, + "median": 1.5548035525134765, + "max": 1.6454418420325965 + } + }, + "decode": { + "calls": 17353, + "errors": 0, + "lane_histogram": { + "1": 13952, + "2": 3299, + "3": 102 + }, + "max_lanes": 3, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 12429, + "calls_with_prefilling": 15457, + "host_call_s": { + "count": 17353, + "sum": 230.8325613248162, + "min": 0.011753769940696657, + "median": 0.013146739918738604, + "max": 0.01930820499546826 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "cbc6739e05c48039d3118b6422d5a1264c7dc44688ee7eca69538d95f267a415" +} diff --git a/experiments/evidence270/atom270-timeline-gemma-pair-summary.json b/experiments/evidence270/atom270-timeline-gemma-pair-summary.json new file mode 100644 index 00000000..ef9b4ff2 --- /dev/null +++ b/experiments/evidence270/atom270-timeline-gemma-pair-summary.json @@ -0,0 +1,142 @@ +{ + "routes": { + "whole": { + "result_sha256": "6a34ace19a9b54cac19e0bfe500e6eb5081fdd76ad4f1bfadfc53b7cb87ecd6b", + "load_s": 840.5813920759829, + "metrics_8192_128": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.845269416924566, + "output_tok_s": 76.28904624473022, + "median_first_sse_s": 13.461700428510085, + "median_tpot_est_s": 0.01794637393712352, + "median_e2e_s": 15.955143106519245, + "all_text_equal": true + }, + "timeline": { + "remote": { + "completed": 264, + "errors": 0, + "submit_to_completion_s": { + "count": 264, + "sum": 1170.3865798402112, + "min": 1.5152255450375378, + "median": 4.479313701973297, + "max": 6.292836716049351 + }, + "completion_gap_s": { + "count": 263, + "sum": 434.28598437004257, + "min": 1.510221569915302, + "median": 1.5562208250630647, + "max": 3.503013251000084 + }, + "backlogged_completion_gap_s": { + "count": 240, + "sum": 373.0109350649873, + "min": 1.510221569915302, + "median": 1.5548035525134765, + "max": 1.6454418420325965 + } + }, + "decode": { + "calls": 17353, + "errors": 0, + "lane_histogram": { + "1": 13952, + "2": 3299, + "3": 102 + }, + "max_lanes": 3, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 12429, + "calls_with_prefilling": 15457, + "host_call_s": { + "count": 17353, + "sum": 230.8325613248162, + "min": 0.011753769940696657, + "median": 0.013146739918738604, + "max": 0.01930820499546826 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "cbc6739e05c48039d3118b6422d5a1264c7dc44688ee7eca69538d95f267a415" + } + }, + "stream": { + "result_sha256": "b4658cbaa41b7e1f74de7cd614ef20b8f99bf6ed2983e6d7023e697bafcdf94d", + "load_s": 18.39854133501649, + "metrics_8192_128": { + "requests": 16, + "output_tokens": 2048, + "wall_s": 26.31547353998758, + "output_tok_s": 77.82493432572929, + "median_first_sse_s": 13.238841598504223, + "median_tpot_est_s": 0.0175043796968287, + "median_e2e_s": 15.669753587979358, + "all_text_equal": true + }, + "timeline": { + "remote": { + "completed": 264, + "errors": 0, + "submit_to_completion_s": { + "count": 264, + "sum": 1142.7042513388442, + "min": 1.510972999036312, + "median": 4.303183654556051, + "max": 6.169583000941202 + }, + "completion_gap_s": { + "count": 263, + "sum": 424.09189840906765, + "min": 1.4572016739984974, + "median": 1.5101190001005307, + "max": 3.5004626109730452 + }, + "backlogged_completion_gap_s": { + "count": 240, + "sum": 363.64755154878367, + "min": 1.4572016739984974, + "median": 1.5085299970232882, + "max": 1.793420379050076 + } + }, + "decode": { + "calls": 17220, + "errors": 0, + "lane_histogram": { + "1": 13676, + "2": 3452, + "3": 92 + }, + "max_lanes": 3, + "calls_with_ready_prefill": 0, + "calls_with_waiting": 12314, + "calls_with_prefilling": 15324, + "host_call_s": { + "count": 17220, + "sum": 224.36575269058812, + "min": 0.011706677032634616, + "median": 0.012880158028565347, + "max": 0.018264282029122114 + } + }, + "limits": "All harness phases included. Completion gaps include client/phase idle periods; submit-to-completion includes producer queueing. Queue states are snapshots at decode entry, not continuous ready-time measurements. Python intervals do not prove DMA overlap; overlapping sums are not wall-clock decomposition.", + "source_sha256": "bebfb02bde6f7c764fd6d9acd792c1621d84ebc7bd486c8c84208c37ab552c8a" + } + } + }, + "limits": "One instrumented timed batch per condition; same physical ATOM2. Not statistical improvement or device overlap proof. Hybrid B4 early device import is deferred.", + "full_reference_matches": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "throughput_ratio": 1.0201324850237614 +} diff --git a/experiments/evidence270/atom270_allcore_probe.py b/experiments/evidence270/atom270_allcore_probe.py new file mode 100644 index 00000000..30367d23 --- /dev/null +++ b/experiments/evidence270/atom270_allcore_probe.py @@ -0,0 +1,15 @@ +"""Task-only diagnostic: append all core-attention outputs; retain all hidden states.""" +from optimum.rbln.transformers.models.qwen3.qwen3_architecture import Qwen3Wrapper +class CoreProbeWrapper(Qwen3Wrapper): + def __init__(self,*args,**kwargs): + super().__init__(*args,**kwargs) + self._observed=[] + def record(module,args,output):self._observed.append(output) + for layer in self.model.model.layers: + layer.self_attn.get_attention_op().register_forward_hook(record) + def forward(self,*args): + self._observed=[] + logits,states=super().forward(*args) + assert len(self._observed)==36 and len(states)==37 + if self.phase!='decode':return logits,states + return logits,(*states,*self._observed) diff --git a/experiments/evidence270/atom270_attention_raw_probe.py b/experiments/evidence270/atom270_attention_raw_probe.py new file mode 100644 index 00000000..5f4867d4 --- /dev/null +++ b/experiments/evidence270/atom270_attention_raw_probe.py @@ -0,0 +1,22 @@ +"""Task-only export instrumentation; installed SDK and production engine unchanged.""" +from optimum.rbln.transformers.models.qwen3.qwen3_architecture import Qwen3Wrapper +LABELS=['embedding','q_proj','k_proj','v_proj','q_norm','k_norm','rotary_q','rotary_k','attention_v','attention_result','o_proj'] +class AttentionProbeWrapper(Qwen3Wrapper): + def __init__(self,*args,**kwargs): + super().__init__(*args,**kwargs) + a=self.model.model.layers[0].self_attn + self._observed=[] + def record(module,args,output): + self._observed.append(output) + def inputs(module,args): + for value in args[:3]:self._observed.append(value) + for module in [a.q_proj,a.k_proj,a.v_proj,a.q_norm,a.k_norm]:module.register_forward_hook(record) + a.get_attention_op().register_forward_pre_hook(inputs) + a.get_attention_op().register_forward_hook(record) + a.o_proj.register_forward_hook(record) + def forward(self,*args): + self._observed=[] + logits,states=super().forward(*args) + assert len(self._observed)==10 and len(states)==37 + if self.phase!='decode':return logits,states + return logits,(states[0],*self._observed,*states[11:]) diff --git a/experiments/evidence270/atom270_layer_probe.py b/experiments/evidence270/atom270_layer_probe.py new file mode 100644 index 00000000..cb498134 --- /dev/null +++ b/experiments/evidence270/atom270_layer_probe.py @@ -0,0 +1,24 @@ +"""Isolated export instrumentation: original arithmetic, selected first-layer outputs. + +This module is imported only by the task-owned diagnostic compiler process. +It never modifies installed SDK files or a serving engine. +""" +from optimum.rbln.transformers.models.qwen3.qwen3_architecture import Qwen3Wrapper + +class LayerProbeWrapper(Qwen3Wrapper): + def __init__(self,*args,**kwargs): + super().__init__(*args,**kwargs) + layer=self.model.model.layers[0] + self._observed=[] + def record(module,args,output): + self._observed.append(output) + for module in [layer.get_pre_attention_layernorm(),layer.self_attn, + layer.get_post_attention_layernorm(),layer.get_mlp(),layer]: + module.register_forward_hook(record) + + def forward(self,*args): + self._observed=[] + logits,states=super().forward(*args) + assert len(self._observed)==5 and len(states)==37 + # Keep the SDK output count/shapes; slots0..5 are diagnostic, not layer indices. + return logits,(states[0],*self._observed,*states[6:]) diff --git a/experiments/evidence270/atom_server270_profile.py b/experiments/evidence270/atom_server270_profile.py new file mode 100644 index 00000000..a52d6a41 --- /dev/null +++ b/experiments/evidence270/atom_server270_profile.py @@ -0,0 +1,33 @@ +"""Separate diagnostic entrypoint; leaves the baseline HTTP server unchanged.""" +import json +import atom_server270_http as server +from stage_probe270 import StageProbe +from timeline270 import TimelineProbe + +probes = [] +timelines = [] +original_observer = server.observe_decode_batches + + +def observe_with_probe(batcher): + histogram = original_observer(batcher) + probe = StageProbe() + probe.install(batcher) + probes.append(probe) + timeline = TimelineProbe() + timeline.install(batcher) + timelines.append(timeline) + return histogram + + +if __name__ == '__main__': + server.observe_decode_batches = observe_with_probe + try: + server.main() + finally: + # server.main closes the producer before return, settling completion callbacks. + for probe in probes: + print('ATOM270_HOST_STAGE_TIMES ' + json.dumps(probe.snapshot()), flush=True) + + for timeline in timelines: + print("ATOM270_HOST_TIMELINE " + json.dumps(timeline.snapshot()), flush=True) diff --git a/experiments/evidence270/atom_server270_quality.py b/experiments/evidence270/atom_server270_quality.py new file mode 100644 index 00000000..8e22974d --- /dev/null +++ b/experiments/evidence270/atom_server270_quality.py @@ -0,0 +1,143 @@ +"""Single assigned ATOM HTTP backend, optionally fed by the established RNGD subprocess adapter.""" +import argparse +import importlib.util +import os +import json +import signal +import sys +from pathlib import Path + + +def parse_args(argv=None): + p = argparse.ArgumentParser(description=__doc__) + p.add_argument('--model', required=True) + p.add_argument('--cache-dir', required=True) + p.add_argument('--port', required=True, type=int) + p.add_argument('--batching', required=True, help='trusted ATOM batching module') + p.add_argument('--http-module', required=True, help='trusted HTTP module') + p.add_argument('--pd-module', required=True, help='trusted subprocess worker module') + p.add_argument('--worker-python') + p.add_argument('--stream-layers', action='store_true') + p.add_argument('--engine-factory') + p.add_argument('--width', type=int, default=8192) + p.add_argument('--kv-blocks', type=int, default=3) + p.add_argument('--batch-size', type=int, choices=(1,4), default=1) + p.add_argument('--tensor-parallel-size', type=int, choices=(1,2), default=1) + p.add_argument('--max-output', type=int, default=128) + a = p.parse_args(argv) + if a.kv_blocks < a.batch_size + 1: + p.error('kv-blocks must reserve one scratch block beyond batch-size') + if bool(a.worker_python) != bool(a.engine_factory): + p.error('worker-python and engine-factory must be supplied together') + if a.stream_layers and not a.worker_python: + p.error('stream-layers requires a subprocess producer') + return a + + +def check_shm_capacity(config, batch_size, width, *, free_bytes=None): + if any(type(x) is not int or x <= 0 for x in (batch_size, width)): + raise ValueError('batch size and width must be positive integers') + head_dim = getattr(config, 'head_dim', None) or config.hidden_size // config.num_attention_heads + required = batch_size * 2 * config.num_hidden_layers * config.num_key_value_heads * width * head_dim * 2 + if free_bytes is None: + stat = os.statvfs('/dev/shm') + free_bytes = stat.f_bavail * stat.f_frsize + if required > free_bytes: + raise RuntimeError(f'Insufficient shared memory: KV slab needs {required} bytes, available {free_bytes}') + return required + + +def validate_compiled_geometry(config, batch_size, blocks, *, tensor_parallel_size=1, artifact_config=None): + counts = [getattr(c, 'num_devices', None) for c in getattr(config, 'compile_cfgs', [])] + if artifact_config is not None: + geometry = tuple(artifact_config.get(k) for k in ('batch_size', 'kvcache_num_blocks', 'max_seq_len')) + if geometry != (batch_size, blocks, 9216) or any(n is not None and n != tensor_parallel_size for n in counts): + raise ValueError('Runtime and saved artifact geometry disagree') + # SDK loading may clear num_devices on config objects; saved compile metadata retains it. + counts = [c.get('num_devices') for c in artifact_config.get('_compile_cfgs', [])] + if ((config.batch_size, config.kvcache_num_blocks, config.max_seq_len) != (batch_size, blocks, 9216) + or not counts or any(n != tensor_parallel_size for n in counts)): + raise ValueError('Compiled batch/block/context geometry does not match experiment') + + +def observe_decode_batches(batcher): + histogram = [0] * (batcher.lanes + 1) + original = batcher._decode_step + def step(): + count = len(batcher.decoding) + result = original() + histogram[count] += 1 + return result + batcher._decode_step = step + return histogram + + +def main(): + a = parse_args() + for name, path in [('opt_prime.accel.pd.pd_batching', a.pd_module), + ('opt_prime.accel.platforms.atom.batching', a.batching), + ('opt_prime.accel.serve.llmd', a.http_module)]: + spec = importlib.util.spec_from_file_location(name, path) + module = importlib.util.module_from_spec(spec) + sys.modules[name] = module + spec.loader.exec_module(module) + from opt_prime.accel.pd.pd_batching import SubprocessPrefillWorker + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from opt_prime.accel.serve.batched import BatchedOwner, BatchedService + from opt_prime.accel.serve.llmd import serve_stage + from transformers import AutoTokenizer, AutoConfig + + if a.worker_python: + config = AutoConfig.from_pretrained(a.model, local_files_only=True).get_text_config() + required = check_shm_capacity(config, a.batch_size, a.width) + print(f'PD_SHM_REQUIRED_BYTES {required}', flush=True) + nodes = list(Path('/dev').glob('rbln[0-9]*')) + if len(nodes) != a.tensor_parallel_size: + raise RuntimeError('assigned ATOM count must equal requested tensor parallel size') + engine = AtomRblnEngine(devices=sorted(int(n.name[4:]) for n in nodes), max_seq_len=9216, + batch_size=a.batch_size, kv_blocks=a.kv_blocks, cache_dir=a.cache_dir) + worker = owner = server = http = None + occupancy = None + try: + engine.start(a.model) + validate_compiled_geometry(engine.model.rbln_config, a.batch_size, a.kv_blocks, tensor_parallel_size=a.tensor_parallel_size, + artifact_config=json.loads((Path(engine.dir) / 'rbln_config.json').read_text())) + print('ATOM270_COMPILED ' + json.dumps(dict(batch_size=a.batch_size, kv_blocks=a.kv_blocks, context=9216, tensor_parallel_size=a.tensor_parallel_size)), flush=True) + c = engine.model.config.get_text_config() + if a.worker_python: + worker = SubprocessPrefillWorker(a.worker_python, + ['--engine', 'factory', '--model', a.model, '--width', str(a.width), '--engine-factory', a.engine_factory], + slots=a.batch_size, layers=c.num_hidden_layers, kv_heads=c.num_key_value_heads, + head_dim=getattr(c, 'head_dim', None) or c.hidden_size // c.num_attention_heads, + max_context=a.width, stream_layers=a.stream_layers) + tok = AutoTokenizer.from_pretrained(a.model, local_files_only=True) + batcher = engine.batcher(prefix_cache=False, prefill_worker=worker, local_prefill=worker is None) + occupancy = observe_decode_batches(batcher) + from transformers import GenerationConfig + eos=GenerationConfig.from_pretrained(a.model,local_files_only=True).eos_token_id + eos=eos if isinstance(eos,list) else ([] if eos is None else [eos]) + eos=sorted(set(eos+([] if tok.eos_token_id is None else [tok.eos_token_id]))) + assert eos, 'Quality validation requires explicit EOS handling' + print('ATOM270_QUALITY_EOS '+json.dumps(eos),flush=True) + service = BatchedService(batcher, model=a.model, vocab_size=c.vocab_size, + max_output=a.max_output, tokenizer=tok, eos_token_ids=eos, + render=lambda ids: tok.decode(ids, skip_special_tokens=True)) + owner = BatchedOwner(service) + # Quality requests use explicit model/tokenizer EOS; performance benchmark entrypoint is unchanged. + signal.signal(signal.SIGTERM, lambda *_: owner.submit('stop')) + server, http = serve_stage(owner, 'decode', a.port) + owner.run() + finally: + if occupancy is not None: + print('ATOM270_ACTIVE_LANES ' + json.dumps(occupancy), flush=True) + if server: + server.shutdown() + server.server_close() + http.join(5) + if worker: + worker.close() + engine.close() + + +if __name__ == '__main__': + main() diff --git a/experiments/evidence270/audit_atom270_abba.py b/experiments/evidence270/audit_atom270_abba.py new file mode 100644 index 00000000..1dcbb4ad --- /dev/null +++ b/experiments/evidence270/audit_atom270_abba.py @@ -0,0 +1,64 @@ +"""Audit completed same-Pod ATOM width ABBA; occupancy is reported independently.""" +import argparse,copy,hashlib,json,statistics +from pathlib import Path +from audit_http265 import audit_result + + +def record(log,prefix): + found=[json.loads(line[len(prefix):]) for line in log.splitlines() if line.startswith(prefix)] + assert len(found)==1, f'Expected one {prefix}' + return found[0] + + +def audit(queue,runs,logs): + assert queue['state']=='complete' and queue['cleanup_exit_code']==0,'Incomplete or failed queue' + rows=queue['runs'];assert len(rows)==len(runs)==len(logs)==4,'Incomplete ABBA' + assert [q['width'] for q in rows]==[8192,1024,1024,8192] + assert len({q['stem'] for q in rows})==4 and all(Path(q['stem']).name==q['stem'] for q in rows) + configs=[];bindings=[];histograms=[] + for q,d,log in zip(rows,runs,logs): + assert q['exit_code']==0,'Failed child' + audit_result(d,'atom','whole',lengths=(128,512),expected_slots=4,expected_atom_devices=1) + assert d['producer_chunk']==q['width'] and d['kv_blocks']==5 and d['tensor_parallel_size']==1 + assert d['prompts']==runs[0]['prompts'] and d['references']==runs[0]['references'] + assert all(d[k]==runs[0][k] for k in ['model','dtype','torch','transformers','harness_sha256']) + cfg=copy.deepcopy(d['server_config']);assert len(cfg['servers'])==1 + argv=cfg['servers'][0];pos=argv.index('--width')+1 + assert argv[pos]==str(q['width']) and cfg['metadata']['producer_chunk']==q['width'] + argv[pos]='WIDTH';cfg['metadata']['producer_chunk']='WIDTH';configs.append(cfg) + assert record(log,'ATOM270_COMPILED ')==dict(batch_size=4,kv_blocks=5,context=9216,tensor_parallel_size=1) + binding=record(log,'ATOM270_BINDING ');assert binding==q['binding'] + assert binding['visible_count']==1 and len(binding['physical_to_visible'])==1 + assert binding['opened']==['/dev/rbln'+next(iter(binding['physical_to_visible']))] + bindings.append(binding) + hist=record(log,'ATOM270_ACTIVE_LANES ') + assert len(hist)==5 and all(type(n) is int and n>=0 for n in hist) and sum(hist)>0 + histograms.append(hist) + assert all(c==configs[0] for c in configs),'Other config changed' + assert all(b==bindings[0] for b in bindings),'Physical binding changed' + comparisons=[] + for n in (128,512): + for o in (32,128): + samples={} + for label,indices in [('wide',(0,3)),('narrow',(1,2))]: + samples[label]=[next(b['summary'] for b in runs[i]['batches'] if (b['input_tokens'],b['output_tokens_per_request'])==(n,o)) for i in indices] + ratio=statistics.median(s['output_tok_s'] for s in samples['narrow'])/statistics.median(s['output_tok_s'] for s in samples['wide']) + comparisons.append(dict(input_tokens=n,output_tokens=o,samples=samples,throughput_ratio=ratio)) + return dict(stage='passed',conditions=16,warmups=16,correctness_responses=256,timed_text_responses=256,binding=bindings[0],active_lane_histograms=histograms,full_batch_observed=[h[4]>0 for h in histograms],rows=comparisons,limits='Two timed batches per width per condition, whole route, short inputs only. No confidence interval, heldout quality acceptance, other-model generalization or device overlap proof. Occupancy failures remain separate from ABBA correctness.') + + +if __name__=='__main__': + p=argparse.ArgumentParser(__doc__);p.add_argument('--queue',type=Path,required=True);p.add_argument('--output',type=Path,required=True);a=p.parse_args() + q=json.loads(a.queue.read_text()) + if q['state']=='failed' or any(r['exit_code']!=0 for r in q['runs']): + print('failed: queue/child failure');raise SystemExit(1) + if q['state']!='complete': + print(f"pending: {len(q['runs'])}/4 recorded rows");raise SystemExit(2) + paths=[a.queue];data=[];logs=[] + for row in q['runs']: + assert Path(row['stem']).name==row['stem'] + rp=a.queue.parent/(row['stem']+'.json');lp=a.queue.parent/(row['stem']+'.server0.log') + data.append(json.loads(rp.read_text()));logs.append(lp.read_text());paths.extend([rp,lp]) + out=audit(q,data,logs);out['source_sha256']={p.name:hashlib.sha256(p.read_bytes()).hexdigest() for p in paths} + with a.output.open('x') as f:json.dump(out,f,indent=2);f.write('\n') + print(json.dumps(out,indent=2)) diff --git a/experiments/evidence270/audit_atom270_allcore_abba.py b/experiments/evidence270/audit_atom270_allcore_abba.py new file mode 100644 index 00000000..6d466c16 --- /dev/null +++ b/experiments/evidence270/audit_atom270_allcore_abba.py @@ -0,0 +1,69 @@ +"""Audit completed ABBA data without converting parity failures to passes.""" +import hashlib,json,math,statistics +from pathlib import Path +A=Path(__file__).resolve().parent +CONDITIONS={(512,128),(8192,128)} + +def audit_arm(d): + assert d['passed'] and d['slots_per_decoder']==4 and d['physical_atom']==1 and d['physical_rngd']==d['physical_gpus']==0 + assert d['context']==9216 and not d['decoder_cache'] and not d['producer_cache'] + assert set(d['prompts'])=={'512','8192'} and all(len(v)==int(k) for k,v in d['prompts'].items()) + assert set(d['references'])=={'512:128','8192:128'} + assert len(d['warmups'])==len(d['correctness_batches'])==len(d['batches'])==2 + assert {(b['input_tokens'],b['output_tokens']) for b in d['correctness_batches']}==CONDITIONS + assert {(b['input_tokens'],b['output_tokens_per_request']) for b in d['batches']}==CONDITIONS + warm=set();flags=[];token_failures=0;text_failures=0 + for r in d['warmups']: + _,endpoint,n,o=r['id'].split('-');n,o=int(n),int(o);warm.add((n,o));assert endpoint=='0' + ref=d['references'][f'{n}:{o}'];assert len(ref['tokens'])==len(r['tokens'])==o + equal=r['tokens']==ref['tokens'];assert r['tokens_equal']==equal;flags.append(equal) + assert warm==CONDITIONS + for b in d['correctness_batches']: + n,o=b['input_tokens'],b['output_tokens'];ref=d['references'][f'{n}:{o}'] + assert b['concurrency']==16 and len(b['rows'])==len({r['id'] for r in b['rows']})==16 + for r in b['rows']: + assert len(r['tokens'])==o + equal=r['tokens']==ref['tokens'];assert r['tokens_equal']==equal;flags.append(equal);token_failures+=not equal + for b in d['batches']: + n,o=b['input_tokens'],b['output_tokens_per_request'];ref=d['references'][f'{n}:{o}'];rows=b['rows'];s=b['summary'] + assert b['concurrency']==16 and b['repeat']==0 and len(rows)==len({r['id'] for r in rows})==16 + equalities=[] + for r in rows: + assert r['input_tokens']==n and r['output_tokens']==o + equal=r['text']==ref['text'];assert r['text_equal']==equal;equalities.append(equal);flags.append(equal);text_failures+=not equal + assert all(math.isfinite(r[k]) for k in ['first_sse_s','e2e_s','tpot_est_s']) + assert 0=0 + assert s['requests']==16 and s['output_tokens']==16*o and s['all_text_equal']==all(equalities) + assert math.isfinite(s['wall_s']) and s['wall_s']>0 + expected={'output_tok_s':16*o/s['wall_s'],'median_first_sse_s':statistics.median(r['first_sse_s'] for r in rows),'median_e2e_s':statistics.median(r['e2e_s'] for r in rows),'median_tpot_est_s':statistics.median(r['tpot_est_s'] for r in rows)} + assert all(math.isclose(s[k],v,rel_tol=1e-10,abs_tol=1e-10) for k,v in expected.items()) + assert d['parity_passed']==all(flags) + return {'responses':66,'parity_passed':all(flags),'failed_token_checks':token_failures,'failed_timed_text_checks':text_failures} + +def main(): + outer=json.loads((A/'atom270-allcore-abba-outer.json').read_text());assert outer['state']=='complete' and outer['cleanup_exit_code']==outer['exit_code']==0 + queue=json.loads((A/'atom270-allcore-abba-arms.json').read_text());assert [x['arm'] for x in queue]==['a1','b1','b2','a2'] + result={'arms':{},'comparisons':{},'limits':'Same-card ABBA, two timed batches per variant/condition. Original parity failures remain failures; no equivalent-output speedup or general-quality claim.'};data={};bindings=[] + for q in queue: + arm=q['arm'];p=A/f'atom270-qwen4b-allcore-abba-{arm}.json';d=json.loads(p.read_text());data[arm]=d + r=audit_arm(d);assert q['parity_passed']==r['parity_passed'] and q['exit_code']==(0 if r['parity_passed'] else 1) + log=(A/f'atom270-qwen4b-allcore-abba-{arm}.server0.log').read_text() + def marker(name): + rows=[json.loads(l.split(name,1)[1]) for l in log.splitlines() if name in l];assert len(rows)==1;return rows[0] + binding=marker('ATOM270_BINDING ');bindings.append(binding) + assert marker('ATOM270_COMPILED ')==dict(batch_size=4,kv_blocks=5,context=9216,tensor_parallel_size=1) + hist=marker('ATOM270_ACTIVE_LANES ');assert len(hist)==5 and hist[4]>0 + r.update(sha256=hashlib.sha256(p.read_bytes()).hexdigest(),binding=binding,active_lane_histogram=hist) + result['arms'][arm]=r + assert all(b==bindings[0] for b in bindings) + assert all(d['prompts']==data['a1']['prompts'] for d in data.values()) + result['reference_parity']={f'{a}-{b}':{k:v['tokens']==data[b]['references'][k]['tokens'] for k,v in data[a]['references'].items()} for a,b in [('a1','a2'),('b1','b2'),('a1','b1')]} + for n,o in sorted(CONDITIONS): + values={arm:next(b['summary'] for b in d['batches'] if b['input_tokens']==n) for arm,d in data.items()} + entry={'measurements':values} + for key in ['output_tok_s','median_first_sse_s','median_e2e_s']: + a=statistics.mean(values[x][key] for x in ['a1','a2']);b=statistics.mean(values[x][key] for x in ['b1','b2']) + entry[key]={'original_mean_of_two':a,'allcore_mean_of_two':b,'allcore_relative_change_pct':(b/a-1)*100} + result['comparisons'][f'{n}:{o}']=entry + (A/'atom270-allcore-abba-audit.json').write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(result,indent=2)) +if __name__=='__main__':main() diff --git a/experiments/evidence270/audit_atom270_allcore_http.py b/experiments/evidence270/audit_atom270_allcore_http.py new file mode 100644 index 00000000..232f11d7 --- /dev/null +++ b/experiments/evidence270/audit_atom270_allcore_http.py @@ -0,0 +1,26 @@ +"""Recompute the fixed matrix and compare recorded baseline reference tokens.""" +import hashlib,json +from pathlib import Path +from audit_http265 import audit_result +A=Path(__file__).resolve().parent +p=A/'atom270-qwen4b-allcore-http.json' +d=json.loads(p.read_text()) +r={'audit':audit_result(d,'atom','native',expected_slots=4,expected_atom_devices=1),'sha256':hashlib.sha256(p.read_bytes()).hexdigest(),'comparisons':{},'measurements':[]} +for label in ['b1','b4']: + b=json.loads((A/f'atom270-qwen4b-tp1-{label}-native.json').read_text()) + assert b['prompts']==d['prompts'] + r['comparisons'][label]={k:{'tokens_equal':v['tokens']==b['references'][k]['tokens'],'first_difference':next((i for i,(x,y) in enumerate(zip(v['tokens'],b['references'][k]['tokens'])) if x!=y),None)} for k,v in d['references'].items()} +log=(A/'atom270-qwen4b-allcore-http.server0.log').read_text() +for marker in ['ATOM270_BINDING ','ATOM270_COMPILED ','ATOM270_ACTIVE_LANES ']: + rows=[json.loads(line.split(marker,1)[1]) for line in log.splitlines() if marker in line] + assert rows,marker + r[marker.strip()]=rows +hist=r['ATOM270_ACTIVE_LANES'][-1] +assert len(hist)==5 and hist[4]>0,hist +outer=json.loads((A/'atom270-allcore-http-outer.json').read_text()) +assert outer['state']=='complete' and outer['exit_code']==outer['cleanup_exit_code']==0 +for b in d['batches']: + r['measurements'].append({'input_tokens':b['input_tokens'],'output_tokens_per_request':b['output_tokens_per_request'],**b['summary']}) +r['limits']='One timed batch per condition; historical references are from other cards. No stable performance gain or general semantic-quality claim.' +(A/'atom270-allcore-http-audit.json').write_text(json.dumps(r,indent=2)+'\n') +print(json.dumps(r,indent=2)) diff --git a/experiments/evidence270/audit_atom270_allcore_pd.py b/experiments/evidence270/audit_atom270_allcore_pd.py new file mode 100644 index 00000000..a7259fc6 --- /dev/null +++ b/experiments/evidence270/audit_atom270_allcore_pd.py @@ -0,0 +1,38 @@ +"""Audit transport parity separately from full compiled-batch occupancy.""" +import hashlib,json +from pathlib import Path +from audit_http265 import audit_result +A=Path(__file__).resolve().parent + +def audit_mode(data,log,mode): + result={'responses':audit_result(data,'atom',mode,expected_slots=4,expected_atom_devices=1)} + assert data['kv_blocks']==5 + def one(marker): + rows=[json.loads(l[len(marker):]) for l in log.splitlines() if l.startswith(marker)];assert len(rows)==1,marker + return rows[0] + geometry=one('ATOM270_COMPILED ');assert geometry==dict(batch_size=4,kv_blocks=5,context=9216,tensor_parallel_size=1) + binding=one('ATOM270_BINDING ');mapping=binding['physical_to_visible'] + assert binding['visible_count']==1 and len(mapping)==1 and set(mapping.values())=={0} + assert all(k.isdigit() for k in mapping) and set(binding['opened'])=={f'/dev/rbln{k}' for k in mapping} + hist=one('ATOM270_ACTIVE_LANES ');assert len(hist)==5 and all(type(v) is int and v>=0 for v in hist) and sum(hist)>0 + result.update(geometry=geometry,binding=binding,active_lane_histogram=hist,full_batch_observed=hist[4]>0) + return result + +def main(): + outer=json.loads((A/'atom270-allcore-pd-outer.json').read_text());assert outer['state']=='complete' and outer['exit_code']==outer['cleanup_exit_code']==0 + queue=json.loads((A/'atom270-allcore-pd-arms.json').read_text());assert [x['mode'] for x in queue]==['whole','stream'] and all(x['exit_code']==0 for x in queue) + report={'modes':{},'limits':'Within-route HTTP parity and whole/stream parity; occupancy reported independently. Single timed batches, not a matched native/PD speedup or general quality result.'};data={} + for mode in ['whole','stream']: + p=A/f'atom270-qwen4b-allcore-pd-{mode}.json';d=json.loads(p.read_text());data[mode]=d + log=(A/f'atom270-qwen4b-allcore-pd-{mode}.server0.log').read_text() + report['modes'][mode]=audit_mode(d,log,mode);report['modes'][mode]['sha256']=hashlib.sha256(p.read_bytes()).hexdigest() + assert report['modes']['whole']['binding']==report['modes']['stream']['binding'] + assert data['whole']['prompts']==data['stream']['prompts'] + matches={k:v['tokens']==data['stream']['references'][k]['tokens'] for k,v in data['whole']['references'].items()} + assert all(matches.values());report['whole_stream_references_equal']=matches + native=json.loads((A/'atom270-qwen4b-allcore-http.json').read_text());assert native['prompts']==data['whole']['prompts'] + report['historical_native_reference_matches']={k:v['tokens']==native['references'][k]['tokens'] for k,v in data['whole']['references'].items()} + report['full_batch_observed_both_modes']=all(x['full_batch_observed'] for x in report['modes'].values()) + report['timings']={mode:[{'input':b['input_tokens'],'output_per_request':b['output_tokens_per_request'],'summary':b['summary']} for b in d['batches']] for mode,d in data.items()} + (A/'atom270-allcore-pd-audit.json').write_text(json.dumps(report,indent=2)+'\n');print(json.dumps(report,indent=2)) +if __name__=='__main__':main() diff --git a/experiments/evidence270/audit_atom270_allcore_pd_quality.py b/experiments/evidence270/audit_atom270_allcore_pd_quality.py new file mode 100644 index 00000000..7ff5ff7c --- /dev/null +++ b/experiments/evidence270/audit_atom270_allcore_pd_quality.py @@ -0,0 +1,44 @@ +"""Re-score frozen raw answers; do not conflate completed execution with quality passing.""" +import hashlib,json +from pathlib import Path +from heldout_quality270 import score,compare +A=Path(__file__).resolve().parent + +def audit_rows(rows,fixture): + cases={c['case']:c for c in fixture['cases']};assert len(rows)==len({r['case'] for r in rows})==len(cases)==36 + for r in rows: + c=cases[r['case']] + assert 'error' not in r and r['input_sha256']==c['input_sha256'] and r['expected']==c['expected'] and r['input_tokens']==len(c['ids']) + assert hashlib.sha256(json.dumps(c['ids'],separators=(',',':')).encode()).hexdigest()==c['input_sha256'] + choice=r['response']['choices'][0];assert r['text']==choice['text'] and r['tokens']==choice['token_ids'] and r['finish_reason']==choice['finish_reason'] + assert all(r[k]==v for k,v in score(r['text'],c['expected']).items()) + assert 00 + r.update(active_lanes=hist,full_batch_observed=hist[4]>0,historical_pd_comparison=compare(baseline,rows),native_allcore_comparison=compare(native,rows),raw_sha256=hashlib.sha256(p.read_bytes()).hexdigest()) + result['modes'][mode]=r + assert bindings[0]==bindings[1];result['whole_stream_comparison']=compare(records['whole'],records['stream']) + result['all_answers_and_formats_pass']=all(r['answer_correct']==r['format_correct']==36 for r in result['modes'].values()) + (A/'atom270-allcore-pd-quality-audit.json').write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(result,indent=2)) +if __name__=='__main__':main() diff --git a/experiments/evidence270/audit_atom270_allcore_quality.py b/experiments/evidence270/audit_atom270_allcore_quality.py new file mode 100644 index 00000000..90557dc0 --- /dev/null +++ b/experiments/evidence270/audit_atom270_allcore_quality.py @@ -0,0 +1,25 @@ +"""Independent frozen-fixture, EOS, raw-text scoring and baseline comparison.""" +import hashlib,json +from pathlib import Path +from heldout_quality270 import score,compare +A=Path(__file__).resolve().parent +p=A/'atom270-qwen4b-allcore-quality-retry1.jsonl' +rows=[json.loads(l) for l in p.read_text().splitlines()] +f=json.loads((A/'quality263-qwen4b-fixtures.json').read_text()) +mapped=json.loads((A/'quality270-qwen4b-mapped-fixtures.json').read_text()) +assert f['cases']==mapped['cases'] and mapped['model']=='Qwen/Qwen3-4B' +base=[json.loads(l) for l in (A/'quality263-qwen4b-native-atom.jsonl').read_text().splitlines()] +assert len(rows)==len(f['cases'])==36 and len({r['case'] for r in rows})==36 +for r,c in zip(rows,f['cases']): + assert r['case']==c['case'] and r['input_sha256']==c['input_sha256'] and r['expected']==c['expected'] and 'error' not in r + assert all(r[k]==v for k,v in score(r['text'],c['expected']).items()) + assert r['finish_reason']=='stop' and r['tokens'][-1]==151645 +log=(A/'atom270-qwen4b-allcore-quality-retry1.server0.log').read_text() +def value(marker): + lines=[l.split(marker,1)[1] for l in log.splitlines() if marker in l];assert len(lines)==1 + return json.loads(lines[0]) +assert value('ATOM270_QUALITY_EOS ')==[151643,151645] +outer=json.loads((A/'atom270-allcore-quality-retry1-outer.json').read_text()) +assert outer['state']=='complete' and outer['exit_code']==outer['cleanup_exit_code']==0 +r={'cases':36,'answer_correct':sum(x['answer_correct'] for x in rows),'format_correct':sum(x['format_correct'] for x in rows),'baseline_comparison':compare(base,rows),'eos_stopped':36,'active_lanes':value('ATOM270_ACTIVE_LANES '),'geometry':value('ATOM270_COMPILED '),'raw_sha256':hashlib.sha256(p.read_bytes()).hexdigest(),'cleanup_exit_code':0,'scope':'Sequential fixed36cases, short/2K/8K; no concurrent B4 quality, general quality or speedup claim.'} +(A/'atom270-allcore-quality-audit.json').write_text(json.dumps(r,indent=2)+'\n');print(json.dumps(r,indent=2)) diff --git a/experiments/evidence270/audit_atom270_allcore_quality_c12.py b/experiments/evidence270/audit_atom270_allcore_quality_c12.py new file mode 100644 index 00000000..9e160ba5 --- /dev/null +++ b/experiments/evidence270/audit_atom270_allcore_quality_c12.py @@ -0,0 +1,31 @@ +"""Independent frozen-fixture, EOS, raw-text scoring and baseline comparison.""" +import hashlib,json +from pathlib import Path +from heldout_quality270 import score,compare +A=Path(__file__).resolve().parent +p=A/'atom270-qwen4b-allcore-quality-c12.jsonl' +rows=[json.loads(l) for l in p.read_text().splitlines()] +f=json.loads((A/'quality263-qwen4b-fixtures.json').read_text()) +mapped=json.loads((A/'quality270-qwen4b-mapped-fixtures.json').read_text()) +assert f['cases']==mapped['cases'] and mapped['model']=='Qwen/Qwen3-4B' +base=[json.loads(l) for l in (A/'quality263-qwen4b-native-atom.jsonl').read_text().splitlines()] +assert len(rows)==len(f['cases'])==36 and len({r['case'] for r in rows})==36 +by_case={r['case']:r for r in rows} +for c in f['cases']: + r=by_case[c['case']] + assert r['case']==c['case'] and r['input_sha256']==c['input_sha256'] and r['expected']==c['expected'] and 'error' not in r + assert all(r[k]==v for k,v in score(r['text'],c['expected']).items()) + assert r['finish_reason']=='stop' and r['tokens'][-1]==151645 +log=(A/'atom270-qwen4b-allcore-quality-c12.server0.log').read_text() +def value(marker): + lines=[l.split(marker,1)[1] for l in log.splitlines() if marker in l];assert len(lines)==1 + return json.loads(lines[0]) +assert value('ATOM270_QUALITY_EOS ')==[151643,151645] +outer=json.loads((A/'atom270-allcore-quality-c12-outer.json').read_text()) +assert outer['state']=='complete' and outer['exit_code']==outer['cleanup_exit_code']==0 +hist=value('ATOM270_ACTIVE_LANES ');assert len(hist)==5 and hist[4]>0, 'Insufficient actual B4 occupancy' +for n in (0,2048,8192): + cohort=[r for r in rows if r['cohort']==n];assert len(cohort)==12 + assert max(r['request_start_monotonic'] for r in cohort) 0, 'No actual victim import before fault' + assert t['slot'] == 3 and not t['admitted'] + assert len(t['active_tokens']) == 3 and all(0 < n < 512 for n in t['active_tokens']) + assert row['survivor_tokens'] == control['tokens'] + assert row['survivors_exact'] and row['idle'] and not row['poisoned'] + assert cancel['replacement_tokens'] == control['replacement_tokens'] and cancel['replacement_exact'] + assert death['victim_error'] + return dict(model=d['model'], raw_file=path.name, raw_sha256=hashlib.sha256(raw).hexdigest(), + runtime_sha256=d['source_sha256'], survivor_token_comparisons=3072, + replacement_token_comparisons=16, + cases=[dict(case=r['case'], trigger=r['trigger'], histogram=r['histogram']) for r in [cancel, death]]) + + +if __name__ == '__main__': + print(json.dumps([audit(p) for p in sys.argv[1:]], indent=2)) diff --git a/experiments/evidence270/audit_four_model_current270.py b/experiments/evidence270/audit_four_model_current270.py new file mode 100644 index 00000000..72c083ec --- /dev/null +++ b/experiments/evidence270/audit_four_model_current270.py @@ -0,0 +1,41 @@ +"""Re-audit raw serving/quality evidence; keep missing and failed routes visible.""" +import hashlib,json +from pathlib import Path +from audit_http265 import audit_result +from heldout_quality270 import score +A=Path(__file__).resolve().parent +result={'http':{},'route_reference_matches':{},'quality':{},'limits':'Historical task runtimes and isolated artifacts, not current-main acceptance. Same-route parity is not cross-engine parity; timings are not a matched speedup comparison. Missing routes remain missing.'} +for model,label,tp in [('qwen06b','qwen06b',1),('qwen4b','qwen4b',1),('koni','koni',2),('gemma','causal-gemma',1)]: + for backend,batch in [('gpu',4),('atom',1),('atom',4)]: + records={} + for mode in ('native','whole','stream'): + key=f'{model}/{backend}/b{batch}/{mode}' + name=f'stream265-{model}-http-c16-{mode}.json' if backend=='gpu' else f'atom270-{label}-tp{tp}-b{batch}-{mode}.json' + if (model,backend,batch,mode)==('gemma','atom',4,'stream'):name='atom270-timeline-gemma-b4-stream.json' + p=A/name + if not p.exists():result['http'][key]={'state':'missing','file':name};continue + d=json.loads(p.read_text());entry={'file':name,'sha256':hashlib.sha256(p.read_bytes()).hexdigest()};records[mode]=d + try:entry.update(state='passed',audit=audit_result(d,backend,mode,expected_slots=batch,expected_atom_devices=tp if backend=='atom' else 0)) + except (AssertionError,KeyError,ValueError) as e:entry.update(state='failed',reason=str(e),runtime_complete=d.get('passed'),parity_passed=d.get('parity_passed')) + entry['failed_token_checks']=sum(not r['tokens_equal'] for b in d.get('correctness_batches',[]) for r in b['rows']) + if name=='atom270-timeline-gemma-b4-stream.json':entry['limits']='Instrumented causal-export run; HTTP parity passes but max active lanes3, so full B4 occupancy remains unverified.' + result['http'][key]=entry + for a,b in [('native','whole'),('whole','stream')]: + if a not in records or b not in records:continue + x,y=records[a],records[b];assert x['prompts']==y['prompts'] + matches={k:v['tokens']==y['references'][k]['tokens'] for k,v in x['references'].items()} + result['route_reference_matches'][f'{model}/{backend}/b{batch}/{a}-{b}']=matches + fixture=json.loads((A/f'quality263-{model}-fixtures.json').read_text());cases={c['case']:c for c in fixture['cases']};assert len(cases)==36 + for route in ['native-gpu','rngd-gpu','native-atom','rngd-atom']: + p=A/f'quality263-{model}-{route}.jsonl';rows=[json.loads(l) for l in p.read_text().splitlines()] + assert len(rows)==len({r['case'] for r in rows})==36 + for r in rows: + c=cases[r['case']];assert r['input_sha256']==c['input_sha256'] and 'error' not in r + assert all(r[k]==v for k,v in score(r['text'],c['expected']).items()) + result['quality'][f'{model}/{route}']={'file':p.name,'sha256':hashlib.sha256(p.read_bytes()).hexdigest(),'cases':36,'answer_correct':sum(r['answer_correct'] for r in rows),'format_correct':sum(r['format_correct'] for r in rows)} +(A/'four-model-current270-audit.json').write_text(json.dumps(result,indent=2)+'\n') +from collections import Counter +print('HTTP',dict(Counter(r['state'] for r in result['http'].values()))) +for k,v in result['http'].items(): + if v['state']!='passed':print(k,v) +for k,v in result['quality'].items():print(k,v['answer_correct'],v['format_correct']) diff --git a/experiments/evidence270/audit_http265.py b/experiments/evidence270/audit_http265.py new file mode 100644 index 00000000..9f9a6a8d --- /dev/null +++ b/experiments/evidence270/audit_http265.py @@ -0,0 +1,78 @@ +"""Audit HTTPc16 evidence. Token checks and timed SSE text checks are distinct.""" +import json,math,statistics,sys +from pathlib import Path +CONDITIONS={(n,o) for n in (128,512,2048,8192) for o in (32,128)} +MODELS=('qwen06b','qwen4b','gemma','koni') + +def audit_result(d,backend,mode,*,lengths=(128,512,2048,8192),expected_slots=None,expected_atom_devices=None): + assert lengths and len(set(lengths))==len(lengths) and all(type(n) is int and n>0 for n in lengths) + conditions={(n,o) for n in lengths for o in (32,128)} + count=len(conditions) + assert d['passed'] and d['parity_passed'], 'runtime or parity failure' + slots=(1 if backend=='atom' else 4) if expected_slots is None else expected_slots + assert type(slots) is int and slots>0 + assert d['slots_per_decoder']==slots + atom_devices=int(backend=='atom') if expected_atom_devices is None else expected_atom_devices + assert type(atom_devices) is int and atom_devices>=0 + assert d['physical_atom']==atom_devices and d['physical_gpus']==int(backend=='gpu') + assert d['physical_rngd']==int(mode!='native') and d['context']==9216 + assert not d['decoder_cache'] and not d['producer_cache'] + assert set(d['prompts'])=={str(n) for n in lengths} + assert all(len(v)==int(k) for k,v in d['prompts'].items()) + assert set(d['references'])=={f'{n}:{o}' for n,o in conditions} + assert len(d['warmups'])==len(d['correctness_batches'])==len(d['batches'])==count + warm=set() + for r in d['warmups']: + _,endpoint,n,o=r['id'].split('-');n,o=int(n),int(o);warm.add((n,o));ref=d['references'][f'{n}:{o}'] + assert endpoint=='0' and r['tokens_equal'] and r['tokens']==ref['tokens'] and len(r['tokens'])==o + assert warm==conditions + assert {(b['input_tokens'],b['output_tokens']) for b in d['correctness_batches']}==conditions + for b in d['correctness_batches']: + n,o=b['input_tokens'],b['output_tokens'];ref=d['references'][f'{n}:{o}'] + assert b['concurrency']==16 and len(b['rows'])==16 and len({r['id'] for r in b['rows']})==16 + assert all(r['tokens_equal'] and r['tokens']==ref['tokens'] and len(r['tokens'])==o for r in b['rows']) + assert {(b['input_tokens'],b['output_tokens_per_request']) for b in d['batches']}==conditions + for b in d['batches']: + n,o=b['input_tokens'],b['output_tokens_per_request'];ref=d['references'][f'{n}:{o}'];rows=b['rows'];s=b['summary'] + assert b['concurrency']==16 and b['repeat']==0 and len(rows)==16 and len({r['id'] for r in rows})==16 + for r in rows: + assert r['input_tokens']==n and r['output_tokens']==o and r['text_equal'] and r['text']==ref['text'] + assert all(math.isfinite(r[k]) for k in ['first_sse_s','e2e_s','tpot_est_s']) + assert 0=0 + assert s['requests']==16 and s['output_tokens']==16*o and s['all_text_equal'] + assert math.isfinite(s['wall_s']) and s['wall_s']>0 + expected={'output_tok_s':16*o/s['wall_s'],'median_first_sse_s':statistics.median(r['first_sse_s'] for r in rows),'median_e2e_s':statistics.median(r['e2e_s'] for r in rows),'median_tpot_est_s':statistics.median(r['tpot_est_s'] for r in rows)} + assert all(math.isclose(s[k],v,rel_tol=1e-10,abs_tol=1e-10) for k,v in expected.items()) + return dict(stage='passed',conditions=count,warmups=count,correctness_responses=count*16,timed_text_responses=count*16) + +def main(): + root=Path(sys.argv[1]) if len(sys.argv)>1 else Path(__file__).resolve().parent + report={'scope':'HTTPc16 smoke, one measured batch per condition. Nonstreamed correctness retains token arrays; timed SSE retains text/counts. No statistical speedup claim.','cases':{},'route_reference_comparisons':{}} + failed=False;pending=False + for backend in ('gpu','atom'): + queuepath=root/('stream265-http-atom-c16-queue.json' if backend=='atom' else 'stream265-http-c16-queue.json') + queue=json.loads(queuepath.read_text()) if queuepath.exists() else [] + for model in MODELS: + refs={} + for mode in ('native','whole','stream'): + stem=f'stream265-{model}-http-'+('atom-' if backend=='atom' else '')+f'c16-{mode}';p=root/(stem+'.json');key=f'{backend}/{model}/{mode}' + if not p.exists():report['cases'][key]={'stage':'not_collected'};pending=True;continue + try: + d=json.loads(p.read_text());result=audit_result(d,backend,mode);refs[mode]=d + q=[r for r in queue if r['model']==model and r['mode']==mode] + if not q:result['stage']='awaiting_child_exit';pending=True + else: + assert len(q)==1 and q[0]['exit_code']==0 and all(x==0 for x in q[0].get('copy_exit_codes',[])) + report['cases'][key]=result + except (AssertionError,ValueError,KeyError) as e:report['cases'][key]={'stage':'failed','error':str(e) or type(e).__name__};failed=True + for a,b in [('native','whole'),('whole','stream')]: + if a in refs and b in refs: + assert refs[a]['prompts']==refs[b]['prompts'] + matches={k:refs[a]['references'][k]['tokens']==refs[b]['references'][k]['tokens'] for k in refs[a]['references']} + report['route_reference_comparisons'][f'{backend}/{model}/{a}-vs-{b}']=matches + if a=='whole' and not all(matches.values()):failed=True + (root/'stream265-http-c16-audit.json').write_text(json.dumps(report,indent=2)+'\n') + print(json.dumps({k:v for k,v in report['cases'].items() if v['stage']!='not_collected'},indent=2)) + raise SystemExit(1 if failed else 2 if pending else 0) + +if __name__=='__main__':main() diff --git a/experiments/evidence270/build_atom270_allcoreprobe.py b/experiments/evidence270/build_atom270_allcoreprobe.py new file mode 100644 index 00000000..0ab36938 --- /dev/null +++ b/experiments/evidence270/build_atom270_allcoreprobe.py @@ -0,0 +1,15 @@ +"""Compile an isolated diagnostic artifact using SDK output_hidden_states support.""" +from pathlib import Path +import json,time +import optimum.rbln as rbln +root=Path(__file__).resolve().parent +artifact=root/'atom270-artifacts-allcoreprobe'/'Qwen--Qwen3-4B-tp1-s9216-b4-k5' +assert not artifact.exists(), 'Preserve existing diagnostic artifacts' +config=dict(batch_size=4,max_seq_len=9216,tensor_parallel_size=1,kvcache_num_blocks=5,kvcache_block_size=9216,output_hidden_states=True,create_runtimes=False) +from atom270_allcore_probe import CoreProbeWrapper +rbln.RBLNQwen3ForCausalLM._decoder_wrapper_cls=CoreProbeWrapper +start=time.perf_counter() +print('HIDDEN_COMPILE_START '+json.dumps(config),flush=True) +model=rbln.RBLNQwen3ForCausalLM.from_pretrained('Qwen/Qwen3-4B',export=True,rbln_config=config) +model.save_pretrained(artifact) +print('HIDDEN_COMPILE_COMPLETE '+json.dumps(dict(seconds=time.perf_counter()-start,config=config)),flush=True) diff --git a/experiments/evidence270/build_atom270_attentionraw.py b/experiments/evidence270/build_atom270_attentionraw.py new file mode 100644 index 00000000..d7a1d8b5 --- /dev/null +++ b/experiments/evidence270/build_atom270_attentionraw.py @@ -0,0 +1,15 @@ +"""Compile an isolated diagnostic artifact using SDK output_hidden_states support.""" +from pathlib import Path +import json,time +import optimum.rbln as rbln +root=Path(__file__).resolve().parent +artifact=root/'atom270-artifacts-attentionraw'/'Qwen--Qwen3-4B-tp1-s9216-b4-k5' +assert not artifact.exists(), 'Preserve existing diagnostic artifacts' +config=dict(batch_size=4,max_seq_len=9216,tensor_parallel_size=1,kvcache_num_blocks=5,kvcache_block_size=9216,output_hidden_states=True,create_runtimes=False) +from atom270_attention_raw_probe import AttentionProbeWrapper +rbln.RBLNQwen3ForCausalLM._decoder_wrapper_cls=AttentionProbeWrapper +start=time.perf_counter() +print('HIDDEN_COMPILE_START '+json.dumps(config),flush=True) +model=rbln.RBLNQwen3ForCausalLM.from_pretrained('Qwen/Qwen3-4B',export=True,rbln_config=config) +model.save_pretrained(artifact) +print('HIDDEN_COMPILE_COMPLETE '+json.dumps(dict(seconds=time.perf_counter()-start,config=config)),flush=True) diff --git a/experiments/evidence270/build_atom270_hidden.py b/experiments/evidence270/build_atom270_hidden.py new file mode 100644 index 00000000..11d5207c --- /dev/null +++ b/experiments/evidence270/build_atom270_hidden.py @@ -0,0 +1,13 @@ +"""Compile an isolated diagnostic artifact using SDK output_hidden_states support.""" +from pathlib import Path +import json,time +import optimum.rbln as rbln +root=Path(__file__).resolve().parent +artifact=root/'atom270-artifacts-hidden'/'Qwen--Qwen3-4B-tp1-s9216-b4-k5' +assert not artifact.exists(), 'Preserve existing diagnostic artifacts' +config=dict(batch_size=4,max_seq_len=9216,tensor_parallel_size=1,kvcache_num_blocks=5,kvcache_block_size=9216,output_hidden_states=True,create_runtimes=False) +start=time.perf_counter() +print('HIDDEN_COMPILE_START '+json.dumps(config),flush=True) +model=rbln.RBLNQwen3ForCausalLM.from_pretrained('Qwen/Qwen3-4B',export=True,rbln_config=config) +model.save_pretrained(artifact) +print('HIDDEN_COMPILE_COMPLETE '+json.dumps(dict(seconds=time.perf_counter()-start,config=config)),flush=True) diff --git a/experiments/evidence270/build_atom270_layerprobe.py b/experiments/evidence270/build_atom270_layerprobe.py new file mode 100644 index 00000000..30635c0e --- /dev/null +++ b/experiments/evidence270/build_atom270_layerprobe.py @@ -0,0 +1,15 @@ +"""Compile an isolated diagnostic artifact using SDK output_hidden_states support.""" +from pathlib import Path +import json,time +import optimum.rbln as rbln +root=Path(__file__).resolve().parent +artifact=root/'atom270-artifacts-layerprobe'/'Qwen--Qwen3-4B-tp1-s9216-b4-k5' +assert not artifact.exists(), 'Preserve existing diagnostic artifacts' +config=dict(batch_size=4,max_seq_len=9216,tensor_parallel_size=1,kvcache_num_blocks=5,kvcache_block_size=9216,output_hidden_states=True,create_runtimes=False) +from atom270_layer_probe import LayerProbeWrapper +rbln.RBLNQwen3ForCausalLM._decoder_wrapper_cls=LayerProbeWrapper +start=time.perf_counter() +print('HIDDEN_COMPILE_START '+json.dumps(config),flush=True) +model=rbln.RBLNQwen3ForCausalLM.from_pretrained('Qwen/Qwen3-4B',export=True,rbln_config=config) +model.save_pretrained(artifact) +print('HIDDEN_COMPILE_COMPLETE '+json.dumps(dict(seconds=time.perf_counter()-start,config=config)),flush=True) diff --git a/experiments/evidence270/build_gemma270_causal.py b/experiments/evidence270/build_gemma270_causal.py new file mode 100644 index 00000000..6c5ef4b5 --- /dev/null +++ b/experiments/evidence270/build_gemma270_causal.py @@ -0,0 +1,25 @@ +"""Controlled Gemma B1/k5 re-export with the previously validated causal correction.""" +import hashlib +import json +from pathlib import Path +import gemma_global_causal_compat +from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + +cache = Path('atom270-artifacts-causal') +artifact = cache / 'unsloth--gemma-3-1b-it-tp1-s9216-b1-k5' +assert not artifact.exists(), 'Preserve existing artifacts; this is a fresh controlled export' +nodes = list(Path('/dev').glob('rbln[0-9]*')) +assert len(nodes) == 1 +engine = AtomRblnEngine(devices=[int(nodes[0].name[4:])], max_seq_len=9216, + batch_size=1, kv_blocks=5, cache_dir=str(cache)) +try: + engine.start('unsloth/gemma-3-1b-it') + assert engine.compiled is True + provenance = dict(model='unsloth/gemma-3-1b-it', batch=1, kv_blocks=5, context=9216, + correction='text prefill causal; image prefill retains bidirectionality', + patch_sha256={name: hashlib.sha256(Path(name).read_bytes()).hexdigest() + for name in ['gemma_causal_compat.py', 'gemma_global_causal_compat.py']}) + (artifact / 'causal_export_provenance.json').write_text(json.dumps(provenance, indent=2)+'\n') + print('CONTROLLED_EXPORT '+json.dumps(provenance), flush=True) +finally: + engine.close() diff --git a/experiments/evidence270/build_gemma270_causal_b4.py b/experiments/evidence270/build_gemma270_causal_b4.py new file mode 100644 index 00000000..9c20de60 --- /dev/null +++ b/experiments/evidence270/build_gemma270_causal_b4.py @@ -0,0 +1,25 @@ +"""Controlled Gemma B4/k5 re-export with the previously validated causal correction.""" +import hashlib +import json +from pathlib import Path +import gemma_global_causal_compat +from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + +cache = Path('atom270-artifacts-causal') +artifact = cache / 'unsloth--gemma-3-1b-it-tp1-s9216-b4-k5' +assert not artifact.exists(), 'Preserve existing artifacts; this is a fresh controlled export' +nodes = list(Path('/dev').glob('rbln[0-9]*')) +assert len(nodes) == 1 +engine = AtomRblnEngine(devices=[int(nodes[0].name[4:])], max_seq_len=9216, + batch_size=4, kv_blocks=5, cache_dir=str(cache)) +try: + engine.start('unsloth/gemma-3-1b-it') + assert engine.compiled is True + provenance = dict(model='unsloth/gemma-3-1b-it', batch=4, kv_blocks=5, context=9216, + correction='text prefill causal; image prefill retains bidirectionality', + patch_sha256={name: hashlib.sha256(Path(name).read_bytes()).hexdigest() + for name in ['gemma_causal_compat.py', 'gemma_global_causal_compat.py']}) + (artifact / 'causal_export_provenance.json').write_text(json.dumps(provenance, indent=2)+'\n') + print('CONTROLLED_EXPORT '+json.dumps(provenance), flush=True) +finally: + engine.close() diff --git a/experiments/evidence270/check_atom270_b4_faults.py b/experiments/evidence270/check_atom270_b4_faults.py new file mode 100644 index 00000000..723a5ee1 --- /dev/null +++ b/experiments/evidence270/check_atom270_b4_faults.py @@ -0,0 +1,125 @@ +"""Task-owned hardware test: three active survivors, fourth-lane cancellation/reuse/death. + +Run through container_bootstrap270.py using the experiment's existing environment. +Only the producer subprocess created here may be killed. No device regrouping. +""" +import argparse +import hashlib +import json +import os +from pathlib import Path +import time + + +def main(): + parser=argparse.ArgumentParser(__doc__) + for name in ['model','family','cache-dir','output']:parser.add_argument('--'+name,required=True) + parser.add_argument('--worker-python',default='./furiosa_prefill_python.sh') + parser.add_argument('--tensor-parallel-size',type=int,choices=[1,2],default=1) + args=parser.parse_args();output=Path(args.output) + if output.exists():raise RuntimeError('Refuse output overwrite') + from transformers import AutoTokenizer, AutoConfig + from opt_prime.accel.pd import PDRequest + from opt_prime.accel.pd.runtime import RequestCancelled + from opt_prime.accel.pd.pd_batching import SubprocessPrefillWorker + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import check_shm_capacity,validate_compiled_geometry,observe_decode_batches + c=AutoConfig.from_pretrained(args.model,local_files_only=True).get_text_config() + check_shm_capacity(c,4,8192) + nodes=list(Path('/dev').glob('rbln[0-9]*'));assert len(nodes)==args.tensor_parallel_size + engine=AtomRblnEngine(devices=sorted(int(n.name[4:]) for n in nodes),max_seq_len=9216,batch_size=4,kv_blocks=5,cache_dir=args.cache_dir) + artifact=Path(engine._dir(args.model)) + assert (artifact/'config.json').exists(),'Load-only test: existing compiled artifact required' + if args.family=='gemma': + provenance=json.loads((artifact/'causal_export_provenance.json').read_text()) + assert provenance['model']==args.model and provenance['batch']==4 + result=dict(model=args.model,batch=4,kv_blocks=5,width=8192,stage='starting',cases=[], + scope='Direct batcher hardware fault test, not HTTP, throughput or semantic quality. Three survivors use512 output tokens; victim/replacement16.') + result['source_sha256']={name:hashlib.sha256((Path('stream265-code/opt_prime')/name).read_bytes()).hexdigest() for name in ['accel/platforms/atom/batching.py','accel/pd/pd_batching.py','accel/pd/prefill_server.py']} + worker=None + def save():output.write_text(json.dumps(result,indent=2)+'\n') + def step_until(batch,condition,timeout=600): + deadline=time.monotonic()+timeout + while not condition(): + assert time.monotonic() 0 + if outcome=='cancel-reuse': + assert batch.cancel('victim');batch.step() + try:victim.result(timeout=0);raise AssertionError('Cancellation succeeded unexpectedly') + except RequestCancelled:pass + replacement=batch.submit(PDRequest('reuse',short,16));batch.step() + assert batch.by_id['reuse'].slot==trigger['slot'],'Replacement did not reuse cancelled lane' + else: + assert worker._proc.poll() is None + worker._proc.kill() + step_until(batch,lambda:batch.idle) + actual=[f.result() for f in alive] + assert actual==reference,'Survivor output changed' + assert not batch.poisoned and sorted(batch.free)==[0,1,2,3] and not batch.by_id + row=dict(case=outcome,trigger=trigger,histogram=hist,survivor_tokens=actual,survivors_exact=True,idle=True,poisoned=False) + if outcome=='cancel-reuse': + assert replacement.result()==replacement_reference,'Reused lane output changed' + row.update(replacement_tokens=replacement.result(),replacement_exact=True) + else: + assert victim.exception() is not None and worker._proc.wait(timeout=10)==-9 and not worker.ready + row['victim_error']=repr(victim.exception()) + result['cases'].append(row);save() + result['stage']='complete' + except BaseException as error: + result.update(stage='failed',error=repr(error));raise + finally: + if worker is not None:worker.close() + engine.close();save() + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/check_gemma270_causality.py b/experiments/evidence270/check_gemma270_causality.py new file mode 100644 index 00000000..7521e1e9 --- /dev/null +++ b/experiments/evidence270/check_gemma270_causality.py @@ -0,0 +1,106 @@ +"""Test native Gemma prefix causality by changing only the prompt suffix.""" + +import argparse +import json +from pathlib import Path + +import numpy as np +import torch +from opt_prime.accel.platforms.atom.engine import AtomRblnEngine +from rebel.kv_cache import aligned_tensor + +p = argparse.ArgumentParser(description=__doc__) +p.add_argument("--cache-dir", required=True) +p.add_argument("--exports", nargs="+", required=True) +p.add_argument("--output", required=True) +p.add_argument("--kv-blocks", type=int, choices=(3, 5), required=True) +a = p.parse_args() +artifact = Path(a.cache_dir) / f"unsloth--gemma-3-1b-it-tp1-s9216-b1-k{a.kv_blocks}" +assert (artifact / "config.json").is_file(), "Existing compiled artifact required; never compile in this diagnostic" +metadata = json.loads((artifact / "rbln_config.json").read_text()) +assert (metadata["batch_size"], metadata["kvcache_num_blocks"], metadata["max_seq_len"]) == (1, a.kv_blocks, 9216) +assert not Path(a.output).exists(), "Preserve existing evidence" +devices = list(Path("/dev").glob("rbln[0-9]*")) +assert len(devices) == 1 +engine = AtomRblnEngine( + devices=[int(devices[0].name[4:])], + max_seq_len=9216, + batch_size=1, + kv_blocks=a.kv_blocks, + cache_dir=a.cache_dir, +) +rows = [] +try: + engine.start("unsloth/gemma-3-1b-it") + assert engine.compiled is False, "Diagnostic must load existing graph" + b = engine.batcher(prefix_cache=False) + model = b.model + decoder = b.decoder + rc = model.rbln_config + assert rc.cache_impl == "hybrid" and rc.batch_size == 1 and rc.sliding_window == 512 + metas = rc.kvcache_metas + assert all(isinstance(m, dict) and m["dtype"] == "float32" for m in metas) + assert len(metas) == 52 + runtime = model.model[1] + buffers = [] + for meta in metas: + count = int(np.prod(meta["shape"])) + size = runtime.get_kv_cache_size_for_layer(meta["name"]) + assert size == count * 2, (meta["name"], size, count) + buffers.append( + aligned_tensor( + count, dtype=np.uint16, alignment=2 << 20, tensor_type="np" + ).reshape(meta["shape"]) + ) + global_table = torch.zeros((1, 1), dtype=torch.int16) + local_table = torch.zeros((1, 1), dtype=torch.int16) + + with torch.inference_mode(): + for path in a.exports: + r = torch.load(path, weights_only=True) + ids = r["ids"] + n = len(ids) + assert n == 128 + cutoff = n // 2 + changed = ids[:cutoff] + [ + (x + 1) % model.config.vocab_size for x in ids[cutoff:] + ] + states = [] + for prompt in [ids, changed]: + model.prefill_decoder( + input_ids=torch.tensor([prompt]), + cache_position=torch.arange(n, dtype=torch.int32)[None], + batch_idx=0, + block_tables=global_table[0], + local_block_tables=local_table[0], + ) + saved = [] + for meta, buf in zip(metas, buffers): + runtime.get_kv_cache_for_layer(meta["name"], 0, 0, n, buf) + saved.append(buf[0, :, :cutoff].copy()) + states.append(saved) + changed_slots = [] + for i, (x, y) in enumerate(zip(*states)): + changed_slots.append( + { + "slot": i, + "layer": i // 2, + "equal": bool(np.array_equal(x, y)), + "changed_fraction": float(np.mean(x != y)), + } + ) + row = { + "artifact": artifact.name, + "kv_blocks": a.kv_blocks, + "input_tokens": n, + "unchanged_prefix": cutoff, + "slots": changed_slots, + "prefix_exact": all(s["equal"] for s in changed_slots), + } + rows.append(row) + print(json.dumps(row), flush=True) +finally: + engine.close() +with Path(a.output).open("x") as output: + output.write(json.dumps(rows, indent=2) + "\n") +assert all(row["prefix_exact"] for row in rows), "Causality violation: changing suffix changed prefix KV" diff --git a/experiments/evidence270/check_gemma270_causality_b4.py b/experiments/evidence270/check_gemma270_causality_b4.py new file mode 100644 index 00000000..c1c1fe88 --- /dev/null +++ b/experiments/evidence270/check_gemma270_causality_b4.py @@ -0,0 +1,120 @@ +"""Test native Gemma prefix causality by changing only the prompt suffix.""" + +import argparse +from itertools import product +import json +from pathlib import Path + +import numpy as np +import torch +from opt_prime.accel.platforms.atom.engine import AtomRblnEngine +from rebel.kv_cache import aligned_tensor + +p = argparse.ArgumentParser(description=__doc__) +p.add_argument("--cache-dir", required=True) +p.add_argument("--exports", nargs="+", required=True) +p.add_argument("--output", required=True) +p.add_argument("--kv-blocks", type=int, choices=(5,), required=True) +a = p.parse_args() +artifact = Path(a.cache_dir) / f"unsloth--gemma-3-1b-it-tp1-s9216-b4-k{a.kv_blocks}" +assert (artifact / "config.json").is_file(), "Existing compiled artifact required; never compile in this diagnostic" +metadata = json.loads((artifact / "rbln_config.json").read_text()) +assert (metadata["batch_size"], metadata["kvcache_num_blocks"], metadata["max_seq_len"]) == (4, a.kv_blocks, 9216) +assert not Path(a.output).exists(), "Preserve existing evidence" +devices = list(Path("/dev").glob("rbln[0-9]*")) +assert len(devices) == 1 +engine = AtomRblnEngine( + devices=[int(devices[0].name[4:])], + max_seq_len=9216, + batch_size=4, + kv_blocks=a.kv_blocks, + cache_dir=a.cache_dir, +) +rows = [] +try: + engine.start("unsloth/gemma-3-1b-it") + assert engine.compiled is False, "Diagnostic must load existing graph" + b = engine.batcher(prefix_cache=False) + model = b.model + decoder = b.decoder + rc = model.rbln_config + assert rc.cache_impl == "hybrid" and rc.batch_size == 4 and rc.sliding_window == 512 + metas = rc.kvcache_metas + assert all(isinstance(m, dict) and m["dtype"] == "float32" for m in metas) + assert len(metas) == 52 + runtime = model.model[1] + buffers = [] + for meta in metas: + count = int(np.prod(meta["shape"])) + size = runtime.get_kv_cache_size_for_layer(meta["name"]) + assert size == count * 2, (meta["name"], size, count) + buffers.append( + aligned_tensor( + count, dtype=np.uint16, alignment=2 << 20, tensor_type="np" + ).reshape(meta["shape"]) + ) + global_table = torch.zeros((1, 1), dtype=torch.int16) + local_table = torch.zeros((1, 1), dtype=torch.int16) + + with torch.inference_mode(): + for lane, path in product(range(4), a.exports): + global_table.fill_(lane) + local_table.fill_(lane) + r = torch.load(path, weights_only=True) + ids = r["ids"] + n = len(ids) + assert n == 128 + cutoff = n // 2 + changed = ids[:cutoff] + [ + (x + 1) % model.config.vocab_size for x in ids[cutoff:] + ] + # A prefix-changing control detects wrong-lane or stale/zero-buffer reads. + prefix_changed = [(x + 1) % model.config.vocab_size for x in ids[:cutoff]] + ids[cutoff:] + assert prefix_changed[:cutoff] != ids[:cutoff] and prefix_changed[cutoff:] == ids[cutoff:] + states = [] + for prompt in [ids, changed, prefix_changed]: + model.prefill_decoder( + input_ids=torch.tensor([prompt]), + cache_position=torch.arange(n, dtype=torch.int32)[None], + batch_idx=lane, + block_tables=global_table[0], + local_block_tables=local_table[0], + ) + saved = [] + for meta, buf in zip(metas, buffers): + runtime.get_kv_cache_for_layer(meta["name"], lane, 0, n, buf) + saved.append(buf[lane, :, :cutoff].copy()) + states.append(saved) + changed_slots = [] + for i, (x, y) in enumerate(zip(states[0], states[1])): + changed_slots.append( + { + "slot": i, + "layer": i // 2, + "equal": bool(np.array_equal(x, y)), + "changed_fraction": float(np.mean(x != y)), + } + ) + control_changes = [not np.array_equal(x, y) for x, y in zip(states[0], states[2])] + assert len(control_changes) == 52 + row = { + "artifact": artifact.name, + "lane": lane, + "kv_blocks": a.kv_blocks, + "input_tokens": n, + "unchanged_prefix": cutoff, + "slots": changed_slots, + "prefix_exact": all(s["equal"] for s in changed_slots), + "prefix_control_changed_slots": [i for i, changed in enumerate(control_changes) if changed], + "prefix_control_passed": all(control_changes), + } + rows.append(row) + print(json.dumps(row), flush=True) +finally: + engine.close() +with Path(a.output).open("x") as output: + output.write(json.dumps(rows, indent=2) + "\n") +assert all(row["prefix_exact"] for row in rows), "Causality violation: changing suffix changed prefix KV" + +assert len(rows) == 4 * len(a.exports) and {row["lane"] for row in rows} == set(range(4)) +assert all(row["prefix_control_passed"] for row in rows), "Prefix control failed: validate lane/cache readback before interpreting causality" diff --git a/experiments/evidence270/container_bootstrap270.py b/experiments/evidence270/container_bootstrap270.py new file mode 100644 index 00000000..389f4c96 --- /dev/null +++ b/experiments/evidence270/container_bootstrap270.py @@ -0,0 +1,67 @@ +"""Use only assigned ATOM nodes and their existing RSD mapping; never regroup devices.""" +import csv +import io +import json +import os +from pathlib import Path +import re +import runpy +import sys + + +def assigned_mapping(nodes, group_text): + names=[Path(n).name for n in nodes] + if len(names) not in (1,2) or len(set(names))!=len(names) or any(not re.fullmatch(r'rbln[0-9]+',n) for n in names): + raise ValueError('Expected one or two unique assigned ATOM nodes') + rows=[r for r in csv.DictReader(io.StringIO(group_text)) if r['DEVICE'] in names] + if len(rows)!=len(names) or {r['DEVICE'] for r in rows}!=set(names): + raise ValueError('Missing or duplicate assigned-device mapping') + mapping={int(r['DEVICE'][4:]):int(r['NPU ID']) for r in rows} + if len({r['GROUP'] for r in rows})!=1 or set(mapping.values())!=set(range(len(names))): + raise ValueError('Assigned devices need one group with contiguous local IDs') + return mapping + + +def main(): + nodes=list(Path('/dev').glob('rbln[0-9]*')) + group_path=Path('/sys/class/rebellions/rsd/rsd_group') + mapping=assigned_mapping(nodes,group_path.read_text()) + os.environ['RBLN_DEVICES']=','.join(str(n) for n in sorted(mapping.values())) + os.environ['RBLN_NPUS_PER_DEVICE']='1' + os.environ.pop('RBLN_DEVICE_MAP',None) + os.environ.pop('RBLN_DUMMY_DEVICE',None) + import rebel + import opt_prime.accel.platforms as platforms + from opt_prime.accel.platforms.registry import DeviceInfo + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + if rebel.device_count()!=len(mapping): + raise RuntimeError('SDK visible device count does not match assigned nodes') + original=platforms.discover + def discover(kinds=None): + if kinds==['atom']: + return [DeviceInfo('atom',physical,'rebellions',extra={'visible_npu_id':visible,'dev':f'/dev/rbln{physical}'}) for physical,visible in sorted(mapping.items())] + return original(kinds) + platforms.discover=discover + start=AtomRblnEngine.start + def checked_start(self,*args,**kwargs): + if set(self.devices)!=set(mapping): + raise RuntimeError('Engine must use exactly the assigned ATOM devices') + if assigned_mapping(nodes,group_path.read_text())!=mapping: + raise RuntimeError('Assigned device mapping changed before startup') + result=start(self,*args,**kwargs) + opened=set() + for f in Path('/proc/self/fd').iterdir(): + try:path=os.readlink(f) + except OSError:continue + if re.fullmatch(r'/dev/rbln[0-9]+',path):opened.add(path) + if opened!={str(n) for n in nodes} or assigned_mapping(nodes,group_path.read_text())!=mapping: + self.close() + raise RuntimeError('Device binding changed or unexpected ATOM descriptors opened') + print('ATOM270_BINDING '+json.dumps(dict(visible_count=len(mapping),physical_to_visible=mapping,opened=sorted(opened))),flush=True) + return result + AtomRblnEngine.start=checked_start + sys.argv=sys.argv[1:] + runpy.run_path(sys.argv[0],run_name='__main__') + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/diagnose_atom270_allcoreprobe.py b/experiments/evidence270/diagnose_atom270_allcoreprobe.py new file mode 100644 index 00000000..133c3f4e --- /dev/null +++ b/experiments/evidence270/diagnose_atom270_allcoreprobe.py @@ -0,0 +1,78 @@ +"""SDK decode replay without AtomBatcher: identical forced tokens, positions and prefill.""" +import json +from pathlib import Path + + +def decode_inputs(active, reference, length, index): + assert 1 <= index <= len(reference) + assert active and len(set(active)) == len(active) and all(0 <= x < 4 for x in active) + return ([reference[index-1] if i in active else 0 for i in range(4)], + [length+index-1 if i in active else 0 for i in range(4)], + [[i if i in active else 4] for i in range(4)]) + + +def main(): + import torch + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import validate_compiled_geometry + root=Path(__file__).resolve().parent + output=root/'atom270-qwen4b-allcoreprobe-sdk.json' + if output.exists():raise FileExistsError(output) + data=json.loads((root/'atom270-qwen4b-tp1-b4-native.json').read_text()) + nodes=list(Path('/dev').glob('rbln[0-9]*'));assert len(nodes)==1 + torch.set_num_threads(4) + engine=AtomRblnEngine(devices=[int(nodes[0].name[4:])],max_seq_len=9216, + batch_size=4,kv_blocks=5,cache_dir=str(root/'atom270-artifacts-allcoreprobe')) + result=dict(complete=False,model='Qwen/Qwen3-4B',rows=[]) + try: + engine.start(result['model']) + model=engine.model;decoder=model.decoders[4] + validate_compiled_geometry(model.rbln_config,4,5, + artifact_config=json.loads((Path(engine.dir)/'rbln_config.json').read_text())) + result['compiled_dir_name']=Path(engine.dir).name + result['decoder_class']=type(decoder).__name__ + with torch.inference_mode(): + for length,target in [(2048,13),(512,50)]: + prompt=data['prompts'][str(length)] + reference=data['references'][f'{length}:128']['tokens'] + for active in [[0,1,2,3]]: + prefill=[] + for block in range(5): + ids=prompt if block<4 else [0] + out=model.prefill_decoder(input_ids=torch.tensor([ids],dtype=torch.int64), + cache_position=torch.arange(len(ids),dtype=torch.int32)[None],batch_idx=0, + block_tables=torch.tensor([block],dtype=torch.int16)) + assert int(out.padded_cache_lengths or 0)==0 + prefill.append(int(out.logits[0,-1].argmax())) + assert len(set(prefill[:4]))==1 + probes=[] + for index in range(1,target+1): + tokens,positions,blocks=decode_inputs(active,reference,length,index) + out=decoder(input_ids=torch.tensor(tokens,dtype=torch.int64)[:,None].contiguous(), + cache_position=torch.tensor(positions,dtype=torch.int32)[:,None].contiguous(), + block_tables=torch.tensor(blocks,dtype=torch.int16)) + logits=out.logits[:,-1].float() + assert torch.isfinite(logits).all() + if index in [target-1,target]: + rows=[] + for lane in active: + values,ids=logits[lane].topk(8) + rows.append(dict(lane=lane,token=int(logits[lane].argmax()), + top_tokens=ids.tolist(),top_logits=values.tolist(), + reference_token=reference[index],reference_logit=float(logits[lane,reference[index]]), + max_abs_from_lane0=float((logits[lane]-logits[0]).abs().max()) if 0 in active else None)) + hidden=out.hidden_states + assert hidden is not None and len(hidden)==73 + hidden_stats=[] + for boundary,h in enumerate(hidden): + assert h.shape[0]==4 and torch.isfinite(h).all() + hidden_stats.append(dict(boundary=boundary,label=('core_attention_layer_'+str(boundary-37) if boundary>=37 else 'original_boundary_'+str(boundary)),shape=list(h.shape),dtype=str(h.dtype),max_abs_from_lane0=[float((h[i].float()-h[0].float()).abs().max()) for i in range(4)])) + probes.append(dict(index=index,logits_dtype=str(out.logits.dtype),rows=rows,hidden=hidden_stats)) + row=dict(input_tokens=length,active_lanes=active,prefill_tokens=prefill[:4],probes=probes) + result['rows'].append(row) + output.write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(row),flush=True) + result['complete']=True;output.write_text(json.dumps(result,indent=2)+'\n') + finally:engine.close() + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/diagnose_atom270_attentionraw.py b/experiments/evidence270/diagnose_atom270_attentionraw.py new file mode 100644 index 00000000..b5753117 --- /dev/null +++ b/experiments/evidence270/diagnose_atom270_attentionraw.py @@ -0,0 +1,80 @@ +"""SDK decode replay without AtomBatcher: identical forced tokens, positions and prefill.""" +import json +from pathlib import Path + + +def decode_inputs(active, reference, length, index): + assert 1 <= index <= len(reference) + assert active and len(set(active)) == len(active) and all(0 <= x < 4 for x in active) + return ([reference[index-1] if i in active else 0 for i in range(4)], + [length+index-1 if i in active else 0 for i in range(4)], + [[i if i in active else 4] for i in range(4)]) + + +def main(): + import torch + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import validate_compiled_geometry + root=Path(__file__).resolve().parent + output=root/'atom270-qwen4b-attentionraw-sdk.json' + if output.exists():raise FileExistsError(output) + data=json.loads((root/'atom270-qwen4b-tp1-b4-native.json').read_text()) + nodes=list(Path('/dev').glob('rbln[0-9]*'));assert len(nodes)==1 + torch.set_num_threads(4) + engine=AtomRblnEngine(devices=[int(nodes[0].name[4:])],max_seq_len=9216, + batch_size=4,kv_blocks=5,cache_dir=str(root/'atom270-artifacts-attentionraw')) + result=dict(complete=False,model='Qwen/Qwen3-4B',rows=[]) + try: + engine.start(result['model']) + model=engine.model;decoder=model.decoders[4] + validate_compiled_geometry(model.rbln_config,4,5, + artifact_config=json.loads((Path(engine.dir)/'rbln_config.json').read_text())) + result['compiled_dir_name']=Path(engine.dir).name + result['decoder_class']=type(decoder).__name__ + with torch.inference_mode(): + for length,target in [(2048,13),(512,50)]: + prompt=data['prompts'][str(length)] + reference=data['references'][f'{length}:128']['tokens'] + for active in [[0,1,2,3]]: + prefill=[] + for block in range(5): + ids=prompt if block<4 else [0] + out=model.prefill_decoder(input_ids=torch.tensor([ids],dtype=torch.int64), + cache_position=torch.arange(len(ids),dtype=torch.int32)[None],batch_idx=0, + block_tables=torch.tensor([block],dtype=torch.int16)) + assert int(out.padded_cache_lengths or 0)==0 + prefill.append(int(out.logits[0,-1].argmax())) + assert len(set(prefill[:4]))==1 + probes=[] + for index in range(1,target+1): + tokens,positions,blocks=decode_inputs(active,reference,length,index) + out=decoder(input_ids=torch.tensor(tokens,dtype=torch.int64)[:,None].contiguous(), + cache_position=torch.tensor(positions,dtype=torch.int32)[:,None].contiguous(), + block_tables=torch.tensor(blocks,dtype=torch.int16)) + logits=out.logits[:,-1].float() + assert torch.isfinite(logits).all() + if index in [target-1,target]: + rows=[] + for lane in active: + values,ids=logits[lane].topk(8) + rows.append(dict(lane=lane,token=int(logits[lane].argmax()), + top_tokens=ids.tolist(),top_logits=values.tolist(), + reference_token=reference[index],reference_logit=float(logits[lane,reference[index]]), + max_abs_from_lane0=float((logits[lane]-logits[0]).abs().max()) if 0 in active else None)) + hidden=out.hidden_states + assert hidden is not None and len(hidden)==37 + hidden_stats=[] + for boundary,h in enumerate(hidden): + assert h.shape[0]==4 and torch.isfinite(h).all() + labels=['embedding','q_proj','k_proj','v_proj','q_norm','k_norm','rotary_q','rotary_k','attention_v','attention_result','o_proj'] + label=labels[boundary] if boundary<11 else 'original_boundary_'+str(boundary) + hidden_stats.append(dict(boundary=boundary,label=label,shape=list(h.shape),dtype=str(h.dtype),max_abs_from_lane0=[float((h[i].float()-h[0].float()).abs().max()) for i in range(4)])) + probes.append(dict(index=index,logits_dtype=str(out.logits.dtype),rows=rows,hidden=hidden_stats)) + row=dict(input_tokens=length,active_lanes=active,prefill_tokens=prefill[:4],probes=probes) + result['rows'].append(row) + output.write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(row),flush=True) + result['complete']=True;output.write_text(json.dumps(result,indent=2)+'\n') + finally:engine.close() + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/diagnose_atom270_direct.py b/experiments/evidence270/diagnose_atom270_direct.py new file mode 100644 index 00000000..90a746de --- /dev/null +++ b/experiments/evidence270/diagnose_atom270_direct.py @@ -0,0 +1,72 @@ +"""SDK decode replay without AtomBatcher: identical forced tokens, positions and prefill.""" +import json +from pathlib import Path + + +def decode_inputs(active, reference, length, index): + assert 1 <= index <= len(reference) + assert active and len(set(active)) == len(active) and all(0 <= x < 4 for x in active) + return ([reference[index-1] if i in active else 0 for i in range(4)], + [length+index-1 if i in active else 0 for i in range(4)], + [[i if i in active else 4] for i in range(4)]) + + +def main(): + import torch + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import validate_compiled_geometry + root=Path(__file__).resolve().parent + output=root/'atom270-qwen4b-direct-sdk.json' + if output.exists():raise FileExistsError(output) + data=json.loads((root/'atom270-qwen4b-tp1-b4-native.json').read_text()) + nodes=list(Path('/dev').glob('rbln[0-9]*'));assert len(nodes)==1 + torch.set_num_threads(4) + engine=AtomRblnEngine(devices=[int(nodes[0].name[4:])],max_seq_len=9216, + batch_size=4,kv_blocks=5,cache_dir=str(root/'atom270-artifacts')) + result=dict(complete=False,model='Qwen/Qwen3-4B',rows=[]) + try: + engine.start(result['model']) + model=engine.model;decoder=model.decoders[4] + validate_compiled_geometry(model.rbln_config,4,5, + artifact_config=json.loads((Path(engine.dir)/'rbln_config.json').read_text())) + result['compiled_dir_name']=Path(engine.dir).name + result['decoder_class']=type(decoder).__name__ + with torch.inference_mode(): + for length,target in [(2048,13),(512,50)]: + prompt=data['prompts'][str(length)] + reference=data['references'][f'{length}:128']['tokens'] + for active in [[0],[1],[2],[3],[0,1,2,3]]: + prefill=[] + for block in range(5): + ids=prompt if block<4 else [0] + out=model.prefill_decoder(input_ids=torch.tensor([ids],dtype=torch.int64), + cache_position=torch.arange(len(ids),dtype=torch.int32)[None],batch_idx=0, + block_tables=torch.tensor([block],dtype=torch.int16)) + assert int(out.padded_cache_lengths or 0)==0 + prefill.append(int(out.logits[0,-1].argmax())) + assert len(set(prefill[:4]))==1 + probes=[] + for index in range(1,target+1): + tokens,positions,blocks=decode_inputs(active,reference,length,index) + out=decoder(input_ids=torch.tensor(tokens,dtype=torch.int64)[:,None].contiguous(), + cache_position=torch.tensor(positions,dtype=torch.int32)[:,None].contiguous(), + block_tables=torch.tensor(blocks,dtype=torch.int16)) + logits=out.logits[:,-1].float() + assert torch.isfinite(logits).all() + if index in [target-1,target]: + rows=[] + for lane in active: + values,ids=logits[lane].topk(8) + rows.append(dict(lane=lane,token=int(logits[lane].argmax()), + top_tokens=ids.tolist(),top_logits=values.tolist(), + reference_token=reference[index],reference_logit=float(logits[lane,reference[index]]), + max_abs_from_lane0=float((logits[lane]-logits[0]).abs().max()) if 0 in active else None)) + probes.append(dict(index=index,logits_dtype=str(out.logits.dtype),rows=rows)) + row=dict(input_tokens=length,active_lanes=active,prefill_tokens=prefill[:4],probes=probes) + result['rows'].append(row) + output.write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(row),flush=True) + result['complete']=True;output.write_text(json.dumps(result,indent=2)+'\n') + finally:engine.close() + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/diagnose_atom270_firstkv_control.py b/experiments/evidence270/diagnose_atom270_firstkv_control.py new file mode 100644 index 00000000..5c975c7b --- /dev/null +++ b/experiments/evidence270/diagnose_atom270_firstkv_control.py @@ -0,0 +1,117 @@ +"""SDK decode replay without AtomBatcher: identical forced tokens, positions and prefill.""" +import json +from pathlib import Path + + +def decode_inputs(active, reference, length, index): + assert 1 <= index <= len(reference) + assert active and len(set(active)) == len(active) and all(0 <= x < 4 for x in active) + return ([reference[index-1] if i in active else 0 for i in range(4)], + [length+index-1 if i in active else 0 for i in range(4)], + [[i if i in active else 4] for i in range(4)]) + + +def main(): + import torch + import numpy as np + import hashlib + from rebel.kv_cache import aligned_tensor + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import validate_compiled_geometry + root=Path(__file__).resolve().parent + output=root/'atom270-qwen4b-firstkv-control-sdk.json' + if output.exists():raise FileExistsError(output) + data=json.loads((root/'atom270-qwen4b-tp1-b4-native.json').read_text()) + nodes=list(Path('/dev').glob('rbln[0-9]*'));assert len(nodes)==1 + torch.set_num_threads(4) + engine=AtomRblnEngine(devices=[int(nodes[0].name[4:])],max_seq_len=9216, + batch_size=4,kv_blocks=5,cache_dir=str(root/'atom270-artifacts-hidden')) + result=dict(complete=False,model='Qwen/Qwen3-4B',rows=[]) + try: + engine.start(result['model']) + model=engine.model;decoder=model.decoders[4] + validate_compiled_geometry(model.rbln_config,4,5, + artifact_config=json.loads((Path(engine.dir)/'rbln_config.json').read_text())) + result['compiled_dir_name']=Path(engine.dir).name + result['decoder_class']=type(decoder).__name__ + runtime=model.model[1] + metas=model.rbln_config.kvcache_metas[:2] + assert all(isinstance(m,dict) for m in metas) + buffers=[] + for meta in metas: + count=int(np.prod(meta['shape'])) + size=runtime.get_kv_cache_size_for_layer(meta['name']) + assert size==count*2 + buffers.append(aligned_tensor(count,dtype=np.uint16,alignment=2<<20,tensor_type='np').reshape(meta['shape'])) + def read_first_kv(n): + records=[] + for meta,buf in zip(metas,buffers): + arrays=[] + for block in range(4): + runtime.get_kv_cache_for_layer(meta['name'],block,0,n,buf) + arrays.append(buf[block,:,:n].copy()) + differences=[a!=arrays[0] for a in arrays] + records.append(dict(name=meta['name'],tokens=n,shape=list(arrays[0].shape), + block_sha256=[hashlib.sha256(a.tobytes()).hexdigest() for a in arrays], + changed_elements=[int(x.sum()) for x in differences], + first_changed_position=[int(np.flatnonzero(x.any(axis=(0,2)))[0]) if x.any() else None for x in differences])) + return records + with torch.inference_mode(): + for length,target in [(2048,13),(512,50)]: + prompt=data['prompts'][str(length)] + reference=data['references'][f'{length}:128']['tokens'] + for active in [[0,1,2,3]]: + prefill=[] + for block in range(5): + ids=prompt if block<4 else [0] + out=model.prefill_decoder(input_ids=torch.tensor([ids],dtype=torch.int64), + cache_position=torch.arange(len(ids),dtype=torch.int32)[None],batch_idx=0, + block_tables=torch.tensor([block],dtype=torch.int16)) + assert int(out.padded_cache_lengths or 0)==0 + prefill.append(int(out.logits[0,-1].argmax())) + assert len(set(prefill[:4]))==1 + probes=[] + kv_checks=[] + for index in range(1,target+1): + if index in [1,target-1,target]: + kv_checks.append(dict(phase='before',index=index,cache=read_first_kv(length+index-1))) + tokens,positions,blocks=decode_inputs(active,reference,length,index) + out=decoder(input_ids=torch.tensor(tokens,dtype=torch.int64)[:,None].contiguous(), + cache_position=torch.tensor(positions,dtype=torch.int32)[:,None].contiguous(), + block_tables=torch.tensor(blocks,dtype=torch.int16)) + logits=out.logits[:,-1].float() + assert torch.isfinite(logits).all() + if index in [target-1,target]: + rows=[] + for lane in active: + values,ids=logits[lane].topk(8) + rows.append(dict(lane=lane,token=int(logits[lane].argmax()), + top_tokens=ids.tolist(),top_logits=values.tolist(), + reference_token=reference[index],reference_logit=float(logits[lane,reference[index]]), + max_abs_from_lane0=float((logits[lane]-logits[0]).abs().max()) if 0 in active else None)) + hidden=out.hidden_states + assert hidden is not None and len(hidden)==37 + hidden_stats=[] + for boundary,h in enumerate(hidden): + assert h.shape[0]==4 and torch.isfinite(h).all() + hidden_stats.append(dict(boundary=boundary,shape=list(h.shape),dtype=str(h.dtype),max_abs_from_lane0=[float((h[i].float()-h[0].float()).abs().max()) for i in range(4)])) + probes.append(dict(index=index,logits_dtype=str(out.logits.dtype),rows=rows,hidden=hidden_stats)) + kv_checks.append(dict(phase='after',index=target,cache=read_first_kv(length+target))) + row=dict(input_tokens=length,active_lanes=active,prefill_tokens=prefill[:4],probes=probes,kv_checks=kv_checks) + changed=list(prompt);changed[0]=(changed[0]+1)%model.config.vocab_size + controls=[] + for label,ids in [('changed',changed),('restored',prompt)]: + model.prefill_decoder(input_ids=torch.tensor([ids],dtype=torch.int64),cache_position=torch.arange(len(ids),dtype=torch.int32)[None],batch_idx=0,block_tables=torch.tensor([3],dtype=torch.int16)) + check=read_first_kv(length) + for c in check: + assert c['changed_elements'][:3]==[0,0,0] + assert (c['changed_elements'][3]>0) if label=='changed' else (c['changed_elements'][3]==0) + controls.append(dict(phase=label,cache=check)) + row['readback_controls']=controls + result['rows'].append(row) + output.write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(row),flush=True) + result['complete']=True;output.write_text(json.dumps(result,indent=2)+'\n') + finally:engine.close() + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/diagnose_atom270_hidden.py b/experiments/evidence270/diagnose_atom270_hidden.py new file mode 100644 index 00000000..26806365 --- /dev/null +++ b/experiments/evidence270/diagnose_atom270_hidden.py @@ -0,0 +1,78 @@ +"""SDK decode replay without AtomBatcher: identical forced tokens, positions and prefill.""" +import json +from pathlib import Path + + +def decode_inputs(active, reference, length, index): + assert 1 <= index <= len(reference) + assert active and len(set(active)) == len(active) and all(0 <= x < 4 for x in active) + return ([reference[index-1] if i in active else 0 for i in range(4)], + [length+index-1 if i in active else 0 for i in range(4)], + [[i if i in active else 4] for i in range(4)]) + + +def main(): + import torch + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import validate_compiled_geometry + root=Path(__file__).resolve().parent + output=root/'atom270-qwen4b-hidden-sdk.json' + if output.exists():raise FileExistsError(output) + data=json.loads((root/'atom270-qwen4b-tp1-b4-native.json').read_text()) + nodes=list(Path('/dev').glob('rbln[0-9]*'));assert len(nodes)==1 + torch.set_num_threads(4) + engine=AtomRblnEngine(devices=[int(nodes[0].name[4:])],max_seq_len=9216, + batch_size=4,kv_blocks=5,cache_dir=str(root/'atom270-artifacts-hidden')) + result=dict(complete=False,model='Qwen/Qwen3-4B',rows=[]) + try: + engine.start(result['model']) + model=engine.model;decoder=model.decoders[4] + validate_compiled_geometry(model.rbln_config,4,5, + artifact_config=json.loads((Path(engine.dir)/'rbln_config.json').read_text())) + result['compiled_dir_name']=Path(engine.dir).name + result['decoder_class']=type(decoder).__name__ + with torch.inference_mode(): + for length,target in [(2048,13),(512,50)]: + prompt=data['prompts'][str(length)] + reference=data['references'][f'{length}:128']['tokens'] + for active in [[0,1,2,3]]: + prefill=[] + for block in range(5): + ids=prompt if block<4 else [0] + out=model.prefill_decoder(input_ids=torch.tensor([ids],dtype=torch.int64), + cache_position=torch.arange(len(ids),dtype=torch.int32)[None],batch_idx=0, + block_tables=torch.tensor([block],dtype=torch.int16)) + assert int(out.padded_cache_lengths or 0)==0 + prefill.append(int(out.logits[0,-1].argmax())) + assert len(set(prefill[:4]))==1 + probes=[] + for index in range(1,target+1): + tokens,positions,blocks=decode_inputs(active,reference,length,index) + out=decoder(input_ids=torch.tensor(tokens,dtype=torch.int64)[:,None].contiguous(), + cache_position=torch.tensor(positions,dtype=torch.int32)[:,None].contiguous(), + block_tables=torch.tensor(blocks,dtype=torch.int16)) + logits=out.logits[:,-1].float() + assert torch.isfinite(logits).all() + if index in [target-1,target]: + rows=[] + for lane in active: + values,ids=logits[lane].topk(8) + rows.append(dict(lane=lane,token=int(logits[lane].argmax()), + top_tokens=ids.tolist(),top_logits=values.tolist(), + reference_token=reference[index],reference_logit=float(logits[lane,reference[index]]), + max_abs_from_lane0=float((logits[lane]-logits[0]).abs().max()) if 0 in active else None)) + hidden=out.hidden_states + assert hidden is not None and len(hidden)==37 + hidden_stats=[] + for boundary,h in enumerate(hidden): + assert h.shape[0]==4 and torch.isfinite(h).all() + hidden_stats.append(dict(boundary=boundary,shape=list(h.shape),dtype=str(h.dtype),max_abs_from_lane0=[float((h[i].float()-h[0].float()).abs().max()) for i in range(4)])) + probes.append(dict(index=index,logits_dtype=str(out.logits.dtype),rows=rows,hidden=hidden_stats)) + row=dict(input_tokens=length,active_lanes=active,prefill_tokens=prefill[:4],probes=probes) + result['rows'].append(row) + output.write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(row),flush=True) + result['complete']=True;output.write_text(json.dumps(result,indent=2)+'\n') + finally:engine.close() + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/diagnose_atom270_layerprobe.py b/experiments/evidence270/diagnose_atom270_layerprobe.py new file mode 100644 index 00000000..3323dcf1 --- /dev/null +++ b/experiments/evidence270/diagnose_atom270_layerprobe.py @@ -0,0 +1,80 @@ +"""SDK decode replay without AtomBatcher: identical forced tokens, positions and prefill.""" +import json +from pathlib import Path + + +def decode_inputs(active, reference, length, index): + assert 1 <= index <= len(reference) + assert active and len(set(active)) == len(active) and all(0 <= x < 4 for x in active) + return ([reference[index-1] if i in active else 0 for i in range(4)], + [length+index-1 if i in active else 0 for i in range(4)], + [[i if i in active else 4] for i in range(4)]) + + +def main(): + import torch + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import validate_compiled_geometry + root=Path(__file__).resolve().parent + output=root/'atom270-qwen4b-layerprobe-sdk.json' + if output.exists():raise FileExistsError(output) + data=json.loads((root/'atom270-qwen4b-tp1-b4-native.json').read_text()) + nodes=list(Path('/dev').glob('rbln[0-9]*'));assert len(nodes)==1 + torch.set_num_threads(4) + engine=AtomRblnEngine(devices=[int(nodes[0].name[4:])],max_seq_len=9216, + batch_size=4,kv_blocks=5,cache_dir=str(root/'atom270-artifacts-layerprobe')) + result=dict(complete=False,model='Qwen/Qwen3-4B',rows=[]) + try: + engine.start(result['model']) + model=engine.model;decoder=model.decoders[4] + validate_compiled_geometry(model.rbln_config,4,5, + artifact_config=json.loads((Path(engine.dir)/'rbln_config.json').read_text())) + result['compiled_dir_name']=Path(engine.dir).name + result['decoder_class']=type(decoder).__name__ + with torch.inference_mode(): + for length,target in [(2048,13),(512,50)]: + prompt=data['prompts'][str(length)] + reference=data['references'][f'{length}:128']['tokens'] + for active in [[0,1,2,3]]: + prefill=[] + for block in range(5): + ids=prompt if block<4 else [0] + out=model.prefill_decoder(input_ids=torch.tensor([ids],dtype=torch.int64), + cache_position=torch.arange(len(ids),dtype=torch.int32)[None],batch_idx=0, + block_tables=torch.tensor([block],dtype=torch.int16)) + assert int(out.padded_cache_lengths or 0)==0 + prefill.append(int(out.logits[0,-1].argmax())) + assert len(set(prefill[:4]))==1 + probes=[] + for index in range(1,target+1): + tokens,positions,blocks=decode_inputs(active,reference,length,index) + out=decoder(input_ids=torch.tensor(tokens,dtype=torch.int64)[:,None].contiguous(), + cache_position=torch.tensor(positions,dtype=torch.int32)[:,None].contiguous(), + block_tables=torch.tensor(blocks,dtype=torch.int16)) + logits=out.logits[:,-1].float() + assert torch.isfinite(logits).all() + if index in [target-1,target]: + rows=[] + for lane in active: + values,ids=logits[lane].topk(8) + rows.append(dict(lane=lane,token=int(logits[lane].argmax()), + top_tokens=ids.tolist(),top_logits=values.tolist(), + reference_token=reference[index],reference_logit=float(logits[lane,reference[index]]), + max_abs_from_lane0=float((logits[lane]-logits[0]).abs().max()) if 0 in active else None)) + hidden=out.hidden_states + assert hidden is not None and len(hidden)==37 + hidden_stats=[] + for boundary,h in enumerate(hidden): + assert h.shape[0]==4 and torch.isfinite(h).all() + labels=['embedding','pre_attention_norm','attention_output','post_attention_norm','mlp_output','layer0_output'] + label=labels[boundary] if boundary<6 else 'original_boundary_'+str(boundary) + hidden_stats.append(dict(boundary=boundary,label=label,shape=list(h.shape),dtype=str(h.dtype),max_abs_from_lane0=[float((h[i].float()-h[0].float()).abs().max()) for i in range(4)])) + probes.append(dict(index=index,logits_dtype=str(out.logits.dtype),rows=rows,hidden=hidden_stats)) + row=dict(input_tokens=length,active_lanes=active,prefill_tokens=prefill[:4],probes=probes) + result['rows'].append(row) + output.write_text(json.dumps(result,indent=2)+'\n');print(json.dumps(row),flush=True) + result['complete']=True;output.write_text(json.dumps(result,indent=2)+'\n') + finally:engine.close() + + +if __name__=='__main__':main() diff --git a/experiments/evidence270/diagnose_atom270_samecard_retry1.py b/experiments/evidence270/diagnose_atom270_samecard_retry1.py new file mode 100644 index 00000000..63ad5157 --- /dev/null +++ b/experiments/evidence270/diagnose_atom270_samecard_retry1.py @@ -0,0 +1,92 @@ +"""Isolate ATOM B4 lane versus KV block under a fixed 2048-token prompt (#270).""" +import json +from pathlib import Path + + +def select_idle(batcher, lane, block): + assert batcher.idle and sorted(batcher.free) == list(range(4)) + assert sorted(batcher.blocks.free) == list(range(4)) + assert batcher.blocks.lane_block == [None] * 4 + assert lane in batcher.free and block in batcher.blocks.free + batcher.free[:] = [lane] + [x for x in batcher.free if x != lane] + batcher.blocks.free[:] = [block] + [x for x in batcher.blocks.free if x != block] + + +def main(fixed_prefill_lane=None, run_label="single"): + import torch + from opt_prime.accel.pd import PDRequest + from opt_prime.accel.platforms.atom.engine import AtomRblnEngine + from atom_server270_http import validate_compiled_geometry + root = Path(__file__).resolve().parent + suffix = '' if fixed_prefill_lane is None else '-prefill0' + output = root / f'atom270-qwen4b-samecard-retry1-{run_label}{suffix}.json' + if output.exists(): + raise FileExistsError(output) + data = json.loads((root / 'atom270-qwen4b-tp1-b4-native.json').read_text()) + prompt = data['prompts']['2048'] + reference = data['references']['2048:32']['tokens'] + torch.set_num_threads(4) + nodes = list(Path('/dev').glob('rbln[0-9]*')) + assert len(nodes) == 1 + engine = AtomRblnEngine(devices=[int(nodes[0].name[4:])], max_seq_len=9216, + batch_size=4, kv_blocks=5, cache_dir=str(root/'atom270-artifacts')) + result = dict(complete=False, model='Qwen/Qwen3-4B', batch_size=4, kv_blocks=5, + input_tokens=2048, fixed_prefill_lane=fixed_prefill_lane, rows=[]) + try: + engine.start(result['model']) + validate_compiled_geometry(engine.model.rbln_config, 4, 5, + artifact_config=json.loads((Path(engine.dir)/'rbln_config.json').read_text())) + batcher = engine.batcher(prefix_cache=False, local_prefill=True) + if fixed_prefill_lane is not None: + assert fixed_prefill_lane == 0 and not batcher.hybrid_cache + def prefill(lane, prompt, start=0): + assert start == 0 + return batcher.model.prefill_decoder( + input_ids=torch.tensor([list(prompt)], dtype=torch.int64), + cache_position=torch.arange(len(prompt), dtype=torch.int32)[None], + batch_idx=fixed_prefill_lane, + block_tables=torch.tensor([batcher.blocks.lane_block[lane]], dtype=torch.int16)) + batcher._prefill = prefill + emit, decoded = batcher._emit, batcher._decoded + for forced in (False, True): + for lane in range(4): + for block in range(4): + select_idle(batcher, lane, block) + probes, placements = [], [] + def on_emit(request, token): + assert request.slot == lane and batcher.blocks.lane_block[lane] == block + placements.append([request.slot, batcher.blocks.lane_block[lane]]) + index = len(request.tokens) + return emit(request, reference[index] if forced and index < 13 else token) + def on_decoded(rows, lengths, logits): + for i, request in enumerate(rows): + index = len(request.tokens) + if index in (12,13): + if forced: + assert request.tokens == reference[:index] + values, tokens = logits[i].float().topk(8) + probes.append(dict(index=index, top_tokens=tokens.tolist(), top_logits=values.tolist(), + logit369=float(logits[i,369]), logit264=float(logits[i,264]))) + return decoded(rows, lengths, logits) + batcher._emit, batcher._decoded = on_emit, on_decoded + future = batcher.submit(PDRequest(f'{forced}-{lane}-{block}', tuple(prompt), 14 if forced else 32)) + batcher.run_until_idle() + tokens = future.result() + assert len(tokens) == (14 if forced else 32) and len(probes) == 2 + row = dict(forced_prefix=forced, lane=lane, block=block, tokens=tokens, + matches_reference=tokens == reference[:len(tokens)], probes=probes, + checked_placements=len(placements)) + result['rows'].append(row) + output.write_text(json.dumps(result, indent=2)+'\n') + print(json.dumps(row), flush=True) + batcher._emit, batcher._decoded = emit, decoded + result['complete'] = True + output.write_text(json.dumps(result, indent=2)+'\n') + finally: + engine.close() + + +if __name__ == '__main__': + import sys + assert sys.argv[1] in ('a1','b1','b2','a2') + main(0 if sys.argv[1].startswith('b') else None, sys.argv[1]) diff --git a/experiments/evidence270/diagnostic-evidence-manifest.json b/experiments/evidence270/diagnostic-evidence-manifest.json new file mode 100644 index 00000000..deeb6020 --- /dev/null +++ b/experiments/evidence270/diagnostic-evidence-manifest.json @@ -0,0 +1,41 @@ +{ + "archive": "Local research-hub registration; remote synchronization not independently verified.", + "raws": [ + { + "file": "atom270-qwen4b-hidden-sdk.json", + "sha256": "fbeaf63365c11bd62d024aa01f62b889c7b3eea4401fde7ccc96826334f0bb6b", + "research_event_id": "b7a8db3a252147529a1215077fd03376" + }, + { + "file": "atom270-qwen4b-firstkv-control-sdk.json", + "sha256": "f1e4e89bbdb7b44fccc64c4590bff328fad6dcdcf63bb59030142c3366287e19", + "research_event_id": "d6d3ec81b1f64913a994fe30ee6cea8d" + }, + { + "file": "atom270-qwen4b-layerprobe-sdk.json", + "sha256": "8ef2432c82c9d39f4fb801c783776e5891fd5577a8b039fb3bcf2a21e3c31685", + "research_event_id": "24858169b62f4e0cbbdc0e3faf61c9e8" + }, + { + "file": "atom270-qwen4b-attentionraw-sdk.json", + "sha256": "95ea83ef9bac821055405ab9aaedd3bc044076b619889caa0590f6d5202d3856", + "research_event_id": "b66fcf032bf348fe92441eed9647fc8a" + }, + { + "file": "atom270-attentionprobe.log", + "sha256": "5e0c97226839462c03e80dd41cf40f5dd7c01c398c16aa95fa5abba9a0a2bcd5", + "research_event_id": "b66fcf032bf348fe92441eed9647fc8a" + } + ], + "sources": { + "build_atom270_hidden.py": "6a68702aca558fea48d7eba32237d6f4675c4b816738d08003c450188115950a", + "diagnose_atom270_hidden.py": "18ded477299a3e3132d966e3490570b0f345b7d3250a4a6cbacea0b8c1ec94fc", + "diagnose_atom270_firstkv_control.py": "ec2e3995f2c9931d4a0660696cfa1462a3d3678aac8606aac280c856d2300264", + "atom270_layer_probe.py": "e1b65c2e0559199c45f0c387d1a157b1090c7d29479ac8bcce04c5b6fbfc73eb", + "build_atom270_layerprobe.py": "da92ab137ffccefff03b37b23c02e92e00582a4db58a3e280ed885213eb6aedf", + "diagnose_atom270_layerprobe.py": "94c1d38b9869daa25a5a1bbaa21e74ee3369f368fa7c8a0ef3afffaa7b647913", + "atom270_attention_raw_probe.py": "f6a77b9d5790c4ad9bca282039d81d08eef249e14aae8f03e41e833c6abb010d", + "build_atom270_attentionraw.py": "408de9e6fe314b4bebbccd1eb0c8f4a31fb1472710f0706e958be886a1541b14", + "diagnose_atom270_attentionraw.py": "9a8ddc28f2034e46a1f03aa550ad9793c271f0d292eab2e219ea30720b8d0671" + } +} diff --git a/experiments/evidence270/fault-evidence-manifest.json b/experiments/evidence270/fault-evidence-manifest.json new file mode 100644 index 00000000..64c7cee4 --- /dev/null +++ b/experiments/evidence270/fault-evidence-manifest.json @@ -0,0 +1,25 @@ +{ + "storage": "research-hub artifact attachments; registered locally, remote sync not independently verified", + "files": { + "atom270-fault-b4-gemma.json": { + "sha256": "83b17d8acf98989a1e1df0dbb263cdf9588b3aa56b1942c29024458985227821", + "bytes": 71897, + "research_event_id": "24e3094043994026b5acb35209abcabe" + }, + "atom270-fault-b4-qwen06b.json": { + "sha256": "e61b9829ed90c84f08186b84ab42c5a6472ed8e5a55cb37e20af92b653f0a0e2", + "bytes": 74403, + "research_event_id": "a5225a891b594a978d45f0464fca32f5" + }, + "atom270-fault-b4-qwen4b.json": { + "sha256": "f73a020b719e01212d239bcf0ce7ea6b0ce7d69c0c08f6d7abb7604e77a92e96", + "bytes": 74401, + "research_event_id": "7b5d169ceaae48a6bbe3c0dbe45c75f6" + }, + "atom270-fault-b4-koni.json": { + "sha256": "5374d6383f09c444c9044a52b9a746f52baf5bb297d8af44817098c9af5a5941", + "bytes": 74289, + "research_event_id": "e40abcf0ec324331b2c04aa34d6c4198" + } + } +} diff --git a/experiments/evidence270/four-model-current270-audit.json b/experiments/evidence270/four-model-current270-audit.json new file mode 100644 index 00000000..54805940 --- /dev/null +++ b/experiments/evidence270/four-model-current270-audit.json @@ -0,0 +1,788 @@ +{ + "http": { + "qwen06b/gpu/b4/native": { + "file": "stream265-qwen06b-http-c16-native.json", + "sha256": "e2e467edfb6ebd2412aec8704e48a96a84563290303c327a0ac40ae74310cbb8", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/gpu/b4/whole": { + "file": "stream265-qwen06b-http-c16-whole.json", + "sha256": "0d0cc35d41f260798cb8d330cead5c19a112c0508f9bfa241d31da72b5b8ea00", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/gpu/b4/stream": { + "file": "stream265-qwen06b-http-c16-stream.json", + "sha256": "e8af822f9b8a11bb94130fc8abec6937907da2a5e411b0567e4ca1d19be68562", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/atom/b1/native": { + "file": "atom270-qwen06b-tp1-b1-native.json", + "sha256": "1375ae8c9eb389b2a14bd70420539d3c48276261bae0d61c55e8e46973ac7b6f", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/atom/b1/whole": { + "file": "atom270-qwen06b-tp1-b1-whole.json", + "sha256": "a1fa82e039e96027508ab0da5b714575632aee09ff2fdbc9ff24c772a5911314", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/atom/b1/stream": { + "file": "atom270-qwen06b-tp1-b1-stream.json", + "sha256": "b0b2211cb820c2fa94f67d2cdee39def66032dcbd806320c0ed4d1fec47dfc9e", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/atom/b4/native": { + "file": "atom270-qwen06b-tp1-b4-native.json", + "sha256": "c9f2a5e4f193b0db8a4d498d332443ceea21b02bf982c6fdb1fe1256d09f96e9", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/atom/b4/whole": { + "file": "atom270-qwen06b-tp1-b4-whole.json", + "sha256": "b789259526fca6a904ef23e1cfbc256f7a22fbc7444b4d9d8f261154fabbb2b6", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen06b/atom/b4/stream": { + "file": "atom270-qwen06b-tp1-b4-stream.json", + "sha256": "c415ce39681718661a61a9f64a9115849210235e9b3ce315e636a2cb58a960e4", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen4b/gpu/b4/native": { + "file": "stream265-qwen4b-http-c16-native.json", + "sha256": "4ac98e035408b6e8ac262569d404db549b5b9cf97b2d86391f7091e9a2f0a392", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen4b/gpu/b4/whole": { + "file": "stream265-qwen4b-http-c16-whole.json", + "sha256": "6dd9ae92178d14d1f26892651518a8ce68c2c8e7728667048f64efd961536368", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen4b/gpu/b4/stream": { + "file": "stream265-qwen4b-http-c16-stream.json", + "sha256": "f95fa88b03d9f2f53f36637b8cbbc9f42f90dbec6acf97a7111ba33f830198ce", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen4b/atom/b1/native": { + "file": "atom270-qwen4b-tp1-b1-native.json", + "sha256": "eb11ac21e4a6f74f2a258aaa319cabfcc7eea0b3b2e30f6dd4f1208d5f5e9c5a", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen4b/atom/b1/whole": { + "file": "atom270-qwen4b-tp1-b1-whole.json", + "sha256": "ca1d7fe3bc2b55461fd4d7ef01bbeae68265ca4a4af03ade5714faaea509cb5c", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen4b/atom/b1/stream": { + "file": "atom270-qwen4b-tp1-b1-stream.json", + "sha256": "a079bf27d380ecb93e1c33d7093b187fed24d96ecaea7cdaede8d34acf3263f3", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "qwen4b/atom/b4/native": { + "file": "atom270-qwen4b-tp1-b4-native.json", + "sha256": "11d07b625862acc161aa95c34cd2d1087e9bd302477c01ff7e7f66677f9234f7", + "state": "failed", + "reason": "runtime or parity failure", + "runtime_complete": true, + "parity_passed": false, + "failed_token_checks": 36 + }, + "qwen4b/atom/b4/whole": { + "state": "missing", + "file": "atom270-qwen4b-tp1-b4-whole.json" + }, + "qwen4b/atom/b4/stream": { + "state": "missing", + "file": "atom270-qwen4b-tp1-b4-stream.json" + }, + "koni/gpu/b4/native": { + "file": "stream265-koni-http-c16-native.json", + "sha256": "cc3a88e2d5c0d23fd791c39beda7a8d2cf374f6b31bef9ef80732fcfffab5a28", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/gpu/b4/whole": { + "file": "stream265-koni-http-c16-whole.json", + "sha256": "6698b52b83c863fd56d03b7b71ed032f88528e66885e5ae5f932bd1addc0fa64", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/gpu/b4/stream": { + "file": "stream265-koni-http-c16-stream.json", + "sha256": "c5d44fd309863d83d7d10422d5b35a798dea2427dd035160a41e29cc86f624de", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/atom/b1/native": { + "file": "atom270-koni-tp2-b1-native.json", + "sha256": "111c24490d36d835efa369d11ba05ce9a63e7c53da89ca4d7d8a72119c688f39", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/atom/b1/whole": { + "file": "atom270-koni-tp2-b1-whole.json", + "sha256": "a21aa944d52a1f0d9527f75e58d6b6f5bbc60cc7ddbef815517069018ceafe4c", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/atom/b1/stream": { + "file": "atom270-koni-tp2-b1-stream.json", + "sha256": "37cdbd2af69c954c9d9b485274a765dda72e3f413e0ebeab239c73d60b895fe7", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/atom/b4/native": { + "file": "atom270-koni-tp2-b4-native.json", + "sha256": "b865d242b87b6a948b81fa3bf97927b302db923c3834f013fbacb0b9de6c509c", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/atom/b4/whole": { + "file": "atom270-koni-tp2-b4-whole.json", + "sha256": "6d71f099b7811884212c4a00dc9ccfd4a0de661bb491a305aecab693d08048bb", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "koni/atom/b4/stream": { + "file": "atom270-koni-tp2-b4-stream.json", + "sha256": "dbc521450419643fd1dbe75c8ed4da5b4081656ae3071addab718b3a73cb2996", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/gpu/b4/native": { + "file": "stream265-gemma-http-c16-native.json", + "sha256": "03ad5688ebb5715bf41b9c7f0f697df132f4c29b9aa902f448d2463a1ef9f1b9", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/gpu/b4/whole": { + "file": "stream265-gemma-http-c16-whole.json", + "sha256": "64ef67de7e66e0c9a1ab047e59702c831ba977e90be119bb605cd4cf6e666a0d", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/gpu/b4/stream": { + "file": "stream265-gemma-http-c16-stream.json", + "sha256": "7efef90f546c15c9ea5eeea09e9cf92f72ed35ca2b63d6eadcc6c46b7584d950", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/atom/b1/native": { + "file": "atom270-causal-gemma-tp1-b1-native.json", + "sha256": "76b81b2728303015358810b1a76391f5bad2f63291ab4b8aad39c44ced7d19b9", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/atom/b1/whole": { + "file": "atom270-causal-gemma-tp1-b1-whole.json", + "sha256": "22fc823a863bd5ea01a80e7cad1ab01ce8519a89f1ab504f2c2eee8fe74bc54b", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/atom/b1/stream": { + "file": "atom270-causal-gemma-tp1-b1-stream.json", + "sha256": "2fa51d84d752b2d370dc73504194835cd282aced9ce980e37557b92ef7c4ada1", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/atom/b4/native": { + "file": "atom270-causal-gemma-tp1-b4-native.json", + "sha256": "d9b5752d5fefe4a26d95118699c7ee50407ab20fc7ea9ee1909fb0a1eec41e78", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/atom/b4/whole": { + "file": "atom270-causal-gemma-tp1-b4-whole.json", + "sha256": "0750435be2d4dc9c7ace05e59fc29a5362153b0018e9698755ef24fd1c78bbf5", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0 + }, + "gemma/atom/b4/stream": { + "file": "atom270-timeline-gemma-b4-stream.json", + "sha256": "b4658cbaa41b7e1f74de7cd614ef20b8f99bf6ed2983e6d7023e697bafcdf94d", + "state": "passed", + "audit": { + "stage": "passed", + "conditions": 8, + "warmups": 8, + "correctness_responses": 128, + "timed_text_responses": 128 + }, + "failed_token_checks": 0, + "limits": "Instrumented causal-export run; HTTP parity passes but max active lanes3, so full B4 occupancy remains unverified." + } + }, + "route_reference_matches": { + "qwen06b/gpu/b4/native-whole": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "qwen06b/gpu/b4/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "qwen06b/atom/b1/native-whole": { + "128:32": true, + "128:128": true, + "512:32": false, + "512:128": false, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "qwen06b/atom/b1/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "qwen06b/atom/b4/native-whole": { + "128:32": true, + "128:128": true, + "512:32": false, + "512:128": false, + "2048:32": false, + "2048:128": false, + "8192:32": true, + "8192:128": true + }, + "qwen06b/atom/b4/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "qwen4b/gpu/b4/native-whole": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": false, + "2048:32": true, + "2048:128": false, + "8192:32": false, + "8192:128": false + }, + "qwen4b/gpu/b4/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "qwen4b/atom/b1/native-whole": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": false, + "2048:32": true, + "2048:128": false, + "8192:32": true, + "8192:128": true + }, + "qwen4b/atom/b1/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "koni/gpu/b4/native-whole": { + "128:32": true, + "128:128": false, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": false, + "8192:32": true, + "8192:128": false + }, + "koni/gpu/b4/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "koni/atom/b1/native-whole": { + "128:32": false, + "128:128": false, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": false + }, + "koni/atom/b1/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "koni/atom/b4/native-whole": { + "128:32": true, + "128:128": false, + "512:32": false, + "512:128": false, + "2048:32": true, + "2048:128": false, + "8192:32": true, + "8192:128": false + }, + "koni/atom/b4/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "gemma/gpu/b4/native-whole": { + "128:32": true, + "128:128": true, + "512:32": false, + "512:128": false, + "2048:32": false, + "2048:128": false, + "8192:32": true, + "8192:128": true + }, + "gemma/gpu/b4/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "gemma/atom/b1/native-whole": { + "128:32": false, + "128:128": false, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "gemma/atom/b1/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "gemma/atom/b4/native-whole": { + "128:32": true, + "128:128": true, + "512:32": false, + "512:128": false, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + }, + "gemma/atom/b4/whole-stream": { + "128:32": true, + "128:128": true, + "512:32": true, + "512:128": true, + "2048:32": true, + "2048:128": true, + "8192:32": true, + "8192:128": true + } + }, + "quality": { + "qwen06b/native-gpu": { + "file": "quality263-qwen06b-native-gpu.jsonl", + "sha256": "7eae1901548995027bdffafc78a9a62cb0a63056bc0d68587b9d781d332b1463", + "cases": 36, + "answer_correct": 32, + "format_correct": 36 + }, + "qwen06b/rngd-gpu": { + "file": "quality263-qwen06b-rngd-gpu.jsonl", + "sha256": "14d2e4ffe69397e14e12a9157d9a4754585a4f1d925a72b68eac4810a7cdf751", + "cases": 36, + "answer_correct": 34, + "format_correct": 36 + }, + "qwen06b/native-atom": { + "file": "quality263-qwen06b-native-atom.jsonl", + "sha256": "ebac616e36356306ed87cadd8f29f0da18e04f5053de45bdd660b516611b4012", + "cases": 36, + "answer_correct": 33, + "format_correct": 36 + }, + "qwen06b/rngd-atom": { + "file": "quality263-qwen06b-rngd-atom.jsonl", + "sha256": "d8f73b70e85874e05b6fa6da55ffacc33ee0d15b264f9d2f074771ec4ee831b8", + "cases": 36, + "answer_correct": 33, + "format_correct": 36 + }, + "qwen4b/native-gpu": { + "file": "quality263-qwen4b-native-gpu.jsonl", + "sha256": "07de535a0b4e07a73a4e8a987349524843952a74c03e2d90daeb64a56e75c592", + "cases": 36, + "answer_correct": 36, + "format_correct": 36 + }, + "qwen4b/rngd-gpu": { + "file": "quality263-qwen4b-rngd-gpu.jsonl", + "sha256": "cd57299f4ac43a51224d05417533f8b0775c164142260c0c3a005528ee8cf50a", + "cases": 36, + "answer_correct": 36, + "format_correct": 36 + }, + "qwen4b/native-atom": { + "file": "quality263-qwen4b-native-atom.jsonl", + "sha256": "8facfb6ce5caf41faba23bee0825a699102743ba19af7679b9eb49c23ec1c032", + "cases": 36, + "answer_correct": 36, + "format_correct": 36 + }, + "qwen4b/rngd-atom": { + "file": "quality263-qwen4b-rngd-atom.jsonl", + "sha256": "988287bc91e8052c69acf17b91e9d11e76c725cf79f639010e24e278224d2ae6", + "cases": 36, + "answer_correct": 36, + "format_correct": 36 + }, + "koni/native-gpu": { + "file": "quality263-koni-native-gpu.jsonl", + "sha256": "21d9569e7e4bcd07f697aa029662238fdfd489af54e32a8d07c7745371a901cf", + "cases": 36, + "answer_correct": 31, + "format_correct": 34 + }, + "koni/rngd-gpu": { + "file": "quality263-koni-rngd-gpu.jsonl", + "sha256": "859b720c37bd2dc885d4849206b53826a068494a23563e6609fb764b0992fc37", + "cases": 36, + "answer_correct": 31, + "format_correct": 34 + }, + "koni/native-atom": { + "file": "quality263-koni-native-atom.jsonl", + "sha256": "87b43860ee2b5e2cbae939661a2ca2a3d03bd9c079d4b112fbeecf12361dc4b8", + "cases": 36, + "answer_correct": 31, + "format_correct": 34 + }, + "koni/rngd-atom": { + "file": "quality263-koni-rngd-atom.jsonl", + "sha256": "3d6c76f1198cba14cc95e2c5e8f1ab555d19899f6f4097aff6a20f9c6c029fdb", + "cases": 36, + "answer_correct": 31, + "format_correct": 34 + }, + "gemma/native-gpu": { + "file": "quality263-gemma-native-gpu.jsonl", + "sha256": "a2db94f0d9e87d01dc89091de3908716ca00c41f7ee8908b0e658def4cf9fe64", + "cases": 36, + "answer_correct": 0, + "format_correct": 0 + }, + "gemma/rngd-gpu": { + "file": "quality263-gemma-rngd-gpu.jsonl", + "sha256": "2c357b75f063f3f40911a6c05b92b039490789ca41704cecdabb2df8060617d7", + "cases": 36, + "answer_correct": 0, + "format_correct": 0 + }, + "gemma/native-atom": { + "file": "quality263-gemma-native-atom.jsonl", + "sha256": "3ce4573bacaff53daf3f2cf7736f04e37c7a38b107606eb76f068984f3cc1e88", + "cases": 36, + "answer_correct": 0, + "format_correct": 0 + }, + "gemma/rngd-atom": { + "file": "quality263-gemma-rngd-atom.jsonl", + "sha256": "f146eef24ba4d1e406f2750b7dc7bc116041c7e6ecc30670c3f86e9516da62fa", + "cases": 36, + "answer_correct": 0, + "format_correct": 0 + } + }, + "limits": "Historical task runtimes and isolated artifacts, not current-main acceptance. Same-route parity is not cross-engine parity; timings are not a matched speedup comparison. Missing routes remain missing." +} diff --git a/experiments/evidence270/gemma_causal_compat.py b/experiments/evidence270/gemma_causal_compat.py new file mode 100644 index 00000000..79ee3dcd --- /dev/null +++ b/experiments/evidence270/gemma_causal_compat.py @@ -0,0 +1,13 @@ +"""Process-local validation of the one-line upstream causal-prefill fix.""" +import inspect,textwrap,linecache +import optimum.rbln.transformers.models.decoderonly.decoderonly_architecture as module +old=module.SlidingWindowAttentionOp.forward +source=textwrap.dedent(inspect.getsource(old)) +needle='op_args["is_bidirectional"] = True' +assert source.count(needle)==1,'SDK source changed; review patch before using it' +source=source.replace(needle,'op_args["is_bidirectional"] = self.phase == "image_prefill"') +filename='' +linecache.cache[filename]=(len(source),None,source.splitlines(True),filename) +namespace=dict(module.__dict__) +exec(compile(source,filename,'exec'),namespace) +module.SlidingWindowAttentionOp.forward=namespace['forward'] diff --git a/experiments/evidence270/gemma_global_causal_compat.py b/experiments/evidence270/gemma_global_causal_compat.py new file mode 100644 index 00000000..3a9a6393 --- /dev/null +++ b/experiments/evidence270/gemma_global_causal_compat.py @@ -0,0 +1,22 @@ +"""Process-local global text-prefill correction; rebuild into a separate cache.""" + +import inspect +import linecache +import textwrap + +import gemma_causal_compat # noqa: F401 - preserve the validated sliding correction +import optimum.rbln.transformers.models.decoderonly.decoderonly_architecture as module + +for name in ("AttentionOp", "FlashAttentionOp"): + cls = getattr(module, name) + source = textwrap.dedent(inspect.getsource(cls.forward)) + needle = 'op_args["is_bidirectional"] = True' + assert source.count(needle) == 1, "SDK changed; review the global causal patch" + source = source.replace( + needle, 'op_args["is_bidirectional"] = self.phase == "image_prefill"' + ) + filename = f"" + linecache.cache[filename] = (len(source), None, source.splitlines(True), filename) + namespace = dict(module.__dict__) + exec(compile(source, filename, "exec"), namespace) + cls.forward = namespace["forward"] diff --git a/experiments/evidence270/heldout_quality270.py b/experiments/evidence270/heldout_quality270.py new file mode 100644 index 00000000..af1ab56e --- /dev/null +++ b/experiments/evidence270/heldout_quality270.py @@ -0,0 +1,133 @@ +"""Preregistered synthetic structured-answer checks; not a general quality benchmark.""" +import argparse +import hashlib +import json +import random +import time +import urllib.request +from pathlib import Path + + +def score(text, expected): + def unique(pairs): + result = {} + for key, value in pairs: + if key in result: + raise ValueError('duplicate JSON key') + result[key] = value + return result + try: + value = json.loads(text, object_pairs_hook=unique) + except (ValueError, TypeError): + value = None + valid = isinstance(value, dict) and isinstance(value.get('answer'), str) + return dict(answer_correct=bool(valid and value['answer'] == expected), + format_correct=bool(valid and set(value) == {'answer'})) + + +def compare(baseline, candidate): + left = {x['case']: x for x in baseline} + right = {x['case']: x for x in candidate} + if len(left) != len(baseline) or len(right) != len(candidate) or left.keys() != right.keys(): + raise ValueError('duplicate or missing paired cases') + result = dict(cases=len(left), correct_to_wrong=0, wrong_to_correct=0, token_equal=0) + for key, a in left.items(): + b = right[key] + if a['input_sha256'] != b['input_sha256']: + raise ValueError('paired input IDs differ') + result['correct_to_wrong'] += bool(a['answer_correct'] and not b['answer_correct']) + result['wrong_to_correct'] += bool(not a['answer_correct'] and b['answer_correct']) + result['token_equal'] += a['tokens'] == b['tokens'] + return result + + +def fixtures(tokenizer): + rng = random.Random(26320261024) + tasks = [] + for variant in range(4): + a, b = rng.randrange(21, 80), rng.randrange(11, 60) + tasks.append(('arithmetic', variant, '', f'Compute {a} + {b}.', str(a + b))) + secret = ''.join(rng.choice('ABCDEFGHJKLMNPQRSTUVWXYZ23456789') for _ in range(8)) + tasks.append(('passkey', variant, f'The access key for archive {variant} is {secret}.', + f'What is the access key for archive {variant}?', secret)) + city = ['춘천', '목포', '전주', '속초'][variant] + tasks.append(('korean_table', variant, + f'배송표: 주문 가{variant}=대전; 주문 나{variant}={city}; 주문 다{variant}=부산.', + f'배송표에서 주문 나{variant}의 도착 도시를 찾으세요.', city)) + filler = 'Background: shelves hold ordinary books; this note contains no task answer.\n' + result = [] + for target in [0, 2048, 8192]: + for task, variant, fact, question, expected in tasks: + def render(repeats): + prompt = (filler * (repeats // 2) + fact + '\n' + filler * (repeats - repeats // 2) + + '\n' + question + '\nReturn only a JSON object with one key "answer" and a string value. No explanation.') + ids = tokenizer.apply_chat_template([{'role': 'user', 'content': prompt}], + tokenize=True, return_dict=False, add_generation_prompt=True, enable_thinking=False) + return prompt, [int(x) for x in ids] + low, high = 0, target + if target and len(render(0)[1]) > target: + raise ValueError('base prompt exceeds context target') + while low < high: + mid = (low + high + 1) // 2 + if len(render(mid)[1]) <= target: + low = mid + else: + high = mid - 1 + prompt, ids = render(low) + digest = hashlib.sha256(json.dumps(ids, separators=(',', ':')).encode()).hexdigest() + result.append(dict(case=f'{task}-{variant}-{target}', task=task, target_tokens=target, + expected=expected, prompt=prompt, ids=ids, input_sha256=digest)) + return result + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest='command', required=True) + export = sub.add_parser('fixtures') + export.add_argument('--model', required=True) + export.add_argument('--output', type=Path, required=True) + run = sub.add_parser('run') + run.add_argument('--fixtures', type=Path, required=True) + run.add_argument('--endpoint', required=True) + run.add_argument('--route', required=True) + run.add_argument('--output', type=Path, required=True) + args = parser.parse_args() + if args.command == 'fixtures': + from transformers import AutoTokenizer + tokenizer = AutoTokenizer.from_pretrained(args.model, local_files_only=True) + data = dict(model=args.model, seed=26320261024, max_tokens=128, + thinking=False, cases=fixtures(tokenizer)) + args.output.write_text(json.dumps(data, ensure_ascii=False, indent=2) + '\n') + return + data = json.loads(args.fixtures.read_text()) + # Exclusive creation prevents silently replacing a prior run after seeing answers. + with args.output.open('x') as output: + for case in data['cases']: + body = dict(model=data['model'], prompt=case['ids'], temperature=0, + max_tokens=data['max_tokens'], ignore_eos=False, stream=False) + request = urllib.request.Request(args.endpoint.rstrip('/') + '/v1/completions', + data=json.dumps(body).encode(), headers={'Content-Type': 'application/json', + 'x-request-id': 'quality263-' + case['case']}) + start = time.perf_counter() + row = dict(case=case['case'], route=args.route, input_sha256=case['input_sha256'], + input_tokens=len(case['ids']), expected=case['expected']) + try: + with urllib.request.urlopen(request, timeout=600) as response: + payload = json.load(response) + choice = payload['choices'][0] + row.update(text=choice['text'], tokens=choice['token_ids'], + finish_reason=choice['finish_reason'], response=payload, + **score(choice['text'], case['expected'])) + except Exception as error: + row.update(error=repr(error), answer_correct=False, format_correct=False) + output.write(json.dumps(row, ensure_ascii=False) + '\n') + output.flush() + raise + row['elapsed_seconds'] = time.perf_counter() - start + output.write(json.dumps(row, ensure_ascii=False) + '\n') + output.flush() + print(case['case'], row['answer_correct'], row['format_correct'], flush=True) + + +if __name__ == '__main__': + main() diff --git a/experiments/evidence270/heldout_quality270_concurrent.py b/experiments/evidence270/heldout_quality270_concurrent.py new file mode 100644 index 00000000..99c93229 --- /dev/null +++ b/experiments/evidence270/heldout_quality270_concurrent.py @@ -0,0 +1,38 @@ +"""Frozen quality cases submitted in twelve-request cohorts by input-length stratum.""" +import argparse,concurrent.futures,hashlib,json,threading,time,urllib.request +from pathlib import Path +from heldout_quality270 import score + +def run(fixtures,endpoint,output): + data=json.loads(Path(fixtures).read_text()) + assert len(data['cases'])==36 and len({c['case'] for c in data['cases']})==36 + cohorts=[[c for c in data['cases'] if c['target_tokens']==n] for n in (0,2048,8192)] + assert all(len(c)==12 for c in cohorts) + with Path(output).open('x') as out: + for cohort in cohorts: + barrier=threading.Barrier(12) + def request(case): + digest=hashlib.sha256(json.dumps(case['ids'],separators=(',',':')).encode()).hexdigest() + assert digest==case['input_sha256'] + body=dict(model=data['model'],prompt=case['ids'],temperature=0,max_tokens=data['max_tokens'],ignore_eos=False,stream=False) + req=urllib.request.Request(endpoint.rstrip('/')+'/v1/completions',data=json.dumps(body).encode(),headers={'Content-Type':'application/json','x-request-id':'quality270-c12-'+case['case']}) + barrier.wait(timeout=30) + start=time.monotonic() + row=dict(case=case['case'],input_sha256=digest,input_tokens=len(case['ids']),expected=case['expected'],cohort=case['target_tokens'],request_start_monotonic=start) + try: + with urllib.request.urlopen(req,timeout=600) as response:payload=json.load(response) + choice=payload['choices'][0] + row.update(text=choice['text'],tokens=choice['token_ids'],finish_reason=choice['finish_reason'],response=payload,**score(choice['text'],case['expected'])) + except Exception as e:row.update(error=repr(e),answer_correct=False,format_correct=False) + row['request_end_monotonic']=time.monotonic();row['elapsed_seconds']=row['request_end_monotonic']-start + return row + with concurrent.futures.ThreadPoolExecutor(max_workers=12) as pool: + futures=[pool.submit(request,c) for c in cohort] + rows=[] + for future in concurrent.futures.as_completed(futures): + row=future.result();rows.append(row);out.write(json.dumps(row,ensure_ascii=False)+'\n');out.flush() + print(row['case'],row['answer_correct'],row['format_correct'],flush=True) + assert len(rows)==12 and not any('error' in r for r in rows), 'Request failure; all completed rows preserved' + +if __name__=='__main__': + p=argparse.ArgumentParser(description=__doc__);p.add_argument('--fixtures',required=True);p.add_argument('--endpoint',required=True);p.add_argument('--output',required=True);a=p.parse_args();run(a.fixtures,a.endpoint,a.output) diff --git a/experiments/evidence270/imported-fault-evidence-manifest.json b/experiments/evidence270/imported-fault-evidence-manifest.json new file mode 100644 index 00000000..6a502edf --- /dev/null +++ b/experiments/evidence270/imported-fault-evidence-manifest.json @@ -0,0 +1,18 @@ +{ + "research_event_id": "0f0d5e4c5fc243f1a8a16b0511518894", + "archive": "Raw JSON registered with local research-hub; remote synchronization not independently verified.", + "files": [ + { + "file": "atom270-imported-fault-b4-qwen06b.json", + "sha256": "e92ca8ba00a8610d4e68b940f325dd9da2588cd9c44143f348088f085a5750de" + }, + { + "file": "atom270-imported-fault-b4-qwen4b.json", + "sha256": "a45e0ec49fa2a1e50554c3f8c41cb9f446164895607344ea8dc8be2bff4689b0" + }, + { + "file": "atom270-imported-fault-b4-koni.json", + "sha256": "37787b68b525c50d62fe69868e61091a33b2af180e0345da9dcac715a8d3112d" + } + ] +} diff --git a/experiments/evidence270/qwen06-regression-evidence-manifest.json b/experiments/evidence270/qwen06-regression-evidence-manifest.json new file mode 100644 index 00000000..99cdb0b0 --- /dev/null +++ b/experiments/evidence270/qwen06-regression-evidence-manifest.json @@ -0,0 +1,38 @@ +{ + "research_event_id": "06f53eaae36443dab95b4b3eb0861283", + "storage": "research-hub attachments registered locally; remote synchronization not independently verified", + "files": { + "atom270-qwen06-regression-retry1-r0-whole.json": { + "sha256": "5fc169b062bd3181b0a6fcfeef06e7586b79b2d0e502c2d485b22a4e020ba940", + "bytes": 319795 + }, + "atom270-qwen06-regression-retry1-r0-whole.server0.log": { + "sha256": "c109d050bdd9023aec04eec148a045715aa0cd8ce56be9ecc18160146ba0bb9c", + "bytes": 845427 + }, + "atom270-qwen06-regression-retry1-r1-stream.json": { + "sha256": "3ff17acf90f08ffe318f60155a0c23a0e3f65f6b0533f81eded290040e8056b5", + "bytes": 319821 + }, + "atom270-qwen06-regression-retry1-r1-stream.server0.log": { + "sha256": "81173dc46ac2c9c6da6230801fce5895e0858437e29091d40eb3bc9bd4023d01", + "bytes": 600651 + }, + "atom270-qwen06-regression-retry1-r2-stream.json": { + "sha256": "0a53097c042a7c46bd55c12c428c99afb792ed579816dd727a239a0dda0adf12", + "bytes": 319811 + }, + "atom270-qwen06-regression-retry1-r2-stream.server0.log": { + "sha256": "34ab0eff903c0f7f3589d7864d473f56bd3bb7470d904b926cf5135ca15e8fb1", + "bytes": 602127 + }, + "atom270-qwen06-regression-retry1-r3-whole.json": { + "sha256": "7621b42a720e44bce355e342ea536578b51fff212a93bf679b21c50704ad7bd7", + "bytes": 319814 + }, + "atom270-qwen06-regression-retry1-r3-whole.server0.log": { + "sha256": "611ba99cae925155b06b967f018bea0d28a25b1ad26b0e983979b38fa0e3f736", + "bytes": 872030 + } + } +} diff --git a/experiments/evidence270/qwen4-lane-fixtures.json b/experiments/evidence270/qwen4-lane-fixtures.json new file mode 100644 index 00000000..24840dcb --- /dev/null +++ b/experiments/evidence270/qwen4-lane-fixtures.json @@ -0,0 +1,2870 @@ +{ + "prompts": { + "512": [ + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 198, + 14582, + 25, + 3555, + 374, + 279, + 6722, + 315, + 9625, + 5267, + 16141, + 25 + ], + "2048": [ + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745, + 5610, + 6467, + 624, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 198, + 14582, + 25, + 3555, + 374, + 279, + 6722, + 315, + 9625, + 5267, + 16141, + 25 + ] + }, + "references": { + "512:128": { + "tokens": [ + 576, + 6722, + 315, + 9625, + 374, + 12095, + 624, + 32313, + 11, + 773, + 279, + 1196, + 4588, + 369, + 279, + 6722, + 315, + 9625, + 11, + 323, + 358, + 18577, + 12095, + 13, + 1988, + 3783, + 11, + 279, + 1196, + 1083, + 3897, + 264, + 15493, + 315, + 11504, + 4004, + 2266, + 911, + 14697, + 8482, + 40676, + 323, + 6467, + 13, + 2938, + 4977, + 45205, + 13, + 10696, + 807, + 4829, + 752, + 311, + 990, + 429, + 2266, + 369, + 279, + 3405, + 30, + 1988, + 279, + 3405, + 572, + 911, + 279, + 6722, + 315, + 9625, + 11, + 892, + 374, + 264, + 4586, + 6540, + 3405, + 13, + 576, + 4004, + 2266, + 2578, + 387, + 264, + 2518, + 305, + 14443, + 476, + 7196, + 949, + 315, + 264, + 2155, + 3383, + 13, + 6771, + 752, + 1779, + 1549, + 382, + 785, + 1196, + 594, + 1943, + 8471, + 448, + 5248, + 84966, + 315, + 279, + 1852, + 4004, + 2266, + 25, + 330, + 1986, + 374, + 4004, + 2266, + 13, + 576, + 2518, + 3745, + 5610, + 40676, + 13, + 576, + 6303, + 3745 + ] + }, + "2048:128": { + "tokens": [ + 576, + 6722, + 315, + 9625, + 374, + 12095, + 624, + 32313, + 11, + 773, + 279, + 1196, + 4588, + 369, + 279, + 6722, + 315, + 9625, + 11, + 323, + 358, + 18577, + 12095, + 13, + 1988, + 3783, + 11, + 279, + 1196, + 3897, + 264, + 15493, + 315, + 11504, + 4004, + 2266, + 911, + 14697, + 8482, + 40676, + 323, + 6467, + 13, + 2938, + 4977, + 45205, + 13, + 10696, + 807, + 2299, + 7497, + 421, + 358, + 646, + 10034, + 39715, + 1995, + 30, + 2521, + 7196, + 1052, + 594, + 264, + 14068, + 1588, + 13, + 6771, + 752, + 1744, + 382, + 785, + 4004, + 2266, + 374, + 1101, + 11504, + 5128, + 911, + 2518, + 323, + 6303, + 14697, + 13, + 576, + 3405, + 374, + 911, + 279, + 6722, + 315, + 9625, + 13, + 576, + 4226, + 374, + 12095, + 13, + 576, + 1196, + 2578, + 387, + 13295, + 421, + 358, + 646, + 5244, + 389, + 279, + 5042, + 3405, + 8818, + 279, + 39715, + 2266, + 13, + 2521, + 8365, + 807, + 1366, + 311, + 1490, + 421, + 358, + 646, + 3705, + 5248, + 37597, + 323 + ] + }, + "2048:32": { + "tokens": [ + 576, + 6722, + 315, + 9625, + 374, + 12095, + 624, + 32313, + 11, + 773, + 279, + 1196, + 4588, + 369, + 279, + 6722, + 315, + 9625, + 11, + 323, + 358, + 18577, + 12095, + 13, + 1988, + 3783, + 11, + 279, + 1196, + 3897, + 264, + 15493 + ] + } + } +} diff --git a/experiments/evidence270/raw-evidence-manifest.json b/experiments/evidence270/raw-evidence-manifest.json new file mode 100644 index 00000000..aeb32c45 --- /dev/null +++ b/experiments/evidence270/raw-evidence-manifest.json @@ -0,0 +1,58 @@ +{ + "research_event_id": "578a57666040434b90f5b76a51871973", + "storage": "research-hub artifact attachments; registered locally, remote synchronization not independently verified", + "files": { + "atom270-timeline-gemma-b4-whole.json": { + "sha256": "6a34ace19a9b54cac19e0bfe500e6eb5081fdd76ad4f1bfadfc53b7cb87ecd6b", + "bytes": 545767 + }, + "atom270-timeline-gemma-b4-whole.server0.log": { + "sha256": "cbc6739e05c48039d3118b6422d5a1264c7dc44688ee7eca69538d95f267a415", + "bytes": 2199092 + }, + "atom270-timeline-gemma-b4-stream.json": { + "sha256": "b4658cbaa41b7e1f74de7cd614ef20b8f99bf6ed2983e6d7023e697bafcdf94d", + "bytes": 545802 + }, + "atom270-timeline-gemma-b4-stream.server0.log": { + "sha256": "bebfb02bde6f7c764fd6d9acd792c1621d84ebc7bd486c8c84208c37ab552c8a", + "bytes": 2182393 + }, + "atom270-abba-gemma-b4-r0-w8192.json": { + "sha256": "40f2702e77e93e120dafb3a46d697575a6a2fbea4b5248562b420f1107492df3", + "bytes": 220936 + }, + "atom270-abba-gemma-b4-r0-w8192.server0.log": { + "sha256": "89bf71fea58f9d11673ef8b71999777a8927d8cc9ea95674eb10b8acafafb0f2", + "bytes": 1263839 + }, + "atom270-abba-gemma-b4-r1-w1024.json": { + "sha256": "9cf1dc2a82ef3e15fb479f2b547b625828f6ab1454f3ae1da065d5e769d7b308", + "bytes": 220928 + }, + "atom270-abba-gemma-b4-r1-w1024.server0.log": { + "sha256": "7bc8be3957f24aba400130619dba6b5c347ac25ce5090f93aaede9fbc83bcbf1", + "bytes": 767179 + }, + "atom270-abba-gemma-b4-r2-w1024.json": { + "sha256": "2939601a01acc5a4885ec5cebe2176a040106e1bb4d0c81b02bb850054085ef0", + "bytes": 220929 + }, + "atom270-abba-gemma-b4-r2-w1024.server0.log": { + "sha256": "a04491e23500acada9a6bfd676d2950f38e69e5f5083498ffe935554af29189e", + "bytes": 757068 + }, + "atom270-abba-gemma-b4-r3-w8192.json": { + "sha256": "cb82aacf02bb2403d2302502b8810db2838e645946843ca602b9e43dbda85efe", + "bytes": 220949 + }, + "atom270-abba-gemma-b4-r3-w8192.server0.log": { + "sha256": "00d8c2924b7cd5338aadbacefd4028b35944852203b1e608035fad49975724d0", + "bytes": 1241723 + }, + "atom270-shortwidth-gemma-b4-whole.json": { + "sha256": "d5b023ca9bddcec08feead487995a505ec09daae804ecf14e79461eb99539db2", + "bytes": 220910 + } + } +} diff --git a/experiments/evidence270/run_atom270_allcore_abba_child.py b/experiments/evidence270/run_atom270_allcore_abba_child.py new file mode 100644 index 00000000..3283d6c1 --- /dev/null +++ b/experiments/evidence270/run_atom270_allcore_abba_child.py @@ -0,0 +1,20 @@ +"""Same-Pod fresh-process ABBA; parity failures remain failures in preserved raw data.""" +import json,subprocess +from pathlib import Path +A=Path(__file__).resolve().parent +queue=A/'atom270-allcore-abba-arms.json' +assert not queue.exists() +state=[] +for arm,config in [('a1','atom270-qwen4b-tp1-b4-native-config.json'),('b1','atom270-qwen4b-allcore-http-config.json'),('b2','atom270-qwen4b-allcore-http-config.json'),('a2','atom270-qwen4b-tp1-b4-native-config.json')]: + name=f'atom270-qwen4b-allcore-abba-{arm}.json';assert not (A/name).exists() + argv=['bash',str(A/'run-http270-atom.sh'),'--model','Qwen/Qwen3-4B','--mode','native','--output',name,'--server-config',config,'--lengths','512','8192','--outputs','128','--concurrency','16','--repeats','1','--slots','4','--startup-timeout','600','--request-timeout','600'] + print('ABBA_START',arm,flush=True) + child=subprocess.run(argv,cwd=A) + record={'arm':arm,'config':config,'exit_code':child.returncode,'argv':argv} + if (A/name).exists(): + d=json.loads((A/name).read_text());record.update(runtime_complete=d.get('passed',False),parity_passed=d.get('parity_passed',False),batches=len(d.get('batches',[]))) + state.append(record);queue.write_text(json.dumps(state,indent=2)+'\n') + # Continue expected parity failures; stop actual runtime failures, preserving the failed arm. + assert record.get('runtime_complete') and record.get('batches')==2,record + assert child.returncode==(0 if record['parity_passed'] else 1),record + print('ABBA_END',arm,json.dumps(record),flush=True) diff --git a/experiments/evidence270/run_atom270_allcore_pd_child.py b/experiments/evidence270/run_atom270_allcore_pd_child.py new file mode 100644 index 00000000..c601c033 --- /dev/null +++ b/experiments/evidence270/run_atom270_allcore_pd_child.py @@ -0,0 +1,16 @@ +"""Validate all-core decode with RNGD whole and streamed KV, retaining failed evidence.""" +import json,subprocess +from pathlib import Path +from audit_http265 import audit_result +A=Path(__file__).resolve().parent +queue=A/'atom270-allcore-pd-arms.json';assert not queue.exists();state=[] +for mode in ['whole','stream']: + stem=f'atom270-qwen4b-allcore-pd-{mode}' + assert not (A/(stem+'.json')).exists() + argv=['bash',str(A/'run-http270-atom.sh'),'--model','Qwen/Qwen3-4B','--mode','pd','--output',stem+'.json','--server-config',f'atom270-qwen4b-allcore-{mode}-config.json','--lengths','128','512','2048','8192','--outputs','32','128','--concurrency','16','--repeats','1','--slots','4','--startup-timeout','1800','--request-timeout','600'] + if mode=='stream':argv+=['--reference','atom270-qwen4b-allcore-pd-whole.json'] + print('PD_START',mode,flush=True);child=subprocess.run(argv,cwd=A) + record={'mode':mode,'exit_code':child.returncode,'argv':argv};state.append(record);queue.write_text(json.dumps(state,indent=2)+'\n') + assert child.returncode==0,'Failed raw data retained; dependent mode not run' + d=json.loads((A/(stem+'.json')).read_text());record['audit']=audit_result(d,'atom',mode,expected_slots=4,expected_atom_devices=1) + queue.write_text(json.dumps(state,indent=2)+'\n');print('PD_END',mode,json.dumps(record),flush=True) diff --git a/experiments/evidence270/run_atom270_allcore_pd_quality_child.py b/experiments/evidence270/run_atom270_allcore_pd_quality_child.py new file mode 100644 index 00000000..19832fe8 --- /dev/null +++ b/experiments/evidence270/run_atom270_allcore_pd_quality_child.py @@ -0,0 +1,56 @@ +"""Frozen concurrent quality gate for the all-core RNGD→ATOM routes.""" +import hashlib,json,os,signal,socket,subprocess,sys,time,urllib.request +from pathlib import Path +A=Path(__file__).resolve().parent +import argparse +p=argparse.ArgumentParser();p.add_argument('--mode',required=True,choices=['whole','stream']);a=p.parse_args() +stem='atom270-qwen4b-allcore-pd-quality-'+a.mode +config=json.loads((A/f'atom270-qwen4b-allcore-pd-quality-{a.mode}-config.json').read_text()) +fixture=A/'quality270-qwen4b-mapped-fixtures.json' +output=A/(stem+'.jsonl');statepath=A/(stem+'-run.json') +assert not output.exists() and not statepath.exists() +f=json.loads(fixture.read_text());assert len(f['cases'])==36 +from transformers import AutoTokenizer +tok=AutoTokenizer.from_pretrained(f['model'],local_files_only=True) +for c in f['cases']: + ids=tok.apply_chat_template([{'role':'user','content':c['prompt']}],tokenize=True,return_dict=False,add_generation_prompt=True,enable_thinking=False) + assert list(ids)==c['ids'] + assert hashlib.sha256(json.dumps(c['ids'],separators=(',',':')).encode()).hexdigest()==c['input_sha256'] +with socket.socket() as sock: + sock.bind(('127.0.0.1',0));port=sock.getsockname()[1] +argv=[str(port) if x=='{port}' else x for x in config['servers'][0]] +endpoint=f'http://127.0.0.1:{port}' +state={'state':'running','argv':argv,'fixtures_sha256':hashlib.sha256(fixture.read_bytes()).hexdigest(),'sequential':False,'concurrency':12} +statepath.write_text(json.dumps(state,indent=2)+'\n') +with (A/(stem+'.server0.log')).open('x') as log: + proc=subprocess.Popen(argv,stdout=log,stderr=subprocess.STDOUT,start_new_session=True) + try: + deadline=time.monotonic()+600 + while True: + if proc.poll() is not None:raise RuntimeError(f'server exited {proc.returncode}') + try: + with urllib.request.urlopen(endpoint+'/ready',timeout=2) as response: + if response.status==200:break + except OSError:pass + if time.monotonic()>deadline:raise TimeoutError('readiness timeout') + time.sleep(2) + subprocess.run([sys.executable,str(A/'heldout_quality270_concurrent.py'),'--fixtures',str(fixture),'--endpoint',endpoint,'--output',str(output)],check=True) + from heldout_quality270 import score + rows=[json.loads(l) for l in output.read_text().splitlines()] + assert len(rows)==36 and len({r['case'] for r in rows})==36 + by_case={r['case']:r for r in rows} + for case in f['cases']: + row=by_case[case['case']] + assert row['case']==case['case'] and row['input_sha256']==case['input_sha256'] and 'error' not in row + assert all(row[k]==v for k,v in score(row['text'],case['expected']).items()) + state.update(state='complete',cases=36,answer_correct=sum(r['answer_correct'] for r in rows),format_correct=sum(r['format_correct'] for r in rows)) + except BaseException as e: + state.update(state='failed',error=repr(e));raise + finally: + if proc.poll() is None: + os.killpg(proc.pid,signal.SIGTERM) + try:proc.wait(timeout=30) + except subprocess.TimeoutExpired:os.killpg(proc.pid,signal.SIGKILL);proc.wait() + state['server_exit']=proc.returncode + statepath.write_text(json.dumps(state,indent=2)+'\n') +print(json.dumps(state),flush=True) diff --git a/experiments/evidence270/run_atom270_allcore_pd_quality_pair.py b/experiments/evidence270/run_atom270_allcore_pd_quality_pair.py new file mode 100644 index 00000000..e2ced9c7 --- /dev/null +++ b/experiments/evidence270/run_atom270_allcore_pd_quality_pair.py @@ -0,0 +1,10 @@ +"""Run both frozen quality routes once; retain every answer including wrong ones.""" +import json,subprocess +from pathlib import Path +A=Path(__file__).resolve().parent +for mode in ['whole','stream']: + print('QUALITY_START',mode,flush=True) + subprocess.run(['bash',str(A/'run-quality270-pd-atom.sh'),'--mode',mode],check=True) + d=json.loads((A/f'atom270-qwen4b-allcore-pd-quality-{mode}-run.json').read_text()) + assert d['state']=='complete' and d['cases']==36 and d['server_exit']==0 + print('QUALITY_END',mode,d['answer_correct'],d['format_correct'],flush=True) diff --git a/experiments/evidence270/samecard-evidence-manifest.json b/experiments/evidence270/samecard-evidence-manifest.json new file mode 100644 index 00000000..a9cfe0c0 --- /dev/null +++ b/experiments/evidence270/samecard-evidence-manifest.json @@ -0,0 +1,28 @@ +{ + "research_event_id": "20b19f6ff2f04da283fd53d98024b4bf", + "raws": [ + { + "file": "atom270-qwen4b-samecard-retry1-a1.json", + "sha256": "0c7ef20fd373eb1993680aaab83d12c63ab0ed1d45098abca30332dfc6c9383f" + }, + { + "file": "atom270-qwen4b-samecard-retry1-b1-prefill0.json", + "sha256": "c8f9b4c29d4f583c9753785f67fc90fb35d0088abb1e8820c75972abada0ba77" + }, + { + "file": "atom270-qwen4b-samecard-retry1-b2-prefill0.json", + "sha256": "c8f9b4c29d4f583c9753785f67fc90fb35d0088abb1e8820c75972abada0ba77" + }, + { + "file": "atom270-qwen4b-samecard-retry1-a2.json", + "sha256": "0c7ef20fd373eb1993680aaab83d12c63ab0ed1d45098abca30332dfc6c9383f" + } + ], + "raw_archive": "Registered in local research-hub; remote synchronization not independently verified.", + "sources": { + "diagnose_atom270_direct.py": "28030bbf0ad20f4ba0c5d4c229a95bee90f5afcb7bf2a95b1af96b97f931f654", + "diagnose_atom270_samecard_retry1.py": "7a0b383797761558ea0c81ccea02bd12dfe43ea4c6537497826ccf52b08abe0f", + "qwen4-lane-fixtures.json": "98ecf202160213ae7591c69d3ba053dab8d786f8ac1fe81147f944881725a154" + }, + "fixture_source_sha256": "11d07b625862acc161aa95c34cd2d1087e9bd302477c01ff7e7f66677f9234f7" +} diff --git a/experiments/evidence270/selfcheck_allcore270_abba_audit.py b/experiments/evidence270/selfcheck_allcore270_abba_audit.py new file mode 100644 index 00000000..390a3af8 --- /dev/null +++ b/experiments/evidence270/selfcheck_allcore270_abba_audit.py @@ -0,0 +1,23 @@ +"""Historical raw failure must stay failed; inconsistent flags/timing must be rejected.""" +import copy,json +from pathlib import Path +from audit_atom270_allcore_abba import audit_arm +A=Path(__file__).resolve().parent +def subset(name): + d=json.loads((A/name).read_text());d['prompts']={k:v for k,v in d['prompts'].items() if k in ['512','8192']};d['references']={k:v for k,v in d['references'].items() if k in ['512:128','8192:128']} + d['warmups']=[r for r in d['warmups'] if '-'.join(r['id'].split('-')[-2:]) in ['512-128','8192-128']] + d['correctness_batches']=[b for b in d['correctness_batches'] if b['input_tokens'] in [512,8192] and b['output_tokens']==128] + d['batches']=[b for b in d['batches'] if b['input_tokens'] in [512,8192] and b['output_tokens_per_request']==128] + return d +bad=subset('atom270-qwen4b-tp1-b4-native.json');assert not audit_arm(bad)['parity_passed'] +good=subset('atom270-qwen4b-allcore-http.json');assert audit_arm(good)['parity_passed'] +for field in ['flag','time','token','missing']: + d=copy.deepcopy(good) + if field=='flag':d['correctness_batches'][0]['rows'][0]['tokens_equal']=False + elif field=='time':d['batches'][0]['summary']['output_tok_s']*=2 + elif field=='token':d['correctness_batches'][0]['rows'][0]['tokens'][0]+=1 + else:d['batches'].pop() + try:audit_arm(d) + except AssertionError:pass + else:raise AssertionError('Bad evidence accepted:'+field) +print('PASS: historical parity failure preserved; valid candidate accepted;4 corrupted cases rejected') diff --git a/experiments/evidence270/selfcheck_analyze_timeline270.py b/experiments/evidence270/selfcheck_analyze_timeline270.py new file mode 100644 index 00000000..cf41aa04 --- /dev/null +++ b/experiments/evidence270/selfcheck_analyze_timeline270.py @@ -0,0 +1,19 @@ +from analyze_timeline270 import summarize +trace={'remote':[dict(id=0,slot=0,input_tokens=128,start=0.,end=2.,error=False),dict(id=1,slot=1,input_tokens=128,start=0.,end=5.,error=False)],'decode':[dict(start=2.1,end=2.2,lanes=1,prefilling=1,ready=0,waiting=14,error=False),dict(start=5.1,end=5.3,lanes=2,prefilling=0,ready=0,waiting=14,error=False)]} +s=summarize(trace) +assert s['remote']['completed']==2 and s['remote']['completion_gap_s']['median']==3. +assert s['remote']['submit_to_completion_s']['median']==3.5 +assert s['decode']['lane_histogram']=={'1':1,'2':1} +assert s['decode']['calls_with_ready_prefill']==0 and s['decode']['calls_with_waiting']==2 +assert s['decode']['max_lanes']==2 +assert abs(s['decode']['host_call_s']['sum']-.3)<1e-8 +try:summarize({'remote':[], 'decode':[dict(trace['decode'][0],end=1.)]}) +except ValueError:pass +else:raise AssertionError('invalid interval accepted') +s2=summarize({'remote':[], 'decode':[]});assert s2['decode']['host_call_s']['median'] is None +print('PASS interval summaries, queue-state counts, empty data and malformed interval rejection') + +assert s['remote']['backlogged_completion_gap_s']['median']==3. +idle={'remote':[trace['remote'][0],dict(trace['remote'][1],start=4.)], 'decode':[]} +assert summarize(idle)['remote']['backlogged_completion_gap_s']['count']==0 +print('PASS completion gap excludes requests submitted after previous completion') diff --git a/experiments/evidence270/selfcheck_atom270_abba.py b/experiments/evidence270/selfcheck_atom270_abba.py new file mode 100644 index 00000000..af804b1f --- /dev/null +++ b/experiments/evidence270/selfcheck_atom270_abba.py @@ -0,0 +1,43 @@ +"""Synthetic fixtures test audit logic without archived hardware artifacts.""" +import copy,json +from pathlib import Path +from audit_atom270_abba import audit +def fixture(width): + d=dict(passed=True,parity_passed=True,slots_per_decoder=4,physical_atom=1,physical_gpus=0,physical_rngd=1, + context=9216,kv_blocks=5,tensor_parallel_size=1,decoder_cache=False,producer_cache=False, + producer_chunk=width,model='synthetic',dtype='bfloat16',torch='fixture',transformers='fixture', + harness_sha256='fixture',server_config={'servers':[['server.py','--width',str(width)]], + 'metadata':{'producer_chunk':width}},prompts={},references={},warmups=[],correctness_batches=[],batches=[]) + for n in (128,512): + d['prompts'][str(n)]=[3]*n + for o in (32,128): + tokens=[7]*o;ref=dict(tokens=tokens,text='fixture');d['references'][f'{n}:{o}']=ref + d['warmups'].append(dict(id=f'w-0-{n}-{o}',tokens_equal=True,tokens=tokens)) + d['correctness_batches'].append(dict(input_tokens=n,output_tokens=o,concurrency=16, + rows=[dict(id=str(i),tokens_equal=True,tokens=tokens) for i in range(16)])) + rows=[dict(id=str(i),input_tokens=n,output_tokens=o,text_equal=True,text='fixture', + first_sse_s=.25,e2e_s=.5,tpot_est_s=.25/(o-1)) for i in range(16)] + summary=dict(requests=16,output_tokens=16*o,all_text_equal=True,wall_s=1.,output_tok_s=16.*o, + median_first_sse_s=.25,median_e2e_s=.5,median_tpot_est_s=.25/(o-1)) + d['batches'].append(dict(input_tokens=n,output_tokens_per_request=o,concurrency=16,repeat=0,rows=rows,summary=summary)) + return d +wide=fixture(8192);narrow=fixture(1024) +binding=dict(visible_count=1,physical_to_visible={'2':0},opened=['/dev/rbln2']) +log='\n'.join(prefix+json.dumps(value) for prefix,value in [('ATOM270_COMPILED ',dict(batch_size=4,kv_blocks=5,context=9216,tensor_parallel_size=1)),('ATOM270_BINDING ',binding),('ATOM270_ACTIVE_LANES ',[0,10,5,1,0])]) +queue=dict(state='complete',cleanup_exit_code=0,runs=[dict(stem=f'r{i}',width=w,exit_code=0,binding=binding) for i,w in enumerate([8192,1024,1024,8192])]) +data=[wide,narrow,copy.deepcopy(narrow),copy.deepcopy(wide)];logs=[log]*4 +result=audit(queue,data,logs);assert result['correctness_responses']==256 and len(result['rows'])==4 +for mutation in ['incomplete','child','order','duplicate','binding','config','tokens','metrics']: + q=copy.deepcopy(queue);d=copy.deepcopy(data);ll=list(logs) + if mutation=='incomplete':q['state']='running' + elif mutation=='child':q['runs'][1]['exit_code']=1 + elif mutation=='order':q['runs'][0]['width']=1024 + elif mutation=='duplicate':q['runs'][1]['stem']='r0' + elif mutation=='binding':ll[1]=ll[1].replace('/dev/rbln2','/dev/rbln3') + elif mutation=='config':d[1]['server_config']['servers'][0].append('--unrelated') + elif mutation=='tokens':d[1]['references']['128:32']['tokens'][0]+=1 + elif mutation=='metrics':d[1]['batches'][0]['summary']['output_tok_s']*=2 + try:audit(q,d,ll) + except (AssertionError,ValueError):pass + else:raise AssertionError('Accepted corrupt '+mutation) +print('PASS synthetic ABBA fixture and eight corruption rejections; not hardware results') diff --git a/experiments/evidence270/selfcheck_atom270_direct.py b/experiments/evidence270/selfcheck_atom270_direct.py new file mode 100644 index 00000000..a94d48f8 --- /dev/null +++ b/experiments/evidence270/selfcheck_atom270_direct.py @@ -0,0 +1,12 @@ +"""Catch replay feeding a newly generated token or writing to the wrong KV position.""" +import importlib.util +from pathlib import Path +p=Path(__file__).with_name('diagnose_atom270_direct.py') +assert p.exists(), 'direct SDK replay missing' +s=importlib.util.spec_from_file_location('direct',p);m=importlib.util.module_from_spec(s);s.loader.exec_module(m) +assert m.decode_inputs([2], [7,8,9], 5, 2) == ([0,0,8,0],[0,0,6,0],[[4],[4],[2],[4]]) +assert m.decode_inputs([0,1,2,3], [7,8,9], 5, 1) == ([7]*4,[5]*4,[[0],[1],[2],[3]]) +try:m.decode_inputs([0], [7], 5, 0) +except AssertionError:pass +else:raise AssertionError('invalid output index accepted') +print('direct SDK replay selfcheck passed') diff --git a/experiments/evidence270/selfcheck_quality270_concurrent.py b/experiments/evidence270/selfcheck_quality270_concurrent.py new file mode 100644 index 00000000..cafc8a17 --- /dev/null +++ b/experiments/evidence270/selfcheck_quality270_concurrent.py @@ -0,0 +1,32 @@ +"""Real localhost requests prove cohort concurrency, fixed EOS policy and raw scoring.""" +import hashlib,json,tempfile,threading +from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer +from pathlib import Path +from heldout_quality270_concurrent import run +barrier=threading.Barrier(12) +class Handler(BaseHTTPRequestHandler): + def do_POST(self): + b=json.loads(self.rfile.read(int(self.headers['Content-Length']))) + assert b['ignore_eos'] is False and b['temperature']==0 and b['max_tokens']==128 + i=b['prompt'][0];barrier.wait(timeout=10) + data=json.dumps({'choices':[{'text':json.dumps({'answer':str(i)}),'token_ids':[i,151645],'finish_reason':'stop'}]}).encode() + self.send_response(200);self.send_header('Content-Length',str(len(data)));self.end_headers();self.wfile.write(data) + def log_message(self,*args):pass +server=ThreadingHTTPServer(('127.0.0.1',0),Handler);thread=threading.Thread(target=server.serve_forever);thread.start() +try: + with tempfile.TemporaryDirectory() as tmp: + p=Path(tmp);cases=[] + for i in range(36): + ids=[i];cases.append(dict(case=str(i),ids=ids,input_sha256=hashlib.sha256(json.dumps(ids,separators=(',',':')).encode()).hexdigest(),expected=str(i),target_tokens=(0,2048,8192)[i//12])) + (p/'fixture.json').write_text(json.dumps(dict(model='fixture',max_tokens=128,cases=cases))) + run(p/'fixture.json',f'http://127.0.0.1:{server.server_port}',p/'rows.jsonl') + rows=[json.loads(l) for l in (p/'rows.jsonl').read_text().splitlines()] + assert len(rows)==36 and all(r['answer_correct'] and r['format_correct'] and r['tokens']==[int(r['case']),151645] for r in rows) + for n in (0,2048,8192): + cohort=[r for r in rows if r['cohort']==n] + assert max(r['request_start_monotonic'] for r in cohort)