diff --git a/README.md b/README.md
index 5636ca0..65d24dc 100644
--- a/README.md
+++ b/README.md
@@ -202,7 +202,8 @@ ImageSharp 默认分配器会把像素拆成 4MB 块，大图上拿不到一整
| CI 架构 | Windows x64 / x86 / ARM64，Linux x64 / ARM64，macOS x64 / ARM64 |
| SIMD | .NET 10 运行时探测 AVX → AVX2 → AVX-512 / VNNI；无对应指令集或 ARM 时用 Vector/标量 |
| 输入 | 交错像素内存（默认 BGR24，也可 RGB24 / BGRA32 / RGBA32）；无图片路径、文件或图片库 API |
-| 设备 | CPU only，无 GPU |
+| 设备 | 默认 CPU；`net10.0` 另有可选 Vulkan 后端（`OcrBackend`），直连系统加载器：Windows `vulkan-1.dll`、Linux `libvulkan.so.1`、Android `libvulkan.so`；`netstandard2.0` 只有 CPU |
+| 安卓 | 目前通过开发用宿主 `test/Sdcb.SimdPaddleOCR.AndroidBench`（`net10.0-android`，引用 `net10.0` 库，adb 驱动）在骁龙 8 Gen 3 上跑 CPU 与 Vulkan，见 [`docs/vulkan-8gen3.md`](docs/vulkan-8gen3.md)；桌面的 Vulkan 路由和 shader 没有变 |
| NativeAOT | 裁剪发布时请保留核心程序集和所用模型程序集 |
## 许可证与第三方组件
diff --git a/README_EN.md b/README_EN.md
index 3afd1c1..358692b 100644
--- a/README_EN.md
+++ b/README_EN.md
@@ -202,7 +202,8 @@ ImageSharp's default allocator splits pixels into 4MB chunks, so large images do
| CI architectures | Windows x64 / x86 / ARM64, Linux x64 / ARM64, macOS x64 / ARM64 |
| SIMD | .NET 10 probes AVX → AVX2 → AVX-512 / VNNI at runtime; Vector/scalar when those ISAs are missing or on ARM |
| Input | Interleaved pixels (BGR24 by default; RGB24 / BGRA32 / RGBA32 also accepted); no image path, file, or image-library API |
-| Device | CPU only, no GPU |
+| Device | CPU by default; `net10.0` also has an optional Vulkan backend (`OcrBackend`) loading the system loader directly: `vulkan-1.dll` on Windows, `libvulkan.so.1` on Linux, `libvulkan.so` on Android; `netstandard2.0` is CPU only |
+| Android | Currently run through the dev host `test/Sdcb.SimdPaddleOCR.AndroidBench` (`net10.0-android`, references the `net10.0` library, driven over adb) on a Snapdragon 8 Gen 3, CPU and Vulkan; see [`docs/vulkan-8gen3.md`](docs/vulkan-8gen3.md). Desktop Vulkan routing and shaders are unchanged |
| NativeAOT | Keep the core assembly and the model assemblies you use when publishing trimmed |
## License and third-party components
diff --git a/docs/perf.md b/docs/perf.md
index 6881d20..ed14ac0 100644
--- a/docs/perf.md
+++ b/docs/perf.md
@@ -162,7 +162,7 @@ x64 / ARM64 本库 peak 大约少 **300 MB**。c 仍然略省，但更慢。Open
### Vulkan GPU（RTX 3080 Ti，`8d62a35`）
-同机同轮：HOME-MAIN 5800X + RTX 3080 Ti（驱动 581.80），`.NET 10.0.11`，`--engine sharp|vulkan --workers 4 --benchmark-kind simd --warmup 1`，同一 `dataset/` 100 张变尺寸图（对 GPU 最不利的逐图新 shape）。墙钟 n=99。**CPU 列是同轮重测**，不要拿上表 1.4.2 的 `97c1448` mean 硬接（那次 tiny 63.1 / small 200 / medium 585）。B580 另一台机、另一份尺子，见 [vulkan-b580.md](vulkan-b580.md)；Radeon 880M 核显见 [vulkan-880m.md](vulkan-880m.md)。Intel UHD 770 没有协作矩阵，`Auto` 仍走 CPU，显式 `Vulkan` 走较慢的无矩阵档，见 [vulkan-uhd770.md](vulkan-uhd770.md)。
+同机同轮：HOME-MAIN 5800X + RTX 3080 Ti（驱动 581.80），`.NET 10.0.11`，`--engine sharp|vulkan --workers 4 --benchmark-kind simd --warmup 1`，同一 `dataset/` 100 张变尺寸图（对 GPU 最不利的逐图新 shape）。墙钟 n=99。**CPU 列是同轮重测**，不要拿上表 1.4.2 的 `97c1448` mean 硬接（那次 tiny 63.1 / small 200 / medium 585）。B580 另一台机、另一份尺子，见 [vulkan-b580.md](vulkan-b580.md)；Radeon 880M 核显见 [vulkan-880m.md](vulkan-880m.md)。Intel UHD 770 没有协作矩阵，`Auto` 仍走 CPU，显式 `Vulkan` 走较慢的无矩阵档，见 [vulkan-uhd770.md](vulkan-uhd770.md)。骁龙 8 Gen 3（Adreno 750，安卓）同样没有协作矩阵，但无矩阵档三档都快于本机 CPU，`Auto` 走 GPU，见 [vulkan-8gen3.md](vulkan-8gen3.md)。
端到端（median ms/图，越低越好；加速 = CPU median / Vulkan median）：
@@ -227,6 +227,7 @@ JSON：`bench-out/local-5800x-c-{tiny,small,medium}-4w.json`。
| 本机 5800X | — | `97c1448` / `68a009a` | 墙钟/内存 `97c1448`；CER 左 1:4 后同机复测（1.4.2） |
| 本机 Vulkan | — | `8d62a35` | 5800X + 3080 Ti 同轮 sharp/vulkan 4w；tiny/small/medium；A/B 基线 `d6c260c` |
| 880M Vulkan | — | `2520c77` | 锐龙 AI 9 H365 + Radeon 880M 同轮 sharp/vulkan 4w；A/B 基线 `713fc03` / `d6c260c` |
+| 8 Gen 3 Vulkan | — | `vulkan-android-8gen3` 分支 | 真我 GT5 Pro，`test/Sdcb.SimdPaddleOCR.AndroidBench`，本机 CPU / Vulkan 交替 3 轮 4w；见 [vulkan-8gen3.md](vulkan-8gen3.md) |
推送或手动触发 [`.github/workflows/test.yml`](../.github/workflows/test.yml)，下载 `perf-report` artifact。本地同一套数据：
diff --git a/docs/vulkan-8gen3.md b/docs/vulkan-8gen3.md
new file mode 100644
index 0000000..e4621b4
--- /dev/null
+++ b/docs/vulkan-8gen3.md
@@ -0,0 +1,145 @@
+# SimdPaddleOCR — 骁龙 8 Gen 3（Adreno 750）Vulkan
+
+实测机：真我 GT5 Pro（RMX3888），骁龙 8 Gen 3（SM8650），Adreno 750，16 GB LPDDR5X 与 GPU 共享，Android 16（API 36）。驱动报 Vulkan 1.3.128，`driverVersion`=0x802fa029（GLES 同版本串 V@0762.41）。进程是安卓应用 `test/Sdcb.SimdPaddleOCR.AndroidBench`（`net10.0-android`，CoreCLR，.NET 10.0.11），引用库的 `net10.0` 构建：CPU 走 AdvSimd 内核，Vulkan 直连系统 `libvulkan.so`。数据集用仓库 `dataset/` 100 张，`--workers 4 --warmup 1`，选项与桌面 `Tests` 相同，n=99。
+
+**结论：Adreno 750 没有协作矩阵，compute subgroup 只能是 64 或 128，走无矩阵档（5.3 第 3 条）。原样上机时三档都算错（DET 概率图几乎全零，会话却报告 GPU 正常），修掉三处只在这颗 GPU 上暴露的问题后三档精度与本机 CPU 一致；再按 Adreno 调 GEMM 和两处路由，纯 GPU 快 14–33%。三档端到端都快于本机 CPU（tiny 约 1.3×，small 约 3.3×，medium 约 2.2×），所以 `Auto` 在“无协作矩阵且 subgroup 最小 64”的设备上改走 GPU。桌面 sg16 / sg32 / sg32l 的路由和 spv 没有改。**
+
+## 设备能力（`--caps` 探针）
+
+| 项 | 值 |
+|---|---|
+| 设备 | `Adreno (TM) 750`，vendor 0x5143，deviceId 0x43051401，`INTEGRATED_GPU`；加载的库是 `libvulkan.so` |
+| subgroup | 默认 64；`VK_EXT_subgroup_size_control`，`sgRange` **64–128**，`requiredSubgroupSizeStages` 含 compute |
+| 实际 lane 数 | 同一 `VkDevice.NewPipeline` 路径的探针 shader：请求 0 → `gl_SubgroupSize`=64，请求 64 → 64，请求 128 → 128；16 / 32 在范围外，按规范不能请求，没有请求 |
+| cooperative matrix | 没有 `VK_KHR_cooperative_matrix`（`vkGetPhysicalDeviceCooperativeMatrixPropertiesKHR` 不导出），协作矩阵列表为空 |
+| fp16 | `shaderFloat16`、16-bit storage 都有 |
+| 共享内存 | `maxComputeSharedMemorySize` = **32 KB**；`maxComputeWorkGroupInvocations` 1024 |
+| 绑定限制 | `maxStorageBufferRange` = **128 MB**；`minStorageBufferOffsetAlignment` = 64 B |
+| 内存 | heap0 15203 MB、heap1 4095 MB，都 `DEVICE_LOCAL`；有 DL\|HV\|HC 和 DL\|HV\|HC\|CACHED 类型（输入缓冲落前者，回读落后者） |
+| 队列 | family 0 `0x19f`×3（图形+计算），family 1 `0x2`×1（仅计算，被选中），family 2 `0x8`；三族 `timestampValidBits`=48，周期 52.08 ns，时间戳可用（探针 0.26 ms 一次 dispatch 读回正常）；`globalPriority` 未授予（0） |
+| push descriptor | 有 |
+| 峰值 | `--peak`：纯寄存器 fp32 FMA **2.09 TFLOPS**；f16vec4 FMA 只有 0.91 TFLOPS（没有翻倍）。GEMM 必须 fp32 累加，按 2.09 算 |
+
+`CoopGemm` 为假、`SubgroupMin` > 16，所以 `GpuGraphModel` 进 `_nocm`，不创建任何 `conv1x1_cm*` / `convk_cm*` 管线。锁不住 8 lane，所以是新加的 `_ncWide` 子档。
+
+## 走了哪一档
+
+5.3 第 3 条：无协作矩阵。大 GEMM（1×1 里没被 dot 接住的、MatMul）走 `gemm_nc` 的 `-DDIRECT` 构建（`gemm_nc_d` / `gemm_nc_ds`），不请求 subgroup 大小（驱动给 64），内核不用 subgroup 内置量；kxk 稠密卷积一律走直接卷积 `convk_dot`；`conv1x1_dot` / `mmdot_sk` / `conv_dw*` / 其余内核照旧。能锁 8 lane 的 UHD 770 仍走原来的 `gemm_nc` / `gemm_nc_s`（`requiredSubgroupSize=8`），spv 逐字节不变。
+
+## 上机后修掉的正确性问题
+
+逐层对比用 `--layers`（把 GPU 图截断到节点 k，读回该节点输出的统计量），同一份代码链接进桌面 GpuBench，在 3080 Ti 上取参照；再用 `--gemm` 把单个 GEMM 和 fp64 参考比。
+
+1. **`gemm_nc` 的寄存器预取在 Adreno 编译器下算错。** 第一处分叉就是第一个走 `gemm_nc_s` 的卷积；单测 6400×32×64 有 188871/204800 个输出错。fma 换乘加、if 换 select 都没用；去掉“下一块 A/B 先读进 `f16vec4` 数组”的预取、或把块缩到 32×32 / 4×4，就完全正确。改成 `-DDIRECT`：每块直接从全局读进共享内存。
+2. **标量常量上传只有 2 字节，Adreno 按绑定范围做越界检查。** `elem4` 按 `uint` 读 `b[0]`，桌面驱动不检查，Adreno 越界读回 0：CLS 头里乘常量那一步整个变 0，tiny CLS 只对 306/1022，转错方向的行又把 REC 拖到 CER 57%。`ConstF16` / `VecF16` 补齐到 16 字节的整数倍（只追加 0）。
+3. **im2col 矩阵超过 `maxStorageBufferRange`。** medium DET 960×960 的 7×7 卷积（M 57600，K 1568）im2col 有 180 MB，128 MB 之后全读成 0，medium 在大图上丢框（只检出 818 行，CER 13.2%）。放不进一个绑定时改走 `convk_dot`；仍会超范围的计划在运行前抛异常回退 CPU，不再返回半张零图。
+
+试过但确认不需要的：把 arena slab 起点按 `minStorageBufferOffsetAlignment`（64 B）对齐、SE 计数器按 64 B 步进、把描述符 range 截到 128 MB。在这个驱动上都不改变任何输出（逐图、逐像素相同），所以没有留下；按规范它们仍属于未对齐 / 超范围的用法，换驱动时值得再看。
+
+## 端到端（4 workers，median ms/图）
+
+同一时段 CPU / Vulkan 交替 3 轮（`ab.ps1`，轮间降温 60 s；tiny、small 第 2、1 轮的 CPU 因启动脚本竞态丢了，修好脚本后补测一轮，记在最后）。热状态（`PowerManager.CurrentThermalStatus`）tiny/small 为 2–3，medium 全程 3，机身明显发烫。
+
+| 模型 | 本机 CPU | Vulkan | 对 CPU |
+|---|---|---|---:|
+| tiny | 218.8 / 202.6 / 276.4 | 156.1 / 170.4 / 154.9 / 151.7 | 约 1.3× |
+| small | 867.1 / 785.0 / 811.7 | 251.3 / 247.3 / 246.2 / 255.1 | 约 3.3× |
+| medium | 3073.8 / 3294.1 / 3368.9 | 1466.4 / 1458.1 / 1467.0 | 约 2.2× |
+
+- 发热是最大的误差来源：同一版本 medium Vulkan 在热状态 2 时 median 833 ms（det_graph 213、rec_graph 558），热状态 3 时 1460 ms（385 / 984）。纯 GPU 的最小值基本不受影响，受影响的是持续负载下的频率。表里 CPU / Vulkan 都是同一热状态下交替测的。
+- 工作集峰值：medium CPU 约 1.60 GB，Vulkan 约 1.09 GB；small 约 1.01 / 0.99 GB；tiny 约 0.96 / 0.95 GB。
+
+分阶段 mean ms/图（第 3 轮）：
+
+| 模型 | 阶段 | CPU | Vulkan |
+|---|---|---:|---:|
+| tiny | det_graph | 118.9 | 20.1 |
+| tiny | rec_graph（CPU 为各线程累加） | 270.0 | 64.0 |
+| tiny | lines_wall | 83.2 | 84.5 |
+| small | det_graph | 384.4 | 40.2 |
+| small | lines_wall | 382.5 | 166.2 |
+| medium | det_graph | 1365.1 | 384.8 |
+| medium | lines_wall | 2036.3 | 1010.8 |
+
+tiny 的收益全在检测上，识别墙钟与 CPU 4 线程持平。Vulkan 运行时 `crop` 阶段（CPU 做透视裁剪）从 9–12 ms 涨到 16–27 ms，推测是 GPU 忙时 CPU 频率被压低，没有再查。
+
+### 精度
+
+| 模型 | 本机 CPU exact_lines / CER | Vulkan exact_lines / CER | 逐图 hash/detected/texts 一致 | 框数一致 |
+|---|---|---|---:|---:|
+| tiny | 742 / 2.37% | 743 / 2.40% | 78/100 | 99/100 |
+| small | 950 / 0.41% | 949 / 0.42% | 90/100 | 100/100 |
+| medium | 1004 / 0.14% | 1003 / 0.15% | 97/100 | 100/100 |
+
+本机 CPU 三档与 Windows 基线（742 / 950 / 1004，CER 2.37% / 0.41% / 0.14%）完全一致；CLS 三档都是满分。DET 对拍（同一输入进 CPU / Vulkan 会话）：tiny 10/10 图框数相同、概率图最大偏差 0.05–0.15（3080 Ti 对它自己的 CPU 是 0.03–0.18）；medium 6/6 图框数相同、0.03–0.07。
+
+并发（`--conc`，24 张 × 3）：tiny 4 / 8 线程、small 4 线程、medium 4 线程、`Auto` tiny 4 线程，`mismatches=0`，日志里没有 fallback；medium 4 线程工作集峰值 980 MB，没有被系统杀掉。
+
+## 纯 GPU
+
+`gprof.ps1`（逐 dispatch 时间戳，次数取最小，ms）。“调优前”是三处正确性修复之后、调优之前的版本；两列之间的每一步都做过同一时段交替 A/B，见各提交说明。
+
+| 用例 | 调优前 | 调优后 |
+|---|---:|---:|
+| medium DET 960×960 | 380.5 | 298.6 |
+| medium DET 640×960 | 270.1 | 205.2 |
+| medium REC 8×480 | 358.9 | 242.3 |
+| medium REC 1×320 | 40.0 | 34.4 |
+| small DET 960×960 | 64.5 | 52.7 |
+| small REC 8×480 | 72.6 | 54.5 |
+| tiny DET 960×960 | 27.9 | 23.7 |
+| tiny REC 8×480 | 15.9 | 12.3 |
+
+调优后热点：medium DET 是 `gemm_nc_ds` 92.6、`convk_dot` 88.9、`gemm_nc_d` 53.3、`conv1x1_dot` 19.4、`conv_dw4` 15.8、`conv_dw4t` 11.1；medium REC 是 `gemm_nc_ds` 110.9 + `gemm_nc_d` 95.7（合计 85%），其余是 `conv_dw4t` 20.6、`convk_dot` 6.3。
+
+相对峰值：热点 GEMM 形状合计 0.64 TFLOPS，是 fp32 FMA 峰值 2.09 的 31%；`convk_dot` 在 57600×2304×64 上约 0.56 TFLOPS。
+
+## 调优（每项都在手机上有收益才保留）
+
+1. **GEMM：fp16 共享内存 + BK 8。** `--gemmsweep` 在 11 个最热形状上交替比较（总 ms）：原 64×64 / BK16 / fp32 暂存 109.5；BK 8 83.1；128×128、每线程 8×4 84.1；BK 8 + fp16 暂存 **76.2**（保留）；同样配置换 128×64 / 64×128 块 74.6 / 75.0（噪声内，64×128 在 N=64 上浪费半块）。暂存的值本来就来自 fp16，逐位不变，e2e 与上一版 100/100。
+2. **两阶段空间均值在任意尺寸都用。** `reduce_hw` 一个工作组只算一个通道，在 Adreno 上按 C×2 字节跨步读；medium REC 8×480 269.7 → 243.3，medium DET 324.4 → 317.0。
+3. **kxk 稠密卷积一律直接卷积。** `CONVD_KMAX` 256 / 512 / 1024 / 2304：medium DET 315.2 / 312.3 / 304.0 / 299.3，small 64.6 / 60.7 / 52.8 / 52.8，tiny 32.8 / 29.9 / 23.7 / 23.7；640×960 216.9 → 206.2。直接卷积也不需要 im2col 暂存区。
+
+重扫但保持原值的：
+
+- K≤128 的 dot → GEMM 门槛（M×cout > 2¹² / 2¹⁴ / 2¹⁶ / 2¹⁸ / 2²⁰）：2¹²–2¹⁶ 相差 0.3 ms 以内，2¹⁸ 起 tiny DET 23.7 → 26.5，保持 2¹⁶。
+- `StreamCuts`（small 端到端，两轮）：现有切分 median 234 / 235；不切分 254 / 290；0.3/0.6/0.85 236 / 258；0.7 252 / 262。CPU ArgMax 和后面的 GPU 波重叠在手机上仍然赚，保持原样。
+
+## 试过但没留下的
+
+| 尝试 | 结果 |
+|---|---|
+| 9×9 depthwise 走共享内存分块的 `conv_dw4t`（sg32 档允许到 9） | medium DET 303.7 → 317.0，small 52.7 → 55.3，更慢 |
+| GEMM 不用共享内存，每线程直接按 f16vec4 读自己的 A/B 行 | 慢 2–4 倍；全展开的 8×8 版本又被编译器算错 |
+| GEMM BK 4 / BK 32 / 32×32 4×4 块 | 83.2 / 200.6 / 148.1（对 76.2） |
+| 共享内存按 f16vec4 向量读 | 76.8，与标量读相同 |
+| fp16 乘法 fp32 累加 | 没做：f16vec4 FMA 峰值只有 fp32 的 0.43 倍，且会改变数值 |
+| slab / 计数器按 64 B 对齐、描述符 range 截到 128 MB | 输出不变，没有收益 |
+
+## 留下的优化
+
+- GEMM 只到峰值的 31%。不用共享内存、全靠缓存的版本慢 2–4 倍，推测 SSBO 读在这里命中不了足够近的缓存；下一步可以试通过纹理路径读（需要给管线加 texel buffer 绑定），没有验证。
+- medium DET 的 `convk_dot` 已经第二（88.9 ms），都是 7×7 / 大 K 卷积，M 方向 4 行一组，可以试更大的寄存器块。
+- REC 的 `conv_dw4t` 20 ms、DET 的 `conv_dw4` 16 ms。
+- 发热：持续跑 medium 时热状态到 3，端到端从 833 ms 涨到 1460 ms。更少的 dispatch / 更短的峰值比单纯缩短纯 GPU 时间更值。
+- 规范层面：未对齐的绑定偏移（64 B）和超出 128 MB 的描述符 range 在这个驱动上无害，但属于无效用法。
+
+## `Auto`
+
+三档在所有交替轮次里 Vulkan 都快于本机 CPU，所以 `GpuBackend.UsesGpu` 对 `Auto` 在“没有协作矩阵、`SubgroupMin` ≥ 64”的设备上返回真（按能力判定，没有写死骁龙 / Adreno）。实测 `--backend auto`：usesGpu=True，与 `--backend vulkan` 逐图 100/100；`Auto` + `SIMD_OCR_BACKEND=cpu`：与 CPU 100/100。能锁 8 lane 的 UHD 770 和其它无矩阵设备（最小 subgroup < 64，例如不带协作矩阵驱动的 NVIDIA、Mali）没有在这条规则里，`Auto` 仍走 CPU，显式 `Vulkan` 才走 GPU。
+
+## 桌面路径
+
+sg16 / sg32 / sg32l 的 shader、spv 和选择条件都没动；全量重编后 `gemm_nc.spv`、`gemm_nc_s.spv` 和其余 spv 逐字节不变（`se_fused_dbg.spv` 删掉，不提交）。共享的 `GpuGraph` 改了两处会在桌面执行的地方：fp16 常量补齐到 16 字节、按 `maxStorageBufferRange` 的门槛和保护（桌面范围是 GB 级，不触发）。3080 Ti（驱动 581.80）同一时段对 `feature/2.0` 交替 2 轮，`--engine vulkan` median ms：tiny 18.0 / 19.9 → 18.5 / 20.7，small 26.7 / 30.3 → 29.2 / 28.5，medium 36.5 / 37.6 → 37.3 / 37.3（噪声内）；精度 741 / 950 / 1006 不变；逐图 100/100（三档），`--engine sharp` tiny / small 100/100。B580、880M、UHD 770 没法测：它们都不满足 `SubgroupMin` ≥ 64，走的路由没变，只有常量缓冲补齐这一处共享改动。
+
+## 复现
+
+```text
+cd test/Sdcb.SimdPaddleOCR.AndroidBench
+.\run.ps1 -Build -Install -Push -Bench "--caps" # 推 bench-out/models 和 dataset/，打印能力
+.\run.ps1 -Bench "--e2e --model medium --backend vulkan" # JSON 与 logcat 拉回 bench-out/android
+.\ab.ps1 -tag x -models tiny,small,medium -rounds 3 # CPU / Vulkan 交替
+.\gprof.ps1 -tag x # 纯 GPU
+```
+
+需要 .NET SDK ≥ 10.0.302 和 `android` workload、Android SDK platform 36、JDK 17/21。模型从手机存储读（medium 的 DET+REC 约 130 MB，不打进 APK）。
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuBackend.cs b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuBackend.cs
index cfe9d45..ab1b7c1 100644
--- a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuBackend.cs
+++ b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuBackend.cs
@@ -50,12 +50,14 @@ private static bool IsVulkanExplicit(OcrBackend backend) =>
/// Whether a session for this option runs on the GPU. Without an
/// fp16 cooperative matrix of the cm shaders' shape ()
- /// only an explicit Vulkan choice gets the no-coopmat GEMM tier: it trails
- /// the CPU on every model measured (docs/vulkan-uhd770.md), so Auto stays
- /// on the CPU there.
+ /// Auto takes the no-coopmat GEMM tier only where compute subgroups cannot
+ /// go below 64 lanes: there (Adreno 750) it beats the CPU on every model
+ /// measured (docs/vulkan-8gen3.md), while on UHD 770, which pins 8 lanes,
+ /// it trails the CPU (docs/vulkan-uhd770.md). Other no-coopmat devices
+ /// only get the GPU for an explicit Vulkan choice.
internal static bool UsesGpu(OcrBackend backend) =>
IsVulkanSelected(backend) && TryGetDevice() is { } dev
- && (dev.CoopGemm || IsVulkanExplicit(backend));
+ && (dev.CoopGemm || dev.SubgroupMin >= 64 || IsVulkanExplicit(backend));
/// Creates a session on the resolved backend; CPU on any GPU failure.
internal static IOcrSession CreateSession(CompiledModel compiled, OcrBackend backend)
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuGraph.cs b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuGraph.cs
index 7e475e1..bb3f296 100644
--- a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuGraph.cs
+++ b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/GpuGraph.cs
@@ -81,11 +81,14 @@ internal void Release()
private readonly bool _lite;
// No coopmat of the selected cm set's shape (VkDevice.CoopGemm):
// subgroup-free gemm_nc instead of every cm pipe
- // (Intel's compiler aborts the process on those shaders). Sessions only
- // reach this for an explicit OcrBackend.Vulkan — GpuBackend.UsesGpu.
+ // (Intel's compiler aborts the process on those shaders). Auto sessions
+ // reach this only on wave64-minimum devices — GpuBackend.UsesGpu.
private readonly bool _nocm;
// gemm_nc built with the unrolled no-act / relu / hardswish, single-output tail
private readonly VkPipeline? _pGemmNcS;
+ // no-coopmat tier on a device that cannot pin a narrow subgroup (Adreno
+ // wave64): direct-load GEMM plus the routing measured on Adreno 750
+ private readonly bool _ncWide;
// fp16 copies of constant tensors (weights, biases, scalars), lazy
private readonly Dictionary<(int, int, int, int), VkBuffer> _constF16 = new();
@@ -197,10 +200,13 @@ private GpuGraphModel(VkDevice dev, CompiledModel compiled)
{
// One tile; n64/n32 stay aliases so CmTile's sg16 cout split is
// not reused here. 8 lanes keep its 64 fp32 accumulators in
- // registers (UHD 770: SIMD16 spills, 4x slower).
+ // registers (UHD 770: SIMD16 spills, 4x slower). Without an
+ // 8-lane pin (wave64 Adreno) the register-prefetch form is
+ // miscompiled there; the direct-load build is exact and faster.
bool sg8 = dev.ComputeSubgroupSize && dev.SubgroupMin <= 8 && dev.SubgroupMax >= 8;
- _pConv1x1 = Pipe("gemm_nc", 6, 16, sg8 ? 8u : 0u);
- _pGemmNcS = Pipe("gemm_nc_s", 6, 16, sg8 ? 8u : 0u);
+ _ncWide = !sg8;
+ _pConv1x1 = sg8 ? Pipe("gemm_nc", 6, 16, 8u) : Pipe("gemm_nc_d", 6, 16);
+ _pGemmNcS = sg8 ? Pipe("gemm_nc_s", 6, 16, 8u) : Pipe("gemm_nc_ds", 6, 16);
_pConv1x1N64 = _pConv1x1;
_pConv1x1N32 = _pConv1x1;
}
@@ -314,7 +320,9 @@ private VkBuffer ConstF16(int tensorIndex, int padRows = 0, int rowElems = 0,
}
else
{
- alloc = Math.Max(n, padRows * rowElems);
+ // whole 16 B: kernels read scalars as a packed half pair / f16vec4,
+ // and Adreno bounds-checks every load against the binding range
+ alloc = (Math.Max(n, padRows * rowElems) + 7) / 8 * 8;
h = new Half[alloc];
for (int i = 0; i < n; i++) h[i] = (Half)f32[i];
}
@@ -347,7 +355,7 @@ private ReadOnlySpan CstF32(int tensorIndex) =>
private VkBuffer VecF16((int, int) key, float[] v)
{
if (_vecF16.TryGetValue(key, out VkBuffer? cached)) return cached;
- Half[] h = new Half[v.Length];
+ Half[] h = new Half[(v.Length + 7) / 8 * 8];
for (int i = 0; i < v.Length; i++) h[i] = (Half)v[i];
VkBuffer buf = _dev.NewStorageBuffer((ulong)h.Length * 2, hostVisible: false);
unsafe { fixed (Half* p = h) _dev.Upload(buf, p, (ulong)h.Length * 2); }
@@ -665,6 +673,9 @@ bool Perm(int ni, params int[] perm)
if (i == inIdx && inCinPad != 0)
sz = numel[i] / shapes[i][1] * inCinPad;
slabElems[i] = (sz + 7) / 8 * 8 + 128 * 64;
+ if (slabElems[i] * 2 > (long)_dev.MaxStorageRange)
+ throw new NotSupportedException(
+ $"tensor {i} ({sz * 2} B) exceeds maxStorageBufferRange {_dev.MaxStorageRange}");
cursor += slabElems[i];
}
// the im2col scratch sits past every slab; its size (and the arena
@@ -1407,10 +1418,15 @@ bool Absorbable(int cn, int srcPhys)
// sg32: implicit-GEMM coopmat conv instead (same tap
// addressing, tensor cores; any K, batched or not)
bool convkCm = _sg32 && cinIn % 8 == 0 && scalarBias == 0;
+ // the im2col matrix is one binding: past
+ // maxStorageBufferRange (Adreno: 128 MB) go direct
+ bool im2colFits = (long)M * Kp * 2 <= (long)_dev.MaxStorageRange;
+ // wide no-coopmat parts: direct at every K measured
+ // (Kp <= 2304 in these models), im2col+GEMM never won
if (convkCm || (cout % 4 == 0 && scalarBias == 0 && cinIn % 4 == 0
- && (nb > 1 || Kp <= (Environment.GetEnvironmentVariable(
+ && (nb > 1 || !im2colFits || Kp <= (Environment.GetEnvironmentVariable(
"SIMD_OCR_CONVD_KMAX") is string km
- ? int.Parse(km) : 1024))
+ ? int.Parse(km) : _ncWide ? int.MaxValue : 1024))
&& Environment.GetEnvironmentVariable("SIMD_OCR_NODCONV") == null))
{
// stem conv on the fp32 NCHW input: skip nchw2nhwc
@@ -1494,6 +1510,9 @@ bool Absorbable(int cn, int srcPhys)
if (nb > 1)
throw new NotSupportedException(
$"im2col conv path lacks batch support (n{ni})");
+ if (!im2colFits)
+ throw new NotSupportedException(
+ $"im2col matrix {M}x{Kp} exceeds maxStorageBufferRange (n{ni})");
maxIm2col = Math.Max(maxIm2col, (long)M * Kp);
Emit(_pIm2col, $"im2col n{ni} M{M} K{K}",
[(arena, SlotOf(node.Inputs[0]), 2),
@@ -1717,9 +1736,9 @@ int ConstIn(NodeRecord nd, int other)
seCtr += nb; // one ticket counter per batch
break;
}
- // lite parts take it at any size: reduce_hw's one-channel
- // workgroups read 2 bytes at a C*2 stride
- if (c % 4 == 0 && 256 % cv4 == 0 && (hw >= 4096 || _lite))
+ // lite and wide no-coopmat parts take it at any size:
+ // reduce_hw's one-channel workgroups read 2 bytes at a C*2 stride
+ if (c % 4 == 0 && 256 % cv4 == 0 && (hw >= 4096 || _lite || _ncWide))
{
// two-phase: S pixel partitions → fp32 partials → mean
int s = PartSplits(hw, c);
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc.comp b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc.comp
index e538ff9..9aa76e0 100644
--- a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc.comp
+++ b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc.comp
@@ -15,6 +15,12 @@
// wider SIMD spills them. Bindings and the 16-byte push constant match
// conv1x1_cm.
//
+// -DDIRECT (devices that cannot pin a narrow subgroup, e.g. Adreno wave64):
+// each tile goes straight from global memory into shared memory, no
+// register prefetch (Adreno's compiler miscomputes the prefetch form);
+// tiles stay fp16 in shared memory with BK 8 (Adreno 750: 30% faster than
+// fp32 tiles with BK 16), the tail stages through its own fp32 array.
+//
// flags: bit0 hasBias, bit1 hasResidual, bit3 dual output, bits[4:7] activation
// act: 0=none 1=relu 2=gelu 3=hardswish 4=sigmoid
// bit10 (1024): dual output stores the pre-activation value
@@ -25,6 +31,9 @@
#ifndef BN
#define BN 64u
#endif
+#if defined(DIRECT) && !defined(BK)
+#define BK 8u
+#endif
#ifndef BK
#define BK 16u
#endif
@@ -55,7 +64,15 @@ layout(push_constant) uniform PC { uint mDim; uint nDim; uint kDim; uint flags;
// As = S[0, BK*BM), Bs = S[BK*BM, ...); after the K loop each thread stages
// one accumulator row at S[tid*TN, +TN), so BK*(BM+BN) >= NTHR*TN
+#ifdef DIRECT
+shared float16_t S[BK * (BM + BN)];
+shared float St[NTHR * TN];
+#define LDS(v) float(v)
+#else
shared float S[BK * (BM + BN)];
+#define St S
+#define LDS(v) (v)
+#endif
#define As(i) S[i]
#define Bs(i) S[BK * BM + (i)]
@@ -90,6 +107,7 @@ void main()
for (uint i = 0u; i < TM * TN; i++) acc[i] = 0.0;
f16vec4 z = f16vec4(float16_t(0.0));
+#ifndef DIRECT
f16vec4 ra[AV];
f16vec4 rb[BV];
for (uint t = 0u; t < AV; t++)
@@ -106,6 +124,7 @@ void main()
uint gn = n0 + r, gk = cv * 4u;
rb[t] = (e < BN * AK && gn < p.nDim && gk < p.kDim) ? w[gn * kv + (gk >> 2u)] : z;
}
+#endif
for (uint k0 = 0u; k0 < p.kDim; k0 += BK)
{
@@ -115,7 +134,12 @@ void main()
if (e < BM * AK)
{
uint r = e / AK, cv = e - r * AK;
+#ifdef DIRECT
+ uint gm = m0 + r, gk = k0 + cv * 4u;
+ f16vec4 f = (gm < p.mDim && gk < p.kDim) ? x[gm * kv + (gk >> 2u)] : z;
+#else
vec4 f = vec4(ra[t]);
+#endif
As((cv * 4u + 0u) * BM + r) = f.x;
As((cv * 4u + 1u) * BM + r) = f.y;
As((cv * 4u + 2u) * BM + r) = f.z;
@@ -128,7 +152,12 @@ void main()
if (e < BN * AK)
{
uint r = e / AK, cv = e - r * AK;
+#ifdef DIRECT
+ uint gn = n0 + r, gk = k0 + cv * 4u;
+ f16vec4 f = (gn < p.nDim && gk < p.kDim) ? w[gn * kv + (gk >> 2u)] : z;
+#else
vec4 f = vec4(rb[t]);
+#endif
Bs((cv * 4u + 0u) * BN + r) = f.x;
Bs((cv * 4u + 1u) * BN + r) = f.y;
Bs((cv * 4u + 2u) * BN + r) = f.z;
@@ -136,6 +165,7 @@ void main()
}
}
barrier();
+#ifndef DIRECT
uint k1 = k0 + BK;
if (k1 < p.kDim)
{
@@ -154,12 +184,13 @@ void main()
rb[t] = (e < BN * AK && gn < p.nDim && gk < p.kDim) ? w[gn * kv + (gk >> 2u)] : z;
}
}
+#endif
for (uint kk = 0u; kk < BK; kk++)
{
float a[TM];
float b[TN];
- for (uint i = 0u; i < TM; i++) a[i] = As(kk * BM + ty * TM + i);
- for (uint j = 0u; j < TN; j++) b[j] = Bs(kk * BN + tx * TN + j);
+ for (uint i = 0u; i < TM; i++) a[i] = LDS(As(kk * BM + ty * TM + i));
+ for (uint j = 0u; j < TN; j++) b[j] = LDS(Bs(kk * BN + tx * TN + j));
for (uint i = 0u; i < TM; i++)
for (uint j = 0u; j < TN; j++)
acc[i * TN + j] = fma(a[i], b[j], acc[i * TN + j]);
@@ -202,14 +233,14 @@ void main()
// copies, which push the accumulators into spills
for (uint i = 0u; i < TM; i++)
{
- for (uint j = 0u; j < TN; j++) S[tid * TN + j] = acc[i * TN + j];
+ for (uint j = 0u; j < TN; j++) St[tid * TN + j] = acc[i * TN + j];
uint gm = mBase + i;
if (gm >= p.mDim) continue;
[[dont_unroll]] for (uint j = 0u; j < TN; j++)
{
uint gn = nBase + j;
if (gn >= p.nDim) continue;
- float v = S[tid * TN + j];
+ float v = St[tid * TN + j];
if (hasBias) v += float(bias[gn]);
if (hasRes) v += float(res[gm * p.nDim + gn]);
o[gm * p.nDim + gn] = actf(v, act, p.flags);
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc_d.spv b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc_d.spv
new file mode 100644
index 0000000..0a1a560
Binary files /dev/null and b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc_d.spv differ
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc_ds.spv b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc_ds.spv
new file mode 100644
index 0000000..2353c8e
Binary files /dev/null and b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Shaders/gemm_nc_ds.spv differ
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Vk.cs b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Vk.cs
index 83da462..7a315ee 100644
--- a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Vk.cs
+++ b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/Vk.cs
@@ -1,5 +1,6 @@
// Minimal Vulkan 1.1 P/Invoke for compute-only use (SimdPaddleOCR Vulkan backend).
-// Loads vulkan-1.dll (Windows) / libvulkan.so.1 (Linux) via SetDllImportResolver.
+// Loads vulkan-1.dll (Windows) / libvulkan.so.1 (Linux) / libvulkan.so (Android)
+// via SetDllImportResolver.
// Only the entry points and structs we need; all structs are blittable.
using System.Runtime.InteropServices;
@@ -11,6 +12,10 @@ internal static unsafe partial class Vk
private static int _resolverRegistered;
+ // Android ships only the unversioned NDK name (/system/lib64/libvulkan.so)
+ private static readonly string[] s_candidates = OperatingSystem.IsWindows()
+ ? ["vulkan-1.dll"] : ["libvulkan.so.1", "libvulkan.so"];
+
public static void RegisterResolver()
{
if (Interlocked.Exchange(ref _resolverRegistered, 1) != 0)
@@ -19,11 +24,20 @@ public static void RegisterResolver()
{
if (name != LibName)
return IntPtr.Zero;
- string candidate = OperatingSystem.IsWindows() ? "vulkan-1.dll" : "libvulkan.so.1";
- return NativeLibrary.TryLoad(candidate, out IntPtr h) ? h : IntPtr.Zero;
+ foreach (string candidate in s_candidates)
+ if (NativeLibrary.TryLoad(candidate, out IntPtr h))
+ {
+ LoadedLibrary = candidate;
+ return h;
+ }
+ throw new DllNotFoundException(
+ $"Vulkan loader not found (tried {string.Join(", ", s_candidates)})");
});
}
+ /// File name the resolver loaded, or null before the first call.
+ public static string? LoadedLibrary { get; private set; }
+
public const ulong WholeSize = ~0UL;
public const uint QueueFamilyIgnored = ~0u;
@@ -67,6 +81,10 @@ public uint MaxComputeSharedMemorySize
{
get { fixed (byte* p = LimitsAndSparse) return *(uint*)(p + 4 + 216); }
}
+ public uint MaxStorageBufferRange
+ {
+ get { fixed (byte* p = LimitsAndSparse) return *(uint*)(p + 4 + 28); }
+ }
}
[StructLayout(LayoutKind.Sequential)]
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/VkDevice.cs b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/VkDevice.cs
index a0519e5..e00c3f5 100644
--- a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/VkDevice.cs
+++ b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/VkDevice.cs
@@ -24,6 +24,7 @@ internal unsafe sealed class VkDevice : IDisposable
public bool Coop8x16x16; // same types, MxNxK 8x16x16 (Arc)
public uint SubgroupMin = 1, SubgroupMax = 128;
public uint MaxSharedMemory; // maxComputeSharedMemorySize (bytes per workgroup)
+ public ulong MaxStorageRange = uint.MaxValue; // maxStorageBufferRange (bytes per binding)
public int CoopM, CoopN, CoopK; // best fp16->fp32 subgroup coopmat config
public bool Storage16Bit; // storageBuffer16BitAccess + shaderInt16 enabled
public bool ShaderFloat16; // shaderFloat16 (VK_KHR_shader_float16_int8) enabled — required by fp16 shaders
@@ -107,6 +108,7 @@ public static VkDevice Create(uint deviceIndex = 0)
d.VendorId = props.VendorID;
d.TimestampPeriodNs = props.TimestampPeriodNs;
d.MaxSharedMemory = props.MaxComputeSharedMemorySize;
+ d.MaxStorageRange = props.MaxStorageBufferRange;
uint nqf = 0;
Vk.vkGetPhysicalDeviceQueueFamilyProperties(d.PhysDevice, &nqf, null);
diff --git a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/tools/build-shaders.bat b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/tools/build-shaders.bat
index 10b9624..f6ab17b 100644
--- a/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/tools/build-shaders.bat
+++ b/src/Sdcb.SimdPaddleOCR/Backends/Vulkan/tools/build-shaders.bat
@@ -51,3 +51,6 @@ rem nocm: subgroup-free fp16 GEMM (devices without 16x16x16 coopmat)
echo glslc gemm_nc
"%GLSLC%" -O --target-env=vulkan1.1 "%DIR%\gemm_nc.comp" -o "%DIR%\gemm_nc.spv" || exit /b 1
"%GLSLC%" -O --target-env=vulkan1.1 -DSIMPLE "%DIR%\gemm_nc.comp" -o "%DIR%\gemm_nc_s.spv" || exit /b 1
+rem nocm without a pinnable narrow subgroup (Adreno wave64): no register prefetch
+"%GLSLC%" -O --target-env=vulkan1.1 -DDIRECT "%DIR%\gemm_nc.comp" -o "%DIR%\gemm_nc_d.spv" || exit /b 1
+"%GLSLC%" -O --target-env=vulkan1.1 -DDIRECT -DSIMPLE "%DIR%\gemm_nc.comp" -o "%DIR%\gemm_nc_ds.spv" || exit /b 1
diff --git a/src/Sdcb.SimdPaddleOCR/Properties/InternalsVisibleTo.cs b/src/Sdcb.SimdPaddleOCR/Properties/InternalsVisibleTo.cs
index 6e3d50d..7dfcd06 100644
--- a/src/Sdcb.SimdPaddleOCR/Properties/InternalsVisibleTo.cs
+++ b/src/Sdcb.SimdPaddleOCR/Properties/InternalsVisibleTo.cs
@@ -2,3 +2,4 @@
[assembly: InternalsVisibleTo("Sdcb.SimdPaddleOCR.Tests")]
[assembly: InternalsVisibleTo("Sdcb.SimdPaddleOCR.UnitTests")]
[assembly: InternalsVisibleTo("Sdcb.SimdPaddleOCR.GpuBench")]
+[assembly: InternalsVisibleTo("Sdcb.SimdPaddleOCR.AndroidBench")]
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Bench.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/Bench.cs
new file mode 100644
index 0000000..fb93622
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/Bench.cs
@@ -0,0 +1,49 @@
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+/// Mode dispatch. Relative paths resolve against the app's external files dir.
+static class Bench
+{
+ internal static string OutDir = "";
+
+ internal static int Run(string[] args, string root, string outDir)
+ {
+ Paths.Root = root;
+ OutDir = outDir;
+ if (args.Length == 0) return Usage();
+ return args[0] switch
+ {
+ "--init" => 0,
+ "--caps" => Caps.Run(args),
+ "--e2e" => E2e.Run(args),
+ "--detprof" or "--recprof" => Prof.Run(args),
+ "--detcmp" or "--reccmp" => Compare.Run(args),
+ "--conc" => Prof.Conc(args),
+ "--rawbench" => Caps.RawBench(args),
+ "--gemm" => GemmTest.Run(args),
+ "--gemmsweep" => GemmTest.Sweep(args),
+ "--peak" => GemmTest.Peak(args),
+ // --layers [first] [last]
+ "--layers" => LayerProbe.Run(Paths.ModelPath(args[1], args[2]), int.Parse(args[3]), int.Parse(args[4]),
+ int.Parse(args[5]), args.Length > 6 ? int.Parse(args[6]) : 0,
+ args.Length > 7 ? int.Parse(args[7]) : int.MaxValue),
+ _ => Usage(),
+ };
+ }
+
+ static int Usage()
+ {
+ Console.WriteLine("""
+ --caps
+ --e2e --model tiny|small|medium --backend cpu|vulkan|auto [--workers 4] [--count N] [--warmup N] [--out name.json]
+ --detprof [reps] SIMD_OCR_GPU_PROF=1 for per-kernel times
+ --recprof [reps]
+ --detcmp [maxSide] CPU vs Vulkan prob map + boxes on the same input
+ --reccmp CPU vs Vulkan REC activations + CTC argmax
+ --conc [threads] [backend]
+ --gemm [flags] [tileM] [tileN] [reqSg] [reps]
+ --layers [first] [last]
+ --rawbench [pc uints...]
+ """);
+ return 2;
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Caps.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/Caps.cs
new file mode 100644
index 0000000..1f31e10
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/Caps.cs
@@ -0,0 +1,228 @@
+using System.Runtime.InteropServices;
+using Sdcb.SimdPaddleOCR.Backends.Vulkan;
+
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+/// Device capability probe (M0) and single-dispatch timing.
+static unsafe class Caps
+{
+ static string CompType(uint t) => t switch
+ {
+ 0 => "f16", 1 => "f32", 2 => "f64", 3 => "s8", 4 => "s16", 5 => "s32", 6 => "s64",
+ 7 => "u8", 8 => "u16", 9 => "u32", 10 => "u64", 1000141000 => "bf16",
+ _ => t.ToString(),
+ };
+
+ static string Scope(uint s) => s switch { 1 => "device", 2 => "workgroup", 3 => "subgroup", 5 => "queuefamily", _ => s.ToString() };
+
+ internal static int Run(string[] args)
+ {
+ Console.WriteLine($"phone: {Android.OS.Build.Manufacturer} {Android.OS.Build.Model} ({Android.OS.Build.Device}) " +
+ $"Android {Android.OS.Build.VERSION.Release} API {(int)Android.OS.Build.VERSION.SdkInt} soc={(OperatingSystem.IsAndroidVersionAtLeast(31) ? Android.OS.Build.SocModel : "?")}");
+ Console.WriteLine($"runtime: {RuntimeInformation.FrameworkDescription} rid={RuntimeInformation.RuntimeIdentifier} " +
+ $"cpus={Environment.ProcessorCount} advsimd={System.Runtime.Intrinsics.Arm.AdvSimd.IsSupported} " +
+ $"dotprod={System.Runtime.Intrinsics.Arm.Dp.IsSupported} vector={System.Numerics.Vector.Count}x{System.Numerics.Vector.IsHardwareAccelerated} " +
+ $"lib={typeof(PaddleOcrAll).Assembly.GetCustomAttributes(typeof(System.Runtime.Versioning.TargetFrameworkAttribute), false).OfType().FirstOrDefault()?.FrameworkName}");
+
+ using var dev = VkDevice.Create();
+ Console.WriteLine($"vulkan loader: {Vk.LoadedLibrary}");
+ Vk.VkPhysicalDeviceProperties props;
+ Vk.vkGetPhysicalDeviceProperties(dev.PhysDevice, &props);
+ uint api = props.ApiVersion;
+ Console.WriteLine($"device: {dev.DeviceName} vendor=0x{props.VendorID:x} deviceId=0x{props.DeviceID:x} type={props.DeviceType} " +
+ $"api={api >> 22 & 0x7f}.{api >> 12 & 0x3ff}.{api & 0xfff} driver=0x{props.DriverVersion:x8}");
+
+ byte* lim = props.LimitsAndSparse + 4; // native VkPhysicalDeviceLimits is 8-aligned
+ Console.WriteLine($"limits: maxComputeSharedMemorySize={*(uint*)(lim + 216)} maxWgInvocations={*(uint*)(lim + 232)} " +
+ $"maxWgSize={*(uint*)(lim + 236)}x{*(uint*)(lim + 240)}x{*(uint*)(lim + 244)} " +
+ $"maxWgCount={*(uint*)(lim + 220)}x{*(uint*)(lim + 224)}x{*(uint*)(lim + 228)}");
+ Console.WriteLine($"limits: maxStorageBufferRange={*(uint*)(lim + 28)} minStorageBufferOffsetAlignment={*(ulong*)(lim + 328)} " +
+ $"maxPushConstantsSize={*(uint*)(lim + 32)} maxMemoryAllocationCount={*(uint*)(lim + 36)} timestampPeriod={*(float*)(lim + 424)}ns");
+
+ uint nqf = 0;
+ Vk.vkGetPhysicalDeviceQueueFamilyProperties(dev.PhysDevice, &nqf, null);
+ var qf = stackalloc Vk.VkQueueFamilyProperties[(int)nqf];
+ Vk.vkGetPhysicalDeviceQueueFamilyProperties(dev.PhysDevice, &nqf, qf);
+ for (uint i = 0; i < nqf; i++)
+ Console.WriteLine($"queue family {i}: flags=0x{qf[i].QueueFlags:x} count={qf[i].QueueCount} timestampValidBits={qf[i].TimestampValidBits}{(i == dev.QueueFamily ? " <- selected" : "")}");
+ Console.WriteLine($"queue globalPriority={dev.QueuePriority}");
+
+ Console.WriteLine($"subgroup: default={dev.SubgroupSize} stages=0x{dev.SubgroupStages:x} ops=0x{dev.SubgroupOps:x} " +
+ $"sizeControl={dev.SubgroupSizeControl} sgRange={dev.SubgroupMin}-{dev.SubgroupMax} requiredStagesCompute={dev.ComputeSubgroupSize}");
+ Console.WriteLine($"features: storage16={dev.Storage16Bit} shaderFloat16={dev.ShaderFloat16} coopMatrix={dev.CoopMatrix} " +
+ $"push={dev.PushDescriptors} coop16x16x16={dev.Coop16x16x16} coop8x16x16={dev.Coop8x16x16} best={dev.CoopM}x{dev.CoopN}x{dev.CoopK}");
+ Console.WriteLine($"routing: Sg32Subgroup={dev.Sg32Subgroup} CoopGemm={dev.CoopGemm}");
+
+ for (int i = 0; i < (int)dev.MemProps.MemoryHeapCount; i++)
+ Console.WriteLine($"heap {i}: {dev.MemProps.HeapAt(i).Size >> 20} MB flags=0x{dev.MemProps.HeapAt(i).Flags:x}");
+ for (int i = 0; i < (int)dev.MemProps.MemoryTypeCount; i++)
+ Console.WriteLine($"memtype {i}: flags=0x{dev.MemProps.TypeAt(i).PropertyFlags:x} heap={dev.MemProps.TypeAt(i).HeapIndex}");
+
+ uint next = 0;
+ Vk.vkEnumerateDeviceExtensionProperties(dev.PhysDevice, null, &next, null);
+ var exts = new Vk.VkExtensionProperties[next];
+ fixed (Vk.VkExtensionProperties* pe = exts)
+ Vk.vkEnumerateDeviceExtensionProperties(dev.PhysDevice, null, &next, pe);
+ var names = new List();
+ for (int i = 0; i < next; i++)
+ fixed (byte* n = exts[i].ExtensionName) names.Add(new string((sbyte*)n));
+ Console.WriteLine($"extensions ({names.Count}): {string.Join(' ', names.OrderBy(x => x))}");
+
+ CoopList(dev);
+ SubgroupProbe(dev);
+ TimestampProbe(dev);
+ return 0;
+ }
+
+ static void CoopList(VkDevice dev)
+ {
+ byte[] fn = System.Text.Encoding.ASCII.GetBytes("vkGetPhysicalDeviceCooperativeMatrixPropertiesKHR\0");
+ IntPtr p;
+ fixed (byte* f = fn) p = Vk.vkGetInstanceProcAddr(dev.Instance, f);
+ if (p == IntPtr.Zero) { Console.WriteLine("coopmat: vkGetPhysicalDeviceCooperativeMatrixPropertiesKHR not exported"); return; }
+ var q = (delegate* unmanaged[Cdecl])p;
+ uint np = 0;
+ q(dev.PhysDevice, &np, null);
+ var cm = new Vk.VkCooperativeMatrixPropertiesKHR[np];
+ for (int i = 0; i < np; i++) cm[i].SType = VkConst.StCooperativeMatrixPropertiesKHR;
+ fixed (Vk.VkCooperativeMatrixPropertiesKHR* pc = cm) q(dev.PhysDevice, &np, pc);
+ Console.WriteLine($"coopmat: {np} configurations");
+ foreach (var c in cm)
+ Console.WriteLine($" coopmat {c.MSize}x{c.NSize}x{c.KSize} A={CompType(c.AType)} B={CompType(c.BType)} C={CompType(c.CType)} R={CompType(c.ResultType)} sat={c.SaturatingAccumulation} scope={Scope(c.Scope)}");
+ }
+
+ static byte[] Resource(string name)
+ {
+ using Stream s = typeof(Caps).Assembly.GetManifestResourceStream(name)!;
+ byte[] b = new byte[s.Length];
+ s.ReadExactly(b);
+ return b;
+ }
+
+ /// gl_SubgroupSize as seen by a pipeline built through the same
+ /// VkDevice.NewPipeline path the graph uses, per requiredSubgroupSize.
+ static void SubgroupProbe(VkDevice dev)
+ {
+ byte[] spv = Resource("sgsize.spv");
+ foreach (uint req in new uint[] { 0, 16, 32, 64, 128 })
+ {
+ if (req != 0 && !(dev.SubgroupSizeControl && dev.ComputeSubgroupSize && req >= dev.SubgroupMin && req <= dev.SubgroupMax))
+ {
+ Console.WriteLine($"requiredSubgroupSize={req}: not requested (outside sgRange {dev.SubgroupMin}-{dev.SubgroupMax} or no compute size control; invalid usage)");
+ continue;
+ }
+ try
+ {
+ var sp = dev.NewPipeline(dev.NewShaderModule(spv), 1, 4, req);
+ var set = dev.NewDescriptorSet(sp.SetLayout);
+ const int groups = 64;
+ var ob = dev.NewStorageBuffer(groups * 16, hostVisible: true, preferHost: true);
+ dev.BindBuffer(set, 0, ob);
+ IntPtr cmd = dev.NewCommandBuffer();
+ IntPtr f = dev.NewFence();
+ var begin = new Vk.VkCommandBufferBeginInfo { SType = VkConst.StCommandBufferBeginInfo };
+ Vk.Check(Vk.vkBeginCommandBuffer(cmd, &begin), "begin");
+ Vk.vkCmdBindPipeline(cmd, VkConst.BindPointCompute, sp.Pipeline);
+ Vk.vkCmdBindDescriptorSets(cmd, VkConst.BindPointCompute, sp.Layout, 0, 1, &set, 0, null);
+ Vk.vkCmdDispatch(cmd, groups, 1, 1);
+ Vk.Check(Vk.vkEndCommandBuffer(cmd), "end");
+ dev.Submit(cmd, f); dev.WaitFence(f);
+ uint* o = (uint*)ob.Map();
+ var sizes = new HashSet();
+ for (int g = 0; g < groups; g++) sizes.Add($"gl_SubgroupSize={o[g * 4]} numSubgroups={o[g * 4 + 1]} lastSubgroupId={o[g * 4 + 2]} lastLane={o[g * 4 + 3]}");
+ ob.Unmap();
+ Console.WriteLine($"requiredSubgroupSize={req}: {string.Join(" | ", sizes)}");
+ dev.DestroyFence(f);
+ ob.Free();
+ }
+ catch (Exception ex)
+ {
+ Console.WriteLine($"requiredSubgroupSize={req}: FAILED {ex.Message}");
+ }
+ }
+ }
+
+ static void TimestampProbe(VkDevice dev)
+ {
+ byte[] spv = Resource("sgsize.spv");
+ var sp = dev.NewPipeline(dev.NewShaderModule(spv), 1, 4, 0);
+ var set = dev.NewDescriptorSet(sp.SetLayout);
+ var ob = dev.NewStorageBuffer(1 << 20, hostVisible: false);
+ dev.BindBuffer(set, 0, ob);
+ IntPtr cmd = dev.NewCommandBuffer();
+ IntPtr f = dev.NewFence();
+ var qci = new Vk.VkQueryPoolCreateInfo { SType = 11, QueryType = 2, QueryCount = 2 };
+ Vk.Check(Vk.vkCreateQueryPool(dev.Device, &qci, null, out IntPtr qp), "qp");
+ var begin = new Vk.VkCommandBufferBeginInfo { SType = VkConst.StCommandBufferBeginInfo };
+ Vk.Check(Vk.vkBeginCommandBuffer(cmd, &begin), "begin");
+ Vk.vkCmdResetQueryPool(cmd, qp, 0, 2);
+ Vk.vkCmdBindPipeline(cmd, VkConst.BindPointCompute, sp.Pipeline);
+ Vk.vkCmdBindDescriptorSets(cmd, VkConst.BindPointCompute, sp.Layout, 0, 1, &set, 0, null);
+ Vk.vkCmdWriteTimestamp(cmd, VkConst.PipelineStageBottomOfPipe, qp, 0);
+ Vk.vkCmdDispatch(cmd, 16384, 1, 1);
+ Vk.vkCmdWriteTimestamp(cmd, VkConst.PipelineStageBottomOfPipe, qp, 1);
+ Vk.Check(Vk.vkEndCommandBuffer(cmd), "end");
+ long t0 = System.Diagnostics.Stopwatch.GetTimestamp();
+ dev.Submit(cmd, f); dev.WaitFence(f);
+ double wall = System.Diagnostics.Stopwatch.GetElapsedTime(t0).TotalMilliseconds;
+ ulong* ts = stackalloc ulong[2];
+ var r = Vk.vkGetQueryPoolResults(dev.Device, qp, 0, 2, 16, ts, 8, 1 | 2);
+ Console.WriteLine($"timestamp probe: result={r} t0={ts[0]} t1={ts[1]} delta={(ts[1] - ts[0]) * dev.TimestampPeriodNs / 1e6:F4} ms (wall {wall:F3} ms, 16384 WGs x 256)");
+ Vk.vkDestroyQueryPool(dev.Device, qp, null);
+ dev.DestroyFence(f);
+ ob.Free();
+ }
+
+ // --rawbench [pc uints...]
+ internal static int RawBench(string[] args)
+ {
+ using var dev = VkDevice.Create();
+ byte[] spv = File.ReadAllBytes(Paths.Path_(args[1]));
+ int nbind = int.Parse(args[2]);
+ uint gx = uint.Parse(args[3]), gy = uint.Parse(args[4]);
+ int reps = int.Parse(args[5]);
+ ulong bytes = ulong.Parse(args[6]) << 20;
+ uint reqSg = uint.Parse(args[7]);
+ uint[] pcs = args.Skip(8).Select(uint.Parse).ToArray();
+ int pcBytes = Math.Max(16, pcs.Length * 4);
+ var rp = dev.NewPipeline(dev.NewShaderModule(spv), nbind, pcBytes, reqSg);
+ var set = dev.NewDescriptorSet(rp.SetLayout);
+ var bufs = new List();
+ for (int b = 0; b < nbind; b++)
+ {
+ var buf = dev.NewStorageBuffer(bytes, hostVisible: false);
+ dev.Zero(buf);
+ dev.BindBuffer(set, (uint)b, buf);
+ bufs.Add(buf);
+ }
+ IntPtr cmd = dev.NewCommandBuffer();
+ IntPtr f = dev.NewFence();
+ var qci = new Vk.VkQueryPoolCreateInfo { SType = 11, QueryType = 2, QueryCount = 2 };
+ Vk.Check(Vk.vkCreateQueryPool(dev.Device, &qci, null, out IntPtr qp), "qp");
+ uint[] pcv = new uint[pcBytes / 4];
+ pcs.CopyTo(pcv, 0);
+ double best = double.MaxValue;
+ ulong* ts = stackalloc ulong[2];
+ for (int r = 0; r < reps; r++)
+ {
+ var begin = new Vk.VkCommandBufferBeginInfo { SType = VkConst.StCommandBufferBeginInfo };
+ Vk.Check(Vk.vkResetCommandBuffer(cmd, 0), "reset");
+ Vk.Check(Vk.vkBeginCommandBuffer(cmd, &begin), "begin");
+ Vk.vkCmdResetQueryPool(cmd, qp, 0, 2);
+ Vk.vkCmdBindPipeline(cmd, VkConst.BindPointCompute, rp.Pipeline);
+ Vk.vkCmdBindDescriptorSets(cmd, VkConst.BindPointCompute, rp.Layout, 0, 1, &set, 0, null);
+ fixed (uint* pp = pcv) Vk.vkCmdPushConstants(cmd, rp.Layout, VkConst.StageComputeShader, 0, (uint)pcBytes, pp);
+ Vk.vkCmdWriteTimestamp(cmd, VkConst.PipelineStageBottomOfPipe, qp, 0);
+ Vk.vkCmdDispatch(cmd, gx, gy, 1);
+ Vk.vkCmdWriteTimestamp(cmd, VkConst.PipelineStageBottomOfPipe, qp, 1);
+ Vk.Check(Vk.vkEndCommandBuffer(cmd), "end");
+ dev.Submit(cmd, f); dev.WaitFence(f);
+ Vk.vkGetQueryPoolResults(dev.Device, qp, 0, 2, 16, ts, 8, 1 | 2);
+ best = Math.Min(best, (ts[1] - ts[0]) * dev.TimestampPeriodNs / 1e6);
+ }
+ Console.WriteLine($"rawbench best={best:F4} ms");
+ Vk.vkDestroyQueryPool(dev.Device, qp, null);
+ foreach (var b in bufs) b.Free();
+ return 0;
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Compare.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/Compare.cs
new file mode 100644
index 0000000..553bcce
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/Compare.cs
@@ -0,0 +1,98 @@
+using Sdcb.SimdPaddleOCR.Backends.Vulkan;
+using Sdcb.SimdPaddleOCR.OnnxSharp;
+
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+/// Same input through the CPU and Vulkan paths on the phone.
+static class Compare
+{
+ internal static int Run(string[] args) => args[0] == "--detcmp" ? Det(args) : Rec(args);
+
+ // --detcmp [maxSide]
+ static int Det(string[] args)
+ {
+ string model = args[1];
+ int maxSide = args.Length > 3 ? int.Parse(args[3]) : 960;
+ string[] files = args[2] == "all" ? Paths.DatasetFiles()
+ : int.TryParse(args[2], out int n) ? Paths.DatasetFiles().Take(n).ToArray()
+ : [Paths.Path_(args[2])];
+ var mdl = Model.Load(File.ReadAllBytes(Paths.ModelPath(model, "det")));
+ var compiled = new CompiledModel(mdl, intraOpThreads: 4);
+ var opts = new PaddleOcrDetectorOptions();
+ var scratch = new DbPostprocess.Workspace();
+ using var sCpu = OcrSessionFactory.Create(compiled, OcrBackend.Cpu);
+ using var sGpu = OcrSessionFactory.Create(compiled, OcrBackend.Vulkan);
+ Console.WriteLine($"cpu session={sCpu.GetType().Name} gpu session={sGpu.GetType().Name}");
+ int sameBoxes = 0;
+ foreach (string f in files)
+ {
+ var (bgr, w, h) = Paths.LoadBgr(f);
+ var size = PPOCRPreprocess.ComputeDetSize(w, h, maxSide);
+ int W = size.Width, H = size.Height;
+ float[][] maps = new float[2][];
+ int[] nbox = new int[2];
+ string[] boxStr = new string[2];
+ IOcrSession[] ss = [sCpu, sGpu];
+ for (int k = 0; k < 2; k++)
+ {
+ IOcrSession s = ss[k];
+ s.Reshape([1, 3, H, W]);
+ PPOCRPreprocess.Det(bgr, w, h, w * 3, W, H, s.InputData, s.ResizeWorkspace, 1, s.InputIsNhwc,
+ ImagePixelFormat.Bgr24);
+ maps[k] = s.RunInternal(s.InputData).ToArray();
+ var boxes = DbPostprocess.Run(maps[k], W, H, opts, w, h, size.WidthRatio, size.HeightRatio, scratch);
+ nbox[k] = boxes.Length;
+ boxStr[k] = string.Join(";", boxes.Select(b => $"{b.X1:F0},{b.Y1:F0},{b.X3:F0},{b.Y3:F0}"));
+ }
+ float[] pc = maps[0], pg = maps[1];
+ double md = 0, sum = 0; int up = 0, dn = 0, nz = 0;
+ for (int i = 0; i < pc.Length; i++)
+ {
+ double d = Math.Abs(pc[i] - pg[i]);
+ md = Math.Max(md, d); sum += d;
+ if (pg[i] != 0) nz++;
+ if (pc[i] < 0.3f && pg[i] >= 0.3f) up++;
+ if (pc[i] >= 0.3f && pg[i] < 0.3f) dn++;
+ }
+ bool same = boxStr[0] == boxStr[1];
+ if (same) sameBoxes++;
+ Console.WriteLine($"{Path.GetFileName(f)} {W}x{H}: maxAbs={md:F4} meanAbs={sum / pc.Length:E2} gpuNonZero={nz}/{pg.Length} flips(0.3) up={up} down={dn} boxes cpu={nbox[0]} gpu={nbox[1]} {(same ? "same" : "DIFF")} gpuAlive={((sGpu as GpuSession)?.GpuAlive.ToString() ?? "n/a")}");
+ }
+ Console.WriteLine($"detcmp {model}: identical box lists {sameBoxes}/{files.Length}");
+ return 0;
+ }
+
+ // --reccmp : random input, activations before the CTC projection
+ static int Rec(string[] args)
+ {
+ string model = args[1];
+ int nb = int.Parse(args[2]), W = int.Parse(args[3]);
+ var mdl = Model.Load(File.ReadAllBytes(Paths.ModelPath(model, "rec")));
+ var compiled = new CompiledModel(mdl, intraOpThreads: 4);
+ int[] shape = [nb, 3, 48, W];
+ var inp = new float[nb * 3 * 48 * W];
+ var rr = new Random(1234);
+ for (int i = 0; i < inp.Length; i++) inp[i] = (float)(rr.NextDouble() * 2 - 1);
+ var cpu = new InferenceSession(compiled);
+ cpu.Reshape(shape);
+ if (!cpu.TryRunUntilCtcProjection(inp, out CtcProjectionOperands ops))
+ { Console.WriteLine("cpu: no CTC projection tail"); return 2; }
+ float[] a = ops.Activations.ToArray();
+ int mm = ops.MatMulNodeIndex;
+ using var dev = VkDevice.Create();
+ using var gg = new GpuDetGraph(dev, compiled);
+ float[] g = gg.Run(shape, inp, mm, checked((int)mdl.Nodes[mm].Inputs[0])).ToArray();
+ double md = 0; int nz = 0;
+ for (int i = 0; i < a.Length; i++) { md = Math.Max(md, Math.Abs(a[i] - g[i])); if (g[i] != 0) nz++; }
+ int rows = ops.RowCount;
+ int[] ic = new int[rows], ig = new int[rows];
+ float[] sc = new float[rows], sg = new float[rows];
+ Kernels.MatMul.TryArgMax(a, ops.Weights, ops.Bias, ic, sc, ops.Batch, ops.Rows, ops.Inner, ops.Columns, ops.PackedWeights, 4);
+ Kernels.MatMul.TryArgMax(g, ops.Weights, ops.Bias, ig, sg, ops.Batch, ops.Rows, ops.Inner, ops.Columns, ops.PackedWeights, 4);
+ int bad = 0, badText = 0;
+ for (int r = 0; r < rows; r++)
+ if (ic[r] != ig[r]) { bad++; if (ic[r] != 0) badText++; }
+ Console.WriteLine($"reccmp {model} {nb}x{W}: act n={a.Length} maxAbs={md:F4} gpuNonZero={nz}/{g.Length} argmaxDiff={bad}/{rows} (non-blank {badText})");
+ return 0;
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/E2e.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/E2e.cs
new file mode 100644
index 0000000..04b6b05
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/E2e.cs
@@ -0,0 +1,168 @@
+using System.Diagnostics;
+using System.Runtime.InteropServices;
+using System.Text.Encodings.Web;
+using System.Text.Json;
+using System.Text.Json.Nodes;
+using Sdcb.SimdPaddleOCR.OnnxSharp;
+using Sdcb.SimdPaddleOCR.Tests;
+
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+///
+/// End-to-end dataset run with the desktop Tests harness's options and JSON
+/// layout (rows: file / total_ms / hash / detected / texts / stage_ms ...),
+/// so cmp.ps1, steady.ps1 and Tests --summarize work on the pulled file.
+///
+static class E2e
+{
+ internal static int Run(string[] args)
+ {
+ string model = "tiny", backendName = "cpu";
+ int workers = 4, warmup = 1;
+ int? count = null;
+ string? outName = null;
+ for (int i = 1; i < args.Length; i++)
+ {
+ string Next() => args[++i];
+ switch (args[i])
+ {
+ case "--model": model = Next(); break;
+ case "--backend": backendName = Next(); break;
+ case "--workers": workers = int.Parse(Next()); break;
+ case "--count": count = int.Parse(Next()); break;
+ case "--warmup": warmup = int.Parse(Next()); break;
+ case "--out": outName = Next(); break;
+ default: throw new ArgumentException($"unknown argument {args[i]}");
+ }
+ }
+ OcrBackend backend = Paths.ParseBackend(backendName);
+ string metadataPath = Path.Combine(Paths.Dataset, "metadata.json");
+ string[] files = Paths.DatasetFiles();
+ if (files.Length != 100) throw new InvalidOperationException($"expected 100 images, got {files.Length}");
+ if (count is { } c) files = files.Take(c).ToArray();
+
+ var decoded = files.Select(f => (Name: Path.GetFileName(f), Img: Paths.LoadBgr(f))).ToArray();
+
+ const int CacheEntries = 32;
+ using var ocr = PaddleOcrAll.Load(Paths.ModelPath(model, "det"), Paths.ModelPath(model, "cls"),
+ Paths.ModelPath(model, "rec"), Paths.DictPath(model), new PaddleOcrOptions
+ {
+ LineWorkerCount = workers,
+ Detector = new PaddleOcrDetectorOptions { MaxSessionCacheEntries = CacheEntries, Backend = backend },
+ Recognizer = new PaddleOcrRecognizerOptions { AdaptiveWidth = true, TargetWidth = 320, Backend = backend },
+ Classifier = new PaddleOcrClassifierOptions { Backend = backend },
+ });
+ PipelineProfiler.Enable(true);
+ var prev = PipelineProfiler.Snapshot();
+ bool gpu = Backends.Vulkan.GpuBackend.UsesGpu(backend);
+ double wsLoaded = WorkingSetMb();
+ Console.WriteLine($"loaded model={model} backend={backendName} usesGpu={gpu} workers={ocr.EffectiveLineWorkerCount}/{workers} ws={wsLoaded:F1}MB");
+
+ int warmupCount = Math.Min(warmup, Math.Max(0, decoded.Length - 1));
+ var rows = new List();
+ var boxes = new List();
+ double wsPeak = wsLoaded;
+ for (int index = 0; index < decoded.Length; index++)
+ {
+ var (name, (bgr, w, h)) = decoded[index];
+ var sw = Stopwatch.StartNew();
+ PaddleOcrResult r = ocr.Run(bgr, w, h, w * 3);
+ sw.Stop();
+ var cur = PipelineProfiler.Snapshot();
+ var stageMs = new Dictionary();
+ var stageCalls = new Dictionary();
+ for (int s = 0; s < PipelineProfiler.StageCount; s++)
+ {
+ stageMs[PipelineProfiler.StageNames[s]] = cur[s].Milliseconds - prev[s].Milliseconds;
+ stageCalls[PipelineProfiler.StageNames[s]] = cur[s].Calls - prev[s].Calls;
+ }
+ prev = cur;
+ double ws = WorkingSetMb();
+ wsPeak = Math.Max(wsPeak, ws);
+ rows.Add(new BenchmarkRow
+ {
+ File = name, Width = w, Height = h, Warmup = index < warmupCount,
+ TotalMs = sw.Elapsed.TotalMilliseconds, StageMs = stageMs, StageCalls = stageCalls,
+ Detected = r.DetectedCount, Lines = r.Lines.Length, Hash = $"{r.PackedTextHash:x16}",
+ Texts = r.Lines.Select(x => x.Text).ToArray(),
+ Rotations = r.Lines.Select(x => x.AppliedRotationDegrees).ToArray(),
+ WorkingSetMb = ws,
+ });
+ boxes.Add(r.Lines.Select(x => Aabb(x.Box)).ToArray());
+ Console.WriteLine($"{index + 1}/{decoded.Length} {name} total={sw.Elapsed.TotalMilliseconds:F3} det={stageMs["det_graph"]:F2} lines={r.Lines.Length} ws={ws:F1}MB");
+ }
+
+ var meta = new JsonObject
+ {
+ ["mode"] = backendName == "cpu" ? "sharp" : backendName,
+ ["benchmark"] = true,
+ ["benchmarkKind"] = "simd",
+ ["model"] = model,
+ ["workers"] = workers,
+ ["backend"] = backendName,
+ ["usesGpu"] = gpu,
+ ["effectiveWorkers"] = ocr.EffectiveLineWorkerCount,
+ ["cacheEntries"] = CacheEntries,
+ ["rid"] = RuntimeInformation.RuntimeIdentifier,
+ ["os"] = RuntimeInformation.OSDescription,
+ ["arch"] = RuntimeInformation.OSArchitecture.ToString().ToLowerInvariant(),
+ ["cpu"] = Environment.ProcessorCount,
+ ["machine"] = $"{Android.OS.Build.Manufacturer} {Android.OS.Build.Model}",
+ ["timestamp"] = DateTimeOffset.Now.ToString("yyyy-MM-ddTHH:mm:sszzz"),
+ ["pixelFormat"] = "bgr",
+ ["working_set_mb_loaded"] = wsLoaded,
+ ["working_set_mb_peak"] = wsPeak,
+ ["working_set_mb_last"] = rows.Count > 0 ? rows[^1].WorkingSetMb : wsLoaded,
+ ["advSimdSupported"] = System.Runtime.Intrinsics.Arm.AdvSimd.IsSupported,
+ ["effectiveIsa"] = System.Runtime.Intrinsics.Arm.AdvSimd.IsSupported ? "advsimd" : "vector",
+ ["sampleCount"] = decoded.Length,
+ ["framework"] = RuntimeInformation.FrameworkDescription,
+ ["libraryTfm"] = typeof(PaddleOcrAll).Assembly
+ .GetCustomAttributes(typeof(System.Runtime.Versioning.TargetFrameworkAttribute), false)
+ .OfType().FirstOrDefault()?.FrameworkName,
+ ["thermal"] = Thermal(),
+ };
+ var accuracy = BenchSummary.ComputeAccuracy(rows, metadataPath, boxes);
+ if (accuracy is not null) meta["accuracy"] = BenchSummary.AccuracyNode(accuracy);
+ JsonObject doc = BenchSummary.WrapWithMeta(rows, meta);
+ string outPath = Path.Combine(Bench.OutDir, outName ?? $"e2e-{model}-{backendName}.json");
+ File.WriteAllText(outPath, doc.ToJsonString(new JsonSerializerOptions
+ {
+ WriteIndented = true,
+ Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping,
+ }));
+ Console.WriteLine($"saved: {outPath}");
+ BenchSummary.Print(BenchSummary.Parse(outPath, null, metadataPath));
+ var measured = rows.Where(r => !r.Warmup).Select(r => r.TotalMs).OrderBy(x => x).ToArray();
+ var tail = rows.Where(r => !r.Warmup).TakeLast(50).Select(r => r.TotalMs).OrderBy(x => x).ToArray();
+ if (measured.Length > 0 && tail.Length > 0)
+ Console.WriteLine($"steady: all-med {measured[measured.Length / 2]:F1} last50-med {tail[tail.Length / 2]:F1} ws-peak {wsPeak:F0}MB thermal={Thermal()}");
+ return 0;
+ }
+
+ static float[] Aabb(PaddleOcrDetectionBox b) =>
+ [
+ Math.Min(Math.Min(b.X1, b.X2), Math.Min(b.X3, b.X4)),
+ Math.Min(Math.Min(b.Y1, b.Y2), Math.Min(b.Y3, b.Y4)),
+ Math.Max(Math.Max(b.X1, b.X2), Math.Max(b.X3, b.X4)),
+ Math.Max(Math.Max(b.Y1, b.Y2), Math.Max(b.Y3, b.Y4)),
+ ];
+
+ internal static double WorkingSetMb()
+ {
+ using Process p = Process.GetCurrentProcess();
+ p.Refresh();
+ return p.WorkingSet64 / (1024d * 1024d);
+ }
+
+ /// PowerManager thermal status (0 none .. 6 shutdown) or -1.
+ internal static int Thermal()
+ {
+ try
+ {
+ var pm = (Android.OS.PowerManager?)Android.App.Application.Context.GetSystemService(Android.Content.Context.PowerService);
+ return pm is null || !OperatingSystem.IsAndroidVersionAtLeast(29) ? -1 : (int)pm.CurrentThermalStatus;
+ }
+ catch { return -1; }
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/GemmTest.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/GemmTest.cs
new file mode 100644
index 0000000..b6d6292
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/GemmTest.cs
@@ -0,0 +1,246 @@
+using System.Text.RegularExpressions;
+using Sdcb.SimdPaddleOCR.Backends.Vulkan;
+
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+///
+/// GEMM checks with the graph's conv1x1 binding layout (x, w[N,K], bias,
+/// res, o, o2; pc = M, N, K, flags): one dispatch vs an fp64 reference,
+/// a variant sweep over hot shapes, and an FMA peak probe.
+///
+static unsafe class GemmTest
+{
+ // --gemm [flags] [tileM] [tileN] [reqSg] [reps]
+ internal static int Run(string[] args)
+ {
+ string shader = args[1];
+ int M = int.Parse(args[2]), N = int.Parse(args[3]), K = int.Parse(args[4]);
+ uint flags = args.Length > 5 ? uint.Parse(args[5]) : 0;
+ uint tm = args.Length > 6 ? uint.Parse(args[6]) : 64, tn = args.Length > 7 ? uint.Parse(args[7]) : 64;
+ uint reqSg = args.Length > 8 ? uint.Parse(args[8]) : 0;
+ int reps = args.Length > 9 ? int.Parse(args[9]) : 10;
+ using var dev = VkDevice.Create();
+ using var h = new Harness(dev, (long)M * K, (long)N * K, (long)M * N, N);
+ var pipe = dev.NewPipeline(dev.NewShaderModule(Spv(shader)), 6, 16, reqSg);
+ double best = h.Time(pipe, M, N, K, flags, tm, tn, reps);
+ var (maxErr, bad) = h.Verify(M, N, K, flags);
+ Console.WriteLine($"gemm {shader} M={M} N={N} K={K} flags={flags} tile={tm}x{tn} reqSg={reqSg}: best={best:F4} ms ({2.0 * M * N * K / (best * 1e-3) / 1e12:F3} TFLOPS) maxErr={maxErr:G4} bad={bad}/{(long)M * N}");
+ return bad == 0 ? 0 : 3;
+ }
+
+ static readonly (int M, int N, int K)[] s_hot =
+ [
+ (5760, 1024, 512), (5760, 512, 1024), (2880, 768, 1536), (2880, 1536, 768),
+ (57600, 256, 128), (57600, 128, 256), (14400, 512, 256), (3600, 1024, 512),
+ (900, 896, 1792), (57600, 64, 256), (11520, 256, 512),
+ ];
+
+ // --gemmsweep [rounds] [flags]: every variant is checked on
+ // an odd shape against fp64, then timed on the hot shapes, variants
+ // interleaved per round; min per shape over rounds
+ internal static int Sweep(string[] args)
+ {
+ string[] shaders = args[1].Split(',');
+ int rounds = args.Length > 2 ? int.Parse(args[2]) : 3;
+ uint flags = args.Length > 3 ? uint.Parse(args[3]) : 0;
+ using var dev = VkDevice.Create();
+ using var h = new Harness(dev, s_hot.Max(s => (long)s.M * s.K), s_hot.Max(s => (long)s.N * s.K),
+ s_hot.Max(s => (long)s.M * s.N), s_hot.Max(s => s.N));
+ var pipes = new VkPipeline[shaders.Length];
+ var tiles = new (uint, uint)[shaders.Length];
+ for (int v = 0; v < shaders.Length; v++)
+ {
+ pipes[v] = dev.NewPipeline(dev.NewShaderModule(Spv(shaders[v])), 6, 16, 0);
+ tiles[v] = Tile(shaders[v]);
+ h.Time(pipes[v], 1000, 136, 264, flags | 3u, tiles[v].Item1, tiles[v].Item2, 1);
+ var (err, bad) = h.Verify(1000, 136, 264, flags | 3u);
+ Console.WriteLine($"variant {v} {shaders[v]} tile={tiles[v].Item1}x{tiles[v].Item2}: verify maxErr={err:G3} bad={bad}");
+ }
+ var best = new double[shaders.Length, s_hot.Length];
+ for (int v = 0; v < shaders.Length; v++) for (int s = 0; s < s_hot.Length; s++) best[v, s] = double.MaxValue;
+ for (int r = 0; r < rounds; r++)
+ for (int s = 0; s < s_hot.Length; s++)
+ for (int v = 0; v < shaders.Length; v++)
+ {
+ var (M, N, K) = s_hot[s];
+ best[v, s] = Math.Min(best[v, s], h.Time(pipes[v], M, N, K, flags, tiles[v].Item1, tiles[v].Item2, 3));
+ }
+ Console.WriteLine("shape".PadRight(18) + string.Concat(Enumerable.Range(0, shaders.Length).Select(v => $"{"v" + v,10}")));
+ var sum = new double[shaders.Length];
+ for (int s = 0; s < s_hot.Length; s++)
+ {
+ var (M, N, K) = s_hot[s];
+ var line = new System.Text.StringBuilder($"{M}x{N}x{K}".PadRight(18));
+ for (int v = 0; v < shaders.Length; v++) { line.Append($"{best[v, s],10:F3}"); sum[v] += best[v, s]; }
+ Console.WriteLine(line);
+ }
+ Console.WriteLine("sum ms".PadRight(18) + string.Concat(sum.Select(x => $"{x,10:F2}")));
+ double flop = s_hot.Sum(s => 2.0 * s.M * s.N * s.K);
+ Console.WriteLine("TFLOPS".PadRight(18) + string.Concat(sum.Select(x => $"{flop / (x * 1e-3) / 1e12,10:F3}")));
+ Console.WriteLine($"thermal={E2eThermal()}");
+ return 0;
+ }
+
+ // --peak: fp32 / fp16 FMA throughput of 16 independent vec4 chains per thread
+ internal static int Peak(string[] args)
+ {
+ using var dev = VkDevice.Create();
+ using var h = new Harness(dev, 4096, 4096, 4096, 64);
+ foreach (string name in new[] { "peak.spv", "peak16.spv" })
+ {
+ using Stream s = typeof(GemmTest).Assembly.GetManifestResourceStream(name)!;
+ byte[] b = new byte[s.Length];
+ s.ReadExactly(b);
+ var pipe = dev.NewPipeline(dev.NewShaderModule(b), 6, 16, 0);
+ const uint groups = 4096;
+ double best = h.Time(pipe, 0, 0, 0, 0, 0, 0, 10, groups);
+ double flop = groups * 256.0 * 32768;
+ Console.WriteLine($"peak {name}: {best:F3} ms -> {flop / (best * 1e-3) / 1e12:F3} TFLOPS");
+ }
+ return 0;
+ }
+
+ static int E2eThermal()
+ {
+#if ANDROID
+ return E2e.Thermal();
+#else
+ return -1;
+#endif
+ }
+
+ static (uint, uint) Tile(string shader)
+ {
+ var bm = Regex.Match(shader, @"bm(\d+)");
+ var bn = Regex.Match(shader, @"bn(\d+)");
+ return (bm.Success ? uint.Parse(bm.Groups[1].Value) : 64u, bn.Success ? uint.Parse(bn.Groups[1].Value) : 64u);
+ }
+
+ static byte[] Spv(string shader)
+ {
+ if (shader.EndsWith(".spv")) return File.ReadAllBytes(shader);
+ using Stream s = typeof(VkDevice).Assembly.GetManifestResourceStream(
+ $"Sdcb.SimdPaddleOCR.Backends.Vulkan.Shaders.{shader}.spv") ?? throw new FileNotFoundException(shader);
+ byte[] b = new byte[s.Length];
+ s.ReadExactly(b);
+ return b;
+ }
+
+ /// Random fp16 operands sized for the largest shape, bound once.
+ sealed class Harness : IDisposable
+ {
+ readonly VkDevice _dev;
+ readonly Half[] _x, _w, _bias, _res;
+ readonly VkBuffer[] _binds;
+ readonly IntPtr _cmd, _fence, _qp;
+
+ public Harness(VkDevice dev, long xElems, long wElems, long mn, int maxN)
+ {
+ _dev = dev;
+ var rr = new Random(3);
+ mn = Math.Max(mn, 1000 * 136);
+ _x = new Half[Math.Max(xElems, 1000 * 264) + 128 * 512];
+ _w = new Half[Math.Max(wElems, 136 * 264) + 128 * 2048];
+ _bias = new Half[Math.Max(maxN, 136) + 8];
+ _res = new Half[mn];
+ for (long i = 0; i < _x.Length; i++) _x[i] = (Half)(rr.NextDouble() * 2 - 1);
+ for (long i = 0; i < _w.Length; i++) _w[i] = (Half)((rr.NextDouble() * 2 - 1) * 0.1);
+ for (int i = 0; i < _bias.Length; i++) _bias[i] = (Half)(rr.NextDouble() - 0.5);
+ for (long i = 0; i < _res.Length; i++) _res[i] = (Half)(rr.NextDouble() - 0.5);
+ ulong outBytes = (ulong)mn * 2 + 4096;
+ _binds = [Up(_x), Up(_w), Up(_bias), Up(_res),
+ dev.NewStorageBuffer(outBytes, hostVisible: true, preferHost: true),
+ dev.NewStorageBuffer(outBytes, hostVisible: true, preferHost: true)];
+ _cmd = dev.NewCommandBuffer();
+ _fence = dev.NewFence();
+ var qci = new Vk.VkQueryPoolCreateInfo { SType = 11, QueryType = 2, QueryCount = 2 };
+ Vk.Check(Vk.vkCreateQueryPool(dev.Device, &qci, null, out _qp), "qp");
+ }
+
+ VkBuffer Up(Half[] h)
+ {
+ var b = _dev.NewStorageBuffer((ulong)Math.Max(h.Length, 64) * 2, hostVisible: false);
+ fixed (Half* p = h) _dev.Upload(b, p, (ulong)h.Length * 2);
+ return b;
+ }
+
+ /// Min GPU time (ms) of dispatches. The
+ /// operands are laid out for the call's own (M, N, K).
+ public double Time(VkPipeline pipe, int M, int N, int K, uint flags, uint tm, uint tn, int reps, uint groups = 0)
+ {
+ IntPtr set = _dev.NewDescriptorSet(pipe.SetLayout);
+ for (uint i = 0; i < 6; i++) _dev.BindBuffer(set, i, _binds[i]);
+ uint gx = groups != 0 ? groups : (uint)((M + tm - 1) / tm), gy = groups != 0 ? 1 : (uint)((N + tn - 1) / tn);
+ uint[] pc = [(uint)M, (uint)N, (uint)K, flags];
+ double best = double.MaxValue;
+ ulong* ts = stackalloc ulong[2];
+ for (int r = 0; r < reps; r++)
+ {
+ var begin = new Vk.VkCommandBufferBeginInfo { SType = VkConst.StCommandBufferBeginInfo };
+ Vk.Check(Vk.vkResetCommandBuffer(_cmd, 0), "reset");
+ Vk.Check(Vk.vkBeginCommandBuffer(_cmd, &begin), "begin");
+ Vk.vkCmdResetQueryPool(_cmd, _qp, 0, 2);
+ Vk.vkCmdBindPipeline(_cmd, VkConst.BindPointCompute, pipe.Pipeline);
+ Vk.vkCmdBindDescriptorSets(_cmd, VkConst.BindPointCompute, pipe.Layout, 0, 1, &set, 0, null);
+ fixed (uint* pp = pc) Vk.vkCmdPushConstants(_cmd, pipe.Layout, VkConst.StageComputeShader, 0, 16, pp);
+ Vk.vkCmdWriteTimestamp(_cmd, VkConst.PipelineStageBottomOfPipe, _qp, 0);
+ Vk.vkCmdDispatch(_cmd, gx, gy, 1);
+ Vk.vkCmdWriteTimestamp(_cmd, VkConst.PipelineStageBottomOfPipe, _qp, 1);
+ Vk.Check(Vk.vkEndCommandBuffer(_cmd), "end");
+ _dev.Submit(_cmd, _fence); _dev.WaitFence(_fence);
+ Vk.vkGetQueryPoolResults(_dev.Device, _qp, 0, 2, 16, ts, 8, 1 | 2);
+ best = Math.Min(best, (ts[1] - ts[0]) * _dev.TimestampPeriodNs / 1e6);
+ }
+ _dev.FreeDescriptorSet(set);
+ return best;
+ }
+
+ /// Checks the last dispatch's output (the operands as laid out
+ /// for this M, N, K) against fp64.
+ public (double MaxErr, long Bad) Verify(int M, int N, int K, uint flags)
+ {
+ Half* o = (Half*)_binds[4].Map();
+ bool hasBias = (flags & 1) != 0, hasRes = (flags & 2) != 0;
+ uint act = (flags >> 4) & 7;
+ double maxErr = 0; long bad = 0;
+ for (int m = 0; m < M; m++)
+ for (int n = 0; n < N; n++)
+ {
+ double acc = 0;
+ for (int k = 0; k < K; k++) acc += (double)_x[(long)m * K + k] * (double)_w[(long)n * K + k];
+ if (hasBias) acc += (double)_bias[n];
+ if (hasRes) acc += (double)_res[(long)m * N + n];
+ acc = act switch
+ {
+ 1 => Math.Max(acc, 0),
+ 2 => 0.5 * acc * (1 + Erf(acc / Math.Sqrt(2))),
+ 3 => acc * Math.Clamp(acc * (double)BitConverter.UInt16BitsToHalf((ushort)(flags >> 16)) + 0.5, 0, 1),
+ 4 => 1 / (1 + Math.Exp(-acc)),
+ _ => acc,
+ };
+ double g = (double)o[(long)m * N + n];
+ double e = double.IsNaN(g) ? double.PositiveInfinity : Math.Abs(g - acc);
+ maxErr = Math.Max(maxErr, e);
+ if (e > 0.02 + 0.01 * Math.Abs(acc)) bad++;
+ }
+ _binds[4].Unmap();
+ return (maxErr, bad);
+ }
+
+ public void Dispose()
+ {
+ Vk.vkDestroyQueryPool(_dev.Device, _qp, null);
+ _dev.DestroyFence(_fence);
+ foreach (var b in _binds) b.Free();
+ }
+ }
+
+ // Abramowitz-Stegun 7.1.26 (|err| < 1.5e-7)
+ static double Erf(double v)
+ {
+ double s = Math.Sign(v); v = Math.Abs(v);
+ double t = 1 / (1 + 0.3275911 * v);
+ double y = 1 - ((((1.061405429 * t - 1.453152027) * t + 1.421413741) * t - 0.284496736) * t + 0.254829592) * t * Math.Exp(-v * v);
+ return s * y;
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/LayerProbe.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/LayerProbe.cs
new file mode 100644
index 0000000..9470bc1
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/LayerProbe.cs
@@ -0,0 +1,53 @@
+using Sdcb.SimdPaddleOCR.Backends.Vulkan;
+using Sdcb.SimdPaddleOCR.OnnxSharp;
+
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+///
+/// Bisection aid: truncates the GPU graph after node k and reads that node's
+/// output back through the graph's own fp32 writeback, printing statistics.
+/// Plain .NET, so the desktop GpuBench links it too for reference numbers.
+///
+static class LayerProbe
+{
+ // --layers [firstNode] [lastNode]
+ internal static int Run(string onnx, int n, int h, int w, int first = 0, int last = int.MaxValue)
+ {
+ var mdl = Model.Load(File.ReadAllBytes(onnx));
+ var compiled = new CompiledModel(mdl, intraOpThreads: 4);
+ int[] shape = [n, 3, h, w];
+ var inp = new float[n * 3 * h * w];
+ var rr = new Random(7);
+ for (int i = 0; i < inp.Length; i++) inp[i] = (float)(rr.NextDouble() * 2 - 1);
+ using var dev = VkDevice.Create();
+ using var gg = new GpuDetGraph(dev, compiled);
+ Console.WriteLine($"device={dev.DeviceName} nodes={mdl.Nodes.Length}");
+ var nodes = mdl.Nodes;
+ for (int k = first; k <= Math.Min(last, nodes.Length - 1); k++)
+ {
+ int t = checked((int)nodes[k].Outputs[0]);
+ GpuSchedule s;
+ try { s = gg.Prepare(shape, k + 1, t); }
+ catch (Exception ex) { Console.WriteLine($"n{k,-4} {nodes[k].Operator,-14} plan failed: {ex.GetBaseException().Message}"); continue; }
+ int p = t;
+ while (s.Alias[p] != p) p = s.Alias[p];
+ if (s.Off[p] < 0 || s.Recs.Length < 2)
+ {
+ if (Environment.GetEnvironmentVariable("LAYERS_VERBOSE") == "1")
+ Console.WriteLine($"n{k,-4} {nodes[k].Operator,-14} skipped: t{t} phys{p} off={s.Off[p]} recs={s.Recs.Length}");
+ continue;
+ }
+ float[] v = gg.Run(shape, inp, k + 1, t).ToArray();
+ double sum = 0, sumAbs = 0; float mn = float.MaxValue, mx = float.MinValue; int nz = 0, nan = 0;
+ foreach (float x in v)
+ {
+ if (float.IsNaN(x) || float.IsInfinity(x)) { nan++; continue; }
+ mn = MathF.Min(mn, x); mx = MathF.Max(mx, x); sum += x; sumAbs += Math.Abs(x);
+ if (x != 0) nz++;
+ }
+ string last2 = string.Join(" | ", s.Recs.Take(s.Recs.Length - 1).TakeLast(2).Select(r => r.Tag));
+ Console.WriteLine($"n{k,-4} {nodes[k].Operator,-14} numel={v.Length,-9} mean={sum / v.Length,10:G5} meanAbs={sumAbs / v.Length,10:G5} min={mn,10:G5} max={mx,10:G5} nz={nz}/{v.Length} bad={nan} recs={s.Recs.Length} [{last2}]");
+ }
+ return 0;
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/MainActivity.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/MainActivity.cs
new file mode 100644
index 0000000..0852a78
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/MainActivity.cs
@@ -0,0 +1,118 @@
+using Android.App;
+using Android.OS;
+using Android.Views;
+
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+///
+/// adb entry point. Extras: args (space-separated bench arguments),
+/// env (K=V;K=V, applied before the library is touched so its
+/// static env switches see them) and run (output id). Everything the
+/// bench prints goes to logcat (tag SimdOcrBench) and to
+/// files/out/<run>.log; <run>.done holds the exit code.
+/// The process kills itself afterwards so every run starts cold.
+///
+[Activity(Label = "SimdOcrBench", MainLauncher = true, Exported = true,
+ Name = "com.sdcb.simdocr.bench.MainActivity")]
+public class MainActivity : Activity
+{
+ protected override void OnCreate(Bundle? savedInstanceState)
+ {
+ base.OnCreate(savedInstanceState);
+ Window?.AddFlags(WindowManagerFlags.KeepScreenOn);
+ string args = Intent?.GetStringExtra("args") ?? "--caps";
+ string env = Intent?.GetStringExtra("env") ?? "";
+ string run = Intent?.GetStringExtra("run") ?? "run";
+ string root = GetExternalFilesDir(null)!.AbsolutePath;
+ string outDir = Path.Combine(root, "out");
+ // app-created, so the app can read what adb pushes into them later
+ // (directories made by the shell user are not readable by the app)
+ foreach (string d in new[] { outDir, Path.Combine(root, "models"), Path.Combine(root, "dataset") })
+ Directory.CreateDirectory(d);
+ File.Delete(Path.Combine(outDir, run + ".done"));
+
+ foreach (string kv in env.Split(';', StringSplitOptions.RemoveEmptyEntries))
+ {
+ int eq = kv.IndexOf('=');
+ if (eq > 0) System.Environment.SetEnvironmentVariable(kv[..eq], kv[(eq + 1)..]);
+ }
+
+ var worker = new Thread(() =>
+ {
+ int rc;
+ using (var log = new TeeWriter(Path.Combine(outDir, run + ".log")))
+ {
+ Console.SetOut(log);
+ Console.SetError(log);
+ Console.WriteLine($"args: {args}");
+ if (env.Length > 0) Console.WriteLine($"env: {env}");
+ try
+ {
+ Directory.SetCurrentDirectory(root);
+ rc = Bench.Run(args.Split(' ', StringSplitOptions.RemoveEmptyEntries), root, outDir);
+ }
+ catch (Exception ex)
+ {
+ Console.WriteLine($"FATAL {ex}");
+ rc = 99;
+ }
+ Console.WriteLine($"exit {rc}");
+ }
+ File.WriteAllText(Path.Combine(outDir, run + ".done"), rc.ToString());
+ Process.KillProcess(Process.MyPid());
+ }, 64 << 20);
+ worker.Start();
+ }
+}
+
+/// Line-buffered tee to logcat and a file.
+sealed class TeeWriter(string path) : TextWriter
+{
+ private readonly StreamWriter _file = new(path, append: false) { AutoFlush = true };
+ private readonly System.Text.StringBuilder _line = new();
+ private readonly object _sync = new();
+ public override System.Text.Encoding Encoding => System.Text.Encoding.UTF8;
+
+ public override void Write(char value)
+ {
+ lock (_sync)
+ {
+ if (value == '\n') { FlushLine(); return; }
+ if (value != '\r') _line.Append(value);
+ }
+ }
+
+ public override void Write(string? value)
+ {
+ if (value is null) return;
+ lock (_sync)
+ foreach (char c in value)
+ {
+ if (c == '\n') FlushLine();
+ else if (c != '\r') _line.Append(c);
+ }
+ }
+
+ public override void WriteLine(string? value)
+ {
+ lock (_sync) { Write(value); FlushLine(); }
+ }
+
+ private void FlushLine()
+ {
+ string s = _line.ToString();
+ _line.Clear();
+ Android.Util.Log.Info("SimdOcrBench", s);
+ _file.WriteLine(s);
+ }
+
+ protected override void Dispose(bool disposing)
+ {
+ if (disposing)
+ {
+ lock (_sync) if (_line.Length > 0) FlushLine();
+ _file.Dispose();
+ }
+ base.Dispose(disposing);
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Paths.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/Paths.cs
new file mode 100644
index 0000000..0a11466
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/Paths.cs
@@ -0,0 +1,42 @@
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+/// Model / dataset locations and image decoding shared by the phone
+/// host and the desktop GpuBench (which links this file).
+static class Paths
+{
+ internal static string Root = "";
+ internal static string ModelsDir = "models";
+ internal static string DatasetDir = "dataset";
+
+ internal static string Path_(string p) => Path.IsPathRooted(p) ? p : Path.Combine(Root, p);
+ internal static string ModelPath(string model, string kind) =>
+ kind == "cls" ? Path.Combine(Path_(ModelsDir), "cls.onnx") : Path.Combine(Path_(ModelsDir), $"{model}-{kind}.onnx");
+ internal static string DictPath(string model) => Path.Combine(Path_(ModelsDir), $"{model}-dict.txt");
+ internal static string Dataset => Path_(DatasetDir);
+
+ internal static OcrBackend ParseBackend(string s) => s.ToLowerInvariant() switch
+ {
+ "cpu" or "sharp" => OcrBackend.Cpu,
+ "vulkan" => OcrBackend.Vulkan,
+ "auto" => OcrBackend.Auto,
+ _ => throw new ArgumentException("backend must be cpu, vulkan or auto"),
+ };
+
+ /// Decodes like the desktop Tests harness (ImageSharp, BGR24).
+ internal static (byte[] Bgr, int W, int H) LoadBgr(string path)
+ {
+ using var image = SixLabors.ImageSharp.Image.Load(path);
+ byte[] px = new byte[checked(image.Width * image.Height * 3)];
+ for (int y = 0; y < image.Height; y++)
+ for (int x = 0; x < image.Width; x++)
+ {
+ var p = image[x, y];
+ int o = (y * image.Width + x) * 3;
+ px[o] = p.B; px[o + 1] = p.G; px[o + 2] = p.R;
+ }
+ return (px, image.Width, image.Height);
+ }
+
+ internal static string[] DatasetFiles() => Directory.GetFiles(Dataset, "img-*.jpg")
+ .OrderBy(x => x, StringComparer.OrdinalIgnoreCase).ToArray();
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Prof.cs b/test/Sdcb.SimdPaddleOCR.AndroidBench/Prof.cs
new file mode 100644
index 0000000..eabc087
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/Prof.cs
@@ -0,0 +1,80 @@
+using System.Diagnostics;
+using Sdcb.SimdPaddleOCR.Backends.Vulkan;
+using Sdcb.SimdPaddleOCR.OnnxSharp;
+
+namespace Sdcb.SimdPaddleOCR.AndroidBench;
+
+/// Pure-GPU profiles (GpuBench --detprof / --recprof) and the
+/// concurrent-caller check (GpuBench --conc), on the phone.
+static class Prof
+{
+ internal static int Run(string[] args)
+ {
+ bool det = args[0] == "--detprof";
+ string model = args[1];
+ int a = int.Parse(args[2]), b = int.Parse(args[3]);
+ int reps = args.Length >= 5 ? int.Parse(args[4]) : 20;
+ var mdl = Model.Load(File.ReadAllBytes(Paths.ModelPath(model, det ? "det" : "rec")));
+ var compiled = new CompiledModel(mdl, intraOpThreads: 8);
+ int[] shape = det ? [1, 3, a, b] : [a, 3, 48, b];
+ var inp = new float[shape[0] * shape[1] * shape[2] * shape[3]];
+ var rr = new Random(1);
+ for (int i = 0; i < inp.Length; i++) inp[i] = (float)(rr.NextDouble() * 2 - 1);
+ int nodeLimit = int.MaxValue, outTensor = -1;
+ if (!det)
+ {
+ var cpu = new InferenceSession(compiled);
+ cpu.Reshape(shape);
+ cpu.TryRunUntilCtcProjection(inp, out CtcProjectionOperands ops);
+ nodeLimit = ops.MatMulNodeIndex;
+ outTensor = checked((int)mdl.Nodes[nodeLimit].Inputs[0]);
+ Console.WriteLine($"rec: batch={a} T={ops.Rows} C={ops.Inner} cols={ops.Columns}");
+ }
+ using var dev = VkDevice.Create();
+ using var gg = new GpuDetGraph(dev, compiled);
+ double best = double.MaxValue;
+ var walls = new List();
+ for (int r = 0; r < reps; r++)
+ {
+ long t0 = Stopwatch.GetTimestamp();
+ gg.Run(shape, inp, nodeLimit, outTensor);
+ double ms = Stopwatch.GetElapsedTime(t0).TotalMilliseconds;
+ walls.Add(ms);
+ best = Math.Min(best, ms);
+ }
+ walls.Sort();
+ Console.WriteLine($"best wall {best:F2} ms (median {walls[walls.Count / 2]:F2}), dispatches={gg.DispatchCount(shape, nodeLimit, outTensor)} thermal={E2e.Thermal()}");
+ gg.DumpProfile(shape, nodeLimit, outTensor);
+ return 0;
+ }
+
+ // --conc [threads] [backend]: serial reference texts vs concurrent callers
+ internal static int Conc(string[] args)
+ {
+ string model = args[1];
+ int threads = args.Length > 2 ? int.Parse(args[2]) : 4;
+ OcrBackend backend = args.Length > 3 ? Paths.ParseBackend(args[3]) : OcrBackend.Vulkan;
+ using var ocr = PaddleOcrAll.Load(Paths.ModelPath(model, "det"), Paths.ModelPath(model, "cls"),
+ Paths.ModelPath(model, "rec"), Paths.DictPath(model), new PaddleOcrOptions
+ {
+ Detector = new PaddleOcrDetectorOptions { Backend = backend },
+ Recognizer = new PaddleOcrRecognizerOptions { Backend = backend },
+ Classifier = new PaddleOcrClassifierOptions { Backend = backend },
+ });
+ var imgs = Paths.DatasetFiles().Take(24).Select(Paths.LoadBgr).ToArray();
+ string Texts(int i) => string.Join("|", ocr.Run(imgs[i].Bgr, imgs[i].W, imgs[i].H).Lines.Select(l => l.Text));
+ string[] reference = Enumerable.Range(0, imgs.Length).Select(Texts).ToArray();
+ int bad = 0;
+ double wsPeak = E2e.WorkingSetMb();
+ var tc = Stopwatch.StartNew();
+ Parallel.For(0, imgs.Length * 3, new ParallelOptions { MaxDegreeOfParallelism = threads }, j =>
+ {
+ int i = j % imgs.Length;
+ if (Texts(i) != reference[i]) { Interlocked.Increment(ref bad); Console.WriteLine($"MISMATCH img{i}"); }
+ double ws = E2e.WorkingSetMb();
+ lock (reference) wsPeak = Math.Max(wsPeak, ws);
+ });
+ Console.WriteLine($"conc model={model} backend={backend} usesGpu={GpuBackend.UsesGpu(backend)} threads={threads} runs={imgs.Length * 3} mismatches={bad} wall={tc.ElapsedMilliseconds}ms ws-peak={wsPeak:F0}MB");
+ return bad == 0 ? 0 : 5;
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Sdcb.SimdPaddleOCR.AndroidBench.csproj b/test/Sdcb.SimdPaddleOCR.AndroidBench/Sdcb.SimdPaddleOCR.AndroidBench.csproj
new file mode 100644
index 0000000..235fdec
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/Sdcb.SimdPaddleOCR.AndroidBench.csproj
@@ -0,0 +1,42 @@
+
+
+
+ net10.0-android
+ 29
+ Exe
+ enable
+ enable
+ preview
+ true
+ false
+ Sdcb.SimdPaddleOCR.AndroidBench
+ Sdcb.SimdPaddleOCR.AndroidBench
+ com.sdcb.simdocr.bench
+ SimdOcrBench
+ 1
+ 1.0
+ android-arm64
+ apk
+ true
+ false
+ true
+
+ embedded
+
+ false
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak.comp b/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak.comp
new file mode 100644
index 0000000..83d299e
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak.comp
@@ -0,0 +1,38 @@
+#version 450
+#extension GL_EXT_shader_explicit_arithmetic_types_float16 : require
+#extension GL_EXT_control_flow_attributes : require
+
+// FMA peak probe: 8 independent vec4 chains per thread, fully unrolled
+// inner body, 512 iterations: 2 * 4 * 8 * 512 = 32768 flop per thread.
+// -DF16 runs the same on f16vec4.
+// Rebuild: glslc -O --target-env=vulkan1.1 [-DF16] peak.comp -o peak[16].spv
+
+layout(local_size_x = 256) in;
+layout(std430, binding = 0) buffer O { vec4 o[]; };
+layout(push_constant) uniform PC { float s; } p;
+
+#ifdef F16
+#define T f16vec4
+#define S(x) float16_t(x)
+#else
+#define T vec4
+#define S(x) (x)
+#endif
+
+void main()
+{
+ T m = T(S(p.s)), c = T(S(1e-4));
+ float b = float(gl_LocalInvocationID.x) * 1e-3;
+ T a0 = T(S(b)), a1 = T(S(b + 0.1)), a2 = T(S(b + 0.2)), a3 = T(S(b + 0.3));
+ T a4 = T(S(b + 0.4)), a5 = T(S(b + 0.5)), a6 = T(S(b + 0.6)), a7 = T(S(b + 0.7));
+ for (int it = 0; it < 64; it++)
+ {
+ [[unroll]] for (int u = 0; u < 8; u++)
+ {
+ a0 = fma(a0, m, c); a1 = fma(a1, m, c); a2 = fma(a2, m, c); a3 = fma(a3, m, c);
+ a4 = fma(a4, m, c); a5 = fma(a5, m, c); a6 = fma(a6, m, c); a7 = fma(a7, m, c);
+ }
+ }
+ T r = a0 + a1 + a2 + a3 + a4 + a5 + a6 + a7;
+ if (float(r.x) == 12345.0) o[gl_GlobalInvocationID.x] = vec4(r);
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak.spv b/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak.spv
new file mode 100644
index 0000000..af1f022
Binary files /dev/null and b/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak.spv differ
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak16.spv b/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak16.spv
new file mode 100644
index 0000000..2d86a12
Binary files /dev/null and b/test/Sdcb.SimdPaddleOCR.AndroidBench/Shaders/peak16.spv differ
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/ab.ps1 b/test/Sdcb.SimdPaddleOCR.AndroidBench/ab.ps1
new file mode 100644
index 0000000..9ca434f
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/ab.ps1
@@ -0,0 +1,26 @@
+# End-to-end A/B on the phone: for each model, backends alternate round by
+# round (cpu, vulkan, cpu, vulkan, ...) with a cool-down between runs. Prints
+# median ms (all / last 50 images), thermal status and accuracy per run.
+# .\ab.ps1 -tag final -models tiny,small,medium -rounds 3
+param([string]$tag = "ab", [string[]]$models = @("tiny", "small", "medium"),
+ [string[]]$backends = @("cpu", "vulkan"), [int]$rounds = 3, [int]$cool = 60, [string]$Env = "")
+$here = $PSScriptRoot
+$out = (Resolve-Path "$here\..\..").Path + "\bench-out\android"
+foreach ($m in $models) {
+ for ($r = 1; $r -le $rounds; $r++) {
+ foreach ($b in $backends) {
+ $id = "$tag-$m-$b-r$r"
+ Start-Sleep -Seconds $cool
+ & "$here\run.ps1" -Bench "--e2e --model $m --backend $b --out $id.json" -Env $Env -Run $id -TimeoutSec 3600 | Out-Null
+ $log = "$out\$id.log"
+ $st = (Select-String -Path $log -Pattern "steady: all-med ([\d.]+) last50-med ([\d.]+) ws-peak (\d+)MB thermal=(-?\d+)").Matches
+ $ac = (Select-String -Path $log -Pattern "exact_lines=(\d+)/\d+ .*CER=([\d.]+)%").Matches
+ $gpu = (Select-String -Path $log -Pattern "usesGpu=(\w+)").Matches
+ if ($st) {
+ "{0,-7} {1,-7} r{2} med {3,7} last50 {4,7} ws {5,5}MB thermal {6} lines {7} CER {8}% gpu={9}" -f $m, $b, $r,
+ $st[0].Groups[1].Value, $st[0].Groups[2].Value, $st[0].Groups[3].Value, $st[0].Groups[4].Value,
+ $ac[0].Groups[1].Value, $ac[0].Groups[2].Value, $gpu[0].Groups[1].Value
+ } else { "{0,-7} {1,-7} r{2} FAILED (see $log)" -f $m, $b, $r }
+ }
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/global.json b/test/Sdcb.SimdPaddleOCR.AndroidBench/global.json
new file mode 100644
index 0000000..545c92c
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/global.json
@@ -0,0 +1,6 @@
+{
+ "sdk": {
+ "version": "10.0.302",
+ "rollForward": "latestFeature"
+ }
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/gprof.ps1 b/test/Sdcb.SimdPaddleOCR.AndroidBench/gprof.ps1
new file mode 100644
index 0000000..8afdbed
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/gprof.ps1
@@ -0,0 +1,21 @@
+# Pure-GPU profile on the phone over the GpuBench gprof.ps1 case list
+# (per-dispatch timestamps, min over reps). Logs land in bench-out/android/gp--*.log.
+param([string]$tag = "x", [string[]]$cases = @(
+ "det medium 960 960", "det medium 640 960", "rec medium 8 480", "rec medium 1 320",
+ "det small 960 960", "rec small 8 480", "det tiny 960 960", "rec tiny 8 480"),
+ [int]$reps = 10, [switch]$all, [string]$Env = "")
+$here = $PSScriptRoot
+$out = (Resolve-Path "$here\..\..").Path + "\bench-out\android"
+$envs = "SIMD_OCR_GPU_PROF=1" + $(if ($all) { ";SIMD_OCR_GPU_PROF_ALL=1" }) + $(if ($Env) { ";$Env" })
+foreach ($c in $cases) {
+ $k, $m, $a, $b = $c -split " "
+ $mode = if ($k -eq "det") { "--detprof" } else { "--recprof" }
+ $id = "gp-$tag-$k-$m-$a-$b"
+ & "$here\run.ps1" -Bench "$mode $m $a $b $reps" -Env $envs -Run $id | Out-Null
+ $log = "$out\$id.log"
+ $t = (Select-String -Path $log -Pattern "GPU profile \((\d+) dispatches, ([\d.]+) ms").Matches
+ $w = (Select-String -Path $log -Pattern "best wall ([\d.]+)").Matches
+ $th = (Select-String -Path $log -Pattern "thermal=(\d+)").Matches
+ "{0,-22} gpu={1,8} ms disp={2,4} wall={3} thermal={4}" -f $c, $t[0].Groups[2].Value, $t[0].Groups[1].Value,
+ $(if ($w) { $w[0].Groups[1].Value } else { "?" }), $(if ($th) { $th[0].Groups[1].Value } else { "?" })
+}
diff --git a/test/Sdcb.SimdPaddleOCR.AndroidBench/run.ps1 b/test/Sdcb.SimdPaddleOCR.AndroidBench/run.ps1
new file mode 100644
index 0000000..68df8e1
--- /dev/null
+++ b/test/Sdcb.SimdPaddleOCR.AndroidBench/run.ps1
@@ -0,0 +1,104 @@
+# One command for the phone: [build] -> [adb install] -> [push models/dataset] -> launch ->
+# wait for .done -> pull files/out/* into bench-out/android/. On a crash or a
+# process that dies without writing .done, logcat is saved next to the log.
+#
+# .\run.ps1 -Build -Install -Push -Bench "--caps"
+# .\run.ps1 -Bench "--e2e --model tiny --backend vulkan --out e2e-tiny-vk.json" -Env "SIMD_OCR_GPU_TIME=1"
+param(
+ [string]$Bench = "--caps",
+ [string]$Env = "",
+ [string]$Run = "",
+ [switch]$Build,
+ [switch]$Install,
+ [switch]$Push,
+ [int]$TimeoutSec = 3600,
+ [string]$Adb = $(if ($env:ADB) { $env:ADB } else { "D:\_\3rd\android-sdk\platform-tools\adb.exe" }),
+ [string]$AndroidSdk = $(if ($env:ANDROID_HOME) { $env:ANDROID_HOME } else { "D:\_\3rd\android-sdk" }),
+ [string]$Jdk = $(if ($env:JAVA_HOME -and (Test-Path "$env:JAVA_HOME\bin\javac.exe") -and ($env:JAVA_HOME -notmatch "jdk-2[2-9]")) { $env:JAVA_HOME } else { "C:\Program Files\Android\openjdk\jdk-21.0.8" })
+)
+$ErrorActionPreference = "Stop"
+$here = $PSScriptRoot
+$root = (Resolve-Path "$here\..\..").Path
+$pkg = "com.sdcb.simdocr.bench"
+$remote = "/sdcard/Android/data/$pkg/files"
+$local = "$root\bench-out\android"
+New-Item -ItemType Directory -Force $local | Out-Null
+if (-not $Run) { $Run = "r" + (Get-Date -Format "MMdd-HHmmss") }
+
+if ($Build) {
+ Push-Location $here
+ try {
+ dotnet build -c Release "-p:AndroidSdkDirectory=$AndroidSdk" "-p:JavaSdkDirectory=$Jdk" -v:q -nologo -clp:ErrorsOnly | Out-Host
+ if ($LASTEXITCODE -ne 0) { throw "build failed" }
+ } finally { Pop-Location }
+}
+if ($Install) {
+ $apk = "$here\bin\Release\net10.0-android\android-arm64\$pkg-Signed.apk"
+ & $Adb install -r -g $apk | Out-Host
+ if ($LASTEXITCODE -ne 0) { throw "adb install failed" }
+}
+
+function Start-Bench([string]$id, [string]$argv, [string]$envs) {
+ # a previous run may still be exiting: an intent delivered to it would be
+ # swallowed by the dying activity instead of starting a fresh process
+ & $Adb shell "am force-stop $pkg"
+ for ($i = 0; $i -lt 20 -and ((& $Adb shell "pidof $pkg") -join ""); $i++) { Start-Sleep -Milliseconds 500 }
+ & $Adb shell "rm -f $remote/out/$id.done $remote/out/$id.log"
+ $cmd = "am start -S -n $pkg/$pkg.MainActivity --es run '$id' --es args '$argv'"
+ if ($envs) { $cmd += " --es env '$envs'" }
+ & $Adb shell $cmd | Out-Null
+}
+
+if ($Push) {
+ # the app must create models/ dataset/ out/ itself: directories made by the
+ # shell user are not readable by the app
+ & $Adb shell "rm -rf $remote/models $remote/dataset $remote/out"
+ Start-Bench "init" "--init" ""
+ for ($i = 0; $i -lt 30 -and -not ((& $Adb shell "cat $remote/out/init.done 2>/dev/null") -join ""); $i++) { Start-Sleep 1 }
+ & $Adb push "$root\bench-out\models\." "$remote/models/" | Out-Host
+ & $Adb push "$root\dataset\." "$remote/dataset/" | Out-Host
+}
+
+& $Adb logcat -c
+Start-Bench $Run $Bench $Env
+Write-Output "[$Run] $Bench $(if ($Env) { "(env $Env)" })"
+
+$t0 = Get-Date
+$seen = 0
+$started = $false
+$result = $null
+while (((Get-Date) - $t0).TotalSeconds -lt $TimeoutSec) {
+ Start-Sleep -Seconds 2
+ $done = (& $Adb shell "cat $remote/out/$Run.done 2>/dev/null") -join ""
+ $log = & $Adb shell "cat $remote/out/$Run.log 2>/dev/null"
+ if ($log) {
+ $lines = @($log)
+ for ($i = $seen; $i -lt $lines.Count; $i++) { Write-Output $lines[$i] }
+ $seen = $lines.Count
+ }
+ if ($done -match "^\d+$") { $result = [int]$done; break }
+ $procId = (& $Adb shell "pidof $pkg") -join ""
+ if ($procId) { $started = $true }
+ elseif ($started -or ((Get-Date) - $t0).TotalSeconds -gt 30) {
+ # the app writes .done and then kills itself: look once more
+ Start-Sleep -Seconds 1
+ $done = (& $Adb shell "cat $remote/out/$Run.done 2>/dev/null") -join ""
+ $lines = @(& $Adb shell "cat $remote/out/$Run.log 2>/dev/null")
+ for ($i = $seen; $i -lt $lines.Count; $i++) { Write-Output $lines[$i] }
+ if ($done -match "^\d+$") { $result = [int]$done }
+ break
+ }
+}
+
+& $Adb pull "$remote/out/." "$local" 2>&1 | Out-Null
+if ($null -eq $result) {
+ $lc = "$local\$Run-logcat.txt"
+ & $Adb logcat -d -v time "SimdOcrBench:*" "DOTNET:*" "AndroidRuntime:*" "DEBUG:*" "libc:*" "ActivityManager:*" "lowmemorykiller:*" "*:F" "*:S" | Out-File -Encoding utf8 $lc
+ Write-Output "[$Run] process ended without .done — logcat saved to $lc"
+ exit 98
+}
+if ($result -ne 0) {
+ & $Adb logcat -d -v time "SimdOcrBench:*" "DOTNET:*" "AndroidRuntime:*" "*:F" "*:S" | Out-File -Encoding utf8 "$local\$Run-logcat.txt"
+}
+Write-Output "[$Run] exit $result -> $local\$Run.log"
+exit $result
diff --git a/test/Sdcb.SimdPaddleOCR.GpuBench/Harness.cs b/test/Sdcb.SimdPaddleOCR.GpuBench/Harness.cs
index 3b92b4d..081685f 100644
--- a/test/Sdcb.SimdPaddleOCR.GpuBench/Harness.cs
+++ b/test/Sdcb.SimdPaddleOCR.GpuBench/Harness.cs
@@ -1,5 +1,18 @@
namespace Sdcb.SimdPaddleOCR.GpuBench;
+static class DesktopPaths
+{
+ internal static int Run(Func mode, string[] args)
+ {
+ string root = AppContext.BaseDirectory;
+ while (root.Length > 3 && !File.Exists(Path.Combine(root, "Sdcb.SimdPaddleOCR.slnx")))
+ root = Path.GetDirectoryName(root.TrimEnd(Path.DirectorySeparatorChar))!;
+ AndroidBench.Paths.Root = root;
+ AndroidBench.Paths.ModelsDir = Path.Combine("bench-out", "models");
+ return mode(args);
+ }
+}
+
static class Harness
{
internal static int Usage(int code)
@@ -26,6 +39,10 @@ internal static int Usage(int code)
--sessiso
--reciso ![]()
--detmap ![]()
+ --detcmp [maxSide] CPU vs Vulkan DET map + boxes
+ --reccmp CPU vs Vulkan REC activations
+ --gemm [flags] [tileM] [tileN] [reqSg] [reps] GEMM vs fp64 reference
+ --layers [first] [last] per-node output stats of the truncated GPU graph
--recreal ![]()
[W]
""");
return code;
diff --git a/test/Sdcb.SimdPaddleOCR.GpuBench/Program.cs b/test/Sdcb.SimdPaddleOCR.GpuBench/Program.cs
index 9b9a2c2..4c36010 100644
--- a/test/Sdcb.SimdPaddleOCR.GpuBench/Program.cs
+++ b/test/Sdcb.SimdPaddleOCR.GpuBench/Program.cs
@@ -13,5 +13,11 @@
"--rec" => RecCompare.Run(args),
"--recprof" or "--sessiso" or "--reciso" or "--recreal" => RecBench.Run(args),
"--conc" or "--pipe" => PipelineBench.Run(args),
+ "--gemm" => Sdcb.SimdPaddleOCR.AndroidBench.GemmTest.Run(args),
+ // same checks as the phone host; models from bench-out/models, images from dataset/
+ "--detcmp" or "--reccmp" => DesktopPaths.Run(Sdcb.SimdPaddleOCR.AndroidBench.Compare.Run, args),
+ "--layers" when args.Length >= 5 => Sdcb.SimdPaddleOCR.AndroidBench.LayerProbe.Run(args[1], int.Parse(args[2]),
+ int.Parse(args[3]), int.Parse(args[4]), args.Length > 5 ? int.Parse(args[5]) : 0,
+ args.Length > 6 ? int.Parse(args[6]) : int.MaxValue),
_ => Harness.Usage(2),
};
diff --git a/test/Sdcb.SimdPaddleOCR.GpuBench/Sdcb.SimdPaddleOCR.GpuBench.csproj b/test/Sdcb.SimdPaddleOCR.GpuBench/Sdcb.SimdPaddleOCR.GpuBench.csproj
index 4481ab0..01aaccf 100644
--- a/test/Sdcb.SimdPaddleOCR.GpuBench/Sdcb.SimdPaddleOCR.GpuBench.csproj
+++ b/test/Sdcb.SimdPaddleOCR.GpuBench/Sdcb.SimdPaddleOCR.GpuBench.csproj
@@ -18,5 +18,9 @@
+
+
+
+