Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -306,7 +306,7 @@ GPUs report compute capability 12.0 (`sm_120`) and should use the CPU package or
build from source with `-DCMAKE_CUDA_ARCHITECTURES=120` until a dedicated CUDA
asset is published.

**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.4](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/en/deploy/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)
**Prebuilt binaries:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.5](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.5) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.5/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.5/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.5/funasr-llamacpp-windows-x64-cuda.zip) · **Download & quickstart:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/en/deploy/llama-cpp.html) · **GGUF models:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **Docs & benchmarks:** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API example →](./examples/openai_api/) · [Gradio demo →](./examples/openai_api/GRADIO.md) · [Client recipes →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript recipes →](./examples/openai_api/JAVASCRIPT.md) · [Kubernetes template →](./examples/openai_api/kubernetes/) · [Workflow recipes →](./examples/openai_api/WORKFLOWS.md) · [Postman collection →](./examples/openai_api/POSTMAN.md) · [OpenAPI spec →](./examples/openai_api/OPENAPI.md) · [Security guide →](./examples/openai_api/SECURITY.md) · [Deployment matrix →](./docs/deployment_matrix.md) · [Deployment docs →](./runtime/readme.md) · [Agent integration →](https://modelscope.github.io/FunASR/agent.html)

Expand All @@ -330,6 +330,7 @@ asset is published.

## What's new

- 2026/08/29: **llama.cpp runtime v0.2.5** — uploads host weights into the selected Vulkan backend buffer before graph execution. Q8 and F16 weights pass local Linux Vulkan llvmpipe validation, and all nine Linux, macOS, and Windows archives were built from exact commit `f371370d4c5e4c61d13d4eb9c55cda2f4dd95e4f` and verified against their published SHA-256 values. This does not claim the AMD Windows hardware crash is fixed; [#3479](https://github.com/modelscope/FunASR/issues/3479) remains open for reporter retesting. [Fix ->](https://github.com/modelscope/FunASR/pull/3555) · [Release ->](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.5)
- 2026/08/29: **llama.cpp runtime v0.2.4** — fixes intermittent empty SenseVoiceSmall transcripts with F16 GGUF models. Query embeddings are now decoded according to their GGML F16/F32 type instead of reading F16 storage as F32. The exact v0.2.3 AVX2 asset reproduced 22 blank outputs across 298 completed runs; the fixed head produced one identical non-empty result in 100/100 runs and matched the Q8 model byte-for-byte. The release workflow will build nine Linux, macOS, and Windows archives from one exact commit. [Fix ->](https://github.com/modelscope/FunASR/pull/3550) · [Release ->](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4)
- 2026/08/29: **v1.4.6 on PyPI** — the realtime WebSocket server keeps 20-second pings but no longer closes healthy sessions by default when queued decoding delays pong handling; operators can still configure a positive timeout after measuring production queue and decode latency. This release also avoids `IndexError` on empty English timestamp segments and groups adjacent subtitle words into readable cues. Upgrade with `python -m pip install -U "funasr==1.4.6"`. The GitHub Release carries the verified nine-platform llama.cpp v0.2.3 runtime archives. [Release ->](https://github.com/modelscope/FunASR/releases/tag/v1.4.6)
- 2026/08/29: **llama.cpp runtime v0.2.3** — adds flushed stage boundaries after backend initialization for model loading, audio/VAD, graph construction and allocation, and compute. These diagnostics narrow Windows AMD Vulkan `0xC0000005` failures that occur after `vulkan backend ready`; they do not claim the hardware-specific crash is fixed. Nine Linux, macOS, and Windows archives are published from one exact commit. [Troubleshooting boundaries ->](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [Release ->](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3)
Expand Down
1 change: 1 addition & 0 deletions README_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ Whisper は単一モデルですが、**FunASR はツールキット**です—

## 最新情報

- 2026/08/29:**llama.cpp runtime v0.2.5** — graph 実行前に host weight を選択された Vulkan backend buffer へ upload します。Q8/F16 weight は Linux Vulkan llvmpipe で検証済みで、Linux、macOS、Windows 向け 9 archive は exact commit `f371370d4c5e4c61d13d4eb9c55cda2f4dd95e4f` から build し、公開 SHA-256 と照合しました。AMD Windows hardware crash の修正済みとはしておらず、[#3479](https://github.com/modelscope/FunASR/issues/3479) は reporter の実機再テスト待ちで open のままです。[修正 →](https://github.com/modelscope/FunASR/pull/3555) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.5)
- 2026/08/29:**llama.cpp runtime v0.2.4** — F16 GGUF model で transcript が断続的に空になる問題を修正しました。Query embedding は F16 storage を F32 として読むのではなく、GGML の F16/F32 type に従って decode されます。正確な v0.2.3 AVX2 asset では 298 回の完了 run 中 22 回の空出力を再現しましたが、修正後は 100/100 回で同一の非空出力となり、Q8 model と byte 単位で一致しました。Release workflow は同一の exact commit から 9 platform archive を構築します。[修正 →](https://github.com/modelscope/FunASR/pull/3550) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4)
- 2026/08/29:**v1.4.6 を PyPI に公開** — realtime WebSocket server は 20 秒間隔の ping を維持しつつ、decode queue によって pong 処理が遅れた正常な session を default では切断しなくなりました。運用環境の queue と decode latency を測定したうえで、正の timeout を明示設定することもできます。空の英語 timestamp segment による `IndexError` を防ぎ、隣接する subtitle word を読みやすい cue にまとめます。更新:`python -m pip install -U "funasr==1.4.6"`。GitHub Release には検証済みの llama.cpp v0.2.3 9 platform runtime archive も含まれます。[Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.6)
- 2026/08/29:**llama.cpp runtime v0.2.3** — backend 初期化後に、model 読み込み、audio/VAD、graph の構築と割り当て、compute の各段階を即時 flush する境界ログを追加しました。`vulkan backend ready` 後に起きる Windows AMD Vulkan `0xC0000005` の発生箇所を絞り込むための診断であり、hardware 固有の crash が修正済みとはしていません。同一の exact commit から Linux、macOS、Windows 向け 9 archive を公開します。[トラブルシューティング →](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3)
Expand Down
1 change: 1 addition & 0 deletions README_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ Whisper는 단일 모델이지만, **FunASR는 툴킷**입니다. 용도에 맞

## 최신 소식

- 2026/08/29: **llama.cpp runtime v0.2.5** — graph 실행 전에 host weight를 선택한 Vulkan backend buffer에 upload합니다. Q8/F16 weight는 Linux Vulkan llvmpipe에서 검증했으며, Linux, macOS, Windows용 archive 9개를 exact commit `f371370d4c5e4c61d13d4eb9c55cda2f4dd95e4f`에서 build하고 공개 SHA-256과 대조했습니다. AMD Windows hardware crash가 수정됐다고 주장하지 않으며, [#3479](https://github.com/modelscope/FunASR/issues/3479)는 제보자의 실제 hardware 재검증을 위해 open 상태를 유지합니다. [수정 →](https://github.com/modelscope/FunASR/pull/3555) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.5)
- 2026/08/29: **llama.cpp runtime v0.2.4** — F16 GGUF model에서 transcript가 간헐적으로 비는 문제를 수정했습니다. Query embedding은 F16 storage를 F32로 읽지 않고 GGML F16/F32 type에 맞게 decode됩니다. 정확한 v0.2.3 AVX2 asset은 완료된 298회 실행 중 22회의 빈 출력을 재현했지만, 수정 후에는 100/100회 동일한 비어 있지 않은 결과를 냈고 Q8 model과 byte 단위로 일치했습니다. Release workflow가 동일한 exact commit에서 9개 platform archive를 빌드합니다. [수정 →](https://github.com/modelscope/FunASR/pull/3550) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4)
- 2026/08/29: **v1.4.6 PyPI 공개** — realtime WebSocket server는 20초 간격 ping을 유지하면서 decode queue 때문에 pong 처리가 늦어진 정상 session을 기본적으로 종료하지 않습니다. 운영 환경의 queue 및 decode latency를 측정한 뒤 양수 timeout을 명시적으로 설정할 수도 있습니다. 빈 영어 timestamp segment에서 발생하는 `IndexError`를 방지하고 인접 subtitle word를 읽기 쉬운 cue로 묶습니다. 업데이트: `python -m pip install -U "funasr==1.4.6"`. GitHub Release에는 검증된 llama.cpp v0.2.3 9개 platform runtime archive도 포함됩니다. [Release →](https://github.com/modelscope/FunASR/releases/tag/v1.4.6)
- 2026/08/29: **llama.cpp runtime v0.2.3** — backend 초기화 이후 model load, audio/VAD, graph build와 allocation, compute 단계마다 즉시 flush되는 경계 로그를 추가했습니다. `vulkan backend ready` 이후 발생하는 Windows AMD Vulkan `0xC0000005`의 위치를 좁히기 위한 진단이며, hardware별 crash가 수정됐다고 주장하지 않습니다. 동일한 exact commit에서 Linux, macOS, Windows용 archive 9개를 공개합니다. [문제 해결 경계 →](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [Release →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3)
Expand Down
3 changes: 2 additions & 1 deletion README_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,6 +149,7 @@ Whisper 是单个模型,**FunASR 是一个工具箱**——按场景挑模型

## 最新动态

- 2026/08/29:**llama.cpp runtime v0.2.5** — 在计算图执行前,将 host 权重上传到所选 Vulkan backend buffer。Q8 与 F16 权重已通过本地 Linux Vulkan llvmpipe 验证,九个 Linux、macOS 与 Windows 压缩包均由准确提交 `f371370d4c5e4c61d13d4eb9c55cda2f4dd95e4f` 构建,并按公开 SHA-256 逐一复核。本版本不宣称 AMD Windows 硬件崩溃已经修复;[#3479](https://github.com/modelscope/FunASR/issues/3479) 保持开放,等待报告者实机复测。[修复 →](https://github.com/modelscope/FunASR/pull/3555) · [发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.5)
- 2026/08/29:**llama.cpp runtime v0.2.4** — 修复 F16 GGUF 模型偶发空转写。运行时现在按 GGML 的 F16/F32 类型解码查询 embedding,不再把 F16 存储误读为 F32。精确的 v0.2.3 AVX2 发布包在 298 次完整运行中复现 22 次空结果;修复后 100/100 次输出稳定唯一,并与 Q8 模型逐字节一致。发布工作流将在同一准确提交上构建九个 Linux、macOS 与 Windows 压缩包。[修复 →](https://github.com/modelscope/FunASR/pull/3550) · [发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4)
- 2026/08/29:**v1.4.6 已发布到 PyPI** — 实时 WebSocket 服务继续每 20 秒发送 ping,但当排队解码延迟 pong 处理时,默认不再误关仍健康的连接;运维人员仍可在测量生产队列与解码延迟后显式配置正数超时。本版本同时避免空英文时间戳片段触发 `IndexError`,并将相邻字幕词组合为更易读的字幕段。升级命令:`python -m pip install -U "funasr==1.4.6"`。GitHub 发布页同时提供已验证的 llama.cpp v0.2.3 九平台运行包。[发布页 →](https://github.com/modelscope/FunASR/releases/tag/v1.4.6)
- 2026/08/29:**llama.cpp runtime v0.2.3** — 后端初始化完成后新增可立即刷新的阶段边界,覆盖模型加载、音频/VAD、计算图构建与分配、推理计算,可继续缩小发生在 `vulkan backend ready` 之后的 Windows AMD Vulkan `0xC0000005` 崩溃范围;本版本不宣称已经修复该硬件相关崩溃。九个 Linux、macOS 与 Windows 压缩包由同一准确提交构建并发布。[排障边界 →](./runtime/llama.cpp/README.md#optional-windows-vulkan-backend-for-sensevoicesmall) · [发布页 →](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.3)
Expand Down Expand Up @@ -364,7 +365,7 @@ Windows Vulkan ZIP 使用显卡驱动提供的系统 Vulkan loader,不需要
compute capability 12.0(`sm_120`),在专用 CUDA 产物发布前,请使用 CPU 包,或从
源码构建并设置 `-DCMAKE_CUDA_ARCHITECTURES=120`。

**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.4](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.4) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.4/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/deploy/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/)
**预编译二进制:** [Releases](https://github.com/modelscope/FunASR/releases) · [v0.2.5](https://github.com/modelscope/FunASR/releases/tag/runtime-llamacpp-v0.2.5) · [Linux Vulkan tarball](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.5/funasr-llamacpp-linux-x64-vulkan.tar.gz) · [Windows Vulkan zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.5/funasr-llamacpp-windows-x64-vulkan.zip) · [Windows CUDA zip](https://github.com/modelscope/FunASR/releases/download/runtime-llamacpp-v0.2.5/funasr-llamacpp-windows-x64-cuda.zip) · **下载与快速开始:** [funasr.com/deploy/llama-cpp](https://www.funasr.com/deploy/llama-cpp.html) · **GGUF 模型:** [Hugging Face](https://huggingface.co/FunAudioLLM) · **文档与评测:** [runtime/llama.cpp/](./runtime/llama.cpp/)

[OpenAI API 示例 →](./examples/openai_api/README_zh.md) · [Gradio Demo →](./examples/openai_api/GRADIO_zh.md) · [客户端配方 →](./examples/openai_api/CLIENTS.md) · [JavaScript/TypeScript 配方 →](./examples/openai_api/JAVASCRIPT_zh.md) · [Kubernetes 模板 →](./examples/openai_api/kubernetes/README_zh.md) · [工作流配方 →](./examples/openai_api/WORKFLOWS_zh.md) · [Postman 集合 →](./examples/openai_api/POSTMAN_zh.md) · [OpenAPI 规范 →](./examples/openai_api/OPENAPI_zh.md) · [安全指南 →](./examples/openai_api/SECURITY_zh.md) · [部署选型 →](./docs/deployment_matrix_zh.md) · [部署文档 →](./runtime/readme_cn.md) · [Agent 集成 →](https://modelscope.github.io/FunASR/agent.html)

Expand Down
2 changes: 1 addition & 1 deletion runtime/llama.cpp/current-release.txt
Original file line number Diff line number Diff line change
@@ -1 +1 @@
runtime-llamacpp-v0.2.4
runtime-llamacpp-v0.2.5
4 changes: 2 additions & 2 deletions runtime/llama.cpp/tests/test_download_funasr_model.sh
Original file line number Diff line number Diff line change
Expand Up @@ -170,8 +170,8 @@ assert_readme_quickstart() {
printf 'non-portable Releases link in %s\n' "$readme" >&2
exit 1
fi
if ! grep -F 'runtime-llamacpp-v0.2.4' <<<"$section" >/dev/null; then
printf 'missing current runtime v0.2.4 link in %s\n' "$readme" >&2
if ! grep -F 'runtime-llamacpp-v0.2.5' <<<"$section" >/dev/null; then
printf 'missing current runtime v0.2.5 link in %s\n' "$readme" >&2
exit 1
fi
if ! grep -F 'funasr-llamacpp-linux-x64-vulkan' <<<"$section" >/dev/null; then
Expand Down
Loading
Loading