Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/deployment_matrix.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ Use this page to choose the shortest deployment path for a product, demo, benchm
| Runtime WebSocket service | Live captions, meetings, call-center streams | [Runtime service docs](../runtime/readme.md) | Use when partial results, endpointing, or long-lived audio streams matter. |
| ONNX/C++ runtime | High-concurrency CPU services or embedded realtime ASR | [ONNX runtime docs](../runtime/onnxruntime/readme.md) | Keep this path when latency/concurrency is already proven; add text post-processing for fixed business terms before moving to GPU LLMs. |
| vLLM acceleration | Higher-throughput LLM-based ASR with Fun-ASR-Nano | [vLLM guide](./vllm_guide.md) | Use for LLM decoder throughput; does not apply to non-autoregressive Paraformer. |
| MOSS-Transcribe-Diarize | Long-form multi-speaker transcription with timestamps and speaker labels | [Third-party MOSS deployment guide](./moss_transcribe_diarize.md) | OpenMOSS Apache-2.0 model served with vLLM or SGLang Omni; not a FunASR-owned model or `AutoModel` backend. |
| MCP server | Claude/Cursor/desktop agent speech tools | [MCP example](../examples/mcp_server/) | Good when the ASR result should be exposed as a local tool. |
| Subtitle generator | SRT/VTT from long audio or video | [Subtitle example](../examples/subtitle/) | Use verbose segments and speaker labels when readability matters. |
| Batch ASR script | Archives, meetings, datasets, repeated offline runs | [Batch example](../examples/batch_asr_improved.py) | Add queueing, manifests, and retry logs for production use. |
Expand Down
1 change: 1 addition & 0 deletions docs/deployment_matrix_ja.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
| Kubernetes API | Cluster service 向け internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP` から開始。公開範囲を広げる前に auth、TLS、network policy、GPU scheduling を追加します。 |
| Runtime WebSocket service | Live captions、meeting、call-center stream | [Runtime service docs](../runtime/readme.md) | partial result、endpointing、long-lived audio stream が重要な場合に使います。 |
| vLLM acceleration | Fun-ASR-Nano の LLM-based ASR throughput 向上 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput 向け。non-autoregressive Paraformer には適用しません。 |
| MOSS-Transcribe-Diarize | 長時間の複数話者 transcription、timestamp、speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS の Apache-2.0 model を vLLM / SGLang Omni で提供します。FunASR 所有 model や `AutoModel` backend ではありません。 |
| MCP server | Claude/Cursor/desktop agent の speech tool | [MCP example](../examples/mcp_server/) | ASR 結果を local tool として Agent に渡したい場合に便利です。 |
| Subtitle generator | 長時間 audio/video から SRT/VTT 作成 | [Subtitle example](../examples/subtitle/) | readability が重要な場合は verbose segment と speaker label を使います。 |
| Batch ASR script | Archive、meeting、dataset、繰り返し offline run | [Batch example](../examples/batch_asr_improved.py) | production では queue、manifest、retry log を追加してください。 |
Expand Down
1 change: 1 addition & 0 deletions docs/deployment_matrix_ko.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
| Kubernetes API | Cluster service용 internal speech API | [Kubernetes template](../examples/openai_api/kubernetes/) | private `ClusterIP`부터 시작합니다. 범위를 넓히기 전에 auth, TLS, network policy, GPU scheduling을 추가하세요. |
| Runtime WebSocket service | Live captions, meeting, call-center stream | [Runtime service docs](../runtime/readme.md) | partial result, endpointing, long-lived audio stream이 중요할 때 사용합니다. |
| vLLM acceleration | Fun-ASR-Nano의 LLM-based ASR throughput 향상 | [vLLM guide](./vllm_guide.md) | LLM decoder throughput용입니다. non-autoregressive Paraformer에는 적용되지 않습니다. |
| MOSS-Transcribe-Diarize | 긴 다중 화자 transcription, timestamp, speaker label | [Third-party MOSS guide](./moss_transcribe_diarize.md) | OpenMOSS의 Apache-2.0 model을 vLLM / SGLang Omni로 제공합니다. FunASR 소유 model 또는 `AutoModel` backend가 아닙니다. |
| MCP server | Claude/Cursor/desktop agent speech tool | [MCP example](../examples/mcp_server/) | ASR 결과를 local tool로 Agent에 전달할 때 유용합니다. |
| Subtitle generator | 긴 audio/video에서 SRT/VTT 생성 | [Subtitle example](../examples/subtitle/) | readability가 중요하면 verbose segment와 speaker label을 사용합니다. |
| Batch ASR script | Archive, meeting, dataset, 반복 offline run | [Batch example](../examples/batch_asr_improved.py) | production에서는 queue, manifest, retry log를 추가하세요. |
Expand Down
1 change: 1 addition & 0 deletions docs/deployment_matrix_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@
| Runtime WebSocket 服务 | 实时字幕、会议、客服流式音频 | [Runtime 服务文档](../runtime/readme_cn.md) | 需要中间结果、断句或长连接音频流时选择。 |
| ONNX/C++ Runtime | 高并发 CPU 服务或嵌入式实时 ASR | [ONNX Runtime 文档](../runtime/onnxruntime/readme.md) | 如果延迟和并发已经验证,不要轻易替换;固定业务词优先做文本后处理。 |
| vLLM 加速 | Fun-ASR-Nano 等 LLM-based ASR 高吞吐 | [vLLM 指南](./vllm_guide.md) | 适合 LLM 解码吞吐;不适用于非自回归 Paraformer。 |
| MOSS-Transcribe-Diarize | 长音频多人转写、时间戳和说话人标签 | [第三方 MOSS 部署指南](./moss_transcribe_diarize.md) | OpenMOSS 以 Apache-2.0 发布,通过 vLLM 或 SGLang Omni 服务;不是 FunASR 自有模型或 `AutoModel` 后端。 |
| MCP 服务 | Claude/Cursor/桌面 Agent 语音工具 | [MCP 示例](../examples/mcp_server/) | 适合把 ASR 结果暴露成一个本地工具。 |
| 字幕生成 | 从长音频或视频生成 SRT/VTT | [字幕示例](../examples/subtitle/) | 需要可读性时使用 verbose segments 和说话人标签。 |
| 批处理脚本 | 录音归档、会议纪要、数据集处理 | [批处理示例](../examples/batch_asr_improved.py) | 生产使用时建议增加队列、manifest 和重试日志。 |
Expand Down
153 changes: 153 additions & 0 deletions docs/moss_transcribe_diarize.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,153 @@
# MOSS-Transcribe-Diarize in the FunASR ecosystem

[中文](./moss_transcribe_diarize_zh.md)

This guide connects the third-party
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
model to the FunASR deployment ecosystem. The model is published by OpenMOSS
under Apache-2.0; it is not a FunASR model and is not registered in FunASR
`AutoModel`.

MOSS-Transcribe-Diarize jointly generates transcription, timestamps, and
speaker labels such as `[S01]`. An application therefore does not need to
assemble an external VAD, ASR, and diarization pipeline. This is a deployment
property, not a claim that the model has no internal segmentation or chunking.

## Pinned sources

This guide was verified against:

- source: `OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`
- model: `OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`
- license: Apache-2.0 in the pinned upstream source and model metadata
- vLLM nightly index: `68b4a1d582818e67adc903bf1b8fc5a5447da2fa`

Pin all three revisions. The model uses `trust_remote_code`; do not execute a
floating model revision in a production service.

## Choose a serving path

| Path | Environment | Response contract | Use when |
|---|---|---|---|
| vLLM | CUDA 12 (`cu129`) or CUDA 13 (`cu130`) | `response_format=json` returns raw `[start][Sxx]text[end]` text | You already operate vLLM or need its scheduler |
| SGLang Omni | CUDA 13 in the current upstream guide | `response_format=verbose_json` returns parsed segments | You need structured speaker segments directly from the API |
| Transformers | PyTorch process | Python objects and raw tagged text | Evaluation, debugging, or custom preprocessing |

The two HTTP backends share `/v1/audio/transcriptions`, but their documented
response formats are not interchangeable. Do not promise vLLM `segments`
without testing the exact vLLM revision.

## vLLM

Create an isolated environment and install the upstream-pinned nightly build:

```bash
uv venv --python 3.12 .venv-moss
uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \
--torch-backend=auto \
--extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
```

The `audio` extra is required. A plain `vllm` install can start the server but
returns HTTP 400 (`Invalid or unsupported audio file`) because no audio decoder
is installed.

Start the service with the immutable model revision:

```bash
CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \
OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
--served-model-name moss-transcribe-diarize \
--trust-remote-code \
--host 127.0.0.1 \
--port 8898
```

Submit audio and validate the speaker-tagged text:

```bash
curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=moss-transcribe-diarize \
-F response_format=json \
-F temperature=0 \
| tee moss-transcription.json

python - <<'PY'
import json

with open("moss-transcription.json", encoding="utf-8") as stream:
payload = json.load(stream)
text = payload.get("text", "")
assert text.strip(), payload
assert "[S01]" in text, text
print(text)
PY
```

For long recordings, test `max_completion_tokens` against the longest expected
meeting. A larger limit can increase memory use and tail latency.

### Reproduced vLLM contract

The FunASR ecosystem validation used vLLM
`0.23.1rc1.dev949+g68b4a1d58`, Torch `2.11.0+cu129`, and one H100 80GB:

- the bundled 6.000-second sample
(`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`)
returned HTTP 200 and `[0.96][S01]... [5.94]`;
- an A + 0.8-second silence + B + 0.8-second silence + A probe
(`dbb32bcfed2e8226bedf64248a9f4a44685b293a4696d18fb4cfa701b04db912`)
returned HTTP 200 with `S01 -> S02 -> S01` and timestamps through 19.08 seconds.

This proves the pinned API and speaker-return contract on those inputs. It is
not a diarization accuracy result, overlap test, throughput benchmark, or
production capacity claim.

## SGLang Omni

Follow the pinned upstream SGLang Omni installation guide for CUDA 13, then
download the immutable model snapshot and serve the local directory:

```bash
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
--local-dir .models/moss-transcribe-diarize

sgl-omni serve \
--model-path .models/moss-transcribe-diarize \
--port 8898 \
--max-running-requests 16 \
--cuda-graph-max-bs 16 \
--mem-fraction-static 0.80
```

Request parsed segments:

```bash
curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
-F response_format=verbose_json
```

Verify every segment has start/end timing, text, and the expected speaker
field before wiring the response into subtitles, meeting notes, or analytics.

## Production validation

Use real multi-speaker audio, not only a short single-speaker smoke sample:

- measure speaker consistency across long turns and speaker returns;
- test overlap, crosstalk, music, noise, and long silence;
- verify timestamp monotonicity and full-audio coverage;
- record GPU, CUDA, Torch, backend commit, model revision, audio duration,
generation limit, wall latency, and peak memory;
- keep authentication, TLS, request limits, and retention policy at the API
gateway; bind the model worker to a private address.

When reporting a problem to FunASR, state that this is the OpenMOSS third-party
path and include the exact backend and upstream revisions. Model architecture or
weight issues belong upstream; deployment-center contract and documentation
issues can be filed in FunASR.
141 changes: 141 additions & 0 deletions docs/moss_transcribe_diarize_zh.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
# 在 FunASR 生态中部署 MOSS-Transcribe-Diarize

[English](./moss_transcribe_diarize.md)

本文把第三方
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
模型接入 FunASR 部署生态。模型由 OpenMOSS 以 Apache-2.0 发布,不是 FunASR
自有模型,也没有注册到 FunASR `AutoModel`。

MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话人标签,应用侧
不必再拼接外部 VAD、ASR 和 diarization 管线。这里描述的是部署形态,不表示模型
内部没有分块或分段。

## 固定上游版本

本文按以下不可变版本验证:

- 源码:`OpenMOSS/MOSS-Transcribe-Diarize@cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3`
- 模型:`OpenMOSS-Team/MOSS-Transcribe-Diarize@e8681d68e7042738ffca8ac8212bc8fcb1131ab8`
- 许可证:固定上游源码和模型元数据中的 Apache-2.0
- vLLM nightly 索引:`68b4a1d582818e67adc903bf1b8fc5a5447da2fa`

三者都要固定。模型依赖 `trust_remote_code`,生产服务不要执行浮动的模型 revision。

## 选择服务后端

| 路径 | 环境 | 响应契约 | 适用场景 |
|---|---|---|---|
| vLLM | CUDA 12(`cu129`)或 CUDA 13(`cu130`) | `response_format=json` 返回原始 `[start][Sxx]text[end]` 文本 | 已运行 vLLM,或需要其调度能力 |
| SGLang Omni | 当前上游指南为 CUDA 13 | `response_format=verbose_json` 返回解析后的 segments | API 需要直接返回结构化说话人分段 |
| Transformers | PyTorch 进程 | Python 对象和原始标签文本 | 评测、排障或自定义预处理 |

两个 HTTP 后端都使用 `/v1/audio/transcriptions`,但官方文档中的响应格式不能混为
一谈。没有对 exact vLLM revision 做验证时,不要承诺 vLLM 返回 `segments`。

## vLLM

创建隔离环境并安装上游固定的 nightly build:

```bash
uv venv --python 3.12 .venv-moss
uv pip install --python .venv-moss/bin/python -U 'vllm[audio]' \
--torch-backend=auto \
--extra-index-url https://wheels.vllm.ai/68b4a1d582818e67adc903bf1b8fc5a5447da2fa/cu129
```

必须安装 `audio` extra。只安装 plain `vllm` 时服务可以启动,但因为没有音频解码器,
请求会返回 HTTP 400(`Invalid or unsupported audio file`)。

使用不可变模型 revision 启动:

```bash
CUDA_VISIBLE_DEVICES=0 .venv-moss/bin/vllm serve \
OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
--served-model-name moss-transcribe-diarize \
--trust-remote-code \
--host 127.0.0.1 \
--port 8898
```

提交音频并验证说话人标签文本:

```bash
curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=moss-transcribe-diarize \
-F response_format=json \
-F temperature=0 \
| tee moss-transcription.json

python - <<'PY'
import json

with open("moss-transcription.json", encoding="utf-8") as stream:
payload = json.load(stream)
text = payload.get("text", "")
assert text.strip(), payload
assert "[S01]" in text, text
print(text)
PY
```

长音频需要按业务最长会议验证 `max_completion_tokens`。增大上限也会影响显存和尾延迟。

### 已复现的 vLLM 契约

FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
`2.11.0+cu129` 和一张 H100 80GB:

- 仓库内 6.000 秒样例
(`ea03e1f473ad1618a03da3327a545369cb8f6f06cb0f4115535e5a866167d47e`)
返回 HTTP 200 和 `[0.96][S01]... [5.94]`;
- A + 0.8 秒静音 + B + 0.8 秒静音 + A 的拼接样例
(`dbb32bcfed2e8226bedf64248a9f4a44685b293a4696d18fb4cfa701b04db912`)
返回 HTTP 200、`S01 -> S02 -> S01`,时间戳覆盖到 19.08 秒。

这证明固定版本对这些输入满足 API 和说话人返回契约,不是 diarization 精度、重叠语音、
吞吐或生产容量结论。

## SGLang Omni

按固定上游 SGLang Omni 安装指南准备 CUDA 13 环境,然后下载不可变模型快照并从本地目录启动:

```bash
hf download OpenMOSS-Team/MOSS-Transcribe-Diarize \
--revision e8681d68e7042738ffca8ac8212bc8fcb1131ab8 \
--local-dir .models/moss-transcribe-diarize

sgl-omni serve \
--model-path .models/moss-transcribe-diarize \
--port 8898 \
--max-running-requests 16 \
--cuda-graph-max-bs 16 \
--mem-fraction-static 0.80
```

请求结构化分段:

```bash
curl -fsS http://127.0.0.1:8898/v1/audio/transcriptions \
-F file=@audio.wav \
-F model=OpenMOSS-Team/MOSS-Transcribe-Diarize \
-F response_format=verbose_json
```

接入字幕、会议纪要或分析系统前,验证每个 segment 都包含起止时间、文本和预期的 speaker 字段。

## 上线前验证

不要只用短单人音频,应使用真实多人长音频:

- 检查长轮次及说话人再次出现时的身份一致性;
- 覆盖重叠语音、串音、音乐、噪声和长静音;
- 验证时间戳单调递增并覆盖完整音频;
- 记录 GPU、CUDA、Torch、后端 commit、模型 revision、音频时长、生成上限、
墙钟延迟和峰值显存;
- 在 API 网关完成认证、TLS、请求限制和数据留存策略,模型 worker 只绑定内网。

向 FunASR 报告问题时,请明确这是 OpenMOSS 第三方路径,并附上后端与上游 exact
revision。模型架构或权重问题应反馈给上游;部署中心的契约或文档问题可以在 FunASR 提交。
50 changes: 50 additions & 0 deletions tests/test_moss_transcribe_diarize_docs.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
from pathlib import Path

import pytest


ROOT = Path(__file__).resolve().parents[1]
GUIDES = (
ROOT / "docs" / "moss_transcribe_diarize.md",
ROOT / "docs" / "moss_transcribe_diarize_zh.md",
)
MATRICES = (
ROOT / "docs" / "deployment_matrix.md",
ROOT / "docs" / "deployment_matrix_zh.md",
ROOT / "docs" / "deployment_matrix_ja.md",
ROOT / "docs" / "deployment_matrix_ko.md",
)


@pytest.mark.parametrize("guide", GUIDES)
def test_moss_guides_pin_upstream_and_separate_serving_contracts(guide: Path) -> None:
text = guide.read_text(encoding="utf-8")

for marker in (
"OpenMOSS/MOSS-Transcribe-Diarize",
"cb765f2b0fe6f7a298aa2002e2281ae693d1f3c3",
"OpenMOSS-Team/MOSS-Transcribe-Diarize",
"e8681d68e7042738ffca8ac8212bc8fcb1131ab8",
"68b4a1d582818e67adc903bf1b8fc5a5447da2fa",
"vllm[audio]",
"0.23.1rc1.dev949+g68b4a1d58",
"dbb32bcfed2e8226bedf64248a9f4a44685b293a4696d18fb4cfa701b04db912",
"S01 -> S02 -> S01",
"/v1/audio/transcriptions",
"response_format=json",
"response_format=verbose_json",
"[S01]",
"Apache-2.0",
):
assert marker in text, f"{guide.name} is missing {marker}"

assert "AutoModel" in text
assert "third-party" in text.lower() or "第三方" in text


@pytest.mark.parametrize("matrix", MATRICES)
def test_all_deployment_matrices_link_moss_guide(matrix: Path) -> None:
text = matrix.read_text(encoding="utf-8")

assert "MOSS-Transcribe-Diarize" in text
assert "moss_transcribe_diarize.md" in text
Loading
Loading