Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 53 additions & 0 deletions .github/workflows/test-moss-adapter.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
name: Validate MOSS adapter

on:
pull_request:
paths:
- "funasr/models/moss_transcribe_diarize/**"
- "funasr/auto/auto_model.py"
- "docs/moss_transcribe_diarize*.md"
- "tests/test_moss_transcribe_diarize_*.py"
- ".github/workflows/test-moss-adapter.yml"
push:
branches:
- main
paths:
- "funasr/models/moss_transcribe_diarize/**"
- "funasr/auto/auto_model.py"
- "docs/moss_transcribe_diarize*.md"
- "tests/test_moss_transcribe_diarize_*.py"
- ".github/workflows/test-moss-adapter.yml"

permissions:
contents: read

jobs:
contract:
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.11"
cache: pip
- name: Install contract-test dependencies
run: |
python -m pip install --upgrade pip
python -m pip install \
torch --index-url https://download.pytorch.org/whl/cpu
python -m pip install \
pytest black==25.1.0 numpy soundfile requests packaging \
omegaconf hydra-core kaldiio librosa tqdm pyyaml
- name: Check formatting and syntax
run: |
python -m black --check \
funasr/models/moss_transcribe_diarize/model.py \
tests/test_moss_transcribe_diarize_model.py \
tests/test_moss_transcribe_diarize_docs.py
python -m py_compile funasr/models/moss_transcribe_diarize/model.py
- name: Run adapter and documentation contracts
run: |
python -m pytest -q \
tests/test_moss_transcribe_diarize_model.py \
tests/test_moss_transcribe_diarize_docs.py
72 changes: 70 additions & 2 deletions docs/moss_transcribe_diarize.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@
This guide connects the third-party
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
model to the FunASR deployment ecosystem. The model is published by OpenMOSS
under Apache-2.0; it is not a FunASR model and is not registered in FunASR
`AutoModel`.
under Apache-2.0; it is not a FunASR model. FunASR provides an adapter for its
public Transformers and vLLM interfaces while retaining the OpenMOSS model
name, license, and upstream revision.

MOSS-Transcribe-Diarize jointly generates transcription, timestamps, and
speaker labels such as `[S01]`. An application therefore does not need to
Expand All @@ -25,6 +26,65 @@ This guide was verified against:
Pin all three revisions. The model uses `trust_remote_code`; do not execute a
floating model revision in a production service.

## FunASR AutoModel contract

Use an isolated Python 3.10+ environment with Transformers 5.6 or newer for
the local backend. MOSS performs long-form transcription and speaker
diarization in one generation, so do **not** pass `vad_model` or `spk_model`.
External VAD segmentation would break the model's global speaker identity
across chunks.

```python
from funasr import AutoModel

model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
model_revision="e8681d68e7042738ffca8ac8212bc8fcb1131ab8",
backend="hf",
device="cuda:0",
dtype="bf16",
attn_implementation="sdpa",
disable_update=True,
)

result = model.generate("audio.wav", max_new_tokens=5120)[0]
print(result["text"])
for segment in result["sentence_info"]:
print(segment["start"], segment["end"], segment["spk"], segment["text"])
```

The adapter preserves the raw tagged generation in `raw_text` and returns the
common FunASR fields:

- `text`: readable transcript without MOSS control tags;
- `timestamp`: segment-level `[start_ms, end_ms]` pairs;
- `sentence_info`: `start`, `end`, `text`, `sentence`, `spk`, and `timestamp`;
- `raw_text`: exact `[start][Sxx]text[end]` generation for auditing.

If the parser cannot prove the tagged structure, it leaves the model text
visible in `text` and `raw_text` and returns empty timestamp/segment arrays
instead of silently inventing speaker metadata.

The same result contract can wrap an already running vLLM service without
downloading local weights:

```python
from funasr import AutoModel

model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
backend="vllm",
vllm_base_url="http://127.0.0.1:8898/v1",
vllm_model="moss-transcribe-diarize",
disable_update=True,
)
result = model.generate("audio.wav")[0]
```

The vLLM adapter sends the documented OpenAI-compatible multipart request and
normalizes the returned raw text. Authentication can be supplied with
`vllm_api_key`; keep it in a secret store rather than source code.

## Choose a serving path

| Path | Environment | Response contract | Use when |
Expand Down Expand Up @@ -105,6 +165,14 @@ This proves the pinned API and speaker-return contract on those inputs. It is
not a diarization accuracy result, overlap test, throughput benchmark, or
production capacity claim.

The FunASR adapter was also tested on both `backend="hf"` and
`backend="vllm"` with model revision
`e8681d68e7042738ffca8ac8212bc8fcb1131ab8`. The 15.1685-second two-speaker
probe (`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)
returned two monotonic segments labelled `S01` and `S02` through the common
`AutoModel` result contract. The temporary vLLM worker was stopped after the
test; this is a contract smoke test, not an accuracy benchmark.

## SGLang Omni

Follow the pinned upstream SGLang Omni installation guide for CUDA 13, then
Expand Down
62 changes: 61 additions & 1 deletion docs/moss_transcribe_diarize_zh.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,8 @@
本文把第三方
[OpenMOSS/MOSS-Transcribe-Diarize](https://github.com/OpenMOSS/MOSS-Transcribe-Diarize)
模型接入 FunASR 部署生态。模型由 OpenMOSS 以 Apache-2.0 发布,不是 FunASR
自有模型,也没有注册到 FunASR `AutoModel`。
自有模型。FunASR 只为其公开的 Transformers 与 vLLM 接口提供适配器,并保留
OpenMOSS 模型名称、许可证和上游 revision。

MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话人标签,应用侧
不必再拼接外部 VAD、ASR 和 diarization 管线。这里描述的是部署形态,不表示模型
Expand All @@ -22,6 +23,59 @@ MOSS-Transcribe-Diarize 会联合生成转写、时间戳和 `[S01]` 等说话

三者都要固定。模型依赖 `trust_remote_code`,生产服务不要执行浮动的模型 revision。

## FunASR AutoModel 契约

本地后端应使用隔离的 Python 3.10+ 环境,并安装 Transformers 5.6 或更新版本。
MOSS 在一次生成中完成长音频转写与说话人识别,因此不要传 `vad_model` 或
`spk_model`。外部 VAD 会把长音频切开,并破坏跨分块的全局说话人身份。

```python
from funasr import AutoModel

model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
model_revision="e8681d68e7042738ffca8ac8212bc8fcb1131ab8",
backend="hf",
device="cuda:0",
dtype="bf16",
attn_implementation="sdpa",
disable_update=True,
)

result = model.generate("audio.wav", max_new_tokens=5120)[0]
print(result["text"])
for segment in result["sentence_info"]:
print(segment["start"], segment["end"], segment["spk"], segment["text"])
```

适配器在 `raw_text` 中保留原始标签生成,并返回 FunASR 通用字段:

- `text`:移除 MOSS 控制标签后的可读转写;
- `timestamp`:segment 级 `[start_ms, end_ms]`;
- `sentence_info`:包含 `start`、`end`、`text`、`sentence`、`spk` 和 `timestamp`;
- `raw_text`:用于审计的原始 `[start][Sxx]text[end]` 生成。

当解析器无法确认标签结构时,会保留 `text` 与 `raw_text`,并返回空时间戳和分段,
不会静默编造说话人信息。

同一结果契约也可以包装已经启动的 vLLM 服务,并且不会下载本地权重:

```python
from funasr import AutoModel

model = AutoModel(
model="OpenMOSS-Team/MOSS-Transcribe-Diarize",
backend="vllm",
vllm_base_url="http://127.0.0.1:8898/v1",
vllm_model="moss-transcribe-diarize",
disable_update=True,
)
result = model.generate("audio.wav")[0]
```

vLLM 适配器发送官方文档中的 OpenAI-compatible multipart 请求,并统一解析返回文本。
认证可通过 `vllm_api_key` 提供;请放在 secret store,不要写入源码。

## 选择服务后端

| 路径 | 环境 | 响应契约 | 适用场景 |
Expand Down Expand Up @@ -98,6 +152,12 @@ FunASR 生态验证使用 vLLM `0.23.1rc1.dev949+g68b4a1d58`、Torch
这证明固定版本对这些输入满足 API 和说话人返回契约,不是 diarization 精度、重叠语音、
吞吐或生产容量结论。

FunASR 适配器还分别使用 `backend="hf"` 和 `backend="vllm"`,按模型 revision
`e8681d68e7042738ffca8ac8212bc8fcb1131ab8` 做了真实测试。15.1685 秒的双说话人
样例(`43dccc068506439cb633b382b6b98185baa837363d08cc5f7152ca89b0fdc3c8`)通过
统一 `AutoModel` 结果契约返回两个单调时间分段和 `S01`、`S02`。测试完成后已停止
临时 vLLM worker;这只是契约冒烟测试,不是精度基准。

## SGLang Omni

按固定上游 SGLang Omni 安装指南准备 CUDA 13 环境,然后下载不可变模型快照并从本地目录启动:
Expand Down
6 changes: 6 additions & 0 deletions funasr/auto/auto_model.py
Original file line number Diff line number Diff line change
Expand Up @@ -542,6 +542,12 @@ def build_model(**kwargs):
if kwargs["model"] in {"silero-vad", "silero_vad"}:
kwargs.setdefault("model_conf", {})
kwargs["model"] = "SileroVad"
if kwargs["model"] in {
"MOSS-Transcribe-Diarize",
"OpenMOSS-Team/MOSS-Transcribe-Diarize",
}:
kwargs.setdefault("model_conf", {})
kwargs.setdefault("model_path", kwargs["model"])
if "model_conf" not in kwargs:
logging.info("download models from model hub: {}".format(kwargs.get("hub", "ms")))
kwargs = download_model(**kwargs)
Expand Down
5 changes: 5 additions & 0 deletions funasr/models/moss_transcribe_diarize/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"""OpenMOSS MOSS-Transcribe-Diarize integration."""

from .model import MossTranscribeDiarize

__all__ = ["MossTranscribeDiarize"]
Loading
Loading