RVCBench (accepted at NeurIPS 2026) combines conventional voice-cloning utility evaluation with tests across speakers, languages and real-world audio conditions.
- Utility: speaker similarity, content accuracy, speech quality, intelligibility and emotion consistency.
- Robustness: noisy or overlapping references, compression, anti-cloning protection, long-form speech, unusual text, speaker demographics and multilingual/cross-lingual coverage.
- Automated evaluation: a pip package with seven metrics, prompt export and evaluation reports. Use our datasets or score your own audio.
Linux, Python 3.10–3.13 and FFmpeg are required. CPU and GPU scoring are supported.
python -m pip install "rvcbench[eval]"
rvcbench setup-scorerssetup-scorers downloads and verifies metric models once. For FFmpeg setup, CPU/GPU installation
and download requirements, see the installation guide.
| Your goal | Start here |
|---|---|
| Score generated audio with your own reference data | Score your own audio |
| Test a model on RVCBench datasets | Evaluate your model |
| Reproduce the published experiments | Reproduce the paper |
from rvcbench import metrics
with metrics.Evaluator(["sim", "wer", "speechmos"], device="cpu") as evaluator:
scores = evaluator.score(
"generated.wav",
reference="speaker_reference.wav",
text="Hello there.",
language="en",
)
print(scores)Replace the audio paths and expected text with your own. Available metrics are sim, sva, wer,
speechmos, mcd, stoi and emotion. MCD/STOI also require a same-text target recording.
See the metrics guide for inputs and batch scoring.
Start with the 52-utterance onboarding suite:
rvcbench prompts --suite onboarding-v1 --output prompts/
# Generate each prompt with your model; save outputs as <id>.wav in outputs/my-model/.
rvcbench score --suite onboarding-v1 --generated outputs/my-model --output results/my-model --device cpuRVCBench prepares the data and scores your outputs; you supply model inference. Prompt exports include JSONL, ZipVoice TSV and Seed-TTS-style batch lists. The step-by-step guide covers file formats, generation and reading the report.
For broader evaluation, use core-v1 (480 utterances) or full-v1 (12,724 utterances). See suite coverage and task selection and resume and model comparison. Packaged suites are currently previews and use a separate protocol from the paper. The paper's deepfake-detectability evaluation and audio-LLM expression judge are not included in these suites.
The paper defines 18 evaluations, 14,370 utterances and 204 speakers. Browse interactive results and audio examples, paper result tables, or the paper for the full analysis.
The frozen v1 branch (tag v1.0)
contains the paper-release code. main contains the installable package for evaluating new models.
See versions and reproduction.
- Built-in models — 32 voice-cloning adapters, protection and denoising workflows.
- Datasets — source corpora, Hub folders and preprocessing.
- Python API · CLI reference · Run records.
- Contributing — development setup, checks and adding a model.
@inproceedings{jin2026rvcbench,
title = {RVCBench: Benchmarking the Robustness of Voice Cloning Across Modern Audio Generation Models},
author = {Jin, Ruinan and Liao, Xinting and Yu, Hanlin and Pandya, Deval and Li, Xiaoxiao},
booktitle = {Advances in Neural Information Processing Systems},
url = {https://arxiv.org/abs/2602.00443},
year = {2026}
}CC0-1.0. Models, upstream code and source corpora retain their own licenses. Questions and contributions are welcome through issues or ruinanjin@alumni.ubc.ca.