feat: multi-provider media generation and voice TTS (siliconflow / openai-compatible / local piper) - #600
Open
angri450 wants to merge 1 commit into
Open
feat: multi-provider media generation and voice TTS (siliconflow / openai-compatible / local piper)#600angri450 wants to merge 1 commit into
angri450 wants to merge 1 commit into
Conversation
…enai-compatible / local piper) - media generation: provider + base_url configurable (volcengine / siliconflow / openai-compatible relay); provider-aware API-key verification (/ping vs /models) and model probes with per-dialect payloads; harness_config passes provider through - voice: new presets siliconflow (CosyVoice2 TTS + SenseVoice STT) and openai-compatible (custom endpoint/model/voice); preset id may carry an adapter kind different from its id (both speak the openai wire protocol); STT model configurable separately (extra.stt_model); OpenAI preset stores tts-1 + whisper-1 - voice: new local piper preset (127.0.0.1:8081, zero cost); zh / en / mix voices; wav media type; STT rejects with capability mismatch - tests: media settings provider save/probe dispatch; voice manager preset resolution and piper behavior
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
概要
让模型配置页（Settings → Models）支持多服务商与免费本地 TTS：
volcengine / siliconflow / openai-compatible三选，base_url可编辑（中转站填自己的地址）。生图模型预置 Qwen/Qwen-Image、Kwai-Kolors/Kolors、dall-e-3；视频预置 Wan2.2 T2V/I2V。设计要点
siliconflow/openai-compatible均映射到已有的openaiwire 协议（/v1/audio/speech、/v1/audio/transcriptions），复用现成 adapter 零重复；piper是独立本地 adapter（HTTP GET 到本机服务，WAV 直出，无 key、无 SSRF 担忧——固定 localhost 默认地址可被 base_url 覆盖）。extra.stt_model/extra.model），顺带修正 OpenAI 官方卡片此前只存whisper-1导致 TTS 拿到错误模型的缺陷。size=2K/ siliconflowimage_size=1024x1024/ openai-compatiblesize+n+response_format=url；凭证验证 volcengine 走/ping、其余走GET /models（不产生费用）。视频在 openai-compatible 上如实报不支持（OpenAI 协议无标准），引导换 provider。MediaGenerationConfig.provider透传（本仓只传配置；harness-agent 侧在 PyPI 包维护，上游仓库未公开，PR 合并后需同步发布）。测试
前端构建
dashboard/三处（MediaGeneration / Voice 面板 + locales zh/en），vite build产物不入库（.gitignore），部署时构建。