qiudl
712063071c
skills/ → skills-dev(9), skills-req(10), skills-ops(4), skills-integration(8), skills-biz(4), skills-workflow(7) generate-marketplace.py 改为自动扫描所有 skills-* 目录。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
509 lines
12 KiB
Markdown
509 lines
12 KiB
Markdown
---
|
|
name: doubao-voice
|
|
description: 豆包语音API调用。支持语音合成(TTS)和唱歌。当用户提到语音合成、文字转语音、唱歌、豆包语音相关任务时自动激活。
|
|
---
|
|
|
|
# 豆包语音API技能
|
|
|
|
调用火山引擎豆包语音API，实现文字转语音(TTS)和唱歌功能。
|
|
|
|
## 核心功能 ⭐
|
|
|
|
### 1. 文字转语音 (TTS)
|
|
|
|
```bash
|
|
# 1. 配置环境变量
|
|
export DOUBAO_APP_ID="your_app_id"
|
|
export DOUBAO_ACCESS_TOKEN="your_access_token"
|
|
|
|
# 2. 文字转语音
|
|
python scripts/voice_converter.py tts "你好世界"
|
|
```
|
|
|
|
### 2. 唱歌 🎵
|
|
|
|
```bash
|
|
# 让豆包唱歌
|
|
python scripts/singing.py sing "请唱一首关于春天的歌"
|
|
|
|
# 交互式唱歌模式
|
|
python scripts/singing.py interactive
|
|
```
|
|
|
|
## 功能概述
|
|
|
|
| 模块 | 功能 | 推荐模型 |
|
|
|------|------|---------|
|
|
| **语音合成 (TTS)** | 文字转语音、多种音色 | 豆包语音合成模型2.0 |
|
|
| **唱歌** | 实时语音交互、唱歌、角色扮演 | 豆包端到端实时语音大模型 |
|
|
|
|
---
|
|
|
|
## 环境配置
|
|
|
|
### 1. 获取火山引擎豆包语音凭证
|
|
|
|
1. 访问 [火山引擎控制台](https://console.volcengine.com/)
|
|
2. 开通「豆包语音」服务
|
|
3. 创建应用获取 `App ID` 和 `Access Token`
|
|
4. 开通所需服务：
|
|
- 「语音合成」权限：大模型语音合成
|
|
|
|
### 2. 环境变量配置
|
|
|
|
```bash
|
|
# ~/.zshrc 或 ~/.bashrc
|
|
export DOUBAO_APP_ID="your_app_id"
|
|
export DOUBAO_ACCESS_TOKEN="your_access_token"
|
|
export DOUBAO_CLUSTER="volcano_tts" # TTS服务集群
|
|
```
|
|
|
|
### 3. Python 依赖
|
|
|
|
```bash
|
|
# 推荐使用 uv
|
|
uv pip install requests websocket-client
|
|
|
|
# 或使用 pip
|
|
pip install requests websocket-client
|
|
```
|
|
|
|
---
|
|
|
|
## API 基础
|
|
|
|
### Base URL
|
|
|
|
```
|
|
TTS API: https://openspeech.bytedance.com/api/v1/tts
|
|
```
|
|
|
|
### 认证方式
|
|
|
|
使用 Access Token 进行认证，在请求头中添加：
|
|
```
|
|
Authorization: Bearer {access_token}
|
|
```
|
|
|
|
---
|
|
|
|
## 一、语音合成 (TTS)
|
|
|
|
### 1.1 基础语音合成
|
|
|
|
将文本转换为语音文件。
|
|
|
|
**自然语言示例**:
|
|
- "把这段文字转成语音"
|
|
- "用豆包合成语音"
|
|
- "生成语音：你好，欢迎使用豆包语音"
|
|
|
|
**Python 实现**:
|
|
|
|
```python
|
|
import os
|
|
import requests
|
|
import json
|
|
import base64
|
|
|
|
def text_to_speech(text: str, voice_type: str = "BV700_V2_streaming", output_file: str = "output.mp3"):
|
|
"""
|
|
文字转语音
|
|
|
|
Args:
|
|
text: 要合成的文本
|
|
voice_type: 音色类型 (默认: BV700_V2_streaming)
|
|
output_file: 输出音频文件路径
|
|
|
|
Returns:
|
|
音频文件路径
|
|
"""
|
|
app_id = os.environ.get("DOUBAO_APP_ID")
|
|
access_token = os.environ.get("DOUBAO_ACCESS_TOKEN")
|
|
cluster = os.environ.get("DOUBAO_CLUSTER", "volcano_tts")
|
|
|
|
url = "https://openspeech.bytedance.com/api/v1/tts"
|
|
|
|
headers = {
|
|
"Authorization": f"Bearer {access_token}",
|
|
"Content-Type": "application/json"
|
|
}
|
|
|
|
payload = {
|
|
"app": {
|
|
"appid": app_id,
|
|
"token": access_token,
|
|
"cluster": cluster
|
|
},
|
|
"user": {
|
|
"uid": "user123"
|
|
},
|
|
"audio": {
|
|
"voice_type": voice_type,
|
|
"encoding": "mp3",
|
|
"speed_ratio": 1.0,
|
|
"volume_ratio": 1.0,
|
|
"pitch_ratio": 1.0
|
|
},
|
|
"request": {
|
|
"reqid": "req_" + os.urandom(8).hex(),
|
|
"text": text,
|
|
"text_type": "plain",
|
|
"operation": "query"
|
|
}
|
|
}
|
|
|
|
response = requests.post(url, headers=headers, json=payload)
|
|
data = response.json()
|
|
|
|
if data.get("code") == 3000:
|
|
# 解码音频数据
|
|
audio_data = base64.b64decode(data["data"])
|
|
with open(output_file, "wb") as f:
|
|
f.write(audio_data)
|
|
return output_file
|
|
else:
|
|
raise Exception(f"TTS 失败: {data}")
|
|
|
|
# 使用示例
|
|
audio_file = text_to_speech("你好，我是豆包语音助手")
|
|
print(f"语音已生成: {audio_file}")
|
|
```
|
|
|
|
### 1.2 流式语音合成
|
|
|
|
适用于长文本，边生成边播放。
|
|
|
|
```python
|
|
import websocket
|
|
import json
|
|
import os
|
|
|
|
def stream_tts(text: str, voice_type: str = "BV700_V2_streaming"):
|
|
"""
|
|
流式语音合成
|
|
|
|
Args:
|
|
text: 要合成的文本
|
|
voice_type: 音色类型
|
|
"""
|
|
app_id = os.environ.get("DOUBAO_APP_ID")
|
|
access_token = os.environ.get("DOUBAO_ACCESS_TOKEN")
|
|
|
|
ws_url = f"wss://openspeech.bytedance.com/api/v1/tts/ws?appid={app_id}&token={access_token}"
|
|
|
|
def on_message(ws, message):
|
|
data = json.loads(message)
|
|
if "audio" in data:
|
|
# 处理音频数据
|
|
audio_chunk = base64.b64decode(data["audio"])
|
|
# 播放或保存音频片段
|
|
print(f"收到音频片段: {len(audio_chunk)} 字节")
|
|
|
|
def on_open(ws):
|
|
payload = {
|
|
"app": {
|
|
"appid": app_id,
|
|
"token": access_token,
|
|
"cluster": "volcano_tts"
|
|
},
|
|
"user": {
|
|
"uid": "user123"
|
|
},
|
|
"audio": {
|
|
"voice_type": voice_type,
|
|
"encoding": "mp3"
|
|
},
|
|
"request": {
|
|
"reqid": "stream_" + os.urandom(8).hex(),
|
|
"text": text,
|
|
"text_type": "plain",
|
|
"operation": "submit"
|
|
}
|
|
}
|
|
ws.send(json.dumps(payload))
|
|
|
|
ws = websocket.WebSocketApp(
|
|
ws_url,
|
|
on_message=on_message,
|
|
on_open=on_open
|
|
)
|
|
ws.run_forever()
|
|
|
|
# 使用示例
|
|
stream_tts("这是一段很长的文本，使用流式合成可以边生成边播放...")
|
|
```
|
|
|
|
### 1.3 音色选择
|
|
|
|
豆包语音提供多种音色：
|
|
|
|
| 音色代码 | 描述 | 场景 |
|
|
|---------|------|------|
|
|
| BV700_V2_streaming | 通用女声 | 通用场景 |
|
|
| BV701_V2_streaming | 通用男声 | 通用场景 |
|
|
| BV406_streaming | 温柔女声 | 客服、助手 |
|
|
| BV158_streaming | 活泼女声 | 教育、娱乐 |
|
|
| BV115_streaming | 磁性男声 | 新闻、播音 |
|
|
|
|
**查询可用音色**:
|
|
|
|
```bash
|
|
TOKEN="${DOUBAO_ACCESS_TOKEN}"
|
|
APP_ID="${DOUBAO_APP_ID}"
|
|
|
|
curl -s "https://openspeech.bytedance.com/api/v1/tts/voices?appid=$APP_ID" \
|
|
-H "Authorization: Bearer $TOKEN"
|
|
```
|
|
|
|
---
|
|
|
|
## 完整工具类
|
|
|
|
```python
|
|
import os
|
|
import requests
|
|
import base64
|
|
import json
|
|
from typing import Optional
|
|
|
|
class DoubaoVoice:
|
|
"""豆包语音API工具类"""
|
|
|
|
BASE_URL = "https://openspeech.bytedance.com/api/v1"
|
|
|
|
def __init__(self, app_id: str = None, access_token: str = None):
|
|
self.app_id = app_id or os.environ.get("DOUBAO_APP_ID")
|
|
self.access_token = access_token or os.environ.get("DOUBAO_ACCESS_TOKEN")
|
|
self.cluster_tts = os.environ.get("DOUBAO_CLUSTER", "volcano_tts")
|
|
|
|
@property
|
|
def headers(self):
|
|
return {
|
|
"Authorization": f"Bearer {self.access_token}",
|
|
"Content-Type": "application/json"
|
|
}
|
|
|
|
def text_to_speech(
|
|
self,
|
|
text: str,
|
|
voice_type: str = "BV700_V2_streaming",
|
|
output_file: str = "output.mp3"
|
|
) -> str:
|
|
"""文字转语音"""
|
|
url = f"{self.BASE_URL}/tts"
|
|
|
|
payload = {
|
|
"app": {
|
|
"appid": self.app_id,
|
|
"token": self.access_token,
|
|
"cluster": self.cluster_tts
|
|
},
|
|
"user": {"uid": "user123"},
|
|
"audio": {
|
|
"voice_type": voice_type,
|
|
"encoding": "mp3",
|
|
"speed_ratio": 1.0,
|
|
"volume_ratio": 1.0,
|
|
"pitch_ratio": 1.0
|
|
},
|
|
"request": {
|
|
"reqid": "req_" + os.urandom(8).hex(),
|
|
"text": text,
|
|
"text_type": "plain",
|
|
"operation": "query"
|
|
}
|
|
}
|
|
|
|
response = requests.post(url, headers=self.headers, json=payload)
|
|
data = response.json()
|
|
|
|
if data.get("code") == 3000:
|
|
audio_data = base64.b64decode(data["data"])
|
|
with open(output_file, "wb") as f:
|
|
f.write(audio_data)
|
|
return output_file
|
|
else:
|
|
raise Exception(f"TTS 失败: {data}")
|
|
|
|
def list_voices(self) -> list:
|
|
"""获取可用音色列表"""
|
|
url = f"{self.BASE_URL}/tts/voices"
|
|
params = {"appid": self.app_id}
|
|
|
|
response = requests.get(url, headers=self.headers, params=params)
|
|
data = response.json()
|
|
|
|
if data.get("code") == 0:
|
|
return data["voices"]
|
|
else:
|
|
raise Exception(f"获取音色列表失败: {data}")
|
|
|
|
|
|
# ==================== 使用示例 ====================
|
|
if __name__ == "__main__":
|
|
voice = DoubaoVoice()
|
|
|
|
# 示例1: 文字转语音
|
|
audio_file = voice.text_to_speech("你好，我是豆包语音助手")
|
|
print(f"语音已生成: {audio_file}")
|
|
|
|
# 示例2: 查看可用音色
|
|
voices = voice.list_voices()
|
|
for v in voices[:5]:
|
|
print(f"{v['voice_type']}: {v['description']}")
|
|
```
|
|
|
|
---
|
|
|
|
## 二、唱歌 (豆包端到端实时语音大模型)
|
|
|
|
### 2.1 基础唱歌
|
|
|
|
让豆包唱歌，支持任何歌曲主题。
|
|
|
|
**自然语言示例**:
|
|
- "请唱一首关于春天的歌"
|
|
- "唱一个温柔的摇篮曲"
|
|
- "来一首欢快的儿歌"
|
|
|
|
**Python 实现**:
|
|
|
|
```python
|
|
import asyncio
|
|
from scripts.singing import DoubaoSinging
|
|
|
|
async def main():
|
|
singing = DoubaoSinging()
|
|
|
|
# 让豆包唱歌
|
|
audio_file = await singing.sing(
|
|
"请唱一首关于春天的歌",
|
|
output_file="spring_song.mp3",
|
|
language="zh-CN"
|
|
)
|
|
print(f"唱歌完成: {audio_file}")
|
|
|
|
asyncio.run(main())
|
|
```
|
|
|
|
### 2.2 交互式唱歌
|
|
|
|
与豆包进行实时对话，可以要求她唱歌、讲故事等。
|
|
|
|
**Python 实现**:
|
|
|
|
```python
|
|
import asyncio
|
|
from scripts.singing import DoubaoSinging
|
|
|
|
async def main():
|
|
singing = DoubaoSinging()
|
|
|
|
# 启动交互式模式
|
|
await singing.interactive_singing(language="zh-CN")
|
|
|
|
asyncio.run(main())
|
|
```
|
|
|
|
**交互示例**:
|
|
```
|
|
你: 请唱一首情歌
|
|
豆包: [生成音频] 我会为你唱一首温柔的情歌...
|
|
|
|
你: 能加点方言吗？
|
|
豆包: [用方言重新唱歌]
|
|
|
|
你: quit
|
|
再见!
|
|
```
|
|
|
|
---
|
|
|
|
## 自然语言操作示例
|
|
|
|
### TTS 操作
|
|
|
|
| 用户说 | 执行操作 |
|
|
|--------|----------|
|
|
| "把这段话转成语音：你好世界" | 调用 TTS API 生成语音 |
|
|
| "用温柔女声合成语音" | 使用 BV406_streaming 音色 |
|
|
| "生成一段播音腔的新闻语音" | 使用磁性男声音色 |
|
|
|
|
### 唱歌操作
|
|
|
|
| 用户说 | 执行操作 |
|
|
|--------|----------|
|
|
| "请唱一首关于春天的歌" | 调用端到端实时语音大模型生成唱歌音频 |
|
|
| "唱一首摇篮曲" | 生成温柔的摇篮曲 |
|
|
| "唱歌的同时讲个故事" | 交互式对话中唱歌并讲故事 |
|
|
| "开启交互式唱歌模式" | 启动实时语音交互 |
|
|
|
|
---
|
|
|
|
## 计费说明
|
|
|
|
### TTS 计费
|
|
|
|
- **并发版**: 2000元/并发/月（纯并发计费，不收取字符调用费用）
|
|
- **按量付费**: 按合成字符数计费
|
|
|
|
### 免费试用
|
|
|
|
新用户开通服务后可获得一定免费额度，具体额度以控制台显示为准。
|
|
|
|
---
|
|
|
|
## 注意事项
|
|
|
|
1. **音频格式**: TTS 支持 mp3/wav/pcm
|
|
2. **文本长度**: TTS 单次请求最长支持 5000 字符
|
|
3. **并发限制**: 注意 API 调用频率和并发数限制
|
|
4. **Token 安全**: Access Token 存储在环境变量中，不要硬编码
|
|
|
|
---
|
|
|
|
## 错误处理
|
|
|
|
```python
|
|
def safe_tts(text: str):
|
|
"""带错误处理的 TTS"""
|
|
try:
|
|
voice = DoubaoVoice()
|
|
return voice.text_to_speech(text)
|
|
except Exception as e:
|
|
if "401" in str(e):
|
|
print("认证失败，请检查 Access Token")
|
|
elif "429" in str(e):
|
|
print("请求过于频繁，请稍后重试")
|
|
else:
|
|
print(f"合成失败: {e}")
|
|
return None
|
|
```
|
|
|
|
---
|
|
|
|
## 常见场景
|
|
|
|
### 场景 1: 生成多语言语音
|
|
|
|
```python
|
|
voice = DoubaoVoice()
|
|
|
|
# 中文
|
|
voice.text_to_speech("你好", voice_type="BV700_V2_streaming", output_file="zh.mp3")
|
|
|
|
# 英文
|
|
voice.text_to_speech("Hello", voice_type="EN_001", output_file="en.mp3")
|
|
```
|
|
|
|
|
|
---
|
|
|
|
## 参考资源
|
|
|
|
- [火山引擎豆包语音文档](https://www.volcengine.com/docs/6561/1359369)
|
|
- [豆包语音控制台](https://console.volcengine.com/speech/app)
|
|
- [API 接口文档](https://www.volcengine.com/docs/6561/1359370)
|
|
- [计费说明](https://www.volcengine.com/docs/6561/1359370)
|