Fish Audio Raw API Skill

Use this skill to generate correct, runnable Fish Audio API calls without any SDK. The canonical machine-readable sources are:

• REST: https://docs.fish.audio/api-reference/openapi.json
• WebSocket: https://docs.fish.audio/api-reference/asyncapi.yml

This file condenses those into rules an agent can apply directly.

Global facts

• Base URL: https://api.fish.audio
• WebSocket base: wss://api.fish.audio
• Auth (all endpoints): Authorization: Bearer <FISHAPIKEY>
• Optional distributed tracing for inference APIs: see https://docs.fish.audio/api-reference/observability.
• Get API keys: https://fish.audio/app/api-keys
• Never hardcode keys. Read from an env var like FISHAPIKEY.
• Errors are JSON {status, message} for 401 / 402 / 404, and an array of {loc, type, msg, ctx, in} for 422 (validation).

Endpoint map

Text-to-Speech: POST /v1/tts

Required headers:

• Authorization: Bearer <FISHAPIKEY>
• Content-Type: application/json or application/msgpack

Optional headers:

• model: values s1, s2-pro, s2.1-pro, s2.1-pro-free. If omitted or unrecognized, the server falls back to s2.1-pro (paid). Default to s2.1-pro for production; use s2.1-pro-free for free-tier evaluation and prototyping (same model, no TTFA/DPA guarantees).

Response: streaming audio bytes (Transfer-Encoding: chunked) in the format set by format. Write to a file or pipe to a player. There is no JSON wrapper on success.

Request body fields (TTSRequest)

ReferenceAudio = { audio: <raw bytes>, text: <transcript string> }. 10–30 s of clean speech works best.

Voice source rules

1. Library / custom voice model → set referenceid to the model id. Simplest path.
2. Zero-shot from audio → set references (array of {audio, text}) and use MessagePack body. JSON cannot carry raw audio bytes.
3. Multi-speaker dialogue (s2-pro and the S2.1-Pro family) → reference_id: [id0, id1, ...] and embed <|speaker:0|> / <|speaker:1|> markers inside text. For zero-shot multi-speaker, references is an array-of-arrays, one inner array per speaker.

Single-speaker curl

curl --request POST https://api.fish.audio/v1/tts \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: s2.1-pro" \
  --data '{
    "text": "Hello! Welcome to Fish Audio.",
    "reference_id": "<voice-model-id>",
    "format": "mp3",
    "mp3_bitrate": 128,
    "latency": "normal"
  }' \
  --output out.mp3

Multi-speaker curl

curl --request POST https://api.fish.audio/v1/tts \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: s2.1-pro" \
  --data '{
    "text": "<|speaker:0|>Good morning!<|speaker:1|>Good morning! How are you?",
    "reference_id": ["<speaker-0-id>", "<speaker-1-id>"],
    "format": "mp3"
  }' \
  --output dialogue.mp3

Python (no SDK, streaming to file)

import os, httpx

payload = {
    "text": "Hello from Fish Audio.",
    "reference_id": "<voice-model-id>",
    "format": "mp3",
    "latency": "normal",
}

headers = {
    "Authorization": f"Bearer {os.environ['FISH_API_KEY']}",
    "Content-Type": "application/json",
    "model": "s2.1-pro",
}

with httpx.stream("POST", "https://api.fish.audio/v1/tts",
                  headers=headers, json=payload, timeout=None) as r:
    r.raise_for_status()
    with open("out.mp3", "wb") as f:
        for chunk in r.iter_bytes():
            f.write(chunk)

Python with inline references (MessagePack)

import os, httpx, msgpack

with open("sample.wav", "rb") as f:
    ref_audio = f.read()

payload = {
    "text": "Clone this voice and say this line.",
    "references": [{"audio": ref_audio, "text": "Transcript of sample.wav."}],
    "format": "mp3",
}

headers = {
    "Authorization": f"Bearer {os.environ['FISH_API_KEY']}",
    "Content-Type": "application/msgpack",
    "model": "s2.1-pro",
}

body = msgpack.packb(payload, use_bin_type=True)
with httpx.stream("POST", "https://api.fish.audio/v1/tts",
                  headers=headers, content=body, timeout=None) as r:
    r.raise_for_status()
    with open("out.mp3", "wb") as f:
        for chunk in r.iter_bytes():
            f.write(chunk)

Node.js (fetch, streaming)

import { createWriteStream } from "node:fs";
import { Readable } from "node:stream";
import { pipeline } from "node:stream/promises";

const res = await fetch("https://api.fish.audio/v1/tts", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.FISH_API_KEY}`,
    "Content-Type": "application/json",
    model: "s2.1-pro",
  },
  body: JSON.stringify({
    text: "Hello from Fish Audio.",
    reference_id: "<voice-model-id>",
    format: "mp3",
    latency: "normal",
  }),
});

if (!res.ok) throw new Error(`${res.status} ${await res.text()}`);
await pipeline(Readable.fromWeb(res.body), createWriteStream("out.mp3"));

Speech-to-Text: POST /v1/asr

Required headers: Authorization. Content type: multipart/form-data or application/msgpack.

Form fields:

• audio (binary, required)
• language (string, optional; omit to auto-detect)
• ignore_timestamps (bool, default true; set false to get per-segment timestamps, which adds latency on clips < 30 s)

Response (200):

{
  "text": "full transcript",
  "duration": 12.34,
  "segments": [{"text": "...", "start": 0.0, "end": 1.23}]
}

curl

curl --request POST https://api.fish.audio/v1/asr \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --form "[email protected]" \
  --form "language=en" \
  --form "ignore_timestamps=false"

Python

import os, httpx

with open("input.wav", "rb") as f:
    r = httpx.post(
        "https://api.fish.audio/v1/asr",
        headers={"Authorization": f"Bearer {os.environ['FISH_API_KEY']}"},
        files={"audio": f},
        data={"language": "en", "ignore_timestamps": "false"},
        timeout=120,
    )
r.raise_for_status()
print(r.json()["text"])

Voice Design: POST /v1/voice-design

Required headers:

• Authorization: Bearer <FISHAPIKEY>
• Content-Type: application/json
• model: voice-design-1 (required; currently the only public Voice Design model)

Response: JSON { candidates: VoiceDesignCandidate[] }. Each candidate includes audio_base64; decode it to write the generated audio bytes to a file. The current candidate audio payload is WAV bytes encoded as base64.

Request body fields (VoiceDesignRequest)

Do not send MessagePack, multipart form data, inline reference audio, or service-internal fields such as features, featuresjsonfile, or includeaudiobase64.

curl

curl --request POST https://api.fish.audio/v1/voice-design \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --header "model: voice-design-1" \
  --data '{
    "instruction": "Warm, confident studio narrator with a natural tone",
    "reference_text": "Welcome to Fish Audio.",
    "language": "en",
    "n": 2
  }' | jq -r '.candidates[0].audio_base64' | base64 --decode > voice.wav

Python

import base64
import os
import httpx

r = httpx.post(
    "https://api.fish.audio/v1/voice-design",
    headers={
        "Authorization": f"Bearer {os.environ['FISH_API_KEY']}",
        "Content-Type": "application/json",
        "model": "voice-design-1",
    },
    json={
        "instruction": "Warm, confident studio narrator with a natural tone",
        "reference_text": "Welcome to Fish Audio.",
        "language": "en",
        "n": 2,
    },
    timeout=120,
)
r.raise_for_status()
candidate = r.json()["candidates"][0]
with open("voice.wav", "wb") as f:
    f.write(base64.b64decode(candidate["audio_base64"]))

Billing: one successful generation request is charged once, even when it returns multiple candidates. Authentication, validation, balance, concurrency, and service errors are not billed.

Voice models: /model

List: GET /model

Query params: pagesize (default 10), pagenumber (default 1), title, tag (string or array), self (bool; only your models), authorid, language, titlelanguage, sortby (score | taskcount | created_at, default score).

Returns {total, items: ModelEntity[]}.

Create: POST /model (multipart/form-data)

Required: type=tts, title, train_mode=fast, voices (one or more audio file uploads).

Optional: visibility (public | unlist | private, default public; coverimage is required if public), description, coverimage, texts (transcripts matching each voice; if omitted, ASR is run on the audio), tags (string or array), enhanceaudioquality (bool, default false).

curl --request POST https://api.fish.audio/model \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --form "type=tts" \
  --form "train_mode=fast" \
  --form "title=My Voice" \
  --form "visibility=private" \
  --form "[email protected]" \
  --form "[email protected]" \
  --form "texts=Transcript of sample 1." \
  --form "texts=Transcript of sample 2." \
  --form "tags=en" \
  --form "tags=narration"

Returns 201 with the full ModelEntity including id, state (created | training | trained | failed), visibility, samples, author, counts, timestamps. Use id as reference_id in /v1/tts.

Get / Update / Delete

• GET /model/{id} → ModelEntity
• PATCH /model/{id}: JSON, form-urlencoded, multipart, or msgpack. Nullable fields: title, description, cover_image (binary), visibility, tags.
• DELETE /model/{id} → 200 on success.

curl --request PATCH https://api.fish.audio/model/<id> \
  --header "Authorization: Bearer $FISH_API_KEY" \
  --header "Content-Type: application/json" \
  --data '{"title": "Renamed", "visibility": "unlist"}'

Wallet

• GET /wallet/self/package → {userid, type, total, balance, createdat, updatedat, finishedat}
• GET /wallet/self/api-credit → {id, userid, credit, createdat, updatedat, hasphonesha256, hasfreecredit}. Pass ?checkfreecredit=true to also populate hasfreecredit (default false; the field is null when not checked).

Replace self with a specific user_id if you have permission; otherwise always use self.

WebSocket TTS: wss://api.fish.audio/v1/tts/live

For low-latency / streaming TTS (e.g. LLM token stream → speech). All frames are MessagePack-encoded binary messages.

Connection headers

• Authorization: Bearer <FISHAPIKEY>
• model: optional; same values and fallback behavior as POST /v1/tts (falls back to s2.1-pro when omitted or unrecognized)

Event sequence

Client → server:

1. StartEvent (once, first message): {event: "start", request: <TTSRequest>}. The request object is the same schema as POST /v1/tts above. Usually request.text = "" and the real text streams in TextEvents.
2. TextEvent (one per text chunk): {event: "text", text: "..."}. Send as many as needed.
3. FlushEvent (optional): {event: "flush"}. Forces the server to synthesize buffered text immediately (use for turn-taking / low-latency flushes).
4. CloseEvent (final): {event: "stop"}. Note the literal is stop, not close.

Server → client:

• AudioEvent: {event: "audio", audio: <bytes>}. Many of these; concatenate in order to reconstruct the audio stream in the format set by request.format.
• FinishEvent: {event: "finish", reason: "stop" | "error"}. Exactly one, then the server closes the socket. Ignore unknown events for forward compatibility.

Python example (websockets>=14 + msgpack)

additionalheaders is the parameter name in websockets v14+. On older releases use extraheaders= or import from websockets.legacy.client.

import asyncio, os, msgpack, websockets
from websockets.exceptions import ConnectionClosed

API_KEY = os.environ["FISH_API_KEY"]
URL = "wss://api.fish.audio/v1/tts/live"

start = {
    "event": "start",
    "request": {
        "text": "",
        "reference_id": "<voice-model-id>",
        "format": "mp3",
        "latency": "normal",
    },
}

async def run(text_stream):
    headers = {"Authorization": f"Bearer {API_KEY}", "model": "s2.1-pro"}
    async with websockets.connect(URL, additional_headers=headers,
                                  max_size=None) as ws:
        await ws.send(msgpack.packb(start, use_bin_type=True))

        async def sender():
            try:
                async for chunk in text_stream:
                    await ws.send(msgpack.packb(
                        {"event": "text", "text": chunk}, use_bin_type=True))
                await ws.send(msgpack.packb({"event": "stop"}, use_bin_type=True))
            except ConnectionClosed:
                pass  # server sent finish before the text stream drained

        send_task = asyncio.create_task(sender())
        try:
            with open("out.mp3", "wb") as f:
                async for raw in ws:
                    msg = msgpack.unpackb(raw, raw=False)
                    if msg["event"] == "audio":
                        f.write(msg["audio"])
                    elif msg["event"] == "finish":
                        if msg["reason"] == "error":
                            raise RuntimeError("TTS failed")
                        break
        finally:
            send_task.cancel()
            try:
                await send_task
            except (asyncio.CancelledError, ConnectionClosed):
                pass

async def words():
    for w in ["Hello", " from", " Fish", " Audio."]:
        yield w

asyncio.run(run(words()))

Node.js example (ws + @msgpack/msgpack)

import WebSocket from "ws";
import { encode, decode } from "@msgpack/msgpack";
import { createWriteStream } from "node:fs";

const ws = new WebSocket("wss://api.fish.audio/v1/tts/live", {
  headers: {
    Authorization: `Bearer ${process.env.FISH_API_KEY}`,
    model: "s2.1-pro",
  },
});

const out = createWriteStream("out.mp3");

ws.on("open", () => {
  ws.send(encode({
    event: "start",
    request: { text: "", reference_id: "<voice-model-id>", format: "mp3" },
  }));
  ws.send(encode({ event: "text", text: "Hello from Fish Audio." }));
  ws.send(encode({ event: "stop" }));
});

ws.on("message", (buf) => {
  const msg = decode(buf);
  if (msg.event === "audio") out.write(Buffer.from(msg.audio));
  else if (msg.event === "finish") {
    out.end();
    ws.close();
    if (msg.reason === "error") throw new Error("TTS failed");
  }
});

Emotion / expression control

The S1 model uses (parenthesis) tags inside text, e.g. (happy) What a day!. S2-Pro uses free-form [bracket] natural-language tags, e.g. [slightly sarcastic, rising tone]. Either works through text; there is no separate parameter. Full list: https://docs.fish.audio/api-reference/emotion-reference.md.

Encoding and content-type rules

• Use application/json for normal TTS requests. It's the simplest and works for reference_id flows.
• Use application/msgpack when you need to send raw audio bytes inline (inline references, or the WebSocket protocol).
• Use multipart/form-data for /v1/asr and POST /model because they upload files.
• All WebSocket frames are MessagePack binary, regardless of inner payload.

Error handling checklist

• 401 → missing / bad Authorization header.
• 402 → out of credit. Check /wallet/self/api-credit.
• 404 → bad model/{id} (voice model doesn't exist or isn't visible to you).
• 422 → validation. The response is an array; each item's loc points at the offending field. Most common causes:

- referenceid is an array but model is s1 (multi-speaker requires s2-pro or an S2.1-Pro model). - references sent with Content-Type: application/json (must be msgpack). - Numeric param out of range (temperature, topp, chunklength, minchunklength, prosody.speed, earlystopthreshold). - mp3bitrate / opus_bitrate set without matching format.

• WebSocket: a finish event with reason: "error" means the server failed mid-stream. Surface the message and reconnect rather than retrying on the same socket.

Decision shortcuts

• User just wants audio from text → POST /v1/tts with JSON + reference_id.
• User has a raw voice clip and wants instant cloning → POST /v1/tts with MessagePack + references.
• User wants dialogue between multiple speakers → POST /v1/tts on s2.1-pro with reference_id array and <|speaker:N|> tags.
• User is streaming tokens from an LLM and wants speech to play as it arrives → WebSocket /v1/tts/live.
• User wants a persistent custom voice they can reuse → POST /model first, then reuse the returned id as referenceid.
• User wants a transcript → POST /v1/asr.