Miso TTS 8B — Highly Emotive Text-to-Speech Model (Miso Labs)

TL;DR

Miso TTS 8B is an open-weights ~8.2B-parameter text-to-speech model from Miso Labs, released as code + weights on GitHub and Hugging Face. The architecture is explicitly inspired by Sesame CSM: an RVQ Transformer with a Llama-8B backbone that consumes interleaved text + audio-frame embeddings, plus a small Llama-300M decoder that autoregressively predicts higher-order codebooks within each frame using the Mimi audio tokenizer (32 codebooks, 2,051 audio vocab). It supports voice cloning via audio context conditioning, ships with SilentCipher watermarking on by default, and currently targets English only. The companion launch video — “Launching Miso One: The Most Emotive AI Voice Model” — is filed separately at Launching Miso One — The Most Emotive AI Voice Model.

Key claims

Model is an ~8.2B-parameter RVQ Transformer; backbone is llama-8B and audio decoder is llama-300M [Model Summary].
Text vocabulary is 128,256; audio vocabulary is 2,051 across 32 Mimi codebooks; max sequence length is 2,048 [Model Summary].
Architecture is “inspired by the Sesame CSM” — two transformers, one for interleaved text/audio-frame embeddings, one for higher-order codebook prediction within a frame [Model Introduction, Architecture].
The backbone accepts interleaved text and audio tokens to condition on conversation history [Architecture].
Voice cloning is supported by passing prior Segment(speaker, text, audio) context to generator.generate(...) [Usage / Prompted generation].
Generated audio is watermarked by default via SilentCipher (sony/silentcipher); deployers are told to use a private key [Safety, Weights].
Recommended VRAM is 24 GB at bf16/fp16 and 40 GB+ at fp32; first run downloads ~30–40 GB into the HF cache [System Requirements].
English only at release [Model Summary].

Method

Miso TTS 8B follows the two-transformer RVQ pattern popularised by Sesame CSM: a large backbone (Llama-8B style) consumes a stream of interleaved text-token embeddings and audio-frame embeddings (where each “audio frame” carries the 32 Mimi codebook entries for that timestep), and produces a per-frame hidden state. A smaller Llama-300M decoder then runs autoregressively within that frame to predict the higher-order Mimi codebooks given the lower-order ones — this avoids paying full backbone cost per codebook. The Mimi codec then synthesizes waveform from the predicted codes, and SilentCipher applies a watermark before audio is returned. Inference is exposed via a Generator with generate(text, speaker, context, max_audio_length_ms), where context is an optional list of prior Segment tuples used for prompted/voice-cloned generation.

Results

No benchmark numbers are reported in the repo at filing time — the page is a code + model card, not a paper. Marketed positioning is “state-of-the-art text-to-speech model” and “highly emotive”; the companion blog post is referenced but not transcluded here. The launch YouTube (Launching Miso One — The Most Emotive AI Voice Model) is framed as “the most emotive AI voice model” but is not transcript-available at filing time.

Why it’s interesting

The interesting move is following Sesame CSM’s two-transformer RVQ recipe at the 8B scale and shipping the weights openly — most prior emotive-TTS open releases sit at 1–4B. This complements TTS-1 Technical Report (Inworld TTS-1-Max, also Llama-based autoregressive at 8.8B with X-codec2) by validating the recipe at similar scale with the Mimi codec instead, and contrasts with the omni-model trend (Qwen3-TTS Family is Now Open Sourced: Voice Design, Clone, and Generation, Voxtral TTS — Mistral AI frontier open-weight text-to-speech model (announcement)) where TTS gets folded into a larger multimodal stack — Miso explicitly stays text→speech-only and English-only. Worth tracking whether the “highly emotive” positioning is backed by an actual evaluation in the linked blog post or remains a marketing claim.