Pocket-sized speech, in Go

Kyutai’s PocketTTS is a text-to-speech model small enough to run on a CPU. Its reference implementation is Python and PyTorch. go-pocket-tts runs the same model as a single Go binary: it reads the safetensors weights directly and needs neither Python nor ONNX Runtime.

It got one sentence in Audio on Linux, in Go. It deserves more than that, because upstream has moved on since then: a September model, German, a 24-layer variant and a new kind of sampler. This weekend I brought the port up to upstream v3.3.0. That came to 25 pull requests and 116 commits in two days, and German is part of it.

Listen first

Both clips below were made today with the commands shown under them. Each is a single take, cut only by the model’s own end-of-speech decision. The only processing afterwards was loudness normalisation and MP3 encoding.

./pockettts synth --language english_2026-09 \
  --text "Hello. I am a text to speech model, running in pure Go on an ordinary laptop CPU. No Python, and no ONNX Runtime." \
  --out en.wav

./pockettts synth --language german_24l \
  --text "Hallo Welt. Ich bin Pocket TTS von Kyutai. Ich bin schnell genug, um auch auf kleinen CPUs zu laufen. Ich hoffe, ich gefalle dir." \
  --out de.wav

As a check that does not depend on my ears, I ran both files through Whisper. It returned both texts word for word. The only slips were the proper nouns: “Onyx runtime” and “Qtai”.

Some honest numbers, from a laptop with an AMD Ryzen 5 4600H. The English clip comes from a 6-layer model, and generating it took 10.1 s for 8.2 s of audio, about 0.8× real time, with a peak of 1.6 GB of memory. The German clip comes from the 24-layer model, which is about three times the weights: 10.8 s for 6.1 s of audio, about 0.6× real time, with a peak of 3.4 GB. So neither is faster than real time on this machine. That is consistent with the model being built to fit a CPU.

Or try it in your browser

No need to build anything to hear it: the web demo runs the same Go code, compiled to WebAssembly. Pick a model and a voice, type something and press Synthesize. The page bundles no model of its own. It fetches model, tokenizer and voices straight from Hugging Face, at the same pinned revisions the command line uses.

Five of the six models work in the browser, german_24l included. The sixth, english_2026-09_24l, is left out for now. Its checkpoint is 1.3 GB. It loads, but then the Mimi decoder runs out of 32-bit WebAssembly address space with 4.27 GB in use.

What is in the box

The project is about 50,000 lines of Go, and 131 of its 250 source files are tests. It ships two binaries. pockettts is the one you use:

• synth turns text into a 24 kHz WAV file. It reads from a flag or from stdin and can split long text into sentence chunks.
• serve is an HTTP server with POST /tts, GET /voices and GET /health. It has a worker pool and shuts down gracefully.
• model download and model verify fetch checkpoints from Hugging Face, pinned to a revision and checked against SHA256 sums.
• doctor checks whether everything a language needs is present and consistent before you find out the hard way.
• export-voice turns a WAV prompt into a voice file. Today this still goes through the optional ONNX backend.

The default backend, native-safetensors, is the whole inference stack written in Go. It has the transformer backbone that generates latent audio frames, the flow head that samples them, and the Mimi decoder that turns them into a waveform, with AVX2/FMA kernels for the hot loops. An ONNX Runtime backend also exists as a fallback, and the WebAssembly build is what powers the browser demo above.

The resync

The port was last synced with upstream pocket-tts 2.1.0 in May. Since then, upstream had added 98 commits and reached 3.3.0. Most of that is not code you can copy. It is behaviour you have to match: a different GELU, a new end-of-speech rule, a new tokenizer format, new text handling. Every one of those changes can make the output sound slightly wrong without causing a single error.

So the work ran in numbered phases, one pull request per slice. After every slice the full test suite had to pass, and the reference fixtures were regenerated after any change to the numerics.

Model configs as data

The old port was wired to a single checkpoint, tts_b6369a24, with its quirks hard-coded in Go. Upstream now describes every model in a YAML config: layer count, sampler type, text-preparation rules, tokenizer, default temperature, how many frames to generate after end-of-speech. The first phase ported that config layer and embedded upstream’s configs in the binary. A model is now selected with a flag:

--language	Layers	Sampler	Default voice
english_2026-01	6	LSD	alba
english_2026-09	6	LSD	alba
english_2026-09_24l	24	LSD	alba
english_drifting_26-09	6	drifting	alba
german	6	LSD	juergen
german_24l	24	LSD	juergen

Each language comes with 27 predefined voices. The 6-layer models hold about 210–235 MB of weights, and german_24l about 670 MB. Every file is pinned to a Hugging Face revision and a checksum, so “it worked on my machine” means the same bytes on yours.

Numerics

Some of these changes were small to write and large in effect. Upstream switched its transformer layers from the exact, erf-based GELU to the tanh approximation, so the Go port did too, at all three places it is used. The difference is small per value, but the model runs autoregressively, and small differences compound over hundreds of steps. Newer models expect a BOS token in front of the voice prompt. The stopping rule for generation was changed to match upstream’s exactly. Every chunk now fades in after decoding, using torch-style linspace ramps, because “the same fade” has to mean the same samples.

The flow head was generalised as well. The September models come in two kinds: LSD, which samples with two time conditions, and the new drifting head, which has none and produces a frame in a single step. The Go code now reads from the checkpoint how many time embeddings it has and dispatches on that. A small observation from the first smoke run: the same 11-word sentence took 40 frames with the drifting model and 50 with the January LSD model.

Text and tokens

Before a model sees any text, the text is cleaned, capitalised, split into chunks and tokenised, and each of those steps can go wrong in its own way. I ported upstream’s token-based sentence splitter, and a new loader reads Hugging Face tokenizer.json files (Unigram with byte fallback), which is what the new models ship with.

Both are tested against goldens generated by the Python originals, including the cases where Python and Go disagree about Unicode without saying so. Python’s str.isdigit() accepts ² and ①, but Go’s unicode.IsDigit does not. Normalising … into ... turns one character into three sentence ends. German brings 3,50 €, 24.12. and „these quotes“. The splitter now passes all 29 golden cases. Four of them failed with the old encoder.

Several languages, one server

Upstream’s server runs one language per process. Mine can now serve several from one process. A request picks its model with a language field, and the models live in a small LRU registry:

./pockettts serve --language german --server-languages english_2026-01
curl -s -X POST http://localhost:8080/tts \
  -d '{"text":"Hello there.","language":"english_2026-01"}' -o hello.wav

Each model loads on its first request. At most --server-max-languages stay loaded (two by default). The least recently used model leaves the registry right away but is only freed once its running requests have finished, so memory can briefly go over the cap, and it is honest about that. Review found two problems in the first version, both of which only show up under real traffic:

• A model that was still loading could be evicted, so the next request started a second load of the same weights.
• A request loaded its model before it had a worker slot, so requests waiting in the queue could still fill the memory.

Both are fixed: loads now run on their own goroutine and can be cancelled, eviction skips models that are still loading, and a request only touches the weights once it is allowed to run.

Two other server changes. serve --default-voice accepts a voice ID, a local file, or a pinned hf://…@revision reference. And the server now refuses to start if its default voice cannot be loaded, instead of starting and then failing every request that does not name a voice.

Trying it

The ungated Hugging Face repository is enough: no account and no token needed.

git clone https://github.com/CWBudde/go-pocket-tts
cd go-pocket-tts
go build -o pockettts ./cmd/pockettts
./pockettts model download --language german \
  --hf-repo kyutai/pocket-tts-without-voice-cloning
./pockettts doctor --language german
./pockettts synth --language german --text "Guten Tag." --out hallo.wav

The code is at github.com/CWBudde/go-pocket-tts, and the credit for the model goes to Kyutai: all the port does is take Python out of the picture.