Whistle is a speech-to-text model from Cactus Compute: 55 million parameters and 16.9 MB on disk, covering seven languages (English, German, French, Spanish, Italian, Dutch and Polish). The weights are published under Apache-2.0, but the code that runs them is not. Upstream, in cactus-compute/needle, Whistle is a thin Python wrapper around a prebuilt, closed-source C++ engine. go-whistle is my pure-Go port: no cgo, no Python, no ONNX. It does transcription with language detection, keyword biasing, word timestamps, live streaming and embeddings, from a CLI, an HTTP/WebSocket server, a Go library, or a WebAssembly page in the browser.
Try the demo in your browser. It is published on GitHub Pages from every push. It downloads the 6.6 MiB WebAssembly build and the 16.1 MiB model from Hugging Face, checks the model’s SHA-256, and then transcribes WAV files or your microphone inside the tab. No audio leaves the browser.
Here is a German clip from Google’s FLEURS test set (CC BY 4.0), one of the 32 fixture clips in the repository:
$ whistle transcribe fleurs-de_de-10229344228128634115.wav
Jeder nimmt an der Gesellschaft Teil und benutzt Transportsysteme, was jeder beklagt sich über die Transportsysteme.It gets “Fast jeder” wrong as “was jeder”. The closed engine makes the same mistake, because matching the engine’s output, mistakes included, is how the port is tested.
Nothing to translate
Most of my ports start from source code in some other language. This one couldn’t. The public cactus-compute/needle repository has no speech code at all: no mel front end, no encoder and no decoder search. What it does have is a JAX reference for the shared Needle 3 blocks (the decoder is built entirely from those), the code for the .cact container and its quantisation, and some prose about the pipeline. The weights come in two forms: all 55,151,433 parameters as FP32 safetensors, and the shipped 16.9 MB whistle.cact with 2- and 4-bit weights, the mel filterbank and the tokenizer.
The prebuilt engine then serves as a black-box oracle. needle_embed returns the encoder output, one 512-wide row per 80 ms frame, and needle_transcribe returns the final JSON. The engine never ships with go-whistle, and nothing loads it at runtime. It sits behind a build tag in tools/oracle and writes fixtures that the tests compare against. Every open question was settled by measuring against those fixtures, and each answer is written up in docs/FINDINGS.md, which now has 43 entries: the question, the alternatives, the metric and the winner. Some were quick. The CQ codebook in the file’s header comes from 400,000 samples of numpy’s RandomState(0) and 200 Lloyd-Max iterations, and rerunning that code reproduces all 28 header floats bit for bit. Others took a real search.
Fitting the encoder through one output
The encoder is where the sources leave the most open. The weights give the tensor shapes, but not the order of operations, the padding, the activations, or how the log-mel spectrogram is computed. The harness treats each of those as an axis: 864 front-end variants plus 19 encoder axes, such as stride placement, kernel layout, GLU form and module order. A hypothesis is one value on every axis. Its score is the mean per-frame cosine between our encoder output and the engine’s embedding over the clips. Scoring one hypothesis on all 32 clips takes about 0.3 s on an M5 Pro. A beam search over the stages (stem geometry first, then activations, front end, conv module and block structure) finds the answer in about three minutes.
The scores need a baseline first. Comparing each engine frame with the mean engine frame of the other clips already gives 0.2349, so anything near that carries no signal. Started from a hypothesis that is wrong on all 26 axes (score 0.017903), the search still converges to the same leader, 0.999819 on the inner clips, in 7.5 minutes. The final norm can be checked directly. Every one of the engine’s 2205 rows has unit RMS after dividing by the FP16 gain stored in the .cact, to within 1.8e-7, so the engine uses that gain and not the FP32 one.
Speech can’t settle everything, so the oracle tool also synthesises probe signals: digital silence of six lengths, DC, 440 Hz tones and bursts at falling levels, noise, a slow fade to −140 dB, and single clicks at known offsets. On digital silence the stem’s output is exactly zero, so any position signal added afterwards would be the only input left. That is how pe_gate, a learned gate of σ(−6.1094) = 0.00222 on a sinusoid, turned out to be unused. The fades and quiet tones exposed the log floor. It isn’t Whisper’s hard clamp eight decades below the peak, but a soft knee:
With log10(x + 10−7.6 · max x), the encoder matches the engine at a mean per-frame cosine of 0.999828 over the 32 clips. The final embedding path, from the packed weights, reaches 0.999827, and every clip is at least 0.999.
How close it gets
The engine computes with 2-bit weights and 8-bit activations, so equal bits were never the goal. The goal is equal transcripts. On a 200-utterance subset of LibriSpeech test-clean, the port has 223 errors in 4167 reference words (WER 5.35 %), and the engine has 221 (5.30 %). 181 of the 200 texts are identical. The rest of the parity table:
The decoder search was its own reconstruction job: Whisper’s beam search at five beams, a silence gate (fitted against 340 synthetic signals until it agrees with the engine’s verdicts), and keyword biasing through an Aho-Corasick automaton. The bonus ended up at 2 nats per token, revoked when a partial match breaks. With one keyword, a LibriSpeech clip changes from “Here are some verses of colonies.” to “Here are some verses of Cowley’s.”:
$ printf "Cowley's\n" > keywords.txt
$ whistle transcribe ls-other-3080-5040-0011.wav
Here are some verses of colonies. Tell me how you like them.
$ whistle transcribe --keywords keywords.txt ls-other-3080-5040-0011.wav
Here are some verses of Cowley's. Tell me how you like them.Each remaining difference is written up as a near tie. The last one investigated: with the keyword Paare on a German clip, the engine writes “Adduktionsplan” and the port writes “Adoptionsplan”. The engine’s text actually scores 0.0688 nats per token higher under our model, but five finished hypotheses fill their slots one step before the leader can end. The cause is the keyword bonus being slightly too strong, not a missing bookkeeping rule.
Running straight from the 2-bit file
Dequantised to float32, the weights take 221 MB. The 16.9 MB file stores each matrix row in groups of 128 as indices into a small codebook, times one FP16 norm per group, rotated by a ±1 Hadamard matrix. Because that matrix is symmetric, a product never needs the dequantised row: rotate each 128-wide group of the activation once with a fast Walsh-Hadamard transform, and the rest is table lookups over the packed indices. The assembly kernels (NEON TBL, AVX2 VPSHUFB) round the lookup tables to 14-bit integers in two byte planes, so one shuffle looks up 16 or 32 rows at once, and the sums stay exact. All 152 quantised tensors run this way. The packed path gives the same 200 LibriSpeech transcripts as the float path, and it is faster:
Memory drops too. After loading, the process holds 37.9 MiB with packed weights and 238.8 MiB with float weights. One 30 s transcription peaks at 107.5 MiB, against 69.9 MiB for the engine’s CLI. The engine is still faster: 6.9 ms against 18.8 ms to the first token and 1499 against 1085 tokens per second, median over the 32 clips. That is hand-tuned C++ with int8 activations against Go with float32 activations. On the same 32 clips, a transcription takes 0.0082 of the audio’s duration. The tensor kernels (GEMM, attention, conv, RoPE) came from my go-pocket-tts port and now live in their own module, algo-tensor, which both projects use.
Streaming, and the browser
One call takes at most 30 s of audio, as in the engine. For longer audio and microphones there is the engine’s streaming scheme. Every chunk, by default one second, re-transcribes the buffer, and only the words two consecutive passes agree on are committed. Once committed audio fills more than 15 s, the buffer is trimmed at a sentence end.
In 797 of 930 stream calls the port makes the engine’s decision, and the final text is the engine’s in 98 of 108 runs. The stream JSON matches the engine’s byte for byte in fields, order and number format. That matters when something downstream already parses the engine’s output.
The web demo is the same library compiled with GOOS=js GOARCH=wasm and running in a Web Worker. WebAssembly gets the pure-Go fallback kernels, one thread and no fused multiply-add, so the float32 sums round differently from every native build. The transcripts survive this. The whole package passes its tests under Node, including the 32-clip engine parity, and on four clips the page’s JSON is byte for byte the CLI’s, minus the timings. It is slow, though. On the M5 Pro it does about 22 tokens per second and takes 0.3 to 0.8 s to the first token, against 18.8 ms natively. A 7 s clip takes about 3 s in Chromium.
Trying it
go install github.com/cwbudde/go-whistle/cmd/whistle@latest
whistle download # fetches the pinned model files, checks SHA-256
whistle doctor # kernels, load time, a self-test
whistle transcribe --word-timestamps talk.wav
arecord -q -f S16_LE -r 16000 -c 1 -t raw | whistle stream
whistle serve # POST /transcribe, WebSocket /streamThe library is github.com/cwbudde/go-whistle/whistle: Load, TranscribeWAV, NewStream and Embed. WAV input has to be integer PCM, and a single call is limited to 30 s. The README covers the server protocol and the web demo, and docs/FINDINGS.md is the long version of this post.