{"id":2218,"date":"2026-10-11T17:36:16","date_gmt":"2026-10-11T15:36:16","guid":{"rendered":"https:\/\/www.pcjv.de\/?p=2218"},"modified":"2026-10-11T17:37:11","modified_gmt":"2026-10-11T15:37:11","slug":"go-whistle-whistle-speech-to-text-in-pure-go-rebuilt-from-the-weights","status":"publish","type":"post","link":"https:\/\/www.pcjv.de\/?p=2218","title":{"rendered":"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights"},"content":{"rendered":"\n

Whistle<\/a> is a speech-to-text model from Cactus Compute<\/a>: 55 million parameters and 16.9 MB on disk, covering seven languages (English, German, French, Spanish, Italian, Dutch and Polish). The weights<\/a> are published under Apache-2.0, but the code that runs them is not. Upstream, in cactus-compute\/needle<\/a>, Whistle is a thin Python wrapper around a prebuilt, closed-source C++ engine. go-whistle<\/a> is my pure-Go port: no cgo, no Python, no ONNX. It does transcription with language detection, keyword biasing, word timestamps, live streaming and embeddings, from a CLI, an HTTP\/WebSocket server, a Go library, or a WebAssembly page in the browser.<\/p>\n\n\n\n\n\n\n\n

\"The
The web demo transcribing a German FLEURS clip, run on an AMD Ryzen 5 4600H laptop. The reference text is “Diese Paare k\u00f6nnen sich f\u00fcr einen Adoptionsplan f\u00fcr ihr Baby entscheiden.”; the closed engine produces exactly the same two errors as the port.<\/figcaption><\/figure>\n\n\n\n

Try the demo in your browser.<\/a><\/strong> It is published on GitHub Pages from every push. It downloads the 6.6 MiB WebAssembly build and the 16.1 MiB model from Hugging Face, checks the model’s SHA-256, and then transcribes WAV files or your microphone inside the tab. No audio leaves the browser.<\/p>\n\n\n\n

Here is a German clip from Google’s FLEURS test set (CC BY 4.0), one of the 32 fixture clips in the repository:<\/p>\n\n\n\n

FLEURS de_de, test split, clip 10229344228128634115. Reference text: “Jeder nimmt an der Gesellschaft teil und benutzt Transportsysteme. Fast jeder beklagt sich \u00fcber die Transportsysteme.”<\/figcaption><\/figure>\n\n\n\n
$ whistle transcribe fleurs-de_de-10229344228128634115.wav\nJeder nimmt an der Gesellschaft Teil und benutzt Transportsysteme, was jeder beklagt sich \u00fcber die Transportsysteme.<\/code><\/pre>\n\n\n\n
It gets “Fast jeder” wrong as “was jeder”. The closed engine makes the same mistake, because matching the engine’s output, mistakes included, is how the port is tested.<\/p>\n\n\n\n
Nothing to translate<\/h2>\n\n\n\n
Most of my ports start from source code in some other language. This one couldn’t. The public cactus-compute\/needle<\/a> repository has no speech code at all: no mel front end, no encoder and no decoder search. What it does have is a JAX reference for the shared Needle 3 blocks (the decoder is built entirely from those), the code for the .cact<\/code> container and its quantisation, and some prose about the pipeline. The weights come in two forms: all 55,151,433 parameters as FP32 safetensors, and the shipped 16.9 MB whistle.cact<\/code> with 2- and 4-bit weights, the mel filterbank and the tokenizer.<\/p>\n\n\n\n
The prebuilt engine then serves as a black-box oracle. needle_embed<\/code> returns the encoder output, one 512-wide row per 80 ms frame, and needle_transcribe<\/code> returns the final JSON. The engine never ships with go-whistle, and nothing loads it at runtime. It sits behind a build tag in tools\/oracle<\/code> and writes fixtures that the tests compare against. Every open question was settled by measuring against those fixtures, and each answer is written up in docs\/FINDINGS.md<\/code>, which now has 43 entries: the question, the alternatives, the metric and the winner. Some were quick. The CQ codebook in the file’s header comes from 400,000 samples of numpy’s RandomState(0)<\/code> and 200 Lloyd-Max iterations, and rerunning that code reproduces all 28 header floats bit for bit. Others took a real search.<\/p>\n\n\n\n
Fitting the encoder through one output<\/h2>\n\n\n\n
The encoder is where the sources leave the most open. The weights give the tensor shapes, but not the order of operations, the padding, the activations, or how the log-mel spectrogram is computed. The harness treats each of those as an axis: 864 front-end variants plus 19 encoder axes, such as stride placement, kernel layout, GLU form and module order. A hypothesis is one value on every axis. Its score is the mean per-frame cosine between our encoder output and the engine’s embedding over the clips. Scoring one hypothesis on all 32 clips takes about 0.3 s on an M5 Pro. A beam search over the stages (stem geometry first, then activations, front end, conv module and block structure) finds the answer in about three minutes.<\/p>\n\n\n\n
The scores need a baseline first. Comparing each engine frame with the mean engine frame of the other clips already gives 0.2349, so anything near that carries no signal. Started from a hypothesis that is wrong on all 26 axes (score 0.017903), the search still converges to the same leader, 0.999819 on the inner clips, in 7.5 minutes. The final norm can be checked directly. Every one of the engine’s 2205 rows has unit RMS after dividing by the FP16 gain stored in the .cact<\/code>, to within 1.8e-7, so the engine uses that gain and not the FP32 one.<\/p>\n\n\n\n
Speech can’t settle everything, so the oracle tool also synthesises probe signals: digital silence of six lengths, DC, 440 Hz tones and bursts at falling levels, noise, a slow fade to −140 dB, and single clicks at known offsets. On digital silence the stem’s output is exactly zero, so any position signal added afterwards would be the only input left. That is how pe_gate<\/code>, a learned gate of σ(−6.1094) = 0.00222 on a sinusoid, turned out to be unused. The fades and quiet tones exposed the log floor. It isn’t Whisper’s hard clamp eight decades below the peak, but a soft knee:<\/p>\n\n\n\n
\"Two
The engine’s log floor. A soft knee 7.6 decades below the clip’s mel peak matches the engine’s embeddings better than Whisper’s hard 8-decade clamp, on average and on the worst clip.<\/figcaption><\/figure>\n\n\n\n
With log10(x + 10−7.6<\/sup> \u00b7 max x), the encoder matches the engine at a mean per-frame cosine of 0.999828 over the 32 clips. The final embedding path, from the packed weights, reaches 0.999827, and every clip is at least 0.999.<\/p>\n\n\n\n
How close it gets<\/h2>\n\n\n\n
The engine computes with 2-bit weights and 8-bit activations, so equal bits were never the goal. The goal is equal transcripts. On a 200-utterance subset of LibriSpeech test-clean, the port has 223 errors in 4167 reference words (WER 5.35 %), and the engine has 221 (5.30 %). 181 of the 200 texts are identical. The rest of the parity table:<\/p>\n\n\n\n
\"Horizontal
Parity with the closed engine, measured with the default packed weights.<\/figcaption><\/figure>\n\n\n\n
The decoder search was its own reconstruction job: Whisper’s beam search at five beams, a silence gate (fitted against 340 synthetic signals until it agrees with the engine’s verdicts), and keyword biasing through an Aho-Corasick automaton. The bonus ended up at 2 nats per token, revoked when a partial match breaks. With one keyword, a LibriSpeech clip changes from “Here are some verses of colonies.” to “Here are some verses of Cowley’s.”:<\/p>\n\n\n\n
$ printf \"Cowley's\\n\" > keywords.txt\n$ whistle transcribe ls-other-3080-5040-0011.wav\nHere are some verses of colonies. Tell me how you like them.\n$ whistle transcribe --keywords keywords.txt ls-other-3080-5040-0011.wav\nHere are some verses of Cowley's. Tell me how you like them.<\/code><\/pre>\n\n\n\n
Each remaining difference is written up as a near tie. The last one investigated: with the keyword Paare<\/code> on a German clip, the engine writes “Adduktionsplan” and the port writes “Adoptionsplan”. The engine’s text actually scores 0.0688 nats per token higher under our model, but five finished hypotheses fill their slots one step before the leader can end. The cause is the keyword bonus being slightly too strong, not a missing bookkeeping rule.<\/p>\n\n\n\n
Running straight from the 2-bit file<\/h2>\n\n\n\n
Dequantised to float32, the weights take 221 MB. The 16.9 MB file stores each matrix row in groups of 128 as indices into a small codebook, times one FP16 norm per group, rotated by a \u00b11 Hadamard matrix. Because that matrix is symmetric, a product never needs the dequantised row: rotate each 128-wide group of the activation once with a fast Walsh-Hadamard transform, and the rest is table lookups over the packed indices. The assembly kernels (NEON TBL<\/code>, AVX2 VPSHUFB<\/code>) round the lookup tables to 14-bit integers in two byte planes, so one shuffle looks up 16 or 32 rows at once, and the sums stay exact. All 152 quantised tensors run this way. The packed path gives the same 200 LibriSpeech transcripts as the float path, and it is faster:<\/p>\n\n\n\n
\"Bar
The Go port against the closed C++ engine on the same Apple M5 Pro, medians from the repository’s benchmark run. Packed weights beat the float path everywhere, and load in 6.9 ms instead of 132.5 ms.<\/figcaption><\/figure>\n\n\n\n
Memory drops too. After loading, the process holds 37.9 MiB with packed weights and 238.8 MiB with float weights. One 30 s transcription peaks at 107.5 MiB, against 69.9 MiB for the engine’s CLI. The engine is still faster: 6.9 ms against 18.8 ms to the first token and 1499 against 1085 tokens per second, median over the 32 clips. That is hand-tuned C++ with int8 activations against Go with float32 activations. On the same 32 clips, a transcription takes 0.0082 of the audio’s duration. The tensor kernels (GEMM, attention, conv, RoPE) came from my go-pocket-tts<\/a> port and now live in their own module, algo-tensor<\/a>, which both projects use.<\/p>\n\n\n\n
Streaming, and the browser<\/h2>\n\n\n\n
One call takes at most 30 s of audio, as in the engine. For longer audio and microphones there is the engine’s streaming scheme. Every chunk, by default one second, re-transcribes the buffer, and only the words two consecutive passes agree on are committed. Once committed audio fills more than 15 s, the buffer is trimmed at a sentence end.<\/p>\n\n\n\n
\"Diagram
How streaming commits words: a word is committed only when two consecutive passes over the growing buffer agree on it. Real output of whistle stream on nine LibriSpeech clips joined into 44.91 s.<\/figcaption><\/figure>\n\n\n\n
In 797 of 930 stream calls the port makes the engine’s decision, and the final text is the engine’s in 98 of 108 runs. The stream JSON matches the engine’s byte for byte in fields, order and number format. That matters when something downstream already parses the engine’s output.<\/p>\n\n\n\n
The web demo is the same library compiled with GOOS=js GOARCH=wasm<\/code> and running in a Web Worker. WebAssembly gets the pure-Go fallback kernels, one thread and no fused multiply-add, so the float32 sums round differently from every native build. The transcripts survive this. The whole package passes its tests under Node, including the 32-clip engine parity, and on four clips the page’s JSON is byte for byte the CLI’s, minus the timings. It is slow, though. On the M5 Pro it does about 22 tokens per second and takes 0.3 to 0.8 s to the first token, against 18.8 ms natively. A 7 s clip takes about 3 s in Chromium.<\/p>\n\n\n\n
Trying it<\/h2>\n\n\n\n
go install github.com\/cwbudde\/go-whistle\/cmd\/whistle@latest\nwhistle download                  # fetches the pinned model files, checks SHA-256\nwhistle doctor                    # kernels, load time, a self-test\nwhistle transcribe --word-timestamps talk.wav\narecord -q -f S16_LE -r 16000 -c 1 -t raw | whistle stream\nwhistle serve                     # POST \/transcribe, WebSocket \/stream<\/code><\/pre>\n\n\n\n
The library is github.com\/cwbudde\/go-whistle\/whistle<\/code>: Load<\/code>, TranscribeWAV<\/code>, NewStream<\/code> and Embed<\/code>. WAV input has to be integer PCM, and a single call is limited to 30 s. The README<\/a> covers the server protocol and the web demo, and docs\/FINDINGS.md<\/code> is the long version of this post.<\/p>\n\n","protected":false},"excerpt":{"rendered":"
Whistle is a speech-to-text model from Cactus Compute: 55 million parameters and 16.9 MB on disk, covering seven languages (English, German, French, Spanish, Italian, Dutch and Polish). The weights are published under Apache-2.0, but the code that runs them is not. Upstream, in cactus-compute\/needle, Whistle is a thin Python wrapper around a prebuilt, closed-source C++ […]<\/p>\n","protected":false},"author":0,"featured_media":2219,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[45,41],"tags":[],"class_list":["post-2218","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai","category-go"],"yoast_head":"\ngo-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights - Christian's private site<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.pcjv.de\/?p=2218\" \/>\n<meta name=\"twitter:label1\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data1\" content=\"9 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218\"},\"author\":{\"name\":\"\",\"@id\":\"\"},\"headline\":\"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights\",\"datePublished\":\"2026-10-11T15:36:16+00:00\",\"dateModified\":\"2026-10-11T15:37:11+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218\"},\"wordCount\":1646,\"image\":{\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.pcjv.de\\\/wordpress\\\/wp-content\\\/uploads\\\/teaser-go-whistle.jpg\",\"articleSection\":[\"AI\",\"Go\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218\",\"url\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218\",\"name\":\"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights - Christian's private site\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.pcjv.de\\\/wordpress\\\/wp-content\\\/uploads\\\/teaser-go-whistle.jpg\",\"datePublished\":\"2026-10-11T15:36:16+00:00\",\"dateModified\":\"2026-10-11T15:37:11+00:00\",\"author\":{\"@id\":\"\"},\"breadcrumb\":{\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.pcjv.de\\\/?p=2218\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218#primaryimage\",\"url\":\"https:\\\/\\\/www.pcjv.de\\\/wordpress\\\/wp-content\\\/uploads\\\/teaser-go-whistle.jpg\",\"contentUrl\":\"https:\\\/\\\/www.pcjv.de\\\/wordpress\\\/wp-content\\\/uploads\\\/teaser-go-whistle.jpg\",\"width\":480,\"height\":360},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/?p=2218#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/www.pcjv.de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.pcjv.de\\\/#website\",\"url\":\"https:\\\/\\\/www.pcjv.de\\\/\",\"name\":\"Christian's private site\",\"description\":\"Private site of Christian-W. Budde\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.pcjv.de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights - Christian's private site","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.pcjv.de\/?p=2218","twitter_misc":{"Est. reading time":"9 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.pcjv.de\/?p=2218#article","isPartOf":{"@id":"https:\/\/www.pcjv.de\/?p=2218"},"author":{"name":"","@id":""},"headline":"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights","datePublished":"2026-10-11T15:36:16+00:00","dateModified":"2026-10-11T15:37:11+00:00","mainEntityOfPage":{"@id":"https:\/\/www.pcjv.de\/?p=2218"},"wordCount":1646,"image":{"@id":"https:\/\/www.pcjv.de\/?p=2218#primaryimage"},"thumbnailUrl":"https:\/\/www.pcjv.de\/wordpress\/wp-content\/uploads\/teaser-go-whistle.jpg","articleSection":["AI","Go"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.pcjv.de\/?p=2218","url":"https:\/\/www.pcjv.de\/?p=2218","name":"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights - Christian's private site","isPartOf":{"@id":"https:\/\/www.pcjv.de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.pcjv.de\/?p=2218#primaryimage"},"image":{"@id":"https:\/\/www.pcjv.de\/?p=2218#primaryimage"},"thumbnailUrl":"https:\/\/www.pcjv.de\/wordpress\/wp-content\/uploads\/teaser-go-whistle.jpg","datePublished":"2026-10-11T15:36:16+00:00","dateModified":"2026-10-11T15:37:11+00:00","author":{"@id":""},"breadcrumb":{"@id":"https:\/\/www.pcjv.de\/?p=2218#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.pcjv.de\/?p=2218"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.pcjv.de\/?p=2218#primaryimage","url":"https:\/\/www.pcjv.de\/wordpress\/wp-content\/uploads\/teaser-go-whistle.jpg","contentUrl":"https:\/\/www.pcjv.de\/wordpress\/wp-content\/uploads\/teaser-go-whistle.jpg","width":480,"height":360},{"@type":"BreadcrumbList","@id":"https:\/\/www.pcjv.de\/?p=2218#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/www.pcjv.de\/"},{"@type":"ListItem","position":2,"name":"go-whistle: Whistle speech-to-text in pure Go, rebuilt from the weights"}]},{"@type":"WebSite","@id":"https:\/\/www.pcjv.de\/#website","url":"https:\/\/www.pcjv.de\/","name":"Christian's private site","description":"Private site of Christian-W. Budde","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.pcjv.de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"}]}},"_links":{"self":[{"href":"https:\/\/www.pcjv.de\/index.php?rest_route=\/wp\/v2\/posts\/2218","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pcjv.de\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pcjv.de\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pcjv.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2218"}],"version-history":[{"count":1,"href":"https:\/\/www.pcjv.de\/index.php?rest_route=\/wp\/v2\/posts\/2218\/revisions"}],"predecessor-version":[{"id":2220,"href":"https:\/\/www.pcjv.de\/index.php?rest_route=\/wp\/v2\/posts\/2218\/revisions\/2220"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.pcjv.de\/index.php?rest_route=\/wp\/v2\/media\/2219"}],"wp:attachment":[{"href":"https:\/\/www.pcjv.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2218"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pcjv.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2218"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pcjv.de\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2218"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}