Marimba

A fast, lightweight audio model that delivers typed decisions from speech

Speech-to-text models are often overkill when all you need is a decision.

3 min readMachine Learning

Audio models have long been caught between two extremes: massive, slow systems that transcribe everything and brittle keyword matchers that miss the nuance in human speech. Marimba sidesteps that trade-off entirely. It is a fast, lightweight model that listens for a decision, yes or no, a choice, a score, and returns a typed answer without transcribing the full conversation. That is a genuinely practical shift, and it matters more than another incremental improvement in word-error rate.

What makes Marimba interesting is not the size of its underlying model, a frozen Qwen3-ASR-0.6B is small by current standards, but how it uses that model. Instead of decoding every syllable, it extracts zero-shot scores from the ASR decoder's hypotheses and calibrates them to answer specific questions. Small readouts on the frozen encoder handle voice activity, turn endings, speaker changes, and speaker count. The result is a tool that does not need to understand everything a person says; it only needs to understand the one thing you asked it to hear. That distinction is the difference between a general-purpose transcription engine and a purpose-built decision engine. The approach echoes a lesson we saw in Why Simple Keyword Search Still Beats AI Models for Agent Tool Selection: sometimes the most efficient path is not the one that models the most data, but the one that models the right data.

For anyone building voice-driven workflows, the practical consequence is immediate. Marimba runs locally, installs with `pip install marimba-audio`, and ships under an Apache-2.0 license. That means no API latency, no per-query costs, and no sending audio to a third party. A spreadsheet user who wants to say "add five to column B" and get a typed instruction back can wire this into a macro in minutes. A field researcher who needs to log yes/no responses from interviews can process hours of audio on a laptop. The model's design also opens a path toward the kind of AI-native spreadsheet interactions that our University Research Rankings Reveal AI Spreadsheet Innovation Trends have been tracking, where the tool adapts to the user's natural input rather than forcing the user into a rigid form.

The open question is how well the zero-shot calibration holds up across accents, recording conditions, and ambiguous queries. The paper and benchmarks due this week will answer that. But the architecture itself is worth watching, because it points toward a broader principle: the next generation of AI tools will not be the ones that do everything, but the ones that do one thing so reliably that you stop thinking about the tool and start thinking about the work. Marimba's decision to return typed answers from speech, rather than raw transcripts, is a small bet on that principle. The concrete detail to watch is whether its calibration can handle a speaker who says "maybe" when the model expects only "yes" or "no", because that edge case will determine whether this stays a demo or becomes a daily driver.

From Machine Learning

I built it to be fast and lightweight, and to return typed decisions about speech: yes/no, a choice, or a score.

It runs on a frozen Qwen3-ASR-0.6B. Word and language questions use zero-shot scores from the ASR decoder's hypotheses, followed by calibration. Small readouts on the frozen encoder handle voice activity, turn endings, speaker changes and speaker count.

Read the original at Machine Learning