The Text-to-Speech (TTS) Barrier Was Never the Parameter Count

The model file was 38 megabytes. The stack I had to ship beside it, before a single word reached a speaker, was several times that.

That gap is where offline text-to-speech (TTS) actually lives, and it is the part that rarely survives into the release announcement. The announcement has a parameter count in it. Parameter counts are clean, they fit in a headline, and they have much less to do than you would expect with whether you can put a synthetic voice inside a game build, a kiosk, a handheld, or an app that has to keep working on a train with no signal.

Verdict, up front: for offline and embedded work, the parameter budget is the cheapest line item in your deployment, and the field's habit of ranking compact speech models on size-against-quality is relitigating a barrier that fell some time ago. The barriers still standing are licensing, the linguistic front end, and evaluation. None of them are neural.

The belief, stated plainly

Somewhere in the last decade the field settled into a working assumption: natural-sounding synthetic speech is computationally expensive, and the way a small team gets access to it is by shrinking something that was expensive first. The vocabulary is all subtraction — distillation, pruning, quantization, "tiny" this and "nano" that. Compact releases inherit the framing and present themselves as achievements against scale. Look how little this needs. Look what we gave up and got away with.

It is a good story and it flatters everybody in it. It is also, for the decision in front of you, close to irrelevant. The belief has a specific historical source, and the source is thinner than the structure built on top of it.

A short history of the ladder

Speech synthesis is one of the older applied problems in computing, and its cost curve has never moved in one direction.

Read the middle column downward. The expensive thing keeps moving. In 1985 the barrier was linguistic knowledge encoded by hand. In 2000 it was studio time and a voice talent contract — a good unit-selection voice meant weeks of recording, and if you wanted a second voice you did the whole thing again. In 2016 the barrier became compute, and it stayed there long enough that a generation of practitioners started treating it as the natural order.

The autoregressive raw-audio models of that period earned the reputation. They produced a real jump in naturalness and they were genuinely, structurally slow, because they predicted one audio sample at a time and each sample waited on the last. At 22.05 kHz that is twenty-two thousand sequential steps per second of output. The slowness was not a tuning problem; it was the architecture.

Then it stopped being true. Adversarially trained vocoders learned to emit a waveform in one pass. End-to-end architectures in the VITS family folded acoustic modelling and vocoding into a single network trained together, so you no longer paid for two models and the seam between them. Compact checkpoints in that family now run comfortably faster than real time on ARM boards that cost less than a decent microphone cable. The expensive thing moved again, and the belief did not move with it.

The rung the whole ladder hangs from

Here is the part worth sitting with, because it is where the source turns out to be thinner than the belief.

Every "X sounds better than Y" claim in modern speech synthesis traces back, eventually, to mean opinion score. MOS is a five-point absolute category rating: play a listener a clip, ask how natural it sounded, average the numbers. Its methodology descends from ITU-T Recommendation P.800, a standard written to compare telephone transmission quality — codecs and circuits on narrowband phone lines. It was designed to answer whether one compression scheme degraded a call more than another. It was not designed to rank generative models against human recordings, and it was certainly not designed to be aggregated across papers.

But that is what happened. Listener panels are small, because listener panels are expensive. Each paper picks its own test sentences, its own speakers, its own playback instructions, its own crowd platform. Confidence intervals get reported and then quietly ignored in the summary table, which is a shame, because on the "approaches human recordings" claims those intervals frequently overlap with each other and sometimes with the ground-truth condition. Two systems separated by a tenth of a MOS point in different papers, on different sentences, with different listeners, are not ranked. They are unmeasured relative to each other.

The field knew this, and the fix was reasonable: train a model to predict MOS. Automated MOS predictors — the UTMOS family and its relatives, which came out of the VoiceMOS challenge work — take audio in and emit a number that correlates with what human panels gave for similar audio. They are cheap, repeatable, and remove the crowd-platform noise. They are a real contribution and I use them.

They are also a proxy trained on a proxy. The predictor learned from listener ratings collected under specific conditions, on specific systems, in a specific period. Applied to a compact model trained on different data at a different sample rate, it is extrapolating, and it will hand you a confident-looking decimal while doing it. Compact-model release notes that publish a predicted MOS with its confidence interval and say plainly which comparisons used different test-set sizes are doing the honest thing. Most of what gets repeated downstream on forums and model cards drops the interval and keeps the decimal.

So when someone tells you that a 9-million-parameter system is nearly as good as one thirty times its size, the sentence has a real measurement under it, and that measurement is a telephone-era rating scale, filtered through small panels, approximated by a neural predictor, reported without the error bars. It is not nothing. It is much less than the confidence the sentence carries.

Which means the size-versus-quality ladder that organizes how the field talks about compact synthesis is built on a rung nobody has load-tested for that purpose. Fine for a rough sort. Bad for a shipping decision.

How I'd decide instead

These are the criteria I actually use when I have to put a voice in something that runs on hardware I do not control. None of them are the parameter count.

Real-time factor, measured on the target

Real-time factor is seconds of audio produced per second of wall clock. Above 1× you can stream; below it you must pre-render. Your workstation will tell you a flattering number. Measure on the device, with the CPU governor and thermal profile it actually ships with, while your renderer or your app is under load — not on an idle board plugged into the wall.

# rough RTF: compare wall-clock time against the duration of the output
time ./synth --text "$(cat bark_lines.txt)" --out /tmp/out.wav
soxi -D /tmp/out.wav

Then ask the second question, which matters more for interactive audio: not throughput but time-to-first-sample. A system that renders a ten-second line at 5× still makes the player wait if it will not emit anything until the whole line is done. And do not run inference on the audio callback thread. Synthesize on a worker, into a buffer, with an allocation strategy you have looked at.

The footprint you actually ship

Do the arithmetic yourself, because it is the one number in this piece nobody has to interpret for you: a float32 parameter is four bytes. A 9.36-million-parameter model is somewhere around 37 MB before quantization and around a quarter of that at int8, if the architecture quantizes cleanly, which is a real if.

Now add the inference runtime. Add the phoneme dictionaries and any lexicon files. Add the fallback path for text the front end cannot handle. In my experience the weights are frequently the smallest item on that list, and a release that reports only the parameter count has told you the least interesting part of its own footprint.

The license on the front end, the code, and the voice — separately

This is the one that burns people, and it burns them late.

Many compact neural systems are not end-to-end in the deployment sense. They convert text to phonemes using an external grapheme-to-phoneme component, and for a long stretch the default in open-source speech tooling has been eSpeak-ng, which is a formant synthesizer of the old school pressed into service as a phonemizer. eSpeak-ng is copyleft. If you are bundling it into a proprietary game binary or a closed-source app, that is a licensing question you want answered by someone who reads licenses, before the build, not after the store submission.

And it is three questions, not one. The code license, the weights license, and the provenance of the voice are separate and frequently different. A permissive weights license does not tell you whether the speaker consented to having their voice modelled, and "the source recordings were public domain" is a statement about copyright, not about consent or about your exposure if the voice is recognizable. If a project cannot tell you what it was trained on, that is information about your risk, and you should treat it as such without needing anyone to say it out loud.

Sample rate, and where the voice sits in your mix

A large share of open speech checkpoints are trained at 22.05 kHz, an inheritance from the single-speaker corpora that the field standardized on years ago. Your game or film pipeline is almost certainly running 48 kHz. That means resampling, and it means the synthetic voice arrives with an audibly different top end than the foley and dialogue recorded next to it. In a busy mix you will not notice. In a quiet room, under a single line of dialogue, you will, and no amount of EQ puts back air that was never captured. Plan for it, or plan to sit the voice somewhere in the mix where the difference reads as intentional — radio, PA system, in-world device.

Your text, not the test set

Benchmarks read clean prose. Your application does not. Feed the candidate your actual strings: item names, acronyms, player-entered nicknames, version numbers, times, currency, the fantasy proper nouns your writer invented on a Tuesday. Compact models with a small front end handle these unevenly, and the failure is not a crash — it is a shopkeeper confidently mispronouncing the name of your game's main city for the entire playthrough.

What the constraint buys, and what it costs

A small local model is a scope statement, and the honest ones say so. What you get: no network dependency, no per-character billing, no text leaving the device, no vendor deprecating your voice mid-project, and a latency floor set by your own silicon. For anything that ships offline, those are not conveniences, they are the requirement.

What it costs, stated as plainly: prosody flattens over long sentences. Compact models in this class tend to render a two-clause line with less variation than a human reader would, and questions can arrive with the intonation of statements. Emphasis is hard to control without markup the model may not support. You will likely get one voice, or two, and a project that deliberately cut its voice count to hold quality has made a defensible engineering choice rather than a compromise — but it is still one voice, and your cast needs more.

The alternatives carry their own damage. Cloud synthesis gives you range and multiple speakers and bills you per character forever, sends your text off-device, and can change its terms and its voice inventory on a schedule that is not yours. Old-school formant synthesis is tiny, fast, offline, and free of all of the above, and it sounds like 1986 — which is occasionally exactly what a project wants and usually not.

The barrier that actually moved

Here is what compact synthesis genuinely changed, and it is not quality.

It changed iteration speed and it changed permission. You can hear a line read at two in the morning without booking a booth, without a session fee, without asking anyone. You can rewrite the line and hear it again ten seconds later. For a solo developer, that is the difference between having voiced barks in the vertical slice and cutting them from the design doc in month two. For accessibility work — screen reading, in-app narration on devices with no connection — it is the difference between the feature existing and not.

That is barrier reduction, and it is real. It is also narrower than the way the phrase usually gets thrown around. It did not remove the need for taste, direction, or a human performer when the line carries the scene. A model that has never been in a room where someone died does not know how to read the line about it, and the flat prosody you hear on long sentences is that gap surfacing. Use the small model for the four hundred barks. Hire the actor for the twelve lines that matter.

Who this is for

Indie developers shipping offline builds with hundreds of low-stakes voiced lines. Local-first app builders who need narration without a network round trip or a billing relationship. Edge engineers with a fixed footprint budget and a CPU-only target. Anyone whose text is short, in a well-supported language, and predictable in shape.

Who should skip it

Anyone whose project needs a cast — several distinct, directable voices with emotional range across long-form delivery. Anyone shipping into a language the front end covers thinly, where the phoneme coverage will fail before the acoustics do. Anyone in a regulated context — medical, legal, safety-critical announcements — where a system's published evaluation does not cover the use case, and where "it scored well on read prose" is not a claim about your deployment. And anyone who cannot get a straight answer about the training data, if the thing ships to customers.

The compact-model story that gets told is a story about doing more with less. The truer story is that the field's cost curve moved three times in forty years and the belief only ever caught up to the third move. Parameters stopped being the wall a while ago. What is left in your way is written in plain language — in a license file, in a model card, in a paragraph about what the evaluation did not cover.

The weights got small years ago. Everything still blocking you is prose.