Text-to-speech API platforms all sound good now — the phone agent is where they split

The gap was 900 milliseconds, and it ruined everything else.

I was sitting in on a call test for a client's booking line — the kind of agent that gets built in six weeks and runs for three years. The voice was excellent. Warm, unhurried, no robotic seam in the consonants. Then the tester said "no, wait, wrong date," and the agent talked over her for most of a second before it stopped and apologized. That is where text-to-speech API platforms have landed: the audio stopped being the hard part, and everything around the audio became the hard part.

Here is the verdict in one sentence, since you are probably scanning: if you are shipping both batch narration and live phone agents, consolidate the live path onto a platform that treats streaming synthesis and turn-taking as one product, keep a second vendor behind a thin adapter for the batch path, and choose on latency chain, concurrency ceiling, and license terms rather than on how the demo reel sounds.

I came at this sideways. I score games and short films for a living; I own four synths that do not fully work. I ended up reading API docs because a client's agent sounded right and behaved wrong, and nobody on the build could tell me which of the five services in the chain was eating the second. That is the whole story of synthetic voice in one incident. The instrument got good. The room around it did not.

What most people do

Most teams arrive at voice by accretion. Marketing wants narration on product pages, so someone wires up a speech API and ships it in an afternoon. Support wants a phone agent, so someone else wires up a real-time provider, a transcription service, an LLM, and a SIP trunk or a Twilio number. Two months later there are four vendors, four billing models, four status pages, and a glue service nobody wants to own. Each seam between them is a place where latency, cost, and blame accumulate quietly.

The second pattern is a pricing mental model that works fine at demo scale and breaks at product scale. Per-character rates feel like rounding errors. Then a product lead asks for narration on every SKU, and you multiply: forty thousand products, six hundred characters of copy each, twenty-four million characters for one pass. Then copy changes and you pay for the second pass, and nobody built a cache, so you also pay every time a build re-runs. On the live side, per-minute pricing feels equally harmless until you find out what counts as a minute. Some meters bill wall-clock session time, including the silence while your caller looks for their order number. Some bill generated audio only. Some bundle transcription, model tokens, and synthesis into one agent minute; some pass model tokens through at a markup you find in the invoice rather than the pricing page.

The third pattern is choosing on the demo. Vendor demos are engineered — clean declarative sentences, the kind of copy that shows off breath and sibilance and an emotional turn at the end of a line. Your corpus is not that. Your corpus is "order SKU-4471B ships Tuesday to 1120 NE Halsey," and "your balance is $1,204.50," and a customer named Nguyễn in an otherwise English sentence, and the string "1/2" that could be a fraction or a date. Nobody's reel contains those, which is exactly why you should build a script that does.

And the fourth: "we'll abstract it later." Later is a four-hundred-line conditional in the audio service, with vendor-specific voice IDs hardcoded across three repos, written by someone who has since moved teams.

Worth naming what changed underneath all of this, because it reframes the purchase. Five years ago the objection to synthetic voice was that customers would know. Now they mostly do not, and the ones who do mostly do not care as long as the call resolves. The tester on that booking line told me afterwards she assumed she was talking to a person on a bad connection. Synthetic voice stopped being a novelty and became plumbing, and plumbing gets bought on different criteria: uptime, unit economics, what happens at peak, and what the contract says. The interesting question moved from "is it convincing" to "what does it cost me to run three hundred concurrent calls on a Tuesday afternoon in November."

What the evidence suggests

When I actually put vendors side by side on the same text, four things hold up consistently.

The quality spread narrowed at the top and survives in the ugly cases. Between the leading providers, ordinary prose is close enough that blind comparison turns into coin-flipping. The differences show up in alphanumerics, proper nouns, code-switching, currency, times, abbreviations, and repair turns — the moment where a human would put stress on the corrected word ("no, the fourteenth") and most models put stress nowhere. If your evaluation set is marketing prose, every vendor passes and you learn nothing. If it is your worst two hundred strings, the ranking is stable and it is yours.

Phone calls destroy the fidelity you are paying for. The public switched telephone network is narrowband: G.711 companding at 8kHz, a passband that gives up somewhere around 3.4kHz. Whatever 48kHz render you generated gets crushed into that pipe. Practically, this means the phone path and the content path are two different purchases. On the phone path you are buying prosody, timing, and intelligibility through a codec. On the content path you are buying fidelity, expressive range, and long-form stability. Teams that pick one premium voice vendor for both are usually overpaying on one side and underserved on the other.

Latency is a chain, not a number. A vendor's "time to first audio" is one link. The full budget is endpointing (deciding the caller stopped talking), transcription finalization, model time-to-first-token, synthesis time-to-first-byte, network transit, and whatever jitter buffer sits at the edge. In ordinary conversation, people hand off turns in roughly the time it takes to say a syllable; in my testing, anything past about a second reads as a bad line, and the caller starts talking over the agent. You can have a best-in-class synthesis latency and still lose the call to an endpointing threshold set at 800 milliseconds of trailing silence.

Barge-in is where architectures genuinely differ, and it trades against smoothness. When the caller interrupts, the agent has to stop — which means killing an in-flight synthesis stream and dumping audio that has already been buffered downstream. Platforms that buffer aggressively for glitch-free playback are, by construction, worse at interruption. Ask how much audio sits ahead of the playhead, and whether the cancel is client-side or a round trip. Then test it with a human, because the failure is perceptual, not logged.

Three more things that are less about audio and more about survival.

Concurrency ceilings are the real constraint, and they are frequently not on the pricing page. Ask for the number in writing, and ask what happens when you cross it: hard 429s, queueing, or silent degradation to a smaller model. The third one is the worst, because your agent gets subtly worse at peak load and your monitoring says everything is fine.

Licensing is where people get burned, and it is where vendors differ most. The questions worth answering before signing: does the tier you are on permit commercial use of the output, or does that sit one tier up; who owns the generated audio; what consent and verification process applies to a cloned voice, including a founder's or a customer's; does a cloned voice remain usable if you cancel, or does it live and die with the subscription; is your input text used for training, and is opt-out default or request-only. These live in the master agreement and the DPA, not the pricing page footnote. Read them, or have someone read them, before the phone agent is load-bearing.

Preference leaderboards are a shortlist filter, not a decision. They rank on aggregate human preference over someone else's sentences, they move week to week as models ship, and they cannot tell you how a vendor handles your product names at your concurrency from your region. Use them to pick three candidates. Do not use them to pick one.

Here is the comparison table I keep, because it is the one that actually moves decisions:

What I actually do

I split the workload in two before I look at a single vendor, because the two halves want opposite things.

The deterministic path is everything cacheable: product narration, onboarding, IVR prompts, anything where the same text produces the same audio. It is latency-insensitive and fidelity-sensitive. I key the cache on a hash of text, voice ID, model, and synthesis parameters, and store the rendered audio in object storage rather than caching the request. In most catalogs the long tail never changes, which makes this the single largest cost lever available — larger than any per-character discount you will negotiate.

The live path is the phone agent and anything conversational. It is latency-sensitive, and past 8kHz its fidelity is thrown away by the network. This is the half worth consolidating, because the seams between transcription, model, and synthesis are where the second of silence comes from, and a vendor who owns all three can shave that chain in ways your glue code cannot.

Both sit behind one adapter with a deliberately small surface: synthesize text with a voice and a format, return a stream, plus a passthrough dictionary for vendor-specific parameters. I do not build a lowest-common-denominator abstraction — that throws away the parameters you are paying for. I build a swap that takes a week rather than a quarter, and I am honest with myself that it is a week.

Then I run the same evaluation on every candidate, and it takes about two days:

• Forty lines of the worst real text I can find — order numbers, street addresses, an all-caps SKU, a URL, a price, a date written as "1/2", one name from a language the copy is not in.
• p50 and p95 time-to-first-audio measured from the production region, from a deployed process, not from my laptop over hotel Wi-Fi.
• A twenty-minute soak at target concurrency plus fifty percent, logging rate-limit responses and latency drift over time. Latency drift is the tell; some platforms are fine for four minutes.
• Twenty barge-in attempts with a human interrupting about 300ms into the agent's turn. I count clean stops.
• Listening to the phone path through an actual phone at 8kHz, not to the source WAV in monitors. Half the things you agonize over in headphones do not survive the codec, and one or two things you did not notice become the only thing you hear.
• A twelve-month cost model with three-times traffic growth, including minimums, overage rates, and whatever the meter does with silence.
• Reading the license. Commercial use, output ownership, cloning consent, termination.

What I ship, most of the time, is a consolidated vendor on the live path, a second vendor on the batch path chosen for licensing clarity and long-form stability, and a cache in front of both. The cost of that shape is two contracts and two sets of voice IDs, which is a real tax. The benefit is that the adapter stays honest because it is exercised daily, and no single vendor's Tuesday outage takes down both the phone line and the catalog.

Every class of vendor here has a failure mode worth stating plainly.

Unified agent platforms — the "one API for synthesis and phone agents" pitch — remove the seams, and I do think that is the right default for the live path. You also inherit their orchestration opinions, debugging a bad turn means reading traces from inside their box, their concurrency ceiling becomes yours, and if they change model routing behind the scenes your agent's personality can shift without a deploy on your side. Pin what you can pin.

Specialist speech vendors still tend to hold the lead on expressive range and premium cloning, and if a distinctive voice is the product, that is where it lives. The cost is that you become the systems integrator: you own the latency chain, you write the interruption logic, and you are the one correlating four sets of logs at 2am.

Hyperscaler speech services are strong on regional coverage, compliance paperwork, uptime, and fine-grained pronunciation control through SSML and custom lexicons — which matters more than it sounds when your catalog is full of invented brand names. Conversational prosody has generally trailed the specialists, and the console ergonomics are what they are.

Self-hosted open models hand you the cost curve and full control of your data. They also hand you GPU capacity planning, cold starts, and ownership of a voice bug at 3am. Worth it at sustained high volume or under a data constraint that leaves no other option; a distraction below that.

Who this is for, and who should skip it

Consolidating is worth it if you run both workloads, if peak concurrency is a number your ops team can quote, and if the phone agent touches revenue. Skip it if you are batch-only at modest volume — one vendor and a cache is a complete answer, and adding a second is a hobby. Skip it if a specific cloned voice is your brand and it exists at one vendor; in that case your negotiation is about license terms, not architecture. And skip it if procurement has already blessed a hyperscaler and your compliance surface makes a second processor a three-month review. Architecture that cannot get approved is not architecture.

What this piece did not answer

It did not tell you which vendor wins, and that is deliberate — the ranking at the top moves faster than an article's shelf life, and anything I named today would mislead you in six months. It did not cover non-English quality at scale, which is a genuinely different evaluation and where the gaps between vendors are still wide. It did not model self-hosted economics against managed pricing, because that depends on utilization curves I would have to invent to write down. It did not resolve whether the unified-agent category survives as its own layer or gets absorbed into the model providers, which is a real bet you are making when you consolidate. And it did not touch how cloned-voice consent holds up across jurisdictions, which is a lawyer's question that is going to get more interesting, not less.

Where to look next, in order: your own call logs, segmented by turn latency rather than by average; the vendor's changelog and status page history, which tell you more about operational maturity than any benchmark; the master agreement rather than the pricing page; and then your own evaluation script, re-run quarterly, because the only ranking that matters is the one measured on your text from your region at your concurrency.

The voice is solved. The silence is what your customers will actually hear.