A pharmacist reading a prescription out loud rarely stumbles on a drug name, even an unfamiliar one, because years of training built the muscle memory. Voice AI has no such training, and a benchmark published September 17, 2026 shows just how far behind the leading systems still are. Synthio Labs, a clinical-grade voice and agentic AI company for pharma, released DOSE (Drug-name Oral Synthesis Evaluation), the first public benchmark built specifically to test how accurately commercial text-to-speech systems say medication names. The topline number is blunt: general-purpose voice AI models mispronounce up to one in three newly approved drug names, according to the announcement carried by Yahoo Finance, PYMNTS, NDTV Profit, and other outlets on the same day.
The gap matters because voice interfaces are creeping into pharmacy counters, telehealth calls, and medication-reminder apps faster than most safety testing has kept pace with. DOSE puts a number on a problem clinicians and pharmacists have complained about anecdotally for years: TTS systems tuned for natural-sounding conversation are not the same as systems tuned for clinical accuracy. Synthio Labs put that distinction plainly in its research materials, noting that sounding natural and saying the medicine correctly are different skills.
Don't miss new tech stories on Google
Add Tech Insider once in the Google app and our stories appear in your news suggestions.
What the DOSE Benchmark Actually Tests
DOSE evaluates nine commercial text-to-speech systems against 274 medicines, each one embedded inside a realistic clinical sentence rather than read as an isolated word. That design choice matters for anyone building a speech AI product, because a model that nails a drug name in isolation can still fumble it inside natural phrasing, sentence stress, and surrounding medical vocabulary. Of the 274 medicines in the set, 146 are newly approved drugs and the remaining 128 are established names that have circulated in clinical use for years.
Each pronunciation gets scored on a 0-5 scale against verified reference pronunciations, with a score of 4 or higher counting as a pass. Synthio Labs describes the reference set as clinical-grade, built from pronunciation dictionaries and reviewed manually alongside spectrogram comparison, an approach the company had already outlined in an earlier blog post on its internal evaluation framework for pharma voice AI. The full dataset, including audio clips, has been published on Hugging Face, making DOSE reproducible rather than a one-off press claim.
Nine Systems, One Consistent Weak Spot
The systems named in the announcement include ElevenLabs’ eleven_v3, Google’s Gemini TTS, Microsoft Azure TTS, OpenAI’s text-to-speech stack, Deepgram, and Synthio Labs’ own RxPronounce model, built specifically for pharmaceutical vocabulary. Across all nine systems and all 274 medicines, general-purpose models passed between 63.1% and 80.3% of names overall, according to the benchmark. That range already looks shaky for a clinical setting, but it gets worse once the test narrows to drugs approved in the last few years.
ElevenLabs’ eleven_v3 scored 93.0% on established drug names but dropped to 67.1% on newly approved ones, a 25.9 percentage-point fall. Google’s Gemini TTS followed a similar pattern: 89.1% on established names against 61.6% on new approvals, a 27.5-point drop. Microsoft’s Azure TTS reportedly passed fewer than half of generic, or INN (International Nonproprietary Name), drug names, the weakest category-level result disclosed in the announcement. OpenAI’s TTS models and Deepgram both showed sharp drops on recently approved drugs as well, though the announcement did not break out their exact percentages the way it did for ElevenLabs and Google.
DOSE Benchmark at a Glance
| Metric | Value |
|---|---|
| Benchmark name | DOSE (Drug-name Oral Synthesis Evaluation) |
| Published by | Synthio Labs |
| Publish date | September 17, 2026 |
| TTS systems tested | 9 commercial systems |
| Total medicines tested | 274 |
| Newly approved drugs in set | 146 |
| Established drugs in set | 128 |
| Scoring scale | 0-5 (pass requires 4 or higher) |
| Test format | Drug names embedded in real clinical sentences |
| Dataset access | Public, with audio, hosted on Hugging Face |
Model-by-Model Pass Rates
| System | Established Drugs | Newly Approved Drugs | Point Drop |
|---|---|---|---|
| Synthio Labs RxPronounce | 91.2% overall | 87.0% | Smallest drop disclosed |
| ElevenLabs eleven_v3 | 93.0% | 67.1% | -25.9 points |
| Google Gemini TTS | 89.1% | 61.6% | -27.5 points |
| Microsoft Azure TTS | Not disclosed | Under 50% on generic/INN names | Weakest generic-name score |
| OpenAI TTS | Not disclosed | “Drops sharply” per announcement | Not disclosed |
| Deepgram | Not disclosed | “Drops sharply” per announcement | Not disclosed |
Read the table carefully and a pattern jumps out: RxPronounce’s 87.0% pass rate on newly approved drugs sits 10.9 percentage points ahead of the next-best system on that same category, according to Synthio Labs’ own reporting. That is a meaningful gap, but it is also worth noting that a company publishing a benchmark where its own product wins is not a neutral referee. The value of DOSE lies less in “Synthio wins” and more in the fact that every general-purpose system, including ones from Google, Microsoft, and OpenAI, showed a real and measurable falloff exactly where patient safety risk concentrates: brand-new medications that pharmacists, patients, and call-center scripts have not yet memorized.
The Xofluza Problem: When AI Spells Instead of Speaks
The single most concrete example to come out of the announcement involves Xofluza, an antiviral drug name. According to the benchmark writeup, one of the tested systems did not attempt to pronounce the name at all. Instead, it spelled the drug letter by letter: X-O-F-L-U-Z-A. In a chat interface that failure mode might read as a quirky glitch. Read aloud to a patient on a pharmacy refill call or through a telehealth visit summary, it becomes something closer to a dead end, an AI system that cannot complete the one task it was deployed to do.
That kind of failure is instructive because it shows the limits of statistical language modeling applied to a domain with its own naming conventions. Drug names frequently combine Latin and Greek roots, chemical suffixes, and invented syllables that do not follow standard English phonetic patterns. A model trained mostly on conversational text and news articles has comparatively few examples of these words in its training data, and newly approved drugs by definition have almost no exposure at all before their approval date. That is the structural reason the newly-approved category is where every system stumbles hardest.
Why New Drugs Are the Hardest Case
Established drug names have had years to accumulate transcribed audio, closed captions, pharmacy training material, and general internet mentions that a large TTS model’s training pipeline can absorb. A newly approved drug has none of that head start. It shows up in FDA approval documents, clinical trial registries, and prescribing information, but rarely in the kind of conversational audio corpora that voice models learn pronunciation patterns from. That mismatch is precisely what DOSE was designed to expose, and it is why the newly-approved subset produces the widest gaps in the benchmark.
Synthio Labs’ RxPronounce: A Narrow Lead, Not a Solved Problem
Synthio Labs describes itself as a clinical-grade voice and agentic AI company built for pharma, and RxPronounce is the product at the center of that pitch. The company says its model interprets complex regimens, branded drug names, and chemical molecules with the accuracy pharmacy and telehealth workflows require. An 87.0% pass rate on newly approved drugs is a real improvement over the 61.6% to 67.1% range posted by Gemini TTS and eleven_v3, but it is not a clean pass either. Roughly 13 in 100 new drug names still trip up Synthio’s own system under its own scoring rubric, a reminder that domain-specific tuning narrows the gap rather than closing it outright.
That nuance matters for how the industry should read this release. DOSE is simultaneously a genuine contribution to an underserved area of AI evaluation and a marketing document for the company that built it. Both things can be true at once, and the more durable value is the public, audio-backed dataset now sitting on Hugging Face, which other researchers and vendors can use to check their own systems independent of Synthio’s framing.
The Clinical Stakes: LASA Drugs and Medication Errors
Pharmacy safety programs have tracked look-alike, sound-alike drug pairs for decades. The Institute for Safe Medication Practices (ISMP) maintains a confused-drug-name list precisely because similar-sounding medications, at different doses or for different conditions, have caused real patient harm when misheard or misspoken. A voice AI system that already struggles with a single, isolated drug name faces a harder problem still when two different medications sound nearly identical. Synthio Labs flagged this directly in its earlier evaluation-framework writeup, describing mispronunciation errors as “potentially dangerous, especially with similar-sounding medications.”
Separate research from Roark, a company that tests voice agents for call-center and telehealth use cases, has proposed a related methodology: building a “golden set” of drug names that explicitly includes LASA adversarial pairs drawn from ISMP and USP confused-drug lists, then running snapshot regression testing every time a model or voice changes. That approach and DOSE are not the same benchmark, but they point at the same underlying gap. Off-the-shelf voice AI, optimized for fluent general conversation, was never built with medication-name safety as a first-class requirement, and the industry is only now building the tooling to measure that gap systematically.
Competitive Landscape: How Voice AI Accuracy Benchmarks Compare
DOSE is positioned as the first public benchmark focused specifically on drug-name pronunciation, and the available reporting supports that framing; general speech and TTS benchmarks exist in abundance, but none of the widely cited ones isolate medical vocabulary, let alone the newly-approved-drug subset that DOSE singles out. That said, DOSE does not exist in a vacuum. It sits alongside a small but growing set of adjacent efforts, including Roark’s LASA-pair regression testing approach for voice agents and Synthio’s own earlier internal evaluation framework, which already scored mispronunciation as one dimension among several (alongside empathy, clarity, and clinical accuracy against USMLE-style question sets).
The broader AI industry has spent the past year hammering on general-purpose model accuracy in ways that make DOSE’s narrow focus notable by contrast. Benchmarks comparing hallucination rates across leading chatbots have shown wide gaps between vendors on factual accuracy, and the same competitive dynamic appears to be replaying itself in voice: general-purpose leaders such as Gemini and eleven_v3 dominate naturalness and latency benchmarks, while a narrower, purpose-built model claims the accuracy crown on a specialized task. That is a familiar shape in AI product competition, and pharma voice AI looks likely to follow it.
Market Impact: Pharmacy, Telehealth, and Voice Assistants
Where the Risk Concentrates First
Pharmacies increasingly lean on interactive voice response systems, automated refill reminders, and AI call agents to handle patient volume that outpaces staffing. DOSE’s findings suggest that a meaningful share of those automated calls could be mispronouncing the very medication names patients are calling about, particularly for anything approved in the last year or two. That risk compounds when a pharmacy’s automated system is reading out a name a patient has never heard before, since the patient has no independent way to catch the error.
Telehealth platforms face a similar exposure. Visit summaries, medication counseling calls, and AI-generated voice explanations of a new prescription all depend on the underlying TTS engine getting the drug name right on the first try. A misheard or garbled name in a recorded visit summary is not just an annoyance; it can create a documentation mismatch between what a patient believes they were prescribed and what is actually on file. Companies building self-hosted text-to-speech pipelines for regulated industries will likely see this benchmark cited as a reason to add a medical-vocabulary verification layer before shipping voice features into clinical workflows.
Consumer voice assistants sit at a lower-stakes but higher-volume end of the same problem. When a smart speaker or phone assistant answers a spoken question about a medication, a mispronunciation mostly damages trust rather than creating a documented clinical error. But as more people use general assistants for basic health questions, that trust erosion has its own cost, and it gives purpose-built, pharma-specific voice models a commercial argument for licensing their pronunciation layer into consumer products rather than competing head-on with the platform giants.
Historical Context: Voice AI’s Slow Road to Clinical Trust
Speech synthesis has improved dramatically on naturalness and latency over the past three years, with systems like Gemini TTS and ElevenLabs’ eleven_v3 setting the pace on how human a synthetic voice can sound in real time. That progress has mostly been measured on general conversational benchmarks, where prosody, emotional range, and multilingual fluency dominate the scoring criteria. Clinical accuracy on specialized vocabulary was rarely the headline metric, largely because most TTS vendors built their models for broad consumer and enterprise use cases rather than regulated healthcare specifically.
That gap between “sounds impressive in a demo” and “is safe to deploy in a pharmacy” mirrors an older pattern in healthcare AI more broadly. Diagnostic imaging models, clinical documentation tools, and now voice interfaces have each gone through a phase where general capability outpaced domain-specific validation, followed by a wave of specialized benchmarks built to catch the gap before it caused real-world harm. DOSE reads as the voice-AI chapter of that same cycle, arriving roughly at the point where enough pharmacy and telehealth deployments exist to make the safety question urgent rather than theoretical.
How Developers Might Test for This
Teams building voice features on top of a general-purpose TTS API can borrow the shape of DOSE’s methodology even without access to the full dataset: build a small reference set of drug names relevant to their own patient population, score generated audio on a simple pass/fail rubric, and re-run the check any time the underlying model or voice changes. A minimal version of that regression check might look like this:
drug_reference_set = [
{"name": "Xofluza", "reference_audio": "xofluza_ref.wav"},
{"name": "Ozempic", "reference_audio": "ozempic_ref.wav"},
]
for drug in drug_reference_set:
generated_audio = tts_client.synthesize(drug["name"])
score = compare_pronunciation(generated_audio, drug["reference_audio"])
passed = score >= 4 # DOSE-style 0-5 scale, pass threshold 4
log_result(drug["name"], score, passed)
That pattern will not replace a clinical-grade benchmark like DOSE, but it gives engineering teams a cheap first line of defense before a mispronounced drug name ever reaches a patient call.
Industry Response and Open Questions
None of the vendors named in the DOSE announcement, including ElevenLabs, Google, Microsoft, OpenAI, and Deepgram, had issued a public response to the specific benchmark results at the time of this report. That is not unusual for a same-day press release; vendor responses to third-party benchmarks typically arrive days or weeks later, if at all, and often take the form of updated model releases rather than direct rebuttals. Given how fast general-purpose TTS models iterate, several of the systems named in DOSE could see meaningfully different pass rates within a few release cycles, which is exactly why Synthio Labs published the underlying dataset rather than a static leaderboard.
One open question the announcement does not resolve is how DOSE’s scoring compares against human clinician judgment at scale, beyond the manual review Synthio describes in its methodology. Another is whether regulators, such as the FDA in the context of digital health tools, will eventually reference benchmarks like DOSE when evaluating AI-enabled pharmacy or telehealth products. Neither question has a confirmed answer yet, and any claim otherwise should be treated as speculation rather than reported fact.
What Comes Next
Five Predictions for the Next 18 Months
Based on the pattern DOSE has already set and the broader trajectory of specialized AI benchmarking, a few developments look likely over the next 12 to 18 months, though none of these are confirmed and should be read as informed forecasts rather than announced plans.
- General-purpose TTS vendors will likely publish updated pronunciation dictionaries or fine-tuned medical-vocabulary add-ons in response to competitive pressure from purpose-built systems like RxPronounce.
- Expect more narrow, domain-specific voice AI benchmarks to follow DOSE’s format, applying the same “established vs. newly approved” methodology to other fast-changing vocabularies, such as newly approved medical devices or emerging chemical compounds.
- Pharmacy chains and telehealth platforms piloting AI voice agents will likely add a pronunciation QA step to their vendor evaluation checklists, citing DOSE as a reference point even if they do not run the full benchmark themselves.
- Hugging Face and other open dataset hubs will likely see community-contributed extensions to the DOSE dataset, given that Synthio Labs published it publicly with audio rather than gating it behind a paywall.
- Expect continued competitive positioning between general-purpose voice AI leaders and vertical-specific challengers, echoing the same specialist-versus-generalist dynamic already playing out in AI models built for life sciences and pharma.
Why This Story Is Bigger Than One Press Release
It would be easy to read DOSE as a vendor promoting its own product by showing competitors in a bad light, and that reading is not entirely wrong. But the underlying dataset, published with audio and open for outside verification, is the part that outlasts the marketing framing. Anyone skeptical of Synthio’s 87.0% figure for RxPronounce can pull the Hugging Face dataset and check the claim independently, which is a meaningfully higher bar than most vendor benchmarks clear. That openness is likely why outlets across finance, tech, and healthcare media picked up the story on the same day, rather than treating it as a routine product announcement.
The deeper takeaway is less about any single vendor’s score and more about a category of risk the AI industry has mostly ignored while chasing naturalness and latency. Voice AI got remarkably good at sounding human over the past two years. DOSE is a reminder that sounding human and being safe to deploy in a pharmacy are not the same achievement, and that the industry is only beginning to build the tools to tell the two apart.
Frequently Asked Questions
What is the DOSE Benchmark?
DOSE, short for Drug-name Oral Synthesis Evaluation, is a public benchmark from Synthio Labs that tests how accurately nine commercial text-to-speech systems pronounce 274 medicine names embedded in realistic clinical sentences.
Which voice AI models were tested in the DOSE Benchmark?
The announcement names ElevenLabs’ eleven_v3, Google’s Gemini TTS, Microsoft Azure TTS, OpenAI’s TTS models, Deepgram, and Synthio Labs’ own RxPronounce, tested alongside a total of nine commercial systems.
How often do voice AI models mispronounce drug names?
According to Synthio Labs, general-purpose systems mispronounce up to one in three newly approved drug names, with pass rates ranging from about 61.6% to 67.1% on that category for the two systems with disclosed figures, ElevenLabs’ eleven_v3 and Google’s Gemini TTS.
What is a real example of a drug name mispronunciation from the benchmark?
The DOSE writeup cites Xofluza, an antiviral, as a case where one tested system spelled the name letter by letter, X-O-F-L-U-Z-A, instead of pronouncing it as a word.
Did Synthio Labs’ own model perform better than the competition?
Synthio Labs reports its RxPronounce model scored 91.2% overall and 87.0% on newly approved drugs, which the company says is 10.9 percentage points ahead of the next-best system on new drugs specifically, though it is worth noting Synthio built both the benchmark and the winning model.
Why do newly approved drugs cause more pronunciation errors than established ones?
Newly approved drug names have had almost no time to accumulate transcribed audio, captions, or training data that voice AI models typically learn pronunciation patterns from, unlike established drugs that have circulated in clinical and consumer contexts for years.
Is the DOSE Benchmark dataset publicly available?
Yes. Synthio Labs published the full dataset, including audio clips for the 274 tested medicines, publicly on Hugging Face, allowing outside researchers to reproduce or challenge the results.
What are LASA drugs and why do they matter for voice AI?
LASA stands for look-alike, sound-alike drugs, medications with similar names that pharmacy safety programs like ISMP track separately because confusing one for another can cause patient harm; voice AI systems that already struggle with single drug names face a harder challenge distinguishing LASA pairs.
