Cloud speech-to-text bills by the minute and ships your recordings to someone else’s servers. You can skip both. The whisper cpp project, a C/C++ port of OpenAI’s Whisper model, shipped version 1.9.5 in October 2026, a maintenance release that pulls in ggml v0.26.0. That is the version this tutorial pins, so every command below runs against the same code.
You will build the tool from source, pick a model that fits your hardware, and transcribe audio locally from the command line. Then you will wrap it in an HTTP server and a Python batch script that watches a folder. If you already compiled models with the shared ggml library through our llama.cpp tutorial, the build step will feel familiar. Plan on about 45 minutes, most of it spent waiting on a compiler and a model download.
One clarification before the commands. Version 1.9.5 is a maintenance release that updates the runtime, so do not read it as introducing a new model. Newest here means the newest engine. You still choose among the existing Whisper sizes, and community apps updated in October 2026 still list large-v3 among their supported local models. The 13 steps are marked in the headings so you can jump back to any of them.
Don't miss new tech stories on Google
Add Tech Insider once in the Google app and our stories appear in your news suggestions.
Why Run Whisper Locally in 2026
Three reasons push people toward local speech to text: privacy, cost at volume, and control. Privacy comes first. Interview recordings, clinical notes, board calls and customer support audio often carry contractual limits on where they can travel. When the model runs on your own machine, the audio never leaves it, and there is no vendor retention policy to read.
Cost matters less than the marketing suggests, so here is the honest math. A third-party pricing summary from Cartesia reports OpenAI’s hosted whisper-1 at $0.006 per minute and a newer transcription model at $0.0045 per minute. Treat those as reported figures, not a quote from OpenAI. At the higher rate, 100 hours of audio is 6,000 minutes, which comes to $36. That bill is not scary. Local transcription starts paying off when you process thousands of hours, re-run the same archive after every tweak, or cannot legally upload the files at all.
Control is the third reason, and it is the one developers undervalue. A local binary gives you fixed versions, deterministic builds and the option to patch behavior. You can pin the engine to v1.9.5, keep the model file in your artifact store, and reproduce a transcript from last month byte for byte. Hosted APIs change underneath you. Their model names, prices and limits move on the vendor’s schedule, and your pipeline inherits every change.
Local also has real costs. You maintain the build, you babysit GPU drivers, and you accept slower throughput on a laptop than on a rented accelerator. If you only need a few hours of transcripts and no privacy guarantees, our speech-to-text API comparison will get you answers faster. Latency is the other axis. Always-on products such as the glasses in Meta’s 80ms speech AI story chase near-instant streaming, while file-based jobs like the ones in this guide trade latency for accuracy and price.
Rule of thumb: pick local speech to text when audio is sensitive, when volume is high, or when you want a reproducible pipeline. Pick cloud when you need a one-off transcript today and have no compliance constraints. Everything that follows assumes you picked local.
Prerequisites and Versions
You need a machine with a C++ compiler, a few gigabytes of free disk space and a terminal. A GPU helps but is optional. The table lists what the walkthrough uses and why each piece is there.
| Component | Version or requirement | Why you need it |
|---|---|---|
| whisper.cpp | v1.9.5 (tagged October 6, 2026) | The inference engine and the whisper-cli and whisper-server binaries |
| ggml | v0.26.0 (bundled with v1.9.5) | Tensor library that runs the model on CPU or GPU |
| Whisper model | base.en to start, large-v3 for best accuracy | The weights, stored in ggml format |
| CMake | A current 3.x release | Configures and drives the build |
| C/C++ compiler | GCC, Clang or MSVC with C++17 support | Compiles the engine |
| ffmpeg and ffprobe | Any recent build | Converts mp3, m4a and video files to 16 kHz WAV |
| Python | 3.10 or newer | Runs the batch project in Steps 11 to 13 |
| Operating system | Linux, macOS or Windows 11 (WSL2 works well) | All three are supported by the project |
| Optional GPU | NVIDIA with CUDA toolkit, a Vulkan-capable GPU, or Apple silicon | Speeds up the larger models |
| Free disk space | At least 5 GB | Source tree, build output and one or two models |
A third-party release report on ggml v0.26.0 says the new engine carries GPU optimizations for CUDA, Vulkan, Metal and WebGPU. The official release confirms the version bump but not each of those details, so test your own backend instead of assuming a speedup. ggml has also moved on since. The same ecosystem reportedly reached v0.27.0 on October 15, 2026, but whisper.cpp v1.9.5 is the tag we pin, and pinning is what makes this guide reproducible.
You do not need an OpenAI account, an API key or an internet connection after the model download. The weights run on your hardware. That also means nobody throttles you, but it means your RAM sets the ceiling. As a rough guide, the small models run on almost anything, the medium model wants a recent laptop with 16 GB of memory, and large-v3 is happiest with a discrete GPU or an Apple silicon machine with plenty of unified memory.
Check your tools before you start. These three commands should each print a version without errors.
cmake --version
ffmpeg -version | head -n 1
python3 --versionChoosing a Whisper Model Before You Download
Model choice changes your results more than any flag. Whisper ships in five sizes, and whisper cpp reads them in ggml format. Smaller models load fast and use little memory but make more mistakes on accents, jargon and noisy rooms. Larger ones catch more but cost more time and memory. The table below compares them qualitatively, because real speed depends on your CPU, GPU and audio.
| Model name | Languages | Best for | Trade-off |
|---|---|---|---|
| tiny and tiny.en | Multilingual and English-only | Smoke tests, tiny devices, live captions | Lowest accuracy, struggles with accents |
| base and base.en | Multilingual and English-only | Verifying your pipeline, clear single-speaker audio | Misses names and technical terms |
| small and small.en | Multilingual and English-only | Everyday meetings and podcasts on a laptop CPU | Slower than base, still weak on heavy noise |
| medium and medium.en | Multilingual and English-only | Interviews with mixed accents | Needs real memory, slow on CPU alone |
| large-v3 | Multilingual only | Best accuracy, other languages, noisy audio | Heaviest, wants a GPU or Apple silicon |
| Quantized variants (q5 or q8 in the name) | Same as the base model | Cutting memory and download size | Small accuracy loss that you should test |
The .en suffix matters. English-only models usually beat their multilingual twins on English audio at the same size, so use them when you know the language. Drop the suffix when you need Spanish, German, Hindi or automatic language detection.
My recommendation: download base.en first. It finishes in seconds and proves your build works. Then move to small.en or large-v3 depending on your hardware and how much you care about accuracy. Quantization comes later in Step 9, once you know the model you want to shrink. Resist the urge to download everything up front. Each model is a single file, and fetching the wrong one costs you only a few minutes, but a pile of unused models costs disk space you will want back.
Build whisper.cpp From Source (Steps 1 to 4)
Building takes a few minutes on a modern laptop. We pin the exact release tag so a future commit cannot change behavior under you. If you compiled the engine behind our Mellum 2.1 local setup, you already know most of the toolchain, and the CUDA or Vulkan flags carry over.
Step 1: Install the build toolchain
Install a compiler, CMake, git and ffmpeg with your package manager. On Windows, install Visual Studio Build Tools with the C++ workload plus CMake and ffmpeg, or open a WSL2 Ubuntu shell and follow the Linux commands.
# Debian or Ubuntu
sudo apt update
sudo apt install -y build-essential cmake git ffmpeg
# macOS (Homebrew)
xcode-select --install
brew install cmake ffmpeg gitStep 2: Clone the repository and pin v1.9.5
The project lives under the ggml-org organization on GitHub. Clone it and check out the v1.9.5 tag. A write-up of the release is on Freedom.Tech if you want a second description of what shipped. Detached HEAD warnings are normal here because a tag is not a branch.
git clone https://github.com/ggml-org/whisper.cpp.git
cd whisper.cpp
git checkout v1.9.5
git describe --tagsThe last command should print v1.9.5. If it prints something else, the checkout failed, so stop and fix that before you build.
Step 3: Configure and compile
A plain CPU build works everywhere. Add one flag to enable a GPU backend. On Apple silicon, the Metal backend turns on by default, so the CPU commands already use your GPU. Pass the Release configuration explicitly so you never benchmark a slow debug build by accident.
# CPU build (also uses Metal automatically on Apple silicon)
cmake -B build
cmake --build build -j --config Release
# NVIDIA GPU build (needs the CUDA toolkit installed)
cmake -B build -DGGML_CUDA=1
cmake --build build -j --config Release
# Vulkan build (AMD, Intel and many NVIDIA cards)
cmake -B build -DGGML_VULKAN=1
cmake --build build -j --config ReleaseRun only one of the three blocks. If you switch backends later, delete the build folder first with rm -rf build, because CMake caches the old flags. When the build ends, the binaries sit in build/bin. On Windows with MSVC they land in build\bin\Release instead.
Step 4: Download a Whisper model
The repository includes a script that fetches ggml-format models into the models folder. Start with base.en. Swap the name later for small.en, medium.en or large-v3.
sh ./models/download-ggml-model.sh base.en
ls -lh models/*.binYou should see a file named ggml-base.en.bin. If the download stalls or the file looks tiny, delete it and run the script again. A half-written model file is the number one cause of failed first runs, and the error it triggers looks unrelated to the download.
Run Your First Transcription (Steps 5 to 7)
With the build done, a transcription is one command. The repository ships a short sample clip, so you can test the engine before touching your own recordings. That separates build problems from audio problems, which saves real debugging time later.
Step 5: Transcribe the bundled sample
./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wavThe tool prints model and system details to stderr, then the transcript with timestamps. Your log lines will differ by machine, but the transcript should look like this:
whisper_init_from_file_with_params_no_state: loading model from 'models/ggml-base.en.bin'
...
system_info: n_threads = 4 / 8 | ...
main: processing 'samples/jfk.wav' (176000 samples, 11.0 sec), 4 threads, 1 processors, lang = en, task = transcribe, timestamps = 1 ...
[00:00:00.000 --> 00:00:11.000] And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
whisper_print_timings: total time = ... msRead the system_info line. It lists which backends the binary compiled in. If you built with CUDA or Vulkan and the line does not mention it, the GPU flag did not take effect, and you are running on the CPU. Rebuild from a clean folder. The thread count shown there also matters in Step 8.
Step 6: Convert your own audio to 16 kHz WAV
The command-line tool expects 16 kHz, mono, 16-bit PCM WAV. Phones and recorders produce mp3, m4a or video files, so convert them with ffmpeg first. This single command handles almost every input, including the audio track of an mp4.
ffmpeg -i interview.m4a -ar 16000 -ac 1 -c:a pcm_s16le interview.wavThe flags read in plain terms: -ar 16000 sets the sample rate, -ac 1 folds stereo into mono, and -c:a pcm_s16le writes uncompressed 16-bit audio. The file gets bigger than the original. That is fine, since you can delete the WAV after transcribing. The batch script in Step 12 does exactly that.
Step 7: Write transcripts to text, subtitle and JSON files
Printing to the terminal does not scale. Ask whisper-cli to write files next to a base name of your choice. One run can produce plain text, SubRip subtitles, WebVTT captions and JSON at once.
mkdir -p transcripts
./build/bin/whisper-cli \
-m models/ggml-base.en.bin \
-f interview.wav \
-otxt -osrt -ovtt -oj \
-of transcripts/interviewThe command writes transcripts/interview.txt, .srt, .vtt and .json. Use the text file for search and summaries, the SRT or VTT file for video players, and the JSON file when you need segment timestamps in code. If you only want clean text with no timing noise in the terminal, add -nt to the command.
Open the .srt file and check three things: the first timestamp lines up with the first spoken word, proper nouns look right, and nothing repeats in a loop. If all three pass, your pipeline works end to end. The next section improves accuracy and speed.
Tune Accuracy and Speed (Steps 8 and 9)
The default settings of whisper cpp work, but they leave quality and speed on the table. Two adjustments pay off most: tell the engine what to expect, and shrink the model if memory is tight. Make one change at a time and re-run the same test file, so you can see which tweak actually helped.
Step 8: Set language, threads, a vocabulary prompt and voice activity detection
Language comes first. With a multilingual model, whisper-cli guesses the language from the first seconds of audio, and a wrong guess ruins the whole file. Pass -l with a two-letter code when you know it. Use -l auto only for mixed archives.
Threads come second. The -t flag sets how many CPU threads the engine uses. Match it to your physical core count, not the logical count. Hyper-threading rarely helps this workload, and oversubscribing the CPU can make a run slower. The system_info line from Step 5 shows what the tool picked on its own.
The –prompt flag seeds the decoder with text. Feed it the names, product terms and acronyms that appear in your recording, and the model spells them right far more often. Keep it short, a sentence or a comma-separated list. A long prompt can steer the output in odd directions.
./build/bin/whisper-cli \
-m models/ggml-small.en.bin \
-f interview.wav \
-l en \
-t 8 \
--prompt "Acme Corp, Kubernetes, Grafana, Dr. Okafor" \
-otxt -osrt \
-of transcripts/interview-tunedVoice activity detection, or VAD, skips silence and noise before the model sees it. It cuts wasted compute and reduces the looping and made-up text that Whisper produces on quiet stretches. The project’s models folder includes a download script for a VAD model, and the README for your pinned tag lists the exact model name to pass. Copy the name from there, because it changes between releases.
ls models/ | grep -i vad
./build/bin/whisper-cli \
-m models/ggml-small.en.bin \
-f interview.wav \
--vad --vad-model models/VAD_MODEL_FILE \
-otxt -of transcripts/interview-vadReplace VAD_MODEL_FILE with the file that the download script created. Compare the two transcripts side by side. On recordings with long pauses, the VAD run is usually shorter and cleaner. On dense, continuous speech the two should match closely.
Step 9: Quantize the model to save memory
Quantization stores the weights at lower precision. The file shrinks and the model needs less memory, which lets larger models fit on modest machines. The cost is a small accuracy loss that you should measure on your own audio instead of trusting a rule of thumb. The repository builds a quantize tool alongside the other binaries. In most releases it is named quantize, so check build/bin if yours differs.
ls build/bin | grep -i quant
./build/bin/quantize \
models/ggml-small.en.bin \
models/ggml-small.en-q5_0.bin \
q5_0
ls -lh models/ggml-small.en*.binThe second listing shows both files, and the q5_0 one should be clearly smaller. Run Step 7 again with -m pointing at the quantized file and diff the two text outputs. If the differences are cosmetic, keep the smaller file. If names and numbers start going wrong, step back to the full-precision model or try a gentler quantization type.
Serve Whisper Over HTTP (Step 10)
A command-line tool is fine for one-off jobs, but other programs need an API to get local speech to text on demand. The build also produces whisper-server, a small HTTP service that loads the model once and keeps it in memory. Loading a large model on every request wastes seconds, so a long-running server is the right shape for apps, bots and internal tools.
Step 10: Start whisper-server and call it
Start the server bound to localhost. The –convert flag tells it to call ffmpeg on uploads, so clients can send mp3 or m4a files without converting first. Keep the bind address at 127.0.0.1 unless you put authentication in front of it, because the server has no login of its own.
./build/bin/whisper-server \
-m models/ggml-base.en.bin \
--host 127.0.0.1 \
--port 8080 \
--convertIn a second terminal, post a file to the inference endpoint with curl. The form field must be named file.
curl http://127.0.0.1:8080/inference \
-F file=@samples/jfk.wav \
-F temperature=0.0 \
-F response_format=json{"text":" And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country."}From Python, the same call takes four lines with the requests library. Install it with pip install requests. This client works from any language that can send a multipart form, so a Node, Go or shell client looks much the same.
import requests
with open("interview.m4a", "rb") as audio:
reply = requests.post(
"http://127.0.0.1:8080/inference",
files={"file": audio},
data={"response_format": "json", "temperature": "0.0"},
timeout=600,
)
reply.raise_for_status()
print(reply.json()["text"].strip())Treat the server as a single-user tool at first. Requests queue behind each other on one model instance, so two long uploads at once will take roughly twice as long. If you need real concurrency, run several server processes on different ports behind a reverse proxy, and size them to your memory. Check the server’s –help output on your build for the full flag list, since options evolve between releases.
Build the Batch Transcription Project (Steps 11 to 13)
Now put the pieces together into something you can leave running. The project watches an inbox folder, converts each file with ffmpeg, calls whisper-cli, writes four output formats, moves the source file out of the way, and logs a manifest row with timing. It uses only the Python standard library, so there is nothing extra to install.
Step 11: Create the project layout and the script
Make a project folder with four working directories. Then place your whisper.cpp checkout inside it or symlink it, because the script’s defaults look for whisper.cpp/build/bin/whisper-cli relative to the project root.
mkdir -p ~/transcriber/{inbox,transcripts,done,failed}
cd ~/transcriber
ln -s /path/to/your/whisper.cpp whisper.cpp
touch transcribe.pyPaste the full script below into transcribe.py. It handles one pass over the inbox and exits, which makes it easy to schedule. Failures move to the failed folder with the error tail printed, so one bad file never blocks the rest of the batch.
#!/usr/bin/env python3
"""Batch transcriber built on whisper.cpp v1.9.5."""
import argparse
import csv
import shutil
import subprocess
import sys
import tempfile
import time
from pathlib import Path
AUDIO_EXT = {".mp3", ".m4a", ".wav", ".flac", ".ogg", ".opus",
".aac", ".mp4", ".mkv", ".webm", ".mov"}
SETTLE_SECONDS = 5
def run(cmd):
return subprocess.run(cmd, check=True, capture_output=True, text=True)
def duration_seconds(path):
out = run(["ffprobe", "-v", "error", "-show_entries", "format=duration",
"-of", "default=noprint_wrappers=1:nokey=1", str(path)])
return float(out.stdout.strip())
def to_wav(src, dst):
run(["ffmpeg", "-y", "-loglevel", "error", "-i", str(src),
"-ar", "16000", "-ac", "1", "-c:a", "pcm_s16le", str(dst)])
def transcribe(args, wav, out_base):
cmd = [args.bin, "-m", args.model, "-f", str(wav), "-l", args.lang,
"-t", str(args.threads), "-otxt", "-osrt", "-ovtt", "-oj",
"-of", str(out_base)]
if args.vad_model:
cmd += ["--vad", "--vad-model", args.vad_model]
if args.prompt:
cmd += ["--prompt", args.prompt]
run(cmd)
def process(args, src):
out_base = Path(args.outdir) / src.stem
started = time.time()
with tempfile.TemporaryDirectory() as tmp:
wav = Path(tmp) / (src.stem + ".wav")
to_wav(src, wav)
audio_s = duration_seconds(wav)
transcribe(args, wav, out_base)
return audio_s, time.time() - started
def append_row(manifest, row):
is_new = not manifest.exists()
with manifest.open("a", newline="") as handle:
writer = csv.writer(handle)
if is_new:
writer.writerow(["file", "status", "audio_s", "wall_s",
"speed_x", "finished_at"])
writer.writerow(row)
def main():
p = argparse.ArgumentParser(description="Transcribe an inbox folder with whisper.cpp")
p.add_argument("--inbox", default="inbox")
p.add_argument("--outdir", default="transcripts")
p.add_argument("--done", default="done")
p.add_argument("--failed", default="failed")
p.add_argument("--bin", default="whisper.cpp/build/bin/whisper-cli")
p.add_argument("--model", default="whisper.cpp/models/ggml-base.en.bin")
p.add_argument("--lang", default="en")
p.add_argument("--threads", type=int, default=4)
p.add_argument("--vad-model", default="")
p.add_argument("--prompt", default="")
args = p.parse_args()
for folder in (args.inbox, args.outdir, args.done, args.failed):
Path(folder).mkdir(parents=True, exist_ok=True)
manifest = Path(args.outdir) / "manifest.csv"
failures = 0
for src in sorted(Path(args.inbox).iterdir()):
if not src.is_file() or src.suffix.lower() not in AUDIO_EXT:
continue
if time.time() - src.stat().st_mtime < SETTLE_SECONDS:
print(f"skip (still copying): {src.name}")
continue
stamp = time.strftime("%Y-%m-%dT%H:%M:%S")
try:
audio_s, wall_s = process(args, src)
speed = audio_s / wall_s if wall_s else 0.0
append_row(manifest, [src.name, "ok", f"{audio_s:.1f}",
f"{wall_s:.1f}", f"{speed:.2f}", stamp])
shutil.move(str(src), str(Path(args.done) / src.name))
print(f"ok: {src.name} ({speed:.2f}x real time)")
except subprocess.CalledProcessError as exc:
failures += 1
tail = (exc.stderr or "").strip().splitlines()[-5:]
print(f"FAILED: {src.name}\n " + "\n ".join(tail), file=sys.stderr)
append_row(manifest, [src.name, "failed", "", "", "", stamp])
shutil.move(str(src), str(Path(args.failed) / src.name))
return 1 if failures else 0
if __name__ == "__main__":
sys.exit(main())Two design choices deserve a word. The settle check skips files modified in the last five seconds, so the script never grabs a recording that is still copying in. And the temporary WAV lives in a temp directory that Python deletes on exit, which keeps the big uncompressed files off your disk. Extend the script with your own post-processing hook if you like. A common next step is feeding each .txt file to a local model for summaries, and our Meta Muse Glimmer 30B setup guide shows one way to host a model for that on a 24 GB GPU.
Step 12: Run a test batch and read the manifest
Drop two or three short recordings into the inbox, wait five seconds, and run the script. Use the model and thread count that suit your machine.
cp ~/Downloads/standup.m4a ~/Downloads/call.mp3 inbox/
sleep 6
python3 transcribe.py --threads 8 --lang enYou should see one ok line per file. The values in the next two blocks are placeholders that show the format, not benchmarks. Your speeds depend on your hardware and model.
ok: call.mp3 (X.XXx real time)
ok: standup.m4a (X.XXx real time)file,status,audio_s,wall_s,speed_x,finished_at
call.mp3,ok,AAA.A,WWW.W,X.XX,2026-10-28T14:02:11
standup.m4a,ok,AAA.A,WWW.W,X.XX,2026-10-28T14:03:40The speed_x column divides audio length by processing time. A value above 1.00 means the run finished faster than real time. Use that column to choose between models. If the small model gives you a comfortable margin and the medium model falls below 1.00, you know what your hardware can sustain. Confirm the outputs landed in transcripts/ and the originals moved to done/.
Step 13: Schedule the batch with cron and a lock
Automation finishes the job. A cron entry runs the script every ten minutes. The flock wrapper stops a second run from starting while the first is still busy with a long file, which would otherwise double your CPU load and risk grabbing the same file twice.
crontab -e
# add this line, then save:
*/10 * * * * cd $HOME/transcriber && /usr/bin/flock -n /tmp/transcribe.lock /usr/bin/python3 transcribe.py --threads 8 >> run.log 2>&1On macOS, launchd is the native scheduler and works just as well, and on Windows, Task Scheduler can run the same command inside WSL2. Check run.log after the first scheduled pass. If you see skip lines for files that finished copying long ago, the clock on your storage may be off, so look at the file’s modification time. That completes the 13 steps. The sections that follow cover what good output looks like, the mistakes that waste the most time, and a fix list for when things break.
Expected Output and How to Check Transcript Quality
A finished run leaves four files per recording. The text file holds the plain transcript. The SRT and VTT files carry timestamps for video players. The JSON file stores segments with start and end offsets, which is what you want when you build search or clip-cutting tools on top. Here is the shape of an SRT file, using made-up content to show the format:
1
00:00:00,000 --> 00:00:04,200
Welcome back to the weekly review.
2
00:00:04,200 --> 00:00:09,800
First item is the migration plan for the billing service.Looking at the output is not a quality test. Measure it. Pick one recording of two or three minutes, type a reference transcript by hand, and compute the word error rate, or WER. WER counts substitutions, deletions and insertions against the number of reference words. Lower is better, and the number lets you compare models, prompts and VAD settings on your own audio instead of arguing from general impressions.
import re
import sys
def words(path):
text = open(path, encoding="utf-8").read().lower()
return re.findall(r"[a-z0-9']+", text)
def wer(reference, hypothesis):
rows, cols = len(reference) + 1, len(hypothesis) + 1
dist = [[0] * cols for _ in range(rows)]
for i in range(rows):
dist[i][0] = i
for j in range(cols):
dist[0][j] = j
for i in range(1, rows):
for j in range(1, cols):
cost = 0 if reference[i - 1] == hypothesis[j - 1] else 1
dist[i][j] = min(dist[i - 1][j] + 1,
dist[i][j - 1] + 1,
dist[i - 1][j - 1] + cost)
return dist[-1][-1] / max(len(reference), 1)
if __name__ == "__main__":
ref, hyp = words(sys.argv[1]), words(sys.argv[2])
print(f"WER: {wer(ref, hyp):.1%} over {len(ref)} reference words")Run it as python3 wer.py reference.txt transcripts/interview.txt. Track the score in a spreadsheet as you change one variable at a time. Beyond the number, scan for the failure types that WER hides: dropped sentences at the start of a file, names spelled three different ways, and phrases repeated dozens of times. Those are the problems that matter to a reader, and the next two sections show how to prevent and fix them.
Common Pitfalls That Waste the Most Time
Most failed first attempts trace back to the same handful of mistakes. Avoid these and you skip hours of confusion.
- Feeding compressed audio straight to whisper-cli. The command-line tool wants 16 kHz mono WAV. An mp3 or m4a either fails to load or produces garbage. Convert with ffmpeg as in Step 6, or run the server with –convert.
- Using an .en model on non-English speech. English-only models try to force every sound into English words. The transcript looks plausible and is completely wrong. Switch to a multilingual model and set -l explicitly.
- Building from the master branch and forgetting the version. A fresh clone can include unreleased work. Pin the tag, write it in your notes, and upgrade on purpose. That habit also makes bug reports useful.
- Leaving language on auto for single-language archives. Detection runs on the opening seconds. A file that starts with music or a greeting in another language gets mislabeled, and the whole transcript suffers.
- Ignoring silence. Long pauses trigger looping text and invented sentences. Turn on VAD, trim dead air with ffmpeg, or both.
- Setting threads higher than your physical cores. More threads feels faster but often is not. Test 4, 6 and 8 on the same file and keep the winner.
- Trusting the output blindly. Speech recognition makes errors that read fluently. For legal, medical or financial recordings, have a person review the text before anyone relies on it.
- Committing model files to git. They are large binary blobs that bloat the repository permanently. Add models/*.bin to your ignore file and fetch them in your setup script.
One more habit worth building: keep a small set of three test recordings (clean, noisy, accented) and run them after every upgrade or model swap. Regressions show up in minutes instead of weeks, and you never wonder whether a change helped.
Troubleshooting: 12 Problems and Their Fixes
When something breaks, find the symptom in the table, apply the fix, and re-run the sample from Step 5 before touching your own audio.
| Symptom | Likely cause | Fix |
|---|---|---|
| whisper-cli: command not found | Wrong path, or an old tutorial using the legacy binary name main | Call ./build/bin/whisper-cli from the repo root, or on Windows build\bin\Release\whisper-cli.exe |
| cmake: command not found | CMake is not installed or not on PATH | Install it with apt, brew or the Windows installer, then open a new terminal |
| CUDA build fails with nvcc not found | CUDA toolkit missing or its bin folder is not on PATH | Install the toolkit, add nvcc to PATH, delete the build folder and reconfigure |
| GPU build runs but speed is unchanged | The binary compiled without the backend | Read the system_info line, delete build, and rebuild with the right -DGGML flag |
| failed to open model or load error | Wrong path, or a truncated model download | Check the -m path, re-download the file, and compare its size to a fresh copy |
| Error reading the audio file | Input is not 16 kHz mono 16-bit WAV | Convert with the ffmpeg command from Step 6 |
| Same sentence repeats dozens of times | Decoder loop on silence or noise | Enable VAD, trim silence, set –max-context 0, and try a larger model. Upstream was reportedly still reviewing a decoder fix in late October 2026, so treat it as unreleased |
| Transcript is in the wrong language or translated | Auto detection guessed wrong, or translate mode is on | Pass -l with the language code and remove any translate flag |
| Out of memory or the process is killed | Model too large for available RAM or VRAM | Use a smaller model or a q5 quantized file, and close other heavy apps |
| Very slow on CPU | Too few threads, debug build, or a model that is too big | Confirm the Release build, set -t to physical cores, and move down a model size |
| Server returns 400 or an empty result | Form field not named file, or non-WAV upload without –convert | Use -F file=@name and start the server with –convert |
| Timestamps drift on very long files | Cumulative alignment error over hours of audio | Split the audio into 30-minute chunks and offset each chunk’s timestamps |
If none of those match, run the failing command again with the full stderr visible and read the first error line, not the last. The first failure usually names the real cause, and the lines after it are fallout. Searching that exact line in the project’s issue tracker is the next best step. Mention your tag (v1.9.5), operating system and backend whenever you ask for help, because maintainers cannot guess those.
Advanced Tips for Speed, Streaming and Long Files
Once the basics work, a few upgrades turn the setup into a dependable tool. None of them is required, but each solves a problem you will probably hit within a month.
Live captions with whisper-stream
The project includes a streaming example that transcribes microphone input in near real time. It needs the SDL2 library, so install your platform’s SDL2 development package and rebuild with the SDL2 option. Use a small model for this, since live captions punish slow inference. Expect rougher text than file-based runs, because the engine decides on short windows of audio.
cmake -B build -DWHISPER_SDL2=ON
cmake --build build -j --config Release
./build/bin/whisper-stream -m models/ggml-base.en.bin -t 8 --step 500 --length 5000Split long recordings into chunks
Multi-hour files stress memory and let timestamp errors pile up. Cut them into 30-minute pieces with ffmpeg, transcribe each piece, and add the chunk offset back when you merge subtitles. Chunking also gives you natural resume points, so a crash at minute 170 costs you one chunk instead of the whole job.
ffmpeg -i long-meeting.wav -f segment -segment_time 1800 -c copy chunk_%03d.wavOther upgrades are worth a quick list:
- Pick the backend deliberately. Benchmark CPU, Vulkan and CUDA builds on the same file. Integrated GPUs sometimes lose to a well-threaded CPU run, and the only way to know is to measure.
- Apple silicon. The Metal backend works out of the box. The repository also documents a Core ML path for the encoder, which needs an extra model conversion step, so read the README section for your tag before you try it.
- Translate to English. Multilingual models can translate foreign speech into English text with the translate flag. Use it for rough understanding, and keep the original-language transcript for anything official.
- Close the loop with synthesis. If you want spoken replies from your pipeline, our guide to a self-hosted text-to-speech API covers the other direction, so you can build a fully local voice assistant.
- Lock down the server. If you must expose whisper-server beyond localhost, put it behind a reverse proxy with authentication and TLS, and cap upload size.
- Watch upstream. Changes for decoder loops, short-audio padding and streaming buffers were under review in late October 2026. Read the release notes before moving off v1.9.5, then re-run your three test recordings.
For a team setup, store the pinned tag, the model file’s checksum and your flags in a short README inside the project. Anyone can then rebuild the exact environment, which is the entire point of going local.
Local vs Cloud Transcription: Cost, Privacy and Upkeep
Before you commit your archive to this pipeline and transcribe audio locally for good, weigh the trade once more. Local transcription wins on privacy, on predictable cost at scale, and on reproducibility. Cloud wins on setup time, on elastic capacity, and on zero maintenance. Neither is better in the abstract. The right answer depends on your audio, your volume and who has to approve where the files go.
- Privacy. Local keeps recordings on your disk. Cloud means a contract, a data processing agreement and trust in someone else’s retention rules.
- Cost. Reported hosted rates sit around half a cent per minute, so small jobs cost little. Local spend is electricity plus your time, which pays back only with volume or compliance pressure.
- Accuracy. Large-v3 locally and a hosted Whisper-class service should land in the same neighborhood on clean audio. Test both on your own files before you decide.
- Latency. A good GPU can transcribe far faster than real time. A small laptop CPU with a big model cannot. Your speed_x column from Step 12 tells you which camp you are in.
- Upkeep. Local means you update builds, watch drivers and handle failures. Plan an hour a quarter for that.
Speech models keep getting better at conversation as well as dictation. Research such as Block’s Dessa speech model points toward systems that listen and talk at once, and the local tooling will follow. For more hands-on guides in this area, browse our AI and machine learning coverage. The practical takeaway: build the simple whisper cpp pipeline now, measure it on your audio, and swap components as better open models arrive.
FAQ: whisper.cpp Local Transcription
Is whisper.cpp free to use?
Yes. The project is open source, and running it costs only your hardware and electricity. We found no official paid plan for whisper.cpp or its ggml library. Check the license files in the repository and on each model’s page if you plan commercial use.
What is the latest whisper.cpp version?
As of October 28, 2026, the newest release we could confirm is v1.9.5, tagged October 6, 2026, and it bundles ggml v0.26.0. Later commits exist on the main branch, but they are not releases. Check the repository’s releases page before you upgrade.
Which Whisper model should I start with?
Start with base.en to verify your build in seconds. For real work, move to small.en on a modest laptop or large-v3 if you have a capable GPU or an Apple silicon machine with ample memory. Use multilingual models for any language besides English.
Can I transcribe audio locally without a GPU?
Yes. The CPU build works on any modern machine. Small and base models run comfortably, while the largest models run slowly without acceleration. Quantized files and VAD both help on CPU-only systems.
Does whisper.cpp need an internet connection or an OpenAI account?
Only for the first download of the source code and model files. After that it runs offline. It never calls OpenAI’s hosted service, so it needs no API key or account.
Why does my transcript repeat the same sentence?
That is a decoder loop, and it usually starts in silence or noisy stretches. Turn on voice activity detection, trim dead air, disable context carry-over with –max-context 0, or step up to a larger model. Chunking long files also reduces the risk.
Can whisper.cpp tell speakers apart?
Not reliably out of the box. It has a basic option for stereo recordings where each speaker sits on a separate channel. For true speaker labels on mono audio, pair the transcript with a dedicated diarization tool and merge the results by timestamp.
How do I keep whisper-server safe?
Bind it to 127.0.0.1, and if other machines must reach it, place a reverse proxy with authentication and TLS in front. The server has no login of its own, and an open port invites strangers to burn your CPU.
