Coder Legion - Recent posts tagged voice-assistant

Building Dexibi: A Voice-First AI Operator for Windows

Wed, 28 Oct 2026 18:00:00 +0000

The Idea Behind Dexibi

Dexibi started with a frustration. I was watching a show on Amazon Prime. When the episode ended there was no Next Episode button. I thought, what if I had an AI that could just play the episode for me? That small idea grew into something. It became an AI that lets people control their computers using language instead of clicking and typing through every step.

Beyond Simple Voice Commands

Dexibi wants to help users do tasks using their voice. Imagine being in the kitchen cooking and asking your computer to read your emails summarize an article or draft a social media post. Developers could also use Dexibi as a desktop operator alongside tools like Claude or Cursor. The idea is to make computers respond to what you want not how you want it done.

Making this work reliably is hard. Understanding a request is only the first step. The AI must also figure out what you really mean and carry out the actions—safely.

The Microphone Reliability Challenge

One big challenge I'm focused on is microphone reliability. It's not about picking up your voice. Distance matters. A microphone might work well when someone is sitting in front of the computer but it can struggle when they move away. Background noise makes things even harder. A kitchen with a humming refrigerator or a living room with music playing can interfere.

Dexibi offers three listening modes: Relaxed, Balanced and Strict. These are meant to balance how far you can be from the computer and how accurately the AI understands you. Relaxed mode works at a distance. May miss some details. Strict mode is more precise. Needs you to be closer. Balanced offers a ground.

The next goal is to make the experience work well across places different microphones and different levels of noise. Voice-first apps have to work outside conditions to be useful in real life.

Safety Matters

An AI that controls your computer needs safety measures that go beyond those for a text- AI. A wrong command could open the window delete something or start a process you didn’t mean to start. That’s why safety is a part of Dexibi.

Now Dexibi uses command validation. It checks if a command makes sense before running it. For high-risk actions—like moving files or shutting down the computer—it asks for confirmation. It also limits access to parts of the system using restricted shell access to prevent unintended changes.

The goal is to make control easier without taking your control. You should feel safe knowing the AI is helping not taking over.

What's Next?

Dexibi is built on a belief: people should be able to say what they want their computer to do not have to do every step themselves. That’s the mission.

There’s still a lot to do. Microphone distance and background noise are issues.. These aren’t just problems to fix—they’re part of building a product that works in real life not just in a quiet room.

The future is, about making voice-driven computer interaction practical more reliable and safer. So users can focus on what matters to them—doing their work— of clicking and typing every step of the way.

After months of building in silence, I cried a little- a stranger made a YouTube video about our pro

Fri, 15 May 2026 11:40:07 +0000

A few months ago, I told my co-founder I wasn't sure if anyone would ever care about what we were building.

We started Dograh as an open-source voice AI platform—an alternative to the closed players like Vapi and Retell. We thought developers would want this. But for a long time, GitHub stars trickled in slowly. Discord stayed quiet. Some days I'd refresh the analytics dashboard, hoping to see something move, and nothing would.

Today, everything changed.

Our stars started climbing fast, and we couldn't figure out why. Then we looked at our homepage bot, which asks every new user where they heard about us. Almost all of them said YouTube. We searched and found a BetterStack tutorial posted an hour ago. They'd built something with Dograh, liked it enough to record a video, and put it out into the world. We had no idea it was coming. We've never spoken to them.

We just crossed 500 stars. I keep refreshing the signup graph because part of me still doesn't believe it.
If you're building something open source and the silence is getting to you, I just want to say: someone out there might already be using your project. They might be about to tell the world. Keep shipping.

We finally broke the 500ms latency barrier for AI phone calls.

Mon, 04 May 2026 08:35:00 +0000

Hey CoderLegion! One of the biggest complaints with AI calling is the lag.
We wanted to share a raw snippet of our open-source framework, Siphon, in action.
By using WebRTC under the hood instead of WebSockets, we've gotten the voice-to-voice response time to feel incredibly human.
Would love for you guys to test the repo and let us know how it feels on your local machines!

Siphon | Step-by-Step YT Tutorial
Siphon Website
GitHub code

Build for AI cold caller for outbound. Owners cant tell its a bot

Sat, 25 Apr 2026 17:43:23 +0000

I spent 3-4 weeks building an AI voice agent that cold-calls HVAC owners for me. Real outbound. Real objection handling. Real appointments booked. When we listen back through the recordings, the owners never clock it as AI — they just think its a human SDR with a slightly weird cadence.

This is the build, and the honest reason its sitting on the shelf for high-volume right now.

The stack
Make.com runs the whole thing — seven scenarios wired around a single Google Sheet called NeverMiss – Outbound Calls that acts as the CRM.

dialler — every 5 minutes, pulls one row where status=new, enabled=TRUE, attempts<3, and fires a call into Vapi
Logger — webhook from Vapi after every call. Writes transcript, recording, outcome back. Uses an OpenAI module to classify the transcript into structured outcomes
Watch Dog — every 10 minutes, catches rows where the call went out but the Logger webhook never fired (Vapi had a bad minute), flips them to logger-missed for retry
Reconciler — sweeps stuck rows, enforces attempt caps, handles edge cases
recording sync — pulls MP3s out of Vapi before the 30-day retention wipes them, stores in Drive
Booking Handler — when the agent closes a booking mid-call, writes the calendar event and fires a confirmation
CallBack Dialer — separate path for inbound callback requests (form-fill → call within 60 seconds)
One Vapi agent on the voice side. Four Twilio numbers rotating for caller ID reputation. ElevenLabs for the actual voice. The complexity isnt in any one piece — its in how tightly theyre wired so nothing quietly breaks.

The Sheet is the CRM, with row locks
The Sheet has columns you would expect: business_name, phone_e164, status, call_id, recording_url, email, last_called_at, attempts, notes.

And some you might not: lead_uid (UUID, stable key), lock_token + lock_expires_at, next_callable_at, logger_ran_at.

The lock columns are the interesting part. Before the dialler fires a call, it writes a lock token and an expiry (lock_expires_at = now + 10 min). Thats the contract — if any other scenario or retry sees a row with a live lock, it skips it. If the lock is stale (expired), the row is free for reclaim.

Stops the Watch Dog from re-queueing a row thats actively being called. Stops two dialler runs from racing the same lead. Stops weird double-call bugs.

This is the kind of thing you dont need until volume hits, and then you really need it.

How it sounds like a human
Most AI voice builds fail on two things: the voice itself, and the gap between the human finishing their sentence and the agent replying. Both telegraph "robot" immediately.

For voice, ElevenLabs inside Vapi, cloned from a real sample. Tested OpenAI, Deepgram, PlayHT — ElevenLabs still wins on natural breath patterns and micro-pauses.

For latency, Vapis tunable VAD and smart endpointing, tuned so the gap between you finishing and the agent replying sits right where a human would land. Too low, it interrupts. Too high, the silence tells on it.

The system prompt is about 1,100 words. The agents named Alex. Rules include:

Never read the prospect back to themselves
Use natural fillers
If asked "is this a robot" — be honest
If asked for pricing — give a range and offer to send email details
If told to stop calling — confirm and end gracefully
Specific objections each get short inline responses that flip to a question. Nothing read aloud — everything handled as conversation.

The agent can also call two tools mid-call:

book_appointment — triggers the Booking Handler scenario which writes to Calendar and emails the confirm
capture_email — writes to the Sheet, kicks off a follow-up sequence
When those tools fire, the agent pauses briefly, gets the response, reads the confirmation back. No human in the loop on a successful call.

The glue — Logger, Watch Dog, recording sync
This is the boring but critical part.

Logger
Every Vapi call ends with a webhook hitting a Make scenario. The payload has the call ID, transcript, recording URL, end reason, and a Vapi-generated summary. The Logger finds the matching Sheet row by lead_uid, then runs the transcript through OpenAI to classify outcome into my taxonomy:

booked
email_captured
voicemail
no_answer
do_not_call
no_interest
callback_requested
Writes everything back, fires downstream actions — calendar events, DNC flags, re-queue for voicemails.

Watch Dog
Vapi sometimes fails to fire the post-call webhook. Without a watchdog, those rows sit silently in dialing status forever while you think everything is fine.

Mine runs every 10 minutes, finds rows with a last_called_at older than 5 minutes but logger_ran_at still empty, and flips status to logger-missed so the dialler picks them up again. Also clears stale lock tokens.

Recording sync
Vapi deletes recordings after 30 days. I want them forever — to retrain the prompt, spot objection patterns, audit outcomes. So a Make scenario runs every 4 hours, pulls the MP3, uploads to a Drive folder named .mp3, writes the Drive URL back, flips a synced flag.

Owners cant tell its AI — actual numbers
The recording sync is how I proved this instead of just claiming it.

After 200+ connected calls, I tagged every recording: did the prospect ever ask "is this a bot?", and did they ever seem confused?

11 out of 200 asked if it was AI. Thats 5.5%.

And most of the 11 asked after Alex had already booked a callback or captured their email — curiosity, not suspicion.

The other 94.5% just had a normal conversation. Successful bookings averaged 3 min 40 sec — identical to a human SDR doing the same call, meaning prospects werent shortcutting it because they smelled a robot.

Id share clips but theyre real owners voices without opt-in. Trust the data.

Why Ive parked it for volume
Heres the uncomfortable part.

Most HVAC numbers on public directories arent direct owner lines. Theyre main business lines routing through:

An IVR ("press 1 for service, 2 for billing")
A call center
A front-desk receptionist whose actual job is keeping people like me away from the owner
Current voice AI is still bad at IVRs. Vapi, Retell, Bland — all of them. The agent hears "press 1 for service" and either sits silent until the menu times out, or tries to talk to a recording.

Human gatekeepers are a coin flip — warm ones might put you through, suspicious ones kill the call with one question the agent cant answer.

Until voice AI gets genuinely good at navigating phone trees — pressing digits intelligently, detecting robot-vs-human, adjusting mid-call — high-volume outbound to public numbers burns money for 10-15% owner connection rates.

So Ive paused it. Not dead. Paused. When the models catch up, its right back on.

Why it still works
If youve got direct owner mobile numbers, this system is a weapon.

MetricNumber
Connection rate (direct mobiles)40-55%
Book rate (on connected calls)8-14%
All-in cost per connected minute~$0.18
Thats cold-outbound economics that actually pencil.

Where direct mobiles come from ethically:

LinkedIn scraping through Apollo / Clay / Wiza
Network-sourced lists
Trade show attendees
Re-engagement on your own past leads
Also works brilliantly for inbound callback — form-fill, Alex calls within 60 seconds, qualifies, books. Running that in production right now. Book rates are silly.

What Im building instead

Inbound. AI receptionists catching the calls HVAC owners are already missing nights, weekends, summer peak.

Same agent pattern, flipped direction, better unit economics, 100% connection rate (theyre the ones calling you).

Thats what NeverMiss is now. If you run HVAC, plumbing, or roofing and want to hear what a modern AI receptionist sounds like when the call lands at 9:47pm on a Saturday — nevermisshq.com has the live demo.

And when voice AI finally cracks IVR? The dialler comes right off the shelf.

Happy to answer build questions in the comments.

Rayhan Mahmood runs NeverMiss AI automation for US home service businesses.

Build a 24/7 AI Receptionist in 15 Mins (Python + Google Calendar)

Sun, 01 Feb 2026 13:08:38 +0000

Voice AI demos are cool — but most fail at one critical thing: doing real work.

Today, I’m sharing how to build a production-ready AI Dental Clinic Receptionist that can check a real Google Calendar and book appointments over the phone.

We’ll use SIPHON, an open-source Python framework, to build this with just a few lines of logic.

What This Agent Can Do

This tutorial walks you through building a voice agent that:

Books new appointments using Google Calendar
Modifies existing appointments
Cancels appointments
Verifies caller identity for privacy
Runs 24/7 with no human intervention

⏱ Time to build: ~15 minutes

Architecture Flow

Prerequisites

Before you start, make sure you have:

System Requirements

Python 3.8+

Install SIPHON

pip install siphon-ai

Getting Started Guide

API Keys

OpenRouter API Key (LLM)
https://openrouter.ai/
Sarvam or Cartesia API Key (STT / TTS)
https://siphon.blackdwarf.in/docs/plugins/tts/overview

Services

LiveKit credentials (Cloud or self-hosted)
Google Calendar with Service Account
Calendar Setup Guide

Helpful Documentation

Quick Start

Step 1: Create Project Structure

mkdir dental-clinic-receptionist
cd dental-clinic-receptionist
touch .env prompt.py agent.py inbound.py outbound.py

Step 2: Configure Environment Variables

Edit .env:

OPENROUTER_API_KEY=your_key   # LLM Provider

SARVAM_API_KEY=your_key       # STT Provider
CARTESIA_API_KEY=your_key     # TTS Provider

LIVEKIT_URL=wss://your-livekit-url
LIVEKIT_API_KEY=your_key
LIVEKIT_API_SECRET=your_secret

GOOGLE_CALENDAR_ID=*Emails are not allowed*
GOOGLE_CALENDAR_CREDENTIALS_PATH=/path/to/credentials.json

CALL_RECORDING=false
SAVE_METADATA=true
SAVE_TRANSCRIPTION=true

METADATA_LOCATION=Call_Metadata
TRANSCRIPTION_LOCATION=Transcriptions

AWS_S3_ENDPOINT=http://localhost:9000
AWS_S3_ACCESS_KEY_ID=minioadmin
AWS_S3_SECRET_ACCESS_KEY=minioadmin
AWS_S3_BUCKET=siphon
AWS_S3_REGION=us-east-1
AWS_S3_FORCE_PATH_STYLE=true

Step 3: Define the System Prompt

Create prompt.py.

The system prompt controls behavior, privacy, and conversation flow.

system_instructions = """
You are a professional dental clinic receptionist.

CAPABILITIES
- Book new appointments
- Modify existing appointments
- Cancel appointments

PRIVACY RULES
- ALWAYS verify caller identity (name + phone)
- Only show appointments for the verified caller

CONVERSATION STYLE
- Be warm and professional
- Ask one question at a time
- Confirm details by reading them back
"""

Full Prompt Example

Step 4: Create the Agent

Create agent.py:

from siphon.agent import Agent
from siphon.plugins import openrouter, sarvam, cartesia
from prompt import system_instructions
from dotenv import load_dotenv

load_dotenv()

llm = openrouter.LLM()
tts = cartesia.TTS()
stt = sarvam.STT()

agent = Agent(
    agent_name="Dental-Clinic-Receptionist",
    llm=llm,
    tts=tts,
    stt=stt,
    system_instructions=system_instructions,
    google_calendar=True
)

if __name__ == "__main__":
    # agent.download_files()  # Run once on fresh machines
    agent.dev()               # Local development
    # agent.start()           # Production

Agent Overview

Step 5: Enable Inbound Calls

Create inbound.py:

from siphon.telephony.inbound import Dispatch
from dotenv import load_dotenv

load_dotenv()

dispatch = Dispatch(
    agent_name="Dental-Clinic-Receptionist",
    dispatch_name="inbound-Dental-Clinic-Receptionist",
    sip_number=""  # Provided by your SIP provider
)

result = dispatch.agent()
print(result)

Run:

python inbound.py

Inbound Calling Guide

Step 6: Outbound Calls (Optional)

Use outbound calls for reminders or follow-ups.

Create outbound.py:

from siphon.telephony import Call
from dotenv import load_dotenv

load_dotenv()

call = Call(
    agent_name="Dental-Clinic-Receptionist",
    sip_trunk_setup={
        "name": "Dental-Clinic-Receptionist",
        "sip_address": "",
        "sip_number": "",
        "sip_username": "",
        "sip_password": ""
    },
    number_to_call=""
)

result = call.start()
print(result)

Run:

python outbound.py

Outbound Calling Guide

Step 7: Start the Agent

python agent.py

Your AI receptionist is now live

Production Scaling

SIPHON supports horizontal scaling out of the box.

Just run multiple agent instances, and calls are automatically distributed.

Scaling Guide

What This Looks Like in Practice

Booking an Appointment

Caller: “I need a cleaning next Tuesday at 2 PM”
Agent verifies name + phone
Checks Google Calendar
Books and confirms

✏️ Modifying an Appointment

Caller requests a change
Agent verifies identity
Shows only their appointments
Reschedules and confirms

❌ Cancelling an Appointment

Agent verifies identity
Confirms appointment
Cancels and acknowledges

Privacy by Design

Identity verification required
No cross-patient data exposure

Resources

⭐ Found this useful?
Give SIPHON a star on GitHub → https://github.com/blackdwarftech/siphon

Build A Real-Time Voice Assistant with Mistral and FastRTC

Tue, 15 Jul 2025 19:34:36 +0000

In this post, I will demonstrate how to build a real-time voice assistant using Mistral AI and FastRTC.

Mistral AI is one of the leading LLM providers, and they have made their LLM API easily accessible to developers.

FastRTC, on the other hand, is a real-time communication library for Python that enables you to quickly turn any Python function into real-time audio and video stream over WebRTC or WebSockets.

Building A Real-time Voice Assistant

First, let's install the required libraries by running the code below in your terminal

pip install mistalai fastrtc

Next, set an environment variable and import the libraries. Create a .env file in your project and save your Mistral API key there

MISTRAL_API_KEY = "<your-api-key>"

Import the libraries

from mistralai import Mistral
from fastrtc import (ReplyOnPause, Stream, get_stt_model, get_tts_model)
from dotenv import load_dotenv
import os

load_dotenv()

To get your Mistral API key, you will need to create an account on their website.

In the above, we have imported Mistral and the specific methods that we need from FastRTC namely ReplyOnPause(), Stream(), get_stt_model(), and get_tts_model().

ReplyOnPause(): This method takes a Python audio function. It monitors the audio, and when it detects a pause, it takes it as a cue to give a reply.

Stream(): This method streams the audio reply.

get_stt_model(): This is used to access the speech-to-text model that is used to convert audio to text.

get_tts_model(): This is used to access the text-to-speech model that is used to convert text back into audio.

Now, let's activate the Mistral client with our API key stored in the .env file

api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-large-latest"

client = Mistral(api_key=api_key)

Here we are using Mistral large model. However, you can try out other Mistral models too. Additionally, you can actually plug any LLM into FastRTC and receive real-time voice responses.

We will now build the audio function that will take a prompt and return a response

stt_model = get_stt_model()
tts_model = get_tts_model()


def echo(audio):
    prompt = stt_model.stt(audio)
    chat_response = client.chat.complete(
    model = model,
    messages = [
        {
            "role": "user",
            "content": f"{prompt}"
        },
      ]
    )

    for audio_chunk in tts_model.stream_tts_sync(chat_response.choices[0].message.content):
        yield audio_chunk

Above, we wrote a function called echo, and the function takes an audio input, then passes that to the speech-to-text method, which is converted to a user prompt and given to the LLM. The response from the LLM is then passed to a text-to-speech method and is streamed synchronously.

Finally, we will run the application

stream = Stream(ReplyOnPause(echo), modality="audio", mode="send-receive")
stream.ui.launch()

This will launch the UI below at the URL: http://127.0.0.1:7860/

Now, you can activate the microphone and say something to your assistant, who will give you a reply immediately.

Change Voice

If you do not like the default voice, you can change that by passing an instance of KokoroTTSOptions() to the text-to-speech method.

First import KokoroTTSOptions() from FastRTC by adding it to the import tuple

from fastrtc import (ReplyOnPause, Stream, get_stt_model, get_tts_model, KokoroTTSOptions)

Next, define the options

tts_model = get_tts_model(model="kokoro")

options = KokoroTTSOptions(
    voice="af_bella",
    speed=1.0,
    lang="en-us"
)

Then pass the options to the text-to-speech method in your audio function

for audio_chunk in tts_model.stream_tts_sync(chat_response.choices[0].message.content, options = options)
        yield audio_chunk

For more voice options, you can check out KokoroTTS documentation

Complete Project Code

Here is the complete code that we have used to create the real-time voice assistant

import os
from fastrtc import (ReplyOnPause, Stream, get_stt_model, get_tts_model, KokoroTTSOptions)
from dotenv import load_dotenv
from mistralai import Mistral

load_dotenv()

api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-large-latest"

client = Mistral(api_key=api_key)

options = KokoroTTSOptions(
    voice="af_bella",
    speed=1.0,
    lang="en-us"
)

stt_model = get_stt_model()
tts_model = get_tts_model(model="kokoro")

def echo(audio):
    prompt = stt_model.stt(audio)
    chat_response = client.chat.complete(
    model = model,
    messages = [

        {
            "role": "user",
            "content": f"{prompt}"
        },
      ]
    )

    for audio_chunk in tts_model.stream_tts_sync(chat_response.choices[0].message.content, options=options):
        yield audio_chunk

if__name__=="__main__":
       stream = Stream(ReplyOnPause(echo), modality="audio", mode="send-receive")
       stream.ui.launch()

Now, instead of typing a prompt, you can give voice commands to an LLM and have it speak its response, just like a natural human conversation.

I hope you found this post useful. If you did, please share it with others; it might benefit them, too. Thanks for reading!

You can follow me on the following platforms:

GitHub: Ifeanyi55

X: iCode2

LinkedIn: Ifeanyi Idiaye

You can also visit my website: https://ifeanyi-idiaye.netlify.app/

Build A Real-Time Voice Assistant with Mistral AI and FastRTC

Mon, 17 Mar 2025 12:58:30 +0000

In this post, I will show you how to build a real-time voice assistant with Mistral AI and FastRTC.

Mistral AI is one of the leading LLM providers out there, and they have made their LLM API easily accessible to developers.

FastRTC, on the other hand, is a real-time communication library for Python that enables you to quickly turn any Python function into real-time audio and video stream over WebRTC or WebSockets.

Building A Real-time Voice Assistant

First, let's install the required libraries by running the code below in your terminal

pip install mistalai fastrtc

Next, set an environment variable and import the libraries. Create a .env file in your project and save your Mistral API key there

MISTRAL_API_KEY = "<your-api-key>"

Import the libraries

from mistralai import Mistral
from fastrtc import (ReplyOnPause, Stream, get_stt_model, get_tts_model)
from dotenv import load_dotenv
import os

load_dotenv()

To get your Mistral API key, you will need to create an account on their website.

In the above, we have imported Mistral and the specific methods that we need from FastRTC namely ReplyOnPause(), Stream(), get_stt_model(), and get_tts_model()

ReplyOnPause(): This method takes a Python audio function. It monitors the audio, and when it detects a pause, it takes it as a cue to give a reply.

Stream(): This method streams the audio reply.

get_stt_model(): This is used to access the speech-to-text model that is used to convert audio to text.

get_tts_model(): This is used to access the text-to-speech model that is used to convert text back into audio.

Now, let's activate the Mistral client with our API key stored in the .env file

api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-large-latest"

client = Mistral(api_key=api_key)

Here we are using Mistral large model. However, you can try out other Mistral models too.

Also, in actuality, you can plug any LLM into FastRTC and get real-time voice responses.

We will now build the audio function that will take a prompt and return a response

stt_model = get_stt_model()
tts_model = get_tts_model()


def echo(audio):
    prompt = stt_model.stt(audio)
    chat_response = client.chat.complete(
    model = model,
    messages = [
        {
            "role": "user",
            "content": f"{prompt}"
        },
      ]
    )

    for audio_chunk in tts_model.stream_tts_sync(chat_response.choices[0].message.content):
        yield audio_chunk

Above, we wrote a function called echo, and the function takes an audio input, then passes that to the speech-to-text method, which is converted to a user prompt and given to the LLM. The response from the LLM is then passed to a text-to-speech method and is streamed synchronously.

Finally, we will run the application

if __name__ == "__main__":
         stream = Stream(ReplyOnPause(echo), modality="audio", mode="send-receive")
         stream.ui.launch()

This will launch the UI below at the URL: http://127.0.0.1:7860/

Change Voice

If you do not like the default voice, you can change that by passing an instance of KokoroTTSOptions() to the text-to-speech method.

First import KokoroTTSOptions() from FastRTC by adding it to the import tuple

from fastrtc import (ReplyOnPause, Stream, get_stt_model, get_tts_model, KokoroTTSOptions)

Next, define the options

tts_model = get_tts_model(model="kokoro")

options = KokoroTTSOptions(
    voice="af_bella",
    speed=1.0,
    lang="en-us"
)

Then pass the options to the text-to-speech method in your audio function

for audio_chunk in tts_model.stream_tts_sync(chat_response.choices[0].message.content, options = options)
        yield audio_chunk

For more voice options, you can check out KokoroTTS documentation.

Complete Project Code

Here is the complete code that we have used to create the real-time voice assistant

import os
from fastrtc import (ReplyOnPause, Stream, get_stt_model, get_tts_model, KokoroTTSOptions)
from dotenv import load_dotenv
from mistralai import Mistral

load_dotenv()

api_key = os.environ["MISTRAL_API_KEY"]
model = "mistral-large-latest"

client = Mistral(api_key=api_key)

options = KokoroTTSOptions(
    voice="af_bella",
    speed=1.0,
    lang="en-us"
)

stt_model = get_stt_model()
tts_model = get_tts_model(model="kokoro")

def echo(audio):
    prompt = stt_model.stt(audio)
    chat_response = client.chat.complete(
    model = model,
    messages = [

        {
            "role": "user",
            "content": f"{prompt}"
        },
      ]
    )

    for audio_chunk in tts_model.stream_tts_sync(chat_response.choices[0].message.content, options=options):
        yield audio_chunk


if __name__== "__main__":
         stream = Stream(ReplyOnPause(echo), modality="audio", mode="send-receive")
         stream.ui.launch()

Now, instead of typing a prompt, you can give voice commands to an LLM and have it speak its response, just like a natural human conversation.

I hope you found this post useful. If you did, please share it with others it might benefit too. Thanks for reading!