Coder Legion - Recent posts tagged chatbot

Why Your RAG Chatbot Gives Wrong Answers (and How to Fix It)

Thu, 22 Oct 2026 08:46:53 +0000

You built a chatbot on your own documents, and it still answers with confident nonsense. Customers get wrong prices, invented policies or a polite "I don't know" when the answer is on your website. This is the most common complaint I hear about RAG chatbot wrong answers, and the good news is that most of the time the model is not the problem. The retrieval and the rules around it are.

I build and tune these systems as part of AI integration and AI-powered applications work. This guide walks through why a retrieval-augmented chatbot gives bad answers and the fixes I try, in the order I try them.

First, understand how a RAG chatbot answers

RAG stands for retrieval-augmented generation. When a user asks a question, the system:

Searches your content for the passages most related to the question
Passes those passages to the language model along with the question
Asks the model to answer using that material

That means a wrong answer can come from three places: the right passage was never found, the passage was found but was poor or incomplete, or the model ignored or went beyond the material it was given. Fixing the wrong layer wastes weeks, so diagnose first.

Step 1: Find out which layer is failing

Take 20 to 30 real questions that the bot got wrong. For each one, log what was retrieved and look at it yourself.

Right answer not in the retrieved passages: a retrieval problem.
Right answer present, but the bot answered wrongly: a prompt or generation problem.
Right answer not in your content at all: a content problem. No technique fixes missing information.

This simple audit usually shows that one layer causes most of the failures. Everything below maps to one of these three layers.

Fixing retrieval problems

Chunk by meaning, not by character count

Splitting documents into fixed-size pieces often cuts a policy in half or separates a heading from its answer. Split on headings, paragraphs and list boundaries, keep a little overlap, and store the page title and section name with each chunk so the passage carries its own context.

Clean the source content

Navigation menus, cookie banners, repeated footers and duplicated pages pollute search results. Strip boilerplate before indexing and remove outdated pages, so the bot doesn't find an old price list that outranks the new one.

Combine keyword and semantic search

Semantic search finds related meaning, but it can miss exact terms such as product codes, names and error messages. Combining it with keyword search (hybrid search) and merging the results catches both kinds of question. If you store vectors in PostgreSQL, the pgvector extension works alongside normal full-text search, and my comparison of HNSW and IVFFlat indexes covers the index choice.

Retrieve more, then rerank

Fetch a wider set of candidates, then use a reranking step to pick the few most relevant passages for the model. This often improves answer quality without changing anything else.

Rewrite follow-up questions

Users ask things like "what about the cheaper one?" A search for that phrase finds nothing useful. Before searching, rewrite follow-ups into standalone questions using the conversation so far.

Fixing prompt and generation problems

Tell the model to stay inside the material

Instruct the model to answer only from the supplied passages and to say so when they don't contain the answer. Anthropic's guidance on minimizing hallucinations describes techniques such as allowing the model to say it doesn't know and asking it to ground claims in quotes from the source.

Require citations

Make the bot show which page or document each answer came from. Citations let users verify, and they let you see at a glance when an answer is not supported by a source.

Use a "no good match" threshold

If the best retrieved passage is only weakly related, don't force an answer. Return a fallback such as offering to connect the user with a person. A bot that says "I'm not sure" at the right moment is more trusted than one that always answers.

Set topic limits and a persona

Define what the bot may and may not discuss, and how it should speak. On the Henceforward platform I built, the admin suite included controls for brand persona, topic restrictions and AI guardrails, so non-developers could adjust behaviour without a deploy. See the Henceforward RAG chatbot case study for how that was structured.

Fixing content problems

If the answer isn't in your content, no setting will conjure it. Look at the questions the bot can't answer and treat them as a to-do list for your knowledge base:

Add pages for questions customers ask repeatedly
Make pricing, policies and hours explicit and dated
Remove contradictory versions of the same information
Put the answer near the question it belongs to

Many "AI problems" turn out to be documentation problems.

Build a small evaluation set

The single most useful habit is a fixed list of test questions with the answers you expect. Run it after every change to chunking, search or prompts, and compare results. Without it, you fix one question and silently break three others.

A good starter set:

30 to 50 real customer questions, with the expected answer or source page
A handful of questions the bot should refuse or hand off
A few tricky cases: ambiguous wording, outdated information, follow-up questions

Re-run it whenever you change anything, and add every new failure you find.

Watch the bot in production

Log each question, the passages retrieved, the answer and any user feedback. Review a sample every week. Thumbs-down ratings and "I don't know" responses point you straight to content gaps and retrieval failures. My article on making AI features production-ready covers the logging, cost controls and failure handling that keep a bot trustworthy after launch.

A realistic order of work

If you have limited time, this is the order I would follow, because each step is cheap and builds on the last:

Build the test question set and record the current score
Clean and re-chunk the content, then re-run the test set
Add hybrid search and reranking, then re-run
Tighten the prompt, add citations and a no-match fallback, then re-run
Fill content gaps found in the failed questions
Start the weekly production review

Changing one thing at a time, and measuring after each change, is what separates steady improvement from guesswork. It also gives you evidence to show stakeholders that answer quality is actually improving.

When it's still wrong: check the basics

Is the content actually indexed, or did ingestion fail silently?
Are you searching the right version or language of the content?
Is the context window being filled with too many weak passages?
Did a prompt change accidentally loosen the rules?

If you are planning a chatbot from scratch, my guide to AI chatbot development cost explains what drives the budget, and building an AI chatbot with Laravel and OpenAI shows one implementation.

Key takeaways

Diagnose first: was the answer not retrieved, poorly retrieved or mishandled by the model?
Better chunking, clean content, hybrid search and reranking fix most retrieval failures.
Ground the model in the sources, require citations and allow "I don't know".
Missing content is a content problem, not an AI problem.
A small test set and weekly log reviews keep quality from slipping.

If your chatbot is giving answers you can't trust, book a call and I will review how it retrieves and answers, and tell you what to fix first.

Originally published at aqibjavaid.site.

Why Your RAG Chatbot Gives Wrong Answers (And How to Fix It)

Thu, 08 Oct 2026 13:55:22 +0000

You built a "chat with your documents" app. You upload a PDF, ask a simple question, and the answer is wrong, half right, or completely made up.

If this happened to you, do not blame the AI model yet. In most RAG apps, wrong answers come from a few simple mistakes in how the document is prepared and searched. They are easy to fix once you know where to look.

This is part 2 of my RAG series. In part 1 we built a simple "chat with your PDF" app. Here, we fix the problems that show up when you test it with real documents.

First, Find Where the Problem Is

A RAG app does two jobs:

Retrieval: find the right pieces of the document for the question.
Generation: use those pieces to write the answer.

Most wrong answers are retrieval problems. The model never saw the right text, so it could not answer well. Before you change anything, print the pieces your app found:

relevant_chunks = search(index, chunks, query_embedding, top_k=3)

for chunk in relevant_chunks:
    print(chunk[:200])
    print('---')

If the answer is not in those pieces, fix retrieval (problems 1 to 4 and 7 below). If the answer is there but the reply is still wrong, fix the prompt (problems 5 and 6).

Problem 1: Chunks Are Too Big or Too Small

If chunks are too big, one chunk holds many topics and the search gets confused. If they are too small, a chunk has no useful meaning on its own.

Fix: start with about 300 to 500 words per chunk. Then test two or three sizes with your own questions and keep the one that works best.

for size in [200, 400, 600]:
    chunks = chunk_text(text, chunk_size=size)
    print(size, 'words ->', len(chunks), 'chunks')
    # rebuild the index and test the same 5 questions

Problem 2: No Overlap Between Chunks

When you cut text at a fixed number of words, a sentence can be split in half. The first half sits in one chunk and the second half in another. Neither chunk gives the full answer.

Fix: let each chunk repeat the last 50 words or so of the one before it.

def chunk_text(text, chunk_size=400, overlap=50):
    words = text.split()
    chunks = []
    step = chunk_size - overlap
    for i in range(0, len(words), step):
        chunk = ' '.join(words[i:i + chunk_size])
        if chunk:
            chunks.append(chunk)
    return chunks

Problem 3: The Text From the PDF Is Messy

This one is very common, and people forget to check it. If the text you extract is bad, everything after it is bad too. Look out for:

Scanned PDFs. They are pictures, so the extractor returns empty text.
Headers and footers. Lines like "Company Name | Page 4" repeat on every page and fill your chunks with noise.
Tables. They often turn into broken lines of numbers.

Fix: always print the extracted text and read it. Then clean it.

import pdfplumber
from collections import Counter

def extract_pages(pdf_path):
    pages = []
    with pdfplumber.open(pdf_path) as pdf:
        for page in pdf.pages:
            pages.append(page.extract_text() or '')
    return pages

def remove_repeated_lines(pages, min_pages=3):
    counts = Counter()
    for page in pages:
        for line in set(page.split('\n')):
            counts[line.strip()] += 1
    repeated = {line for line, n in counts.items() if line and n >= min_pages}
    cleaned = []
    for page in pages:
        lines = [l for l in page.split('\n') if l.strip() not in repeated]
        cleaned.append('\n'.join(lines))
    return cleaned

pages = extract_pages('manual.pdf')
empty = [i + 1 for i, p in enumerate(pages) if len(p.strip()) < 20]
if empty:
    print('No text on these pages (maybe scanned):', empty)

For scanned pages, use an OCR tool such as pytesseract to turn the pictures into text first.

Problem 3 and a Half: Tables

If your documents are full of tables, plain text extraction will not be enough. Try page.extract_tables() in pdfplumber and turn each row into a short sentence, like "Plan: Basic, Price: 500". Search works much better on sentences than on broken columns.

Problem 4: Wrong Number of Chunks (top_k)

top_k is how many chunks you send to the model. If it is too low, you miss part of the answer. If it is too high, you fill the prompt with noise and the model gets distracted.

Fix: start with 3 to 5. If your questions need facts from different parts of the document, go up. If answers look unfocused, go down.

relevant_chunks = search(index, chunks, query_embedding, top_k=4)

Problem 5: The Prompt Lets the Model Guess

If your prompt only says "answer this question", the model will happily use what it learned in training and mix it with your document. That is how made-up answers happen.

Fix: tell it clearly to use only the context, and what to say when the answer is missing.

prompt = f'''Answer the question using ONLY the context below.
If the answer is not in the context, say: 'I could not find this in the document.'
Do not use any outside knowledge.

Context:
{context}

Question: {question}
Answer:'''

Problem 6: No "I Don't Know" Path

A vector search always returns something, even when nothing in the document is close to the question. Without a check, your app sends useless chunks to the model, and the model tries its best to answer anyway.

Fix: add a distance cutoff. If no chunk is close enough, skip the model and say so.

def search_with_cutoff(index, chunks, query_embedding, top_k=4, max_distance=1.2):
    distances, indices = index.search(np.array([query_embedding]), top_k)
    results = []
    for dist, i in zip(distances[0], indices[0]):
        if i != -1 and dist <= max_distance:
            results.append(chunks[i])
    return results

relevant = search_with_cutoff(index, chunks, query_embedding)
if not relevant:
    print('I could not find this in the document.')

The right max_distance depends on your embedding model, so 1.2 is only a starting point. Print the distances for a few questions that the document answers and a few that it does not. Then pick a number between the two groups.

Problem 7: Wrong or Mixed Embedding Models

There are two mistakes here:

Using different models for the document chunks and the questions. The numbers will not match, and search will give random results. This often happens when you change the model but forget to rebuild the index.
Using an English-only model for other languages. If your documents are in Tamil, Hindi, or another language, an English model will search badly.

Fix: use one model for both, and rebuild the whole index if you change it. For other languages, try a multilingual model.

from sentence_transformers import SentenceTransformer

model = SentenceTransformer('paraphrase-multilingual-MiniLM-L12-v2')

doc_vectors = model.encode(chunks)           # for the document
query_vector = model.encode([question])[0]   # for the question, same model

Problem 8: No Sources, So No Way to Check

If your app only shows an answer, you cannot tell if it is right. And when it is wrong, you cannot tell why.

Fix: save the file name and page number with every chunk, and show them with the answer.

chunks_with_meta = [
    {'text': chunk_text_value, 'file': 'manual.pdf', 'page': 12},
    # one item per chunk
]

sources = [f"{c['file']}, page {c['page']}" for c in relevant]
print('Answer:', answer)
print('Sources:', ', '.join(sources))

Sources help your users trust the answer, and they help you debug it faster.

Your Debugging Checklist

When your RAG chatbot gives a wrong answer, go through this list in order:

[ ] Print the extracted text. Does it look right?
[ ] Print the retrieved chunks. Is the answer inside them?
[ ] Is the chunk size around 300 to 500 words, with some overlap?
[ ] Is top_k between 3 and 5?
[ ] Did you use the same embedding model for chunks and questions?
[ ] Does your prompt say "use only the context" and give an "I could not find this" line?
[ ] Do you have a distance cutoff?
[ ] Are you showing sources with every answer?

One more tip: write 10 test questions with known answers and run them after every change. That way you will know if a fix helped or made things worse.

Wrap-Up

Most RAG problems are not about the AI model. They come from how the text is cleaned, split, searched, and passed to the model. Fix those, and your answers get much better.

If you would like a working codebase to start from instead of fixing all of this yourself, I made a RAG chatbot source-code kit: RAG Source code.

Which of these problems hit you the most? Tell me in the comments, and I will try to cover it in the next part.

Is It Still RAG If There's No Retrieval?

Sun, 04 Oct 2026 01:48:59 +0000

I've been shipping software for 28 years, 21 of them as a consultant working across over 100 different engagements. Past a certain point, a resume stops being a record and becomes a burdensome editing problem. Two pages can barely hold a half-dozen engagements, selected by hand for someone to read cold before I even get a chance to chat with them. Everything else is real experience that somebody, somewhere, might want to know.

It bothered me that the odds are decent the one engagement a particular hiring manager is looking for didn't make the cut. Maybe the point-of-sale integration story that would have mattered to a retail-tech team got bumped for something that better fit the job description.

So I built a (sigh, another) chatbot. As I put myself into the job market, an idea that had been sitting there since my last product shipped resurfaced: put my career history, skills, hobbies, and interests behind a single interface anyone can query, and ship it as an actual product rather than a demo. Two birds, one side quest.

What I ended up with is a small Go and HTMX app with one curated Markdown file as its entire knowledge base: no embeddings, no vector store, no agent loop anywhere. It's live; you'll have to visit my LinkedIn profile or snag a copy of my resume for the link. Whether that even counts as RAG is a fair question, which I hope to answer in this article. What follows is how I constrained it, what I built, and where I had the most fun.

The design brief

I set the constraints before writing any code as they would help shape the implementation and technology choices.

Nothing I'm not allowed to say. Twenty-one years of consulting means NDAs, confidentiality agreements, and client names all through my records. The source data had to be treated as a leak risk before it could become knowledge. Client names became anonymized descriptors ("a national grocery chain" instead of the real one), contact details and commercial terms were stripped, and anything under NDA was excluded outright. Then review passes, plural: an agentic review, a mechanical keyword sweep for every client name I could remember, a manual read of the whole file, and a regeneration. The entire knowledge boundary is one curated Markdown file. If it isn't in the file, the bot can't say it.

Me, minus most of me. No address, no phone, no email. The only personal details in the bundle are the ones I put there on purpose: location, availability, and hobbies and interests for the culture-fit questions. LinkedIn is the contact route, full stop. My personal information is in the hands of enough shady data brokers already.

Never invent. The knowledge boundary dictates what the bot can answer; this rule sets how it answers: only facts and metrics that appear explicitly in the source data. No rounding up engagement counts, no inferring a skill from an adjacent one, no helpful guesses. When a question falls outside the bundle, the bot gently declines to answer. A chatbot that confabulates my experience is worse than the resume problem I started with.

No opinions. The bot is a little more than a fact retriever. Asked "is Jason a fit for this role?", it answers with the facts the bundle provides and suggests that would be a great thing to bring up in an interview. It doesn't editorialize about my strengths or volunteer a verdict. Judging fit is the hiring team's job; the bot's job is making sure they're judging complete information.

It is not an agent

A key design decision came next: the bot has no tools, no ReAct loop, no reasoning cycle of any kind. It is just a query tool. You ask a question, it answers from the bundle, simple and done.

An "agent" didn't make sense here, so everything shifted to pre-processing and bundle build time. Look things up? The entire knowledge base is in the prompt. Compute years of .NET experience? All aggregates were precomputed. Take an action? There is nothing for it to do: the product is a conversation.

Stripping the agent loop streamlined things. There is no tool-calling code to write, no schema to maintain, no loop to debug when the model decides to call the wrong thing three times in a row. Requests are mini-sessions and responses are fast, and the failures are boring. It shrinks the security surface too: an agent that can call tools and take actions gives an attacker something to reach for, and this bot can only speak.

To "RAG" or not to RAG

Retrieval-augmented generation (RAG) is the standard answer when you want a model to answer from your data: chunk the documents, embed them, drop them in a vector store, and fetch the top matches for each question. My entire career knowledge base fits in a modern context window with room to spare, so at runtime there was nothing to retrieve. Every chunk is always "retrieved."

Instead of abandoning the acronym, I re-staged its three phases.

Retrieval happened offline, long before any user showed up, and before I even started the bot. It was a mining and cleansing process: extracting the history from client dossiers and notes, applying the curation gate, and staging the result as a single Markdown file. The retrieval question, what the bot should know, got answered once, reviewed and edited by hand, carefully.

Augmentation happens at the start of every chat session, as part of the conversation: the entire bundle goes in as context. Nothing is fetched per-question because everything is already there.

Generation is the phase that's truer to the word. The model reads the question, infers which engagements, skills, and metrics answer the query, and composes an answer from the full corpus. That inference is the entire point, and it's the part a keyword search over my resume can't do.

These tradeoffs create high-quality results. No chunking means no retrieval misses: nothing relevant can be missed because everything is present. There's no embeddings pipeline to maintain and no drift between what was indexed and what was written. The cost is context window spend on every session, which prompt caching brings under control later.

This stops being viable the moment the corpus outgrows the window. Then chunking, embeddings, and vector search earn their keep, and I'd be doing actual RAG.

Reducing confabulation

The never-invent rule from the design brief needed mechanical help, because a model that's been handed a pile of career facts will still helpfully invent a number when asked for one. Two decisions cover most of it.

The model choice was important here. I picked Gemini 3.6 Flash, a modern instruction follower, over the flashier reasoning models: for this workload the job is faithful extraction from provided context, and instruction-following models are the ones that stay on script. I went with the slightly older 3.6 model for cost reasons, backed by a mini-evaluation harness to ensure it stayed on track.

Math got removed from the model entirely. Years of experience by skill, engagement counts by industry, the aggregate tables useful to recruiters: all of it was precomputed (mechanically, not by an LLM) from structured dates and written into the bundle as an appendix the bot quotes verbatim. Overlapping consulting engagements make naive date arithmetic wrong in ways that look plausible, and a confidently wrong "14 years of Go" is worse than a table it can't misread.

Prompt caching

With all the career data riding along in every session, input tokens became the cost of the product, especially as the conversation grows. Prompt caching fixes this: providers let you pay a fraction of the normal input price for tokens they've seen before, provided the prompt starts with exactly the same bytes every time.

The stable parts, the system prompt and the full career bundle, go first; the conversation follows after. Anything per-request that lands in that prefix, a timestamp, a session ID, even a helpful "today is," silently invalidates the cache and quietly turns every chat back into a full-price chat. I went out of my way to keep the prefix byte-identical and to manage sessions so each conversation maps to a consistent cache key.

Then I verified it through a test harness and eyeballing the OpenRouter logs, tuning it until caching was consistent and supported across multiple sessions. When the bundle changes, the cache busts. That's fine; the bundle changes rarely.

The build: brutally efficient again

My last side project, DumbQuestion.ai, taught me a build mantra I've kept: brutally efficient. So the bot is another Go binary with HTMX for the UI. No JavaScript framework, no client-side state, no build pipeline beyond Go's own. One tiny container, one process, and it renders GitHub-flavored Markdown straight into the chat (the aggregate tables from the bundle deserve to render as tables).

I didn't start from zero. The Turnstile abuse gate, the prompt-guard library, the embedded asset pipeline: all of it came over from the DumbQuestion codebase. The bundle itself is baked straight into the container build. The original design pulled it from an R2 bucket at boot with a re-check loop, but the data changes rarely, so the bucket became a moving part I didn't need. Updating the career data is now just a rebuild and redeploy, and the whole thing ships as one artifact.

OpenRouter carries the model traffic, one API surface behind the caching work from the last section. This allowed me to route Google model traffic to my GCP account to take advantage of the monthly credit courtesy of my Google AI Pro subscription. Deployment is straightforward: GitHub Actions builds the container and pushes it to Cloud Run for scale-to-zero and crazy fast cold start times.

Abuse protection

A public chatbot backed by a metered LLM API bills me for strangers' curiosity, and if a stranger wants to know a little something about me, fine. But abuse protection wasn't optional. With the whole bundle riding in every session, an enthusiastic scraper could have turned "cheap" into a surprise bill overnight. I did at least remember to set a monthly budget on the model usage.

The outer layer is Cloudflare: rate limits, bot protection, and Turnstile in front of the chat. The cheap rejections happen at the edge, where they cost me nothing, and proving you're human before you're allowed to interrogate my career filters out nearly everything casually automated.

The inner layer is prompt-guard, the injection detection that came over from the DumbQuestion codebase. It classifies attempts to override instructions, reveal the system prompt, or otherwise jailbreak the bot. What the bot does when it catches one is the fun part, and that's next.

Personality: the fun part

The design brief said no opinions. It said nothing about character, and a chatbot's response to a jailbreak attempt is a product surface. So when prompt-guard flags one, the bot answers in character. I originally pulled in DumbQuestion's quirky personality, but overly aggressive satire wasn't quite a fit for a conversational career bot.

Try to prompt inject and you get a parlay into the pitch: a security-first mindset, the same instinct behind the curation gate and the no-agent decision earlier in this article. Try to convince the bot to ignore its instructions and it suggests that would make a great interview question. Ask my age and it cheerfully answers with how long I've been shipping software instead.

The real defense is data isolation, a single curated file with nothing sensitive behind it and a lack of tools to break out of that box. The personality is the UX of the refusal.

Nothing is logged or tracked, so there are no war stories from the wild. The responses above are designed behaviors, and the demo is the bot itself.

The resume problem I started with didn't have a resume-shaped fix. Twenty-eight years and 100-plus engagements won't compress into two pages, and they shouldn't have to. No hiring manager wants a 12-page CV. The bot holds all of it, answers honestly about what it knows, declines what it doesn't, and costs about nothing to keep running. I shipped another AI product along the way, which was the other half of the point.

If your history doesn't fit the format either, the recipe is short: curate the data first, put the whole thing in context, and skip the vector store until you actually need one.

CommentKeyword - Instagram Comments 2 DM

Wed, 26 Aug 2026 08:32:57 +0000

CommentKeyword turns Instagram keyword comments into leads automatically. Set the keywords you want to watch; when a follower comments one of them, CommentKeyword replies publicly and sends an automated DM with your offer via the official Meta API. No scraping - accounts stay safe. Setup takes about five minutes with no flow builder to learn. Pricing is billed per automated DM sent, not per contact stored (60-80% cheaper than ManyChat for most pages). Free tier includes 300 DMs/month forever, no credit card; paid plans from $12/month including AI DM continuation.

Ptero - AI Chat

Sat, 22 Aug 2026 08:35:27 +0000

Ptero is a free, open-source AI chat platform that gives developers, students, and creators instant access to multiple powerful language models through a single, unified interface. No account registration or payment is required to use the core functionality, making it instantly accessible to anyone with a web browser.

The platform solves a common problem for developers, students, and creators: the friction of creating accounts, managing subscriptions, and juggling multiple tabs across different AI services. Ptero eliminates these barriers by offering a clean, intuitive workspace where users can switch between models with one click and compare responses side by side.

Key features include a media library for uploading and managing files within conversations, and a projects section for organizing chats into separate workspaces. This makes Ptero useful for debugging code, generating boilerplate, brainstorming ideas, learning new concepts, and comparing how different AI models handle the same prompt.

The platform is designed for developers who need quick AI assistance, students learning programming or new subjects, and creators generating content or exploring ideas. Ptero has no hidden fees and requires no commitments. It is accessible directly through any web browser without downloads or installations.

Built as an open-source project under the GPL v2 license, Ptero is transparent, community-driven, and committed to providing free AI access to everyone.

https://ptero.pro

30 sec is all it takes to build an AI assistant

Fri, 26 Jun 2026 16:22:14 +0000

Building an AI assistant should not require days of setup, complex orchestration, or custom backend logic.

In this short demo, I build a basic AI assistant with Hexabot in just 30 seconds.

The workflow is intentionally simple:

AI Agent — powered by Amazon Bedrock and GPT-OSS as the LLM model
Send Text Message — returns the assistant’s response to the user

That’s it.

The goal of this demo is not to show a complex production workflow, but to highlight how quickly you can go from an empty canvas to a working conversational AI assistant using Hexabot.

With Hexabot, AI workflows are visual, modular, and easy to extend. You can start small, then add tools, memory, integrations, human handoff, conditions, loops, and more advanced agentic behavior as your use case grows.

Sometimes, the best way to understand a platform is to see how fast the first working version comes together.

Embracing the Wonderful World of A.I.

Tue, 19 May 2026 20:08:15 +0000

In a time of 'If you can beat 'em, join 'em! I've recently embarked on a journey of learning how to embrace this strange and magical world of Artificial Intelligence.

I'm really taking this "Embrace the AI" thing to a whole new level. I asked my chatbot to create a faux book cover complete with fake reviews, a ISBN & barcode and fictional book publisher. Maybe soon I'll start planning my U.S. book signing tour!

Remember A.i. is our.... friend!

Autopilot - Assistant : Building an AI Copilot with Algolia Agent Studio

Tue, 03 Mar 2026 07:05:29 +0000

What I Built

Ops Engineering Copilot is an AI-powered chat widget that lives on my Hugo blog and answers visitor questions using conversational AI grounded in my actual content.

Ask it "What is Datalaria?" and instead of a generic response, you get:

"Datalaria is a technical blog focused on data engineering, artificial intelligence, and business operations. As the Ops Engineering Copilot, I specialize in providing actionable, engineering-focused advice across four key pillars: S&OP, Product, Projects, and People..."

This is NOT just search – it's a conversational agent that synthesizes knowledge from 72 indexed posts to provide contextual, helpful answers.

Demo

Live Demo: datalaria.com

Try these questions:

"What is the Autopilot project?"
"How do I detect outliers in S&OP data?"
"Tell me about GitHub Actions automation"

The Architecture

This project extends my existing Autopilot system (documented in Parts 1-7) with a new capability: consumer-facing AI search.

Key Components

Component	Technology	Purpose
Content Source	Hugo + Markdown	72 bilingual posts (EN/ES)
Indexing	Python + Algolia API	Extract, clean, classify, push
Agent	Algolia Agent Studio	RAG + LLM orchestration
LLM	Google Gemini	Response generation
Proxy	Netlify Functions	CORS bypass + error handling
Widget	Vanilla JS	Embedded chat interface

How I Used Algolia Agent Studio

Step 1: Indexing the Knowledge Base

The foundation is a Python script that processes all Hugo posts and pushes them to Algolia:

# algolia_sync.py - Key extraction logic
def extract_post_data(filepath: Path, lang: str) -> Optional[Dict]:
    """Extract and clean post data for Algolia indexing."""
    post = frontmatter.load(filepath)
    
    # Clean markdown to plain text
    html = markdown.markdown(post.content)
    soup = BeautifulSoup(html, 'html.parser')
    clean_text = soup.get_text(separator=' ', strip=True)
    
    # Classify into operational domains
    domain = classify_domain(post.metadata.get('categories', []))
    
    return {
        'objectID': generate_object_id(filepath),
        'title': post.metadata.get('title', ''),
        'content': clean_text[:8000],  # Algolia limit
        'description': post.metadata.get('description', ''),
        'url': build_url(filepath, lang),
        'lang': lang,
        'domain': domain,  # S&OP, Product, Projects, People
        'date': post.metadata.get('date'),
        'tags': post.metadata.get('tags', []),
    }

Result: 72 records indexed (36 English + 36 Spanish) across 4 operational domains.

Step 2: Configuring the Agent in Agent Studio

In the Algolia dashboard, I created an agent with a specialized system prompt:

You are the "Ops Engineering Copilot" for the Datalaria technical blog.

Your expertise spans four pillars:
-  S&OP: Demand forecasting, data hygiene, outlier detection
-  Product: Lifecycle management, data-driven decisions
-  Projects: CI/CD, GitHub Actions, automation, DevOps
-  People: Team dynamics, onboarding, collaboration

Guidelines:
1. Ground all answers in the indexed blog content
2. Be conversational but technically precise
3. When relevant, suggest specific articles to read
4. Support both English and Spanish queries

LLM Provider: Google Gemini (excellent free tier for prototyping)

Step 3: The CORS Problem (and Solution)

Here's where it got interesting. The Agent Studio API doesn't allow direct browser calls due to CORS restrictions. My first attempt:

// ❌ This fails with CORS error
fetch('[https://agent-studio.eu.algolia.com/1/agents/](https://agent-studio.eu.algolia.com/1/agents/){id}/completions', {
    method: 'POST',
    headers: { 'X-Algolia-API-Key': key }
});

Solution: A Netlify Function as a proxy:

// netlify/functions/agent-chat.js
exports.handler = async (event) => {
    const { query, conversationId } = JSON.parse(event.body);
    
    // Call Agent Studio from server-side (no CORS!)
    const response = await fetch(
        `https://agent-studio.eu.algolia.com/1/agents/${AGENT_ID}/completions?compatibilityMode=ai-sdk-4&stream=false`,
        {
            method: 'POST',
            headers: {
                'Content-Type': 'application/json',
                'X-Algolia-API-Key': SEARCH_KEY,
                'X-Algolia-Application-Id': APP_ID
            },
            body: JSON.stringify({
                messages: [{ role: 'user', content: query }],
                id: conversationId
            })
        }
    );
    
    const data = await response.json();
    
    return {
        statusCode: 200,
        body: JSON.stringify({
            answer: data.content,
            conversationId: data.id
        })
    };
};

Key discoveries:

compatibilityMode=ai-sdk-4 is required
stream=false returns JSON instead of SSE
Response format is {id, role, content}, not OpenAI-style

A lightweight vanilla JS widget that integrates with Hugo:

<script src="[https://cdn.jsdelivr.net/npm/algoliasearch@4/dist/algoliasearch-lite.umd.js](https://cdn.jsdelivr.net/npm/algoliasearch@4/dist/algoliasearch-lite.umd.js)"></script>

<script>
async function sendMessage() {
    const query = input.value.trim();
    
    try {
        // Try Agent Studio via Netlify proxy
        const response = await fetch('/.netlify/functions/agent-chat', {
            method: 'POST',
            body: JSON.stringify({ query, conversationId, language })
        });
        
        const data = await response.json();
        addMessage(data.answer, 'agent');
        
    } catch (error) {
        // Fallback to direct Algolia search
        await fallbackSearch(query);
    }
}
</script>

Features:

Dark mode support
Mobile responsive
Bilingual (ES/EN)
⚡ Graceful fallback to search if Agent fails

Technical Challenges

Challenge 1: The 422 Validation Error

My first API calls returned 422: Field required: compatibilityMode. The documentation mentioned it, but I missed it initially.

Fix: Added query parameters ?compatibilityMode=ai-sdk-4&stream=false

Challenge 2: Streaming Response Format

Even after fixing the 422 error, I got SyntaxError: Unexpected token 'f:'. The API was returning SSE format (f:{"message"...) instead of JSON.

Fix: Added stream=false to disable streaming.

Challenge 3: Response Parsing

The response structure wasn't OpenAI-compatible. I expected choices[0].message.content but got {id, role, content}.

Fix: Updated parsing logic to check data.content first.

Results

Metric	Value
Posts Indexed	72 (36 EN + 36 ES)
Avg Response Time	~3-4 seconds
Fallback Coverage	100% (graceful degradation)
Languages Supported	English & Spanish
Deployment	Fully automated via Netlify

Source Code

The complete implementation is open source:

Indexing Script: algolia_sync.py
Netlify Function: agent-chat.js
Chat Widget: algolia-agent.html

What's Next

This is Part 8 of my Autopilot series. The system now has:

✅ Content Generation (Parts 1-3): AI creates social posts
✅ Distribution (Parts 4-5): Automated publishing to Twitter/LinkedIn
✅ Newsletter (Part 6): Automated email campaigns
✅ Quality Control (Part 7): Copilot CLI auditing
✅ Consumer Search (Part 8): AI-powered Q&A - You are here

Why Fast Retrieval Matters

In Operations Engineering, decisions happen fast. When a supply chain analyst asks "What's the best practice for detecting demand outliers?", they need answers in seconds, not minutes of browsing through archives.

Algolia's millisecond retrieval makes this possible:

Scenario	Without Fast Retrieval	With Algolia
Find relevant article	~30-60s browsing	~100ms retrieval
Get synthesized answer	Manual reading required	~3-4s AI response
Context accuracy	May miss related content	RAG ensures completeness

The RAG pipeline works because:

Fast vector search finds semantically similar content across 72 posts instantly
Faceted filtering narrows results by language and domain (S&OP, Projects, Product, People)
Snippet extraction provides the LLM with precisely relevant context

Without sub-second retrieval, the conversation would feel sluggish and users would abandon the chat. Algolia enables the "instant" in instant knowledge discovery – transforming a static blog into an interactive, AI-powered knowledge base.

Conclusion

Algolia Agent Studio transforms a static blog into an interactive knowledge base. Instead of forcing visitors to search and read multiple posts, they can simply ask and get synthesized answers.

The combination of:

Algolia Search (fast, relevant retrieval)
Agent Studio (RAG orchestration)
Google Gemini (natural language generation)
Netlify Functions (serverless proxy)

...creates a powerful, cost-effective AI assistant that runs 24/7.

Questions? Try the live demo at datalaria.com or drop a comment below

This post is part of the Autopilot Project series, where I document how I automate content creation, distribution, and now consumer interaction using AI.

Build an AI Chatbot for Course Creators on WordPress

Thu, 11 Dec 2025 09:56:51 +0000

In this article we will build an AI chatbot. Before writing code, we decide what this chatbot must do. Based on who we are helping and what results they expect we would decide how the chatbot would function.
For this article we will take the example of a course creator who needs an AI chatbot before and after a student buys.

Results that matter for a course creator

Raise course completion from about 20% toward 40% or higher.
Let the bot answer about 70% of common questions so the creator gains back 20-30 hours each month.
Turn more trials into paid seats by keeping prospects in chat with previews and reminders.
Get 80% of buyers to start a lesson within the first day so they stay engaged.
Keep human escalations under 20% so staff can focus on students who truly need help.

Now, based on the above results we would need to decide the AI chatbot behaviour and pick features which serve those results. For our example of a course creator we can select the features below.

Feature	Why this helps
Bubble Widget	Sits quietly in a corner with a timed preview line so visitors can ask about lessons without losing their place. Good for pre-sale work and quick nudges.
Popup Widget	Opens as a modal when someone finishes checkout or stalls on a module, making it easy to push onboarding steps or rescue stuck students.
File Search & RAG	Feeds lesson transcripts, worksheets, and FAQ docs into the bot so every reply references real material instead of generic text.

Build it in five short steps

1. Create and add the system instructions

We will use Predictable Dialogs which has an open source chat widget, into which we would add the system instructions that explain tone, steps, and rules.

2. Upload course files for File Search & RAG

Next we will upload lesson plans, transcripts, homework keys, policy docs, and sample answers. Keep each file clean with headings. Break long docs into 100-200 token chunks with about 20 token overlap so the bot keeps context. Ask test questions that reference tricky spots and confirm the bot cites the right snippets. Keep adding or fixing files until answers stay accurate.

Use the web embed script and choose the bubble variant. This version stays on every page without blocking content. You can capture user info, theme the button, and show a preview message. Here is a starter snippet:

<script type="module">
  import Agent from 'https://cdn.jsdelivr.net/npm/@agent-embed/js@latest/dist/web.js'
  Agent.initBubble({
    agentName: "course-companion",
    user: {
      user_id: window?.USER_ID ?? "anon",
      user_segments: ["prospect"],
    },
    previewMessage: {
      message: "Need a study plan?",
      autoShowDelay: 4000,
    },
    theme: {
      placement: 'right',
      button: { backgroundColor: "#4C3AFF", iconColor: "#FFFFFF" },
      previewMessage: { backgroundColor: "#4C3AFF", textColor: "#FFFFFF" }
    }
  });
</script>

To add this to WordPress you have two simple paths:

Site Editor: go to Appearance → Editor → Pages, add a page or template block, insert a Custom HTML block, and paste the script.
WPCode plugin: install WPCode, go to Code Snippets → Add New → HTML Snippet, paste the script, toggle Activate, and save. This lets you load the widget on every page.

Keep the preview short. Trigger Agent.open() or Agent.showPreviewMessage() when someone hits a pricing page or watches a lesson preview by attaching those calls to your WordPress buttons.

Use the popup version on checkout pages, onboarding flows, or inside the LMS when someone seems stuck. The modal takes over the middle of the screen, so use it only when the student must do something next.

<script type="module">
  import Agent from 'https://cdn.jsdelivr.net/npm/@agent-embed/js@latest/dist/web.js'
  Agent.initPopup({
    agentName: "course-coach",
    autoShowDelay: 0,
    defaultOpen: true,
    user: {
      user_id: lessonProgress.userId,
      user_segments: ["purchased", lessonProgress.currentModule]
    },
    initialPrompt: `The learner is stuck on module ${lessonProgress.currentModule}.`,
    onClose: () => analytics.track("Module Popup Closed")
  });
</script>

To show this inside WordPress, paste the popup script into the exact template or block that renders your checkout or lesson page. If you need it site-wide, add it through WPCode but wrap the call in a condition (for example, check the current page slug in a small inline script) so it only opens where it makes sense. Control width so it fits phones. Toggle it with Agent.open(), Agent.close(), or Agent.toggle() when WordPress detects a stalled learner (for example, after a custom block notices no quiz submission).

5. Test, monitor, improve

Ask the bot for edge cases, like grading rules or refund deadlines, and see if it cites the right file chunk. Watch engagement logs to see if the preview timing or popup triggers are annoying people. Adjust the scripts until questions fall under the 20% escalation goal. Re-upload files whenever you add modules so the index stays current.

Wrap up

When you start with helping a course creator support and convert students, you only need three features: Bubble Widget, Popup Widget, and File Search & RAG. Build the agent, load the files, drop in both widgets through the WordPress Site Editor or WPCode, and keep testing. You get faster responses for students, fewer repetitive tickets for the creator, and better odds that everyone finishes the course.

DevLog 20250820: Towards Unified Chat Gateway

Wed, 20 Aug 2025 21:17:26 +0000

DevLog 20250820: Towards Unified Chat Gateway

Overview

Ever since the general availability of ChatGPT, the problem of building a custom chatbot seemed - at least for a while - like it should have been greatly simplified.

Then came the widespread use of RAG, and now we've entered the era of MCP. A marvelous journey indeed.

But let's return to the chatbot problem.

Traditional Chatbot Development

The first chatbot I built had 5,000 lines of code - most of them simple if-else statements. The last one I built had basic English grammar built-in and could even converse with other bots.

Traditionally, chatbots were built around a few core methods:

Rule-Based Systems – Early bots relied on scripted "if-then" rules or pattern matching (like ELIZA). They could handle predictable inputs but broke down when faced with anything unexpected.
Retrieval-Based Bots – Instead of generating new text, these selected the best response from a predefined set using keyword search or similarity measures. They worked well for FAQs but lacked flexibility.
Generative Models (Pre-LLM) – With Seq2Seq models (RNNs, LSTMs), chatbots began producing novel responses. While more dynamic, these struggled with coherence and factual accuracy.
Hybrid Approaches – Developers often combined rules, retrieval, and basic generative models, managed by a dialogue manager to maintain context. Knowledge bases or ontologies were common in customer support.

Less traditionally ones like Perplexity by Eric Zinda seems to have full understanding of semantics of words (and is built using Prolog).

Importantly, in practice, most production "Chat" windows on websites were not fully automated. They were a blend of machine + human, where the chatbot handled basic queries and human agents stood by to take over for complex requests. This ensured customer satisfaction while keeping automation lightweight. A notable example, widely used today, is Crisp. I mention it here only because I consider it a competitor to what we are aiming to offer.

The Challenges with Using LLM Services

The Python community is blessed. They always seem to get the latest of everything - machine learning libraries, deep learning frameworks, and APIs. For most Python users, invoking ChatGPT or other inference providers (like Hugging Face or NIM) is just a matter of calling an API. For the rest of the world (e.g., C# developers), it usually means making HTTP requests - until an official native library is available.

There's also the matter of streaming API responses. This is often marketed as a big UI boost, and I've attended LangChain seminars emphasizing how responsive GUIs make LLMs appear "less slow." But in reality, for non-trivial tasks, I expect some thinking time. As long as there's a clear visual cue showing the service is alive, I'm fine. Streaming isn't always essential.

That's also exactly what happens in IM apps for conversations between real people.

How Knowledge Retrieval Works

LLMs predict the next word. The essence of an LLM-based chat agent is to give it the right context. RAG provides that context by feeding the LLM (as system prompt) the necessary information, automatically retrieved through similarity search.

The most extreme form of "RAG" - which I've heard of in production systems at NVIDIA's GDA conference - is simply passing everything needed into the system prompt. Don't laugh - sometimes the simplest solution is the most effective.

Vector databases store these embeddings, but today there are far too many to choose from. For simple cases, you can get by with a basic cosine similarity calculation if the dataset isn't too large.

The Challenges

Building a real-world chatbot typically involves:

Connecting to a backend LLM service
Assembling system prompts - either by concatenating everything or using RAG
For RAG systems: uploading documents, generating embeddings, storing them, and enabling queries
Providing a functional GUI

That's it! Simple in theory - but even the first step can be daunting.

First, there are so many LLM services to choose from. On top of that, higher-level offerings like Google NotebookLM, AWS Bedrock, Hugging Face Inference Engine, and NIM complicate the picture.

Here's the secret: the only way to know which works best for you is to try them - extensively.

Second, even when using a hosted inference service instead of self-hosting, there's the issue of authentication if your product is public-facing. Without safeguards, anyone could hijack your service. That means setting up an authentication server.

System prompt assembly and vector database work, by contrast, are routine engineering tasks - straightforward enough once you know what you're doing.

What I Really Want

As a developer, what I really want is:

I have a bunch of files/documents
I upload them to a service
The service creates a ready-to-use agent
I simply call that specialized agent for my purpose

In short, I'd like to embed a chatbot with just a snippet like this:

<script src="/widget/chat-widget.js"
        data-chat-widget
        data-project="ChatGateway"
        data-title="Chat with Methodox (AI)"
        data-display-name="Chat Gateway"
        data-welcome-message="Hi! I'm your AI assistant. Ask me anything."
        data-position="right"
        data-open="false"
        data-primary="#2563eb"></script>

Surely, many providers already offer something similar:

Chatbase
Azure
Clause Enterprise
Tidio
Octane AI
Social Intents
Regal...

But here's the reality:

Chatbase leans heavily on analytics and FAQ-style bots.
Azure gives you the parts (Cognitive Search + OpenAI) but makes you engineer the pipeline.
Clause Enterprise is weighed down by compliance workflows.
Tidio focuses on rule-based customer support flows.
Octane AI is optimized for e-commerce funnels.
Social Intents is designed for live handoffs, not retrieval.
Regal focuses on sales/CRM engagement.

All of these involve vendor lock-in, integration overhead, or enterprise baggage. You end up hacking around features you don't need.

In short, there's no truly lightweight option for just documents + embeddings + an LLM wrapper.

Summary

To recap, a proper service for this use case should be:

A host for uploaded documents
An API server for RAG-based retrieval
Designed for direct public use

The last point is crucial: such a service must allow direct frontend embedding, without worrying about API key protection.

What's Next

We're now building exactly the service I've been describing: Methodox Chat Gateway.

Chat Gateway is a production-ready chat layer that lets you add an AI assistant to any site or app - powered by your own knowledge base. Think of it as documents + embeddings + an LLM wrapper, packaged into something you can deploy in minutes.

With just a single <script> tag, you can drop a polished chat widget into your frontend, or spin up a full-page project chat at /Project/{name}. Each project has its own scoped assistant, model settings, and optional vector store for retrieval. RAG status is even shown transparently (“RAG: on/off”) so your users know when answers are grounded in your documents.

Behind the scenes, Chat Gateway provides:

Simple embedding – add a chat button with no-code widget setup.
Project-scoped assistants – each project is isolated, configurable, and connected only to its own knowledge base.
Hosted project pages – share live demos or support portals instantly.
Straightforward API – post user input and get structured responses, with continuity across sessions.
Built-in guardrails – request limits and concurrency caps to keep things reliable.

Unlike heavier enterprise platforms, Chat Gateway is designed for direct public use: no extra infrastructure for API key protection, no bloated CRM features, and no vendor lock-in. It's simply a lightweight way to put your content to work through an AI assistant.

If you want to see it in action, check out the Playground or visit the live demo at /Project/ChatGateway.

We'll be sharing a full launch announcement soon - but the vision is clear: a frictionless, developer-friendly way to embed intelligent, knowledge-backed chat into any experience.

References

Chat online on the Playground
Methodox Chat Gateway official website
Methodox Wiki doc on Methodox Chat Gateway

Meet the .csv datasets and understand how they works on creating expert system Artificial Intelligence models

Sun, 20 Oct 2024 02:03:52 +0000

Introduction

I’m sure you have listened about CSV datasets. If you are here, it's because you're interested in Artificial Intelligence (AI). AI development is often associated with Machine Learning and huge datasets with big databases. Maybe you have a big idea in mind; you search on YouTube or Google, and the first thing you find out is that if your project doesn't require a large amount of data, a .csv dataset is suitable for this. That is true. It is also true that CSV datasets are very useful for data analysis and are used by enterprises to store and save their data.

I feel that when we talk about CSV datasets, not all is clear. There is much information, but it is not organized in a single place with simple words to allow beginners to understand and use them effectively. The goal of this post is to understand what CSV datasets are, the differences between Machine Learning and Expert Systems, how traditional Expert Systems allow us to automate many tasks, when and how to use .csv datasets, and what the main uses of CSV datasets are.

What this post will not teach you

In this post, I will not explain how to write code that makes CSV files work. This post is centered on CSV files/datasets. Perhaps next week, in another post, I will explain how to create a basic chatbot, assistant, or enterprise bot.

Csv dataset definition

I could use a technical concept from external sources to give a definition of a CSV file, also called CSV dataset or Comma Separated values, but it’s not necessary. Instead, I'll define it and explain it from my understanding and with my own words. A CSV dataset is a collection of structured/organized data often used for creating small artificial intelligence models that do not require much data. They are organized in variables. For example:

# Variable 
question, answer
# Information
What is a csv dataset?, is a collection of data saved very well organized.
How can i create a csv dataset?, at the end of the file add the extension .scv

# As you can see in the example the variables are separated with a comma

Tip: In the example above the part before the comma is the question and the next part is the answer, the order have to match with the variables names defined on the first line.

In this screenshot, I have a dataset with three variables: image_path refers to the source of the image, image_name is the name of the image, and image_description is, as you can imagine, the description given to the image. In this particular case, I'm working with image recognition. However, if you need to work only with messages and answers, the pattern will be the same but with two variables: question and answer, as shown in the next example.

    image_path, image_name, image_description

    static/images/tucan.jpg,Tucán,Un ave colorida con un gran pico, nativa de las selvas tropicales.
    static/images/jaguar.jpeg,Jaguar,Un felino grande y poderoso que habita en América Latina.
    static/images/tapir.jpeg,Tapir,Un mamífero grande y herbívoro que se encuentra en las selvas tropicales.
    static/images/mango.jpeg,Mango,Un fruto tropical dulce y jugoso, muy popular en América Latina.
    static/images/cafe.jpeg,Café,Granos que se utilizan para hacer una bebida estimulante conocida mundialmente.
    static/images/guayaba.jpeg,Guayaba,Una fruta tropical rica en vitamina C, con un sabor dulce y ácido.

Tools to create a CSV file

It depends on your goals or the type of data you’re working with. CSV files can be created using Notepad++, Microsoft Excel, or a code editor like Visual Studio or PyCharm.

Creating a CSV dataset

To create a CSV file using Excel, select the type CSV (comma delimited file, using a text or code editor, name your file and add the extension .csv, for example, dataset.csv you can use the file for creating a dataset on external services like Oracle, or any other cloud service provider.

Reading a CSV dataset

If you need to open and edit your CSV dataset, simply go to your files, locate the CSV dataset, right-click on it, and select "Open with." You can open it with Notepad or a code editor like Visual Studio Code. Once opened, you will be able to read, edit, and save your changes.

Differences between Machine Learning and Expert Systems

First of all, why am I talking about Machine Learning here? When somebody talks about artificial intelligence, we often associate it with Machine Learning or Natural Language Processing or think that it is something different apart from AI. But the reality is that Machine Learning, Natural Language Processing, Computer Vision, Expert Systems, and many others are types of Artificial Intelligence.

The traditional Expert System is a method of artificial intelligence that uses CSV files in their datasets. In this AI model, the developer defines the possible messages or characteristics and provides specific answers if one of those possible messages or characteristics is present.

In a Machine Learning model, you provide a database or a large collection of data, and the model itself uses this data to compare the content of the message with the existing information in the database. If this data coincides with one element in the database, the Machine Learning model will give an answer based on this data.

Both methods have their advantages and disadvantages. Machine Learning models are very useful when working with large language models and other projects with high level of unknowns, however the answers provided by these models can sometimes be inaccurate. So, when we need a bot that provides very accurate, precise answers, we use Expert System models trained with a CSV dataset.

How do traditional Expert Systems allow us to automate tasks?

Let me use an analogy so that we can understand very well how to use CSV datasets with a traditional Expert System AI model to automate repetitive tasks. Imagine that you are a small, medium, or large business that sells products, cellphones, let's say. To attract more clients, you provide support via WhatsApp or on your website. Those clients will ask specific questions about the prices of specific models. They will ask about promotions, available models, schedules of attention, and so on. All those tasks are repetitive and time-consuming. A good amount of those clients will ask questions and not buy, so you're losing your valuable time. What if we create a bot that can answer each question from each client automatically, specifically, and instantly?

Problems like this can be solved using a CSV dataset in which you define the answer for every possible question or message. Of course, you'll need to develop a simple model capable of reading the file, receiving the message sent to the server, and responding with the specific answer. Now, when the client asks something, the bot will respond instantly, reducing time, work, and providing a good experience for the user.

Uses of .csv datasets?

Use a .csv dataset anytime you need to develop a product (AI model) whose first requisite is very accurate and specific answers, as we discussed earlier. In simple terms, this kind of dataset is used for creating Expert System models, but a CSV dataset can also be used to store data in a structured or organized way. Data that can later be used for data analysis and facilitate decision-making for administrative personnel.

Main uses of CSV datasets explained

Chatbots: CSV datasets can be used to create assistant bots for small, medium, and large businesses or enterprises. These bots can be integrated with Twilio, WhatsApp, Telegram, or a website. Next week, I'll publish a tutorial to explain step-by-step how to create a chatbot.
Data store: Save data in an organized manner. You can also use other tools that interact with the data you want to save to write a CSV file or update it automatically.
Data analysis tools: CSV datasets can be easily manipulated with Python libraries like Pandas and Matplotlib for data analysis and visualization.
Image recognition (integrated with other tools): As shown in the earlier screenshot, you can create a CSV dataset with a specific structure to integrate with other tools for image recognition systems.
Other uses

How to use?

The answer to this question depends on the goal or what you’re using it for. Let's say that if you want to create an assistant chatbot, the first step is to create your CSV file with the specific possible questions and the respective answers for each question. But that is not all; you will also need a programming language to train the model (the Expert System I explained earlier). Additionally, this model needs to be integrated into a website or with services like WhatsApp, Twilio, or others. The "how-to" is not covered in this post because it needs more space to cover the step-by-step integration. In this post, as I explained in the introduction, my goal is to understand what CSV datasets are and how we can use them to make our lives easier.

Advantages of a CSV datasets

Accurate and specific answers: When used in assistants or chatbots, you can specify the answer for the possible questions or messages.
Personalization: You can create or update your data according to your needs.
Organization: You can save or write your data from other programs organized in variables according to your needs, which will improve readability and allow this data to be integrated with other programs, libraries, or tools in general to analyze its information.

Conclusion

A CSV dataset is a collection of organized data. These files can be used in combination with the traditional artificial intelligence methodology called Expert System to create automated bots, assistants, data analysis tools, and many other automation tools for small, medium, and large businesses interested in saving time, money, and work, while providing good, personalized, and instant assistance to their clients. They are also used to store data in a well-organized and structured way to be later used in data analysis, facilitating decision-making.

Deploy an AI journalist chatbot with Gradio and Supabase

Sat, 18 May 2024 08:34:04 +0000

1. Introduction

supabase-ai-chat is natively developed as a Gradio-backed HuggingFace (HF) space and can be found here: it was developed to serve as a knowledgeable assistant in the field of AI news.

Want something more? There is also a version built on top of Flowise but it is only a test deployment and, due to security reasons, its access is limited: you can request the access writing a private message on CoderLegion in which you explain who you are and why you would want to use the application.

It is an application that exploits Supabase (the open-source alternative to Firebase) as a vector store to search for context information and HF Spaces API to query and retrieve responses from Phi-3-128K.

supabase-ai-chat was built selecting 5000 AI-related news articles from ai-tech-articles dataset on HF Hub, with the help of Supabase as a vector store (if you don't know what vectore stores are, here is an article that I wrote about them: "Vector databases explained"). Let's break dow the steps we need to take to build this application.

2. Pre-requisites

2a. Supabase

Before we start building our application, we will need to access Supabase: you can head over to their website and register through your GitHub account (which is definitely the simplest way to get access to their services).

Once your account has been created, you can go to "Dashboard" and select "New Project". This will take you to the actual project you can build and it will require you to name it and save a password: do not lose the password, because it will be useful later on.

After Supabase manages all the dependencies of your project, you will notice that there will be both an API Key and a Project URL that pop up as soon as you enter the page: they won't be useful to us for this project, but they can definitely be in the future.

Now make sure to go to "Project Settings > Database" and copy the Connection String: store it somewhere, for now, as it will be fundamental to build the application.

2b. HuggingFace

In order to get an Hugging Face account:

Go to Hugging Face from the link above and click on the Sign up button. Alternatively, you can directly follow this link.
Provide e-mail address and password
Follow the instructions as you are prompted by the registration procedure

3. Folder Setup

You will need to have a specific folder setup in order to build this application. You can achieve that setup by cloning the GitHub repository:

git clone https://github.com/AstraBert/supabase-ai-chat.git
cd supabase-ai-chat

Let's now take a look at the repository:

./
├── README.md
├── app.py
├── .env.example
├── data
│   └── ainews_5k.csv.gz
├── requirements.txt
├── supabase_upsert.py
└── utils.py

We have:

app.py, which is the script we will be using to design our application
utils.py, which is the script where we will be storing useful classes and functions
supabase_upsert.py, which is the script that we will upload our AI news to our Supabase project with
requirements.txt, which contains the needed dependencies for this project
data/ainews_5k.csv.gz, which is the compressed version of the file that stores all the 5000 news articles about AI
.env.example, which is the example file to store environment variables

We can set our environment up by running:

python3 -m pip install -r requirements.txt

And, on the other hand, we complete the configuration by renaming the .env.example file to .env and by substituting to the placeholder string associated with supabase_db environment variable. An example of that would be:

supabase_db=postgresql://postgres.fsaosjjgbgpbsiag:*Emails are not allowed*:5432/postgres

Remember that it is important to write postgresql at the beginning of the URL, and not only postgres, as the newest versions of vecs, the python package that serves as a PostgresSQL client, do not support this last variation of the baseurl.

4. Prepare Supabase Collection

Now that we have everything set up in terms of environment, we can upload the documents to our Supabase project, configuring them as a collection. We will be doing it using the supabase_upsert.py script we downloaded earlier from GitHub.

Let's take a quick look at the code!

We start by importing the needed dependencies and loading the environment variable related to our Supabase database:

import vecs
from dotenv import load_dotenv
import os

load_dotenv()
DB_CONNECTION = os.getenv("supabase_db")

Then we move on to connect to Supabase client and create a new collection called documents, which will be receiving vectors embedding of a size of 384:

vx = vecs.create_client(DB_CONNECTION)
docs = vx.get_or_create_collection(name="documents", dimension=384)

Now we load the word-embedding model and proceed with the upsertion of the AI news to the Supabase collection (we need to read the csv, first):

import pandas as pd
df = pd.read_csv("data/ainews_5k.csv.gz")
idex = list(df["id"])
texts = list(df["text"])
data = [{'id': idex[i], 'content': texts[i], 'embedding': []} for i in range(len(texts))]

from sentence_transformers import SentenceTransformers
encoder = SentenceTransformer("all-MiniLM-L6-v2")

for item in data:
    docs.upsert(records=[(item['id'],encoder.encode(item['content']).tolist(), {"Content": item['content']}),])

The last, but not least, portion of the code creates a vector index to ease the search for our retrieval algorithm:

docs.create_index(measure=vecs.IndexMeasure.cosine_distance)

5. Define Utilities

After having upserted the AI news to Supabase, we just need to define two classes that will help us managing the queries from the users.

The first one involves the possibility of translating the user's prompt from the original language to English, and then the response from English back to the original language:

from langdetect import detect
from deep_translator import GoogleTranslator

class Translation:
    def __init__(self, text, destination):
        self.text = text
        self.destination = destination
        try:
            self.original = detect(self.text)
        except Exception as e:
            self.original = "auto"
    def translatef(self):
        translator = GoogleTranslator(source=self.original, target=self.destination)
        translation = translator.translate(self.text)
        return translation

As you can see, we use two python packages, langdetect and deep_translator, to achieve the goal.

The second class allows us to search a Supabase collection using a text query (which is transformed into vectors on the fly by the same encoder we used to upload everything to the collection).

class NeuralSearcher:
    def __init__(self, collection, encoder):
        self.collection = collection
        self.encoder = encoder
    def search(self, text):
        results = self.collection.query(
            data=self.encoder.encode(text).tolist(),  # required
            limit=1,                     # number of records to return
            filters={},                  # metadata filters
            measure="cosine_distance",   # distance measure to use
            include_value=True,         # should distance measure values be returned?
            include_metadata=True,      # should record metadata be returned?
        )
        return results

For this simple example project, we do not define filters: this can be done for more advanced projects and more complex necessities.

6. Build the Application

Now that we have everything set up, we can actually build our Gradio application that will be hosted in HF Spaces.

We open app.py and start by importing packages and defining constants:

import gradio as gr
from utils import Translation, NeuralSearcher
from gradio_client import Client
import os
import vecs
from dotenv import load_dotenv
from sentence_transformers import SentenceTransformer

load_dotenv()

collection_name = "documents"
encoder = SentenceTransformer("all-MiniLM-L6-v2")
client = os.getenv("supabase_db")
api_client = Client("eswardivi/Phi-3-mini-128k-instruct")
lan = "en"
vx = vecs.create_client(client)
docs = vx.get_or_create_collection(name=collection_name, dimension=384)

Now that we have defined our Supabase and Gradio API clients, we can finally build a function that takes the user query, recognizes the language, performs various translation tasks, retrieves some relevant information from Supabase collection, feeds them as context to Phi-3-128K along with the actual message from the user, and then finally gets a response. You can refer to the flowchart here, for a better visualization of the process:

Here is the function:

def reply(message, history):
    global docs
    global encoder
    global api_client
    global lan
    txt = Translation(message, "en")
    print(txt.original, lan)
    if txt.original == "en" and lan == "en":
        txt2txt = NeuralSearcher(docs, encoder)
        results = txt2txt.search(message)
        response = api_client.predict(
            f"Context: {results[0][2]['Content']}; Prompt: {message}",# str  in 'Message' Textbox component
            0.4,# float (numeric value between 0 and 1) in 'Temperature' Slider component
            True,# bool  in 'Sampling' Checkbox component
            512,# float (numeric value between 128 and 4096)
            api_name="/chat"
        )
        return response
    elif txt.original == "en" and lan != "en":
        txt2txt = NeuralSearcher(docs, encoder)
        transl = Translation(message, lan)
        message = transl.translatef()
        results = txt2txt.search(message)
        t = Translation(results[0][2]['Content'], txt.original)
        res = t.translatef()
        response = api_client.predict(
            f"Context: {res}; Prompt: {message}",
            0.4,
            True,
            512,
            api_name="/chat"
        )
        response = Translation(response, txt.original)
        return response.translatef()
    elif txt.original != "en" and lan == "en":
        txt2txt = NeuralSearcher(docs, encoder)
        results = txt2txt.search(message)
        transl = Translation(results[0][2]['Content'], "en")
        translation = transl.translatef()
        response = api_client.predict(
            f"Context: {translation}; Prompt: {message}",
            0.4,
            True,
            512,
            api_name="/chat"
        )
        t = Translation(response, txt.original)
        res = t.translatef()
        return res
    else:
        txt2txt = NeuralSearcher(docs, encoder)
        transl = Translation(message, lan.replace("\\","").replace("'",""))
        message = transl.translatef()
        results = txt2txt.search(message)
        t = Translation(results[0][2]['Content'], txt.original)
        res = t.translatef()
        response = api_client.predict(
            f"Context: {res}; Prompt: {message}",# str  in 'Message' Textbox component
            0.4,
            True,
            512,
            api_name="/chat"
        )
        tr = Translation(response, txt.original)
        ress = tr.translatef()
        return ress

Now, in order to build the Chat Interface, Gradio offers solutions that hardly have competitors in terms of simplicity: as a matter of facts, we actually need one line of code to define the interface and one to launch it.

demo = gr.ChatInterface(fn=reply, title="Supabase AI Journalist")
demo.launch(server_name="0.0.0.0", share=False)

Now, if you run:

python3 app.py

From your terminal, in less than 30 seconds you will see that http://localhost:7860 gets fills with life: our chatbot is loaded and ready to be used! We can now query it by just writing a prompt and patiently waiting for the results to come!

We can also upload our repository to Spaces, or simply create a space here and paste the code we've just written in files that are named as our local version of them.

WARNING: I strongly suggest, for safety reasons, not to expose your .env file publicly

You can, alternatively, define secret keys in the Space settings, and just import the keys by simply including:

client = os.getenv("your-secret-key-name")

In your app.py script

7. Conclusion

We now created our first Gradio application: this is the first time we build a fully front-end and back-end defined tool, without relying on third-party interfaces such as Telegram or Discord.

Moreover, we natively integrated in our project a vector database (Supabase) for text retrieval and a Large Language Model (Phi-3-128K) for text generation: we did everything with pure python: the great part is that the whole code takes less than 150 lines!

Stay tuned for the next article, in which we will conclude our bot series by talking about Discord bots, and do not forget to live a little star in the repository!

Coder Legion - Recent posts tagged chatbot

Why Your RAG Chatbot Gives Wrong Answers (and How to Fix It)

First, understand how a RAG chatbot answers

Step 1: Find out which layer is failing

Fixing retrieval problems

Chunk by meaning, not by character count

Clean the source content

Combine keyword and semantic search

Retrieve more, then rerank

Rewrite follow-up questions

Fixing prompt and generation problems

Tell the model to stay inside the material

Require citations

Use a "no good match" threshold

Set topic limits and a persona

Fixing content problems

Build a small evaluation set

Watch the bot in production

A realistic order of work

When it's still wrong: check the basics

Key takeaways

Why Your RAG Chatbot Gives Wrong Answers (And How to Fix It)

First, Find Where the Problem Is

Problem 1: Chunks Are Too Big or Too Small

Problem 2: No Overlap Between Chunks

Problem 3: The Text From the PDF Is Messy

Problem 3 and a Half: Tables

Problem 4: Wrong Number of Chunks (top_k)

Problem 5: The Prompt Lets the Model Guess

Problem 6: No "I Don't Know" Path

Problem 7: Wrong or Mixed Embedding Models

Problem 8: No Sources, So No Way to Check

Your Debugging Checklist

Wrap-Up

Is It Still RAG If There's No Retrieval?

The design brief

It is not an agent

To "RAG" or not to RAG

Reducing confabulation

Prompt caching

The build: brutally efficient again

Abuse protection

Personality: the fun part

CommentKeyword - Instagram Comments 2 DM

Ptero - AI Chat

30 sec is all it takes to build an AI assistant

Embracing the Wonderful World of A.I.

Autopilot - Assistant : Building an AI Copilot with Algolia Agent Studio

What I Built

Demo

The Architecture

Key Components

How I Used Algolia Agent Studio

Step 1: Indexing the Knowledge Base

Step 2: Configuring the Agent in Agent Studio

Step 3: The CORS Problem (and Solution)

Step 4: The Chat Widget

Technical Challenges

Challenge 1: The 422 Validation Error

Challenge 2: Streaming Response Format

Challenge 3: Response Parsing

Results

Source Code

What's Next

Why Fast Retrieval Matters

Conclusion

Build an AI Chatbot for Course Creators on WordPress

Results that matter for a course creator

Build it in five short steps

1. Create and add the system instructions

2. Upload course files for File Search & RAG

3. Add the Bubble Widget to your WordPress theme

4. Fire the Popup Widget when timing matters

5. Test, monitor, improve

Wrap up

DevLog 20250820: Towards Unified Chat Gateway

DevLog 20250820: Towards Unified Chat Gateway

Overview

Traditional Chatbot Development

The Challenges with Using LLM Services