Coder Legion - Recent posts tagged vector-search

AI Fundamentals - Part 3: Giving AI Knowledge Beyond Its Training

Sun, 12 Jul 2026 06:45:12 +0000

In Part 2, we learned why AI sometimes hallucinates. One of the biggest reasons is that an LLM can only answer based on what it learned during training and the information available in its context window. We also introduced grounding-providing the model with reliable information at runtime instead of expecting it to know everything.

But that raises an important question: Where does that information come from? Modern AI applications don't simply dump an entire database or a thousand-page PDF into the prompt. Instead, they first identify the most relevant pieces of information and only send those to the model. In this article, we'll learn how that works.

Running Example

Let's continue building our AI-powered Travel Planner. So far, it can answer general travel questions using the knowledge it learned during training. Now we want to make it much smarter by uploading several documents into our application:

Lonely Planet's Japan travel guide
A PDF containing train schedules
A document listing recommended local restaurants
Hotel information
Internal travel policies for our company

Together, these documents contain hundreds of pages. Now the user asks:

I'm staying near Tokyo Station. Which ramen restaurant from our travel guide is within walking distance and is known for vegetarian options?

Somewhere in those hundreds of pages is the answer. The challenge is no longer generating text-it's finding the right information first.

The Problem: An LLM Can't Read Your Entire Knowledge Base Every Time

A common misconception is that AI applications simply send all their documents to the model. Imagine our travel guide contains 450 pages, thousands of restaurant listings, hotel descriptions, transportation details, and sightseeing recommendations. Sending all of that to the LLM every time someone asks "Where should I eat tonight?" creates several problems.

First, many documents are simply too large to fit inside the model's context window. Second, even if they did fit, making the model read hundreds of irrelevant pages just to answer a simple question is expensive, slow, and inefficient. Instead, we want to send only the information that matters. The question becomes: How do we find the right few paragraphs from hundreds of pages?

Your first instinct might be: "Why not just search the documents?" That's exactly what we'll explore next.

Keyword Search: The Traditional Approach

Before AI became popular, most applications relied on keyword search. The idea is simple: the user searches for a word, and the application finds documents containing that exact word.

For example, given this travel guide text:

Tokyo has several excellent vegetarian ramen restaurants...
Mount Fuji is one of Japan's most iconic landmarks...
The Shinkansen connects Tokyo and Kyoto...

If the user searches for "vegetarian ramen", the search engine finds the first paragraph because it contains both words. While traditional keyword search is simple, fast, and easy to understand, it falls short when matching meaning instead of exact text.

When Keyword Search Falls Short

Synonyms and Meaning (Example 1): If a user asks "Where can I eat meat-free ramen?" and our document says "Vegetarian ramen is available near Tokyo Station", a traditional keyword search may completely miss this result because it searches for exact words, not meaning. It doesn't know that meat-free, vegetarian, and plant-based are closely related concepts.
Name Variations (Example 2): If a user asks to "Suggest a place near Fujisan", and the document says "A day trip to Mount Fuji is highly recommended", keyword search often fails because it doesn't understand that Fujisan and Mount Fuji refer to the same place.
Terminology Differences (Example 3): If a user asks "Where can I find authentic local noodle shops?" and the guide says "Several ramen restaurants popular with residents...", keyword search fails to connect noodle shop with ramen restaurant.

The real limitation is that keyword search compares text, while humans compare meaning. This is why search systems often return dozens of irrelevant results or miss the best result entirely, as you've likely experienced when searching documentation using a word that differs from the author's chosen terminology.

Semantic Search: Searching by Meaning

This is where AI changes everything. Instead of asking "Does this document contain the same words?", semantic search asks "Does this document mean the same thing?"

Let's return to our example. The user asks "Where can I eat meat-free ramen near Tokyo Station?" and the travel guide says "Ichiran offers vegetarian ramen a short walk from Tokyo Station." A semantic search understands that meat-free and vegetarian express nearly the same idea, allowing it to retrieve the correct passage even though the exact words are different.

But computers don't naturally understand meaning; they understand numbers. How can a computer determine that vegetarian and meat-free are closely related? That's where embeddings come in.

Embeddings: Turning Meaning into Numbers

An embedding is a numerical representation of text that captures its meaning. Think of it as a fingerprint-not of the exact words, but of what those words represent.

For example, these three phrases all have different wording but express similar ideas, so their embeddings end up being very similar:

Vegetarian ramen
Meat-free noodles
Plant-based ramen

Conversely, phrases like Mount Fuji, Flight booking, and Currency exchange produce embeddings that are quite different because they represent completely unrelated concepts. The key takeaway is: embeddings represent meaning, not exact wording.

Why Not Compare the Original Text?

Let's go back to our Travel Planner. Suppose our travel guide contains:

Ichiran offers vegetarian ramen near Tokyo Station.

Now imagine two different users:

User A: Recommend vegetarian ramen.
User B: Where can I find meat-free noodle restaurants?

A computer comparing plain text sees very few matching words between User B's query and the guide. A traditional keyword search might decide they are unrelated. Embeddings solve this by converting both pieces of text into numerical representations where similar meanings end up close together.

Semantic Similarity: Measuring Meaning Instead of Words

Once text is converted into embeddings, comparing sentences becomes much easier. Instead of asking if they contain the same words, we ask: How similar are their embeddings? If two embeddings are close together, they are likely talking about similar ideas; if they are far apart, they are probably unrelated.

A useful mental model is to imagine every sentence placed on a giant map. Similar ideas appear close together, while unrelated concepts are placed farther apart:

                  Travel

                      ●
                Visit Japan

           ●                   ●
  Vegetarian Ramen      Tokyo Station

                ●
         Meat-free Noodles


--------------------------------------------


                      ●
              JavaScript Closures

                                ●
            Docker Networking

Notice how the travel-related concepts naturally cluster together, while completely unrelated software concepts appear elsewhere. This is exactly what embeddings help achieve.

Cosine Similarity: Finding the Closest Match

Now that every sentence has an embedding, we need a way to compare them mathematically. One of the most common techniques is cosine similarity.

Suppose our Travel Planner has these restaurant descriptions stored:

A: Vegetarian ramen near Tokyo Station.
B: Traditional sushi restaurant.
C: Italian pizza in Osaka.

If the user asks for "Meat-free noodles near Tokyo Station", the embeddings might compare like this:

Restaurant	Similarity
Vegetarian ramen near Tokyo Station	0.96
Traditional sushi restaurant	0.41
Italian pizza in Osaka	0.09

The higher the score, the closer the meanings. Notice that the user never mentioned the word vegetarian, yet Restaurant A is still identified as the best match because the meaning is similar.

Mental Model

In a library, keyword search works like looking up every book containing the word ramen. Semantic search works by finding books discussing the same idea, even if they use completely different words.

Where Do These Embeddings Come From?

Fortunately, you don't have to calculate these numbers yourself. Just as LLMs specialize in generating text, there are embedding models that specialize in generating embeddings. Instead of producing a paragraph, an embedding model takes text and produces a list of numbers representing its meaning:

User Text ("Vegetarian ramen...") ──► Embedding Model ──► [0.18, -0.42, 0.91, ...]

The resulting list (or vector) usually contains hundreds or thousands of numbers. You never need to interpret them yourself; their only purpose is to make mathematical similarity comparisons possible.

Chunking: Breaking Large Documents into Pieces

How do we generate embeddings for a 500-page travel guide? We don't feed the entire book into the embedding model at once. Embedding models have strict input limit constraints (typically measured in tokens), and comparing a whole book's meaning is too vague to be useful. If a user asks about a specific ramen restaurant, we want to retrieve the exact paragraph discussing it, not a 500-page book embedding.

So, we break the document down into smaller, logical pieces in a process called chunking. A chunk could be a single paragraph, a section of 100-200 words, or a sliding window of text (e.g., 200 words with a 20-word overlap to ensure context isn't cut off at boundaries).

Mental Model

Think of chunking like cutting a long movie reel into short, thematic scenes. If you want to find the scene with Mount Fuji, it is much easier to search and index short clips than searching the entire three-hour film.

💡 Developer's Takeaway

Choosing the right chunk size is a balancing act. If chunks are too small, they might lose context (e.g., a sentence saying "It is delicious" without mentioning what is delicious). If they're too large, the specific details get averaged out, and search accuracy drops.

Vector Databases: Storing and Indexing Meanings

Once we have broken our travel guide into thousands of chunks and generated an embedding for each, we need a place to store them. Traditional databases (like PostgreSQL or MySQL) are optimized for finding exact matches like numbers, dates, or strings. They aren't designed to compare thousands of high-dimensional lists of numbers to find which ones are "closest" in meaning.

That's where Vector Databases come in. A vector database is designed specifically to store and index high-dimensional vectors (embeddings) and search through them based on mathematical similarity (like cosine similarity) extremely fast.

Common vector database setups include:

Dedicated vector databases (Pinecone, Milvus, Qdrant, Chroma)
Vector extensions for traditional databases (like pgvector for PostgreSQL)

💡 Developer's Takeaway

You don't need to write complex search algorithms yourself. A vector database handles the indexing and allows you to query it using an embedding, returning the most semantically similar chunks in milliseconds.

Embedding Dimensions: Understanding the Scale

When you configure a vector database, it will ask you to specify the number of dimensions. Dimensions refer to the size of the list of numbers generated by the embedding model. For example:

A lightweight model might output vectors with 384 dimensions.
A standard model (like OpenAI's text-embedding-3-small) outputs 1536 dimensions.
Large models can output 3072 dimensions or more.

Each dimension represents an abstract feature of meaning that the model learned during training. You don't need to know what each dimension represents; the database only cares that you are comparing vectors of the exact same size.

💡 Developer's Takeaway

An embedding model and a vector database must match: you cannot store a 1536-dimension embedding in a database index configured for 384 dimensions.

RAG (Retrieval-Augmented Generation): Putting It Together

Now we have all the pieces to solve our Travel Planner's original challenge: answering questions using our travel guides without sending the entire book to the LLM. We use a pattern called RAG (Retrieval-Augmented Generation).

RAG combines Retrieval (finding relevant chunks from our vector database) and Generation (having the LLM write a response using those chunks). Here is the step-by-step workflow:

Ingestion (Happens once in advance):
- Break documents into chunks.
- Generate embeddings for each chunk.
- Store the chunks and their embeddings in a vector database.
Querying (Happens at runtime):
- User asks: "Recommend a vegetarian ramen shop near Tokyo Station."
- The application generates an embedding for the user's question.
- The application queries the vector database using this question embedding.
- The database returns the top 2 or 3 most similar text chunks.
- The application merges the user's question and the retrieved chunks into a single prompt (grounding context).
- The LLM reads the context and generates a correct, verified answer.

               User Question
                     │
                     ▼
             Generate Embedding
                     │
                     ▼
    Query Vector DB ──► Stored Chunks
                     │
                     ▼
            Get Relevant Chunks
                     │
                     ▼
    Build Prompt (Question + Chunks)
                     │
                     ▼
             LLM Generates Text
                     │
                     ▼
             Grounded Response

💡 Developer's Takeaway

RAG is the most popular way to connect proprietary data to LLMs. It ensures the model's response is grounded in facts, reduces hallucinations, and works without needing to retrain the model.

CAG (Cache-Augmented Generation): The New Alternative

While RAG is powerful, it requires setting up chunking pipelines, embedding models, and vector databases. This can be complex. With modern LLMs having massive context windows (often supporting 1 million to 2 million tokens), another approach has emerged: CAG (Cache-Augmented Generation).

Instead of retrieving matching chunks at runtime, CAG preloads the entire document set (e.g., our 500-page travel guide) directly into the model's context window. It leverages KV (Key-Value) Caching to precompute and cache the mathematical states of the documents once. When a user asks a question, the LLM reads the cached documents instantly, bypassing the retrieval pipeline entirely.

RAG is like looking up a topic in a library catalog, retrieving three specific books, and reading them to find the answer.
CAG is like having the entire encyclopedia already open on your desk with the pages instantly readable.

💡 Developer's Takeaway

CAG simplifies the architecture by eliminating vector databases and chunking. However, it is limited by the size of the context window and is best suited for relatively static datasets that fit comfortably within the model's limit. RAG remains essential for massive or constantly changing data.

Recap

In this article, we explored how to give AI knowledge beyond its training. We covered:

Keyword Search vs. Semantic Search
Embeddings, Semantic Similarity, and Cosine Similarity
Chunking, Vector Databases, and Embedding Dimensions
RAG (Retrieval-Augmented Generation) and CAG (Cache-Augmented Generation)

In Part 4, we will answer the next logical question: How do we connect our model to the outside world so it can execute tasks, use external APIs, and remember users over time? We will explore tool calling, memory, Model Context Protocol (MCP), and AI agents.

SurrealDB: The "One-Size-Fits-All" Database for Lazy Developers (Part 1)

Wed, 27 May 2026 17:11:00 +0000

Intro: We need to talk about your "Modern Stack"
Let’s be real for a second. We’ve all been there: You have a "simple" project idea. Two hours later, you’re staring at a Docker Compose file with PostgreSQL, Redis for caching, Neo4j because "graphs are cool," and a Node.js backend just to move JSON from A to B.

Why are we doing this to ourselves?

I’m a medior dev. I’ve reached the point in my career where I’m too tired to write the same CRUD boilerplate for the 100th time. I just want to build things that work. And that’s when I stumbled upon SurrealDB.

What is SurrealDB? (Besides a cool name)
Imagine if SQL, MongoDB, and GraphQL had a baby, and then that baby was raised by Firebase but grew up to be actually flexible.

SurrealDB is a multi-model database. It doesn't care if you want documents, relational tables, or complex graph structures. It just says: "Sure, why not both?"

Why you (as a lazy dev) should care:
The "Backend Killer" (No, seriously): SurrealDB has built-in Row-Level Security (RLS) and authentication. You can connect your frontend (React, Vue, whatever) directly to the database. No more writing 500 lines of Express.js just to check if user_id === owner_id.

Schema-Flexible (The "I'll decide later" feature):
You can start schema-less (just dump data) and then enforce a schema once you actually know what you're building. It's perfect for prototyping without the "oops, I forgot a migration" headache.

Graph Relations without the PhD:
Remember trying to write a recursive CTE in SQL to find "friends of friends"? In SurrealDB, it's literally just an arrow: SELECT ->knows->person FROM user:me. My brain thanked me.

A Quick Reality Check
Is it magic? Almost. Is it perfect? Well, it's still evolving. But if you're building a startup or a side project and you're a "team of one," using a traditional stack is like bringing a tank to a knife fight. It’s overkill and you’ll probably hurt yourself.

What’s next?
In the next part of this mini-series, I’m going to show you how I deleted 70% of my backend code using SurrealDB’s auth layer.

Spoiler: It felt amazing.

Oracle Database 23ai: Vector Similarity Search - Exact, Approximate, and Multi-Vector Strategies

Mon, 16 Mar 2026 08:59:34 +0000

Oracle Database 23ai's AI Vector Search provides multiple strategies for finding similar vectors, each with different trade-offs between accuracy, speed, and resource usage. Understanding when to use exact search, approximate search, or multi-vector search—and knowing the essential vector functions—is crucial for building high-performance semantic search applications.

Understanding Similarity Search Types

1. Exact Similarity Search (Flat Search)

Exact similarity search calculates a query vector's distance to all other vectors. It's also called flat search or exhaustive search because every vector in the dataset is compared.

Characteristics:

Gives the most accurate results
Perfect search quality (100% recall)
Involves potentially significant time as dataset grows
No indexes required
Suitable for small to medium datasets (thousands to hundreds of thousands of vectors)

SQL Example:

SELECT 
  product_id,
  product_name,
  VECTOR_DISTANCE(embedding, :query_vector, COSINE) AS similarity
FROM products
ORDER BY similarity
FETCH FIRST 10 ROWS ONLY;

When to Use:

Small datasets where performance is acceptable
When perfect accuracy is required
Development and testing phases
Benchmarking approximate search results

2. Approximate Similarity Search

Approximate similarity search uses vector indexes to dramatically speed up searches with minimal accuracy loss. Instead of checking every vector, it leverages specialized data structures to narrow the search space.

Key Requirements:

You must enable vector pool in the SGA for HNSW indexes (in-memory neighbor graph indexes).

ALTER SYSTEM SET vector_memory_size = 800M SCOPE=SPFILE;
-- Restart required

Characteristics:

Can be more efficient but less accurate than exact search
Uses target accuracy setting (typically 90-99%)
Requires vector indexes (HNSW or IVF)
Scales to millions or billions of vectors
Typical accuracy: 95%+ with proper configuration

SQL Example:

-- Using FETCH APPROXIMATE
SELECT 
  product_id,
  product_name
FROM products
ORDER BY embedding less-than equals greater-than :query_vector
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

Performance Comparison:

According to Oracle benchmarks, exact search on 50,000 vectors took 1.50 seconds versus 0.47 seconds with HNSW index—over 3x faster with the same top-10 results.

Vector Index Types

Oracle AI Vector Search supports two main types of vector indexes:

HNSW (Hierarchical Navigable Small World)

In-Memory Neighbor Graph vector index that creates a navigable graph structure for ultra-fast similarity search.

Creating HNSW Index:

CREATE VECTOR INDEX products_hnsw_idx
ON products(embedding)
ORGANIZATION NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

Characteristics:

Fully built in-memory in the vector pool
Extremely fast queries (sub-100ms)
Requires substantial memory
No DML operations allowed after creation
Not available in RAC environments

Memory Calculation:

Formula: 1.3 times number_of_vectors times number_of_dimensions times dimension_byte_size

Example for 1M vectors, 768 dimensions, FLOAT32 (4 bytes):

1.3 × 1,000,000 × 768 × 4 = 3.99 GB

IVF (Inverted File Flat)

Neighbor Partition vector index built on disk with blocks cached in the buffer cache.

Creating IVF Index:

CREATE VECTOR INDEX products_ivf_idx
ON products(embedding)
ORGANIZATION NEIGHBOR PARTITIONS
DISTANCE COSINE
WITH TARGET ACCURACY 95;

Characteristics:

Disk-based, blocks cached in buffer cache
More scalable for very large datasets
Supports DML operations (may require periodic rebuild)
Works in RAC environments
Slightly slower than HNSW but still very fast

How IVF Works:

The index partitions vectors into clusters based on similarity. By default, the number of partitions equals the square root of the dataset size. During search, only relevant clusters are examined.

3. Multi-Vector Similarity Search

Multi-vector similarity search is usually used for multi-document search where documents are split into chunks, and chunks are embedded individually into vectors.

Use Cases:

Long documents split into paragraphs or sections
Product catalogs with multiple descriptions
Research papers chunked for semantic search
Multi-modal data (text + images)

How It Works:

Documents are split into chunks
Each chunk is embedded individually into a separate vector
Chunks are stored with partition keys linking them to parent documents
Search retrieves top chunks across all documents
Results can be aggregated by document

Uses partitions to organize chunks belonging to the same document.

Table Structure Example:

CREATE TABLE document_chunks (
  doc_id NUMBER,
  chunk_id NUMBER,
  chunk_text CLOB,
  embedding VECTOR(768, FLOAT32),
  PRIMARY KEY (doc_id, chunk_id)
) PARTITION BY RANGE (doc_id) (
  PARTITION p1 VALUES LESS THAN (1000),
  PARTITION p2 VALUES LESS THAN (2000),
  PARTITION p3 VALUES LESS THAN (MAXVALUE)
);

Multi-Vector Search Query:

-- Find top chunks across all documents
SELECT 
  doc_id,
  chunk_id,
  chunk_text,
  VECTOR_DISTANCE(embedding, :query_vector, COSINE) AS score
FROM document_chunks
ORDER BY score
FETCH APPROXIMATE FIRST 20 ROWS ONLY;

Aggregating by Document:

-- Get top documents based on best chunk match
SELECT 
  doc_id,
  MIN(VECTOR_DISTANCE(embedding, :query_vector, COSINE)) AS best_score
FROM document_chunks
GROUP BY doc_id
ORDER BY best_score
FETCH FIRST 10 ROWS ONLY;

Narrowing Search Results

Attribute Filtering with WHERE Clause

Use the WHERE clause to filter results based on metadata or business attributes. You are not limited by the use of the ORDER BY clause.

This powerful combination enables hybrid search: semantic similarity (vector search) plus exact filters (traditional SQL).

Example: Filter by Category and Date

SELECT 
  product_id,
  product_name,
  category,
  COSINE_DISTANCE(embedding, :query_vector) AS similarity
FROM products
WHERE category = 'Electronics'
  AND launch_date greater-than DATE '2024-01-01'
ORDER BY embedding less-than equals greater-than :query_vector
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

Example: Price Range Filter

SELECT 
  product_id,
  product_name,
  price,
  L2_DISTANCE(embedding, :query_vector) AS distance
FROM products
WHERE price BETWEEN 100 AND 500
  AND in_stock = 'Y'
ORDER BY embedding less-than hyphen greater-than :query_vector
FETCH APPROXIMATE FIRST 5 ROWS ONLY;

Best Practices for Filtering:

Apply filters in WHERE clause before vector search
Use indexed columns for better performance
Combine multiple conditions as needed
Leverage partition pruning for partitioned tables

Testing Other Distance Functions

Oracle provides shorthand distance functions that simplify syntax and improve code readability.

Distance Function Equivalents

L1_DISTANCE(v1, v2) is similar to Manhattan distance

SELECT L1_DISTANCE(
  VECTOR('[1, 2, 3]'),
  VECTOR('[4, 5, 6]')
) AS manhattan_dist
FROM DUAL;

L2_DISTANCE(v1, v2) is similar to Euclidean distance

SELECT L2_DISTANCE(
  VECTOR('[3, 0]'),
  VECTOR('[0, 4]')
) AS euclidean_dist
FROM DUAL;
-- Result: 5.0

COSINE_DISTANCE is the same as cosine

SELECT COSINE_DISTANCE(
  VECTOR('[1, 0]'),
  VECTOR('[1, 1]')
) AS cosine_dist
FROM DUAL;

INNER_PRODUCT(v1, v2) is the same as dot product

SELECT INNER_PRODUCT(
  VECTOR('[2, 3]'),
  VECTOR('[4, 5]')
) AS dot_product
FROM DUAL;
-- Result: 2*4 + 3*5 = 23

Note on DOT vs INNER_PRODUCT:

The VECTOR_DISTANCE function with DOT metric returns the negated inner product, while the INNER_PRODUCT function returns the actual dot product.

-- These are NOT equivalent:
SELECT VECTOR_DISTANCE(v1, v2, DOT) FROM table;     -- Returns -1 * dot_product
SELECT INNER_PRODUCT(v1, v2) FROM table;             -- Returns dot_product

Other Essential Vector Functions

1. Vector Constructors

TO_VECTOR() converts a string or a character large object (CLOB) to a vector

-- From string
SELECT TO_VECTOR('[1.5, 2.3, 4.1]') AS vec FROM DUAL;

-- From CLOB
DECLARE
  my_clob CLOB := '[0.1, 0.2, 0.3, 0.4]';
  my_vector VECTOR;
BEGIN
  my_vector := TO_VECTOR(my_clob);
END;
/

TO_VECTOR also takes another vector as input, adjusts its format, and returns the adjusted vector as output.

VECTOR() converts a string or CLOB into a vector

SELECT VECTOR('[1, 2, 3]', 3, FLOAT32) AS vec FROM DUAL;

TO_VECTOR and VECTOR are synonymous—they perform the same function.

2. Vector Serializer

VECTOR_SERIALIZE() converts a vector into a string or a CLOB

SELECT VECTOR_SERIALIZE(embedding) AS vec_string
FROM products
WHERE product_id = 1;

-- Result: '[0.12, 0.45, 0.78, ...]'

Use Cases:

Exporting vectors for external processing
Debugging and inspection
Logging and auditing
Integration with non-Oracle systems

3. Vector Norm

VECTOR_NORM() returns the Euclidean norm of a vector—the distance from the origin to the vector.

Also known as magnitude or length, it's calculated as the square root of the sum of squared components.

Formula:

norm = square_root(x1-squared + x2-squared + ... + xn-squared)

SQL Example:

SELECT VECTOR_NORM(TO_VECTOR('[3, 4]')) AS magnitude
FROM DUAL;
-- Result: 5.0 (because square_root(9+16) = 5)

Practical Use: Normalizing Vectors

-- Normalize vector to unit length
SELECT 
  embedding / VECTOR_NORM(embedding) AS normalized_vector
FROM products
WHERE product_id = 1;

Normalized vectors have a magnitude of 1.0, which is required for meaningful dot product similarity comparisons.

4. Vector Dimension Count

VECTOR_DIMENSION_COUNT() returns the number of dimensions of a vector

SELECT VECTOR_DIMENSION_COUNT(embedding) AS dimensions
FROM products
WHERE product_id = 1;

-- Result: 768 (for BERT-style embeddings)

Use Cases:

Validating embedding dimensions
Debugging dimension mismatches
Dynamic schema inspection
Migration validation

5. Vector Dimension Format

VECTOR_DIMENSION_FORMAT() returns the storage format of the vector

SELECT VECTOR_DIMENSION_FORMAT(embedding) AS format
FROM products
WHERE product_id = 1;

-- Possible results: 'INT8', 'FLOAT32', 'FLOAT64', 'BINARY'

Use Cases:

Schema documentation
Storage optimization analysis
Migration planning
Model compatibility verification

Complete Working Example

Here's a comprehensive example demonstrating all three search types:

-- Create table
CREATE TABLE research_papers (
  paper_id NUMBER PRIMARY KEY,
  title VARCHAR2(500),
  abstract CLOB,
  category VARCHAR2(100),
  publish_date DATE,
  embedding VECTOR(768, FLOAT32)
);

-- Insert sample data
INSERT INTO research_papers VALUES (
  1,
  'Advances in Vector Search',
  'This paper explores efficient algorithms...',
  'Computer Science',
  DATE '2024-06-15',
  VECTOR('[0.1, 0.2, ...]', 768, FLOAT32)
);
COMMIT;

-- 1. EXACT SIMILARITY SEARCH
-- Most accurate, slower for large datasets
SELECT 
  paper_id,
  title,
  COSINE_DISTANCE(embedding, :query_vector) AS similarity
FROM research_papers
WHERE category = 'Computer Science'
ORDER BY similarity
FETCH FIRST 10 ROWS ONLY;

-- 2. CREATE VECTOR INDEX for approximate search
CREATE VECTOR INDEX papers_hnsw_idx
ON research_papers(embedding)
ORGANIZATION NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- 3. APPROXIMATE SIMILARITY SEARCH
-- Faster, 95%+ accuracy
SELECT 
  paper_id,
  title,
  publish_date
FROM research_papers
WHERE category = 'Computer Science'
  AND publish_date greater-than DATE '2024-01-01'
ORDER BY embedding less-than equals greater-than :query_vector
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

-- 4. MULTI-VECTOR SEARCH (for chunked documents)
-- Create chunked version
CREATE TABLE paper_chunks (
  paper_id NUMBER,
  chunk_id NUMBER,
  chunk_text CLOB,
  embedding VECTOR(768, FLOAT32),
  PRIMARY KEY (paper_id, chunk_id)
);

-- Find best matching chunks
SELECT 
  paper_id,
  chunk_id,
  L2_DISTANCE(embedding, :query_vector) AS distance
FROM paper_chunks
ORDER BY distance
FETCH APPROXIMATE FIRST 20 ROWS ONLY;

-- Aggregate to document level
SELECT 
  p.paper_id,
  p.title,
  MIN(COSINE_DISTANCE(c.embedding, :query_vector)) AS best_match_score
FROM research_papers p
JOIN paper_chunks c ON p.paper_id = c.paper_id
GROUP BY p.paper_id, p.title
ORDER BY best_match_score
FETCH FIRST 10 ROWS ONLY;

Best Practices

1. Choose the Right Search Strategy

Exact search: Less than 100K vectors, need perfect accuracy
Approximate with HNSW: 100K to 10M vectors, need sub-100ms latency, have memory available
Approximate with IVF: 10M+ vectors, limited memory, can tolerate slightly higher latency
Multi-vector: Document chunks, multi-modal data, detailed granularity needed

2. Optimize Index Configuration

-- For high accuracy (slower but better results)
CREATE VECTOR INDEX idx1 ON table(embedding)
ORGANIZATION NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 99;

-- For speed (faster but slightly lower accuracy)
CREATE VECTOR INDEX idx2 ON table(embedding)
ORGANIZATION NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 90;

3. Use Attribute Filtering Effectively

Combine vector similarity with traditional filters to narrow results:

-- Efficient: Filter first, then search
SELECT * FROM products
WHERE price less-than 100              -- Traditional filter
  AND category = 'Electronics'         -- Traditional filter
ORDER BY embedding less-than equals greater-than :query_vector  -- Vector search
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

4. Monitor and Tune Performance

-- Check if index is being used
EXPLAIN PLAN FOR
SELECT * FROM products
ORDER BY embedding less-than equals greater-than :query_vector
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY);

5. Handle Dimension Mismatches

-- Validate dimensions before comparison
SELECT 
  VECTOR_DIMENSION_COUNT(embedding) AS dims,
  VECTOR_DIMENSION_FORMAT(embedding) AS format
FROM products
WHERE VECTOR_DIMENSION_COUNT(embedding) != 768;

Common Pitfalls and Solutions

Pitfall 1: Forgetting to Enable Vector Pool

Problem: HNSW index creation fails with memory error

Solution: Configure vector pool before creating HNSW indexes

ALTER SYSTEM SET vector_memory_size = 1G SCOPE=SPFILE;
-- Restart database

Pitfall 2: Using Wrong Index Type

Problem: Slow performance despite having an index

Solution: Match your workload to index type:

HNSW for read-heavy, memory-available scenarios
IVF for write-heavy or memory-constrained scenarios

Pitfall 3: Metric Mismatch

Problem: Index not being used despite proper syntax

Solution: Ensure query metric matches index metric

-- Index uses COSINE
CREATE VECTOR INDEX idx ON table(embedding)
DISTANCE COSINE;

-- Query must also use COSINE
-- Correct: Uses less-than equals greater-than (COSINE operator)
ORDER BY embedding less-than equals greater-than :query_vector

-- Wrong: Uses less-than hyphen greater-than (EUCLIDEAN operator)
-- This will NOT use the index!
ORDER BY embedding less-than hyphen greater-than :query_vector

Pitfall 4: Not Using FETCH APPROXIMATE

Problem: Query is slow despite having vector index

Solution: Use FETCH APPROXIMATE to enable index usage

-- Without APPROXIMATE - performs exact search
FETCH FIRST 10 ROWS ONLY;

-- With APPROXIMATE - uses index
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

Oracle Database 23ai provides a comprehensive vector search framework with multiple strategies to fit different use cases:

Search Strategies:

Exact: Perfect accuracy, suitable for smaller datasets
Approximate: 95%+ accuracy with dramatic speed improvements using HNSW or IVF indexes
Multi-vector: Chunk-level granularity for document search

Key Requirements:

Enable vector pool for HNSW indexes: vector_memory_size
Use FETCH APPROXIMATE for index-accelerated queries
Match distance metrics between index and query

Essential Functions:

Constructors: TO_VECTOR(), VECTOR()
Serializer: VECTOR_SERIALIZE()
Utilities: VECTOR_NORM(), VECTOR_DIMENSION_COUNT(), VECTOR_DIMENSION_FORMAT()
Distance Functions: L1_DISTANCE(), L2_DISTANCE(), COSINE_DISTANCE(), INNER_PRODUCT()

Best Practices:

Choose search strategy based on dataset size and accuracy requirements
Combine vector search with WHERE clause filters for powerful hybrid queries
Monitor index usage with EXPLAIN PLAN
Normalize vectors when using dot product similarity

By understanding these search strategies and vector functions, you can build high-performance semantic search applications that scale from thousands to billions of vectors while maintaining excellent accuracy and query speed.

Oracle Database 23ai: Creating Vectors and Understanding Distance Metrics for Similarity Search

Mon, 09 Mar 2026 08:00:00 +0000

Oracle Database 23ai introduces native vector capabilities that enable semantic search directly within SQL. Understanding how to create vectors, calculate distances, and choose appropriate metrics is fundamental to building effective AI-powered applications. This comprehensive guide explores vector operations in Oracle 23ai with practical examples and best practices.

The VECTOR Data Type

Oracle 23ai introduces a native VECTOR data type designed specifically to store and manage vector embeddings efficiently within the database.

Declaration Syntax:

-- Flexible: any dimensions and format
embedding VECTOR

-- Specific dimensions, flexible format
embedding VECTOR(512)

-- Fully specified: dimensions and format
embedding VECTOR(512, FLOAT32)

Format Options:

INT8: 8-bit integers
FLOAT32: 32-bit floating-point (IEEE standard, most common)
FLOAT64: 64-bit floating-point (IEEE standard, higher precision)
BINARY: Binary vectors for specialized use cases

Oracle Database automatically casts values if needed between formats.

Creating Vectors with the VECTOR Constructor

The VECTOR constructor is a function that allows us to create vectors without storing them. It's particularly useful for learning purposes, testing, and ad-hoc queries.

Basic Syntax

VECTOR(array_literal [, number_of_dimensions] [, format])

Parameters:

array_literal: String representation of vector values
number_of_dimensions: Optional, inferred from array if not specified
format: Optional (INT8, FLOAT32, FLOAT64, BINARY)

Examples

Simple 2D Vector:

SELECT VECTOR('[0, 0]');

Output: [0, 0]

Vector with Scientific Notation:

SELECT VECTOR('[10, 0]');

Output: [1.0E+001, 0] (equivalent to [10, 0])

Specifying Dimensions:

SELECT VECTOR('[1, 2, 3]', 3, FLOAT32);

Creating Vectors from Variables:

DECLARE
  my_vector VECTOR;
BEGIN
  my_vector := VECTOR('[0.5, 0.8, 0.2]');
  DBMS_OUTPUT.PUT_LINE(my_vector);
END;
/

Best Practices for Learning

Use small number of dimensions (2-4) when learning vector concepts, as they're easier to visualize and understand. For production, match the dimensions to your embedding model (typically 384-1536).

The VECTOR_DISTANCE Function

VECTOR_DISTANCE is the main function for calculating distance between two vectors. It allows you to calculate distance between two parameters and therefore takes 2 vectors as params.

Syntax

VECTOR_DISTANCE(expr1, expr2 [, distance_metric])

Parameters:

expr1, expr2: Two vector expressions to compare
distance_metric: Optional; defaults to COSINE (or HAMMING for BINARY vectors)

Returns: BINARY_DOUBLE representing the distance

Default Behavior

If you do not specify a distance metric, then the default distance metric is cosine. If the input vectors are BINARY vectors, the default metric is hamming.

Basic Example

SELECT 
  VECTOR_DISTANCE(
    VECTOR('[1, 0]'),
    VECTOR('[0, 1]'),
    COSINE
  ) AS distance;

Vector Distance Metrics: Choosing the Right One

Oracle AI Vector Search supports multiple distance metrics, each suited for different use cases and data characteristics.

1. Euclidean Distance (L2)

Euclidean distance gives us the straight-line distance between two vectors. It uses the Pythagorean theorem and is sensitive to both vector size and direction.

Formula:

d = square_root[(x2-x1)squared + (y2-y1)squared]

SQL Example:

SELECT TO_NUMBER(
  VECTOR_DISTANCE(
    VECTOR('[3, 0]'),
    VECTOR('[0, 4]'),
    EUCLIDEAN
  )
) AS distance;

Result: 5 (forming a 3-4-5 right triangle)

Use Cases:

Spatial data and geographical coordinates
Physical measurements
Image similarity (pixel-level comparisons)
When absolute distance matters

2. Euclidean Squared Distance (L2_SQUARED)

The Euclidean distance without taking the square root. When ordering is more important than the actual distance values, squared Euclidean distance is very useful as it's faster to calculate, avoiding the square root computation.

Advantage: Faster computation for ranking/sorting

SQL Example:

SELECT TO_NUMBER(
  VECTOR_DISTANCE(
    VECTOR('[3, 0]'),
    VECTOR('[0, 4]'),
    EUCLIDEAN_SQUARED
  )
) AS distance_squared;

Result: 25 (5 squared)

3. Cosine Similarity / Cosine Distance

Cosine similarity is the most widely used similarity metric, especially in natural language processing (NLP). The smaller the angle, the more similar the vectors are.

Cosine measures the angle between two vectors rather than their magnitude. It's ideal for text embeddings where direction matters more than magnitude.

Formula:

cosine_similarity = (A dot B) / (magnitude_A times magnitude_B)
cosine_distance = 1 - cosine_similarity

SQL Example:

SELECT 
  VECTOR_DISTANCE(
    VECTOR('[1, 0]'),
    VECTOR('[1, 1]'),
    COSINE
  ) AS cosine_dist;

Key Characteristics:

Range: 0 (identical) to 2 (opposite)
Normalized: Insensitive to vector magnitude
Best for: Text embeddings, document similarity, semantic search

Why It's Popular:

Cosine is one of the most useful metrics since it measures the angle between two vectors instead of the difference in size or position. This makes it perfect for comparing text embeddings where the semantic meaning is encoded in direction, not magnitude.

4. Dot Product Similarity

Dot product allows us to multiply the size of each vector by the cosine of their angle. It is equivalent to the sum of the vectors' coordinates. Larger values mean more similar; smaller values mean less similar.

Formula:

dot_product = sum(ai times bi)

Note: Oracle's DOT metric calculates the negated dot product, so more negative values indicate greater similarity.

SQL Example:

SELECT 
  VECTOR_DISTANCE(
    VECTOR('[2, 3]'),
    VECTOR('[4, 5]'),
    DOT
  ) AS dot_distance;

Use Cases:

Recommendation systems
Similarity ranking with normalized vectors
Fast approximate nearest neighbor search

Important: For meaningful results, vectors should be normalized to unit length.

5. Manhattan Distance (L1)

Manhattan distance is useful for describing uniform grids. It's useful for city blocks, power grids, chessboards, and is faster than Euclidean metrics.

Also known as taxicab distance or L1 distance, it calculates the sum of absolute differences between vector components.

Formula:

manhattan = sum of absolute(ai - bi)

SQL Example:

SELECT 
  VECTOR_DISTANCE(
    VECTOR('[1, 2]'),
    VECTOR('[4, 6]'),
    MANHATTAN
  ) AS manhattan_dist;

Result: absolute(4-1) + absolute(6-2) = 7

Use Cases:

Grid-based problems
Route planning on city streets
Feature selection in machine learning
When diagonal movement isn't allowed

6. Hamming Distance

Hamming distance describes where vector dimensions differ. They are binary vectors and tell us the number of bits that require change to match.

Hamming distance computes the position of each bit in the sequence and is used for network error detection and correction.

SQL Example:

SELECT 
  VECTOR_DISTANCE(
    VECTOR('[1, 0, 1, 1, 0]', 5, BINARY),
    VECTOR('[1, 1, 1, 0, 0]', 5, BINARY),
    HAMMING
  ) AS hamming_dist;

Result: 2 (positions 2 and 4 differ)

Use Cases:

Error detection and correction
Genetic sequence comparison
Digital communication
Binary classification problems

7. Jaccard Distance

Jaccard distance measures dissimilarity between binary vectors based on the ratio of intersection to union.

Requirements: Both vectors must be BINARY

Formula:

jaccard = 1 - (size_of_intersection / size_of_union)

SQL Example:

SELECT 
  VECTOR_DISTANCE(
    VECTOR('[1, 1, 0, 1]', 4, BINARY),
    VECTOR('[1, 0, 1, 1]', 4, BINARY),
    JACCARD
  ) AS jaccard_dist;

Use Cases:

Set similarity comparison
Document deduplication
Recommendation systems
Clustering binary data

Shorthand Distance Operators

Oracle provides convenient shorthand operators for common distance calculations, making SQL queries more concise and readable.

Available Operators

Operator for Euclidean Distance: less-than hyphen greater-than

SELECT '[1, 2]' <-> '[0, 1]' AS euclidean_dist;

Equivalent to:

SELECT L2_DISTANCE(VECTOR('[1, 2]'), VECTOR('[0, 1]')) AS euclidean_dist;

SELECT VECTOR_DISTANCE(VECTOR('[1, 2]'), VECTOR('[0, 1]'), EUCLIDEAN) AS euclidean_dist;

Operator for Cosine Distance: less-than equals greater-than

SELECT '[1, 0]' <=> '[0, 1]' AS cosine_dist;

Equivalent to:

SELECT COSINE_DISTANCE(VECTOR('[1, 0]'), VECTOR('[0, 1]')) AS cosine_dist;

Operator for Negative Dot Product: less-than hash greater-than

SELECT '[2, 3]' <#> '[4, 5]' AS neg_dot_product;

Equivalent to:

SELECT -1 * INNER_PRODUCT(VECTOR('[2, 3]'), VECTOR('[4, 5]')) AS neg_dot_product;

Practical Example

-- Compare products by embedding similarity
SELECT 
  p1.product_name,
  p2.product_name,
  p1.embedding <=> p2.embedding AS similarity_score
FROM products p1, products p2
WHERE p1.product_id = 100
  AND p2.product_id != 100
ORDER BY similarity_score
FETCH FIRST 5 ROWS ONLY;

Shorthand Distance Functions

In addition to operators, Oracle provides shorthand functions for cleaner code:

L1_DISTANCE - Manhattan distance
L2_DISTANCE - Euclidean distance
COSINE_DISTANCE - Cosine distance
INNER_PRODUCT - Dot product (not negated)
HAMMING_DISTANCE - Hamming distance
JACCARD_DISTANCE - Jaccard distance

Example:

SELECT 
  L2_DISTANCE(v1.embedding, v2.embedding) AS euclidean,
  COSINE_DISTANCE(v1.embedding, v2.embedding) AS cosine,
  INNER_PRODUCT(v1.embedding, v2.embedding) AS dot_prod
FROM vectors v1, vectors v2
WHERE v1.id = 1 AND v2.id = 2;

Performing Similarity Search

The VECTOR_DISTANCE function can be used to perform similarity search by ordering results based on vector proximity.

Exact Similarity Search

SELECT 
  id,
  description,
  VECTOR_DISTANCE(embedding, :query_vector, COSINE) AS distance
FROM documents
ORDER BY distance
FETCH FIRST 10 ROWS ONLY;

Approximate Similarity Search (with Index)

-- Using FETCH APPROXIMATE with vector index
SELECT 
  id,
  description
FROM documents
ORDER BY embedding <=> :query_vector
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

Key Difference:

EXACT: Compares query vector with every vector (slower, 100% accurate)
APPROXIMATE: Uses vector indexes (HNSW/IVF) for fast search with approximately 95% accuracy

Choosing the Right Distance Metric

Decision Matrix

Use Case	Recommended Metric	Why
Text embeddings	COSINE	Captures semantic similarity, magnitude-invariant
Image similarity	EUCLIDEAN	Pixel-level comparisons benefit from absolute distance
Recommendation systems	DOT (normalized vectors)	Fast computation, works well with normalized data
Grid/route problems	MANHATTAN	Natural fit for grid-based navigation
Binary classification	HAMMING	Direct bit difference counting
Error detection	HAMMING	Counts differing positions
Set similarity	JACCARD	Measures intersection/union ratio

Performance Considerations

Fastest to Slowest:

DOT (simple multiplication and sum)
EUCLIDEAN_SQUARED (avoids square root)
MANHATTAN (absolute values and sum)
EUCLIDEAN (includes square root)
COSINE (normalization overhead)

Matching Index and Query Metrics

If a similarity search query specifies a distance metric that conflicts with the metric in a vector index, the vector index is not used and an exact search is performed instead.

Best Practice: Ensure your query metric matches your index metric for optimal performance.

-- Index created with COSINE
CREATE VECTOR INDEX docs_idx ON documents(embedding)
ORGANIZATION NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- Query should also use COSINE for index usage
SELECT id FROM documents
ORDER BY embedding <=> :query_vector  -- Uses COSINE operator
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

Complete Working Example

Here's a comprehensive example demonstrating vector creation, storage, and similarity search:

-- Create table with vector column
CREATE TABLE product_embeddings (
  product_id NUMBER PRIMARY KEY,
  product_name VARCHAR2(100),
  description CLOB,
  embedding VECTOR(384, FLOAT32)
);

-- Insert sample data
INSERT INTO product_embeddings VALUES (
  1,
  'Laptop Computer',
  'High-performance laptop for developers',
  VECTOR('[0.2, 0.8, 0.5, ...]', 384, FLOAT32)
);

INSERT INTO product_embeddings VALUES (
  2,
  'Wireless Mouse',
  'Ergonomic wireless mouse',
  VECTOR('[0.1, 0.3, 0.7, ...]', 384, FLOAT32)
);

COMMIT;

-- Create vector index for fast similarity search
CREATE VECTOR INDEX product_emb_idx 
ON product_embeddings(embedding)
ORGANIZATION NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- Perform similarity search
DECLARE
  query_vec VECTOR;
BEGIN
  -- Create query vector (in practice, from embedding model)
  query_vec := VECTOR('[0.15, 0.75, 0.6, ...]', 384, FLOAT32);
  
  -- Find similar products
  FOR rec IN (
    SELECT 
      product_name,
      description,
      COSINE_DISTANCE(embedding, query_vec) AS similarity
    FROM product_embeddings
    ORDER BY embedding <=> query_vec
    FETCH APPROXIMATE FIRST 5 ROWS ONLY
  ) LOOP
    DBMS_OUTPUT.PUT_LINE(
      'Product: ' || rec.product_name || 
      ' | Similarity: ' || rec.similarity
    );
  END LOOP;
END;
/

Best Practices

1. Choose Appropriate Dimensions

Match embedding dimensions to your model:

384: MiniLM, lightweight models
768: BERT, sentence transformers
1024: Cohere embedding models
1536: OpenAI ada-002

2. Normalize Vectors When Using DOT

-- Normalize vector to unit length
SELECT 
  VECTOR_NORM(embedding) AS original_magnitude,
  embedding / VECTOR_NORM(embedding) AS normalized_vector
FROM documents
WHERE id = 1;

3. Use Appropriate Formats

FLOAT32: Default, balances precision and performance
FLOAT64: When high precision is critical
INT8: For quantized models, saves storage

4. Monitor Index Accuracy

-- Check if index is being used
EXPLAIN PLAN FOR
SELECT id FROM documents
ORDER BY embedding <=> :query_vector
FETCH APPROXIMATE FIRST 10 ROWS ONLY;

SELECT * FROM TABLE(DBMS_XPLAN.DISPLAY);

5. Benchmark Different Metrics

Test multiple metrics on your data to find the best performer:

-- Compare metrics
SELECT 
  'COSINE' AS metric,
  AVG(COSINE_DISTANCE(e1.embedding, e2.embedding)) AS avg_dist
FROM embeddings e1, embeddings e2
WHERE e1.id != e2.id
UNION ALL
SELECT 
  'EUCLIDEAN',
  AVG(L2_DISTANCE(e1.embedding, e2.embedding))
FROM embeddings e1, embeddings e2
WHERE e1.id != e2.id;

Common Pitfalls and Solutions

Pitfall 1: Dimension Mismatch

Problem: Comparing vectors with different dimensions

-- This will error
SELECT VECTOR('[1, 2]') <=> VECTOR('[1, 2, 3]');

Solution: Ensure all vectors have the same dimensions

Pitfall 2: Wrong Metric for Data Type

Problem: Using JACCARD on non-binary vectors

Solution: Use JACCARD only with BINARY vectors

Pitfall 3: Not Using Indexes

Problem: Slow similarity searches on large datasets

Solution: Create appropriate vector indexes (HNSW for speed, IVF for scale)

Pitfall 4: Metric Mismatch with Index

Problem: Query metric conflicts with index metric, causing full table scan

Solution: Match query metric to index metric

Conclusion

Oracle Database 23ai's native vector capabilities provide a powerful, integrated platform for semantic search and AI-powered applications. Key takeaways:

Vector Creation:

VECTOR constructor for creating vectors directly in SQL
Support for multiple formats (INT8, FLOAT32, FLOAT64, BINARY)
Flexible dimension specification

Distance Metrics:

COSINE: Default, best for text embeddings and semantic similarity
EUCLIDEAN: Straight-line distance, good for spatial data
DOT: Fast for normalized vectors, recommendation systems
MANHATTAN: Grid-based problems, faster than Euclidean
HAMMING: Binary vectors, error detection
JACCARD: Set similarity with binary vectors

Shorthand Operators:

Euclidean distance operator: less-than hyphen greater-than
Cosine distance operator: less-than equals greater-than
Negative dot product operator: less-than hash greater-than

Best Practices:

Match metric to your use case and embedding model
Create appropriate vector indexes
Ensure metric consistency between index and queries
Use approximate search for large datasets
Benchmark different metrics on your data

By understanding these vector operations and distance metrics, you can build efficient, accurate similarity search applications entirely within Oracle Database 23ai.

Oracle AI Vector Search: DML and DDL Operations on Vector Columns

Thu, 26 Feb 2026 08:19:03 +0000

Oracle Database 23ai introduces the VECTOR data type, enabling you to store AI embeddings alongside traditional business data. Understanding how to perform Data Manipulation Language (DML) and Data Definition Language (DDL) operations on vector columns is essential for building effective AI-powered applications. This guide covers everything you need to know about working with vector columns, including supported operations and important restrictions.

Understanding the VECTOR Data Type

The VECTOR data type can store vectors with:

Arbitrary number of dimensions: From 1 to 65,535 dimensions (65,528 for BINARY format)
Multiple formats: INT8, FLOAT32, FLOAT64, or BINARY
Flexible or fixed specifications: Can be declared with or without constraints

Declaration Examples:

-- Flexible: any dimensions and format
CREATE TABLE flexible_vectors (
    id NUMBER,
    embedding VECTOR
);

-- Fixed: specific dimensions and format
CREATE TABLE fixed_vectors (
    id NUMBER,
    embedding VECTOR(384, FLOAT32)
);

DML Operations on Vectors

1. INSERT Operations

You can directly insert vectors into tables using several methods:

Method A: Insert Using Vector Literals

-- Insert vector as array literal
INSERT INTO products (product_id, name, embedding)
VALUES (1, 'Laptop', '[0.1, 0.8, 0.5, 0.3]');

-- Insert with TO_VECTOR function
INSERT INTO products (product_id, name, embedding)
VALUES (2, 'Mouse', TO_VECTOR('[0.2, 0.7, 0.4, 0.6]'));

Method B: Insert with Embedding Generation

-- Generate embeddings during insert
INSERT INTO products (product_id, name, description, embedding)
VALUES (
    3,
    'Keyboard',
    'Mechanical gaming keyboard with RGB',
    VECTOR_EMBEDDING(doc_model USING 'Mechanical gaming keyboard with RGB' AS data)
);

Method C: Insert from Another Table

-- Copy vectors from another table
INSERT INTO products_archive (product_id, name, embedding)
SELECT product_id, name, embedding
FROM products
WHERE created_date < ADD_MONTHS(SYSDATE, -12);

2. UPDATE Operations

You can directly update vector columns:

-- Update with vector literal
UPDATE products
SET embedding = '[0.15, 0.85, 0.55, 0.35]'
WHERE product_id = 1;

-- Update using embedding generation
UPDATE products
SET embedding = VECTOR_EMBEDDING(doc_model USING description AS data)
WHERE embedding IS NULL;

-- Conditional update
UPDATE products
SET embedding = VECTOR_EMBEDDING(doc_model USING description AS data)
WHERE category = 'Electronics'
    AND embedding IS NULL;

3. DELETE Operations

Delete operations work normally with tables containing vector columns:

-- Delete specific rows
DELETE FROM products
WHERE product_id = 100;

-- Delete based on relational criteria
DELETE FROM products
WHERE created_date < ADD_MONTHS(SYSDATE, -24)
    AND status = 'ARCHIVED';

-- Delete all rows
DELETE FROM products;
-- or
TRUNCATE TABLE products;

4. Loading Data Using SQL*Loader

SQL*Loader can load vector data from external files:

Control File Example (products.ctl):

LOAD DATA
INFILE 'products.dat'
INTO TABLE products
FIELDS TERMINATED BY ','
(
    product_id,
    product_name,
    description,
    embedding CHAR(4000)
)

Data File Example (products.dat):

1,Laptop,High-performance laptop,"[0.1,0.8,0.5,0.3,0.6]"
2,Mouse,Wireless gaming mouse,"[0.2,0.7,0.4,0.6,0.5]"
3,Keyboard,Mechanical keyboard,"[0.15,0.85,0.55,0.35,0.65]"

Load Command:

sqlldr userid=username/password@db control=products.ctl log=products.log

5. DML on Tables with HNSW Indexes

Important Update (23ai Release 23.6+):

In Oracle Database 23ai releases 23.4 and 23.5, DML operations were not allowed on tables with HNSW indexes. Starting with Release 23.6, this restriction has been lifted:

Transactional consistency: DML modifications are now supported on tables with HNSW indexes
RAC support: Guarantees transactional consistency even in Oracle RAC environments
Consistent results: Vector search queries using HNSW indexes see transactionally consistent results based on their read snapshot

-- These operations now work with HNSW indexes (23.6+)
INSERT INTO products VALUES (101, 'New Product', '[0.1, 0.2, 0.3]');
UPDATE products SET embedding = '[0.2, 0.3, 0.4]' WHERE product_id = 101;
DELETE FROM products WHERE product_id = 101;

Vector DDL Operations

Tables with Multiple Vector Columns

Tables can have:

More than one column of VECTOR data type
Different formats and dimensions in different columns

CREATE TABLE multimedia_content (
    content_id NUMBER PRIMARY KEY,
    title VARCHAR2(500),
    description CLOB,
    -- Text embedding: 384 dimensions, FLOAT32
    text_embedding VECTOR(384, FLOAT32),
    -- Image embedding: 512 dimensions, FLOAT32
    image_embedding VECTOR(512, FLOAT32),
    -- Audio embedding: 256 dimensions, INT8
    audio_embedding VECTOR(256, INT8),
    -- Flexible dimension column
    metadata_embedding VECTOR
);

Adding Vector Columns

You can add vector columns to existing tables:

-- Add vector column with specific dimensions
ALTER TABLE products
ADD description_vector VECTOR(384, FLOAT32);

-- Add flexible vector column
ALTER TABLE customers
ADD preference_vector VECTOR;

-- Add with default NULL
ALTER TABLE documents
ADD content_vector VECTOR(768, FLOAT32) DEFAULT NULL;

After Adding, Populate the Column:

-- Generate embeddings for existing data
UPDATE products
SET description_vector = VECTOR_EMBEDDING(doc_model USING description AS data);

Dropping Vector Columns

-- Drop a specific vector column
ALTER TABLE products
DROP COLUMN description_vector;

-- Drop multiple columns
ALTER TABLE products
DROP (description_vector, image_vector);

Dropping Tables with Vector Columns

Tables containing vector columns can be dropped normally:

-- Drop table
DROP TABLE products;

-- Drop table with CASCADE CONSTRAINTS
DROP TABLE products CASCADE CONSTRAINTS;

-- Drop and purge from recycle bin
DROP TABLE products PURGE;

Prohibited Operations on Vector Columns

Oracle Database 23ai has specific restrictions on where and how vector columns can be used. Understanding these limitations is crucial for proper database design.

1. Table and Storage Restrictions

Cannot Define Vector Columns In:

External Tables:

-- This will fail
CREATE TABLE ext_vectors (
    id NUMBER,
    embedding VECTOR
)
ORGANIZATION EXTERNAL (...);

Note: As of Oracle Database 26ai, external tables CAN be created with VECTOR columns, allowing vector embeddings in text or binary format stored in external files to be rendered as the VECTOR data type. Check your database version for availability.

Index-Organized Tables (IOTs):

-- Cannot use as primary key
CREATE TABLE iot_vectors (
    embedding VECTOR PRIMARY KEY,  -- Not allowed
    data VARCHAR2(100)
)
ORGANIZATION INDEX;

-- Cannot use as non-key column either
CREATE TABLE iot_vectors (
    id NUMBER PRIMARY KEY,
    embedding VECTOR,  -- Not allowed
    data VARCHAR2(100)
)
ORGANIZATION INDEX;

Clusters and Cluster Tables:

-- Vectors cannot be part of clusters
CREATE CLUSTER vector_cluster (
    id NUMBER,
    embedding VECTOR  -- Not allowed
);

Global Temporary Tables:

-- This will fail
CREATE GLOBAL TEMPORARY TABLE temp_vectors (
    id NUMBER,
    embedding VECTOR  -- Not allowed
)
ON COMMIT DELETE ROWS;

Manual Segment Space Management (MSSM) Tablespaces:

Only the SYS user can create vectors as BasicFiles in MSSM tablespaces. Regular users should use Automatic Segment Space Management (ASSM) tablespaces:

-- Create ASSM tablespace for vectors
CREATE TABLESPACE vector_data
DATAFILE '/u01/app/oracle/oradata/vector_data01.dbf' SIZE 1G
SEGMENT SPACE MANAGEMENT AUTO;  -- Required for non-SYS users

2. Partitioning Restrictions

Sub-partitioning Keys:

-- Vector columns cannot be sub-partition keys
CREATE TABLE sales_data (
    sale_id NUMBER,
    sale_date DATE,
    embedding VECTOR(384, FLOAT32)
)
PARTITION BY RANGE (sale_date)
SUBPARTITION BY HASH (embedding)  -- Not allowed
(...);

Vectors in Partitioned Tables (Allowed):

-- Vectors CAN exist in partitioned tables
CREATE TABLE sales_data (
    sale_id NUMBER,
    sale_date DATE,
    product_name VARCHAR2(200),
    embedding VECTOR(384, FLOAT32)
)
PARTITION BY RANGE (sale_date) (
    PARTITION p2023 VALUES LESS THAN (DATE '2024-01-01'),
    PARTITION p2024 VALUES LESS THAN (DATE '2025-01-01')
);

3. Constraint Restrictions

Primary Keys:

CREATE TABLE vectors (
    embedding VECTOR PRIMARY KEY  -- Not allowed
);

Foreign Keys:

CREATE TABLE vectors (
    id NUMBER PRIMARY KEY,
    embedding VECTOR
);

CREATE TABLE related (
    id NUMBER PRIMARY KEY,
    vector_ref VECTOR REFERENCES vectors(embedding)  -- Not allowed
);

Unique Constraints:

CREATE TABLE vectors (
    id NUMBER PRIMARY KEY,
    embedding VECTOR UNIQUE  -- Not allowed
);

Check Constraints:

CREATE TABLE vectors (
    id NUMBER PRIMARY KEY,
    embedding VECTOR CHECK (embedding IS NOT NULL)  -- Not allowed
);

Default Values:

CREATE TABLE vectors (
    id NUMBER PRIMARY KEY,
    embedding VECTOR DEFAULT '[0,0,0,0]'  -- Not allowed
);

4. Column Modification Restrictions

Cannot Modify Vector Column Definition:

-- Cannot change dimensions or format
ALTER TABLE products
MODIFY (embedding VECTOR(512, FLOAT64));  -- Not allowed

-- Cannot change to/from VECTOR type
ALTER TABLE products
MODIFY (embedding CLOB);  -- Not allowed

Workaround - Add New Column:

-- Instead, add a new column
ALTER TABLE products
ADD embedding_new VECTOR(512, FLOAT64);

-- Migrate data (with conversion if needed)
UPDATE products
SET embedding_new = VECTOR_EMBEDDING(model USING description AS data);

-- Drop old column
ALTER TABLE products DROP COLUMN embedding;

-- Rename new column
ALTER TABLE products RENAME COLUMN embedding_new TO embedding;

5. Index Restrictions

Non-Vector Indexes:

Vector columns cannot be part of traditional indexes:

-- B-tree index
CREATE INDEX idx_embedding ON products(embedding);  -- Not allowed

-- Bitmap index
CREATE BITMAP INDEX idx_embedding ON products(embedding);  -- Not allowed

-- Reverse key index
CREATE INDEX idx_embedding ON products(embedding) REVERSE;  -- Not allowed

-- Function-based index
CREATE INDEX idx_embedding ON products(UPPER(embedding));  -- Not allowed

Vector Indexes Only:

-- HNSW vector index
CREATE VECTOR INDEX idx_hnsw ON products(embedding)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- IVF vector index
CREATE VECTOR INDEX idx_ivf ON products(embedding)
ORGANIZATION NEIGHBOR PARTITIONS
DISTANCE COSINE
WITH TARGET ACCURACY 95;

6. Other Restrictions

Continuous Query Notification (CQN):

Vector columns are not supported in CQN queries.

Comparison Operators:

Standard comparison operators cannot be used:

-- These will fail
SELECT * FROM products WHERE embedding = '[0.1, 0.2]';
SELECT * FROM products WHERE embedding > '[0.1, 0.2]';
SELECT * FROM products WHERE embedding < '[0.1, 0.2]';

Use Vector Distance Functions Instead:

-- Correct approach
SELECT product_name
FROM products
ORDER BY VECTOR_DISTANCE(embedding, '[0.1, 0.2, 0.3]', COSINE)
FETCH FIRST 10 ROWS ONLY;

Best Practices for Vector DML and DDL

1. Design Considerations

-- Good: Specify dimensions and format upfront
CREATE TABLE products (
    product_id NUMBER PRIMARY KEY,
    name VARCHAR2(200),
    embedding VECTOR(384, FLOAT32)  -- Clear specification
);

-- Less optimal: Flexible vectors harder to optimize
CREATE TABLE products (
    product_id NUMBER PRIMARY KEY,
    name VARCHAR2(200),
    embedding VECTOR  -- May lead to performance issues
);

2. Batch Operations

-- Efficient: Batch update
UPDATE products
SET embedding = VECTOR_EMBEDDING(doc_model USING description AS data)
WHERE embedding IS NULL;

-- Less efficient: Row-by-row updates in loop

3. Transaction Management

-- Good practice: Commit after large operations
BEGIN
    INSERT INTO products_archive
    SELECT * FROM products WHERE created_date < DATE '2023-01-01';
    
    COMMIT;
END;
/

4. Use ASSM Tablespaces

-- Create tablespace with ASSM for vector tables
CREATE TABLESPACE vector_ts
DATAFILE '/u01/oradata/vector_ts01.dbf' SIZE 10G
AUTOEXTEND ON NEXT 1G MAXSIZE UNLIMITED
SEGMENT SPACE MANAGEMENT AUTO;

-- Create table in ASSM tablespace
CREATE TABLE products (
    product_id NUMBER PRIMARY KEY,
    embedding VECTOR(384, FLOAT32)
) TABLESPACE vector_ts;

5. Monitor Vector Column Usage

-- Check vector column statistics
SELECT 
    table_name,
    column_name,
    data_type,
    nullable
FROM user_tab_columns
WHERE data_type = 'VECTOR';

-- Check table size with vectors
SELECT 
    segment_name,
    segment_type,
    bytes/1024/1024 AS size_mb
FROM user_segments
WHERE segment_name = 'PRODUCTS';

Summary

DML Operations Supported:

INSERT vectors directly or with embedding generation
UPDATE vector columns
DELETE rows from tables with vectors
Load data using SQL*Loader
DML on tables with HNSW indexes (23.6+)

DDL Operations Supported:

Create tables with multiple vector columns
Different formats and dimensions per column
ADD vector columns to existing tables
DROP vector columns
DROP tables containing vectors

Key Restrictions:

External tables (except 26ai+)
Index-Organized Tables (IOTs)
Clusters and cluster tables
Global temporary tables
Sub-partitioning keys
Primary keys, foreign keys, unique constraints
Check constraints, default values
Column modification (dimensions/format)
MSSM tablespaces (non-SYS users)
Non-vector indexes (B-tree, bitmap, etc.)
Standard comparison operators (=, >, <)

Understanding these operations and restrictions ensures you can effectively design and manage vector-enabled applications in Oracle Database 23ai, combining the power of AI embeddings with traditional relational data operations.

Oracle AI Vector Search: Querying Vectors and Optimizing with Indexes

Mon, 16 Feb 2026 08:23:55 +0000

Oracle Database 23ai's AI Vector Search enables powerful semantic search capabilities, but to truly harness its potential, you need to understand how to query vector data efficiently and when to use vector indexes. This guide explores querying vectors and optimizing performance with specialized vector indexes.

Running Basic Queries on Vectors

Combining Vector and Relational Data

One of Oracle AI Vector Search's most powerful features is the ability to combine vector data with traditional relational data in a single query. This unified approach eliminates data fragmentation and allows you to perform semantic searches alongside business logic filters.

Basic Query Pattern:

-- Create table with both relational and vector data
CREATE TABLE products (
    product_id NUMBER PRIMARY KEY,
    product_name VARCHAR2(200),
    description CLOB,
    price NUMBER(10,2),
    category VARCHAR2(100),
    in_stock VARCHAR2(1),
    description_vector VECTOR(384, FLOAT32)
);

-- Insert sample data
INSERT INTO products VALUES (
    1001,
    'Wireless Headphones',
    'Premium noise-cancelling Bluetooth headphones with 30-hour battery life',
    299.99,
    'Electronics',
    'Y',
    VECTOR_EMBEDDING(doc_model USING 
        'Premium noise-cancelling Bluetooth headphones with 30-hour battery life' AS data)
);

-- Query combining vector similarity with relational filters
SELECT 
    product_name,
    price,
    category,
    VECTOR_DISTANCE(
        description_vector,
        VECTOR_EMBEDDING(doc_model USING 'wireless audio device' AS data),
        COSINE
    ) AS similarity_score
FROM products
WHERE price < 500
    AND in_stock = 'Y'
    AND category = 'Electronics'
ORDER BY similarity_score
FETCH FIRST 10 ROWS ONLY;

Selecting Vector Data

You can select vector data using the asterisk (*) keyword just like any other column:

-- Select all columns including vectors
SELECT * FROM products;

-- Select specific columns including the vector
SELECT product_id, product_name, description_vector 
FROM products;

-- Vector dimensions can be extracted
SELECT 
    product_name,
    VECTOR_DIMENSION_COUNT(description_vector) AS dimensions,
    VECTOR_DIMENSION_FORMAT(description_vector) AS format
FROM products;

Important Restriction: No Comparison Operators Between Vectors

Standard comparison operators (such as =, >, <, >=, <=, !=) are not allowed between vectors. You cannot directly compare vectors using these operators.

Invalid Operations:

-- These will cause errors
SELECT * FROM products WHERE description_vector = other_vector;
SELECT * FROM products WHERE description_vector > other_vector;
SELECT * FROM products WHERE description_vector < other_vector;

Valid Operations:

Instead, use specialized vector distance functions:

-- Correct: Use VECTOR_DISTANCE function
SELECT 
    product_name,
    VECTOR_DISTANCE(description_vector, :query_vector, COSINE) AS distance
FROM products
ORDER BY distance
FETCH FIRST 5 ROWS ONLY;

-- Correct: Use shorthand operators
SELECT product_name
FROM products
ORDER BY description_vector <=> :query_vector  -- Cosine distance
FETCH FIRST 5 ROWS ONLY;

Understanding Vector Indexes

Vector indexes are specialized data structures that dramatically improve query performance on vector data. Without indexes, every similarity search performs an exhaustive comparison against all vectors in the table—a process that becomes prohibitively slow as datasets grow.

Why Vector Indexes Matter

The Problem:

Exhaustive Search: Compares query vector against every vector in the table
Time Complexity: O(n) - grows linearly with dataset size
Real Impact: Searches that take seconds can become hours-long operations

The Solution:
Vector indexes use sophisticated techniques to reduce the search space:

Clustering: Groups similar vectors together
Partitioning: Divides the vector space into manageable regions
Neighbor Graphs: Creates efficient navigation paths through vector space

The Result:
Vector indexes greatly reduce search space, making searches appear quick and extremely efficient, often reducing search time from hours to milliseconds.

Types of Vector Indexes

Oracle AI Vector Search supports three types of vector indexes:

1. In-Memory Neighbor Graph (HNSW) Index

HNSW (Hierarchical Navigable Small World) is a graph-based index that provides the fastest query performance.

Characteristics:

In-memory only index that can require a lot of memory for large datasets
Extremely fast query performance (milliseconds)
Uses hierarchical layers for efficient navigation
Best for datasets that fit in available memory

Creating an HNSW Index:

CREATE VECTOR INDEX product_hnsw_idx ON products (description_vector)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

Memory Requirements:

To roughly determine the memory size needed to store an HNSW index, use the following formula: 1.3 number of vectors number of dimensions * size of your vector dimension type

Example Calculation:

1 million vectors × 384 dimensions × 4 bytes (FLOAT32) × 1.3 = ~2 GB

2. Neighbor Partition (IVF) Index

IVF (Inverted File Flat) is a storage-based index that balances search quality with reasonable speed.

Characteristics:

Storage-based index not constrained by memory like HNSW
Uses buffer cache and disk storage
Can be used for very large datasets and still provide excellent performance compared to exhaustive similarity search
Supports global and local partitioning

Creating an IVF Index:

CREATE VECTOR INDEX product_ivf_idx ON products (description_vector)
ORGANIZATION NEIGHBOR PARTITIONS
DISTANCE COSINE
WITH TARGET ACCURACY 95;

When to Use IVF:

Dataset is too large to fit in memory
RAM is limited or constrained
Need to support very large (billion-scale) datasets
Working with partitioned tables

3. Hybrid Vector Index

Hybrid indexes combine full-text search with semantic vector search in a single index.

Creating a Hybrid Index:

CREATE VECTOR INDEX product_hybrid_idx ON products (description)
ORGANIZATION HYBRID
PARAMETERS('EMBEDDING MODEL doc_model');

Benefits:

Single index for both keyword and semantic searches
Run textual queries, vector similarity queries, or hybrid queries
Ideal for applications needing both search approaches

Vector Pool Memory Configuration

The Vector Pool is memory allocated in SGA to store Hierarchical Navigable Small World (HNSW) vector indexes and all associated metadata. Proper configuration is essential for HNSW index performance.

Configuring Vector Memory Size

At CDB (Container Database) Level

At the CDB level, VECTOR_MEMORY_SIZE specifies the current size of the Vector Pool:

-- Set vector memory size at CDB level
ALTER SYSTEM SET VECTOR_MEMORY_SIZE = 1G SCOPE=BOTH;

Note: A database restart is required to apply this parameter change.

At PDB (Pluggable Database) Level

The maximum PDB VECTOR_MEMORY_SIZE value is limited to 70% of the PDB sga_target. PDBs inherit settings from the CDB but can specify their own maximum usage.

-- Check current SGA target
SELECT value FROM V$PARAMETER WHERE name='sga_target';

-- PDB inherits from CDB and can set maximum
-- Maximum = 70% of PDB's sga_target

Automatic Vector Pool Growth

Oracle 23ai supports automatic vector pool management:

If VECTOR_MEMORY_SIZE is set to 1 and the sga_target is greater than 0 at CDB initialization, HNSW index creation will automatically grow the vector memory pool to satisfy the new index

-- Enable automatic growth
ALTER SYSTEM SET VECTOR_MEMORY_SIZE = 1 SCOPE=SPFILE;

Memory Considerations

Important Points:

Vector indexes are stored in this pool along with vector metadata
HNSW indexes require significant RAM
RAM constrains the vector index size
IVF indexes use buffer cache and disk, not heavily dependent on Vector Pool
IVF indexes use memory either from the vector pool if defined or the shared pool to speed up index creation and during DML operations

Monitoring Vector Pool:

-- Check vector pool allocation and usage
SELECT 
    pool,
    alloc_bytes,
    used_bytes,
    populate_status
FROM V$VECTOR_MEMORY_POOL;

-- Check individual index memory usage
SELECT 
    owner,
    index_name,
    index_organization,
    allocated_bytes,
    used_bytes,
    num_vectors
FROM V$VECTOR_INDEX;

Choosing the Right Index Type

Decision Matrix

Factor	HNSW	IVF	Hybrid
Speed	Fastest (milliseconds)	Fast (seconds)	Medium
Memory	High (in-memory)	Low (disk-based)	Medium
Accuracy	Highest (95-99%+)	High (90-99%)	Variable
Dataset Size	Small to Medium	Large to Huge	Medium
Best For	Speed-critical apps	Memory-constrained environments	Mixed query types

Practical Guidelines

Use HNSW When:

Dataset fits comfortably in available memory
Query speed is critical (real-time applications)
You can allocate sufficient Vector Pool memory
Working with up to millions of vectors

Use IVF When:

Dataset is too large for available memory
Working with billions of vectors
RAM is limited or expensive
Using partitioned tables for very large datasets

Use Hybrid When:

Application needs both keyword and semantic search
Users perform mixed query types
Want to simplify index management

Performance Optimization Tips

1. Index Configuration

Target Accuracy:

-- Higher accuracy = slower but more precise
WITH TARGET ACCURACY 99;  -- Very precise

-- Lower accuracy = faster but less precise  
WITH TARGET ACCURACY 90;  -- Still good for most cases

Balance accuracy with performance needs. TARGET ACCURACY between 90-99% provides excellent results.

2. Memory Allocation

-- Allocate adequate memory for HNSW indexes
-- Use formula: 1.3 × vectors × dimensions × format_size

-- Example for 1M vectors, 384 dimensions, FLOAT32:
-- 1.3 × 1,000,000 × 384 × 4 = ~2 GB
ALTER SYSTEM SET VECTOR_MEMORY_SIZE = 2G SCOPE=BOTH;

3. Query Optimization

-- Use FETCH FIRST for better performance
SELECT product_name, similarity_score
FROM (
    SELECT 
        product_name,
        VECTOR_DISTANCE(description_vector, :query_vec, COSINE) AS similarity_score
    FROM products
    WHERE category = 'Electronics'  -- Apply filters early
)
ORDER BY similarity_score
FETCH FIRST 10 ROWS ONLY;  -- Limit results

4. Approximate vs. Exact Search

-- Approximate search (uses index, faster)
SELECT product_name
FROM products
ORDER BY VECTOR_DISTANCE(description_vector, :query_vec, COSINE)
FETCH FIRST 10 ROWS ONLY;

-- Exact search (no index, 100% accurate, slower)
SELECT product_name
FROM products
ORDER BY VECTOR_DISTANCE(description_vector, :query_vec, COSINE)
FETCH EXACT FIRST 10 ROWS ONLY;

Use approximate search for most queries; reserve exact search for critical accuracy requirements.

Real-World Example: E-commerce Product Search

-- Step 1: Create table
CREATE TABLE products (
    product_id NUMBER PRIMARY KEY,
    name VARCHAR2(500),
    description CLOB,
    price NUMBER(10,2),
    category VARCHAR2(100),
    rating NUMBER(3,2),
    reviews_count NUMBER,
    desc_vector VECTOR(384, FLOAT32)
);

-- Step 2: Configure memory (1M products × 384 dims × 4 bytes × 1.3 = ~2GB)
ALTER SYSTEM SET VECTOR_MEMORY_SIZE = 2G SCOPE=BOTH;

-- Step 3: Create HNSW index
CREATE VECTOR INDEX products_hnsw_idx ON products (desc_vector)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- Step 4: Query with combined filters
SELECT 
    name,
    price,
    rating,
    VECTOR_DISTANCE(
        desc_vector,
        VECTOR_EMBEDDING(model USING 'comfortable running shoes' AS data),
        COSINE
    ) AS relevance
FROM products
WHERE price BETWEEN 50 AND 200
    AND category = 'Footwear'
    AND rating >= 4.0
    AND reviews_count > 100
ORDER BY relevance
FETCH FIRST 10 ROWS ONLY;

Key Takeaways

Queries can combine vector and relational data in a single SQL statement, eliminating data fragmentation
Standard comparison operators (=, >, <) cannot be used between vectors - use VECTOR_DISTANCE functions instead
Vector indexes dramatically improve performance by reducing search space through clustering, partitioning, and graphs
HNSW indexes provide fastest queries but require significant memory (Formula: 1.3 × vectors × dimensions × format_size)
IVF indexes work for very large datasets using disk storage, not constrained by RAM
Vector Pool memory must be configured for HNSW indexes at CDB level with VECTOR_MEMORY_SIZE parameter
Database restart required when changing VECTOR_MEMORY_SIZE (unless using automatic growth)
PDB maximum is 70% of sga_target, inheriting from CDB configuration
Choose index type based on dataset size and memory availability - HNSW for speed, IVF for scale

By understanding these concepts and properly configuring vector indexes, you can build high-performance semantic search applications that seamlessly integrate AI capabilities with traditional relational database operations.

Oracle AI Vector Search Workflow: From Data to Semantic Search

Fri, 06 Feb 2026 07:12:33 +0000

Oracle AI Vector Search Workflow: From Data to Semantic Search

Oracle Database 23ai introduces AI Vector Search, enabling semantic search capabilities directly within the database. Unlike traditional keyword-based searches, vector search understands the meaning and context behind your queries, allowing you to find relevant information even when exact keywords don't match.

What Are Vector Embeddings?

Vector embeddings are mathematical representations that capture the semantic meaning of your data. They convert unstructured data like text, images, or audio into numerical vectors that can be mathematically compared.

Key Properties:

Each embedding is an array of numbers (typically 128 to 1536 dimensions)
Semantically similar content has vectors that are close together in vector space
Distance between vectors indicates similarity in meaning

The Oracle AI Vector Search Workflow

The workflow consists of four primary steps:

Step 1: Generate Vector Embeddings

You can generate embeddings either within or outside Oracle Database.

Option A: Within Oracle Database (ONNX)

Load ONNX embedding models directly into the database:

-- Load the embedding model
EXECUTE DBMS_VECTOR.LOAD_ONNX_MODEL(
    'DM_DUMP',
    'all_MiniLM_L12_v2.onnx',
    'doc_model'
);

-- Generate embedding from text string
SELECT TO_VECTOR(
    VECTOR_EMBEDDING(doc_model USING 'hello world' AS data)
) AS embedding;

Common Embedding Patterns:

1. Convert Text String to Embedding:

SELECT VECTOR_EMBEDDING(doc_model USING 'semantic search' AS data) AS vec;

2. Convert File → Text → Chunks → Embeddings:

INSERT INTO doc_chunks (doc_id, chunk_id, chunk_data, embedding)
SELECT 
    d.id,
    JSON_VALUE(c.column_value, '$.chunk_id' RETURNING NUMBER),
    JSON_VALUE(c.column_value, '$.chunk_data'),
    VECTOR_EMBEDDING(doc_model USING 
        JSON_VALUE(c.column_value, '$.chunk_data') AS data)
FROM documents d,
    DBMS_VECTOR_CHAIN.UTL_TO_CHUNKS(
        DBMS_VECTOR_CHAIN.UTL_TO_TEXT(d.pdf_content),
        JSON('{"by":"words", "max":"100", "overlap":"20", 
              "split":"recursively", "language":"american"}')
    ) c;

Chunking Parameters:

by: Chunking method (words, sentences, paragraphs)
max: Maximum size per chunk (e.g., 100 words)
overlap: Overlapping units between chunks (prevents context loss)
split: Splitting strategy (recursively, custom)

3. Convert Files to Embeddings:

SELECT VECTOR_EMBEDDING(image_model USING file_content AS data) AS vec
FROM media_files;

4. End-to-End Pipeline:

INSERT INTO vectorized_docs
SELECT 
    d.doc_id,
    chunks.chunk_data,
    VECTOR_EMBEDDING(doc_model USING chunks.chunk_data AS data)
FROM source_documents d,
    DBMS_VECTOR_CHAIN.UTL_TO_CHUNKS(
        DBMS_VECTOR_CHAIN.UTL_TO_TEXT(d.pdf_blob),
        JSON('{"by":"words", "max":"100", "overlap":"20"}')
    ) chunks;

Option B: Outside Oracle Database

Generate embeddings using third-party services or local models:

from sentence_transformers import SentenceTransformer
import oracledb

model = SentenceTransformer('all-MiniLM-L6-v2')
embedding = model.encode("Oracle AI Vector Search")

connection = oracledb.connect(user="user", password="pwd", dsn="dsn")
cursor = connection.cursor()
cursor.execute(
    "INSERT INTO documents (id, text, embedding) VALUES (:1, :2, :3)",
    [1, "Oracle AI Vector Search", embedding.tolist()]
)
connection.commit()

In-Database Advantages: No data movement, reduced latency, simplified architecture, enterprise security

External Advantages: Flexibility in model choice, access to latest models, independent scaling

Step 2: Store Vector Embeddings

Store embeddings in columns with the VECTOR data type within standard relational tables:

CREATE TABLE products (
    product_id NUMBER PRIMARY KEY,
    product_name VARCHAR2(200),
    description CLOB,
    price NUMBER(10,2),
    category VARCHAR2(100),
    description_vector VECTOR(384, FLOAT32)
);

-- Insert with inline embedding generation
INSERT INTO products (product_id, product_name, description, description_vector)
VALUES (
    1001,
    'Wireless Headphones',
    'Noise-cancelling Bluetooth headphones',
    VECTOR_EMBEDDING(doc_model USING 'Noise-cancelling Bluetooth headphones' AS data)
);

Supported Formats: INT8 (memory efficient), FLOAT32 (balanced), FLOAT64 (highest precision)

Step 3: Create Vector Indexes

For large datasets, vector indexes are essential for performance.

Without Index: O(n) complexity - compares against every vector (slow for large datasets)
With Index: O(log n) complexity - checks subset of vectors (dramatically faster)

Index Types:

1. HNSW (In-Memory, Fast):

CREATE VECTOR INDEX product_hnsw_idx ON products (description_vector)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- Configure memory
ALTER SYSTEM SET VECTOR_MEMORY_SIZE = 2G SCOPE=BOTH;

2. IVF (Storage-Based, Scalable):

CREATE VECTOR INDEX product_ivf_idx ON products (description_vector)
ORGANIZATION NEIGHBOR PARTITIONS
DISTANCE COSINE
WITH TARGET ACCURACY 95;

3. Hybrid (Text + Semantic):

CREATE VECTOR INDEX product_hybrid_idx ON products (description)
ORGANIZATION HYBRID
PARAMETERS('EMBEDDING MODEL doc_model');

Choosing the Right Index:

HNSW: Dataset fits in memory, need maximum speed
IVF: Very large datasets or limited memory
Hybrid: Need both keyword and semantic search

Step 4: Query with Similarity Search

Use native SQL with specialized functions for similarity searches:

Basic Similarity Search:

SELECT 
    product_name,
    description,
    VECTOR_DISTANCE(
        description_vector,
        VECTOR_EMBEDDING(doc_model USING 'wireless audio device' AS data),
        COSINE
    ) AS similarity_score
FROM products
ORDER BY similarity_score
FETCH FIRST 5 ROWS ONLY;

Shorthand Operators:

-- <=> is cosine distance
SELECT product_name
FROM products
ORDER BY description_vector <=> 
    VECTOR_EMBEDDING(doc_model USING 'wireless audio device' AS data)
FETCH FIRST 5 ROWS ONLY;

Combining Semantic and Relational Searches:

This is Oracle's key advantage—combining semantic similarity with business logic:

-- Find similar products within price range and category
SELECT 
    product_name,
    price,
    category,
    VECTOR_DISTANCE(description_vector, :query_vec, COSINE) AS similarity
FROM products
WHERE price BETWEEN 100 AND 500
    AND category = 'Electronics'
ORDER BY similarity
FETCH FIRST 10 ROWS ONLY;

Advanced Hybrid Query:

SELECT 
    p.category,
    COUNT(*) AS matching_products,
    AVG(p.price) AS avg_price,
    MIN(VECTOR_DISTANCE(p.description_vector, :search_vec, COSINE)) AS best_match
FROM products p
WHERE p.price < 1000 AND p.in_stock = 'Y'
GROUP BY p.category
HAVING COUNT(*) > 5
ORDER BY best_match
FETCH FIRST 3 ROWS ONLY;

Distance Metrics:

Metric	Operator	Best For
COSINE	`<=>`	Text embeddings
EUCLIDEAN	`<->`	General purpose
DOT	N/A	Normalized vectors
MANHATTAN	N/A	Specific applications

Complete End-to-End Example

Building a semantic search system for customer support:

-- 1. Load model
EXECUTE DBMS_VECTOR.LOAD_ONNX_MODEL('MODEL_DIR', 'all-MiniLM-L12-v2.onnx', 'support_model');

-- 2. Create table
CREATE TABLE support_articles (
    article_id NUMBER PRIMARY KEY,
    title VARCHAR2(500),
    content CLOB,
    category VARCHAR2(100),
    content_vector VECTOR(384, FLOAT32)
);

-- 3. Insert with embeddings
INSERT INTO support_articles (article_id, title, content, category, content_vector)
VALUES (
    1,
    'How to Reset Your Password',
    'To reset your password, navigate to the login page...',
    'Account Management',
    VECTOR_EMBEDDING(support_model USING 'To reset your password...' AS data)
);

-- 4. Create index
CREATE VECTOR INDEX support_vec_idx ON support_articles (content_vector)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- 5. Semantic search
SELECT 
    title,
    category,
    VECTOR_DISTANCE(
        content_vector,
        VECTOR_EMBEDDING(support_model USING 'cannot log into my account' AS data),
        COSINE
    ) AS relevance_score
FROM support_articles
ORDER BY relevance_score
FETCH FIRST 5 ROWS ONLY;

Even though the query "cannot log into my account" doesn't match exact keywords, it will find articles about password reset, account recovery, and login troubleshooting.

Document Processing Pipeline

Process PDFs into searchable chunks:

-- Source documents table
CREATE TABLE source_docs (
    doc_id NUMBER PRIMARY KEY,
    filename VARCHAR2(500),
    file_content BLOB
);

-- Chunked content table
CREATE TABLE doc_chunks (
    chunk_id NUMBER PRIMARY KEY,
    doc_id NUMBER,
    chunk_text VARCHAR2(4000),
    chunk_vector VECTOR(384, FLOAT32)
);

-- Process: PDF → Text → Chunks → Vectors
INSERT INTO doc_chunks (chunk_id, doc_id, chunk_text, chunk_vector)
SELECT 
    ROW_NUMBER() OVER (ORDER BY d.doc_id),
    d.doc_id,
    JSON_VALUE(c.column_value, '$.chunk_data'),
    VECTOR_EMBEDDING(doc_model USING 
        JSON_VALUE(c.column_value, '$.chunk_data') AS data)
FROM source_docs d,
    DBMS_VECTOR_CHAIN.UTL_TO_CHUNKS(
        DBMS_VECTOR_CHAIN.UTL_TO_TEXT(d.file_content),
        JSON('{"by":"words", "max":"100", "overlap":"20"}')
    ) c;

Best Practices

Chunking:

Use 20-30% overlap to prevent context loss
50-100 words for precise matching, 200-300 for broader context

Index Configuration:

Set TARGET ACCURACY between 90-99%
Allocate sufficient memory for HNSW indexes
Monitor and rebuild IVF indexes periodically

Query Optimization:

Use FETCH FIRST n ROWS ONLY for better performance
Combine vector search with filters early
Use bind variables for query vectors
Prefer approximate search for most queries

Vector Formats:

FLOAT32: Best balance (default choice)
INT8: When memory is constrained
FLOAT64: Rarely needed

Performance Considerations

Memory Requirements (HNSW):

Memory = 1.3 × (format_size) × (dimensions) × (row_count)
Example: 1M vectors × 384 dims × 4 bytes × 1.3 = ~2 GB

Typical Performance:

Exact search: Seconds to hours
Approximate (HNSW): Milliseconds to seconds
Approximate (IVF): Seconds for billion-scale datasets

Use Cases

E-commerce product search
Customer support article matching
Document management systems
Fraud detection
Recommendation engines
Image similarity search
Code search with natural language

Workflow Summary

Generate Vector Embeddings: Load ONNX model or use external APIs, convert data to vectors
Store Vector Embeddings: Create tables with VECTOR columns alongside business data
Create Vector Indexes: Choose HNSW, IVF, or Hybrid based on dataset size and requirements
Query with Similarity Search: Use native SQL combining semantic and relational searches

Key Advantages:

Unified system (vectors + business data)
Native SQL with vector extensions
Enterprise features (security, availability, performance)
Flexible deployment (in-database or external)
Scalable (thousands to billions of vectors)
Hybrid queries (semantic + relational filters)

Oracle Database 23ai eliminates data fragmentation by keeping AI algorithms where the data lives, reducing complexity while providing enterprise-grade security and performance.

Understanding Semantic Search: Vector Embeddings and Similarity Search

Mon, 02 Feb 2026 07:14:35 +0000

Semantic search represents a fundamental shift in how we retrieve information from databases and search engines. Unlike traditional keyword-based search that relies on exact text matches, semantic search understands the meaning and context behind queries, enabling more intuitive and accurate information retrieval.

What is Semantic Search?

Semantic search goes beyond keyword matching to understand the intent and contextual meaning behind a query. Instead of looking for exact word matches, it retrieves results based on semantic similarity—finding content that means the same thing, even when different words are used.

For example, searching for "healthy dinner ideas" could return results like "nutritious meal prep for busy nights" even though the exact keywords don't match. This is possible because semantic search operates on the underlying meaning of the content.

Understanding Vector Data Distribution

Vector embeddings, which power semantic search, have unique characteristics:

Uneven Distribution

Vector data points are not uniformly distributed across vector space. Instead, they cluster around regions of semantic similarity, reflecting how related concepts group together in meaning.

Semantic Clustering

Vectors representing similar concepts naturally cluster together:

Words like "king," "queen," "prince," and "princess" form a royalty cluster
Technical terms like "algorithm," "function," and "code" cluster in programming regions
Synonyms and semantically related phrases position close to each other

This clustering property is fundamental to how semantic search works—we find related content by finding nearby vectors in this space.

How Similarity Search Works: k-NN Principle

At its core, semantic search relies on k-Nearest Neighbors (k-NN) search. When you perform a similarity search:

Convert your query into a vector embedding
Find the k nearest vectors to your query vector in the vector space
Retrieve the corresponding documents or data points

The result is an ordered list ranked by similarity, with the most semantically similar items appearing first.

Distance Metrics

Similarity between vectors is measured using:

Cosine Similarity: Measures the angle between vectors (commonly used for text)
Euclidean Distance: Straight-line distance between points
Dot Product: Useful for normalized vectors
Manhattan Distance: Sum of absolute differences along each dimension

Types of Similarity Search

1. Exact Search (Exhaustive Search)

How It Works: Compares the query vector against every single vector in the database.

Characteristics:

Accuracy: 100% accurate—guarantees finding actual nearest neighbors
Speed: Slow for large datasets (can take hours for millions of vectors)
Use Cases: Small datasets (< 10,000 documents) or when perfect accuracy is critical

2. Approximate Search (ANN)

How It Works: Uses specialized algorithms (like HNSW, IVF, or LSH) to efficiently search through large datasets by narrowing down the search space.

Characteristics:

Accuracy: High accuracy (typically 90-99%)
Speed: Dramatically faster—searches that take 65 hours with exact search complete in seconds
Use Cases: Large datasets (hundreds of thousands to billions of vectors)

Popular ANN Algorithms:

HNSW: Graph-based, extremely fast for queries
IVF: Cluster-based, good for very large datasets
LSH: Hash-based, excellent for high-dimensional data

Aspect	Exact Search	Approximate Search
Accuracy	100%	90-99%
Speed	Slow (linear)	Fast (sub-linear)
Scalability	Poor for large datasets	Excellent
Best For	< 10K documents	> 10K documents

Vector Embedding Models

Vector embeddings are the "translation layer" that converts human-readable content into machine-understandable numerical representations.

What Are Embedding Models?

Embedding models are machine learning models—typically based on transformer architectures—that convert data into dense vector representations trained on massive datasets to understand semantic relationships.

Key Capabilities

Contextual Understanding: Assign meaning based on context (e.g., "bank" in "river bank" vs. "financial bank" gets different embeddings)

Feature Extraction: Identify and quantify relevant features:

In text: semantic meaning, sentiment, topic
In images: shapes, colors, textures, objects
In audio: pitch, rhythm, timbre, speech patterns

Transformer Architecture: Excel at processing sequences, capturing long-range dependencies, and parallel processing for efficiency.

Popular Models

For Text:

Sentence Transformers (all-MiniLM-L6-v2): 384 dimensions, fast and lightweight
BERT: 768 dimensions, general-purpose language understanding
GPT Embeddings (text-embedding-ada-002): 1536 dimensions, production-grade

For Images:

CLIP: Jointly embeds images and text in the same space
ResNet: Deep convolutional neural network for image features
ViT: Transformer-based image understanding

For Audio:

Wav2Vec 2.0: Speech and audio embeddings
CLAP: Contrastive Language-Audio Pre-training

Types of Embedding Models

1. Pre-trained Open Source Models

Advantages: Zero training cost, proven performance, large community support

When to Use: General semantic search, quick prototyping, resource-constrained environments

Examples: Sentence Transformers library (15,000+ models), BERT variants, Universal Sentence Encoder

2. Custom Models Based on Your Dataset

Advantages: Optimal performance for specific use cases, understands proprietary terminology

Process: Start with pre-trained model → Fine-tune on labeled data → Evaluate → Iterate

Use Cases: Medical terminology, legal documents, e-commerce catalogs, scientific research

Generating Vector Embeddings

1. Outside the Database

Third-Party APIs: OpenAI, Cohere, Google Vertex AI, Hugging Face

Local Inference: Python libraries (sentence-transformers, transformers), ONNX Runtime, TensorFlow/PyTorch

Example:

from sentence_transformers import SentenceTransformer
model = SentenceTransformer('all-MiniLM-L6-v2')
embeddings = model.encode(["Semantic search is powerful"])

Advantages: Flexibility in model choice, control over pipeline

Disadvantages: Requires additional infrastructure, data movement between systems

2. Within the Database (ONNX)

ONNX (Open Neural Network Exchange) is an open format enabling models trained in one framework to be deployed in another.

Supported Databases: Oracle Database 23ai, PostgreSQL (with extensions), Microsoft SQL Server, SingleStore

Example (Oracle):

BEGIN
  DBMS_VECTOR.LOAD_ONNX_MODEL(
    directory => 'MODEL_DIR',
    file_name => 'all-MiniLM-L6-v2.onnx',
    model_name => 'text_embedding_model'
  );
END;

Advantages: No data movement, reduced latency, simplified architecture

Disadvantages: Limited to ONNX-compatible models, database computational overhead

Factor	External Generation	In-Database (ONNX)
Model Flexibility	High	Medium
Latency	Higher	Lower
Data Security	Requires data export	Data stays in DB
Best For	Batch processing, custom models	Real-time apps, integrated systems

Real-World Applications

Document Search: Find relevant documents based on meaning rather than keywords
Recommendation Systems: Suggest products/content based on similarity
Question Answering: Match user questions to relevant answers
Content Moderation: Identify similar or duplicate content
Image Search: Enable search by visual similarity
Customer Support: Route tickets and find similar past issues
Fraud Detection: Identify unusual patterns
Code Search: Find similar code snippets or search by natural language

Performance Optimization Tips

Choose the Right Balance: Use exact search for < 10K documents, approximate for > 100K
Tune Parameters: Adjust ef_search (HNSW) or nprobe (IVF) for accuracy vs. speed
Optimize Dimensions: Choose appropriate dimension counts, consider quantization
Implement Caching: Cache frequent embeddings, pre-compute for static content
Batch Processing: Generate embeddings in batches, leverage GPU acceleration

Key Takeaways

Vector embeddings capture semantic meaning in numerical form
Vector data naturally clusters by semantic similarity
Choose between exact search (accurate, slow) and approximate search (fast, 90-99% accurate) based on your needs
Transformer-based embedding models provide state-of-the-art semantic understanding
Models can be pre-trained, custom-trained, or fine-tuned for specific domains
Embeddings can be generated externally or within databases using ONNX

Whether you're building a search engine, recommendation system, or AI-powered application, understanding these concepts is crucial for leveraging the full power of modern semantic search technologies.

Oracle AI Vector Search in Oracle Database 23ai

Mon, 26 Jan 2026 15:52:40 +0000

Oracle Database 23ai (formerly 23c) represents a significant leap forward in integrating artificial intelligence capabilities directly into the database. One of its most groundbreaking features is AI Vector Search, which enables the database to understand conceptual similarity between pieces of data. This powerful feature eliminates the need for separate vector databases and allows you to perform semantic searches alongside traditional business queries within a unified system.

What is Oracle AI Vector Search?

Oracle AI Vector Search enables searching both structured and unstructured data by semantics or meaning, rather than just by values, fundamentally changing how we approach information retrieval. Instead of traditional keyword-based searches that match exact terms, vector search understands the context and meaning behind your queries.

Key Benefits of Oracle AI Vector Search

1. Semantic Search on Unstructured Data

Vector search is considered better than keyword search because it's based on the meaning and context behind words, not the actual words themselves. This allows you to find relevant information even when the exact keywords don't match.

2. Unified Relational and Semantic Search

One of the biggest advantages is the ability to combine semantic search on unstructured data with relational search on business data in a single system. This allows search results based on meaning, not just text matching – a fundamental shift for information retrieval.

3. Eliminates Data Fragmentation

No need to maintain separate vector databases or move data between systems. Your vectors live alongside your business data, benefiting from Oracle's robust security, availability, and performance features.

4. Enhanced Security and Governance

Oracle Database's security features (encryption, RBAC, auditing, Data Vault policies) extend automatically to vector data types. Vectors and graph data can be secured using the same roles and privileges as your tables.

Understanding the Core Components

The VECTOR Data Type

The VECTOR data type was introduced with Oracle Database 23ai, providing the foundation to store vector embeddings alongside business data. To use the VECTOR data type, the COMPATIBLE initialization parameter must be set to 23.4.0 or higher.

Here's a basic example:

CREATE TABLE documents (
    doc_id INT,
    doc_text CLOB,
    doc_vector VECTOR
);

The VECTOR data type can store vectors in different formats:

INT8 (8-bit integers)
FLOAT32 (32-bit floating-point numbers)
FLOAT64 (64-bit floating-point numbers)

You can specify the number of dimensions and storage format when declaring a column:

CREATE TABLE products (
    product_id NUMBER,
    description VARCHAR2(4000),
    embedding VECTOR(1536, FLOAT32)
);

Vector Embeddings: The Foundation

Vector embeddings are mathematical vector representations of data points that describe the semantic meaning behind content such as words, documents, audio tracks, or images. They translate semantic similarity, as perceived by humans, into proximity in a mathematical vector space.

How Vector Embeddings Work:

Unstructured data (text, images, audio) is converted into numerical vectors
Semantically similar content has vectors that are close together in vector space
Searching semantic similarity in a dataset is equivalent to searching nearest neighbors in a vector space

Creating Vector Embeddings:

You have several options for generating embeddings:

ONNX Embedding Models: Import pre-trained models into Oracle Database using the ONNX (Open Neural Network Exchange) format
Third-Party REST APIs: Access external embedding services
Local Models: Use models like all-MiniLM-L12-v2 for text embeddings or ResNet for image embeddings

Example of loading an ONNX model:

BEGIN
  DBMS_VECTOR.LOAD_ONNX_MODEL(
    directory => 'DM_DUMP',
    file_name => 'all-MiniLM-L6-v2.onnx',
    model_name => 'doc_model'
  );
END;
/

Vector Indexes: Accelerating Similarity Search

While exact similarity searches are accurate, they can be slow for large datasets. Oracle AI Vector Search supports three types of vector indexes that use Approximate Nearest Neighbor algorithms to reduce distance calculations:

1. HNSW (Hierarchical Navigable Small World) Index

HNSW is an In-Memory Neighbor Graph vector index that's very efficient for approximate similarity search. It creates a layered graph structure that enables fast navigation through high-dimensional space.

Characteristics:

In-memory only (stored in Vector Memory Pool in SGA)
Extremely fast query performance
Memory can be estimated as 1.3 times the product of vector format size, number of dimensions, and number of rows
Best for datasets that fit in available memory

Example:

CREATE VECTOR INDEX product_hnsw_idx ON products (embedding)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

2. IVF (Inverted File Flat) Index

The IVF index is a storage-based Neighbor Partition vector index that enhances search efficiency by narrowing the search area through neighbor partitions or clusters.

Characteristics:

Storage-based (not constrained by memory like HNSW)
Can be used for very large datasets and still provide excellent performance compared to exhaustive similarity search
Supports both global and local partitioning on partitioned tables
Ideal for large datasets where HNSW won't fit in memory

Example:

CREATE VECTOR INDEX product_ivf_idx ON products (embedding)
ORGANIZATION NEIGHBOR PARTITIONS
DISTANCE COSINE
WITH TARGET ACCURACY 95;

3. Hybrid Vector Index

Hybrid Vector Index combines full text and semantic search in one index, allowing users to run textual queries, vector similarity queries, or hybrid queries. This is particularly useful when you need both keyword matching and semantic understanding.

Practical Implementation Example

Here's a complete example of using vector search:

-- Step 1: Create table with vector column
CREATE TABLE knowledge_base (
    id NUMBER PRIMARY KEY,
    content CLOB,
    content_vector VECTOR(384, FLOAT32)
);

-- Step 2: Insert data and generate embeddings
INSERT INTO knowledge_base VALUES (
    1,
    'Oracle Database 23ai introduces AI Vector Search',
    NULL
);

-- Step 3: Update vectors using embedding model
UPDATE knowledge_base
SET content_vector = VECTOR_EMBEDDING(doc_model USING content AS DATA);

-- Step 4: Create vector index for performance
CREATE VECTOR INDEX kb_vector_idx ON knowledge_base (content_vector)
ORGANIZATION INMEMORY NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

-- Step 5: Perform similarity search
SELECT id, content
FROM knowledge_base
ORDER BY VECTOR_DISTANCE(
    content_vector,
    VECTOR_EMBEDDING(doc_model USING 'AI features in databases' AS DATA),
    COSINE
)
FETCH FIRST 5 ROWS ONLY;

Distance Metrics

Oracle supports multiple distance metrics for vector comparisons:

COSINE: Measures angular distance (best for text embeddings)
EUCLIDEAN: Measures straight-line distance
DOT_PRODUCT: Useful for normalized vectors
MANHATTAN: Sum of absolute differences

Use Cases

Oracle AI Vector Search enables powerful applications:

Semantic Document Search: Find documents by meaning, not just keywords
Recommendation Systems: Suggest products based on similarity
Image Similarity Search: Find visually similar images
Chatbots and Q&A Systems: Build context-aware conversational AI
Anomaly Detection: Identify outliers in high-dimensional data
Natural Language Processing: Power advanced text analysis applications

Performance Considerations

Vector Pool Sizing

For HNSW indexes, you need to configure the Vector Memory Pool:

ALTER SYSTEM SET VECTOR_MEMORY_SIZE = 2G SCOPE=BOTH;

Choosing the Right Index

Use HNSW when: Dataset fits in memory and you need maximum speed
Use IVF when: Dataset is very large or memory is limited
Use Hybrid when: You need both keyword and semantic search

Target Accuracy

The TARGET ACCURACY parameter balances speed vs. precision. Higher accuracy (e.g., 95-99) returns more precise results but requires more computation.

Integration with Existing Features

One of the most powerful aspects of Oracle AI Vector Search is its deep integration with existing Oracle Database features:

Oracle RAC: Full support for clustered environments
Data Guard: Vector data replicates alongside business data
Partitioning: IVF indexes support local partitioning
Security: All Oracle security features apply to vectors
ORDS: Expose vector search through REST APIs
Oracle APEX: Build AI-powered applications with minimal code

Getting Started

Oracle AI Vector Search is available in:

Oracle Autonomous Database (Always Free tier available)
Oracle Database 23ai Free (Developer edition)
Oracle Exadata Database Service
Oracle Base Database Service

To start experimenting:

Download Oracle Database 23ai Free from Oracle Container Registry
Set COMPATIBLE parameter to 23.4.0 or higher
Configure Vector Memory Pool for HNSW indexes
Load an ONNX embedding model
Create tables with VECTOR columns
Start building AI-powered applications!

Oracle Database 23ai brings AI algorithms to where the data lives, instead of having to move the data to where the AI algorithm lives. This architectural approach provides significant advantages in terms of security, performance, and simplicity.

By combining semantic search capabilities with traditional relational data operations in a single, converged database, Oracle AI Vector Search represents a fundamental shift in how we build AI-powered applications. Whether you're implementing RAG-based chatbots, recommendation engines, or advanced search systems, Oracle Database 23ai provides the foundation for secure, scalable, and high-performance AI applications.

The elimination of data fragmentation, coupled with enterprise-grade security and proven Oracle reliability, makes Oracle AI Vector Search an compelling choice for organizations looking to integrate AI capabilities into their data infrastructure.

Additional Resources

RAG Pipeline Deep Dive: Ingestion, Chunking, Embedding, and Vector Search

Wed, 21 Jan 2026 06:23:00 +0000

Retrieval-Augmented Generation (RAG) has become the default approach for providing context and memory to AI applications. Understanding the RAG pipeline—from document ingestion to final response generation—is essential for building high-performance AI systems. This comprehensive guide explores each stage of the RAG workflow, with special focus on Oracle Database 23ai's native vector capabilities.

The RAG Pipeline: An Overview

A complete RAG pipeline consists of two main phases:

Ingestion Phase (Offline):
Documents → Chunking → Embedding → Index/Store in Vector Database

Retrieval Phase (Runtime):
User Query → Embed Query → Similarity Search → Top-K Results → LLM Generation → Response

The RAG pipeline connects the model to the information it needs at query time. When a user asks a question, the pipeline retrieves the most relevant documents, prepares that text as context, and includes it in the prompt so the model can generate an answer grounded in retrieved material rather than training data alone.

Phase 1: Document Ingestion

Ingestion is the process of loading documents from multiple sources and in multiple formats, transforming them into a structured form suitable for embedding and retrieval.

Document Loading

Raw documents—whether PDFs, Word documents, web pages, or database records—must be transformed into manageable chunks suitable for retrieval.

Common Sources:

Local files (PDF, DOCX, TXT, CSV)
Web pages and APIs
Databases and data warehouses
Cloud storage (S3, GCS, Azure Blob)
Enterprise systems (SharePoint, Confluence, Notion)

Loading Libraries:

PyPDF2: For PDF documents
python-docx: For Word documents
LangChain Document Loaders: Unified interface for multiple formats
LlamaIndex Data Connectors: Specialized loaders for various sources
Oracle OracleDocLoader: Native loader for Oracle Database 23ai

Text Extraction and Cleaning

A clean corpus improves both retrieval accuracy and embedding quality. Use parsing libraries to extract only the primary content from web pages or PDFs, and remove boilerplate, navigation links, and disclaimers.

Cleaning Steps:

Remove Noise: Headers, footers, navigation, ads
Normalize Structure: Convert to consistent format (JSONL, Parquet)
Preserve Metadata: Source, timestamp, author, document type
Handle Special Characters: Normalize encoding (UTF-8)
Maintain Logical Structure: Keep headings, lists, tables

Metadata Enrichment

Each chunk should carry metadata such as topic, date, and source, so that retrieval can filter and rank results more effectively.

Important Metadata Fields:

source_id: Unique document identifier
title: Document title
created_date: Document creation timestamp
author: Document creator
document_type: PDF, email, webpage
topic/category: For domain filtering
page_number: For PDF sources
chunk_id: Unique identifier for the chunk

Phase 2: Chunking - The Critical Step

Chunking is simply the act of splitting larger documents into smaller units ("chunks"). Each chunk can be individually indexed, embedded, and retrieved.

Why Chunking Matters

The chunking process directly feeds into the embedding step, where each chunk is converted into vector representations that capture semantic meaning.

A common source of inaccurate or unstable answers lies in a structural conflict within the traditional "chunk-embed-retrieve" pipeline: using a single-granularity, fixed-size text chunk to perform two inherently conflicting tasks:

Semantic matching (recall): Smaller chunks (100-256 tokens) needed for precise similarity search
Context understanding (utilization): Larger chunks (1024+ tokens) needed for coherent LLM generation

This creates a difficult trade-off between "precise but fragmented" and "complete but vague."

Pre-Chunking vs Post-Chunking

Pre-chunking is the most common method. It processes documents asynchronously by breaking them into smaller pieces before embedding and storing them in the vector database. This enables fast retrieval at query time since all chunks are pre-computed.

Post-chunking takes a different approach by embedding entire documents first, then performing chunking at query time only on the documents that are actually retrieved. The chunked results can be cached, so the system becomes faster over time.

Chunking Strategies

Fixed-Length Chunking: This is the simplest approach, segmenting text into equally sized pieces using character, token, or word counts.

Common Chunk Sizes:

300-500 tokens: Typical chunks for most use cases, depending on the model and content type
100-256 tokens: For high-precision recall in semantic search
1024+ tokens: For providing complete context to LLMs

Adopt recursive/semantic chunkers with 10-20% overlap to maintain context across boundaries.

Advanced Chunking Approaches:

Recursive Chunking: This method is a bit smarter. It splits text hierarchically based on cues like paragraphs, line breaks, or sentences. If a section is too long, it recursively divides it further.

Semantic Chunking: Instead of letting token limits decide where ideas get chopped, it uses embeddings to detect where the meaning actually shifts—grouping semantically related sentences together.

Document-Aware Chunking: Preserves document structure (sections, paragraphs, tables) while chunking.

Chunk Overlap

Chunk overlap is the number of overlapping characters (or tokens) between adjacent chunks. This ensures that context isn't lost when information spans chunk boundaries.

Recommended Overlap:

10-20% overlap is standard practice
For 500-token chunks, use 50-100 token overlap
Maintains continuity without excessive redundancy

Example:

Chunk 1: [Tokens 0-500]
Chunk 2: [Tokens 450-950]  ← 50 token overlap
Chunk 3: [Tokens 900-1400] ← 50 token overlap

Splitting Process

Documents are split recursively until the chunks are small enough, using text splitters that respect:

Sentence boundaries (don't break mid-sentence)
Paragraph boundaries (maintain logical grouping)
Section boundaries (preserve document structure)

Popular Chunking Libraries:

The LLM community often turns to two powerful open-source libraries: LangChain and LlamaIndex:

LangChain: Broad framework with flexible chunking strategies
LlamaIndex: Designed specifically for RAG pipelines with sophisticated NodeParsers
Chonkie: Lightweight, dedicated chunking library focusing solely on text splitting

Phase 3: Embedding and Storage

Chunking makes information retrievable; embeddings make it searchable by meaning rather than keywords.

What Are Embeddings?

Embeddings are numerical representations that capture semantic relationships between text. A piece of text—whether a word, phrase, sentence, or paragraph—is converted to a vector of numbers (typically 384-1536 dimensions).

Key Properties:

Semantic similarity: Embeddings that are numerically similar are also semantically similar
Dense representation: High-dimensional vectors capture nuanced meaning
Model-specific: Different embedding models produce different vector spaces

Embedding Models

Popular Options:

OpenAI text-embedding-3-large: 3072 dimensions, high quality
Cohere embed-v4.0: 1024 dimensions, multimodal (text + images)
Sentence-BERT (SBERT): 768 dimensions, open-source
all-MiniLM-L6-v2: 384 dimensions, lightweight and fast

Choose MiniLM/SBERT/Instruct/E5 for text, CLIP for images. Normalize embeddings for consistent similarity calculations.

Embedding Generation: Outside vs Inside the Database

Option 1: Generate Embeddings Outside Oracle Database 23ai

Vector embeddings can be generated outside Oracle Database 23ai using third-party embedding models, then imported into the database for storage and search.

Workflow:

from sentence_transformers import SentenceTransformer

# Load embedding model
model = SentenceTransformer('all-MiniLM-L6-v2')

# Generate embeddings
chunks = ["First chunk text", "Second chunk text"]
embeddings = model.encode(chunks)

# Store in Oracle Database 23ai
# INSERT INTO documents (text, embedding) VALUES (?, ?)

Advantages:

Flexibility in choosing any embedding model
Can use cloud-based embedding services
Easy integration with existing ML pipelines

Option 2: Generate Embeddings Inside Oracle Database 23ai

Embeddings can also be generated inside Oracle Database 23ai by downloading pretrained embedding models, converting them to ONNX format, and importing the ONNX format models into Database 23ai.

ONNX (Open Neural Network Exchange) is an open-source format designed to represent deep learning models. It aims to provide interoperability between different deep learning frameworks.

Process:

Download Pretrained Model from Hugging Face or similar source
Convert to ONNX Format using Oracle OML4Py utility
Import into Database 23ai using DBMS_VECTOR package

The command can be run to import the ONNX file into the database:

EXEC dbms_vector.load_onnx_model(
  'DM_DUMP',
  'all_MiniLM_L12_v2.onnx',
  'minilm_l12_v2',
  JSON('{"function": "embedding", "embeddingOutput": "embedding", "input": {"input": ["DATA"]}}')
);

Generate Embeddings in SQL:

SELECT VECTOR_EMBEDDING(minilm_l12_v2 USING 'Your text here' AS data) AS embedding
FROM DUAL;

Advantages:

Oracle Database implements an ONNX runtime directly within the database. This allows you to generate vector embeddings directly within Oracle Database using SQL
Eliminates data movement between systems
Simplified architecture and security
Native integration with Oracle's vector capabilities

Phase 4: Vector Indexes for Fast Search

Once embeddings are generated and stored, vector indexes are specialized data structures designed for similarity searches in high-dimensional vector spaces. They help accelerate vector similarity searches by reducing the number of distance calculations needed.

Why Vector Indexes?

Exact search to find the closest matches for a given vector are accurate but can be slow, since costly vector distance computations are needed for all vectors in a column.

Vector indexes use techniques like clustering, partitioning, and neighbor graphs to group similar vectors together, reducing the search space dramatically.

Oracle AI Vector Search: Supported Index Types

Oracle AI Vector Search supports Hierarchical Navigable Small World (HNSW) indexes and Inverted File (IVF) indexes.

1. HNSW: In-Memory Neighbor Graph

Hierarchical Navigable Small World (HNSW) creates a navigable graph structure to locate similar vectors with exceptional speed and accuracy.

Characteristics:

HNSW is an In-Memory Neighbor Graph Vector Index fully built in-memory. You need to setup a new memory pool with VECTOR_MEMORY_SIZE in the SGA to accommodate it.

Since the HNSW index consumes a lot of memory, Oracle Database 23ai makes this index available by allocating a memory area called Vector Pool.

Memory Calculation:

To roughly determine the memory size needed to store an HNSW index, use the following formula: 1.3 × number of vectors × number of dimensions × size of your vector dimension type (for example, a FLOAT32 is 4 bytes).

Creating HNSW Index:

-- Configure vector pool
ALTER SYSTEM SET vector_memory_size=800M SCOPE=SPFILE;
-- Restart required

-- Create HNSW index
CREATE VECTOR INDEX documents_hnsw_idx 
ON documents (embedding)
ORGANIZATION NEIGHBOR GRAPH
DISTANCE COSINE
WITH TARGET ACCURACY 95;

Performance:

HNSW enables sub-100ms retrieval at 95%+ recall—extremely fast for real-time applications.

Limitations:

DML operations (insert, update, delete) are not allowed after creating an HNSW index—designed for read-heavy workloads.

HNSW indexes are not available in RAC environments.

2. IVF: Neighbor Partition Index

Inverted File Flat (IVF) indexes partition data points into clusters, enabling efficient searching within relevant clusters.

Characteristics:

IVF Flat index is a Neighbor Partition Vector index built on disk, and its blocks are cached in the regular buffer cache.

A significant advantage of an IVF vector index is that it is not constrained by the amount of memory available in the vector pool like an HNSW vector index is.

How IVF Works:

It is designed to enhance search efficiency by narrowing the search area through the use of neighbor partitions or clusters. The center of each partition, or the centroid, represents the average vector for each partition. By default, the number of partitions and their corresponding centroids is set to the square root of the dataset's size.

Creating IVF Index:

CREATE VECTOR INDEX documents_ivf_idx 
ON documents (embedding)
ORGANIZATION NEIGHBOR PARTITIONS
DISTANCE COSINE
WITH TARGET ACCURACY 95;

Advantages:

Works with massive datasets
Supports global and local indexes on partitioned tables
DML operations can degrade accuracy over time. Periodically rebuilding the index may be necessary
More memory-efficient than HNSW

Performance:

Although an IVF index won't be as fast as an equivalent HNSW index, it can be used for very large data sets and still provide excellent performance when compared to an exhaustive similarity search.

3. Hybrid Vector Indexes

Hybrid vector index combines full text and semantic search in one index, allowing users to easily index and query their documents using a combination of full text search and semantic vector search.

Use Case:
When you need to match both semantic meaning AND specific keywords/terms in a single query.

GPU Acceleration

At Oracle CloudWorld 2024, Oracle demonstrated GPU-accelerated creation of vector embeddings and HNSW index creation using NVIDIA GPUs.

When a user sends a request to create an HNSW vector index to Oracle Database, the database intelligently compiles and redirects the task to a Compute Offload Server process that leverages GPUs and the CAGRA algorithm from NVIDIA cuVS library to rapidly generate a graph index.

Phase 5: Retrieval - Finding Relevant Context

At runtime, when a user submits a query, the retrieval phase locates the most relevant chunks to provide as context to the LLM.

Query Embedding

The system encodes the user's query with the same model used to embed documents during ingestion.

-- Generate query embedding
SELECT VECTOR_EMBEDDING(minilm_l12_v2 USING 'What is RAG?' AS data) AS query_vector
FROM DUAL;

Similarity Search

The query vector is compared against all document vectors in the database using a distance metric.

Distance Metrics:

Common options include EUCLIDEAN, L2_SQUARED, COSINE, DOT, MANHATTAN, and HAMMING. You can specify the metric during index creation or within the search query itself.

Cosine Similarity: Most common for text (measures angle between vectors)
Euclidean Distance: Straight-line distance in vector space
Dot Product: Efficient for normalized vectors

Performing Similarity Search:

SELECT 
  text,
  VECTOR_DISTANCE(embedding, :query_vector, COSINE) AS distance
FROM documents
ORDER BY distance
FETCH APPROX FIRST 5 ROWS ONLY;

Top-K Results

The system searches the index for the nearest vectors, retrieves the top-k chunks, and includes those chunks with the query in the prompt.

Choosing K:

K=3-5: For focused, specific queries
K=10-20: For broader research queries
K=50+: For comprehensive document analysis

Challenges:

Top-k returns near duplicates or shallow snippets, so the prompt lacks diversity. Retrieval misses proper nouns, IDs, or acronyms because they occur rarely.

Solutions:

Reranking: Rerank with cross-encoders (Cohere ReRank, ColBERT) when precision matters
Hybrid Search: Combine BM25 (keyword) with dense vectors to boost tail recall
Metadata Filtering: Filter by date, source, document type before retrieval

Monitoring Index Accuracy

A valuable feature is Oracle's provision of functions to report index accuracy based on query vectors used in approximate searches. This allows you to continuously monitor and optimize the accuracy of your vector search implementation.

Phase 6: Generation - Creating the Final Response

Once relevant chunks are retrieved, they're combined with the user's query and sent to an LLM for response generation.

Prompt Assembly

Retrieved chunks are formatted into a coherent context block:

Context:
[Chunk 1 text]
[Chunk 2 text]
[Chunk 3 text]

Question: [User query]

Answer based on the provided context:

LLM Generation

The complete prompt (context + query) is sent to the LLM, which generates a response grounded in the retrieved information rather than relying solely on parametric knowledge.

Benefits:

Reduces hallucinations
Provides traceable sources
Enables up-to-date information
Maintains factual accuracy

Best Practices for Production RAG Pipelines

1. Optimize Chunking Strategy

Align chunk size with context windows, use task-specific sentence transformers, and default to HNSW + metadata filtering for sub-100ms retrieval at 95%+ recall.

2. Choose Appropriate Embedding Models

Match embedding dimensions to your dataset size
Consider domain-specific models for specialized content
Test multiple models and measure retrieval quality

3. Implement Proper Indexing

Small-Medium datasets (< 1M vectors): HNSW for maximum speed
Large datasets (1M+ vectors): IVF for scalability
Keyword + Semantic: Hybrid vector indexes

4. Monitor and Iterate

Monitor recall@k, latency, and index memory; re-embed on model upgrades.

Track metrics:

Retrieval accuracy (precision@k, recall@k)
Query latency (p50, p95, p99)
Index build time
Storage requirements

5. Handle Edge Cases

Query reformulation for better retrieval
Fallback strategies when retrieval fails
Confidence thresholds for generated responses
Citation and source tracking

Advanced RAG Techniques

1. Self-Correction (CRAG)

When strong retrieval pipelines return incomplete or irrelevant context, CRAG adds a lightweight feedback loop: before generating, the system checks whether the retrieved set is good enough. If it looks weak, the system triggers another retrieval pass.

2. Agentic RAG

Agentic RAG systems plan retrieval strategies and iterate based on partial results—dynamically deciding what to retrieve next.

3. GraphRAG

Build an advanced RAG system by first structuring data into a knowledge graph, then layering graph-aware retrieval to capture entity relationships.

4. Multimodal RAG

Combining text, images, audio, and video retrieval for richer context.

A robust RAG pipeline requires careful attention to every stage:

Ingestion: Clean, structured data with rich metadata
Chunking: Balanced chunk sizes with appropriate overlap (10-20%)
Embedding: Semantic representations using appropriate models
Indexing: Fast similarity search with HNSW or IVF indexes
Retrieval: Accurate top-K results with metadata filtering
Generation: Grounded responses with proper citations

Oracle Database 23ai provides a complete, integrated platform for RAG with native vector support, ONNX model runtime, advanced indexing (HNSW, IVF, Hybrid), and SQL-native operations—eliminating the need for separate vector databases.

By mastering these components and following best practices, you can build production-grade RAG systems that deliver accurate, relevant, and trustworthy AI-powered experiences.

Why a Vector Database Is Critical for Real AI Implementation

Mon, 15 Dec 2025 17:11:17 +0000

AI is only useful when it answers with the right information.
A vector database makes that possible.

Instead of searching by keywords, it searches by meaning.
That’s how AI finds the most relevant data from your docs, pages, chats, and files.

Without a vector database:

AI guesses
answers feel random
hallucinations increase

With a vector database:

answers stay accurate
context is preserved
responses feel reliable

At byteoniclabs.com every serious AI system we build starts with a proper vector database, because that’s what turns AI from a chatbot into a business tool.

If your AI gives mixed answers, your data layer is the first thing to fix.

OCI Introduction to AI: Transforming Enterprise Intelligence

Fri, 29 Aug 2025 14:52:46 +0000

Oracle Cloud Infrastructure (OCI) is revolutionizing how enterprises approach artificial intelligence, providing comprehensive AI services that integrate seamlessly with existing business processes. This introduction explores the fundamental concepts of AI, Oracle's AI ecosystem, and practical applications that are transforming modern enterprises.

Understanding Artificial Intelligence

Artificial Intelligence represents the ability of machines to mimic cognitive abilities and problem-solving capabilities of humans. This transformative technology has evolved from simple automation to sophisticated systems that can understand context, make decisions, and continuously improve their performance.

How AI Systems Learn

AI systems demonstrate remarkable adaptability by learning new skills through observation. This learning process involves:

Pattern recognition from large datasets
Algorithmic adaptation based on feedback
Continuous improvement through iterative training
Knowledge transfer across different domains

Artificial General Intelligence (AGI)

While current AI is specialized, Artificial General Intelligence can mimic human skills, sensory and motor skills, and use these skills to carry out tasks without human interruption. AGI represents the future where AI systems can:

Understand context across multiple domains
Perform complex reasoning similar to human thought processes
Adapt to new situations without explicit programming
Integrate multiple capabilities for comprehensive problem-solving

The Modern AI Revolution

Today's enterprises leverage AI for automation, decision-making, and creative support across numerous applications. The technology has moved beyond experimental phases to become essential for competitive advantage.

AI Applications Across Industries

AI domains span a wide range of practical applications:

Communication and Language

Language translation: Breaking down global communication barriers
Text-to-speech: Converting written content to natural-sounding audio
Natural language processing: Understanding and generating human-like text
Sentiment analysis: Interpreting emotional context in communications

Visual Intelligence

Image classification: Automatically categorizing and tagging visual content
Computer vision: Understanding and interpreting visual information
Object detection: Identifying specific items within images or video
Facial recognition: Secure identification and authentication systems

Business Intelligence

Product recommendations: Personalized suggestions based on behavior patterns
Fraud detection: Real-time identification of suspicious transactions
Predictive analytics: Forecasting trends and outcomes
Customer behavior analysis: Understanding user preferences and patterns

Autonomous Systems

Self-driving cars: Autonomous navigation and decision-making
Robotic automation: Intelligent manufacturing and logistics
Smart infrastructure: Adaptive city and building management
Predictive maintenance: Anticipating equipment failures before they occur

Oracle Cloud Infrastructure AI Services

OCI provides a comprehensive suite of AI services designed to accelerate enterprise AI adoption without requiring deep machine learning expertise.

OCI Generative AI

OCI Generative AI service is a fully managed service that seamlessly integrates large language models (LLMs) from Cohere and Meta Llama 2 to address diverse business use cases. Key capabilities include:

Multilingual support: Supporting over 100 languages
Text generation: Creating human-like content for various purposes
Summarization: Condensing large documents into key insights
Code generation: Assisting developers with automated code creation
Chat interfaces: Building intelligent conversational experiences

OCI AI Agents

Oracle has expanded its AI offerings significantly, with over 50 AI agents and a new generative AI RAG Agent in OCI announced at CloudWorld 2024. These agents provide:

Specialized functionality for specific business domains
Pre-built solutions reducing development time
Integration capabilities with existing enterprise systems
Customization options for unique business requirements

Machine Learning Services

Build, train, and deploy machine learning models. Or use AI services to add prebuilt chatbot, anomaly detection, NLP, and speech capabilities to applications and operations:

AutoML: Automated model building and optimization
Custom model training: Bring your own algorithms and data
Model deployment: Scalable inference endpoints
MLOps integration: Complete machine learning lifecycle management

Select AI in Autonomous Database

Oracle Autonomous Database's Select AI feature represents a breakthrough in data accessibility, enabling business users to interact with their data using natural language.

Natural Language Query Interface

Select AI uses your language to query data, revolutionizing how users interact with databases. The DBMS_CLOUD_AI package, with Select AI, facilitates and configures the translation of natural language prompts to generate, run, explain SQL statements.

How Select AI Works

Select AI takes a question in natural language and translates it into SQL:

Natural language input: Users ask questions in plain English
Intent interpretation: AI understands the query requirements
SQL generation: Automatic creation of optimized database queries
Result presentation: Data returned in user-friendly formats
Query refinement: You can keep refining your questions for better results

Interactive Learning

Select AI combines the capabilities of Large Language Models (LLM) with metadata from your ADB schemas to enable database-specific SQL to be generated. This approach ensures:

Context-aware queries based on your specific database schema
Accurate results tailored to your data structure
Learning from interactions to improve future responses
Domain-specific understanding of your business terminology

Vector Search and AI Integration

Oracle Database 23ai introduces powerful vector capabilities that form the foundation of modern AI applications.

Vector Database Capabilities

SQL support for vector generation enables developers to:

Store vector embeddings alongside traditional relational data
Perform similarity searches using SQL syntax
Integrate with machine learning pipelines seamlessly
Scale vector operations using Oracle's proven database technology

Supported Vector Data Types

Vector data types are supported natively in Oracle Database, including:

Dense vectors: Optimized for machine learning embeddings
Sparse vectors: Efficient storage for high-dimensional data
Binary vectors: Specialized for specific AI applications
Flexible dimensions: Supporting various embedding models

Approximate Search Indexes

Approximate search indexes provide fast similarity search capabilities:

High-performance queries on large vector datasets
Configurable accuracy: Balance between speed and precision
Parallel processing: Leveraging Oracle's parallel query capabilities
Memory optimization: Efficient use of system resources

Generative AI Pipeline Integration

AI Vector Search powers generative AI pipelines by enabling:

Retrieval-Augmented Generation (RAG): Implementing a complete Retrieval-Augmented Generation (RAG) pipeline
Contextual responses: Grounding AI responses in your enterprise data
Real-time search: Finding relevant information during conversations
Multi-modal search: Combining text, image, and structured data

Advanced AI Features in Oracle Database 23ai

Comprehensive AI Toolkit

Leveraging Oracle Database 23ai technologies and assisted by AI, developers can focus on building app functionality instead of data infrastructure needs. The platform includes:

Integrated vector database: Native support for AI workloads
Machine learning algorithms: Built-in ML capabilities
Graph analytics: Advanced relationship analysis
JSON document support: Flexible data modeling for AI applications

Enterprise-Grade AI Infrastructure

Oracle Database 23ai is available in Oracle Cloud Infrastructure (OCI) on Oracle Exadata Database Service, Oracle Exadata Cloud@Customer and Oracle Base Database Service, as well as on Oracle Database@Azure, ensuring:

High availability: Enterprise-grade reliability for AI workloads
Scalability: Handle growing AI data requirements
Security: Protect sensitive AI models and training data
Performance: Optimized infrastructure for AI computations

Real-World AI Implementation Strategies

Starting Your AI Journey

Identify Use Cases

Process automation: Streamline repetitive tasks
Customer experience: Enhance user interactions
Data insights: Extract value from existing data
Operational efficiency: Optimize resource utilization

Build AI Capabilities

Start with pre-built services: Leverage Oracle's AI services
Develop custom models: Build specialized solutions for unique needs
Integrate existing systems: Connect AI with current business processes
Train your team: Develop internal AI expertise

Best Practices for Enterprise AI

Data Strategy

Data quality: Ensure clean, well-organized training data
Data governance: Implement proper data management policies
Privacy compliance: Maintain regulatory compliance for AI systems
Data integration: Connect disparate data sources effectively

Infrastructure Planning

Scalable architecture: Design for growing AI workloads
Security framework: Protect AI models and data
Performance monitoring: Track AI system effectiveness
Cost optimization: Manage AI infrastructure expenses

Oracle's AI Ecosystem Advantages

Integrated Platform

Oracle provides a complete AI ecosystem that eliminates the complexity of managing multiple vendors and integration points:

Single vendor solution: Unified support and accountability
Seamless integration: Built-in connectivity between AI services
Consistent security: Unified security model across all components
Simplified management: Single control plane for AI operations

Enterprise Focus

Oracle's AI solutions are designed specifically for enterprise requirements:

Regulatory compliance: Built-in features for industry regulations
High availability: Enterprise-grade uptime guarantees
Scalable architecture: Handle enterprise-scale workloads
Professional support: Dedicated support for mission-critical AI applications

The Future of AI in Enterprise

Emerging Trends

Multimodal AI: Systems that process text, images, and audio simultaneously
Autonomous operations: Self-managing AI systems requiring minimal human intervention
Ethical AI: Responsible AI development with built-in bias detection
Edge AI: Bringing AI capabilities closer to data sources

Oracle's AI Roadmap

Oracle continues to invest heavily in AI innovation, with recent developments including:

Enhanced vector search: More sophisticated similarity algorithms
Expanded language models: Additional LLM options and capabilities
Improved automation: Reduced complexity for AI implementation
Industry-specific solutions: Tailored AI applications for vertical markets

Conclusion

Oracle Cloud Infrastructure's approach to AI combines cutting-edge technology with enterprise-grade reliability and security. From Select AI's natural language database queries to comprehensive vector search capabilities, OCI provides the tools enterprises need to harness artificial intelligence effectively.

The integration of AI capabilities directly into Oracle Database 23ai, combined with powerful cloud-native AI services, creates an ecosystem where organizations can build sophisticated AI applications without the complexity traditionally associated with artificial intelligence implementations.

Whether you're just beginning your AI journey or looking to scale existing AI initiatives, Oracle's comprehensive AI platform provides the foundation for transforming your business through intelligent automation, enhanced decision-making, and innovative customer experiences.

Ready to explore AI capabilities in your organization? Start with Oracle's pre-built AI services to gain immediate value, then expand into custom AI solutions as your team develops expertise with the platform.

The Future of Search: How AI is Revolutionizing Online Discovery

Tue, 28 Jan 2025 07:56:13 +0000

Have you ever wondered how search engines like Google are changing with the rise of artificial intelligence (AI)? Platforms like ChatGPT and Perplexity are leading the way in this new era of search. For SEO experts, this shift presents a huge opportunity to adapt and thrive.

The impact of AI-powered search tools is already evident. We experienced a significant increase in website traffic from ChatGPT at the end of 2024.

But standing out in this new search landscape requires more than just traditional SEO tactics. AI search operates differently, prioritizing conversational queries and delivering more dynamic results.

So, how can you ensure your website gets noticed and attracts traffic in this AI-driven age? In this article, I'll break down the different types of AI search platforms, provide tips for optimizing your content for search engines and large language models (LLMs), and equip you with the tools you need to succeed. Let's get started!

How AI search differs from conventional search engines

AI-powered search tools like ChatGPT function in a unique way compared to traditional search engines such as Google. These platforms rely on large language models (LLMs) to process queries and generate results. Here's a simple explanation:

Query processing

While search engines rank web pages based on relevance, AI tools like ChatGPT use pre-trained LLMs to understand user intent and provide answers. Instead of fetching real-time results from the web, LLMs predict responses based on their extensive training data, resulting in highly contextual and conversational outputs.

Results format

Google displays a list of links, snippets, and rich results for users to explore. AI tools, on the other hand, offer direct, conversational answers that save time but may reduce the need for multiple clicks.

Freshness of information

Search engines constantly update their index to reflect the latest content. AI search tools, unless connected to live updates, rely on the data within their LLMs, which can sometimes lead to outdated information.

Transparency

Google provides links to original sources for verification. Similarly, ChatGPT Search, when linked to browsing capabilities, includes source links for the information it provides. This helps users trace the origin of the answers. However, when ChatGPT operates without live browsing, it does not directly cite sources.

To optimize for both traditional search engines and AI platforms, you need to create content that is clear, authoritative, and rich in context. This type of content is easily processed by AI and can be incorporated into search results effectively.

Understanding different types of AI search

Various AI-powered search platforms have emerged, each with its unique strengths and approaches, driven by major tech companies. Let's take a closer look at some of the key players in the market.

ChatGPT search

Developed by OpenAI, ChatGPT revolutionizes the search experience by providing direct, conversational responses to user queries. Users can refine their questions in real-time, turning a single query into an ongoing conversation.

ChatGPT quickly gained popularity, reaching over 100 million users within two months of its launch in November 2022. By late 2024, it had over 300 million weekly active users, generating over 1 billion messages daily.

How it delivers search results: ChatGPT is powered by OpenAI's GPT-4 model, an LLM trained on vast text datasets. It doesn't crawl the web in real-time but relies on its pre-trained knowledge, periodically updated. When connected to live search capabilities, it provides real-time results and cites sources, offering a hybrid search experience.

Google's AI Overviews

Google AI Overviews integrate AI-powered summaries directly into search results, offering users quick insights by consolidating information from reliable sources. These summaries appear above traditional search listings, providing concise answers to help users find information efficiently.

How it works: Google AI Overviews utilize the advanced Gemini AI model, previously known as Bard, along with Google's Knowledge Graph. By analyzing content from authoritative sources, it generates precise summaries based on factors like search intent, semantic relevance, and content credibility.

Google Gemini

Gemini, developed by Google, is a generative AI chatbot that aims to compete with platforms like ChatGPT. Google plans to integrate Gemini more deeply into its products by 2025, leveraging its capabilities in language and visual processing.

How it delivers search results: Gemini combines Google's search algorithms with AI-powered technology to provide interactive, multimedia-rich answers, going beyond text-based responses.

Bing AI search

Microsoft's Bing has positioned itself as an AI-powered alternative to Google, incorporating OpenAI's GPT technology. Bing offers conversational AI features alongside traditional search results, emphasizing its commitment to AI-driven search.

How it delivers search results: Bing AI utilizes OpenAI's GPT-4 model to provide conversational answers and real-time web results. Unlike ChatGPT, Bing actively retrieves and ranks web pages while offering AI-generated summaries.

Other major players

Apple's AI-driven Spotlight offers web search capabilities directly on devices, personalizing results based on user behavior within the Apple ecosystem. Meta's AI-powered search focuses on personalized content discovery within its social platforms, while Amazon's AI search enhances eCommerce with personalized product recommendations.

In China and Russia, Baidu and Yandex lead AI searches, catering to region-specific results and integrating localized content and cultural nuances.

AI search optimization strategies: keyword research, content, and beyond

Now that we've covered the major AI search platforms, let's discuss practical strategies for optimizing your website to attract traffic in this evolving landscape.

To stay ahead in the AI-driven search environment, focus on mastering key strategies such as keyword research, creating high-quality content, and adapting to the changing search landscape.

1. Master Keyword Research for Conversational Queries

AI search platforms are designed to understand natural, conversational language, which makes traditional keyword tactics less effective. Instead of focusing solely on high-volume, generic terms, adapt your approach to match how users naturally phrase their queries.

Focus on Questions and Long-Tail Keywords
AI tools can handle more specific and detailed queries. Use tools like Rank Tracker to find question-based and long-tail keywords. Start with the Related Questions tool, then gather more ideas from ranking keywords and autocomplete suggestions.

If you already have a large list of keywords in your project and want to view only the long-tail phrases, you can apply the keyword length filter and then sort them by search volume or keyword difficulty.

Tip: Think Like Your Audience AI queries mimic how people speak. Explore “how-to” questions, comparisons, and specific needs to align with user intent.

2. Optimize Content for AI Friendliness

Producing well-researched, insightful content that establishes authority increases your chances of being referenced by AI-powered search engines. A strong foundation is aligning your content strategy with Google’s E-E-A-T principles: Experience, Expertise, Authoritativeness, and Trustworthiness. These guidelines emphasize the importance of creating content that both informs and builds credibility with users and search algorithms.

Once you have E-E-A-T in mind, shift your focus to crafting valuable, actionable content that resonates with your audience. Your content should go beyond surface-level information and include elements that enhance its credibility, such as:

Expert Insights: Incorporate quotes or perspectives from industry professionals to lend credibility and context.
Citations from Authoritative Sources: Reference reliable studies and reports to strengthen your claims.
Balanced Viewpoints (Pros and Cons): Presenting a well-rounded discussion makes your content more comprehensive and trustworthy.
Data-Backed Analysis: Use statistics and case studies to provide tangible evidence supporting key arguments.

Additionally, align your content with search intent—the underlying reason behind a user's query. Whether someone is looking for information, a product, or a solution to a problem, your content should anticipate and fulfill their needs. Failing to match search intent can result in high bounce rates and lost opportunities.

Work on Your Content Format and Structure:

Use a Question-Answer Format: Write headings and subheadings as questions to match AI-generated responses.
For example, use “How does AI search work?” instead of “AI Search Overview.”

Be Concise and Direct: AI platforms summarize content, so lead with short, valuable answers and follow up with more details. For instance, start with a 2-3 sentence summary, then expand with supporting information.
Structure Content Clearly: Use bullet points, numbered lists, and clear subheadings. AI favors well-organized content that’s easy to extract and present.

3. Optimize for Featured Snippets

Featured snippets are a key entry point for AI platforms like Google AIO. To increase your chances of being featured:

Answer Questions Clearly and Succinctly: Provide a direct, concise response at the start of your content, followed by supporting details.
Use Structured Data: Mark up your content with FAQ schema to make it more snippet-friendly.
Write Snippable Paragraphs and Lists: Ensure to include lists and short paragraphs that Google could use to generate a featured snippet. For example, instead of burying the answer in paragraphs, you could start with: “The best laptops for video editing in 2025 are [Brand A] and [Brand B].” Then expand with features and comparisons.

4. Focus on Entities and Topics for Google’s Knowledge Graph

Google’s Knowledge Graph plays a significant role in AI search, helping to connect entities like people, brands, and locations with related concepts. Optimizing for entities ensures Google understands your content in context.

How to Align with the Knowledge Graph:

Use Precise Terminology: Clearly define and mention entities (brand names, product categories, locations) in your content to enhance relevance.
Develop Content Clusters: Build comprehensive, interlinked content around core topics to signal topical authority.
Leverage Structured Data: As mentioned earlier, use schema markup (such as Organization, Product, and Event schema) to provide additional context about your content.

Focusing on entities improves your chances of appearing in AI-generated summaries and carousel-style search results.

5. Gain Mentions and Reviews

AI search engines consider brand mentions and user-generated content as credibility signals. Building a strong digital presence through mentions and reviews can boost your search visibility. Here’s how to do it:

Monitor Mentions: Use tools like Awario to create alerts for your brand name and target keywords to discover what people are saying about you.

6. Leverage content diversity

AI search engines prioritize multimedia-rich content, which enhances engagement and provides more comprehensive answers. Expanding beyond text can improve visibility and reach.
Content formats to consider:
Video tutorials, explainers, and interviews can rank in AI-generated results.
Infographics and visuals: simplify complex topics with engaging visual content.

Audio content: Podcasts and audio summaries resonate with voice search users and multitaskers.
Diversifying your content portfolio ensures your information is accessible across different AI search formats.

7. Optimize for technical SEO

Even in the AI era, a strong technical SEO foundation remains crucial. Ensuring your site is technically sound allows AI-powered search engines to crawl and interpret your content efficiently.
Here’s what you can do to improve your technical SEO:
Fix crawl errors: regular audits with WebSite Auditor can help you identify and resolve crawl and indexing issues.
Enhance page speed: use tools like PageSpeed Insights to identify and fix slow-loading elements.
Ensure mobile-friendliness: responsive design is critical, as AI search is heavily used on mobile devices.
Make sure your content is interconnected through strategic internal linking, guiding AI models to understand content hierarchies and topical relevance.
A technically optimized site enhances your chances of appearing in AI-powered search results and improves user experience.

Conclusion

AI-powered search engines are reshaping the SEO landscape, favoring conversational queries, multimedia content, and well-structured answers. Whether you’re optimizing for ChatGPT’s conversational responses, Google AIO’s featured snippets, or Google’s Knowledge Graph entities, the strategies remain rooted in creating user-focused, authoritative, and technically sound content.
By adopting these best practices, you’ll position your website to succeed not just in traditional search engines but also in the evolving world of AI-driven platforms. The key is to stay adaptable, keep your content valuable, and always think ahead to meet the needs of AI-powered search users.

Beyond Elasticsearch: The Rise of Unified Vector and Keyword Search

Wed, 22 Jan 2025 11:21:23 +0000

This post was written by James Luan, VP of Engineering at Zilliz and key maintainer of the Milvus open source project.

The Evolution of Modern Search Architecture

You're debugging a critical production issue, and you know the solution exists in your documentation. Your vector search returns semantically relevant results about error handling and system recovery but somehow misses that one crucial Stack Overflow answer you saved last month with the exact error code. Sound familiar?

This common frustration highlights a fundamental challenge in modern search technology: the need to balance semantic understanding with precise term matching. The rise of AI and large language models has brought powerful semantic search capabilities through vector embeddings while revealing the continued importance of traditional keyword matching. When supporting customers, for instance, vector search brilliantly handles natural language queries like "my payment isn't going through" by understanding user intent. Still, keyword search remains essential for matching specific product codes and error messages.

Recent experiences with RAG implementations across the industry have revealed interesting insights about search technology. While dense vector embeddings excel at capturing semantic relationships and contextual understanding, traditional BM25 keyword matching still proves superior in certain scenarios, particularly with domain-specific queries and rare terms. This complementary strength of different search approaches has pushed the community toward exploring unified solutions.

However, implementing hybrid search at scale presents its own set of challenges. While frameworks like LangChain or LlamaIndex have made proof-of-concept implementations straightforward, scaling these solutions to production reveals significant architectural complexities. The conventional approach requires maintaining separate vector databases and search engines, resulting in the following:

Data redundancy across multiple storage systems
Complex consistency management between separate indices
Fragmented security controls and access patterns
Increased latency due to cross-system synchronization
Higher operational complexity in high-throughput environments

An Example of Complex Architectures of Hybrid Search

Elasticsearch's Technical Limitations

As the long-standing leader in enterprise search, Elasticsearch has powered everything from e-commerce platforms to logging systems for over a decade. Their dominance in keyword-based search and their extensive enterprise feature set, made them an obvious candidate for bridging the gap between traditional and AI-powered search. Recognizing this opportunity, Elasticsearch made a significant move by introducing vector search capabilities in version 8.0, aiming to unify traditional keyword search with modern vector operations under a single platform.

However, despite its market leadership and ambitious steps toward vector search capabilities, Elasticsearch faces several fundamental constraints that impact its effectiveness in modern search architectures:

Performance and Resource Management

The tight coupling between write operations, index construction, and query processing creates substantial overhead during data ingestion and indexing. This architectural decision leads to CPU and I/O contention, particularly during bulk updates. The "near real-time" search design introduces latency that can impair AI applications requiring rapid interactions or dynamic decision-making.

Architectural Constraints

The JVM-based architecture introduces additional complexity through memory management overhead and garbage collection pauses. Organizations often find themselves over-provisioning resources to maintain stable performance as data volumes grow. The non-cloud-native design, with tightly coupled storage and compute, requires simultaneous scaling of both resources, reducing deployment flexibility.

Scaling Limitations

The shard management system presents a fundamental dilemma: too many shards in small datasets lead to performance degradation. However, too few shards in large datasets limit scalability and cause data distribution imbalances. In multi-replica scenarios, the requirement for independent index construction per shard multiplies compute costs and reduces resource efficiency.

Vector Search Performance

The vector search implementation, built on Lucene's core, shows notable performance gaps compared to specialized engines, particularly in high-dimensional spaces. Internal benchmarks indicate performance degradation becomes acute beyond 128 dimensions, with query latency increasing exponentially in high-concurrency scenarios.

Technical Innovation: A New Approach to Unified Search

Recent advances in search technology have emerged from extensive experience with real-world search challenges across thousands of production deployments. A consistent pattern has emerged through deep engagement with the open-source community. While vector search excels at semantic matching, organizations often need to maintain separate systems for traditional keyword search, leading to architectural complexities and operational overhead.

This insight led to a fundamental reimagining of how unified search capabilities could work in the open source vector database Milvus. The result is Sparse-BM25, introduced by Milvus 2.5, an enhanced approach that leverages vector operations while preserving the precision of traditional keyword matching. This technical innovation builds on Milvus's expertise in vector operations while addressing the broader community's need for unified search capabilities.

Core Technical Architecture

The implementation introduces several key technical innovations:

Advanced text processing pipeline powered by Tantivy, delivering
sophisticated stemming, lemmatization, and stop word filtering
capabilities with support for over 30 languages
Distributed term frequency management system that efficiently handles
corpus-wide statistics at scale, with automatic rebalancing and
optimization
Novel sparse vector generation utilizing Corpus TF for documents and
Query TF with global IDF for queries, optimized for high-cardinality
vocabularies
Custom BM25 distance function optimized for high-dimensional sparse
vector spaces, with configurable saturation parameters

The system implements a sophisticated indexing hierarchy that adapts to query patterns. The WAND-based inverted index provides efficient retrieval for standard queries, while the Block-Max WAND algorithm development promises further optimization for long-tail queries. Through careful pruning of long-tail terms and vector quantization, the system achieves more than 5x performance improvement while keeping recall degradation under 1% and reducing memory usage by over 50%.

The graph index support extends beyond basic HNSW implementation, incorporating dynamic graph pruning and rebuilding based on query patterns. This approach has shown particular effectiveness in handling high-dimensional sparse vectors, where traditional inverted indices struggle with the curse of dimensionality. The system automatically maintains optimal graph connectivity while minimizing index maintenance overhead.

Technical Advances in Modern Search Systems

Algorithm Flexibility

Modern search systems transform similarity calculations into vector distance computations, enabling more complex queries and corpus distance analysis. Recent research like "End-to-End Query Term Weighting" has introduced innovations such as the Term Weighting BERT (TW-BERT) algorithm, which uses BERT to infer n-gram term weights from queries and construct query representations. When combined with BM25 for calculating document relevance, these approaches demonstrate significant improvements over traditional token-based methods in both in-domain and out-of-domain scenarios.

Efficiency Considerations

Advanced search architectures implement lexical search through sparse vectors, combining traditional inverted index compression techniques with lossy compression for dense embeddings. Through careful pruning of long-tail terms and vector quantization, these systems can achieve substantial performance improvements while maintaining high recall rates and reducing resource usage. Ongoing research continues to optimize data compression to further reduce storage costs and query I/O overhead.

Handling Complex Queries

While traditional search engines commonly use WAND (Weak AND) technology to optimize inverted index queries, its effectiveness can diminish with long queries due to excessive inverted list intersections and reduced pruning efficiency. Modern approaches address this by combining sparse embeddings with graph indices like HNSW, showing particular promise for sparse vectors with high dimensionality.

These technical advances in unified vector and keyword search are particularly significant for RAG applications, where queries often combine natural language questions with specific technical terms or metadata filters. Efficiently processing these mixed queries while maintaining high performance is crucial for production deployments.

Looking Forward: The Future of Search

Remember that frustrated developer from our opening scenario? Today's reality is that teams shouldn't have to choose between semantic understanding and precise matching, performance and flexibility, or simplicity and scale. As our industry continues to push the boundaries of AI and search technology, we're moving toward a future where search infrastructure becomes truly intelligent - understanding not just what users are searching for but why they're searching for it.

The next frontier isn't just about combining vector and keyword search - it's about creating systems that can dynamically adapt to different types of queries, learn from user interactions, and seamlessly integrate with the growing ecosystem of AI tools and frameworks. As we continue to build and experiment with these technologies in Milvus, we're not just solving technical challenges - we're reshaping how humans interact with and make sense of the vast amounts of information at our disposal.

This is an incredibly exciting time for the developers and engineers working on these challenges every day. The tools and capabilities we're building today will form the foundation of how future generations will access and understand information. And that's worth getting right.