You have embeddings to store and you want similarity search that stays fast as the collection grows, without renting a managed service and shipping your vectors to someone else’s cloud. To install Milvus on your own Ubuntu box gives you exactly that: an open-source vector database that indexes billions of vectors and answers nearest-neighbour queries in milliseconds, running entirely on hardware you control.
This guide walks the whole path on Ubuntu. You deploy Milvus in standalone mode with Docker Compose, confirm it works with a real Python client that creates a collection and runs a vector search, then install the Attu web dashboard and put it behind Nginx with a valid TLS certificate. Along the way I flag the one requirement that trips people up on virtual machines, because I hit it myself before this ran clean.
I ran this end to end on Ubuntu 26.04 (Docker 29.6, Milvus 2.6) in July 2026, and the same commands work on Ubuntu 24.04.
How Milvus is put together
Milvus ships in two shapes. Standalone runs the whole database on one machine and is what you want for a homelab, a single app, or anything up to a few million vectors. Distributed spreads the query, data, and index nodes across a Kubernetes cluster for the billion-vector tier. This guide covers standalone, which is the sensible starting point and the one most self-hosters actually need.
Even in standalone mode Milvus is not a single process. The Compose file brings up three containers: milvus-etcd holds metadata, milvus-minio is the S3-compatible object store for the actual vector data and index files, and milvus-standalone is the engine that ties them together and serves the API. The distributed deployment leans on a separate Pulsar or Kafka broker for its write-ahead log, but standalone has always embedded its own message queue (Milvus calls the current one Woodpecker), so there is no broker to run alongside these three. Client applications talk to the engine on TCP port 19530, and a health and metrics endpoint sits on 9091.
Prerequisites
Milvus is more demanding than a typical web app because the engine memory-maps index segments and the object store buffers writes. Size the host from the working set: the vectors and indexes you expect to keep hot should fit comfortably in RAM, so a small RAG store of a few hundred thousand embeddings is happy on modest hardware, while a multi-million-vector collection wants far more headroom. Milvus recommends 8 GB of RAM and 4 CPU cores as a practical minimum and 16 GB or more for real workloads, backed by an SSD because index builds and etcd are both disk-latency sensitive. The lab for this guide ran on 8 GB and 4 vCPUs, which is a floor for following along rather than a production recommendation.
- Ubuntu 26.04 or 24.04 with a user that has
sudo - A CPU that supports the AVX2 and SSE4.2 instruction sets (nearly every physical CPU since 2013; see the note in step 2 if you are on a VM)
- At least 8 GB RAM, 4 cores, and 20 GB of free SSD space
- Outbound internet access to pull container images
1. Install Docker Engine and the Compose plugin
Milvus standalone is distributed as containers, so Docker Engine plus the Compose plugin is the only runtime you need. Skip the docker.io package in the Ubuntu archive; it lags well behind. Add Docker’s own repository instead, which is keyed to your release codename and works the same on 26.04 (resolute) and 24.04 (noble):
sudo apt-get update
sudo apt-get install -y ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.listRefresh the package index and install the engine, CLI, and the Compose plugin:
sudo apt-get update
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-pluginAdd your user to the docker group so you can run the client without sudo, then log out and back in for the group change to take effect:
sudo usermod -aG docker $USERConfirm both pieces are present before moving on:
docker --version
docker compose versionThe two lines report the engine and the Compose plugin independently:
Docker version 29.6.2, build dfc4efb
Docker Compose version v5.3.1If you want the reasoning behind each step or run into a repository hiccup, the dedicated Docker Compose install guide covers it in more depth.
2. Deploy Milvus standalone with Docker Compose
Milvus publishes a ready-made Compose file with every release. Grab the one attached to the latest stable build, which always resolves to the current release, and keep it in its own directory so the data volumes land somewhere predictable:
mkdir -p ~/milvus && cd ~/milvus
wget https://github.com/milvus-io/milvus/releases/latest/download/milvus-standalone-docker-compose.yml -O docker-compose.ymlBring the stack up in the background:
docker compose up -dDocker pulls etcd, MinIO, and the Milvus image, then starts all three in order. Give it a minute on the first run, then check the state of each container:
docker compose psAll three should read healthy, with the engine exposing the client port 19530 and the metrics port 9091:
NAME STATUS PORTS
milvus-etcd Up 29 minutes (healthy) 2379-2380/tcp
milvus-minio Up 29 minutes (healthy) 0.0.0.0:9000-9001->9000-9001/tcp
milvus-standalone Up 29 minutes (healthy) 0.0.0.0:9091->9091/tcp, 0.0.0.0:19530->19530/tcpThe engine exposes a health endpoint that is handy for scripts and uptime checks. A ready instance answers with a plain OK:
curl http://localhost:9091/healthzIf milvus-standalone keeps restarting, check for AVX
Here is the gotcha that cost me time. Milvus computes vector distances with hand-tuned SIMD code, and the engine refuses to start on a CPU that lacks the AVX instruction set. On bare metal you almost never hit this. On a virtual machine you can, because many hypervisors present a generic emulated CPU by default that hides the host’s AVX flags. When that happens the milvus-standalone container crash-loops and its logs end in Illegal instruction.
Check what your CPU advertises:
grep -o -m1 'avx2\|avx' /proc/cpuinfoIf that prints avx2 (or at least avx), you are fine. If it prints nothing on a VM, the fix is to pass the host CPU through instead of the emulated default. On Proxmox set the VM’s CPU type to host; on plain KVM or libvirt use -cpu host or the host-passthrough model, then fully power the guest off and on so the new CPU model takes effect. A reboot alone is not enough. If you genuinely cannot expose AVX, Milvus ships an alternative Compose file that swaps MinIO and etcd for a lighter backend, but exposing the host CPU is the clean answer.
3. Verify Milvus with the Python client
A healthy container is not the same as a working database. The honest test is to connect, create a collection, insert vectors, and get sensible results back from a search. The official pymilvus client makes that a short script. Install it in a virtual environment so it does not touch the system Python:
sudo apt-get install -y python3-venv python3-pip
python3 -m venv ~/mv-demo
~/mv-demo/bin/pip install --upgrade pip pymilvusOpen a small demo script:
vim ~/milvus/quickstart.pyPaste in the following. It creates an eight-dimension collection, inserts four short documents with random vectors, flushes them so they are searchable, then runs a nearest-neighbour query:
import random, time
from pymilvus import MilvusClient
client = MilvusClient(uri="http://localhost:19530")
print("Milvus server:", client.get_server_version())
if client.has_collection("demo_collection"):
client.drop_collection("demo_collection")
client.create_collection(collection_name="demo_collection", dimension=8)
docs = [
"Milvus is an open-source vector database",
"Docker Compose runs multi-container apps",
"Ubuntu 26.04 ships with Linux kernel 7.0",
"Vector search powers RAG pipelines",
]
data = [{"id": i, "vector": [random.random() for _ in range(8)], "text": docs[i]}
for i in range(len(docs))]
print("Inserted entities:", client.insert("demo_collection", data)["insert_count"])
client.flush("demo_collection")
time.sleep(3)
query = [[random.random() for _ in range(8)]]
results = client.search("demo_collection", data=query, limit=3,
output_fields=["text"], consistency_level="Strong")
for hits in results:
for hit in hits:
print(" id={} distance={:.4f} text={}".format(
hit["id"], hit["distance"], hit["entity"]["text"]))Run it against the running instance:
~/mv-demo/bin/python ~/milvus/quickstart.pyThe client reports the server version, confirms the insert, and ranks the documents by cosine distance to the random query vector:
Milvus server: 2.6.21
Inserted entities: 4
id=3 distance=0.9196 text=Vector search powers RAG pipelines
id=2 distance=0.7475 text=Ubuntu 26.04 ships with Linux kernel 7.0
id=1 distance=0.6982 text=Docker Compose runs multi-container appsThat is a real round trip through the engine: schema creation, an insert, a flush, and an indexed search. Swap the random vectors for embeddings from a local model served by Ollama and you have the storage layer for a retrieval pipeline. If you are wiring one together, the walkthrough on building a self-hosted RAG with Ollama and LangChain shows how the pieces fit, and Milvus drops into the same slot a vector store like pgvector would fill.
4. Install the Attu web GUI
Working entirely from the client is fine for automation, but a dashboard makes it far easier to inspect collections, watch segment counts, and eyeball your data. Attu is the official GUI for Milvus, shipped as a single container. Point it at the engine with the MILVUS_URL variable (use the machine’s LAN address, not localhost, since the value is read inside the container) and map its web port to 8000 on the host:
docker run -d --name attu --restart unless-stopped \
-p 8000:3000 \
-e MILVUS_URL=http://10.0.1.50:19530 \
zilliz/attu:v2.6Replace 10.0.1.50 with your server’s address. Confirm the container is up and serving:
docker ps --filter name=attu --format "{{.Image}} {{.Status}} {{.Ports}}"You can now reach Attu at http://your-server:8000. That is enough to browse locally, but exposing an unencrypted dashboard on the network is a bad habit, so the next step wraps it in HTTPS.
5. Secure Attu with Nginx and HTTPS
Attu talks to your database and shows every collection you own, so it belongs behind TLS. Nginx in front of the container handles the certificate and proxies traffic to port 8000. Point a DNS A record at the server first, then set two shell variables so you change them once and paste the rest as is:
export SITE_DOMAIN="milvus.example.com"
export ADMIN_EMAIL="[email protected]"Install Nginx and the Certbot client:
sudo apt-get install -y nginx certbot python3-certbot-nginxCreate a server block for the dashboard. Attu is a single-page app that expects connection upgrades to pass straight through, so the proxy forwards the WebSocket headers to keep every panel responsive:
sudo vim /etc/nginx/sites-available/attuAdd the following, using a placeholder for the hostname that a later command fills in:
server {
listen 80;
server_name SITE_DOMAIN_HERE;
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}Substitute your real hostname into the file, enable the site, drop the default one, and reload:
sudo sed -i "s/SITE_DOMAIN_HERE/${SITE_DOMAIN}/" /etc/nginx/sites-available/attu
sudo ln -s /etc/nginx/sites-available/attu /etc/nginx/sites-enabled/attu
sudo rm -f /etc/nginx/sites-enabled/default
sudo nginx -t && sudo systemctl reload nginxNow issue a Let’s Encrypt certificate. With port 80 reachable from the internet, the Nginx plugin validates the domain over HTTP-01 and rewrites the server block for TLS in one shot:
sudo certbot --nginx -d "${SITE_DOMAIN}" --non-interactive --agree-tos --redirect -m "${ADMIN_EMAIL}"Open the firewall for HTTP and HTTPS if UFW is active:
sudo ufw allow 'Nginx Full'If the server sits on a private network with no public port 80, skip the plugin and validate over DNS instead. Certbot has plugins for Cloudflare, Route 53, DigitalOcean, Google Cloud DNS, and others; install the matching one first (for Cloudflare that is python3-certbot-dns-cloudflare), then the flow is certbot certonly --dns-cloudflare --dns-cloudflare-credentials /etc/letsencrypt/cloudflare.ini -d "${SITE_DOMAIN}" after dropping your API token in that file. Substitute your provider’s plugin if you are not on Cloudflare, then reference the issued certificate in the Nginx ssl_certificate directives by hand.
Browse to https://milvus.example.com and Attu greets you with its connection screen. The Milvus address is prefilled from the container environment, so click Connect:
Once connected, the landing page confirms the deployment: the server version, the standalone deploy mode, uptime, and the single default database. This is the fastest way to prove the whole stack is talking end to end.
6. Manage collections and run vector search in Attu
Open the default database and the demo_collection you created earlier. The schema view lays out the fields, the primary key, and the index. Notice that the vector field carries an AUTOINDEX with a cosine metric, which is what let the Python search return ranked results without you defining an index by hand:
The Data tab is where the dashboard earns its place. It lists the entities you inserted, the raw vectors, and the dynamic text field, and it runs queries with a filter expression right in the browser. The Vector Search tab next to it lets you paste a query vector and see the nearest neighbours without writing any code, which is genuinely useful when you are debugging why a search returns what it does:
Milvus also ships a lightweight built-in web console at http://your-server:9091/webui for quick health and segment inspection, but Attu is the tool you will actually live in for day-to-day collection work.
What I’d change before production
The setup above is solid for a homelab, a staging environment, or a single application, but a few things are worth tightening before you point real traffic at it. Milvus starts with authentication disabled, so anyone who reaches port 19530 owns your data; turn on user authentication in the config and never expose 19530 to the open internet. The bundled MinIO and etcd are single instances with no redundancy, so for anything you cannot lose, back the object store with a real S3 bucket or an external MinIO cluster and snapshot the etcd data directory on a schedule.
When one host stops keeping up, that is the signal to move from standalone to the distributed deployment on Kubernetes, where query and data nodes scale independently. Milvus is not the only option at this tier either. If you would rather run a database that is a single binary with no etcd or object store to manage, the Qdrant vector database covers that ground, and it is worth weighing the two against your own workload before you commit. Whichever you land on, you now have a tested, self-hosted starting point that costs nothing but the box it runs on.
