Vector Databases: The Memory Layer of Modern AI

Before RAG was on every slide deck, the infrastructure that makes it work was already maturing in the open. Vector search isn't new. What changed is the part where embedding models got good enough that the math actually pays off.

Every retrieval-augmented system, every semantic search engine, every recommender that understands meaning instead of keywords leans on the same primitive: find the nearest points to a query in a high-dimensional space. Vector databases exist to make that operation fast, scalable, and not a research project. If you're building anything that retrieves by meaning, you'll end up using one. Worth knowing how they actually work before you pick.

Why traditional databases fail for semantic search

A PostgreSQL WHERE clause matches exactly. LIKE matches patterns. Full-text search matches tokens. None of those capture meaning.

"How do I fix a memory leak in Python?" and "Python application consuming too much RAM" mean the same thing, but they share almost no words. Keyword search misses the connection. Their embedding vectors, on the other hand, will sit right next to each other in vector space.

That's the shift. Search by what text means instead of what it contains. And "closeness in meaning" maps to "closeness in vector space," which becomes a nearest-neighbor problem over high-dimensional vectors.

Traditional databases aren't built for this. A B-tree index assumes a natural ordering it can binary-search through; in 1,536 dimensions there's nothing to binary-search. Computing the distance between a query and every stored vector works fine for thousands of documents, falls apart at millions, and is hopeless at billions.

Embeddings recap

Quick reminder of what we're storing.

An embedding is a dense vector, typically 256 to 3,072 floating-point numbers, that represents the semantic content of a piece of text or image. The model that produced it was trained so that semantically similar inputs land near each other, measured by cosine similarity — the cosine of the angle between two vectors:

\cos(\theta) = \frac{\mathbf{a} \cdot \mathbf{b}}{\lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert} = \frac{\sum_i a_i b_i}{\sqrt{\sum_i a_i^2}\,\sqrt{\sum_i b_i^2}}

It ranges from $-1$ (opposite) through $0$ (orthogonal, unrelated) to $1$ (identical direction). For unit-normalized vectors the denominator is 1, so cosine similarity reduces to the plain dot product $\mathbf{a} \cdot \mathbf{b}$ — which is why most vector stores normalize on the way in.

from openai import OpenAI

client = OpenAI()

response = client.embeddings.create(
    model="text-embedding-3-small",
    input="How do I fix a memory leak in Python?"
)

vector = response.data[0].embedding  # list of 1536 floats
print(len(vector))  # 1536
print(vector[:5])   # [0.0023, -0.0091, 0.0142, ...]

Your retrieval quality is bounded by the embedding model. The best vector database in the world can't find a relevant document if the embedding model didn't put it near the query. Picking the right model and chunking the documents well matters at least as much as picking the right database.

What a vector database actually does

At its core, four operations:

Insert — store a vector with metadata (document ID, source URL, timestamps, tags).
Index — build a structure that enables fast approximate nearest-neighbor (ANN) search.
Query — given a query vector and a value of k, return the k most similar stored vectors.
Filter — combine vector similarity with metadata predicates ("most similar 10 docs that are also tagged 'engineering' and created after 2024-01-01").

The hard part is operation 3. Exact nearest-neighbor search is O(n) with high constants because of the dimensionality. A million vectors at 1,536 dimensions means over 6 billion floating-point ops per query. Approximate algorithms trade a small bit of accuracy for orders of magnitude in speed.

ANN algorithms: the engine room

Three algorithm families dominate.

HNSW (Hierarchical Navigable Small World)

The most-used ANN algorithm in production. It builds a multi-layer graph where each node is a vector and edges connect nearby ones. The top layer is sparse with long-range connections. The bottom is dense with local connections.

A query starts at the top, greedily walks toward the target vector, drops to a lower layer when it stops making progress. Like zooming in on a map: continent, country, city, street.

Pros: very high recall at very high speed, works well in memory, no training step. Cons: high memory use because of the graph, insert time grows logarithmically, not great for datasets that change rapidly.

The two tuning knobs are M (edges per node — higher means better recall and more memory) and ef_construction / ef_search (candidates explored — higher means better recall and slower queries).

IVF (Inverted File Index)

IVF partitions the vector space into clusters using k-means. At query time, only the clusters near the query are searched. That cuts the comparisons from N to roughly N/k * nprobe, where nprobe is how many clusters you check.

Training phase:
  1. Run k-means on a sample of vectors → k centroids
  2. Assign each vector to its nearest centroid

Query phase:
  1. Find the nprobe closest centroids to the query
  2. Search only vectors assigned to those clusters
  3. Return top-k results

Pros: lower memory overhead than HNSW, scales to very large datasets. Cons: needs a training phase, recall degrades when clusters are uneven, weak at low nprobe.

PQ (Product Quantization)

PQ compresses vectors to save memory and speed up distance math. It splits each vector into subvectors and quantizes each one into one of 256 centroids (1 byte). A 1,536-dimensional float32 vector (6,144 bytes) shrinks to 192 bytes — 32x smaller.

PQ is almost always paired with IVF as IVF-PQ: IVF for coarse search, PQ for compressed distance computation inside clusters. That combination powers billion-scale systems at Meta and Spotify.

Pros: massive memory savings, datasets that would never fit in RAM start to fit. Cons: lossy compression hurts accuracy, training step required, overkill for small datasets.

The key players

The market has gotten crowded. A pragmatic breakdown:

Pinecone — fully managed, serverless, zero ops. You hit an API and it handles indexing, scaling, replication. Best for teams that want to ship without running infrastructure. Pays for it in cost and lock-in.

Qdrant — open source, written in Rust, fast. Rich filtering, sparse vectors, hybrid search (dense plus sparse). Self-host or use their managed cloud.

Weaviate — open source, focused on developer ergonomics. Has built-in vectorization modules so you can hand it raw text and it embeds for you. Convenient if you want that, extra abstraction if you don't.

Chroma — lightweight, Python-native, designed for prototyping and small-to-medium loads. In-process or as a server. The fastest way to a working RAG demo in an afternoon.

pgvector — a PostgreSQL extension. Worth its own section.

pgvector: vector search in your existing stack

Most teams already run PostgreSQL. If that's you, pgvector is hard to argue with. Instead of adding a new database to the architecture, you add an extension to the one you already operate.

-- Enable the extension
CREATE EXTENSION vector;

-- Create a table with a vector column
CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    title TEXT,
    content TEXT,
    source TEXT,
    created_at TIMESTAMP DEFAULT NOW(),
    embedding vector(1536)
);

-- Insert a document with its embedding
INSERT INTO documents (title, content, source, embedding)
VALUES (
    'Memory Management in Python',
    'Python uses reference counting and a cyclic garbage collector...',
    'internal-docs',
    '[0.0023, -0.0091, 0.0142, ...]'  -- 1536-dimensional vector
);

Queries are familiar SQL with vector operators:

-- Find the 5 most similar documents to a query vector
SELECT id, title, content,
       embedding <=> '[0.0031, -0.0087, ...]'::vector AS distance
FROM documents
ORDER BY embedding <=> '[0.0031, -0.0087, ...]'::vector
LIMIT 5;

The <=> operator is cosine distance. pgvector also supports <-> (L2) and <#> (negative inner product).

For real performance, you need an index. pgvector supports both IVFFlat and HNSW:

-- Create an HNSW index (default for most use cases)
CREATE INDEX ON documents
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 200);

-- Or IVFFlat for larger datasets with memory constraints
CREATE INDEX ON documents
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);

The strength of pgvector is transactional consistency. Vectors and metadata live in the same database, participate in the same transactions, get backed up together. You can join vector results with relational data in one query:

-- Semantic search with metadata filtering
SELECT d.id, d.title, d.content, t.name AS tag
FROM documents d
JOIN document_tags dt ON d.id = dt.document_id
JOIN tags t ON dt.tag_id = t.id
WHERE t.name = 'engineering'
  AND d.created_at > '2024-01-01'
ORDER BY d.embedding <=> $1::vector
LIMIT 10;

The limit is scale. pgvector performs well up to roughly 1-5 million vectors, depending on dimensionality and hardware. Past that, the dedicated distributed vector databases (Qdrant, Pinecone, Milvus) handle the load better.

Metadata filtering: the hybrid pattern

Pure vector search retrieves by meaning. Real apps almost always combine that with structured constraints: "Find similar documents, but only from this department, created in the last 30 days, with a confidence score above 0.8."

How a database handles this matters a lot for query performance. Two strategies:

Pre-filtering applies metadata predicates first, then searches the filtered subset. Efficient when filters are very selective. Risk is missing semantically relevant results that fall outside the filter.

Post-filtering does vector search first, retrieves more candidates than needed, then applies metadata filters. Preserves recall, wastes compute on candidates that get thrown out.

Most production systems use a hybrid: loose pre-filtering to narrow the candidate set, then vector search, then post-filter for any remaining predicates.

Picking the right tool

The decision tree is simpler than the marketing implies:

Already using PostgreSQL and under 5M vectors? Start with pgvector. Less operational surface, transactions for free.
Need managed simplicity and okay paying for it? Pinecone gets you to production fastest.
Need control, performance, open source? Qdrant or Milvus, self-hosted.
Prototyping this afternoon? Chroma in-memory, zero config.

The vector database is the memory layer of modern AI. It's where embeddings live, where semantic search happens, where RAG finds the context that keeps an LLM grounded. The algorithms are mature, the tooling is production-ready, and the hardest part is no longer the infrastructure. It's the embedding model and chunking strategy you feed it.