"How do I fix a memory leak in Python?" Your documentation has an article called "Python application consuming too much RAM." That's a promising match, even though the wording differs. Now imagine the search finds ten related pages, but nine belong to another department. The one page you needed never made the list.

Vector search helps with the wording problem. Making it fast introduces a second problem: deciding which documents to examine at all. Understanding that decision explains why a search can be impressively quick and still leave useful material behind.

Turning a question into coordinates

An embedding model converts text into a list of numbers, called a vector. Its training makes some relationships in language appear as geometric proximity. Our memory-leak question and RAM article may land near each other because they concern related concepts. Whether they actually do is something to check with the chosen model; the titles alone don't establish that the article contains a fix.

Keyword search still earns its place, especially for exact error messages and identifiers. Embeddings add a way to retrieve relevant wording that the query did not contain. The database's job is to search those coordinates efficiently.

A common comparison is cosine similarity, the cosine of the angle between two vectors:

cos⁡(θ)=a⋅b∥a∥ ∥b∥=∑iaibi∑iai2 ∑ibi2\cos(\theta) = \frac{\mathbf{a} \cdot \mathbf{b}}{\lVert \mathbf{a} \rVert \, \lVert \mathbf{b} \rVert} = \frac{\sum_i a_i b_i}{\sqrt{\sum_i a_i^2}\,\sqrt{\sum_i b_i^2}}

For nonzero vectors, the mathematical range runs from −1-1 for opposite directions to 11 for the same direction; 00 means perpendicular. Those are geometric descriptions. A score of zero doesn't certify that two documents have nothing to do with each other. With unit-normalized vectors, the denominator is one and the calculation reduces to a dot product.

Here's a Python example that embeds the question:

from openai import OpenAI client = OpenAI() response = client.embeddings.create( model="text-embedding-3-small", input="How do I fix a memory leak in Python?" ) vector = response.data[0].embedding # list of 1536 floats print(len(vector)) # 1536 print(vector[:5]) # inspect the actual values returned

The default output of text-embedding-3-small has 1,536 dimensions, and OpenAI documents its embeddings as unit-normalized. Other models can use different dimensions and normalization conventions, so check what your distance calculation expects. OpenAI embedding guide.

Documents need embeddings too, produced in a compatible space. Chunking matters here: bury the useful Python explanation inside a large, mixed-topic passage and its vector may be a poor representative of the answer. A search index can retrieve nearby vectors efficiently; it cannot repair a representation that obscures what you need.

The price of examining everything

The simplest search compares the question's vector with every document vector, sorts by distance, and returns the nearest five or ten. That gives us an exact reference result for the chosen distance metric.

The arithmetic grows with both the document count and the vector length. One million vectors with 1,536 float32 coordinates contain 1.536 billion coordinates, or 6.144 billion bytes of raw vector data. That's a storage calculation, not a measured query time. Hardware, batching, memory bandwidth, and how many searches arrive together determine whether scanning them is acceptable.

An ordinary B-tree index doesn't directly solve this distance-ranking problem. A database can, however, add specialized vector indexes. There's no need to declare relational databases obsolete to explain why a different index helps.

Approximate nearest-neighbor search, or ANN, avoids examining every vector. It tries to find a good candidate set quickly, accepting that some exact nearest neighbors may be missed. We measure that loss with recall. In an illustrative search for ten neighbors, recovering eight of the exact top ten gives recall@10 of 80%. That says how well the index reproduced the exact vector ranking. It doesn't say that 80% of the results answer the user's question.

This distinction is useful when debugging: poor agreement with exact search points toward the index settings; poor answers even from exact search point toward the representation, document collection, or relevance criteria.

How the shortcuts work

HNSW follows connections between nearby vectors

Hierarchical Navigable Small World, or HNSW, organizes vectors into a graph with several layers. Upper layers contain fewer points and help a search move across the space. Lower layers refine the search among more local neighbors. Think of using major roads to get near a destination before navigating side streets. The graph links are the roads; similarity tells the search which direction looks promising. Original HNSW paper.

The shortcut can miss a useful neighborhood. Exploring more candidates improves the chance of finding it, at the cost of more work. Common parameters separate graph construction from querying: M controls connectivity, ef_construction controls construction effort, and ef_search controls query exploration. More connections also consume memory. HNSW can accept incremental insertions without a separate clustering-training phase, but sustained insertion, deletion, and querying still need workload testing. HNSW implementation parameters.

IVF chooses neighborhoods before searching them

An inverted file index, or IVF, groups vectors around representative centers, often learned with k-means. It searches selected groups instead of the entire collection:

Training phase:
  1. Run k-means on a sample of vectors → nlist centroids
  2. Assign each vector to its nearest centroid

Query phase:
  1. Find the nprobe closest centroids to the query
  2. Search only vectors assigned to those clusters
  3. Return top-k results

Using nlist for the number of groups avoids confusing it with k, the number of answers requested. With equally sized groups, the candidate count is roughly N × nprobe / nlist. Uneven groups change that estimate. The important failure is easy to picture: if the RAM article belongs to a group we didn't search, it cannot appear in the answer. Increasing nprobe searches more groups and costs more time. Faiss IVF explanation.

PQ makes each vector cheaper to store

Product quantization, or PQ, addresses a different cost. It divides a vector into smaller pieces and replaces each piece with a code identifying a learned representative. With 256 representatives per piece, each code needs one byte. IVF can select groups while PQ compresses the vectors inside them. Compression adds approximation to the distance calculation. Faiss index encodings.

The compression ratio depends on the chosen configuration. For an illustrative 1,536-dimensional float32 vector, the uncompressed size is 1,536 × 4 = 6,144 bytes. Split it into 192 pieces of eight dimensions and assign an eight-bit code to each: the codes occupy 192 bytes, a 32-fold reduction. That calculation excludes codebooks, document identifiers, index structures, metadata, and any retained original vectors. It describes the vector codes, not a promise that the whole database becomes 32 times smaller.

Ten neighbors can become one usable result

Return to the Python question. Suppose we need ten documents tagged engineering, created after January 1, 2024. These conditions define which documents are eligible.

Consider a deliberately simplified candidate list: ANN returns 20 documents, and only two pass both conditions. Returning those two does not mean only two eligible documents exist. The other eight we wanted may sit just beyond the candidate cutoff, or in a neighborhood the approximate search skipped.

Filtering first removes ineligible documents before similarity ranking. If that leaves a small collection, an exact scan may be quite reasonable. Excluding documents outside a required filter is correct behavior. The relevant recall comparison is against the nearest neighbors inside the eligible set.

Filtering afterward discards candidates that fail the conditions. Asking for extra candidates can help, but it does not guarantee enough results or preserve recall. Even an exact top-20 search across all documents can miss the best ten engineering documents if most of the first 20 belong elsewhere.

Implementations can combine filtering with index traversal. In pgvector, approximate index scans apply filters after scanning; iterative scans can continue looking for matches, subject to scan limits. That gives a concrete reason to test restrictive filters alongside the unrestricted query. pgvector filtering documentation.

The practical question is how much extra work it takes to find enough eligible neighbors. A benchmark with no filters leaves that question unanswered.

Following the query in SQL

These pgvector examples make the path visible: store the document and vector, request nearest neighbors, add an index, then apply metadata conditions. The separate PostgreSQL article covers the integration decision.

These SQL examples use $1 as an application-bound parameter containing a complete vector. An abbreviated string with ... is not valid vector data. The document insertion requires its own embedding; the nearest-neighbor queries receive the question's embedding.

-- Enable the extension CREATE EXTENSION vector; -- Create a table with a vector column CREATE TABLE documents ( id SERIAL PRIMARY KEY, title TEXT, content TEXT, source TEXT, created_at TIMESTAMP DEFAULT NOW(), embedding vector(1536) ); -- Application binds $1 to the complete document embedding INSERT INTO documents (title, content, source, embedding) VALUES ( 'Memory Management in Python', 'Python uses reference counting and a cyclic garbage collector...', 'internal-docs', $1::vector -- all 1536 values, not an abbreviated vector );
-- Find the 5 most similar documents to a query vector SELECT id, title, content, embedding <=> $1::vector AS distance FROM documents ORDER BY embedding <=> $1::vector LIMIT 5;

<=> computes cosine distance, so smaller values sort first. The other operators include <-> for L2 distance and <#> for negative inner product. These examples and index options follow the pgvector documentation.

-- HNSW example: tune these values against your workload CREATE INDEX ON documents USING hnsw (embedding vector_cosine_ops) WITH (m = 16, ef_construction = 200); -- Alternative: build IVFFlat after loading representative vectors CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops) WITH (lists = 100);

Choose one index alternative for a comparison rather than assuming that creating both measures either one fairly. The numerical settings above are example configurations, not benchmark winners.

The filtered query joins document metadata, assuming the application also has document_tags and tags tables:

-- Semantic search with metadata filtering SELECT d.id, d.title, d.content, t.name AS tag FROM documents d JOIN document_tags dt ON d.id = dt.document_id JOIN tags t ON dt.tag_id = t.id WHERE t.name = 'engineering' AND d.created_at > '2024-01-01' ORDER BY d.embedding <=> $1::vector LIMIT 10;

Now the search must satisfy both similarity and eligibility. Inspect its execution plan and results with representative data; SQL expresses what you want, while the plan reveals how the database tries to get it.

A short list of implementations to investigate

The algorithms explain the choices more usefully than a universal vector-count cutoff. These implementations provide several operational choices:

  • Pinecone provides a managed vector database.
  • Qdrant supports vector search and payload filtering, with self-hosted and cloud deployment paths.
  • Weaviate combines a vector database with model integrations, including vectorization.
  • Chroma stores documents, embeddings, and metadata, with local and hosted options.
  • pgvector adds vector similarity search to PostgreSQL.

If PostgreSQL already holds the documents and permissions, I'd include pgvector in the comparison. If operating the search service is work the team wants to delegate, I'd include a managed option. Those are operational reasons to shortlist a tool. Whether it handles the workload comes from measurement.

Measure the search your readers will actually use

For the memory-leak question, success means finding a useful, eligible explanation quickly. ANN earns its place when skipping comparisons saves enough time without losing too many of the results that matter.

I would build a benchmark around representative questions, including paraphrases and exact technical identifiers. For each, establish an exact nearest-neighbor baseline under the same filters. Then vary candidate exploration, recording recall against that baseline, result count, latency at realistic concurrency, and memory use. Separately judge whether the retrieved passages answer the questions. Repeat while documents are inserted, updated, and deleted, because a static collection cannot show how search behaves during change.

That produces a defensible decision: this configuration retrieves enough useful material, under these filters and this load, within our latency and memory budget. It also tells you what to adjust when the Python article disappears. Increase the search effort if ANN missed a good neighbor. Revisit the embedding or chunk if exact search ranked it poorly. Investigate candidate exhaustion if the department filter emptied the results. Each failure has a different remedy, and knowing which one occurred is more useful than choosing a database by its headline scale.