Vector Index Cheat Sheet

Quick reference. Pair with lessons/0002-vector-search-at-scale.html.

Algorithm families at a glance

Algorithm	Mental model	Build cost	Query cost	Memory
Exact k-NN (brute force)	Compare against all vectors	0	O(N · d)	O(N · d)
HNSW	Multi-layer graph; navigate "social network"	O(N · log N)	O(log N)	O(N · d + N · M)
IVF (flat)	Cluster vectors; search only nearest clusters	O(N · k) (k-means)	O((N/nlist) · nprobe)	O(N · d)
IVF + PQ	IVF + compressed vectors (~50 bytes each)	IVF + codebook build	Faster than IVF (cache-friendly)	O(N · d / 64)

LanceDB index types (v0.33)

Config class	Composed of	Best for
`IvfFlat`	IVF + no compression	Highest recall, more memory
`IvfPq`	IVF + Product Quantization	Billion-scale, memory-bound
`IvfRq`	IVF + RaBitQ	Max compression
`IvfSq`	IVF + Scalar Quantization	Light compression, fast
`IvfHnswFlat`	IVF + HNSW + no compression	Highest recall, low latency
`IvfHnswPq`	IVF + HNSW + PQ	Low latency + low memory
`IvfHnswSq`	IVF + HNSW + Scalar Quantization	Default recommendation: best latency/recall balance

API quick reference

Create an index

from lancedb.pydantic import IvfPq, IvfHnswSq

# IVF + PQ
table.create_index("vector", config=IvfPq(
    num_partitions=256,    # ~ rows // 4096
    num_sub_vectors=96,    # ~ dim // 8
    metric="cosine",
))

# IVF + HNSW + SQ (default recommendation for 100k–10M)
table.create_index("vector", config=IvfHnswSq(
    num_partitions=256,    # ~ rows // 4,096
    m=16,
    ef_construction=150,
    metric="l2",
))

Search (with or without index)

# With index, ANN is used automatically
results = table.search(query_vec).limit(10).to_list()

# Force exact k-NN (bypass index, useful for ground-truth benchmarking)
results = table.search(query_vec).bypass_vector_index().limit(10).to_list()

Parameter cheat sheet

Param	Algorithm	Phase	Effect of raising
`M`	HNSW	Build	↑ recall, ↑ memory, ↑ build time
`ef_construction`	HNSW	Build	↑ graph quality, ↑ build time
`ef_search`	HNSW	Query	↑ recall, ↓ query speed
`nlist`	IVF	Build	↑ query speed (up to a point), ↑ build time
`nprobe`	IVF	Query	↑ recall, ↓ query speed
`num_partitions`	IVF	Build	Same as `nlist`
`num_sub_vectors`	PQ	Build	↑ recall, ↑ memory

When to use what (decision tree)

< 100k rows
Don't create an index. Exact k-NN is faster overall and uses less memory.

100k – 10M rows
IvfHnswSq with default params. Tune M and ef_construction if recall is low.

10M+ rows, memory-bound
IvfPq or IvfRq. Accept 3–5% recall drop for 30–60× memory savings.

Filtered queries (where(...))
HNSW-backed indexes can be slower with filters. IvfPq or IvfRq handle filters more predictably.

The "magic triangle"

You can only pick two of three:

Recall (did we get the right top-k?)
Speed (how fast is the query?)
Memory (how much RAM does the index use?)

Index parameter tuning is the art of finding the best 2-of-3 point for your workload.

Recall benchmark recipe

import random
import lancedb
from rag_pipeline import Embedder

embedder = Embedder()
db = lancedb.connect("data/lancedb")
table = db.open_table("rag_chunks")

# Sample 50 random query texts from your real index
contents = table.to_pandas()["content"].tolist()
samples = random.sample(contents, 50)

recalls = []
for text in samples:
    qv = embedder.embed_one(text)

    # Ground truth via exact k-NN
    gt = table.search(qv).bypass_vector_index().limit(10).to_list()
    gt_ids = {r["id"] for r in gt}

    # Approximate
    ann = table.search(qv).limit(10).to_list()
    ann_ids = {r["id"] for r in ann}

    recalls.append(len(gt_ids & ann_ids) / len(gt_ids))

print(f"recall@10 mean = {sum(recalls) / len(recalls):.3f}")
print(f"recall@10 min  = {min(recalls):.3f}")

Targets: mean ≥ 0.95, min ≥ 0.85. If min is too low, the index is missing queries that the user's intuition would say "should obviously work." Raise ef_search (HNSW) or nprobe (IVF) until you hit your target.

Glossary

Term	Meaning
Exact k-NN	Brute-force scan of every vector. O(N) per query. Default in LanceDB without an index.
ANN	Approximate Nearest Neighbor. Trades recall for speed.
Recall@k	Fraction of the true top-k that the index returned.
HNSW	Hierarchical Navigable Small World. Graph-based ANN.
IVF	Inverted File Index. Cluster-based ANN.
PQ	Product Quantization. Lossy vector compression.
Quantizer	The compression strategy inside an index: Flat (none), SQ (scalar), PQ (product), RQ (RaBitQ).
nprobe	How many IVF clusters to search at query time.
nlist	How many IVF clusters to build at index time.
Centroid	The "center" of an IVF cluster. Vectors assigned to the cluster are near it.
Cosine vs L2	Both are distance metrics. Cosine is direction-only; L2 is straight-line. With normalized vectors, they agree up to scale.
bypass_vector_index()	LanceDB method to force exact k-NN even when an index exists. Use it for ground-truth benchmarking.