Embeddings & Vector Search
Compressed essence for Day 8 of the 30-day AI learning plan.
1. What an embedding is
A function embed(text) → number[] of fixed length (e.g. 384).
The list is a coordinate in N-dimensional space. Texts with similar meaning
get nearby coordinates. That is the entire trick.
2. Cosine similarity
How close two embeddings are, measured by angle (not distance):
cosine(a, b) = (a · b) / (|a| · |b|) ∈ [-1, 1]
if a, b are unit-length: cosine = a · b (just the dot product)
1.0 = same direction → very similar meaning
0.0 = unrelated
-1.0 = oppositeWith a normalized embedder you can skip the magnitudes — rank by dot product.
3. Vector search in 3 steps
- Index: embed every document once → store its vector.
- Query: embed the query text → one vector.
- Rank: cosine-similarity(query, each doc) → return top-k.
4. Semantic vs lexical (the key contrast)
| Lexical / bag-of-words | Semantic embedding |
|---|---|
| Matches shared words | Matches shared meaning |
| "car" ≠ "automobile" | "car" ≈ "automobile" |
| Fast, offline, no model | Needs a model (e.g. SBERT) |
The library's
LexicalEmbedder is lexical — fine for fixtures,
but it cannot "search by meaning." A real model (SBERT / all-MiniLM-L6-v2) is
what makes RAG retrieval work.5. Minimal cosine in TypeScript
function cosine(a: number[], b: number[]): number {
let dot = 0, ma = 0, mb = 0;
for (let i = 0; i < a.length; i++) {
dot += a[i] * b[i];
ma += a[i] * a[i];
mb += b[i] * b[i];
}
return dot / (Math.sqrt(ma) * Math.sqrt(mb));
}