Skip to content
AI Engineer Guide.

concept · basics · intermediate · concept 5 of 27

Cosine similarity

0 of 27 complete

Why it matters

Almost every embedding system needs to answer one question: how close are these two things? Retrieval ranks passages against a query; deduplication compares documents; clustering groups users; evaluation checks whether a paraphrase landed near its reference.

The naive answer is Euclidean distance, but raw distance depends on how long the vectors are. Some encoders produce vectors whose length grows with document length, frequency or confidence. If you rank by distance, the longest document wins regardless of topic. Cosine similarity removes that dependence: only direction counts. That property is why it is the default similarity for text embeddings, and why the choice must still be made deliberately rather than by habit.

Mental model: the angle between two arrows

Put both vectors' tails at the origin. Cosine similarity is the cosine of the angle between the arrows. Point them the same way and the score is 1; make them perpendicular and it is 0; point them in opposite directions and it is −1.

This is a good picture for reasoning about margins. Two passages both "about billing" sit in a narrow cone and score highly against a billing query, while a passage about a refund lands a few degrees away. Retrieval quality is often about the size of that angle difference, not the absolute score.

The analogy breaks when you read the score as a percentage. An angle of 60 degrees is a cosine of 0.5, which sounds like "half relevant" and is not. Cosine is a geometric quantity; relevance is a property of the task, judged by humans or a labelled set. The mapping between the two is empirical and model-specific.

How it works

Mechanism, step by stepInteractive
  1. 1Take the dot product. Multiply matching components and sum them. It is large when the vectors agree on many dimensions.
  2. 2Divide by both lengths. This is the normalisation step that removes magnitude, leaving only direction.
  3. 3Read the result as an angle. For real vectors the value lies in $[-1, 1]$.
  4. 4Bulk-compute with a matrix product. Score one query against $N$ documents as a single matrix–vector product instead of a loop. This is what a vector index does internally.
  5. 5Rank, then threshold. Ranking is usually robust; a fixed cutoff is not. Choose cutoffs from labelled data for that specific encoder.
  6. 6Keep the metric aligned with the index. Decide once whether vectors are stored normalised, and configure the index to match. Mixed conventions change rankings silently.

Step 1 of 6

Take the dot product. Multiply matching components and sum them. It is large when the vectors agree on many dimensions.

Step 1/6 · step 1

Figure. The mechanism in this page's own steps. The figure adds sequence, not new claims.

What to notice. A step can be read alone, but the order is the mechanism. The control lets you stop on any one.

Text version

The page's steps: 1. Take the dot product. Multiply matching components and sum them. It is large when the vec… 2. Divide by both lengths. This is the normalisation step that removes magnitude, leaving on… 3. Read the result as an angle. For real vectors the value lies in $[-1, 1]$. 4. Bulk-compute with a matrix product. Score one query against $N$ documents as a single mat… 5. Rank, then threshold. Ranking is usually robust; a fixed cutoff is not. Choose cutoffs fr… 6. Keep the metric aligned with the index. Decide once whether vectors are stored normalised…

  1. Take the dot product. Multiply matching components and sum them. It is large when the vectors agree on many dimensions.
  2. Divide by both lengths. This is the normalisation step that removes magnitude, leaving only direction.
  3. Read the result as an angle. For real vectors the value lies in [−1,1][-1, 1].
  4. Bulk-compute with a matrix product. Score one query against NN documents as a single matrix–vector product instead of a loop. This is what a vector index does internally.
  5. Rank, then threshold. Ranking is usually robust; a fixed cutoff is not. Choose cutoffs from labelled data for that specific encoder.
  6. Keep the metric aligned with the index. Decide once whether vectors are stored normalised, and configure the index to match. Mixed conventions change rankings silently.

The math

cos⁡(q,d)=q⋅d∥q∥2 ∥d∥2=∑i=1nqidi∑iqi2 ∑idi2.\cos(\mathbf{q}, \mathbf{d}) = \frac{\mathbf{q} \cdot \mathbf{d}}{\lVert \mathbf{q} \rVert_2 \, \lVert \mathbf{d} \rVert_2} = \frac{\sum_{i=1}^{n} q_i d_i} {\sqrt{\sum_i q_i^2}\,\sqrt{\sum_i d_i^2}}.

For unit vectors the relationship to Euclidean distance is exact:

∥q−d∥22=2−2cos⁡(q,d).\lVert \mathbf{q} - \mathbf{d} \rVert_2^2 = 2 - 2\cos(\mathbf{q}, \mathbf{d}).

So on normalised vectors, ranking by L2 distance and ranking by cosine produce the same order. The two metrics only disagree when magnitudes differ.

A minimal implementation

import numpy as np
 
def cosine_scores(query: np.ndarray, docs: np.ndarray, eps: float = 1e-12) -> np.ndarray:
    """query: (d,), docs: (n, d) -> (n,) cosine scores."""
    q_norm = np.linalg.norm(query)
    d_norms = np.linalg.norm(docs, axis=1)
    # Guard against zero vectors: a zero vector has no direction.
    denom = np.maximum(q_norm * d_norms, eps)
    return (docs @ query) / denom
 
docs = np.array([[1.0, 0.0], [1.0, 1.0], [-1.0, 0.0]])
query = np.array([2.0, 0.0])          # twice as long, same direction as docs[0]
 
print(cosine_scores(query, docs))      # [ 1.   0.707 -1. ]
print(np.linalg.norm(query))           # 2.0 — magnitude ignored by cosine

The zero-vector guard matters in practice: padding rows, all-zero embeddings from a failed encoder call, and empty documents all produce a norm of zero, and the resulting division yields nan that propagates through sorting and comparisons without raising.

Trade-offs

MetricGood forCostFails when
CosineText embeddings, direction-only similarityIgnores magnitude, needs an extra norm passVector length itself encodes importance or confidence
Dot productCheap, fast on unit vectorsMagnitude leaks into the scoreVectors are not normalised and lengths vary
Euclidean (L2)Clusters, coordinates with real unitsSensitive to scaleEmbedding norms correlate with document length
Cosine on centred vectorsReducing a shared mean directionNeeds corpus statistics at query timeCorpus mean drifts or is expensive to recompute
Learned rerankerFine distinctions, negation, qualifiersOne forward pass per candidateThe right passage was never retrieved
Trade-off mapInteractive

Good for

Text embeddings, direction-only similarity

Cost

Ignores magnitude, needs an extra norm pass

Fails when

Vector length itself encodes importance or confidence

  1. Vector length itself encodes importance or confidence
  2. Vectors are not normalised and lengths vary
  3. Embedding norms correlate with document length
  4. Corpus mean drifts or is expensive to recompute
  5. The right passage was never retrieved

Figure. Each choice with what it buys, what it costs, and the case where it loses.

What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.

Text version

Four trade-offs: Cosine is good for Text embeddings, direction-only similarity, costs Ignores magnitude, needs an extra norm pass, and fails when Vector length itself encodes importance or confidence. Dot product is good for Cheap, fast on unit vectors, costs Magnitude leaks into the score, and fails when Vectors are not normalised and lengths vary. Euclidean (L2) is good for Clusters, coordinates with real units, costs Sensitive to scale, and fails when Embedding norms correlate with document length. Cosine on centred vectors is good for Reducing a shared mean direction, costs Needs corpus statistics at query time, and fails when Corpus mean drifts or is expensive to recompute. Learned reranker is good for Fine distinctions, negation, qualifiers, costs One forward pass per candidate, and fails when The right passage was never retrieved

Failure modes

Metric and index disagree. Vectors are written unnormalised into an inner-product index, but the application code assumes cosine. Long documents dominate the top-k. Debugging step: take one query, compute cosine in Python and compare its top 10 with the index's top 10; any divergence is a normalisation bug.

A threshold was copied from another model. A 0.78 cutoff that worked for one encoder admits almost everything for another. Debugging step: plot score distributions for labelled positive and negative pairs and pick the cutoff where precision meets the requirement, per model version.

The space is anisotropic. Some encoders place all sentence embeddings in a narrow cone, so unrelated pairs still score around 0.7 and the usable margin collapses. Debugging step: sample a thousand random pairs and look at the score histogram; a high mean with small variance means you need a different encoder or mean-centering.

Zero vectors reach the scorer. A failed embedding call returns zeros, which score 0.0 against everything and can sort into the middle of the list unnoticed. Debugging step: assert every row has a norm above a floor at write time, and alert on the count of rejected rows.

In production

Pin the metric in one place — index configuration and client code should not each decide. Store whether vectors are normalised in index metadata, and validate it on the first query after a deploy.

Recalibrate whenever the encoder changes. Similarity thresholds, deduplication cutoffs and clustering radii are all functions of the model's geometry, not universal constants. Keep a small labelled pair set, and recompute the threshold when the model version changes.

Watch the score distribution, not just the top score. A shift in the median similarity across queries is an early signal of an encoder change, a corpus shift, or a query distribution change — often before answer quality metrics move.

Common misconceptions

People think: A cosine score of 0.9 means the passage is 90% relevant.

Actually: It means the vectors point in nearly the same direction. Relevance depends on the task, and the mapping from score to correctness is empirical for each encoder.

People think: Cosine similarity has a universal 'good match' threshold.

Actually: Thresholds depend on the encoder, the domain and the length distribution of the corpus. They must be measured on labelled pairs for that specific setup.

A third one: that cosine is always the right metric. If an encoder is trained with a dot-product objective and produces meaningful magnitudes, cosine throws away signal the model was trained to use.

Interview angle

Cosine questions are usually a proxy for whether you understand vector search end to end: the geometry, the normalisation convention, the index metric, and the difference between a ranking score and a calibrated probability. A strong answer explains why normalisation makes cosine and dot product agree, then names the failure that follows when the convention is mixed. Practise with the retrieval questions at /interview/rag and read similarity metrics next.

Sources