Why it matters
Comparing two vectors needs a single number that says how alike they are, and the dot product is the cheapest one that is differentiable and hardware-friendly. It appears in attention scores, embedding similarity and the linear layer itself, where it is computed for every output feature at once. Its weakness is that magnitude leaks into the score, so a long vector can look similar to everything until you normalise.
Mental model
The dot product is a tally of agreement: for each coordinate, multiply the two values and add up the results, so matching large values contribute a lot. The metaphor breaks because agreement in one coordinate can be cancelled by disagreement in another, and the total says nothing about which coordinates drove it.
How it works
The interactive figure loads with JavaScript. The text version below describes the same frame.
Figure. Five synthetic vectors plotted by their two coordinates. Each candidate's dot product with the query vector is computed from those coordinates; the cosine button divides both lengths out. The axes are dimensionless, labelled dim 1 and dim 2, with ticks from −1 to 1; no embedding model produced these positions. Attention scaling and batched layouts are not in the figure.
What to notice. Under the dot product, b leads at 0.80 because it is the longest vector, and c at 0.38 outranks the perfectly aligned but short a at 0.20; at k = 2 the returned pair would be b and c. Switch to cosine and a and b tie at 1.00, because both lie on the query's ray, while c falls to 0.66 and is third. Only the score changes; the five points never move. Dot mixes alignment with length, and cosine keeps the alignment term alone.
Text version
A scatter of five labelled synthetic vectors around a query vector at 0.30, 0.55 on a dimensionless grid. The default frame uses the dot product and k = 3: b, the same angle at twice the length, is returned first at 0.80; c, a wider angle but a longer vector, second at 0.38; and a, the same angle at half the length, third at 0.20. d at 0.19 and e, pointing the opposite way, at minus 0.66 stay outside k, so the two vectors on the query's ray are not the top two. The metric control switches to cosine, where a and b tie at 1.00, c falls to 0.66 and the order becomes a, b, c. Two callouts restate the aligned-short and wider-long pair. A slider sets k from one to five.
- Confirm the two vectors have the same length and are indexed the same way; misaligned features produce a meaningless score.
- Multiply coordinate by coordinate and sum, or call the library routine so the reduction runs in optimised code.
- Interpret the sign as direction of alignment and the magnitude as strength times lengths.
- Normalise both vectors to unit length when you want pure directional similarity.
- For batches, arrange the operands so the contraction dimension is contiguous, since layout dominates speed.
- In attention, divide the result by the square root of the dimension before the softmax to control variance.
Trade-offs
| Choice | Good for | Cost | Fails when |
|---|---|---|---|
| Raw dot product | Cheap, sensitive to confidence and magnitude | Long vectors dominate the ranking | Embeddings have varying norms |
| Cosine similarity | Pure direction, bounded range | Loses magnitude information and needs normalisation | Magnitude itself is meaningful |
| Learned bilinear score | Can weight coordinates differently | Extra parameters and overfitting risk | Data per pair is scarce |
| Euclidean distance | Interpretable as physical distance | Not a bounded similarity, sensitive to scale | You need a comparable score across pairs |
Good for
Cheap, sensitive to confidence and magnitude
Cost
Long vectors dominate the ranking
Fails when
Embeddings have varying norms
- Embeddings have varying norms
- Magnitude itself is meaningful
- Data per pair is scarce
- You need a comparable score across pairs
Figure. Each choice with what it buys, what it costs, and the case where it loses.
What to notice. The fourth column is the decision: if the losing case describes your situation, the choice is wrong.
Text version
Four trade-offs: Raw dot product is good for Cheap, sensitive to confidence and magnitude, costs Long vectors dominate the ranking, and fails when Embeddings have varying norms. Cosine similarity is good for Pure direction, bounded range, costs Loses magnitude information and needs normalisation, and fails when Magnitude itself is meaningful. Learned bilinear score is good for Can weight coordinates differently, costs Extra parameters and overfitting risk, and fails when Data per pair is scarce. Euclidean distance is good for Interpretable as physical distance, costs Not a bounded similarity, sensitive to scale, and fails when You need a comparable score across pairs
Failure modes
- Unnormalised comparison. A frequently occurring document with a large embedding norm outranks better matches. Debug: plot score against vector norm on a sample; correlation between them indicates the confound.
- Feature misalignment. Two vectors built from different column orders produce a low score for identical items. Debug: score a vector against itself after each preprocessing path; it should be maximal in both.
- Overflow in low precision. Summing many large products saturates the accumulator. Debug: compare the dot product in float32 against the reduced-precision result on the same inputs.
In production
Log the distribution of vector norms in your index; a long tail means raw dot-product ranking is partly a popularity ranking. When switching between dot and cosine, re-tune thresholds, because scores are no longer on the same scale.
Common misconceptions
People think: Dot product and cosine similarity are the same thing.
Actually: They agree only when both vectors are unit length; otherwise the dot product also encodes magnitude, which changes rankings.
People think: A higher dot product always means more similar.
Actually: It can mean longer. Without normalisation the score mixes alignment with scale.
Interview angle
A strong answer gives both the algebraic and geometric definitions, explains the magnitude confound, and connects it to attention scaling and retrieval normalisation. It should mention numerical accumulation. Weak answers define it as multiplication and addition without saying why it measures similarity or where it misleads.
Sources
- 3Blue1Brown, Essence of linear algebra — the geometric meaning of the dot product.
- NumPy, numpy.dot — the operation and its shape rules.
- scikit-learn, cosine_similarity — normalised dot-product similarity in practice.