Interview: Embeddings and Vector Similarity
Embeddings and similarity in simple words: what a vector is, how embedding models differ from chat models, dimensions, cosine vs dot product vs Euclidean, what embeddings miss, and how to choose and change an embedding model - from basic definitions to production decisions.
What you will be able to do
- Explain embeddings with a simple picture
- Compare embedding models and chat models
- Explain dimensions and storage cost
- Compare cosine, dot product and Euclidean distance
- Name what embeddings miss and how to choose a model
The idea, in plain English
"Explain embeddings" is the second question in most RAG interviews. Interviewers want a simple picture first - a map of meaning - and then the details: dimensions, similarity measures and limits.
The best answers use real numbers. Module 2 of this course measured everything on the bakery documents - use those results.
Draw it if you can: two arrows pointing in similar directions for "refund" and "return", and one pointing away for "chocolate cake".
Worked example: Twelve questions on embeddings, answered with the real measurements from RAG Module 2.
Basic: What is an embedding?
A list of numbers that represents the meaning of a text. A trained model puts texts with similar meaning close together, so we can search by meaning instead of exact words.
- Like a place on a map of meaning.
- "refund" vs "return" scored 0.698; "refund" vs "chocolate cake" 0.331 (Lesson 2.1).
Basic: What is the difference between an embedding model and a chat model?
An embedding model (an encoder) reads a whole text and returns one vector. A chat model (a decoder) generates text token by token. RAG uses both: one to find, one to answer.
- nomic-embed-text: 137M parameters; llama3: 8B.
- llama3 refused to embed in this setup (Lesson 2.3).
Basic: What is cosine similarity?
A measure of how much two vectors point in the same direction: 1 means the same direction, 0 unrelated. It ignores the vectors' lengths.
- The usual measure for text embeddings.
Intermediate: What do the dimensions of an embedding mean?
The number of values in each vector - 384, 768, 1536 and so on. Single numbers have no readable meaning; meaning is spread across all of them. More dimensions can hold finer differences but cost more to store and compare.
- 768 x 4 bytes = 3,072 bytes per vector; a million vectors = 3.1 GB (Lesson 2.5).
Intermediate: Cosine, dot product or Euclidean distance - which do you use?
Use what the model was trained for. For normalised (length-1) vectors all three give the same ranking, so dot product is the cheapest. For other vectors, cosine is safest - dot product favours longer vectors.
- Lesson 2.6: a vector made 3x longer jumped to first place with dot product.
- Choose the metric when you create the index.
Intermediate: What do embeddings not capture well?
Negation and logic ("can" vs "cannot be refunded" scored 0.971), word order, exact numbers and codes, and languages or domains the model was not trained on.
- Combine with keyword search; let the LLM read the text for logic.
Intermediate: Why must the question and the documents use the same embedding model?
Each model has its own vector space. Vectors from different models are not comparable - a question embedded by one model cannot find documents embedded by another.
- Changing the model means re-embedding everything.
- Some models also need task prefixes (search_query / search_document).
Advanced: How do you choose an embedding model?
Shortlist models for your language and domain from benchmarks, then test them on your own questions with known answers. Weigh quality against dimensions, speed, cost, context length, hosting and licence.
- Leaderboards are a starting point, not a decision.
- Check language support - Telugu collapsed to one vector in Lesson 2.2.
Advanced: What happens to text that is longer than the embedding model's limit?
Depending on the API it errors or is cut off silently. Ollama cut it silently: a fact at the end of a long text scored the same as no fact at all.
- Chunk documents to fit the limit, and check token counts.
Advanced: How can you reduce the storage and memory cost of vectors?
Use fewer dimensions (some models support shortening), quantise the numbers (float16, int8 or binary), store vectors in binary not JSON, and remove duplicate chunks.
- nomic-embed-text kept every answer with 256 of 768 numbers (Lesson 2.5) - test before you shrink.
Professional: You need to switch to a better embedding model in production. How?
Re-embed the whole collection into a new index in the background, evaluate old vs new on the test set, switch traffic (or dual-read) when quality is better, keep the old index for rollback, then delete it.
- Never mix vectors from two models in one index.
- Budget the time and cost of re-embedding.
Professional: How would you explain to a manager why search found the wrong document?
Search compares meaning, not exact facts. Words that appear in every question - like the company name - or documents that are "about everything" can pull results the wrong way. We fix it by cleaning queries, improving chunks, adding keyword search, and measuring on real questions.
- Lesson 2.7: removing the shop name from questions took semantic search from 7/12 to 11/12.
Key points
- Embeddings are numbers that place texts on a map of meaning.
- Embedding models find; chat models answer.
- For normalised vectors, cosine, dot and Euclidean rank the same.
- Embeddings miss negation, exact numbers and unknown languages.
- Choose models by testing on your own questions; never mix models.
Interview questions
What does a similarity of 1.0 mean?
The vectors point in exactly the same direction.
Is a 0.7 score '70% the same'?
No - scores are for ranking, not percentages.
What is a normalised vector?
A vector scaled to length 1.
Does the same text always give the same embedding?
Yes, for the same model - store it once.
What is a task prefix?
A label like 'search_query: ' that some models expect.
Why batch embedding requests?
Fewer round trips: 100 texts took 0.99 s instead of 1.77 s.
What is semantic search?
Search by meaning using embeddings.
What is quantisation of vectors?
Storing the numbers with fewer bits to save space.
Comments
Sign in to leave a comment. Your name and photo come from Google; nothing else is shared.
Loading comments...