Lesson 2.5 · Embeddings and Semantic Search

Vector Dimensions and Similarity

What do 768 dimensions mean - and cost? One vector is 3,072 bytes as float32 (10,288 as JSON text), so a million chunks need 3.1 GB. Single numbers mean nothing on their own. And with this model you can keep only the first numbers: 256 and 128 still found the right document for all four questions; 64 and 16 began to fail.

Intermediate25 min

What you will be able to do

Explain what a dimension is
Calculate the storage a collection of vectors needs
Understand why vectors are normalised to length 1
See that single dimensions have no readable meaning
Test shortening vectors to save space

The idea, in plain English

A dimension is one number in the vector. nomic-embed-text gives 768; other models give 384, 1,024, 1,536 or more. More dimensions can hold finer differences in meaning - but every number must be stored and compared, for every chunk.

Embedding vectors here are normalised: their length is exactly 1. So only their direction matters, and comparing two vectors is a single multiply-and-add (Lesson 2.6). The numbers themselves are not features you can read - "number 3" does not mean "about money".

We measure the storage cost of our vectors, look at single numbers, and then keep only the first 256, 128, 64 and 16 numbers to see how far we can shrink before search breaks. nomic-embed-text was trained so that its first numbers carry the most meaning; for other models this may not work.

Worked example: The bakery vectors from Lesson 2.4, measured for size and shortened step by step until search breaks.

workflowHow short can a vector be?step 1 / 3

768 numbers

3,072 bytes

✓

256 / 128

1,024 / 512 bytes

256 bytes

64 bytes

React Flow

1 - Full size

All four questions find the right document.

right

4 of 4

1M vectors

3.1 GB

Four questions searched with shortened vectors. Real results.

Words you will see in this lesson

A few size words.

Small dictionary

DimensionOne position (one number) in a vector.

float32A common way to store one decimal number in 4 bytes.

NormalisedScaled so the vector’s length is exactly 1.

Vector lengthThe square root of the sum of all numbers squared.

ShorteningKeeping only the first numbers of a vector to save space and time.

An everyday example: describing a person

Describe a friend with 3 facts - "tall, glasses, curly hair" - and many people match.
With 50 facts you can pick them out of a crowd. More facts, finer differences - that is more dimensions.
But a police file with 768 facts per person needs a big cupboard - that is storage.
If the first 10 facts are the most useful ones, you can often stop there - that is shortening.

Example 1 - size, length and single numbers

dimensions.py loads the vectors saved in Lesson 2.4:

Example 1 - dimensions.py

import json
import math
from array import array
from pathlib import Path

import ollama

records = json.loads(Path("vectors.json").read_text())
vector = records[0]["vector"]

# 1. Length and storage
print("numbers per vector:", len(vector))
print("length of the vector:", round(math.sqrt(sum(x * x for x in vector)), 6))
binary = len(array("f", vector).tobytes())                   # 4 bytes per number
as_json = len(json.dumps(vector))
print(f"one vector: {binary:,} bytes as float32, {as_json:,} bytes as JSON text")
print(f"1,000,000 vectors as float32: {binary * 1_000_000 / 1e9:.1f} GB")

# 2. One number means nothing on its own
for word in ["refund", "return", "delivery"]:
    v = ollama.embed(model="nomic-embed-text", input=word).embeddings[0]
    print(f"  '{word}': number 1 = {v[0]:+.3f}, number 2 = {v[1]:+.3f}, number 3 = {v[2]:+.3f}")


# 3. Keep only the first N numbers - does search still work?
def shorten(v, n):
    part = v[:n]
    length = math.sqrt(sum(x * x for x in part))
    return [x / length for x in part]                       # make it length 1 again


questions = ["Can I return my cake?", "Is there a charge for bringing the order to my house?",
             "When do you close on Sunday?", "Are there nuts in your cakes?"]
for n in [768, 256, 128, 64, 16]:
    best = []
    for question in questions:
        q = ollama.embed(model="nomic-embed-text", input="search_query: " + question).embeddings[0]
        top = max(records, key=lambda r: sum(a * b for a, b in zip(shorten(q, n), shorten(r["vector"], n))))
        best.append(top["source"])
    print(f"first {n:>3} numbers ({n * 4:>4} bytes): {best}")

Output

numbers per vector: 768
length of the vector: 1.0
one vector: 3,072 bytes as float32, 10,288 bytes as JSON text
1,000,000 vectors as float32: 3.1 GB
  'refund': number 1 = +0.060, number 2 = +0.039, number 3 = -0.169
  'return': number 1 = +0.041, number 2 = +0.019, number 3 = -0.190
  'delivery': number 1 = -0.050, number 2 = +0.061, number 3 = -0.206
first 768 numbers (3072 bytes): ['refunds.md', 'delivery.md', 'hours.md', 'allergens.md']
first 256 numbers (1024 bytes): ['refunds.md', 'delivery.md', 'hours.md', 'allergens.md']
first 128 numbers ( 512 bytes): ['refunds.md', 'delivery.md', 'hours.md', 'allergens.md']
first  64 numbers ( 256 bytes): ['refunds.md', 'refunds.md', 'hours.md', 'allergens.md']
first  16 numbers (  64 bytes): ['refunds.md', 'menu.md', 'hours.md', 'allergens.md']

Reading the size

768 numbers x 4 bytes = 3,072 bytes per vector as float32.
As JSON text the same vector took 10,288 bytes - over 3 times more. Fine for 7 documents; vector databases store binary numbers (Modules 3 and 4).
One million chunks: 3.1 GB of vectors alone, before the text. A model with 1,536 dimensions would need twice that.
The length of every vector is 1.0 - Ollama returns them normalised. Only direction matters.

Reading single numbers

"refund", "return" and "delivery" all have number 3 around -0.17 to -0.21.
"refund" and "return" are similar overall (0.698), but no single number "means" refund.
Meaning is spread across all 768 numbers together. Do not try to read one; compare whole vectors.

Example 2 - keep only the first numbers

shorten() keeps the first n numbers and makes the length 1 again. Then four questions are searched at each size:

768, 256 and 128 numbers: all four right - refunds, delivery, hours, allergens.
64 numbers: the delivery question ("a charge for bringing the order to my house") found refunds.md. Wrong.
16 numbers: it found menu.md. Still wrong; the other three held on.
So with this model, 256 numbers (a third of the space) kept every answer here. Test on YOUR questions before you shrink - four questions are not proof.
This works because nomic-embed-text was trained to put the most important information first. With most other models, cutting numbers off breaks search.

Choosing a dimension size

Bigger is not automatically better - a well-trained 768 model can beat a weak 1,536 one. Measure on your own questions (Module 11).
Size matters at scale: storage, memory for the index, and time for every comparison grow with the dimensions.
Fix the size per collection: every vector in one search must have the same length, from the same model.

Dimensions at a glance

Bytes per vector

float32.

768 x 4 = 3,072 bytes

Collection size

1,000,000 chunks.

3,072 x 1,000,000 = 3.1 GB

Vector length

Normalised.

math.sqrt(sum(x * x for x in v)) == 1.0

Shorten

Only for models trained for it.

v[:256], then make length 1 again

Try it yourself

The code does not change. Swap the content string and the program does something else entirely.

What usually goes wrong

Storing vectors as JSON at scale

10,288 bytes instead of 3,072 per vector. Use binary float32 storage.

Shortening any model

It worked for nomic-embed-text, which was trained for it; most models break.

Mixing lengths

All vectors in one search must have the same number of dimensions.

Choosing by dimension count

Measure quality on your questions instead.

Practice

Write these yourself before opening anything. Getting them wrong first is most of how this sticks.

Write a small benchmark: embed the 7 bakery documents and 10 questions with known right answers, then for sizes 768, 512, 256, 128, 64 and 32 print how many of the 10 find the right document first, and the bytes per vector. Plot nothing - a table is enough. Which size would you choose, and why?

Show hint

Reuse shorten() and the questions from Lesson 1.4 (with the shop name removed - Lesson 2.7 explains why).

Key points

A dimension is one number; nomic-embed-text gives 768.
One float32 vector is 3,072 bytes; a million are 3.1 GB.
Vectors here have length 1, so only direction matters.
Single numbers mean nothing alone - meaning is spread over all of them.
This model kept all answers with its first 256 numbers; 64 and 16 failed.

Quick check before you move on

How many bytes is one 768-number vector as float32?

3,072.

What is the length of the vectors Ollama returned?

1.0.

At which size did the first question fail?

64 numbers - the delivery question found refunds.md.

Interview questions

How do embedding dimensions affect a vector search system?

They set storage and memory per vector, index build and query cost, and often (but not always) representational quality; choose by measured retrieval quality versus cost, and consider dimension reduction or quantisation at scale.

What is Matryoshka representation learning?

A training method that makes the leading dimensions of an embedding a useful smaller embedding on their own, so vectors can be truncated (and renormalised) to trade quality for size.

Quiz

1.
Why does the JSON file take more space than float32?
2.
Why can you not read meaning from number 3 of a vector?
3.
Why did shortening work with nomic-embed-text?

Comments

Loading comments...