Lesson 2.4 · Embeddings and Semantic Search

Generating Embeddings with an API

Embed all the bakery documents through Ollama’s API, save the vectors with their source, and search them - "Is there a charge for bringing the order to my house?" found delivery.md with no shared keyword. Measured: one batch request is as exact as one request per text, and 100 texts took 0.99 s instead of 1.77 s. Plus the raw HTTP call, the model’s task prefixes, and how a hosted API differs.

Intermediate30 min

What you will be able to do

Create embeddings with the Python library and the HTTP API
Send texts in batches
Use the task prefixes a model expects
Store vectors with their text and source
Search stored vectors with a new question
Know what changes with a hosted embedding API

The idea, in plain English

An embedding model is used through an API: you send text, you get vectors back. Ollama runs that API on your laptop at localhost:11434. Hosted services (OpenAI, Google, Cohere and others) offer the same idea over the internet, for a key and a price per token.

In a RAG app you call it in two places. During ingestion you embed every chunk once and store the vector next to the text and its source. At query time you embed only the question and compare it with the stored vectors.

We do both for the bakery: save the seven documents as vectors in a JSON file, then search them. Along the way we measure batching, look at the raw HTTP request, and use the task prefixes nomic-embed-text was trained with. nomic-embed-text on Ollama.

Worked example: embed_docs.py saves the bakery’s seven documents as vectors; search.py answers questions against them.

workflowEmbed once, search many timesstep 1 / 3

7 documents

search_document: ...

Ollama API

POST /api/embed

vectors.json

text + source + vector

question

search_query: ...

best match

delivery.md 0.576

React Flow

1 - Ingestion: one batch

All seven documents in one request, saved with their source.

tokens

215

file

7 x 768 numbers, 72,905 bytes

Ingestion stores vectors; each question is embedded and compared. Real run.

Words you will see in this lesson

A few API words.

Small dictionary

APIA way for programs to call a service - here: send text, get vectors.

EndpointOne address of an API, like POST /api/embed.

BatchMany texts sent in one request.

Task prefixA short label a model was trained with, like "search_query: ".

Hosted APIAn embedding model run by a company on its servers, used with a key.

An everyday example: a photo studio

You take 100 passport photos to a studio for printing.
Handing them over one at a time, waiting at the counter each time, takes all afternoon. One envelope with all 100 is much faster - that is a batch.
You keep each print in a labelled sleeve: the photo, the name, the date - that is storing the vector with its text and source.
The studio down the road (a hosted API) prints too - for a fee, and you have to carry the photos there.

Example 1 - embed and save the documents

embed_docs.py is the ingestion side:

Each text gets the prefix "search_document: " - nomic-embed-text was trained with this label for stored text, and "search_query: " for questions. Other models use other labels or none; check the model’s page.
It embeds the texts one request each, then all in one batch, and compares: the biggest difference is 0.0000000. Batching changes nothing in the result.
It saves each record with the original text (without the prefix), its source and its vector.
7 vectors as JSON text take 72,905 bytes - numbers written as text are big. Lesson 2.5 looks at size.

Example 1 - embed_docs.py

import json
import time
from pathlib import Path

import ollama

from bakery import DOCS

MODEL = "nomic-embed-text"
names = list(DOCS)
texts = ["search_document: " + DOCS[name] for name in names]   # nomic's label for stored text

# One request per document
start = time.perf_counter()
one_by_one = [ollama.embed(model=MODEL, input=text).embeddings[0] for text in texts]
print(f"one request per document: {(time.perf_counter() - start) * 1000:.0f} ms")

# One request for all documents (a batch)
start = time.perf_counter()
reply = ollama.embed(model=MODEL, input=texts)
print(f"one batch request:        {(time.perf_counter() - start) * 1000:.0f} ms "
      f"({reply.prompt_eval_count} tokens)")

same = max(abs(a - b) for v1, v2 in zip(one_by_one, reply.embeddings) for a, b in zip(v1, v2))
print(f"biggest difference between the two ways: {same:.7f}")

# Save: the text, where it came from, and its vector
records = [{"source": name, "text": DOCS[name], "vector": vector}
           for name, vector in zip(names, reply.embeddings)]
Path("vectors.json").write_text(json.dumps(records))
print(f"saved {len(records)} vectors x {len(records[0]['vector'])} numbers -> vectors.json "
      f"({Path('vectors.json').stat().st_size:,} bytes)")

Output

$ python embed_docs.py
one request per document: 116 ms
one batch request:        95 ms (215 tokens)
biggest difference between the two ways: 0.0000000
saved 7 vectors x 768 numbers -> vectors.json (72,905 bytes)

Example 2 - search the saved vectors

search.py is the query side. It reads vectors.json, embeds only the question with "search_query: ", and prints the best two:

"Can I return my cake?" - refunds.md 0.794. With the prefixes it scored a little higher than without them in Lesson 2.1 (0.759).
"Is there a charge for bringing the order to my house?" - delivery.md first. No word in common with "Delivery costs Rs 50".
The documents were not embedded again. In a real app, ingestion runs when documents change; search runs for every question.

Example 2 - search.py

import json
import sys
from pathlib import Path

import ollama

records = json.loads(Path("vectors.json").read_text())        # made once by embed_docs.py
question = sys.argv[1]
q = ollama.embed(model="nomic-embed-text", input="search_query: " + question).embeddings[0]

scored = sorted(((sum(a * b for a, b in zip(q, r["vector"])), r) for r in records),
                key=lambda pair: pair[0], reverse=True)
print(f"question: {question}")
for score, record in scored[:2]:
    print(f"  {score:.3f}  {record['source']:<13} {record['text'][:70]}")

Output

$ python search.py "Can I return my cake?"
question: Can I return my cake?
  0.794  refunds.md    Refunds: custom cakes can be refunded only if cancelled at least 24 ho
  0.663  allergens.md  Allergens: all cakes may contain nuts and eggs. Eggless cakes must be 
$ python search.py "Is there a charge for bringing the order to my house?"
question: Is there a charge for bringing the order to my house?
  0.576  delivery.md   Delivery: we deliver within 8 km of MG Road between 10 am and 8 pm. De
  0.548  menu.md       Menu: the best seller is the pineapple pastry at Rs 45. A 1 kg black f

Batches pay off at scale

With 7 documents batching saved little (95 ms vs 116 ms). batch_100.py tries 100 texts:

One request per text: 1.77 s. One batch: 0.99 s - almost half.
Each request has a fixed cost (sending, waiting, starting work). A batch pays it once.
With a hosted API the gap is bigger: every request crosses the internet.
Keep batches to a sensible size (tens to a few hundred texts) and retry a failed batch, not the whole job.

batch_100.py

import time

import ollama

from bakery import DOCS

texts = [f"search_document: {text} (copy {i})" for i in range(15) for text in DOCS.values()][:100]

start = time.perf_counter()
for text in texts:
    ollama.embed(model="nomic-embed-text", input=text)
loop_s = time.perf_counter() - start

start = time.perf_counter()
ollama.embed(model="nomic-embed-text", input=texts)
batch_s = time.perf_counter() - start
print(f"100 texts - one by one: {loop_s:.2f} s   one batch: {batch_s:.2f} s")

Output

100 texts - one by one: 1.77 s   one batch: 0.99 s

The raw HTTP call

The Python library is a thin wrapper around one HTTP request. With curl:

POST /api/embed with JSON: the model name and the input.
The answer has "embeddings" (a list of vectors), token count and timings in nanoseconds.
Any language that can send HTTP can make embeddings - JavaScript, Go, Java.
We shortened the vector for the page; the real answer has all 768 numbers.

Output (vector shortened)

$ curl -s http://localhost:11434/api/embed -d '{"model": "nomic-embed-text", "input": "search_query: Can I return my cake?"}'
{"model": "nomic-embed-text", "embeddings": [[0.043061, 0.05391, -0.162077, -0.043141, "... 764 more numbers"]], "total_duration": 11138500, "load_duration": 911125, "prompt_eval_count": 12}

A hosted embedding API

Hosted APIs work the same way: text in, vectors out. What changes:

You need an API key (from the provider’s dashboard) and you pay per token.
Your text leaves your machine - check what you may send (customer data, contracts).
Vectors from different models cannot be mixed. If you change the embedding model, re-embed every document.
The code below is the shape of a hosted call with the OpenAI library. It was NOT run for this lesson - it needs a key and a paid account. Everything else in this module runs locally.

hosted.py (not run - needs a key)

import os

from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])     # never write the key in the code
reply = client.embeddings.create(model="text-embedding-3-small",
                                 input=["Refunds: custom cakes can be refunded ...", "Delivery: ..."])
vectors = [item.embedding for item in reply.data]

Watch out: Never mix vectors from two models in one search. A question embedded with one model cannot be compared with documents embedded by another - the numbers mean different things.

Embedding API at a glance

Python

One vector per text.

ollama.embed(model="nomic-embed-text", input=texts).embeddings

HTTP

Any language.

POST http://localhost:11434/api/embed {"model": ..., "input": ...}

Prefixes (nomic)

Stored text / questions.

"search_document: " / "search_query: "

Batch

Fewer round trips.

input=[text1, text2, ...]

Store

Never just the vector.

{"source": ..., "text": ..., "vector": [...]}

Try it yourself

The code does not change. Swap the content string and the program does something else entirely.

Without prefixes

“Remove both prefixes and run embed_docs.py and search.py again. Do the scores change?”

What usually goes wrong

One request per text

100 texts took 1.77 s instead of 0.99 s. Batch them.

Saving only the vectors

Without the text and source you cannot show or cite what you found.

Forgetting the model’s prefixes

nomic-embed-text expects search_document / search_query labels.

Mixing models

Vectors from different models are not comparable. Re-embed everything when you switch.

Practice

Write these yourself before opening anything. Getting them wrong first is most of how this sticks.

Turn embed_docs.py into embed_folder.py: read every .md file in a folder, split it by "## " sections (like Lesson 1.6), embed the chunks in batches of 16, and save source, text and vector. Then change search.py to print the top 3 chunks. Test it on the docs folder from Lesson 1.6.

Show hint

Use range(0, len(texts), 16) to make batches, and extend one list with each batch’s embeddings.

Key points

An embedding API takes text and returns vectors - locally via Ollama or hosted with a key.
Embed documents once during ingestion; embed only the question at query time.
Batches give the same vectors with fewer round trips: 0.99 s vs 1.77 s for 100 texts.
Use the task prefixes your model was trained with.
Store each vector with its text and source; never mix models.

Quick check before you move on

What was the biggest difference between batched and one-by-one vectors?

0.0000000 - identical.

Which document did "a charge for bringing the order to my house" find?

delivery.md.

Which prefix does nomic-embed-text want for questions?

"search_query: ".

Interview questions

How would you embed a large document collection efficiently?

Chunk, deduplicate, batch requests within provider limits, parallelise carefully with rate limiting and retries, cache by content hash, store vectors with metadata, and re-embed only changed chunks.

What are the trade-offs between local and hosted embedding models?

Local: no per-token cost, data stays in-house, but you manage hardware and scaling. Hosted: strong models and easy scaling, but cost, latency, rate limits and data-privacy constraints, plus lock-in since vectors are model-specific.

Quiz

1.
Why does batching save time?
2.
Why must you re-embed all documents when changing the embedding model?
3.
What should be stored next to each vector?

Comments

Loading comments...