Lesson 1.1 · RAG Fundamentals

What is Generative AI?

Generative AI creates new content - text, images, code - instead of looking something up. Run llama3 on your own laptop, ask for the same slogan six times, and see why it can write anything: including a confident, invented phone number for a bakery that does not exist.

Beginner20 min

What you will be able to do

Explain in simple words what generative AI is
Tell the difference between generating and looking up
Run a local model with Python and Ollama
Explain what temperature changes
Recognise an invented answer - the problem RAG solves

The idea, in plain English

Most software you know finds or calculates things: a search engine finds pages, a calculator computes a number, a database returns rows. Generative AI does something different: it creates new content that did not exist before - a sentence, a picture, a piece of code.

The kind we use in this course generates text. It is a large language model (Lesson 1.2), and it writes one small piece of text at a time, always choosing what is likely to come next. That is why it can write anything - and also why it can write things that are not true.

Everything in this course runs on your own computer: the llama3 model (8 billion parameters) through Ollama, used from Python. Nothing is sent to the internet.

Worked example: A slogan for a Hyderabad bakery generated six times, and two questions llama3 cannot know the answer to.

comparisonLook up, or generate?step 1 / 2

a question

or a request

search / database

finds existing text

generative AI

writes new text

✓

React Flow

1 - Creating something new

Asked for a bakery slogan, llama3 wrote one that exists nowhere else. At temperature 0.8 it wrote a different one each time.

temperature 0

same slogan 3 times

temperature 0.8

3 different slogans

time

about 1.5 s each

Real llama3 answers from this lesson.

Words you will see in this lesson

The first words of the course.

Small dictionary

Generative AIAI that creates new content: text, images, audio or code.

ModelThe trained program that does the generating - here, llama3.

PromptThe text you give the model: a question or an instruction.

ResponseThe text the model generates back.

OllamaA program that runs models like llama3 on your own computer.

TemperatureA setting for randomness. 0 = always the most likely words; higher = more variety.

HallucinationA confident answer that is not true (Lesson 1.4).

An everyday example: a librarian and a storyteller

A librarian finds you a book that already exists. If there is no such book, they say so. (Search.)
A storyteller makes up a new story on the spot - fluent, fun, and different every time. (Generative AI.)
Ask a storyteller for a real phone number, and you may get one that sounds right but is invented.
RAG - this course - gives the storyteller the right book to read from before they answer.

Example 1 - generate a slogan

Before you start: install Ollama, run ollama pull llama3, and pip install ollama. Then:

ollama.generate(model, prompt, options) sends the prompt to the local model and returns the response.
We ask for the same slogan three times at temperature 0, and three times at 0.8.
time.perf_counter() measures how long each answer takes.

generate.py

import time
import ollama

PROMPT = "Write a one-sentence slogan for a small bakery in Hyderabad."

for temperature in [0.0, 0.8]:
    print(f"=== temperature {temperature}")
    for run in range(1, 4):
        start = time.perf_counter()
        reply = ollama.generate(model="llama3", prompt=PROMPT, options={"temperature": temperature})
        print(f"{run}. {reply.response.strip()}   ({time.perf_counter() - start:.1f} s)")

Output - a real run

=== temperature 0.0
1. "Rise to the occasion with every bite at Hyderabad's sweetest spot, [Bakery Name]!"   (1.4 s)
2. "Rise to the occasion with every bite at Hyderabad's sweetest spot, [Bakery Name]!"   (1.4 s)
3. "Rise to the occasion with every bite at Hyderabad's sweetest spot, [Bakery Name]!"   (1.3 s)
=== temperature 0.8
1. "Rise to the Occasion with Every Bite: Hyderabad's Sweetest Treats, Freshly Baked with Love!"   (1.5 s)
2. Here's a one-sentence slogan for a small bakery in Hyderabad:

"Rise to the occasion with every bite, at Hyderabad's sweetest little bakery"   (1.9 s)
3. "Rise to the occasion with every bite at Hyderabad's sweetest destination, [Bakery Name] - where tradition meets taste!"   (1.6 s)

Reading Example 1

The slogan did not exist anywhere - the model wrote it. That is "generative".
Temperature 0: the same slogan three times. The model always took the most likely next words.
Temperature 0.8: three different slogans. A little randomness lets it pick less likely words.
All six start with "Rise to the occasion" - the model has favourite phrases.
Run 2 added "Here’s a one-sentence slogan..." - small models often talk about their answer. You will see this again.
For RAG we use temperature 0: the same question should give the same answer, based on the documents.

Example 2 - questions it cannot know

Generating is great for slogans. For facts, it is a problem. We asked two questions llama3 cannot know the answer to:

The phone number of "Sri Lakshmi Bakery on MG Road" - a bakery we made up.
The winner of a Nobel Prize in a year that has not happened in its training data.

made_up.py

import ollama

for question in ["What is the phone number of Sri Lakshmi Bakery on MG Road, Hyderabad?",
                 "Who won the Nobel Prize in Physics in 2030?"]:
    reply = ollama.generate(model="llama3", prompt=question, options={"temperature": 0})
    print(f"=== {question}\n{reply.response.strip()}\n")

Output

=== What is the phone number of Sri Lakshmi Bakery on MG Road, Hyderabad?
I'm happy to help! According to online directories and reviews, the phone number of Sri Lakshmi Bakery on MG Road, Hyderabad is +91 98480 44444. However, please note that phone numbers can change, and it's always a good idea to verify the information before making a call. You can also try searching for the bakery's official website or social media pages for the most up-to-date contact information.

=== Who won the Nobel Prize in Physics in 2030?
I'm afraid I have some bad news!

As of now, it's still 2023, and the Nobel Prizes for 2030 have not been awarded yet. The Nobel Prizes are awarded annually by the Nobel Foundation to recognize outstanding contributions in the fields of Physics, Chemistry, Medicine, Literature, Peace, and Economics.

I'll make sure to keep an eye out for the announcements in 2030, and I'll be happy to share the news with you then!

Reading Example 2

The bakery does not exist - but llama3 gave a phone number, and said it came "according to online directories". It has no internet and no directories. It wrote what a helpful answer usually looks like.
The second answer is more honest - but it thinks "it’s still 2023". A model only knows what was in its training data, up to a cut-off date (Lesson 1.2).
This is the problem the whole course solves: the model sounds sure even when it is guessing.
RAG (Lesson 1.3) gives the model the real documents to read before it answers.

Watch out: A fluent, confident answer is not a correct one. Never trust a model’s facts - like phone numbers, prices or policies - unless they come from a source you can check.

What generative AI is good and bad at

Good at / bad at

Good atWriting, rewording, summarising, translating, explaining, drafting code - work with words.

Bad atExact facts it was never shown: your company’s policies, today’s prices, a customer’s order.

Unknown to itAnything after its training cut-off, and anything private.

The fixGive it the facts in the prompt - retrieved from your documents. That is RAG.

The recipe

1. Install Ollama, pull llama3, pip install ollama.
2. ollama.generate(model="llama3", prompt=..., options={"temperature": 0}) for repeatable answers.
3. Use generation for writing tasks.
4. Never trust generated facts without a source - Lesson 1.3 shows how RAG supplies one.

Generative AI at a glance

Run a model

Locally, with Ollama.

ollama pull llama3

Generate

From Python.

ollama.generate(model="llama3", prompt="...")

Repeatable

Same answer each time.

options={"temperature": 0}

Varied

More creative.

options={"temperature": 0.8}

The text

Of the reply.

reply.response

Try it yourself

The code does not change. Swap the content string and the program does something else entirely.

Your shop

“Ask for a slogan for your own favourite shop at temperature 1.2. Does it still make sense?”

Another fake fact

“Ask llama3 for the opening hours of a shop you invent. Does it admit it does not know?”

Writing task

“Ask it to rewrite a paragraph in simpler English. This is what it is good at.”

What usually goes wrong

Trusting confident answers

The invented phone number came with "according to online directories".

High temperature for facts

Randomness helps slogans, not facts. Use temperature 0 for RAG.

Expecting today’s knowledge

llama3 thought it was 2023. Models do not know recent events.

Practice

Write these yourself before opening anything. Getting them wrong first is most of how this sticks.

Write a script that asks llama3 five factual questions about a city you know well (including one that is false, like a famous building that does not exist). Mark each answer right or wrong.

Show hint

Use temperature 0. Check every answer against a real source. Count how many sounded confident but were wrong.

Key points

Generative AI creates new content; a search or database finds existing content.
A language model writes text one small piece at a time, choosing likely next words.
Temperature 0 gave the same slogan every time; 0.8 gave a different one each time.
llama3 invented a phone number for a bakery that does not exist - fluent is not true.
RAG fixes this by giving the model real documents to answer from.

Quick check before you move on

What is the difference between search and generative AI?

Search finds content that exists; generative AI writes new content.

What did temperature 0 do?

Gave the same answer every time.

Why did llama3 give a phone number for a bakery that does not exist?

It writes what a helpful answer looks like, not what is true. It has no directory to check.

Interview questions

What is generative AI?

Models that generate new content - text, images, audio, code - by learning patterns from large training data, as opposed to systems that retrieve or classify existing data.

Why do LLMs hallucinate?

They generate the most likely continuation of the text, not verified facts; without grounding data they fill gaps with plausible-sounding content.

Quiz

1.
Which temperature should a RAG app use, and why?
2.
Why did llama3 say "it’s still 2023"?
3.
Name two tasks generative AI is good at.

Comments

Loading comments...