Use Cases and Limitations of RAG
The same small RAG pipeline answered a support bot, an HR policy search and a library’s docs - five questions, each with a source. Then we went looking for where it breaks, and found four real limits - a different word, counting across chunks, an old policy file, a right document with a wrong conclusion - and one hard test it passed.
What you will be able to do
- Recognise good use cases for RAG
- Run one RAG pipeline on different kinds of documents
- Find the limits of RAG with targeted tests
- Tell retrieval limits from generation limits
- Choose a fix for each limit
The idea, in plain English
RAG fits any job of the form "answer questions from these documents": a customer support bot, a search over company policies, a helper for a library’s documentation. The documents change, the answers must be checkable, and nobody wants a model that invents rules.
But RAG is not magic. It can only answer from what it retrieves, and the model can still read the right text wrongly. Some questions do not fit at all - "how many?" and "list all" need EVERY document, not the top two.
We reuse the pipeline from Lesson 1.6 on three document sets, then write five questions designed to break it, and see what each failure teaches. llama3 on Ollama, temperature 0.
Worked example: One pipeline, three document sets - a bakery, an HR handbook and a code library’s README - then five tests built to break it.
1 - Three use cases
Support, policy search and code docs - the same code, different folders.
One pipeline, three document sets, five stress tests. Real results.
Words you will see in this lesson
A few words about limits.
Use caseA real job a tool is used for.AggregationA question about ALL the data: count, total, list every...Multi-hopA question that needs two or more facts from different places.Conflicting documentsTwo documents that say different things about the same rule.Vocabulary mismatchThe question and the document use different words for the same idea.An everyday example: a reference desk
- A good reference librarian finds you the right page fast - that is RAG at its best.
- Ask "how many books in this library mention rice?" - they cannot read every book for you. That is an aggregation question.
- If the shelf has both the 2023 and the 2025 rule book, they may hand you the wrong one.
- And even with the right page open, a tired reader can misread a number.
Example 1 - one pipeline, three use cases
minirag.py is the Lesson 1.6 pipeline in one file: ingest(folder) and ask(chunks, question). use_cases.py runs it on three folders:
- bakery - a customer support bot (the documents from Lesson 1.6).
- hr - an HR policy search: leave.md and remote.md.
- code - a library’s documentation: one README with three sections.
- Nothing in the code is bakery-specific. Change the folder, and you have a different assistant.
# Leave policy
## Annual leave
Every employee gets 18 days of paid annual leave per year. Up to 5 unused days can be carried into the next year.
## Sick leave
Employees get 10 days of sick leave per year. More than 2 sick days in a row need a doctor's note.
# shopcart - a small Python library
## Installation
Run pip install shopcart. Python 3.10 or newer is required.
## add_item
cart.add_item(name, price, qty=1) adds an item. qty must be at least 1, or ValueError is raised.
## total
cart.total(discount=0) returns the sum of price times qty, minus the discount in percent.# The pipeline from Lesson 1.6 in one file: ingest(folder) once, then ask(question)
import re
from pathlib import Path
import ollama
STOP = {"a", "an", "the", "is", "are", "i", "my", "can", "do", "does", "you", "for", "to", "of", "in", "on",
"it", "what", "if", "be", "how", "much", "many", "must", "at", "and", "we", "sri", "lakshmi", "bakery",
"which", "will", "get", "me", "per"}
def words(text):
return {w for w in re.findall(r"[a-z0-9_]+", text.lower()) if w not in STOP}
def ingest(folder):
chunks = []
for path in sorted(Path(folder).glob("*.md")):
for section in re.split(r"\n## ", path.read_text())[1:]:
title, _, body = section.partition("\n")
source = f"{path.name}#{title.strip()}"
chunks.append({"source": source, "text": body.strip(), "words": words(source + " " + body)})
return chunks
def ask(chunks, question, k=2):
asked = words(question)
scored = sorted(chunks, key=lambda c: len(asked & c["words"]), reverse=True)
found = [c for c in scored[:k] if asked & c["words"]]
if not found:
return [], "I don't know - our documents do not cover this."
context = "\n".join(f"[{c['source']}] {c['text']}" for c in found)
prompt = (f"Answer the question using only the context below. Apply the numbers and rules to the question. "
f"Answer in one or two sentences and name the source in brackets.\n\n"
f"Context:\n{context}\n\nQuestion: {question}")
reply = ollama.generate(model="llama3", prompt=prompt, options={"temperature": 0})
return [c["source"] for c in found], reply.response.strip()from minirag import ask, ingest
CASES = {
"bakery": ["Can I get a refund for a custom cake I cancelled 2 days before pickup?"],
"hr": ["How many unused leave days can I carry into next year?",
"Do I need a doctor's note for 3 sick days?"],
"code": ["What happens if I call add_item with qty=0?",
"Which Python version does shopcart need?"],
}
for folder, questions in CASES.items():
chunks = ingest(folder)
print(f"=== {folder}: {len(chunks)} chunks")
for question in questions:
sources, answer = ask(chunks, question)
print(f"Q: {question}\n found: {sources}\n A: {answer}")=== bakery: 8 chunks
Q: Can I get a refund for a custom cake I cancelled 2 days before pickup?
found: ['policies.md#Refunds', 'menu.md#Cakes']
A: No, you cannot get a refund for a custom cake you cancelled 2 days before pickup, as the policy requires cancellation at least 24 hours before pickup. [policies.md#Refunds]
=== hr: 4 chunks
Q: How many unused leave days can I carry into next year?
found: ['leave.md#Annual leave', 'leave.md#Sick leave']
A: According to the context, up to 5 unused days can be carried into the next year. [leave.md#Annual leave]
Q: Do I need a doctor's note for 3 sick days?
found: ['leave.md#Sick leave', 'remote.md#Working from home']
A: According to the context, since you're taking more than 2 sick days in a row, you would need a doctor's note. [leave.md#Sick leave]
=== code: 3 chunks
Q: What happens if I call add_item with qty=0?
found: ['README.md#add_item', 'README.md#total']
A: If you call `add_item` with `qty=0`, a `ValueError` is raised because `qty` must be at least 1. [README.md#add_item]
Q: Which Python version does shopcart need?
found: ['README.md#Installation']
A: Shopcart requires Python 3.10 or newer. [README.md#Installation]Reading the use cases
- HR: "up to 5 unused days" and "more than 2 sick days need a doctor’s note" - both right, each with its section.
- Code: the ValueError for qty=0 and "Python 3.10 or newer" - right. Developers get answers from the docs instead of guesses.
- Bakery - a surprise: the right section was found, but the answer was "No, you cannot get a refund ... the policy requires cancellation at least 24 hours before pickup". 2 days IS more than 24 hours, so the answer should be yes. In Lesson 1.3 a different prompt got this right.
- Lesson: retrieval can be perfect and generation still wrong. Comparisons of numbers and times are a known weak spot of models - test them, and keep a prompt that works.
Example 2 - five tests built to break it
limits.py asks five questions, each aimed at one possible weak spot:
import shutil
from pathlib import Path
from minirag import ask, ingest
bakery = ingest("bakery")
TESTS = [
("1. different words", bakery, "Can I return my cake?"),
("2. two facts needed", bakery, "Is delivery free if I order one 1 kg black forest cake?"),
("3. counting everything", bakery, "How many items on the menu cost more than Rs 50?"),
]
# 4. two versions of the same rule: an old file nobody deleted
shutil.copytree("hr", "hr_old", dirs_exist_ok=True)
Path("hr_old/leave_2023.md").write_text(
"# Leave policy 2023\n\n## Annual leave\nEvery employee gets 15 days of paid annual leave per year.\n")
hr_old = ingest("hr_old")
TESTS.append(("4. old and new policy", hr_old, "How many days of annual leave do I get?"))
TESTS.append(("5. not in the documents", ingest("hr"), "What is the salary of a team manager?"))
for label, chunks, question in TESTS:
sources, answer = ask(chunks, question)
print(f"{label}\nQ: {question}\n found: {sources}\n A: {answer}\n")1. different words
Q: Can I return my cake?
found: ['menu.md#Cakes']
A: According to the context, there is no mention of any return policy or rules, so it is not possible to determine if you can return your cake.
2. two facts needed
Q: Is delivery free if I order one 1 kg black forest cake?
found: ['menu.md#Cakes', 'policies.md#Delivery']
A: No, delivery is not free if you order one 1 kg black forest cake, as the cost of the cake (Rs 650) is less than the minimum order value of Rs 1000. [Source: policies.md#Delivery]
3. counting everything
Q: How many items on the menu cost more than Rs 50?
found: ['menu.md#Pastries', 'menu.md#Cakes']
A: Two items on the menu cost more than Rs 50: the chocolate pastry at Rs 55 and the 1 kg black forest cake at Rs 650. [menu.md]
4. old and new policy
Q: How many days of annual leave do I get?
found: ['leave.md#Annual leave', 'leave_2023.md#Annual leave']
A: According to the context, you get 15 days of paid annual leave per year. [leave_2023.md#Annual leave]
5. not in the documents
Q: What is the salary of a team manager?
found: ['remote.md#Working from home']
A: I apologize, but there is no information provided about the salary of a team manager in the given context. The context only mentions the policy on working from home, not salaries.Reading the five tests
- 1. Different words - "return" is not "refund". The search found the cakes section instead, and the model honestly said there is no return policy. Wrong for the customer. Fix: embeddings that match meaning (Module 2).
- 2. Two facts needed - it found the cake price AND the delivery rule, and combined them: Rs 650 is under Rs 1000, so delivery is not free. Right - RAG handled this one well.
- 3. Counting - "how many items cost more than Rs 50?" The answer was 2; the truth is 3 (chocolate pastry 55, black forest 650, butterscotch 600). Even with both menu chunks, the model miscounted. Fix: for counts, totals and lists, use code or a database query - not RAG.
- 4. Old and new policy - the old file leave_2023.md said 15 days, the current one 18. Both were retrieved, and the model chose the OLD one. Fix: remove old documents, or store a date with each chunk and prefer the newest (Module 5).
- 5. Not in the documents - "salary of a team manager" matched "manager" in the remote-work policy, but the model said there is no salary information. Right - and a reminder that the code’s "nothing found" check does not catch weak matches.
When to use RAG - and when not
- Good fit: questions answered by a passage in your documents - support, policies, product docs, contracts, manuals, internal wikis.
- Good fit: answers that must be current and checkable, with a source.
- Poor fit: counting, totals, rankings and "list all" - use SQL or code over structured data.
- Poor fit: messy, duplicated or outdated documents - RAG repeats what the documents say, good or bad.
- Poor fit: deep reasoning over many documents at once - see advanced patterns (Module 9) and agents.
- Always: measure on real questions (Module 11). Our five tests found three failures in five minutes.
RAG limits at a glance
Different wordsRetrieval misses.
"return my cake" -> refunds not found -> embeddings
Counting / listingNeeds all data.
said 2, truth 3 -> SQL or code
Old documentsConflicting facts.
15 days (2023) won over 18 -> clean up, dates
Reasoning slipRight text, wrong conclusion.
2 days vs 24 hours -> test prompts
Two factsOften fine.
cake Rs 650 + delivery rule -> not free (right)
Try it yourself
The code does not change. Swap the content string and the program does something else entirely.
“Make a folder with three .md files from your own work or college and ask five questions with minirag.”
“Use the Lesson 1.3 prompt in minirag.ask. Is the refund answer right now - and do the others stay right?”
“Write count_items(chunks, above=50) in plain Python that reads the prices with a regex. Compare with RAG’s answer.”
“Delete hr_old/leave_2023.md and ask again. What does the answer say now?”
What usually goes wrong
It said 2 items; there are 3. Use code or SQL for aggregation.
The 2023 policy (15 days) won over the current one (18).
The refund section was found, and the conclusion was still wrong.
Five targeted questions found three failures. Test the weak spots on purpose.
Practice
Write these yourself before opening anything. Getting them wrong first is most of how this sticks.
Write a test file for minirag with ten questions about the HR folder: six answerable, two that need two sections, and two not covered. For each, store the expected key words, run the questions, and print a score (like Lesson 1.4). Then add the old 2023 policy and run it again - how many answers get worse?
Show hintHide hint
Reuse grade() from Lesson 1.4. Run each version twice to make sure temperature 0 gives the same results.
Key points
- RAG fits "answer from these documents" jobs: support, policies, documentation.
- The same pipeline works for any folder of documents.
- Retrieval limits: different words, and questions that need every chunk.
- Document limits: old and conflicting files are repeated, not fixed.
- Generation limits: the right text can still lead to a wrong conclusion.
- Use code or SQL for counting and listing; test the weak spots on purpose.
Quick check before you move on
Interview questions
What are the main limitations of RAG?
Retrieval failures (vocabulary mismatch, poor chunking), questions needing global or aggregate reasoning, conflicting or stale sources, context window limits, and generation errors even with correct context; plus latency and cost.
How would you handle "how many" or "list all" questions in a RAG product?
Route them to structured data - SQL, metadata filters or code tools - via query classification or an agent, instead of top-k retrieval over text.
Quiz
- 1.
Why is counting a poor fit for RAG?
- 2.
The refund section was retrieved but the answer was wrong. Which part failed?
- 3.
Name two fixes for conflicting documents.
Comments
Sign in to leave a comment. Your name and photo come from Google; nothing else is shared.
Loading comments...