Interview: LLM and Prompting Basics

What you will be able to do

• ✓Answer the basic LLM questions clearly in under a minute each
• ✓Explain tokens, context windows and temperature with examples
• ✓Explain hallucination and how to reduce it
• ✓Compare prompting, RAG and fine-tuning
• ✓Use a simple answer structure for any question

The idea, in plain English

Every agent interview starts with the model underneath. If you cannot explain tokens, the context window or why a model hallucinates, the interviewer will not trust your answers about agents.

This module collects the questions that come up again and again in agentic AI interviews - from basic to advanced - with answers in plain words. Each answer has the same shape: a one-line definition, how it works, a short example (often from a lesson in this course), and a trade-off.

Read the answer, then close the page and say it out loud in your own words. The key points under each answer are what an interviewer listens for.

Worked example: Fifteen basic-to-advanced questions on the model under every agent, answered in simple words with examples from this course.

How to use this module - and how to answer

Interviewers check two things: do you understand it, and have you built it. A good answer shows both.

• •1. Start with a one-line definition - no long warm-up.
• •2. Explain how it works in two or three sentences.
• •3. Give a concrete example - best from something you built (the lessons of this course count).
• •4. Name a trade-off or a limit. This is what separates a senior answer from a junior one.
• •5. Stop. Let the interviewer ask a follow-up.
• •Levels: Basic = you must know it; Intermediate = expected for a working engineer; Advanced = senior and design questions.

Basic: What is a large language model (LLM)?

An LLM is a neural network trained on a very large amount of text to predict the next token. Given some text, it gives a probability for every possible next token, picks one, adds it, and repeats.

• •It does not look anything up - it generates from patterns learned in training.
• •"Large" means billions of parameters (llama3 has 8 billion).
• •Everything else - chat, tools, agents - is built on next-token prediction.

Basic: What is a token, and why does it matter?

A token is a piece of text the model reads and writes - a word, part of a word, or a symbol. Text is split into tokens before the model sees it.

• •Cost: hosted APIs charge per input and output token.
• •Speed: output is generated one token at a time.
• •Limits: the context window is counted in tokens.
• •Languages differ: many non-English languages need several times more tokens for the same meaning.

Basic: What is the context window?

The context window is the maximum number of tokens the model can look at in one call - the prompt plus the answer. Anything outside it does not exist for the model.

• •Agents fill it fast: system prompt, tool descriptions, history, tool results.
• •Too long can be cut silently - some runtimes drop the start of the prompt without an error.
• •Fixes: summarise history, retrieve only what is needed, trim tool results.

Basic: What do temperature and top-p do?

They control how the next token is chosen. Temperature 0 always takes the most likely token, so the same prompt gives the same answer. Higher temperature makes less likely tokens possible - more varied, more creative, less predictable.

• •Top-p keeps only the most likely tokens whose probabilities add up to p, then samples from those.
• •Agents and tool calls: low temperature, for repeatable decisions.
• •Brainstorming or writing: higher temperature.

Basic: What is a hallucination, and why does it happen?

A hallucination is a fluent, confident answer that is not true - an invented fact, number, name or source. It happens because the model is trained to produce likely-sounding text, not checked facts.

• •Worst on private data, recent events and rare facts.
• •It can even invent a source ("according to the website").
• •Reduce it with RAG, tools that fetch real data, instructions to say "I don't know", and checks in code.

Basic: What is a system prompt?

The system prompt is the instruction that sets the model's role and rules for the whole conversation, sent before the user's messages.

• •Typical content: role, tone, rules, output format, what to refuse.
• •It shapes behaviour but does not enforce it - important rules must also be checked in code.

Basic: What is zero-shot, few-shot and chain-of-thought prompting?

Zero-shot gives only the instruction. Few-shot adds a few input-output examples so the model copies the pattern. Chain-of-thought asks the model to reason step by step before answering.

• •Few-shot is the cheapest way to fix a format.
• •Chain-of-thought helps with multi-step problems but costs more tokens and time.
• •In agents, ReAct is chain-of-thought plus actions.

Intermediate: How do you get reliable structured output (like JSON) from an LLM?

Ask for an exact schema, use the model's JSON or structured-output mode if it has one, then parse and validate the result in code - and retry or repair when it fails.

• •Validate with Pydantic or JSON Schema - never trust the raw text.
• •Keep schemas small and give an example.
• •Lesson 1.3 and Lesson 2.5 do this with a local model.

Intermediate: Prompting, RAG or fine-tuning - how do you choose?

Start with prompting. Add RAG when the model needs knowledge it does not have - private, recent or changing facts that must be cited. Fine-tune when you need a behaviour prompting cannot give - a fixed style, format or narrow skill.

• •RAG changes what the model sees; fine-tuning changes its weights.
• •RAG is easy to update and can cite sources; fine-tuning is slow to change.
• •Many systems use prompting + RAG, and fine-tune only later.

Intermediate: What are open-weight and closed models, and why would you run a model locally?

Closed models (most hosted APIs) are used only through a provider. Open-weight models (like llama3) can be downloaded and run on your own machine, for example with Ollama.

• •Local: data stays in-house, no per-token cost, works offline.
• •Hosted: usually stronger models, no hardware to manage, easy scaling.
• •This course runs everything locally with Ollama for that reason.

Intermediate: Why are the same prompts sometimes worse on a different model?

Models are trained differently - on different data, prompt formats and tool-calling formats. A prompt tuned for one model is not guaranteed to work for another.

• •Re-test prompts and tool calls when you switch models.
• •Smaller local models need clearer, shorter instructions.
• •Keep an evaluation set so the comparison is measured, not guessed.

Advanced: How would you reduce the cost and latency of LLM calls in an agent?

Send fewer tokens and make fewer calls: shorter prompts, trimmed history and tool results, a smaller model for easy steps, caching repeated calls, and streaming so the user sees output early.

• •Measure first - trace where tokens and time go (Lesson 5.6).
• •Route: cheap model for classification, strong model for hard reasoning.
• •Run independent tool calls in parallel.
• •Set limits: max steps, max tokens, timeouts.

Advanced: What is prompt caching, and when does it help?

Prompt caching lets a provider reuse the processed form of a prompt prefix that repeats - like a long system prompt or tool list - so later calls are faster and cheaper.

• •It works only for an identical prefix, so put stable content first and changing content last.
• •Big win for agents, which resend the same instructions every step.

Advanced: How do you evaluate whether a prompt change made things better?

Run the old and new prompt on the same fixed set of test inputs with known good answers, score both automatically, and compare - never judge from one or two examples.

• •Keep the test set in version control.
• •Score with exact checks where possible, LLM-as-judge where not.
• •Watch for regressions on cases that used to work.

Interview questions