RTE logo
← Back to Agentic AI map
Lesson 5.2 · LangSmith & Tooling

LangSmith Setup & Tracing

Lesson 5.1 built a tiny tracer by hand. LangSmith does the same job for you: create an API key, set three environment variables, and every LangChain and LangGraph run is traced - no code changes. Add @traceable for plain Python, tags and metadata, hide private data, and see exactly what is sent.

observability

What you will be able to do

  • Create a LangSmith account and an API key, and keep the key out of your code
  • Turn on tracing with environment variables
  • Trace plain Python functions with @traceable
  • Add tags and metadata so traces are easy to find
  • Hide inputs or outputs, sample traces, and turn tracing off
  • Know what happens when the key is wrong or the service is unreachable

The idea, in plain English

In Lesson 5.1 we wrote a 60-line tracer to see inside an agent. LangSmith is LangChain’s hosted service that does this properly: it records every run, keeps the history, and shows it in a web app where you can click through each step.

For LangChain and LangGraph code, you change nothing in your program. You set a few environment variables, and the library sends each run to LangSmith in the background.

You need a free account and an API key (the steps are below). To test every example on our own machine, we pointed the SDK at a small local server that saves exactly what would be sent. langsmith 0.14.7, langchain-core 1.6.9, LangGraph 1.2.14, llama3 on Ollama.

Worked example: The Lesson 5.7 helpdesk agent, traced without changing one line of it - plus @traceable functions, a failing tool, and four settings checked by capturing every request the SDK sent.

request flowWhat happens when tracing is onstep 1 / 3
React Flow

1 - The agent runs as usual

LangChain and LangGraph report each step (graph, node, model call, parser) to the SDK. Your code does not change.

runs recorded
18
code changes
none

The helpdesk agent with LANGSMITH_TRACING=true. Requests recorded by our local stand-in server.

Words you will see in this lesson

A few LangSmith words.

Small dictionary
LangSmithLangChain’s hosted service for tracing, evaluating and monitoring LLM apps.
API keyA secret string that proves who you are to LangSmith. Personal keys start with lsv2_pt_.
ProjectA folder for traces - one per app or experiment. Default name: "default".
TraceEverything that happened in one run of your app, as a tree (Lesson 5.1).
RunOne step inside a trace: a graph, a node, a model call, a tool.
@traceableA decorator that turns any Python function into a traced run.
Metadata / tagsExtra labels on a run (user id, version) so you can search and filter.
SamplingTracing only some runs - for example half of them - to save cost.

An everyday example: a flight recorder

  • Every airliner has a flight recorder. The pilots do not fly differently because of it - it records in the background. (Tracing needs no code changes.)
  • After a problem, experts replay the recording step by step. (Reading a trace - Lesson 5.3.)
  • If the recorder fails, the plane still flies. (A wrong key gave a warning, not a crash.)
  • Some conversations are private, so not everything is recorded. (Hiding inputs and outputs.)

Step 1 - create a free account and an API key

Do this once. The LangSmith website changes from time to time - if a button has moved, look for "API Keys" in the settings.

  • 1. Go to smith.langchain.com and sign up - with Google, GitHub or an email address. The free plan is enough for this course.
  • 2. In the app, open Settings (the gear icon).
  • 3. Open "API Keys" and click "Create API Key".
  • 4. Choose a Personal Access Token, give it a description such as "agentic-ai-course", and create it. The key starts with lsv2_pt_.
  • 5. Copy the key NOW. It is shown only once. If you lose it, delete it and create a new one.
  • 6. Your account is in the US region by default. If you chose the EU region, you also need LANGSMITH_ENDPOINT=https://eu.api.smith.langchain.com.

Watch out: The key gives access to your traces. Never paste it into code, a notebook you share, a chat, or git. Keep it in an environment variable or a .env file that is listed in .gitignore.

Step 2 - turn tracing on

Three environment variables. In the code examples, lsv2_pt_your_key_here is a placeholder - use your own key:

  • LANGSMITH_TRACING=true - switch tracing on.
  • LANGSMITH_API_KEY - your key.
  • LANGSMITH_PROJECT - where the traces go. Without it, they went to a project called "default".
  • A .env file is NOT read by itself. With only a .env file, the SDK sent nothing. Call load_dotenv() first (python-dotenv), or export the variables in your shell.
  • Older tutorials use LANGCHAIN_TRACING_V2 and LANGCHAIN_API_KEY. The SDK still reads the LANGCHAIN_ names, but LANGSMITH_ is the current spelling.
In the terminal (macOS / Linux)
export LANGSMITH_TRACING=true export LANGSMITH_API_KEY=lsv2_pt_your_key_here # placeholder - paste your own key export LANGSMITH_PROJECT=helpdesk-agent
Or in a .env file - and load it
# .env (add ".env" to .gitignore!) # LANGSMITH_TRACING=true # LANGSMITH_API_KEY=lsv2_pt_your_key_here # LANGSMITH_PROJECT=helpdesk-agent from dotenv import load_dotenv load_dotenv() # reads .env into the environment - before anything is traced

Example 1 - trace the helpdesk agent, unchanged

The agent from Lesson 5.7, without one changed line. Only two things are new:

  • The environment variables from Step 2.
  • wait_for_all_tracers() at the end. The SDK sends runs in the background; a short script can end before they are sent. A long-running server does not need it.
trace_agent.py
from helpdesk_agent.agent import build_graph, start_state graph = build_graph() result = graph.invoke(start_state("Should I bring an umbrella in Mumbai, and what is 12 * 8?"), {"recursion_limit": 20}) print(result["answer"]) from langchain_core.tracers.langchain import wait_for_all_tracers wait_for_all_tracers() # send everything before the program ends
What the SDK sent - recorded by our local stand-in server
GET /info POST /runs/multipart 11112 bytes x-api-key: lsv2_pt_your_key_here POST /runs/multipart 27795 bytes x-api-key: lsv2_pt_your_key_here POST /runs/multipart 28273 bytes x-api-key: lsv2_pt_your_key_here POST /runs/multipart 21147 bytes x-api-key: lsv2_pt_your_key_here 18 runs, project: helpdesk-agent

How we tested without an account

LANGSMITH_ENDPOINT tells the SDK where LangSmith is. We set it to a small local server that accepts every request and saves the body:

  • This lets you see exactly what leaves your machine - useful before you send real data to any service.
  • Our server did not answer the /info question about compression, so the SDK printed "Run compression is not enabled ... Falling back to regular multipart ingestion." With the real LangSmith you will not see this.
  • With your real key, simply do not set LANGSMITH_ENDPOINT (or set the EU address).
fake_langsmith.py - a local stand-in that saves what is sent
"""A local stand-in for the LangSmith API: accepts every request and saves what was sent.""" import json, time from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer class H(BaseHTTPRequestHandler): def _save(self): n = int(self.headers.get("Content-Length") or 0) body = self.rfile.read(n) if n else b"" with open("requests.log", "a") as f: f.write(json.dumps({"t": time.time(), "method": self.command, "path": self.path, "content_type": self.headers.get("Content-Type"), "encoding": self.headers.get("Content-Encoding"), "api_key": self.headers.get("x-api-key"), "bytes": len(body)}) + "\n") with open(f"body-{time.time_ns()}.bin", "wb") as f: f.write(body) self.send_response(200); self.send_header("Content-Type", "application/json"); self.end_headers() self.wfile.write(b"{}") do_GET = do_POST = do_PATCH = do_PUT = _save def log_message(self, *a): pass ThreadingHTTPServer(("127.0.0.1", 1984), H).serve_forever() # run the agent against it: # LANGSMITH_TRACING=true LANGSMITH_ENDPOINT=http://127.0.0.1:1984 \ # LANGSMITH_API_KEY=lsv2_pt_your_key_here LANGSMITH_PROJECT=helpdesk-agent python trace_agent.py

Example 2 - trace your own functions with @traceable

Not everything is LangChain. @traceable turns any Python function into a run:

  • run_type="tool" marks get_weather and calculate as tools, so they show as tools in the app.
  • Calls inside a traced function become its children: plan_trip -> get_weather, calculate.
  • name, tags and metadata label the run. Children inherit the tags and metadata.
  • An exception is recorded on the run as an error - with the traceback - and still raised to your code.
traceable_demo.py
from langsmith import traceable @traceable(run_type="tool") # any Python function can become a run def get_weather(city: str) -> str: return {"mumbai": "Rainy, 27°C"}.get(city.lower(), "unknown") @traceable(run_type="tool") def calculate(expression: str) -> str: if "/ 0" in expression: raise ZeroDivisionError("division by zero") return str(eval(expression, {"__builtins__": {}})) @traceable(name="plan_trip", tags=["demo"], metadata={"user_id": "u-42", "app_version": "0.1.0"}) def plan_trip(city: str) -> str: weather = get_weather(city) budget = calculate("1200 * 3") return f"{city}: {weather}, budget {budget}" print(plan_trip("Mumbai")) try: calculate("10 / 0") except ZeroDivisionError as error: print("caught:", error) from langsmith.run_trees import get_cached_client get_cached_client().flush() # send before exit
Output - the runs that were sent
Mumbai: Rainy, 27°C, budget 3600 caught: division by zero plan_trip [chain] inputs={"city": "Mumbai"} outputs={"output": "Mumbai: Rainy, 27°C, budget 3600"} tags=['demo'] metadata={'user_id': 'u-42', 'app_version': '0.1.0'} get_weather [tool] inputs={"city": "Mumbai"} outputs={"output": "Rainy, 27°C"} tags=['demo'] metadata={'user_id': 'u-42', 'app_version': '0.1.0'} calculate [tool] inputs={"expression": "1200 * 3"} outputs={"output": "3600"} tags=['demo'] metadata={'user_id': 'u-42', 'app_version': '0.1.0'} calculate [tool] inputs={"expression": "10 / 0"} outputs={"output": null} ERROR="ZeroDivisionError('division by zero')\n\nTraceback (most recen"

Tip: Put the user id, app version and model name in metadata. In LangSmith you can then filter: "all traces of user u-42" or "all traces since version 0.1.0".

Example 3 - four settings, checked

We ran the same scripts with different settings and recorded what was sent:

  • LANGSMITH_HIDE_INPUTS=true: every run was still sent, but with inputs={}. Outputs, tags and metadata stayed. (LANGSMITH_HIDE_OUTPUTS works the same way.) Use it when inputs contain personal data.
  • LANGSMITH_TRACING_SAMPLING_RATE=0.5: of 10 calls, one run traced 6 and another 4. Each trace is kept or dropped at random - good for busy apps.
  • Tracing off (no LANGSMITH_TRACING): no runs were sent. The only request was GET /info, caused by our own flush() call; without that call, nothing at all.
  • A wrong key with the real LangSmith: 403 Forbidden and one warning line - the program finished normally.
Output - LANGSMITH_HIDE_INPUTS=true
plan_trip [chain] inputs={} outputs={"output": "Mumbai: Rainy, 27°C, budget 3600"} tags=['demo'] metadata={'user_id': 'u-42', 'app_version': '0.1.0'} get_weather [tool] inputs={} outputs={"output": "Rainy, 27°C"} tags=['demo'] metadata={'user_id': 'u-42', 'app_version': '0.1.0'} calculate [tool] inputs={} outputs={"output": "3600"} tags=['demo'] metadata={'user_id': 'u-42', 'app_version': '0.1.0'} calculate [tool] inputs={} outputs={"output": null} ERROR="ZeroDivisionError('division by zero')\n\nTraceback (most recen"
Output - the placeholder key against the real LangSmith
Failed to multipart ingest runs: langsmith.utils.LangSmithError: Failed to POST https://api.smith.langchain.com/runs/multipart in LangSmith API. HTTPError('403 Client Error: Forbidden for url: https://api.smith.langchain.com/runs/multipart', '{"error":"Forbidden"}\n') Mumbai: Rainy, 27°C, budget 3600 caught: division by zero

Watch out: Tracing sends your prompts, tool results and answers to a hosted service. Check what your data rules allow. Hide inputs/outputs, or keep personal data out of traces, before you trace real users.

The recipe

  • 1. Create a free account and a Personal Access Token at smith.langchain.com. Copy it once; keep it secret.
  • 2. Set LANGSMITH_TRACING=true, LANGSMITH_API_KEY and LANGSMITH_PROJECT (export, or .env + load_dotenv()).
  • 3. Run your LangChain / LangGraph code unchanged. Short scripts: wait_for_all_tracers() at the end.
  • 4. Add @traceable to your own important functions, with tags and metadata.
  • 5. Before real users: decide what may be sent - hide inputs/outputs, and sample if traffic is high.
  • Next: Lesson 5.3 reads the trace we just made, step by step.

LangSmith setup at a glance

Turn on

Environment variable.

LANGSMITH_TRACING=true
Your key

Never in code.

LANGSMITH_API_KEY=lsv2_pt_...
Project

Default: "default".

LANGSMITH_PROJECT=helpdesk-agent
EU region

Only if you chose EU.

LANGSMITH_ENDPOINT=https://eu.api.smith.langchain.com
Own function

Becomes a run.

@traceable(run_type="tool")
Labels

For filtering.

@traceable(tags=[...], metadata={...})
Private data

Sent as {}.

LANGSMITH_HIDE_INPUTS=true
Only some traces

0 to 1.

LANGSMITH_TRACING_SAMPLING_RATE=0.5
Short scripts

Send before exit.

wait_for_all_tracers()

Try it yourself

The code does not change. Swap the content string and the program does something else entirely.

Your first trace

“With your real key, run trace_agent.py and open the helpdesk-agent project in LangSmith.”

Find by metadata

“Run traceable_demo.py, then filter the project by the metadata user_id = u-42.”

See the error

“Open the failing calculate run. Where does LangSmith show the traceback?”

Two projects

“Run the agent once with LANGSMITH_PROJECT=helpdesk-dev and once with helpdesk-test. Why keep them apart?”

What usually goes wrong

The key in the code

Anyone who sees the code or the git history can use it.

✗ os.environ["LANGSMITH_API_KEY"] = "lsv2_pt_8f3..."
✓ export LANGSMITH_API_KEY=...   # or .env + load_dotenv(), .env in .gitignore
Expecting .env to load itself

With only a .env file, nothing was traced. Call load_dotenv() first.

Short scripts that end too early

Runs are sent in the background. Call wait_for_all_tracers() (or flush) before the script ends.

Tracing personal data without thinking

Prompts and answers go to a hosted service. Hide inputs/outputs or remove personal data first.

Key points

  • LangSmith traces LangChain and LangGraph runs with three environment variables and no code changes.
  • Create a Personal Access Token once; keep it in the environment, never in code or git.
  • A .env file needs load_dotenv(); short scripts need wait_for_all_tracers().
  • @traceable traces any Python function; tags and metadata make traces easy to find.
  • HIDE_INPUTS sent inputs={}; SAMPLING_RATE=0.5 kept about half; a wrong key gave 403 but the app kept working.

Quick check before you move on

Which three variables turn tracing on?
LANGSMITH_TRACING=true, LANGSMITH_API_KEY and LANGSMITH_PROJECT.
Where did traces go without LANGSMITH_PROJECT?
To a project called "default".
What did a wrong key do to the program?
Nothing serious: a 403 warning was printed and the program finished normally.
What did LANGSMITH_HIDE_INPUTS=true change?
Runs were still sent, but every inputs field was {}.

Quiz

  1. 1.

    Why did our .env file alone not turn tracing on?

  2. 2.

    Why call wait_for_all_tracers() in a short script?

  3. 3.

    How did we test tracing without an account?

  4. 4.

    What happens to the children of a @traceable function with tags?

Interview questions

How do you add tracing to a LangChain or LangGraph app?

Set LANGSMITH_TRACING, LANGSMITH_API_KEY and LANGSMITH_PROJECT; runs are captured via callbacks automatically. Use @traceable for non-LangChain code, and tags/metadata for filtering.

How do you trace responsibly in production?

Keep keys in a secret store, mask or hide sensitive inputs/outputs, sample high-volume traffic, separate projects per environment, and make sure tracing failures cannot affect the request path.

Comments

Sign in to leave a comment. Your name and photo come from Google; nothing else is shared.

Loading comments...