What Is an AI Trace? A Guide for Product Teams

A trace captures the full sequence of what an AI system did in response to a single user request. Take a concrete example: a user opens an AI analytics assistant and asks, "Why did sign-ups drop last week?" The system doesn't return a single response. It queries a database, retrieves context from prior conversation, reasons over the results, generates an answer, and records whether the user acted on it or asked a follow-up. All of that is the trace.

Most traces contain some version of these fields:

• User input and detected intent. The raw query and the system's classification of what the user was trying to do (investigate a metric, find a segment, debug a funnel, etc.)
• Model response. The text or structured output the system returned
• Tool calls and results. Each external function, API, or data query the agent triggered, along with what it returned
• Retrieved context. Documents, prior conversation turns, or data rows the system pulled before generating its response (often called RAG, or retrieval augmented generation)
• Latency and cost. How long each step took and what it cost to run
• User feedback signals. Whether the user gave a thumbs up or down, asked a follow-up, abandoned the session, or converted on the action the AI recommended

One thing that surprises many PMs: a single trace can take seconds and involve dozens of nested steps, whereas a click event is a single timestamp with a handful of properties. The depth is what makes traces so useful, and why standard event tracking tools don't capture them.

A product event records what a user did. A trace records what the system did in response.

When someone clicks a button, your product analytics stack logs a flat record: event name, timestamp, user ID, and a set of properties. That's the whole thing. An AI interaction works differently. The user's input is a starting point, not the event. Everything that happens between input and output — the reasoning, the retrievals, the tool calls — is invisible to an event-tracking system.

Two differences matter most for PMs. First, the output is non-deterministic: two users asking the identical question can receive different responses, because the model's output depends on context, temperature, and retrieval results. You can't define "correct" the way you can verify a form submission. Second, the structure is nested rather than flat. A single interaction might contain five tool calls, three retrieval steps, and two model turns, each with its own latency and outcome.

That's why evaluating AI features requires traces rather than events. Events tell you whether users engaged with the feature. Traces tell you what quality of experience they actually got.

Trace analysis is the practice of inspecting traces, individually and in aggregate, to understand AI behavior and identify what to fix.

At the single-trace level, it works like reading a session recording or a stack trace: you open one interaction and walk through it step by step to understand why it went wrong. Say a user asked a question and got a confidently wrong answer. The trace shows you exactly where the failure happened: whether the model hallucinated an entity, a tool call returned an empty result, or the retrieval step pulled an outdated document.

At the aggregate level, trace analysis looks at patterns across many interactions: which intents fail most often, which tool calls return errors at high rates, which session shapes correlate with user satisfaction. This is how PM teams build a failure taxonomy — a named, structured set of failure modes that lets engineering and product prioritize what to fix.

The output of trace analysis is usually one of two things: a fix that goes into the system (a prompt change, a new retrieval strategy, a tool correction), or a new AI evaluation (eval) that captures the failure so it can be tracked over time. Trace analysis feeds the eval process; evals make the failure pattern measurable.

Pass rates and quality scores tell you whether the model performs on test cases. They don't tell you whether high-quality AI interactions drive retention, whether failure modes concentrate in your highest-value segments, or whether the most expensive query types convert at a meaningful rate.

Answering those questions requires joining trace data to the behavioral and product engagement data your team already tracks. That join, matching a trace to the same user identity whose retention curve, funnel conversion, and feature adoption you measure in product analytics, is what turns observability into a business case.

Amplitude's AI Agents product is built for this workflow. AI interactions are treated as events in the same behavioral event stream as every other action in your product, so you can segment by AI quality tier, correlate trace failure modes with downstream churn, or run an experiment where one variant gets a better prompt and measure whether the quality improvement shows up in a retention lift. The trace doesn't live in a separate observability silo; it connects to the same cohorts, funnels, and retention charts you already use.

A span is one unit of work within a trace. If a trace is the full record of an AI interaction, a span is a single step inside it: one tool call, one model turn, one retrieval request.

When you open a trace in a visualization tool, you'll typically see a tree or waterfall diagram where each row is a span. The parent span is the full interaction; child spans are the individual steps it triggered. Each span has its own start time, end time, status (success or error), and metadata.

For PMs, the key thing to know is that spans are what engineers debug when something goes wrong in a specific step. The distinction matters when you're talking to your engineering team: "the trace" refers to the whole interaction; "the span" refers to the individual step that failed.