Analyze agent results

After events are flowing, Agent Analytics gives you three ways in, one for each item in its left navigation: Monitor for the big picture, Sessions for individual diagnosis, and Evaluators for measuring the criteria you define. Each item groups related views into tabs. Everything also works in standard Amplitude charts, funnels, and cohorts.

Monitor

Monitor is the first item in the Agent Analytics left navigation. It has five tabs: Overview, Charts, Topics, Insights, and Costs. Monitor opens on Overview. Overview, Charts, and Costs share one date range. Topics and Insights each use their own reporting window.

Overview

Overview is the tab Monitor opens on, and it gathers the project's headline numbers onto one screen:

• Key metrics for your agents.
• Activity and cost charts, side by side.
• A feed of recent insights, with pinned findings first.
• A table of top-level topics, sized by how many sessions fall in each.

Select an insight to open it on the Insights tab, or select a topic row to open that cluster on the Topics tab.

Charts

The Charts tab on Monitor summarizes agent health across the selected project and time range. It shows agent activity and session volume, session quality from the always-on signals, and cost and latency trends. Use these charts to spot a shift before you open individual sessions, and apply the agent filter to focus on one agent or compare agents.

Topics

The Topics tab on Monitor groups your closed sessions by what they were about, so you can see the main things users bring to your agent without defining categories yourself. Amplitude reads each session's content, clusters similar sessions together, and gives each cluster an auto-generated label. Clustering runs periodically, so it fills in as sessions arrive (usually within a day or two) and keeps updating.

Amplitude discovers topics project-wide, building one shared topic map across every agent in the project. This means that the Topics tab doesn't filter by a single agent. Every session belongs to both a topic and an agent. To view one agent's topics, filter the Sessions tab by that agent and a topic. If you need fully separate topic maps per agent, instrument each agent into its own project.

The tab has two parts:

• Topic clusters (the map). A scatter plot where each point is a session, and sessions about similar things sit closer together, forming clusters. Each cluster is a labeled topic, with a ranked list alongside showing how large each one is. Zoom into a cluster to focus it, or click a point to open that session. It's a bird's-eye view of what people bring to your agent and how big each theme is.
• Scorecard. A per-topic breakdown of how many sessions fall in each topic and how they score on the quality signals, so you can see which themes are doing well and which are struggling.

Insights

The Insights tab on Monitor lists findings Amplitude generates about your agents' cost, efficiency, and quality, such as an agent getting more expensive or task completion dropping. Amplitude re-evaluates findings periodically; they aren't live queries. Each finding names the agent it applies to, or shows Project-wide.

Each finding has a severity: Severe, Moderate, Low, or No Issue. Severity is 100 × magnitude × confidence, and a trend icon shows whether the finding is trending up or down. Filter the list by severity, by ticket status, or by search text, and apply the Monitor agent filter to see one agent's findings. When an agent doesn't have enough sessions yet, the tab says so instead of showing findings.

Open a finding to see its description, its history over time, a recommendation, and sample sessions. From a finding you can:

• Pin it: Pin a finding to keep it at hand. Pinned findings sort to the top of the Insights tab and the Overview feed. Pins are personal, and they follow you across browsers and devices.
• Ask AI about it: Select Help Me Understand This Insight to start a Global Agent chat about the finding.
• Create a ticket: Create a Jira or Linear issue from the finding, or link an existing one. This requires a connected Jira or Linear workspace.
• Copy it: Copy the finding formatted for a ticket description or for a coding agent prompt, or copy a link to it.
• Build a dataset: Save the sample sessions as a dataset to run evaluators against them. The dataset doesn't update as the finding changes.
• Dismiss it: Dismiss a finding and say why, such as Not Relevant, Already Known, or Incorrect. Dismissing hides it for everyone in the project. To bring it back, show dismissed findings and restore it.

To read findings programmatically, use the List saved Agent Analytics insights and Get a saved Agent Analytics insight endpoints.

Costs

The Costs tab on Monitor breaks down LLM spend across the project. Access to the Costs tab depends on your permissions. If you don't have access, the tab doesn't appear, and Overview shows total LLM spend in place of the cost detail.

Explore sessions

Sessions is the second item in the Agent Analytics left navigation. It has two tabs: Sessions for working through individual sessions, and Datasets for collecting sessions into curated sets.

The Sessions tab lists every agent session with its signal results, cost, turns, and duration. Filter by agent, by any signal outcome (for example, sessions where Task Completed is false), by user, by time range, or by whether you've manually reviewed a session.

Opening a session shows the full transcript-level detail including every turn with its user message, tool calls with inputs and outputs, AI responses with model, tokens, cost, latency, and the session's signal results with their rationales. What's visible depends on your privacy mode. For example: in metadata_only mode, the structure and metrics are visible but not the content. Clicking the Info tab displays the signals. Clicking the Review tab on a session displays customer evaluators along with manual tags and comments.

As you inspect sessions, you can mark the ones you've reviewed. The Manually Reviewed filter can then narrow the list to reviewed or not-reviewed sessions. A reviewer can work through a backlog without re-reading sessions they've already confirmed. Reviewed sessions and the notes you leave on them feed the datasets you use to calibrate evaluators.

When your agent shows a user a chart, summary, table, or a set of choices, it can emit an artifact so the session view renders that same visual instead of raw tool output — useful when the answer was the picture and the JSON doesn't tell you whether it was any good. Refer to Render agent artifacts.

From a session you can jump to the user's other sessions, their Session Replay (when linked), or to their full event stream in User Lookup.

How quality is measured

Agent Analytics measures quality in three layers. All three are queryable in any Amplitude chart.

Signals: automatic, on every session

After a session closes, Amplitude scores it on seven built-in signals: Task Completed, Response Quality, Session Safety, User Intent, User Friction, Has Negative Feedback, and Data Quality Issues. Results land as properties on the session's [Agent] Session Record event, each with a rationale. For what each signal answers and its labels, refer to Agent Analytics built-in signals.

To trend a signal, chart [Agent] Session Record grouped by the signal property, for example the share of records where [Agent] Task Completed is true. To build a cohort, use the same property: users with three or more sessions where Task Completed is false in the last 30 days.

Evaluators: your criteria, calibrated

Custom evaluators are checks you define for criteria specific to your product. LLM evaluators judge each session against a prompt you write, and code evaluators apply deterministic pattern rules with no model calls. You calibrate an evaluator against your own reviews, then activate it, and it writes one [Agent] Evaluator Result event per closed session with the result, the judge model, the evaluator version, and the evaluation cost. Session dimensions such as agent, cost, turns, and topic ride along on each result, so you can segment outcomes directly.

To write, calibrate, and activate an evaluator, refer to Create and calibrate evaluators. Evaluator Result events count against your Amplitude event volume, unlike core agent events, which Amplitude meters as agent sessions.

Scores: the user's voice

Scores are feedback your application sends: a thumbs control, a CSAT survey, your own offline LLM judge, or human review. They arrive as [Agent] Score events targeting a message or a session, with an evaluation source of user, ai, or reviewer. Amplitude never generates Score events.

Two things make Scores powerful:

• A thumbs down sent as a user-feedback Score counts as explicit dissatisfaction for Negative Feedback.
• Scores share the project with everything else, so "sessions the model judged fine but the user thumbed down" is one chart: Session Records with Response Quality true, joined by session to Scores with value 0.

Datasets

A dataset is a curated set of up to 1,000 sessions with ground-truth labels that you use to calibrate and back-test evaluators before you activate them. The Datasets tab on the Sessions page lists your datasets and how far you've reviewed each one. Build a dataset in any of these ways:

• From the Sessions tab: Select sessions and add them to a new or existing dataset.
• From a suggested filter: Start from Has Negative Feedback, User Friction, Task Completed, or Has Tool Errors, optionally for one agent. Amplitude adds up to 50 recent matching sessions.
• From an insight: Save an insight's sample sessions. Refer to Insights.
• From a calibration run: Running an evaluator on your manually tagged sessions saves them as a calibration set.

Amplitude also keeps a built-in All Reviewed Sessions dataset.

Open a dataset to label ground truth on each session, add comments or tags, and track review progress. To work with the data outside Amplitude, export a dataset two ways:

• Summary export: the dataset's metadata, its sessions, and the review comments and tags on each session.
• Full export: session metadata, review comments and tags, traces, and enrichments for every session, matching a single-session JSON export. A full export can take a few minutes and produce a large file.

Datasets are the input to evaluator calibration: draft an evaluator, dry-run it against a dataset, and compare its output to your ground-truth labels. Refer to Evaluators for that workflow and Runs for the log of every scoring run.

Runs

The Runs tab on the Evaluators page is the log of every evaluator run. Each time an evaluator scores a set of sessions it creates a run, and this tab lists them so you can track scoring as it happens and review what has already been scored.

There are two kinds of runs: dry runs, which you use to test and calibrate an evaluator, and one-off runs, which score whatever sessions you pick. For each run, you can see its status, which evaluators and agents it used, how many sessions it covered, what it cost, and when it started. You can narrow the list down by agent, evaluator, type, or status, click into any run for more detail, or start a new one from the Start Run dialog. The Start Run modal allows you to choose the sessions, the date range, or the dataset to run the evaluator on.

Connect to product analytics

This is the point of putting agent data in Amplitude. Some patterns to start with:

• Funnels: First Agent Session to your conversion event. Or segment an existing signup funnel by whether the user touched the agent.
• Cohorts: Users with repeated task failures, for targeting with Guides or measuring churn impact. Users with high agent engagement, for expansion analysis.
• Retention: Retention curves segmented by [Agent] Task Completed on the user's first session.
• Session Replay: When linked at instrumentation time, jump from any agent session to the recording of the user's visit.
• Cost accountability: Session Cost USD on the Session Record, grouped by Topic Summary or agent, tells you where spend goes and whether those sessions succeed. Refer to how cost is calculated for where the numbers come from and why the provider bill remains the financial source of truth.

Agent events are ordinary Amplitude events. This means anything you can do with product events, you can do with them, including exporting them and querying them alongside everything else.