Analyze agent results

Once events are flowing, Agent Analytics gives you three ways in: dashboards for the big picture, sessions for individual diagnosis, and evaluators for measuring the criteria you define. Everything also works in standard Amplitude charts, funnels, and cohorts.

Monitor

Open Monitor from the Agent Analytics left navigation. Monitor summarizes agent health across the selected project and time range. The page charts agent activity and session volume, session quality from the always-on signals, and cost and latency trends. Use these charts to spot a shift before you open individual sessions, and apply the agent filter to focus on one agent or compare agents.

Insights

The Insights tab on Monitor lists findings Amplitude generates about your agents' cost, efficiency, and quality, such as an agent getting more expensive or task completion dropping. Amplitude re-evaluates findings periodically; they aren't live queries. Each finding names the agent it applies to, or shows Project-wide.

Each finding has a severity: Severe, Moderate, Low, or No Issue. Severity is 100 × magnitude × confidence, and a trend icon shows whether the finding is trending up or down. Filter the list by severity, by ticket status, or by search text, and apply the Monitor agent filter to see one agent's findings. When an agent doesn't have enough sessions yet, the tab says so instead of showing findings.

Open a finding to see its description, its history over time, a recommendation, and sample sessions. From a finding you can:

• Pin it: Pin a finding to keep it at hand. Pins are personal and stay in your browser.
• Ask AI about it: Select Help Me Understand This Insight to start a Global Agent chat about the finding.
• Create a ticket: Create a Jira or Linear issue from the finding, or link an existing one. This requires a connected Jira or Linear workspace.
• Copy it: Copy the finding formatted for a ticket description or for a coding agent prompt, or copy a link to it.
• Build a dataset: Save the sample sessions as a dataset to run evaluators against them. The dataset doesn't update as the finding changes.
• Dismiss it: Dismiss a finding and say why, such as Not Relevant, Already Known, or Incorrect. Dismissing hides it for everyone in the project. To bring it back, show dismissed findings and restore it.

To read findings programmatically, use the List saved Agent Analytics insights and Get a saved Agent Analytics insight endpoints.

Explore sessions

The Sessions view lists every agent session with its signal results, cost, turns, and duration. Filter by agent, by any signal outcome (for example, sessions where Task Completed is false), by user, by time range, or by whether you've manually reviewed a session.

Opening a session shows the full transcript-level detail including every turn with its user message, tool calls with inputs and outputs, AI responses with model, tokens, cost, latency, and the session's signal results with their rationales. What's visible depends on your privacy mode. For example: in metadata_only mode, the structure and metrics are visible but not the content. Clicking the Info tab displays the signals. Clicking the Review tab on a session displays customer evaluators along with manual tags and comments.

As you inspect sessions, you can mark the ones you've reviewed. The Manually Reviewed filter can then narrow the list to reviewed or not-reviewed sessions. A reviewer can work through a backlog without re-reading sessions they've already confirmed. Reviewed sessions and the notes you leave on them feed the datasets you use to calibrate evaluators.

When your agent shows a user a chart, summary, table, or a set of choices, it can emit an artifact so the session view renders that same visual instead of raw tool output — useful when the answer was the picture and the JSON doesn't tell you whether it was any good. Refer to Render agent artifacts.

From a session you can jump to the user's other sessions, their Session Replay (when linked), or to their full event stream in User Lookup.

Topics

The Topics page groups your closed sessions by what they were about, so you can see the main things users bring to your agent without defining categories yourself. Amplitude reads each session's content, clusters similar sessions together, and gives each cluster an auto-generated label. Clustering runs periodically, so it fills in as sessions arrive (usually within a day or two) and keeps updating.

Amplitude discovers topics project-wide, building one shared topic map across every agent in the project. This means that the Topics page doesn't filter by a single agent. Every session belongs to both a topic and an agent. To view one agent's topics, filter the Sessions view by that agent and a topic. If you need fully separate topic maps per agent, instrument each agent into its own project.

The page has two parts:

• Topic clusters (the map). A scatter plot where each point is a session, and sessions about similar things sit closer together, forming clusters. Each cluster is a labeled topic, with a ranked list alongside showing how large each one is. Zoom into a cluster to focus it, or click a point to open that session. It's a bird's-eye view of what people bring to your agent and how big each theme is.
• Scorecard. A per-topic breakdown of how many sessions fall in each topic and how they score on the quality signals, so you can see which themes are doing well and which are struggling.

How quality is measured

Agent Analytics measures quality in three layers. All three are queryable in any Amplitude chart.

Signals: automatic, on every session

After a session closes, Amplitude scores it on seven built-in signals: Task Completed, Response Quality, Session Safety, User Intent, User Friction, Has Negative Feedback, and Data Quality Issues. Results land as properties on the session's [Agent] Session Record event, each with a rationale. For what each signal answers and its labels, refer to Agent Analytics built-in signals.

To trend a signal, chart [Agent] Session Record grouped by the signal property, for example the share of records where [Agent] Task Completed is true. To build a cohort, use the same property: users with three or more sessions where Task Completed is false in the last 30 days.

Evaluators: your criteria, calibrated

Custom evaluators are checks you define for criteria specific to your product. LLM evaluators judge each session against a prompt you write, and code evaluators apply deterministic pattern rules with no model calls. You calibrate an evaluator against your own reviews, then activate it, and it writes one [Agent] Evaluator Result event per closed session with the result, the judge model, the evaluator version, and the evaluation cost. Session dimensions such as agent, cost, turns, and topic ride along on each result, so you can segment outcomes directly.

To write, calibrate, and activate an evaluator, refer to Create and calibrate evaluators. Evaluator Result events count against your Amplitude event volume, unlike core agent events, which Amplitude meters as agent sessions.

Scores: the user's voice

Scores are feedback your application sends: a thumbs control, a CSAT survey, your own offline LLM judge, or human review. They arrive as [Agent] Score events targeting a message or a session, with an evaluation source of user, ai, or reviewer. Amplitude never generates Score events.

Two things make Scores powerful:

• A thumbs down sent as a user-feedback Score counts as explicit dissatisfaction for Negative Feedback.
• Scores share the project with everything else, so "sessions the model judged fine but the user thumbed down" is one chart: Session Records with Response Quality true, joined by session to Scores with value 0.

Datasets

A dataset is a curated set of up to 1,000 sessions with ground-truth labels that you use to calibrate and back-test evaluators before you activate them. The Datasets page in the left navigation lists your datasets and how far you've reviewed each one. Build a dataset in any of these ways:

• From the Sessions view: Select sessions and add them to a new or existing dataset.
• From a suggested filter: Start from Has Negative Feedback, User Friction, Task Completed, or Has Tool Errors, optionally for one agent. Amplitude adds up to 50 recent matching sessions.
• From an insight: Save an insight's sample sessions. Refer to Insights.
• From a calibration run: Running an evaluator on your manually tagged sessions saves them as a calibration set.

Amplitude also keeps a built-in All Reviewed Sessions dataset.

Open a dataset to label ground truth on each session, add comments or tags, and track review progress. To work with the data outside Amplitude, export a dataset two ways:

• Summary export: the dataset's metadata, its sessions, and the review comments and tags on each session.
• Full export: session metadata, review comments and tags, traces, and enrichments for every session, matching a single-session JSON export. A full export can take a few minutes and produce a large file.

Datasets are the input to evaluator calibration: draft an evaluator, dry-run it against a dataset, and compare its output to your ground-truth labels. Refer to Evaluators for that workflow and Runs for the log of every scoring run.

Runs

The Runs page is the log of every evaluator run. Each time an evaluator scores a set of sessions it creates a run, and this page lists them so you can track scoring as it happens and review what has already been scored.

There are two kinds of runs: dry runs, which you use to test and calibrate an evaluator, and one-off runs, which score whatever sessions you pick. For each run, you can see its status, which evaluators and agents it used, how many sessions it covered, what it cost, and when it started. You can narrow the list down by agent, evaluator, type, or status, click into any run for more detail, or start a new one from the Start Run dialog. The Start Run modal allows you to choose the sessions, the date range, or the dataset to run the evaluator on.

Connect to product analytics

This is the point of putting agent data in Amplitude. Some patterns to start with:

• Funnels: First Agent Session to your conversion event. Or segment an existing signup funnel by whether the user touched the agent.
• Cohorts: Users with repeated task failures, for targeting with Guides or measuring churn impact. Users with high agent engagement, for expansion analysis.
• Retention: Retention curves segmented by [Agent] Task Completed on the user's first session.
• Session Replay: When linked at instrumentation time, jump from any agent session to the recording of the user's visit.
• Cost accountability: Session Cost USD on the Session Record, grouped by Topic Summary or agent, tells you where spend goes and whether those sessions succeed. Refer to how cost is calculated for where the numbers come from and why the provider bill remains the financial source of truth.

Agent events are ordinary Amplitude events. This means anything you can do with product events, you can do with them, including exporting them and querying them alongside everything else.