Amplitude Agent Analytics
Observe what users ask your AI agent, evaluate where it fails and what those cost your business. Agent Analytics is now available to all customers.
What it is
Description: Traditional analytics tells you a session happened. LLM observability can show you the trace. Neither immediately answers the question you care about: Did the agent give the user a good answer, and did that make them convert or churn?
Amplitude launched our specialized AI agents in 2025 and we faced the same issue. We built Agent Analytics for ourselves to close this gap. Each trace is scored for task success, failure type, and tool-call errors. Every conversation becomes agent behavior events you can query in the same funnels, cohorts, and retention charts you already use.
What it does: • Score every conversation. Inspect agent traces, take advantage of online evaluations such as task success and helpfulness, and filter quickly by behavioral cohorts. • Rank failing topics. Conversations are clustered by intent so you can see which topics fail most and fix the expensive ones first. • Connect agents to outcomes. View journeys and session replays inclusive of agent usage, compare different model providers on conversion, and find failure modes that cost you users.
Why: Most teams verify agents with offline evals, ship, and then have no way to see what the agent actually did with a real customer. Offline evals measure the model in a lab; Agent Analytics measures the behavioral trace downstream of the agent's decision, which is exactly the layer that's missing. We know how important it is to measure AI products so every Amplitude plan, including yours, gets a default allocation to use Agent Analytics. Go beyond observability. Get started today.
Where: amplitude.com/agent-analytics or check out the Agent Analytics product
Available in EUDC: Yes
When: Now! (Launched 8/20 for all customers)