Can Agents Take On Enterprise Analytics?

This post was co-authored by Jacob Newman, Principal AI Product Manager at Amplitude.

Every vendor claims their analytics platform is “AI-powered.” But there’s a massive gap between auto-generated chart captions and intelligent systems that investigate your data, diagnose root causes, and recommend what to do next. Without clear benchmarks, buyers can’t separate real capability from noise.

We introduce a task-based evaluation for assessing AI analytics agents across four foundational question types: descriptive, diagnostic, predictive, and prescriptive.

Amplitude’s Global Agent achieves an overall score of 76% across all four tasks, which is a 7x+ improvement over six months of building and iterating. It excels at descriptive and diagnostic analytics with emerging capabilities in predictive and prescriptive analytics.

Figure 1. Global Agent’s overall success across all analytics tasks, showing consistent improvement over time. The baseline of 9% represents best-in-class warehouse Text-to-SQL performance on multi-query production-grade workflows.

By publishing our framework and scores, we establish a shared language for the industry, help teams evaluate solutions rigorously, and elevate customer expectations for what an AI analytics agent must do.

Defining AI analytics agents

AI analytics agents use statistical analysis and large language models to automatically surface insights from behavioral and business data. These agents are designed to interpret natural-language questions, query relevant data, apply the right analytical workflows, and reliably explain results and next steps.

A mature AI analytics agent will deliver true, meaningful insights to end users. We define an “insight” as a single data-driven observation that deepens understanding of users, products, campaigns, or processes and enables better decision making.

Eight-step agent workflow for responding to analytics tasks

Strong AI analytics agents should follow these eight steps when doing analytics tasks.

AI analytics tasks

A strong AI analytics agent must handle a wide spectrum of questions, from basic “What happened?” questions to sophisticated “Why did it happen?” and “What should we do?” investigations.

We define four key tasks reflecting the types of analytics questions an agent should be able to answer:

• Descriptive analytics: What happened?
• Diagnostic analytics: Why did it happen?
• Predictive analytics: What will happen?
• Prescriptive analytics: What should we do?

Figure 2. The four AI analytics tasks by task complexity

These tasks define the distinct categories of manual analytics work that AI agents should reduce, in order of increasing complexity. An agent that only answers descriptive questions still forces teams to manually investigate causes, evaluate impact, and determine next steps, shifting the hardest analytical work back to users.

For each task, we define representative questions in business language and the characteristics of a good AI answer. This grounds the framework in how real analysts work while remaining intuitive for product, marketing, and data leaders evaluating AI analytics platforms.

Descriptive analytics task

At a minimum, an AI analytics agent should be able to answer “What happened?” by describing user behavior and business performance across metrics, time periods, and segments. This task determines whether an agent can extract insights from data and convey them in plain language.

This task validates whether an AI analytics agent understands the underlying data, taxonomy, and metric definitions, and can translate natural language questions into accurate analyses. If an agent struggles with descriptive analytics, it does not yet have a robust system.

Diagnostic analytics task

An AI analytics agent should also be able to answer “Why did this happen?” by doing multi-step analysis, exploring multiple hypotheses, and identifying plausible root causes. This task evaluates whether an agent can move beyond description to provide explanation, without requiring users to manually investigate.

This task validates whether an AI analytics agent can systematically investigate causes, rule out false signals, and return evidence-backed root causes. If the agent can describe what happened but not explain why, it is not true AI analysis.

Predictive analytics task

An AI analytics agent should be able to answer “What will happen?” by forecasting future outcomes based on historical patterns, behavioral signals, and current trends. This task evaluates whether an agent can generate credible predictions grounded in data rather than speculation.

True predictive capability is more than a churn score. An AI analytics agent should anticipate future outcomes, communicate prediction drivers, provide confidence ranges, and adapt as conditions change.

Prescriptive analytics task

At the highest level of maturity, an AI analytics agent should be able to answer “What should we do?” by recommending actions informed by descriptive, diagnostic, and predictive insights. This task evaluates whether an agent can translate analysis into actionable next steps.

Prescriptive analytics capabilities guide decision making and help teams determine next steps by tying recommendations directly to historical user behavior and predicted outcomes. Look for AI analytics agents that can propose data-backed actions, articulate expected impact, and communicate potential risks or tradeoffs.

Evaluating Amplitude’s Global Agent on AI analytics tasks

To set an industry baseline, we scored Amplitude’s Global Agent on all four tasks by asking representative questions across real analytics workflows. We evaluated each response using an LLM-as-a-judge approach against human-defined criteria. The scores represent the percent of responses that successfully captured the desired insight.

Figure 3. Global Agent’s success by task, highlighting its strength in descriptive and diagnostic analytics tasks, with emerging capabilities in predictive and prescriptive tasks

Overall, we would expect most AI analytics agents to do reasonably well at descriptive analytics tasks, with sharp drop-offs for diagnostic ones. Global Agent demonstrates strength in both, reliably explaining what happened and why through consistent diagnostic reasoning. Other AI analytics agents are unable to meet this diagnostic bar, let alone provide predictive or prescriptive offerings. Global Agent shows emerging capabilities in both areas.

Score summary

Descriptive analytics score

Score: 80%

Global Agent scores highest in descriptive analytics. The agent reliably answers “What happened?” across user behavior, product usage, and revenue growth. This includes accurately selecting metrics and taxonomy, applying time ranges and comparisons, and running the correct analyses.

Global Agent reduces manual analytical work by ensuring descriptive questions are answered clearly and accurately without requiring users to configure charts.

Diagnostic analytics score

Score: 76%

Global Agent scores highly at the diagnostic level. The agent can investigate why changes occur by exploring multiple hypotheses and returning evidence-backed root cause explanations.

This task reflects Global Agent’s strength in multi-step, multi-dimensional analysis and behavioral reasoning. This is an area where many AI analytics tools still rely on manual investigation or surface-level explanations.

Predictive analytics score

Score: 63%

Global Agent’s advanced predictive capabilities are continuing to evolve. The agent can anticipate future outcomes for select scenarios in a way that many AI analytics tools or manual analyses cannot. But predictive analytics remains an area of active investment and refinement.

For this task, our focus is on building forecasts that are credible, grounded in meaningful signals, and transparent about uncertainty.

Prescriptive analytics score

Score: 50%

Prescriptive analytics represents the frontier of AI analytics. Global Agent can propose data-backed next steps in many contexts, and even take some actions itself. But we do not treat our prescriptive capability as “solved.”

Our approach emphasizes improved decision making and action by tying recommendations back to observed behavior, expected impact, and tradeoffs. This task reflects where the industry is heading and where Amplitude is actively building.

AI analytics evaluation in practice

This evaluation defines what “good” looks like across descriptive, diagnostic, predictive, and prescriptive analytics tasks. To make this concrete, we provide example questions and Global Agent’s responses below.

Together, these examples illustrate how AI analytics agents should be evaluated and what a high-scoring AI-generated insight looks like.

Evaluations: Example questions and high-scoring responses

The table below presents a set of representative questions spanning user behavior, product usage, funnel conversion, and marketing performance, and Global Agent’s response to those questions.

For clarity, we’ve summarized Global Agent’s non-deterministic responses in the table, with the full analyses and charts linked throughout.

Anatomy of a high-scoring AI insight

While the evaluation cases show what questions an AI analytics agent should be able to answer, the visual below illustrates what a strong AI-generated response should look like.

We break down a single Global Agent example, from question to output to follow up, and highlight the essential components that distinguish a “gold-standard” insight from a surface-level response. The fully annotated diagram is available here.

Figure 4. Preview of Global Agent’s response

Setting the standard for best-in-class AI analytics agents

As product and marketing data grow more complex, the value of AI analytics agents comes from the ability to reduce manual analytical work, surface meaningful insights, and support better decision making and action throughout the business.

This evaluation offers a clear, task-based framework to assess whether an AI analytics agent meets that bar by defining what “good” looks like across analytics tasks.

Using this framework to evaluate Amplitude’s Global Agent highlights our ability to reliably answer what happened and why through accurate data interpretation, multi-step reasoning, and evidence-backed explanations. The evaluation also surfaces Global Agent’s emerging capabilities in predictive and prescriptive analytics, where we are actively building today.

For buyers navigating a crowded market, this evaluation offers a practical framework to guide your decision and a clear example of what a mature AI analytics agent should be able to do. Amplitude best-in-class AI analytics platform delivers analytical rigor and enables teams to move faster and act on insights with confidence.

As AI analytics continues to evolve, the gap between surface-level automation and true analytical intelligence will only widen. We are committed to leading the category forward and invite people to share their thoughts with us on X.

Learn more about how Amplitude AI Agents are changing the way teams build products in our webinar.

Acknowledgements

This evaluation is the result of close cross-functional collaboration and domain expertise from engineering, product, and design. Thank you to the following individuals for their contributions: Adam Lohner, Adam Struck, Amogh Dikshit, Andrew Chou, Ashkan Aghdai, Avery Chan, Ben Feigin, Brian Cooper, Brian Giori, Chamila Amithirigala, Chris Yu, Eric Carlson, Eric Kim, Frank Lee, Henry Arbolaez, Jacob Newman, Jake McKibben, James Evans, Janaki Vivrekar, Jin Wan, Jingshu Zhao, Jimmy Wilson, Karan Johar, Kevin Wu, Larry Xu, Marvin Liu, Nhi Dao, Nirmal Utwani, Peyton Rose, Pranjal Joshi, Ram Soma, Richard Fang, Ryan Busk, Shawn McKay, Skylar Givens, Steven Cheng, Tommy Keeley, Tyler Wanlass, Vinay Ayyala, Vinay Goel, Will Newton