Anomaly + Forecast: Find anomalies in your data

When core metrics fluctuate, you need to know whether the changes are meaningful or random noise. Amplitude's Anomaly + Forecast feature highlights statistically significant deviations from expected metric values, based on historical data.

This can help you:

• Determine whether a change is truly meaningful
• Catch instrumentation errors
• Study seasonal trends
• Monitor the impact of product releases

Forecast projects metrics into the future, which helps you set realistic goals for your team and product.

Before you begin

Anomaly detection and forecast work with time-series data within the following Amplitude charts: Event Segmentation, Conversion over Time (when counting by unique users for the entire funnel), User Sessions, Retention over Time, and Stickiness over Time.

Within Event Segmentation, Anomaly + Forecast works with rolling windows, rolling averages, growth percentage, and custom formulas that support time-series analyses.

Anomaly detection isn't compatible with cumulative time series charts, or charts comparing two different time periods.

If you're using an hourly interval, Anomaly + Forecast supports a maximum of one group-by.

Set up anomaly detection

Anomaly + Forecast uses an anomaly detection technique based on the extensively tested open source tool Prophet. Prophet forecasts time-series data and handles missing data points, trend shifts, and large outliers.

You can find the control for this feature on the left hand side, right above the main chart area. If your chart isn't supported by the Anomaly + Forecast feature, the button isn't clickable.

To set up anomaly detection, follow these steps:

1. Click Anomaly + Forecast to enable the feature. The button turns orange when you engage the feature.

2. Click the drop-down arrow to the right of the Anomaly + Forecast button.

3. Select the mode. Your options are agile, robust, and custom. Agile mode adjusts more quickly to recent trends, using a 95% confidence interval and 120 days of training data before the chart's date range. Robust mode works best for stable metrics because it uses a full year of extra training data and better accounts for seasonality. Custom lets you change the confidence interval and training duration. Higher significance levels usually result in fewer anomalies on the chart.

4. Add a forecast, if you prefer. Forecast projects your metrics into the future, while anomalies apply only to historical data. To add a forecast, enter the number of months for the forecast in the Forecast Period field.
5. Click Apply to begin detecting anomalies.

Set up automated project alerts

You can request an emailed digest of all anomalies in a project. These digests are automated, and you can set them up for multiple projects. After you switch them on, the digests continue until you turn them off.

To set up automated alerts for anomalies in your projects, navigate to Organization settings > Projects, find the project you're interested in, and click it. Then click the Automatic Alerts tab and flip the toggle switch to subscribe to project alerts.

When alerts are active, Amplitude displays a table on this page that lists all alerts automatically generated by the project. Click an alert to open the chart where the alert occurred and view the alert in context.

Interpret your results

Charts with a single series display a light blue band (the confidence interval) and a dashed line representing the expected value beside the solid blue line for your actual data. Detected anomalies appear in orange outside the confidence band. You can describe an anomaly this way: "Based on 120 days of training data, this data point represents an unexpected change with 95% confidence."

If no orange dots are present, all data points are within the confidence interval.

You can run Anomaly + Forecast with multiple series on a chart. Hover over each series to view its confidence band (these appear in different colors as well).

For forecasts, you see solid lines representing actual data and the confidence intervals until the current date. After the current date, the forecast displays only dashed lines representing anticipated future values. Prophet projects metrics by assuming the size and frequency of past changes are similar in the future, with a certain degree of confidence.

You can describe forecast results this way: "Based on the trend from the last 120 days of data, we're 95% confident that this metric is between [high value] and [low value] on [a future date]."

Determine anomaly causes

Identifying an anomaly is only the first step. You probably want to know what caused it in the first place.

Start by looking at a few related metrics to see if you observe anomalies on those as well. In particular, look at the events that fire before or after the step in the funnel. You could also use group-bys on any properties that might yield more insights into why these anomalies occurred.

A third option is to examine the business context surrounding the anomaly. For example, did a new feature ship that day? Could that have been the cause?

Finally, Amplitude's Root Cause Analysis feature is a powerful tool for tracking down the causes of anomalies.

Training data

Amplitude uses different default training durations for different time intervals and modes. In custom mode, you can configure this and add it to the chart date range.

For example, using a daily interval and looking at the last 30 days of data on the chart, the default training data duration for daily charts is 120 days before the chart date range starts. In this case, Amplitude uses a total of 150 days of data to train the model.

In agile mode, Amplitude uses the following default data training durations:

In robust mode, Amplitude uses the following default data training durations:

The upper limit of training data (prior periods + chart duration) used for each interval is as follows:

If you have a specific training duration in mind that agile or robust modes don't offer, you can set that duration by choosing the custom mode.

Insights package

If you need alerting for anomalies, Amplitude's Insights package includes automatic and custom monitor alerts. Refer to Insights for more information.