What Is Incrementality Testing for Modern Marketers

Incrementality testing is a controlled experiment that measures the true causal impact of marketing campaigns. It compares outcomes between two groups: one exposed to your campaign (treatment group) and one that isn’t (control group).

The control group shows what happens without your marketing, while the treatment group shows what happens with it. The difference between these groups is your incremental lift—the results actually caused by your campaign.

Here’s what each term means:

• Lift: Additional outcomes caused by marketing activity above baseline behavior
• Control group: Audience withheld from the marketing campaign
• Treatment group: Audience exposed to the campaign being tested
• Causal effect: Changes in outcomes directly attributable to the campaign

Traditional , like last-click, often credit campaigns for conversions that would have happened anyway. Incrementality testing reveals which results your campaign actually created versus which it just touched along the way.

Most marketing platforms overstate their impact. , , and other channels report conversions that include people who would have converted without seeing your ads.

Privacy changes make this problem worse. iOS updates limit tracking through apps, while browsers restrict third-party cookies. These changes create gaps in attribution data, making platform reports less reliable.

Marketing incrementality testing solves this by measuring actual cause and effect:

• True impact measurement: Proves campaigns cause specific outcomes rather than just correlating with them
• Budget optimization: Identifies which channels create new customers versus capture existing demand
• Future-proof approach: Works without user-level tracking or cookies

For example, a retailer might see high conversions from branded search ads. However, an incrementality test could reveal that 80% of those conversions would have happened through organic search anyway. The real incremental value is just 20%—dramatically different from what the platform reports.

Incrementality measurement compares outcomes between test groups using a simple formula. The calculation shows how much your campaign actually moved the needle above baseline behavior.

Incrementality calculation formula

The basic formula is: (Treatment Group Results - Control Group Results) / Control Group Results.

This gives you the percentage lift caused by your campaign. For example, if your treatment group had a 5% conversion rate and your control group had a 4% conversion rate, your incremental lift is 25%.

You can apply this formula to any metric:

• (CLTV)

Reading incremental revenue in analytics platforms

Platforms like Amplitude display incrementality results across multiple dimensions. You can see the lift broken down by audience segment, geography, time period, and campaign element.

The key metrics to track include:

• Absolute lift: Raw difference in outcomes between groups
• Percentage lift: Relative improvement over baseline
• : Confidence that results aren't due to chance
• Net incremental revenue: Total additional revenue caused by the campaign

Amplitude's experimentation features calculate these automatically and show to help you understand the reliability of your results.

Different marketing channels benefit from incrementality testing in specific ways. The approach works best for channels where the relationship between exposure and conversion isn't immediately apparent.

Paid social and display campaigns

Social media and display ads often influence customers over time rather than driving immediate clicks. Traditional attribution misses this delayed impact.

Incrementality tests for these channels typically measure:

• Brand awareness lift through surveys or branded search volume
• changes over 30–90 day windows
• for truly new customers

Search ads and shopping feeds

Search campaigns capture demand that might exist organically. An incrementality test reveals how much additional traffic and revenue your ads actually create.

Common tests include:

• Pausing ads in specific geographic regions
• Testing different keyword match types
• Comparing bid levels to measure true incremental ROAS

In-product feature rollouts with holdout groups

Product teams use incrementality testing to measure feature impact. A holdout group continues using the old experience while the test group gets the new feature.

This approach measures effects on:

• and session depth
• Conversion through key funnels
• Long-term retention and customer lifetime value

Running effective incrementality tests requires careful planning and execution. These actions help avoid common mistakes that can invalidate your results.

1. Right-size your groups for statistical power

Your test groups need enough people to detect meaningful differences. Too small, and you won’t see significant results even if your campaign works. Too large, and you’re withholding campaign exposure unnecessarily.

Use your baseline conversion rate and expected lift to calculate . Most tests need at least 1,000 people per group to detect a 10% lift confidently.

2. Isolate one variable per test

Test one campaign, creative, or channel at a time. Multiple changes make it impossible to know what caused any differences you observe.

For example, don't launch new creative and increase budgets simultaneously. Run separate tests for each change so you can measure their individual impact.

3. Control for seasonality and external factors

Holidays, competitor actions, and market changes can skew results. Run tests during stable periods or ensure both groups experience the same external conditions.

Account for factors like:

• Seasonal shopping patterns
• Competitor promotional campaigns
• Economic news that might affect buying behavior
• Product launches or PR events

4. Connect lift to long-term customer value

Immediate conversions don't tell the whole story. Track how incremental customers behave over time to understand true campaign value.

Use to measure:

• Repeat purchase rates for incrementally acquired customers
• Customer lifetime value differences between the test and control
• Retention patterns over 90+ day periods

5. Automate reporting with integrated platforms

Manual analysis introduces errors and delays insights. Platforms like Amplitude automate incrementality calculations and provide real-time results as your test runs.

Integrated tools maintain consistent user assignment, track exposure accurately, and calculate automatically.

The most effective approach to incrementality testing integrates measurement with your existing analytics stack. This eliminates data silos between campaign management and results analysis.

Point solutions require manual data exports and custom analysis for each test. A like Amplitude connects with experimentation features, so you can measure incrementality using the same data that powers your other analytics.

With unified analytics and experimentation, you can run continuous incrementality tests across channels and campaigns. Results feed directly into budget allocation decisions without waiting for separate analysis cycles.

This approach makes marketing incrementality testing a standard part of campaign optimization rather than a special project. Teams can quickly identify what's working, scale effective tactics, and eliminate wasteful spending.