Feature and Web Experiment Functional Comparison

The following tables describe which Experiment functionality is available for Feature experimentation, Web experimentation, or both. For more information on the differences and use cases between Feature and Web Experiment, go to Feature and Web Experiment Use Cases.

Planning


Functionality	Feature	Web
Client-side or Server-side implementation	✅	❌
Local or remote evaluation	✅	✅
Stats Method
Sequential	✅	✅
T-test	✅	✅
Thompson Sampling (MAB)	✅	✅
Reduce chance of Error
Bonferroni Correction	✅	✅
Group Experiments
Mutual Exclusion	✅	❌
Holdouts	✅	❌
% of audience to rollout	✅	❌
Exposure Event Type
Exposure Event	✅	✅
Custom Exposure	✅	❌
Bucketing Salt
Bucketing salt	✅	✅

Creating experiments


Functionality	Feature	Web
Name	✅	✅
Project	✅	✅
Template	✅	❌
Recommended Settings	✅	✅
Link	✅	✅
Tag	✅	✅
Variants	✅	✅
Payload	✅	❌
When to Run
When to start	✅	✅
When to end	✅	✅
Traffic estimate, Control mean estimator, Power duration estimator for each day	✅	❌
OOTB web editor widgets	❌	✅
Project-level user permissions	✅	✅
Notifications through Slack channel or webhook	✅	✅

Experiment types


Functionality	Feature	Web
A/B	✅	✅
Multi-Armed Bandit	✅	✅

Goals


Functionality	Feature	Web
Primary Metric	✅	✅
Secondary Metric	✅	✅
Guardrail vs success	✅	✅
Direction	✅	✅
Minimum Detectable Effect	✅	✅
Winsorization	✅	✅
CUPED	✅	✅
Attribution	✅	✅
Window	✅	✅

Targets


Functionality	Feature	Web
User or Behavioral Cohort	✅	✅
Stratified Sampling	✅	✅
Rollout	✅	✅
Sticky Bucketing	✅	❌


Functionality	Feature	Web
Deployment	✅	✅
Dependencies	✅	❌
Testers	✅	❌

Analysis


Functionality	Feature	Web
Assignment and Exposure charts	✅	❌
Variant Jumping	✅	✅
Anonymous Exposures	✅	✅
Exposures without Assignments	✅	❌
Rollout options	✅	✅