Introduction
An incremental test is one of the simplest ways to measure the efficacy of your campaign. By running an incremental test, you expose a portion of your audience (traditionally 50%) to the creative and maintain a control group that does not see anything.
These types of tests are valuable to measure the impact of long term, core strategies. They are not as helpful when run for a short period of time, or where the goal is content engagement (clicks, lead capture).
The primary metrics we measure in the platform are conversion rate uplift (which dictates the overall success and confidence, this is not determined by revenue), AOV uplift and revenue uplift.
Key Metrics Used in Calculations
Sessions - The number of sessions where a user successfully triggered a campaign event. If a user triggers a campaign multiple times in the same session, this will still only count as one session.
Sales - The number of sessions in which a user triggers a campaign event and completes a sale. This does not include any sales made in future sessions.
Revenue - The amount of revenue generated from sales attributed to a campaign.
Clicks - Any time a user clicks on any CTA on a campaign. A CTA must be selected as Count Towards CTR to be tracked as a click.
Leads - Any event in which a user fills out a form and submits the information via a Submit CTA. This includes both campaigns that request an Email/SMS or survey campaigns.
Closes - Any time a user clicks on a campaign close button. This does not include Dismiss CTAs (a click) or if someone clicks outside of an overlay.
Key Formulae
Conversion Rate = Sales / Sessions
AOV = Revenue / Sales
Revenue per Session = Revenue / Sessions
Conversion Rate Uplift = (Treatment CR / Control CR) - 1
AOV Uplift = (Treatment Average Order Value / Control Average Order Value) - 1
Revenue per Session Uplift = (Treatment RPS / Control RPS) - 1