Confidence is the likelihood that the conversion rate for the treatment group and the control group display a significant difference. Whilst confidence can be used to measure this significance for any binary metric, we only apply our formula to conversion rate (i.e. sales, not revenue).

We use a frequentist model to evaluate confidence. Frequentist represents the statistical likelihood of two groups being different, based on repetitive testing.

It is important to note that confidence does not equal uplift. If a campaign displays high confidence, it means that we have high certainty that the conversion rates between our test and control groups are different. In order to be certain that this difference is stable and consistent, Yieldify applies confidence validation.

You can find the confidence level for any of your tests by clicking on the performance tab of a campaign in the YCP platform.

Any campaign is marked as statistically significant at 95% confidence and after 100 sales have been captured in the control group. Whilst arbitrary, this is standard practice and ensures that there's only a 1-in-20 chance that the result does not coincide with the conversion rate uplift shown.

For certain websites a lower level of statistical significance may be appropriate for your tests. Please contact your CSM to determine if a different significance level is right for your website.

Key Formulae