Yieldify users can now set a default test significance at the website level in the Account Settings. The selected test significance will be applied to all tests that report on incremental performance for the selected website.
Additionally, users can see the date at which a change to website-level test significance was last made, and by whom.
What is test confidence?
Confidence (AKA statistical significance) is the likelihood that the conversion rate for the treatment group (website visitors who are shown a Yieldify experience) and the control group (website visitors who are not shown a Yieldify experience) displays a statistically significant difference. By setting your test significance to 95%, we can say with 95% certainty that the conversion rate uplift observed was directly caused by the Yieldify experience.
Read more about how Yieldify calculates confidence significance here.
Choosing the right test confidence for a website
Contact your CSM to calculate the right test significance for your website such that tests can reach that level of confidence within a reasonable time frame. They will need the following website-level metrics: ‘Monthly Sessions’, ‘Session Conversion Rate’, and ‘Monthly Sales’.
How to change a website’s default test significance
Navigate to your website’s Account Settings on the bottom left of your screen (gear icon)
Select the Reporting tab and scroll down to the Test Confidence card
Click on the drop-down menu to view the test significance levels available
Select the test significance level you want to set as default across all tests that report on incremental performance for this website
Click “Apply”
Considerations
All websites' test confidence will be set to 95% by default
The settings will be reflected in your website’s reporting on the 2nd of January 2023
Historical reporting for each website will be reflected in the selected significance