As you continue boosting your traffic to your website, you are likely to begin seeing statistically noteworthy results quicker. In case the process of getting the website setup and operating is still going on, be patient. Outcomes do take some time to get visible. It’s actually not unusual for many marketers to have the A/B tests running for a couple of months simultaneously.
When you’re fresh to the A/B testing, the outcomes could appear overpowering. Probably you are buried in lots of data, at a loss on how to go about interpreting the results of the test. The following easy and useful step-by-step plan will assist you in navigating the process.
At the time of setting up the individual A/B tests, we did request you to select precisely a single conversion metric that you would watch. Below are several cases of what perhaps you may have been keeping tabs on:
- Sign-ups for email list
- Sign-ups for free trial
- Product sign-ups
- E-commerce transactions
- eBook downloads
- Video plays
- Demo completions
- Revenue per visitor
- Shares on social media
- Blog post comments
This list could go on and on. Prioritize those conversion events which make your business grow.
Prior to analyzing the results, go back to the original study question you had set. Statistical significance implies that achieved outcomes could not have occurred by mere chance. There should be some justification or explanation for those trends which you are observing.
Once you have analyzed the results from the A/B test, then you should get a strong perception of what really happened. Your original hypothesis is the vital step towards understanding the ‘why’.
What behavior of the user to anticipate to monitor and even why do it? This is the crucial question that will assist you in translating the observed quantitative insights into some tangible considered and strategic best practices.
As an illustration, take a preview of the A/B testing client story gotten from Optimizely and the Backcountry. This is online retailing business selling outdoor gear, accessories and clothing.
Backcountry wished to optimize on its forthcoming shipping strategy targeting the holidays. They decided to test a sequence of deals on shipping during some other high-traffic seasons. Deals on shipping frequently aid in enticing buyers to transact that final buy.
The product team from Backcountry hypothesized that giving free, 2-day shipping during their yearly 4th of July sale could boost returns per visitor on their site.
What is the Reason for this?
The lowered shipping costs could motivate buyers to purchase more.
These are the variation and original Shopping carts:
Backcountry in the above study illustration did not openly share their results. Could you try and guess what the results could have been?
Keep in mind that the objective of the test was to find out which option on shipping would enhance revenue or returns per every website visitor.
For purposes of discussion, let us make assume that the option offering variation shipping won.
Maintain the Process Running
The A/B testing is a nonstop process and you should not stop having gotten the initial set of outcomes/results.