A/B TESTING

1. Do's and Don'ts

1.1. Don'ts: - Don't wait to test the variation until after you have tested the control. Always test both version simultaneously. - Don't conclude too early. This is to prevents you from reading too much into the results if you have only a few conversion or visitors during each variation. - Don't surprise regular visitors. If you are testing a core part of the website, include only new visitors to avoid shocking regular visitors. - Don't let your gut feeling overrule test results. The winners in A/B Tests are often surprising or unintuitive.

1.2. Do's: - Know how long to run a test before giving up because giving up too early can cost you because you may have gotten meaningful results if you waited a little longer. - Show repeat visitors the same variations to prevent blunders such as showing a user a different price or a different promotional offer. - Make your A/B test consistent across the whole website as showing one variation on page 1 and another variation on page 2 will skew the results. - Do as many A/B tests as an A/B tests can have only three outcomes: no result, a negative results and a positive result. The key to optimize conversion rates is to do a ton of A/B tests

2. Pros and Cons of A/B Testing

2.1. Prosof website A/B Testing: - Fast; out of all types of test, A/B is way faster because it takes very little time to create a modified version of an existing web page that includes a modified item and throw it up on your site. - Test reality, not theory; A/B testing on a live web site is that you will obtain real results from real users doing real things. Meaning you’re not using theory, estimates etc to base decisions. - Quantifiable; A/B testing provides actual numbers to be compared, sliced and diced to evaluate results. - Accurate; A/B Testing is 100% accurate assuming you have statistically significant data.

2.2. Cons of web site A/B Testing: - Can hurt web site results; in A/B testing what you thought was an excellent B test item in an A/B Test may turn out terribly wrong. When it happens, you’re going to end up hurting your overall web site results. - Missing the “why”; A/B web site testing does not explore the rationale behavioural decisions that are being made by the web site visitors. - Not predictive; it can’t be used to predict future design change impacts. - Needs traffic; in order to provide quick, consistent and reliable results, you will need a pretty good amount of traffic to your web page to run an A/B testing.

3. What is A/B Testing?

3.1. A/B testing is to determine which version of elements is better through experimentation. At the end, you measure which version is more successful and use it for real-world usage.

4. What is tested in A/B testing?

4.1. The choice of what to test is usually depended on your goals. E.g If the goal is to increase the number of sign-ups, then we will test on length of the sign-up form, types of fields.

4.1.1. Elements that are usually tested: - The call to action’s (i.e. the button’s) wording, size, color and placement, - Headline or product description, - Form’s length and types of fields, - Layout and style of website, - Product pricing and promotional offers, - Images on landing and product pages, - Amount of text on the page (short vs. long).

5. How to create A/B Test?

5.1. Two ways to set up an A/B test

5.1.1. 1. Replace element to be tested before the page loads; if we are testing a single element on a web-page, the A/B tool will randomly replace the original element on the page with one of the variations before display the page to the visitor

5.1.2. 2. Redirect to another page; if we are testing the entire page, then we need to create and upload a new page for the website