Ultimate Guide to A/B Testing
by Wee Sin Tan
1. Things to test
1.1. Common elements that are tested are: - The call to action’s (i.e. the button’s) wording, size, color and placement, - Headline or product description, - Form’s length and types of fields, - Layout and style of website, - Product pricing and promotional offers, - Images on landing and product pages, - Amount of text on the page (short vs. long).
2. Tools for A/B Testing
2.1. - Google Website Optimizer - A/Bingo and Vanity - Visual Website Optimizer - Unbounce and Performable - Vertster, SiteSpect, Webtrends Optimize and Omniture’s Test&Target
3. What is it?
3.1. Two versions of an element and a metric that defines success.
3.2. Which is better? Subject both versions to experimentation simultaneously. Which version was more successful and select that version.
4. Do's and Don'ts
4.1. Don'ts
4.1.1. - Don’t let your gut feeling overrule test results
4.1.2. - Don’t surprise regular visitors
4.1.3. - When doing A/B testing, never ever wait to test the variation until after you’ve tested the control
4.1.4. - Don’t conclude too early.
4.2. Do's
4.2.1. - Do many A/B tests
4.2.2. - Make your A/B test consistent across the whole website
4.2.3. - Show repeat visitors the same variations
4.2.4. - Know how long to run a test before giving up