A/B testing

1. Pros of Web Site A/B Testing

1.1. Fast

1.1.1. it takes very little time to create a modified version of an existing web page

1.2. Tests reality, not theory

1.2.1. live web site is you’re obtaining real results from real users doing real things

1.3. Quantifiable

1.3.1. provides actual numbers that can be compared, sliced and diced to evaluate results

1.4. Accurate

1.4.1. A/B testing is 100% accurate if you have statistically significant data

2. Cons of Web Site A/B Testing

2.1. Can Hurt Web Site Results

2.1.1. In A/B testing that bad decision, meaning what you thought was an excellent B test item in an A/B test, may go terribly wrong

2.2. Missing the “Why”

2.2.1. No understanding of number and result

2.3. Not Predictive

2.3.1. Can’t be used to predict future design change impacts

2.4. Needs Traffic

2.4.1. In order to provide quick, consistent and reliable results, you’re going to need a pretty good amount of traffic to your web page