A/B and Multivariable testing

How to document your A/B or multivariate test (Based on article by Josh Baker, http://blog.joshbaker.com/2009/01/17/how-to-document-your-ab-or-multivariate-test/)

Get Started. It's Free
or sign up with your email address
A/B and Multivariable testing by Mind Map: A/B and Multivariable testing

1. Master Test Worksheet

1.1. Column Headers

1.1.1. Test Name

1.1.2. Test Date Range

1.1.3. Test Hypothesis

1.1.4. Results

1.2. Each Row is a single test

1.2.1. Link Test Name cell for each test to the other spreadsheet built for that test

2. Individual Test Worksheet

2.1. First Tab

2.1.1. Test Name

2.1.2. Test Date Range

2.1.3. Test Hypothesis

2.1.4. What kind of test this is (a/b, multivariate), and how many panels or combinations

2.1.5. Traffic data (source of traffic, and current traffic stats)

2.1.6. What type of metrics I will be using to determine the results and how to determine the metrics (is conversion impressions divided by sales, or impressions divided by clicks on a certain button etc?)

2.1.7. A space to record final metric results (control performed as such, top performers identified individual performed as such)

2.1.8. Learning’s (both as the test is live and from the results)

2.1.9. Ideas for future tests based off of this information

2.1.10. Next actions (will this be rolled out etc.)

2.1.11. Miscellaneous notes

2.2. Additional Tabs

2.2.1. Screenshot of control

2.2.2. Screenshots of test panels or combinations (depending on how many there are). If there are too many panels or page combinations to take screenshots of, after the test is ended I take screenshots of the top performing test panels for future comparisons)

2.2.3. Screenshots of Test statistics When I am using Google Website Optimizer I take daily screenshots of the stats admins. and store in a separate folder The final screenshot from the point at which we end the test

2.3. Various others

2.3.1. Reference such as more detailed metrics information, etc.

3. Folders for each test (named as test)

3.1. Test specification (PowerPoint)

3.2. Analytics data

3.3. Screenshots