Useful Usability

1. Pros

1.1. Fast of all the test types

1.1.1. takes very little time to create a modified version of an existing web page that includes a modified item (like a new picture, new copy or other new element) and throw it up on your site.

1.2. Test Reality, not theory

1.2.1. a live web site is you’re obtaining real results from real users doing real things.

1.3. Quantifiable

1.3.1. Provide actual numbers that can be compared, sliced and diced to evaluate results

1.4. Accurate

1.4.1. 100% accurate assuming

2. Cons

2.1. Can hurt web site results

2.2. Missing the "why"

2.2.1. does not explore the rationale behavioral decisions that are being made by the web site visitors.

2.3. Not predictive

2.3.1. can’t be used to predict future design change impacts.

2.4. Need traffics

2.4.1. need a pretty good amount of traffic to your web page to run an A/B test.