1.1. Donts When doing A/B testing, never ever wait to test the variation until after you’ve tested the control. Always test both versions simultaneously. If you test one version one week and the second the next, you’re doing it wrong. Don’t conclude too early. There is a concept called “statistical confidence” that determines whether your test results are significant (that is, whether you should take the results seriously). Don’t surprise regular visitors. If you are testing a core part of your website, include only new visitors in the test. Don’t let your gut feeling overrule test results. The winners in A/B tests are often surprising or unintuitive. On a green-themed website, a stark red button could emerge as the winner. Even if the red button isn’t easy on the eye, don’t reject it outright.
1.2. Dos Know how long to run a test before giving up. Giving up too early can cost you because you may have gotten meaningful results had you waited a little longer. Show repeat visitors the same variations. Your tool should have a mechanism for remembering which variation a visitor has seen. This prevents blunders, such as showing a user a different price or a different promotional offer. Make your A/B test consistent across the whole website. If you are testing a sign-up button that appears in multiple locations, then a visitor should see the same variation everywhere. Showing one variation on page 1 and another variation on page 2 will skew the results. Do many A/B tests. Let’s face it: chances are, your first A/B test will turn out a lemon. But don’t despair. An A/B test can have only three outcomes: no result, a negative result or a positive result. The key to optimizing conversion rates is to do a ton of A/B tests, so that all positive results add up to a huge boost to your sales and achieved goals.
2. Pros and Cons of A/B Testing
2.1. Pros Fast - Of all the test types, A/B is way, way fast. That’s because it takes very little time to create a modified version of an existing web page that includes a modified item (like a new picture, new copy or other new element). Tests reality, not theory The good news about A/B testing on a live web site is you’re obtaining real results from real users doing real things. Quantifiable A/B web site testing provides actual numbers that can be compared, sliced and diced to evaluate results. Interaction, conversion, number of abandonments – all those numbers are accessible during and after testing. No guessing required! Accurate Unlike other forms of web site testing, A/B testing is 100% accurate ASSUMING you have statistically significant data.
2.2. Cons Can Hurt Web Site Results - In A/B testing that bad decision, meaning what you thought was an excellent B test item in an A/B test, may go terribly wrong. Missing the “Why” - not able to guess what the user is thinking Not Predictive A/B testing is great and all, but it can’t be used to predict future design change impacts. To a certain extent this means that you’re always stuck doing A/B testing. At some point it would be handy to be able to predict if a whole new web page, web site or application will (or won’t) work. Needs Traffic In order to provide quick, consistent and reliable results, you’re going to need a pretty good amount of traffic to your web page to run an A/B test.
3. The Pros and Cons of Alpha or Beta Testing
3.1. pros: Large Volume of Testers Live Testing Real User Testing Beta Means Continual Tweaking
3.2. cons: The Web site or Application Must be Built Beta Testing is Chaotic Beta Testers May Not Match Expected End-Users Beta Testing Exposes Your Secret Sauce
4. Pros and cons of Usability Testing a Web Site or Application
4.1. pros: Tests Real Users Tests Reality, Not Opinions Can test throughout design & development Provides the Why of user behavior
4.2. Cons: Trained Usability Professionals required Testing won’t reveal all issues Usability Testing results can vary Testing & fixing are different things
5. What Is A/B Testing?
5.1. At its core, A/B testing is exactly what it sounds like: you have two versions of an element (A and B) and a metric that defines success. To determine which version is better, you subject both versions to experimentation simultaneously. In the end, you measure which version was more successful and select that version for real-world use.
6. what are some items that are frequently tested?
6.1. The call to action’s (i.e. the button’s) wording, size, color and placement Headline or product description Form’s length and types of fields Layout and style of website Product pricing and promotional offers Images on landing and product pages Amount of text on the page (short vs. long)
7. 2 Commons ways to set up A/B Testing
7.1. Replace the element to be tested before the page loads If you are testing a single element on a Web page—say, the sign-up button—then you’ll need to create variations of that button (in HTML) in your testing tool. When the test is live, the A/B tool will randomly replace the original button on the page with one of the variations before displaying the page to the visitor.
7.2. Redirect to another page If you want to A/B test an entire page—say, a green theme vs. a red theme—then you’ll need to create and upload a new page on your website. For example, if your home page is http://www.example.com/index.html, then you’ll need to create a variation located at http://www.example.com/index1.html. When the test runs, your tool will redirect some visitors to one of your alternate URLs.
