Meaningful Statistical Analysis and Improved ROI: Web Hosting Provider Tips

If you suffered the nightmare called Statistics 101 freshman year of college, you’re familiar with baselines, test populations, A/B testing, multi-variable testing and other statistical apparatus that’s been taking up brain space for the past 20 years. And, if you were able to avoid the “pleasure” of calculating white balls and black balls in imaginary urns, congratulations. You dodged a bullet.

But, guess what. As a site owner, you’re now in the statistics business. They may have failed to mention that part at Webmaster University, but if you don’t start tracking your site’s growth, there won’t be any.

Now, this doesn’t mean that you have to pull out the old Stat 101 textbook (yet), but it does mean that you have to design test methods that can be validated and used to develop strategies for improving SEO and conversion rates, and consequently, your site’s return on investment.

So welcome to Statistical Analysis 000, designed especially for the numerically impaired, and that’s most of us. This article is offered by your web hosting provider and will address these important issues.

Decide What You’re Measuring

Duh! By simply compiling some data, and then trying to figure out what all of those numbers mean, is meaningless (pun intended). Instead, you start with an objective – the activity, behavior or site element to be measured – then design your test around it.

For example, let’s say you want to measure the impact of two different shopping carts on buying practices of visitors to your recently launched commercial site. That’s an objective, and it’s pretty easy to determine which shopping cart site visitors prefer after testing, right? Umm, no. Please read on.

Determine Your Test Population

If you base your site growth strategy on a test population of 10 site visitors (five of whom are friends of your mother who’s just so proud) you won’t develop meaningful statistical analysis because the test pool doesn’t represent a broad enough range of individual tastes, preferences, biases and so on.

Your designated test population should be a percentage of your weekly visitors, so the actual size of the test pool will change depending on whether you get 10,000 hits a week (lucky you) or 100 a week.

Also, consider other aspects of your test population instead of randomly selecting the activities of the first 100 visitors through the door. Before conducting your test, define other characteristics of your test group, i.e., only AOL users, only visitors through links, only organic SERP visitors and so on. In other words, define your test demographic before you launch the test. You can use a finite number, the first 500 visitors you get, for example. That’s called a numerical test population. Or, you can create a temporal test population based on the time factor, e.g., all visitors for the next seven days.

Keep Statistical Analysis Simple

The simplest test is comparing A and B where A is shopping cart A and B is shopping cart B. By swapping out A for B, and collecting data based on your pre-defined test population, it’s pretty easy to tell which one visitors prefer – if that was your test objective. More people completed transactions (fewer shopping cart abandonments) with shopping cart B than A. So you go with software B, right? Not so fast.

Meaningful Statistical Analysis

For those feeling light-headed, breathe deeply before reading on.

Once again, the easiest testing method is the A-B comparison – changing one website variable, testing it, comparing test results and selecting A or B based on number of visitors who used A versus B.

However, what if you dig a little deeper into the numbers? You might discover some other interesting data. Sure, fewer people abandoned their shopping carts (average percent of abandonments per site = 70%, btw) using software B, but with additional, deeper statistical analysis, you might also discover that visitors using shopping cart software A spent more on each visit. Or viewed more pages. Or, returned more frequently. Or returned fewer items, or required less client care. That’s why it’s important to keep it simple and define what it is your measuring.

Meaningful metrics analysis: (1) must have a clearly-defined, variable for testing and (2) all data should be analyzed using methodology that can be replicated in identical tests. This avoids leaping to conclusions not founded in the collected data.

Slow But Sure?

Using simple A-B comparison tests, it’s going to take you between 40 and 50 years to fully optimize your site, swapping out type font Y for font Z and color Q for color R. Not to mention the testing of your back office – content management systems (which is better for your business needs?), auto email (which gets the job done at the lowest cost?), database (which gives you the most flexibility and ease of use?).

You get the idea. Simple A-B comparisons are perfect for one-run tests to see if that new demo video is pulling in more high ticket item sales, thus giving you an improved rate of return (ROI) to justify the costs associated with creating and adding the demo. So, in order to optimize your site before the start of the next century, you’re going to have to employ multi-variable testing – changing more than one item at a time and, through the use of statistical formulae, determining which variable created the optimum impact.

Or, you can use metrics analysis software (whew!) that crunches the numbers for you and delivers the results of multi-variable testing in a variety of formats including:

Heat maps that provide a graphic showing site “hot spots” in reds and yellows, and site elements that aren’t getting much action in blues and greens. This visual representation of visitor activity is easy to interpret, even for the complete novice. It’s no more complex than looking at the temperature range on a weather map.

The list format provides a full, collated summary of all collected data. It’s perfect for number crunchers and “outside-the-box” creative types. However, to perform a meaningful analysis of raw data, you’ll have to conduct the test several times to, first, establish your measurement baseline. Then, subsequent testing will show increases or declines in activity compared to the initial data – your baseline data.

Finally, the overlay view provides an overlay of numerical percentages of gains or losses for each element of each site page. Using a simple GUI, you see the percentage of increase or decrease of every design element. This enables you to analyze multi-variable test data at a glance to see which variables have had a positive impact and which haven’t. Change accordingly. Then test again.

The Bottom Line – Yours

Sure it’s fun to check out the heat map after you’ve added some cool graphics, or to spend hours reviewing raw data in list format – at least for some site owners. But here’s the bottom line: it’s your bottom line.

All of your testing has one, over-arching objective – to squeeze more dollars out of your pixels, i.e. to increase your return on investment (ROI).

Some owners make changes to their sites to experiment. And when they get the pre-defined result they were after, they’ll refine the changes even more. Why bother? It’s a big waste of time if it doesn’t boost ROI. A recent study compared two checkout software packs. One could be set up for single-click checkout by repeat buyers. The second pack required six clicks to get through the checkout regardless of how many times a visitor stopped by to make a purchase.

You’d think the “one-click and out” software would out-perform the six click version that executed the exact same function on line. Surprise! It didn’t. Visitors didn’t care whether it took one or six clicks, as long as there was plenty of signage along the way to tell buyers where they were in the checkout process and how to get to the next step.

So, why swap out one checkout for another if it isn’t going to improve your ROI. Why risk changing the appearance of your checkout or shopping cart and confusing repeat buyers? (Not good.)

You can run test after test, but if the results aren’t contributing to a significantly improved ROI, you’re spinning your wheels.

Maybe it is time to dust off the old Statistics 101 text from your halcyon school days. Meaningful statistical analysis – analyses that lead to site improvements that, in turn, boost your ROI – are now part of your business as site owner and webmaster.

Stay tuned to this web hosting blog for more webmaster tips.