For online marketers like me, numbers are fundamental to the way we make decisions. Whether you split test your Facebook ad creative, run A/B or multi-variate tests on landing pages or analyse pre-defined goals in Google Analytics, you need to be on top of your numbers if you are to get the most bang for your marketing buck. But just how much do you know about statistical models and how confident are you that you’re applying the right one’s in the right situations?
Below are four statistical concepts that are crucial for all online marketers to know.
1.The Pareto Principal: Every eighty percent of effect comes from twenty percent of cause, otherwise known as the 80/20 rule. In other words, twenty percent of your work accounts for eighty percent of positive results. This principle helps with time and money management. Identify what works and focus on that first.
So, for example, you’re running a bunch of Facebook ads for a client. You’ve got some ad sets that are delivering great returns and others that are not quite bringing in the same ROI. As such, focus your time in further optimising the top performing ad sets and get to the rest when you have time.
2. The Law of Large Numbers: You ever wonder why onsite optimisation platforms like Convert.com advise you run a test to a large enough audience before jumping to conclusions? It’s because you need a broad enough sample to act as your comparable baseline.
Think of it in terms of an A/B landing page test. You want to improve the number of people that add products to their basket on your ecommerce site. The result of your test will be rolled out to some 1,000 product pages.
You pick your best selling product to test your new layout on. Now if you only tested landing page A to 5 people and landing page B to 5 people, do you think you’d be confident to roll out the results site wide? Of course not. At a minimum you’d want 1,000 impressions before making such a decision. Moreover, you’d want to test the layout on other top selling product pages so you could base your decision on multiple pages each experiencing 1,000 page impressions.
So don’t assume generalisations from research. Skewed hours and possible conversions can be just that, skewed. You may miss a great opportunity for sitewide improvement if your decisions are based on too small a sample.
3. Relative and Absolute Numbers: Numbers in regards to conversion and marketing are relative to the situation. With out using the absolute number as base, percentage readings can make a one percentage drop seem much worse than it really is, misleading the marketer. You need to make sure everyone seeing the data understands what the number entails.
Let’s continue the A/B test scenario. What if you had run the A/B test to just 20 people, 10 people/impressions to each page. Page A, your current page design, continues to enjoy a 20% add to basket action rate as before. But maybe test B, your new page design, saw a drop from 2 to 1 of people adding an item to their basket.
If you report to stakeholders that page B saw a 50% drop in basket adds, that sounds pretty extreme. But if you gave your stakeholders the full facts of the matter and that only 20 people were sampled…well you’d probably be told to stop wasting their time! You’d be told to come back when you had a greater sample and to report your percentage findings in conjunction with absolute numbers.
4. Simpson’s Paradox: As stated on Wikipedia this is a paradox “in which a trend appears in different groups of data but disappears or reverses when these groups are combined.”
Keeping with our A/B test scenario let’s say we have two pages that are experiencing quite different conversion rates even though the layout is essentially the same. If you wanted to run A/B tests on each page, could you be sure your outcomes would be valid given both pages have quite different conversion rates?
Armed with a knowledge of Simpson’s Paradox you might first decide to investigate the source of traffic to each page before running the tests and jumping to conclusions.
So, for example, page A may receive similar traffic as most pages on the rest of your site, i.e. from internal links, from direct Google traffic, from remarketing ads, etc. But page B, which has a much higher conversion rate, actually receives over 50% of it’s traffic from a link on a major publishers site. Page B is kind of unique on your site and so should not be included in your A/B test. By all means run an A/B test on that page alone, but do not combine the results with the results of your other landing page A/B tests.
In Summary
Data can help lead marketers in the right direction and increase sales. But first you need to get your sample sizes right, go armed with a working knowledge of the above statistical models and check and recheck your numbers before presenting to stakeholders or you could end up leading people astray.
Key Takeaways:
- Marketing decisions need to be based on statistically reliable sample sizes.
- Conversion rates can be a common misunderstood statistic in marketing.
- Knowledge of statistics is a key component to a successful online marketing campaign.
“If you can figure out which 20% of your time produces 80% of your business’ results, you can spend more time on those activities and less time on others.”
