Let me begin by telling you a story about shark attacks. The summer of 2001 was christened the “summer of the shark” by Time Magazine because the focus was on shark attacks that year. This story was then picked up and further propagated by other news channels. Before we knew it, the big story that summer was about an unexpected increase in shark attacks that year.
Before I go on, let me say that I do not intend to minimize the gravity of shark attacks on anyone during that time or any other time. The shark is a top predator of the ocean and behaves as such. The point I want to make is this – the media used a useless metric (number of shark attacks) as a way to drum up interest in a pre-9/11 summer. If they had actually put it in the context of relative measure such as the number of shark attacks in 2000, they might have opted to focus on the lifecycle of the ant instead.
This brings me to the first rule of creating a great metric. That’s what you are here to read, right?
The First Rule
Numbers without context are notoriously poor measures of performance. Performance is best understood in relation to another quantity. So the next time someone throws out a metric “number of blah blah blah,” your first question should be “In relation to what?”
Here’s the first rule about building a great metric – It should be a relative measure.
Let’s work through a few examples:
- There were 3,000 visitors to bayleafdigital.com in Jan 2015.
- eCommerce firm ACME made $5 million in income in Q4 2014.
- AAPL stock was trading at $119 on Feb 5, 2015.
Do you know the action you can take based on these statements? None. Nada. Here’s why – There is no comparison for any of the numbers listed above. Bay Leaf Digital’s visitors were 3,000 in January compared to what? Was that a good thing or a bad thing? Do I need to panic or celebrate??? Apple’s stock was at $119. Was that an increase because of the recent earnings call? Or was that pretty much a flat day compared to the day before? So metrics make sense only in relation to something else.
That relative measure can be the same metric for a different time or a directly related metric for the same time. For example (using Bay Leaf Digital’s 3,000 visitors in Jan 2015):
| Root Measurement | Interesting Metric | Type of Comparison |
| 3,000 visitors in Jan 2015 vs 2,200 visitors in Jan 2014 | 800 more visitors in Jan YOY | Time series for the same metric |
| 3,000 visitors of which 2,500 were new visitors | 5x new visitors compared to returning visitors | Ratio comparison |
| 3,000 visitors of which 1,800 saw more than 1 page. | 60% visitors are engaged | Percent based comparison to a directly related metric |
The relative measures above provide contexts to these metrics. Now we know the visitor count in January changed for the better. Or that we are getting more new visitors than repeats. Or that there is a small and interesting segment of highly engaged visitors. Now we have something to chew on. In web analytics, relative measures are way more useful than absolute measures.
The Second Rule
Of the above relative measures, the percentage based metric feels the most intuitive because of its use in everyday life. From announcing sales (50% off!) to discussing interest rates (11% APR), the use of a percent based metric is widespread. That brings us to the second rule of a great metric.
The second rule about building a great metric – It should to be simple to understand.
This rule “It should be simple to understand” sounds easy, but there’s a lot more to it than you may imagine. By simple, I mean it should have the following characteristics:
- A great metric should be easy to communicate. In a conversation, you should not have to pause to explain what it means. In the example above, if we said “60% of our visitors saw more than 1 page on the site,” people will get it. There is little room for misinterpretation.
- It should be linear. When a change occurs, the size of the change is directly related to the impact on the business goal. In other words, the metric should not have a complex exponential or a logarithmic relationship to the goal. Below is a simple example of how linear and non-linear metrics behave with respect to a business goal.
- It should be directionally intuitive. An increase/up is good. A decrease/down is bad. This is how we have grown up to understand life. Keep it that way.
The Third Rule
The third key characteristic of a great metric is that is can be applied in a variety of contexts. These could be time, marketing channels, visitor type, content, etc. So the greater the ability of the metric to be universally applicable, the better the metric is.
We are going to continue using our earlier example of the percent based engagement rate metric to explain further:
| Context – Time | Interesting Metric – Engagement Rate | Derived Metric – Rate of change | Context – Visitor Type | Interesting Metric – Engagement Rate | Derived Metric – % difference | |
| Nov 2014 | 57% | – | First Time | 50% | ||
| Dec 2014 | 53% | -7% | Repeat | 75% | 50% | |
| Jan 2014 | 60% | 13% |
In the above two tables, we have applied the context of time and visitor type to the engagement metric. In both cases, the reports make a lot of sense. We see that engagement rate was higher in January compared to December and that repeat visitors have a higher engagement rate compared to first time visitors.
Because we were able to apply these contexts, we have even more information about the factors affecting engagement rate.
So here’s the third rule about building a great metric – It should have universal applicability
A great metric can transcend the silos of teams in an organization. It can be used as a universal language across marketing, operations, design, and many other teams. Think about a designer looking about engagement rate of the home page before and after an upgrade. Now think about the SEO expert analyzing engagement rates of her highest performing pages compared to the engagement rate of her channel as a whole. And across all these teams, the definition of the metric remains unchanged, but yet it is widely accepted and applicable.
The Fourth Rule
The most important characteristic that sets great metrics apart from poor metrics is their actionability. The report you generate should drive action from its recipients. The simple question to ask yourself or the person requesting the report is “What action are you going to take when there is a change in this metric?” If the answer is vague, then you know this is not a great metric. It might not even be a good metric. More on that topic in my article on how to identify poor metrics.
So here’s the fourth rule about building a great metric – It has to be actionable
By actionable, I mean a change in this metric should spur further segmentation and analysis to identify possibly causation. One more thing about being actionable, a great metric is closely tied to an organization’s business goal. So when it is time to action, the actions have a direct impact on the organization’s ability to achieve its business goal.
This fourth rule wraps up the key rules to be used when building great metrics. Remember, all four rules are needed to build a great metric. Satisfy three and you have a good metric. Satisfy four and you’ll get a great metric.
A Last Word
Remember every metric you build will not be a great metric and every report will not be a great report. However, you should have the one report that becomes the reference report across the organization. And that report should have all your great metrics.
To recap, here are the four rules to create a great web analytics metric:
- It should be relative
- It should be simple
- It should be universally applicable
- It has to be actionable
What are your thoughts on how to create great web metrics? Let us know in the comments below or on Twitter @Amplytics.