In a 1918 edition of the New Republic, a music critic lamented: "Strictly considered, writing about music is as illogical as singing about economics." As true as the statement is it did not, of course, result in the end of music criticism.
When examining the health of digital marketing effort or even a business itself, we’ll hopefully turn our gaze to analytics and numbers. Measuring things is good and gives a different perspective that sometimes is missing from our more narrative natural language.
The challenge, of course, is that in discussing performance as collection of numbers and metrics we are, by nature, abstracting and translating from the real meaning of number into the narrative forms required for decision-making and inspired action.
Let’s examine some definitions of words and phrases about numbers and analytics. That way, when we’re talking about numbers we can increase, and not disperse, our understanding.
General analytics terms:
Metric: Something that can be measured. Remember though, just because something can be measured doesn’t mean that it is beneficial to do so. For example "bounce rate" and "visits from mobile devices" are both metrics.
KPI: Key performance indicator. This is a metric that has a known relationship to your business objectives. When a key performance metric changes, then some impact is expected to be felt by the whole organization. For example, the number of "thank you for filling out the lead form" pages served is a KPI.
Vanity metric: A metric in which the clear relationship to business objectives is overshadowed by a personal desire of someone to see the metric change in one direction or another. Often a vanity metric will have no clear relationship to business objectives. Resources spent improving vanity metrics could be used to improve the business. For example, "site wide bounce rate" is often a vanity metric leading to expenditures on a total site redesign, when a redesign of specific landing pages that are poor performers might be a better option.
Segment: A subset of all available data that shares a common characteristic. For example "mobile traffic," or "visits that filled out the lead form," are segments.
Specific metrics which are commonly discussed:
Bounce rate: The percentage of visits in which only a single page on the website is viewed.
Unique visitors: Should be the number of individual people who view a website. In practice this is actually the number of unique devices used to access a website. For example, one individual could access a website from a laptop, their work desktop computer, their mobile phone and an iPad. This one person would register as four unique visitors.
Visits: The number of visits to a website, regardless of whether they were from unique individuals or not. For example, one person visiting a site 200 times would generate 200 visits. So would 100 people visiting a site twice, or 200 people visiting a website once.
Mobile visits: Visits accessing a website through a known mobile device including smartphones and tablets.
Volume: A number or count. Used to gather absolute data on how many times something occurred. Traffic volume, for example, is a count of how much traffic a website is receiving.
Percent: A ratio or portion. Used to gather relative data on the frequency or relative size something occurred. For example, "exit rate" measures how often a page is the last page viewed on a site.
Kinds of data
Panel data: data generated by asking people to fill out a survey, or by getting them to agree to have their Web usage monitored via software for a study. For example, Hitwise and Nielsen rely on survey populations to represent all Web traffic.
Server data: data generated by a Web server and output to server logs. This data includes a variety of non-human traffic types such as bots and scrapers.
Javascript data: data generated by inclusion of a bit of code on every page. This data only counts devices which can use Javascript and requires simple though diligent maintenance of the code on each page.
Ad vendor data: data provided by a third-party source, often to encourage deeper use of their tools or services. This methods used to generate this data are often opaque.
Words that suggest statistics:
Average: adding up all of the items and dividing by the items. Statistics nerds may refer to this as the "mean." In some cases a metric is only available as a metric. However, a single outlier in a data set can severely distort averages. This effect gets more pronounced the smaller the data size.
Median: the item in the middle of a data set (or an average of the two middle items, if the number of items is even). This method minimizes the impact of outlier data. For example, a set of daily visits to a website from social media might be: 1, 4, 4, 4, 5, 6, 6, 7, 8, 200. The median daily visits from social media would be 5.5 (the average of the two middle items in the data set, 5 and 6). For comparison, the average daily visits in this example would be 24.5. The outlier — 200 — distorts the average.
Typical: This could refer to average, median or mode (mode is the number that appears in a data set with the greatest frequency. In the example above illustrating the concept of the median, the mode is 4). Given the potential difference and meaning of the different methods, it’s important to gain clarification when this word is used.
Significant: Tests and analysis have been performed and it is unlikely that the results are due to chance. In fact, there should be percentage chance number associated with significance. For example, a statistician should be able to say, "There is a 95 percent certain that was not a chance occurrence." Or, "There’s a 5 percent possibility that this was a chance occurrence." When dealing with numbers, if you believe something is important but have not performed tests, then simply say you believe it’s important. But don’t call it "significant."
Trend: The direction that a metric is moving. In many cases, the trend may be more important than the absolute value. In particular, how fast or slow (steep or flat, if represented on a chart with a time-based x axis).
