Six reasons internet measurement is rubbish

The greatest fallacy in digital marketing is that the internet is the most measurable medium. It’s not. Yes, it’s the medium with the most data (absolutely phenomenal amounts), but making sense of that data to get a true picture of what consumers are actually doing (ie measuring it) is still far from an exact science.

Last week I journeyed into the heart of data geekery, spending two days at the biennial I-com conference in Lisbon, a conference dedicated solely to online measurement. I’d be lying if I pretended my brain didn’t hurt by the end of it. But several themes emerged over the course of the conference that deserve recognition outside this specialist world. For marketers (precious few of whom actually attended the conference), sorting this stuff out will be vital if they’re ever going to spend serious money online. And for publishers, this stuff is key to the way they will sell online performance.

1. Even the things you think you know, you don’t. One of the key problems with measuring online activity is that the data comes from machines, not humans. Linking up what we know from humans (ie panels) with the data deluge from the internet is tricky. The result is that we often assume the machine data tells us things it doesn’t.

Take unique users for example. Sounds like a pretty basic piece of information, and something most campaign metrics seem to look at these days. But as one speaker pointed out, measurement of unique users is usually based on cookies. Now people use multiple devices to access the web (mobile, home computer, work computer etc etc), and each has a different cookie. Also, they are increasingly using multiple browsers on single device, adding even more cookies. Some users, said the speaker, had around seven separate cookies. So what does that make a unique user?

2. Clicks are pointless. This isn’t new news, but that doesn’t stop click-through rates being used as a measure of campaign success. Yes they’re easy, but they mean absolutely nothing unless an ad is purely a direct-response piece. What’s more, click-through rates are declining fast; that means campaign measures based on clicks are looking at a diminishing audience. Also, there are, said one panellist at the conference, no correlations between click-through rates and other brand scores.

3. The last click tells you little. All the action is still around the last click – this particularly applies to search, where search engines make their money from people clicking on links. But much of the hard work in terms of taking people through the purchase funnel comes much earlier – and this bit is really hard to measure. Doron Wesley from Cheil/Samsung admitted at the conference that the brand had not even started to look at so-called ‘attribution’ – working out where and when a consumer interacted with a brand online and what the influences on purchase were. If a brand like Samsung can’t do this, not many others can either.

Search is an interesting one, because on the surface it is so measurable. But in truth it is still extremely difficult to measure search’s total impact. How do you measure ROPO (research online, purchase offline)? Or even the branding impact of search activity?

4. People can’t even agree what needs measuring. Many years ago, when I first started writing about online marketing, an agency boss told me that the medium needed reach and frequency measures like TV to get marketers interested. Imagine my surprise when I find the exact same argument is raging in 2010. In Lisbon it was the GRP system that holds such sway in the US that caused debate – does the internet need to adopt GRPs to be able to get marketers interested? There’s no consensus here – everyone knows the internet needs some sort of standard measurement currency, but nobody can agree on what that should be.

5. Increasingly the action is in social, but that’s even less measurable. Social media is increasingly used as a PR tool. Terms such as ‘engagement’ and ‘conversation’ abound. But what does that mean and how do you work it’s delivering.

The last couple of years have seen all sorts of buzz-tracking services launch, and some were at the conference. But they didn’t really have an answer when one delegate asked why, when he asked five different word-of-mouth measurement companies to track his Superbowl ad, he got five different responses. These services sell themselves as tracking tools, but actually they are hugely reliant on human input to categorise brand mentions on social media. They’re better than nothing, but they’re still flawed.

6. Don’t even start on mobile. The mobile expert that appeared on stage simply described mobile measurement as ‘icky’. Nuff said.

As Geoff Ramsey, CEO of eMarketer, put it:  “Right now, we’re in the dark ages.” And as one of the few clients at the conference told me, without a proper currency online that allows some sort of comparison between media, he cannot advise heavy spend in digital.

So next time you hear an agency lazily praise the measurability of online media, it’s time to ask a few hard questions.

[tweetmeme source=”davidtiltman” only_single=false]