Why can’t ComScore & Neilsen do simple math?

The short answer is that it’s not the math that is the problem.

The numbers have long been a source of major squabbles for a lot of web publishers. The internal numbers often are much larger than what the outside monitors report. I have been asked for years why ComScore and Nielsen‘s numbers are so much lower than our internal numbers. Sometimes in extreme cases I have seen the comparison be as much as 75% lower.

Nielsen and ComScore’s can’t even agree. Last year Nielsen measured Wired News’ audience at 1.1 million, while ComScore reported an equally precise-sounding 1.9 million.

So why use ComScore and Nielsen, why not just use an industry standard like an Omniture?

The answer is our advertisers. ComScore and Nielsen panels create a comfort level with the advertisers. It’s what they know and because they match up well with the traditional television-measurement approach. Nielsen claims its television ratings have a margin of error of 4 percent.

So who can you trust? What numbers are accurate?

I wish that was an easy question. ComScore’s canned response claims that cookies overstate traffic. Recently with the help of Omniture they said they had detected 1.5 billion U.S. cookies, which is seven times larger than the number of Americans online.

What I find is that the real numbers are somewhere between the internal reports and what ComScore and Nielsens report.

Why all the confusion?

Let’s start with understanding the problem.

Understanding the Nielsen and ComScore problem.

• They base their numbers on sample size. This is called Panel-based measurement. These panelists get modest compensation for installing tracking software on their computers. ComScore solicits a larger panel of 120,000 and Nielsen only has 28,000.
• Nielsen and ComScore use different formulas to extrapolate total traffic.
• Both count only U.S. Visitors and a lot of media of media sites can have 20% – 30% of their total traffic outside the U.S. Obviously there is a growing number of page-views coming fromAfghanistan. My own blog reports show that only 61% of my own traffic is from the US.
• Neither have a way of accurately measuring people surfing from universities, government agencies, and the corporate professionals. That is because many such places often prohibit workers from installing the tracking software.

• Nielsen Conducts random-dial phone surveys to even out the work-surfing numbers.
• There are lots of industry questions. E-mail newsletters, should they be counted most of which is read on smartphones.

Understanding why internal numbers are just as confusing.

• Most internal counts are actually unique browsers instead of unique visitors. Which oftens causes an apples-to-oranges comparison.
• Some sites adjust down for slide shows that rack up a dozen page-views for one article.
• In most cases I have found that sites don’t adjust the numbers to account for slide shows.
• Flash and other interactive content often only counts a single pageview. Some sites have configured code to count clicks in flash but the the 3^rd party monitors do not account for them.
• Robots crawling sites to index them for search or price-comparison sites will increases the count.
• Google and some other often identify themselves but most publishers sites that don’t configure their internal analytics to exclude them. – A lot of content-scrapers deliberately camouflage themselves to avoid being blocked.
• Internal staffs should they be counted? A lot of sites do not count them to due to the traffic created from internal Quality Control process or for research purpose. Some sites do include them.
• A lot of companies outsource Qualtity Control and if they don’t configure internal systems to ignore them it will inflate the numbers.

I would call this flying in the dark.

Another problem is just an industry problem. There is a lot of confusion by publishers on the definitions of what a page-view, a unique visitor or a visit are.

Page views are the number of individual Web pages the site served. – If one person reads a three-page story, that’s three page views. Publishers use this number to measure available inventory for advertisers, who typically pay based on how many times their ads will be shown.

Unique visitors are the number of people who looked at the site. If you look at the a websites home page at home and at work, that counts as two page views but only one unique visitor. Publishers use this number to give advertisers an idea of the site’s “reach”—that is, how many people might see an ad

So the answer to Why can’t ComScore & Neilsen do simple math? The problems go way beyond the math.

Next week I will I’ll include some of my suggestions to the problem as well as some technologies that can help.

*This article doesn’t account for Comscores Media Metrix 360 which isn’t fully adopted yet.

/Justin Makler

The Short Link: http://wp.me/pOR8I-5W

Articles I used in my research:

• The Trouble With Web-Traffic Numbers (blogs.wsj.com)
• Calacanis Take Down of ComScore, Advocates Short:- Comscore’s CMO Responds (benzinga.com)
• Biometrics firm confirms: User counts for websites are 2-4 times too high (venturebeat.com)
• ComScore, Omniture Join Up to Measure Audiences (thaibrother.com)
• Web Analytics Startup Effective Measure Gets $4 Million (paidcontent.org)
• How unique is a unique visitor? (vator.tv)
• Dot-Complicated: Tracking Web Traffic (online.wsj.com)
• Yahoo Says We’re Wrong, “It’s Y!ou” Is A Big Success (YHOO) (businessinsider.com)
• As Challenges Grow, Nielsen Looks To Explain Its Digital Media Abilities (paidcontent.org)

Advertisement