If I understand their offer correctly - $5000 semi-annually - it seems a lot like yesterday's post - a gated community for conferences, is now a gated community for measurement.
I would say they have entirely missed the larger opportunity to accurately measure the internet in all it's manifestations, facebook apps, blog conversations, casual games, online comics... all those pieces of transaction and non-transaction based content that are real indicators of what the web is at a given point, and where it will go in the future.
And that's what big companies, and conferences, always do. They miss opportunities. Really really big ones like IBM did with Microsoft, and what Microsoft did with Google and like Google is missing now with 'fill in the blank."
Companies need to think more holo-graphically about the internet, and William Gibson keeps popping in my mind here, and see web in 3 dimensional living breathing terms instead of what can be translated into a flattened power point presentation and an excel spread sheet.
I would tell all the boards of these companies to go see more 3D cartoons with their kids. Perhaps then they would will start thinking more HOLO about how to measure, enable, and monetize the web.
fredwilson
· 6 months ago
I hope someone from comScore reads this comment and responds to it. You raise a very important point but I don't feel capable of responding intelligently and comprehensively.
Jon Miles
· 6 months ago
Yes, they should measure 'the all' of it, in order to monetize the important "some of it."
The analogy I'd use is a fisherman casting his net. He throws it wide and catches lots of different fish, and perhaps even a lobster. The spear fisherman, with similar effort, just catches one.
It is a function of the tool they designed.
Perhaps comscore is too fixated on their "spear" concept.
Mary Hodder
· 6 months ago
The beacon idea is innovative. The question is will small services be motivated to share that kind of information in exchange for the trade off of getting Comscore to backup what they claim as numbers. Google Analytics does this now, discretely, where a small service can just give screenshots of the results to funders and advertisers -- essentially to those it's dealing with directly, without having to give data to everyone publicly.
The other question I have about the beacon is will anyone else be able to use it besides Comscore for good or evil (in the google sense of evil). My sense is that if others can take the data too, and possibly use it maliciously, small firms won't do it.
Ideally, if a small firm did use it, they would have some control over who got the data and what they did with it in order that it be a data partnership verses just a give away to Comscore and anyone else.
mary
LaurelPapworth
· 6 months ago
If currency is that which we value - store of value or measure of value - then Comscore may be laying infrastructure for assessing what we value online. Pretty important for the coming social economy no? Laurel @SilkCharm
amerkatz
· 6 months ago
Fred, this is a huge deal indeed. No longer can the web 1.0 monopolies like Ancestry.com hide from the truth that innovative social networking companies like Familybuilder.com are taking huge audience share. Heretofore this share was discredited since it only came from internal server logs. Kudos to Comscore for leveling the playing field!
(disclaimer: I sit on the BOD of Familybuilder)
jasonspalace
· 6 months ago
great move for the internet. wonder if you leaving the board had anything to do with the creation of Media Matrix 360. wouldn't be surprised if it did.
fredwilson
· 6 months ago
I did not leave the board out of frustration. I just can't work on investments for more than one decade!
jasonspalace
· 6 months ago
oops... the rules of venture capital slipped my mind for a moment :)
Vladimir Vukicevic
· 6 months ago
This seems like a good development for comScore. But I also see a good deal of competition in this market. How can comScore protect its dominant position? It seems that both the panel centric method and the server-side method (even when turned into a hybrid model) can be relatively easily replicated? Has the brand become large enough to protect comScore's position?
fredwilson
· 6 months ago
The panel model is hugely expensive to operate. This is a highly constructed representative global panel. Before comScore, no company had built a panel of more than 100k members. A 2mm global panel is a huge undertaking
Vladimir Vukicevic
· 6 months ago
I see - so a head-on approach would probably be difficult.
Maybe a more open method provides greater opportunity for new entrants looking to view the Internet in a 3-dimensional (or even 4-dimensional) way - i.e. a crowdsourced approach to tracking and measuring - in line with what I think Jon Miles is mentioning below.
Jon Miles
· 6 months ago
Yes - Youtube is user contributed. So why not metrics?
Vladimir Vukicevic
· 6 months ago
There's definitely something in the idea - a combination of Digg and Wikipedia and YouTube.
greghills
· 6 months ago
comScore is the gold standard in ad agencies, but Quantcast has been operating on a hybrid approach and already has a solid list of sites that have implemented their pixel. I also like how Quantcast allows marketers to tag campaign landing pages as well as sales pages, so there can be a feedback loop between who's visiting the site, who's buying, and the media planning process itself. That way, when you plan a campaign, you can consider the target demographic in the brand brief as well as the demographic info for who is already buying the product.
koranp23
· 6 months ago
I believe this will be excellent for all people that use the internet, by way of so many hackers going onto indivduals accounts and stealing their information this I think will limited the problems that we as users are having. great job.I for one use the site zynga to play poker texas hold'em
ShanaC
· 6 months ago
This sounds amazing-
quick question...is anyone out there measuring Q scores and whether there is such thing as
High q scored ad item on a q score site (high imopression)= both high click through + high external buys?
Say for I dunno, swatch watches (or something of that sort)
Comscore is the closest I have seen, but I have yet to see the internet version of Q
Jay Meattle
· 6 months ago
It's a step in the right direction for comScore, but less relevant now that Google Analytics has API's which site owners can turn to publicly expose their site stats for free vs. pay $10k a year, a fee that most startups would like to avoid.
Mike Su
· 6 months ago
Very glad to see this move and am eager to see the results. I've always had two major beefs with the panel based approach:
1) While all your points regarding server logs are valid, a *pageview* is a pageview, regardless of unique visitors. So if comscore says I have a million uniques, but my logs tell me i get 100mm pageviews (and third party ad servers that serve our ads confirm this), that means each and very unique visitor is consuming 100 pages/visit. While I'd love to think our properties are just that sticky and engaging, I know that is fundamentally incorrect, and comscore has never ever had an explanation for that, especially when it comes to blogs, which should have low pages/visit because of the format.
2) Panel works great for TV because of a finite number of channels. But on the web, given an infinite supply of sites, a panel based system simply cannot extrapolate a representative set of visits to all these destinations.
Hopeful that Media Metrix 360 goes some way to bridge that gap.
bernardlunn
· 6 months ago
That hits the nail squarely on the head, from my experience running a niche site.
Michael E. Rubin
· 6 months ago
"A *pageview* is a pageview, regardless of unique visitors."
*sigh* Mike, no, it's not. This has been an issue for several years, especially as Web 2.0 applications that do not generate page views (AJAX, especially) have come to the forefront.
Page views just aren't relevant in 2009. They need to be retired to the dustbin of web measurement like other 1.0-era traffic metrics. After all, you aren't still insisting on "clicks" and "hits" as being relevant today, do you? So, too, with pageviews.
Disclaimer: I used to work for comScore but don't anymore. This opinion is my own.
Mike Su
· 6 months ago
Right, but your point only reinforces the point - if we counted AJAX/Flash etc. interactions, that 100 pages/visit would look more like 150~200 pieces of content consumed per visit, which would make the uniques that comScore counts look even more out of whack.
My point is not that PV's are the end all be all, but that either a page is loaded from the web server or not. If you work the math, it should be obvious that comScore severely undercounts uniques.
Finally, as a guy working in web, I'm with you, PVs do not fully represent modern web interaction. But as an ad-supported publisher, we have to juggle between adhering to standards that are important to our advertisers (like page impressions), and helping to influence change in those standards.
ShanaC
· 6 months ago
Mike- it will be very hard to count Flash/Ajax interactions. Can we switch the meaning of the word impression to a slightly more intutive one: Not just how many times one has seen the web page- but whatever the webpage does, is that use strong enough for you to remember it, remember it favorably, and ultimately, make you want to come back, with friends?
Remember AltaVista- how much value is still in that name and domain as a result of this new definition? There are a lot of people who remember the name- would it be more favorable, for example, to buy off the domain and redevelop a search engine than the current model of developing new ones?
I've always wanted to see true Q scores done for websites. Call me old fashioned for a young person, but Viacom makes a ton of money through some of the more old fashioned methods of literally trudging through their various demographics' houses, getting Q score data for the demographics' beliefs about products, and asking loads of questions about those products. We are talking about various sub-demographics' choice of shoes here (how many pairs, where they buy, why the colors they choose to buy, what brands).
This is how we find out what colors become "in", for example, when it comes to housepaint. We track those people and go into their houses, and see what they buy, and find out all sorts of extraneous information about them.
It is far more expensive, and the companies that do this move like lumbering giants- but there is a reason they are around. The information is very accurate. It gives a lot of meaning to the datapoints of broader surveys.
I recognize this is the idea behind FocusSight, Admetrix, among other products- but without actually tracing down real life behavior and matching it to online behavior in real life focus groups, and then winnowing down the focus groups into people one would want to follow around to see what the leading edge for your target demographic is, even Media Metrix 360 won't be so helpful.
People lie too much (usually not on purpose, but enough to make a statistical difference)- it is always better to observe them in the wild. Pure sampling won't fix the problem, even if they weight, because the data set will still only be correlative and not causative. You won't see causative answers with qualitative type studies such as very close up focus groups- but the mixture together tends to be more positive.
Jon Miles
· 6 months ago
For $5000 semi-annually it better do all that and a cup of coffee.
CardiffGiant
· 6 months ago
A couple things I don't understand about this:
a) This surely isn't a solution for start-ups as it costs $10k a year which is a real dollar amount for many start-ups b) Since Google Analytics just allowed website operators to display their Analytics data publicly on their sites, why wouldn't site operators simply use that instead of paying $10k to Comscore for this new service?
Mike Su
· 6 months ago
regarding (b) - because advertisers care about/trust/go by comScore. GA data is generally considered "internal" data rather than 3rd party data.
CardiffGiant
· 6 months ago
But WHY should advertisers care about/trust/go by ComScore, which is an imperfect attempt to measure traffic, when publishers can provide them direct access to server-level data in the form of Google Analytics which is essentially a third party verification of ACTUAL traffic to a site?
As someone who has managed mult-million dollar online ad budgets, I'd rather see Analytics data for each site I buy for than ComScore data.
fredwilson
· 6 months ago
Except server side overcounts as I explained in my post. A hybrid model is better
Mike Su
· 6 months ago
Server side overcounts *uniques* (for all the reasons you explained), but it does not overcount page impressions - the page is either requested and served, or it isn't. When you take that server page impression count and divide it by the undercounted uniques from a panel system, you get an abnormally high pages/visit number.
fredwilson
· 6 months ago
Correct
Steven Kane
· 6 months ago
Hi Fred. This does seem like a great new wrinkle. But isn't it back to the future? Media Metrix as one of the first (maybe the first?) server-side measurement companies, no? Through their acquisition way back when of I-Pro?
fredwilson
· 6 months ago
I'm not sure steve. I think they had a panel too. Just a very small one
Lee Drucker
· 6 months ago
I realize that this is a bit off-topic, but I thought that the people who read this blog might have some suggestions. I am a JD/MBA Student at NYU- I have an idea for an on-line investment tool, and I am looking for a web developer who would be able to create it. Does anyone have any suggestions?
Thanks for your help.
Vladimir Vukicevic
· 6 months ago
I find that personal referrals are the best for those kinds of services. NYU also has a computer science/design program - why not start there if you're looking at doing this on a tight budget.
Dave Hendricks
· 6 months ago
Fred: You should check out Datran Media's Aperture. It's based on validated and verified third party data.
This new version of Comscore looks alot like a combination of legacy Comscore and Quantcast. Both have major flaws in their methodology, that do not start and end with the self-reported panel data. Websites report vistors in an inaccurate fashion as well. People think Nielsen is bad with it's very small sample and extrapolation, but Comscore and Quantcast as just as bad.
Aperture is beacon-based already and has coverage of more than 70 million households in the US, and growing. A 2 million sample sounds like alot until you realize that it's less than 1% of the US Audience.
If you want, we'd be happy to show it to you. It's real. It's released. And it can retarget too. Oh, and it's compliant with privacy standards.
E. David Zotter
· 6 months ago
Isn't this what Red Sheriff was doing in 2000? ...or am I missing something?
fredwilson
· 6 months ago
I don't think they had the panel that comscore has
E. David Zotter
· 6 months ago
..... one more thought. They should use relevantID in the mix... that would increase the accuracy of their beacon dramatically.
All Comments
I would say they have entirely missed the larger opportunity to accurately measure the internet in all it's manifestations, facebook apps, blog conversations, casual games, online comics... all those pieces of transaction and non-transaction based content that are real indicators of what the web is at a given point, and where it will go in the future.
And that's what big companies, and conferences, always do. They miss opportunities. Really really big ones like IBM did with Microsoft, and what Microsoft did with Google and like Google is missing now with 'fill in the blank."
Companies need to think more holo-graphically about the internet, and William Gibson keeps popping in my mind here, and see web in 3 dimensional living breathing terms instead of what can be translated into a flattened power point presentation and an excel spread sheet.
I would tell all the boards of these companies to go see more 3D cartoons with their kids. Perhaps then they would will start thinking more HOLO about how to measure, enable, and monetize the web.
The analogy I'd use is a fisherman casting his net. He throws it wide and catches lots of different fish, and perhaps even a lobster. The spear fisherman, with similar effort, just catches one.
It is a function of the tool they designed.
Perhaps comscore is too fixated on their "spear" concept.
The other question I have about the beacon is will anyone else be able to use it besides Comscore for good or evil (in the google sense of evil). My sense is that if others can take the data too, and possibly use it maliciously, small firms won't do it.
Ideally, if a small firm did use it, they would have some control over who got the data and what they did with it in order that it be a data partnership verses just a give away to Comscore and anyone else.
mary
(disclaimer: I sit on the BOD of Familybuilder)
Maybe a more open method provides greater opportunity for new entrants looking to view the Internet in a 3-dimensional (or even 4-dimensional) way - i.e. a crowdsourced approach to tracking and measuring - in line with what I think Jon Miles is mentioning below.
quick question...is anyone out there measuring Q scores and whether there is such thing as
High q scored ad item on a q score site (high imopression)= both high click through + high external buys?
Say for I dunno, swatch watches (or something of that sort)
Comscore is the closest I have seen, but I have yet to see the internet version of Q
1) While all your points regarding server logs are valid, a *pageview* is a pageview, regardless of unique visitors. So if comscore says I have a million uniques, but my logs tell me i get 100mm pageviews (and third party ad servers that serve our ads confirm this), that means each and very unique visitor is consuming 100 pages/visit. While I'd love to think our properties are just that sticky and engaging, I know that is fundamentally incorrect, and comscore has never ever had an explanation for that, especially when it comes to blogs, which should have low pages/visit because of the format.
2) Panel works great for TV because of a finite number of channels. But on the web, given an infinite supply of sites, a panel based system simply cannot extrapolate a representative set of visits to all these destinations.
Hopeful that Media Metrix 360 goes some way to bridge that gap.
*sigh* Mike, no, it's not. This has been an issue for several years, especially as Web 2.0 applications that do not generate page views (AJAX, especially) have come to the forefront.
Even back in 2006, we were deailng with this issue:
http://arstechnica.com/business/news/2006/12/84...
Page views just aren't relevant in 2009. They need to be retired to the dustbin of web measurement like other 1.0-era traffic metrics. After all, you aren't still insisting on "clicks" and "hits" as being relevant today, do you? So, too, with pageviews.
Disclaimer: I used to work for comScore but don't anymore. This opinion is my own.
My point is not that PV's are the end all be all, but that either a page is loaded from the web server or not. If you work the math, it should be obvious that comScore severely undercounts uniques.
Finally, as a guy working in web, I'm with you, PVs do not fully represent modern web interaction. But as an ad-supported publisher, we have to juggle between adhering to standards that are important to our advertisers (like page impressions), and helping to influence change in those standards.
Remember AltaVista- how much value is still in that name and domain as a result of this new definition? There are a lot of people who remember the name- would it be more favorable, for example, to buy off the domain and redevelop a search engine than the current model of developing new ones?
I've always wanted to see true Q scores done for websites. Call me old fashioned for a young person, but Viacom makes a ton of money through some of the more old fashioned methods of literally trudging through their various demographics' houses, getting Q score data for the demographics' beliefs about products, and asking loads of questions about those products. We are talking about various sub-demographics' choice of shoes here (how many pairs, where they buy, why the colors they choose to buy, what brands).
This is how we find out what colors become "in", for example, when it comes to housepaint. We track those people and go into their houses, and see what they buy, and find out all sorts of extraneous information about them.
It is far more expensive, and the companies that do this move like lumbering giants- but there is a reason they are around. The information is very accurate. It gives a lot of meaning to the datapoints of broader surveys.
I recognize this is the idea behind FocusSight, Admetrix, among other products- but without actually tracing down real life behavior and matching it to online behavior in real life focus groups, and then winnowing down the focus groups into people one would want to follow around to see what the leading edge for your target demographic is, even Media Metrix 360 won't be so helpful.
People lie too much (usually not on purpose, but enough to make a statistical difference)- it is always better to observe them in the wild. Pure sampling won't fix the problem, even if they weight, because the data set will still only be correlative and not causative. You won't see causative answers with qualitative type studies such as very close up focus groups- but the mixture together tends to be more positive.
a) This surely isn't a solution for start-ups as it costs $10k a year which is a real dollar amount for many start-ups
b) Since Google Analytics just allowed website operators to display their Analytics data publicly on their sites, why wouldn't site operators simply use that instead of paying $10k to Comscore for this new service?
As someone who has managed mult-million dollar online ad budgets, I'd rather see Analytics data for each site I buy for than ComScore data.
Thanks for your help.
This new version of Comscore looks alot like a combination of legacy Comscore and Quantcast. Both have major flaws in their methodology, that do not start and end with the self-reported panel data. Websites report vistors in an inaccurate fashion as well. People think Nielsen is bad with it's very small sample and extrapolation, but Comscore and Quantcast as just as bad.
Aperture is beacon-based already and has coverage of more than 70 million households in the US, and growing. A 2 million sample sounds like alot until you realize that it's less than 1% of the US Audience.
If you want, we'd be happy to show it to you. It's real. It's released. And it can retarget too. Oh, and it's compliant with privacy standards.