DISQUS

Jesse Farmer · 1 year ago

Hey Fred,

I'm Jesse, the creator of Adonomics, although I'm no longer working for the company that now owns it. A few points.

1. DAU is not a very honest metric, either. It includes both new users and returning users. If your app is growing virally the "activity" can be fueled almost entirely by the acquisition of new users who do nothing more than install the app, invite their friends, and leave.

On top of that DAU obscures some user behavior. For example, if I get 1,000 DAU today and 1,000 DAU tomorrow, how many users do those two days have in common? Is it the same 1,000 users on both days? 800? Or did only 100 users visit both days?

2. Installs are useful, but not for what people think — they measure the application's network saturation. An application with 30MM installs has saturated almost 50% of Facebook. The growth characteristics and considerations of an application in this state are much different than an application that hasn't already made the rounds.

Also, for what it's worth, the "install" numbers on Adonomics are off-the-wall wrong. It calculates them by dividing DAU by %DAU. Facebook doesn't care much about the accuracy of these numbers so you can get day-by-day swings of over 2-3MM installs. Trust me, no app gains or loses that many installs in a day.

That said, the way people blindly follow other people's metrics is a problem. Metrics should support business goals, not the other way around. I think people see it as a game: get the "most" installs, get the "most" followers, get the "most" pageviews, etc.

Just some food for thought.

Cheers,
Jesse

fredwilson · 1 year ago

Jesse ­ thanks for stopping by and adding to the discussion

This is terrific feedback and information

John · 1 year ago

I would throw out the idea that what's needed is more specific measurements for many of these. Take, for example, Twitter followers. The number is generally meaningless, unless you could determine what % of those followers have notifications turned on... and what's more, if you could actually measure the number of impressions of folks who actually saw or were delivered your tweet. The sad part is... all of these numbers could be reported, they just aren't!

bfernald · 1 year ago

Statistics and metrics are NEVER meaningful with out context. It just doesn't matter how "accurate" they are, they don't have meaning until you put them to work. Batting averages mean nothing until you're faced with choosing a pinch hitter. Gas mileage (mpg) doesn't help if you don't know what kind of driving you expect to do.

Subscriber counts are like horsepower. It's latent potential. If you're buying a Prius, you probably don't care until that one day you need everything it's got to make a left turn in front of traffic. Everyday activity on your blog may trickle as you broadcast your thoughts, but when you ask that burning question, suddenly everyone turns out to comment.

fredwilson · 1 year ago

Subscribers = horsepower.
I like it

Ross Mayfield · 1 year ago

Sounds like you are about to discount my cashflows

fredwilson · 1 year ago

;)

jmspool · 1 year ago

I think you're right. I suspect that subscribers and followers are the 2008 version of 1999's page hits.

Great post.

moon · 1 year ago

Looking at the conversations on Friendfeed and there mostly about Friendfeed

pixites · 1 year ago

on the spot!

AndyBeard · 1 year ago

That attention from RSS is actually very much on the low side, I would have expected even for a more mature blog at least 10% especially with the way Feedburner seems to spike on post attention when someone shares a post on Google reader.

Scobleizer · 1 year ago

Absolutely true! But that probably explains why I spend so much more time over on FriendFeed than anywhere else lately. The most engaging people are hanging out there. Twitter is a great place to do something other than engage with a lot of people. I'm still there too, but less and less, especially when it's down.

dahowlett · 1 year ago

Disagree if we're taking about those who spend - it's not the number who engage but who engages.

Anthony V · 1 year ago

Imagine then, the people who start doing anything specifically to grow these bogus numbers! Creating value vs numbers :)

gregorylent · 1 year ago

which is why calculating return on advertising spend will increasingly be a crap shoot .. (my father's phrase)

NICCAI · 1 year ago

It would be nice to know which tweeters actually have a pulse! I'd venture to say that 70%+ of mine are bots.

Joshua March · 1 year ago

Good comments. However I would just point out that Daily Active Users on Facebook is that - daily. The actual number of Active users is likely to be much higher (e.g. active once a week, or once a month). These numbers are just being introduced by Facebook, although currently it's only the developers who can see them.

Chris · 1 year ago

While no doubt there are huge discrepancies between "Initial Interest Indications" (III's, perhaps) and "Ultimate Visit Indications" (UVI's, perhaps), a ratio that divides the latter into the former might be a useful metric of health. I would envision a slide (to the board or to investors) that showed a "comparables" chart of similar apps/services that showed the relative ratios of UVI's as a percentage of III's.

To inflate a service's "value" using only III's is indeed a "lie," but to discredit these factors as "useless" seems to discourage more creative (or retention-driving) efforts that leverage RSS, FB and/or Twitter/etc. If I were running or measuring a "Retention Marketing" team (which is a hard job), I would have one component of that team's bonus tied to the improvement of the aforementioned ratios.

fredwilson · 1 year ago

The daily actives/total subs number is interesting

But it tends to go down over time

My feed has been out there for almost five years

If I had started blogging six months ago, the ratio would be much higher

fred

seemsArtless · 1 year ago

But aren't the statistics useful when comparing one blog to another? It is valid to say that a blog with 133,000 RSS subscribers is twice as popular as one with 66,500 subscribers. Sure, you'd have to keep some factors in mind, like how long the blog has been around, but I think the comparison is relevant and useful.

fredwilson · 1 year ago

I don't think it's valid to say that a blog with 133k RSS subs is twice as
popular as a blog with 66.5k RSS subs.

I think you need to look at average daily readers (web+feed) to see which is
more popular

It's entirely possible that a blog with 66.5k RSS subs has more average
daily readers than this blog

fred

cfinke · 1 year ago

It sounds like FeedBurner needs to re-think how they calculate RSS subscribers - anyone who hasn't accessed the feed in over a month definitely should not be counted as a subscriber.

davidcushman · 1 year ago

amen Fred. The truly valuable are those who share your purpose and are responding to you right now.

Dario Salvelli · 1 year ago

I think that is important also how people spent time to follow. It's an important number, above all for the RSS subscribers.

Ken Yarmosh · 1 year ago

A technicality here, although I agree with your overall premise. Here is how FeedBurner (Google) describes "Feed Subscribers" for a single day subscriber total:

"FeedBurner’s subscriber count is based on an approximation of how many times your feed has been requested in a 24-hour period."

The key point is the 24-hour period. You will probably notice that the RSS subscriber one day count still drops on the weekend.

So, while I agree that reach is more important than subscribers, I think it is a bit much to say that the subscriber count is "basically useless." You can also look at your subscriber count over 7 / 30 days / all time, which then shows average subscribers during that time period.

fredwilson · 1 year ago

Yeah, but a lot of those requests are from software, not users

chudson · 1 year ago

I don't think I would say these statistics lie. I think they're just easy to misinterpret. In the earlier days of the web, people got really excited about pageviews. Then they realized there were more meaningful ways to track audience engagement (7-day actives, 30-day actives, unique users per month, etc) - crude tools like pageviews and installs are only good for first-order guesses as to how well a service is doing. I'd argue the statistics you cite (installs, followers, etc) allow lazy people to stay lazy if they choose to rely solely on those without asking more questions about the underlying level of activity.

centernetworks · 1 year ago

Great post Fred - the only thing that I would disagree with is that the numbers don't matter in the long run. I wish they didn't but from what I can tell, advertisers sometimes make decisions based on these "lies" - especially the RSS one for bloggers. Your analysis though is spot on. One additional note, just like I wrote about Friendfeed defaults, there are many RSS apps that provide a set of defaults - that provides probably the highest boost for some blogs.

I wrote about this a year ago on a post titled "are rss subscribers equal to hits" - rss subs as a metric reminds me of 1995 and the usage of "hits" as the publicly reported web metric.
http://www.centernetworks.com/are-rss-subscribe...

kidmercury · 1 year ago

IMO some better metrics:

% of traffic that comes from type-ins & bookmarks
viral metrics -- i.e. % of posts that go viral
average time on site
avg # of non-spam comments
percentage of posts made by passionate users (i.e. repeat commenters)

though ultimately i personally am not a huge metrics person and favor more intuitive stuff like feeling out which communities have the most passionate users.

Blogwatch · 1 year ago

Didn't FriendFeed add Robert Scoble (and a few other "A listers") to EVERYONES Friends list? Basically counting on people's laziness in not de-friending them, similar to "Tom" being everyone's default friend on MySpace?

scottshapiro · 1 year ago

totally - this definitely distorts the metrics as well

James Joyner · 1 year ago

Another thing that distorts RSS subscriber counts is automatic includes. Several of us are on various "recommended" lists put out by various readers and RSS interactive products, totally inflating our numbers despite, as you say, many of those people never reading, let alone engaging with, the blog.

Kyle · 1 year ago

Now that we know what doesn't matter - what are the 3 web statistics that "don't lie"? Or said another way, what are the 3 most important web statistics that my social media site should rely on?

fredwilson · 1 year ago

Daily active unique visitors
Average length of stay
% of return visitors

Those would be my votes

jer979 · 1 year ago

Kyle beat me to the point on the question, but looking at a data point of one (me) and my daily interaction with you.

I subscribe via RSS and read your posts pretty much daily, but I don't always go to your site. Some days I don't comment at all, today I'm commenting via disqus only. In all scenarios, I'm "engaged" w/the AVC brand.

So, while those three metrics are certainly important, it relies on the website as the destination. That flies in the face of the distributed web/syndication value prop.

You've posted before on the value of re-purposing and syndicating your content. There's got to be some metric for that.

blogcosm · 1 year ago

But the number of people who read this blog via the feed every day averages less than 4,000.

Do you have any details behind that number? I'm very interested in understanding where it comes from.

Scott

fredwilson · 1 year ago

Its the feedburner daily reach stat for this blog's feed

sent2null · 1 year ago

nailed it, good article.

spira · 1 year ago

I hate the phrase ""there are lies, damned lies, and statistics" - it's nonsense. "There are lies, damned lies, and words" is just as meaningful. Statistics, by definition, can't lie - they just are, just like words. It's people who lie.

Statistical illiteracy is a more widespread problem in our society than language illiteracy, but that's the fault of our educational systems that think it's far more important to teach high schoolers trigonometry than statistics. But that just means that more people can be fooled by statistics; it has nothing to do with the intrinsic nature of statistics.

So please try not to slander statistics in the future - they're victims here too.

And yes, I'm (mostly) serious)

Boudicca · 1 year ago

Spira

Sorry if you are a statistician and that phrase upsets you and I strongly agree that statistical illiteracy is a Huge problem in our society. But I have to disagree that the phrase in nonsense.

The line isn't "there are liars, damn liars and statisticians" : )

Of course it is people who lie - lies are told by people,damned lies are told by people and its people who lie with statistics (sometimes intentionally and sometimes simply because of incompetence).

fredwilson · 1 year ago

Thank you for the correction

As usual the commenters have the right answer

Jay Cuthrell · 1 year ago

This is also why 90% of the peer reviews of 50% of all studies show that 75% of most statistics are fabricated and the remaining 25% are mired in pseudoreplication.

And yes, I'm (mostly) joking.

Ken Yarmosh · 1 year ago

"Subscribers is not computed for browsers and bots that access your feed."

http://www.google.com/support/feedburner/bin/an...

(btw, this was supposed to follow in response to the above comment but it got placed at the end of the thread)

Hootan · 1 year ago

Fred, your post makes me think of web statistics in general, like monthly uniques. Entrepreneurs quote me "millions" all the time, but a quick check of alexa or quantcast reveals another story (I know those sites are rough estimators, but decent nevertheless).

I would imagine that the "big" numbers quoted by some of these new companies are meant to excite investors. I wonder whether a small but loyal following would be as exciting to investors as a massive but "undefined" following. The truth is more important, but in this day and age, it look like the market rewards huge growth in numbers, not quality of visitors.

Douglas Karr · 1 year ago

There are a few infamous bloggers who have exploited these vulnerabilities and become as large as their numbers said they previously were. I won't mention names here, but they've even bragged about what they did. The result? Bigger and better advertisers than they'd ever imagined with 6-digit incomes.

You and I may be aware of these, but advertisers are not.

Paul Edmondson · 1 year ago

One stat I've found interesting is a velocity rate. Take the number of subscribers of a feed or installs of an app and divide it by the number of days (time). That will give you a ratio that you can track over time. I use this for several things in our business. If you graph this over time, it can be telling.

fredwilson · 1 year ago

Totally agree

DorianBenkoil · 1 year ago

Paul, what's it tell you?

Chase Barfield · 1 year ago

Mr. Wilson - Very insightful and so true.

Vijay Goel, M.D. · 1 year ago

You've hit the lazy thing straight on. While I was at McKinsey, I did a ton of customer segment mining through existing company data trying to match business results with customer data. Its not that the data doesn't exist in most cases, its that people don't actually want to make the numbers mean something, because then they'll have to get off their rears to fix the issues that emerge.

It's important to track what's happening to individuals to really be able to understand what your customer base is doing (e.g., rapid subscribes may mask an unhealthy early attrition rate if you're only monitoring overall retention).

Its also important for the company to know which customers (and their activities) make them money and what causes that loyalty to disappear. Nothing like having customer service maximizing their handle time per average caller when you're an account decisionmaker thinking about renewing your relationship with a company...

Boudicca · 1 year ago

Love the opening line!
The question of how to quantify the number of people actually interact in a meaningful way is a big question made up of numerous pieces. I really like one of the pieces you brought up here - what is the ratio of of users/viewers and what are the variable that contribute to the variance in ratio between different sites.

Kalpesh Khivasara · 1 year ago

Hi Fred, couldn't agree more with your point. BTW, can you let me know how did you
"guess" that the people reading your feeds number less than 4000? I'd be very interested to know.

fredwilson · 1 year ago

Its not a guess

Feedburner reports daily reach numbers

Fred

john@doe.org · 1 year ago

I DISAGREE STRONGLY.

Two thumbs up for sexy seo commenter, I like your call sign/trademark. I am going to create my own called SEXY BEAST!

---

Anyways, back to the topic at hand. After reading all the comments, I have to disagree with you Fred.

STATS do not lie at all, they are what they are. It is the cultural perception to take it for its face value that has become the stinker. We could go a step further and include more detailed stats should an inquiring mind explore. Maybe have the feedburner count turn into a link where you can get far more detail stats for the public like someother companies currently offer. I think sitemeter.com, I am not to sure of the top of my head.

I think your argument though has good points and got me thinking about influence. One solution can be the one I describe above were details stats are always are available for the curious minds where it is simple and easy to access.

This then got me thinking about how influential a person is within their domain. There stats are insight, but what it comes down to is how much influence do you truly have.

I was just reading about that company earlier this week about how they are working with Meebo to identify how much influence a certain individual has for ad purposes. I think that is just brilliant that a company is finally pursuing this next step in marketing, BRILLIANT!

Breaking it down to the person :-)

Sorry for the rants, great post!

AngelaE8654 · 1 year ago

I am still just learning about all these social bookmarking places. LOL



Angela

Chris · 1 year ago

This makes me feel good since I'm not really active in Twitter and I just opened a Facebook account. Now at least I know that the people who visit my blog, albiet modestly small, are quality visitors. But don't you have to drive a lot of traffic to your site in order to attract a few good ones?

Melanie Baker · 1 year ago

Hi Fred - Thanks so much for this, and excellent timing! Served as the perfect example to illustrate some work we've been doing on measuring social engagement as opposed to just "storyless" numbers: Storytelling ROI: social engagement metrics for Marketing & Social Media bloggers (As a side note, your overall engagement score currently is 28919, very respectable.)

I think sorting out all these metrics and providing relevant context for people is still in embryonic stages, and there is so much potential for really cool developments. At the same time, though, the true connections between people, the incredible value of what you can learn from others -- I don't think stuff like that will ever be accurately analyzed. Nor does it need to be. :)

Hashim Warren · 1 year ago

another number that lies - unique visitors and pageviews.

What really matters is engagement. What really matters is loyal visitors.

Sagar · 1 year ago

i'm a facebook developer and yes, facebook install numbers are HORRIBLE...hahah

DorianBenkoil · 1 year ago

One thing that bothers me a tad about this is it seems to assume a user is a user, period. That it's all about aggregate numbers. That may be true in mass media (Desperate Housewives, the Super Bowl), but in a targeted medium, 3,000 of the "right" people may be more valuable than 3 million of the general public.

fredwilson · 1 year ago

That is very true and I am particularly blessed with the quality of the people who read this blog

Although I do run advertising on this blog and in the feed, which goes to charity, the real value is in the relationships it has allowed me to develop with the readers

DorianBenkoil · 1 year ago

Thanks for the nice words, Fred. I was talking in purely economic terms, as well.

Just one example: For IT-oriented lead generation, each individual lead can be worth hundreds of dollars, and it's better to have a few high-qualified leads than a mass of unqualified.

hannes · 1 year ago

Interesting points - and very true.

But don't forget that your 877 followers on Tumblr might only view your content on the Tumblr dashboard without going to your site/tumblelog - so you got probably a bit more than 250 people reading the content you post on Tumblr.

Antman · 1 year ago

But they make real good marketing numbers . . . perception is reality and for many they create one hell of a perception.

Marcel LeBrun · 1 year ago

To use a high school science analogy: it is like the difference between potential energy and kinetic (or motion) energy. Someone with 5,152 followers on twitter has a greater potential for engagement than someone with 100 followers, but not the guarantee. The right way to measure engagement or influence is by measuring the actual conversational dynamics which truly indicate engagement (comments, votes, bookmarks, links, etc.) around a topic or post.

I think your point is especially visible with services where users/followers/subscribers constantly accumulate (like the facebook installs - since users may never remove them, even though they are rarely used). Good post.

ajayinsead03 · 1 year ago

I just stumbled at your blog. Although I agree with the general idea behind your ascertain: I dont agree that the numbers are completely useless or lies -- misleading - perhaps. Perhaps, because it depends on where you want these numbers to LEAD you ?

While many have given different analogies..How about an analogy, from the world of Finance - from options.. Real Options..

When people install an app, or tune in - > its an option they excersize - are they not? If yes, then that option has to be worth something... is it not? Think of the economic value ( not the monetization ) that such an option entails: * X millions have downloaded this bar --> u could be the next :( *

My 2 cents

Cheers,
ajay

SexySEO · 1 year ago

Figures by themselves don't mean anything: figures are just figures. And statistics were (and always are) the way of .... manipulation of figures :)))

Leonard Boord · 1 year ago

Not so quick here... I agree RSS and Followers. But...

Apps: When a user downloads and app he leaves the door open for more inbound communications. You can now devise different strategies to recoup the user, at a very low cost. So it's not worthless, or meaningless your foot is in the door.