A VC - Latest Comments in The Cohort Analysis

Re: The Cohort Analysis

Christine Brumback — Fri, 23 Oct 2009 23:23:41 -0000

having only seen cohort data in more of a research setting with much smaller user groups, i love seeing this for such large volumes of customer usage data.

since Twitter is a network, i'd wonder how the the cohorts behave in relation to the network size increasing...and also, where are the pre 2007 folks? (aka the ones who were mocked for tweeting about burrito lunches in south park? )

Re: The Cohort Analysis

rfreeborn — Sat, 10 Oct 2009 11:08:44 -0000

What I don't see within these graphs, and don't know cohort analysis well enough, is to understand how this relates to correlation and/or causation. The fact that these groups joined @ the same time has nothing, other then that date in common, to show the who/what/where/when/why of being engaged with any service.

Another caution I would think of re: using cohort analysis when in startup mode is the sample size. What's the number of people in that Jan 2007 group vs. April 2009? Were there any comments made during the meeting re: applying a weighted average to the analysis? January 2007 has a cohort of 10 early adopters and April has 10,000 "mainstream" users.....

Re: The Cohort Analysis

Ivan Kirigin — Wed, 07 Oct 2009 23:59:28 -0000

How do you track loyalty? And how would you track it for those referred from google? Does chartbeat or google analytics do cohort analysis? Have you ever seen auto-cohorts for sites like this that don't really have a sign in - but just index by the first time the machine has visited the site and their referrer?

Re: The Cohort Analysis

Jeff Pester — Wed, 07 Oct 2009 22:31:48 -0000

I get that, I just think you'll get 10x the participation from users if you don't require them to create lists in the first place. Just let them start out by being able to mark tweets as "I like" and accounts as "I recommend" and let them group things in to lists when it makes sense to them.

As an app developer I'm much more interested in getting access via the API to those two silos of information than I am to some much smaller data set of user-defined lists, for a variety of reasons.

Again, I think lists are fine. I just don't think that by themselves they're going to deliver on their promise for either end-users or app developers. The bar is unnecessarily too high for initial user engagement.

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 21:14:05 -0000

I can't wait to see all the list tools that are built on the lists api

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:56:55 -0000

Thanks andrew. I ran out of time on this one. That's the hazards of posting in the morning when I have to be done at 7am so I can wake the kids, shower, and get to work

Re: The Cohort Analysis

daryn — Wed, 07 Oct 2009 20:52:39 -0000

yes :)

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:52:30 -0000

Social media mining might be helpful

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:51:13 -0000

On this blog twitter is king in loyalty, volume of traffic, and time of visit and pages viewed but that might have something to do with my relationship with twitter

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:43:56 -0000

Oooh. The wrong side of the corporate firewall. I'm so going to use that line john

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:41:07 -0000

Having dave on your board is probably entertaining in addition to being valuable

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:39:39 -0000

That's exactly what twitter is doing. They create the lists api and the ecosystem develops all sorts of ways to crate them

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:37:58 -0000

Lists is an api like most things in twitter. Many people will create tools to build lists in all sorts of ways, including all the third party apps

Re: The Cohort Analysis

fredwilson — Wed, 07 Oct 2009 20:37:17 -0000

Two great comments in one fraser

I think you can easily get to complicated with measurement

Re: The Cohort Analysis

Michael Broukhim — Wed, 07 Oct 2009 18:08:27 -0000

Great thoughts here Joe. Would love to hear any more ideas you'd be looking for; we'd be happy to incorporate into Twillist (http://alpha.twillist.com) ...feel free to email hello@twillist.com

Re: The Cohort Analysis

Michael Broukhim — Wed, 07 Oct 2009 17:58:32 -0000

This is exactly what we're aiming to do with Twillist (http://twillist.com); we'll be adding a layer of meta-data to lists and automation / suggestion to list creation & discovery. We're very bullish on the possibilities for discovery that Twitter lists creates.

Great post Fred on cohort analysis... it's something we've been working on perfecting @totspot... no easy task to nail this type of analysis, but leads to very rich understanding of your users and changes to your app.

Update: btw, here's the alpha link to Twillist: http://alpha.twillist.com

Re: The Cohort Analysis

leigh — Wed, 07 Oct 2009 06:31:07 -0000

Great content in Dave's presentation with a lot of learning that large brands can utilize as well (on my to do list to write a presentation of lessons Corporations can learn from start ups). But man, he needs to get some SERIOUS powerpoint creative help (Dave please - no more colour coding your content! :)

Re: The Cohort Analysis

andrewparker — Tue, 06 Oct 2009 17:37:18 -0000

I don't think it's obvious from your post why this type of analysis is so important.

When a development team makes design choices and then goes back to review traffic data to evaluate the success of their choices, they have to use a cohort analysis for their analysis to be accurate. You can't just look back over 2 months worth of uniques in Google Analytics to determine if a change implemented 4 weeks ago was actually successful. By using a cohort analysis you can isolate the variable of where a user is in their lifecycle of using their service, which allows you to more accurate assess how new features affect users.

Additionally a cohort analysis is a great way to assess the lifetime value of an acquired user. As you look at older cohorts, you can measure, on average, how long a user will stick with your service, and, depending on your business model, how much a user is worth to you. Once you know that number, you're golden because you know your allowable you can spend in marketing to acquire new users sustainably.

I agree that cohort analysis are our firm's favorite measurement, and the reasons above just scratch the surface of the valuable conclusion you can draw from a cohort analysis.

Re: The Cohort Analysis

ShanaC — Tue, 06 Oct 2009 16:23:41 -0000

I second this. I'm not the early user. I'm still having trouble figuring out what is twitter doing to fill my needs (oddly right now it is ridiculous real time search)

Yet I know it should be more than that...without good metrics, I am unsure what to mold it to.

Re: The Cohort Analysis

ShanaC — Tue, 06 Oct 2009 16:21:24 -0000

It's extremely difficult to get real world feedback longitudinally of the people who dislike you- but those people are often the most useful people. Especially if they are part of the target group you are going for.

How do you get their advice? I'm finding interviewing people on the street is not enough...

Re: The Cohort Analysis

ceonyc — Tue, 06 Oct 2009 16:15:25 -0000

I try to very actively find relevant folks, so it's pretty balanced.

Re: The Cohort Analysis

Ivan Kirigin — Tue, 06 Oct 2009 14:47:30 -0000

I just read this, and liked it. Then I read this story:
http://mashable.com/2009/10...

...which ranks social sites by how loyal the traffic they send is. This is related to your points about twitter vs. google in referrers - and is relevant here in talking about cohort analysis.

I would imagine the readers for this blog come in bumps when you have a popular post. How many of those readers from twitter vs. google vs. some other service stay to read more, or subscribe to your RSS feed, etc.

I've been designing a blog platform for myself (because I can), and will make a it a lot like a webapp - with sign on, user tracking, custom tools etc.

It will be interesting to integrate http://mixpanel.com and some other tools to track how readers on a blog behave.

Re: The Cohort Analysis

Jason — Tue, 06 Oct 2009 14:00:29 -0000

timing of this post is impeccable, thanks for the reminder!

Re: The Cohort Analysis

John Mahoney — Tue, 06 Oct 2009 12:28:29 -0000

Despite the tongue in cheek tone, its a great point Andy. Any of the large search engines/social communication platforms have incredible insight in both macro and micro trends. They are certainly predictive, with accuracy improving as the population sample increases. The statistics collected by any of these firms could provide incredible actionable intelligence if applied externally to markets or investments. In some ways the ultimate "insider information" is sitting on the wrong side of the corporate firewall.

Imagine if you could build your own analytics to mine it...

-john

Re: The Cohort Analysis

andyswan — Tue, 06 Oct 2009 12:19:28 -0000

Really great post. The only downside is the inescapable feeling that at some point in the not-too-distant future, Google and Twitter are going to know what I'm going to do before I do.