Quantifying Facebook Engagement: More than Just Counting Fans and Likes

[#1]

Michael Wu, Ph.D. is Lithium's Principal Scientist of Analytics, digging into the complex dynamics of social interaction and group behavior in online communities and social networks.

Michael was voted a 2010 Influential Leader by CRM Magazine for his work on predictive social analytics and its application to Social CRM. He's a regular blogger on the Lithosphere's Building Community blog and previously wrote in the Analytic Science blog. You can follow him on Twitter at mich8elwu.

---

Since our annual Lithium Network Conference (LiNC 2011) is this week, I’m going to take a little detour from our gamification journey. Rather than trying wrap up my mini-series on gamification, I am going to show you some exciting data from a little secret project (code name Project Atlas) that I’ve been working on. The details of this project will be revealed at LiNC, so stay tuned. And if you get the chance, come join us at LiNC.

A few months ago, I was posed an interesting question: “Can we quantify the level engagement on a Facebook fan page that is a little deeper than just the number of fans or likes?” My default answer was, “Probably... give me some data and some time to play with it.” Subsequently, I got a small data set from our Social Media Monitoring (SMM) platform.

This dataset consisted of ~39 million participation records from 11+ million unique fans on 3,050 fan pages spanning from Feb-2009 to May-2011. This may sound like a lot of data, but really it is a tiny data set compare to the data volume processed by our SMM platform. Today, I will show you some of the results from playing with this small data set.

A Facebook Fan Page is Structurally a Community

Before I go into the data, I want to take you back to a mini-series I wrote on Cyber-Anthropology. It examines social media from a relational perspective and describes the complementary roles of social networks and communities in the development of interpersonal relationships throughout human history. In that mini-series, I characterized the structural and functional differences between social networks and communities. If you missed those posts, I will summarize the key point here (feel free to skip ahead to the next section if you are familiar with the structural and functional differences): 

Social Networks:

1. Held together by pre-established interpersonal relationships between individuals
2. People know everyone that is in their social network (i.e. direct connections)
3. Each person has one social network. But a person can have many different social graphs depending on which relationship we want to focus on (see Social Network Analysis 101)
4. The primary anthropological function of social networks is to maintain people’s pre-existing relationships
5. Social networks have a network structure
6. Within each person’s social networks there are sub-communities with different interests

Communities:

1. Held together by the common interests of a group of people
2. Pre-existing relationships may exist, but are not required, so new members generally do not know any or most of the people in the community
3. Any one person may be part of many communities at a given time
4. The primary anthropological function of communities is to develop people’s weak ties into strong relationships
5. Communities have overlapping and nested structures
6. Within each communities, social networks develop naturally as people build their tie strength

From this perspective, we can see that although Facebook is definitely a social network, a fan page is structurally more like a community. It is held together by the common interest (e.g. around a brand), and most of the fans don’t know each other when they join. Moreover, people can be part of many fan pages at any given time. So a fan page is really a community within the Facebook social network.

The Depth of Engagement on Your Fan Page

If we are treating a fan page as a community, how can we measure the engagement of fans on that fan page? Well, to start with, clearly you need to have fans! People realized this and many have used fan count as a way to measure engagement, but fan count is in reality, like the total registration or membership of your fan page. When someone liked your fan page, they merely joined your fan page as a community member. However, as I described in an earlier post (i.e. No Game, No Gain: Realizing the ROI of Your Facebook Fans), the true value of your fans cannot be realized until they take actions to interact with you and with others. Therefore fan count is only the most superficial characterization of engagement, because it says nothing about the fans’ subsequent action and their interactions.

I consider fan count the level 0 engagement metric. Figure 0 shows the distribution of fan counts in my data set. We see a wide variety of pages with fans counts spanning over 7 orders of magnitude (from tens to 39 millions) with a median level around 3,400. Note: Most of the distributions we deal with are power-law distributed and must be displayed on a logarithmic scale.

I am going to describe several deeper engagement metrics for your fan page and show you some data. I will focus on the most visible action (i.e. posting a message or comment) for now, and describe other actions, such as likes, in subsequent articles. For the rest of this article, I’ve used fan pages that have 1,000 posts or more.

If fan count is level 0, then level 1 is should focus on the active fans of your fan page (i.e. those who posted something). With all things being equal, it is clear that a fan who posts something is probably more engaged than a fan who doesn’t post. Figure 1a shows the distribution of active fans across all the fan pages I examined. The median number of active fans is around 2,900. However, the number of active fans may be biased by the age of the fan page. Younger fan pages that haven’t been around long enough may not have the time needed to develop a large active-fan base. Figure 1b shows the distribution of age normalized active fans. After normalization, the median level for the number of active fans per day is only about 19, but the most active pages can still have up to 2,200 active fans per day.

As we have learned from observing the 90-9-1 rule, the majority of your fans are inactive at any given time and only a small fraction of your fans are actively participating. Figure 1c shows that a conservative estimate of the fractional active fans (i.e. active fans divided by the total fans). This distribution has a median value of 3.45%. So on average, only 3.45% of your fans are actively engaging (i.e. posting). This is slightly less than what the 90-9-1 rule would have predicted, but the distribution definitely covers the expected 10% active fans with a wide margin.

If your fans are sufficiently engaged to post messages on your fan page, then the next level of engagement (level 2) digs deeper and looks at what fraction of the posts are interactive. That is, what fraction of posts have a comment? Figure 2 shows the distribution of interactive posts. The data shows that a significant portion of the posts on fan pages are not interactive. The median level for the fraction of interactive posts is about 66.8% (with a pretty large standard deviation). That means on average, over 1/3 of the posts on any fan page will never get a response before disappearing down the stream and off the wall.

Conclusion

Alright, that is probably enough data for you today. The full spectrum of engagement can go very deep and we need to quantify each level of engagement to get a full picture of how well your fans engage. Unfortunately, fan count is merely the shallowest (level 0) of all engagement metrics and doesn’t tell you very much. We can go much, much deeper.

In this post I covered the first two levels, but there are actually eight levels of engagement, and each level goes deeper than the previous. I will cover the deeper level engagement metrics in the subsequent posts. After I introduce all the components, I will show you how to combine these different engagement levels into a single score that quantifies the overall engagement of your fans. But for now:

• Level 0: Total fan counts
• Level 1: Active fans
• Level 2: Interactivity through comments

I must say that one of the best things that Lithium did for me, as a scientist, is acquiring Scout Labs (now Lithium SMM). Through our SMM platform, I basically get an unlimited supply of social and behavior data. To me, that’s data heaven! Project Atlas is just the beginning of my intellectual playground. I certainly look forward to sharing more data and deeper insights in the future.

 

By the way, after LiNC, I will be traveling in Europe starting next week for about 3 weeks. I will be participating in several speaking engagements, interviews, launch events, and a little bit of vacation between them. So I apologize in advance if I am unusually slow in responding. Coming up next week are:

1. May 24th: European Customer Experience World 2011
2. May 26th: Digital Surrey on the Science of Gamification

If you are around London, I’d love to meet you. See you later and stay tuned for deeper engagement!

[#2]

Level 3 - density of the community of 'posters'?

[#3]

Helllo Laurence,

Thanks for the comment.

That is a good guess. But level 3 actually tries to quantify the amount of interaction in each interactive post (i.e. post with comments). Remember you have to go deeper each level. Please stay tune for deeper engagement metrics on your facebook fan page.

Hope to see you next time.

[#4]

Great Post Mike! An cool next step would be to measure brand advocacy of community members based on the nature of their post (positive/negative or maybe a likert scale). Then follow that person out of the community to see their social network and see how many individuals they personally know are part of the brand community. You could then extrapolate that person's primary influence on the brand (e.g. people who they know in the community) and community influence (e.g. people who they don't know but who responded to their post) within the Facebook Fan Page community.

[#5]

Hello Doug,

Thank you for the comment.

The Facebook Engagement Index (FEI) is already developed. I just haven't got the chance to write up all the detail and post them yet. The intent of this index is to measure engagement: how fans engage, how much, how deep, etc. It is NOT supposed to measure the influence of fans. For that, there are other analytics that I've developed last years. If you are interested, you can check out this chapter, which that contains links to some of my early works on influence. For a complex list of blog post, you can access them through the influencer label on this blog.

And we are working on implementing some of these influencer algorithms for the social web. We are just need to build the computing hardware infrastructure that enables such large computation. So they will come soon, and I will write about them as they get rolled out. So that is a great suggestion, and it is already on its way, and it will be coming soon.  :-)

Thanks again for stopping by and commenting. See you next time.

[#6]

Hi Mike!

Your post is very relevant to our latest research. We've analyzed contagion of the content on FB fan-pages walls (2 autobrands). Additionally we've made content analysis of posts to understand what themes, media-types (video, foto, text etc) in the posts stimulate fans to comment and like content (some kind of engagement).

But the next big question is to understand how relationships (social side of contagion of the content) between fans can influence engagement of fans. I see value of your first step in demonstraiting real, middle level of engagement on fan-pages (fast 4% in your results). And this level can say me what means my data about commenting and liking posts on walls. 

I am really interesting in next steps!

[#7]

Hello Andrei,

Thank you for the comment.

I'm glad that you are finding my work relevant to your research. I hope that you will find my later posts more useful, as they do go very deep. Much much deeper. It is great that you are analyzing the content type too as different content type drives different levels of engagement. But as in my reply to Doug (above), the FEI i

s design specifically to measure engagement, not influence. For engagement, there are other influence analytics that I've developed already.

And I will say that the two deepest level of engagement metrics (level 7 and level 8) has to do with how relationship develop between fans. However, I must ask you to wait a few week before I reveal more details about those metrics. So stay tuned.

Thanks again for stopping by. I'll see you later.

[#8]

Mike - A very good post demonstrating the value/differences of both engagement between fans and the page and fans with other fans. I'm curious, has there been any work done to take into account the impact of prior relationships (people who are also friends outside of a fan page) on fan page activity?

Thx again.

 Fred

[#9]

Hello Fred,

Thank you for the nice comment.

Very interesting question. It is one that I've been asking a year ago when I write that mini-series on cyber-anthropology. All it comes down to is getting the friend data from Facebook. However, Facebook doesn't give out the social graph data. So we have no way of knowing who are friends with whom on a fan page. That is we don't know which pairs of fans has pre-existing relationships.

However, strickly from a social psychology perspective. People tend to engage more and engage at a deeper level with people they already know. That is one of the reason why Facebook is so sticky and is able to take so much of our attention. Unfortunately we cannot get the data from Facebook to prove this.

Anyway, thank you for stopping by and commenting. I hope to see you again next time.

[#10]

Mike - I understand completely the Facebook "who are friends" issue. However, two thoughts come to mind.

The first is looking for conversations between fans of a page and seeking out other fan pages where they are both fans and have engaged (a likely indicator of a deeper relationship).

The second thought is that (yes, this is not simple task) it might be possible to cross reference Facebook patrons with other social sites, such as Twitter/FriendFeed, and identify people who are both Facebook users AND follow each other on these other social networks. Even a purely random sample would be an interesting data point. I know a non-trivial number of people within my own social graph on Twitter that have migrated to more discussion-oriented groups on Facebook to deepen their conversations and level of "mutual interest" discussions, so such dual-membership relationships might be a way to provide a baseline sample (I'm constantly flooded with Twitter/LinkedIn associates who want to be friends on Facebook as well). Just a thought.

Best,

Fred

[#11]

Fred, Mike:

Data consistency for identities across communities is a challenge. However, the proliferation of single-signon capability (like the Connect with Facebook you have on this site) may lead to interesting correlations. 

-Alan

[#12]

Hi Mike!

Thx!

My little research is more about engagement (driving actions of fans through content value) as about influence (driving actions of fans through actions of another fans). 

Explaine me please how do you distinguish engagement and influence?

And where can I know more about FEI?

[#13]

Hello Fred,

Thank you for coming back and continuing the conversation.

Thank you for the suggestion. However, the first idea is probably not going to work, because the homophily / tie strength relationship is not symmetric. People who are connected by deeper (stronger) relationships tend to have similar interest (due to homophily), but people have same interest can have a whole spectrum of relationship from very strong to absolutely no relationship at all.

In other words, deep and strong relationship implies similar interest, but the converse is NOT true mathematically (from mathematical logic). That is, similar interest does NOT necessarily and cannot imply deeper/stronger relationship. So using overlapping fan pages as an indication of greater similarity of interest cannot infer anything about the strength of relationship between the two person. In general, as we know from sociology, many interpersonal relationships are not symmetric, and causal inference can only be apply in one direction and not the converse.

The second idea may work, but as you realize, it is a lot of work, and probably not worth the trouble to go get all those data just to infer Facebook friendship, which is really an aggregation of many relationships: childhood friends, relatives, schoolmates, colleagues, etc. (see Social Network Analysis 101). I think I would have to think of a better, simpler and more elegant solution before I will move forth to address the question of our interest.

Anyway, thank you for the discussion. Hope to see you again.

[#14]

Hello Alan,

Thank you for the comment.

Single Sign-On (SSO) can certainly help solve the entity disambiguation problem, but it still cannot identify relationships, between two entities. That is, you can know whether two user profiles represents the same entity or different entities, but it doesn't tell you anything about the relationship between one entity to another entity.

Still, I appreciate your suggestion. See you again next time.

[#15]

Hello Andrei,

Concerning the Facebook Engagement Index (FEI), your inquiry is probably one that many people want to ask also. Unfortunately... I don't think you are going to find more info about it elsewhere at this point.

It is a novel analytic development, and I haven't came across anything remotely like it during the R&D of the FEI. However, I am just so busy with my daily engineering work, architecture responsibilities, external speaking engagements, etc. that I haven't had much time to write it up yet. So please be patient and give me a little bit of time. It will be reveal here (on my blog) slowly over the next few weeks/months bit by bit with data and all the details you want to know.

Stay tuned. And please come back next time for more detail about the FEI.

[#16]

As always, an excellent post with lots of good information.  However, one thing does puzzle me.  You said your data set consisted of 11+million unique fans, yet you also said that the fan counts for the pages in your data set ranged from tens to 39 million.  How is it possible for a page to have more than 11 million fans if that is the total number of unique fans in your data set?

Thanks,

Kaushal

[#17]

Hello Kaushal,

Thank you for commenting. I am in London now. In fact, I'm at Edelman Digital's London office on Victoria St.

You posed a great question. Glad to see that someone is paying attention to the details.

Let me explain the data set a bit. The data set consist of records of all the participations from 11+ million unique fans, so these are actually the activity records from the fans. Therefore, the unique fans we are talking about here are really the active fans. The distribution of fan counts across all the pages is not obtained by summing the active fans within a particular fan page. They are obtained from the fan count that showed up on the fan page. So the 39 million fans on the most popular fan page consist of active as well as the inactive fans. That is why the fan counts are in general much larger than the active fans in our data set.

I hope this explains the data a bit more. I'm sure this must be a confusing point for a lot of people, so really havce to thank you for asking the question. Hope to see you again next time.

[#18]

Hello Mike, 

Enjoyed the read. Looking forward to the subsequent posts in this series. I agree with you that there are many more factors that can be looked at in addition to the # of fans, # of active fans, & # of interactive posts -- like the breadth & depth of interaction, quality etc.

To me it is interesting to gauge the level of engagement & quantify it; equally fascinating is how that "metric" can be leveraged for other assessments like customer ltv, retention modeling etc. 

Ned

[#19]

Hello Ned,

Glad to see you back. I'm actually in Florence, Italy now. So I give a quick reply, as Wifi is usually not free here. I actually quite surprise by this, but i'm spoiled by all the free wifi in US.

Anyway, you are right. There are a lot more metrics that quantifies deeper levels of engagement, and I will show you more data on them after I return to the states. Specifically, both quantity, breadth, depth and objectively quantifiable qualities will all be covered. So please do come back in a couple weeks.

Once we have a good reliable metric for quantifying engagement, then, we can do a correlation study to see if there are any correlation with customer loyalty, retention, lifetime value, referral value, network value, etc. So that is something to be explore later. That is an excellent idea.

Thank you for the comment and see you again next time.