After 3 weeks of Health Factors, I guess (from the fact that my last post received no kudos) you are probably a little fed up with it. So let's take a break!

It's been a while since we talk about rank design, so this is a good time to revisit the topic.  This is the 4th article in the miniseries on designing the optimal ranking structure for your community. Previous blog articles from this miniseries can be found here: 

1. Spacing the rungs of your ranking ladder

2. Know your superusers!

3. Flow with your most prolific superusers

Give your superusers a little surprise

Part of the fun and the challenge in gaming is the unpredictable elements in games. The player can never truly know the outcome of his play. When the gamer has just figured out the game, he moves into the control state. When he is able to predict the outcome, the game is probably too easy for him. And soon he will find the game boring and move onto something more challenging. Likewise, a fun and challenging ranking ladder should be cryptic, and it is best when there are some elements of surprise built in. In a previous post, I talked about switching after year 2 to a very regular and predictable arithmetic progression (a.k.a. linear progression) for our ranking ladder. The problem is that your smart superusers will most likely figure out this ranking scheme.

So, how do you keep the superusers engaged under such a predictable (boring) ranking criteria? I will describe 4 things that you can do to spice up your ranking ladder even when it is a predictable and boring arithmetic progression.

1. Name your ranks creatively.

It is inevitable that some superusers will eventually figure out the post requirement for promotion, and will expect a promotion at the right time. But if you name your ranks cleverly, they still won't know what they are going to get next. So, don't name your ranks in any obvious progression. For example, a terrible choice of rank names would be bronze, silver, etc. Without even writing it down, it is blatantly obvious that the one after silver is going to be gold, then probably platinum, titanium, and then diamond. Give your superusers some serendipitous joy when they get promoted and use your imagination when naming your ranks.

2. Make some noise.

For statisticians, there is a simple trick to make things less predictable. Just add some noise (or randomness)! Rather than following any formula strictly, you just need to randomly jitter the number a little bit. For example, it is very easy for anyone to figure out the pattern in the following linear progression: 10, 20, 30, 40, 50, 60, 70, etc. By add some random noise, the sequence becomes much harder to predict: 11, 18, 32, 41, 49, 63, 74, etc. Notice that the challenges between the ranks will remain roughly the same (compare the figure of the jagged ranking criteria here with the figure of the smooth ranking criteria from my previous blog post). The shape is virtually the same, but this one is much harder to figure out. You can start jittering the post criteria from the first rank if you wish, but please be sure to "make some noise" when you switch over to the highly regular linear progression.

3. Privileges do matter.

Even with added noise and the most humorous and interesting rank names, superusers may still get tired of the routine rank changes without other incentives. This is especially true after you switch the ranking criteria over to the highly predictable linear progressions to avoid over challenging your superusers. If you follow the recommendation in this miniseries of blogs, this should happen roughly 2 years after the superuser first participated in the community. After 2 years, don't you think it is time to show your superusers that you recognize how special they are? So tell them (it only takes an email) and grant them access and customization privileges that are not available to others. Start with something simple, such as allowing your superusers to use a personalized icon. Then as they move up the rather boring rank ladder, they will be intermittently rewarded with different privileges that are totally unpredictable.

Attaching special privileges to a rank through our permission system is an extremely effective way to engaging your superusers. However, you might want to let time test their loyalty and goodwill before giving out too many privileges. Therefore the perfect time to give out special privileges is after the superusers have moved through the first 24 ranks. However, if you really trust your superusers, you may do this earlier on your ranking ladder. This also applies when switching your superusers into a MVP program.

4. Show your trust.

An important reason to give special permissions to your superusers is to establish trust. A deep relationship built on trust is likely to be more lasting than a superficial relationship that is built on other incentives. So show your superusers that you trust them. Invite them to beta test programs in private boards and treat them as your most valuable assets. Give them power to take action in the community, such as moving posts and perhaps deleting inappropriate content. Let them help you moderate your community. Can you really trust the superusers with that much power? I believe that you should. Because it is highly improbable that a superuser would just throw away all the reputations and privileges they've earned through hard work over long period of time. The more they have invested, the more they will treasure their unique social status.

Now that your rank ladder is all spiced up, you should be able to keep your superusers engaged "indefinitely" (contrast the above figure with the figure from my previous blog post). Next time we'll wrap up with the grand finale: synthesizing all the rank design principles we've explored to build a complete ranking structure. Stay tuned at mich8elwu.

Last time we talk about the content health factor and how we plan to reformulate it to enable drill down capability for the health factors and CHI. This time, I will explore the third health factor: Members.

The Members Health Factor

After achieving a steady stream of traffic and copious amount of high-value content, a healthy community should accumulate registered members, grow, reach critical mass, and then become self-sufficient and self-sustained. The members health factor is intended to measure the growth rate of the community.

Recall that traffic is a measure of passive engagement, and content is a measure of active participation. In Lithium-powered communities, login is not required for passive engagement, but it is required for active participation. This means that users must register to become members of the community and must sign in before they can post. However, because people tend not to register  or log in unless they have to, the members health factor is good estimate for the conversion rate from passive engagement to active participation.

What about Member Churns and Lurkers

Although growth is a sign of a healthy community, member attrition is inevitable. Some users register merely to post a question, and they will leave as soon as they have the solution to their problem. Therefore, even healthy communities will have some natural member churn rate. But a high attrition rate is indicative of poor community health because the community is not able to retain members and grow its active core of user base. This issue has been brought to my attention at the Persuasive2009 conference.

Another question I often hear is that "what should we do about lurkers?" If you are not familiar with social media you might think that lurking is an undesirable behavior. But in a series of studies, Prof. Blair Nonnecke and Prof. Jenny Preece have shown that lurkers do provide value, and they are important to the success of community. So let your lurkers lurk! When the time comes--when they have a specific question to ask or some information to contribute--some of them will de-lurk and become members.

The Current and Future of Members

Currently, the members health factor is the same as the weekly registration metric, so it only measures the passive to active conversion rate. To get a complete picture of community health, it is important to track member attrition in addition to member accumulation. So in our next iteration of CHI, the members health factor will account for member churn.

This covers the three diagnostic health factors: Traffic, Content, and Members. They are called diagnostic because they describe the result of user behaviors and tell you the current health of the community. As such, they are lagging indicators of community health. Next time, we will start with the predictive health factors: Liveliness, Interaction, and Responsiveness. They are referred to as predictive because they describe the intrinsic social dynamics within the community, so they tend to be leading indicators of community health.

Let me close by reaffirming that all comments are welcome, so let me know what's on your mind. We do take your comments seriously, and you can help us make CHI more accurate and better suited to everyone's needs.

My previous post focused on the "traffic" health factor - where it is today and where it's headed in the near future in terms of reformulation. Today I'll be talking about another health factor: Content.

The Content Health Factor

Once critical mass is reached in terms of human traffic, the next thing to focus on is building content within the community. Visitors won't return time and time again without an abundance of interesting, useful and highly desirable content. The content health factor is a measure of both the quantity and the quality of the messages posted within your community. Contrary to traffic, which measures the passive engagement of visitors, content measures the active participation of your community members. Because posting (whether it is a message, a reply, or a comment) adds consumable information that is persistent within the community. This is a form of active participation.

Measuring the Quantity and Quality of the Posts

The post count metric provides a straightforward measure of the quantity of posts, but how do you measure their quality? We leave this decision to the readers. Using a marketplace metaphor, when the number of consumers (readers) in a community is large, the "economics" of the community can give an accurate estimate of the relative demand (whether they are useful or interesting) for the posts. Since the demand for a post strongly correlate with its viewership, the demand for a post must be reflected in the page view metric. However, highly viewed pages tend to draw more random views. This snowball effect will inflate the estimate of consumer demand. Therefore, the post quality can be approximated by a dampened version of the page view metric, which we call viewership.

The Current and Future of Content

In the current implementation of CHI, post counts and page views are aggregated over the weekly window, and then computed at the community level. However, this computation prohibits any drill down capability for this health factor. Yet, drilling down to a category or a board and seeing the content health factor at different hierarchy of the community can provide actionable intelligence for the community manager. Because this is a common use-case, and personally I've been asked this specific question at our customer conference earlier this year, I will make sure that we address this issue in the next reformulation of CHI. This drill-down view can be achieved by computing the product before the aggregation.

Can you guess what is coming? Yes, we will talk about the member health factor next. In the mean time, please do tell me if there is a feature you want with regards to measuring community health. Stay tuned at mich8elwu.

Welcome back. I hope everyone had a relaxing Fourth of July holiday. Last week, Lithium delivered a new report, the community health report, to our clients. This report contains two important pieces of new information: 

1. The community health index (CHI): a score that reflects the "health" of your community.
2. The community health compass: a radar chart that shows the relative value of the six health factors (traffic, content, members, liveliness, interaction, and responsiveness) that went into the computation of CHI.

Earlier this year we published a whitepaper on CHI. Since then, however, we've refined the CHI algorithm based on data from real-world use cases along with customer feedback. So for this post, I'd like to talk about the current state and future direction of CHI.

The Traffic Health Factor

Let me begin with the health factor you are most familiar with: Traffic. This health factor is most important when you are launching a community. If no one comes to your new community, the community will fail. As the community matures and acquires a large core of active users, the effect of traffic may become less important. This is because traffic reflects only the passive engagement of the visitors with the community. In a public community, visitors can accomplish quite a bit without any active contribution. For example, they can read messages, search for answers, navigate the community, checkout other members' public profiles, etc. These activities are considered "passive" because the visitors merely consume existing community content, but do not add content of their own. 

Measuring Human Traffic is Not That Simple

In short, the traffic health factor is intended to measure the amount of human visits to the community. However, traditional page-view metrics also counts non-human visits by web robots and crawlers. Moreover, counting page views on a modern AJAX-filled website is not trivial. Because Web 2.0 technology furnishes users with a wide variety of interactions on a very dynamic webpage, users can potentially visit many places and perform many activities without refreshing the page. As a result, behind-the-scene REST API calls, RSS feeds, or any server-rendered pages go unnoticed to conventional page-view trackers such as Google Analytics.

The Current and Future of Traffic

In the current formulation of CHI, traffic is measured by Lithium's own PageView metric, which includes REST calls and any server-rendered pages. Although our preliminary study suggested that robots and crawlers do not significantly affect the final CHI score of the community, we intend to remove their contribution to our PageView metric in our next formulation of CHI. However, since robots do not always declare their identity (in fact some crawlers may even intentionally disguise themselves as human visitors), it is not possible to completely remove the effect of robots and crawlers. Despite this, through iterative reformulation, we hope to derive a traffic health factor that can track the passive engagement of human visitors more accurately.

I hope this post gave you a better understanding of the traffic health factor. Next time, let's talk about the content health factor. In the meantime, feel free to continue this conversation in the comments section below.

This week, I'm going to digress from the topic of flow and rank ladders in order to share an interesting talk I heard at a conference. Last week, I was at the C&T2009 meeting at Penn State University with our Chief Community Officer, Joe Cothrel. Although this meeting was rather relaxing for me, because I didn't have to present, I still can't believe it took me 12 hours to get there (9 hours of travel from SFO to State College connecting at DC, plus 3 hours lost from PST to EST). I stayed on campus at the Nittany Lion Inn that is a 5 minute walk to the IST building, the meeting venue location. I will not bother to recap the meeting, since a concise summary can be found in the conference program. However, one talk sparked some thoughts in my head that I'd like to share with you.

Day 2 of the conference opened with a keynote, titled "Knowledge Reuse and Novelty in Community Settings," delivered by Prof. Karim R. Lakhani from Harvard Business School. Prof. Lakhani presented an interesting experiment on collaborative innovation in the form of a MATLAB programming contest for solving an NP hard problem. Each code entry's performance is evaluated, scored, ranked, and displayed immediately with the contestant's name. Since this is a collaborative effort, any contestant may reuse and modify code submitted by others and then resubmit it as their own entry. The contest is closed after about a week and the top score at that time wins regardless of how many times one submits, how many lines of code one adds, or how much performance gain one contributes. The competition is all about reputation, collaboration, and learning; the winner only gets a T-shirt or a cap.

The results of this experiment are quite interesting!

• Novelty per entry is quite low: 3.5% on average.
• Borrowed code per entry is rather high: 71% on average.
• Small chunks of novelty code are often reused, so they tend to have high social values. But code entries that are too novel (have too many novel blocks of code) are often not reused because they are too hard to understand. Therefore their social value decreases.
• In contrary, small chunks of borrowed code are not often reused, so they tend to have lower social values. However, as the sizes (number of lines) of the borrowed blocks of code increase, they become reused more often, so their social value increases.
• Winning entries tend to have few lines of novel code and many chunks of borrowed code. In fact, the amount of borrowed code is twice as predictive of top performance as novelty.
• Finally, collaborative innovation almost always leads to a more optimal solution in shorter amount of time.

Since all communications in a community are persistent and are made available through the internet to the rest of the world, a community is a fertile ground for collaborative innovation. Although the amount of novelty per post is usually negligible, through many iterative refinements by many users from different backgrounds, the solution is often highly optimized and very innovative. This method of innovation and optimization is actually very similar to how evolution optimizes certain biological motifs through natural selection. Computer scientists have found this optimization method so effective that they invented the field of evolutionary computing through biomimicry.

Now, how would you like to run a similar type of collaborative innovation "contest" on your community? Lithium is geared up for a new product that will enable you to reuse the great content in your community, collaborate, innovate and produce highly valuable knowledge base articles. Watch out for our Tribal Knowledge Base (TKB) products announcement soon!