CHI Compass Update

[#1]

Dr Michael Wu Ph.D. is Lithium's Principal Scientist of Analytics, digging into the complex dynamics of social interaction and online communities.

He's a regular blogger on the Lithosphere and previously wrote in the Analytic Science blog.

You can follow him on Twitter at mich8elwu.

---

Hello everyone, and welcome to Building Community, Lithium's blog about the platform. Just as in the past, I will continue to cover analytic science and research here at Lithium. Today, I will talk about a refinement in the community health report that slightly altered the information displayed on the CHI compass that many community managers are receiving today.

The CHI compass is a radar chart that is a great for giving a compact presentation of multi-dimension data, namely the six health factors (members, content, traffic, liveliness, interaction and responsiveness) that goes into the computation of CHI.

However, some health factors have very different ranges of values, for example, weekly values for members may range in the hundreds, where as content (posts weighted by views) and traffic (views) may have values over 10K or 100K. On the other hand, liveliness, interaction and responsiveness, have much smaller values in the tens or, in some cases, less than one. With such large variation in the scale of the data, it means we have to normalize these values before we display the CHI compass. Otherwise the small weekly variations in the smaller values would not be visible against the large variations in the diagnostic health factors.

Here is an example that I've generated with Excel. In this example, liveliness had a very significant 100% weekly increase (from 1 to 2, 2 to 3, then 3 to 4) during the first 4 weeks. But it is relatively unnoticeable when plotted in the radar chart against the other diagnostic health factors, which have much higher un-normalized values. 

Community Intention

One of the insights that came out of Lithium's research was that even though you may launch a community with a specific purpose (such as key benefits like Innovate, Promote, or Support,) depending on the interaction at any particular period in time, you will find that a community can adopt the multiple traits, and these traits can change over time. For example, support communities often behave like enthusiast communities after product launches. Likewise marketing and sales oriented community can also behave like a support community and answer technical questions from the enthusiasts.

The shape of the CHI compass was designed to actually show what type community behavior is taking place. Thus, the shape can tell you whether your community is trending towards an innovation, promotion, or support community. However, this design requires a particular normalization scheme that normalizes the predictive health factors as a group. Because the liveliness factor generally has a smaller range of values than interaction, it appears much smaller when normalized and put on the same scale as interaction. People often interpret this low value as the community not being healthy whereas the community might be perfectly fine. The relative position on the compass is purely an artifact of trying to show the typology information on the CHI compass.

Under this normalization, the same community would look like the following. The outward bulge in interaction is the characteristic signature of an promotion community.

Rethinking the Visualization

So, on to the changes. After hearing many inquiries about how to interpret the CHI compass, we've decided to go with a simpler and more intuitive normalization scheme and remove the community typology information from the CHI compass.

We simply normalize each health factor to its best previous performance (over a 6 month window). If you outperformed the 6 month best score, your health factor will be 100% that week. This will redefine the new standard for which to normalize your future scores. So a growing community should hit 100% periodically, which indicates improvements over the last 6 months.

With the new normalization, the very same community would look as follow.

Some of you may ask..., if the shape of the compass doesn't indicate the community type anymore, where do we get that information? Don't worry, the community typology information is not lost, we will design another widget specifically for displaying that data!

I hope that gives you an insight into the changes in the CHI compass, and why we thought it is important. As always please let me know if you have any questions or would like me to cover particular topics.

[#2]

I like it! Much easier to see how the factors are trending 'at a glance'. Look fwd to the new 'community type' widget you mention.

[#3]

Hello Mike,

Thanks for you comment. Glad you like the idea of the new widget.

Let me try to reply to some of your other comments that I think might be useful for other readers here.

"The un-normalized grap only seems to show week 3 and 4 (green and purple), whereas the normalized shows all the weeks/colors. Is something wrong with the graphic, or is that the whole problem with un-normalized data?"

Actually all 4 weeks are on there. They are just so similar that they pretty much lay on top of the earlier weeks. That is why you don't see them. That is another problem with using un-normalized data. The scale for the traffic health factors is so large that the weekly variation in other factors are virtually unnoticeable.

"I totally get the w/o typology form. I didn't get the typology... like most people, it sounds"

Yeah, in retrospect. I think that could be confusion for people who are not statisticians. My apology to you all. But here is how I compute the percentages for week 1. We take the un-normalized predictive factors and divied each by the best of the group in the past 6 month. Since we are only at week 1, the best among the predictive factors is interaction, which has a score of 5. So we divided 1, 5 and 2 by 5 to get that 20%, 100% and 40% for liveliness, interaction, and responsiveness respectively.

Now for week 2. The value drop, the predictive factors are 2, 3 and 3. But the best score among the predictive group is still interaction, which has a score of 5. So we still normalizes 2, 3 and 3 by 5, which gave us the 40%, 60% and 60% for liveliness, interaction, and responsiveness respectively.

Moving on for week 3. Now the best score in the past 6 month among the predictive factors is 8 because interaction jumped to 8. So we divided the week 3 scores: 3, 8 and 2 by 8, which gave 38%, 100% adn 25% for liveliness, interaction, and responsiveness respectively. I think you get the idea.

By normalzing the predictive factors as a group, the relative difference between the 3 predictive factors are preserved. And this difference is precisely what gave us the typology information, which used to be encode in the shape of the CHI compass.

For example, enthusiast communities tend to have have a higher level of interaction relative to their liveliness and responsiveness because the dominant mode of interaction is discussion, where as in a support community the dominant mode of interaction is Q&A. By normalizing the predictive factors as a group we can preserve the fact that interaction is always higher because all three predictive factors are always divided by the same number each week. If they were normalize individually, they would have been divided by different numbers depending on each factor's past performance. In this case, interaction may no longer be the highest score week-to-week. You can see this in the "normalized w/o typology" figure above. So the typology information, that characteristic bulge along the interaction axis is lost in this figure.

I hope this made some sense, rather than made it more confusing.