Tamara Barber [Posted by Tamara Barber]

I’m currently working on a piece of research we here at Forrester call the Wave, which is our vendor comparison methodology. I’m examining vendors in the Market Research Online Community (MROC) space and am very much looking forward to sharing my findings once the research is published.

In the meantime, I’ve been doing some soul searching on what differentiates a research community from a typical online panel. Vendors and clients alike generally have very specific views on the difference. Here’s a sampling of quotes from a recent survey I fielded on MROCs[i]:

“An online panel suggests that you'll be using it solely for survey research. While an online community is more involved – discussions, community building activities, as well as surveys.” 

“An online community is an ongoing social networking forum which I would consider qualitative. A quantitative online panel allows you to do one off quantitative research projects.”

“Panels are large (often a thousand or more) and are often utilized once or twice per research project (i.e. a survey and a follow-up).  MROC[s] are long-standing communities (often a year or more) with a smaller group (300-500) of participants.”

All valid points, which I think highlight three key differences between communities and panels. Communities are:

  • Qualitative. Results are directional and usually biased toward a sub-population of a researcher’s target market. Community research should not take the place of statistically sound, representative quantitative research, but it should definitely be used for idea generation, ethnographic exercises, or getting a quick pulse on a certain type of consumer.
  • Interactive. Companies lose the community feel if they don’t engage their members in an interactive manner. This means researchers should manage communities so that they ask for and provide feedback to the members, and also allow and encourage members to interact amongst themselves.
  • Iterative. While communities can certainly run for short periods of time, the actual research exercises that happen in a community are based on building ideas off of each other. Rather than a typical radio-button survey that elicits further questions but are never followed up on.

But does size matter? Most vendors in this space have the capability to host tens of thousands of members in one community, but I’ve found that, the larger a ‘community’ is, the more often it’s used for quantitative insights only – just as a panel. And the larger a community gets, the harder it is to manage. My conclusion for the time being is that there is such a thing as a community that is too big for doing proactive market research exercises. I am very interested to hear your views, since the topic is a little controversial at the moment. I welcome (and encourage) your feedback and comments!

[i]These quotes are from Forrester’s Q3 2009 Global MROC Online Survey, which we fielded among 78 market researchers.