Focus groups vs. analytics in measuring social media ROI


Tom Gerace’s recent post in imedia connection, attempts to measure the true brand value of a social campaign.  What I found interesting about it was that here you have one of the  leading social platforms, Gather, with access to best analytics tools in the space, choosing to use predominately traditional sampling and focus testing techniques. It just goes to show that there are many ways to approach measurement and as marketers, it is our job to pick the right horse for the right course.

Client Goals:

The campaign objective was to build awareness, purchase intent and trial of a new mass market consumer packaged goods product.

Measurement Methodology:

  • Reach: measured by the unique users exposed to some aspect of the campaign.  Individual activities like reviewing and commenting were broken-out separately.
  • Impact: measured based on a control group study.  Responses compared those exposed to some aspect of the campaign with those not exposed on the same site.  Similar studies were completed on traditional media sites running media campaigns as well for cross site comparison.
  • Studies were completed by third party surveying groups and analyzed by the media buying agency.  We also operated our own internal study to calibrate results of our internal tools against those of our partners.

Engagement Methodology:

Gather engineered conversation about the product through product sampling and reviews by approximately 100 members.  Because members are connected to friends on the site, their conversation was seen hundreds of thousands of times by others on the site in their content and conversation feeds.

  • Sampling by Influencers: Gather identified 25 influential Gather women to try the product in place of substitute goods.  Members were identified based on algorithmic analysis of their standing within the community.  These members then came back to the site to write about their week long trial and discussed the experience with other members.
  • By Request: Gather distributed additional product samples to Gather members to try when they needed a break during their busy days.  Members requested product samples by discussing their busy days and why they need the break.
  • Take-a-Break Chats: Gather created a series of weekly online chats for women to discuss their busy lives, tips for relaxation and their short-term and long-term goals.  Chats were held in the afternoon when people tend to need a break. Product samples were given out during the chats.
  • To-Do Surveys: Gather posted regular surveys throughout the program to engage members in discussions and to produce member-generated content for the group.  Members filled out surveys discussing their long- and short-term goals, they submitted to-do lists and playlists, and they shared motivational tips with each other.
  • Member Submissions: Members were allowed to submit their own content and start their own discussions in the group as well, and the community was moderated to ensure that all content aligned with the theme and focus of the community.

Ker-ching…er…I mean “Results”

I’ll admit, we were nervous about what the numbers might say.  We had observational and anecdotal information about the power of product sampling on the site.  We had seen members truly engage in conversation around a product.  But we had not measured the mass-market impact of these conversations.  We didn’t know if they moved the needle for the friends and friends-of-friends that were discussing products they had tried.  Then the cash register rang.

Gather’s social media campaign showed a 30% increase in purchase intent by those exposed to the campaign.  According to the measurement data collected by both Gather and the media buying organization, the social media campaign operated on Gather outperformed other sites involved in the campaign.  It significantly exceeded results seen in these types of studies on traditional media sites.  People who were exposed to the program on Gather performed 15-20% higher across every campaign metric than the control group.

As we move into this brave new world, let me close with a couple of questions: How do you measure the campaigns you run?  What metrics do you find most credible?



    Track comments via RSS 2.0 feed. Feel free to post the comment, or trackback from your web site.

  2. 12/8 2009

    Doing all this online work and then using surveyed purchase intent seems kind of backward.

    Why not observe changes in online brand advocacy – which we (MotiveQuest) have proven are related to changes in sales?

    More here:



  3. 12/11 2009

    Twitter Comment

    Focus groups vs. analytics in measuring social media ROI | Content …: Take-a-Break Chats: Gather created a series… [link to post]

    Posted using Chat Catcher

  4. 12/11 2009

    Twitter Comment

    RT @SocialMediaComm: Focus groups v. analytics in measuring #socialmedia ROI | Content to Commerce [link to post]

    Posted using Chat Catcher


We Are... BIG FUEL

Hello! Big Fuel is a full-service marketing and communications company based in New York that takes brands from Content To Commerce. A unique approach that bridges "people stories" to "product stories" through social media and branded content.

Big Fuel is one part marketing agency, building brands through consumer insight; one part entertainment company, creating content that people love; and one part distribution company, driving guaranteed results by delivering content to targeted audiences. We work with major brands, leading agencies, publishers and platforms to help marketers achieve true consumer engagement.

test Latest from Twitter
Watch Videos