Is There a Reliable Way to Measure Word-of-Mouth Marketing?
Word-of-mouth (WOM) marketing is one of the recent darlings of the marketing industry. It’s becoming increasingly more common to find large firms retaining specialized WOM agencies in an effort to counteract the rapidly rising CPMs of the fractured media landscape. Some are even hiring marketing managers specifically for word of mouth. Industry associations are tripping over one another with conferences and workshops to share best practices and develop standardized codes of ethics, while marketing trade publications extol the virtues of harnessing the holy grail — getting one person to “objectively” influence the opinions (and hopefully buying behavior) of another.
Beyond all the energy, enthusiasm, and hype, only two things are known for certain:
- WOM can have a very positive impact on a brand marketing plan. Case studies abound — both B2B and B2C — that bear this out.
- Metrics are all over the lot. We count almost 50 different measures being used by WOM “experts” to calculate the value of word of mouth.
Most telling in underscoring the challenges in measuring WOM are the publications of the Word of Mouth Marketing Association (WOMMA) itself. In their comprehensive 2005 report on the subject, they readily admit that there is no simple or direct way to understand the financial payback from WOM investments. Even experts quoted in the book disagree on which metrics make sense and which don’t.
 |
Defining Word of Mouth
Try to define word of mouth and you’ll quickly hit the first gap. There’s no consistent lexicon yet for understanding how it works or how to talk about planning, executing, and measuring a campaign that exploits its capabilities. In WOMMA’s report, their executives do a good job of explaining WOM simply. Brad Fay, managing director of GfK NOP says: “The purpose of … word-of-mouth marketing is … to enlist target consumers to become senders of marketing-relevant information to other consumers, who will ultimately go on to buy products and services.”
Try to define word of mouth and you’ll quickly hit the first gap. There’s no consistent lexicon yet for understanding how it works or how to talk about planning, executing, and measuring a campaign that exploits its capabilities. In WOMMA’s report, their executives do a good job of explaining WOM simply. Brad Fay, managing director of GfK NOP says: “The purpose of … word-of-mouth marketing is … to enlist target consumers to become senders of marketing-relevant information to other consumers, who will ultimately go on to buy products and services.”
Karthik Iyer, senior vice president of business development for Intelliseek, adds this clarifying thought: “If marketing is the science of moving things from a producer to consumer, then word-of-mouth marketing is the science of understanding the kind of moving that goes on, one consumer at a time, from consumer to consumer to consumer.”
Most expert-practitioners agree there are two types of word of mouth: organic — that which happens naturally — and amplified — a campaign that the marketer facilitates. The former can be measured through traditional brand tracking, reputation surveys, and customer experience monitoring, while the latter can also be measured through traditional campaign tracking tools.
However, some marketers lean heavily toward measuring the online portion of WOM to the detriment of their offline segment, because online is easier to track. But WOM research and consulting firm Keller Fay notes that nearly 80% of word of mouth occurs offline. For Vocalpoint, a social networking website for mothers, that number approaches 90%.
The Intuit Experience
Intuit practices amplified word of mouth extensively and even has an executive dedicated to it. Kira Wampler, word-of-mouth senior marketing manager for Intuit’s Home & My Business segment, treats WOM the same way she would any other marketing discipline — first by understanding the key customer insights and marketing goals for the product being featured — Quicken, QuickBooks, or TurboTax.
Intuit practices amplified word of mouth extensively and even has an executive dedicated to it. Kira Wampler, word-of-mouth senior marketing manager for Intuit’s Home & My Business segment, treats WOM the same way she would any other marketing discipline — first by understanding the key customer insights and marketing goals for the product being featured — Quicken, QuickBooks, or TurboTax.
 |
Intuit’s recent efforts provide as close to a best-practice example of how to measure an amplified word-of-mouth campaign as you’ll find. The firm had recently launched its new portal Web site jackrabbit.intuit.com, which provided new small-business owners with actionable content and tools they could find useful in starting and building their businesses. The content came from both Intuit and other industry sources.
 |
Since the site was in beta, Intuit wanted to create buzz, but wasn’t ready to commit to a full-blown marketing campaign. They started building relationships with five influential bloggers, giving them access to the site before it went public. Then, two days before the public launch, they gave an exclusive to two bloggers, asking them to look at the Web site and then share their thoughts. Intuit didn’t attempt to influence their comments. Wampler notes that one of the bloggers did a lot of her own legwork to determine whether the site was relevant to her readers. Her post, which was positive, was picked up by 13 other bloggers, creating exponential visitor growth.
As measures of success, Intuit counted the number of blogs that picked up the story or wrote their own and noted the velocity (whether it took a month or happened in two days); share of voice (how much talk occurred in the blogosphere); voice quality (what was said and the extent to which those comments were positive or negative); and sentiment (how meaningful the comments were) of each.
Solid execution; yet, like Intuit, most WOM efforts today focus on “stroke counting” — page views, number of eyeballs, etc., rather than ROI or financial payback.
The Difference between B2B and B2C WOM
In the B2B environment, word of mouth historically has manifested itself in the amplified category through conferences, sales calls, and special-interest communities. In the tech sector, customer reference programs are common, where potential clients are introduced to existing clients they can grill about how well the product works and meets their needs.
In the B2B environment, word of mouth historically has manifested itself in the amplified category through conferences, sales calls, and special-interest communities. In the tech sector, customer reference programs are common, where potential clients are introduced to existing clients they can grill about how well the product works and meets their needs.
 |
But the main difference between B2B and B2C is that B2C WOM occurs pretty much anywhere, according to Steven Nicks, EVP of client services for The Phelon Group, a customer intelligence consultancy. For instance, a self-acclaimed TiVo buff, he admits that while sitting on an airplane, he might go on and on to the person next to him about how great he feels TiVo is. But opportunities like that are few and far between in the B2B space. As a result, he notes that organizers of conferences such as EMC World in Boston have begun providing such WOM vehicles as specialty cafés that foster the ability for people to congregate, connect, and chat. Five years ago, such an idea might have been considered a waste of space. Today, it would fall under the category of “marketing opportunity.”
Walking a Thin Line
There’s been a lot of press recently about how some WOM tactics walk a thin line between what is ethical and what is shady. Questionable methods include hiring people to go into bars and loudly order a specific branded drink, or sending an “actor” out with the latest cell phone camera to ask tourists to take his picture as a means of getting the device into their hands and creating opportunities to talk about how wonderfully miniature it is.
There’s been a lot of press recently about how some WOM tactics walk a thin line between what is ethical and what is shady. Questionable methods include hiring people to go into bars and loudly order a specific branded drink, or sending an “actor” out with the latest cell phone camera to ask tourists to take his picture as a means of getting the device into their hands and creating opportunities to talk about how wonderfully miniature it is.
In addition, there are companies that charge advertisers to take their message and distribute it out to a “network” of agents (for instance, teens or seniors) they maintain. Agents get paid to promote advertiser messages on blogs, in chat rooms, at events, or to talk it up with their friends. Teens are paid to talk up certain candy bars, clothing items, or music to their friends, while grandmothers earn points for sharing their knowledge of over-the-counter medications with others at the senior center.
Nicks says that unethical word of mouth is not as prevalent or noticeable in the B2B space as in B2C. However, there are a number of B2B initiatives he would classify as bad ideas. First, if a company is going to connect two people together, telling Person A what Person B is concerned about before introducing the two is a good idea, and prepares Person A to speak intelligently; however, telling Person A what’s important to say is not.
Second, companies exist that offer discounts to current customers for talking to prospects, or for allowing their case study in the media. There are also sophisticated blogs that compensate people around their activities. Anything that removes the genuineness and openness and dilutes the quality of the message, Nicks feels, devalues what you’re trying to do, noting that word of mouth has to be truly genuine to have real impact.
While the vast majority of WOM network companies employ clear rules of conduct, it would be difficult to ignore the enormous potential for abuse here, particularly where measurement is concerned. First, the quality of the context for these types of in-person contacts cannot be monitored, so a small spike in sales can be easily be misconstrued as a success. We wouldn’t even know if the agents were just buying more of the product themselves. Even more importantly, such a program could give word-of-mouth a bad reputation, quickly equating it to spam, exposing the category to potential regulation and legislation and effectively killing it before it even has a chance to mature.
The WOMMA Framework
The Word of Mouth Marketing Association came together in 2005 to create a framework around word-of-mouth marketing, including defining it, and developing standards, metrics, and measurement tools. They concluded that a WOM “episode” involves four components:
The Word of Mouth Marketing Association came together in 2005 to create a framework around word-of-mouth marketing, including defining it, and developing standards, metrics, and measurement tools. They concluded that a WOM “episode” involves four components:
- Participants: Creators, senders, and receivers who can be measured on their propensity, demographics, credibility, and reach;
- WOMUnit: A single unit of (media-agnostic) marketing-relevant information;
- Action: What participants do to create, pass along, or respond to a WOMUnit; actions can be measured on velocity, distribution/spread, and source diversity; and
- Venue: The medium or physical location where the communication takes place; venues can be measured on total potential population vs. actual audience received.
 |
A single “episode” achieves one in five outcomes, each of which serves as a trackable event:
- Consumption: The receiver absorbs the information but takes no action;
- Inquiry: The receiver seeks additional information;
- Conversion: The receiver completes a desired action;
- Relay: The receiver redistributes the WOMUnit to another person; or
- Recreation: The receiver creates a new WOMUnit.
WOMUnits can also be broken down into more distinct measurable parts, including:
- Topicality: The degree to which the marketing message is contained in the WOMUnit;
- Timeliness: Whether the WOMUnit arrives in time to be relevant to a specific campaign;
- Polarity: The positive vs. negative content of the WOMUnit;
- Clarity: Determines if a message is understood by the receiver in the way it was intended by the sender; and
- Depth: The amount of visual, written, or verbal information included in a WOMUnit presumed to increase message persuasiveness.
Tying WOM to Experience
For organic word-of-mouth, both Wampler and Nicks use Fred Reichheld’s Net Promoter Score®1 (NPS) as their main measurement tool. The score is based on the concept that all customers can be divided into three categories: promoters (your best customers), passives (satisfied but unenthusiastic), and detractors (unsatisfied customers at the highest risk of defection and producing negative WOM).
For organic word-of-mouth, both Wampler and Nicks use Fred Reichheld’s Net Promoter Score®1 (NPS) as their main measurement tool. The score is based on the concept that all customers can be divided into three categories: promoters (your best customers), passives (satisfied but unenthusiastic), and detractors (unsatisfied customers at the highest risk of defection and producing negative WOM).
To build the score, companies must first ask their customers the “ultimate” question: “How likely is it that you would recommend our company to a friend or colleague?” Answers range from 0 (least likely) to 10 (most likely). Nines and 10s are promoters — your best referral sources.
 |
Your Net Promoter Score would then be determined by subtracting the percentage of detractors from the percentage of promoters (%P – %D = NPS). The resulting number should serve as your starting benchmark as you try to turn more customers into promoters. WOM programs are measured on their ability to impact the NPS.
But even the Net Promoter Score requires a lot of effort and back-end work, says Gary Spangler, e-business leader for the electronic & communication technologies platform at DuPont. Wampler cautions that often, when people evaluate their NPS, they are simply handed a number and don’t do anything with it. To really get value from it, companies must look at the underlying drivers that would make someone more or less likely to offer a recommendation. Getting a handle on those drivers, she says, will, in turn, drive the decisions used to affect change within an organization.
The flaw, says Nicks, is that the score only tracks what people are saying they would say about you, not what they’re actually saying about you. A better measure to have in a marketing dashboard, he feels, would be the percentage of deals that word of mouth is being leveraged on and how those deals compare with non-WOM deals — a true tie-back to sales.
Alternate Approaches
An alternate metric suggested by The Phelon Group is “return on reference,” which measures the impact of satisfied customers on recruiting new customers and on shaping the impressions of influentials. Phelon often measures, for instance, whether a customer-reference program is able to shorten the sales cycle.
An alternate metric suggested by The Phelon Group is “return on reference,” which measures the impact of satisfied customers on recruiting new customers and on shaping the impressions of influentials. Phelon often measures, for instance, whether a customer-reference program is able to shorten the sales cycle.
While we agree it may be a good idea to measure the correlation between WOM and the sales cycle time, we don’t recommend using vague phrases such as “return on reference,” as they tend to undermine the credibility of the effort in the eyes of the CFO for whom “return on …” is a sacred pretext to “investment,” “capital,” or “sales.”
 |
DuPont, in the early stages of WOM adoption, intends to look at measures such as campaign scale (how far it reached); speed (how fast it spread); share of voice in that space; whether it achieved positive lift in sentiment; whether the message was understood; whether the messages was relevant; whether the message had sustainability (something you can add to or a one-shot); and how far it moved from its source. Spangler is also trying to define financial metrics, such as whether WOM will increase sales, revenue, inquiries, and lead conversions, but admits it’s tough to articulate a tried-and-true method for getting there.
Nicks feels that the ROI of WOM may be easier to track in the B2B space because, while it’s hard to single out its impact by specific account, it is possible to measure overall lift for campaigns with WOM against campaigns without WOM, something he calls a lift “in combination.” That’s a hard notion for executives to accept though, he says, because everyone wants to know the exact revenue impact.
The Paradox of Measuring Financial Return on WOM
Even if you could isolate the specific revenue (or profit) created by WOM, the measurement effort itself could eat up all your profits. Consequently, many advocate either the stroke-counting approaches mentioned above, or other “strategic” frameworks.
Even if you could isolate the specific revenue (or profit) created by WOM, the measurement effort itself could eat up all your profits. Consequently, many advocate either the stroke-counting approaches mentioned above, or other “strategic” frameworks.
Dr. Walter Carl, assistant professor at the Department of Communication Studies of Northeastern University, measures the “credibility” of word of mouth in six ways 1 :
- Credibility effect: Whether the information provided by the participant’s conversational partner made information heard from another source (such as the media) more or less believable;
- Thinking change: Whether the episode resulted in a change of thinking or ideas about the product or service and the action a person plans to take;
- Inquiries: The likelihood that a person will seek additional information after a WOM episode, such as visiting a Web site;
- Purchase likelihood and behavior: The likelihood of the participant to purchase or use the product or service;
- Pass-along likelihood and behavior: The extent to which the participant will tell other people about the product or service; and
- Relationship consequences: Whether the WOM episode has an effect on the participant’s relationships when they engage with their friends and family, etc.
 |
Measurement solutions provider CRM Metrix recommends companies measure “the perceived newness and differentiation of their messages” and message “talkability,” while Brandimensions suggests companies focus on “buzz,” which is a proxy for awareness, and “sentiment,” which Wampler uses. On a 1-5 scale, sentiment reflects the degree to which consumers are feeling negatively (1) or positively (5) toward the brand and its attributes.
Others use automated media analysis solutions from organizations such as Biz360 and Delahaye to measure the number and nature of media references picking up the WOM message.
A Prescription for WOM Measurement Progress
Most vendors and consultants working in WOM circles today profess to tie word of mouth to sales or some other form of financial payback. However, too often, these efforts fall back on traditional tracking methods such as web metrics, pre and post surveys, and publicity monitoring. These types of measures are a good start, but they don’t go far enough. Too many companies are settling for what they can get, rather than trying to develop processes that directly correlate to increased sales.
Most vendors and consultants working in WOM circles today profess to tie word of mouth to sales or some other form of financial payback. However, too often, these efforts fall back on traditional tracking methods such as web metrics, pre and post surveys, and publicity monitoring. These types of measures are a good start, but they don’t go far enough. Too many companies are settling for what they can get, rather than trying to develop processes that directly correlate to increased sales.
 |
It seems to us that the right process for measuring WOM would include the following steps:
- Define objectives. Clearly and succinctly state the intended outcome of the campaign expenditure in economic or behavioral terms. If your primary goal is to influence awareness or attitudes, attempt to forecast how that will translate into an increase in profitable customer behavior patterns. If the objective is to shorten the sales cycle, develop hypotheses about how much shortening you can do and what the economic value of a shorter sales cycle would be. If you’re intending to influence the net promoter score, be clear on your expectations for the financial outcome associated with a 1%, 5%, or 20% improvement.
- Test message strategy effectiveness on behavioral intentions. There are many choice/options techniques available today that can tease out the potential impact of subtle changes in message execution. Much of this can be done quickly and inexpensively via online research panels.
- Construct experimental designs to validate the relationships between intended behavior change and actual behavior change. Develop test/control constructs to determine the true predictive value of the awareness or attitude change. Try to control WOM message exposure either geographically or, if that’s not possible, on the basis of targeted delivery channels, demographic sub-segments, or simply time (pre-launch vs. post-launch). Methods of controlling for extraneous environmental factors are readily available from your local statistician or academic.
- Conduct post-campaign interviews with current and new customers, and those who still resist your value proposition to find out what did/did not influence their decisions to act/not act as you desired.
- Review your proposed measurement methodology with key constituents of the outcome in advance. If the campaign results are positive, who is most likely to challenge your eventual findings? Ensure that finance, sales, and operations are able to air their concerns about the validity of the approach before you launch and ask if they have any better ideas.
- Keep the proper perspective in mind. Effectiveness and efficiency are only relevant in the context of your expectations. State them clearly and in as close-to-financial terms as you can. Did you achieve the specific result you expected? Did the result come at a favorable cost? If the answers are both yes, you likely have a successful WOM campaign on your hands.
|
Word of Mouth in Action:
What are marketing agencies tracking and measuring? |
|
We asked a few leading marketing agencies to tell us what they track in word-of-mouth marketing campaigns. They told us:
Blog, social networking, and other buzz metrics include:
Sources: MRM Worldwide, Starcom Mediavest Group, and Wunderman.
|
Protecting the Future
Measuring word of mouth is certainly a work in progress and companies have gone to great lengths to drill down below the surface and understand its value drivers. Now it’s time to link those drivers to results. Key WOM metrics that can move the needle on brand equity, customer franchise value enhancement, or short-term cash flow should be elevated to the dashboard level and monitored frequently. The potential impact of WOM is enormous if we can keep it from being discredited before it gets a chance to mature as a valuable marketing tool.
Measuring word of mouth is certainly a work in progress and companies have gone to great lengths to drill down below the surface and understand its value drivers. Now it’s time to link those drivers to results. Key WOM metrics that can move the needle on brand equity, customer franchise value enhancement, or short-term cash flow should be elevated to the dashboard level and monitored frequently. The potential impact of WOM is enormous if we can keep it from being discredited before it gets a chance to mature as a valuable marketing tool.
Net Promoter is a registered trademark of Satmetrix Systems Inc., Bain & Company, and Fred Reichheld.
1 Source: Measuring Word of Mouth, WOMMA, Summer 2005.
For a word-of-mouth metrics glossary, click here.
