Calculating the Value of Referrals: Easier Said Than Done

 

There is still much to be learned about calculating the economic value of referrals and other word-of-mouth (WOM) activities, both online and offline. Current methodologies and research give us small glimpses of a great universe, but in many ways our current approach to charting WOM is like being an astronomer back in the pre-Galilean era. We can see the moon and the stars, but we have no real frame of reference around how big the universe actually is or the activity that’s taking place outside of our view.

Consider, for instance, that even the most advanced third-party tools primarily track online WOM — a severe limitation when you look at studies like the 2006 Keller Fay Group survey, which concludes that only 8% of brand-related conversations take place online. “Online tracking mechanisms make it easier to track who’s recommending what,” says V. Kumar, the ING Chair Professor in Marketing at the University of Connecticut’s School of Business and executive director of the school’s ING Center for Financial Services. “But still the hole is there, because the offline activities are not captured.”

The good news is that the tools and research methods are improving — much as the modern telescope has evolved from the early models built by Galileo — helping us to gain more informed insight around the real drivers of WOM. In this article, we’ll look at two emerging bodies of work that advance the discussion about the business impact of WOM and offer some takeaways for marketers who might, understandably, be struggling to get their arms around calculating the economic value of WOM.

Ultimate Question, Elusive Answers
Fred Reichheld’s “ultimate question” — would you recommend us to a friend? — helped to catalyze the word-of-mouth movement for corporate marketers. Social media tools such as blogs and user rankings have pushed the concept further from a very personal interaction — a friend recommending something to another friend — into a one-to-many activity online. (Call it the Oprah’s Book Club model brought to the Web, where anyone can be Oprah.)

Scores of studies point to the benefits of personal referrals vs. marketing promotions for influencing purchase decisions. No argument there. The burning issue for marketers is that many of these studies only scratch the surface of the true impact of word of mouth. Truth be told, while we understand at a conceptual level the benefits of WOM, we don’t really have our arms around its behavioral or economic impact.

First, one has to accept the fact that a referral is going to work very differently in B2B vs. B2C environments, or even within B2C or B2B categories. The power of referrals in consumer electronics, for example, is great. But does it make sense to measure WOM for toothpaste, or aircraft engines? Not so much.

If you’re in a category where WOM is important, how do you get a handle on the value of those referrals? Right now your primary option is to try to replicate external studies — the ones that, for example, tell you that an average “influencer” refers 4.6 people, etc. But you’re not likely to reach such a tidy conclusion for your own business — that is, not without a significant research investment — and even if you do, it’s probably only a brief snapshot that reflects the truth in place at the time you took it. You can’t assume your conclusions will hold up over the length of any resource allocation and planning process.

Another option is to use a service provider that offers tools and methodologies for measuring “buzz” for your business. Several credible companies have staked claims in this space, offering what they see as quantifiable evidence of both the volume and type of referral activity around a particular product or brand — at least the activity taking place on the Web. Companies such as Nielsen BuzzMetrics, for example, use data-mining software to index online consumer-generated media such as blogs, message boards, and discussion groups, and then analyze the findings to determine measures such as the volume, reach, and even the credibility of the source.

Measuring the Value of Referrals
Many marketers, however, remain unsure how to use this data to improve the effectiveness of their marketing programs. Part of the challenge lies with the data itself. Consider the Net Promoter Score (NPS), the output of Reichheld’s “ultimate question.” NPS measures the number of customers who say they would (or wouldn’t) recommend a company to other people. Lauded for its simplicity, NPS nevertheless is limited by its reach, because it only tracks whether people say they will recommend a company or not. Net Promoter doesn’t track whether those customers actually follow through on their recommendations. In terms of determining the economic value of referrals, this doesn’t provide much insight.

“There’s a big discrepancy between people who say they are willing to recommend and those who end up recommending,” says UConn’s Kumar. “Rather than looking at the top line, we decided there was a need to measure the value a customer brings in through referrals.”

That’s why Kumar and two associates undertook a study that pulls the Net Promoter thread in an attempt to unravel some of the mystery around referral value. The group polled 9,900 customers at a telecommunications company and 6,700 at a financial

services firm on their referral intentions — and then tracked the prospects that those referrals brought into each company. Their conclusions, published in the Harvard Business Review, let some air out of the NPS balloon: While 81% of customers doing business with the telecom company said they would recommend the company to others (resulting in a high NPS), only 30% actually followed through with a referral. In addition, only 12% of those referrals generated new customers, and only 8% became profitable new customers. The group found similar results at the financial services firm.

No Correlation between CLV and Referrals
Further analysis revealed that customers with high customer lifetime value (CLV) rates were not necessarily the most passionate (or most effective) influencers. Customers with the best referral rates, the study showed, had surprisingly low CLV calculations. “That was our biggest revelation,” says Kumar. “The expectation was that a high CLV would correlate with a high CRV [customer referral value]. But we discovered that high-CLV customers tend to mind their own business, or they don’t see the value in telling their peers. And they don’t fall for incentives.”

The authors concluded that a new metric — customer referral value — is a far better proxy for a customer’s total value to a company. CRV is a complicated formula that takes into account past referral behavior, estimates of future referrals driven by marketing incentives, and estimates of how many referrals would have become customers anyway, thereby weighting the value of a customer’s referrals. “Rather than looking at the top line, CLV lets us assign a value to referrals, which lets us correlate the return on investment for marketing campaigns to influence or encourage those referrals.”

Leveraging “Talk Value”
Other researchers are also attempting to put a finer point around measuring the outcomes of referrals and the effectiveness of WOM campaigns. Northeastern University and BzzAgent, for example, have collaborated on a new methodology for measuring conversational reach. The two have published a paper, titled “Measuring the Ripple,” that describes their proposed industry standard for leveraging “talk value,” which the researchers define as “highly credible marketing-relevant conversations beyond the initial brand touchpoint or conversation with a participant in an organized WOM marketing program.”

The researchers claim that the methodology will enable marketing firms to not only tally the number of people reached through a campaign, but also to quantify the outcomes of those connections, in terms of intentions (to inquire further about, use, purchase, or refer the product) as well as behaviors (inquiry, use, purchase, or referral).

The research was meant to extend the acknowledged limitations of BzzAgent’s own business model, in which volunteer “buzz agents” are incented to promote various products and services from BzzAgent clients and keep logs of their individual WOM activities, which BzzAgent tracks and analyzes. The current BzzAgent model (as well as those from other service providers) only tracks that first line of referrals from the agent to the people in his or her network. It does not measure the “ripple” of WOM that may occur beyond those initial pass-alongs.

To quantify any additional conversations that may occur beyond that initial referral, Northeastern and BzzAgent created a “conversation card” that first-line referrers were asked to give to anyone to whom they spoke about a targeted product or service. The card directed recipients to a Web site where they could take a survey about the WOM “episode,” and included instructions for the first-line referrals to pass the card along to anyone else with whom they had a subsequent conversation about the product or service. (Using both physical cards along with URLs enabled the researchers to track both online and offline conversations.)

The “conversational reach” was calculated using the number of original participants in the WOM campaign, the percentage of those people who made an initial referral, and the number of participants in each subsequent “generation” of referrals (the ripple) over a period of time. That total generates the “relay rate” — or the total number of people reached through the campaign. 

Driving Economic Value
These studies represent an important evolution in the WOM tool kit. But there is no silver bullet for determining how much growth WOM really drives for your business; the answer will be different for every company. CMOs looking for these answers must first ascertain how much of a driver WOM is for their business, what those drivers are, and, finally, what they can do to influence those drivers.

Determining who is most influential by looking at the relationships between customers, referral recipients and the offers and behaviors that drive the best conversion rates is a critical piece of the “viral value” equation, says Jim Calhoun, CEO of PopularMedia, a WOM marketing company whose services include a technology platform for launching and optimizing viral campaigns. “The best customers are not necessarily the best referrers,” says Calhoun. “By putting a program in place to observe customers, engage their social networks and their conversations about your brand, and hone in on those who are most influential in getting what you want to get done, you can get lots of insight about how to monetize [referrals].”

In the Harvard study, Kumar and his fellow researchers demonstrated how targeted marketing campaigns can influence WOM behavior and, as a result, improve payback. Using direct-mail promotion and incentives for referrals, the telecom company turned 4% of its high-CLV/low-CRV customers into high-CRV “champions” without compromising their CLVs. In effect, the researchers have developed method for building new customer value segmentation models based on current and future WOM-driven transactions.

Doing the Legwork
Secondary studies are one option for obtaining insight into what others are finding, but you very likely won’t be able to plug those external results into your business. To get actionable results, you’ll have to do the legwork yourself.

More important than simply determining the average referral rate — the number of people that a customer interacts with on your behalf — is determining the economic benefit of increasing the number of referrals per customer and then identifying the actions you can take to cause customers to be more prolific. In other words, what’s the economic value of the gap between referral rates of 2.6 vs. 5.9? And how do you focus on the drivers that convince your customers to make more referrals?

(Kumar notes that the next round of research around CRV, the results from which are expected this summer, involves examining the drivers of referrals and how marketers can use those drivers to improve the economic contributions of customers. The researchers are also looking at the WOM effect in B2B environments.)

One of the big challenges associated with measuring a chain of referrals is that in each successive step in the chain, you almost certainly are underemphasizing the importance of all the other indirect environmental chatter that’s occurring naturally around your product or brand. If your research fails to control for that indirect chatter, you’ve left your findings open to significant criticism. Which is why, if you really want to understand referrals, you will need to build the research into your regular tracking studies. (And you may need to adjust the frequency of the research if the WOM number bounces around too much annually.) This will enable you to monitor the chatter so you can see when the environmental aspects have moved above or below the control band of normal. You can then correlate customer self-reported referrals to the level of environmental noise.

It’s quite possible that you won’t be able to forecast the financial implications of the actions you take with any great certainty. If that’s the case, your best approach is to inform management consensus regarding the perceived value of customer referrals, which points you to the drivers. Even if you don’t have the means to calculate specific referral rates, if you agree that a referral is an important driver of shareholder value, you can work backward to understand what’s driving those referrals.

As with any measure of customer engagement, keep in mind that the value of referrals does not have to be calculated strictly in top-line results. Consider PETCO. After the pet products retailer added user-generated product ratings and reviews to its Web site (the review platform is hosted by Bazaarvoice), it discovered that products featuring user reviews had a 17% lower return rate than those without reviews. PETCO’s e-mail marketing campaigns showed impressive gains as well: After the company began incorporating review excerpts into its e-mail newsletters, click-through rates increased by 50% and conversation rates improved by 20%.

Five Important Takeaways
Granted, this is pretty heady stuff. Some might call it dense. To help you see the forest (instead of just the trees), here are five key points to consider.

1.         Research in this area can be highly suspect. Beware of pre-formed “networks of influencers” compensated for viral efforts.

2.         Online behavior is still, in most categories, only a portion of the overall buying process. If you only measure those things that are easy to track, you’ll surely draw the wrong conclusions.

3.         Early studies suggest that customers with the highest lifetime value (defined transactionally) may pale in comparison to those who buy less but refer more prolifically.

4.         A major challenge for measurement of WOM is that, in each successive step in a referral chain, you almost certainly underemphasize the importance of all the other indirect environmental chatter.

5.         To understand the payback on WOM and social media, you need to experiment to determine the real drivers of economic value in your business. No other business will exhibit the same drivers in the same ways.

In the frenzy of activity to measure new media and social networking, keep in mind the old saying, “If you can keep your head while those about you are losing theirs, you’ll ... be the only one with a head.” You get the idea.