How Do You Measure Engagement?

Start by Defining It in the Right Context

As the dynamics of both consumers and communication channels change, marketers are seeking new and better ways to measure the effectiveness of their programs and justify their marketing spend. One concept that has elbowed its way to the front of the metrics pack is “engagement,” which is rapidly gaining favor among marketers and media buyers as a critical measurement of customer value and brand strength.

But while just about everyone agrees on the importance of measuring engagement — who doesn’t want to know how well your customers are receiving your message or interacting with your brand? — the common ground pretty much ends there.

Ask a marketer how they measure engagement — or even, for that matter, how they define it — and the answers are as far-reaching as an Iowa cornfield.

There’s no question of the potential benefits of positioning your marketing strategy around the concept of engagement. But as in all areas of marketing measurement, the approach must be framed in the proper context in order to deliver useful insights, which — when combined with existing data and a methodical testing program — lead to positive outcomes.

Two Types of Engagement

There are two generally accepted engagement “types”: emotional engagement and behavioral engagement. The former is more popular; the latter is more important.

1. Emotional Engagement

When a consumer makes some conscious or subconscious connection with a brand based on some marketing stimulus — usually advertising — they are engaging emotionally with that brand. The Advertising Research Foundation (ARF), among others, has adopted this perspective in its latest definition of engagement: “Turning on a prospect to a brand idea enhanced by the surrounding context.” In its 2006 whitepaper “Measures of Engagement,” the ARF further defines engagement as “an integrative concept that has to span engagement with the brand, engagement with the idea or ‘creative’ and engagement with the media or context.” A turned-on customer, they continue, results from stimulating co-creation, which leads to a more personal, deeper brand meaning.

The research tools for measuring engagement within this emotion-driven context have been available for the past 50 years and for the most part do an adequate job. Brand recognition and recall studies, association techniques, customer satisfaction surveys, and the like all help marketers determine how customers feel about a product, service, or brand, and therefore measure various levels of emotional engagement.

This means that, other than the nomenclature, nothing much has changed over the past half-century when it comes to measuring emotional engagement. Metrics developed in this context, with a focus primarily on advertising, do not give marketers any deep insight into the linkage between engagement and financial outcomes; rarely are they predictive of future buying behavior. If your goal is to sell things, these types of engagement metrics don’t cut it. “Necessary but not sufficient” comes to mind for this approach.

Even newer metrics such as Net Promoter (and yes, customer satisfaction is a form of engagement measurement), despite their growing popularity, don’t go far enough in linking customers’ feelings to their wallets. “Net Promoter gives you a good longitudinal means to study your progress, particularly in a quantitative world,” says Tom Nightingale, vice president of communications and chief marketing officer at Con-way Inc., a $4.7 billion transportation and logistics company. “We’re probably measuring the current state as well as we can today. Long-term linkage between engagement and economic value is something we’d love to get to, but realistically we’re a long way from that.”

2. Behavioral Engagement

Nevertheless, there are tangible benefits to be gained from framing a marketing strategy around the concept of engagement — but only if the engagement is defined differently.

In the most literal sense of the word, “engagement” should be focused on the evolution of the behavioral relationship between consumer and company, not simply the emotional attachment one has to the other.

“Companies that realize there’s a much deeper interaction between customers or prospects with the brand are recognizing that there are other elements to measure there,” says Brian Haven, a senior analyst with Forrester Research and author of the 2007 report “Marketing’s New Key Metric: Engagement.”

It’s important to note that behavioral engagement is not limited to a purchase of a product or service; it encompasses all the interactions that a prospect or customer has in relation to a brand. There are any number of pre- or post-sale activities that can be (directly or indirectly) predictive of a future purchase or re-purchase; they include visiting a Web site, downloading a whitepaper, calling customer service, recommending a product, or even commenting on a blog.

Such behavioral measures of engagement hold the potential to displace awareness and brand preference as interim measures of marketing effectiveness. So why aren’t more marketers adopting this approach? For one, the ever-increasing complexity of the purchase funnel, which for most industries encompasses multiple channels and non-linear activities, makes it impossible to adopt a one-size-fits-all formula. “Anyone who believes there’s a set formula is crazy,” says Haven. “If I’m a retailer, I have a different type of interaction with a customer than the manufacturer does. The data points I’m collecting are different. The variables are different.”

At the same time, creating linkages between marketing programs and customer activities is no longer a linear, one-to-one process. The relationships among the different channels, and their collective impact on a customer’s buying decision, are far more difficult to gauge (and influence) than they once were. Consider the intricate relationship between just two channels: the Internet and a retail store. A consumer can visit a store to look at a product, purchase it, and bring it home. Or she may look at the product in the store, then go home to compare prices on the Internet, and purchase from a lower-priced competitor. Alternatively, she may do her homework first on the Internet (comparing prices or features, reading customer reviews), buy the product from a trusted vendor, and then have it shipped to her house or pick it up at one of the retailer’s local outlets. Two channels, multiple behaviors, multiple types of engagement. Throw additional channels into the mix — the call center, for example, or reseller partners — and what once was a fairly linear purchase path becomes a tangled jungle trail.

Arnaud De Bruyn, assistant professor of marketing at France’s ESSEC School, explains the complexity of this modern purchase funnel — and its impact on resource allocation decisions — with a simple car-buying example: “No one buys a car on the Internet, which might lead one to conclude that the Web site is an ineffective channel for selling cars,” he says. “But 70% of people who buy a car do their research online first. So if you stop investing in the site, you indirectly lose customers.”

How Do You Measure Behavioral Engagement?

Marketers, therefore, need to develop more effective methods for tracking how customer segments or individuals are progressing through one or more desired paths toward a closer, more commercial relationship with a product or brand. The goal is to justify spending in areas that will influence the behaviors that lead to specific, positive outcomes, be they more referrals, more prospects, more purchases, or repeat purchases.

Forrester Research has proposed a metric that defines engagement as “the level of involvement, interaction, intimacy, and influence an individual has with a brand over time.” The metric is built from both online and offline data, using quantitative and qualitative measures and what Forrester calls “the fuzzy areas in the middle best characterized by social media.”

Behavioral engagement metrics such as Forrester’s can give companies a window into not only which stage individuals are at in their progression from prospects to customers, but also the velocity at which they are moving toward the end goal. How does one measure pace? Try comparing how many of a given universe of customers/prospects are at various stages of the engagement chain at two time intervals. This tells you not just how many moved from one stage to the next, but the velocity with which they are moving, which in turn helps you forecast what might happen at the next time interval.

Such insights into the value of all stages of the customer pipeline can help marketers determine the impact that marketing efforts may have on increasing that value. Understand, however, that the end point doesn’t have to be a direct (or immediate) economic transaction. Equally important are the myriad indirect methods for increasing value, such as a referral that leads to a new customer, or a white-paper download that educates a prospect and influences a future product decision. There’s plenty of data available to track how customers or prospects are engaging with a company; the key is to synthesize it into a clear model for demonstrating either short- or long-term economic benefit.

As noted earlier, there’s no set formula for developing an engagement methodology. But there are steps marketers can take to develop a more comprehensive view of engagement and a process for capturing the most relevant insights from customer behaviors.

Rules of Engagement

1. Develop a vision. Don’t attempt to measure engagement until determining the outcomes you want to achieve. The way you design your campaigns and make allocation decisions will be influenced by your objectives. Are you trying to sell more of Product X to Customer Segment A? Are you trying to retain more customers from Segment B? The narrower your initial focus, the less daunting the task will seem.

With the objectives defined, map out the different components of the purchase funnel and the non-linear pathways that customers or prospects may take to get to a point where economic value is created. How do customers find/enter your Web site? What steps do they take to download a whitepaper? At what points does your sales team interact directly or indirectly with customers or prospects? What role do channel partners play?

This exercise will provide insight not only into the paths themselves; it will also begin to shed light on how different behaviors influence one another and create purchase opportunities or other value-creating activities. To probe more deeply into these drivers, your next step will be to identify places on the map where you have good data and where you don’t. Look beyond the traditional customer survey information, brand-tracking studies, and the CRM system. What Web analytics are you capturing? Do you have access to point-of-sale data or call-center transcripts?

2. Create a methodical testing process. For the areas in which you have little or no data, make assumptions based on experience — and then test your assumptions by constructing experimental designs. Understanding the chain better can provide insights into the value of specific activities. This will enable you to begin validating previously fuzzy relationships between, for example, word-of-mouth referrals and cash-register sales. Focus these experiments on one or two areas at a time, replacing assumptions with facts as you go along to plug holes in the map.

“So many companies want an ‘e=mc2’ formula for engagement,” says Forrester’s Haven. “But it’s really a matter of testing. It may be quantitative, it may be ethnographic, but with so many touchpoints, you need testing to get the answers you need.”

Applying more disciplined, scientific techniques will help marketers understand the net impact of specific interactions, and thereby determine which levers are the most beneficial to pull for a positive outcome.

3. Look for predictive validity of upstream behaviors. As you fill in the data gaps, you’ll be in better position, through multivariate, linear, or non-linear regression techniques, to add more rigor to your predictive modeling. For example, a customer who downloads a whitepaper gives us 80% confidence that she will speak positively to someone who’s considering our brand. And if a customer speaks positively to three people, there’s a 75% likelihood that one of those people will have a direct transaction with us.

Be careful not to blow your credibility by holding up small, isolated pieces of the puzzle as predictive. True predictive models encompass all the drivers of engagement and, importantly, the interrelations among them. Trying to draw a straight line between awareness, for example, and sales puts marketing in a vulnerable position and makes it easy for other stakeholders to question allocation decisions. Finance knows that awareness alone doesn’t put food on the table.

4. Leverage your engagement drivers. Once you find the correlations between upstream behaviors and economic value (e.g., transactional behavior), you can isolate the likely drivers of more economic behavior and begin focusing marketing activities on extending or leveraging those drivers. For example, if you see that word-of-mouth references actually make the referrer 60% more likely to re-purchase, then you might want to invest more in promoting referrals amongst your current customer base. Alternatively, if you find that e-commerce activity at your site spikes when certain influential bloggers post news of your wonderful products, you may elect to increase the frequency of your news releases and provide those bloggers with advance release notice.

The goal is not just to find the engagement behaviors that are indicative of profitable economic outcomes, but then to build your marketing (and potentially selling) programs around the goal of stimulating more of those behaviors. 

Breaking It Down

There’s no question that developing a set of useful metrics around an amorphous topic such as engagement is a significant challenge for any organization. That’s why it’s best to break the big, scary engagement monster into a series of smaller elements that are more easily tackled.

By nature, the components of engagement will be very dynamic; very often, change will be the only constant. So you’ll need to develop ways to test and learn quickly and use that knowledge to build the foundation for a broader engagement strategy. Guided by a solid upfront vision, this type of engagement learning platform just might be the difference between distinguished success and momentum-killing mediocrity for your marketing programs.