We are a specialty consulting firm exclusively focused on measuring and improving the financial return from marketing investments. With experience in dozens of industries, we use a broad toolkit of unique approaches to find just the right way to break through the political, cultural, and structural obstacles to help you crack your toughest measurement challenges.

Featured Webcast: Tony Palmer, SVP & CMO, Kimberly-Clark

View our one-on-one interview with Tony Palmer as he discuss marketing performance measurement in the latest installment of Measured Thoughts.

How Can You (Re)build Trust in Research as a Measurement Tool?

It’s a given that some elements of marketing effectiveness measurement are only handled through market research. Sound, insightful, directive marketing measurement is highly dependent upon research-driven insights to fill in data gaps about where things are today and hone assumptions about where they are headed. Used in this manner, research is a valuable contributor of insights about the potential payback of investments across the marketing mix.

Somewhere along the way, however, market research seems to have drifted off its strategic course. Instead of being used as a forward-looking instrument to increase the value (and reduce the risk) of marketing investments, research departments may find themselves relegated to the role of scorekeeper, used to rationalize decisions or track the status quo. In some organizations, the research function is entrusted to junior-level marketers who possess neither the technical expertise nor the political stature to raise its profile.

In short, research is, in the eyes of many marketers, suffering from a credibility gap that undermines its influence and impact as a strategic tool for discovery, learning, and the deployment and measurement of marketing resources. The result: Some executives are losing faith in the capabilities of their marketing research groups.

It may be an opportune time for CMOs to take a hard look at their research departments. By redirecting research toward a contributing role to help improve marketing effectiveness measurement, marketing leaders can begin to rebuild trust in the research department throughout the organization.

Research’s Multifaceted Role in Marketing Measurement
Companies have no shortage of data at their disposal to fuel marketing-mix models. But corporate data warehouses don’t tend to spit out reports on customer experience/satisfaction or brand equity evolution, since those elements are not tracked at the transaction level. (At best, they are “sampled” at the transaction level, such as when a retailer gives a receipt that encourages the buyer to fill out an online survey.) Research is required to fill these information gaps by sampling and reporting on elements for which there is no “systemic” data, including: levels of salient awareness; levels and nature of pre-purchase consideration of the brand, product, or service vs. competitors; and distinguishing characteristics, attributes, or perceptions of the brand or company. There is no other way to collect this information than through research.

Research also plays an important role in illuminating the decision process that precedes purchase (or re-purchase). This process includes mapping the various influences that cause groups of prospects to become customers, or groups of customers to stay with a brand, and then testing the relative sensitivity of those key influences and the tradeoff relationships between them. By stringing those influences together and examining the funnel progression of various segments of a marketplace, research helps to identify the stages prospects/customers pass through on their way to purchase. By monitoring how many and which types of prospects are at each stage, researchers then can develop insights into what tools or strategies marketing should be focusing on to accelerate the process. The outcomes of this work form the bedrock of assumptions required to build forward-looking simulation tools incorporating transactional data.

Beyond the buying process, research also helps shed light on the influence of elements that lie beyond the marketer’s direct control. These elements cover a broad spectrum, from word of mouth and reputation to interest rates, carbon emission levels, or any cocktail of micro- and macro-environmental factors. The role of research here is to help set and manage expectations for how the marketer’s toolkit must be stocked in order to meet the nature of the evolving challenge.

Research can apply similar filters to the competitive environment, so that management teams always understand how customers perceive their company or brand in the broader context of the marketplace. By providing insight into the minds of those who really matter — the customers — research makes management less likely to fall into the trap of its own perceptions, which can become highly insulated over time.

If marketers view their dashboard as a pair of headlamps for illuminating the business direction in otherwise dark and foggy conditions, the role of research is to tune and sharpen those headlamps so they are pointed in the right directions at the right targets. Without research looking ahead into the uncertainties, marketers run the risk of ending up precisely where the dashboard is pointing — which, by the time they arrive, may no longer be the desired destination.

Challenges and Opportunities
Unfortunately, several forces have contributed to undermining the credibility of research in fulfilling this measurement directive. The challenges span organizational, cultural, and functional boundaries; we summarize four here, and offer suggestions for addressing each barrier.

1. Extreme delegation. Too often, responsibility for research programs is pushed toward increasingly junior-level staff, or outsourced completely. We’re not talking about delegating responsibility in order to execute more efficiently; the problem arises when leadership delegates responsibility for defining the questions that need to be answered. People who lack the subtle, contextual understanding of how many variables might coalesce to form a clear perspective on marketing’s payback are essentially being asked to both define the questions and direct the methodology to answer them. And that’s a major obstacle. “If you’re just getting an order handed to you without questioning why you need to do it, you won’t understand the problem clearly up front,” says Mike Lotti, AMA chairman and retired director of business research and vice president of corporate marketing at Eastman Kodak.

If you’re one of the few people (read: a CMO) who can see how all the angles intersect, you’d best be sure your research is delivering the answers you need to fill in the mosaic. A periodic review of key questions facing the business and a process for prioritizing how to go about answering them will put research initiatives on the proper path. Spending a few hours each quarter focusing on deriving semantic precision and clarity to ensure the highest probability of retrieving useful insights should pay big dividends. Painful, perhaps, but the payback can be significant.

2. Dilution of resources. Increasing demand for research support across business units stretches the research staff (and budget) so thin, it’s no wonder they are permanently caught in a cycle of finding lower-cost, faster-response solutions. “When you’re emphasizing speed and lower price, then you stop looking at how you can make the research more valuable,” says Bill Cook, senior vice president of research and standards at the Advertising Research Foundation.

A research group that tries to be responsive to all corners of the company can actually produce poorer results. One-off projects, commissioned by individual business units, tend to get lost in the shuffle and quickly forgotten, minimizing the prospect of any insights emerging around long-term payback of investments. Internal studies pile up, with little pollination across businesses to provide a more robust picture of customers or market segments. Not only does this result in missed opportunities, it actually drives up costs because multiple units are quite possibly commissioning redundant studies. The sins of the past decade in this area are now coming back to haunt many companies that lack continuity of knowledge, as ad hoc efforts are started and then abandoned due to “lack of insight.”

To address this problem, consider developing a framework to assess the relative potential economic value of each proposed research initiative. It doesn’t have to be too precise — orders of magnitude will do — but by measuring each project on the net expected incremental value (impact less cost), one can clarify which projects really warrant the attention of senior staff and the additional cost of better samples, techniques, and/or methodologies.

The framework should be reviewed quarterly, with projects scoring lowest on the scale either outsourced or killed. Empower your researchers to turn down requests for projects low on the expected value chain or those for which reasonable, proximate answers already exist. And give them some air cover to protect their political exposure in doing so. Remember, the single greatest cost to the organization isn’t the cost of doing research; it’s the cost of not doing the right research, or not doing the right research correctly.

3. Fear of the uncertain. Note that we said “uncertain,” not “unknown.” Researchers generally love wading into the acknowledged unknown. But it’s also safe to say that most researchers, particularly those at the middle or lower levels of the organization, want to keep their jobs. That’s why, when they take on a project, they look “in the numbers” for answers, wrapping themselves in a statistical security blanket. This survivalist motivation often results in fitting solutions to problems not on the basis of the validity of the answer, but on the statistical significance of it. “Saying that 52% of consumers will like a new product isn’t enough,” says Lotti. “You have to walk out from behind the data and demonstrate value.”

More valid results to hard questions can often emerge from qualitative, even internal decision-calculus efforts to frame and simulate questions and answers. However, these techniques are increasingly a lost art in a world driven by quick statistical support to justify a new project. The result: knee-jerk tendencies toward familiar methods and vendors, even if they are a poor fit for the problem at hand. “Marketers don’t buy what they need — they buy what’s available,” says Don Schultz, professor (emeritus-in-service) of integrated marketing communication at Northwestern University. “The suppliers are driving the content based on the tools, not the insights you’re trying to get.”

To break out of this value-destroying cycle, the research department should be tasked with studying and proposing a playbook of decision-making techniques to be used when the expected value of proprietary or secondary research isn’t clear. Make them responsible for working with their constituent groups to develop greater clarity of purpose, richer understanding of expected value, and more efficient use of time and money by having some tools to move projects from subjective “wants” to more objectively prioritized “needs.”

4. Vision impairment. Many research teams struggle with an inability to see the comprehensive picture of how research needs to fit with analytics, experimental designs, and structured guessing in the overall approach to triangulating on the question of marketing’s payback. This can lead to a well-known syndrome: Every problem looks like a nail, so hit it with your hammer. How else to explain why so many tracking studies are done annually or semi-annually; research groups fail to realize the irrelevance of semi-annual data input into a model attempting to optimize quarterly (or perhaps even monthly) resource allocation.

Some CMOs have attempted to address this problem by changing the title of research to “marketing intelligence” and giving their research directors responsibility for analytics too. But you can’t change a zebra’s stripes just by calling it a lion. It still thinks and sees the world like a zebra. To grow and evolve, an experiential filter requires exposure.

The lesson here: Only mushrooms and tree frogs grow in the dark. CMOs who need more from their research team need to bring them into the light (or better yet, ask them to help create the light). Make sure their activities are coordinated with the functional and line-of-business owners and integrated into an overall plan to attack the prioritized needs. “Researchers have to understand how the business makes its money, how it uses its assets, the intellectual property stream that goes into it, how brand equity plays into the purchase, the sales force, channels — the whole nine yards,” says Lotti. “You have to be able to make your point on more than just the data. You have to bring in how it’s going to play with everything the business uses to go to market.”

A Long Road Back to Credibility
Marketing research’s credibility problem didn’t spring up overnight — and it’s unlikely to get back to its mission-critical perch anytime soon. Here’s what CMOs can do to at least get the ship pointed in the right direction:

Set the right frameworks for clarifying the questions you need to answer and the methods you’ll use to prioritize your constrained resources (time, money, management attention). Revisit it quarterly to ensure continued relevancy to the issues at hand.
Invest in building the skills of your research teams to learn new ways to answer questions beyond the traditional tool set.
Engage your fellow constituents in sales, finance, and the business units in the upfront planning of research so everyone understands and supports the inevitable tradeoffs being made in the design and execution of research projects. And empower your research team to say “no” when there are more cost-effective ways to pursue answers.
Resist the temptation to delegate too quickly (or outsource entirely) sensitive tasks where questions of bias or subjective interpretation may undermine the credibility of a larger set of conclusions derived from the component parts. Remember, a measurement framework is only as credible as the weakest assumption among many.

With these baseline elements in place, marketing research can once again begin to play a vital role in optimizing marketing investments and contributing to the long-term value creation that businesses need to survive.