5 Ways Marketing Forecasts Flounder

 

Most people think of forecasting in terms of revenue levels and associated budgets. For marketers, however, forecasting extends well beyond expense projection into interim effects such as brand equity, price elasticity, response rates, and channel development. In other words, forecasting isn’t just about the top and bottom line, but every line in between where we define metrics and milestones.

Yet forecasting is something in which so few marketers are well versed, never mind educated or trained. For some, avoidance of formalized forecasting emanates from a lack of expertise in what they perceive as a rigid mathematical science. Decades of realizing that their intuitive instincts aren’t always the best predictor of outcomes beyond their control has taught them that it’s better to sit in judgment of the forecasts of others than to originate them.

That’s what makes forecasting so hard: Its success lies not so much in the numbers, but in human dynamics. People across an organization have different viewpoints on what should be forecast and the most effective methods for doing so. Withholding information that would aid the forecasting process, opting instead to use that knowledge to their own advantage down the road (people like to look smart in meetings), is a common practice. Others create sophisticated models that focus only on the data they have at their fingertips, when in reality the data they don’t have could be far more relevant for a given forecast.

Forecasting should be a critical element in the strategic planning arsenal, and a CMO who allows it to be marginalized by conflicting agendas or the marketing team’s aversion to statistical analysis will be hobbled. A structured approach to forecasting, on the other hand, can improve the effectiveness of marketing programs and help avoid costly mistakes traced to faulty projections — which in turn do serious harm to marketing’s credibility across the organization.

To improve forecasting proficiency in your marketing department, consider these five common missteps, along with suggestions on how to avoid them.

Mistake #1: Delegating to the “quants.”
For many marketers, “forecasting” conjures visions of hard-core quantitative analysis. This frequently leads to one of two reactions: avoiding the practice altogether or passing responsibility to data analysts to crunch and re-crunch the numbers. The latter has picked up steam recently as companies collect increasingly more data and feel obligated to mine it heavily for every available insight.

By focusing too much on the data, however, CMOs may be ignoring other elements that are critical to increasing the accuracy of their forecasts: namely, judgment and experience. “Using stepwise regression on historical data to pick your variables is a bad way to do forecasting,” says Scott Armstrong, a Wharton marketing professor and author of Principles of Forecasting. “The added complexity will lead to reduced accuracy.”

Instead, Armstrong prefers a blend of structured judgment and statistical analysis, which he sees as the wave of the future. One form of this approach is “rule-based forecasting,” which integrates the real-world experience of experts into the decision-making process, providing often critical information that traditional data modeling may not take into account.

Armstrong offers one simple example: A quantitative study using a technique like exponential smoothing shows that sales of a product are trending up. Based solely on that trend line, management might decide to invest more in that product to maximize the growth opportunity. In reality, however, other causal forces at work — outside of the data set — might have a negative impact on sales of the product. If a sales manager hears that a new competitor is about to enter the space, then that knowledge must be captured and fed into the forecasting model as another variable that could change the outcome.

“You can have your managers record their expectations about the direction of the trend and use this to override the model’s predictions,” Armstrong explains. “Any time the trend in the data is contrary to the expectations, you don’t use the trend. You just say, ‘The best we can tell is that sales forecasts will stay flat.’ That simple rule has been shown to lead to enormous improvements in accuracy.”

Such an override function does not mean that a sales manager can simply ignore a given data point; it must be incorporated into a structured process in which the judgment is explained, recorded, and weighted as any other variable. That way, post-analysis (with the benefit of hindsight) can better gauge the quality of the judgment (as well as the judge) and promote continuous improvement in both the process and the individual skills.

“By structuring human judgment, you can improve the accuracy of the forecast,” Armstrong says.

Mistake #2: Reinventing the wheel.
For centuries, mankind has been attempting to accurately forecast everything from weather to lunar cycles to presidential elections. And certainly for the better part of the last century, much of the emerging science of forecasting has made its way into contemporary management practices and been thoroughly tested and vetted in millions of scenarios. So why do marketers keep trying to build entirely new models from scratch in the belief that their situation is unique?

“Most people have no idea how rich a set of techniques is already out there in the science of forecasting,” says Armstrong. “The knowledge is there, and it’s easily accessible. It’s mind-boggling why practitioners don’t use it.”

Granted, Armstrong has an agenda here: He created a Web site, forecastingprinciples.com, which aggregates 50 years of knowledge about forecasting methods. The site is designed to help practitioners, researchers, and educators learn about various techniques and choose the methods most relevant to their needs. The content is based heavily on real-world uses of forecasting techniques, much of it culled from the principles outlined in Armstrong’s book. (The site is maintained as a public service by the International Institute of Forecasters.)

“The site is part of the effort to move toward evidence-based management,” Armstrong says, citing similar efforts by the health care, education, and social welfare sectors to take complex studies and academic research, translate them into practical, intelligible advice, and post the information on a central clearinghouse that anyone can access.

There are literally dozens of possible ways to forecast something, depending upon what information is (or isn’t) available and the degree of accuracy or certainty needed. Developing a better understanding of the options (see “Methodology Tree”) and selecting one that meets the needs will help promote a more efficient, credible, and productive outcome.

Mistake #3: Locking in on a single forecasting method.
Using a single technique — regardless of whether it is regression analysis, a Delphi study, a prediction market, structured judgment, or something else — rarely provides the full picture needed to make accurate projections. On its own, each may provide a limited view of the future (or the past). Combined, however, these methods can lead to far richer, and more accurate, projections.

Econometric models will help unearth relationships and historical trends, for example, while Delphi studies (comprising a panel of experts from inside and outside the organization) can lead to additional insights about competitors, perhaps, or a shift in consumer behavior, that won’t show up in historical data. “There’s a lot to gain by [combining] structured uses of managers’ judgment and quantitative methods you can use on historical data,” Armstrong says.

The key here, of course, is getting those managers to buy into the program and share their knowledge. A CMO’s relationship-building and influence skills are as important here as his acumen in statistical analysis. Getting the primary stakeholders to agree on the objectives and the proper mix of methods for the forecast — up front, before the project begins — will greatly increase its probability of success.

Mistake #4: Using as many variables as possible.
Managers often try to anticipate politically motivated challenges to the credibility of their forecasts by incorporating every possible variable that may be even remotely relevant to the question at hand. Others, in an altruistic effort to engage key stakeholders in the forecasting process, solicit the input of multiple parties and quickly end up with a model that appears comprehensive but in reality is far too convoluted to understand. In either case, the result is usually the same: death by complexity.

Generally speaking, forecasting models need to be thorough enough to be credible, but not overly complex at the expense of clarity.

“People think that the more variables they throw in, the better the forecast. But when you do that, you actually harm the forecast,” says Armstrong. Instead, he suggests working with managers and other experts to choose only the variables that will have the greatest impact (positively or negatively) on the forecast.

“Take the problem, write down what you think are the most important variables, ask some experts [both within and perhaps outside your firm] to rank the variables and the expected impact of a significant increase or decrease for each,” he explains. “Separating the key variables from the noise and establishing clear hypotheses about the direction of the effect for each variable is the most important part of developing a causal model.”

Bottom line: Think quality of variables, not quantity. The former gets lost in the latter.

Mistake #5: Missing the opportunity to learn from mistakes.
It’s human nature to conveniently forget the predictions that don’t turn out as expected. Weathermen and sports columnists do it all the time; marketing executives do so as well. But tracking the performance of forecasting methods is the only way to improve their future success. Continuous improvement comes from monitoring not just what works, but also the elements that are not working.

Forecasting is all about iterative learning — the experience gained from methodically adding incremental insight into the collective knowledgebase of the management team based on every success, failure, and ambiguous outcome. Unfortunately, time-pressed (and tenure-challenged) CMOs rarely have an opportunity to step back to review those outcomes with any rigor. This exacerbates the gaps in the learning process.

While it may be unrealistic to expect a marketing team to perform a detailed analysis of historical error rates of various forecasts over a span of several years, a CMO can (and should) schedule a quarterly or semi-annual review of forecasts made vs. actual outcomes observed. Such a meeting can be used to discuss how the forecast might have been more accurate; how that knowledge can be used to improve future forecasts; and what reasonable expectations for accuracy should be established in the name of building credibility.

Summary
Marketers frequently attend conferences and workshops to learn about competitive strategies and the latest marketing tactics, rationalizing their attendance as an investment in making smart decisions about what to do and how to do it. But by comparison, they spend virtually no time learning about methods to improve their ability to predict what will actually happen as a result of their actions. A solid understanding of forecasting methodologies and techniques is a critical step in improving the effectiveness of resource allocation strategies and enhancing corporate credibility.

“Effectiveness” is a function of performance vs. expectation. In order to gauge how effective investments are, marketing leaders must first have a clearly formed expectation, which in most cases should be backed by quantitative evidence. Regardless of whether you’re planning a new advertising campaign, targeting a new customer segment, or creating a go-to-market strategy for a new product, the ability to develop an effective forecast — while avoiding the mistakes that marketers often make — may hold the key to success.