Measuring the Impact of Loyalty Initiatives

By Kelvin Taylor 

In an ideal world, you would measure the impact of your loyalty program initiatives just like any other marketing initiative — create a comprehensive test and control experimental design in which you "control" for the significant variables and attempt to get a pure read on the incremental benefit of just the loyalty initiative. Comparing the difference in post- vs. pre-levels for the test group against those of the control group is referred to as repeated measures.

Unfortunately, reality often intrudes into our ideal world, making it difficult or impossible to carve out carefully designed test/control experiments. You may not, for example, be able to isolate the loyalty stimulus to just a subset of customers. But there is another approach you can take. It's called survivor modeling, and it can be appropriate for measuring loyalty initiatives when a significant component of the anticipated economic value lies in higher rates of customer retention.

 

In survivor modeling, we look at the expected value of a cohort group of customers over time, absent any loyalty intervention. Taking into account historical customer purchase and retention rates, we can create a forecast of the future value of the cohort customers at various points in time. This forecast can be modified by expectations about competitive activities, interest rates, or other external influencing factors that might have an impact on the business.

Next, we develop a forecast for the expected value of the cohort group with the added impact of the loyalty initiative to determine the level of improvement in purchase rates and/or retention we would have to see in order to justify the expense of the initiative. Once launched, we then carefully track the effect of the loyalty initiative on purchase and retention rates against our acceptable target levels to see if the initiative is, in fact, delivering at least as much as we expected or required. If yes, we continue. Otherwise, we consider options for fixing it or killing it altogether. Chart 2 shows one way the survival curves can be plotted as a graphic illustration.

Pros & Cons
One of the benefits of survivor modeling is that you needn't wait long to track results. As soon as you can confirm the desired change in behavior is taking place and "sticking," you can calculate the expected value of the initiative.

However, while survival modeling can be a very useful tool for measuring the impact of loyalty initiatives, it does have some significant limitations. First, this approach isn't likely to discriminate between small magnitudes of economic value change. If you're expecting only a modest difference in purchase or retention rates, survival modeling might not adequately measure the impact within a "short" time horizon. Second, survival modeling isn't a very good method for dissecting the interaction effects between the loyalty program and other activities simultaneously impacting customer behavior. If, for example, a new advertising campaign is launching at the same time as the loyalty initiative, survivor models won't effectively discriminate between the relative contributions of the two stimuli unless coupled with test and control groups.

 

Finally, developing the forecasts of future value of a group of customers can be chock-full of big assumptions. To build better, more credible forecasts, engage opinion leaders from finance, operations, sales, etc. in the process to get the best possible insights into assumptions. That way, while the mathematically "right" answer may still elude you, you will at least be assured of converging on a "better" answer than you started with, and one that more accurately considers your stakeholders.