Measured Thoughts: Sean Hagerty, CMO, Vanguard

A dialogue on marketing measurement, featuring Sean Hagerty, CMO, Vanguard

Vanguard is a mutual fund company — with about $1.3 trillion in assets under management — that serves the investment needs of multiple audiences, including individuals, institutions and financial advisers. Such diverse target groups make life challenging for CMO Sean Hagerty, who oversees what he describes as “strategic process.” Hagerty sat down with MarketingNPV Managing Partner Dave Reibstein to talk about Vanguard’s business, the role of marketing, and the processes in place for measuring marketing’s success. The interview is part of MarketingNPV’s “Measured Thoughts” webcast series; the edited transcript follows. To view the video, click here.

Dave Reibstein: Tell me about your background and how you got to where you are today.

Sean Hagerty: I actually grew up in the business side of the institutional business at Vanguard and managed sales and client services organizations earlier in my career. At Vanguard we believe in management by rotation. I had rotated into a few different operational areas and client service areas. About five years ago I was asked to head up the marketing organization, which was almost a brand new discipline for me. It was an interesting assignment because there was so much for me to learn, but also so much of just basic business knowledge that I could leverage.

Reibstein: Tell us about the role of marketing at Vanguard.

Hagerty: Marketing at Vanguard is pretty well defined in that we are organized into several businesses based on the segments that we serve. And marketing is a centralized organization that is responsible for a few things in what I think of as “strategic process.” First, we have a pretty large client insight organization that is there to help inform those business segments and help them set strategy. Obviously, some part of what we're doing in marketing is trying to think forward ... what are those things that we might be anticipating? What are some of those insights we might want to generate to help set business strategy?

The business units are responsible for setting that strategy. We’re responsible for then working with that to translate it into a marketing strategy and marketing plan to help the businesses achieve their goals, and then measuring those results. We think of ourselves being in tight alignment with the businesses and helping to set and execute the business strategy. Oftentimes we say – and I stole this from a colleague – that business strategy and marketing strategy are two sides of the same coin. And you have to have both working well in order to achieve business goals.

Reibstein: Has the role of marketing changed much in the last five years?

Hagerty: I think it has changed. Marketing is moving from an expense that has to be managed to an investment in the future. We're now saying, how can marketing give us competitive advantage? How can we invest in it appropriately and thoughtfully to gain competitive advantage? And if it can't do that for us, then we shouldn't spend. That’s a different mindset than, “Let's manage that expense.”

Reibstein: If I asked the same question about the role of marketing at Vanguard of your CEO, would I get the same answer?

Hagerty: I think you would. I think the CEO would answer it the same way – that we’re on a journey towards marketing excellence. We've had that conversation and I think we share that vision.

Reibstein: And would your CFO view it as an investment and not just an expense?

Hagerty: I think the CFO would look at it maybe a little bit more with a numbers orientation: Is it generating positive NPV and the kind of ROI benchmarks that we set for ourselves? I think that would be a little more precise, show-me-the-numbers discussion compared to what the CEO and I might say.

Reibstein: That takes me to the next question, which is; how are you measuring whether you’re adding positive NPV or any of the other goals that you're trying to accomplish?

Hagerty: To say that there is one catchall measure that says marketing is or isn’t effective is probably a little pie in the sky. There are some things we measure incredibly well and have a level of precision where we know exactly what the ROI or the NPV is on a particular investment. In others, where there is a lot more intuition involved as to whether we should spend that money, we try to build at least to some composite on a dashboard that tells us, based on all of the inputs and all the things we're trying to achieve for the business segments, whether that composite is trending up or down or staying the same.

Reibstein: So I suspect for direct mail pieces, for example, you see what direct response rate you get. But some of the efforts don't have an immediate response.

Hagerty: That's right. Also, in our business-to-business segments it's harder to say, “I need a suite of sales collateral to equip our sales team with.” That is a marketing expense, after all. But can I say that that piece of collateral or presentation or the positioning of work we do in a B2B environment is responsible for the next sale? It's hard to tie those things together. So we have other measures in terms of the effectiveness of the piece itself, but it's harder to link the effectiveness of the piece to actual cash flow. That's where we struggle a little bit.

Reibstein: Understandable. Just so that we understand Vanguard completely, what’s the split roughly between B2B verses B2C business?

Hagerty: The direct retail business is a little bit more than 50 percent of the business. Our institutional businesses, which include dispersing our funds through other financial intermediaries and directly to institutions, represent just less than half of the business. In terms of trajectory, the financial adviser business probably has the highest growth rate.

Reibstein: I want to make a transition for us to think about setting your marketing budget. How do you go about determining what that budget should be?

Hagerty: We start with, what are the business goals? We first want to work with our partners in the business segments and ask them what they are trying to accomplish.

Reibstein: Give me an example of one of those business goals.

Hagerty: For us an important measure is net cash flow. How much are we getting from clients or prospects in terms of contributions into the mutual funds? So we might have a goal that says we're hoping to achieve a certain amount of cash flow sales from our sales team. Or we might have a goal of improving loyalty among the client base, or increasing retention. If it’s improving loyalty, then we’ll start to think about what we might want to do from a marketing perspective to help engender loyalty or to help increase share of wallet or whatever that goal might be.

And we’ll essentially build the budget from that [based on] what we think is going to be necessary to accomplish those goals, both long-term and short-term. We’ll build the budget bottom-up and have a discussion at a senior management level about both the level of spend and spend across business segments. We can shift money between business segments depending on what we think the priority is in that particular year.

Reibstein: You described the process you go through for setting budget and then also how you determine the level that you should have. How do you determine how to allocate that?

Hagerty: There's media mix for advertising. If we're doing project-based work, we are using NPV tools and other measurements. Different activities have different measures. So if we have an awareness goal, is advertising the right thing to do to or is it some other mechanism? And within advertising, what channels work best? [We also look at] what we have learned from past activities. We build based on prior knowledge.

Reibstein: So it sounds like you have it fairly well developed. What tools or models do you have?

Hagerty: I think we're getting there. We try hard. I think that most of our spend we do this with, but with some of the spend it's still a little bit gut check more than pure tools. But essentially what we do is try and run experiments on a constant basis and help learn from those experiences and then apply that to future activities.

Reibstein: So would that experiment be, we’re going to do a limited direct mail and we’re going to see what the response is?

Hagerty: Yes. And we hold control groups in every single piece of direct mail we do. As an example, we did a design of experiments in the first quarter of last year where we did a significant increase in weight of advertising in two local markets. We varied the different channels in those markets and then had two control markets and tried to see does weight make a difference at certain levels? And what happens with respect to the allocation of the money across different mediums? We got some pretty significant learnings from it that we are now applying to next year.

The reason I think it's so important to test is, you can read all the textbooks you can listen to a lot of colleagues, but I think it’s different for every industry and probably every company. And so you have to find out what works for your audience and for your business model.

Reibstein: Again, to give me a sense, how much of your overall budget roughly is for direct mail vs. more general marketing efforts?

Hagerty: Direct mail is actually a very small percentage of our budget. For our business model, in the retail business, direct mail has not been all that successful for prospecting as a strong mix of media from an advertising perspective.

In addition, Vanguard happens to be a very significant word-of-mouth company. That is how we've grown up over the past 30 years. We're fortunate in that we have that and therefore probably don't have to spend to some levels of our competitors because our clients are incredibly loyal and they're willing to tell others.

Reibstein: Do you measure word of mouth?

Hagerty: We are just starting to measure word of mouth and trying to understand it better. It's an interesting dilemma in that, do you actually want to do anything about word-of-mouth or do you let that stay as is? We’re having a lot of discussions about that now.

We do try and understand, for clients who are interested in Vanguard or who are new to Vanguard, how did they know about Vanguard in the first place? And it's interesting that for new clients, word of mouth is still the predominant source of the initial awareness. And our website and advertising become more important actually as they move into a decision-making mode.

Reibstein: After a test convinces you to roll something out, how do you evaluate whether it was worth it?

Hagerty: I like to think of long-term and short-term measures. I’ve seen the pendulum swing back and forth based on our own sentiments of what we want. Sometimes I’ve said, “The marketing spend really has to generate positive NPV and I want to see that immediate response.” And other times we’ve swung to, “Well, it’s not about that, it’s about brand health measures, awareness and image attributes and space between us and the competition on particular image attributes.”

I think we’re settling on some combination of the two. We’ll take some part of the spend and we’ll say, what part of the spend do we think should be NPV-oriented and what part of this do we think is image and brand oriented? And we set goals beforehand in terms of how far we want to see them move or this is what kind of a gap we want in between us and our competition. So we will measure against those.

Reibstein: So how are you, or others, held responsible? Is it how much of your goals did you accomplish?

Hagerty: Ultimately how I'm held responsible is in the composite: does it all make sense? We're encouraged to take risks – not foolish risks – but to try new things and to experiment. It's an important part of our culture.

Reibstein: So as a company that learns so much from experimentation, I take it it’s just as valuable to have experiments that don't work so that you can learn what not to do.

Hagerty: Absolutely. Learning from failure is something that makes sense whether you’re doing it as an individual or an institution. We don't try and say, let's ignore our failures. We say, let's actually learn a ton from them and bring them up to the surface.

Reibstein: What I’d like to do is think about some the tools that you might use for measuring the effectiveness of any of the marketing. In particular I’m trying to find out what is it that you really look at for measuring intermediary results vs. some long-term results. You mentioned you've got objectives on both.

Hagerty: I think an important part of what we do is that we have a partnership between the marketing organization, our business units and our finance group. And we have finance professionals who are dedicated to the marketing group. They sit with us, so they are part of the team, but they report up through the CFO. We work with them to think about lifetime value models and differences based on business segments.

Reibstein: On a customer-specific basis, you would have a lifetime value of that customer? Or groups?

Hagerty: I don't want to say that we would have it per individual. But we would be able to look at certain segments, people who come to Vanguard at a certain level. And it’s fairly predictive that if somebody comes in at level X and they look like this they're probably going to have a [particular lifetime value].

In the institutional business it’s much more pure cost accounting because they are very large relationships that require a fair amount of analysis to even think about how do we want to price that piece of business. So we have a pretty good understanding of what the lifetime value of that client is. And we’ll use that in conjunction with marketing activities to start thinking about, all right, well how much can I spend to attract the next incremental client? And we’ll look at both cash flow and some of the long-term measures.

I know the next question might be, and how do you link the long-term measures to the short-term measures? So does awareness and image attributes translate to sales? And I don't know how to answer the question. That I think is the Holy Grail. We have not really solved that. But we are very comfortable in saying we understand the value. What we do at least is say, this is what you would have to believe. And I think that's an important concept that even if you don't have the precision tools, that you make people go through the spreadsheets and go through the analysis to say this is what you would have to believe in order to make this investment.

Reibstein: Sort of the inference of, if we get this it will evolve awareness that will translate to ...

Hagerty: Exactly. So at least make the assumptions and think about it. In our ETF [Exchange Trade Funds] business as an example we said, if we invest in this business what would you have to believe in terms of cash flow, market share, in order to justify this amount of spending? And so we ran those models and said, well it's reasonable to think that if we spend this we can get this amount in cash flow and market share. And in fact we are.

That was an experiment where it was difficult to understand exactly what we were spending and what we were getting for it on a short-term basis, primarily because with financial advisers and intermediaries it's a little more opaque.

Reibstein: Thinking about the purchase funnel, there are intermediary sorts of measures that we have which ultimately lead to some financial measures. And then there is a time sequence – how long it’s going to take. Do you have as much faith in the long-term consequences as you do in the short term?

Hagerty: Obviously, no. There has to be some leap of faith to say that those long-term consequences matter. And interestingly some of the data we use to help us there is when we get into a client we ask them, When did you first become aware of us as a company? It’s self reported so it is always subject to some amount of risk. Amazingly, often it is years between first awareness and initial purchase. So it helps you believe that getting that first level of awareness in a prospect is an important long-term goal even though the client might not buy from you for three or four years. But I will tell you, if there’s any place that’s hard to justify spend, that’s it.

Reibstein: Do you measure the Vanguard brand value?

Hagerty: We don't measure a numerical brand value primarily because we're not a publicly traded company. We're owned by the shareholders of the firm. Sometimes we would toy around with, well if a similar company with this amount of assets just got sold on the marketplace at X maybe we might be worth Y. The fact is what we do know is that the value of the brand is astronomical compared to the value of the assets of the company. And therefore we think it is incredibly important to think about maintaining and protecting that brand value. And that brand value has been built up primarily by serving now millions of clients whom are very loyal to Vanguard. And that we take with an incredible amount seriousness. We measure satisfaction and loyalty almost at a religious level and make sure that those measures are going the way we want them to go.

Reibstein: How do you measure loyalty?

Hagerty: We went from satisfaction to loyalty about 10 years ago. We derive loyalty through multiple measures. Regression analysis, to try and figure out what was loyal vs. not. We have moved to the Net Promoter Score, which you might hear from other companies. We did so for a few reasons. One is we found that some of the research we were doing was starting to get voluminous. There was one survey where we asked something like 90 questions of one group of individuals, and that starts to border on the ridiculous. So we like the simplicity of asking one question with Net Promoter. Though we think it's also important not to just measure Net Promoter Score but also to understand the drivers of the Net Promoter Score. So we cheat. We don't ask one question. We ask one question on kind of a pulse level. But we’ve set it up to where at least every other year we're going to do research to understand the drivers of that goal.

Reibstein: And I'm finding a lot of companies do that because they don't want to just know where they are, they need to know what they need to do to change that.

Hagerty: That's right. You really have to think about what drives that outcome.

Reibstein: Do you see the role of the research changing in any way to help support some of your measurement efforts?

Hagerty: I think research has changed over the past several years for us. Research has become more strategic in terms of helping to understand where we should be going and more scientific in measuring the results. Some of the things like Net Promoter Score and some of the brand awareness and image attribute work that we have done has gotten better.

The other thing that's important is that research is not research anymore for us. It’s client insight. For us, client insight is the marrying of data with research. So it's one thing to get a Net Promoter Score. It's another thing to take that score and marry it with the underlying data and behavioral attributes to help you understand a lot more about that Net Promoter Score. I’ll give you one example: we have advice services that we provide to our retail investors. We had an NPS score that we thought was OK but we wanted to improve the score. We did not just driver analysis but we looked at the underlying individuals and we looked at their holdings, their asset level, their age. We started to really slice and dice the data and by doing that, find out that there was a segment of the population that was less satisfied than others. We then changed the service model in order to accommodate that and saw a doubling of Net Promoter Score.

Marrying data with the research is where research has to go. We have a ton of data. We have a lot of research. Putting that together and creating insights that can actually change your business model, that's where the power is.

Reibstein: Where do you think the analytics are lacking the most?

Hagerty: I'm not there yet with predictive analytics. To date I haven't been as happy with our ability to say, let me run a lot of regressions and I’m going to tell you who's likely to leave as an example. I think there are a lot of false positives in that kind of stuff. And so far I haven't been willing to use it all that much in the service model. I might use it for targeting for marketing purposes. I wouldn’t want to really incorporate it in the service model because I think sometimes you just look silly.

Reibstein: You mentioned earlier that sometimes you have to make gut decisions. I’m curious that in the absence of data, what’s the process you go through for making some of those decisions?

Hagerty: It's a combination of things. There's always some data available. When we talk about client insight we talk about that sweet spot between intuition and data, because too much data can start to send you in odd directions. So whether you have tons of data or no data, we think that you have to find that balance between whatever data you can find and business intuition. And importantly, I think the way to make those decisions is to encourage debate and to encourage lively discussion. So we’ll get in a room sometimes on a topic where we’re not 100% sure which direction we want to go and we’ll set up formal debates – a group for and a group against – and we’ll listen to it. Ultimately it's not democratic – the person in charge makes the decision and that's the direction we take. But I think the best way to make those intuitive decisions is to be open to a lot of thought, have a lot of healthy debate and then ultimately move forward, get everybody on the same page and go one way or the other.

Reibstein: And that's where you make those implicit assumptions more explicit so you can debate the underlying assumptions.

Hagerty: That's exactly right. So you have to challenge those assumptions and then look for some underpinning of them. Is there some reasonable expectation as to what you believe or is it flimsy? And that's why the debate’s important.

Reibstein: We've talked about some of the measures that you have and how you go about doing that. I want to move to talking about how it gets reported. Let me start by asking who’s responsible for measuring marketing’s performance?

Hagerty: Measuring marketing’s performance is done in two areas. Our client insight organization has a lot of tools to help measure things, so that organization is responsible for helping to put together a lot of the data, whether it's research data or actual client behavior. But the finance folks are the ones who really oversee that. And we have a metrics steering committee, where there’s business folks, finance folks and marketing folks. We try to agree on common definitions of success. We agree on goals and then we review the activity as a group and have some broad-based steering of what goes on in terms of marketing measurement.

Reibstein: Is the performance of the marketing also shared through your dashboard?

Hagerty: That's what we review at those meetings. We will look at individual campaigns and then we’ll look at what happens on the outcomes-oriented dashboard. I have a desire to have outcome-oriented dashboards for each of our business segments. Those dashboards are basically the funnel. So we look at the components of the funnel that are important to that business segment, because one business segment might have complete and total awareness. In our 401K business [for example], there are not that many human resources or treasury managers that don't know that Vanguard’s in the 401K business. So I don't even measure awareness, but I do care there about leads and client loyalty, so I’ll measure that.

We’ll measure whatever components are important to that business, put that on the dashboard, and then we’ll look at all the campaigns and other activities that we think are drivers of those outcomes, and we’ll look at the outcomes of those campaigns and then we we’ll look at whether they are driving the measures broadly that are represented by the funnel.

Reibstein: It sounds really fabulous. So what's missing?

Hagerty: Sometimes there's not as strong a connection between the individual activities and those outcomes. It's harder to prove because you need a lot of data points to say, OK, we’ve done the regression analysis and I know the activities I'm moving on the driver side are the reason I get that outcome. As an example, we had a significant increase in awareness earlier in this decade. I could have said, well it’s the advertising that did it. And somebody else could have said, well it was the PR. And somebody else could have said, it was word-of-mouth.

Reibstein: Or the new CMO.

Hagerty: Or the new CMO [laughs]. It wasn't that, for sure. So the perfect attribution of the advertising as the element that drove that awareness is what's missing. So that still engenders a ton of debate and it’s where the intuition has to come into play to say, Even though I don't have perfect data there I believe that the advertising is a substantial contributor to the change in awareness.

The other place we have misses is within the business lines. Some business lines have it easier than others in terms of the way they measure things. I'm going to meet with the head of our financial adviser business in the next few weeks to talk about whether I should spend that next incremental marketing dollar on sales or should I spend it on marketing? What’s going to help me to drive that next incremental ETF cash flow? I don't think we have a perfect tools that would help her decide, all things being equal, should I increase my marketing spend or my sales spend and what's going to get me that next dollar? Some of those tools are missing.