Quantipulation: ROI Versus Success

[This is a follow-up post to Quantipulation. I thought I could get away with just floating a few ideas out there, but some comments I’ve seen suggest that there’s a lot more to quantipulation than I wrote about, and those comments are correct.]

Quantipulation — the art and act of using unverifiable math and statistics to convince people of what you believe to be true — is commonplace in the marketing world, but perhaps nowhere more so than in the social media environment. Especially when it comes to everyone’s favorite topic: Social media ROI.

Whenever I use the term ROI in my reports, the editor where I work asks me to spell it out. As she rightly says, there may be people who aren’t familiar with the term. I don’t tell her this, but if you don’t know what ROI is, I don’t want you reading my reports.

There’s another reason why she’s right: There may be people who define ROI differently than I do. I won’t tell her this, either, but those people don’t deserve to read my reports.

ROI = return on investment. It doesn’t mean return on influence or any other “I” word you can dream up. And despite what some quantipulators would have us believe there’s only one formula for ROI: Financial return divided by financial investment. The only “variable” piece to the formula is the timeframe you use to quantify these variables.

That won’t stop some people from trying to redefine the formula, however.

The most egregious example comes from a firm called Digital Royalty. I won’t besmirch my blog by linking to the offending post. Instead, I’ll point you to Anna O’Brien’s brilliant (and very funny) critique of it.

Here’s another example of ROI quantipulation:

My bet is that tthe firm that put this chart together wanted to include other ROI components, but since it would have messed up their inverted hour glass figure, they decided to leave them out.

Then there’s attempt at redefining social media ROI:

This guy has decided that the ROI unit of measure should be “conversation”. He goes on to tell us that we can measure the “value” of conversation by looking at participation, engagement, influence, imagination, energy, and stickiness. But not increased revenue or decreased cost. Sweet.

There are (at least) two things going on with these attempts to redefine ROI. One is bad, the other is good. 

The bad: An annoying attempt to demonstrate thought leadership. Ugh. Not the way to do it. Anna O’Brien said it best in her blog post: “Random metric names and symbols is not an equation.” (Maybe she didn’t say it best, because it should be “are not an equation”).

There is a good aspect to what the ROI quantipulators are doing, however. They’re raising the very valid point that there are other measures of success beyond ROI. 

There’s a formula for that, too. The one I like is from Pat LaPointe who writes a blog called Marketing NPV. Pat’s formula says that success can be measured by dividing the value added by the resources used. And as this formula implies, “value” can take on the form of many of those measures that those other people wanted to use to calculate ROI.

But this isn’t the whole formula.

Pat added something on to this formula that, as far as I’m concerned, qualifies Pat as a marketing genius. Pat’s formula for calculating success is:

(Value Added/Resources Used) * Perception

What Pat recognized was that what you might consider to be “value” might not be viewed as valuable by other people. Other people like, say, your CEO or CFO.

We’re living in an ROI culture. Suggest that your company do something, and somebody will ask “what’s the ROI on that?” If you want to get up in front of your management team and suggest that your company do something because you “feel” it’s the best thing for the company to do, go for it. Just don’t send me your resume when you’re on the street. 

That doesn’t make your feeling wrong. But being right doesn’t make you successful. Persuading others to do the right thing does. 

This is why quantipulation is so important:  Quantipulation is an attempt to influence perception. To be a successful leader, innovator, or change agent, you have to shape, change, and confirm people’s perceptions.

There’s a reason I call quantipulation an art. Successful quantipulators know that it’s about more than just the data – it’s about logic and emotion. And there’s no formula or recipe for figuring out how much logic and emotion to mix in with the data.

The examples of ROI quantipulation shown above fail not because they’re wrong, but because they fail to influence perception. Those formulas simply confirm for the social media believers what they already believe. That’s easy. Converting the heathen is hard.

Had those social media ROI formulas made any attempt to link social media results to the conventional definition of ROI — financial return — they might have been more persuasive.

Last thought: Quantipulation is not inherently bad or evil. Yes, it’s a play on the word manipulative, which doesn’t have positive connotations. But I prefer to take a more realistic view: It is what it is. And it’s a necessary skill for today’s business world.

Advertisement

Quantipulation

A guy named John Wanamaker is famous for something he said 100 years ago. He said:

“Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

Unfortunately, he’s wrong. I mean, if he didn’t know which half was wasted, how did he know it was half and not three-quarters or one-quarter of it?

He’s also wrong because it’s conceivable that 100% of his advertising dollars were wasted.

A century ago there were no ad ratings or measurement services. So how he could possibly know if ANY of his advertising spend was effective? It’s quite possible that any increase he saw in sales was due to exogenous factors like the weather, the economy, the competition raising prices or going out of business, or word of mouth among customers.

Ah, but hold on here a second. I guess it’s possible that 100% of his advertising spend was effective – or at least, not wasted – depending on what measure of success you use. If you don’t believe me, ask DeBeers.

Is it likely that the advertising he did had absolutely NO effect at all? Probably not. Just because someone didn’t make a bee line for the department store after seeing an ad, doesn’t mean the ad had no effect and should be considered wasted dollars. Some might have seen the ad and learned about the store, or the ad might have left others with a positive impression of the store.

Wanamaker thought half his advertising spend was wasted because he had no way to measure its effectiveness and didn’t even know what to measure.

Today’s advertisers have some measurement tools and services available to them, but none can claim to be totally accurate. And marketers are dreaming up new metrics every day, so you can be sure that no one measure is perfect, nor can we safely assume that even a group of commonly used metrics can truly give us a reliable picture of the effectiveness of advertising.

Bottom line: Any claim on what percentage of your advertising is wasted and what isn’t is just a random guess. We simply don’t know – and can’t know.

Here’s another claim to consider: Have you heard that its costs five times more to acquire a customer than to keep or retain one? How did they figure that? You could double the number of insurance, credit card, or mortgage customers you have by simply tweaking your underwriting guidelines, risk guidelines, or interest rates. No big cost associated with that.

But to retain those customers, you have to incur some big costs to keep branches open, provide call center support, and deliver service in an ever-growing number of channels. Many of the costs you incur to keep the business running are costs that help keep your customers  satisfied – and, hence, keeping them as customers. There’s simply no way the cost of acquisition is five times greater than the cost of retention.

But, wait, that’s not right either. Because all those costs you incur to retain your customers help to make your company the great company that it is. It’s what you’ve built your reputation upon. And without that reputation you couldn’t retain OR attract customers.

Bottom line: There’s simply no way to accurately calculate the cost of acquisition or retention. It involves making too many judgments and decisions on which activities contribute to acquisition and retention. It can’t be done.

———-

These claims – that half of advertising is wasted, or that acquisition costs are five times greater than retention costs – are examples of what I call Quantipulation:

The art and act of using unverifiable math and statistics to convince people of what you believe to be true.

The examples I just gave are just two examples of this widespread practice. In fact, the incidence of quantipulation has grown by 1273% compounded annually since 2003. And I have the math to prove it:

What’s driving this growth in quantipulative activity?

The false legitimacy that quantipulation provides gives quantipulators confirmation that the things they WANT to believe are really true.

In addition, there are many people who want to lay claim to having the secret sauce for marketing success, and sadly, many people who want that special sauce. Quantipulation provides the “scientific” proof that their sauce tastes best.

There are at a lot different flavors of this special sauce that people quantipulate about, especially about customer loyalty, influence, performance metrics and ROI.

I’ll be discussing those things in more detail during the conference. Hope you’ll be there.

Oh, and in the mean time, if I catch you doing anything quantipulative, I’ll be sure to call you out on it. 

The Marketing Metrics Of High-Performing Integrated Marketers

I published a report called The Hallmarks Of High-Performing Integrated Marketers In Financial Services which analyzes the integrated marketing practices of 175 firms across a range of industries. I thought I’d share some interesting findings from the report, which looked at just the financial services respondents.

Not surprisingly, 83% of financial services firms said that they have made a strategic commitment to integrating marketing campaigns across the channels in which they market in. Interestingly, financial services firms are split — almost down the middle — between those that have achieved a 10% or greater lift in their marketing performance as a result of their integrating marketing efforts, and those firms that haven’t.

My report looks at the business practices and strategies that distinguish the high-performers from the rest of the pack. One finding that I’ll share here relates to the use of performance metrics: High-performers have a different view of what constitutes integrated marketing success than other firms.

High-performers are more likely to measure the lift in customer satisfaction and average customer spend than other financial services firms. And less likely to measure likelihood to refer the brand and brand awareness.

My take: Measurement costs money. It doesn’t matter how simple the metric is. The act of collecting data, measuring results, and making a metric actionable and usable takes time, effort, and money.

The findings from the survey shows that high-performing firms put their emphasis on measuring the metrics that matter — those that measure and drive bottom-line performance. The under-performers can continue to look at NPS and brand awareness at their own risk.

Technorati Tags: Banking, Marketing, Marketing ROI, Net Promoter Score