Quantipulation

A guy named John Wanamaker is famous for something he said 100 years ago. He said:

“Half the money I spend on advertising is wasted; the trouble is I don’t know which half.”

Unfortunately, he’s wrong. I mean, if he didn’t know which half was wasted, how did he know it was half and not three-quarters or one-quarter of it?

He’s also wrong because it’s conceivable that 100% of his advertising dollars were wasted.

A century ago there were no ad ratings or measurement services. So how he could possibly know if ANY of his advertising spend was effective? It’s quite possible that any increase he saw in sales was due to exogenous factors like the weather, the economy, the competition raising prices or going out of business, or word of mouth among customers.

Ah, but hold on here a second. I guess it’s possible that 100% of his advertising spend was effective – or at least, not wasted – depending on what measure of success you use. If you don’t believe me, ask DeBeers.

Is it likely that the advertising he did had absolutely NO effect at all? Probably not. Just because someone didn’t make a bee line for the department store after seeing an ad, doesn’t mean the ad had no effect and should be considered wasted dollars. Some might have seen the ad and learned about the store, or the ad might have left others with a positive impression of the store.

Wanamaker thought half his advertising spend was wasted because he had no way to measure its effectiveness and didn’t even know what to measure.

Today’s advertisers have some measurement tools and services available to them, but none can claim to be totally accurate. And marketers are dreaming up new metrics every day, so you can be sure that no one measure is perfect, nor can we safely assume that even a group of commonly used metrics can truly give us a reliable picture of the effectiveness of advertising.

Bottom line: Any claim on what percentage of your advertising is wasted and what isn’t is just a random guess. We simply don’t know – and can’t know.

Here’s another claim to consider: Have you heard that its costs five times more to acquire a customer than to keep or retain one? How did they figure that? You could double the number of insurance, credit card, or mortgage customers you have by simply tweaking your underwriting guidelines, risk guidelines, or interest rates. No big cost associated with that.

But to retain those customers, you have to incur some big costs to keep branches open, provide call center support, and deliver service in an ever-growing number of channels. Many of the costs you incur to keep the business running are costs that help keep your customers  satisfied – and, hence, keeping them as customers. There’s simply no way the cost of acquisition is five times greater than the cost of retention.

But, wait, that’s not right either. Because all those costs you incur to retain your customers help to make your company the great company that it is. It’s what you’ve built your reputation upon. And without that reputation you couldn’t retain OR attract customers.

Bottom line: There’s simply no way to accurately calculate the cost of acquisition or retention. It involves making too many judgments and decisions on which activities contribute to acquisition and retention. It can’t be done.

———-

These claims – that half of advertising is wasted, or that acquisition costs are five times greater than retention costs – are examples of what I call Quantipulation:

The art and act of using unverifiable math and statistics to convince people of what you believe to be true.

The examples I just gave are just two examples of this widespread practice. In fact, the incidence of quantipulation has grown by 1273% compounded annually since 2003. And I have the math to prove it:

What’s driving this growth in quantipulative activity?

The false legitimacy that quantipulation provides gives quantipulators confirmation that the things they WANT to believe are really true.

In addition, there are many people who want to lay claim to having the secret sauce for marketing success, and sadly, many people who want that special sauce. Quantipulation provides the “scientific” proof that their sauce tastes best.

There are at a lot different flavors of this special sauce that people quantipulate about, especially about customer loyalty, influence, performance metrics and ROI.

I’ll be discussing those things in more detail during the conference. Hope you’ll be there.

Oh, and in the mean time, if I catch you doing anything quantipulative, I’ll be sure to call you out on it. 

Advertisements

13 thoughts on “Quantipulation

  1. If someone’s willing to pay for an answer, there’s always going to be somebody willing to take their money and give them one, whether or not it has any real validity. Sometimes, as you say, it’s just not possible to give an accurate answer to a question, no matter how much time or money you have, but many analysts are reluctant to say so for fear of losing their aura of expertise. Lest I come off as too cynical, however, I do think there is value to even the most imprecise estimate, as long as the proper qualifiers are used. After all, you’ve got to plug something into the business case spreadsheet, and you know the answer isn’t zero. Just make sure that the number doesn’t leak into the wild without attribution; that’s when you get things like the universal 80/20 law, or the “fact” that 30% of bank revenues are attributable to payments. If you trace these claims back to their original source, you invariably find loads of qualifiers and assumptions that didn’t get passed on.

  2. Aaron: Couldn’t agree with you more. So as not to give away my whole presentation before I present it at the conference, I didn’t include a number of examples in the blog post that I’ll talk about in the presentation. What characterizes some (many?) of those examples is the un-usefulness of the estimate, metric, or formula.

    This is the acid test, as far as I’m concerned: If an estimate, metric, or formula is useful (i.e., managers can use it to make intelligent decisions about the the business) AND the limitations of the estimate/metric/formula are understood and recognized, then it probably ISN’T quantipulation.

    What I was trying to point out is that there are many instances of using math/statistics to sound legit or influence opinions. And too many people simply don’t take a critical enough look at the math behind those instances.

  3. Ron
    Found your blog through Avinash. I like it that you question not just the data but the method it was collected and the fact you are looking for lurking variables when someone attributes causation (like the sales increase from ad spend).
    None of the overreaching studies that attribute success to result of moving just one metric (say loyalty, satisfaction, etc etc) do not ask the question, what percentage of changes can be explained just by these vs. other environmental factors.

    Very close to the cost of customer acquisition claim is the 5% increase in loyalty increases profit by 85% claim (my take on this here: http://iterativepath.wordpress.com/2009/12/12/loyalty-reality/ )

    We are surrounded by such claims, there is little hope for evidence based marketing.

    -Rags

  4. Great article, I agree with you completely, but clients expect and demand some sort of metric on their advertising spend so the quantification of their spend is necessary.

  5. William: Thanks for commenting. You’ll get no argument from me on your point. In fact, don’t get me wrong: I quantipulate ALL THE TIME. It’s basically my job to quantipulate. I’.m not making a judgement statement. However, I think it’s important that if you’re going to be quantipulative, that you be transparent about the inputs and assumptions. If you can do that, you’ll be incredibly influential.

  6. Rags: Great points. And PLEASE don’t get me going about the stuff that Reichheld has published. Between the 5% increase in loyalty claim and the whole Net Promoter Score stuff, there’s no bigger snake oil salesman in the consulting business.

  7. Even when dealing with a single variable, such as “revenue,” there are all kinds of creative, nebulous and sometimes dubious factors that can skew the number. Introduce a second variable and the math becomes very slippery.

    When measuring advertising, it’s important to remember that ads take many different forms and seek to accomplish very different things. There are billboards, online ads, TV ads, print ads, etc. Some ads have objectives that are very hard to measure/quantify, such as “brand advertising.” Then there are direct response ads, like those for the Supper Shammy, where it can be much easier to calculate a direct and immediate correlation. If you aren’t asking people to do anything in your ads besides “remember us” or “consider us,” obviously you’re going to have a much harder time quantifying results than if you ran ads with an offer/call-to-action, e.g., “Hurry! Redeem this coupon for your 2-for-1 offer before December 1st!”

  8. Now that you mention it. On a scale of 0-10 (no less, and not 1-10), how likely are you to recommend my comment to your friends and colleagues.

    and what can I do to make it 9 or 10?

  9. Rags: On a scale of 0-10, I’d give a 8. However, if you RT the link to this blog post 5 times tomorrow, I’ll bump that up to 9, and if you RT 10 times, I’ll give it a 10. 🙂

  10. I’m feeling almost ashamed that I’ve never asked myself about the validity of claims re: costs of acquisition or retention. The idea that it costs a bunch more to get new business than keep current business – that’s been gospel truth… the rationale for many huge marketing and business development decisions. Thanks for making us think!

  11. Pingback: User Investment | The Nunamaker Group

  12. Ken: Nice name by the way. I thought the same thing too until I read this post.

    Dangit Ron, see what you’re doing now? You’re making us think. Gah… 🙂

  13. Ron…
    Bravo. Well stated. You and your readers may find this (Bryan Eisenberg’s summary of Avinash Kaushik’s post) to be a valuable, useful read

    Snip:
    “… 78 percent of companies are just hoping for success by guessing how well they are at providing their customers quality experiences. While we may all be suffering from data diarrhea, making decisions based on analysis of our metrics is just unclear, and they fear failure. Some call this assumption marketing. For over a decade, I’ve called this a symptom of accidental marketing.

    http://www.bryaneisenberg.com/2011/08/data-rich-optimization-poor

Comments are closed.