It's The Research That's Irrelevant

In an article titled “U.S. Consumers Feel Loyalty Reward Programs Not Relevant“, MediaPost reported that:

… far too many mailings, emails and Facebook messages that companies send to their loyalty reward program members miss the mark. Only 32% of U.S. consumers rated reward program communications 8 or higher on a 1 to 10 scale for measuring relevancy to their personal needs, with 68% giving a score of 7 or below.”

I guess loyalty programs must really be failing, eh? Unfortunately, this conclusion is hard to support with the data cited. Here’s why:

  • There’s no context. The question to address is not whether or not loyalty program communications are relevant — but whether or not those communications are more or less relevant than other marketing communications. If 32% of U.S. consumers rated all marketing communications 8 or higher on a 10-point scale, then yes, I would agree that loyalty programs were falling short. But what if it’s only 10%? Then three times as many consumers would have found loyalty program communications relevant as those that found other marketing communications relevant. And if that were the case, then we might conclude that loyalty programs were actually doing a pretty good job.
  • The scale is open to interpretation. The article puts the “relevance” cutoff at 8 on the 10-point scale. Who told them to do that? What if a high percentage of respondents rated loyalty program communications at a 7 on that scale? How do we know that consumers who gave a score of 7 were lodging a negative opinion?

This study is a good example of what’s wrong with the slew of market research-related press releases that go out everyday: A few data points are taken out of context to support a conclusion which is sure to generate some press interest, but rests on some flimsy assumptions and analysis.

That’s OK, though. I’ve got no beef with the firms putting out these press releases.

What isn’t OK, however, is that few marketers scrutinize the findings. Instead, they blindly accept the “findings” as fact. That needs to change.

5 thoughts on “It's The Research That's Irrelevant

  1. Ron,

    Welcome back. I wish you had this blog during the last several months. We could have had our twitter exchanges about bad research here and more than 140 characters.

    @dmgerbino

  2. The internet is much more valuable starting today. I would rate my excitement level about your new blog an 8 or higher on a scale between 1 and 10.

Comments are closed.