I got an email from Tim Keiningham, an SVP and head of consulting with Ipsos Loyalty. Tim let me know that a paper he co-authored, called A Longitudinal Examination of Net Promoter and Firm Revenue Growth (which I wrote about here) won the 2007 Marketing Science Institute/H. Paul Root Award given to the Journal of Marketing paper that had “the most significant contribution to the advancement of the practice of marketing.”
To quote from the paper:
We find no support for the claim that Net Promoter is the single most reliable indicator of a company’s ability to grow. We found that when making ‘apples-to-apples’ comparisons, Net Promoter does not perform better than the ACSI (American Customer Satisfaction Index). Managers have adopted the Net Promoter metric on the basis that is superior to other metrics. Our research suggests that such presumptions are erroneous.”
My take: If the Marketing Science Institute says that the “most significant contribution to the advancement of the practice of marketing” is a study that refutes one of today’s most popular marketing fads, namely the Net Promoter Score, what does this say for the credibility and long-term success of NPS?
It says that the beginning of the end has begun.
Management fads go through a predictable cycle of four stages. They:
- Begin to show up in the management press with articles (or books) from early proponents;
- Gain acceptance when credible case studies highlight the [allegedly] superior performance firms realize from deploying the concept;
- Come under more rigorous scrutiny which produces studies and examples that refute and challenge the claims of success; and then
- Begin a decline in popularity (but can become part of the fabric of management if — and only if — they’re truly a worthwhile concept).
In addition to the Keiningham study, comes stories like the one from Paul Schwartz who wrote:
I’ve been using NPS with clients for a couple of years, and I’m not convinced yet that it is the best indicator of a company’s ability to grow. The real work is figuring out what drives the likelihood to recommend for each business, and then measuring the actual recommendation and purchase behavior of customers.”
Not convinced yet, Paul? You may never be. More studies like Keiningham’s will be done, and more stories like yours will be told in 2008. 2008 will be the year that NPS moves into stage 3.
Dr. Seuss once wrote:
The storm starts
When the drops start dropping
When the drops stop dropping
Then the storm starts stopping.”
The NPS storm is starting to stop.
Congrats to Tim and his colleagues on their award, and Tim — if you need help carrying your bags when you go to Austin, TX to pick up the award, just let me know.
Technorati Tags: Marketing, Net Promoter Score, Marketing Science Institute, Tim Keiningham
Very ‘prognisticatory’ of you Ron. It sure appears that it is very hard, and maybe not even appropriate, to pick a single measure for predicting growth or assessing the overall health of a company’s customers. Just as in other areas of assessing a business, a single measure (e.g. revenues) won’t give you the whole picture. When it comes to customers, we are talking about human beings, and there is nothing simple or singular in being able to predict human/customer behavior. Accepting this premise means it’s not so easy to develop a single business measure to predict a company’s growth.
So, what will be the next management fad???
I read the research Ron and I read your post regarding it. I am so happy they won. Research and testing won too.
Ron,
You keep missing the point of NPS. It is NOT a survey, it is a discipline. Like accounting. The score is math – like ROI. It’s not a strategy – it is what it is. BUT, the discipline that Satmetrix has built around the survey results is what makes it so important in predicting future profits of a company.
NPS will be a fad for these Fortune 1000 companies that have adopted it – just like accounting.
Denise —
My co-authors and I are only concerned with the research presented in our scientific journals, and the Harvard Business Review (HBR) is one most important in terms of its impact. The research presented in HBR (and in the book The Ultimate Question) in support of Net Promoter pertained to the metric’s superiority in predicting growth. It is important to note that the reason for the metric’s existence and adoption was the research presented in HBR, and it is sold as the “single most reliable indicator of a company’s ability to grow.” Our research presented in the Journal of Marketing clearly challenges these claims, and does so using a rigorous scientific approach. [The Journal of Marketing is the top scientific journal in all of management and economics, as measured by the citation index.]
While it is fine to support the discipline you speak of in your response, we must also never be apologists or revisionists when it comes to the quality of the research presented in our scientific journals.
Sincerely,
Tim Keiningham
[...] JL: I’ve heard you talk about Net Promoter Scores (NPS)* as a metric. Do you encourage this type of evaluation, or do you agree with those who think it’s too simplistic? [...]
As I filled out a 1-10 rating of a user-created quiz on Flixster (I gave the quiz an 8, and then the site told me that the average review was a 5.5), I realized a huge problem in NPS. And that is, that different people have very different views about judging that are entirely personal and subjective. In my Flixster case, I realized that while I have done a few of these quizzes before, I have not done many of them, and it had been a while since I had done one. So my evaluation was not an informed one.
Turning to NPS as it applys to business, there are optimistic people who tend to give out high scores on all of these types of evaluations, and there is another camp of pessimistic people who tend to always give very low scores (like Olympic judges who must score low in case a superior performance occurs later), and everything in between. So the exact same transaction experience could be judged an 8 by the first category of people, and a 5 by the second category. NPS has no way of differentiating or calibrating for this innate characteristic of scorers.
This is akin to the study of people’s habits on tipping waitstaff at restaurants. The study discovered that the amount of tip had much more to do with the tipper than with the service received, i.e. there are people who generally tip generously and others who tip conservatively, and the service experience itself could do little to impact a shift in a person’s tipping habits.
While I understand the discussion about NPS and its flaws I guess time will tell. As for the comments I’ve seen that some have a NPS of 75 and while ours is only 31 it is a bother, to say the least.
Yet, I would have to disagree with Morriss’ analogy with tipping, since my wife was a waitress for 20 years and always averaged a much higher tip ratio than the other waitresses due to great service and personality and somewhat with upselling them for a larger average meal tab. She had just as many good tippers and bad tippers as everyone else.
The Net Promoter score does identify the level of Loyalty that might exist in your customer base. But it does not give you the tools to improve your Customer Loyalty over the long run. Brookeside can do that