The Satisfaction Trap

I recently ran across an old Harvard Business School article by Frederick Reichheld ("Learning from Customer Defections", March-April 1996) that has some interesting thoughts that I often forget. He notes that satisfaction surveys have two principle problems. Some of the comments below are direct quotes from the article. I have tried to maintain the context of the quotes.

The first problem is that satisfaction scores become an end in themselves. I have seen this in many of my consulting assignments where the satisfaction score is one, if not the only, factor in computing salary increases and bonuses. This can also be seen by noting the emphasis on the J.D.Power survey results for just about everything - especially cars. The score is all that matters! This is a problem because in operating a business we can't manage scores, we can only manage people and processes. Another reason why this is a problem is that most surveys represent only a sample of customers and each sample has a different set of outcomes. In fact, sample statistics will absolutely change every time a sample is taken. When companies do not understand the notion of sampling error, chaos can occur within the management ranks when the sample scores fall (within the range of the sampling error). Management believes that customers are not as satisfied as they were in the previous period and hence it is time to agressively attack the problem. In fact, there is no problem, just sampling error. This can best be described as the "keystone cop approach to management" and is one management philosophy I do not support.

The second problem is that surveys of measure the wrong activity or the wrong customers. The fact is that designing a survey requires some significant training. Most companies believe that they can assign the survey to an employee with little or no training in survey design because "all you have to do is write down a couple of questions." When I am asked to review surveys and survey results from these companies, most of the time I find the survey flawed in one or more ways. The worst case is when a group designs a survey with a specific outcome in mind and constructs the survey to verify the outcome. No need to discuss this situation the any further. A point I almost always make to these companies is that even a poorly designed survey will provide a measurement and if that measurement is incorrect, it may lead to wrong conclusions and decisions.

Here are some of the excellent points that are made in the article:
1. Whenever rewards are based on satisfaction scores decoupled from repurchase loyalty and profits, the result is unproductive behavior.
2. Surveys almost never provide the information that managers need to pick the investments that will maximize customer value and, in turn, cash flow.
3. Surveys ignore critical distinctions among customer segments.
4. In business after business, 60% to 80% of lost customers reported on a survey just prior to defecting that they were satisfied or very satisfied.

This article was written about the same time Mr. Reichheld's book "The Loyalty Effect" was published and prior to his later work on NPS. I am not a fan of NPS but that is left for a future blog.