Digging a little Deeper into NPS metrics

Previously I discussed how the NPS metric is flawed by including satisfaction and dissatisfaction within the same metric. I pointed out that the satisfaction scale and the dissatisfaction scale do not have the same metrics/components and hence cannot be combined into a single metric such as the NPS metric. Of course, there may be other metrics that may be able to combine satisfaction and dissatisfaction based on the assumption that they will be represented accurately on the same scale. In order to understand the differences between these two scales, it is necessary to fully describe the parameters of each of the scales (satisfaction and dissatisfaction).

Although both satisfaction and dissatisfaction are often described using a Likert scale, there is no reason to believe that the units on each scale have the same value or meaning. For example, a measure on the satisfaction scale of "satisfied" (second-highest to "very satisfied") may not have the same intensity or value with a value of "dissatisfied" (second-highest to "very dissatisfied") on the dissatisfaction scale.  From another perspective, one might want to believe that the amount of customer involvement of moving from "satisfied" to "very satisfied" is equivalent to the amount of customer intention of moving from "dissatisfied" to "very dissatisfied".  Of course, this movement can be expected when we are using ordinal scales but that does not transfer to equality.  Remember that ordinal scales only indicate order but does not include a metric of distance along the scale.  Thus, the movement from one value either up or down on the scale does not require that the distance be the same.  Likewise, movement on a Likert scale for satisfaction does not require the same value for movement on the dissatisfaction scale.  Although the scales are numbered sequentially, there is no requirement for the linear relationship of the ordinal scale to accurately reflect equivalent linear values for both the satisfaction and the dissatisfaction scales. 

If in fact, decreasing satisfaction does not generally lead to dissatisfaction, then a better terminal value for least satisfied would more likely be the customer is "indifferent".  Hence a more accurate satisfaction scale would follow the sequence very satisfied, satisfied, slightly satisfied, and indifferent.  A similar scale for dissatisfaction would follow the same sequence of very dissatisfied, dissatisfied, slightly dissatisfied, and indifferent.  I am using the term "indifference" to describe in non-psychological terms that point in the customer's attitude where the customer stops caring. In this zone of "indifference".  At this point, the customer is no longer connected with the evaluation process and has become indifferent. 

As an example, I once had a consulting relationship with the company that had very high satisfaction scores and believed that their customer relationships were all strong. However, once we analyzed all the customer relationships we found that a relationship that we thought had no meaning was in fact very important. It turns out that one of the company’s largest customers had been put on credit hold because of some delay in the paperwork. Even though the customer represented the largest account, personnel in accounts receivable were trained only to collect accounts. There’s no customer training included for personnel in the Accounts Receivable department. Although the large customer did not change the supplier, it did introduce some significant stress to the company management.  The satisfiers were well taken care of and the areas of dissatisfaction that were measured were small. Hence, the NPS score was very high.  The simple answer might be that the company did not include the Accounts Receivable department as part of its metric. In that sense, a better NPS score may have been computed. However, the values of satisfaction and dissatisfaction on the metric scale would not have given had a significant cause for concern since it would not have been included in the calculated NPS metric. In this case, it almost led to the loss or at least a diminished relationship with a very large customer.

The bottom line is that using a single Likert scale for satisfaction and dissatisfaction is attempting to mix two very different parameters. Previously I have indicated that lower levels of satisfaction do not generally lead to values of dissatisfaction. Similarly, using the same logic, reducing levels of dissatisfaction does not generally lead to satisfaction. It is time to create complementary scales for satisfaction and dissatisfaction so that more accurate assessment of the satisfiers and the satisfiers of the customer relationship can be presented and understood.  It is usually reasonable to believe there is no perfect customer relationship. All customer relationships have strengths and weaknesses.   Not only is a time to create complementary scales for satisfaction and dissatisfaction, it is also time to incorporate into the metrics of the customer relationship, multiple factors that more accurately describe the company – customer relationship. Simplicity is no longer the answer.

Low-Hanging Fruit - Part 3

In the previous two blogs that discuss the elephant in the room, this blog takes the next step in describing the elephant. As we describe the elephant using such terms as customer satisfaction, customer experience, customer loyalty, net promoter score, etc. various metrics are provided with the hope of describing the elephant. Metrics used by each of these majors often contain similar measures.

Each of the metrics that researchers derived for the various characteristics of the customer have assumed that the individual metrics captured sufficient information to describe the company/customer relationship. However, as described in parts one and two of this series of blogs, are only characterizing a fraction of the elephant. What has really been captured in the various metrics noted above is the “low hanging” descriptors of the elephant. The elephant, which I have used symbolically to describe the entirety of the company/customer relationship, has only been defined using easily measured parameters.  These parameters are generally limited to the direct interaction between the customer and the company and have only short-term and limited impact on long-term memory.

First Conclusion:  current metrics represent only the low hanging measures of the company/customer relationship

The first question that comes to mind once it’s recognized that only low hanging metrics are being used to describe the company/customer relationship is what measures are missing.  Within that question are the questions relating to other customer contacts not included with the low hanging measures and what measures are missing that are not direct measures.

A comprehensive measure of the company/customer relationship requires two components; namely, direct metrics and indirect metrics.  

            The DIRECT METRICS consist of the metrics of the direct interaction between the customer and the company.

            The INDIRECT METRICS consist of the non-interactive relationship be the customer and the company continues to have an impact on both the customer and the company.

Second Conclusion: the perfect relationship between a company and a customer occurs when all metrics are maximized.

Until all the metrics, between direct and indirect, are known, measured, and maximized the likelihood of understanding the state of the customer/company relationship can not be assessed.

is there an elephant in the room? part 2

Each of the individual metrics appears to capture the chaos embedded in the relationship between the customer and the company. So when each of the metrics is carefully examined, it appears to be possible to gain insight into how well the metric works. More importantly, it becomes possible to see where the metric identifies aspects of the elephant, even though that metric is known to contain errors; either in the metric itself or the variation in the shape of the elephant.

A fact: all measurements contain errors.

Hence when all these metrics are combined, the shape of the elephant becomes more difficult to visualize and understand.  One of the reasons for this lack of visibility describing the elephant is that each of the metrics may contain different forms of measurement error which may pose a challenge to understanding the impact of each error. While it would be convenient and yield better understanding to know all aspects of the elephant, the real objective for these measurements previously described, is to increase the understanding between the customer and the company.

A fact: it is not necessary to have a perfect vision of the elephant

The real value is knowing how well the processes that are being used by the company to provide its products and services to the customer are performing. If the processes are perfect and there are no errors in the implementation of the processes, the company has delivered what it believes to be the best way of improving relationships with the customer.  There are two ways in which the company can fail to maximize its relationship with the customer.  The two ways it can fail are:

1.    1. Unable to deliver the products and services perfectly with no error

2.   2. Unable to provide the best products and services.

This information noted above provides some of the key objectives for managers and executives; namely, managers must constantly assure the processes continue to be provided errorless, and executives must constantly review the products and processes to ensure that each one meets and hopefully exceeds the expectations of the customer.

However, there one aspect that has not been considered; namely, the customer.  Only those aspects of the customer that has an active connection with the company have been considered.  BUT, is there more?   

The picture is not complete.  The metric needs further definition.

Part 3  What is the metric?

Is there an elephant in the room? Part 1

There are numerous metrics that measure the different relationships between a customer and the company.  Customer satisfaction,  customer experience, customer effort,  customer rage, customer loyalty, there may be others.

I am reminded of an elephant in the room described by several people each with a limited perspective. Each is asked to describe the animal in the middle of the room. One person grabs the tail and gives a comprehensive description. Another touches the leg and provides a comprehensive description of the leg. Another touches the trunk and like the others gives a comprehensive description of the elephant’s trunk. I could go on with more descriptions of the elephant by others with similar limited vision, but I think you get the idea, 

While it is true that each description of one aspect of the elephant is accurate there is no consistency between the metrics to define the entire elephant in the room. This analogy has some relevance to the metrics that are being used today to understand the relationship between the customer and the company. There are many people who have carefully constructed metrics that are statistically accurate and allow hypotheses to be extracted and yield valid data to describe an aspect of a customer/company relationship.

There appears to be a lot of dissent between those who have rigorously developed a specific metric to describe the elephant with others who also developed a metric different from all the others. In fact, virtually all of the metrics have been rigorously developed to describe the elephant. However, the sum of all the metrics don’t seem to accurately describe the elephant.  What is missing is a comprehensive metric that would support the many individual metrics from the tail, leg, trunk, and other specific measures of the elephant.  However, at this time there appears to be no single metric that integrates the various specific metrics currently used.

Part 2 will take the next step.

Everything is broken

  Another way of saying that everything is broken is by saying that nothing is perfect. The next several blogs are going to describe how entropy can be used as one of the most efficient ways of increasing performance in the service organization. The following list of quotes has been extracted from several articles that support this focus on entropy.

1  1. Entropy is a measure of disorder. It’s a physical law of nature that left untouched, everything will steadily deteriorate.  Taken from head scratchers.com/January 2013 post

    2.  Managing entropy in business: a demon that slowly erodes, disrupts, and destroys organizations. Taken from bizshifts.com/July 2016

    3. Entropy can show you where additional resources would make your business more efficient.  Taken from yourbusiness.azcentral.com.  No date identified.

    4. Entropy occurs in every aspect of the business. Taken from FS.blog/2018

    5. Understanding the law of nature about entropy can help prevent business deterioration. Entropy is the potential energy that is not available for work. Taken from slideshare.net/June 2013

    6. Entropy is a quantifiable value analogous to the amount of randomness in the system. Taken from fastCompany.com.  no date identified

    7. Entropy is the tendency of ordered systems to move towards disorder if left unattended. All organized systems are like this. You have to continually work to maintain them or they’ll fall into disarray. Taken from nfx.com/post/entropy. No date identified

    8. Entropy is the tendency of ordered systems to move towards disorder if left unattended.  Taken from nfx.com/entropy

    9. We find that the tendency of operational routines to decay is widespread. Taken from pubsonline.informs.org. No date identified.

As you can see in these comments taken from various articles entropy is a measure of disorder or reduction in productivity.  The purpose of this blog and following blogs is to identify the various components of entropy that can be found within the service organization. As we identify each component of entropy, the metrics needed for that component of entropy will likewise be identified.

Finally, we will create a model to combine the various components of entropy to provide management with sufficient information to reduce the disorder in the appropriate areas of the business.

Some of the components of entropy that have been identified are:

     1. Cultural entropy – measure dysfunction     

     2. Economic entropy – measure waste of capital, human resources, and materials

     3. Process entropy – measure scale entropy, capability entropy, and speed entropy.

The bottom line is that while many authors have addressed the various aspects of entropy in service, there is no integrated perspective that I have found in the literature.  Until we identify the components of entropy, we cannot develop processes that will minimize possible degradation and increase entropy.  

A closer look at apples and oranges

I recently wrote a blog about apples and oranges as a comparison with satisfaction and dissatisfaction. In that blog, I particularly related how the NPS metric was making a large assumption about satisfaction and dissatisfaction that doesn’t appear to be appropriate. I am going to take a deeper look at these two parameters. I think when we examine them we can see what makes them different.

It all starts with the fact that metrics such as satisfaction and dissatisfaction do not naturally fall onto a numeric scale. Most often these parameters are measured on what is called an ordinal scale. An ordinal scale is known for its ability to provide order to a series of values such as flavor (a little sweet, sweet, very sweet), color (dull, flat, bright) or shape (straight, slight curves, very curvy). The parameters such as satisfaction (slightly, somewhat, very satisfied) and dissatisfaction (slightly, somewhat, very dissatisfied) are most often also characterized on ordinal scales. Most customer satisfaction surveys use some form of an ordinal scale with both satisfaction and dissatisfaction identified on the scale. A simple five-point scale usually consists of the ranking from dissatisfied, slightly dissatisfied, neutral, slightly satisfied, and satisfied.

This is where the trouble begins. The first assumption is that the customer attitude change between dissatisfied and slightly dissatisfied is approximately the same as the attitude change between slightly dissatisfied and neutral. This assumption follows between two consecutive majors on this ordinal scale. Of course, is not possible to use arithmetic operations on these numbers. One of the strongest violations of this first assumption is the assumption that the attitude change between slightly satisfied and satisfied is equivalent (and this is the real problem) between slightly dissatisfied and dissatisfied.

The fallacy of this first assumption is that customer attitudes of satisfaction have the same impact of dissatisfaction. What is happening is that when the scale is designed we are saying that satisfaction and dissatisfaction on the same ordinal scale implicitly assumes is that reduction in dissatisfaction leads to satisfaction and similarly, reduction in satisfaction leads to dissatisfaction.

Tom Peters and other authors have noted that dissatisfied customers appear to have stronger feelings about dissatisfaction than satisfaction. They measured this by the number of people they communicate with. Those who are dissatisfied will generally tell more people about their dissatisfaction than those who are satisfied. The ratio has most often been noted as three or more times more people are told of a dissatisfaction event to those that are told of a satisfaction event. The old wives' tale would say “bad news travels fast.”

The bottom line is that satisfaction scales and dissatisfaction scales are more than numbers in a sequence. Satisfaction causes significantly different responses in customers than does dissatisfaction. It is time that we start to pay attention to the reality that measuring satisfaction and dissatisfaction on the same scale is not just misleading but wrong.

The Tyranny of the Urgent

I have a somewhat unusual combination of education and job experience.  I think I am one of the few college professors who have worked in industry as a service manager in a line management position.  From that background, I have had the opportunity to consult with a number of service managers in the computer industry.  In all of my consulting perhaps the most common problem I see with service managers is what I call the “tyranny of the urgent.”  I am sure that the phrase "tyranny of the urgent" has been used many times I am not the creator of this phrase but a firm believer in it. I believe it is a phrase that accurately captures the business of service today

Before I became a service manager with line management responsibility I would drink one or two cups of coffee a day.  Within 6 months of becoming a line manager, I was drinking six to eight cups a day.  I always had a cup of coffee in my hand.  There was always something going on that demanded my attention.  I am not sure if I got more of a rush from the adrenaline of urgent activities or caffeine.  A customer would be down and there would be no one available in the field or the call might come from the President of the company that he just received a call from a customer and thought I should look into the matter.

Urgency is a way of life in the service business.  We never seem to get involved until things aren’t working right or the customer is in trouble or the new product that was shipped has a design error that has to be fixed at the customer site.  With all of these activities crying for our attention, we become experts at handling the urgent.

It is ironic that this urgency can become a personal dictator if we let it.  This is the “tyranny of the urgent.”  The sooner you recognize it, the sooner you can stop it.  In a way, urgent activities are good because we can get an immediate “attaboy’ when we finish them.  Most of us get great satisfaction from “attaboys” and thus the cycle begins.  Perhaps that is what led us into the service profession.  I have seen service managers thrive on constant crisis.  They are not happy unless they have all of their employees working as hard as possible, a stack of calls to return (ASAP) and negotiations going on for credit on out-of-warranty returned goods.

On the other hand, there are activities that can best be described as important rather than urgent. Spending time strictly on urgent matters will not improve the performance of the service organization. It is only when time is spent on important activities (such as service planning, training, and teambuilding) that improvements can be designed and implemented which will lead to a better service organization.

The bottom line is that those urgent activities come and go, but never diminish in number or intensity.  This is the nature of the business of service.  Unfortunately, that is not the only nature of our business.  There is another type of activity in our business – IMPORTANT activities.  All of our business activities can generally be classified into one of these two groups, but usually not into both.  Success comes to service when both urgent and important activities are well-managed.