The Measurement Trap

Taking measurements are important. There is also a degree of risk in taking measurements. I refer to this risk as “the measurement trap. The objective of this blog is to identify and discuss a management behavior that appears valid but which can lead to “paralysis-by-analysis.” We normally assume that a measurement is an accurate assessment of some aspect of our business. But the measurement trap derives from a false belief that we can fully understand all aspects of our business strictly through measurements. In my view,

The measurement trap occurs when an executive believes that he can understand the business and customers in his market by increasing the number of measures and level of sophistication within his measurement system.

The four measurement traps

First trap – all aspects of the business can be measured.
To understand the subtlety of this trap consider the most obvious implication – that everything of importance in your business can be measured. This suggests that there is no aspect of the business that cannot be measured. While this may be a dream of people like me who deal the quantitative aspects of business, it is nevertheless naïve. Is it possible to capture the essence of the customer interaction or the level of trust that builds between people with quantitative measurement? Some would offer the examples of employee and customer satisfaction surveys as evidence that the essence of relationships can be measured. To rebut this idea, consider that satisfaction surveys of new car buyers indicate very high levels of satisfaction and yet the loyalty of car buyers, (indicated by repurchase of the same make again) is very low (the highest is less than 50%).

Another example of not being able to capture all aspects of employees is to consider the measures of employee satisfaction as they relate to customers.
Consider measures of courtesy and professionalism. Each measure is incomplete and only an indication at best and misleading at worst. For example, courtesy measures almost always show high levels of satisfaction (with rare exception). It is generally believed that they include a bias from the customer that may be based on the fear of retribution from a low rating or, on the other hand, the employee may have a good working relationship with the customer and the customer is biased to helping a “friend.”

Second trap – you don’t need to work with people.
Perhaps one of the most negative aspects of the measurement trap is that it relieves the executive from the responsibility/necessity of employee interaction with the service employee. If, in fact, all aspects of the business can be measured, there is then no longer a need to interact with employees reporting directly to the executive. The reason most executives interact with employees is to get additional information and details about the operation. They probe to discover the information not available through the measurement system. But, if all the information is available through measurement, he only needs to review the computer printouts. For this executive, the only role of the employee is to create the measurement systems and to assure that the measurements are accurate and timely.

At one time I worked for an executive who was an advocate of measurements. He was also one of the major influences that caused me to understand the concept of the measurement trap. He spent his day pouring over computer printouts and was always thinking of new measurements to take. Actually I have him to thank for getting me into the measurement of customer satisfaction so I guess his efforts were not entirely off based. (One great result of his obsession with measuring was the that the branch service managers were not spending enough time with customers. One of the measures that he took was of the time that branch service managers actually spent with customers. When I was asked to look into this situation and present solutions to get the branch service managers more involved with customers, we initiated an analysis of the activities of branch service managers. We discerned some rather dramatic workload situations:
1. The average service manager was signing about 1000 documents per month.
2. The average service manager was receiving about 20,000 pages of computer output per month.
3. Each service manager had to inspect the lease cars of every technician twice each year and report such items as worn out wiper blades, burned out light bulbs and general cleanliness.

Our findings were that (1) the average service manager needed to work overtime just to maintain his paperwork and (2) there was no time available during a standard workweek to meet with customers. With this information that we uncovered from digging into the operation, we did reduce the level of paperwork at the branch level. (Our digging did not require a measurement system). There is a positive from his perspective, he did discover through the measurement of time with the customer, that there was a problem with the branch service managers. The measurement did not disclose the nature of the problem. It took some detailed investigation to uncover the situations noted above.

One very important aspect of working with this executive was that he did not need to meet with me to discuss the business – all he needed to do was spend more time looking over the measurements. In fact, he rarely met with me or the rest of his staff (no regular staff meetings). My role changed from a planning and analysis role to one of designing more information systems. The executive did not need to work with me, only to pass on his requirements for information and to receive current information. Needless to say, it didn’t take long to figure out that this was not the place for me (and he agreed with me).

Third trap - company culture is not important
A variation on the second trap is that if you don’t need to work with the people around you, you don’t have to worry about what kind of people they are. By this I mean you can ignore the character and personalities of the employees. If the business is completely measurable, then there is no need to deal with employees and who they are. If the executive is really caught up in the measurement trap, he needs only have the measurements delivered to his office. The people on his staff can be the best in the world or a bunch of “goofballs.” The key is that there is no need for employee interaction when you are in the measurement trap. This leads to the obvious conclusion that the kind of people you have working for you is not important as long as you have the right measurements in place.

The factors which this trap overlooks are employee development and employee morale. If the quality of the employee is incidental and not an important aspect of the operation, what value will the employee perceive of himself? If the executive is busy spending his time behind a closed door analyzing measurements, he is not spending time to develop his replacement and is not giving those who report directly to him, the opportunity to work with him and learn more about the business to become more valuable employees.

Fourth trap - given enough measurements all problems can be understood and solved with quantitative information.
When an executive enters the measurement trap one of the largest impacts on the organization is the implementation of the measurement trap. When the executive believes that the business can be defined by measurements, then when a problem occurs, the executive concludes that more measurements are needed to analyze and solve the problem. The first thing the executive does is to implement additional measurements regarding the problem area. A spiral continues when the additional measurements do not provide the solution to the problem so the executive begins to add further measurements. For example, if the executive was measuring time of the branch service manager on the customer site as noted above, he might add an additional measurement of what other ways the branch service manager spends his time. If that did not provide sufficient information, he might add another level of detail to identify the specific activities associated with each of the ways the branch service manager is spending his other time. As a result, the executive gets more and more information and the branch service manager spends more and more of his time supplying the information.

In the extreme case, this continued search for the measurements necessary to solve a problem can create a spiral of ever-increasing efforts to measure aspects of the business to the point where the cost of measurement exceeds the cost of the problem. This may be the process that led to the popular phrase “paralysis by analysis.” The flaw in this aspect of the measurement trap is that all business problems can be understood by having more and more quantitative information and that because the information is not available there is a need to develop further information gathering systems.


Avoiding and Escaping the Measurement Trap

Now that you know what the measurement trap is and some of its obvious consequences, namely; the four traps noted in the preceding paragraphs, the question is how can you protect yourself from getting in it and what do you do when you in it to get out of it.

Staying out of the “measurement trap”

To keep from getting into the measurement trap, the first step is to be aware of the measurements you are currently making and examine each measurement to verify that it is necessary. The fact is that many companies take unnecessary measurements that are costly and have little or no impact of the successful operation of the business. (Consider the auto inspection noted above – was an examination of the wiper blades and associated documentation on a specific form a good use of the time of the service manager?) One of the outcomes from my consulting has been to note how much data companies have and how little information is actually useful. The point is that most companies take measurements without thinking about the cost, value or usefulness.

The second step to staying out of the measurement trap is to be judicious about measurements; select them carefully and constantly audit them for accuracy and relevance. While many measurements can be expensive, time consuming and superfluous, there are measurements necessary to successfully run a business. Remember, measurements are like looking out the rear view mirror, they only tell you what has happened in the past, they do not necessarily predict what is going to happen in the future.

The third step to staying out of the measurement trap is to look at the cost of your measurement systems. If the cost of measurement is disproportionate to your operation, you are probably making too many measurements and probably heading into the trap. When looking at the cost of measurement, don’t forget to include the cost of data collection.

Getting out of the “measurement trap”

The first step to getting out of the measurement trap is to see that you are in it. If you have noticed after reading the four traps noted above that you are in the measurement trap, you have already taken the first step.

The second step to getting out is to change your perspective from an internal perspective based merely on quantitative measurements to an external perspective based on qualitative considerations of the interaction with employees and customers as well as quantitative measurements.

The final step is to always be aware that measurements can be valuable and should be used where necessary but that they will not provide the total solution to any problem.

Conclusion

The measurement trap is easy to fall into because it generally is the easiest way to approach a problem. It doesn’t require the time and energy to work with people and besides who can argue with numbers. One of the corporate games I saw during my tenure in the corporate world was “he who has the numbers usually wins.” In fact, that is why I started to measure customer satisfaction. It was the easiest way to win the argument of how we were doing when the sales organization would introduce anecdotal stories of unhappy customers. In the business world the football adage of “winning is the only thing” doesn’t work because in the corporate world when one wins someone else loses. And that too often is the customer.