Similarities and Differences in Satisfaction within Companies

In my last blog I began a discussion on multi-dimensional customer satisfaction. The reason for offering up this topic now is my on-going concern that measurement of customer satisfaction today has changed so very little from the measurement ten to fifteen years ago even though there have been some great improvements in measurement. Some companies still rely on a single questionnaire mailed out at regular intervals to a specific individual in each of a selected number of customer organizations. While this process in and of itself is not incorrect, the conclusions and directions that may result as an outcome of the statistical results may be very misleading. In this case of a single questionnaire, it would be because the person completing the survey questionnaire may not be sufficiently familiar with the product and/or service to express the views of the company. Equally as important as the single view is the notion that one view may not adequately express the overall perception of the product and/or service. The strengths and weaknesses of the company performing the survey may vary from department to department within the company. The likelihood of capturing all of this information within a single survey is highly unlikely.

This blog will discuss some simple techniques for examining information from multiple sources within a customer company. In particular, the following paragraphs will describe the requirements for relating results from multiple departments of the customer, what statistical procedures can be used to detect real differences and how those differences might look on a graph or chart.

Requirements for Combining Data

Since satisfaction measurements are taken on an ordinal scale (which means only the order has value - the interval between intervals on the scale is undefined), and the responses are perceptual, absolute values are not available for comparison. It is relatively easy to compare weights of two or more food items since absolute values for 1 pound, 1 ounce, etc. are well defined. However, when it becomes necessary to compare the measure of customer satisfaction by two or more different individuals or groups, there is no absolute measure upon which to base the comparison. Thus, any measure must be on a relative scale and the scale must be the same for every individual or group. The scale used should be completely anchored in order to reduce variation.

Whenever possible the best way to compare results from different groups is to have each group answer the same question with the same wording and the same scale. Although there may be influence from the questionnaire (where the question was located), the same question with the same scale eliminates most of the difficulty. The other factors that may cause different responses such as knowledge of the product and/or service may not be known. When this situation occurs, the low cost solution is to ignore all these other factors, the high cost solution is to take the time and expense to measure them.

If the survey instrument has been properly developed, there will be common questions on each of the surveys so that comparison of results in areas of concern can be made.
The obvious response to the plan of having several survey instruments is to just send the same questionnaire to multiple persons within the customer company. This would be the best solution if the survey was sufficiently general so that each respondent could adequately respond to each question. However, the idea of asking customers questions is to evaluate the strategy and tactics of the company from the customer perspective. When the questions are reduced to a minimum set of general questions, the results have little or no information content with which to evaluate the strategy and tactics of the company performing the survey. Thus, surveys to different individuals within the customer organization should have unique questions specific to that organization along with some common questions from which to draw comparisons and a combined perspective of satisfaction.

Multiple Surveys - A Simple Example

In order to compare satisfaction levels within a company, the obvious step, as noted above, is to create surveys that can be sent to different organizations within the company. Since each organization will have different roles to play with regard to the product and/or service, questions appropriate for one organization may not be appropriate for another organization. However, in order to compare the perceptions of the different organizations there must be sufficient commonality of questions in each survey to allow for comparison of key factors within the survey.
The easiest common question to include on all surveys is the overall satisfaction question. Since this question is usually designed to perform multiple functions within the survey, one more use makes it that much more valuable. Consider for a moment the impact of having a measure of overall satisfaction from each organization for each of the customers you have surveyed. The results might appear as follows if a 10 point scale is used:
Purchasing – average satisfaction is 9.0
Operations – average satisfaction is 8.2
Management – average satisfaction is 6.5

While I don’t subscribe to the use of average satisfaction as a valid measure, I have used it in this case to demonstrate that differences may occur within an organization.

Although there may be many reasons for the differences between the three measures of overall satisfaction, first assume that the differences are statistically significant. That means that the differences are not the result of sampling error. (Many companies see every measured satisfaction difference as significant and immediately jump to conclusions that the difference is the result of something going on when often the reason for the difference is simply sampling error. From statistics sampling error arises because a sample result hardly ever exactly measures the population characteristic. Each time a sample is drawn from the population, a different sample result occurs. The nice thing about sampling is that the results from samples are generally normally distributed about the mean of the measure being taken so that as more samples are drawn, the population mean becomes bounded by the samples.

Data for the chart above was created by taking all the responses from the Purchasing Department and computing an average satisfaction. Similarly, average satisfaction responses were computed for Operations and Management. Thus, the chart indicates the average differences for different groups of customers for the sample. (A second chart, to be discussed later, will examine the differences between each measure within a specific customer and accumulate those differences on a chart.)
The results from the chart above indicate that the differences noted between Purchasing, Operations and Management are not the result of sampling error.

Therefore, there must be some other factors influencing the differences. One simple possibility is that the sales organization has an excellent relationship with the purchasing departments and has developed very positive relationships over time and hence the high average satisfaction level from purchasing personnel within the customer companies sampled. The Management personnel seem to rate their overall satisfaction lower than the other two groups. This might be interpreted to mean that management personnel might mean that the contacts with Management personnel have not developed strong relationships. The statistics indicate that Operations personnel are, on average, scoring satisfaction lower than purchasing personnel. Since this is a pattern for the entire sample response to the survey, it may indicate a need to train sales and service personnel to spend more time with customer personnel in Operations and Management in order to discover possible reasons for the difference in overall satisfaction scores.

The difference between the average satisfaction for Operations and the other departments should also be evaluated. Survey results from the previous chart indicate that satisfaction levels are generally lower for Operations and Management than for Purchasing.

So What

Now that there is statistical evidence that Purchasing, Operations and Management have different perceptions of overall satisfaction what is the next step. The first point is that this is news. Whereas a single measure would have never uncovered this difference, there is now a measure which allows a further step into understanding what is going on within your customer organizations.
The second point is that each of these measures can prioritize where to invest additional resources to provide the greatest improvement in customer perceptions of your company. From the simple example, Management appears to be the best candidate for application of resources. While there may be extenuating circumstances that may direct the efforts into other areas, the statistical results provide signposts of this possible opportunity.

In my next blog I will analyze the measure of differences within individual customers and discuss how to interpret the results. The key will be on how to combine individual differences to get macro measures of the entire customer base along with the micro measures of individual customers.