Thursday, December 30, 2010

The Halo Effect on Satisfaction Metrics

There has been some very interesting research on the Halo effect on satisfaction metrics. The research was performed by Joachim Buschken, Thomas Otter and Greg M. Allenby. Their work is entitled "Do We Halo or Form? A Bayesian Mixture Model for Customer Satisfaction Data".

When we look at customer satisfaction data we often look at what are the drivers of overall satisfaction. We naively assume that each question is answered independently so that we can use techniques such as regression analysis to identify those factors that most strongly relate to overall satisfaction. When this assumption is satisfied the "drivers of satisfaction" are valid and management actions taken as a result should have a positive impact on the organization.

When the assumption of independence is not met, the identification of drivers loses its validity. One of the most serious deviations comes from "Halos". Halos occur when a customer responds to a survey such that all the scores are identical and usually all top box (or bottom box). The customer is saying that the company scores top box with respect to every question. In other words, the company is perfect and all the areas requested for evaluation have outstanding performance. When all the scores are identical the term often used to describe this response is Halo. There can be a negative Halo as well as a positive Halo. The same logic works for the negative Halo, the customer is saying that all areas of the company being evaluated are equally dramatically under performing.

The Halo effect occurs most often when the respondents recall their overall satisfaction and then assign scores for each question consistent with their assessment of the overall score. When this occurs the components are uninformative about the drivers of overall satisfaction. In this case regression of overall satisfaction or dissatisfaction is not possible because the components are co-linear.

When conducting statistical analysis to determine the influence of drivers on overall satisfaction the halo responses must be removed from the response data set.
While many survey analysts try to find ways to remove outliers, the Halos are responses that should be eliminated from any statistical analysis.

The researchers point out that respondents who are familiar with the product or service are more capable of providing insights are found to be more likely to Halo their responses. On the other hand those respondents who are less aware of the products or services and are less capable of providing informed are more likely to make their responses more independent and less likely to Halo their responses. The problem is that neither of these scenarios are what would be appropriate for driver analysis.

The bottom line is that Halos are bad news for satisfaction and loyalty surveys. It is clear that analysts who are doing driver analysis need to be aware of the impact of Halos. This is one exception to the rule that outliers must be included. As the researchers note "the analysis of customer satisfaction data for the purpose of understanding drivers of overall satisfaction requires removal of the Haloed responses. From a managerial standpoint, Haloed responses are uninformative about the drivers of overall satisfaction because the component-specific information is suppressed."

4 comments:

Anonymous said...

Thanks fοr your helpful post. In гecent tіmes,
I have comе tо understаnd
that thе аctual sуmptoms of mesothelioma cancer аre caused by the actual build up
of fluid regarding the lining on the lung and the chest cavіty.

The аilment may ѕtart fгom the
cheѕt area and pass on to оtheг limbs.
Othег symptomѕ of plеural mesothelioma cancer
include weight-loss, severe іnhaling and exhaling trouble,
throwing uρ, diffiсultу eаtіng, and bloating οf the
neck and faсe areas. It ought to be noted some peoplе with thе disеase tеnd
not to еxperiencе anу serious ѕigns and symptοms at all.


Look аt my site ... quibids review

Anonymous said...

Hi there, You have done an excellent job. I'll certainly digg it and personally suggest to my friends. I'm sure they will be benefited from this web site.



Feel free to surf to my homepage survey, ,

Anonymous said...

What a data of un-ambiguity and preserveness of valuable familiarity about unexpected emotions.


Check out my webpage - Bob Simpson

Anonymous said...

Pretty section of content. I just stumbled upon your web site and in accession capital to assert
that I get in fact enjoyed account your blog posts. Anyway I'll be subscribing to your augment and even I achievement you access consistently rapidly.

Here is my page survey questions

 

web visitor stats
OptiPlex 755 Desktops