There has been some very interesting research on the Halo effect on satisfaction metrics. The research was performed by Joachim Buschken, Thomas Otter and Greg M. Allenby. Their work is entitled "Do We Halo or Form? A Bayesian Mixture Model for Customer Satisfaction Data".
When we look at customer satisfaction data we often look at what are the drivers of overall satisfaction. We naively assume that each question is answered independently so that we can use techniques such as regression analysis to identify those factors that most strongly relate to overall satisfaction. When this assumption is satisfied the "drivers of satisfaction" are valid and management actions taken as a result should have a positive impact on the organization.
When the assumption of independence is not met, the identification of drivers loses its validity. One of the most serious deviations comes from "Halos". Halos occur when a customer responds to a survey such that all the scores are identical and usually all top box (or bottom box). The customer is saying that the company scores top box with respect to every question. In other words, the company is perfect and all the areas requested for evaluation have outstanding performance. When all the scores are identical the term often used to describe this response is Halo. There can be a negative Halo as well as a positive Halo. The same logic works for the negative Halo, the customer is saying that all areas of the company being evaluated are equally dramatically under performing.
The Halo effect occurs most often when the respondents recall their overall satisfaction and then assign scores for each question consistent with their assessment of the overall score. When this occurs the components are uninformative about the drivers of overall satisfaction. In this case regression of overall satisfaction or dissatisfaction is not possible because the components are co-linear.
When conducting statistical analysis to determine the influence of drivers on overall satisfaction the halo responses must be removed from the response data set.
While many survey analysts try to find ways to remove outliers, the Halos are responses that should be eliminated from any statistical analysis.
The researchers point out that respondents who are familiar with the product or service are more capable of providing insights are found to be more likely to Halo their responses. On the other hand those respondents who are less aware of the products or services and are less capable of providing informed are more likely to make their responses more independent and less likely to Halo their responses. The problem is that neither of these scenarios are what would be appropriate for driver analysis.
The bottom line is that Halos are bad news for satisfaction and loyalty surveys. It is clear that analysts who are doing driver analysis need to be aware of the impact of Halos. This is one exception to the rule that outliers must be included. As the researchers note "the analysis of customer satisfaction data for the purpose of understanding drivers of overall satisfaction requires removal of the Haloed responses. From a managerial standpoint, Haloed responses are uninformative about the drivers of overall satisfaction because the component-specific information is suppressed."
Thursday, December 30, 2010
Monday, December 27, 2010
Impact of Lying on Surveys
Have you ever wondered what the impact of respondents lying on your survey might be? Repeated lying on surveys is known as farming and refers specifically to repeatedly completing a survey (correctly or otherwise). Thomas Chesney of Nottingham University Business School in Nottingham, UK and Kay Penny of Edinburgh Napier University in Edinburgh, UK have recently completed an interesting study to examine this possibility.
Web based businesses such as SurveyMonkey allow researchers or businesses to create and distribute online questionnaires quickly and at low costs. One of the main detractors to this method is the lack of certainty about who responded and how many times. As a secondary concern, how accurate are the demographic responses. Are the respondents who they claim to be? Some of the more sophisticated survey processes will not allow the same code to be used more than once; however, there may still be ways of providing multiple respondents. For example, it has been suggested that some respondents voted more than once for Sarah Palin's daughter when she was competing on the TV program "Dancing with the Stars."
There have been studies that show that an incentivize survey often leads to an increased response rate and the greater the incentive,the greater the increase in response. In the same way others have found that one big prize increases the response more than many small prizes despite the lower odds of winning. Another way to increase participation to use a prize draw rather than prepaid or promised monetary incentives.
A particular study by Morabia and Zheng in 2009 investigated the influence of entry into a raffle as an incentive for participation to an urban transportation survey in New York and found that approximately 4.7% of the participants were thought to have responded twice since they gave the same email address (are we talking dumb here).
A more subtle concern about lying on a survey is the respondent may be asked to answer questions that my impact their job. Something that business people often overlook is the possibility that one or more competitors may be responding to their surveys in ways that would distort the results and increase the possibility that decisions made as a result of the responses would be in error.
In reality there appears to be three basic types of farming; namely, the first is to repeatedly tell the truth, the second is to provide a random answer to each question and the third is to give perceived average responses to every response.
The finding of these researchers is that farming can lead to real and potential problems which cannot be predicted. The errors can be either a type I or type II error with no simple way to reduce or eliminate the errors.
The bottom line is that surveys on the web present real opportunities for lying and distorting the truth The best way to remove this possibility is good management. Of course, it can be virtually eliminated with phone or other direct survey techniques which are both more timely and expensive. Just another concern for those of us who attempt to extract accurate information from surveys.
Web based businesses such as SurveyMonkey allow researchers or businesses to create and distribute online questionnaires quickly and at low costs. One of the main detractors to this method is the lack of certainty about who responded and how many times. As a secondary concern, how accurate are the demographic responses. Are the respondents who they claim to be? Some of the more sophisticated survey processes will not allow the same code to be used more than once; however, there may still be ways of providing multiple respondents. For example, it has been suggested that some respondents voted more than once for Sarah Palin's daughter when she was competing on the TV program "Dancing with the Stars."
There have been studies that show that an incentivize survey often leads to an increased response rate and the greater the incentive,the greater the increase in response. In the same way others have found that one big prize increases the response more than many small prizes despite the lower odds of winning. Another way to increase participation to use a prize draw rather than prepaid or promised monetary incentives.
A particular study by Morabia and Zheng in 2009 investigated the influence of entry into a raffle as an incentive for participation to an urban transportation survey in New York and found that approximately 4.7% of the participants were thought to have responded twice since they gave the same email address (are we talking dumb here).
A more subtle concern about lying on a survey is the respondent may be asked to answer questions that my impact their job. Something that business people often overlook is the possibility that one or more competitors may be responding to their surveys in ways that would distort the results and increase the possibility that decisions made as a result of the responses would be in error.
In reality there appears to be three basic types of farming; namely, the first is to repeatedly tell the truth, the second is to provide a random answer to each question and the third is to give perceived average responses to every response.
The finding of these researchers is that farming can lead to real and potential problems which cannot be predicted. The errors can be either a type I or type II error with no simple way to reduce or eliminate the errors.
The bottom line is that surveys on the web present real opportunities for lying and distorting the truth The best way to remove this possibility is good management. Of course, it can be virtually eliminated with phone or other direct survey techniques which are both more timely and expensive. Just another concern for those of us who attempt to extract accurate information from surveys.
Thursday, December 16, 2010
Customer Advocacy
With the increasing use of social media, the concept of customer advocacy is starting to have legs. The Gartner research group has reported that more than $1.2 billion is spent on loyalty programs in the US. They also suggest that more than 75% of consumers have at least one loyalty card and more than 1/3 of the shopping population has two or more loyalty cards.
The idea of customer advocacy is based on the notion that someone who espouses a product, service or company is an advocate. Some loyalty scales used in the market describe people who score high on satisfaction surveys as advocates. The bad news is creating an advocate may not necessarily follow a score on a satisfaction survey. We have not seen enough evidence to validate the fact that a high satisfaction score creates an advocate; however, it is fairly certain that an advocate should be very satisfied before that person starts advocating a product, service or company.
The good news is that WOM is relatively inexpensive. There is some interesting research that bears prominently on this advocacy topic. The research was published in the September, 2009 issue of the Journal of Marketing. The study was titled "Effects of Word-of-Mouth Versus Traditional Marketing: Findings from an Internet Social Networking Site." The major finding was that WOM has a much longer carryover period. This suggests that WOM will persist long after normal activities designed to increase customer loyalty. The technical description of the result is that WOM is about 2.5 times higher than the average advertising elasticity.
The bottom line is that companies should start to focus their programs toward WOM on social networking sites and reward customer advocacy behavior. The challenge is going to be how best to get customers to become advocates. We will be searching the market for successes in finding and tracking customer advocates on the social networking sites.
The idea of customer advocacy is based on the notion that someone who espouses a product, service or company is an advocate. Some loyalty scales used in the market describe people who score high on satisfaction surveys as advocates. The bad news is creating an advocate may not necessarily follow a score on a satisfaction survey. We have not seen enough evidence to validate the fact that a high satisfaction score creates an advocate; however, it is fairly certain that an advocate should be very satisfied before that person starts advocating a product, service or company.
The good news is that WOM is relatively inexpensive. There is some interesting research that bears prominently on this advocacy topic. The research was published in the September, 2009 issue of the Journal of Marketing. The study was titled "Effects of Word-of-Mouth Versus Traditional Marketing: Findings from an Internet Social Networking Site." The major finding was that WOM has a much longer carryover period. This suggests that WOM will persist long after normal activities designed to increase customer loyalty. The technical description of the result is that WOM is about 2.5 times higher than the average advertising elasticity.
The bottom line is that companies should start to focus their programs toward WOM on social networking sites and reward customer advocacy behavior. The challenge is going to be how best to get customers to become advocates. We will be searching the market for successes in finding and tracking customer advocates on the social networking sites.
Tuesday, December 7, 2010
Endowed Progress
Have you ever heard of "endowed progress"? If you never have, you might want to read a research paper by authors Nunes and Dreze entitled "The Endowed Progress Effect: How Artificial Advancement Increases Effort." These researchers uncovered another people trait that improves sales, profit and (probably) loyalty. Endowed progress means that people who are given an artificial advancement toward a goal (such as a free trip) show greater persistence toward reaching the goal than they otherwise would. Artificial advancement is a way of appearing to give a customer a head start toward a goal even though it still takes the same effort to reach the goal. The example they use is an offer of a free airline ticket after 8 trips will not have as many people reaching the goal of 8 trips as when the offer is for a free trip after 10 trips but with a two trip head start. While both offers are the same, these researchers proved that customers were more likely to complete the 8 trips in a 10 trip program with a two trip head start than just doing 8 trips.
What is happening is that the task is reframed as a program that is already started. The customer tends to increase the chance that 8 trips will be taken and in a shorter period of time. They found that as people progress toward a goal, their effort will increase to get completion and as a result completion time will decrease given the head start.
Another finding from the research is that customer persistence depends on the relative progress made by the customer but NOT on the amount of the reward points or miles or other goal. thus, we might expect the customer will work harder as the goal comes into sight.
Their third finding was that endowed progress appears to be more effective when the customers are provided with a reason for the endowment. The customers want to know that there is a reason you are giving them a head start. The reason may be as simple as "we are starting a new loyalty program and are offering you this incentive to join our rewards program".
While I have never been a proponent of rewards programs, if you must use them, you might want to take advantage of this research. Although it suggests we are like sheep, their research is compelling. Try it, it just might provide a way of increasing sales, profit and loyalty.
What is happening is that the task is reframed as a program that is already started. The customer tends to increase the chance that 8 trips will be taken and in a shorter period of time. They found that as people progress toward a goal, their effort will increase to get completion and as a result completion time will decrease given the head start.
Another finding from the research is that customer persistence depends on the relative progress made by the customer but NOT on the amount of the reward points or miles or other goal. thus, we might expect the customer will work harder as the goal comes into sight.
Their third finding was that endowed progress appears to be more effective when the customers are provided with a reason for the endowment. The customers want to know that there is a reason you are giving them a head start. The reason may be as simple as "we are starting a new loyalty program and are offering you this incentive to join our rewards program".
While I have never been a proponent of rewards programs, if you must use them, you might want to take advantage of this research. Although it suggests we are like sheep, their research is compelling. Try it, it just might provide a way of increasing sales, profit and loyalty.
Saturday, November 20, 2010
Loyalty versus Trust
Edelman is the world's leading independent Public relations firm (at least that is what they advertise). Of interest is their Edelman Trust Barometer. They have just released their 2010 Edelman Trust Barometer Survey results. They report that they sampled 4,875 informed participants in two age groups (25-34) and (35-64). In order to qualify for the survey each must be college-educated with a household income in the top quartile for their age in their country. They must also read or watch business/news media at least several times a week and follow public policy issues in the news at least several times a week. Each interview participates in a 25-minute telephone. The results published in the 2010 survey was based on interviews between September 29 and December 6, 2009. The measurement is based on a 9-point Likert-type scale for trust. The scale notes that 1 denotes the lowest trust and d9 represents the highest trust.
With this background, it appears that their measurement system appears to follow sound survey techniques. They have summarized some of the general results that maybe of interest, namely,
1. Global trust in business is up modestly for the year.
2. The gain may be due to a spiked increase in a handful of Western countries. The trust measurement increased 18 points to 54% for the United States.
3. Trust also remains high in Brazil, India and China who each have a trust measurement higher than 60%.
4. The quality of products and services ranks higher than financial returns. according to Mr. Edelman,"Trust is now an essential line of business to be developed and delivered.
5. There appears concern by the majority that there will be a return to business as usual by companies as the economy recovers.
6. Sweden, Canada and Germany remain the most trusted countries for global headquarters.
7. Trust in government appears to be stable. However, the trust in the US is up 16 points to 46% and in Russia trust has decreased by 10 points to 38%.
8. In 20 countries corporate or product advertising continues to be the least credible source of information at 17%.
9. Reports from industry analysts and articles in business magazines remain the most credible sources of information about a company. However, the trust in mainstream media is waning.
Mr. Edelman notes that "we're seeing a vastly different set of factors driving reputation that we did 10 years ago". This is consistent with measures of loyalty that appear to have very different factors driving loyalty with an individual company.
The bottom line is that we now have another measure by which we can assess a company's strength in the market. The next step will be to compare these measures of trust with financial performance as has already been done using the ACSI measurement.
To learn more about the Trust Barometer you can find Edelman on the web at www.edelman.com.
With this background, it appears that their measurement system appears to follow sound survey techniques. They have summarized some of the general results that maybe of interest, namely,
1. Global trust in business is up modestly for the year.
2. The gain may be due to a spiked increase in a handful of Western countries. The trust measurement increased 18 points to 54% for the United States.
3. Trust also remains high in Brazil, India and China who each have a trust measurement higher than 60%.
4. The quality of products and services ranks higher than financial returns. according to Mr. Edelman,"Trust is now an essential line of business to be developed and delivered.
5. There appears concern by the majority that there will be a return to business as usual by companies as the economy recovers.
6. Sweden, Canada and Germany remain the most trusted countries for global headquarters.
7. Trust in government appears to be stable. However, the trust in the US is up 16 points to 46% and in Russia trust has decreased by 10 points to 38%.
8. In 20 countries corporate or product advertising continues to be the least credible source of information at 17%.
9. Reports from industry analysts and articles in business magazines remain the most credible sources of information about a company. However, the trust in mainstream media is waning.
Mr. Edelman notes that "we're seeing a vastly different set of factors driving reputation that we did 10 years ago". This is consistent with measures of loyalty that appear to have very different factors driving loyalty with an individual company.
The bottom line is that we now have another measure by which we can assess a company's strength in the market. The next step will be to compare these measures of trust with financial performance as has already been done using the ACSI measurement.
To learn more about the Trust Barometer you can find Edelman on the web at www.edelman.com.
Friday, November 19, 2010
Customer Facts to Consider for Improved Loyalty
I am close to the edge of doing what offends me. I am tired of reading 5 ways to increase loyalty or 7 steps to improve customer satisfaction or 6 ways to avoid losing customers, etc. I am about to walk on the edge by noting some information that may help you improve your customer loyalty. I have taken these facts from a blog by Loyalty 360 titled "11 Key Customer Loyalty Trends for 2011". I will leave the blog alone except that within the blog there were some statistics that might provide some guidance for the new year.
Here are some statistics that, in my opinion, might have relevance to understanding customers:
1. Behavioral economists note that economic decision making is 70% emotional and 30%rational.
2. A 20009 Gallup survey found that those companies that are in the upper half of both customer and employee engagement get a 240% boost in bottom line results compared with those who are in the upper half of customer engagement or in the upper half of employee engagement who only get a 70% boost.
3. Customer engagement is the holy grail for loyalty initiatives because engagement yields loyalty, advocacy, trust and passion which are the components that directly impact the bottom line.
4. The 18 to 35 year-olds' are particularly responsive to social and green issues. 85% say they would switch brands because of such marketing and 73% said they would try a new brand.
5. The National Restaurant Association found that 84% of those members who responded plan to invest more in their loyalty initiatives in the future because of the proven ability to drive business growth.
6. A surprise finding is that traditional incentive marketing does not drive consumer participation as much as achieved by gaming. The statistics are that 200 million people play games on Facebook every month and 24 games have more than 10 million users per month. The point is that a good game can impact consumer participation and may help build a lasting relationship and/or brand loyalty.
While there is no BIG message in this blog, it does provide some insight about customers. The bottom line here is that customers continue to change and are becoming more connected using social media. Companies must be prepared to adjust their strategies to meet the changing face of the customer.
Here are some statistics that, in my opinion, might have relevance to understanding customers:
1. Behavioral economists note that economic decision making is 70% emotional and 30%rational.
2. A 20009 Gallup survey found that those companies that are in the upper half of both customer and employee engagement get a 240% boost in bottom line results compared with those who are in the upper half of customer engagement or in the upper half of employee engagement who only get a 70% boost.
3. Customer engagement is the holy grail for loyalty initiatives because engagement yields loyalty, advocacy, trust and passion which are the components that directly impact the bottom line.
4. The 18 to 35 year-olds' are particularly responsive to social and green issues. 85% say they would switch brands because of such marketing and 73% said they would try a new brand.
5. The National Restaurant Association found that 84% of those members who responded plan to invest more in their loyalty initiatives in the future because of the proven ability to drive business growth.
6. A surprise finding is that traditional incentive marketing does not drive consumer participation as much as achieved by gaming. The statistics are that 200 million people play games on Facebook every month and 24 games have more than 10 million users per month. The point is that a good game can impact consumer participation and may help build a lasting relationship and/or brand loyalty.
While there is no BIG message in this blog, it does provide some insight about customers. The bottom line here is that customers continue to change and are becoming more connected using social media. Companies must be prepared to adjust their strategies to meet the changing face of the customer.
Friday, November 12, 2010
Is the Impact of the Internet on Loyalty Positive or Negative
Forrester Research has completed a study in the UK that suggests the Internet has created a platform that is having a negative effect on customer loyalty. A summary of the research was reported in ComputerWeekly.com in their 11/10/2010 issue. Some of the statistics that were derived from a survey of 500 shoppers in the UK are:
1. 60% of UK shoppers have not decided on the brand they will purchase before buying a product online.
2. 86% of the shoppers use ratings and reviews for online purchases.
3. 44% go online before buying products in-store.
4. 42% use their mobile phone while shopping of whom 16% used their phone to compare prices with other stores.
The UK is a relatively mature market. The average UK shopper spends about $900 annually compared with $750 in the US.
Based on these statistics one might conclude that shopping has become more of a commodity and buyers are either price or value shopping. There is little, if any, loyalty in price or value shopping unless the shopper adds value for a particular store or company.
On the other side of the coin there is an interesting note in facebooksniper.com that describes how companies can use Facebook and Twitter to increase incoming traffic to a website. Here are the steps that are suggested:
1. Sign up to Facebook.
2. Make friends in Facebook by joining groups or communities that match your market niche.
3. Send a friend request to people in the community or social network.
4. Use your existing customer's contact email in your email account.
5. Start promoting your business on Facebook - but do it tactically. One way is to promote your website is to create Fan pages representing your business and add people there.
6. Update your Fan pages with the latest happenings in your business.
7. Use Tweets to get the same message out.
8. Let users post their message so that it will be visible for all buddies in the friend list.
The bottom line is that the Internet can be a friend or enemy. It can be a friend if you learn how to use it to attract customers. It becomes an enemy when you allow your business to become a commodity with all your competitors. You choose.
In this new world of instant communication, those who understand the Internet will have a distinct and superior advantage to those who do not.
1. 60% of UK shoppers have not decided on the brand they will purchase before buying a product online.
2. 86% of the shoppers use ratings and reviews for online purchases.
3. 44% go online before buying products in-store.
4. 42% use their mobile phone while shopping of whom 16% used their phone to compare prices with other stores.
The UK is a relatively mature market. The average UK shopper spends about $900 annually compared with $750 in the US.
Based on these statistics one might conclude that shopping has become more of a commodity and buyers are either price or value shopping. There is little, if any, loyalty in price or value shopping unless the shopper adds value for a particular store or company.
On the other side of the coin there is an interesting note in facebooksniper.com that describes how companies can use Facebook and Twitter to increase incoming traffic to a website. Here are the steps that are suggested:
1. Sign up to Facebook.
2. Make friends in Facebook by joining groups or communities that match your market niche.
3. Send a friend request to people in the community or social network.
4. Use your existing customer's contact email in your email account.
5. Start promoting your business on Facebook - but do it tactically. One way is to promote your website is to create Fan pages representing your business and add people there.
6. Update your Fan pages with the latest happenings in your business.
7. Use Tweets to get the same message out.
8. Let users post their message so that it will be visible for all buddies in the friend list.
The bottom line is that the Internet can be a friend or enemy. It can be a friend if you learn how to use it to attract customers. It becomes an enemy when you allow your business to become a commodity with all your competitors. You choose.
In this new world of instant communication, those who understand the Internet will have a distinct and superior advantage to those who do not.
Subscribe to:
Posts (Atom)
Posts
