Words have value too!

A lot of time and energy is spent quantitatively analyzing the results of surveys. The survey may be based on metrics such as customer satisfaction, NPS, customer effort, or customer experience. This quantitative analysis may take on many dimensions of statistical methodology. However, the comments that are made associated with the individual scores are seldom included in the statistical quantitative analysis. There are several methodologies available for including the analysis of the comments (referred to as content analysis) with the numerical analysis.  It can be a worthwhile endeavor to compare the results of the quantitative and content analyses. 

A McKinsey study suggests that positive emotions correlate strongly with profits. The study indicated that after a positive customer experience more than 85% of the customers purchased more and after a negative experience more than 70% purchased less.

In this blog two different types of qualitative analyses are discussed; namely, descriptive measures of word usage and content analysis (which is also referred to as sentiment analysis).

Analysis of word usage includes grouping for individual words or word groups into categories. Words or word groups are usually grouped into three categories; namely, negative comments, positive comments or general comments. If the comment relates to a specific activity then those comments will be grouped by specific survey question(s). Within this grouping of word usage, descriptive measures provide a complementary presentation to the quantitative results. The descriptive measures are limited primarily to the number of comments associated with a specific question or questions on the survey.

The general hypothesis is that the descriptive measure of positive comments will be similar to that measured quantitatively. Obviously, the hypothesis is also extended to the negative remarks which should be reflected by the negative scores on the survey. This qualitative procedure does not interpret the intensity of any feelings implicit in the wording. In adition, the words will not pick up such subtleties as sarcasm or other emotions.

Content analysis (Sentiment analysis) is used to determine how customers feel with respect to a product or service. The primary purpose of sentiment analysis is to capture strong feelings that may be embedded in emotionally laden words. The two dimensions of sentiment analysis identify the feelings as being either positive or negative and the individual words or word phrases can be used to calibrate the magnitude of the sentiment.

Content analysis for product support is generally focused on the statistics associated with word usage (positive and negative). Typically the primary goal of content analysis is to validate the quantitative aspects of the survey metrics. Hence, the need for sentiment analysis is often limited. Only when word usage analysis indicates conflicting results with the quantitative analysis does sentiment analysis become a worthwhile addition to add to the perspective of the customers.

Too often surveys are conducted to understand the strengths and weaknesses of product support. The comments included with surveys are often reviewed individually but rarely analyzed in sufficient detail to provide verification of the quantitative measures taken.

The bottom line is that qualitative analysis is often overlooked when examining the relationship between customers and the products and services provided to them by the company. It may be time to reconsider what aspects of survey analysis should be included. It is likely that the best answer is both quantitative and qualitative analyses provide the greatest insight about the customers.

A note to remember is that feelings persist much longer than the score presented on the survey. An unkempt restroom at a restaurant can have a lasting effect on the customers that use it. It may be the only reason the customer never comes back. An emergency service provided beyond expectation may create a customer for life. Emotions are powerful.