Overcoming Non-Response Bias

Chris Thomas explains how speech analytics is able to provide a representative sample from people who don’t take part in customer surveys.

The presidential election of 1936 pitted Alfred Landon, the Republican governor of Kansas, against the incumbent president, Franklin D. Roosevelt.

The Literary Digest was one of the most respected magazines of the time and had a history of accurately predicting the winners of presidential elections that dated back to 1916.

For the 1936 election, the Literary Digest prediction was that Landon would get 57% of the vote against Roosevelt’s 43%. The actual results of the election were 62% for Roosevelt against 38% for Landon. The sampling error in the Literary Digest poll was 19%, the largest ever in a major public opinion poll. (Squire, 1988)

There were two issues with the Literary Digest poll.

1. Their contact lists came mainly from telephone directories. In the depression-hit 1930s, possession of a telephone indicated a level of wealth which determined a bias in the voting patterns of those surveyed.
2. While their survey contact list was huge (they sent polls to 1 in 4 voters across the country) their non-response rate was extremely high – giving high bias towards those with strongly polarised opinions.

Post-call surveys of Customer Satisfaction and Net Promoter Score are subject to these same sample error problems because of the self-selection of the responders and the likelihood that responses will come mainly from those with strong positive or negative opinions.

Sample bias and non-response bias mean that contact centre survey tools are likely to provide a highly unrepresentative measure of customer sentiment, which would be potentially disastrous if used as the basis for business decisions on products, processes, staffing and training.

Speech analytics is able to remove all sample bias and non-response bias by tracking and measuring customer sentiment in all interactions without requiring the knowledge or participation of the customer.

In the 1936 election a new upstart polling company did give the correct prediction. By taking a representative sample and removing bias, the Gallup organisation were able to accurately predict the Roosevelt landslide.

In the same way, speech analytics is able to provide a representative sample and accurate customer satisfaction measures by harvesting opinion from all those people – like you – who don’t take part in customer surveys.

With thanks to Chris Thomas, Sales Director at CallMiner