AI in Survey Based Market Research
Michael Wagstaff • 7 August 2023
AI is here already and there's no need to be afraid
Articles on artificial Intelligence (AI) are commonplace at the moment. And one of the really interesting things is the number that are illustrated with a robot in human form or a half robot - half human hybrid. In fact, rather like the illustration we have used for this article!
AI is often associated with a post apocalyptic world of sentient robots ruling over a dystopian landscape. This just goes to show how people associate AI with sentient beings that can act and think for themselves and unless we ACT NOW, will take over the world.
The reality is a lot more mundane. In fact AI is already deeply embedded in the survey process, particularly in relation to sampling and analysis, in a very non threatening way. Here's how.
- Sampling Optimisation: AI can help in designing effective sampling strategies that ensure a more representative and unbiased sample. Machine learning algorithms can analyse historical survey data to determine the best sampling techniques that can yield the most informative results.
- Improved Representativeness: With AI, it is possible to improve participant response by analysing demographic data, past survey responses, and other relevant data. This ensures that the surveys reach the most relevant audience, thus improving the quality of the sample.
- Bias Reduction: AI can help in identifying and reducing various kinds of biases that can creep into surveys, such as selection bias, nonresponse bias and coverage bias. By identifying patterns in who is not responding or how certain groups are over- or under-represented, adjustments can be made to reduce these biases.
- Data Cleaning: Machine learning algorithms can help in cleaning the survey data by identifying and handling outliers, missing values and other anomalies that can skew the results.
- Adaptive Survey Design: AI can help design adaptive surveys where the questions posed to the respondents are dynamic and change based on their previous answers. This not only increases the relevance of the survey but also improves response rates and the overall quality of the sample.
- Response Quality Check: AI can assess the quality of the responses in real-time, checking for inconsistencies or indications that the respondent is not providing thoughtful answers (for example, always choosing the same option). It can then flag these for review or potentially discard them to maintain the integrity of the sample.
- Automated Follow-ups: AI can automate the process of following up with respondents who have not completed the survey, increasing the chances of obtaining a complete and representative sample.
- Predictive Analysis: AI can analyse the survey data and predict potential trends or patterns in the responses, allowing researchers to modify the survey in real-time to better meet their objectives. This is particularly helpful during a soft launch phase.
- Enhanced Analysis: Post-survey, AI can assist in data analysis, employing advanced techniques such as sentiment analysis, topic modelling and clustering to deliver more nuanced insights from the survey data.
None of the above points are things to fear. In fact, all of them are being done right now in most market research agencies. AI is being used to make the survey process smoother, better and more accurate.
AI is taking over the world but in a good way.
Housing affordability is getting worse and there is a need to balance policies relating to demand with those addressing supply.
After yet another market research conference calls for great emphasis on storytelling, we discuss why researchers find it so difficult and what can be done to get better at it.
A university degree doesn't necessarily make you a better researcher but it does help. We have a look at whether a degree should be a requirement for market research roles.
Given the huge amount of brand insight that can be derived by applying techniques such as text analytics to existing data, we ask whether the end of the survey for brand metrics is in sight?
With the insights industry creaking under poor quality data, generative AI adds to the mix by creating convincing synthetic respondents. How should we treat these fascinating fakers?
Do tech people know more about market research than social scientists? In this article we discuss whether big tech is taking over the insights industry and what role is there for human researchers.
Does curating a personal brand make a difference? In this article, I consider my own lack of brand image and reflect on what might have been.
In an effort to improve quality, the market research industry's current focus is on survey length and question engagement. This overlooks a more fundamental issue: respondent authenticity.
The true value of review sites lies in going beyond the stars and analysing what reviewers are actually saying.
Predictive analytics is reshaping brand insights by moving beyond descriptive dashboards to proactive decision-making