Are The Days Of The Survey Numbered?

I've worked in market research for 40 years and what we're currently witnessing with AI is unprecedented. AI has enabled us to analyse vast datasets quickly and this poses a fundamental challenge to the continued existence of the market research industry.

The answer's out there

My gut feel is that about 80% of survey questions asked today already have answers tucked away in previous surveys, business management databases or in publicly accessible sources. If we can reuse, recontextualise and re-analyse this data to give us the insight we need, why do we need to run a survey? All we need is a system capable of sorting through the huge amount of existing data to get the answers we need.

This is where Large Language Models (LLM) such as ChatGPT, DeepSeek and Grok come in. LLMs can synthesise large datasets swiftly and far quicker than a human can. They mine existing and historic data to provide immediate, (mostly) reliable answers to questions that previously would have required new surveys and can undertake predictive analysis to make future forecasts of where trends are taking us.

The research bell curve

To visualise where I think we're at, think of a bell curve distribution. Most research questions are situated comfortably in the middle of that distribution. This is my 80% that are already answered by existing information. The ends of the bell curve - the niche inquiries and highly nuanced issues - represent the outliers. These are precisely the cases for which market research surveys are most needed.

But this in turn raises a paradox because it is here that  surveys are at their weakest. The subtlety, nuance and highly specific context and samples needed to address outlying questions often evade traditional survey methods. This means they are ineffective precisely when they’re most needed.

I think this raises a troubling question for market research: if surveys are ill-suited for addressing the questions that remain genuinely unanswered and unnecessary for those that are readily answered, then what exactly is their future?

Market research reinvented

It seems clear that market research requires reinvention rather than a bit of tinkering around the edges. Surveys will not disappear entirely but their relevance will diminish unless they evolve to target precisely those difficult areas at the extremes. This transformation will mean fewer surveys but ones of deeper quality, insight and sophistication. The focus will shift from routine fact-finding to precision exploration and the deep understanding of human motivations, feelings and reactions that quantitative data currently struggles to capture.

The market research industry has to ask itself whether the traditional survey's twilight is approaching. My feeling is that it  is. The answer lies not in clinging nostalgically to the past but in reinventing the whole survey process to more precisely address the 20% of research questions for which we don't already have answers.

In that future, market research will look very different.