Forty Years On

Michael Wagstaff • 2 September 2024

How my research skills have evolved

It's almost 40 years since I started working in research and over the course of my career, I’ve seen some pretty big changes in the skills and approaches needed to make a difference in this field. From my early days in academia to the fast-paced world of online surveys and now to text and data analytics for brands, the skills I’ve picked up along the way have definitely evolved.



The learning years

I started out in academic research, which is all about precision and rigour. Back then, my work was heavily focused on hypothesis testing, literature reviews and applying statistical techniques to ensure every finding was backed by solid evidence. This time was all about applying a methodical approach to problems, which really set the stage for everything that came after. It also introduced me to SPSS for analysing data which was an essential tool for uncovering insight.


Academia laid the foundations for a career in research and gave me a deep appreciation for data accuracy and the importance of evidence-based decision making.


The listening years

After academia, I joined a local government research team and that’s where things started to change. I had to learn to really listen to people to fully understand their needs. This time showed me the importance of empathy and communication—skills that have remained important throughout my career.


And as a commissioner of research, I also got to see what it's like being a client of a research agency. This showed me the importance of only promising what you can actually deliver!


The agency years

Next was a move into market research, mainly because it paid more! The pace was much faster and there was pressure on getting things done quickly and efficiently. And doing them online. For some of this time, I worked for the agency that pioneered online research and convincing clients (and indeed myself) of its merits was a challenge.  I still feel uncomfortable about applying probability based statistics to non-probability samples but acknowledge that that ship has well and truly sailed.


During the agency years, I had to be able to write questions that were both accurate and unbiased and surveys that were short and sharp. On top of that, I learnt how to blend qualitative insights with quantitative data to tell a more complete story. I also had to learn how to present insights in PowerPoint without sending the audience to sleep.


There was also a need to have a more commercial mindset. This meant making sure that the research had to directly help client businesses while at the same time making the agency money.


The analytics years

Today my work involves using text analytics, predictive modelling and techniques such as network analysis to extract customer insights. The tools I use have changed a lot, too—gone are the days of SPSS and PowerPoint; now it’s all about Python and advanced analytical techniques I didn’t even know existed when I started out.


It’s been hard work getting up to speed with all these new tools but it’s also been really exciting to see how much more we can do with the data we already have.


Looking back

Looking back, it’s clear that while the hard skills needed for research have evolved with the tools and technologies, the soft skills have remained pretty much constant. Being able to listen, communicate effectively and understand the needs of different people is still a huge part of the job. As an insight professional today, it’s not just about being technically skilled; it’s also about being able to translate complex data into insights that can really drive decision-making and make a difference.


The insight world is always changing and so are the skills we need to succeed in it. From the rigour of academic research to the fast-paced demands of online research and now to the data analytics focus of my current role, adaptability and a willingness to learn have been key.


It’s a constantly evolving field and I think that’s what makes it so exciting to still be be a part of 40 years on.

by Michael Wagstaff 2 June 2025
With the insights industry creaking under poor quality data, generative AI adds to the mix by creating convincing synthetic respondents. How should we treat these fascinating fakers?
by Michael Wagstaff 26 May 2025
Do tech people know more about market research than social scientists? In this article we discuss whether big tech is taking over the insights industry and what role is there for human researchers.
by Michael Wagstaff 12 May 2025
Does curating a personal brand make a difference? In this article, I consider my own lack of brand image and reflect on what might have been.
by Michael Wagstaff 9 May 2025
If surveys are ill-suited for addressing the questions that remain genuinely unanswered and unnecessary for those that are readily answered, then what exactly is their future?
by Michael Wagstaff 8 May 2025
In an effort to improve quality, the market research industry's current focus is on survey length and question engagement. This overlooks a more fundamental issue: respondent authenticity.
by Michael Wagstaff 7 May 2025
The true value of review sites lies in going beyond the stars and analysing what reviewers are actually saying.
by Michael Wagstaff 28 April 2025
Predictive analytics is reshaping brand insights by moving beyond descriptive dashboards to proactive decision-making
by Michael Wagstaff 28 April 2025
The uncovering of large scale survey fraud has shaken the market research industry. In this article, we discuss how a much more rigorous panel sign up procedure is needed to reduce the impact of fraudulent respondents.
by Michael Wagstaff 8 April 2025
The huge volume of data available through consumer comments, reviews and surveys can make cutting through the noise difficult. In this article we discuss how text analytics combined with human expertise can uncover the insight.
by Michael Wagstaff 10 March 2025
Market research agencies are going all in on AI based models to generate next level consumer insight. But are these just more illusion than substance?
Show More