Forty Years On

Michael Wagstaff • 2 September 2024

How my research skills have evolved

It's almost 40 years since I started working in research and over the course of my career, I’ve seen some pretty big changes in the skills and approaches needed to make a difference in this field. From my early days in academia to the fast-paced world of online surveys and now to text and data analytics for brands, the skills I’ve picked up along the way have definitely evolved.



The learning years

I started out in academic research, which is all about precision and rigour. Back then, my work was heavily focused on hypothesis testing, literature reviews and applying statistical techniques to ensure every finding was backed by solid evidence. This time was all about applying a methodical approach to problems, which really set the stage for everything that came after. It also introduced me to SPSS for analysing data which was an essential tool for uncovering insight.


Academia laid the foundations for a career in research and gave me a deep appreciation for data accuracy and the importance of evidence-based decision making.


The listening years

After academia, I joined a local government research team and that’s where things started to change. I had to learn to really listen to people to fully understand their needs. This time showed me the importance of empathy and communication—skills that have remained important throughout my career.


And as a commissioner of research, I also got to see what it's like being a client of a research agency. This showed me the importance of only promising what you can actually deliver!


The agency years

Next was a move into market research, mainly because it paid more! The pace was much faster and there was pressure on getting things done quickly and efficiently. And doing them online. For some of this time, I worked for the agency that pioneered online research and convincing clients (and indeed myself) of its merits was a challenge.  I still feel uncomfortable about applying probability based statistics to non-probability samples but acknowledge that that ship has well and truly sailed.


During the agency years, I had to be able to write questions that were both accurate and unbiased and surveys that were short and sharp. On top of that, I learnt how to blend qualitative insights with quantitative data to tell a more complete story. I also had to learn how to present insights in PowerPoint without sending the audience to sleep.


There was also a need to have a more commercial mindset. This meant making sure that the research had to directly help client businesses while at the same time making the agency money.


The analytics years

Today my work involves using text analytics, predictive modelling and techniques such as network analysis to extract customer insights. The tools I use have changed a lot, too—gone are the days of SPSS and PowerPoint; now it’s all about Python and advanced analytical techniques I didn’t even know existed when I started out.


It’s been hard work getting up to speed with all these new tools but it’s also been really exciting to see how much more we can do with the data we already have.


Looking back

Looking back, it’s clear that while the hard skills needed for research have evolved with the tools and technologies, the soft skills have remained pretty much constant. Being able to listen, communicate effectively and understand the needs of different people is still a huge part of the job. As an insight professional today, it’s not just about being technically skilled; it’s also about being able to translate complex data into insights that can really drive decision-making and make a difference.


The insight world is always changing and so are the skills we need to succeed in it. From the rigour of academic research to the fast-paced demands of online research and now to the data analytics focus of my current role, adaptability and a willingness to learn have been key.


It’s a constantly evolving field and I think that’s what makes it so exciting to still be be a part of 40 years on.

by Michael Wagstaff 18 July 2025
Housing affordability is getting worse and there is a need to balance policies relating to demand with those addressing supply.
by Michael Wagstaff 23 June 2025
After yet another market research conference calls for great emphasis on storytelling, we discuss why researchers find it so difficult and what can be done to get better at it.
by Michael Wagstaff 21 June 2025
A university degree doesn't necessarily make you a better researcher but it does help. We have a look at whether a degree should be a requirement for market research roles.
by Michael Wagstaff 19 June 2025
Given the huge amount of brand insight that can be derived by applying techniques such as text analytics to existing data, we ask whether the end of the survey for brand metrics is in sight?
by Michael Wagstaff 2 June 2025
With the insights industry creaking under poor quality data, generative AI adds to the mix by creating convincing synthetic respondents. How should we treat these fascinating fakers?
by Michael Wagstaff 26 May 2025
Do tech people know more about market research than social scientists? In this article we discuss whether big tech is taking over the insights industry and what role is there for human researchers.
by Michael Wagstaff 12 May 2025
Does curating a personal brand make a difference? In this article, I consider my own lack of brand image and reflect on what might have been.
by Michael Wagstaff 8 May 2025
In an effort to improve quality, the market research industry's current focus is on survey length and question engagement. This overlooks a more fundamental issue: respondent authenticity.
by Michael Wagstaff 7 May 2025
The true value of review sites lies in going beyond the stars and analysing what reviewers are actually saying.
by Michael Wagstaff 28 April 2025
Predictive analytics is reshaping brand insights by moving beyond descriptive dashboards to proactive decision-making
Show More