GOSH Food and MMR have teamed up to investigate the impact of emerging technologies, focusing on synthetic AI personas, on early-stage R&D. This presentation will cover the insights from our cutting-edge experiment, examining if and how these AI-generated personas can streamline the ideation process for brands. Join us as we share the outcomes of this venture, offering a glimpse into the practical applications and challenges of integrating AI into the initial stages of product development and consumer research.

One of the somewhat underappreciated use case for Large Language Models such as chatGPT is classification in zero-shot or few-shots learning. Based on their prior understanding of target classes and semantic content, LLM are able to categorize verbatims on the basis of zero or few examples into standard (think ‘positive’ or ‘negative’) or less standard (‘vague’ vs ‘precise enough’) categories. In this talk we will present two use cases of this in market research, highlighting the benefits and pitfalls of the chatGPT approach compared to standard machine learning:

a. sentiment analysis, with an eye to the distinction between verbatims that express a neutral sentiment and those that do not express a sentiment at all. We will compare the performance of chatGPT with those of a vanilla BERT based language model and a fine-tuned model utilizing panel previous panel respondents data.

b. online qual moderation, showing how priming chatGPT into fine-grained classifications helps provides the basis for a successful automation of online qual moderation.

Explore the innovative use of generative AI and vector databases in the presentation “Every Library Deserves a Librarian.” This session will delve into an ongoing project with the Archive of Market and Social Research (AMSR), an essential resource for the market research industry. The presentation will showcase an AI-driven solution designed to enhance the way users interact with the decades of digitized reports contained in the archive. By utilizing technologies such as large language models, sentence-transformers, and full-text search, users can effortlessly query the archive with conversational-style questions and receive both a conversational response and links to the original documents. The discussion will also cover the benefits, risks, and future directions of this approach, providing a more efficient and engaging experience for both professional researchers as well as those outside of the industry. 

Automated reporting has reached new levels from just basic “auto-charting”. In this joint case-study and presentation we’re going to explore new breakthroughs in automation: We will explore case study examples of complete, client-ready decks with every detail automated – whether in PowerPoint, Excel, PDF or even Google Slides and Sheets. Additionally we will see open cloud-based technology that integrates powerful on-demand  automation from online dashboard platforms.  E-Tabs CEO Benjamin Rietti will be joined virtually by Susan Collicott, information designer at Escalent.

What does AI mean for the kids? Richard Owen will walk us through 6 guiding principles for deploying this new research technology among an audience of under-eighteens. Learn how to adapt youth research best practices for AI tools and anticipate considerations from child respondents and their parents.

Data Quality is a top priority for both market research agencies and their clients, as the industry relies on delivering reliable and trustworthy data. At the same time, guaranteeing quality data in online research with our traditional quality systems is becoming increasingly difficult. It often feels like we are upgrading quality checks just to find that fraudsters have already found a way to circumvent them.

This is a shared challenge across all sample providers, and it is clearly accelerating. On the one side we face an inherent imbalance between the demand for respondents and the supply of individuals willing to participate. While post-pandemic cultural shifts, inflation and increasing economic uncertainty all contribute to this changing ecosystem.

On the other hand, bad actors now have easier access to technologies to overcome the quality checks we have in place. They employ a range of methods, from using language models to generate fake open-ended responses, to harnessing multi-account management systems that create artificial survey participants, to automated systems that transfer data across different survey pages. The ease and speed with which these technologies are used and developed, poses a significant challenge to maintaining the integrity of

our research.

In this talk, we’ll explore how we see the challenge of quality evolving and will showcase how rapid advancements in technology are impacting our combat against fraud. The aim is to raise awareness among researchers, clients, and the wider research industry to help us all recognize and respond to fraudulent patterns in data more swiftly. Finally, this is a challenge that we will need to address as an industry.

This presentation will examine the challenges of generating reliable synthetic data, which at its heart centres around having reliable training data.  It will look at how reliably or not data can be synthetically generated from different types of survey and different types of survey content and look at how it can be improved by improving these surveys and augmenting them with additional datapoints.

The online data collection and research community has been hit hard over the last couple of years with an influx of fraudulent participants. The sector under the Global Data Quality initiative with leadership from the likes of MRS, Insights Association, ESOMAR have come together to address ongoing and emerging risks to data quality in market and social research, consumer insights and analytics industry with the goal of increasing information and building trust.

MRS is overseeing work streams focussed on fraud detection. One key area is to understand some aspects of “poor data” and MRS have been conducting an in-depth audit of a badly hit survey. The results so far have indicated the need for additional views on the data and more transparency on the data flowing into the survey

The presentation will cover

• Results from the audit
  • Including what additional data points highlighted the issues
• Sample ecosystem
• Information that can help improve the quality of data
• Potential impact on the research process
• Next steps for the Global Data Quality / MRS

In line with one of the conference topics of interest, we would like to rely on the distinction between AI as a tool and AI as teammate in order to frame a meaningful way to think about AI and the future of market research. If AI is a tool, we need to master that tool, finding out the right moves that will predictably result in the right effects. If AI is a teammate, we need to cooperate with that teammate, finding out the right division of labor and learning to trust her. Based on our experience with using generative AI in qual research, particularly for moderation and analysis, we wish to argue that generative indeed is a teammate of sorts. In both cases, using gen AI as a tool to produce predictable results carved to our likings fails. In both cases, we need to invent a new division of labor where gen AI takes on the right tasks, leaving our teammate do her job as she knows to do it, and taking up on ours when need be. However, AI is no ordinary teammate: she does not spontaneously take responsibility for what she does, so that it remains our job to held her accountable for an explanation of her choices.

We will illustrate the tenets and the outcome of this approach on a handful of very concrete examples, in particular appealing to a demo qual  research we have conducted in the UK, France and Germany about fashion and sustainability, involving 40 fashion conscious participants per country. As a moderator dedicated to helping respondents to themselves go to the bottom of what they mean, AI was the right teammate during fieldwork. As a meticulous and tireless reader dedicated at identifying trends, AI was the right teammate during analysis. Throughout these processes, cooperation remained paramount—neither side should overstep their bounds, and each should fulfill their respective responsibilities. Moreover, accountability emerged as equally essential—dissociating from AI is not the optimal approach to harnessing the benefits of the AI revolution.

In this interactive workshop, Paul Hudson will lead a guided, interactive debate in which teams explore the complex trade-offs between stakeholder and participant needs. We’ll highlight the common tactics for improving PX and the ways in which benchmarking survey quality can give stronger voice to participants before research goes to field. Together, we’ll investigate the merits of each tactic and build a framework for a programme of survey data collection that is truly customer-centric.

“Legends is not a research company” words I’ve heard over and over, yet our competitors are finding their feet and becoming a viable alternative.

During this presentation, I would like to focus on how I am on a journey to change the shape of how the company approach Market Research, think bigger and look to shake off the shackles of being Data Rich Information Poor.

I would like to focus on some horror stories and how a degree of diplomacy has needed to be applied to turn the ship (albeit very slowly) and adjust our approach beyond the white elephant to a model befitting the largest company in our field.

Innovation comes through experimentation and trialling new and different approaches. But not every experiment is going to lead to success, and businesses are naturally reluctant to dedicate time and money to ideas that may not ever result in a commercial product. 

Niall Smith, founder of Sketchbook will host the following panel of experts from across the sector who will discuss the research industry’s attitude to failure and risk, and how to foster creativity and innovation in the sector: 

Jack Bowen, Founder of CoLoop.ai  

Richard Owen, Head of Innovation at Firefish 

Rebecca Cole, Managing Director at Cobalt Sky  

Danny Russell, Owner DRC 

Our industry’s participant crisis is a problem of our own making. By going back to first principles of human behaviour and psychology that our work is rooted in, embracing lessons from SaaS product management, and embracing technology, we can start to solve the problem from recruitment through to retention – including taking significant steps towards tackling fraud.

Price compression in the cost per interview has caused a decline in the value of survey incentives paid to research participants.  At the same time, the growth of programmatic sampling has caused an increase in uncompensated screening work by research participants.   Research panel members are not happy and it has become difficult and expensive to retain members or to recruit new members.  As a result, sample providers are increasingly relying on non-panel, intercept samples which has caused a rash of quality problems in the industry.  Quality problems have led to enhanced validation processes for people to join panels which has increased the friction and further reduced the value proposition of doing surveys.  Based upon research among panel member participants in the UK, recommendations will be made on how to win back and satisfy research participants.

Generative AI is going to transform research technology so it can be aimed at business questions, not just research questions. Instead of platforms that allow us to efficiently script surveys and view basic data visualisation dashboards, it will interrogate business questions in order to identify information gaps, recommend and execute research and circle back to the business question to deliver compelling insights. Bounce AI models have been trained by researchers on real-world business problems, market research methodologies and data, broadening the scope of impact for research technology and delivering more relevant answers to business questions. This is how Researchers and Generative AI together are going to transform the research industry.

This paper will principally address the category of “A new lens on engagement and fraud”. However, it will also overlap with other submission categories, namely “Practical considerations of living with AI” and “What does ‘good’ look like today?”, as the paper will demonstrate how AI-powered approaches, specifically Conversational AI, can provide quality and value. There are many reasons why survey data quality seems to be in perpetual decline, including lazy and professional respondents and bots. However, poor quality, boring surveys certainly don’t help. If people are bored or disinterested, they pay less attention, provide less considered answers and data quality suffers as a result.

This paper will discuss how FAR – Fun, Anonymity and Relevance – can drive survey engagement and improve data quality. We’ll start with how we make surveys more FUN. One approach is to reimagine surveys as conversations. With a traditional survey, people receive a set of questions and provide their answers. They respond and so we call them respondents. There is no interaction and often it is not a very engaging experience for people. On the other hand, conversations are everything that surveys aren’t; they are two-way rather than one-way, conversations involve listening as much as asking, conversations can be emotional, surprising and FUN. And when you are part of a conversation, you are a participant, not a respondent. We’ll demonstrate how conversational research builds engagement and we’ll show how participants describe the experience (hint, they find it better and more fun than other surveys!)

Conversational AI uses AI to probe the answers people give to open-ended questions. This leads to richer, more insightful open-ended data compared to typical online surveys. Two really interesting aspects of conversational research are the evidence that the delta between conversational AI and regular surveys is greater both when a) the topic is a sensitive one and b) when the audience is from under-represented or disadvantaged communities.

Why is this? We don’t know for sure, but we hypothesise that the ANONYMITY of talking with a chatbot makes people feel more comfortable in sharing their feelings and behaviours and that this is particularly so when people are asked about sensitive topics and among under-represented audiences who may feel marginalised by traditional approaches. So we’ll discuss if conversational research could be an enabler of better representation in research.

As well as being used to ask open-ended questions, AI, powered by large language models, can be trained to measure the quality of open-ended responses. There are different aspects to this including identifying uninformative or gibberish answers which is relatively straightforward. However, detecting irrelevant answers is harder yet important when bots have been trained to get around checks on response length and the use of gibberish by proving longer, yet often irrelevant answers. So we’ll explain how AI can be used to measure RELEVANCE of answers. Whilst the issue of survey fraud is complex and addressing it requires a number of solutions, measuring for relevance provides a useful weapon in the researchers armoury to help combat fraud.

Crafting a local heartbeat for global brands in every corner of the world can be slow and costly. Our paper will showcase how a global personal care brand, in partnership with Hello Ara, leverages Conversational AI, generative AI, Metaverse co-creation, and human imagination to transition from insight to rapid action.

Key Takeaways:

1. Experience the synergy of generative AI, Metaverse Qual, and Conversational AI quant.
2. See generative AI, pre-trained with Brand DNA, in action in real-time qual and quant research.
3. Discover the value of an iterative design process with AI and imagination to create differentiation.

The latest “trend” in the AI world are RAGs. No, not Ragtime – the catchy music from the early 1900’s from composers such as Scott Joplin. These RAGs are the “catchy” named “Retrieval Augmented Generation” interfaces for data. They are a mixture of the latest form of database – vector databases and LLMs (Large Language Models). The talk is intended as practical introduction to the concepts behind RAGs, how to build one and how they can help with the analysis of open end data and text derived from focus groups et al. RAGs are, infact, a unique form of modelling that are a wholly new way to present clients with results and allow them to navigate text data of any form. RAGs allow client to “talk” to their data in a totally new way.

Recent research by the GRBN highlighted ways in which companies can improve the business impact of their insights functions. This involves making the journey from being traditional insights functions, where they act mainly as a supplier, taking briefs from stakeholders and delivering on them; to a source of competitive advantage, where they are corporate leaders, focused on foresight and innovation. 

One of the ways in which it’s been shown companies can make the transition to being a source of competitive advantage is by aligning their insights and analytics functions and bringing the two together. This is the main reason the AIA (Advanced Insights and Analytics) Council of the MRS was born – to encourage and support companies to bridge that gap between insights and analytics, instead of the two functions acting as separate entities, often in silo. 

The members of the AIA have come together to develop their top pieces of advice on exactly how to do this. Actionable take-aways you can implement in your in-house insight teams straight away, as well as the evidence behind why it’s so important to do, and the benefits that can be gleaned from putting in the effort.  

Join us for an in-depth workshop on creating a data warehouse tailored specifically for survey data. This session is designed to help you understand the unique challenges and key decisions involved in integrating survey data into a data warehouse.   

This workshop is ideal for anyone involved in survey data who wants to understand the foundational structures that are required to take advantage of the modern data stack. 

In the workshop we’ll cover:

1. Why we need data warehouse models for survey data
2. Understanding data warehousing concepts
3. A model structure for survey data warehousing
4. Identifying key attributes for our data
5. Putting it into practice

An insightful session where we delve into the transformative power of interdisciplinary collaboration in bringing a bold innovative design that boosts experiences and evoke emotions to life. Through a holistic and collaborative approach, we leverage modelling, and gaming software, cutting-edge AI to meticulously craft an beautiful and innovative 3D environment. Explore the seamless fusion of creativity, technology, and the concepts of education, entertainment, and socialization as we unveil the captivating journey of harvesting interdisciplinary collaboration. Gain a deep understanding of how this collaborative effort reshapes traditional notions of playing, learning, creativity, and collaboration in the digital age.

In this session, we invite you to glimpse into the future of digital spaces and discover the limitless possibilities of creativity and innovation within the case study process of the creation of Dom World – a Hybrid Creative Hub, alongside other inspiring projects of ours. Witness firsthand the symbiotic relationship between technology and creativity as we showcase the outcomes of our collaborative endeavours.

Join us as we embark on this exciting journey towards reshaping the way we learn, create, and collaborate in the virtual world, where the boundaries of imagination are boundless. Be part of this transformative experience as we chart new territories and redefine the landscape of virtual collaboration and creativity. Engage in discussions, share insights, and connect with fellow enthusiasts as we collectively envision the future of immersive experiences and digital creativity.

Market research is a highly devolved process, with lots of different people involved in a project from end to end.  From the research buyer (and their internal stakeholders), the research agency executive team, data processing, sample providers (and maybe interviewers), survey software suppliers and research participants themselves.  For those “working at the coal face” of survey data, data quality is rightly of high priority, with concerns around the rise of survey bots, and survey fraud being “hot topics”.  But what does “good” look like to other people involved in the process?

This session will look to explore the agendas of different stakeholders in the research process through the lens of the project management triangle of time, cost and quality.

This 20 minute presentation will use short video vox pops from people working at different stages of the process discussing what good looks like to them (and what they think good looks like at other stages of the process) to weave this narrative together and highlight the sometimes competing agendas of different research stakeholders.

ASC has long been an active supporter in the drive towards better interoperability within survey research and between different platforms. This workshop is intended as a first step towards creating a new manifesto for the industry, capturing the essence of what developers and implementers need to address in today’s more diverse technology and data landscapes.

This session looks at what is next for the research industry. The panel will reflect on the content of the conference and give their expert opinions on how they believe our industry will continue to transform. In sixty minutes, they will explore the impact of AI, The role of technology as a driver for change, the future of insights, the future of participant engagement, data quality, what skills will be required, and how the structure of the industry might change. Should we be concerned for our survival or look forward to a prosperous future?