AI systems present both a existential challenge to surveys and a mechanism for their re-birth. AI’s make the analysis of “natural” conversations far easier and faster. Surveys were invented because we could not easily analyse conversational data and we need it to be more “formalized” so we could fit it into statistical analysis. There is nothing natural about 5 point scales, multiple choice questions and NPS etc. People just do not think or act in those terms. We need data from AI’s to be in that form because we need to perform analysis on data “the way we have always done it”. And then there is the question of “synthetic data”, AI’s can simulate respondents or respondent segments in an incredibly life like way. Once we have these simulacra of consumers what do we do with them ? Do we use an approach based on the variability of respondents – surveys – to elicit data from them ? Or do we just ask them direct questions and skip the interminable 5 point scales ? The talk will discuss the development of an AI surveying system – Tullius – which is designed to interact with AI’s/personas and “interview” them. The talk will also discuss if this approach is just an appeasement to the needs of analysts so used to humans and their variability that we see the need to simulate that flawed behaviour with AI’s just so we feel comfortable with the data. The talk will also discuss the performance of AI’s when presented with “traditional” questionnaires. Finally we will discuss the whole concept of interviewing personas – is it just “faster horses”, are we simply not using the AI’s to their full potential by imposing an outmoded approach to data elicitation. We are treating the AI’s as people, but they are not people. 

AI moderation has the potential to genuinely democratise research – rewriting the rule book on how we design surveys, ask questions, and interpret results. Using a WhatsApp-based AI moderator they’ve developed, Maria and Andrew will share examples of real conversations with participants and how they unfold. They’ll put human-written and AI-generated questions head-to-head, sharing an honest take on what works and what doesn’t. Expect insights on engagement, usability, ethics, and what participants really think. You’ll leave with clear guidance on when and how to use AI moderation, what it means for the future of surveys, and the skills researchers will need next. 

AI is embedded in our creative workflows, powering text, image, sound, and code generation at unprecedented speeds. But how do we move beyond the “demo effect” and make these systems meaningful, ethical, and truly useful? 

This talk explores vibecoding: a fast, improvisational approach where existing AI platforms are remixed to sketch, test, and prototype new creative experiences. Through a series of live and recent projects, from a rapid synthetic survey generator and a school-safe learning companion to real-time poetry-to-image performance tools and an interactive WebAR business card; Chris demonstrates how ideas can shift from concept to working prototype in hours, not months. 

Attendees will leave with a clearer picture of both the potential and pitfalls of creative AI, and how vibecoding can help us explore the future with speed, imagination, and responsibility. 

Generative AI is being built into platforms like Databricks, Snowflake, Microsoft and Tableau, promising easier data access and faster insights. Survey data is a tougher challenge, with metadata, weighting and open text to handle. This talk looks at what works, what does not, and what it means for anyone trying to use AI responsibly with surveys. 

Understanding how to capitalise on the potential for Generative Artificial Intelligence to transform survey practice in ways that enhance data quality while reducing costs is a key priority. One area of application garnering particular interest is the use of Large Language Models (LLMs) in questionnaire design, evaluation, and testing (QDET) procedures. This presentation will present results of research investigating the effectiveness of LLMs at applying the ‘Question Appraisal System’ (Willis and Lessler, 1999), which consists of a structured framework for identifying question features likely to give rise to response challenges. The research consisted of three components: 1) a comparison of six alternative LLMs to assess within- and between-model variation in applying the QAS with different prompts, and benchmarking model performance against expert and novice human assessments of the same questions; 2) the development and testing of a pipeline for conducting QAS evaluations at scale using GPT-4 via the OpenAI API, and validating outputs for n= 130 draft survey questions with expert and novice evaluations of the same questions. 3) assessing improvement in model’s performance of the QAS task with a revised promp addressing usability of the original QAS instrument. The results demonstrate the effectiveness of both prompts at yielding human-like QAS evaluations from LLMs, with agreement rates between LLMs and humans equivalent to those observed when comparing novice and expert humans. The revised ‘usercentric’ prompt was successful in generating more detailed and critical QAS evaluations, and highlights key lessons for effective prompt engineering. 

When everyone has access to the same models and capabilities and AI has stops being a differentiator, what actually sets you apart?

This session argues that generative AI doesn’t eliminate the need for research expertise. It demands better researchers. Through examples of where AI has failed spectacularly in research and consulting contexts, the talk examines what happens when we outsource our thinking to machines.

The question isn’t how much time we can save. It’s what we choose to do with the time we have. In an efficiency-obsessed world, how do you make the case for the slower, harder work that actually differentiates your research from everyone else’s?

See how the new world of agentic AI will turbocharge the power of research. The future will be one where the consumer insights industry is leading our clients into the sunny uplands of our AI world. Because we know data, research, business and AI we can be the ones bringing new AI technology into our organisation and help them create competitive advantage. 

This presentation explores a novel approach to survey testing: using AI personas, powered by OpenAI’s agent mode (released 17 July 2025), to act as synthetic respondents that can test surveys before launch. We will demonstrate how AI agents can be assigned a persona — covering demographics, attitudes, knowledge levels, and then navigate a live survey just as a real participant might. A key extension of this method is instructing the AI agent to compare the live survey against its original questionnaire document or design brief. This enables the agent to flag discrepancies—such as missing questions, altered wording, methodological problems —before the survey reaches the field. It’s an additional safeguard against errors introduced during programming or translation. This talk will appeal to researchers, operations teams, and technologists looking to improve survey quality, reduce launch risk, and explore the emerging role of agentic AI systems in research operations. 

Generative AI has captured enormous attention in survey research, but its application has been uneven. While qualitative tasks such as coding open-ends have seen dramatic gains, quantitative analysis has remained more resistant. The reason is straightforward: quantitative research relies on mathematical precision, reproducibility, and statistical rigour, which are qualities that large language models are notoriously poor at delivering. When business-critical decisions depend on numbers, “approximately correct” is not good enough. This paper addresses the central question: can generative AI be trusted with numbers? Drawing on practical implementations in survey research, we argue that it can, but only if designed with determinism and transparency at its core. The paper contrasts conventional LLM approaches, which generate text probabilistically, with a hybrid architecture that embeds validated statistical methods within an autonomous AI system. Rather than producing answers that “sound right,” such systems produce findings that can be independently verified, replicated, and trusted. We will review the methodological principles underpinning this approach, including automated crosstabulations, significance testing, regressions, and anomaly detection. Special attention will be given to how the system prioritises meaningful findings, avoiding the pitfalls of surface-level dashboards while reducing analysis time from weeks to minutes. Case studies from organisations such as De Beers, Bose, and leading agencies illustrate how rigorous automation changes practice: from improving data quality, to accelerating delivery, to supporting exploratory analysis. Beyond technical detail, the paper reflects on broader implications for the research industry. If reproducible quant insights can be produced at speed and scale, what does this mean for the role of analysts? How can insight teams ensure that automation enhances, rather than erodes, their professional standards? And how might we reconcile the flexibility of generative AI with the discipline of statistical science? 

In addressing these questions, the paper contributes to an urgent debate. Generative AI promises efficiency and accessibility, but only by ensuring rigour and reproducibility can it be trusted as a foundation for quantitative research. 

This talk will deep dive on two internal agentic tools we have made available to all STRAT7 staff over the past few months.  The first of these is the STRAT7 ‘Deep Researcher’ tool and the second is ‘Crowd Tracks’. Both of these tools make use of agent-based architectures, which work with researchers to create bespoke reports, all using natural language prompts. These tools have been designed to maximise user trust, confidence and security vs. freely available deep researcher tools.  

This talk will explain how we’ve built these products from scratch to drive user adoption and reduce user friction. It will also touch on our STRAT7 Confidence score framework, which we are actively developing into all of our AI Tools.