When AI keeps giving the same answer: what synthetic participants miss

604 people. 120 artificially generated participants. Almost the same headline finding. At first glance, that might look like a breakthrough: why go to the trouble of recruiting people if AI can deliver the same answer?

Yet the decisive difference was not in the most common answer. It was in the answers hardly anyone gave. That is where the real story of this study begins, and with it an important debate about the future of qualitative research.

What the new study actually shows

On 16 October 2026, Nielsen Norman Group (NN/g) published a comparison of answers from 604 real technology and design professionals with those of 120 synthetic people simulated by an AI platform. Respondents were asked what they would call the discipline that studies people’s needs and experiences and turns them into recommendations for products and services. [1]

In both datasets, ‘UX’ or ‘user experience’ remained the dominant term. UX-related terms appeared even more often in the synthetic answers: 84 per cent, compared with around 70 per cent among the human respondents. But while the real people used more than 50 different terms, the simulated answers clustered around roughly twelve. Rare and sometimes surprising terms were largely missing. [1]

The AI got the direction right, but not the range.

That is more than a linguistic detail. In exploratory research, we are interested not only in majorities but also in exceptions, divergent interpretations and new ways of thinking. These can point to unrecognised needs, contexts of use or emerging market segments.

The problem of apparent agreement

In AI research, the narrowing towards a few dominant response patterns is often described as ‘mode collapse’. For synthetic participants, this means that simulated answers can convincingly reproduce a common pattern without adequately reflecting the diversity of real people. [1]

Imagine a study on a new banking app. Most participants want a clear, simple interface. A synthetic sample could reproduce that consensus very plausibly. But it might miss the perspective of someone with unusual security concerns, a rare usage situation or contradictory habits. Cases like these are often what alert a product team to critical design questions.

This banking example is hypothetical. It illustrates the methodological consequence, not a result observed in the NN/g study.

Why this study does not prove everything

Scientific rigour also means naming the limits of the comparison. The study examined a single open question about what to call a professional field, not a full qualitative interview study. The human and synthetic groups were also composed differently: 67 per cent of the real respondents were UX professionals, compared with only 20 per cent of the synthetic ones. [1]

In addition, the simulated people answered within AI-moderated interviews, while the humans answered a survey question, and the AI platform partly rephrased the question. These differences may also have influenced how people responded. The study therefore does not show that every synthetic approach is always worse. It does, however, demonstrate a concrete risk that needs to be checked whenever such approaches are used.

Another research project from 2026 compared human and AI-generated viewpoints using a Q-methodology analysis. Its authors also found clear problems in representing the diversity of human perspectives. Whether these findings transfer to other models, populations and tasks needs to be assessed case by case. [2]

Where synthetic participants can be useful

None of this means synthetic participants are useless. In early project phases, they can help surface assumptions, formulate hypotheses, test discussion guides or structure familiar lines of argument. Such uses, however, are something different from the empirical study of real users. [3]

One distinction is crucial: an AI can describe a plausible experience. That does not mean it has had that experience. If you want to know how people actually behave, which trade-offs they make and why they depart from the supposed norm, you need data from real people.

What this means for UX research and market research

For clients, there are three practical tests. First, the purpose of a study should be clear: is it about generating ideas, or about reliable statements on real target groups? Second, it must be disclosed which insights come from human research and which were merely simulated. Third, rare perspectives and relevant subgroups in particular should be studied deliberately with real participants.

This applies especially to complex B2B decisions, sensitive healthcare topics and specialist UX questions. Here, the quality of insight often depends on concrete life and work experience that a statistically plausible profile cannot replace.

That is why, for us at Insight Fox, participant recruitment is not an interchangeable step before the ‘real’ research. It is a methodological part of the study: who is interviewed, what experience these people genuinely bring and how their suitability was checked all help determine which insights can emerge at all.

Not every surprising answer is right. But every perspective smoothed away too early can be a missed insight.

Conclusion: real people remain the benchmark

The new NN/g study is not an argument against using AI in market research. It is an argument against confusing simulation with observation. AI can help prepare, structure and streamline research. Whether synthetic answers validly represent a particular target group, however, has to be demonstrated independently and for the specific task.

Qualitative research is not only about confirming what we expect. Its value often lies in uncovering contradictions, edge cases and previously unknown perspectives. So the decisive question is not: ‘Can AI give a plausible answer?’ It is: ‘Which human experiences are we missing if this is the only answer we hear?’

About Insight Fox

Insight Fox specialises in recruiting suitable participants for qualitative market research and UX research. The brand is backed by the team at WMM GmbH, which has worked in qualitative market research since 2004. Our standard: careful screening, personal verification and traceable recruitment quality, so that research rests on real experience.

Editorial note: the studies cited examine different questions and methods. The statements in this article are limited accordingly; hypothetical examples are labelled as such.