By David Schneer
15-Minute Read
Somewhere along the way, an idea calcified into doctrine: if you want to know whether an ad works, you gather eight to ten strangers in a room with a two-way mirror, dim the lights, play the spot, and ask them what they think. But is this really the right way to evaluate an ad?
A Short History (Or: Why the Inventor Hated His Invention)
The “focused interview” was developed in the early 1940s by Robert K. Merton and Paul Lazarsfeld at Columbia University’s Bureau of Applied Social Research, originally to study the wartime propaganda films the U.S. War Department was producing for soldiers and civilians. Merton and Kendall (1946) first formalized the method in print, and Merton, Fiske, and Kendall (1956) elaborated it into a full manual. The protocol was rigorous. Respondents experienced the stimulus individually, then sat for a structured interview about specific moments of reaction. Group sessions existed, but they were a supplement to the individual data — never a replacement for it.
Eventually, Madison Avenue got its hands on the methodology, dropped the rigor, kept the conviviality, added a buffet complete with M&Ms, and called it a focus group.
By 1987, Merton himself was writing a corrective in Public Opinion Quarterly, warning that what the industry was practicing under the name “focus group” had drifted so far from his original method that the two should not be confused (Merton, 1987). He was, by all accounts, irritated. The methodology he co-invented for careful study of reaction had become, in the wrong hands, a virtual groupthink party with snacks. We have been ignoring his complaint for nearly forty years.
A Famous Experiment You May Not Remember
In 1951, around the same time Merton was refining his focused interview, Solomon Asch ran what is now one of the most replicated findings in social psychology. He showed groups of college students a card with a single line on it, then a second card with three lines of obviously different lengths and asked them to identify which of the three matched the first. The right answer was always plain. The catch: every other person in the room was a confederate, instructed to give the wrong answer in unison. The lone real participant gave a wrong answer to conform with the group roughly a third of the time — even when the right answer was sitting in front of his face (Asch, 1951; Asch, 1955).
These were not stupid people. These were intelligent young adults publicly contradicting their own eyes because seven other people in the room had agreed on something different.
Now picture that dynamic in a focus group. The first respondent says, “I thought the ad was kind of condescending.” Now the cat is out of the bag. It does not matter whether she is right. It does not matter whether anyone else agrees. From that moment on, the rest of the room is not reacting to your ad. They are reacting to her opinion of your ad. You just paid forty thousand dollars to run Solomon Asch’s experiment with crappy controls.
The Argument
Let’s take a Point-Counterpoint approach to the most common defenses of the focus group as the venue for ad testing.
Point: Focus Groups Surface Insights Individuals Miss
Proponents argue that the magic of a group is the chemistry between respondents — that one person’s comment unlocks another’s and outpours a richness no individual interview produces.
Counterpoint: True, but for the Wrong Job
Group dynamics absolutely surface things. They surface social proof, conformity, and sometimes, status maneuvering—the particular comments a respondent thinks will make her sound thoughtful in front of strangers. These are real human phenomena, and qualitative researchers should take heed. But none of them is what an ad has to overcome. An ad must land on one person, alone, in under seconds, while the person is preoccupied. If you want to know whether your ad survives that encounter, you must recreate that encounter—not host a salon for eight.
Point: The Group Helps People Open Up
This argument goes that strangers feed off each other, that hearing one person be honest gives the others permission, and that the group is a kind of accelerant for candor.
Counterpoint: The Group Is Also an Accelerant for Performance
Sometimes it accelerates candor. Other times, it accelerates performance — respondents auditioning for the moderator, the client behind the glass, or each other. They reach for the witty line, the morally safe answer, the answer that sounds sophisticated. Others follow and agree. Janis (1972) named the phenomenon “groupthink”, and Sunstein and Hastie (2015) demonstrated that group deliberation can head toward the direction that the room leans first. Ad reactions are exactly the kind of thing groups distort fastest, because the social cost of admitting “that one made me cry” or “I didn’t get it” is higher than the social cost of just going along.
Point: We Need to See Real Emotional Reactions
Some clients want to see laughter, confusion, the eyeroll, the tear—the entertainment of it all. The group, defenders say, is the only place those moments occur.
Counterpoint: They Happen More Honestly in One-on-Ones
Heath (2012) and Wood (2019) document at length that advertising response is largely pre-verbal — a System 1 process in Kahneman’s (2011) framework — happening in the first second or two of exposure, before the prefrontal cortex assembles a sentence about it. Barden (2013) makes the same case for marketing decisions broadly. You see those reactions cleanest when nothing else is competing for the respondent’s attention: not seven other faces, not a moderator’s smile, not the imagined audience behind the mirror.
Point: Respondents Will Tell Us Why They Reacted That Way
The whole point of qualitative research is the why, defenders argue. And the group, they say, is where the why comes out.
Counterpoint: Asking Why Produces Answers That are Seen Before They Are Heard.
I am, for the record, a defender of asking why (Schneer, 2026). Used well, it is the most natural probe we have. But there is a difference between asking why someone holds a considered opinion and asking why someone reacted, in three-quarters of a second, to a piece of pre-verbal stimulus. The first question has an answer. The second one does not — at least not one the respondent has access to. Zaltman (2003) argues that the great majority of consumer cognition happens below conscious awareness. Wilson (2002) goes further: people are systematically poor narrators of their own mental processes. Asked why an ad made them feel a certain way, respondents will answer. They will sound coherent. The answer will be a confabulation. In a group, it will additionally be tuned to the room.
Point: Clients Need to Watch — and Focus Groups Are More Convenient
This is the most honest defense, even if its proponents rarely claim it. The two-way mirror, the conference room, the buffet, the client smirking at the loud talker’s comments — that is the product. Clients want to feel close to the consumer, and the focus group is a venue that delivers the feeling; that is, if the learning is compressed into two hours, runs in the evening, and does not interfere with the rest of their schedule. Sit through eight 60-minute In-Depth Interviews (IDI) spread over two days? As they say in New York, fuhgeddaboudit.
Yes, this has happened to me and more than once, with clients who would not budge off a group format the procurement department had pre-approved. We compensated by having respondents fill in self-administered questionnaires before they spoke — capturing the private reaction before the room could contaminate it — but the cost of that workaround was real. The exercise in front-loaded silence poured water on the energy of the group, and some clients found that quite uncomfortable.
Counterpoint: Silent Opinions
There is a version of the group format that can work, even for evaluative jobs, and it is worth naming. In a clinic-style group, respondents enter the room, see the stimulus, and immediately record their reactions privately — on a tablet, on a paper questionnaire, on whatever device captures the answer before another voice in the room can shape it. Discussion comes way after. The group then becomes a place to interpret reactions that have already been committed in private, not a place to manufacture them in public.
This worked well, per Asch (1955), who observed that conformity ebbs when participants can record their answer privately before group exposure. It also binds them to their written answers. The individual data tell you what the respondent actually thought; the discussion that follows tells you how she will defend and rationalize her view among peers. Two different things, both useful.
The catch is that some clients will not budge. They want the room hot and the discussion live, not eight people silently typing into iPads for the first ten minutes. The silent-vote front end is a workaround, not a solution. The right answer is still to use the right tool for the question. Silent Opinions is the honorable second-best.
Counterpoint: Comfort Is Everything (and Groups Are Not Comfortable)
I noted in Backbone that respondents need to feel safe before any probe lands well, and that comfort and perceived safety drive almost everything in qualitative work (Schneer, 2024). The focus group fails this test on its own terms. A respondent in a 1:1 with a warm moderator is, after a few minutes, more candid than she ever will be in a room of strangers. The group is not a place of comfort. It is a place of discussion, monitored by a stranger, observed through one-way glass, watched by people whose names are unknown. The “comfort” of the group is the comfort of conformity. That is exactly what you do not want when you are trying to understand whether a thirty-second commercial is moving.
When the Group Is the Right Tool (and When It Isn’t)
The choice is not whether focus groups are good or bad. They are tools. The question is whether the tool fits the job.
Use the group when the unit of analysis is the conversation itself — how consumers talk about a category, what language they use among peers, where they agree and disagree, which features of an idea they elaborate on when they hear someone else’s reaction. Groups earn their keep for category exploration, hypothesis generation, stress-testing positioning territories, and workshop-style co-creation. They are right when peer dynamics are the phenomenon you came to study: B2B influence networks, household decision dynamics, social consumption.
Use individual interviews — and, for advertising specifically, individual exposures with structured probing afterward — when the unit of analysis is the person. IDIs are the right tool for ad evaluation, concept testing, pricing research, sensitive topics (money, health, identity, parenting), B2B work with senior executives (where peers will not be candid in front of one another or get into a pissing match), and anything where social desirability bias would distort the answer. If your decision rests on the individual reaction, the group will get in the way of seeing it.
A short version, because clients usually want one. Group: “How do people in this category talk to each other?” IDI: “What does this person actually think?” Confuse those two questions, and the methodology will not save you.
My Take
Focus groups are not useless. They are excellent for the right job. The point is to know which job that is.
Ad evaluation is not it. The format optimizes for the wrong things. It contaminates the very reaction you came to measure. It rewards the loudest voice, punishes the softest, and produces a deliverable so rich with quotes that it feels like data when it is mostly transcript.
Test your ad the way it will actually be encountered: alone, distracted, briefly. Pair the depth of a 1:1 with implicit response measurement to capture what the respondent cannot tell you (Nelson-Field, 2020). Binet and Field (2013) have shown that emotional response is the strongest predictor of in-market effectiveness. If you measure it in the wrong room, you will optimize against the wrong signal. This is why nonverbal intelligence training is critical for qualitative researchers.
The focus group will tell you many useful things. Whether your ad works is not one of them.
If your business is making creative decisions based on what a room full of strangers said in the dark, it might be time to test those assumptions. The team at Merrill Research builds qualitative designs that match the question, including ad evaluation that respects how ads are actually consumed. https://merrillresearch.com/contact/
References
Asch, Solomon E. 1951. “Effects of Group Pressure upon the Modification and Distortion of Judgments.” In Groups, Leadership, and Men, edited by Harold Guetzkow, 177–190. Pittsburgh: Carnegie Press.
Asch, Solomon E. 1955. “Opinions and Social Pressure.” Scientific American 193, no. 5: 31–35.
Barden, Phil. 2013. Decoded: The Science Behind Why We Buy. Chichester: Wiley.
Binet, Les, and Peter Field. 2013. The Long and the Short of It: Balancing Short and Long-Term Marketing Strategies. London: Institute of Practitioners in Advertising.
Heath, Robert. 2012. Seducing the Subconscious: The Psychology of Emotional Influence in Advertising. Chichester: Wiley-Blackwell.
Janis, Irving L. 1972. Victims of Groupthink: A Psychological Study of Foreign-Policy Decisions and Fiascoes. Boston: Houghton Mifflin.
Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar, Straus and Giroux.
Merton, Robert K. 1987. “The Focussed Interview and Focus Groups: Continuities and Discontinuities.” Public Opinion Quarterly 51, no. 4: 550–566.
Merton, Robert K., and Patricia L. Kendall. 1946. “The Focused Interview.” American Journal of Sociology 51, no. 6: 541–557.
Merton, Robert K., Marjorie Fiske, and Patricia L. Kendall. 1956. The Focused Interview: A Manual of Problems and Procedures. Glencoe, IL: Free Press.
Nelson-Field, Karen. 2020. The Attention Economy and How Media Works: Simple Truths for Marketers. Singapore: Palgrave Macmillan.
Schneer, David M. 2024. Backbone: Surviving the Road Less Quantified — Toward a Deeper Understanding of Qualitative Research.
Schneer, David M. 2026. “To Why or Not to Why: That Is the Question.” Merrill Research blog.
Sunstein, Cass R., and Reid Hastie. 2015. Wiser: Getting Beyond Groupthink to Make Groups Smarter. Boston: Harvard Business Review Press.
Wilson, Timothy D. 2002. Strangers to Ourselves: Discovering the Adaptive Unconscious. Cambridge, MA: Belknap Press of Harvard University Press.
Wood, Orlando. 2019. Lemon: How the Advertising Brain Turned Sour. London: Institute of Practitioners in Advertising.
Zaltman, Gerald. 2003. How Customers Think: Essential Insights into the Mind of the Market. Boston: Harvard Business School Press.