David Schneer

Can Market Research Be a Guide Through Difficult Geopolitical Times?

November 29, 2023 by David Schneer

Can Market Research Be a Guide Through Difficult Geopolitical Times?

By Angela Burtch, Vice President, Merrill Research

4-Minute Read

It’s well known that when some companies experience economic downturns they trim their sales force, and eliminate PR, advertising, and eventually market research.

But other companies continue to spend on advertising and market research with an eye toward maintaining a presence and using market intelligence to ride the eventual hockey stick up as markets recover. They use research to make sense out of seeming chaos.

But what about geopolitical flare-ups?

The current global landscape is marked by geopolitical tension and conflict in regions such as, but certainly not limited to, Ukraine, Russia, Israel, and Gaza, just to name a few.

How can market research become a critical asset? Here are a few ways in which market research has been used in this regard:

Navigating Uncertainty

Geopolitical conflicts can lead to significant economic shifts. Market research helps businesses understand these changes and navigate the uncertainty. It provides insights into how conflicts are affecting consumer behavior, supply chains, and market dynamics.

Risk Management

Market research can identify potential risks and challenges that businesses may face due to geopolitical unrest. This may include changes in regulatory environments or preferred suppliers. In recent research we conducted in the technology components space we found that companies have a changing and growing list of countries that procurement is not allowed to do business with.

Strategic Decision Making

Informed decision-making is crucial during these times. Market research provides data that helps businesses make strategic decisions, such as entering new markets, developing new products, or pivoting business strategies during uncertain times.

Staying Competitive

Understanding the market landscape during geopolitical unrest can give businesses a competitive edge. It can uncover opportunities that may not be apparent amidst the chaos. In a recent project we completed for a consumer software company we learned that a competitor based in a country experiencing conflict saw dips in brand usage and trust.

Recommendations for Market Research

Here are some examples of studies conducted that may be of use to you and your organization:

Customer Research: Understanding how your customers’ behavior is changing due to the geopolitical climate can help you adapt strategies accordingly. We’ve done hundreds of studies among consumers, businesses, association members, etc., to monitor these changes—especially during the pandemic.
Supply Chain Analysis: Researching potential and current supply chain disruptions can help businesses plan for contingencies. Maybe this is the time to conduct some IDIs (in-depth interviews) among channel partners?
Competitor Analysis: Understanding how competitors are responding to the geopolitical situation can provide valuable insights.

Conducting market research during difficult geopolitical times is not just beneficial—it’s essential. It equips businesses with the knowledge and insights they need to navigate through uncertainty and remain resilient.

Looking for a trusted partner to help you navigate through uncertainty? Let’s talk.

How Many Interviews Do We Need to Do?

November 15, 2023 by David Schneer

By David M. Schneer, Ph.D./CEO

4-Minute Read

We’ve been asked that question so many times that I was compelled to write about it.

The real answer is, “it depends.”

But “it depends” to our clients is as useful as a sliding glass door in a submarine. We must give them an answer.

So, here are the variables we consider when providing a research approach plus insights gleaned from a dynamic discussion paper written by The National Centre for Research Methods. The report was written to help guide graduate students on the number of qualitative interviews they would need to do for a dissertation.

This report included the opinions of 19 academic research experts. A good number of them also concluded that “it depends.” But they also raised some excellent points to consider.

For Academia

The paper indicated that the number of qualitative interviews in academia hovers between 20 and 60 interviews with 30 being the average. Still, others indicated 20 for an M.A. thesis and 50 for a Ph.D. dissertation.

For Market Research

But what about applied consumer or B2B qualitative research? Here are some variables to consider when determining the actual number:

Research objectives. See below.
The complexity of the topic. For example, if you’re trying to understand how the process of photosynthesis works in trees, you don’t need to talk to a hundred trees. You can talk to a handful of trees and understand this process, even though the process can differ from tree to tree.
The size of your population/interview pool. I once conducted a focus group among supercomputer users and at the time there may have been a total of 20-30 users of such equipment in the US. But we were able to recruit 6 scientists in Southern California for one focus group.
Theoretical Data Saturation, meaning that new interviews no longer provide substantially different information or insights. [1] That is, if you continue to talk to a certain number of similar people, you are likely to begin to hear the same thing.
Cost.
Timing.
Homogeneity of your sample.

To us, homogeneity is key.

To illustrate, here’s a simple example. Let’s suppose your client wants to market to people named Karen. Yes, we know. Karen’s are extremely difficult, but then this is why the client wants to interview them.

Ideally, you could talk to 20 or 30 Karens, but is that too much? The question then becomes how many Karens do you need to talk to before you start to experience theoretical data saturation?

Here’s what we think, and it is backed up by the research and our experience. If you talk to 12 Karens there is a pretty good chance that Karens 13-20 or even 30 will begin to say the same thing, as we have discussed earlier. So, in this example, 12 Karen’s would suffice and 30 is likely too much, wasting the client’s budget, time, and money.

But now suppose the client wants to also market to Bobs. Now the sample becomes heterogeneous, and tradeoffs need to be made.

If the client has budget for 30, the problem can be easily solved by conducting 24 total interviews, 12 Karens and 12 Bobs.

But what if the client has budget and timing for only 12 interviews?

You can likely get away with six interviews among Karens and six among Bobs, but you may not get to theoretical data saturation or even begin to see patterns develop. But it may be better than no investigation at all.

Often, we will conduct a small pilot interview of three to five interviews and determine if more are warranted based on early results.

But remember, qualitative research is based on depth of information and not statistical reliability or predictability.

Let us help you determine the optimal number of qualitative interviews you need.

AI-Based Fault Tolerant Qualitative Research

May 17, 2023 by David Schneer

By David M. Schneer, Ph.D./CEO

4-Minute Read

Imagine if you could conduct an interview and observe when the respondent’s words do not match their body language.

Now, imagine that, at the end of the interview, you could feed the video into an AI-based software program that would also highlight verbal-physical inconsistencies.

What would that mean?

It would be like the notion of fault-tolerance in the computing sector. Webster defines the term as, “Relating to or being a computer or program with a self-contained backup system that allows continued operation when major components fail.” ^[1] Or, in finance parlance, the term “Belt and Braces” was used in early 19^th Century Britain to describe an ultra-conservative approach to a deal. ^[2]

Of course, the term could apply to any system—even mechanical. But now we think the term can also stand next to qualitative research.

Well, you don’t have to imagine this scenario anymore. At Merrill Research, we have conducted the first fault-tolerant qualitative research study using a highly trained nonverbal intelligence expert (me) backed up by a highly tuned AI-based analytical tool (LightBulb AI).

And so, we put on our belts and suspenders for this study.

And what did we learn?

Of course, given that our research is custom I cannot offer specific details about the project, but this much I can tell you. I observed concrete instances where respondents’ words did not match their facial expressions and the AI-based software was tracking right along with me, second by second. At the end of the interview, we had a complete longitudinal record of the respondent’s facial reactions complete with verbatim transcripts. As such, we were able to pinpoint which questions elicited the most engagement, joy, and surprise, or, on the contrary, contempt, disgust, fear, and anger. Neutral facial expressions were also measured.

This is how we did it.

Respondents in this study agreed to be digitally recorded, whether virtually or in person. We conducted 18, 60-minute in-depth interviews, six via Zoom video and 12 in-person interviews, six in a facility on the East Coast and six in a facility on the West Coast. These respondents were technical professionals. Respondents in this study agreed to be digitally recorded, whether virtually or in person. All video and in-person interviews were conducted by me. Professional videographers were hired to record in 4K video. In both venues, the videographers shot from the observation rooms to mitigate intrusion.

Once digitally recorded, the raw video was fed into an AI-based software program designed to detect the slightest emotion: joy (happiness), surprise, contempt, fear, anger, sadness, disgust and, neutral.

The software also derived an overall engagement measure.

While all these emotions were measured, some (fear, sadness, anger, and disgust) barely registered, if at all. This made sense, given the context of the conversation.

Would the respondents react emotionally like other consumers, or would they live up to their reputation as being linear, rational, and objective?

Spoiler alert. Our research proved undoubtedly that these individuals emoted just like the rest of us, although that did not come through in their verbal comments.

I, as well as the AI-based software, was looking for the seven universal human micro-expressions as defined below.

Emotion	Definition
Joy/Happiness	As measured by a symmetrical smile with crow’s feet engagement. Usually accompanied by surprise.
Surprise	As measured by a quick rise in eyebrows, and eyelids with mouth agape. An indication that one has been caught off guard, either positively or negatively. Often accompanied by Joy/Happiness
Contempt	As measured by an asymmetrical smile. Those showing contempt may either feel negatively toward stimulus, or contempt can be a feeling of superiority.
Disgust	As measured by a “crinkled” nose and mouth agape. Intense dislike.
Anger	As measured by eyebrows together and down with lip compression.
Fear	As measured by a quick but subtle rise in eyebrows and eyelids.
Sadness	As measured by the eyebrows moving up and in with a pout or pressed lips.
Neutral	The absence of the above-mentioned emotions.

Next, we combined these emotions into three main categories: engagement, likability, and dislike.

Engagement comprised focus or interest and was measured on a 100-point scale. Any time engagement dipped below 80%—this was cause for further evaluation.

Next, we combined happiness or surprise on a 100-point likability scale, and the higher the score the better. Any time emotion spiked above 40% it was considered for further review.

We measured dislike as defined by contempt, disgust, fear, and sadness. Here, the lower the score the better, and anytime dislike rose about 40%, it was considered for further evaluation.

And finally, much like me, the AI software was trained to identify contradictions between speech and expression.

This is what the output looked like (again, redacted for privacy):

So, what did we learn?

For the most part, respondents exhibited a very high level of engagement throughout the interview. This made sense, given that the questions we asked were provocative.
We observed a low level of dislikes (negative emotions). Again, this makes sense since our topics, while provocative, were not incendiary or controversial.
On those questions where I noticed inconsistencies, the AI software corroborated my observations. We were able to conclude that indeed some of the answers we were getting were contradictory to what respondents’ emotions were showing.
We matched minute-by-minute, longitudinal verbatim quotations with observed emotional states. This allowed us to pinpoint the cause of the emotion and add quotations with context.
The AI-based software was far more accurate than me. I expected this because not only did I have to interpret the answers, I had to ask the questions. And probe. Not so for the AI software.
The AI software was highly accurate in identifying contradictions.
We did not see any so-called AI “hallucinations”, but we did see some aberrations. We registered false negatives not because of the software mistakes but rather study design. For example, we noticed considerable drops in engagement when respondents looked away from the camera.

We believe that this is the future of qualitative research—highly trained moderators backed up by AI makes for mighty powerful insights with confidence.

Need help with designing AI-based qualitative research? Let Merrill Research guide you through your next research study!

[1] https://www.merriam-webster.com/dictionary/fault-tolerant

[2] https://www.phrases.org.uk/meanings/belt-and-braces.html

How to Lather, Rinse, and Repeat Your Way to Clean Research Results

May 4, 2023 by David Schneer

By Angela Burtch, Vice President, Merrill Research

4-Minute Read

In Part 1 of this series we touched on a few tips on how to ensure the integrity of data when conducting a market research study.

A lot has changed in the past few years and we thought it was time to add more content. More bots, frauds, and other illegitimate means are increasingly common and able to circumvent the safety nets we’ve cast to detect poor data. Everyone is working hard together to mitigate bad survey data. So, with this blog, we’d like to continue to offer up more suggestions that will help all of us to this end.

Lather

Before you even see a respondent in a survey, make sure your sample provider has done all the pre-conditioning to ensure the integrity of the participant. These are just a few of the myriad factors that impact respondent quality. Your respondents need to be squeaky clean. Here are some questions to ask your vendor:

How are they sourcing their sample? Proprietary panel? APIs? if so which ones? Affiliate publishers? River/intercept sampling? Some other way? Make sure you know the source.
If panels are used, what tools are used in the registration process to mitigate fraud? Digital fingerprinting? 3^rd party validation? Double opt-in validation? IP validation?

We could write an entire series of blogs just on this first lather. Go to the sample provider’s website and see what they say they do!

Rinse

Even when the legwork is done up-front by your sample provider to vet your respondents, as researchers we are responsible for ensuring that quality checks continue post-recruitment.

So, it’s time to rinse and get rid of anything that wasn’t taken out in the suds by your sample vendor. How do you do that? Make sure your questionnaire can stand up to fraud.

It used to be that we would sprinkle a “gotcha” question or two into the respondent screening to rinse out the duds. Not anymore. For a typical 10-minute survey we will include all types of trap questions starting in the screening section all the way through the main questionnaire and we use elaborate decision trees to decide what flags or completely terminates a respondent.

Here are a few tips that we highly recommend you incorporate into your next questionnaire:

✔ Red Herring questions. Most researchers know about these. As an example, in a branding study you could include a “fake brand” in an aided awareness list and flag or terminate respondents who select this option. Be careful with this one. Make sure you do your due diligence and make sure that your red herring brand is not too similar to a real-life brand. For example, if the subject matter is awareness of social media sites, a question with an artificial brand “Tweeter” could potentially be selected because of its very close similarity to “Twitter”. Always remember, you don’t want to entrap and confuse respondents, you just want to make sure that you’re rinsing away the unengaged and potentially bad data responses.

✔ Skill testing questions. An actual legal requirement in Canada for sweepstakes. A simple mathematical problem slows down respondents and adds another level of security. Don’t ask respondents to solve a differential calculus problem (unless they are engineers!), but instead, ask a simple problem like (10 x 10) – 5. BTW – that’s 95. Give yourself a pat on the back if you got it right. This may not work with AI created bots, but it can help detect human or click-farm respondents. Stay tuned for my next blog on that..

✔ Attention check questions. These work really well to wash out speeders. In this instance, you tell a respondent to pick a specific response option. For example: “We want to make sure you are taking your time to read all the survey questions, for this question please select ‘Not Very Familiar’ below.” If they didn’t follow the instruction, then what else were they speeding through? Rinse them out of your hair.

✔ Look at the open ends. If you have any open-text questions in your survey, these often provide tell-tale indicators of junk you want to flush out. Nonsense answers (FDSKALFDAKJFSA). Obvious multiple copy and pastes. Or out-of-context answers (e.g., you are asking about car brands, and the responses are about bicycles). Even if you didn’t budget time or expense for coding an open end, including one just for the purpose of monitoring data quality can be a good idea. Example: “In a few words, please type below the kind of movies you like to watch.” Limit the character count to 50 characters so you aren’t using valuable survey taking time, but you can still gauge respondent presence in the question.

✔ Just way too positive. If in a battery of questions, everything is “Extremely Favorable”, “Extremely Important”, etc. Consider you might have an over-endorsing respondent seeking to please the research sponsor and stay in the respondent pool. This is somewhat similar to straight lining, which we addressed in this previous blog.

✔ Speaking of straight lining answers, make it harder to do. Instead of long boring batteries of attributes to rate on a scale, consider mixing up your question structure. How about a drag and drop exercise asking respondents to sort the attributes by Like, Dislike, and Neutral emojis instead of having them cruise down a page clicking numbers.

✔ Look for contradictions. Do the survey answers contradict themselves or do they make sense? Often this means deciding to ditch a respondent on a case-by-case basis. For example, if someone answers that they drink beer once a month and subsequently answers that they buy beer daily, is this valid? Cross-check with some of the other tips listed above.

And Repeat

Yes, repeat. Repeat with every prospective sample partner you are considering. Repeat with every questionnaire you design. Lather, rinse, repeat.

Need help with designing research mousetraps? Let Merrill Research guide you through your next research study!

Law & Order SVU Body Language Fact Check

April 26, 2023 by David Schneer

By David M. Schneer, Ph.D./CEO

3-Minute Read

DUN-DUN!!

Oh man, once I hear that sound[1], I’m like Pavlov’s Dogs running to the couch. It’s Law & Order SVU time, and I’m a rabid fan. The rich and complicated characters are experiential. Like we sometimes do in our research studies, I did a quick word association for some of my favorite characters:

ICE-T? Sarcastic.

Elliot? Skeptical.

Detective Rollins? Captivating.

Lieutenant Benson? Balanced.

Barba? Pugnacious.

Dr. Huang? Uber-Psychiatrist.

I rarely miss an episode. So, I was surprised when the other day, I stumbled upon an episode I had not previously seen, SEASON 6, EPISODE 18–“PURE”, which originally aired on March 8, 2005.

Suddenly the plot turned to body language. I sat up. My Spidey senses were tingling.

This episode featured former SNL comedian, Martin Short, who plays a psychic, Sebastian Ballentine. The psychic claims to know what happened to a missing 18-year-old girl. Stabler smells a rat and thinks Ballantine is dropping clues because he’s the culprit. But Dr. Huang explains to Elliot that the psychic is likely reading facial cues to obtain his information.

Then a picture of the major facial areas from my FACS manual[2] flashed across the screen.

That got me off the couch.

After watching this episode, I decided to fact-check the research and writing staff at Law & Order. Did they get the FACS facts right?

Let’s see.

Below are the verbatim 10 lines (“italicized, bold faced and in quotes”) taken directly from the script of this episode. The scale? True, Mostly True, Mostly False, or False.

“We give away volumes with our faces.” TRUE There are seven universal emotions that have very clear facial signs: anger, sadness, fear, surprise, disgust, contempt, and happiness. Neutral is the lack of any emotion.
“There are 43 distinct facial movements.” MOSTLY TRUE but a cursory survey of the literature regarding the number of facial muscles varies by source. The Interviewer’s Guide sent along with the FACS manual mentions a 1977 study by Young and Decaries that resulted in “42 facial gestalts”,[3] but this refers to two different ways of measuring visible facial movements, the minimal units of behavior method or the list of possible facial gestalts approach. Dr. Ekman was interviewed in a 2003 New York Times article that referenced 43 facial muscles.[4] Interestingly, this article appeared two years before the episode aired, and it is conceivable that the Law & Order researchers/writers tapped into it. An article in the CBC references 43 distinct facial muscles.[5] Smithsonian Magazine mentions 42.[6] The HowStuffWorks website claims 43[7] The Medical University of Southern California hedged its bet and records “over 40 muscles.”[8] And finally, Atlanta Plastic Surgery weighs in with 43.”[9] So, why the discrepancies? One has to do with the process of quantifying facial muscles. Explains Duquesne’s Dr. Anne Burrows, an expert in the subject area, “The problem with quantifying facial musculature is that they’re not like other muscles. They’re fairly flat, difficult to separate from surrounding connective tissue, and they all attach to one another. They are very unlike muscles of the limbs, for example.”[10] The other issue? The number of facial muscles in humans can vary (cf. Waller et al., 2008). So, what is the actual number? I’m going with 43, and the Law & Order staff did their homework on a complicated subject.
“A psychologist named Paul Ekman catalogued 3,000 possible combinations…making up the entire spectrum of human emotion.” MOSTLY TRUE. Technically, he identified 10,000 facial muscle movements but only 3,000 were related to emotion.
“Why [was FACS invented]? To see if someone’s lying.” MOSTLY TRUE. Yes, FACS was created to help with deception detection, but the rubric is also used in the movie industry by computer animators to create characters with real emotions. Law enforcement also uses it.
“FACS breaks facial movement down into action units.” TRUE. These are alphanumeric designations such as AU-1, AU-4, etc.
“AU-1 is raising the frontalis par medialis. It’s the inner eyebrow. It’s a sign of distress.” TRUE The muscle behind the Inner Brow Raiser is the frontalis (pars medialis), which can be visible with the emotions of sadness, surprise, and fear.
“FACS teaches you how to pick up on fleeting micro-expressions that most people don’t even see.” MOSTLY TRUE FACS also helps animators make characters come to life.
“Turns you into a human lie detector.” MOSTLY FALSE Multitudes of studies collected by numerous behavioral scientists over the years have demonstrated conclusively that humans are poor lie detectors. No one single facial action unit is a reliable indicator of deceit. However, the astute observer will look for a cluster of micro-expressions surrounding a stimulus, a possible indicator of deception.
“It’s being taught at the FBI and CIA.” TRUE FACS has been used by local police, the military, TSA, CIA, and the FBI.
“The CD-ROM is sold over the Internet.” MOSTLY TRUE. Back when this episode aired it was available via CD-ROM. Today, you can obtain the FACS manual online (www.paulekman.com)

So, how did Law and Order SVU do? I’d give them an A. They took a complicated subject and boiled it down to its essence for a quick and accurate portrayal of FACS. It is good to watch TV get it right, at least from this perspective and confirms my training on the subject.

We teach micro-expression detection and look for them in our research studies. We can tell who is engaged or comfortable and who isn’t. For those who are not engaged or uncomfortable, we can intensify our efforts to draw them out with stimuli such as refreshments, new content, or a change in topic.

Contact us today to see how we can help you or your organization become proficient at finding out what people are really thinking when they communicate with you.

Most Communication is Nonverbal. Are You Fluent?

REFERENCES

[1] https://www.today.com/popculture/tv/law-order-dun-dun-sound-story-famous-noise-rcna17526

[2] FACS Investigator’s Guide, 1978. Paul Ekman, Wallace V. Friesen, Joseph C. Hager, Salt Lake City, UT p. 3.

[3] Young,G., & Decarie, .T G. An ethology-based catalogue of facial/vocalbehaviorsinfancy. Educational Testing Service, Princeton, N.)., 1974.

[4] https://www.nytimes.com/2003/08/05/health/conversation-with-paul-ekman-43-facial-muscles-that-reveal-even-most-fleeting.html

[5] https://www.cbc.ca/natureofthings/features/the-seven-universal-emotions-we-wear-on-our-face

[6] https://www.smithsonianmag.com/smart-news/human-faces-might-only-express-four-basic-emotions-180949598/

[7] https://science.howstuffworks.com/life/inside-the-mind/emotions/muscles-smile.htm

[8] https://muschealth.org/medical-services/ent/fprs/facial-paralysis

[9] https://www.atlplastic.com/blog/2018/07/11/how-facial-muscles-contribute-to-facial-wrinkles/

[10] https://www.sciencedaily.com/releases/2008/06/080616205044.htm

Practical Applications for Nonverbal Intelligence and the Indications of Facial Half Framing.

April 12, 2023 by David Schneer

By David M. Schneer, Ph.D./CEO

3-Minute Read

I’m often asked how you can tell when someone is engaged in the conversation. Certainly, the head tilt is one. And a smile is another. But what if you’re talking with someone and suddenly, they frame the side of their face with their thumb and forefinger without any support? What does that mean?

Well, that’s actually a good sign. Facial half-framing, as I unofficially call it, is a sign of engagement, and positive sentiment.

This is what it looks like.

However, if the person rests their head in the web of their thumb and forefinger, that is a different story. That can mean tiredness, boredom, or disagreement.

We teach these indications and look for them in our research studies. We can tell who is engaged and who isn’t with body language signs like the facial half-frame. For those who are not engaged, we can intensify our efforts to draw them out with stimuli such as refreshments, new content, or a change in topic.

Contact us today to see how we can help you or your organization become proficient at finding out what people are really thinking when they communicate with you.

Most Communication is Nonverbal. Are You Fluent?