The say/feel gap in research; how big is the problem actually, and what do you do about it?
There's a well-documented phenomenon in consumer and user research that most practitioners know intuitively but rarely name directly: people don't report their experience accurately.
Not because they're dishonest. Because self-reporting is hard. In the moment of an interview, participants are performing a version of themselves. They round off hesitation. They describe their behaviour more charitably than it actually was. They say "yes, I'd probably use this" when what they felt was closer to "maybe, under the right circumstances, if the price were different." The social pressure of being in a conversation, even with an AI, shapes what gets said.
This is what we call the Say-Do Gap. The distance between what someone tells you and what they actually feel or do.
It shows up everywhere. In concept tests where participants say they love a product they'd never buy. In usability sessions where someone says "this makes sense" while visibly struggling. In brand perception studies where stated attitudes don't match purchasing behaviour.
The traditional workarounds are imperfect. Projective techniques add noise. Implicit association tests are hard to run at scale. Follow-up probing helps but depends entirely on the moderator catching the right moments. And even the best human moderator misses things, they can't track vocal confidence, facial micro-expressions, and conversation content simultaneously across 40 interviews.
At Decode, the reason we built an emotion layer into Mira wasn't to replace researcher judgement. It was specifically to give researchers a way to see the moments where what someone said and what they felt came apart, and to surface those moments as evidence, not as a resolved verdict.
I'm curious how others in this room handle it.
Do you design studies specifically to work around the say/do gap?
Do you treat self-reported data with a standard discount?
Or have you found methods that get closer to what people actually feel?
Replies
@productrambler This is a very real problem we see in moderated tests. From being a target of attention or feeling that users are themselves on trial, it's difficult to gauge the response.
Some of the methods we tried over the years.
- Look at nonverbal cues: how do they react, what do they do with their hands, etc.?
- On purpose we stage some easy tasks at the start of the session, to build trust and confidence
- We mix the questions, change order, run different sets etc.
Traditional user tests work well, but I think they don't reflect real users. A/B tests on a fraction of the real audience for specific journeys or cohorts lead to much better results.