Product discovery with AI: synthesis that keeps the quote attached
I’m a compulsive listener and an opportunist among the constraints, and research synthesis is where both habits get tested. For most of my career, the slowest part of discovery wasn’t talking to people. It was the days afterward spent turning notes into something a team could act on. AI has collapsed that time. It can cluster a week of interviews, support threads and session notes into themes in minutes.
It can also invent confidence just as fast. Themes that sound right, consensus that wasn’t there, and quotes nobody actually said. As a Head of Design, I care less about how fast synthesis happens than whether the team can trust it. The rule that makes AI synthesis trustworthy is simple: every theme keeps a verbatim quote attached, traceable to a real person and session.
Where AI genuinely helps
I use AI in discovery every week, mostly for the tedious parts:
- Clustering. Grouping dozens of notes into candidate themes, which a person then names and judges.
- Deduping. Spotting that six differently worded complaints are the same problem.
- First-pass summaries. A rough read of what came in this week, so I know where to look first.
- Cross-referencing. Checking whether a theme from interviews also shows up in support tickets or search queries.
That’s real time saved. It means more of the week goes to talking with people and less to sorting sticky notes.
Where it quietly hurts
The failure modes are subtle, which makes them dangerous:
- Smoothing away outliers. Clustering rewards the common case. The one user who described a problem nobody else noticed gets averaged out, and sometimes that user was early, not wrong.
- False consensus. A summary that says "users want X" may rest on two loud sessions out of twenty. The confident tone hides the thin evidence.
- Invented quotes. Ask a model for "representative quotes" and it may produce plausible sentences that no participant said. Those end up in a deck, then a roadmap.
- Flattened context. A complaint from a power user and the same complaint from a first-time user can mean opposite things. Summaries lose who said it.
None of these are reasons to avoid AI in research. They’re reasons to change how you check its output.
The "quote attached" rule
The rule I run, and wrote into How I research, is that no theme survives without at least one verbatim quote and a pointer to its source. Not a paraphrase. The actual words, with who said it and where.
That one constraint does a lot:
- It makes invented quotes easy to catch, since every quote must trace back to a transcript.
- It keeps the human texture in the work. A real sentence from a frustrated customer moves a room more than a bullet point.
- It exposes thin themes. If you can only find one quote for a "major" theme, it isn’t major yet.
Auditing a synthesis in fifteen minutes
Before an AI-assisted synthesis goes to the team, I run a quick audit:
- Pick three themes at random. Trace each quote back to the source. If any don’t exist, the whole synthesis gets redone with stricter instructions.
- Count the sources per theme. A theme supported by one or two sessions is labeled as a signal, not a finding.
- Look for what’s missing. Read a few raw notes the clustering didn’t use. Did it drop an outlier worth keeping?
- Check who’s speaking. Are the quotes from the segment the decision is about?
- Rewrite the headline. Turn each theme into an opportunity with more than one possible answer. If only one solution fits the wording, it’s a solution in disguise.
That last step is where synthesis hands off to framing: outcomes, bets and the evidence that would change them, which I cover in Product thinking.
Match the method to the risk
AI makes synthesis cheap, which tempts teams to run the same method for everything. Don’t. The right research method depends on what you’re unsure about. A usability question needs people trying a prototype. A question about which of two directions performs better might need an experiment. A strategy question might need ten deep conversations. I wrote a whole piece on planning what to test next, so I won’t repeat it here. The AI-era addition is just this: faster synthesis doesn’t make the wrong method right.
What years of prototype nights taught me
For several years I ran UX Speed Dating, a monthly night I took over from a former Atlassian designer and kept going. Designers brought prototypes, from paper to high fidelity to board games, and tested them on as many people as possible in timed rotations. Its playful sibling, UI/UF, was about ideating fashionable new app interfaces. You can read about both in the case study.
Those nights taught me things AI synthesis still can’t replace:
- Watching beats asking. The most useful observations came from what testers did, not what they said they’d do.
- Fresh eyes every round. New people catch what the last five missed.
- Synthesize before the room empties. Insights fade fast. The best findings were written down the same night.
Today I’d add AI to that last step, clustering notes before people leave, with every theme carrying a tester’s actual words. The speed is new. The discipline isn’t.
A one-page synthesis template
The template I use for any AI-assisted synthesis:
| Section | What goes in it |
|---|---|
| Question | What decision this research informs |
| Sources | Sessions, tickets, data used, with counts |
| Themes | Name, number of sources, one verbatim quote each, source pointer |
| Outliers | Notes that didn’t cluster but might matter |
| Opportunities | Each theme rewritten with more than one possible answer |
| Riskiest assumption | The thing the next prototype or test must answer |
| Confidence | Signal or finding, and why |
One page. Quotes attached. If it can’t fit, it isn’t synthesized yet.
AI made research synthesis fast. Keeping the quote attached keeps it honest.
Related First designer to Head of Design: owning product, brand and the team end to endThe Spreadsheet That Tells You What to Test NextPapercuts, Tech Debt and Stalled Features: How Design Teams Ratchet Quality Forward