How to Run Concept Tests That Don’t Turn into Opinion Polls

Concept tests often fail because teams ask the wrong question:

• “Do you like it?”
• “Would you use this?” These invite polite opinions, not reliable evidence.

A good concept test is a structured way to learn:

• whether users understand the value
• whether it solves a real job
• what tradeoffs they accept

Here’s a simple playbook:

1) Start with the job, not the concept

Ask:

• “Tell me about the last time you did X.”
• “What was frustrating?” This ensures the concept is anchored in reality.

2) Present the concept briefly

Use a one-minute pitch:

• problem
• solution idea
• expected benefit Avoid showing UI too early. You want to test comprehension first.

3) Ask for interpretation

Before asking opinions, ask:

• “What do you think this does?”
• “When would you use it?”
• “What would you expect to happen next?”

Misinterpretation is gold—it shows where value isn’t clear.

4) Force tradeoffs

Ask:

• “What would you give up to get this?”
• “Would you switch from your current approach?”
• “What’s the minimum it must do to be useful?”

5) Use willingness signals, not compliments

Better signals:

• “I’d use this weekly.”
• “I’d send this to my boss/client.”
• “I’d pay for this add-on.”
• “I’d replace spreadsheet X.”

Concept tests are about decision-making under uncertainty. If you leave with “they liked it,” you learned nothing. If you leave with “they’d use it if…” you have product direction.

Interview line:

“I run concept tests to measure comprehension, job relevance, and tradeoffs—not opinions. The goal is to decide, not to be validated.”