Sampling, Selection, Summarization: Scaling Feedback Without Losing Signal

TL;DR: When feedback volume grows, you need rules for what enters the system, what gets sampled, and what gets summarized—without hiding reality.

The scaling problem

At 50 inputs/week, you can read everything.
At 500/week, you can’t. If you don’t change the process, you’ll either drown or over-summarize.

Sampling rules

Use sampling to keep representation:

• sample by segment (SMB, mid-market, enterprise)
• sample by journey stage (onboarding, daily use, renewal)
• sample by channel (support, sales, research)

Selection rules

Select based on:

• severity (blocks task vs annoyance)
• frequency (seen across sources)
• leverage (fix improves multiple workflows)

Summarization rules

Summarize on cadence:

• weekly: candidate themes
• monthly: top themes + evidence pack
• quarterly: strategic narrative

Takeaways

Scale requires rules. Rules create trust in the signal.

---