← Writing

Problem Severity: The 2-Question Test to Decide If It’s Worth Building

December 12, 2024 · Feature Analytics

Problem Severity: The 2-Question Test to Decide If It’s Worth Building

Hook

Most feature debates stall because everyone is arguing from a different definition. One person is thinking “all users.” Another is thinking “power users.” Someone else is thinking “buyers.” Then you pull a chart… and it “proves” whatever each person already believed.

Feature strategy gets dramatically easier when you standardize the definitions and review features the same way, every time.

Thesis

You can’t measure or prioritize a feature without clearly defining: who it’s for, what outcome it delivers, and how you’ll detect value. Once those are explicit, feature metrics become decision tools—not debate fuel.

Problem severity: the 2-question test

Severity is the “how bad is it when it happens?” measure.

The test

  1. “What’s the consequence if this goes wrong or is delayed?”
  2. “Who feels the pain and what do they do because of it?”

Severity levels (practical)

  • Level 1: annoyance (minor friction)
  • Level 2: time cost (manual work, delays)
  • Level 3: business risk (lost revenue, churn, compliance, trust loss)

Decision rule

High severity problems deserve:

  • guardrails,
  • observability,
  • and reliability investment.

Not just “more features.”

What this post covers

  • The core concept
  • The common failure modes
  • A simple operating method you can reuse
  • A decision checklist you can apply immediately

1) The core concept

A feature is a promise to a specific user segment. Your job is to make that promise measurable.

The sequence is always:

  1. Define the segment (Target)
  2. Define the job + outcome (Value)
  3. Define the evidence (Signals)
  4. Decide what to do next (Action)

2) The common ways teams get this wrong

  • Vague target (“everyone”)
  • Vague value (“better experience”)
  • Metrics detached from outcomes (“clicks” instead of “success”)
  • One-size-fits-all windows (weekly retention for a quarterly job)

3) A practical method you can run in 20 minutes

Use this template:

Target

  • Who should use it?
  • What % of active users is that?

Job

  • What are they trying to accomplish?
  • What does “success” look like?

Signals

  • Adoption: first successful outcome
  • Retention: repeat usage in the right window
  • Satisfaction: confidence + ease signal

Decision

  • Enhance / reposition / bundle / stop

4) A decision checklist

Use this checklist before you ship (or keep investing):

  • Is the target behavioral and measurable?
  • Is “success” defined as an outcome?
  • Are adoption and retention windows aligned to the job cadence?
  • Do we have at least one satisfaction/trust signal?
  • If this goes wrong, do users have a recovery path?

5) Suggested CTA

Copy the template from “A practical method” into your next PRD and run a 20-minute review with Design + Eng. Your goal is not agreement—it’s a shared definition.