Product research + analytics · Survey + SQL
A 1,257-response survey and 50 interviews I designed and led, analyzed to answer one question: is satisfaction driven by new features, or by whether the product simply works.
The question
The instinct when satisfaction slips is to add content. Before accepting that, I tested whether satisfaction tracks features at all, or whether it tracks something more basic: how often the product fails under the user.
The finding
Joining survey responses to reported error frequency showed a clean, steep relationship. The experience is not rated on what is added. It is rated on what breaks.
Satisfaction by error frequency
10-point scale · survey n=1,257
Satisfaction nearly halves as errors move from none to near-daily, a larger swing than any feature preference in the survey.
The priorities
Top priority, ranked by respondents
count of respondents · n=1,257
The top three priorities are all reliability. New content and new maps sit far below, despite being where roadmaps usually spend.
The call
Reliability, not feature expansion, should lead the roadmap. Satisfaction is tied to error frequency, and users rank stability, bug fixes, and performance above anything new. Protecting retention starts there.
How I'd measure it
Limits and next steps
Cross-check reported error frequency against telemetry so the driver is measured, not just recalled.
Once errors are rare, confirm features re-enter as the next driver rather than assuming reliability scales forever.
Check whether heavy users weight reliability differently from casual ones before a blanket roadmap call.
Use the reliability work itself as the test: fix, then watch satisfaction and retention move.