How to Read Complaint Data Without Fooling Yourself

Who complains in public

People who complain online skew technical, English-speaking, and concentrated in industries with active communities. Nurses, plumbers and warehouse supervisors have plenty of problems and post about them far less than developers do.

This means absence of complaint is not evidence of absence of problem. It is often evidence of an under-sampled industry — which, if you have access to it, may be the least competitive opportunity available.

Volume versus breadth

Forty complaints in one subreddit can be an artefact of that community's culture. Ten complaints spread across a forum, a review site and a job board almost never are. Weight breadth over volume, always.

Corrections worth making

• Check dates before concluding a gap still exists — the complaint may predate a fix
• Check whether the complainer is the buyer; pain and budget often sit with different people
• Read the quote, not the summary, whenever the two feel different
• Treat a category with no complaints as unexplored rather than solved