Mutterbox

How many reviews are enough to draw conclusions

At twenty reviews a single bad one moves the average by half a star, and one irritated customer looks like a systemic problem. The threshold where numbers start to mean something is lower than most people assume — but it is measured on something other than the average.

The Mutterbox team · · 4 min read

The most common question after a first report is whether a hundred reviews is a lot. The answer depends on what you want to conclude — and that is not a dodge, it is the whole point.

Half of the reviews say nothing

The first number that surprises owners: roughly half of Google Maps reviews are a star rating with no words at all. Across industries it runs between forty and sixty percent.

Those reviews count towards the average and towards the number on the listing. They count towards conclusions not at all, because there is nothing in them to extract. Someone gave three stars — and that is everything known. Not whether it was the waiting, the price, the noise, or the full car park.

Hence the first practical conclusion: when someone says "I have four hundred reviews", they have about two hundred reviews that anything follows from. And it is that second number the analysis depends on.

The threshold is twenty — per topic, not per business

A repeating problem counts as repeating when at least a dozen or so independent statements stand behind it. Below twenty reviews with text on a given topic, a single statement carries enough weight that one exceptionally bad day looks like a permanent state.

The distinction matters. A business with a thousand text reviews is comfortable analysing cleanliness complaints if there are a hundred and fifty of them. With six complaints about parking there is nothing to analyse — a thousand reviews overall does not change that.

This does not mean six complaints are meaningless. It means they are not yet a statistic; they are a prompt to go and look.

The star average is the worst number in the set

The average mixes two populations — writers and clickers — and reduces everything to one figure that does not say what to do. A drop from 4.6 to 4.4 points at no area, no cause, and not even at whether anything actually got worse rather than a different kind of customer having arrived.

The number that says something useful is the share of reviews mentioning a given topic. "Forty-one of two hundred and eighteen text reviews mention waiting at reception" is a sentence a task follows from. "Average 4.4" is not.

Raw counts lie when traffic grows

The second trap is sneakier, because it looks like common sense. A business compares complaint counts month over month: it was twelve, now it is twenty-three. Conclusion: things got worse.

Meanwhile traffic tripled. The share of complaints fell. Objectively things improved, and the decision taken on the raw count — replacing the shift manager, say — was taken on a measurement error.

So trends are computed on the share of complaints among reviews from a period, never on their count. That single rule removes most false alarms.

Old reviews: when they count

A review from four years ago does not describe today's business. For the question "how is it now" it is useless.

For the question "what changed" it is priceless. A complaint about air-conditioning noise running from 2021 to today is different information from the same complaint ending in the month of a refurbishment. The first says the problem persists. The second says the refurbishment worked — and it is the only available evidence that the money did anything.

That is where the point of pulling the whole review history comes from, rather than the last ten. The last ten cannot show a trend, because a trend needs a time axis.

A practical threshold to remember

Three numbers are enough:

  • under 20 reviews with text — read them one by one, count nothing,
  • 20–200 — you can talk about topics repeating, but not about seasonality,
  • over 200 — seasonality works, period comparison works, and so does measuring the effect of a specific change.

If you are in the first band, the best thing you can do is start collecting reviews systematically — from everyone, the same way, without selection. In six months there will be something to read.

Common questions

How many reviews does an analysis need?

Around twenty reviews WITH TEXT on the topic you want to draw a conclusion about. Below that, a single statement carries too much weight and a repeating problem cannot be told apart from a one-off incident.

Why is the star average misleading?

Because it mixes two different populations: people who wrote why they gave a rating, and people who clicked a star and said nothing. The second group is usually about half, and nothing is known about them beyond the number — so the average describes something that cannot be explained.

Do old reviews still count?

For conclusions about the business today, no. For conclusions about a trend, very much yes. A complaint from four years ago that still recurs means something entirely different from one that stopped after a refurbishment.

Should I look at the number of complaints or their share?

The share. Complaint counts rise with traffic, so twice as many complaints against three times as many customers is in fact an improvement. A trend computed on raw counts shows decline where there is none.

How do I tell a seasonal problem from a permanent one?

Lay the complaints about a topic out over time in monthly buckets and look at the share, not the count. A seasonal problem peaks in the same months every year; a permanent one sits at a steady level all year.

Related posts

Be first in when we launch

Join the waitlist. We'll write once, on launch day, with a link to create your account.