Anime Semantle

Ratings Are Not a Clue

← Back to the game

One common approach is to start with highly rated shows on the theory that "today's answer is probably a classic." The game displays ratings, so it is a natural instinct.

It does not work. There are two reasons, and both show up in the data.

1. The pool barely varies in rating

The answer pool is the top 800 titles by MAL global popularity. Which means they are all highly rated to begin with.

MeasureValue
Lowest rating7.97
Highest rating9.25
Median8.24
Standard deviation0.254
10th–90th percentile8.02 – 8.67

The entire pool fits inside a 1.28-point band, and 80% of it is squeezed into the 0.65-point range from 8.02 to 8.67.

For "start with the highly rated ones" to work, the filter has to eliminate candidates. When nearly everything is an 8-point-something, nothing gets eliminated. Titles above 9 are a handful, and the only 7s sit right at the 7.97 floor.

2. Similarity and rating are unrelated

So what about narrowing to "titles rated similarly to the one that just scored 0.7"? That fails too.

Across 599,852 pairs — same-franchise pairs excluded, both sides having a rating — we computed the correlation between similarity and the gap in the two ratings.

Correlation r = +0.004

The closer to zero, the more unrelated. +0.004 is statistically indistinguishable from no relationship at all.

Breaking it down by similarity band gives the same answer. In every band the average rating gap is flat at roughly 0.27.

Similarity bandPairsAvg rating gap
0.1 – 0.22,5780.258
0.2 – 0.319,0600.270
0.3 – 0.495,7990.276
0.4 – 0.5205,5860.280
0.5 – 0.6191,4600.283
0.6 – 0.774,5200.274
0.7 – 0.810,5400.268
0.8 – 0.93020.273

A pair at 0.85 differs in rating by exactly as much as a pair at 0.25. Which makes sense: similarity is computed from synopsis text and ratings come from viewer votes. The two are measuring completely different things.

3. Very close, very split on score

Pairs that tell nearly the same story yet sit almost a full point apart.

SimilarityTitle ATitle BGap
0.822Kaichou wa Maid-sama! (7.98)Kaguya-sama wa Kokurasetai: Ultra Romantic (8.95)0.97
0.804Tsuki ga Kirei (8.02)Kaguya-sama wa Kokurasetai: Ultra Romantic (8.95)0.93
0.770Kimetsu no Yaiba: Katanakaji no Sato-hen (8.14)Bleach: Sennen Kessen-hen (8.98)0.84
0.759Kidou Senshi Gundam 0080 (8.02)Ginga Eiyuu Densetsu (9.02)1.00
0.750Majo no Takkyuubin (8.24)Sousou no Frieren (9.25)1.01

"Majo no Takkyuubin" and "Sousou no Frieren" sit at a respectable 0.750 — both are stories of a magic-using girl travelling and meeting people. Yet their ratings, 8.24 and 9.25, are close to the widest gap this pool allows.

So "it is near Frieren, therefore it must be a 9-rated title" does not follow. There are plenty of 8.2s among Frieren's 0.75 neighbors.

4. What to look at instead

5. The one time ratings are useful

There is one. After the answer is revealed.

Ratings are useless for narrowing candidates but perfectly good for deciding what to actually watch. Find similar shows by similarity, then pick among them by rating — in that order it works. That method is written up in Similar Anime Recommendations.

6. Method

Ratings are the MAL scores from the source data, unmodified. The correlation is a Pearson sample correlation over anime pairs with the same franchise excluded and a rating present on both sides — 599,852 of them, counted directionally. The 0.9-and-above band held only 2 pairs and was dropped from the table. The scoring method and the evidence behind the 70:30 weighting are in About the Game.

Further reading

Every number here comes from the data the game actually runs on. The main strategy guide has the similarity band reference table and a full walkthrough, and each day's answer and its nearest neighbors accumulate in the past answers archive.