John's nerd corner

Review Score Philosophy

I do not see myself as a serious critic. Online reviewers who seem to think very highly of themselves are quite obnoxious, honestly.

However, I think years of following online reviews has given me a particular interest in the thought process and methodology behind picking a score.

Anytime on this blog I've talked about anime or video games, I haven't even used scores. But I do use scores on AniList. And thinking about my scores on AniList led me to want to write an explanation of what my scores mean, which I originally did over there—but frankly, I kept getting embarrassed about having so much text in the bio box, soooooo now this page exists.

But yeah, let's talk about review scores.

In the first place, anytime a review has a score attached, the internet can't handle it very well. Most people just have a reaction to the number and ignore the actual content of the review. Look at the comments on any review of a super-popular release from theneedledrop or IGN.com and you'll see what I mean. People skip to the number, and most likely if it's not high enough, they'll start complaining.

Even so, I do think scores are somewhat useful. They're a quick way to tell you the reviewer's broad feelings about a work. But that's it. To really understand what thoughts a reviewer has about a work of art, they have to tell you. You've got to read the actual paragraphs/listen to the rest of the video.

I think numbers also fail to be particularly useful when people only use a range of 7 - 10. I once saw an AniList review that said something like, "This show is bad. 7/10."

It does seem like many people see scores as being similar to a grade in school, and thus they see anything less than a 90% as unacceptable.

I don't think our school mindset prepares us very well for reviewing art, however. Generally my experience with school is that they want you to do things a certain way, and they have specific requirements you're supposed to meet, and as long as you meet those requirements, you get the full grade. But there's no rubric for art. Ok, so you want to be "entertaining" maybe. But that's so subjective. Not everyone is even entertained by the same things. And furthermore, a lot of art is not even primarily trying to entertain. There are other feelings to explore. But let's say you meet the bar of "being super fun to watch." Does that really make for a masterpiece? It's up for each critic to decide for themselves.

The internet may have an unspoken and dorky obsession with establishing definitive rankings for every single piece of media in existence (why else do people just bring up how movies or shows are bad totally unprompted so often?) but I think that we should stop holding on to the idea of consensus so strongly. Different critics have different perspectives because they have different life experience, and if you actually listen to what they have to say, who knows? Maybe listening to different voices can change the way we all think about art.

But enough about all that. Let me give you the summary of how I use the 10-point scale.

Before I do that though, I will just admit, when I look back at scores I've given, I often don't understand what I was thinking either. It'd probably be easier if I wrote a review every single time. Feelings and impressions kind of morph when they sit in your brain for a while. Even if I haven't actually rewatched something recently, I find myself asking, "Didn't I rate that too low three years ago??"

Anyway.

The goal of these numbers is simple: to communicate my honest reaction.

I think when I am picking a number, there is a subconscious scale of excitement versus disappointment. What I mean is, if an anime does really cool stuff but then does things I dislike, you could call that high excitement, but with a large dose of disappointment weighing against it. So it'll end up at a 7/10. Or, maybe an anime is consistently enjoyable and doesn't do anything I dislike, but it also just doesn't really elicit strong emotions from me. We might call that low disappointment, but only medium excitement. 7/10. So there are different reasons things get the numbers they do, but it ultimately comes down to my gut feeling, really.

When we're talking abut anime and manga, story definitely gets the most weight in how I feel about a work, but I factor in other elements too.

So here's the list of what the numbers are supposed to mean to me (basically):

Alright yeah. That's it. Bye.

JUST KIDDING HERE'S SOME BONUS THOUGHTS

The different types of 7s (also 6s)

The feeling of a 7 is, "I truly like this, but not enthusiastically." There are several reasons this can happen.

A lot of stuff has me like, "This is cool and funny, but not that cool or funny." That's the most basic reason I can give a 7.

When I rate something, I am of course trying to think about my feelings through the entire story. I think often a manga/anime is most exciting in the beginning because they have a fun idea, but then eventually the idea runs out of juice or the story turns into something that doesn't interest me that much.

Like, why'd I give Dr. Stone (manga) a 7? The initial concept of rebuilding civilization from scratch was very interesting to me. But towards the end I kinda felt like I broke out of the spell and was just reading a formulaic thing of "Ok kids now we're gonna teach you about how cell phones are made," and I dunno, even though the series had always been educational, I just didn't feel that tension anymore with how fast science was moving (and maybe some answers to mysteries were not that interesting to me).

And why'd I give Horimiya (anime) a 7? Well, after the titular couple gets together it's like the main hook for watching goes away, and it's still good, but not as gripping anymore. That feeling of anticipation is gone. Plus, the character writing throughout all of it feels pretty surface-level.

So yeah! But it really comes down to my gut feeling, and my words are simply my desperate attempt to explain why I felt the way I felt. It's not easy.

Now there is another type of 7: it's for a story that is silly and kind of frustrating at times—but boy is it fun. That's how I describe Nisekoi (manga). It's like, yeah, this isn't doing anything that amazing, except for being a BLAST. The romance payoff in the end is just adorable, and sure parts were frustrating, but when I look back on it, those became fond memories too.

Finally, a very rare type of 7 is: this story was actually super compelling, but then it made me mad. That's why I gave Heavenly Delusion/Tengoku Daimakyou (anime) a 7. It was very cool and interesting but I thought one part near the end of the anime was just super poor taste, so I had to dock major points. Which doesn't happen often, luckily. But in that case, it was hard to ignore.

By the way, these distinctions exist among 6s too. Obviously the difference is that I just overall think a 6 is not as good as a 7, but what I'm saying is that there are 6s that I think have weird problems, and then there are 6s that are consistently just... I dunno, kinda dumb (but kinda good too).

So are your ratings trying to be objective?

Are my ratings "objective" in the sense that they determine scientifically the quality of a work? No, because there's no such thing as an objective measurement of whether a work of art is good or not. And yet I've had internet randos take issue with that. Unsurprisingly, they hate modern art. Huh.

Are the ratings "objective" in a less literal sense, meaning I try to remove my own biases and just give credit to stuff that seems good regardless of how I personally feel? Also no. I like what I like.

The whole "not for me, but I think it's good" thing seems like a strategy to avoid internet people's anger. Have movie critics in newspapers ever really employed that reasoning?

When I think about this stuff, I worry that people on the internet believe that all discussion about media is just a process of collectively deciding what Truth is (as if "movie good/bad" is a matter of objective truth). They'll go along with what other people say, pick a side, repeat some talking points that sounded convincing. People aren't confident in their own experience; they feel like they have to be told by someone else if it's ok to like something. It's not how it should be. If you're gonna write about a work, it's gotta be your thoughts, not someone else's. Shouldn't a person's rankings of their favorite anime & manga tell us about who they are?

But if all their ratings are just agreeing that the most popular and beloved shows are indeed good, what it tells you is that maybe they're afraid of going against the crowd.

I'm not saying you should intentionally be a contrarian or massive hipster, of course. I mean, heck, my favorite anime and manga are not obscure. The large English-speaking anime community of today mostly does not know much about Love Live!, sadly, but the franchise still has lots of fans worldwide and I will continue to tell people how great Aqours is. So yeah I mean it's not like my tastes are the most basic, but they're not that hipster either. Anyway, if you're someone who maybe wants people to listen to your reviews, if you want to establish credibility as a critic, then I think you gotta know not just the stuff that everybody's seen, but lesser-known works too. Definitely a greater number of lesser-known works than I've finished. See, I'm not that credible. I'm not a tastemaker. But if someone can eloquently describe what a great work that I've never heard of offers to its audience, then I'll be like, dang, this guy is someone who loves and understands art. I'm inclined to take their recommendation.

Did you ever notice the majority of reviews published on AniList seem to be really bad?

Anyway, let me get back to the "objective" question. If I don't like a show, I don't like a show. But I'll admit, with anime in particular, sometimes the production values are just so good that I have to give props even if I didn't care for the story. It's happened. So am I overlooking my own biases in that case? Kind of. But the score I give would be way higher if I actually liked the story.