I’m not a big fan of numbered reviews.
For one, the numerical scale is arbitrary. A 94/100 doesn’t differentiate much from a 9/10 or a 4/5. Yes, those are technically different (I do understand math, after all), but the increased granularity of a percentile or base ten model doesn’t change much from the basic five point model. It’s still just variations on “really like, sorta like, neutral, sorta dislike, really dislike” spectrum, but those are statements of fuzzy opinion. We can’t have that when Metacritic is waiting in the wings. (And insanely, some game studios tie bonuses and even salaries to Metacritic scores. This industry is so messed up…)
Two, the numbers tend to be inflated. A normal distribution of scores would suggest a bulge in the middle of the data set, but game review scores tend to exhibit this data bulge around the three quarters mark. A game that scores in the middle of a number scale, a 5 or 50 (or bizarrely, a 2.5 in a five star system… why bother with halves?), will be seen as a failure, when it is precisely in the middle of the possible scores. Maybe that means the game gets a resounding “meh”, but that’s neither a “retch” nor a “rapture”. Some may well like the game, despite its warts, while others can’t get past the small bra size.
Again, it often comes back to the simple “like-dislike” scale, but since the numbers are arbitrary and weirdly shifted, they don’t correlate well to a simple scale like that.
More than that, though, there is also the concern of personal preference. I happen to really like Cogs, a brilliant little sliding tile puzzler I picked up from Steam for $2 during the Christmas blitz. Its Metacritic score of 73 puts it squarely in “meh” territory in the typical scoring of games. Braid, a game that seemed poised to bring about the second golden age of gaming simply by its mere existence, has a nice 90 score on Metacritic, but it gets a resounding “meh” from me. Grand Theft Auto IV and Halo proudly wear their “Universal Acclaim” badge bestowed by Metacritic (complete with breathless fanboy reviews flooding the internet, tallying to 98 and 97 scores, respectively), but I can’t work up more than a resounding “retch” for either of them. Maybe they mean “universal” the same way that Miss Universe is always an Earth human, or the World Series is pretty much just a United States thing.
Far more useful to me are well-written reviews that clearly state opinions and facts about a game. I may not agree with them, but if they are well-written and clearly reasoned, I can get a fair bead on what the game offers, especially if I read a spread of favorable to unfavorable reviews and parse the commonalities. I wrote about this a little bit when I suggested a Punnet Square of difficulty, and how it colored player reactions to Valkyrie Profile: Covenant of the Plume (a game I really liked).
Even some “real” journalists don’t seem to like numbers, but wind up using them anyway. What good is a system that doesn’t accurately convey information, and that the users don’t trust?
So, henceforth, if I’m going to be pinned down to giving a particular game a “rating”, I will be using the following system:
Games I really like will be given the rating of “Fudge“
Games I sorta like will be given the rating of “Peach“
Games I am ambivalent toward will be given the rating of “Celery“
Games I sorta dislike will be given the rating of “Onion“
Games I really dislike will be given the rating of “Chitlins“
If you don’t like my particular choices, well, perhaps it’s just a matter of taste after all.
The only review site I trust is Game Revolution, ( http://www.gamerevolution.com/ ).
They use a graded scoring system, A+ to F-. It’s simple, and works exactly as you’d expect. Best of all, you don’t have to readjust your world view to understand 7 as being the point exactly between 1 and 10, since most people who’ve been through at least an American school system understand the concept that C means average.
Other than that I don’t really pay attention to reviews, only word of mouth from friends and bloggers with similar tastes.
On second thought, trust is perhaps too strong a word. Lets just say it’s the only review site I can ever be arsed to visit of my own accord.
(And insanely, some game studios tie bonuses and even salaries to Metacritic scores. This industry is so messed up…)
I once worked for a small indie company poised to do a deal with – lets say a big publisher whose name begins with L – over one of their biggest franchises. We were (obviously) hugely excited, until we learned that they wouldn’t pay us any royalties *at all* unless we got a metacritic rating of 80% or higher.
We pointed out that by that metric, they shouldn’t have paid themselves any royalties for the past couple of years. Negotiations did no go well from that point on.
I don’t think it’s that they use numbers that matters – what matters is how seriously these numbers are taken.
And yes, there is a trend for people to forgo critical thought and instead follow a number someone, somewhere, decided to make up.
Sometimes I think what raises us above other creatures, like moths for example, is only that our flames are more complicated.
My scale is simple: 0 or 1.
1 Yes
0 No
Serves me well in games, life, and women….
…wait. Scratch that last one before the wife sees it.
Ha, I love it! I mean… I fudge it!
I agree about the whole scoring system. It seems very aribitary but then human beings love to quanitify things. It’s why we do exams and tests constantly at school and college.
I remember when IGN gave GTA IV a 10/10… thus bascially saying it was the *best* game ever and better than anything else ever. I thought it was OK.
Games, books, movies – rate them on a 1-100 of 100 basis, yeaaaah.
I read review sites that stated they were forced to do this, because people demanded it. There is some truth to this, see my final paragraph.
I know several german book review sites that do not give a numerical rating, but only the opinion of the reviewer. They have some editorial picks but often not even 1-5 star-ratings. I know some sites that sort by ratings like that, but they are often one-man-shows, for which I find that quite acceptable, as I can directly relate from the likes and dislikes of the reviewer to the score he gives certain kinds of literature.
But for games? Most gamers are kids. But thanks god not all kids are like that. The silly ones want their favorite game to have a 100/100. If you disagree, you are too dumb to understand the game. They even don’t want to read what the reviewer actually wrote or criticized, they just want to get a confirmation that their favorite game is great. If you say no, you are ruining their fun and they hate you. So in order to appease the readers, all games are great in the upper margins. But this also means a 79% game is actually NOT GOOD! :>
The worst game review pages are those with the “objective” but totally abitrarily weighted criteria: They rate gameplay with 50%, graphics with 25%, sound with 10%, and other things with X % and the game cannot get better than the gameplay rating, so if the gameplay is shitty not even great graphics can save it.
Just for those who need high score lists… why can’t people leave measuring things to stuff where it is pretty viable. You can measure the size of certain things in inches… but try to measure “FUN” with a number.
Take the approach of literature science: You can claim everything you want, as long as you can provide enough supporting evidence.
I would like to add that I prefer to read private blogs when it comes to game reviews and often also news.
Journalists by now take their information from the net and bloggers without checking the sources and not only the standards of web journalism have reached an alarmingly low level, also printed magazines of supposedly high renown and standards print incredible crap full of rumors, unreliable sources and mixed up facts that I stopped reading them.
(The blessings of the internet and globalization – you get news within seconds from all around the world, the quality however leaves a lot to be desired???)
I rate your blog entry as chocolate mousse. Dunno how this translates to the “fudge-scale”.
[...] Of course, that’s all a matter of taste… [...]
Awww, man, you can’t put onions as “sorta dislike” and Celery as “Ambivalent” !
Onions are just a “must add” thing for so many plates, even if, I admit, taken alone they aren’t the first thing I’d eat…
Onions are the… the… I don’t know, crafting for an MMO? Not essential but adds so much flavor?
Or… Or the tutorial: first thing you add, and go with it until it’s done before adding more.
Brocoli. Please change it with broccoli !
Aw, taste differs, do they
…
Simon, what a lovely understatement. *sigh* Some of these industry hacks really are clueless, and arrogant in their cluelessness. That’s fine if you’re a nutter on the production floor like me, where damage can be minimized, but when promotions give dangerous people power, urgle, that’s bad news.
Longasc, great points, thanks! I really do wonder how much of the numerical obsession is cultural.
Modran, if it helps, I was thinking of putting onions in the “Really Dislike” field, but chitlins are worse. Much worse.
I really can’t stand onions, though I’ll occasionally concede that onion powder is good in *very small* doses with other spices. That said, yes, broccoli could also slot into the “sorta dislike” field. So, if I say Onion, think “Broccoli”, and it all still works.
Thanks for chiming in, everyone. I know, this sort of “fluff piece” is sort of pointing out the obvious, and tastes differ wildly, but hey, it’s always nice that we can be civil about it. (And yes, I know my tastes are weird. So it goes.)
[...] pollinating and cherry picking the best parts. It’s like buying ice cream and then buying a chocolate pie. Either is great alone, but together, they can be more than the sum of their [...]
[...] I’m a developer. I’m not looking for cookies (though sending me fudge would be OK) or cards, but let me tell you a little bit about this side of the [...]
[...] a great game, a curious mix of shop sim and dungeon crawler that manages to be more like fudge mixed with peanut butter rather than anchovies mixed with onions. It came out of left field for [...]
[...] I’d even like to dig into Fract, apparently a sort of Myst-Tron hybrid. Sounds like yummy cinnamon and dark chocolate fudge to [...]
[...] out… well… partially fudge, partially onion, and the two don’t exactly mix. (Apologies to the fudge-covered onion lovers out [...]