On game reviews, ratings and recommendations

Jul
26

On game reviews, ratings and recommendations

Game reviews are, arguably, one of the most important wheels in the giant machine that is the gaming industry. Many people make their purchasing decisions based upon reviews or previews of games, which is why shows such as E3 and PAX are so important and monumental to games journalists. But as our attention spans grow ever shorter, and our reliance upon visual indicators grows stronger, are we actually reading reviews and understanding them? In fact, do people even know how to read game reviews?

Crunching numbers.

reporter at typewriter 100109 lg On game reviews, ratings and recommendations

VGW's Brian on his wireless Smith Corona.

The problem with 1-5 and 1-10 scales is that they lead to the dangers of uncorrelated data. Let’s use Brian’s review of Uncharted, as an example. Brian gave the Sony-exclusive title a shocking 2/5; so shocking that we received a legion of comments wondering how dare we give the majestic Drake a 2/5, which is, essentially, an F (see below). Well, looking at Brian’s review, he begins to qualify his feelings on his dislike of upgraded quick time events, his distaste of the predictable gun play, and various other factors. After reading the review, you begin to see why he did not like it. But therein lies the operative word: he.

Likewise, Chris recently gave Zelda a 5/5. Does that mean the game was utterly flawless and perfect? Of course not because there is no such thing, but it does mean that Chris thought Zelda was so mindblowingly amazing, it will permanently be etched into his gaming psyche. He quantifies in his review that he has always loved the Zelda franchise, and it becomes clear by reading it that this game had little chance of not being a 5/5 for him.

Reviews are excellent ways to gather information on a game and see if it’s something you would like to purchase. But you have to remember that reviews are inherently personal. Realize that every game the reviewer has ever played is at work in their psyche, and on some level, this game is being compared and categorized in relation to this information. “Oh it has this feature? That’s like this game.” Also, a gamer’s particular tastes weigh heavily upon their review. While you can certainly look at games objectively, and ultimately decide whether or not someone out there may like a particular game, in the end, a review is based on the reviewer’s experience with it.

thumbs On game reviews, ratings and recommendations

Read between the thumbs

And that is a great thing. Just as a reviewer’s experience of a game is personal, so too is a reader’s opinion of the review personal. You may be thinking “what the hell does that mean?” I’m so glad you asked. The reason so many multitudes of game review sites exist is that not everyone is going to review a game the exact same way. The key to reading reviews is not just looking at the score or what Person A thought of the game, the important thing is to read between the lines and say “this person seems to like the same things I do, I’m probably safe to listen to their opinion,” or “I do not agree with this person, therefore, I should find another review,” or even better, “what features do I look for in a game, and what does this person think about them?:

Only when we truly begin to analyze reviews do we start to move away from the impact of a number. Really, 8/10 means nothing about a game if you don’t know how they arrived at that conclusion.

We’re not in elementary school anymore.

Of course the current number system also leads to another problem within gaming culture and that is the thought that anything below a 7/10 or 3/5 is not worth playing. At all. Recently, I rated Hunted: The Demon’s Forgeat a 2.5/5. I received a number of comments wondering how I could give this title an ‘F,’ and I spent a great deal of time reminding people to put aside the number-to-letter system we had ingrained in us as school kids, and look upon the broader scale. A fifty-percent is exactly in the middle of the road. This means the game wasn’t great, but it wasn’t horrible either; it was a monument to mediocrity. Some people can find a great deal of joy in games such as this, while others find nothing but pain and misery. Others, such as I, found it to be entirely “meh.”

dunce cap 300x225 On game reviews, ratings and recommendations

Turns out, in grown-up land, a 2.5/5 is NOT an F.

Because of this ingrained letter conversion, gamers tend to fume over low scores, and write them off as not worthy of my money. But wait a minute! Zookeeper, a movie that has received a whopping 30/100 on Metacritic, and has been widely panned by multiple sources, grossed over $43million so far at the box office. So why do movie consumers fork out money to see low-rated movies, but we scoff at buying low-rated games?

The Money Trap

Many people will argue the price. Let’s say, for example, you go to a Friday evening showing of Zookeeper. You shell out $10 for a movie that runs 104 minutes. This works out to approximately $6/hr of entertainment. If you enjoyed the movie, that’s $10 well spent. If you didn’t, well, then, that’s $10 you may as well have flushed down the toilet. On the other hand, let’s say you spent $60 for Duke Nukem Forever. Duke’s campaign lasts about 14 hours, depending on your play style which works out to approximately $4.28/hr of entertainment. If you had any inkling of fun playing Duke, that’s a pretty damn good value in entertainment. If you didn’t enjoy it, sell it on eBay or back to GameStop and trade for something you’d rather play. This, to me, makes the price point argument moot.

I chose these two titles for a very specific reason. Both titles were widely panned and loathed by reviewers, but did well in sales numbers and have had pretty decent user reviews. Likewise, both titles have these high sales and user ratings because of an important thing: fanbase. Chances are if you went to see Zookeeper, it was because you like Kevin James. If you bought Duke Nukem Forever, chances were you were an oldschool Duke fan. For diehard fans of a developer or series, you need to ask yourself whether or not reviews have any effect on your purchases. If not, chances are you read reviews to know that someone else out there agrees with you, or you like being angered by people not agreeing with you. Believe it or not, this is part of the fun of reviews.

It ain’t easy bein’ an elitist.

Writing a game review is a rather tricky process. Reviewers strive to walk the line between producer and consumer — that is to say we are middlemen of sorts. We want to play and enjoy, but also to inform and dissect. This is why, so often, video games, movies and books can be loathed by critics but loved by fans, and vice versa. (Back when Psychonauts was released, it met with critical acclaim and lackluster sales, just as Homefront met with critical panning and decent sales.)

high+society+Celeste+Holm+lace+dress+50s+pink+vintage 300x168 On game reviews, ratings and recommendations

What shall we tell the rabble to play this week, darling?

We here at VGW do something called a Postscript. You may have read Brian’s postscript on Duke Nukem Forever, and my postscript on Dragon Age II. The point of a postscript is to allow for the candor that reviews do not. From a critical perspective, Brian found DNF to be pretty damn bad, but when he decided to just sit back and let himself enjoy it as a fan, he found more enjoyment. Mostly because he wasn’t under the burden of looking for flaws in order to warn potential players what they were getting into. Adversely, I was a little more lenient in my review of DAII because most of my niggling postscript annoyances were that of a fan, and I didn’t want my review to sound like the ravings of a fanboy. Because no one wants that.

So remember, when you read a review, look beyond the number, and read both the meat of the article, and between the lines. What is the reviewer actually telling you? What can you gleam from that reviewer as a gamer? Take these factors into account and realize that no review is The One True Review (except for my review of Kill Team, obviously). It’s why so many of our sites grow and thrive — so you can find reviewers you like and follow them. And for reviews you don’t like or agree with, figure out why you disagree, beyond the number, and you may find a truly engaging conversation in the making.

About Jen Bosier

Jen lives with her husband, daughter and super-villain cats. If she's not reading comics or engrossed in a WH40k novel, she's probably telling you which horror games you should be playing. She's a recovering member of the PC Master Race, and a reluctant Xbox fangirl. You can also find her on the Furious Fourcast.

8 Comments

  • Jul 26, 2011 @ 7:59 am

    Jen, Resident Fanboi Ambassador and Counselor. Motto: I understand your problem, let me help you fix it.

    Reply
  • Jul 26, 2011 @ 8:39 am

    First, it is human nature to desire a simple score. The reviewer should not expect gamers to read the review, unless the gamer wants to debate the review. “Brevity is the sole of wit”

    Second, there are two major scoring systems that everyone understands. Think of this as a “Standard Weights and Measures” for scoring. Letter grades/100 percent system and the four star system in movies.

    Everybody knows that a “C” is 70% which means it’s “good enough”. Everybody knows that 2 1/2 stars out of four is also “good enough”. Anything less is not worth your time.

    Websites, like 1up, claim that 50% is good. They changed to the letter system, and claim that a C is 50%. (Metacritic’s interpretation, undisputed by 1up.) G4TV uses a 5 star system. These systems lead to inconsistencies within their own review team and confusion for everyone else.

    So, before we tackle the “subjectivity vs objectivity” debate. We need a standard system for video games. I don’t see what’s wrong with the 100 point percent system, nor the 4 star system.

    Finally, Roger Ebert became the defacto opinion for movies because people agreed with his reviews. He was consistent and communicated to his audience. We don’t have such trusted reviewers in video games.

    Reply
    • Jul 26, 2011 @ 8:55 am

      From my understanding, the point of a review isn’t to be brief. Most people are given an impression of what the plot is in a movie and also are given an idea of some the basic elements of the movie (ie. action, horror, drama, sci-fi). You can’t get that impression from just video game commercials, you have to dig into a lot more. I find scores to be useful, but not always what I’m looking for. People are developing a taste for video games beyond the score. Sure there will always be people who will only look at scores and not read anything, I don’t think that’s the target audience for VGW. Those individuals can go to metacrtic, read the scores and skip the reviews. People like me read multiple reviews and read reivews for games on systems I don’t own or games I can’t play (PC). I think eventually people are going to look beyond the score and want a review that they can relate to because they’ve realized that a 10 point system or a 5 star system just doesn’t give the description they want. Dynasty Warriors 7 is a really fun game to me but if I went off of IGN’s score then I wouldn’t have bought it. I’d rather have a system that gives a reason why a game is so many stars over all, but could be more stars for a targeted audience.

      Sure Brevity is the soul of wit, but rarely are intelligent and wise things simplistic.

      Reply
    • Jul 26, 2011 @ 9:09 am

      Thanks for weighing in Timothy, you made some very valid points. I believe we already have a standard, globally accepted system for video game reviews, and that is called text. I’m not trying to be abrasive; I’m quite serious. The kind of readers we attract here at VGW are the types of gamers who WANT to read varying opinions, and want to latch on to reviewers they can trust, whether that is through a system of common interests, or just quality criticisms and observations. Your suggestion that as a game reviewer, I shouldn’t expect my readers to do anything more than digest a number insults both my profession and the readers who benefit from it. 

      “Finally, Roger Ebert became the defacto opinion for movies because people agreed with his reviews.”

      There is way too much wrong with this assumption, by the way. People respect Roger Ebert, but I challenge you to find even 75% of the population who AGREE with his reviews. Rather, people respect his opinions and his experience.

      Reply
      • Jul 28, 2011 @ 7:29 am

        To all who replied to me above, thanks for your replies.
        My point is that the reviewer should be serving the reader. Roger Ebert (and Greg Kasavin of Gamespot and producer of Bastion) served the reader by taking their job seriously, reviewing with consistency, and communicating to the reader.

        When Roger Ebert says “4 stars” or Greg Kasavin said “9.0”, it’s a must see or must buy.

        P.S.
        On a side note, frame rate is a serious issue that get very little coverage in reviews. I’d like to see reviewers emphasize it as a distinct issue. Most reviews break down into “Gameplay, Sound, Story, Graphics, etc.” I’d like to see “Frame Rate” explained in more detail. A technical analysis would be best.

        Reply
    • Brian Shea
      Jul 26, 2011 @ 9:49 am

      Tim, it would be irresponsible and lazy of a reader to not look at the review and decide solely based on the score given to it, but unfortunately it happens all too often (just look at my lowly-scored Uncharted: Drake’s Fortune review, for instance).

      I gave Duke Nukem Forever a 2.5/5, but I fully stated in the review that if you just want mindless fun, you could do a whole lot worse. Somebody may like DNF, but if they see a lower score and decide to skip it based only on that, they are missing out on an experience they would otherwise enjoy.

      The same could be said about highly rated games. Bastion, for example, got rated very highly on this website. I trust the reviewer of that game wholeheartedly, but I read the review and realized that this game wouldn’t be my cup of tea.

      Looking only at a review score and claiming to know whether or not a game is good is the equivalent to looking at a slideshow of somebody’s vacation on Facebook and claiming you know whether or not they had a great time. Sure, you have an idea based on the limited scope of the way you’re receiving your information, but you’ll surely learn a lot more from talking to that individual directly and hearing the good and the bad.

      Reply
  • Jul 26, 2011 @ 10:07 am

    Great article!! If people cant take the time to read half a page of text here at VGW than they are only doing themselves a disservice. Star systems do not work one bit in my opinion. If a review is going to be any type of deciding factor for what i play i want an explanation for why it was either good or bad. If you’d like just a simple number or thumb up or down there’s plenty of sites for that. Review’s that i have read here at VGW are what I would consider ‘to the point’. They aren’t to long but get the important details that stood out to them as a reviewer across with great accuracy. Personally I don’t think you should let reviews make the call for you EVER, if you want to play the game and your interested than just go play it. Every time you play a game there’s a chance of both like or dislike. I know for myself there’s been times where I stayed away from a game due to bad reviews but when i played it at a later date i found to actually enjoy it and disagree with quite a few of the reviews or reviewers points. Oh and Roger Ebert (IMO) is a joke. Nothing that guy has ever said about a movie have i agreed with, i think hes got horrible taste but who am i? Just a guy, and hes fucking Ebert so that explains that. Great job Jen.

    Reply
    • Jen Bosier
      Jul 26, 2011 @ 13:53 pm

      Kenn — Thanks for the comment. I think Ebert has become marginally obsolete simply because his criteria on which he bases his scores is wiiiiiiiide. He’ll say the plot isn’t great, and the acting is bad, but the costumes and sets are amazing so, thumbs up! I understand his desire to reach a wide audience, but in doing so, he has made his reviews suspect. Am I his target? Are you? Is the dingbat who sits next to me at work?

      As always, sir, your thoughtful commentary is appreciated by we at VGW. :)

      Reply

Leave a comment