Features

7/10: The review problem

Opinion: Metacritic, 'bribes' and the MW3 storm - John Dean thinks the attack on game reviewers has got out of hand

Up until just a few years ago, the only shudders you'd associate with the word 'seven' revolved around Brad Pitt and a small cardboard box. Now however, it seems that times have changed.

In an industry in which the average has been twisted to represent the epitomy of failure, 7/10 signifies pure disappointment. To be blunt, the whole system is a f***ing mess. If we're going to move towards fixing it though, then we're going to need to work together.

Skim past the idiotic notion that individual journalists are being offered bribes to alter their scores, and the truth behind game reviewers is actually very simple: Game critics aren't as different from you as you might suspect. Just like you, they desperately want exciting and innovative games to succeed - and by the same merit often want to watch the stale products in our industry fade into the obscure.

Zoom

Fundamentally though, there's a greater responsibility at play: Opinion pieces like this are perfect for when it comes to venting a secondary agenda, but it's important that reviews are kept pure. A good reviewer won't allow politics to infringe on the core question of whether or not the game they're playing is great fun to play.

This method might not always result in the most satisfying pieces to read, but it's objectivity like this that we need to embrace. Reviewers most definitely need to be questioned, derided, and occasionally attacked - but it's important that we do so for all the right reasons.

In the past few weeks, we've seen three fascinating cases in which reviewers have been attacked for slightly different reasons. Eurogamer's 8/10 Uncharted 3 review caused an uproar because people were expecting higher scores. A similar thing happened this week when Gamespot's US team awarded Skyward Sword a 7.5/10. Modern Warfare 3's critical acclaim (read our Modern Warfare 3 review) caused a huge backlash across the industry, presumably because people were expecting the game to receive lower scores than EA's excellent Battlefield 3.

Expectations play a huge part in the problem, but it's important to be aware that you aren't the only people moved by these decisions.

Whilst games journalists might not always be as brutally honest about it as some of the commenters below tend to be, most of the sentiments are exactly the same. Everybody wanted Uncharted 3 to be perfect, for example, but it's important to make sure these expectations don't get in the way of making an entirely fair judgement.

Zoom

A lot of Modern Warfare 3 reviewers would have loved to walk away from the latest Call of Duty game having given the franchise a kicking, but the product on show simply didn't allow it - it's a fantastic game.

Claims that Modern Warfare 3 reviewers had been somehow bribed are hilariously ironic: The Metacritic score floats where it does because reviewers have specifically restricted their opinions of the publisher getting in the way of their judgement of the game. The same is true with games like Uncharted 3. Giving a lower score than you'd like to a developer that you love is a genuinely horrible thing to have to do: No good critic takes pleasure in kicking people that they like, but that doesn't make it any less essential sometimes.

Having said that, it's important to question why people have come to decide that an 8/10 score is in any way a negative thing. There's an easy answer to this one, but the long answer is more interesting. As with any disease, it's best to try and identify the root of the infection - and there's one group in particular that have regarded 8 as being a terrible score for far longer than the rest of us.

Ever since games like Gears of War and Bioshock proved that a 90-plus Metacritic rating could have such a huge impact on the sales of a game, internal review-benchmarks suddenly became far more important. It's remarkably common these days to see publishers hiring experienced games journalists to write mock-reviews of the product a long time before the game is sent out to media - if these initial judgements suggest that the game isn't likely to score well, this usually impacts the entire marketing campaign.

Games that have previously seen a decent amount of buzz will suddenly cease to exist, with publishers attempting to cut their losses by slashing advertising and PR budgets at the very last minute.

Zoom

Activision's Singularity is a great example of this in action, but can they really be blamed for such drastic action? Singularity was a fantastic amount of fun, but with games like Bulletstorm failing to make an impact despite a strong marketing campaign and a Metacritic score of 84, would Singularity's average of 76 have really fared any better?

Metacritic can be a remarkably deceptive service, and it's important that you're able to understand the primary function it serves.

Whilst it's useful to get a general gist of whether or not something's worth investigating further, the fuzzy nature of the statistics it provides make it more appropriate for use as a metric tool for marketing departments. What most people outside of industry don't often tend to realise, is that marketing is a science. These guys aren't just throwing money at a wall in the hope that something sticks - they have cold and hard number-proof that tells them exactly how much more money they'll make by achieving certain scores.

Invest another six months into development, and you'll be able to bump it from 70 to 80. If the projected sales increase makes the extra development cost a worthwhile payoff, you'll do it. If it doesn't, you won't. It's that simple.

  1 2
  Next

Comments