Been busy this week – lots of Dragon Commander presentations going on, so I’m happy that I found a bit of time to write this entry.
An interview Sean Ridgeley from Neoseeker did with me went online today. The topic is games journalism and we agreed that once the interview was live I’d post some extra thoughts here, basically the notes I made for myself prior to the interview.
Games journalism obviously is a sensitive topic for a game developer because it’s like a girlfriend asking – do you think there’s something wrong with how I cook? If you’re honest you’re doomed, if not, you can look forward to things like oversalted steak for the rest of your life
Before delving into this, I ‘d like to say that the best thing players can to do when it comes to judging which game they should buy, is to find reviewers that like the games they like, and stay aways as far as they can from sites like Metacritic. I say this because the list of factors that can affect a review is enormous.
First off, you should be extremely wary about day 1 reviews – the probability that something stinks about those reviews is rather high. Just go to metacritic if you’re into the scoring game, look at the critic scores and then look at user scores. Also look at the quantity of reviews on metacritic and check out the dates of the review. For bad games with initially high metacritic ratings, you’ll see a pattern emerging.
There’s a RPG that was released not that long ago and which I haven’t played yet, but the claims that were made about that game in certain reviews are so unbelievable, that I don’t believe a word of what the early day reviewers have been writing. There were even previews of the review, with some of the wording completely inconsistent with previous reviews from the same magazine when it came to RPGs.
As I said I still need to play it, so I’ll find out for myself – but despite having a high meta critic ratings, it has a 6 as a user rating, so that’s probably an indication I won’t like it. If this were booking.com actually, I wouldn’t book it, based on the user reviews.
It was the PR management around that game that got me thinking again about what affects a review. Here’s a few observations:
Big studios, big publishers: As a reviewer, do you dare go against big publisher X who might happen to be the company indirectly paying a large part of your paycheck through ads? A lot of reviewers say they do, but when you look at their scoring behavior, you can see that in a lot of cases – they don’t. There’s often a little scoring bonus I think.
Not that that should surprise you, the recording industry has been doing this for ages. Money just matters, as does size. I got reminded of this the other day when after doing an interview, we got called by the the advertising manager with the question– so how much pages will you book ?
Another problem is that a lot of journalists are working in a stressed environment, typically understaffed or underpaid, with too many games to review and not enough time to review a game. I don’t think that helps them to write a reasonable review, especially if their part of a publication that wants to cover everything.
Then there’s the feeling that’s created around a game. If a big publisher takes you to 5 star hotel , organizes insanely cool things for you to do, shows you the game for a day and then lets you spend their money as you enjoy whatever exotic location for the next 3 days – it’s hard not to feel benevolent.
Personally, I think that if you participated in this it should be mentioned at the start of your review. Environmental conditions always affect how you think about something, just like first impressions.
Another problem is that because a lot of publications are often under financial stress, even the metacritic ones, a lot of reviews for lesser-known games are written by the interns or volunteers. These obviously aren’t used to being bombarded with great proposals, and it’s easy to see that they might be easily impressed.
They also might not be aware of their responsibility. If you’re the unlucky developer who crosses a game reviewer who has opinions that don’t necessarily correspond with the target audience of your game, nor your own for that matter, you might be doomed. I’ve heard of one RPG reviewer who gives a -15% penalty to a RPG that doesn’t have a trade system that fits his taste..
Trends are also a dangerous thing. Early reviews are sometimes completely desynchronized with how the players perceive a game, because a big publication set the tone. This can have a disasterous impact on the developers. If reviewers are stressed for time, it’s imaginable that they might have a look at what others think, and this could lead to an unfair review. Or if the early reviews are part of a well-organized campaign with limited access given to the code, the trend might lead to inflated scores.
The region in which the game was developed and where it’s being reviewed also matters. Games that are overhyped in the US sometimes get smashed to pieces in the Germany, and vice versa. Yet, the taste in games is not that different when it comes to RPGs for instance. Why is that?
I also think that reviewers that really want to be developers shouldn’t review a game – at least not if they’re on metacritic. But that’s just me
I also get very frustrated by the lack of consistency in some of these publications. We actually used to listen to reviewers to adjust our designs, but we gave up, because there’s no consistency. Even not within a specific publication.– they are measuring creativity on a scale, used by everybody in the magazine. That doesn’t mingle well with the subjectivity of a reviewer
I can rant on, but I’ll stop now and perhaps continue in a follow up post.
Here’s one last thought though. Perhaps somebody should do a metacritic of the reviewers individually. After all, a review is always going to be subjective, nothing to be done about that. But people should know of each reviewer how different his opinion is from the rest, and preferably according to target audience.
Anyway, that’s part of what I think on the subject. I’m not against game journalism, on the contrary. I discovered many games because of positive reviews, but I do have a dislike for how the playing field is organized nowadays, and would prefer that particular part of my industry to be extremely objective. It has an enormous influence on the types of games that get played and made. I realize that that’s a futile dream, but I’m an idealist