NEW ARTICLES
Why You Should Avoid Metacritic

Why You Should Avoid Metacritic

At one time or another I’m sure we have all found ourselves looking at Metacritic. Sometimes we are only looking for some of the most critically acclaimed games for a particular year, another time we may just be looking at past popular games to get an idea of something we may be interested in, and yet another time we may, hopefully not too often, go to Metacritic so that we can use it as a tool for judging the worthiness of a game.

Other than being somewhat of a review aggregator to easily find a bunch of reviews piled together, Metacritic does not have any legitimately good features. Uses to be sure, but not any you should be looking for.

All review aggregator like sites in any medium – places like Rotten Tomatoes- have a problem in just the idea itself. There will always be some kind of distortion in validity or “truth” behind the statistics that will be warped depending on the various aggregator’s approach.

Taking Rotten Tomatoes as an example, movies are rated in a variety of ways, but the one that gets shown the most often is the “fresh” or “rotten” rating. To be given a “fresh” review, some threshold must be met (or it is chosen by staff). A movie may then be “certified fresh” but the average review from critics is only 6.1/10. Misleading to say the least.

Why avoid Metacritic? Well there are two major reasons to avoid it. The first is just in the way that the reviews themselves are chosen and inserted into their formula to give what they call a more “accurate” number to the review. It is far more manipulative, less-subjective, and misleading than Rotten Tomatoes- at least Rotten Tomatoes gives you the actual average review behind the “fresh” one.

So how does Metacritic come to their number? Well first, the publications/sites included in the metascore are limited. No big surprise there, and there is no real issue with some kind of arbitration process to prevent hundreds of unproven sites onto the list. No problem there.

Obviously, then, big sites like IGN will be on there and some smaller sites will be shut out. But this is where I have a problem. Metacritic’s metric/algorithm to reach the metascore is unknown (and they will not share it aside from a brief explanation). But they do tell us one piece of interesting information about how they reach the score: each site/publication is weighted in the score.

Which means that Metacritic is subjectively, not only choosing which reviews are included in their score, but they are also choosing the worth/merit of each review site. How can that lead to any kind of “accuracy?” Just the arrogance of choosing one site’s worth over another is astounding to me. Even if they do it with something as simple as seniority/popularity of said site. Just because IGN is more popular that some other site doesn’t mean that somehow the review they give for a game is worth more than another. That is assuming it is done that way (which it likely isn’t), regardless Metacritic will not say how they weight their scores anyway.

Metacritic has put themselves on a dangerous path. They try to explain themselves by arguing the merits of quantity over quality (their assured quality), and of course take it to the extreme when using examples like having 100s of reviews and the dangers of doing having so many and how having so many makes adding more statistically insignificant.

However, where is the explanation for weighting scores? I see the merit, as said previously, in some kind of arbitration process in choosing which sites to put into the score. That makes sense. But why then give some sites more value over others? There is no real reason to do so. It makes the score so subjective to the point of worthless.

They do offer some explanation for their reasoning behind the reviews and where they are rated:

 For games and music, we work to identify publications that (1) are well-regarded in the industry and are known for quality reviews; (2) actually seem to produce quality reviews (or, if not, are so influential in the industry that they have to be included); and (3) have published a good quantity of reviews.

Aside from the subjectivity, just look at what they have in the parentheses in #2, “So influential in the industry that they have to be included.” That seems like a significant problem to me, as it opens up doors for all kinds of issues in what is included. Attempting to avoid some logical fallacies, could that not point to a big issue? Maybe letting those slip through could be argued as a “margin of error” but considering the potential for harm to a review, especially when it might be a heavily weighted site, is it worth it? Is the potential for reducing the score by as little as a few points be worth it?

Before totally evaluating Metacritic’s worth, just look at subjectivity in general. Subjectivity is a part of all reviews to be sure, but with a single review you still have the opportunity to read the review itself and understand why a reviewer came to their conclusions. But with Metacritic you have absolutely no idea why the score ends up being what it is. In other words, you have no review to read to evaluate for yourself.

Why not then just have each review have equal weight while still choosing which sites to include? Would that really take away from the “quality” of the score that Metacritic gives? Since they seem so worried about quality, they can still ensure it through their process of which sites they included (that of course could be an issue as well, but a far lesser evil than this one). How do they measure prestige or respect in the industry? How do they evaluate a “quality” review over a poor one? Every part of the score that leads to the score itself is ridiculously subjective to the point, as said before, of being worthless.

Metacritic adds absolutely no value to the gaming world, other than, as mentioned before, providing a reasonable list of releases by date.

The second reason to avoid Metacritic is far more sinister. Metacritic has been used by publishers as a kind of measuring stick to determine the success of their games. This made big news two years ago when Obsidian missed out on bonuses due to Fallout: New Vegas missing the Metacritic goal. Think back to the hypothetical earlier, of the potential for a drop in even one or two points. Tell Obsidian that is an insignificant problem.

For some reason, publishers see Metacritic as valuable and popular enough to be influential. Because publishers put so much stock into it, Metacritic becomes incredibly dangerous in how it can affect the gaming industry. Some developers jobs may be on the line, some games may not be made when publishers judge how games in that genre do on Metacritic, and a whole host of issues and ways of evaluation when using Metacritic.

That is of course speculation, but that does not mean that the potential does not exist. And seeing the Fallout example, I don’t think it is too far out there, too much of a stretch, to suggest that publishers are utilizing Metacritic as an objective tool to try and treat making games strictly as a business. Reducing games and their effects/value to numbers.

Metacritic and its metascore is incredibly misleading, and Metacritic has for more influence and effect on the gaming world as a whole than it should.

That could be, maybe not solved, but significantly helped if we all avoided Metacritic – strip it of its influence.

About Andrew Otton

Writing, movies, and games are the main passions of my life. Gaming has been the biggest part for most of my life. I'm a recent History graduate and hope to be a published author someday (novels). For now, I spend a lot of time trying to look at the industry critically. Feel free to reach out to me at anytime to have a discussion.
  • born_naughty

    I look at metacritic regularly. First off all just to see which games have been released. When I’m actually interested in one I don’t look at the score alone but I actually do read some reviews. I’m also very interested in the reviews from the users. Many issues are never really talked about in press reviews. I’m talking about stuff like bugs in Fallout and Battlefield 4. Or issues gamers have with the ending of Mass effect 3. That’s why I weigh in both sides. And when used correctly (reading reviews that give you a feeling you know what the game is about) metacritic can be a great place to find the games you want.

    • http://techraptor.net/author/aotton/ Andrew Otton

      Yeah, that is basically what I said in the article, but I think that is the only use of Metacritic, a place to find links to reviews. However, I would still urge you to use something else like gamerankings because metacritic carries just too much weight in the industry. Gamerankings lacks the user reviews, but I think that most of the things you talked about in the user reviews, like bugs, can be found on any game’s official forums.

      I’m not criticizing you, I just think that metacritic is a detriment to the gaming industry.

      • born_naughty

        I do agree but I don’t see how that applies to how I use it. What you’re saying is basically about review scores. But I don’t really look at that. The games with 90+ scores don’t really interest me. I don’t want to adapt to another way of looking for games because it woukd be very bothersome. What I’m doing now works for me.

        • http://techraptor.net/author/aotton/ Andrew Otton

          I completely understand that way works for you and why you do it. I was just saying that anytime you go there you give metacritic traffic and allowing it as an influential site to continue.

          I would wish nobody used the site, but I can understand why you would.

          • born_naughty

            Yes I know what you mean. But I believe it metacritic is not there, something else will take it’s place. And it’s not just with videogames. IMDb is not much better. They have some kind of weighed average too.
            But you do make valid points. Maybe I should find another way to find games to play. I always try to make a stand and be critical about everything so I should.