Gaming Industry Reacts To Steam's Review Overhaul

This article, which was written in 2016, goes into detail regarding the then-current Steam review overhaul and its impact on developers.


Published: September 14, 2016 9:30 PM /

By:


The Steam logo against a backdrop of many of the games available on the service

 

Earlier this week, Valve announced and implemented changes to their reviews system that were designed to combat developers propping up their games' ratings with free key giveaways. This is the second update to the system this year, after the changes in May that made a clear distinction between paying customers and reviewers of all stripes with an added disclaimer.

Since the announcement, there has been much talk regarding the unintended repercussions of the new changes. In addition to those who work with Steam groups and developers to review games, this update obscures reviews written by all manner of key activations. This includes Kickstarter backers, Humble Bundle customers, and even professional and amateur reviewers who choose to advertise their content within the system. Several developers have already posted updates to their game pages urging their fans to support indie games of all stripes with reviews, displaying a real sense of camaraderie among the development community.

YouTuber SirTapTap has already written a great post on their blog highlighting some of these problems with the new system and how they interact with Steam's ratings and store placement. Their commentary may come from a place of passion, but the information laid out within is solid.

I personally reached out to a number of developers and publishers myself, and their opinions about the changes were mixed.

vidar-ice-cave

Dean Ravazi (who you may know as the developer of Kickstarter success Vidar) found these changes had come at the worst time, as he has just recently begun the process of distributing his game to his backers.

A lot has been written about crowdfunding and games, but one consistent theme is that crowdfunding is a way to build a community around your game, and that you need to do this before you can successfully crowdfund as well. And I was thrilled to do that. I worked my butt off to try to find people who I thought would like the game (because that's what marketing is all about, finding your audience), and getting them excited about Vidar. These are people that, I should hope, would buy Vidar on Steam had I not Kickstarted. But I looked to them to help crowdfund and spread the word about Vidar early, and now because of that, I'll be penalized. It's hard to explain, but in short, my early outreach to the people I believed would be interested in Vidar means that the people most likely to give it a positive review - the people the game is designed for - are cut out, and so my reviews will be skewed as a result.

Here's another way to think about it: reviews are generally only left by people who really liked the game, or really disliked it. Because I crowdfunded, I've already made sure that a bucket of people who really liked the game (I hope!) are now excluded. 

Dean goes on to acknowledge that the rule applies to every developer, which ensures that "on some level we'll still be equal." He then makes a pretty valid point regarding the blog post that announced the new rules.

According to their own article, this was to resolve a type of fraud that they identified in roughly 160 games. Out of how many, tens of thousands? It strikes me that this is a very overbroad way to fix that kind of fraud. It also strikes me that if they had a way to identify the 160, then why not just fix those 160, rather than apply this rule to everyone?

march-of-the-living-night

Dave Toulouse is the developer of zombie roguelike March of the Living, a game that currently sits at an 80% Very Positive ranking on the Steam store. While admitting that he wasn't hit hard by Valve's changes, he still has some concerns regarding their widespread effects.
 
Today I just lost 26 legit positive reviews because of this change. This might not seem a lot but out of 389 reviews this matter a lot. The slightest change in reviews can have a dramatic impact on devs with modest success (my game isn't a hit but made me a salary for 2016 so it's kinda neat). This change will mostly hurt small indie teams that were doing "just okay" while the hits will barely see any impact as they have thousands of reviews anyway.
 
As a dev I'm not concerned if someone else try to cheat the system by giving away free copies as it doesn't affect my own score. What I'm concerned about is when Valve influence my own reviews when trying to "fix" the situation.
 
The result right now is that even if the score of my game is "very positive", the first 4 reviews displayed on my game's page are 4 negative reviews. This is showing a distorted picture of the real appreciation of the game and at the same time hurts my single source of income to make a living.
sombrero
 
 Nick Robalik of PixelMetal is the developer of Sombrero, a game we've been covering since its early days. He admit's that Valve's policy is overzealous, but questions the usefulness of the whole endeavor at the same time.
It means that only people who paid for the game can review it. I don't really see to many issues with that. I understand it may cause some issues for those who buy steam keys from 3rd parties, such as a Humble bundle, but overall I don't see many issues with the new requirements.
 
It may be a bit of a scorched earth policy in regards to reviews, but if it helps end or reduce false, paid-for reviews, I'm okay with it.
 
People really shouldn't be paying attention to only aggregate scores anyway. The best resource for honest reviews is always going to be people you know. Getting info about the quality of a game should be coming from more than just a bunch of randoms on the internet.
AVGN II Combat
 
Sam Beddoes is the man behind FreakZone Games, best known for the Angry Video Game Nerd Adventures duology as well as a retro gaming tribute to Manos: The Hands of Fate. 
Well, I guess it's good that people can't bribe people with Steam keys now! I value my customers from other stores like Humble but, they can review on those stores. This prevents devs faking good scores through payments, bribery with free keys, leaving their own reviews etc.
Party Hard 1
 
Alex Nichiporchik is the CEO of TinyBuild, a notable publisher of indie games on Steam like the pictured Party Hard. He has spoken out in the past about how unchecked Steam key distribution affects the gaming ecosystem.
I think it's a really good step forward. Outside of the abuse issues, it's important that people leaving the reviews have bought the game at the properly set price by the Steam Partner, and not potentially an unfair price via shady key resellers. 
 
The issue is perception of value -- when you pay full price for something, odds are you will spend the time to properly evaluate it. Whereas if you buy games in bulk for very low price, you might not appreciate each one on the same level. 
 
I'm digging it. 
I asked Alex about what he thought about customers who pay for TinyBuild games outside of the main Steam store and the fact that their ability to review games is also affected.
That's the trade-off people make if they buy games from non-Steam stores, and in a way it makes sense to me.
 
If I buy something in a store, why should I be able to review it on Amazon? 
incredible baron gameplay
 
Raghav Mathur is one of the people behind Black Shell Media, a prolific Steam publisher who regularly utilizes bundles to promote games such as Zombie Party and The Incredible Baron.
I think Steam is making a push to avoid some of the controversy that has arisen in recent years regarding ethics in the games industry -- ethical journalism, scummy business practices, aggressively bundling games and more. The new system is certainly a step to mitigate these issues. Whether or not that step is in the right direction we shall see. The new system definitely benefits more niche titles that have a harder time getting picked up by major bundles and promotions, as those players who actively seek out the game and buy it on Steam will be the most dedicated fans. However for games with mainstream potential with a smaller number of reviews this definitely hurts, as often times bundle purchasers will nit-pick if the game in question isn't a favorite of theirs. The filtering options definitely help a ton and are a good compromise between what Valve envisioned for this update and what developers are used to.
Gratuitous Space Battles 2 Space Ship
 
Positech Games' Cliff Harris brings up a good point regarding one of the big problems with having a reviews system that is populated by customers. There are people who will take the time to analyze the game properly, but others will just run into a problem with the game and use the storefront as a place to vent.
I think the changes sound ok. Although TBH, for me the biggest complaint I have with the review system is not that people get to leave reviews if they bought the game elsewhere, but the fact that so many gamers seem to treat it like a tech support forum. They run the game on their recompiled copy of Red hat Linux 0.1754652 running on a modified  overclocked toaster, load it with 500 mods, then if the game crashes, they leave a 'game crashes 0/10' review rather than contacting the developer. That gets old really quick.
avadon-3-dragon
 
I'll give Jeff Vogel of Spiderweb Software the final word. Jeff has been in the industry since 1994, writing games such as Nethergate: Resurrection, the Exile/Avernum series, the Geneforge series, and the just released Avadon 3. I think his points represent a swath of developers who are caught in the middle of this latest controversy and would much rather get back to serving their audience what they want.
In a sense, I’m the wrong person to ask. I never look at my Steam reviews. Why would I? Online review systems, even at their best and most carefully tended-do, are weird, unpredictable, full of negativity, and uncontrollable by me.

Also, I, like most indie devs, write niche games. Niche games, by definition, will have a lot of people who don’t like them. This will be reflected in reviews. I get suspicious if an indie game’s reviews are too good. A game like Darkest Dungeon is really good, but you should expect most gamers to not like it. The specific goal of the project, according to its devs, was to create a game for very specialized tastes.

I can’t say for sure, but I think the current freak-out (the extent that their is one) is just a reflection of the fear of indie devs about the gruesome competitiveness of their industry. We, as a rule, are very easy to panic right now. If you write a great game, it is sellable without perfect control over your Steam reviews. If you write games nobody wants, Steam reviews will never save you.

Thanks again to every developer and publisher who got back to me for inclusion in this article.
 
What are your thoughts about Steam's review system changes? Let us know in the comments below!

Have a tip, or want to point out something we missed? Leave a Comment or e-mail us at tips@techraptor.net


Alex Santa Maria TechRaptor
| Staff Writer

Alex Santa Maria is TechRaptor's former Reviews Editor (2015-2020) and current occasional critic. Joining the site early in its life, Alex grew the review… More about Alex