The issue that I want to discuss today has to deal with one of most anticipated games of 2016: No Man’s Sky. For those not following, it’s been a little bit of a rocky road up to the launch on August 9th, 2016. However, the lead up to this game’s release has been special, as not a lot of detail has left the lips of Hello Games during that time. We’ve gotten teaser trailers sprinkled in here and there, and up to a leak of gameplay about two weeks ago, not many people could say exactly what they should expect out of the game. A lot of claims were made on how epic this game was going to be, but there wasn’t a lot of proof of those claims in the end.
However, some of that changed on August 5th, 2016. Mysteriously, Polygon had started to stream the first hour of the game, and people were wondering why. Well, it turned out that an unnamed New York City vendor had broken the street date for the game. Now, sadly this is less uncommon than you think, as this will happen from time to time with the most anticipated games out there. What was curious here, however, was that it wasn’t small unknown sites or little Billy who just happened to convince a local clerk to grab the game. The big fish of the games journalism pond were getting involved, as not only did Polygon get a copy, but Kotaku ended up doing it as well. Now, that stream was taken down on Polygon, and the following statement was provided by Sony:
Hello Games has a tremendous vision for No Man’s Sky.
Sony Interactive Entertainment understands that some users may have obtained early copies of the game and the only reason we have not sent out review copies yet is due to a significant pre-release patch currently scheduled for Monday, August 8. No Man’s Sky is an ambitious game and the patch is a culmination of the studio’s day-1 aspirations.
Sending out early copies of the game prior to the patch would not be a fair depiction of the game as it’s intended for consumers. For those users who have obtained early copies of No Man’s Sky, their save progress will not carry over to the final game after the universe has been regenerated for launch.
SIE stands by the development studio’s vision for the game. We request that media and consumers respect Hello Games’ efforts, waiting for official review copies and adhering to the global embargo set for Monday, August 8 at 9:00pm PT/12:00am ET Tuesday, August 9.
Now, here’s the thing: what’s clear is that No Man’s Sky has been doing their best to hold back information from getting to the public. What exactly the reason for that is can be debated. Could it be that the game is bad? It’s possible. Could it be what the developer claimed it was: “that they’d worked hard to allow for players to find out the secrets together when the game came out?” Yeah, it’s possible. But one thing was clear with the DMCA take downs of even DISCUSSING the leaks: Sony didn’t want the game to hit the public eye just yet. But with this leak, it’s pretty much become impossible. Once it hits the Internet, it’s out there. It’s somewhere on Pornhub now, even as we speak.
But what’s the problem here? So a company gave a game out earlier than expected to journalists. Who cares. Big Deal. That’s Sony’s problem to deal with it, and you can’t force people to take down the footage. It’s for the greater good of consumers, and given the history of games that over promise things, it’s a good thing for the industry as a whole.
However, I think it’s a big deal for the industry as a whole in a negative way. And I want to go into exactly why I think that is the case, and how I think it hurts consumers in the long run.
Review Embargoes: Their Effect on the Industry (even when there isn’t one involved)
Now let’s get one thing straight off the bat: this specific case doesn’t directly deal with review embargoes. However, it’s something I want to start of with, because it sets the landscape for the rest of this article. Review embargoes do have a purpose in today’s gaming journalism environment. However, they are tools, and tools can be used in a lot of ways. Some of them good, and some of them bad.
For those who don’t know what the term I’m referring to is, a review embargo is an agreed upon controlled release of coverage between a publisher and a reviewer. Basically, a reviewer receives a game with an agreement that no coverage related to the game will be published until a specific date. TechRaptor does receive review copies (properly disclosed of course), and will agree to embargoes on the coverage. Granted, we won’t agree to embargoes that ask for changes in the editorial process (as indicated by our Ethics and Standards page). Now, If you break the embargo, there’s consequences; the foremost being that you probably aren’t going to receive any review copies within the industry in the near future, unless you’re big enough to overcome that slight.
Now, in some cases, embargoes can definitely be anti-consumer. A good example of this was post-launch embargo set on Assassin’s Creed Unity. Considering that the embargo happened 12 hours after the game launched, it helped suppress the information that was available to viewers before they could make a day one purchase decision. With all the technical problems that developed during that time, no one knew of the horrid state that the game would launch in. This was despite there being gameplay footage and “influencer” coverage out there already, which seemed to portray the game in a positive light. This is where review embargos are at their worst: when pre-determined or misleading footage by people outside the company are allowed, but actual critical analysis of the game is squashed until after it may be too late.
However, embargoes do have a lot of positives to go along with them. First of all, embargoes allow for the playing field to be even. It allows for sites like TechRaptor, NicheGamer, and Defunct Games to be able to actually give their opinions without having to rush things and still compete with each other. Without that embargo, one of the things that can happen is a race to coverage. If all outlets receive the game at the same time, the first to publish a review will likely get a majority of the hits, which is important to keep a journalistic outlet afloat (in most cases). This could lead to rushed coverage, and rushed coverage means that there’s a definite possibility that mistakes could be made. That, in turn, may harm the game and may harm the consumer.
When embargoes are used right, it allows those organizations the time to dedicate to the game, being able to cover it at a reasonable pace, and in several cases give the game the due time that it deserves. With more time put into the game, there should be more analysis of the game and more accurate information that develops because of it. Which goes to serving the whole point of what you’re trying to deliver to your audience: the truth. You are able to hand them more information based on more hours with the game and in a more relaxed environment. You may not be forced to rush things in those cases to get them out for the sake of clicks on your website. You deliver a better product because you know more about what you’re talking about.
This is the key element of why review embargoes are important for an industry. Sure, it levels the playing field, but it pales in comparison to the accuracy of information. The conditions in which a review embargo (when used correctly or even semi-correctly) is used should result in better information for ALL readerships. Sure, there’s going to be people who still rush things and make mistakes, but that would happen anyway without the review embargo system. The review embargo at least allows for the conditions to exist, and it’s up to the publications to take advantage of them or not.
But why talk about embargoes in this piece? The fact is there was no embargo agreed to in this case. Copies were sold by a store to the journalists in question, and there was no agreement with the publisher and the journalistic outlet. Even if they were in contact about the review copies and possible embargoes that had been planned, they had not received the copy yet and were by no means obligated to listen to their requests regarding taking the footage down or stream the game.
Well, part of the reason to talk about embargoes is to understand why embargoes exist in the first place—reasons that will definitely come into play during the rest of this article. And as you’re about to find out shortly, there’s consequences in the long term regarding using that street copy, and what it means for the industry as a whole.
Serving your Readers in the Short Term Versus the Long Term
One of the main arguments that I’ve seen in defense of the bought copies has to do with serving your readership. Because the copies are available by some means, it’s a publication’s duty to get a copy and be able to accurately give information to their reader base. By the Society of Professional Journalist guidelines, that seemingly would be the case, as it’d fall under the “Act Independently” segment. The main guideline reads that: “The highest and primary obligation of ethical journalism is to serve the public.” You could justify getting that information to the public and the consumer base that’s looking for any sort of information. Getting that information would outweigh any sort of obligation to the publisher or developer of No Man’s Sky.
@DragnixMod You're misunderstanding. The ethical issue would be if I decided to kowtow to publishers/developers instead of serving readers.
— Jason Schreier (@jasonschreier) August 5, 2016
No laws were broken, and you’re trying to get information to your reader base. You’re looking for the truth. So it’s an open and shut case right?
Not so fast.
There’s one key factor that I think always gets overlooked regarding the SPJ guidelines. And it’s one point that I think comes into play here: “Consider the long-term implications of the extended reach and permanence of publication. Provide updated and more complete information as appropriate.” Now, you’d think that just relates to the coverage specifically for the item being reported on. If you publish something that could damage a person’s career, for example, you really need to consider what lasting effect it could have, and to update that story when circumstances change.
However, I don’t think the guideline is that simple. I think that it applies to everything you do as a journalist, pointing out the long-term affects of what you publish, and what it could mean to the industry and your own reputation. You have to be able to deliver the truth, but if your pursuit of the truth ends up damaging your ability to search for the truth in the future, then you’ve got to question if it’s worth it. Is the risk of getting that piece of information that you’re looking at now worth the possible hit that you take going forward.
I’d argue that for the general public’s sake buying that copy to get that early coverage actually HARMS the information and the primary obligation of ethical journalism in the long run. While there may be a short term benefit with the information specifically for the release of No Man’s Sky, the amount of damage that this behavior encourages could lead to worse coverage for all games in the long run. Now I know that’s a bold claim for me to prove, and I’m going to do my best to show how this could turn into a Pandora’s Box situation. I’m not saying that this one moment in video games journalism ends up turning into a slippery slope of bad coverage, but I am saying it’s taking a step down the path.
As I stated before, some of the big reasons why embargoes and review copies exist have to do with fair coverage. Those reviewing a game with an embargo all get a fair shake at giving their impressions of the game and their experiences to the user. While yes, the embargo is used for other reasons as well, this is one of the reasons why journalistic outlets would even agree to restrictions in terms of coverage in the first place. In this case, there’s nothing that’s restricting the truth being reported on, and you do get the information out sooner rather than the case that you don’t get a review copy.
By buying yourself the copy that wasn’t supposed to be in the hands of the public, the playing field is now skewed. With a lack of reasonable choices that exist in terms of coverage, you now have only one or two voices available opinion wise, as opposed to multiple. You can say that those other publications can buy the copy themselves, but considering where it took place, others would get a jump in terms of that coverage just due to circumstances.
Now one can easily argue that “hey, they got a copy of the game and they didn’t go about doing anything illegal in order to obtain it. It’s not their fault that the game was available and broke the street date. They just capitalized on the situation in question.” To a certain extent, that’s right.
But despite that, I’d like to point out one part of the SPJ guidelines that I feel is relevant for the argument that I’m about to make: “Recognize that legal access to information differs from an ethical justification to publish or broadcast.” Now, I believe that this guideline is mainly referring to cases that deal with public information that may cause problems if released. The prime example has to do with sharing information regarding a victim/suspect’s address and telephone numbers. Yes, you found the information publicly and legally, but sharing the information with the general public is something that’s ethically wrong. In that case, it’s a clear cut case of doxxing, and you shouldn’t be giving out that info.
But in the situation of the broken street date copy, things are a lot more grey. While Kotaku and Polygon both didn’t break the law or any agreements that happened with the vendor, they had undeniable proof that the goods they were buying were tainted. They had to know from their experience in the industry those copies of the game should never have been sold. It’s not like a random stranger walked into the store and bought the game without knowledge that the game shouldn’t have been out yet. Ethically, there is a problem with Kotaku’s and Polygon’s at its core. You’re buying goods that had broken agreements, that’s for sure.
However, the counter-argument is that they’re buying that copy for the greater good. That the return on information that can be given to consumers and readers is higher than the moral dilemma. They’ve slightly dirtied their hands, but they did it for their readership and for the pursuit of the truth. As shown in Mr. Schreier’s earlier tweet, they don’t serve publishers or developers (which I will talk about a little later). They are there for their readers.
Now, in some cases, this is actually the right move from a moral dilemma standpoint, given the right circumstances. You do serve your readership more than anything else. However, I don’t think it is that clear in THIS case.
What’s really important to understand from a ethical standpoint here has to do with the human element of an ethical boundary. There will be times in journalism where you may have to push certain standards in order to determine the truth about a situation. Now, what I’m talking about is not your own ethical guidelines for your journalism site, or even just the basic laws of your country. What I’m talking about is “ethical” elements of real life, and the levels of grey that develop because of the pursuit of the truth. Have no idea what I’m talking about? Well, let me use one of the most famous examples provided by the Society of Professional Journalists to describe exactly where the dilemmas lie.
The example I’m referring to is the 1992 PrimeTime Live report that ABC did on the Food Lion chain of supermarkets. In their investigation, they found that the supermarket chain was willingly selling spoiled meat to customers. This was a story that had the public interest at heart: the well-being of those buying the meat were directly affected health-wise by the decisions made by Food Lion. But here’s the thing, in order to secure part of the information for that report, ABC undercover reporters ended up falsifying employment applications and not actually fulfilling their assigned duties when they got the job. So, basically, they willingly violated elements of the law (regarding being truthful on employment applications, etc.) in order to get the truth behind the situation. That’s where the moral question at the center of journalism lies: at what point does honesty and underhanded tactics get overlooked for the sake of giving the public the information they need? In the case of the ABC News report, it’s actually a debate to this day. There is no “right” side in this case, as both decisions have reasonable consequences to them. It boils down to a basic argument: did the ends justify the means?
Now you’re probably sitting here going: “Shaun, why would you use this as an example of the stakes of ethical journalism? The world of investigation journalism when it comes to health risks versus the world of video games are completely different!” Well, I used it specifically for a couple of reasons. First, it’s one of the best examples that I’ve seen where both sides are justified in terms of an argument for and against the actions that were taken but had a clear beneficial outcome for the general public in the end. The use of the specific tactics can be argued here on whether or not it was justified, because there was nothing stopping the reporters in question (in most cases) of doing the work at the company, in conjunction with the work they were attempting to do for investigation.
It’s also one that had a time element forcing action sooner rather then later, due to the fact that the information may have had more people affected if not obtained sooner. There is a comparison that can be made here with No Man’s Sky, as information that could directly lead to a cancel of a pre-order has a time element to it.
However, let’s now circle back and compare some elements about the Food Lion case versus the street copy situation.
First and foremost: there’s a clear difference of overall stakes between the two cases. Look, gaming is my life and all, but let’s be honest about what the outcomes of breaking the street date of No Man’s Sky. At worse, the information regarding the review will determine whether a lot of individuals buy a 60 dollar game or not. There’s no lives DIRECTLY on the line, despite the fact that some may indicate that’s the case. While yes, the developers and publishers are relying on the sales of their game to keep them afloat, there are other jobs. I hate to say that because it makes it sound like I don’t respect their work, because I respect their work immensely. But, it doesn’t necessarily have the long term impact like the case with the spoiled meat could have. If certain people ate that meat, they’d be dead. Now, it would suck if inaccurate information caused the game to do poorly when in normal circumstances it would do well, and there is an effect on those involved. It’s just no where near the terms of “someone may actually die if they eat this.”
Now, one can easily argue that Kotaku/Polygon’s case is a case of specialized reporting. 90% of the time, they aren’t dealing with situations that are normally going to have the well-being of individuals on the line. Sure, you’ve got your cases of G2A and stolen credit cards and working conditions within the video game industry. But in terms of the subject of what was being reported on here, they’re not as high stakes in terms of a person’s individual well-being. Because the action related to review coverage, the argument regarding the moral elements of the case have to be adjusted. Because Kotaku/Polygon are acting like reviewers and specialized consumer specialists, there’s more emphasis on information and delivering it, compared to a normal news organization.
But here’s the thing. Despite both situations having different goals in terms of reporting, the end goal of both should be the same: the pursuit of the truth. And in particular, the pursuit of the truth for journalism as a whole. That every action you take needs to step toward that, in both the short term and hopefully the long term. This is where I think the two situations vastly differ.
Whenever you face a moral decision in the case of journalism, you have to consider if the line you cross will hurt your reporting as a whole in the future. Not from a “well, they won’t talk to me” kind of situation. But more so from a “will the decisions I made end up making it harder for the truth to be delivered going forward?” In some cases, once you cross a particular line, you can’t uncross it. With the broken street copy case, you have to consider what kind of effects as a leader of the games journalism industry your decision will have. You know full well that a question will be asked by some sites that want to be the biggest in the industry: “If Kotaku and Polygon are doing it, why shouldn’t our publication do it to?” You’ve set some precedent to the decision you’ve made on BOTH sides: yours and the person who broke the street date copy. You can say that you didn’t break the agreement, but you helped support it being broken by justifying the action for your own purposes.
And this leads to one of the biggest issues that I have here. Specifically, that the actions of buying the street copy directly underminded an industry standard that’s used to get the truth to your readership. In the Food Lion case, there was no way to get that information that was wildly agreed on and known to be used to always get the truth. But in the case of No Man’s Sky, that information was still going to be available for your readership if the regular review system was in place … or even if you had to buy the copy on the review day. The truth was still able to reported in the second case, and as I’ll go into later on, it may have been more accurate in the case of No Man’s Sky.
But the conditions between the review copy and the broken street copy are different. The review copy has a set of conditions that at least try to create an environment of accuracy. The embargo system itself may not have been used well in the case of No Man’s Sky, but the system was used—the system being one of the conditions to allow for more accuracy for the entire industry. However, the broken street copy case did not allow for more accuracy. Even if you say that you wouldn’t rush coverage, that’s not the point. The protections of the embargo system can protect journalists against themselves. Sometimes, we make decisions in the heat of the moment, it does happen. The embargo system helps in those scenarios where you’re on the edge of whether or not you think you got everything about the game after your latest play session. If there’s an embargo in place, that you gives you more time to make a wiser decision, to re-evaluate what you know. When you take more time on a review, you get a clearer picture, which can determine whether or not you need to play more or find something else that you can try. You’re willingly giving up that protection in the broken street copy case, even if the embargo is less than ideal. If you willingly give up that protection, you have to accept that it’s possible that you’ll be less accurate because of it. The only thing you gain in the broken street copy case is a time advantage, but does time in this scenario make a difference to the pursuit of truth? No, it doesn’t. The truth and consequences of the truth do not change for YOUR reporting. And if the pursuit of the truth is truly most important here, then you want to choose the conditions where truth is more likely to happen.
If the embargo system is as important as you say it is for the industry (and I believe it is as well), you’ve got to be willing to stand up against the twisting of the current system, when there’s a real cost of course (take the post-launch embargo portion indicated by reviewers such as TotalBiscuit). But even when the embargo system is not directly involved, you have to understand that your decisions could end up affecting the system in different ways. And to me, the case with the broken system date is one of those cases.
The Big Bad Publisher
Another argument that’s at the core of several defenses of the broken street date has to do with the idea of the evil publisher. Publishers in the video game world generally take on the form of the generic villainous and faceless corporation. They are out to make money, regardless of whether the truth or justice stands in the way. They are going to screw you over, they are going to make sure that whatever the truth is—they will bury it like your pet Fido at a the age of 8. OK, maybe I’m being a little over dramatic here, but when you see some of the reactions by journalists in terms of publishers, you can understand where that analogy could come from.
@DragnixMod I think long term our readers are harmed if we allow publishers to walk all over us.
— philko (@pkollar) August 5, 2016
Now let’s be clear: Some publishers ARE nefarious regarding the kinds of restrictions and the level of access that they allow journalists to get. Even when all the rules are followed and no moral boundaries were broken, there are publishers that will imply that they are unwilling to work with you if you don’t bow down to their whims. So the fear of publishers walking all over you? Yeah, that’s a justifiable fear.
My best way of describing my experience with bad publishers: you’re their PR tool. Regardless of what questions you have or what kind of journalism you’re trying to do, they will only answer questions that promote their game in a positive light. Trying to follow up on a set of PC port problems that users are experiencing? Radio silence. Actually want a copy of their hit game? Well, you’d probably do better if you wrote up some pieces that help boost the game up, regardless of whether or not the quality of your work and site speaks for itself.
So not letting a publisher walk all over you? Yeah, I couldn’t agree more. The kind of stuff that was pulled by Sony regarding the take downs of discussion of No Man’s Sky leaked footage? Not the actual leaked footage, but discussion regarding the leaked footage without actually using it? That’s a line that Sony crossed, and Sony should be ashamed for going that route. That’s when you’re being stepped on as a reporter and as a critical reviewer. When your writing and your ability to accurately talk about the information out there gets threatened. That’s a case of “OK, I’ve had enough of your shenanigans. This ends now Sony.”
But on the flip side, that doesn’t mean you don’t work with the publisher as well. Not in terms of a “you scratch my back, and I’ll scratch yours” kind of way. What I’m talking about is an expectation of professional courtesy. Where you’d hope that you can be honest with each other, and that you’re not trying to backstab each other while doing it. That you’re going to cross paths, that you are going to disagree on things, and that’s to be expected when it comes to the media and the subject that they cover, especially when it’s in a negative light. Yes, I’m trying to get a story or get answers, and in the cases of a person’s well-being involved, I will step up and maybe defy you. But when it comes to things that are more day to day operations? Nah, we’re cool, as long as you’re cool with the truth.
Now how does this apply to the broken street copy situation? Well, part of the justification is that if review copies of the game were given in a reasonable time, this wouldn’t have happened. That you can’t expect that reporters, who have had to report on all the small sets of information that you’ve given out piecemeal, won’t jump on an opportunity to get actual answers out there to the users. We’ve seen cases that the hold back of information and coverage ended up causing games to succeed that really shouldn’t have gotten the sales that they got. Assassin’s Creed Unity is probably the biggest offender on that list. And I get that those reporters who love the video game industry and games as a whole would want to do everything in their power to make sure that a case like Assassin’s Creed Unity doesn’t happen again.
However, there is no obligation for a publisher to give ANYONE a review copy. It’s not a right of games journalism publications. Just because a publication is of a certain size or of a certain flavor doesn’t mean that we deserve the right to review any game that is released out there. They could let the product at any point speak for itself and hope that the coverage that comes out of the decision works for them in the process. Bethesda’s handling of Doom is a prime example of that; where the single player gameplay was fantastic, and most everything regarding the game indicated that reviewers would love what the game had to offer. But it was decided that only at the 11th hour would review copies go out. That was Bethesda’s decision, and that may or may not have worked to their benefit in the end. Is it better that review copies exist as a whole? I argue yes: it leads to more balanced coverage and more detailed coverage in the long run while still serving the gaming public’s need for quicker information. But this is something that publishers have offered to help in the process of getting accurate information out there.
To me, you’ve got to choose your battles carefully. And no, I’m not indicating to let the publisher walk on you. I’m not going to change my wording of an article because you want some spin on it. I’m not going to avoid talking about a problem that I saw in the game even if it’s an alpha build. Of course, I’ll mention that it’s an alpha build, but I won’t avoid talking about it. If you’re trying to influence how I talk to my audience: then yes, we’re going to have a problem.
But when it comes to get a copy of a game through a means that isn’t supposed to be exist? Where a moral element was broken in order for it to exist? Isn’t it reasonable to think that hey, if the roles were reversed, you’d want them NOT to go ahead and screw the hard work that you’ve put in, just because some guy in New York decided that he was going to break the date for reasons unknown? If you’re willing to break a trust there, what else are you willing to break in order to get the news first? Yes, you do it for the truth, but again, the truth in this case WOULD get out about the game. Why should I bother with giving you review copies going forward if you’re willing to go through other means just because someone else broke the rules?
Again, the subjects that you report on aren’t your friends. but they aren’t necessarily your enemies either. You’ve got to take everything in stride. And while yes, publishers will screw you in several cases, you’ve got to be willing to accept some ground rules. Basically: “Don’t be a dick.”
Production is Different from the Test Environment
This is one where personal experience in the software engineering world comes into play. I understand that, yes, you’d hope that a game would be completely done and finished, tested, and ready for sale for release. But even in the last moments, developers are adding elements to their game, fixing bugs and maybe even adding 21:9 support for the game, so that they can eke out the last few iterations of the software project plan. QA is basically doing everything they can to try to break the system, just in case they missed something along the way. With a background in Software Engineering, I know the stress of the production release—you’re waiting for everything and anything to go wrong. Being a software project manager for a city wide release of a new transit ticket system, I can say that, yes, you’re pulling your hair out at this point in the process, even if you did all the planning in the world. That you’re adding features at the last minute that you promised, even if you gave your bosses a deadline for that feature that’s nine months from now, and they said they didn’t need it. And now they do.
I'm here tonight adding support for 21:9 monitors. We don't even have one. I've stuck some cardboard to the side of mine ¯_(シ)_/¯
— Sean Murray (@NoMansSky) August 5, 2016
Now, here’s the problem. The environment in which you do testing and software changes should be reasonably different from the one in which the production (aka the consumer focused one) server should be, whether it be tools or additional information needed that could have effects performance-wise on the system. During testing, you’re willing to accept some performance issues if it ends up giving you valuable information that you need to make the system work better in the long run. You may have a debugger attached in the test version, for example, which may help step you through elements of the game in real time and allow triggering of certain elements to cause very specific behavior. That debugger has a cost in terms of performance and requires you to actively monitor it, so it’s not going to be seen in a production version.
Would the retail copy of the game, even if there’s a day one patch on it, have a set of software that has those extra features like a debugger on it? No, of course not, at least on the game disc itself. That’s the big difference between a test release and production release candidates. Both are hopefully versioned differently, but each have a different set of requirements in terms of its intended goal. If the game goes out the door to be sold to a consumer, it better be a production version. Or someone MAJORLY screwed up.
However, as many people are aware, the disc doesn’t necessarily have everything the game will be running. There’s server side functionality, and whether i;ts connected to a test OR production server …. well, that’s a different story. The game that you are playing on the disc should be a production version, but that doesn’t necessarily mean it’s not talking to a test version of the server side functionality. In fact, there’s legitimate reasons why the two would be different at times. Maybe you want to use the production version of one to act a control, so that you can focus on the test version of the other in terms of problems. That you know how the production one SHOULD act, given a specific version, and attaching it to a test version on the other side may give you some valuable experimental information. Now, the general public shouldn’t have a way to see that test version of the server either, but it’s a whole difference process there.
Playing a test version of part of the system without knowing it may lead to some bad consequences for a critic’s review of the game. They won’t know about the tools—they won’t be available to him—but they could end up with unexpected outputs that could cause bugs or even crashes. In addition, it could easily lead to performance issues, which would be acceptable in an environment where you’re developing the software but not for the consumer who’s playing an expected “perfect” copy. Sure, you may get some information about the base elements of the game and what the game has to offer from a feature set perspective, but the devil is always in the details, and those details are definitely going to be different in both the production and a test environment.
If you are reviewing/playing our game without our update, on a leaked copy, then please don't. It's not what players will experience.
— Sean Murray (@NoMansSky) August 6, 2016
So, could the copies between production and the test environments be drastically different? Well, it’ll be different, but It shouldn’t be TOO different in the end. After all, if you’re testing the game’s main mechanics and what not, you’ve got to be testing something that’s representing what the real scenarios are going to be. But it’s not entirely the same, especially depending on your feature set.
In the case of No Man’s Sky, they’ve specifically indicated that those who played the game before the upcoming 1.03 update will not get the choice on the three unique paths that you can follow in the game. Now you’re probably wondering, well hold on, how did that not exist before hand? Well, it probably did in a different form. If I was a smart developer, I would extract that layer of events and choices out of the main engine of the game and make it so that I can just drop in that element at any time via a configuration or data file. The series of eventsthat’s present now is probably one that hits all elements in testing: hitting a wide array of procedural generation for example.
Considering how detailed games have become now a days, any small details that are different could make a huge difference in the enjoyment of the game going forward. With frame rates and network connectivity being huge in terms of the PC gaming world, for example, any of those aforementioned tools like the debugger that have a purpose during the development will fall to the wayside when it comes to the real release. When you’ve got network capabilities that you’re probably testing up to now, if someone ends up getting a copy and playing on that environment, that could lead to a significantly different experience. Maybe longer loading times. Maybe elements that don’t work right now, due to some specialized testing going on. But here’s the thing, it’s going to be different, even if there’s no day one patch. And because you don’t have the information to know how different it’s going to be are you able to actually give accurate information to your readership in that case?
There’s a reason why you wait for the developer/publisher to give the go ahead on when you can start playing their game. Because despite all you think you know, they are the ones who know best in the end. In fact, No Man Sky’s case, that was already proven. It took the developer indicating that a day one patch that had significant functionality in order for consumers/critics to realize that the broken street date copies wouldn’t be representative of what the final product was. If that information hadn’t been released, it’s quite possible that publications may have easily misled readership by providing their reviews and impressions before that patch. Sure, that information may have been useful to those who didn’t have an Internet connection in the game, but a majority of your users (considering you’re an online site), are going to get that update. Even without the functionality that was mentioned, any sort of software update can lead to a different experience, especially when it comes to performance. So you need to give your most accurate representation of what the release day candidate could be.
Caught in the Crossfire: The Developer
And here’s where my personal tie in that I eluded to earlier comes into play. I have a history of being a software engineer. Not as a game developer, but as a developer who worked on public systems for transportation. I’m not going to say I’m an expert of any sort, but I’ve seen multiple layers of the process—from the testing, to QA, right down to the design process. And of course, actually coding along the way. I know the struggles of deadlines, I know the hard work it takes to make a good product, and I know how decisions beyond your control can really screw with that on a regular basis. I also know how at every step of the way, that work can be threatened by bad decisions and people who are less concerned about being truthful or even giving your product a chance. Regardless of what you think about big publishers, this is a special situation regarding No Man’s Sky. This is one of the biggest “indie” games that is coming out. The actual studio in question has less than 20 people employed.
I don’t care whether or not you think that it’s just a job or not, but when you’ve got a project that you’re working on, you want to tell everyone about the things you’re doing. You know perfectly well you can’t, whether you’d be leaking industry secrets or end up putting your foot in your mouth when it doesn’t get into the final version. You do your best to hold back your excitement, planning things in detail, and hoping that nothing in the process goes wrong. All those workers who you see fly by in the credits while you pretend to actually pay attention, many of them worked their butt off to make this game happen. They are the hard workers who’ve put their life into this work. You’d hope that people would respect your wishes, respect the decisions you made. And while you understand that yes, some people may not understand or may be critical of the decisions you made, there was a reason for it in the end. You could think those reasons were nefarious, and if you find evidence of that, then call them out and investigate. Call out the fact that you’re getting review copies really late, and QUESTION why they made that decision. Discuss why that was a bad decision and ask them to respond. But in the end, respect that they made the decision that they did, and don’t willingly go against that decision because someone out there in New York decided to break an agreement.
I can understand press calling out games that don't send preview copies, but I am sincerely disappointed at leaked, pre-release streams.
— Rami Ismail (@tha_rami) August 6, 2016
You’ve disrespected the work that people have put in. The developers that have toiled to bring the work of art that many people say that video games are now a days. You’ve disrespected their wishes and have now caused them massive panic they’ve got to deal with at the 11th hour. For what reason? The truth that could have easily been obtained when the game came out. Now note, I’m assuming the truth is the reason considering that’s the bread and butter of real journalism, and there’s always going to be people who put the clicks and fear-mongering over anything else. But if this is what the goal of your decision behind the buying of street copies before the release is, realize that you’ve caused damage that you really can’t take back.
This is a game that’s been worked on for YEARS. That has been planned out down to the last detail of the box art. A plan that has been a long time developing that you helped break in an instant, with a decision that doesn’t really stand up against the weight of the work being presented. For example, think about the backlash that Titan Souls had when it launched. In one instant, thanks to a comment by the game’s artist, many people wrote off the game because of his treatment of TotalBiscuit. Did it have anything to do with the quality of the game? No, it didn’t, but because of one comment, it most definitely had an impact on sales. We’ve seen masterpieces that end up failing not on technical merit, but by one decision that may have overshadowed the actual product. This one decision could have led to that, as any inaccurate information, especially if its the first to get out there, can spread like wildfire. And people then don’t go looking for a second opinion.
With the lack of gameplay footage out there and vagueness about how the mechanics work, it’s easy to believe that Sony was promising a lot more than they could deliver. We’ve seen constant incidents in the past where a game like Spore promised so much and under-delivered. We can take the time and dissect the decisions after the game comes out, pointing out what possibly went wrong and how we bought into the hype train. We can express caution to our readership in pre-ordering a game with that little information, and wait for those who get a hold of the game and have the trust of their readership to give the information to make a consumer purchase decision. And we can do our best to get the answers that our readers need without breaking moral boundaries along the way.More About This Game