Vlambeer’s Rami Ismail Speaks Out About Mysterious Console Certification Process

"Anybody arguing that a game should be done when it goes ‘gold’ is living in the 90s," writes Vlambeer's Rami Ismail. With No Man's Sky days from release, the definition of the word "final" has been thrown into disarray, as conversation turned heated over on-disc content versus the hefty day-zero patch. In response to the controversy surrounding Hello Games' upcoming space adventure, Ismail wrote a lengthy blog post that sheds light on the behind closed doors process of console certification, giving insight into Hello Games' circumstances with launching the game.

Over the past week, at least one retailer broke the No Man's Sky street date, selling copies to eager fans before its official launch. To make matters worse, review builds aren't being sent out to the media until tomorrow, the day before launch, because the game was deemed unfinished by the development team. Now, a massive pre-launch patch is hitting with big changes. Questions have arose, from critics and the public alike: What can cause a situation like this? Who is at fault? These are difficult inquiries to answer, but Rami Ismail of the Dutch studio Vlambeer hopes his experiences and point of view can help clear things up. 

In the post, he begins by clarifying that he is in no way affiliated with No Man's Sky, and that the discussion he wishes to have, while it pertains to Hello Games' difficulties, is not necessarily specific to this one game. Ismail offers a perspective on Day One patches on a whole, and that he often must keep the details vague for various reasons. "You also need to realize a lot of what I’m about to discuss cannot be discussed by a developer during the development process for various reasons, including legal contracts we have to sign to be allowed to release a game," he writes. "This is also the reason I have to be vague about the details, and each of my statements is not about any specific console manufacturer but sort of about all of them and none of them at once."

He points out that console games come with an expectation of quality, and that studios who develop for the platforms are expected to meet a minimum threshold. However, developers are creative people, meaning that its a rule of thumb to believe that they will "always be two weeks late for the deadline." For those that make console games, Ismail points out a stern reminder that both platforms and systems surrounding the bureaucratic side of development are not necessarily "built for 2016." These are systems that were put in place years ago. This means that sometimes, the systems used by developers can be unrealistically complicated. He writes, "It’s like the system you are or were forced to use at your day job, if you’ve ever worked in retail, stock or booking systems, or at any checkout."

Ismail describes the most "egregious" example of this to be the process of certification, often nicknamed as "cert" in the industry. It's a "'Seal of Quality' mindset," meaning that there are certain checkboxes that must be filled in order for a console game to be approved for launch. This could be anything from whether the game is stable to withstand crashes, or pertain more to legal concerns or bug fixes. Ismail states that some are more absurd requests, such as, "if you rapidly plug and unplug the controller, does the game know what to do?" All of these fixes need to be sorted and checked off in an orderly fashion via a submission form. From there, you book an appointment to have your game tested, often just a few days later, to see if it meets these demands. "If any of them exceed a certain level, your submission fails, and you have to start from square one and fill out the form, find a free day to book a slot or wait until certification starts, and submit a new build," he writes.

Firmware updates can be problematic, and at times "impossible" to solve, because developers don't have a hand in when a console decides to update its firmware. Cases such as these can allow developers to be exempt from. Ismail, however, believes that the certification process is poorly documented, which can be another headache of its own. "One console has a field that says ‘assets file’. It doesn’t mention what the assets file is, nor what it does, or what these assets are. If you don’t add the file, it can’t process your submission. If you add it, but it isn’t ‘right’, your build can fail." 

This is all to say that it can be easy to lose a week, or even a month, towards attempting to meet that seal of quality. Ismail is quick to say that this may or may not be the case for No Man's Sky, but "developers with a launch date need to make sure they can hit that launch date." Certification begins at an early stage, in hopes a launch date can still be met, but because of its overly complicated and technical nature, problems may arise.

Game development is often long and laborious, and certification is an added hoop to jump through. "So you submit it, it goes through cert, and you stage it for download and launch. For disc games, the game needs to go through certification in time to be printed, boxed and distributed. At that point, developers are usually still one to three months from release, which tends to mean you need one and a half to three and a half months to get the game done, and then you still need to keep in mind that unpredictable amount of time you’ll spend in certification," he writes. Even with day one patches, those also need to first make their way through certification before they can be revealed publicly. 

This can explain why these kind of patches can seem large and game changing. By the time a game launches, the 'gold' build can already be a few months old. With No Man's Sky, the public was able to see what was offered in-game before that day one patch hit, which is a rare opportunity and a potentially damaging one for the developers. Are most games lacking or facing tremendous bugs before a day one patch? We don't know, but it's likely that No Man's Sky is far from alone.

"Developers care about the games they make, and we’re trying to make the best game we can for our players," writes Ismail. "We’ll take every opportunity we can get for that."

You can read Rami Ismail's full blog post here.

[Source: Rami Ismail (Blog), Photo courtesy of develop-online.net]

Our Take
Ismail gives an incredibly fascinating take on the process of console certification; a topic that isn't often spoken about so candidly, even if he had to be vague and not point the finger at specific console manufacturers or publishers. As I mention above, it's entirely possible that a lot of games have issues before that day one patch hits, but often we don't know about it. Understanding this process is eye opening, and can help us empathize with developers in these kind of situations.