Outriders Server Issues, What Went Wrong? People Can Fly Explains
Outriders is out now and while the servers are more stable now, that wasn't exactly the case at launch. With the arrival of the game as an Xbox Game Pass day one title and the interest surrounding what the latest adventure from People Can Fly has to offer, the servers buckled under pressure. The studio did an amazing job at being transparent about any and all issues with detailed updates on what's happened and what's next. Now, People Can Fly is elaborating a little more on what went wrong.
In a recent Reddit post, the studio went into more detail about the issues surrounding the server. We previously covered the "appreciation" package on the way for players as a "thank you" for being patient with the team, but they also added some clarity surrounding the mess to begin with.
In the studio's most recent Reddit post, the team worked all through Easter weekend and "around the clock" to get the servers stable for players eager to jump into Enoch. Furthering their promise to be completely transparent with players, the post details the more technical deviations that caused such a chaotic launch. Instead of giving the standard "the server demand was too much for us" explanation, People Can Fly took it one step further.
"We managed to understand that many server calls were not being managed by RAM but were using an alternative data management method ("swap disk"), which is too slow for the flow of this amount of data," explains PCF. "Once this data queued back too far, the service failed. Understanding why it was not using RAM was our key challenge and we worked with staff across multiple partners to troubleshoot this."
The post continues, saying, "We spent over two days and nights applying numerous changes and improvement attempts: we both doubled the database servers and vertically scaled them by approximately 50% (“scale-up and scale out”). We re-balanced user profiles and inventories to new servers. Subsequent to the scale-up and scale-out, we also increased disk IOPS on all servers by approximately 40%. We also increased the headroom on the database, multiplied the number of shards (not the Anomalous kind) and continued to do all we were able to in order to force data into RAM."
The steps detailed helped to ensure a more resilient database that can withstand heavy pressure, but that still wasn't quite enough for the dev team as far as a solution goes.
The team added, "At this moment in time we are still waiting for a final Root Cause Analysis (RCA) from our partners, but ultimately what really helped resolve the overloading issue was configuring our database cache cleaning, which was being run every 60 seconds. At this frequency the database cache cleaning operation demanded too many resources which in turn led to the above mentioned RAM issues and a snowball effect that resulted in the connectivity issues seen."
As for right now, the database has been reconfigured regarding cache cleanup operations, which has seen the success PCF has been working towards. There is still more work on the way, but for now? At least players can jump into the game and see what all of the fuss is about.
I'm glad everything is more stable now and I'm eager to learn what's next. I couldn't get enough of my time in Enoch, so much so that I'm already mostly complete with my second run through the story. You can learn more about why I had so much fun with Outriders and how it offers some surprising twists and turns with our full review right here, alongside a small blurb seen below:
“Outriders isn’t revolutionary, and it’s not trying to be. Outriders is a well-thought-out shooter with expertly tailored RPG twists. It's an enjoyable ride that is made infinitely better by playing with friends, but not inherently hurt by running solo either. This game takes the best parts from other looter-shooters and the best aspects of sci-fi RPGs and blends these elements into an experience that is worth diving into because of its high replayability, customizable play style, and fun story. I'm already strapping in for another run.”