Balancing Gears Of War With Epic's QA Team

Ever since the first installment, the Gears of War series has had a reputation for being one of the most polished and technically sound franchises in gaming. This didn’t happen by accident, as Epic’s in-house QA department is a key part of the equation. While we were at the studio for our recent cover story, I had a chance to speak with senior QA manager Prince Arrington about the process.

Game Informer: What are some things that set Epic's QA process apart from other developers?

Prince Arrington: The main thing is, there are a lot of studios. I know people in the industry where their test team is just that...they test a game. They have test cases, and they run them. Ours is very organic. We run test cases, and we do the typical things that your average test team does, but there are a lot of things we do that other test teams don't necessarily do. We're in charge of running the playtests, we're in charge of all the performance and memory testing on the game, and the team also gives subjective feedback. We're in charge of all the general bugging.

In short, the answer is that we're more integrated in the process than your average test team. A lot of our testers are artists, programmers, and level designers. They're able to give feedback that your average tester wouldn't be able to do. When we hire people, we make sure we're hiring people that actually bring value to the team, as opposed to someone who can just sit there and play the game for 8 hours. Our interview process is a little more rigid than your average test team. Most people will take you if you’re a warm body. The interview process here at Epic is pretty intense. We make sure we're hiring the best.  I know there are other studios that have very technical testers, but everybody here understands software development, everybody here understands game development. It makes it very easy for us to talk to developers and have them actually buy into what we're saying. My guys actually do know what they're talking about. 

We're cross-training. Everybody's learning how to use the editor. Everybody's learning how to use the tools. Everybody's learning to how to work performance and memory. There's not one person on our team that couldn't go to another studio and basically become a lead there. We're not just a test house where we throw numbers at the game. I'd rather have 30 tests on Gears locally than 100 testers in some other country. It's just, from a value service, we're more valuable. We don't have drones. You see in this industry, a lot of the test teams tend to be drones. That's probably going to get me flamed at some point.

What did you do QA on before Gears?

Last game I did QA for was Summit Strike, I think. Then I was a producer on GRAW and GRAW 2. Then I got the opportunity to come to Epic.

How many testers do you have here at Epic?

In total, 53.

Tell me a little bit about the actual room where most of the testing is done.

It's a big room we call the "hollow," and it has about thirty testers. They're basically set up in rows, and we try to segregate them by discipline. Multiplayer will be here, and the single-player team will be over here. Every day around four, we basically have a free-for-all. Happy fun time, where everybody plays together. Gets everybody involved and fresh eyes on the project, because the single player team typically doesn't look at multiplayer often. So it gives them an opportunity where people that don't play it all the time get fresh eyes on it. It's a good way for them to just kind of unwind and relax. Just doing single player all the time, while it's fun, it's nice to kind of get that release and curse at your friends and talk a lot of smack. It's set up in such a way that the new testers are integrated with the veteran testers so they're learning on the fly and they're having good role models to learn from. When we put people in the room, we put them in there with a purpose. The lead is segregated off from everyone else, but easily accessible for everybody to get to.

When did you start working on QA? Which Gears game?

I was here early in Gears 2. 

After playing so much on the old control scheme, was there a certain adjustment period before you felt comfortable with the new weapons switching system?

It actually wasn't that bad, because I was aware of the planning.  So I knew they were making changes. Initially, it was a little difficult because you'd try to check your objectives and you'd throw grenades. But for the most part, once I realized that I was doing better with the offhand grenades, I picked it up pretty quickly. It wasn't that bad.

OverRun is the marquee multiplayer mode this time around. Does it get tiring to spend so much time on one mode?

We try to make sure that if we don't have fun with it, we're not gonna ship it. A lot of people here play a lot of different games, and we're all fans. If it is fun, we really do feel like we don't wanna stop playtesting it, and then we know we have something. We do that a lot. We play OverRun and we don't want to leave. That's not because there's beer there...it's because the game's actually fun. There are sessions we've played until seven or eight at night, well after the playtest ends at five, just because it's fun.

QA obviously looks into bugs and balancing issues, but how much input do you have on things like the pacing of campaign?

We're involved in everything from fighting bugs to mesh issues to balancing to pacing. All of that. If something doesn't feel right, then we're able to give that subjective feedback. Every time we do a test pass on Horde, or OverRun, we not only supply a list of issues, but we supply a list of subjective feedback. It involves everything, pacing included.

With OverRun being asymmetrical, how do you go about balancing Cogs vs. Locust? 

The big thing is just grinding on the game, right? One of the cool things about Epic is that our test team is deeply integrated into the design process. One of the things I wanted to make sure that we had when I got here was a get a test team that didn’t just find bugs. That's not very valuable. Anybody can do that. But when you have a team that plays the game a lot and loves the game, and a development team that's interested in hearing QA's feedback, that helps them better balance the game. They're not relying on their design skills to balance the game. They're actually taking feedback from people that play the game all the time. They're actually making changes to the map, making changes to the weapon spawns. I know what happens at other studios, but I've never seen it to the point where, when Quinn (DelHoyo, Epic’s multiplayer director) says he's best friends with those guys, he really is. He's down there all the time, and the feedback is really important to him. We playtest the game with the development. We supply feedback to them after the playtests. The next day, the changes will be in there if they're valuable. QA isn't just testers, they're the developers. They actually have an active role in how the game ends up when it gets into the consumer’s hands.

How many hours in a day does a tester have a controller in their hand, actively playing?

Ruling out potty breaks and soda runs, all day. It's all day. They're either playing the game or entering issues into the database. It's all day, every day. That's one of the reasons we do a palette cleanser at four...just so people can kind of relax. That's not the period of time where people are actively bug-hunting. If something comes up, obviously, we stop the presses and call it game-off We'll track down the issue, but the main purpose of that time is the palette cleanser. Eight hours a day of just gaming sounds awesome, and it is. It's an amazing job, but it's a very different beast than when you're actually doing it. It's all day. All day every day.

What's an example of an issue that you had problems with, and through QA, you just kept hammering and hammering on it until it was significantly improved?

Horde, OverRun. You name it...any new game type. Any of the game types we've shipped with have gone through several iterations. I distinctly remember testing Horde into the night until we got it to where we hit that sweet spot. Same with Beast. I played Overrun quite a bit, and from where it started to where it is just now, it’s loads better. That’s because we're doing those continual playtests and they're really, really taking our feedback to heart. Every game mode that we've had has started off and there wasn't necessarily that sweet spot. The one I remember most was Horde because we spent a lot of time trying to make that perfect. Once we instituted Horde 2.0, there were a ton of changes that we thought were cool that ended up not being cool. You know, we always had an issue with OverRun because we were trying to figure out how to make it cool. How to take the best of Horde, take the best of Beast, and kind of integrate them. We tried it several times before and it just didn't work, it didn't play. Through a ton of iteration, months and months of iteration, it's gotten to the point where you get in there and it's a blast. With QA, you play a game that's broken a lot of the time, and if you're only focusing on the bugs, it's hard to find the fun. Having a development team like ours that really, really cares about our input makes it a lot easier. Developing Horde, Beast, and OverRun would have been very different if they didn't take our input.