Honing the Blade: How Focus Groups Influenced Red Steel 2

In March 2007, I attended a focus group commissioned by Ubisoft regarding the first Red Steel game. The title had launched with the Wii in November of the previous year, and the company wanted additional feedback for Red Steel 2 – a game which hadn't been announced at the time, but which Ubisoft was starting to develop. With the game's recent release, I thought it would be interesting to go back to the original article and see if any of the focus group's suggestions made it into the final product.

I've broken out a few specific aspects that came up in the focus group to see how Red Steel 2 handled them (you can read the original article down below). We'll never know whether or not the focus group was instrumental in making changes for the game, as companies like Ubisoft use focus groups as only one part of the equation when developing a title. But the gamers in the group clearly were not alone in identifying which aspects of Red Steel needed to be changed, as can be seen in the final product.

Gore: Almost everyone on the post-Red Steel focus group wanted to see more gore in the sequel. In some ways the first Red Steel advertised a blood bath by having dripping blood on the cover, although the game itself wasn't bloody. Regardless of what our panelists wanted to see, Red Steel 2 hasn't included more bloody gore even if it is still relatively violent; featuring some shocking deaths. Interestingly, the cover of Red Steel 2 no longer has the blood dripping off the sword like the logo from the first game.

More Environmental Interactivity: The first Red Steel was conspicuous for its simple, linear gameplay focus around sword and gunplay, and it's no stretch to imagine that gamers wanted more. One focus group member, Angela, said she liked shooting barrels in the first game, saying, "It's the one thing I have control over that they are not making me do." Red Steel 2 doesn't fully deliver on the request for more environmental interactivity, although there are crates and barrels you can destroy as well as medallions and badges you can collect.

Sword Power: Combining sword swinging with the Wii controller is a prime reason for being for the Wii Remote, and not being able to freely use their sword whenever was a prime complaint for those on the panel. Thankfully, Red Steel 2 developer Ubisoft Paris came to its senses and let players in the game switch between their sword and gun whenever they wanted. Not only that, but the sword has different powers and there are different types of firearms.

More Control: The Wii Remote and nunchuk are infamous for not being responsive enough, hence the creation of the WiiMotion Plus peripheral. Red Steel 2 wisely requires the use of add-on, and while not necessarily one-to-one with your movements, the action onscreen is definitely better than the first game.

Story: It's debatable whether or not Red Steel 2 improves upon the story from the first game like those in the focus group wanted, but at least its post-apocalyptic East-meets-Western setting is an attempt at being more interesting than the tried-and-true Japanese location from the first game. I'd go a step further and say that very few video games in general have good stories, so I guess this franchise isn't alone at missing the mark.

The following story first appeared in the May 2007 issue of Game Informer (#169), and was written less than six months after the launch of the first Red Steel and the Nintendo Wii.

WEIRD SCIENCE

“I want barrels of it.” It’s a late Wednesday afternoon at Greenberg Studios in downtown San Francisco, and 19-year-old gamer Raul (the names of the participants in the focus group have been changed for legal reasons) has just unequivocally described his attitude towards blood in a first-person shooter. Sitting before a moderator and among a group of his video gaming peers, Raul hasn’t been the most vocal member of this post-release focus group for Ubisoft’s Wii title Red Steel, but he is articulate in his opinions. This is exactly what the people behind the proverbial one-way glass wanted. In the end, it’s just one comment of many that the research team will field about the game from forums, reviews, and sessions like this one. In that regard, this focus group is very similar to many others that occur for different companies throughout the industry. It’s neither the beginning nor the end of the game development process, but simply one part along the way.

It’s easy to fear companies misusing focus groups in order to develop games by marketing and demographic fiat, but in our time watching a focus group in action and in our conversations with research experts in the industry, their power is consciously kept in context. “The point of consumer research isn’t to say, ‘This is what you should do,’” explains LucasArts director of consumer insights Sean Denny. “Because consumers are great, but they aren’t game designers.” Denny says that LucasArts often uses focus groups to get a read on specific gameplay mechanics, character design, environments, etc. These one-on-one sessions can start a year out from launch and involve hands-on time with the game or perhaps the showing of a demo video for those times when a playable build isn’t yet ready.

Any focus group hinges on the people you recruit for it. “If you don’t get the right people, you’re wasting your money,” says Denny. This much is made clear as we talked to the Red Steel session moderator, Gary Rudman, president of GTR Consulting. Today he’s going to oversee two back-to-back sessions. The first, which we sat in on, was comprised of people who hadn’t played the game until Ubisoft sent them copies with the instructions that they play it for at least six hours. The second contained those that bought the game on their own.

Not only are potential focus group participants filtered to make sure they meet desired criteria (whether that be casual vs. hardcore gamer, age, gaming history, etc.), but Rudman says that there is an additional vetting process just before the group starts. He likes to ask these potential panel members about something that’s currently pissing them off. “If they can’t communicate one of those things, or their answer is a downward glance with a shrug, or is so obnoxious that they keep interrupting, then those are the people I might not take into the session.” Rudman says he always over-recruits. Today he expects to meet with up nine kids before finding six that are suitable. The others will be paid for their time and dismissed.

Then again, this pre-screening process doesn’t always weed out the weirdos. One Ubisoft research member told us a story about a time when a guy in a Tom Clancy focus group started freaking out that he was seeing ghosts. It turned out that some of the faces of the Ubisoft employees behind the one-way mirror were being illuminated by the blue glow of their LCD laptops and were faintly visible on the other side. Not fazed by this guy claiming he was seeing ghosts, the moderator of the session told him nonchalantly, “Yes, you’re right. We’re here to talk about Ghost Recon.”

Red Steel

Behind the one-way mirror

The moderator is an important part of the process. Publishers go outside to third parties at offsite premises in order to maintain some objectivity. For Denny, he says you can never underestimate the power of the Force. “You bring a Star Wars fan to our office, and it might be a little biased or skewed. They see the statue of Yoda when they walk in and that’s pretty much it. If they knew that George Lucas was behind the screen, they might not say, ‘That’s no good.’ I’d rather hear the bad news than not.”

In Ubisoft’s case, they’ve had Rudman sub-contract a focus group facility on his own, and even have the facility recruit the people for the groups according to the specs that Ubisoft has outlined. Nicki Shovar, Ubisoft’s director of consumer knowledge, says these kinds of small details are important. “Even for us to do the interviewing allows for bias to happen.” Publishers may pay for the services of moderators like Rudman (who will also write up a final report regarding the focus group sessions for Ubisoft), but he is far from a company stooge. He always tells those chosen for the groups he “has nothing to do with the stuff they see today.” “The important thing I’m going to explain to them is that I’m looking for them to be brutally honest,” he says.

As the session starts, it’s clear that Rudman and the researchers at Ubisoft have culled the right people to deliver the honesty they are looking for. The five gamers that have been chosen represent a mix of gamers at different ages, and as a group their conversations throughout the afternoon make the session feel more like a bunch of friends talking about gaming and not as if they are participating in a study.

One of the first topics of conversation Rudman brings up is the Wii itself. The console holds an interesting significance in the proceedings. Due to the way you play games on the Wii, the panel’s opinions of Red Steel are clearly tied to what they think of the system itself. Angela, a 25-year-old admitted Nintendo fan thinks that “anything that gets us closer to the Star Trek Holodeck is cool,” while the aforementioned Raul doesn’t want to move around like a five-year-old. Sixteen-year-old Darrell voices a similar thought. “I actually have to move my lazy ass to play.”

These opinions of the Wii appear to color Red Steel, but Shovar and her associates aren’t sure how relevant they are. They point out that since so few people actually have the system, it’s not only likely that these are hardcore early adopters, but perhaps not even Nintendo knows much about the demographics of the Wii’s owners at this early stage in the platform’s life. Rudman points out that trying to find the right Wii owners to fill two groups – on a Wednesday night even, adds Shovar – is like looking for a needle in a haystack. At the same time, it’s currently no longer the system’s launch period, and the expectations of the gamers like those in the group may have already grown. In her estimation, Shovar believes that a good mix of gamers can’t be found for any title that sells less than a million copies.

As the panel starts discussing Red Steel itself and the opinions are flying fast, it’s clear that Rudman is crucial to the proceedings. Sometimes he pushes for clarification of a point, other times he makes sure the outspoken members of the group don’t drown the others out. Shovar says that there’s always one person who portrays him or herself as an expert (people may even defer to those with a high-falutin’ British accent!), and this time’s not an exception. Mark, at 19, fits this bill. Very specific in his observations, at times Mark would threaten to take over the conversation, but his affability made him non-threatening. Once he even mugged for us behind the mirror, fixing his hair with a comic touch while he knew we were watching.

A possible danger in focus groups, particularly with strong characters like Mark, is “groupthink.” During the course of the session, several times Rudman had the panel write down their one-word impressions of a topic before it was discussed. This helped solidify each person’s opinion before it could possibly be influenced by the discussion of someone else’s ideas. For this panel, groupthink doesn’t seem to be a problem, even for Darrell and Angela, who weren’t as loud as Mark or 26-year-old techie Andy, but whose opinions were respected among their peers nonetheless.

Rudman starts asking them specific questions about Red Steel. What did they think of the setting? Did they use the spare enemy or stop time feature? Does it have the qualities that a shooter should have? No punches are pulled in the ensuing answers, but what’s more interesting is when the gamers express something deeper about their time with the game. Angela liked the seemingly mundane task of shooting barrels because “It’s the one thing I have control over that they are not making me do.” On the other hand, almost everyone in the panel wished there was more blood – not just for gore’s sake, but because they felt it was a touch of realism that the genre demanded. Someone even points out how there’s dripping blood on the game’s cover.

Focus groups testing out gameplay concepts are usually one-on-one sessions videotaped for additional feedback

Fans wanted to be able to access their guns or sword at anytime, and Red Steel 2 lets them do that

Although the group discusses the game’s controls, this aspect of a title is usually handled differently in pre-release focus groups. Sean Denny says LucasArts videotapes these one-on-one sessions in order to provide an extra dimension of feedback. “We can watch their hands on the controller, and we can see their facial expressions, and we actually have them verbalize and talk out loud – which is a little weird at first. It’s that sort of ongoing verbal dialogue we ask people for so we can understand their thought process of, ‘You know, I want to use the Force, but the control scheme is such that I don’t know how to use the Force.’ We need that level of detail so the team can go back and balance to make it as accessible as possible.”

At various times in the Red Steel focus group, the knives come out and the feedback gets negative. Ugly, even. But finding a bunch of gamers to say something bad about a game isn’t hard. What’s more valuable is the panel’s concrete ideas of what would have made the game better. You should be able to use your sword at all times, not just when the game lets you. “Why?” Rudman asks. “Well, you know,” answers Andy, “Why are lightsabers still popular, man?” Raul says it would make him “excited,” while Darrell expands, “It’s like you’re the freakin’ Highlander or something.” A better story, more environmental interactivity, improved controls, and of course, more blood are all targeted as some necessary improvements.

As the session ends, we’re particularly interested to find out what the researchers thought about some of the more negative comments in the session. They were totally unfazed. In fact, during the focus group the team behind the mirror would often laugh out loud at a comment. Rudman is used to this. “Once you go down a negative path, it’s pretty hard to go back because people are paid to come in here and be critical. We always say, ‘When you come out of a focus group session, and you feel like your product doesn’t suck, you’ve done very well, because [this age group] is very critical.’”

In many ways, this has been a typical focus group in that it’s just data that can be interpreted and used in a variety of ways. There are even contradictory statements those in the group have made which have to be reconciled. “You don’t want polling,” says Shovar. “You never want to be counting people in a focus group. ‘Well, three of the guys didn’t like it and three did.’ You are only seeing one group. In order to analyze this, and I hesitate to even use that word because you don’t do that in qualitative [study], you wait until you’ve finished everything and see what are the patterns. What are the things, that after we see it, are: ‘These are things that we can act on.’ You’re never quite sure if you just happened to get a weird bunch of Splinter Cell fans that nobody agrees with in the Splinter Cell community. So you do have to be careful about using focus groups to make ‘either/or’ decisions.”

Shovar tells us that any info taken from this session will be augmented by the company’s play-test lab in Montreal, comments taken from forums, and a wealth of other data as the future of the franchise is contemplated. As we prepare to leave the facility, the Ubisoft team is getting ready for the next session. A buffet of Thai food has been set up and people help themselves as Rudman enters the room and begins to discuss changes to the question sheet the team has prepared in advance to tweak how this next panel is approached. There is talk about adding new questions to see how the focus group’s recommendations for improvements would differ if Red Steel were approached as an action game and not a shooter, but some think this is unnecessary. We reach the lobby and see a dozen or so kids waiting to be vetted by Rudman for the next session, talking about video games as is natural. It’s just another Wednesday night.