EEDAR dishes data on achievements, new IP

Who was there: Electronic Entertainment Design and Research was represented by president Geoffrey Zatkin (a member of the original EverQuest design team) and VP of analyst services Jesse Divnich, who spoke about gaming trends in intellectual property, as well as what they've learned from sifting through Xbox Live achievement data.

What they talked about: In one Thursday presentation at the Game Developers Conference, former EA/Maxis developer Chris Hecker grappled with reward systems like Xbox Live achievements and whether or not they might ultimately take the fun out of playing games. EEDAR's Zatkin and Divnich have been likewise preoccupied with achievements, but in a very different way.

But before tackling the achievement issue, Zatkin dove into new intellectual property trends in games. EEDAR data shows that the percentage of games establishing an original IP has actually been on the rise since 2006, up from 16 percent to 22 percent. However, that growth is due mostly to the Wii, which in 2009 had 27 percent of its releases count as new IPs, Zatkin said. For the Xbox 360 and PlayStation 3, the number in 2009 was only 17 percent.

Zatkin attributed that imbalance to the "game-changing technology" of the Wii's motion-sensing controller. He also told the audience to expect a surge in new IPs for the Xbox 360 and PS3 once Project Natal and the Move controller come out.

New IPs were particularly prevalent in the role-playing game, action, shooter, and general entertainment genres, Zatkin said. Since 2005, about 38 percent of role-playing games have been based on new IP, while only 8 percent of fighting games in that span were based on new IP.

Zatkin also broke down the disparities of new IP and old IP based on Mature titles. On the Xbox 360 and PS3, 47 percent of new IP were rated M for Mature. For the Wii, only 2 percent of new IP received an M rating.

Zatkin then showed the top five publishers of new IP on each platform. On the PS3, it was Sony, EA, Codemasters, Sega, and Ubisoft. On the Xbox 360, the top five were EA, Microsoft, SouthPeak, Codemasters, and Sega. For the Wii, it was Destineer, Ubisoft, Zoo Games, Konami, and Activision. Zatkin said Destineer had published more than 50 new IPs for the system on its own.

Then Zatkin turned to review scores and pointed out that new IP review scores were trending a little bit lower on all systems. There were some exceptions, as new IPs in action games were as successful or more with reviewers, while new IP in the sports and strategy genres did considerably worse than their established counterparts.

Divnich then took over the talk to present EEDAR's findings on achievement data. Working with mygamercard.net and Microsoft, Divnich said he was able to mine about 32 million data points on Xbox Live achievements relating to a random sample of 100 different Xbox 360 games.

EEDAR broke down achievements into 16 different categories, like completion (beat a level) or exploration (find the skulls in Halo 3). The group found that only 4 percent of gamers on any given game were getting all of the achievements on their games. With AAA titles, that number drops to 2 percent. On average, only 27 percent of players managed to get half of the available achievements in each game. Divnich suggested that with numbers that low, most games' achievements are probably a little too hard.

Divnich posed the question of whether or not achievements actually motivate gamers to play the game. He said they did, but only at the very beginning and the very end of gameplay. He likened it to a runner trying to finish a marathon. Even if runners get off to a bad start and know right away they aren't going to finish the race, they're unlikely to quit so soon after starting. And when they're in the home stretch (80 percent or more of the achievements completed), seeing the finish line motivates them to tough it out, no matter how much of a struggle it is.

Achievements aren't just a motivator, Divnich said. They're also a monitor, as demonstrated by a Microsoft presentation earlier this week. Zatkin emphasized that achievements could be used as a feedback mechanism. For example, if there are achievements given for exploration, combat, and driving in a certain game, the developers can use the data to infer what players enjoyed doing most and allow that to guide their decisions on a sequel.

When asked specifically about Chris Hecker's talk from the previous day, Zatkin suggested achievements were unlikely to be a de-motivator in most cases, with one caveat. He said that role-playing gamers had been trained for years that once they'd finished the story, maxed out their levels, and filled every progress bar they could, the game was over. So if a game had achievements that were too easy to obtain, he acknowledged that could signal to players that the game was effectively done and serve as a de-motivator.

Quote: "Achievements are a reward. People use reward mechanisms in games to get players to do what you want them to do. If there were no rewards for collecting coins in Super Mario, you wouldn't do it. Achievements can be a very powerful tool that has a very low impact on a development budget."--Zatkin, offering advice to game makers.

Takeaway: Sales aren't the only data for publishers and developers to consider when it's time to make their next game. They can get a different perspective on key decisions by mining data to unearth trends across a wide spectrum of categories, including achievements and intellectual property trends.