|"" It is not sufficient to simply test the game itself; is equally important to test the quality of the relationship between the player and the game""|
With traditional Quality Assurance, testers search for bugs and problem within the game, of which there are thousands; Borderlands for example, had over 35,000 bugs found during its QA process. With the hundreds of millions of pounds spent on games today, it is unsurprising that many developers will go beyond the usual bug-finding expedition and will also seek to find more qualitative ways of improving their game. This usually entails asking testers what they thought of various parts of the pre-release game, though there are several problems inherent in this method. Most notably, the tester’s inability to objectively or even accurately convey what he was thinking is frustrating, especially as he himself may not even know which parts he was enjoying or he may not be forthcoming about elements of the game that elicited a fear or confusion response for psychosocial reasons.
UX and usability testing is a relatively old field of research, which has been applied to testing the effectiveness of UCD (User Center Design) in websites for over a decade. It is only in recent years that the video games industry has looked to this type of research to better the quality of its games.
Instead of relying solely upon direct oral feedback from testers, some UX companies utilize technology such as biometrics and other Human-Computer Interaction (HCI) measurements in order to biologically measure the effects that games have upon their players. Using sensors that record responses such as Galvanic Skin Response (GSR), pulse rate, eye-tracking and brain-waves (EEG), researchers can see exactly what exact effects each specific moment in a game is having upon the player. Further analysis of the bio-feedback is performed using interviews with the participant and other psychological methods.
One example of this can be seen in a case study presented on 8-9 April 2010 in Tampere, Finland by Vertical Slice (a British UX research company), in which, the responses from participants playing game segments from Haze and Modern Warfare 2 were compared. Two very similar sections from each game were used; one featuring a vehicle and another during one of each game’s many gun-fight set-pieces. Although the games themselves were obviously bug-free and presumably fairly close to the idea the developers had set out to achieve, the responses from the players were significantly different.
The vehicle section in Haze provoked a strong reading of confusion and despite an initial sense of excitement in both of the Haze scenes, the responses soon stagnated and gave no particular indication of interest or intrigue. In contrast, despite the superficial and visual similarities, the Modern Warfare 2 vehicle section (the snowmobile scene) maintained a level of excitement almost throughout, only dipping in the middle section. Some degree of confusion was noticed when the player mounted the vehicles, and during the gun reloading sequences in the gunfight, some players’ readings showed that they were somewhat frustrated (especially with guns like the M240).
This specific level of objective measurement of game quality is unprecedented in the game industry and affords the developer far greater vision of the game’s strengths and shortcomings, both in the explicit moment and in a more general sense.
Further applications of UX testing can be seen in the studies and subsequent paper published by academics at Canadian universities in the Behaviour and IT journal. In this, Mandryk et al analyse the relationship between difficulty and enjoyment, measuring the GSR and heart rate of players on different levels of challenge in EA Sport’s NHL series.
Also of interest in this paper, are the different responses measured when players were playing against a friend or when competing against the AI. The results denote that players experience far greater feelings of joy when scoring against another person than when playing against computer-controlled teams. This perhaps goes some way to explaining the success of online play in recent years – though again it should be noted that online play provokes far weaker joy responses from players than same-room multiplayer experiences.
In summary, UX is a buzz word in the industry for a reason. Microsoft and Sony already have large in-house UX teams and publishers and developers that do not use the process are finding out the hard way that their games are not good enough. It is not sufficient to simply test the game itself; it is equally important to test the quality of the relationship between the player and the game.