Exploring the Effects of Higher-Fidelity Display and Interaction for Virtual Reality Games

In recent years, consumers have witnessed a technological revolution that has delivered more-realistic experiences in their own homes. Expanding technologies have provided larger displays with higher resolutions, faster refresh rates, and stereoscopic capabilities. These advances have increased the level of display fidelity—the objective degree of exactness with which real-world sensory stimuli are reproduced by a display system. Similarly, the latest generation of video game systems (e.g., Nintendo Wii and Xbox Kinect) with their natural, gesture-based interactions have delivered increased levels of interaction fidelity—the objective degree of exactness with which real-world interactions can be reproduced in an interactive system. Though this technological revolution has provided more realistic experiences, it is not completely clear how increased display fidelity and interaction fidelity impact the user experience because the effects of increasing fidelity to the real world have not been empirically established.

The goal of this dissertation is to provide a better understanding of the effects of both display fidelity and interaction fidelity on the user experience. For the context of our research, we chose virtual reality (VR) games because immersive VR allows for high levels of fidelity to be achieved while games usually involve complex, performance-intensive tasks. In regard to the user experience, we were concerned with objective performance metrics and subjective responses such as presence, engagement, perceived usability, and overall preferences.

We conducted five systematically controlled studies that evaluated display and interaction fidelity at contrasting levels in order to gain a better understanding of their effects. In our first study, which involved a 3D object manipulation game within a three-sided CAVE, we found that stereoscopy and the total size of the visual field surrounding the user (i.e., field of regard or FOR) did not have a significant effect on manipulation times but two high-fidelity interaction techniques based on six degrees-of-freedom (DOF) input outperformed a low-fidelity technique based on keyboard and mouse input. In our second study, which involved a racing game on a commercial game console, we solely investigated interaction fidelity and found that two low-fidelity steering techniques based on 2D joystick input outperformed two high-fidelity steering techniques based on 3D accelerometer data in terms of lap times and driving errors.

Our final three studies involved a first-person shooter (FPS) game implemented within a six-sided CAVE. In the first of these FPS studies, we evaluated display fidelity and interaction fidelity independently, at extremely high and low levels, and found that both significantly affected strategy, performance, presence, engagement, and perceived usability. In particular, performance results were strongly in favor of two conditions: low-display, low-interaction fidelity (representative of desktop FPS games) and high-display, high-interaction fidelity (similar to the real world). In the second FPS study, we investigated the effects of FOR and pointing fidelity on the subtasks of searching, aiming, and firing. We found that increased FOR affords faster searching and that high-fidelity pointing based on 6-DOF input provided faster aiming than low-fidelity mouse pointing and a mid-fidelity mouse technique based on the heading of the user. In the third FPS study, we investigated the effects of FOR and locomotion fidelity on the subtasks of long-distance navigation and maneuvering. Our results indicated that increased FOR increased perceived usability but had no significant effect on actual performance while low-fidelity keyboard-based locomotion outperformed our high-fidelity locomotion technique developed for our original FPS study.

The results of our five studies show that increasing display fidelity tends to have a positive correlation to user performance, especially for some components such as FOR. Contrastingly, our results have indicated that interaction fidelity has a non-linear correlation to user performance with users performing better with "traditionalThe results of our five studies show that increasing display fidelity tends to have a positive correlation to user performance, especially for some components such as FOR. Contrastingly, our results have indicated that interaction fidelity has a non-linear correlation to user performance with users performing better with "traditional", extremely low-fidelity techniques and "natural", extremely high-fidelity techniques while performing worse with mid-fidelity interaction techniques. These correlations demonstrate that the display fidelity and interaction fidelity continua appear to have differing effects on the user experience for VR games.

In addition to learning more about the effects of display fidelity and interaction fidelity, we have also developed the Framework for Interaction Fidelity Analysis (FIFA) for comparing interaction techniques to their real-world counterparts. There are three primary factors of concern within FIFA: biomechanical symmetry, control symmetry, and system appropriateness. Biomechanical symmetry involves the comparison of the kinematic, kinetic, and anthropometric aspects of two interactions. Control symmetry compares the dimensional, transfer function, and termination characteristics of two interactions. System appropriateness is concerned with how well a VR system matches the interaction space and objects of the real-world task (e.g., a driving simulator is more appropriate than a 2D joystick for a steering task).

Although consumers have witnessed a technological revolution geared towards more realistic experiences in recent years, we have demonstrated with this research that there is still much to be learned about the effects of increasing a system's fidelity to the real world. The results of our studies show that the levels of display and interaction fidelity are significant factors in determining performance, presence, engagement, and usability.