The Effect of Interaction Method and Vibrotactile Feedback on User Experience and Performance in the VR Games

Recent hand tracking systems have contributed to enhancing user experience in the virtual environment (VE) due to its natural and intuitive interaction. In addition, wearable haptic devices are another approach to provide engaging and immersive experiences. However, controllers are still prevalent in VR (Virtual Reality) games as a main interaction device. Also, haptic devices are rare and not widely accepted by users because they get bulky to implement sophisticated haptic sensation. To overcome this issue, I conducted experiments (Study 1 and Study 2 of this Thesis) to investigate the effect of interaction method (controller and whole-hand interaction using hand tracking) and vibrotactile feedback on user experience in the VR game. In Study 1 of this Thesis, I recruited 36 participants and compared the user's sense of presence, engagement, usability, and task performance under three different conditions: (1) VR controllers, (2) hand tracking without vibrotactile feedback, and (3) hand tracking with vibrotactile feedback at fingertips through the gloves I developed. The gloves deliver vibrotactile feedback at each fingertip by vibration motors. I observed that whole-hand interaction using hand tracking enhanced the user's sense of presence, engagement, usability, and task performance. Further vibrotactile feedback increased the presence and engagement more clearly. Based on the participants' feedback, I could further modify the form factor to make it more usable in the VR game and comfortable to wear on a regular basis. In this sense, in Study 2 of this Thesis, I developed a new thimble-shape device to deliver vibrotactile feedback only at one fingertip rather than ten fingertips. Further, social VR is an emerging VR platform where multiple users can interact with one another. However, most social VR applications have not provided a sense of touch. I recruited 24 participants and conducted an experiment that explored the effects of interaction method and fingertip vibrotactile feedback on the user's sense of social presence, presence, engagement, and task performance in a cooperative VR game under four different conditions: (1) VR controllers without vibrotactile feedback, (2) VR controllers with vibrotactile feedback, (3) hand tracking without vibrotactile feedback, and (4) hand tracking with vibrotactile feedback with the fingertip vibrotactile device. The results showed that whole-hand interaction using hand tracking increased the level of presence. In addition, multiple items in the presence questionnaire indicated that vibrotactile feedback enhanced the level of presence as well. However, I could not observe the significant difference in social presence due to the unique setting of this experiment. Unlike the previous studies, my task was sufficiently cooperative, and thus, the participants felt high level of social presence regardless the conditions, which led to the ceiling effect. I also observed that there was no significant difference in engagement. Controller conditions had higher performance than hand tracking due to the technological limitations in hand tracking. Results are discussed in terms of implications for the components of interaction in the VR with hands, a touch in social VR, cooperative VR game, and practical design guidelines.