Handheld Virtual Reality for Informal STEAM Education: Exploring its Application, User Experience, and User Interface

Abstract

Virtual reality (VR) is an emerging technology that offers immersive and novel experiences. Despite its increasing adoption in public and social settings, such as museums and exhibitions, traditional VR head-mounted displays (HMDs) present barriers for certain user groups. These challenges include motion sickness, hygiene concerns associated with shared devices, discomfort with public use, and the inconvenience of limited device availability. Handheld VR presents a promising alternative by leveraging widely available devices, such as tablets and smartphones, as viewports for interacting with virtual environments. Functioning similarly to augmented reality (AR), handheld VR allows users to remain connected to both the virtual and physical worlds, offering a safer experience with spatial awareness and facilitating richer interpersonal communication through nonverbal cues (e.g., gestures, facial expressions). While prior research has largely focused on enhancing the immersive qualities of HMD-based VR, limited attention has been given to understanding handheld VR experiences. This dissertation addresses this gap by conducting a series of studies exploring the potential application of handheld VR across different domains and settings. The first study examined how handheld VR facilitates engagement in an informal learning context by analyzing young learners' interactions with a handheld VR-based program at a science museum. The findings provide design guidelines for creating social and engaging learning experiences through interactive virtual environments. The second study explored the application of handheld VR in the arts domain through focus group and expert interviews. Based on the findings, this study developed a taxonomy outlining fundamental components of handheld VR art and established design guidelines for stakeholders interested in creating inclusive and creative experiences. Building on these application-focused studies, the third and fourth studies investigated user interface design for handheld VR, given the unique challenge of interacting with a three-dimensional environment through a two-dimensional display. The findings offer design considerations for developing user interfaces for room-scale handheld VR experiences. Collectively, this dissertation expands the understanding of how handheld VR promotes socially engaging and more accessible experiences, particularly in informal learning and creative domains. Additionally, it contributes to the theoretical understanding of user behavior in handheld VR, including its alignment with established frameworks such as Fitts' Law.