ICoN: Immersive Computational Notebook for Data Science

Abstract

Computational notebooks are widely used in data science, offering an interface that integrates code, documentation, visualizations, and data within a single environment. However, as data analysis becomes increasingly complex, the traditional WIMP (Windows, Icons, Menus, Pointers) interface faces limitations in supporting advanced, embodied workflows. To address this, we initially adapted the computational notebook into the immersive environment to leverage the embodiment and immersiveness provided by immersive technologies. While our adoption of computational notebooks in immersive environments improved navigation performance, their standard interfaces were less effective, which required extensive text input for tasks such as data transformation and visualization. To overcome these challenges, we initially explored embodied data transformation, focusing on gesture-based, direct data manipulation within immersive environments. Embodied data transformation reduced cognitive load, enabling more intuitive data analysis without extensive text input and programming expertise. Building on these successful explorations, we developed ICoN, a system that integrates immersive computational notebooks, embodied data transformation, and visualization within a unified workspace. Through controlled comparisons of desktop vs. VR and separated vs. unified workspaces, we found that ICoN not only improves navigation performance but also enables intuitive data transformation and visualization capabilities. However, despite ICoN's potential, the organizational strategies where execution order plays an important role in immersive environments remain underexplored. Our next research addresses this gap by examining how users spatially arrange and interact with computational notebooks in immersive contexts. In a user study, participants favored organizing their work in half-cylindrical layouts and engaged more frequently in non-linear analysis compared to traditional setups. This shift suggests that immersive environments encourage new approaches to managing data science workflows, leading to a more flexible and efficient use of computational notebooks. Yet, general limitations, such as scalability, must be addressed for ICoN to be applied to real-world, complex data analysis scenarios, where tasks are performed over longer periods and with larger datasets. Manually adjusting large notebooks for different tasks can be time-consuming and exhausting. To overcome these challenges, we introduced organizational guidance, which enables the system to assist analysts in building well-defined structures with consistent spacing and alignment. In our user study, we found that organizational guidance significantly improved the effectiveness of constructing large-scale analyses within immersive computational notebooks. By enabling participants to quickly reorganize their workspace, the system facilitated faster initiation of analysis. To isolate the effect of organizational guidance, we annotated specific cells in the study to highlight embedded structural patterns, helping participants focus on organization rather than code comprehension. However, in real-world analysis, the process often involves a sensemaking process, such as understanding the underlying execution logic, which remains necessary even with guidance. As a result, navigating large, immersive spaces remains inevitable for performing large-scale analyses. We envision incorporating lens techniques commonly used in the field of visualization, allowing analysts to closely inspect small or detailed data elements, and employing near-far proxies that reveal different levels of information based on the distance between the analyst and the target content.