The Effect of Sonification in Informal Learning: From Fairy Tales to Art Galleries

Abstract

As technology continues to advance in the present era, lifelong learning has become essential across all stages of life. Learning occurs not only in formal institutional settings but also in informal contexts embedded in daily life. Within these informal learning environments, sonification can play a pivotal role by facilitating learning through auditory interaction, interpretation, and engagement. However, the effectiveness of sonification can vary based on contextual factors and individual capabilities. To investigate the potential of sonification in informal learning, this dissertation presents four empirical studies across two domains: human-robot interaction (HRI) and art appreciation in gallery settings. The first study examined how emotion-reflecting sonification, compared to emotion mitigating sonification, influences the experience of listening to fairy tales, focusing on participants' enjoyment, empathy, and immersion. The second study compared sentiment based sonification with classical music in robot storytelling, evaluating their impact on comprehension, emotional interpretation, and engagement. The third study investigated the role of Augmented Audio in an art gallery environment, assessing its influence on participants' emotional responses and engagement during art appreciation. The fourth study explored how Augmented Audio and Augmented Visual elements influence comprehension, emotional reaction, and sustained engagement with artworks. Collectively, this dissertation offers insights into how sonification can be effectively applied across varied informal learning contexts, from narrative storytelling to visual art appreciation, to foster immersive, emotionally resonant, and cognitively enriching educational experiences.