SentimentVoice: Integrating Emotion AI and VR in Performing Arts

Abstract

SentimentVoice is a live VR performance art project that repurposes emotion-tracking AI to amplify immigrant narratives. While emotion Ai is often used for surveillance or commercial gain, SentimentVoice subverts this technology into an empathetic tool, actively listening to immigrant stories instead of exploiting them. The project integrates human voice and facial expression as core materials, analysed through emotion AI using face tracking, the ChatGPT API, voice analysis, and speech-to-text conversion. While technologies like lie detectors and facial recognition have been used to monitor and target marginalized communities, SentimentVoice reclaims emotion data as a medium for fostering empathy, deep connections, and mutual understanding. Rooted in oral history, the project collects real immigrant stories from the Richmond community, structured around five key locational memories: airports, schools, immigration offices, hospitals and workplaces. These spaces represent critical moments of transition and adaptation in the immigrant experience.