VTechWorks staff will be away for the winter holidays starting Tuesday, December 24, 2024, through Wednesday, January 1, 2025, and will not be replying to requests during this time. Thank you for your patience, and happy holidays!

Browsing by Author "Jaimovich, Javier"

Now showing 1 - 7 of 7
  • Thumbnail Image
    AffecTech-an affect-aware interactive AV Artwork
    Coghlan, Niall; Jaimovich, Javier; Knapp, R. Benjamin; O’Brien, Donal; Ortiz, Miguel A. (ISEA International, 2009)
    New developments in real-time computing and body-worn sensor technology allow us to explore not just visible gestures using inertial sensors, but also invisible changes in an individual’s physiological state using bio-sensors (Kim & André 2008). This creates an opportunity for a more intimate interaction between the observer and technology-based art (Gonsalves 2008). We present a technical overview of the AffecTech system; a bio-signal based interactive audiovisual installation commissioned as part of the pre-ISEA symposium in November 2008. Observers were invited to sit on one of 2 sensor-enhanced chairs (Coghlan & Knapp 2008), which transmitted physiological data about the occupant to a central control system. This data was used to control and modulate interactive visuals, live video feeds and a surround sound score, with events and interactions dependent on the observers’ affective/emotional state and the disparity or similarity between the bio-signals of the chairs occupants. This technical overview is followed by an examination of the outcomes of the project, from both the artistic and technical viewpoints, with recommendations for modification in future implementations.
  • Thumbnail Image
    Biosignal-driven Art: Beyond biofeedback
    Ortiz, Miguel A.; Coghlan, Niall; Jaimovich, Javier; Knapp, R. Benjamin (CMMAS, 2011)
    Biosignal monitoring in interactive arts, although present for over forty years, remains a relatively little known field of research within the artistic community as compared to other sensing technologies. Since the early 1960s, an ever-increasing number of artists have collaborated with neuroscientists, physicians and electrical engineers, in order to devise means that allow for the acquisition of the minuscule electrical potentials generated by the human body. This has enabled direct manifestations of human physiology to be incorporated into interactive artworks. This paper presents an introduction to this field of artistic practice and scientific research that uses human physiology as its main element. A brief introduction to the main concepts and history of biosignal-driven art is followed by a review of various artworks and scientific enquiry developed by the authors. This aims at giving a complete overview of the various strategies developed for biosignal-driven interactive art.
  • Thumbnail Image
    Contagion of Physiological Correlates of Emotion between Performer and Audience: An Exploratory Study
    Jaimovich, Javier; Coghlan, Niall; Knapp, R. Benjamin (2010)
    Musical and performance experiences are often described as evoking powerful emotions, both in the listener/observer and player/performer. There is a significant body of literature describing these experiences along with related work examining physiological changes in the body during music listening and the physiological correlates of emotional state. However there are still open questions as to how and why, emotional responses may be triggered by a performance, how audiences may be influenced by a performers mental or emotional state and what effect the presence of an audience has on performers. We present a pilot study and some initial findings of our investigations into these questions, utilising a custom software and hardware system we have developed. Although this research is still at a pilot stage, our initial experiments point towards significant correlation between the physiological states of performers and audiences and we here present the system, the experiments and our preliminary data.
  • Thumbnail Image
    Creating Biosignal Algorithms for Musical Applications from an Extensive Physiological Database
    Jaimovich, Javier; Knapp, R. Benjamin (NIME, 2015)
    Previously the design of algorithms and parameter calibration for biosignal music performances has been based on testing with a small number of individuals - in fact usually the performer themselves. This paper uses the data collected from over 4000 people to begin to create a truly robust set of algorithms for heart rate and electrodermal activity measures, as well as the understanding of how the calibration of these vary by individual.
  • Thumbnail Image
    The Emotion in Motion Experiment: Using an Interactive Installation as a Means for Understanding Emotional Response to Music
    Jaimovich, Javier; Ortiz, Miguel A.; Coghlan, Niall; Knapp, R. Benjamin (NIME, 2012)
    In order to further understand our emotional reaction to music, a museum-based installation was designed to collect physiological and self-report data from people listening to music. This demo will describe the technical implementation of this installation as a tool for collecting large samples of data in public spaces. The Emotion in Motion terminal is built upon a standard desktop computer running Max/MSP and using sensors that measure physiological indicators of emotion that are connected to an Arduino. The terminal has been installed in museums and galleries in Europe and the USA, helping create the largest database of physiology and self-report data while listening to music.
  • Thumbnail Image
    Emotion in Motion: A Reimagined Framework for Biomusical/Emotional Interaction
    Bortz, Brennon; Jaimovich, Javier; Knapp, R. Benjamin (NIME, 2015)
    Over the past four years Emotion in Motion, a long running experiment, has amassed the world’s largest database of human physiology associated with emotion in response to the presentation of various selections of musical works. What began as a doctoral research study of participants in Dublin, Ireland, and New York City has grown to include over ten thousand emotional responses to musical experiences from participants across the world, from new installations in Norway, Singapore, the Philippines, and Taiwan. The most recent iteration of Emotion in Motion is currently underway in Taipei City, Taiwan. Preparation for this installation gave the authors an opportunity to reimagine the architecture of Emotion in Motion, allowing for a wider range of potential applications than were originally possible with the initial development of the tools that drive the experiment. Now more than an experiment, Emotion in Motion is a framework for developing myriad emotional/ musical/biomusical interactions with co-located or remote participants. This paper describes the development of this flexible, open-source framework and includes discussion of its various components: hardware agnostic sensor inputs, refined physiological signal processing tools, a public database of data collected during various instantiations of applications built on the framework, and the web application frontend and backend. We also discuss our ongoing work with this tool, and provide the reader with other potential applications that they might realize in using Emotion in Motion.
  • Thumbnail Image
    Using Music and Emotion to Enable Effective Affective Computing
    Bortz, Brennon Christopher (Virginia Tech, 2019-07-02)
    The computing devices with which we interact daily continue to become ever smaller, intelligent, and pervasive. Not only are they becoming more intelligent, but some are developing awareness of a user's affective state. Affective computing—computing that in some way senses, expresses, or modifies affect—is still a field very much in its youth. While progress has been made, the field is still limited by the need for larger sets of diverse, naturalistic, and multimodal data. This work first considers effective strategies for designing psychophysiological studies that permit the assembly of very large samples that cross numerous demographic boundaries, data collection in naturalistic environments, distributed study locations, rapid iterations on study designs, and the simultaneous investigation of multiple research questions. It then explores how commodity hardware and general-purpose software tools can be used to record, represent, store, and disseminate such data. As a realization of these strategies, this work presents a new database from the Emotion in Motion (EiM) study of human psychophysiological response to musical affective stimuli comprising over 23,000 participants and nearly 67,000 psychophysiological responses. Because music presents an excellent tool for the investigation of human response to affective stimuli, this work uses this wealth of data to explore how to design more effective affective computing systems by characterizing the strongest responses to musical stimuli used in EiM. This work identifies and characterizes the strongest of these responses, with a focus on modeling the characteristics of listeners that make them more or less prone to demonstrating strong physiological responses to music stimuli. This dissertation contributes the findings from a number of explorations of the relationships between strong reactions to music and the characteristics and self-reported affect of listeners. It demonstrates not only that such relationships do exist, but takes steps toward automatically predicting whether or not a listener will exhibit such exceptional responses. Second, this work contributes a flexible strategy and functional system for both successfully executing large-scale, distributed studies of psychophysiology and affect; and for synthesizing, managing, and disseminating the data collected through such efforts. Finally, and most importantly, this work presents the EiM database itself.