Browsing Institute for Creativity, Arts, and Technology (ICAT) by Issue Date
Now showing 1 - 20 of 83
Results Per Page
- Multimodal Interaction in Music Using the Electromyogram and Relative Position SensingTanaka, Atau; Knapp, R. Benjamin (NIME, 2002)This paper describes a technique of multimodal, multichannel control of electronic musical devices using two control methodologies, the Electromyogram (EMG) and relative position sensing. Requirements for the application of multimodal interaction theory in the musical domain are discussed. We introduce the concept of bidirectional complementarity to characterize the relationship between the component sensing technologies. Each control can be used independently, but together they are mutually complementary. This reveals a fundamental difference from orthogonal systems. The creation of a concert piece based on this system is given as example.
- The integral Music controller: Introducing a Direct Emotional Interface to gestural control of sound synthesis.Knapp, R. Benjamin; Cook, Perry R. (Michigan Publishing, 2005-09)This paper describes the concept of the integral music controller (IMC), a controller that combines gestural interface with direct emotional control of a digital musical instrument. This new controller enables the performer to move smoothly between direct physical interaction with an acoustical musical instrument, and gestural/emotional control of the instrument’s physical model. The use of physiological signals to determine gesture and emotion is an important component of the IMC. The design of a wireless IMC using physiological signals is described and possible mappings to sound synthesis parameters are explored. Controlling higher level musical systems such as conducting and style modelling is also proposed.
- Creating a Network of Integral Music ControllersKnapp, R. Benjamin; Cook, Perry R. (NIME, 2006)In this paper, we describe the networking of multiple Integral Music Controllers (IMCs) to enable an entirely new method for creating music by tapping into the composite gestures and emotions of not just one, but many performers. The concept and operation of an IMC is reviewed as well as its use in a network of IMC controllers. We then introduce a new technique of Integral Music Control by assessing the composite gesture(s) and emotion(s) of a group of performers through the use of a wireless mesh network. The Telemuse, an IMC designed precisely for this kind of performance, is described and its use in a new musical performance project under development by the authors is discussed.
- Learning When Less is More: “Bootstrapping” Undergraduate Programmers as Coordination DesignersLin, Strong; Tatar, Deborah; Harrison, Steve; Roschelle, Jeremy; Patton, Charles (Computer Professionals for Social Responsibility, 2006)In this paper, we describe an undergraduate computer science class in the United States that we started with the intention of creating a participatory design experience to create distributed mobile collaborative technologies for education. The case highlights the ways in which programmer understanding of an innovative new technology can depend on understanding the context of use. The students were to use Tuple-spaces, a language for coordination. However, it soon became clear that while the coordination of machines may be thought of as a computer science problem, the students could not understand the technical system without richer models of how, why, or when coordination is desirable. We were in the ironic position of teaching human coordination at the same time as describing the technical properties of a system to support it. To “bootstrap” the learning process, we asked the students to draw on their own coordination expertise by implementing familiar coordinative games. We propose games as an addition to the PD toolkit when implementers need help in stepping outside their everyday mindset.
- Using Place as Provocation: In Situ Collaborative Narrative ConstructionSchaefer, Matthew; Tatar, Deborah; Harrison, Steve; Crandell, Alli (Research Center for Educational Technology, 2008)This paper describes a unique model for mobile, collaborative learning embodied in the use of a new software tool called PlaceMark©. The model is overtly intended to help learners reflect on their relationship to particular places, and the relationship between their own experience and other people’s experiences of those spaces. PlaceMark does this not by telling people what a place is, but by instead asking them, as they reflect and write about their own experiences of place. This paper describes how PlaceMark facilitates distributed control in coordinated classroom activities. We believe that the stance to knowledge embodied in the system encourages student responsibility within the learning process and helps teach about multiplicity of perspective in a visceral way. Additionally, as cell phones and other technologies become part of ordinary life, it is increasingly important that children (and all of us) come to have a deeper consciousness of place. This work reports on a pilot study of the software conducted with middle school students, and provides an analysis of the study activity.
- Sensory Chairs: A System for Biosignal Research and PerformanceCoghlan, Niall; Knapp, R. Benjamin (NIME, 2008-06)Music and sound have the power to provoke strong emotional and physical responses within us. Although concepts such as emotion can be hard to quantify in a scientific manner there has been significant research into how the brain and body respond to music. However much of this research has been carried out in clinical, laboratory type conditions with intrusive or cumbersome monitoring devices. Technological augmentation of low-tech objects can increase their functionality, but may n ecessitate a form of context awareness from those objects. Biosignal monitoring allows these enhanced artefacts to gauge physical responses and from these extrapolate our emotions. In this paper a system is outlined, in which a number of chairs in a concert hall environment were embedded with biosignal sensors allowing monitoring of audience reaction to a performance, or control of electronic equipment to create a biosignal-driven performance. This type of affective computing represents an exciting area of growth for interactive technology and potential applications for ‘affect aware’ devices are proposed.
- AffecTech-an affect-aware interactive AV ArtworkCoghlan, Niall; Jaimovich, Javier; Knapp, R. Benjamin; O’Brien, Donal; Ortiz, Miguel A. (ISEA International, 2009)New developments in real-time computing and body-worn sensor technology allow us to explore not just visible gestures using inertial sensors, but also invisible changes in an individual’s physiological state using bio-sensors (Kim & André 2008). This creates an opportunity for a more intimate interaction between the observer and technology-based art (Gonsalves 2008). We present a technical overview of the AffecTech system; a bio-signal based interactive audiovisual installation commissioned as part of the pre-ISEA symposium in November 2008. Observers were invited to sit on one of 2 sensor-enhanced chairs (Coghlan & Knapp 2008), which transmitted physiological data about the occupant to a central control system. This data was used to control and modulate interactive visuals, live video feeds and a surround sound score, with events and interactions dependent on the observers’ affective/emotional state and the disparity or similarity between the bio-signals of the chairs occupants. This technical overview is followed by an examination of the outcomes of the project, from both the artistic and technical viewpoints, with recommendations for modification in future implementations.
- Inner-Active Art: An examination of aesthetic and mapping issues in physiologically based artworksCoghlan, Niall; Knapp, R. Benjamin (ISEA, 2009)Much art seeks to describe or stimulate the feelings and emotions of the viewer, through both abstract and literal representation. With the exponential increase in computing power over recent years we also seek new ways of interacting with technology and exploring the virtual world. Physiological signals from the human body provide us with a view into the autonomic nervous system, that part of the nervous system largely unmediated by the direct intentions of the viewer. With the appropriate choice of signals and processing, we can even develop systems with the ability to interact with us on an emotional level - machines that know how we feel and can react accordingly (Haag et. al., 2004). This gives us the ability to see into and map the interior worlds of artists and viewers through a direct and visceral connection, the human body itself. A key issue in the development of physiologically based artwork is to make the observer-artwork dialogue meaningful to the observer, a question of translating the input bio-signals to visual, auditory or experiential events. We have yet to develop a suitable language for this dialogue and so this paper seeks to explore some potential mappings for bio-signal art, illustrated using several case studies from past and current works (Knapp et.al., 2008) (Gonsalves, 2009). We also examine some of the other philosophical and artistic issues involved in 'affective' and bio-art such as monitoring emotion v. engendering emotion, the involvement of the observer in creating and contributing to bio-signal art and strategies for effectively developing such works.
- Contagion of Physiological Correlates of Emotion between Performer and Audience: An Exploratory StudyJaimovich, Javier; Coghlan, Niall; Knapp, R. Benjamin (2010)Musical and performance experiences are often described as evoking powerful emotions, both in the listener/observer and player/performer. There is a significant body of literature describing these experiences along with related work examining physiological changes in the body during music listening and the physiological correlates of emotional state. However there are still open questions as to how and why, emotional responses may be triggered by a performance, how audiences may be influenced by a performers mental or emotional state and what effect the presence of an audience has on performers. We present a pilot study and some initial findings of our investigations into these questions, utilising a custom software and hardware system we have developed. Although this research is still at a pilot stage, our initial experiments point towards significant correlation between the physiological states of performers and audiences and we here present the system, the experiments and our preliminary data.
- Review of user interface devices for ambient assisted living smart homes for older peopleO’Mullane, Brian A.; Knapp, R. Benjamin; Bond, Rod (International Society for Gerontechnology, 2010)Smart homes generally focus on issues to do with security, health, energy savings and entertainment, issues which grow in importance as we age. The sensors, actuators and entertainment devices required to build such a system add significantly to its complexity. Hence, the Man-Machine Interface (MMI) to the smart home systems is often acknowledged to be the most sensitive area for acceptance. Smart homes can allow the user modify the house via a unified control, additionally assisted living smart homes gather information about the subjects health, information that can be used to feedback to the user to modify their behaviour via the device. Increasingly these interface device present information from the internet, such as weather and news. With the internet fast becoming the first source of information for many services, such as shopping, or care workers access, these devices may additionally help bridge the digital divide between the young and old(²), if the principles of universal design are addressed(³)(⁴). The purpose of this study is to examine user interfaces devices that can perform these tasks and analyse them with regard to the particular requirements of the older user.
- Biosignal-driven Art: Beyond biofeedbackOrtiz, Miguel A.; Coghlan, Niall; Jaimovich, Javier; Knapp, R. Benjamin (CMMAS, 2011)Biosignal monitoring in interactive arts, although present for over forty years, remains a relatively little known field of research within the artistic community as compared to other sensing technologies. Since the early 1960s, an ever-increasing number of artists have collaborated with neuroscientists, physicians and electrical engineers, in order to devise means that allow for the acquisition of the minuscule electrical potentials generated by the human body. This has enabled direct manifestations of human physiology to be incorporated into interactive artworks. This paper presents an introduction to this field of artistic practice and scientific research that uses human physiology as its main element. A brief introduction to the main concepts and history of biosignal-driven art is followed by a review of various artworks and scientific enquiry developed by the authors. This aims at giving a complete overview of the various strategies developed for biosignal-driven interactive art.
- Office of Research Annual Report 2011; Institute for Creativity, Arts, and TechnologyKnapp, R. Benjamin (The Research Institutes of Virginia Tech: Institute for Creativity, Arts, and Technology, 2011-08-01)
- Rapid Prototyping with Simple and Compound Machines Instructional PlanWilliams, Christopher B. (Institute for Creativity, Arts, and Technology: Center for the Arts, 2012)
- App Factor: Inventing to Solve Problems Instructional PlanEvans, Michael A. (Institute for Creativity, Arts, and Technology: Center for the Arts, 2012)OVERVIEW OF LESSON: Students will investigate the nature of inventions and articulate their knowledge of problems in the 1800’s and inventions that resulted to solve those problems. Next, students explore the Rhythmatical iPod® app, an “invention” designed to solve the problem of engaging students in math concepts. Finally, students will collaborate to design a storyboard for their own iPod® app invention.
- YourWellness: Designing an Application to Support Positive Emotional Wellbeing in Older AdultsDoyle, Julie; O’Mullane, Brian A.; McGee, Shauna; Knapp, R. Benjamin (BISL, 2012)Emotional wellbeing is an important indicator of overall health in adults over 65. For some older people, age-related declines to physical, cognitive or social wellbeing can negatively impact on their emotional wellbeing, as can the notion of growing older, the loss of a spouse, a loss of sense of purpose or general worries about coping, becoming ill and/or death. Yet, within the field of technology design for older adults to support independence, emotional wellbeing is often overlooked. In this paper we describe the design process of an application that supports older adults in monitoring their emotional wellbeing, as well as other parameters of wellbeing they consider important to their overall health. This application also provides informative and useful feedback to support the older person in managing their wellbeing, as well as clinically-based interventions if it is determined that some action or behaviour change is required on the part of the older person. We outline findings from a series of focus groups with older adults that have contributed to the design of the YourWellness application.
- The Emotion in Motion Experiment: Using an Interactive Installation as a Means for Understanding Emotional Response to MusicJaimovich, Javier; Ortiz, Miguel A.; Coghlan, Niall; Knapp, R. Benjamin (NIME, 2012)In order to further understand our emotional reaction to music, a museum-based installation was designed to collect physiological and self-report data from people listening to music. This demo will describe the technical implementation of this installation as a tool for collecting large samples of data in public spaces. The Emotion in Motion terminal is built upon a standard desktop computer running Max/MSP and using sensors that measure physiological indicators of emotion that are connected to an Arduino. The terminal has been installed in museums and galleries in Europe and the USA, helping create the largest database of physiology and self-report data while listening to music.
- Tai Chi for Children: Developing a Healthy Mind and Body Instructional PlanBukvic, Ivica Ico (Institute for Creativity, Arts, and Technology: Center for the Arts, 2012)
- Critical and Creative Thinking by Design: IDEA StudioBaum-Combs, Liesl; Cennamo, Katherine S.; Newbill, Phyllis; Wagner, Teri (Institute for Creativity, Arts, and Technology: Center for the Arts, 2012)
- Design of a Mobile Brain Training Tool for Seniors: Motivational RequirementsO’Brien, Donal; Knapp, R. Benjamin; Thompson, Oonagh; Craig, David; Barrett, Suzanne (ICST, 2012)The overall goal of this project is to design, develop, and validate a mobile phone-based ‘brain training’ software suite targeted at senior users, using iterative person-centered design methodologies, that will permit a subsequent clinical trial of cognitive stimulation efficacy known as theSONIC²S Study (Stirling-Oregon-Northern Ireland-Chicago Cognitive Stimulation Study). The SONIC²S Study will represent a long term (c. 15 year), very large scale (n=12,000), embedded clinical trial that aims to definitively establish whether or not brain training acts to prevent dementia or cognitive decline. It is anticipated that participant compliance in such a study will be a significant concern. This study reports on a series of focus groups, usability studies and field trials which sought to identify the key motivational factors influencing seniors’ engagement with mobile brain-training technology in order to inform the design of a bespoke tool which is acceptable/enjoyable to target users.