Experiential Graphic Sonification for Visual and Auditory Communication Design and Musical Expression
Files
TR Number
Date
Authors
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
This dissertation explains two sonification platforms designed for image-sound association study and art, and the study results. A platform for user study was developed first and an artistic audiovisual platform was derived based on it. First, the five image-sound association studies were conducted to see whether people can successfully associate sounds and fundamental shapes (i.e., a circle, a triangle, a square, lines, curves, and other custom shapes) and the correct answer rate was high. Then, the same sonification platform was transformed by adding colors to the audiovisual platform for artistic/musical expression. A line-by-line sonification method and an object-oriented method were newly developed to sonify the background and the shapes separately. To enhance user experience, the sound of each shape was spatialized in a multi-layer speaker environment or a virtual listening environment.