Applying Art-based Embodied Knowledge to Further Artistic Objectives with Technology and Support Creative Thinking in Computing

Abstract

As computers have advanced, their use cases have expanded to support many human-centric activities. Computing's place in the creation and consumption of art has been called into question in recent years with the emergence of generative artificial intelligence (GenAI), which can produce creative works such as paintings, music, and creative writings almost instantly. However, before the introduction of GenAI, software had been used for decades across all vantages of influence to expedite human expression. Society is grappling with the question of how to approach technology's influence on all forms of art: how to effectively apply computing's capabilities without impeding the agency of the artist. Furthermore, there is a lack of influence from the world of art on directing the immense force of technology on the modern world. This dissertation sought to build a tighter bidirectional relationship between art and technology by understanding how technology can further the objectives of the artist through studies 1 and 2 and investigating how embodied logic in art can be used to support the algorithmic thinking required in computing in Study 3.

Study 1 focused on capturing embodied knowledge in the procedures of an art form through motion sensors to communicate physical routines through sound. This work was completed by building a prototype sonification system which dynamically classified segments of a stitch in crochet, and then conducting co-design sessions with stakeholders. Results demonstrated that using novel sensory-perceptual modalities such as audition to express abstract concepts can be used as a novel UX design tool. In study 2 I sought to understand the potential for human biases against technology's use in art. I built two different technologically-augmented dance performances, one with artificial intelligence (AI) and one without, and withheld information on how the performances were built in order to capture audience responses on the creative merits of the technological components of the two works. Results highlighted significantly higher ratings on questions of artistic merit for the AI performance version when implementation information was withheld compared to when this information was divulged, suggesting a potential for bias. This offered insight into the deployment of AI art in the future. Study 3 focused on teaching and using embodied logic in art to assess analytical thinking skills in computing. The study surrounded teaching individuals how to interpret and write crochet patterns using two different methods: with or without the accompanying physical procedures. After the lessons, written tests on patterns in crochet and computer programming were administered. Results showed a positive correlation between programming and crochet test scores of individuals for the majority of participants across all condition groups. Interview transcriptions pointed to similarities in approaching questions on the two tests and a desire for visual aids from participants who didn't learn the physical procedures.

The present body of work examined how the latest technologies can be used to further the human objectives of art, and how constructs from art can be used to advance computing objectives. The present work offers theoretical insights into using multiple sensory-perceptual modalities as system design tools, as well as practical guidance for how technologies can effectively be applied in creative, art-based contexts.