Expressive Communication: A Common Framework for Evaluating Developments in Generative Models and Steering Interfaces

by   Ryan Louie, et al.

There is an increasing interest from ML and HCI communities in empowering creators with better generative models and more intuitive interfaces with which to control them. In music, ML researchers have focused on training models capable of generating pieces with increasing long-range structure and musical coherence, while HCI researchers have separately focused on designing steering interfaces that support user control and ownership. In this study, we investigate through a common framework how developments in both models and user interfaces are important for empowering co-creation where the goal is to create music that communicates particular imagery or ideas (e.g., as is common for other purposeful tasks in music creation like establishing mood or creating accompanying music for another media). Our study is distinguished in that it measures communication through both composer's self-reported experiences, and how listeners evaluate this communication through the music. In an evaluation study with 26 composers creating 100+ pieces of music and listeners providing 1000+ head-to-head comparisons, we find that more expressive models and more steerable interfaces are important and complementary ways to make a difference in composers communicating through music and supporting their creative empowerment.


page 5

page 6

page 9


An Autoethnographic Exploration of XAI in Algorithmic Composition

Machine Learning models are capable of generating complex music across a...

New Musical Interfaces and New Music-making Paradigms

The conception and design of new musical interfaces is a multidisciplina...

Body, Clothes, Water, and Toys: Media Towards Natural Music Expressions with Digital Sounds

In this paper, we introduce our research challenges for creating new mus...

AI Song Contest: Human-AI Co-Creation in Songwriting

Machine learning is challenging the way we make music. Although research...

Formal models of Structure Building in Music, Language and Animal Songs

Human language, music and a variety of animal vocalisations constitute w...

Avatar Fusion Karaoke: Research and development on multi-user music play VR experience in the metaverse

This paper contributes to building a standard process of research and de...

Compose Embellish: Well-Structured Piano Performance Generation via A Two-Stage Approach

Even with strong sequence models like Transformers, generating expressiv...

Please sign up or login with your details

Forgot password? Click here to reset