Painting Music uses artificial intelligence (AI) to create music from live-painted drawings, in real time and unique for each performance. The production is a collaboration with AI engineers Dr. Starkey and Jack Caven from the University of Aberdeen.
Most AI techniques that are used in artistic endeavours are off-the-shelf methods that are black box and cannot explain what they are doing. Painting Music however uses algorithms that are explainable, which is innovative even in the field of AI, and the combination of AI, painting and music represents to the best of our knowledge a world’s first. Painting Music requires the AI to be transparent in how it understands a piece of painted art and then converts this into a musical motif. The AI’s application within Painting Music is therefore incredibly exciting and ground-breaking.
Photo credit Ray Interactive
Under the 2021 Connected Innovators‘ Award (from Creative Edinburgh and Creative Informatics) a portable and autonomous system was engineered in collabration with Ray Interactive. The system enables live broadcasting of Painting Music through any physical and digital platform.
This bespoke product revolutionises the act-of-making by my control over both visual and aural dimensions facilitating a multifaceted and multidisciplinary artistic exploration; transforms my relationship with creative communities and audiences being accessible world-wide; and empowers my practice to contextualise AI, as it is increasingly driving current affairs, through a unique high-quality audio-visual deliverance.
Painting Music taps into the public’s fear – or excitement of AI and the change that this will bring to all facets of society. The production is centred around the question Is AI good or bad?
A prototype of Painting Music was tested through a live stage performance as part of Aberdeen University May Festival 2019 and funded by Creative Scotland. The AI created music from live drawing unfolding on a screen through a projector.
The production explored and reflected on the impact of AI on society using the visual and aural outputs of the process. The perception of AI by the public has changed over time, and that although the public has been more optimistic than pessimistic when discussing AI, the fear of loss of control of AI has been increasing in recent years.
Photo credit – Aberdeen University May Festival.
Dr. Starkey’s research is focused on bridging the gap between humans and computers, and ensuring that the AI is fully explainable to the end user.
BSL Interpreter Elaine Campbell (Above)
Can artificial intelligence, a computer brain that is based on our own biological brains, replace a human? Is this a good idea? And what does this mean? What is artificial intelligence? To answer that question we firstly need to define what is ‘intelligence’. This is surprisingly difficult. Most explanations for intelligence are simply synonyms for the word intelligence like wisdom, cleverness, or being smart.
AI Developer Jack Caven and Paddy the Labrador
So is intelligence then the ability to learn? And learn in particular like a human? To take in some form of sensory input and to learn how to respond? An amoeba in sea water responds to light to determine where to swim to find food – do we consider this intelligent? In some ways amoebas are already more intelligent than even the most sophisticated form of artificial intelligence available today, as they are fully autonomous, make decisions on their own, grow and reproduce.
Painting Music was also transformed into a 20-minute film by filmmakers Albert Lucas and Kim Beveridge funded by Aberdeen City Council and Creative Scotland. With special thanks to videographer Calum McCready and voice-over performed by Shane Strachan.
The production shows close-ups of the artistic process and includes interviews with those involved in the production through live stage performance, film and a body of visual artwork. All music in Painting Music is created by AI from live painting.
Painting Music is a tool to develop a critical response on human-AI ecosystems with many finding the techniques compelling, disturbing, productive, and thought-provoking in equal measure.
Painting Music moves visual art as a traditionally static experience into a dynamic, interactive artform that is transformational, transient, and temporal.
A scene from the performance entitled Top dog
A scene from the performance entitled The physical brain