Painting music

6th Sep, 2019Live productions

Painting Music

Painting Music uses artificial intelligence (AI) to create music from live-painted drawings. The production shows close-ups of the artistic process and includes interviews with those involved in the production through live stage performance, film, and a body of visual artwork.

Painting Music allows audience appreciation of this powerful AI tool; and creates intrigue hooking the viewer around their own questions – “is AI good or bad?”.

This production moves visual art as a traditionally static experience into an artform that is dynamic, interactive, transformational, transient and temporal.

A unique experience is offered by creating a multifaceted and multi-disciplinary artistic expression, that contains elements of risk and reveals the artistic process to the audience.

All music in Painting Music is created by AI from live painting.

Painting Music

Many artists have explored multi-sensory ways of perceiving and to a certain extent have examined the analogy of mark-making to sound. Painting Music, however, uses cutting-edge AI techniques to examine the correlation and harmonious balance between drawing and music. The AI software developed by Jack Caven and  AI specialist Dr. Starkey translates tangible painted marks into audible sounds, in real time and unique for each performance.

Most AI techniques are relatively ‘run-of-the-mill’ methods that use statistical approaches and ‘machine learning’ algorithms. Painting Music however uses algorithms that are based on the type of learning used by the human brain.

Painting Music

Dr. Starkey’s (pictured above) research is focused on bridging the gap between humans and computers, and ensuring that the AI is fully explainable to the end user – this production is a perfect example, which requires the AI to be transparent in how it understands a piece of painted art and then converts this into a musical motif.

Painting Music live stage performance where the AI created music from live drawing unfolding on a screen through a projector was shown as part of Aberdeen University May Festival 2019 and funded by Creative Scotland.

The production explores and reflects on the impact of AI on society using the visual and aural outputs of the process.  The perception of AI by the public has changed over time, and that although the public has been more optimistic than pessimistic when discussing AI, the fear of loss of control of AI has been increasing in recent years. 

Photo credit – Aberdeen University May Festival.

Painting Music

BSL Interpreter Elaine Campbell (Above)

13-year old Labrador Paddy starred in the live stage performance representing the animal kingdom alongside humanity and artificial intelligence.
Painting Music

AI Developer Jack Caven and Paddy the Labrador

Can artificial intelligence, a computer brain that is based on our own biological brains, replace a human? Is this a good idea? And what does this mean? What is artificial intelligence? To answer that question we firstly need to define what is ‘intelligence’. This is surprisingly difficult. Most explanations for intelligence are simply synonyms for the word intelligence like wisdom, cleverness, or being smart.

Painting Music

So is intelligence then the ability to learn? And learn in particular like a human? To take in some form of sensory input and to learn how to respond? An amoeba in sea water responds to light to determine where to swim to find food – do we consider this intelligent? In some ways amoebas are already more intelligent than even the most sophisticated form of artificial intelligence available today, as they are fully autonomous, make decisions on their own, grow and reproduce.

Painting Music was also transformed into a 20-minute film by filmmakers Albert Lucas and Kim Beveridge funded by Aberdeen City Council and Creative Scotland. With special thanks to videographer Calum McCready and voice-over performed by Shane Strachan!

The production shows close-ups of the artistic process and includes interviews with those involved in the production through live stage performance, film and a body of visual artwork.  All music in Painting Music is created by AI from live painting. The software exploits areas of similarity within the two distinct artforms to respond to the live-painted elements and produce musical notes that reflect the development of the evolving artwork.

Painting Music has received funding through the 2021 Connected Innovators’ programme run by Creative Edinburgh! Operating as part of the wider Creative Informatics programme, Connected Innovators is an opportunity for emerging leaders in the creative industries for research and professional development around data and data-driven innovation.

The  Connected Innovators‘ funding for Painting Music would allow firstly to develop closer ties with Edinburgh’s creative community in order to significantly enhance the Painting Music product, particularly in the musical output; and secondly to evolve the product so that the process can be undertaken alone without the need for direct support from others. This second point has the advantage that other artists can also use the product either on their own or in conjunction with myself in performances.

Scene – Child’s play

During the Connected Innovators‘ programme a portable standalone and autonomous system will be engineered in collaboration with art studio Ray Interactive to enable live broadcasting of the artist’s practice to audiences and creatives through any physical and digital platform.

painting music

Scene – Top dog