Painting Music

Home >> Portfolio >> Live Stage Drawing >> Painting Music

Painting Music uses Artificial Intelligence (AI) to create music from live-painted drawings. The production shows close-ups of the artistic process and includes interviews with those involved in the production through live stage performance, film and a body of visual artwork. Each output is briefly discussed below:

Painting Music allows audience appreciation of this powerful AI tool; and creates intrigue hooking the viewer around our own questions Is AI good or bad?.

This production moves visual art as a traditionally static experience into an artform that is dynamic, interactive, transformational, transient and temporal.

Painting Music offers a unique experience by creating a multifaceted and multi-disciplinary artistic expression, that contain elements of risk and reveal the artistic process to the audience.

Visual artist Kate Steenhauer

Many artists have explored multi-sensory ways of perceiving and to a certain extent have examined the analogy of mark-making to sound. Painting Music, however, uses cutting-edge AI techniques to examine the correlation and harmonious balance between drawing and music. The AI software developed by Jack Caven and AI specialist Dr. Starkey translates tangible painted marks into audible sounds, in real time and unique for each performance.

Most AI techniques are relatively ‘run-of-the-mill’ methods that use statistical approaches and ‘machine learning’ algorithms. Painting Music however uses algorithms that are based on the type of learning used by the human brain.

AI specialist Dr. Starkey

Dr. Starkey’s research is focused on bridging the gap between humans and computers, and ensuring that the AI is fully explainable to the end user – this production is a perfect example, which requires the AI to be transparent in how it understands a piece of painted art and then converts this into a musical motif.

Painting Music live stage performance where the AI created music from live drawing unfolding on a screen through a projector was shown as part of Aberdeen University May Festival 2019 and funded by Creative Scotland.

The production explores and reflects on the impact of AI on society using the visual and aural outputs of the process.  The perception of AI by the public has changed over time, and that although the public has been more optimistic than pessimistic when discussing AI, the fear of loss of control of AI has been increasing in recent years. Painting Music creates intrigue hooking the viewer around our own questions centred around the narrative Is AI good or bad?. Photos credit Aberdeen University May Festival.

BSL Interpreter Elaine Campbell

13-year old Labrador Paddy starred in the live stage performance representing the animal kingdom alongside humanity and Artificial Intelligence.

AI Developer Jack Caven and 13-year old Labrador Paddy

Can Artificial Intelligence, a computer brain that is based on our own biological brains, replace a human? Is this a good idea? And what does this mean? What is Artificial Intelligence? To answer that question we firstly need to define what is ‘intelligence’. This is surprisingly difficult. Most explanations for intelligence are simply synonyms for the word intelligence like wisdom, cleverness, or being smart. So is intelligence then the ability to learn? And learn in particular like a human? To take in some form of sensory input and to learn how to respond? An amoeba in sea water responds to light to determine where to swim to find food – do we consider this intelligent? In some ways amoebas are already more intelligent than even the most sophisticated form of Artificial Intelligence available today, as they are fully autonomous, make decisions on their own, grow and reproduce.

Painting Music 20-minute film is produced by Glasgow filmmaker Kim Beveridge and associate editor Albert Lucas funded by Aberdeen City Council. All music in Painting Music is created by AI from live painting. Japanese ink drawing as well as AI visual and aural outputs evolve onto a screen. The film premiere was at Haddo House Art Festival as part of the exhibition Experimental Use of Space 2019. The film is due to be screened at Belmont Filmhouse, George Washington Wilson Centre at Aberdeen University, and Royal Society of Edinburgh as part of Science Festival, and on the 25th June 2020 live streamed via / /

Painting Music offers a unique experience and taps into current demands for interactive and evolving ‘stories’, that contain elements of risk and reveal the artistic process to the audience. The production process brings attention to the art of live drawing and highlights drawing’s dynamic capacity with other artforms, technology and their relationship with its audience. The production explores thoughts and fears over the application and impact of AI, and its prevalence both today and in the future.

Painting Music Visual Artwork: hybrid art pieces consisting of printmaking and oriental techniques, liquid metal paint and more traditional art material such as compressed charcoal on oriental rice-paper along with a silk red thread.

The artwork shows figures play, struggle, fight with, or escape from, search for, and appeal to ‘intelligence’. Data bias is considered here: the intelligence of an imbalanced (unequal) society, who’s logic is intrinsically informing the ‘intelligence’ of our computer algorithms, only amplifying the biases.

Community, Linoprint and liquid gold on oriental paper, 400 x 600mm

Painting Music Visual Artwork Close-up Videoclip

Stages of AI (I), Japanese ink and liquid gold on oriental scroll, 1200 x 400mm
Stages of AI (I), Japanese ink and liquid gold on oriental scroll, 1200 x 400mm
Fighting Intelligence, Charcoal, liquid metal paint and monoprint, 650 x 800mm
In the midst of the 4th industrial revolution, 1200 x 700mm, Japanese ink, oil pastel, linoprint and red thread
Top dog II, 500 x 300mm, Compressed charcoal, liquid metal paint, oil pastel, linoprint and red thread
Searching for learning, Japanese ink, oil pastel and monoprint, 1200 x 400mm
Child’s play, Japanese ink and coloured pencil, 800 x 650mm
Top dog, Japanese ink on oriental scroll, 700 x 750mm
A perpetual sense of uncertainty, linoprint on liquid metal paint,850 x 400mm
The physical brain, Japanese ink, liquid metal paint on oriental scroll, 650 x 600mm
Fighting Intelligence in Red, 900 x 700mm, Japanese ink, oil pastel, linoprint and red thread