Painting Music

Home > Portfolio > Live Stage Drawing > Painting Music

Painting Music uses Artificial Intelligence (AI) to create music from live-painted drawings. Painting Music is a live stage performance, a film and a body of visual artwork. Each output is briefly discussed below.

Many artists have explored multi-sensory ways of perceiving and to a certain extent have examined the analogy of mark-making to sound. Painting Music, however, uses cutting-edge AI techniques to examine the correlation and harmonious balance between drawing and music. The AI software developed by Jack Caven and AI specialist Dr. Starkey translates tangible painted marks into audible sounds, in real time and unique for each performance.

Most AI techniques are relatively ‘run-of-the-mill’ methods that use statistical approaches and ‘machine learning’ algorithms. Painting Music however uses algorithms that are based on the type of learning used by the human brain.

Dr. Starkey’s research is focused on bridging the gap between humans and computers, and ensuring that the AI is fully explainable to the end user – this production is a perfect example, which requires the AI to be transparent in how it understands a piece of painted art and then converts this into a musical motif.

Painting Music live stage performance where the AI created music from live drawing unfolding on a screen through a projector was shown as part of Aberdeen University May Festival 2019 and funded by Creative Scotland.

Intrigue was created hooking the viewer around our own questions centred around the narrative Is AI good or bad?. Photo credit Aberdeen University May Festival.

13-year old Labrador Paddy starred in the performance representing the animal kingdom alongside humanity and Artificial Intelligence.

Can Artificial Intelligence, a computer brain that is based on our own biological brains, replace a human? Is this a good idea? And what does this mean? What is Artificial Intelligence? To answer that question we firstly need to define what is ‘intelligence’. This is surprisingly difficult. Most explanations for intelligence are simply synonyms for the word intelligence like wisdom, cleverness, or being smart. So is intelligence then the ability to learn? And learn in particular like a human? To take in some form of sensory input and to learn how to respond? An amoeba in sea water responds to light to determine where to swim to find food – do we consider this intelligent? In some ways amoebas are already more intelligent than even the most sophisticated form of Artificial Intelligence available today, as they are fully autonomous, make decisions on their own, grow and reproduce.

Painting Music has also been transformed into a 15-minute film produced by Glasgow filmmaker Kim Beveridge and associate editor Albert Lucas funded by Aberdeen City Council. Japanese ink drawing as well as AI visual and aural outputs evolve onto a screen. The film premiere was at Haddo House Art Festival as part of the exhibition Experimental Use of Space 2019. The film will be further shown at Belmont Filmhouse and Aberdeen University in 2020.

Painting Music offers a unique experience and taps into current demands of interactive and evolving ‘stories’, holding elements of risk and revealing behind-the-scene-glimpses with at the core high quality fine art in a distinctive style. The production process brings attention to the art of live drawing and highlights drawing’s dynamic capacity with other artforms.

Painting Music Visual Artwork: hybrid art pieces consisting of printmaking and oriental techniques, liquid metal paint and more traditional art material such as compressed charcoal on oriental rice-paper along with a silk red thread.

The 5th technological revolution, Linoprint and liquid gold on oriental paper, 400 x 600mm

Painting Music Visual Artwork Close-up Videoclip

Stages of AI, Japanese ink and liquid gold on oriental scroll, 1200 x 400mm
Fighting Intelligence, Charcoal, liquid metal paint and monoprint, 650 x 800mm
Searching for learning, Japanese ink, oil pastel and monoprint, 1200 x 400mm
Child’s play, Japanese ink and coloured pencil, 800 x 650mm
Top dog, Japanese ink on oriental scroll, 700 x 750mm
Uncertainty, Linoprint, liquid paint on oriental scroll, 650 x 300mm
The physical brain, Japanese ink, liquid metal paint on oriental scroll, 650 x 600mm
Mother Nature, Japanese ink, oil pastel, liquid metal paint on oriental scroll, 700 x 500mm
Charcoal, oil pastel, liquid metal paint on oriental scroll, 600 x 400mm
Charcoal, oil pastel, liquid metal paint on oriental scroll, 450 x 400mm
Charcoal, Liquid metal paint, oil pastel, 850 x 600mm
Physical struggles and mental battles I, Japanese ink, liquid metal gold paint, oil pastel and silk thread on oriental scroll, 1200 x 400mm
Physical struggles and mental battles II, Japanese ink, liquid metal gold paint, oil pastel and silk thread on oriental scroll, 1200 x 400mm