What does it feel like to improvise with AI?

Pianist Zubin Kanga returns to the Southbank Centre on 17 October 2026 with a host of premieres in a  double bill of  Ghost in the Machine + After Dark: Club Cyborg. Zubin Kanga is currently based at Royal Holloway University where he is leading a £1.4m grant-funded research project 'Cyborg Soloists,' which explores AI and technology in music. [see Robert's 2024 interview with Zubin Kanga]

In advance of Zubin Kanga's premiere of George Lewis's Blombos Playground on 17 October at the Purcell Room, Southbank Centre, London, he asks the question 'What does it feel like to improvise with AI?'

I’ve often experienced a deep connection with other musicians onstage when playing chamber music or improvising together, like we're having a musical conversation. But how does it feel to have a musical conversation with AI? Does it feel anything like playing with a human musician on stage? And what can it tell us about what makes human creativity unique? These are questions I’ve been grappling with while working with American composer George Lewis on his new work composed for human and AI keyboardists, Blombos Playground, which I’m about to premiere at the Southbank Centre in London.

While many people today think of AI art as being a recent development and associate it with the lazy production of bland results, trained via the internet through unethical means, it actually has a long and much more interesting history in experimental music, going back to the 1950s. Early developers of AI music systems never sought to replace human musicians, but instead to explore new ways for human performers to interact with musical technologies – and with each other.

George Lewis [Photo: Eileen Barroso]

George Lewis is one of these pioneers, a legendary figure in the field of human-computer interaction in music. Lewis first started exploring the use of musical machines that could independently interact with human musicians in 1979, developing into his first AI experiments in 1984 with Rainbow Family. His iconic breakthrough system, Voyager, was initially developed in 1985-87 and he continues to develop it to this day – it’s one of the two AI systems in his new piece for me, Blombos Playground. Voyager is completely different from the type of AI we’ve all become familiar with in recent years. Rather than machine learning, the system uses a set of musical rules for listening and playing that Lewis has programmed, based on his own years of experience as an improvising musician (he’s performed on trombone with many of the world’s leading improvisers since the 1970s, including Anthony Braxton, John Zorn, Derek Bailey and Evan Parker). The result of this approach is a computer program that seems to have its own personality as an improvisor. Voyager listens to the human co-performers, and sometimes follows and accompanies, but at other times pushes to the fore, like a human improvisor who senses it might be time for their ‘solo’. Over the four decades since Lewis first programmed it, he’s improved it with every new development in computing, and many of the world’s leading performers and ensembles have performed with it – from saxophonist Roscoe Mitchell, to the International Contemporary Ensemble (USA), to the SWR Symphony Orchestra (Germany). It’s with Voyager’s latest developments that I enter the picture.

It’s been one of the privileges of having a career as a contemporary music performer that I’ve been able to work with the legends I learnt about as a student. As an undergraduate 20 years ago, I read articles about Lewis's AI experiments, and I've listened to many recordings of musicians playing with Voyager over the years. So it was a career highlight when Lewis said ‘yes’ to a new commission as part of my Cyborg Soloists music technology research project. Lewis’s central idea was about pushing Voyager to the next level by combining it with the Somax improvising software created by IRCAM, a renowned music research institute in Paris.

Though Voyager and Somax are both types of ‘AI’, they work completely differently. Whereas Voyager uses rule-based approaches to improvisation, Somax uses machine- learning on a set of user-provided recordings. In my case, it was trained on a couple of hours of improvisations that I sent to the IRCAM team, who then created an avatar twin, a musical doppelgänger. This new combination of the two AIs has been coined ‘SoVo’ by Lewis and his research team (Damon Holzborn at Columbia University, and Marco Fiorini and Gérard Assayag at IRCAM).

Blombos Playground puts me at the centre of a complex weaving of these two AIs. Each section provides me with scored material, drawing on musical gestures from the improvisations I had sent the team, ranging from meditative chorales to jazzy clusters to quicksilver passagework. At the end of each section, Lewis provides guidance on the many different ways of using this material to improvise with the AI co-performers. Using a MIDI keyboard, the sounds I make are as malleable as the AI’s. Sometimes we’re all using piano sounds, sometimes we’re duetting on harpsichord and celeste, and every combination of these three sounds. The spatialization of the sounds adds further layers to this musical conversation, using multiple speakers to place me and the different AIs in different corners of the room, have us chase each other about the room, or spin an AI’s sounds around me.

Although this might seem complicated and technical, in performance the duetting between the human and AI keyboardists feels visceral and immediate. There’s a logic to both the combinations of sounds and their movement around the space. The audience can sometimes hear the clear differences between human and AI, and sometimes they blend together, challenging the listener to distinguish between the real and virtual musicians.

But what does it feel like to play? For me, it feels like being inside the minds of these artificial improvisors as they interact with each other, and with me. I know that these software systems can’t think, but they seem to have personalities: Somax’s improvising persona is closer to mine (since it’s trained on my recordings), while Voyager is like a maverick free jazz improviser who’s comfortable being both an accompanist and a soloist. But they also play in a way that’s subtly different from human co-performers, and their ‘intelligence’ is unmistakably artificial and non-human. They don’t understand my intentions, and I know that these systems can’t intend or understand anything, despite seeming to have musical empathy.

The systems aren’t made to replace human performers, but to reveal to us new ways of thinking about improvisation, musical personality and musical communication. Real musicians are less virtuosic, and more fallible, but there’s a natural and deep understanding and connection when improvising with a real musician that can never be replicated. And what makes these systems interesting is not the AI technology itself, but the musical insights and choices that George Lewis and the Columbia/IRCAM team have programmed into them, using their own musical experience and instincts to create these virtual co- performers. What inspires me about Blombos Playground is that it’s pushed me to new kinds of virtuosity and made me think afresh about my own approaches to both performance and improvisation. It’s challenged my creative habits – and this artificial musical intelligence has made me a better musician.

Zubin Kanga 

Zubin Kanga will perform the world premiere of Blombos Playground by George Lewis on 17 October at the Purcell Room, Southbank Centre, London.

Never miss out on future posts by following us

The blog is free, but I'd be delighted if you were to show your appreciation by buying me a coffee.

Elsewhere on this blog

• Beautifully curated: Carolyn Sampson & Joseph Middleton explore all manner of obsession in Reason in Madness at Oxford Song Festival - review
• Britten in Love: Robin Tritschler & Malcolm Martineau explore the composer's emotional life through his music - concert review
• Immortal Mozart: Katharina Ruckgaber, Martin Häßler & Sholto Kynoch bring a light touch to their exploration of Schubert's admiration for Mozart - review 
• Lived experience: Matthias Goerne & Tamara Stefanovich in Schubert's Winterreise at Oxford International Song Festival - review
• Words & music: at Oxford International Song Festival's opening weekend academics & performers combine for Defining Love - concert review 
• Reinventing the lute song: countertenor Iestyn Davies & harpist Oliver Wass in John Dowland, Purcell, Nico Muhly & Britten at Hatfield House Music Festival  - review
• Delight in small things: the Telekom Beethoven Competition winner, Seth Schultheis in recital at Hatfield House Music Festival - review
• A real festival programme: varied lineups in music by Mozart, Britten, Shostakovich & Brahms at Hatfield House Music Festival - review
• Home