Living Archive: Creating Choreography with Artificial Intelligence
Delve into the ground-breaking experiment of creating a tool for choreography powered by machine learning
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
For his latest world premiere, Living Archive: An AI Performance Experiment, visionary choreographer Wayne McGregor utilised a revolutionary, artificially intelligent choreographic tool, created in collaboration with Google Arts and Culture Lab, and trained on 25 years of his video archive. The groundbreaking, new dance work performed by Company Wayne McGregor, is set to In Seven Days, a gripping and dynamic concerto by progressive British composer, Thomas Adès, and premiered at The Music Center in Los Angeles in July 2019 as part of Adès and McGregor: A Dance Collaboration.
Becoming by Wayne McGregor, Marc Downie, Nick RothwellStudio Wayne McGregor
Deeply invested in technological advancement and scientific research into creativity and choreographic thinking throughout his stellar career, McGregor has long been fascinated by artificial intelligence as a potential dance-making tool.
Plus Minus Human by Wayne McGregorStudio Wayne McGregor
Previously, he collaborated with cognitive and social scientists on interactive, choreographic software, and later worked with Random International’s programmed flock of drones for +/- Human, an immersive installation and performance by Company Wayne McGregor and dancers from The Royal Ballet at London’s Roundhouse.
Jenna Sutela | n-Dimensions by Jenna SutelaSomerset House
Google’s Arts and Culture Lab facilitates extraordinary, interdisciplinary breakthroughs by bringing intrepid artists and curators together with world-class engineers and creative technologists to foster experiments at intersections of art and technology.
Nemesis (2002-01-25) by Wayne McGregorStudio Wayne McGregor
McGregor initially approached The Lab seeking an innovative way to harness his vast video archive, to place it in dialogue with his present-day practice in a bid to generate something wholly new, ahead of the 25th anniversary of his pioneering dance company. At the forefront of his mind, were the questions: What is choreography? Who has to make choreography? What are the potentials of choreography?
Living Archive by Google Arts and Culture Lab, Studio Wayne McGregorStudio Wayne McGregor
Damien Henry, Head of Innovation at Google Arts and Culture for the past five years, works with artists and curators, engineers and creative coders to explore bleeding edge possibilities of Virtual Reality and Machine Learning technologies. In his words, his abiding intention is to “create things that have never been done before … It’s interesting for artists to work with us as we help by providing technology and, in return, we gain insight into what we do and what we can do – into what technology is and isn’t good for.”
Sulphur 16 (1998-01-29) by Wayne McGregorStudio Wayne McGregor
As Google’s mission is to organise information, rendering it understandable and useful, Wayne McGregor’s video archive, which would take a human being a fortnight to watch, was an attractive proposition as a highly specific data set. But what really excited Henry about collaborating with the choreographer was his affinity for technology and tireless pursuit of progress.
Living Archive by Google Arts and Culture Lab, Studio Wayne McGregorStudio Wayne McGregor
Having christened their joint project, Living Archive, Henry and McGregor soon alighted on the idea of creating an algorithm that predicts future movements such as a dancer’s next gesture. “Wayne and I were completely enlivened by this [prospect],” says Henry. “I found some technology related to handwriting prediction, which focused on hand [motions] to predict the shape of the next letter, and the idea was to attempt to extend this technology to the full body.”
brainstate by Wayne McGregorStudio Wayne McGregor
Henry and his team, which includes creative coder, Bastien Girschig, began by training machine learning algorithms, classed as recursive neural networks (RNN), on McGregor’s video database.
The algorithms had to parse every movement within 25 years’ worth of dance so that they might learn, as Henry states, “how the body behaves, how dancers, specifically, behave – to be able to predict the next movement from any given [action]. This was really the core of the project.”
Living Archive by Google Arts and Culture Lab, Studio Wayne McGregorStudio Wayne McGregor
As this algorithmic education progressed, iteration by iteration, McGregor provided additional video material of current company members in rehearsal, so that the nascent AI choreographic tool might fully absorb and mimic the precise physical signatures of the ten dancers who would eventually make up the cast of Living Archive: An AI Performance Experiment.
During these processes, the tool’s composite algorithms detect and extract a skeletal render of the lines between key points in the human body – much like motion capture software.
Living Archive by Google Arts and Culture Lab, Studio Wayne McGregorStudio Wayne McGregor
After extensive development work by the team at The Lab, the tool grew capable of presenting 30 predictive, choreographic options to follow any given movement sequence by any single dancer – one option a piece in the unique style of each of the ten cast, as devised by three key algorithms. The first, and most fundamental, an RNN that ‘memorises’ past data to suggest future gestures.
Living Archive by Google Arts and Culture Lab, Studio Wayne McGregorStudio Wayne McGregor
While the other two algorithms, one two-dimensional (a t-SNE network), the other multi-dimensional, are focused on the proximity of dance poses within the database.
“You can think about these algorithms as reorganising the archive by similarity,” says Henry. “So you connect different gestures from different videos,
and then build a gigantic network of [affiliated] poses. And the way they continue a sequence is by navigating abstract representations of all the poses within these networks.”
Nemesis (2002-01-25) by Wayne McGregorStudio Wayne McGregor
McGregor was duly impressed with the results. “In a way, it’s like a massive atlas that allows you to zone in on any one physical movement, or to move through movements across bodies, to make the representation of a body do something that’s never been done before,” he shares. “It’s a brilliant way of organising physical intelligence that you then use to develop new physical ideas.”
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
The generation of movement possibilities as a responsive source of inspiration for human creators was always the goal of the Living Archive project – rather than the crafting of technology to replace any single soul. “What's interesting is that what we've given to Wayne is just one tool in the creation process,” says Henry. “It offers him and his dancers multiple options, but there is always a human in the loop, because there isn’t an algorithm that can judge the quality of the choreography.”
UNDANCE (2011-12-01) by Wayne McGregorStudio Wayne McGregor
As McGregor explains, “When I’m working with dancers normally, what I’m asking them to do, through their own creativity, is to make iterative versions of an idea that I might have proposed.
Living Archive by Google Arts and Culture Lab, Studio Wayne McGregorStudio Wayne McGregor
What this tool does is make the canvas way, way bigger for each little moment … It allows us to go: I’m starting with this phrase; I’d like the AI to invent the next, but in the style of Jordan, or in the style of Jess. And you can get combinations of those, and then it’s [constantly] learning and feeding back, so this iterative version gives you all these possibilities that you couldn’t have imagined.”
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
A fundamental facet of the AI choreographic tool is that it’s able to ingest new movement material and suggest continuations in real time. While McGregor was making Living Archive: An AI Experiment, in the studio with his dancers, this novel, digital collaborator was able to capture three-five seconds of input via webcam, and instantaneously display 30 original movement sequences of approximately five-ten seconds on screen that the choreographer and/or dancers might then opt to work with.
Living Archive by Google Arts and Culture Lab, Studio Wayne McGregorStudio Wayne McGregor
These AI-derived movement are rendered on screen via both stick figures and (intentionally) partially-abstracted, human forms – any one of which might momentarily act as a supplementary ‘eleventh dancer’.
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
Rather than copying the AI-derived movements, McGregor and the Company’ dancers translate inspiring elements of these suggested motions through their own thinking bodies: real-time, human-machine interaction.
Autobiography by Wayne McGregorStudio Wayne McGregor
The experimental new dance also features further, phenomenal work stemming from the Living Archive project, in the guise of a video piece designed by a collaborator of McGregor’s Ben Cullen Williams. The prolific, British visual artist, who has a keen interest in the sculptural qualities of the human body in space, and relationships between the physical and digital realms, works across mediums to, as he says, “explore our engagement with the world’s phenomenological elements … how we connect with space, light, materials and form to create emotional change.”
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
For the video component, created by Cullen Williams in collaboration with McGregor, twenty new, fifteen-to-twenty minute sequences of dance were autonomously produced by algorithms devised by Henry and his team, each borne of a different ‘seed’ of raw material. “From the root movements, we generated potential continuations in the styles of the dancers,” says Henry. “This raw data, our source, is then transformed into visuals by several, specific algorithms that are able to recreate video using machine learning.”
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
The presentation of every individual video sequence sits along a spectrum, from inescapably digital, abstract imagery to a composite figure that closely represents a human. “When I sat down with Henry’s team, I said, ‘Well if a computer could dance, what would the computer want to look like?’” says Cullen Williams. “It's actually the computer code that's dancing, and then we’re slowly moving up through a spectrum of code [until, eventually] it resembles the outline of a dancer.”
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
Cullen Williams then manipulated these videos using a suite of Google software, before adjusting them to Adès’ music. After which, he edited his materials in tandem with McGregor, his dancers and the choreographic tool making Living Archive: An AI Experiment in the studio. “There's a synergy in everything being created within the same week,” says the artist. “It's a testing lab of ideas … so rather than being a perfect [whole], there'll be moments when things synch up and points where everything diverges.”
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
On stage at The Music Centre in LA, the intricate video piece was presented on a huge, semi-transparent LED screen with visible wiring, specially selected by Cullen Williams for its avowedly technological aesthetic. “Wayne's dancers are like super humans,” he says, “They're so unbelievably toned and supple, so I thought having them [perform] next to this screen made for interesting, conversational starting point for the viewers.”
Living Archive: An AI Performance Experiment (2019-07-12) by Wayne McGregorStudio Wayne McGregor
Beyond the Living Archive’s artistic applications, the choreographic tool is being assessed for educational purposes by Studio Wayne McGregor’s Director of Learning and Engagement, Jasmine Wilson. How this aspect of the project will manifest is yet to be defined, but Wilson posits that it might function as “a creative stimulus, as a way of getting people moving”.
'Living Archive' text written by Suze Olbrich.
Images of Living Archive experiment, courtesy of Google Arts & Culture.
Images of Company Wayne McGregor in rehearsal for Living Archive: An AI Performance Experiment, taken at Studio Wayne McGregor by Camilla Greenwell.
Images of Company Wayne McGregor performing Living Archive: An AI Performance Experiment, captured at the Dorothy Chandler Pavilion by Cheryl Mann. Used by permission of the LA Phil.
