How Magic Leap, Video Games Are Defining Future of Royal Shakespeare Company

At the Royal Shakespeare Company in Stratford upon Avon, Sarah Ellis has the difficult job of figuring out where theater of the 1500s fits into the 21st century. As Director of Digital Development, a title which might seem out of place in an industry ruled by live, human performances, Ellis represents a recent seachange on the hallowed grounds of the RSC, to incorporate new virtual and augmented technologies with the work of a centuries-old playwright.

This spring, the RSC released its latest project made in partnership with creative technology company Magic Leap – a re-enactment of Shakespeare’s Seven Ages of Man from “As You Like It,” a soliloquy whose famous line “All the world’s a stage” encapsulates the company’s intentions for the piece. The RSC refers to this as “Tabletop Theatre,” pint-sized productions which can be staged on any surface.

“We wanted to start with a scene from Shakespeare and use it as a statement of intent for a wider ambition in this space,” Ellis tells “Variety.” “It’s a starting point for us to explore how we make theatre with the new technologies found in gaming and other industries, such as motion capture, photogrammetry, and volumetric capture.”

“Seven Ages of Man” begins after pulling on the Magic Leap One augmented reality headset, then unfolds on a small pedestal-size stage in front of you. The three-minute monologue is performed by Shakespearean actor Robert Gilbert, miniaturized for the size of the tiny set but still pacing the scene with the intensity of a live performer. Gilbert’s performance was initially recorded using volumetric capture, a process that takes video and creates a digital, three-dimensional space from it, making it possible for a live staged performance to be translated into augmented reality and played anywhere. Behind the actor grows a large tree, which ages as the performance pushes on, turning from a spring-time green to autumnal red. Leaves start to fall off its branches, leaving the tree barren as the monologue comes to an end.

Popular on Variety

“Theatre and technology have always had a relationship,” says Ellis. “And so this technology adds to our theatre-making tool kit and extends the possibilities for performance through the tools we have today. The experience of telling a story has always been through the best technology there is, which is the live actor. What’s important to us is how these new technologies add to that performance, take the storytelling further, give new options for directors and designers, allow us to shape, refine and redefine the experiences for our audiences is key.”

“What we’ve found with mixed reality is that it offers a shared experience whilst having a digital world presented to you through the technology. This allows us to think about the real world and the digital world simultaneously and bring those realities together. We’ve found audiences focus on the performance but also look at each other through the device and enjoy that collective and connected experience.”

The RSC first began exploring this space in collaboration with Intel for the 2017 production of “The Tempest,” which featured a full sensor-laden motion-capture performance for the play’s spirit-servant Ariel. Two years later, mixed reality entertainment is becoming more prevalent in the mainstream as the technology becomes more widely available.

“Over the past decades, we’ve seen our storytelling industries work in silos,” says Ellis of the often-dueling creative industries of cinema, television, theater, music and the Internet. “Audiences over time have been crossing these silos and defining their own parameters for content with their own storytelling channels such as YouTube, social media, online platforms such as Netflix and iPlayer for Broadcast, and now “Fortnite” has entered the entertainment space with their latest Marshmallo performance. These are the spaces audiences congregate and it’s how we position theatre in those spaces.”

Senior Director of Developer & Creator Relations at Magic Leap Greg Rinaldi imagines a future where all entertainment is consumed through mixed reality. No screens, no controllers, no mediation beyond a headset.

“I’m a huge sci-fi nerd,” he says. “I think that’s why I joined the company.”  

Prior to joining Magic Leap, Rinaldi worked in the video games industry with Electronic Arts creating massively multiplayer games, enormous online worlds for thousands of players to congregate. But after 15 years in a landscape mediated by controllers and screens, he now views these as impediments to virtual entertainment.

“Most of us experience the world with our faces in our phones. I don’t want to be doing this for the next 20 years. That doesn’t have to be the way I interact with the digital world. Our hope is that as the technology evolves, as it gets smaller and more powerful, that it will be natural to just wear a pair of glasses or interact with some new visual medium that gets created.”

“What we’re trying to work out here is how we make the interaction more natural where there isn’t a steep learning curve.”

Mixed reality, it should be said, isn’t exactly like its virtual counterpart. For one, there are technical differences. Unlike VR which uses digital stereo images to create its virtual worlds, the Magic Leap One harnesses “digital light fields” – a technique that provides the wearer with depth cues similar to those generated by objects in the real world. But perhaps the biggest difference is philosophical. VR envelopes you in another world; XR injects virtuality into what’s already really there.

Since its founding in 2010, Magic Leap has led a somewhat mysterious existence. The technology company is the brainchild of Rony Abovitz, a bioengineer involved in the creation of Mako Surgical Corp.’s surgery-assisting robot arms, and boasts staff members like Science-fiction author Neal Stephenson who maintains the curious job title of “Chief Futurist”. The technology company has also found success among the investor class, going on to rake in an estimated $1.4 billion in investments from the likes of Google and J.P. Morgan.

The design for Magic Leap One has evolved wildly throughout that period. Early on, during a time in the development when the team was referring to the device as The Beast, the Magic Leap One was the size of an industrial refrigerator and required users to stick their heads inside of it as if having their eyes checked by a monolithic optometrist. But the fundamental technology has always remained the same, says Rinaldi.

“I think VR is a fantastic technology frankly. It’s just fundamentally different. It’s about escapism,” says Greg Rinaldi, head of Magic Leap. “You can create really rich worlds in VR.”

“In virtual reality, you can create the space. You have complete control as the creator to make this other world. But with mixed reality, you have to respect the space that you’re in and you have to really understand how the digital content is interacting with the space, and then how the viewers are interacting with that space.”

“I think the reason we’re so excited about mixed reality is because you don’t have to remove people from the world,” he continues. “But I wouldn’t consider them competing technologies, they’re just different immersive mediums. It’s like film and radio almost.”

“What our technology allows you to do is we can map your space with sensors and cameras. We can create a three dimensional model of the room in real time. So the character will understand when they’re moving through the space: there is a table here, I can’t walk through the table, I have to stay on this side of the table, there’s a chair here, I can sit in the chair.”

“Storytelling over the next 10 to 20 years will be defined by the events of the day, what the generation before us has to say, and Shakespeare plays will continue to be entwined in society’s discourse of the day,” says Ellis.

“Over the next ten years, we as industries, sectors, genres will respond to audiences and their new meeting spaces. And we will see those stages converge, and the future of entertainment will be less siloed or defined by a single platform or experience. Content will become distributed, circulated and curated around the spectacle of an event. Theatre will hold this space as strongly as it always had, and it will be how theatre holds the multiple platforms and stages that are opening up to it. What will be important is to remember that with this opening up comes how we curate and architect those stories not forgetting the rituals that have been so intrinsic to how we experience a story for centuries.”

In January, the RSC announced the Audiences of the Future program, which explores the future of live performance with Marshmallow Laser Feast, Magic Leap, Intel, Epic Games, and others. The RSC launched its next exploration into theater and spatial computer in March, with the addition of six fellowships with the University of Portsmouth, the University of Goldsmiths, and i2Media Research.

“I think the possibilities for the Magic Leap device coming into how we make theatre are many, and we are currently exploring those avenues together,” Ellis says. “Exploring this collaboratively gives us an opportunity to imagine what a future production or performance could be whilst thinking about the many ways audiences now connect with content and storytelling. How theatre can exist in those spaces and also how can theatre drive some of the innovation in these areas.”

“As theatre-makers, we’ve been working in virtual reality for centuries so you could argue that the technology has caught up.”

More Legit

  • The Lehman Trilogy review

    Sam Mendes' 'Lehman Trilogy' Kicks off Ahmanson's New Season

    Sam Mendes’ “The Lehman Trilogy,” which took London’s West End by storm will be part of the Ahmanson’s lineup for the 2020-21 season. It will be joined by Broadway hits “Hadestown” and “To Kill a Mockingbird.” Artistic director Michael Ritchie announced the season that will also feature four fan favorites and another production to be [...]

  • Zoe Caldwell Dead

    Zoe Caldwell, Four-Time Tony Winner, Dies at 86

    Zoe Caldwell, an Australian actress with a talent for illuminating the human side of imposing icons such as Cleopatra and Maria Callas in a career that netted her four Tony Awards, died on Sunday due to complications from Parkinson’s disease, according to her son Charlie Whitehead. She was 86. Caldwell occasionally appeared in television and [...]

  • Cambodian Rock Band interview

    Listen: How 'Cambodian Rock Band' Became One of the Most Produced Plays in the U.S.

    One of the hottest trends in American theater this season is Cambodian surf rock from the 1970s — and that’s thanks to “Cambodian Rock Band.” Listen to this week’s Stagecraft podcast below: Playwright Lauren Yee’s genre-bending stage show, part family drama and part rock concert, has become one of the most-produced plays in the U.S. this season. [...]

  • Revenge Song

    Vampire Cowboys' 'Revenge Song': L.A. Theater Review

    There’s highbrow, there’s lowbrow, and then there’s however you might classify Vampire Cowboys, the anarchic New York City theater company whose diverse productions . It’s radical, “good taste”-flouting counter-programming for the vast swaths of the population left unserved by high-dollar, stiff-collar theater options. Vampire Cowboys’ raucous new show, “Revenge Song,” is unlike anything else that’s [...]

  • THE VISIT review

    'The Visit': Theater Review

    Director Jeremy Herrin’s extraordinary take on Friedrich Dürrenmatt’s 1956 play “The Visit” is less of a production and more of a show. A wordy one, to be sure, which is no surprise since it’s an adaptation by Tony Kushner that, including two intermissions, comes in at three-and-a-half hours. It’s never going to be described as [...]

  • Freestyle Love Supreme review

    'We Are Freestyle Love Supreme': Film Review

    For any Lin-Manuel Miranda fans whose hearts sank almost as quickly as they rose upon hearing that, yes, there’s a “Hamilton” movie, and no, it won’t be out for another 20 months, succor may be on the way in the form of a probably faster-arriving movie that features Miranda in almost as big a role, [...]

  • Unmasked review

    Andrew Lloyd Webber's 'Unmasked': Theater Review

    It takes guts to admit you were wrong — especially when you have been so right, so often. Take composer Andrew Lloyd Webber, whose successes with  “Joseph and the Amazing Technicolor Dreamcoat,” “Jesus Christ Superstar,” “Evita,” “The Phantom of the Opera,” “Sunset Boulevard,” and “School of Rock” have made him a musical-theater uber-Lord. Early on during [...]

More From Our Brands

Access exclusive content