Opera of the future

Robotic and music combo gives audience something new

For years, Tod Machover has run the “Opera of the Future” group at the MIT Media Lab in Cambridge, Mass. But he’s lived a double life. Not content to be a well-regarded composer, he’s also an inventor, creating instruments and robots that are transforming the way music is performed.

His latest futuristic opus, “Death and the Powers,” is a 90-minute one-act opera, with libretto by Robert Pinsky, directed by Diane Paulus. It incorporates cutting-edge robotics, and uses a technology Machover dubs “disembodied performance” that requires its star, baritone James Maddalena, to act out almost the entire program out of sight or earshot of the audience — even while a high-tech translation of his performance appears on the stage.

Production designer Alex McDowell, who worked with MIT to create the gestural computer interfaces for the film “Minority Report,” says of “Death and the Powers”: “I was taken places that were outside my experience or even my understanding, in some cases.”

Popular on Variety

“Death and the Powers” tells the story of Simon Powers, a businessman and inventor nearing the end of his life who decides to upload his consciousness into the computerized/robotic home he’s created, so the very walls come alive with his thoughts and feelings.

That means the singer playing Powers, Maddalena, has one scene onstage, then exits and never returns. Offstage, he dons a set of sensors that measure how he moves his arms and how fast and deeply he breathes, as he enters a miked isolation booth beneath the stage. Computer software analyzes his singing as his voice is delivered to the audience through an innovative speaker system that produces powerful sound without massive speaker enclosures.

“You really become part of the orchestra,” he says. “When you’re in front of an audience, there’s an energy that goes back and forth, and when you’re in a booth, there’s none of that. You really are isolated.”

Isolated, but not absent. Maddalena’s voice is also altered electronically, turning him into what Machover calls a “hyperinstrument” that produces its own natural sound and controls an electronic sound at the same time.

McDowell explains that since the star’s personality and performance needed to be conveyed by the set itself, the challenge was “to bring an inert design physically to life.” That begins with the set’s walls, which represent bookshelves but are actually LED videoscreens. Once Powers “goes into the System” (and Maddalena is hooked up to his sensors) software analyzes the performer’s singing, breathing and gestures to get a sense of his emotions, then translates those emotions to the LED videoscreens.

Says Machover, “A combined measurement of all elements from the singer give us information about how agitated or smooth, jumpy or continuous, geometric or blob-like, focused or diffuse, the visual patterns will be. Arm movement determines which direction the large-scale patterns move on the walls.”

“It’s a very direct, one-to-one, intuitive theatrical approach,” says McDowell.

Machover draws a distinction between disembodied performance and the performance-capture process used in such movies as “King Kong” and “Avatar.”

Performance capture depends on an actor’s movements to essentially drive a puppet, be it a CG character or a robot. But, says Machover, “What I wanted was the feeling of that muscle tension, of the gesture, the sense that we know that it’s this man having this particular emotion, but what we actually see is something very different.

“We might see a calligraphy of light on the screen that doesn’t actually look like his arm moving but has the same feeling to it. The result is you get closer to the feeling of that character.”

It was up to McDowell to turn that abstract idea into something the audience could grasp.

“I started by designing a number of objects that represented physical extremities. Robots for fingers, chandelier for brain, walls for skin,” he says.

The robots, dubbed “operabots,” which look a bit like a new-age drum pad on wheels, engage in a “Sorcerer’s Apprentice”-style dance with one of the characters in one scene.

In another, a stringed chandelier with a shape similar to a venus fly trap descends to interact with Powers’ wife in a strangely sensual duet as the couple recall how they met. She can play music directly on its strings, and the way she touches those strings affects the sound of Maddalena’s voice.

Machover sees applications for this technology in large arena performances, like rock concerts, where the aud watches the performers on large videoscreens.

“Frankly, to me, (videoscreens) usually make the live performers look small rather than making them look more present,” he says. Machover envisions concerts where the movement of the performances onstage determines the way the visuals look and the sound is in the hall.

McDowell agrees that the new art has enormous potential. “What’s really exciting to me is there’s … no limit now to what we can measure. The twitches of fingers or breath or acrobatics or position in space can turn into sound or color or animatronics or robotic performance.

“The implication in live performance is vast.”

“Death and the Powers” makes its American debut March 18 at the Cutler Majestic in Boston as a joint presentation of the American Repertory Theater and Opera Boston. It will run for a week in Boston and then move to the Harris Theater in Chicago for an April 2-10 engagement.

More Digital

  • Any Given Wednesday With Bill Simmons

    Spotify in Talks to Acquire Bill Simmons' The Ringer: Report

    Spotify is in early talks to acquire The Ringer, the digital content and podcast network launched by ESPN alum Bill Simmons in 2016, according to a report in the Wall Street Journal. A representative for Spotify declined to comment on the report. Reps for Ringer did not immediately respond to a request for comment. Spotify’s [...]

  • Ted Sarandos Milestone Award PGA

    Netflix Leader Ted Sarandos Plans Broader Creative Push for the Future

    Ted Sarandos, who has helped to upend the way audiences receive and consume entertainment as Netflix’s chief content officer, will be honored with the Milestone Award at the 31st annual Producers Guild Awards on Jan. 18. Game-changers including Steven Spielberg, Sherry Lansing, Robert Iger, Jeffrey Katzenberg and Ron Meyer have previously received the recognition, and [...]

  • peacock-ads-NBCU

    NBCU's Peacock Marks Media's Latest Try to Connect Consumers, Commercials

    In an era when consumers feel more empowered to avoid TV commercials, NBCUniversal is the latest media company to try to get them to bring ads back into their TV lives. One of the key selling points of big streaming-video services like Netflix and Disney Plus is that subscribers don’t have to endure the commercial [...]

  • OTT.X - Mark Fisher

    Entertainment Merchants Association (EMA) Gets Renamed OTT.X With Expanded Scope

    The Entertainment Merchants Association (EMA) home-video retailing trade group has changed its name to OTT.X, The Over The Top Exchange. The new name reflects the org’s broader scope and community to adapt to the evolution of the home-entertainment ecosystem, according to Mark Fisher, president and CEO of OTT.X. The group’s new website is at ottx.org. [...]

More From Our Brands

Access exclusive content