For years, Tod Machover has run the “Opera of the Future” group at the MIT Media Lab in Cambridge, Mass. But he’s lived a double life. Not content to be a well-regarded composer, he’s also an inventor, creating instruments and robots that are transforming the way music is performed.
His latest futuristic opus, “Death and the Powers,” is a 90-minute one-act opera, with libretto by Robert Pinsky, directed by Diane Paulus. It incorporates cutting-edge robotics, and uses a technology Machover dubs “disembodied performance” that requires its star, baritone James Maddalena, to act out almost the entire program out of sight or earshot of the audience — even while a high-tech translation of his performance appears on the stage.
Production designer Alex McDowell, who worked with MIT to create the gestural computer interfaces for the film “Minority Report,” says of “Death and the Powers”: “I was taken places that were outside my experience or even my understanding, in some cases.”
“Death and the Powers” tells the story of Simon Powers, a businessman and inventor nearing the end of his life who decides to upload his consciousness into the computerized/robotic home he’s created, so the very walls come alive with his thoughts and feelings.
That means the singer playing Powers, Maddalena, has one scene onstage, then exits and never returns. Offstage, he dons a set of sensors that measure how he moves his arms and how fast and deeply he breathes, as he enters a miked isolation booth beneath the stage. Computer software analyzes his singing as his voice is delivered to the audience through an innovative speaker system that produces powerful sound without massive speaker enclosures.
“You really become part of the orchestra,” he says. “When you’re in front of an audience, there’s an energy that goes back and forth, and when you’re in a booth, there’s none of that. You really are isolated.”
Isolated, but not absent. Maddalena’s voice is also altered electronically, turning him into what Machover calls a “hyperinstrument” that produces its own natural sound and controls an electronic sound at the same time.
McDowell explains that since the star’s personality and performance needed to be conveyed by the set itself, the challenge was “to bring an inert design physically to life.” That begins with the set’s walls, which represent bookshelves but are actually LED videoscreens. Once Powers “goes into the System” (and Maddalena is hooked up to his sensors) software analyzes the performer’s singing, breathing and gestures to get a sense of his emotions, then translates those emotions to the LED videoscreens.
Says Machover, “A combined measurement of all elements from the singer give us information about how agitated or smooth, jumpy or continuous, geometric or blob-like, focused or diffuse, the visual patterns will be. Arm movement determines which direction the large-scale patterns move on the walls.”
“It’s a very direct, one-to-one, intuitive theatrical approach,” says McDowell.
Machover draws a distinction between disembodied performance and the performance-capture process used in such movies as “King Kong” and “Avatar.”
Performance capture depends on an actor’s movements to essentially drive a puppet, be it a CG character or a robot. But, says Machover, “What I wanted was the feeling of that muscle tension, of the gesture, the sense that we know that it’s this man having this particular emotion, but what we actually see is something very different.
“We might see a calligraphy of light on the screen that doesn’t actually look like his arm moving but has the same feeling to it. The result is you get closer to the feeling of that character.”
It was up to McDowell to turn that abstract idea into something the audience could grasp.
“I started by designing a number of objects that represented physical extremities. Robots for fingers, chandelier for brain, walls for skin,” he says.
The robots, dubbed “operabots,” which look a bit like a new-age drum pad on wheels, engage in a “Sorcerer’s Apprentice”-style dance with one of the characters in one scene.
In another, a stringed chandelier with a shape similar to a venus fly trap descends to interact with Powers’ wife in a strangely sensual duet as the couple recall how they met. She can play music directly on its strings, and the way she touches those strings affects the sound of Maddalena’s voice.
Machover sees applications for this technology in large arena performances, like rock concerts, where the aud watches the performers on large videoscreens.
“Frankly, to me, (videoscreens) usually make the live performers look small rather than making them look more present,” he says. Machover envisions concerts where the movement of the performances onstage determines the way the visuals look and the sound is in the hall.
McDowell agrees that the new art has enormous potential. “What’s really exciting to me is there’s … no limit now to what we can measure. The twitches of fingers or breath or acrobatics or position in space can turn into sound or color or animatronics or robotic performance.
“The implication in live performance is vast.”
“Death and the Powers” makes its American debut March 18 at the Cutler Majestic in Boston as a joint presentation of the American Repertory Theater and Opera Boston. It will run for a week in Boston and then move to the Harris Theater in Chicago for an April 2-10 engagement.
