×
You will be redirected back to your article in seconds

Disney Researchers Create Mixed-Reality ‘Magic Bench’ for Shared Experiences

You’re sitting down in front of a screen looking at an image of yourself, as if facing a mirror. Then a tiny elephant saunters into the frame, plops down next to you — and proffers you a golden orb.

That’s one of the demos for Disney Research’s Magic Bench, a new spin on mixed-reality that lets multiple users interact with animated characters and other CGI elements together. Unlike typical augmented-reality or VR implementations, the prototype doesn’t use headsets or handheld devices.

“This demonstrates [human-computer interaction] in its simplest form: a person walks up to a computer, and the computer hands the person an object,” the researchers write in their paper describing the Magic Bench.

People seated on the Magic Bench, developed by a Disney R&D team in Pittsburgh, see a mirrored image on a large display in front of them, creating a third-person point of view. The scene is reconstructed using a depth sensor, allowing participants to occupy the same actual digital 3D space as computer-generated characters or objects, rather than superimposing one video feed onto another.

The goal of the Mouse House researchers was to create a “walk-up-and-play” experience. The bench contains haptic actuators, which provide vibrations that let users feel the presence of the CGI characters and objects. The Magic Bench also identifies the location and the number of participants, and can infer where they’re looking. In addition, different formations of people seated create different types of experiences.

“Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you,” Moshe Mahler, principal digital artist at Disney Research, said in a video about Magic Bench.

To crack the problem of “depth shadows” — which occur in areas where the depth sensor has no corresponding line of sight with the camera — the Magic Bench system uses a modified algorithm to create a 2D backdrop. The 3D and 2D reconstructions are positioned in virtual space and populated with 3D characters and effects in such a way that the resulting real-time rendering is a composite image capable of interacting with virtual physics, light, and shadows.

The Magic Bench’s 3D reconstruction uses a combination of the depth and color sensors on an off-the-shelf Microsoft Kinect motion-detection device. Here’s the technical explanation, for those interested in this level of detail: “We draw polygons using each point in the point cloud as a vertex, creating the appearance of a solid mesh. The mesh is then aligned to the RGB camera feed of the scene from the same Kinect. This alignment gives the mesh color, and completes a 3D reconstructed video feed.”

The Magic Bench was produced by a team of Disney Research scientists. In addition to Mahler, it includes Kyna McIntosh, John Mars, James Krahe, Jim McCann, Alexander Rivera, Jake Marsico, Ali Israr and Shawn Lawson.

The researchers will demo Magic Bench and present their paper on the project at the Siggraph 2017 computer graphics and interactive tech conference next week in L.A.

Watch a video explaining the concept:

More Digital

  • Jim PackerPRESS PLAY: Variety Home Entertainment

    FilMart: Jim Packer Says Liongate Ready to Support Starz Global Rollout

    A keynote speaker at Hong Kong’s FilMart this week, Jim Packer, Lionsgate’s president of worldwide TV and digital distribution, shares plans to support Starz’ international expansion. And he recounts his experience of watching Netflix change up through the gears. Back in 2012, when Lionsgate was still casting “Orange is the New Black,” Jim Packer, Lionsgate’s [...]

  • Devin Nunes

    Devin Nunes Files $250 Million Defamation Lawsuit Against Twitter

    U.S. Rep. Devin Nunes filed a $250 million defamation lawsuit against Twitter on Monday, alleging that he has been the victim of hundreds of hate-filled tweets. The suit also takes aim at Liz Mair, a Republican communications consultant who has attacked Nunes on Twitter, and the anonymous accounts “Devin Nunes’ Mom” and “Devin Nunes’ Cow.” [...]

  • Reed Hastings Netflix

    Netflix Won't Be Part of Apple's Video Service, CEO Reed Hastings Confirms

    Netflix CEO Reed Hastings confirmed at a press event in Los Angeles Monday that his company won’t be taking part in Apple’s upcoming video service. “Apple is a great company,” Hastings said. However, Netflix wanted to control its experience within its own app, which is why it wasn’t working with Apple. “We have chosen not [...]

  • mark Beaven

    If Spotify Is Holding Town Halls for Songwriters, They Must Be Open (Guest Column)

    Earlier this month, Amazon, Google, SiriusXM and Spotify challenged the Copyright Royalty Board’s decision to increase the compulsory mechanical rates paid to songwriters by 44% over the next five years. The streamers have come under fierce criticism for the move, which they claim is over the complexities of the CRB’s rules but is widely assumed [...]

  • iHeartMedia Promotes Angel Aristone to Executive

    iHeartMedia Promotes Angel Aristone to Executive VP of Communications

    Angel Aristone has been promoted to executive vice president of communications for iHeartMedia, the company announced today. According to the announcement, Aristone will continue to position iHeart as a media and entertainment leader through proactive strategic communication efforts on both a local and national level. She will also continue to oversee media relations and external [...]

More From Our Brands

Access exclusive content