×

Disney Researchers Create Mixed-Reality ‘Magic Bench’ for Shared Experiences

You’re sitting down in front of a screen looking at an image of yourself, as if facing a mirror. Then a tiny elephant saunters into the frame, plops down next to you — and proffers you a golden orb.

That’s one of the demos for Disney Research’s Magic Bench, a new spin on mixed-reality that lets multiple users interact with animated characters and other CGI elements together. Unlike typical augmented-reality or VR implementations, the prototype doesn’t use headsets or handheld devices.

“This demonstrates [human-computer interaction] in its simplest form: a person walks up to a computer, and the computer hands the person an object,” the researchers write in their paper describing the Magic Bench.

People seated on the Magic Bench, developed by a Disney R&D team in Pittsburgh, see a mirrored image on a large display in front of them, creating a third-person point of view. The scene is reconstructed using a depth sensor, allowing participants to occupy the same actual digital 3D space as computer-generated characters or objects, rather than superimposing one video feed onto another.

The goal of the Mouse House researchers was to create a “walk-up-and-play” experience. The bench contains haptic actuators, which provide vibrations that let users feel the presence of the CGI characters and objects. The Magic Bench also identifies the location and the number of participants, and can infer where they’re looking. In addition, different formations of people seated create different types of experiences.

“Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you,” Moshe Mahler, principal digital artist at Disney Research, said in a video about Magic Bench.

To crack the problem of “depth shadows” — which occur in areas where the depth sensor has no corresponding line of sight with the camera — the Magic Bench system uses a modified algorithm to create a 2D backdrop. The 3D and 2D reconstructions are positioned in virtual space and populated with 3D characters and effects in such a way that the resulting real-time rendering is a composite image capable of interacting with virtual physics, light, and shadows.

The Magic Bench’s 3D reconstruction uses a combination of the depth and color sensors on an off-the-shelf Microsoft Kinect motion-detection device. Here’s the technical explanation, for those interested in this level of detail: “We draw polygons using each point in the point cloud as a vertex, creating the appearance of a solid mesh. The mesh is then aligned to the RGB camera feed of the scene from the same Kinect. This alignment gives the mesh color, and completes a 3D reconstructed video feed.”

The Magic Bench was produced by a team of Disney Research scientists. In addition to Mahler, it includes Kyna McIntosh, John Mars, James Krahe, Jim McCann, Alexander Rivera, Jake Marsico, Ali Israr and Shawn Lawson.

The researchers will demo Magic Bench and present their paper on the project at the Siggraph 2017 computer graphics and interactive tech conference next week in L.A.

Watch a video explaining the concept:

More Digital

  • USA Today Mobile Apps Get Oscars

    USA Today Launches Oscars AR Experience to Highlight Work of Costume Designers (EXCLUSIVE)

    USA Today is getting ready for the Academy Awards with an augmented reality (AR) experience dedicated to the work of the costume designers on some of the Oscar-nominated movies. The experience, which went live in USA Today’s Android and iOS app Monday, presents wardrobe from 6 movies in augmented reality, including costumes worn on “Mary [...]

  • Amazon Prime

    Amazon Prime India Greenlights ‘Bandits’ Music Series

    Amazon Prime Video India has greenlit original series “Bandish Bandits.” The show is a musical created by Still and Still Media Collective. The series will follow an Indian classical musician bound by tradition and a pop star whose performance skills are greater than her talent. A bandish is a term used to describe a musical [...]

  • Alibaba Buys 8% Stake in Chinese

    Alibaba Buys 8% Stake in Chinese Video Platform Bilibili

    Alibaba has purchased an 8% stake in the Chinese online video platform Bilibili, the official Xinhua news agency reported. Bilibili is one of China’s top video streaming and entertainment platforms, with about 92 million monthly active users and 450 million page-views per day. Founded in 2009, it was listed on the NASDAQ last March. Alibaba’s [...]

  • Clevver-Logo

    Hearst Magazines Buys Clevver's Pop-Culture YouTube Channels After Defy's Demise

    Hearst Magazines has snapped up Clevver, a network of female-skewing lifestyle and pop-culture news YouTube channels that had been owned by now-defunct Defy Media. Clevver was left homeless after Defy’s sudden shutdown in November; its principals said at the time they were looking for a new home. Hearst Magazines sees a digital fit with Clevver’s [...]

  • "Brother" -- Episode 201-- Pictured (l-r):

    CBS Interactive's Marc DeBevoise on Streaming Boom, Content Strategy, and Apple

    Not everyone wants or needs to be Netflix to succeed in the streaming space. And not everyone sees Apple’s enigmatic new service as a threat. Even as rival streaming services offer gobs of content, CBS Interactive’s president and COO Marc DeBevoise sees the company’s targeted original programming strategy continuing to attract viewers to its All [...]

  • Rhett-Link-Good-Mythical-Morning

    Rhett & Link's Mythical Entertainment in Talks to Acquire Smosh (EXCLUSIVE)

    Smosh, the YouTube comedy brand left stranded after parent company Defy Media went belly-up, may be about to get a new business partner. Mythical Entertainment, the entertainment company founded by top YouTube comedy duo Rhett & Link, has been in talks about acquiring the Smosh brand, sources told Variety. Multiple potential buyers came forward to [...]

  • Pokemon Go

    Proposed 'Pokémon Go' Lawsuit Settlement May Remove Poké Stops, Gyms

    A proposed settlement in the class action lawsuit against “Pokémon Go” developer Niantic could remove or change a number of Poké Stops and Gyms in the popular augmented reality game. The proposed settlement was filed in a California court on Thursday and applies to anyone in the U.S. who owns or leases property within 100 meters [...]

More From Our Brands

Access exclusive content