×
You will be redirected back to your article in seconds

Disney Researchers Create Mixed-Reality ‘Magic Bench’ for Shared Experiences

You’re sitting down in front of a screen looking at an image of yourself, as if facing a mirror. Then a tiny elephant saunters into the frame, plops down next to you — and proffers you a golden orb.

That’s one of the demos for Disney Research’s Magic Bench, a new spin on mixed-reality that lets multiple users interact with animated characters and other CGI elements together. Unlike typical augmented-reality or VR implementations, the prototype doesn’t use headsets or handheld devices.

“This demonstrates [human-computer interaction] in its simplest form: a person walks up to a computer, and the computer hands the person an object,” the researchers write in their paper describing the Magic Bench.

People seated on the Magic Bench, developed by a Disney R&D team in Pittsburgh, see a mirrored image on a large display in front of them, creating a third-person point of view. The scene is reconstructed using a depth sensor, allowing participants to occupy the same actual digital 3D space as computer-generated characters or objects, rather than superimposing one video feed onto another.

The goal of the Mouse House researchers was to create a “walk-up-and-play” experience. The bench contains haptic actuators, which provide vibrations that let users feel the presence of the CGI characters and objects. The Magic Bench also identifies the location and the number of participants, and can infer where they’re looking. In addition, different formations of people seated create different types of experiences.

“Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you,” Moshe Mahler, principal digital artist at Disney Research, said in a video about Magic Bench.

To crack the problem of “depth shadows” — which occur in areas where the depth sensor has no corresponding line of sight with the camera — the Magic Bench system uses a modified algorithm to create a 2D backdrop. The 3D and 2D reconstructions are positioned in virtual space and populated with 3D characters and effects in such a way that the resulting real-time rendering is a composite image capable of interacting with virtual physics, light, and shadows.

The Magic Bench’s 3D reconstruction uses a combination of the depth and color sensors on an off-the-shelf Microsoft Kinect motion-detection device. Here’s the technical explanation, for those interested in this level of detail: “We draw polygons using each point in the point cloud as a vertex, creating the appearance of a solid mesh. The mesh is then aligned to the RGB camera feed of the scene from the same Kinect. This alignment gives the mesh color, and completes a 3D reconstructed video feed.”

The Magic Bench was produced by a team of Disney Research scientists. In addition to Mahler, it includes Kyna McIntosh, John Mars, James Krahe, Jim McCann, Alexander Rivera, Jake Marsico, Ali Israr and Shawn Lawson.

The researchers will demo Magic Bench and present their paper on the project at the Siggraph 2017 computer graphics and interactive tech conference next week in L.A.

Watch a video explaining the concept:

More Digital

  • Lauren Dolgen Exits as BuzzFeed Studios

    Lauren Dolgen Exits as BuzzFeed Studios Boss After a Year

    Veteran TV producer Lauren Dolgen has departed as head of BuzzFeed Studios after one year in the job. BuzzFeed had hired Dolgen, a longtime MTV producer who worked on shows including “Teen Mom” and “16 and Pregnant,” as head of BuzzFeed Studios to oversee the company’s slate of original content. Based in L.A., she had [...]

  • Snap-Derek-Andersen-Lara-Sweet

    Snap Fills Out Senior Ranks With CFO, HR Chief Appointments

    Snap CEO and co-founder Evan Spiegel has turned in house to fill the company’s CFO and head of human resources positions, completing an overhaul of the Snapchat parent’s executive ranks after a series of high-level departures. The company announced the appointment of Derek Andersen, currently Snap’s VP of finance, as chief financial officer. Andersen will [...]

  • Oculus Quest Gets Netflix App, But

    Oculus Quest Will Have Netflix, but Not Plex or HBO

    Facebook’s new Oculus Quest headset is launching with a number of high-profile gaming titles this Tuesday. Media apps on the other hand will take a bit of backseat, with some key players sitting this latest headset out for the time being. First, the good news for anyone who has pre-ordered the headset: One of the [...]

  • Lester Holt

    Chris Berend Tapped to Oversee NBC News' Digital Efforts

    Chris Berend, an executive who has been overseeing digital video for CNN and helped launch Great Big Story, a streaming-video site aimed at younger audiences, will jump to NBC News to lead its digital efforts. Berend will replace Nick Ascheim, who will move into a new role that is described as “improving digital and product [...]

  • Vogue, Conde Nast Entertainment Make Robert

    Robert Semmer Joins Condé Nast Entertainment & Vogue as Vice President of Video

    Condé Nast Entertainment and Vogue have named former Vice and Fader video executive Robert Semmer as their new vice president of video. Semmer will be based in New York and report to Croi McNamara, senior vice president of programming for Condé Nast Entertainment. “Anna and I are thrilled to welcome Robert to our award-winning video [...]

  • CEO of T-Mobile John Legere (L)

    FCC Chairman Backs T-Mobile, Sprint Merger With New Conditions

    FCC Chairman Ajit Pai gave a thumbs-up to T-Mobile and Sprint’s proposed $26 billion merger, after the companies committed to enhanced 5G buildout commitments and agreed to spin off Sprint’s Boost Mobile. T-Mobile and Sprint first announced their plans to merge in April 2018, looking to combine forces to take on industry leaders AT&T and [...]

More From Our Brands

Access exclusive content