×
You will be redirected back to your article in seconds

Disney Researchers Create Mixed-Reality ‘Magic Bench’ for Shared Experiences

You’re sitting down in front of a screen looking at an image of yourself, as if facing a mirror. Then a tiny elephant saunters into the frame, plops down next to you — and proffers you a golden orb.

That’s one of the demos for Disney Research’s Magic Bench, a new spin on mixed-reality that lets multiple users interact with animated characters and other CGI elements together. Unlike typical augmented-reality or VR implementations, the prototype doesn’t use headsets or handheld devices.

“This demonstrates [human-computer interaction] in its simplest form: a person walks up to a computer, and the computer hands the person an object,” the researchers write in their paper describing the Magic Bench.

People seated on the Magic Bench, developed by a Disney R&D team in Pittsburgh, see a mirrored image on a large display in front of them, creating a third-person point of view. The scene is reconstructed using a depth sensor, allowing participants to occupy the same actual digital 3D space as computer-generated characters or objects, rather than superimposing one video feed onto another.

The goal of the Mouse House researchers was to create a “walk-up-and-play” experience. The bench contains haptic actuators, which provide vibrations that let users feel the presence of the CGI characters and objects. The Magic Bench also identifies the location and the number of participants, and can infer where they’re looking. In addition, different formations of people seated create different types of experiences.

“Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you,” Moshe Mahler, principal digital artist at Disney Research, said in a video about Magic Bench.

To crack the problem of “depth shadows” — which occur in areas where the depth sensor has no corresponding line of sight with the camera — the Magic Bench system uses a modified algorithm to create a 2D backdrop. The 3D and 2D reconstructions are positioned in virtual space and populated with 3D characters and effects in such a way that the resulting real-time rendering is a composite image capable of interacting with virtual physics, light, and shadows.

The Magic Bench’s 3D reconstruction uses a combination of the depth and color sensors on an off-the-shelf Microsoft Kinect motion-detection device. Here’s the technical explanation, for those interested in this level of detail: “We draw polygons using each point in the point cloud as a vertex, creating the appearance of a solid mesh. The mesh is then aligned to the RGB camera feed of the scene from the same Kinect. This alignment gives the mesh color, and completes a 3D reconstructed video feed.”

The Magic Bench was produced by a team of Disney Research scientists. In addition to Mahler, it includes Kyna McIntosh, John Mars, James Krahe, Jim McCann, Alexander Rivera, Jake Marsico, Ali Israr and Shawn Lawson.

The researchers will demo Magic Bench and present their paper on the project at the Siggraph 2017 computer graphics and interactive tech conference next week in L.A.

Watch a video explaining the concept:

More Digital

  • Nick Eh 30, StoneMountain64

    CAA Signs Gaming Influencers Nick Eh 30, StoneMountain64

    CAA has signed two popular live-streaming battle royale gamers — Nicholas Amyoony, better known as Nick Eh 30, and David Steinberg, aka StoneMountain64 — for representation. CAA said it will work to create opportunities for Nick Eh 30 (above left) and StoneMountain64 (above right) in all areas, including TV, motion pictures, touring, digital distribution and [...]

  • Amazon Adds NBA League Pass to

    Amazon Adds NBA League Pass Live-Streaming Games to Prime Video Channels

    Amazon is bringing pro hoops action to Prime Video Channels: The ecommerce giant now offers NBA League Pass as part of its lineup of 150-plus subscription options for Prime members in the U.S. It’s the first U.S. live-sports subscription service on Prime Video Channels. Prime members can subscribe to NBA League Pass via Prime Video [...]

  • Fifty Shades of Grey

    International Piracy Ring Stole Over 25,000 Movie and TV Digital Files, U.S. Feds Say

    A five-member international hacking crime ring stole more than 25,000 files for Hollywood movies and TV shows and illegally offered hundreds of them for sale online, according to U.S. law enforcement officials. A federal grand jury in L.A. on Wednesday (Dec. 12) indicted five men, identified as residing in the U.K., India, Dubai and Malaysia, [...]

  • Phil Schiller, Apple's senior vice president

    Apple to Update iPhones in China to Avoid Sales Ban

    Apple is pushing out a software update to iPhones in China to address a recent court order obtained by Qualcomm that banned the sale of certain iPhone models in the country. The update will allow the company to continue selling its phones in China, company representatives told Reuters Friday morning. The Fuzhou Intermediate People’s Court [...]

  • Kevin Reilly Variety Cover

    Kevin Reilly Named Content Chief for WarnerMedia Streaming Service

    Kevin Reilly — a veteran television exec who has led programming at NBC, FX, Fox, and most recently Turner Broadcasting — has been tapped to head content strategy for the still-nascent streaming service that WarnerMedia plans to launch next year. Reilly, who has led programming at Turner brands TNT and TBS since 2015, will serve [...]

  • Apple Culver City

    Apple Expects to Have Over 1,000 Employees in Culver City by 2022

    Apple announced plans to boost its L.A. presence, saying it expects to have more than 1,000 employees in Culver City, Calif., over the next three years, including its growing entertainment team. The move is part of the tech giant’s broader initiative to create 20,000 jobs in the U.S. by 2023 and includes a new $1 [...]

More From Our Brands

Access exclusive content