×
You will be redirected back to your article in seconds

Disney Researchers Create Mixed-Reality ‘Magic Bench’ for Shared Experiences

You’re sitting down in front of a screen looking at an image of yourself, as if facing a mirror. Then a tiny elephant saunters into the frame, plops down next to you — and proffers you a golden orb.

That’s one of the demos for Disney Research’s Magic Bench, a new spin on mixed-reality that lets multiple users interact with animated characters and other CGI elements together. Unlike typical augmented-reality or VR implementations, the prototype doesn’t use headsets or handheld devices.

“This demonstrates [human-computer interaction] in its simplest form: a person walks up to a computer, and the computer hands the person an object,” the researchers write in their paper describing the Magic Bench.

People seated on the Magic Bench, developed by a Disney R&D team in Pittsburgh, see a mirrored image on a large display in front of them, creating a third-person point of view. The scene is reconstructed using a depth sensor, allowing participants to occupy the same actual digital 3D space as computer-generated characters or objects, rather than superimposing one video feed onto another.

The goal of the Mouse House researchers was to create a “walk-up-and-play” experience. The bench contains haptic actuators, which provide vibrations that let users feel the presence of the CGI characters and objects. The Magic Bench also identifies the location and the number of participants, and can infer where they’re looking. In addition, different formations of people seated create different types of experiences.

“Our mantra for this project was: hear a character coming, see them enter the space, and feel them sit next to you,” Moshe Mahler, principal digital artist at Disney Research, said in a video about Magic Bench.

To crack the problem of “depth shadows” — which occur in areas where the depth sensor has no corresponding line of sight with the camera — the Magic Bench system uses a modified algorithm to create a 2D backdrop. The 3D and 2D reconstructions are positioned in virtual space and populated with 3D characters and effects in such a way that the resulting real-time rendering is a composite image capable of interacting with virtual physics, light, and shadows.

The Magic Bench’s 3D reconstruction uses a combination of the depth and color sensors on an off-the-shelf Microsoft Kinect motion-detection device. Here’s the technical explanation, for those interested in this level of detail: “We draw polygons using each point in the point cloud as a vertex, creating the appearance of a solid mesh. The mesh is then aligned to the RGB camera feed of the scene from the same Kinect. This alignment gives the mesh color, and completes a 3D reconstructed video feed.”

The Magic Bench was produced by a team of Disney Research scientists. In addition to Mahler, it includes Kyna McIntosh, John Mars, James Krahe, Jim McCann, Alexander Rivera, Jake Marsico, Ali Israr and Shawn Lawson.

The researchers will demo Magic Bench and present their paper on the project at the Siggraph 2017 computer graphics and interactive tech conference next week in L.A.

Watch a video explaining the concept:

Popular on Variety

More Digital

  • David Nevins, President & CEO of

    CBS Creative Chief David Nevins on His Plans to Compete With HBO Max

    Appearing at Recode’s Code Media conference in Los Angeles on Monday afternoon, CBS chief creative officer David Nevins didn’t hold back criticizing one of his biggest competitors: “If you are going to put out one main product, you have to be as mainstream as you can,” Nevins said about WarnerMedia’s plans for HBO Max. Minutes [...]

  • Amazon Music Free

    Amazon Music Expands Access to Free Streaming Service, Spotify Stock Falls

    The free version of Amazon’s music-streaming service is now available across more platforms, bolstering competition with Spotify’s free tier. Starting Monday, Amazon said, customers can now listen to Amazon Music’s ad-supported selection of top playlists and thousands of stations for free. It’s available to anyone in the U.S., U.K. and Germany who doesn’t have a [...]

  • John T. Stankey WarnerMedia CEO

    WarnerMedia CEO John Stankey on Early Disney Plus Success: 'Good for Them'

    WarnerMedia CEO John Stankey told the audience of Recode’s Code Media conference in Los Angeles Monday that Disney “did a good job” with Disney Plus, which announced that it reached 10 million sign-ups the day after the service’s launch. “Good for them,” Stankey said. “They are off to a good start.” However, he also cautioned [...]

  • Google Stadia

    Google Stadia Reviews: Game-Streaming Service Not Ready for Primetime

    The first takes on Google Stadia, the internet giant’s new game-streaming service that debuts Nov. 19, are out. And the consensus is that Stadia is needed more time in the oven before it was served up to the gaming crowd. Google’s big promise with Stadia is that players can access top-tier game titles without a [...]

  • 60 Minutes" reporter Lara Logan takes

    Lara Logan Will Host Fox Nation Documentary Series

    Lara Logan made a name for herself by reporting from global trouble spots for CBS News. Soon, she will be holding forth on a four-part docuseries for Fox Nation, the Fox News streaming-video service. Logan will present “No Agenda,” which will debut in January on Fox Nation, and is produced by Warm Springs Productions. She [...]

  • warnermedia lab hires

    WarnerMedia Hires Jeremy Toeman, Matthew Davis as Innovation Lab Execs (EXCLUSIVE)

    WarnerMedia is building out the leadership of its Innovation Lab: The AT&T-owned media company has hired Jeremy Toeman as vice president, technology and product, and Matthew Davis as vice president, creative. Both will be based in New York and report to WarnerMedia executive vice president Jesse Redniss, who is also the general manager of the [...]

  • ABC News Live LG

    Xumo Adds ABC News Live Just In Time for the 2020 Election

    Ad-supported streaming video service Xumo is doubling down on news: Xumo is adding ABC News Live to both its owned-and-operated apps as well as to white-labeled services like LG’s Channels video app in the U.S. and Canada. With Aermicans glued to their TVs for the impeachment hearings, and the 2020 election fast approaching, the launch [...]

More From Our Brands

Access exclusive content