×
You will be redirected back to your article in seconds

Adobe Is Visualizing Sound to Simplify VR Video Editing (EXCLUSIVE)

The clapperboard has been a staple of Hollywood filmmaking for decades. But in an era of virtual reality (VR) 360-degree video, the simple tool may not be enough anymore, and new tools are needed.

One of these could be Adobe’s SonicScape project, which is set to be unveiled at the company’s Adobe Max conference in Las Vegas Thursday. “We’ve developed a new way of visualizing ambisonic audio,” explained Adobe Design lab lead senior experience developer Yaniv De Ridder during an exclusive preview of the project for Variety.

In many ways, SonicScape is a kind of 21st century extension of the clapperboard. First introduced when Hollywood started to add sound to its movies, the iconic tool is primarily used to synchronize recorded video with audio, allowing editors to precisely align the visual of the moment the board is closing with the resulting sharp clapping noise.

But with 360-degree video, time isn’t the only factor to consider. VR productions regularly combine 360-degree video cameras with special 3D microphones, with multiple individual capsules to capture sound from all directions. “Audio can come from anywhere,” said De Ridder.

This leaves video editors with the challenge to not only synchronize audio by time, but also by space. They have to make sure that what you see while wearing a VR headset actually matches what you hear, and that the sound for the action in front of you isn’t accidentally placed behind you.

Adobe’s approach with SonicScape is to visualize the sound by overlaying colorful spots over the video that also correspond to the sound’s frequency and intensity. With these visual cues, editors simply have to adjust the sound so those spots are where the action is. They can also use it to add new assets to an existing video, and for example include ambient sounds to improve the soundscape of a 360-degree scene. “We are trying to take the guesswork out of it,” said Adobe Design Lab senior experience designer Michael Cragg.

SonicScape is currently a prototype developed by Adobe’s Design Lab, and not yet part of any of the company’s products. But the team behind it had a clear mandate to solve real-world problems, which is why it did its research before working on the product. “We talked to a lot of VR video editor,” said Cragg. The team hopes that SonicScape could one day help not only to simplify high-end productions, but also make 360-video with binaural audio more accessible to YouTube video producers and others with limited budgets.

Adobe has prototyped a number of VR projects in recent months. The company has experimented with VR advertising as well as using cutting-edge tech to bring depth to flat 360-degree videos. In the case of SonicScapes, the prototype could find its way into actual products sooner rather than later, said Adobe’s director of Immersive Chris Bobotis. “Audio is extremely important.”

Bobotis joined Adobe this summer when it acquired Mettle Skybox tools, a VR video production tool suite developed by Mettle, the company he had co-founded two decades ago. Now, he is focused on adding VR tools to all of Adobe’s editing products, all the way down to PhotoShop. “Our goal is to make it faster, make it easier,” he said.

That’s especially important in an emerging field like VR video where the problems quickly multiply, and files are by nature huge and unwieldy. “Too much of the budget is eaten up by data management,” said Bobotis. Adobe’s role in VR would be to simplify some of the fundamentals, so that editors can spend more time on creative work — even if that means reinventing the clapperboard for the age of 360-degree video.

Correction: 10:20am: An earlier version of this story stated that Adobe had acquired Mettle. The company actually just acquired Mettle Skybox tools.

More Digital

  • Missing Link Laika Studios

    ‘Missing Link’ Again Tops Studios’ TV Ad Spending

    In this week’s edition of the Variety Movie Commercial Tracker, powered by the TV ad measurement and attribution company iSpot.tv, Annapurna Pictures claims the top spot in spending for the second week in a row with “Missing Link.” Ads placed for the animated film had an estimated media value of $5.91 million through Sunday for [...]

  • Sirius Logo

    SiriusXM Unveils $8 Essential Plan for Consumers Without Cars

    SiriusXM wants to cater consumers without cars, or cars without compatible stereos, with a new $8 plan for mobile and in-home listening. Dubbed SiriusXM Essential, the plan offers access to 200+ channels featuring the network’s entire music programming, as well comedy, news and select sports channels. Consumers will be able to test the new plan [...]

  • Mueller Report Book Editions Top Amazon's

    Mueller Report Book Editions Shoot to Top of Best-Seller Lists at Amazon, Barnes & Noble

    Robert Mueller is now a best-selling author. Book publishers’ forthcoming editions of the special counsel’s report zoomed to the top of the Amazon’s and Barnes & Noble’s lists of book best-sellers Friday. That comes a day after the report was publicly released, culminating the nearly two-year investigation into Russian interference in the 2016 U.S. election [...]

  • Marques Brownlee - Retro Tech

    YouTube Orders Marques Brownlee 'Retro Tech' Original Series

    YouTube has turned to one of its homegrown stars — technology vlogger Marques Brownlee, aka “MKBHD” — for its newest original series. The video platform has greenlit series “Retro Tech” featuring Brownlee, in which he’ll unbox and review vintage technology products that have defined pop culture. The show, slated to debut in December 2019, follows [...]

  • Netflix Tests Random Episode Button in

    Netflix Starts Testing Random Episode Button

    Netflix is testing a button to play random episodes of select TV shows, the company confirmed Friday morning. “We are testing the ability for members to play a random episode from different TV series on the Android mobile app,” a Netflix spokesperson told Variety. “These tests typically vary in length of time and by region, [...]

  • Netflix Our Planet Sophie Darlington

    Netflix's 'Our Planet' Roars to Life With Work by Top Wildlife Cinematographers

    In terms of scope, production time and — very likely — budget, Netflix’s “Our Planet” is one of the most ambitious projects from the streaming service to date. Narrated by David Attenborough and made available worldwide on April 5, the goal of the eight-part series is to capture diverse habitats across the globe and highlight [...]

More From Our Brands

Access exclusive content