At Siggraph, Otoy unveils video pipeline for 'holodeck' tech.
The Emerging Technology exhibit alone features free-standing three-dimensional images that almost seem ready to plead “Help us, Obi-wan”; a combination of Oculus Rift virtual reality goggles and a treadmill-like system that makes you feel as if you’re walking through a virtual world; and “Birdly,” a combination of VR goggles and a contraption that resembles an upside-down dentist chair that lets viewers embody a bird flying over San Francisco, complete with a fan to simulate the rush of the wind.
But this year, one of the most mind-blowing things to be seen at Siggraph isn’t in Emerging Technologies, but in a corner of the trade show floor itself: A “holographic” video pipeline from Otoy that seems to be bringing futuristic tech to life surprisingly fast.
“We’re building the holodeck,” Otoy founder and CEO Jules Urbach says flatly, and while the experience he is showing isn’t exactly what Captain Picard and the Enterprise-D crew enjoyed, it is utterly different from anything consumers can experience today.
“In the holodeck, every square foot has a cube of light, and you can navigate through it and look anywhere,” says Urbach. “That is essentially what this technology is.”
Technically, Otoy isn’t showing “holographic” technology, but rather an “emissive light field.” “But we use the word hologram because most people have no idea what a light field is,” says Urbach unapologetically.
Light-field video reproduces every ray of light within a scene, so a viewer can look around, move through a scene and peek behind objects. That is the “holodeck” aspect of light-field tech. (The first consumer light-field still camera was unveiled at this year’s Consumer Electronics Show; it allows shutterbugs to adjust the focus after the photo has been shot. That’s only one thing that light-field cameras can offer.)
“Holographic video is it’s one of those Jetson-type of things, where it’s something everybody knows you have to have for you to feel like you’re in the future,” says Urbach, as he demonstrates light-field scenes on a conventional flatscreen TV (see video above).
“Basically, every single pixel looks like a real hologram,” he explains. “The entire scene is baked into the pixel. So you have everything you need to look at the scene from any angle in even the smallest fragment of that scene. And it decodes on a phone.”
A pixel in Otoy’s system is about half a millimeter across, says Urbach. That’s much larger than a pixel on a TV or phone, but it still takes tens of thousands of them for displays similar to today’s televisions.
Reproducing such complex information may seem like too great a computing challenge for today’s laptops or phones, let alone TVs, but Otoy solves that problem by taking rendering to the cloud. The company has long favored cloud-based rendering for its advanced video, and in this case, the heavy computing for holographic video is handled in the cloud. It takes just milliseconds to solve the math, and then the rendered image is sent to the viewer.
“That means that the screen can be a set of VR goggles,” says Urbach, “or it can be a cylinder, which we’re working on to create the Princess Leia kind of holographic display.”
Otoy’s booth on the Siggraph floor is showcasing a portable (but still bulky) light-field capture system and the cloud-based pipeline for rendering images. Otoy introduced a file format for holographic video in March. A true light-field display is on tap for 2015, the company says, and Urbach says a light-field capture system suitable for entertainment production is in the works and he expects it to be ready in 2015 as well. A true light-field capture system would give live-action directors options now available only in CG animation: The ability to place the “camera” and choose focus and depth-of-field settings in post, and to choose stereo 3D settings in post without complex conversion.
Urbach, who got into Harvard but was too successful at videogames to take time off to matriculate, has been working on advanced video for years. His company has backing from William Morris Endeavor; Ari Emanuel is on Otoy’s advisory board.
The first applications for Otoy’s tech are likely to be games, architectural walkthroughs, and other situations using CG worlds. But with a true light-field camera, live-action content would follow.
Urbach envisions light-field webcams, allowing real-time holographic viewing of scenes around the world. He says he hopes some day to replace his windows with screens showing light-field views of Central Park or other such scenes.