Experimental Light-Field Camera Could Drastically Change the Way Cinematographers Work

Experimental Lightfield Camera Lytro
Courtesy of Lytro

Cinematographers who attended NAB in Las Vegas this past April were intrigued by a new device that could not only revolutionize camera technology, but could change jobs in their profession — and possibly eliminate some.

The object of their attention: the Lytro Cinema professional light-field camera, on display as prototype, large and unwieldy enough to remind DPs of the days when cameras and their operators were encased in refrigerator-sized sound blimps. But proponents insist the Lytro has the potential to change cinematography as we know it.

The Lytro captures a holographic digital model of a scene 300 times per second via its “plenoptic” sensor, which sees objects from multiple points of view. In contrast with a conventional camera, which captures pictures by recording light intensity, Lytro also captures information about the light field emanating from a scene, recording the direction of the light rays.

It produces vast amounts of data, allowing the generation of thousands of synthetic points of view. With the resulting information, filmmakers can manipulate a range of image characteristics, including frame rate, aperture, focal length, and focus — simplifying what can be a lengthy, laborious process.

For example, Lytro’s ability to measure the depth of every object in a scene gives filmmakers the ability to simply delete anything beyond a certain distance from the camera, letting them do green-screen work without green screens. Another bonus: Lytro can gather enough data to produce left- and right-eye views for 3D.

The photographic concepts behind Lytro have been around for more than a century, but advancements in optics, sensor technology, and processing power renewed interest a decade ago. Stanford alum Ren Ng founded the company, simply called Lytro, to commercialize these concepts.

DP David Stump, chair of the camera subdivision of the Technology Committee of the American Society of Cinematographers, helped make the demo film that screened at NAB. Like many, he’s optimistic about the device’s potential to become a standard filmmaking tool.

Others are more cautious, and there is some concern about the effect on employment prospects for camera crews, despite assurances from many quarters that the device cannot simply operate itself; it requires a cinematographer’s trained eye and sensibility.

STAR CAMERA The Lytro display at April’s NAB confab drew
a crowd.

“The technology of light-field capture is pretty universally misunderstood,” Stump says. “Many people think it’s about the ability to refocus the image in post, and that’s one nifty capability, but [the Lytro] yields much more knowledge about the world in front of the camera. To me, the value comes in knowing where pixels are in space relative to the camera. You’re deriving accurate scene geometry from a single lens capture.”

Camera Guild president Steven Poster, who is also vice-chair of the ASC Technology Committee, says that while the camera’s potential is “tremendous,” it’s applicability is limited to situations that call for very specific visual effects. Even with a smaller camera, and “even if the data-handling challenges are overcome — and those are big ifs — filmmakers aren’t going to want to leave artistic decisions to post-production.”

In other words, the creative process will still take place mostly during filming. “In most cases, decisions about framing, focus, and other image traits are best done in the moment, in concert with the actors and other artists on the set,” says Poster. “Filmmaking is a live, collaborative event; it’s not just craft. Operating and focus-pulling are both considered arts.”

Cinematographer Christopher Chomyn, too, sees promise and peril in the new technology. “There are some big hurdles ahead,” he says. “There’s the matter of the physics, the size of the individual photo sites, and their light sensitivity, for example. It’s not a slam dunk. But technology progresses so quickly. And what it does is incredible.”

Recalling his last film, Chomyn sees the upside of the Lytro. “We had a big car stunt, with a huge green screen set up on a street corner, and all the complications and costs in time and manpower that go with that,” he says. “This technology is potentially much more efficient.”

He also brings up the issue of control, which cinematographers have been vigilant about throughout the digital evolution. “There are already so many hands trying to stir the pot,” he says. “With so much additional malleability in the image, it’s imperative that the cinematographer be involved through post, ensuring a consistent and coherent film that remains true to its concept and design.”

Stump agrees that the camera needs considerable development, but insists that the new utilities will be worth it. “It’s another example of our increasing human ability to learn,” says Stump. “And if there’s anything the last 15 years have proven, it’s that cinematographers know how to learn and adapt.”

“Don’t fear it,” he adds. “It’s not here to replace anything. It’s here to enhance everything we do. Like other advancements, it requires a creative mind, a visual storyteller who understands communicating with images. So I say, ‘Embrace it.’ ”