How the Lytro Light-Field Camera Works
Light-field photography has the potential to revolutionize photography. Ren Ng, the founder of Lytro, explains how a camera can capture images that are never out of focus.
Lytro has big plans. This week the Mountain View, CA-based startup said it would soon bring to market a that's based on light-field photography. The result: photographs that you can focus after you take them. Simply click your mouse on the spot on the picture you want in focus, and it changes before your eyes (check out the effect in the photo below).
The idea of light-field photography isn't new. Other companies, including Adobe and another Mountain View startup called Pelican Imaging, have dabbled in the technology. But the Lytro camera, assuming it debuts later this year as planned, will mark the first time the tech makes an appearance in a consumer camera. Its creator, Ren Ng (pronounced "ing") was inspired to make the camera when he found he couldn't capture properly focused pictures of his friend's five-year-old daughter. If a camera existed that could capture all the light information—direction, intensity, and color—of the entire field of view, he thought, it would make getting the "perfect" shot incredibly easy.
"If you can shoot first, focus later, it's going to be the fastest camera you've ever used," Ng said in an interview with PCMag. "Because when you press the shutter button, it takes the shot instantly. It doesn't have to wait for the lens to move."
That's the promise of Lytro's upcoming light-field camera, which the company says will be on store shelves by the end of the year. Besides taking pictures extremely fast, the camera can use the light-field information to create 3D images, and with so much data being gathered, it has improved performance in low light.
How It Works
It all sounds a bit magical, but it opens up a lot of questions. How exactly does it work? What are the tradeoffs, and how will these so-called "living photographs" interact with existing viewing and sharing software?
The basic premise of the light-field camera is to gather all data about the visible light in the camera's field of view so that software can manipulate the photo later. While the concept has been used previously to create imagery like the "bullet time" special effects in the Matrix movies, it required a room full of cameras, Ng says, and the power of a supercomputer. With special optics and sensors, Lytro has built the technique into a single, portable device.
"Regular photographs just don't tell the whole story. If you think about all the light that enters that enters the lens of a camera, that's much more than a photo. The light-field is all the higher-dimensional information that's lost in a regular photo. When we record all this information, that provides us the opportunity in software after the fact."
What happens "after the fact" is the big breakthrough: once the light-field data is captured, Lytro's algorithms can do some impressive tricks. First and foremost is the ability to focus on any point the viewer wishes. You can see the effect in the photo on the right (pic by Eric Cheng/Lytro).
"When a regular camera focuses physically, what the regular camera is doing is adjusting the lens relative to the sensor to bring different parts of the scene into focus," says Ng. "So if we have the whole light field, what we can do what that physical lens would normally have done, but in computation."
Beyond Focus: 3D and More
3D pictures work similarly. With complete light-field information, software can discern how a scene would have looked to two separate cameras (which is how most 3D is shot). The algorithms separate the light from the left and right sides of the camera to create the 3D effect. Ng says the result has improvements over conventional ways of shooting 3D.
"It goes beyond the 3D you see in the movies because we can also change the perspective in the scene. It's what would happen if you were standing at that scene, and you were kind of moving your head from side to side.
"For low light, all the light rays participate. We're using all the light coming through a large aperture to make a picture with a large depth of field—totally impossible with a conventional camera. To give a sense of perspective, there can be 16 times less light for [a specific] depth of field."
Tradeoffs
It all sounds like a big improvement over normal photography, and it is, but it doesn't come without tradeoffs, the main one being picture resolution. The nature of light-field photography makes it difficult capture high-resolution images without making the optics extremely complicated. Although Ng says his team has found ways to mitigate the issue, the Lytro camera won't be competing with others on the market in the megapixel arena. Ng says it ultimately doesn't matter, however.
"Huge advances have been made," he says. "The resolution issue from the research side of things was one of the early big breakthoughs at the company. The thing about resolution, by the time people share pictures online, you're throwing away 90 to 95 percent of those pixels. And the vast majority of picture use today goes through the Web."
The Sharing Question
Ng makes a good point, but the Web issue touches on another question: how will Lytro users be able to share their photos? The file format is a new, proprietary type, Ng says, and Lytro will offer online storage for anyone who owns a camera. Users can then share their focus-anywhere pics via embed codes for viewing online. It's doubtful that popular photo services like Flickr and Snapfish will support Lytro photos anytime soon, but users will always have the ability to "flatten" the images as JPEGs, though they'll lose the interactivity.
While Ng wouldn't talk about any specifics for the upcoming camera, he did say it would be "competitively priced" and targeted at the everyday user. For the future, he has plans to spread the technology out to video cameras, which he says are on the product road map. Initial response to Lytro's plans has been "enormous," he says, and he plans to build on that interest to revolutionize the entire photography industry.
"Cameras are so important to people becuase as human beings we have this fundamental need to share our stories visually. You can see that in the size of the camera market. Not including phones, the camera market did $38 billion in sales last year, and in spite of the lack of innovation it's supposed to grow to $44 billion in 2015. From lytro's perspective, this is a really rich space from which we can build a large independent company that will forever change the way we take and experience pictures."