Dr. Ren Ng of Refocus Imaging gave a tremendous talk at last week’s COBA meeting, the final one for 2006. I’ve been following his research for quite some time, but until last night, I hadn’t seen a first-hand demonstration of his technology. Suffice it to say, I left extremely impressed. The prototype camera — a medium format digital camera that was fitted with a custom microlens array sitting on top of the imaging sensor — allow Ng to set the focus point in the image after the fact. A photo of a flower, for instance, could be refocused to isolate the buds in the foreground or the pedals in the background. A portrait where the ears are nice and sharp could be fixed instantly so that the eyes are in focus.
As he slid the refocus slider, he explained that his Digital Lens allowed for offset depth focus, per pixel focus, and six stops of better sensitivity.
One COBA member called this a disruptive technology, and I couldn’t have agreed more. The biggest hurdle that I can see is getting enough resolution from the modified sensor. From what I understand, the microlens array projects an image onto the imaging sensor that contains more light information about the subject at the expense of resolution. The resulting image is physically smaller — think 10 sensor pixels for every Digital Lens pixel — but has information that allows for refocusing after the fact. I believe Dr. Ng was using a 22MP digital back on his camera, meaning his images were at most 2MP in resolution. For this to be commercially viable, I think the sweet spot for a Digital Lens sensor would have to be at least in the 6-8MP range.
Ng showed a slide which hinted at a possible solution to the reduced image resolution problem. He compared the sensor pixel size between DSLR and consumer point-and-shoot cameras. Digital photography enthusiasts know that DSLR sensors are larger and have greater pixel sizes than their consumer counterparts. If you create a sensor with the same pixel pitch (2.2 microns) of a consumer camera at the same size of a DSLR sensor (6.4-7.2 microns), you would have a sensor on the order of 70-178 megapixels (using 30D and 1Ds Mark II sensor sizes). Assuming the microlenses would project information about one effective pixel onto 10 pixels, this results in an effective 7-18 megapixel image.
As Foveon has argued and the Digital Lens demonstrates, it’s not about the number of pixels but the quality of pixels. Ng emphasized that the process to create hundred megapixel sensors is within grasp today. I have a feeling that his company has been talking with camera and sensor manufacturers about assistance on creating a more functional prototype of his Digital Lens. Right now, the camera needs to be FireWire tethered to a computer to function.
I enjoyed hearing his story on what brought him down this path. He was in the CS Department at Stanford, and wanted to do research on digital photography. The fixed focus lens was a fundamental part of traditional photography, but did it have to be this way? His interest in taking candid portrait meant using long lens, shooting wide-open apertures, and having frequent out of focus or slightly off-focus shots. With this background Ng set out on his journey to create the Digital Lens. This is revolutionary stuff, and I am looking forward to seeing it in professional and consumer products within the next 3-5 years.
Next COBA Meeting
With last week’s meeting, we conclude the COBA year for 2006. The next meeting will be held on Valentine’s Day, February 14, 2007. There will be no January meeting for COBA. See you all next year!
tow.com 
That’s pretty amazing stuff. I’ve sometimes wondered about using a stereoscopic camera so that a computer could calculate distance-to-subject and then choose the in-focus distance from that. Of course, the originals have to have as much in-focus to start with as possible for that to work.
One point to consider is that although you could make a DSLR sensor today with a huge pixel count, it would not come for free. You’d have to give up sensitivity. In essence, your maximum usable ISO would drop from 1600 to 400 (or less) just like the point-and-shoot cameras that use pixels of that small size.