In the first part of the talk, I will introduce how to recover 3D surface shape from consumer images using photometric stereo (PS). PS, a classic research topic in computer vision, estimates surface model of a static scene from multiple images captured under a fixed camera position but various lighting directions. Unlike the classic photometric stereo algorithms, our 3D shape recovery algorithm doesn't require the light source to be far away - we handle the near light illumination with novel optimization methods. This work has helped to resolve longstanding art historical questions about the evolution of the artist Paul Gauguin's printing techniques, which was reported by Newsweek.

In the second part of the talk, I will present compressive reconstruction of 3D volumetric images in a fluorescent microscopy scene. 3D snapshot imaging in a live cell environment is important and may provide new insights for biological applications. Classic 3D microscopy methods such as confocal imaging is not applicable in such a dynamic environment due to the time consuming scan process. To capture a 3D dynamic scene, we consider Fresnel Incoherent Correlation Holography (FINCH) to capture a 2D movie sequence. Each 2D frame consists of the incoherent correlation hologram of the 3D volumetric object. By exploiting the sparsity prior of the volumetric object, we use compressive sensing reconstruction algorithm to recover the 3D dynamic scene from the 2D movie. We show how compressed sensing enables reconstruction without out-of-focus artifacts, when compared to conventional back-propagation recovery. We also analyze the reconstruction guarantees of the proposed approach bo th numerically and theoretically and compare that with coherent holography. In the end, I will present a real reconstruction video showing the internal structure of a moving bacteria using another mystery method. The exact method will be presented in the next talk after we file the patent.

During the talk, I will also lead an open discussion of future work on 3D reconstructions, such as using a more realistic surface reflectance model, combing photogrammetry with PS, and joint spatio-temporal optimization. I also want to discuss with audience (especially those from optimization, signal processing and imaging background) on how to improve the optimization methods, their relationship to 3D tomography, and theoretical and practical limits of compressive sensing.