ADVISOR: Frédéric Devernay
TEL: 04 76 61 52 58
EMAIL: frederic.devernay at inria.fr
TEAM AND LAB: PRIMA team, Inria Rhône-Alpes & LIG
THEMES: Image, Vision
DURATION: 3 to 6 months
Recently, light-field cameras arrived on the consumer market in the form of the Lytro camera (Raytrix also sells a light-field camera more targeted on the industry and research). Conventional cameras follow the pinhole camera model inherited from the biological eye: the image records the radiance of all the optical rays that pass through a single point in space, the optical center. The light-field camera [1,2] samples what is called the plenoptic function, which contains radiance information about all the 3D rays in space. This camera is usually built by placing an array containing thousands of lenslets between the main lens and the image plane. The result is somewhat equivalent (depending of the respective position of the lens array and the image plane [3]) to a 2D array of thousands of very low-resolution cameras (the Lytro raw image is 3280x3280, and each lenslet occupies about 10x10 pixels).
The advantage of this configuration is that for some applications, such as digital refocusing (demonstrated in the images above), almost no computation is necessary, at the expense of low final image resolution (the Lytro software produces 1080x1080 images). On the opposite, other computational imaging techniques such as view interpolation from a few viewpoints require more computation, but produce images with a resolution which is much closer to the original images [5]. For example, the TCM9518MD sensor, which was recently released by Toshiba [9], claims to be able to do digital refocusing using a stereoscopic sensor.
This work will focus on developping a refocusing (or "synthetic apperture") algorithm from a few high-resolution cameras (two to seven) by combining view interpolation techniques with other techniques [8], comparing the results with what would be obtained from a lightfield camera with thousands of lenslets (using ground-truth light-field data [6,7]), and optimizing the camera configuration to get the best results.
Please send a motivition letter and a resumé (CV) by email to frederic.devernay at inria.fr
[1] Adelson, E. H., and Wang, J. Y. A. 1992. Single lens stereo with a plenoptic camera. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 99.106.
[2] Ng, R., Levoy, M., Bredif, M., Duval, G., Horowitz, M., and Hanrahan, P. (2005). Light field photography with a hand-held plenoptic camera. Stanford University Computer Science Tech Report CSTR 2005-02.
[3] Georgiev, T., Zheng, K. C., Curless, B., Salesin, D., Nayar, S., and Intwala, C. 2006. Spatio-angular resolution tradeoffs in integral photography. In Rendering Techniques 2006: 17th Eurographics Workshop on Rendering, 263-272.
[4] Reverse Engineering the Lytro .LFP File Format http://eclecti.cc/computervision/reverse-engineering-the-lytro-lfp-file-format
[5] Frédéric Devernay and Adrian Ramos Peon. 2010. Novel view synthesis for stereoscopic cinema: detecting and removing artifacts. In Proceedings of the 1st international workshop on 3D video processing (3DVP '10). ACM, New York, NY, USA, 25-30.
[6] The (new) Stanford Light Field Archive http://lightfield.stanford.edu/lfs.html
[7] Wetzstein, G., Ihrke, I., Gukov, A., Heidrich, W., (2011) Towards a database of high-dimensional plenoptic images In Proc. ICCP
[8] Berent, J.; Dragotti, P.L.; 2007, "Plenoptic Manifolds," Signal Processing Magazine, IEEE , vol.24, no.6, pp.34-44, Nov. 2007
[9] "Two 1/4 inch 5M pixel 1.4 µm CMOS image sensors: TCM9518MD", TOSHIBA Semiconductor, http://www.semicon.toshiba.co.jp/eng/product/new_products/sensor/1325631_37652.html