We present a method for scene flow estimation from a calibrated stereo image sequence. The scene flow contains the 3-D displacement field of scene points, so that the 2-D optical flow can be seen as a projection of the scene flow onto the images. We propose to recover the scene flow by coupling the optical flow estimation in both cameras with dense stereo matching between the images, thus reducing the number of unknowns per image point. The use of a variational framework allows us to properly handle discontinuities in the observed surfaces and in the 3-D displacement field. Moreover our approach handles occlusions both for the optical flow and the stereo. We obtain a partial differential equations system coupling both the optical flow and the stereo, which is numerically solved using an original multi-resolution algorithm. Whereas previous variational methods were estimating the 3-D reconstruction at time t and the scene flow separately, our method jointly estimates both in a single optimization. We present numerical results on synthetic data with ground truth information, and we also compare the accuracy of the scene flow projected in one camera with a state-of-the-art single-camera optical flow computation method. Results are also presented on a real stereo sequence with large motion and stereo discontinuities.
Please note that this source code is only intended for educational and research purposes. The copyright to this code is owned by INRIA, so please contact me for any other use.
The ground truth for the "Ball" data is actually provided in the groundtruth.cpp source file, and the POV-Ray files used to generate this data are also available separately.
Important note: the GMRES solver is enabled by default with the above code, but the results may be unstable. To disable it, change "#define GMRES 1" in Resolution.cpp to "#define GMRES 0".
The results were published in the ICCV'07 Paper refrenced below. A technical report is also available with more details, but please use the ICCV paper for reference.
@InProceedings{Huguet-Devernay-varsceneflow-2007,
author = {Huguet, Fr\'ed\'eric and Devernay, Fr\'ed\'eric},
title = {A Variational Method for Scene Flow Estimation from Stereo Sequences},
booktitle = {Proc. Intl. Conf. on Computer Vision},
organization = {IEEE},
address = {Rio de Janeiro, Brasil},
month = oct,
year = 2007,
url = {http://hal.inria.fr/inria-00262139}
}
These results were partially funded by the project ANR MD GEOLSTEREO, funded by the "Masses de données" program of the Agence Nationale de la Recherche (French national research agency).
The ANR MD GEOLSTEREO was a collaborative project between Laboratoire Géosciences Azur (UMR 6526 - CNRS - UNSA - UPMC- IRD) and INRIA Grenoble Rhone-Alpes, during the years 2004-2007.