Issue No. 05 - May (2011 vol. 33)
Antonio Torralba , Massachusetts Institute of Technology, Cambridge
Jenny Yuen , Massachusetts Institute of Technology, Cambridge
Ce Liu , Microsoft Research New England, Cambridge
While image alignment has been studied in different areas of computer vision for decades, aligning images depicting different scenes remains a challenging problem. Analogous to optical flow, where an image is aligned to its temporally adjacent frame, we propose SIFT flow, a method to align an image to its nearest neighbors in a large image corpus containing a variety of scenes. The SIFT flow algorithm consists of matching densely sampled, pixelwise SIFT features between two images while preserving spatial discontinuities. The SIFT features allow robust matching across different scene/object appearances, whereas the discontinuity-preserving spatial model allows matching of objects located at different parts of the scene. Experiments show that the proposed approach robustly aligns complex scene pairs containing significant spatial differences. Based on SIFT flow, we propose an alignment-based large database framework for image analysis and synthesis, where image information is transferred from the nearest neighbors to a query image according to the dense scene correspondence. This framework is demonstrated through concrete applications such as motion field prediction from a single image, motion synthesis via object transfer, satellite image registration, and face recognition.
Scene alignment, dense scene correspondence, SIFT flow, coarse to fine, belief propagation, alignment-based large database framework, satellite image registration, face recognition, motion prediction for a single image, motion synthesis via object transfer.
Antonio Torralba, Jenny Yuen, Ce Liu, "SIFT Flow: Dense Correspondence across Scenes and Its Applications", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 33, no. , pp. 978-994, May 2011, doi:10.1109/TPAMI.2010.147