The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2013 vol.35)
pp: 1565-1576
J. Bazin , CVG/CGL, ETH Zurich, Zurich, Switzerland
Hongdong Li , NICTA &RSISE, Australian Nat. Univ. (ANU), Canberra, ACT, Australia
In So Kweon , Dept. of Electr. Eng., KAIST, Daejeon, South Korea
C. Demonceaux , Le2i, Le Creusot, France
P. Vasseur , Lab. LITIS, Univ. of Rouen, St. Étienne du Rouvray, France
K. Ikeuchi , Comput. Vision Lab., Univ. of Tokyo, Tokyo, Japan
ABSTRACT
Data correspondence/grouping under an unknown parametric model is a fundamental topic in computer vision. Finding feature correspondences between two images is probably the most popular application of this research field, and is the main motivation of our work. It is a key ingredient for a wide range of vision tasks, including three-dimensional reconstruction and object recognition. Existing feature correspondence methods are based on either local appearance similarity or global geometric consistency or a combination of both in some heuristic manner. None of these methods is fully satisfactory, especially in the presence of repetitive image textures or mismatches. In this paper, we present a new algorithm that combines the benefits of both appearance-based and geometry-based methods and mathematically guarantees a global optimization. Our algorithm accepts the two sets of features extracted from two images as input, and outputs the feature correspondences with the largest number of inliers, which verify both the appearance similarity and geometric constraints. Specifically, we formulate the problem as a mixed integer program and solve it efficiently by a series of linear programs via a branch-and-bound procedure. We subsequently generalize our framework in the context of data correspondence/grouping under an unknown parametric model and show it can be applied to certain classes of computer vision problems. Our algorithm has been validated successfully on synthesized data and challenging real images.
INDEX TERMS
Optimization, Feature extraction, Geometry, Computer vision, Electronic mail, Pattern matching, Educational institutions, branch-and-bound, Mixed integer programming, quadratic constraint, bilinearities, global optimization
CITATION
J. Bazin, Hongdong Li, In So Kweon, C. Demonceaux, P. Vasseur, K. Ikeuchi, "A Branch-and-Bound Approach to Correspondence and Grouping Problems", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1565-1576, July 2013, doi:10.1109/TPAMI.2012.264
REFERENCES
[1] S. Agarwal, N. Snavely, I. Simon, S.M. Seitz, and R. Szeliski, "Building Rome in a Day," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[2] M. Lhuillier, "Automatic Scene Structure and Camera Motion Using a Catadioptric System," Computer Vision and Image Understanding, vol. 109, pp. 186-203, 2008.
[3] A.C. Berg, T.L. Berg, and J. Malik, "Shape Matching and Object Recognition Using Low Distortion Correspondences," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[4] D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints" Int'l J. Computer Vision, vol. 60, pp. 91-110, 2003.
[5] J. Shi and C. Tomasi, "Good Features to Track," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994.
[6] H. Bay, A. Ess, T. Tuytelaars, and L.V. Gool, "SURF: Speeded Up Robust Features," Computer Vision and Image Understanding, vol. 110, pp. 346-359, 2008.
[7] K. Mikolajczyk and C. Schmid, "An Affine Invariant Interest Point Detector," Proc. European Conf. Computer Vision, 2002.
[8] L. Torresani, V. Kolmogorov, and C. Rother, "Feature Correspondence via Graph Matching: Models and Global Optimization," Proc. European Conf. Computer Vision, 2008.
[9] T.S. Caetano, J. McAuley, L. Cheng, Q.V. Le, and A.J. Smola, "Learning Graph Matching," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 6, pp. 1048-1058, June 2009.
[10] M. Leordeanu and M. Hebert, "Unsupervised Learning for Graph Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[11] M. Leordeanu and M. Hebert, "A Spectral Technique for Correspondence Problems Using Pairwise Constraints," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[12] P. Besl and N. McKay, "A Method for Registration of 3-D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[13] M.A. Fischler and R.C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. ACM, vol. 24, pp. 381-395, 1981.
[14] D.P. Huttenlocher and S. Ullman, "Object Recognition Using Alignment," Proc. IEEE Int'l Conf. Computer Vision, 1987.
[15] J. Maciel and J. Costeira, "A Global Solution to Sparse Correspondence Problems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 187-199, Feb. 2003.
[16] R. Oliveira, R. Ferreira, and J. Costeira, "Optimal Multi-Frame Correspondence with Assignment Tensors," Proc. European Conf. Computer Vision, 2006.
[17] A. Akbarzadeh et al., "Towards Urban 3D Reconstruction from Video," Proc. Int'l Symp. 3D Data Processing, Visualization, and Transmission, 2006.
[18] S. Lazebnik, C. Schmid, and J. Ponce, "Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[19] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[20] H. Li, "Consensus Set Maximization with Guaranteed Global Optimality for Robust Geometry Estimation," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[21] G. McCormick, "Computability of Global Solutions to Factorable Nonconvex Programs: Part I—Convex Underestimating Problems," Math. Programming, vol. 10, pp. 147-175, 1976.
[22] H. Sherali and A. Alameddine, "A New Reformulation-Linearization Technique for Bilinear Programming Problems," J. Global Optimization, vol. 2, pp. 379-410, 1992.
[23] M. Chandraker and D. Kriegman, "Globally Optimal Bilinear Programming for Computer Vision Applications," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[24] C. Olsson, F. Kahl, and M. Oskarsson, "Branch-and-Bound Methods for Euclidean Registration Problems," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 783-794, May 2009.
[25] F. Kahl, S. Agarwal, M. Chandraker, D. Kriegman, and S. Belongie, "Practical Global Optimization for Multiview Geometry," Int'l J. Computer Vision, vol. 79, pp. 271-284, 2008.
[26] T.M. Breuel, "Implementation Techniques for Geometric Branch-and-Bound Matching Methods," Computer Vision and Image Understanding, vol. 90, pp. 258-294, 2003.
[27] L. Lovasz and M.D. Plummer, Matching Theory. North-Holland, 1986.
[28] R. Hartley and F. Kahl, "Global Optimization through Rotation Space Search," Int'l J. Computer Vision, vol. 82, pp. 64-79, 2009.
[29] Y. Seo, Y. Choi, and S. Lee, "A Branch-and-Bound Algorithm for Globally Optimal Calibration of a Camera-and-Rotation-Sensor System," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[30] O. Chum and J. Matas, "Optimal Randomized RANSAC," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 8, pp. 1472-1482, Aug. 2008.
[31] Visual Geometry Group, Univ. of Oxford, http://www.robots. ox.ac.ukvgg, 2013.
[32] B. Gendron and T. Crainic, "Parallel Branch-and-Bound Algorithms: Survey and Synthesis," Operations Research, vol. 42, pp. 1042-1066, 1994.
[33] D. Spampinato and A.C. Elstery, "Linear Optimization on Modern GPUs," Proc. IEEE Int'l Symp. Parallel and Distributed Processing, 2009.
[34] C.N. Vasconcelos and B. Rosenhahn, "Bipartite Graph Matching Computation on GPU," Proc. Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2009.
[35] A. Bartoli and P. Sturm, "The 3D Line Motion Matrix and Alignment of Line Reconstructions," Int'l J. Computer Vision, vol. 57, pp. 159-178, 2004.
[36] J.-S. Kim, P. Gurdjos, and I.-S. Kweon, "Geometric and Algebraic Constraints of Projected Concentric Circles and Their Applications to Camera Calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 637-642, Apr. 2005.
[37] D. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Pearson, 2002.
[38] P. O'Leary, M. Harker, and P. Zsombor-Murray, "Direct and Least Square Fitting of Coupled Geometric Objects for Metric Vision," Image Vision and Signal Processing, vol. 152, no. 6, pp. 687-694, Dec. 2005.
[39] J. Illingworth and J. Kittler, "A Survey of the Hough Transform," Computer Vision, Graphics, and Image Processing, vol. 44, pp. 87-116, 1988.
[40] G. Foresti and C. Regazzoni, "A Hierarchical Approach to Feature Extraction and Grouping," IEEE Trans. Image Processing, vol. 9, no. 6, pp. 1056-1074, June 2000.
[41] G.J. McLachlan and T. Krishnan, The EM Algorithm and Extensions. John Wiley & Sons, 2008.
[42] A. Delong, A. Osokin, H.N. Isack, and Y. Boykov, "Fast Approximate Energy Minimization with Label Costs," Int'l J. Computer Vision, vol. 100, pp. 1-27, 2012.
[43] H.N. Isack and Y. Boykov, "Energy-Based Geometric Multi-Model Fitting," Int'l J. Computer Vision, vol. 97, pp. 123-147, 2012.
[44] N. Thakoor and J. Gao, "Branch-and-Bound Hypothesis Selection for Two-View Multiple Structure and Motion Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[45] H. Li, "Two-View Motion Segmentation from Linear Programming relaxation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[46] J.Y. Bouguet, "Camera Calibration Toolbox for Matlab," http://www.vision.caltech.edu/bouguetjcalib_doc /, 2013.
[47] Q. Zhang and R. Pless, "Extrinsic Calibration of a Camera and Laser Range Finder (Improves Camera Intrinsic Calibration)," Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems, 2004.
[48] W.B. Powell and I.O. Ryzhov, Subset Selection Problems in Optimal Learning. John Wiley & Sons, 2012.
[49] J. Matas and O. Chum, "Randomized RANSAC with Sequential Probability Ratio Test," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[50] M. Wang and X.-S. Hua, "Active Learning in Multimedia Annotation and Retrieval: A Survey," ACM Trans. Intelligent Systems and Technology, vol. 2, article 10 2011.
[51] L. Jonasson, P. Hagmann, C. Pollo, X. Bresson, R. Meuli, and J.-P. Thiran, "Coupled Region Based Level Sets for Segmentation of the Thalamus and Its Subnuclei in DT-MRI," Proc. Int'l Soc. Magnetic Resonance in Medicine, 2005.
146 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool