This Article 
 Bibliographic References 
 Add to: 
A Noniterative Greedy Algorithm for Multiframe Point Correspondence
January 2005 (vol. 27 no. 1)
pp. 51-65
This paper presents a framework for finding point correspondences in monocular image sequences over multiple frames. The general problem of multiframe point correspondence is NP-hard for three or more frames. A polynomial time algorithm for a restriction of this problem is presented and is used as the basis of the proposed greedy algorithm for the general problem. The greedy nature of the proposed algorithm allows it to be used in real-time systems for tracking and surveillance, etc. In addition, the proposed algorithm deals with the problems of occlusion, missed detections, and false positives by using a single noniterative greedy optimization scheme and, hence, reduces the complexity of the overall algorithm as compared to most existing approaches where multiple heuristics are used for the same purpose. While most greedy algorithms for point tracking do not allow for entry and exit of the points from the scene, this is not a limitation for the proposed algorithm. Experiments with real and synthetic data over a wide range of scenarios and system parameters are presented to validate the claims about the performance of the proposed algorithm.

[1] M. Asada and S. Tsuji, “Representation of Three-Dimensional Motion in Dynamic Scenes,” Computer Vision, Graphics, and Image Processing, vol. 21, no. 1, pp. 118-144, Jan. 1983.
[2] S.T. Barnard and W.B. Thompson, “Disparity Analysis of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, no. 4, pp. 333-340, July 1980.
[3] F.T. Boesch and J.F. Gimpel, “Covering the Points of a Digraph with Point Disjoint Paths and Its Application to Code Optimization,” J. ACM, vol. 24, no. 2, pp. 192-198, Apr. 1977.
[4] D. Chetverikov and J. Verestóy, “Feature Point Tracking for Incomplete Trajectories,” Computing, Devoted Issue on Digital Image Processing, vol. 62, no. 4, pp. 321-338, 1999.
[5] I.J. Cox, “A Review of Statistical Data Association Techniques for Motion Correspondence,” Int'l J. Computer Vision, vol. 10, no. 1, pp. 53-66, Feb. 1993.
[6] I.J. Cox and S.L. Hingorani, “An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 138-150, Feb. 1996.
[7] I.J. Cox and M.L. Miller, “On Finding Ranked Assignments with Applications to Multi-Target Tracking and Motion Correspondence,” IEEE Trans. Aerospace and Electronic Systems, vol. 32, no. 1, pp. 486-489, Jan. 1995.
[8] R. Danchick and G.E. Newnam, “A Fast Method for Finding the Exact N-Best Hypotheses for Multitarget Tracking,” IEEE Trans. Aerospace and Electronic Systems, vol. 29, no. 2, pp. 555-560, Apr. 1993.
[9] M.R.W. Dawson, “The How and Why of What Went Where in Apparent Motion: Modeling Solutions to the Motion Correspondence Problem,” Psychological Rev., vol. 98, pp. 569-603, 1991.
[10] S. Deb, M. Yeddanapudi, K. Pattipati, and Y. Bar-Shalom, “A Generalized S-D Assignment Algorithm for Multisensor-Multitarget State Estimation,” IEEE Trans. Aerospace and Electronic Systems, vol. 33, no. 2, pp. 523-538, 1997.
[11] T.E. Fortmann, Y. Bar-Shalom, and M. Sheffe, “Sonar Tracking of Multiple Targets Using Joint Probabilistic Data Association,” IEEE J. Oceanic Eng., vol. 8, no. 3, pp. 173-184, July 1983.
[12] M.R. Gary and D.S. Johnson, Computers and Intractability. New York: Freeman, 1979.
[13] J.J. Gibson, The Ecological Approach to Visual Perception. Boston: Houghton Mifflen, 1979.
[14] J.E. Hopcroft and R.M. Karp, “An $n^{2. 5}$ Algorithm for Maximum Matchings in Bipartite Graphs,” SIAM J. Computing, vol. 2, no. 4, pp. 225-230, Dec. 1973.
[15] B.K.P. Horn and B.G. Schunck, “Determining Optical Flow,” Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[16] O. Javed, Z. Rasheed, K. Shafique, and M. Shah, “Tracking Across Multiple Cameras with Disjoint Views,” Proc. Ninth IEEE Int'l Conf. Computer Vision, 2003.
[17] O. Javed and M. Shah, “Tracking and Object Classification for Automated Surveillance,” Proc. European Conf. Computer Vision, pp. IV: 343 ff., 2002.
[18] M.R.M. Jenkin, “Tracking Three Dimensional Moving Light Displays,” Proc., Workshop Motion: Representation Contr., pp. 66-70, 1983.
[19] G. Johansson, “Spatio-Temporal Differentiation and Integration in Visual Motion Perception,” Psychological Research, vol. 38, pp. 379-393, 1976.
[20] H.W. Kuhn, “The Hungarian method for Solving the Assignment Problem,” Naval Reseach Logistics Quarterly, vol. 2, pp. 83-97, 1955.
[21] D. Marr, Vision: A Computational Investigation into the Human Representation and Processing of Visual Information. New York: W.H. Freeman, 1982.
[22] U. Neisser, Cognition and Reality: Principles and Implications of Cognitive Psychology. San Francisco: W.H. Freeman, 1976.
[23] A.B. Poore, “Multidimensional Assignments and Multitarget Tracking,” Proc. Partitioning Data Sets; DIMACS Workshop, pp. 169-196, 1995.
[24] A.B. Poore and X. Yan, “Data Association in Multi-Frame Processing,” Proc. Second Int'l Conf. Information Fusion, pp. 1037-1044, 1999.
[25] K. Rangarajan and M. Shah, “Establishing Motion Correspondence,” Computer Vision, Graphics, and Image Processing, vol. 54, no. 1, pp. 56-73, July 1991.
[26] R.F. Rashid, “Towards a System for the Interpretation of Moving Light Display,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 2, no. 1, pp. 574-581, Jan. 1980.
[27] D.B. Reid, “An Algorithm for Tracking Multiple Targets,” IEEE Trans. Automatic Control, vol. 24, no. 6, pp. 843-854, Dec. 1979.
[28] V. Salari and I.K. Sethi, “Feature Point Correspondence in the Presence of Occlusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 87-91, Jan 1990.
[29] I.K. Sethi and R. Jain, “Finding Trajectories of Feature Points in a Monocular Image Sequence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp. 56-73, Jan. 1987.
[30] P.J. Shea and A.B. Poore, “Computational Experiences with Hot Starts for a Moving Window Implementation of Track Maintenance,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 1, pp. 87-91, Jan. 1990.
[31] J. Shi and C. Tomasi, “Good Features to Track,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[32] J.T. Todd, “Visual Information about Rigid and Nonrigid Motion: A Geometric Analysis,” J. Experimental Psychology: Human Perception and Performance, vol. 8, pp. 238-252, 1982.
[33] R.Y. Tsai and T.S. Huang, “Uniqueness and Estimation of Three-Dimensional Motion Parameters of Rigid Objects with Curved Surfaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 1, pp. 13-27, Jan. 1984.
[34] S. Ullman, The Interpretation of Visual Motion. Cambridge, Mass.: MIT Press, 1979.
[35] C.J. Veenman, M.J.T. Reinders, and E. Backer, “Resolving Motion Correspondence for Densely Moving Points,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 54-72, Jan. 2001.
[36] J. Verestóy and D. Chetverikov, “Experimental Comparative Evaluation of Feature Point Tracking Algorithms,” Proc. Workshop Evaluation and Validation of Computer Vision Algorithms, pp. 183-194, 2000.
[37] D.B. West, Introduction to Graph Theory. Prentice Hall, 2001.
[38] M. Yachida, M. Asada, and S. Tsuji, “Automatic Analysis of Moving Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 1, pp. 12-20, Jan. 1981.

Index Terms:
Point correspondence, target tracking, motion, occlusion, point trajectory, data association, bipartite graph matching, path cover of directed graph.
Khurram Shafique, Mubarak Shah, "A Noniterative Greedy Algorithm for Multiframe Point Correspondence," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 51-65, Jan. 2005, doi:10.1109/TPAMI.2005.1
Usage of this product signifies your acceptance of the Terms of Use.