Subscribe

Issue No.05 - May (2011 vol.23)

pp: 655-668

Ninad Thakoor , University of California, Riverside, Riverside

Jean Gao , University of Texas at Arlington, Arlington

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TKDE.2010.156

ABSTRACT

Branch-and-bound methods are used in various data analysis problems, such as clustering, seriation and feature selection. Classical approaches of branch-and-bound based clustering search through combinations of various partitioning possibilities to optimize a clustering cost. However, these approaches are not practically useful for clustering of image data where the size of data is large. Additionally, the number of clusters is unknown in most of the image data analysis problems. By taking advantage of the spatial coherency of clusters, we formulate an innovative branch-and-bound approach, which solves clustering problem as a model-selection problem. In this generalized approach, cluster parameter candidates are first generated by spatially coherent sampling. A branch-and-bound search is carried out through the candidates to select an optimal subset. This paper formulates this approach and investigates its average computational complexity. Improved clustering quality and robustness to outliers compared to conventional iterative approach are demonstrated with experiments.

INDEX TERMS

Clustering, segmentation, combinatorial optimization, branch-and-bound, model selection.

CITATION

Ninad Thakoor, Jean Gao, "Branch-and-Bound for Model Selection and Its Computational Complexity",

*IEEE Transactions on Knowledge & Data Engineering*, vol.23, no. 5, pp. 655-668, May 2011, doi:10.1109/TKDE.2010.156REFERENCES

- [1] J. Han and M. Kamber,
Data Mining: Concepts and Techniques. Morgan Kaufmann Publishers Inc., 2006.- [2] D.A. Forsyth and J. Ponce,
Computer Vision: A Modern Approach. Prentice Hall, 2002.- [3] S. Theodoridis and K. Koutroumbas,
Pattern Recognition. Academic Press, 2008.- [4] S. Mitra and T. Acharya,
Data Mining: Multimedia, Soft Computing, and Bioinformatics. Wiley-Interscience, 2003.- [5] S. Zhong and J. Ghosh, "A Unified Framework for Model-Based Clustering,"
J. Machine Learning Research, vol. 4, pp. 1001-1037, 2003.- [6] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data Via the Em Algorithm,"
J. Royal Statistical Soc. B, vol. 39, no. 1, pp. 1-38, http://dx.doi.org/10.23072984875, 1977.- [7] K.P. Burnham and D.A. Anderson,
Model Selection and Multi-Model Inference. Springer, 2003.- [8] H. Li, "Two-view Motion Segmentation from Linear Programming Relaxation,"
Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2007.- [9] K. Schindler and D. Suter, "Two-view Multibody Structure-and-Motion with Outliers through Model Selection,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 983-995, June 2006.- [10] N. Thakoor and J. Gao, "Branch-and-Bound Hypothesis Selection for Two-View Multiple Structure and Motion Segmentation,"
Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1-6, June 2008.- [11] R.I. Hartley and A. Zisserman,
Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.- [12] P.H.S. Torr, "Geometric Motion Segmentation and Model Selection,"
Philosophical Trans. Royal Soc. London A, vol. 356, no. 1740, pp. 1321-1340, http://dx.doi.org/10.1098rsta.1998.0224, May 1998.- [13] F. Glover and M. Laguna,
Modern Heuristic Techniques for Combinatorial Problems. John Wiley & Sons, Inc., pp. 70-150, 1993.- [14] F.A. Chudak and D.B. Shmoys, "Improved Approximation Algorithms for the Uncapacitated Facility Location Problem,"
SIAM J. Computation, vol. 33, no. 1, pp. 1-25, 2004.- [15] D.S. Hochbaum, Ed.,
Approximation Algorithms for NP-hard Problems. PWS Publishing Co., 1997.- [16] G. Schwarz, "Estimating the Dimension of a Model,"
Annals of Statistics, vol. 6, no. 2, pp. 461-464, http://www.jstor.org/stable2958889, 1978.- [17] M. Brusco and S. Stahl,
Branch-and-Bound Applications in Combinatorial Data Analysis. Springer, 2005.- [18] K. Fukunaga,
Introduction to Statistical Pattern Recognition. Academic Press Professional, Inc., 1990.- [19] P. Somol, P. Pudil, and J. Kittler, "Fast Branch and Bound Algorithms for Optimal Feature Selection,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 900-912, July 2004.- [20] L.-K. Shark, A.A. Kurekin, and B.J. Matuszewski, "Development and Evaluation of Fast Branch-and-Bound Algorithm for Feature Matching Based on Line Segments,"
Pattern Recognition, vol. 40, no. 5, pp. 1432-1450, 2007.- [21] M. Roder, J. Cardinal, and R. Hamzaoui, "Branch and Bound Algorithms for Rate-Distortion Optimized Media Streaming,"
IEEE Trans. Multimedia, vol. 8, no. 1, pp. 170-178, Feb. 2006.- [22] J. Jonsson and K. Shin, "A Parametrized Branch-and-Bound Strategy for Scheduling Precedence-Constrained Tasks on a Multiprocessor System,"
Proc. Int'l Conf. Parallel Processing, pp. 158-165, 1997.- [23] S. Fujita, M. Masukawa, and S. Tagashira, "A Fast Branch-and-Bound Algorithm with an Improved Lower Bound for Solving the Multiprocessor Scheduling Problem,"
Proc. Int'l Conf. Parallel and Distributed Systems, pp. 611-616, 2002.- [24] W.L.G. Koontz, P.M. Narendra, and K. Fukunaga, "A Branch and Bound Clustering Algorithm,"
IEEE Trans. Computation, vol. 24, no. 9, pp. 908-915, 1975.- [25] N. Thakoor, V. Devarajan, and J. Gao, "Multi-Stage Branch-and-Bound for Maximum Variance Disparity Clustering,"
Proc. Int'l Conf. Pattern Recognition, pp. 1-4, Dec. 2008.- [26] N. Thakoor, J. Gao, and V. Devarajan, "Multistage Branch-and-Bound Merging for Planar Surface Segmentation in Disparity Space,"
IEEE Trans. Image Processing, vol. 17, no. 11, pp. 2217-2226, Nov. 2008.- [27] L. Devroye and C. Zamora-Cura, "On the Complexity of Branch-and Bound Search for Random Trees,"
Random Structures and Algorithms, vol. 14, no. 4, pp. 309-327, July 1999.- [28] D.R. Smith, "On the Computational Complexity of Branch and Bound Search Strategies," PhD dissertation, Duke Univ., 1979.
- [29] D.R. Smith, "Random Trees and the Analysis of Branch and Bound Procedures,"
J. ACM, vol. 31, no. 1, pp. 163-188, 1984.- [30] H.S. Stone and P. Sipala, "The Average Complexity of Depth-First Search with Backtracking and Cutoff,"
IBM J. Research and Development, vol. 30, no. 3, pp. 242-258, 1986.- [31] W. Zhang, "Branch-and-Bound Search Algorithms and Their Computational Complexity," Univ. of southern California/ Information Sciences Inst., Tech. Rep. ISI/RR-96-443, May 1996.
- [32] W. Zhang and R.E. Korf, "Performance of Linear-Space Search Algorithms,"
Artificial Intelligence, vol. 79, pp. 241-292, 1995.- [33] J.L. Crassidis and J.L. Junkins,
Optimal Estimation of Dynamic Systems. Chapman & Hall/CRC, 2004.- [34] A.D.R. McQuarrie and C.-L. Tsai,
Regression and Time Series Model Selection. World Scientific, 1998.- [35] T.H. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein,
Introduction to Algorithms. The MIT Press, 2001.- [36] W. Feller,
An Introduction to Probability Theory and Its Applications, second ed. vol. 2, John Wiley and Sons, 1966.- [37] P. Torr,
A Structure and Motion Toolkit in Matlab, http://cms.brookes.ac.uk/staff/PhilipTorr/ Betatorrsam.zip, 2010.- [38] P.H.S. Torr and A. Zisserman, "MLESAC: A New Robust Estimator with Application to Estimating Image Geometry,"
Computer Vision and Image Understanding, vol. 78, no. 1, pp. 138-156, 2000.- [39] R. Vidal and S. Sastry, "Optimal Segmentation of Dynamic Scenes from Two Perspective Views,"
Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 281-286, June 2003.- [40] R. Vidal, S. Soatto, Y. Ma, and S. Sastry, "Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix,"
Proc. European Conf. Computer Vision Workshop Visual Modeling of Dynamic Scenes, 2002.- [41]
The Hopkins 155 Dataset, http://www.vision.jhu.edu/datahopkins155 /, 2010. |