Subscribe
Issue No.10 - October (2009 vol.31)
pp: 1804-1816
Minwoo Park , Pennsylvania State University, University Park
Kyle Brocklehurst , Pennsylvania State University, University Park
Robert T. Collins , Pennsylvania State University, University Park
Yanxi Liu , Pennsylvania State University, University Park
ABSTRACT
We propose a novel and robust computational framework for automatic detection of deformed 2D wallpaper patterns in real-world images. The theory of 2D crystallographic groups provides a sound and natural correspondence between the underlying lattice of a deformed wallpaper pattern and a degree-4 graphical model. We start the discovery process with unsupervised clustering of interest points and voting for consistent lattice unit proposals. The proposed lattice basis vectors and pattern element contribute to the pairwise compatibility and joint compatibility (observation model) functions in a Markov Random Field (MRF). Thus, we formulate the 2D lattice detection as a spatial, multitarget tracking problem, solved within an MRF framework using a novel and efficient Mean-Shift Belief Propagation (MSBP) method. Iterative detection and growth of the deformed lattice are interleaved with regularized thin-plate spline (TPS) warping, which rectifies the current deformed lattice into a regular one to ensure stability of the MRF model in the next round of lattice recovery. We provide quantitative comparisons of our proposed method with existing algorithms on a diverse set of 261 real-world photos to demonstrate significant advances in accuracy and speed over the state of the art in automatic discovery of regularity in real images.
INDEX TERMS
Belief propagation, MRF, mean shift, lattice detection, wallpaper patterns.
CITATION
Minwoo Park, Kyle Brocklehurst, Robert T. Collins, Yanxi Liu, "Deformed Lattice Detection in Real-World Images Using Mean-Shift Belief Propagation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 10, pp. 1804-1816, October 2009, doi:10.1109/TPAMI.2009.73
REFERENCES
[1] Y. Liu, W.C. Lin, and J. Hays, “Near-Regular Texture Analysis and Manipulation,” ACM Trans. Graphics, vol. 23, no. 3, pp. 368-376, 2004.
[2] J.J. Gibson, The Perception of the Visual World. Houghton Mifflin, 1950.
[3] B. Julesz, “Visual Pattern Discrimination,” IRE Trans. Information Theory, vol. 8, no. 2, pp. 84-92, 1962.
[4] J. Malik, S. Belongie, J. Shi, and T. Leung, “Textons, Contours and Regions: Cue Integration in Image Segmentation,” Proc. Seventh IEEE Int'l Conf. Computer Vision, vol. 2, pp. 918-925, 1999.
[5] D. Forsyth, “Shape from Texture without Boundaries,” Proc. Seventh European Conf. Computer Vision, pp. 43-66, 2002.
[6] W.C. Lin, J. Hays, C. Wu, V. Kwatra, and Y. Liu, “Quantitative Evaluation of Near Regular Texture Synthesis Algorithms,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 427-434, 2006.
[7] B. Ghanem and N. Ahuja, “Phase Based Modelling of Dynamic Textures,” Proc. IEEE 11th Int'l Conf. Computer Vision, pp. 1-8, 2007.
[8] B. Grunbaum and G. Shephard, Tilings and Patterns. W.H. Freeman and Company, 1987.
[9] H. Coxeter, Introduction to Geometry, second ed. Wiley, 1980.
[10] Y. Liu, R.T. Collins, and Y. Tsin, “A Computational Model for Periodic Pattern Perception Based on Frieze and Wallpaper Groups,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 3, pp. 354-371, Mar. 2004.
[11] Y. Liu, Y. Tsin, and W.C. Lin, “The Promise and Perils of Near-Regular Texture,” Int'l J. Computer Vision, vol. 62, nos. 1/2, pp.145-159, 2005.
[12] J. Hays, M. Leordeanu, A. Efros, and Y. Liu, “Discovering Texture Regularity as a Higher-Order Correspondence Problem,” Proc. Ninth European Conf. Computer Vision, pp. 522-535, 2006.
[13] W.C. Lin and Y. Liu, “Tracking Dynamic Near-Regular Textures under Occlusions and Rapid Movements,” Proc. Ninth European Conf. Computer Vision, pp. 44-55, 2006.
[14] W.C. Lin and Y. Liu, “A Lattice-Based MRF Model for Dynamic Near-Regular Texture Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 777-792, May 2007.
[15] M. Park, Y. Liu, and R.T. Collins, “Efficient Mean Shift Belief Propagation for Vision Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[16] D. Ramanan and D.A. Forsyth, “Finding and Tracking People from the Bottom Up,” Computer Vision and Pattern Recognition, vol. 2, pp. 467-474, 2003.
[17] J. Coughlan and S. Huiying, “Shape Matching with Belief Propagation: Using Dynamic Quantization to Accommodate Occlusion and Clutter,” Proc. Conf. Computer Vision and Pattern Recognition Workshop, p. 180, 2004.
[18] P.F. Felzenszwalb, “Efficient Belief Propagation for Early Vision,” Int'l J. Computer Vision, vol. 70, no. 1, pp. 41-54, 2006.
[19] E.B. Sudderth, A.T. Ihler, W.T. Freeman, and A.S. Willsky, “Nonparametric Belief Propagation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 605-612, 2003.
[20] M. Isard, “Pampas: Real-Valued Graphical Models for Computer Vision,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 613-620, 2003.
[21] T.X. Han, N. Huazhong, and T.S. Huang, “Efficient Nonparametric Belief Propagation with Application to Articulated Body Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 214-221, 2006.
[22] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[23] Y. Weiss and W. Freeman, “Correctness of Belief Propagation in Gaussian Graphical Models of Arbitrary Topology,” Neural Computation, vol. 13, no. 10, pp. 2173-2200, 2001, ISSN 0899-7667.
[24] L. Sigal, Pampas/Non-Parametric Belief Propagation Toolbox for Matlab v0.1, 2005.
[25] R. Frost, Simulated Annealing Tools for Matlab v1.03, http://www.frostconcepts.com/softwaresatools. pdf , 2009.
[26] P. Salamon, P. Sibani, and R. Frost, Facts, Conjectures, and Improvements for Simulated Annealing. SIAM, 2002.
[27] W.T. Freeman, “Learning Low-Level Vision,” Int'l J. Computer Vision, vol. 40, no. 1, pp. 25-47, 2000.
[28] Y. Weiss, “Correctness of Local Probability Propagation in Graphical Models with Loops,” Neural Computation, vol. 12, no. 1, pp. 1-41, 2000.
[29] J. Shi and C. Tomasi, “Good Features to Track,” Proc. IEEE Conf. Computer Vision and Pattern Recognitions, pp. 593-600, 1994.
[30] G. Schindler, P. Krishnamurthy, R. Lublinerman, Y. Liu, and F. Dellaert, “Detecting and Matching Repeated Patterns for Automatic Geo-Tagging in Urban Environments,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[31] M. Park, R.T. Collins, and Y. Liu, “Deformed Lattice Discovery via Efficient Mean-Shift Belief Propagation,” Proc. 10th European Conf. Computer Vision, 2008.
[32] K.C.J.W.B.A. Canada, G.K. Thomas, and Y. Liu, “Automatic Lattice Detection in Near-Regular Histology Array Images,” Proc. IEEE Int'l Conf. Image Processing, 2008.
[33] Y. Liu, T. Belkina, J. Hays, and R. Lublinerman, “Image de Fencing,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[34] E. Chastain and Y. Liu, “Quantified Symmetry for Entorhinal Spatial Maps,” Neurocomputing J., special issue, vol. 70, nos. 10-12, pp. 1723-1727, 2007.
[35] Y. Tsin, Y. Liu, and V. Ramesh, “Texture Replacement in Real Images,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 539-544, 2001.