The Community for Technology Leaders
RSS Icon
Issue No.07 - July (2012 vol.34)
pp: 1263-1280
Guy Ben-Yosef , Ben-Gurion University of the Negev, Beer-Sheva
Ohad Ben-Shahar , Ben-Gurion University of the Negev, Beer-Sheva
Visual curve completion is a fundamental perceptual mechanism that completes the missing parts (e.g., due to occlusion) between observed contour fragments. Previous research into the shape of completed curves has generally followed an “axiomatic” approach, where desired perceptual/geometrical properties are first defined as axioms, followed by mathematical investigation into curves that satisfy them. However, determining psychophysically such desired properties is difficult and researchers still debate what they should be in the first place. Instead, here we exploit the observation that curve completion is an early visual process to formalize the problem in the unit tangent bundle R²xS¹, which abstracts the primary visual cortex (V1) and facilitates exploration of basic principles from which perceptual properties are later derived rather than imposed. Exploring here the elementary principle of least action in V1, we show how the problem becomes one of finding minimum-length admissible curves in R²xS¹. We formalize the problem in variational terms, we analyze it theoretically, and we formulate practical algorithms for the reconstruction of these completed curves. We then explore their induced visual properties vis-à-vis popular perceptual axioms and show how our theory predicts many perceptual properties reported in the corresponding perceptual literature. Finally, we demonstrate a variety of curve completions and report comparisons to psychophysical data and other completion models.
Visual completion, curve completion, tangent bundle, inpainting.
Guy Ben-Yosef, Ohad Ben-Shahar, "A Tangent Bundle Theory for Visual Curve Completion", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 7, pp. 1263-1280, July 2012, doi:10.1109/TPAMI.2011.262
[1] O. Ben-Shahar, "The Perceptual Organization of Visual Flows," PhD thesis, Yale Univ., 2003.
[2] O. Ben-Shahar and S.W. Zucker, "The Perceptual Organization of Texture Flows: A Contextual Inference Approach," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 4, pp. 401-417, Apr. 2003.
[3] O. Ben-Shahar and S.W. Zucker, "Geometrical Computations Explain Projection Patterns of Long Range Horizontal Connections in Visual Cortex," Neural Computation, vol. 16, no. 3, pp. 445-476, 2004.
[4] G. Ben-Yosef and O. Ben-Shahar, "Minimum Length in the Tangent Bundle as a Model for Curve Completion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[5] W.H. Bosking, Y. Zhang, B. Schofield, and D. Fitzpatrick, "Orientation Selectivity and the Arrangement of Horizontal Connections in the Tree Shrew Striate Cortex," The J. Neuroscience, vol. 17, no. 6, pp. 2112-2127, 1997.
[6] M. Brady, W.E.L. Grimson, and D.J. Langridge, "Shape Encoding and Subjective Contours," Proc. Am. Assoc. for Artificial Intelligence Conf., pp. 15-17, 1980.
[7] G. Citti and A. Sarti, "A Cortical Based Model of Perceptual Completion in the Roto-Translation Space," J. Math. Imaging and Vision, vol. 24, no. 3, pp. 307-326, 2006.
[8] S. Coren, "Subjective Contours and Apparent Depth," Psychological Rev., vol. 79, no. 4, pp. 359-367, 1972.
[9] J.H. Elder and R.M. Goldberg, "Ecological Statistics of Gestalt Laws for the Perceptual Organization of Contours," J. Vision, vol. 2, pp. 324-353, 2002.
[10] C. Fantoni and W. Gerbino, "Contour Interpolation by Vector-Field Combination," J. Vision, vol. 3, pp. 281-303, 2003.
[11] J.M. Fulvio, M. Singh, and L.T. Maloney, "Precision and Consistency of Contour Interpolation," Vision Research, vol. 48, pp. 831-849, 2008.
[12] J.M. Fulvio, M. Singh, and L.T. Maloney, "An Experimental Criterion for Consistency in Interpolation of Partly Occluded Contours," J. Vision, vol. 9, pp. 1-19, 2009.
[13] W.S. Geisler and J.S. Perry, "Contour Statistics in Natural Images: Grouping across Occlusions," Visual Neuroscience, vol. 26, pp. 109-121, 2009.
[14] W.S. Geisler, J.S. Perry, B.J. Super, and D.P. Gallogly, "Edge Co-Occurrence in Natural Images Predicts Contour Grouping Performance," Vision Research, vol. 41, no. 6, pp. 711-724, 2001.
[15] W. Gerbino and C. Fantoni, "Visual Interpolation in Not Scale Invariant," Vision Research, vol. 46, pp. 3142-3159, 2006.
[16] C.D. Gilbert, "Horizontal Integration and Cortical Dynamics," Neuron, vol. 9, pp. 1-13, 1992.
[17] D.H. Grosof, R.M. Shapley, and M.J. Hawken, "Macaque v1 Neurons Can Signal 'Illusory' Contours," Nature, vol. 365, pp. 550-552, 1993.
[18] S.E. Guttman and P.J. Kellman, "Contour Interpolation Revealed by a Dot Localization Paradigm," Vision Research, vol. 44, pp. 1799-1815, 2004.
[19] R.K. Hladky and S.D. Pauls, "Minimal Surfaces in the Roto-translational Group with Applications to a Neuro-Biological Image Completion Model," J. Math. Imaging and Vision, vol. 36, pp. 1-27, 2010.
[20] W.C. Hoffman, "The Visual Cortex Is a Contact Bundle," Applied Math. and Computation, vol. 32, pp. 137-167, 1989.
[21] B.K.P. Horn, "The Curve of Least Energy," ACM Trans. Math. Software, vol. 9, no. 4, pp. 441-460, 1983.
[22] J. Hoschek and D. Lasser, Fundamentals of Computer Aided Geometric Design. A.K. Peters, Ltd., 1989.
[23] D. Hubel and T. Wiesel, "Functional Architecture of Macaque Monkey Visual Cortex," Royal Soc. of London, Series B, vol. 198, pp. 1-59, 1977.
[24] Y.M. Jung and J. Shen, "First-Order Modeling and Stability Analysis of Illusory Contours," J. Visual Comm. and Image Representation, vol. 19, pp. 42-55, 2008.
[25] G. Kanizsa, Organization in Vision: Essays on Gestalt Perception. Praeger Publishers, 1979.
[26] P.J. Kellman and T.F. Shipley, "A Theory of Visual Interpolation in Object Perception," Cognitive Psychology, vol. 23, pp. 141-221, 1991.
[27] B.K. Kimia, I. Frankel, and A.M. Popescu, "Euler Spiral for Shape Completion," Int'l J. Computer Vision, vol. 54, nos. 1-3, pp. 159-182, 2003.
[28] D.E. Knuth, "Mathematical Typography," Bull. of the Am. Math. Soc., vol. 1, pp. 337-372, 1979.
[29] W. Köhler, "Physical Gestalten," A Source Book of Gestalt Psychology, W.D. Ellis, ed., pp. 17-53, Routledge & Kegan Paul Ltd., 1920.
[30] L.D. Landau and E.M. Lifshitz, Theory of Elasticity. Pergamon, 1986.
[31] T.S. Lee and M. Nguyen, "Dynamics of Subjective Contours Formation in the Early Visual Cortex," Proc. Nat'l Academy of Sciences USA, vol. 98, no. 4, pp. 1907-1911, 2001.
[32] R. Levien, "The Elastica: A Mathematical History," Technical Report UCB/EECS-2008-103, Electrical Eeng. and Computer Science Dept., Univ. of California, Berkeley, Aug. 2008.
[33] R. Levien, "The Euler Spiral: A Mathematical History," Technical Report UCB/EECS-2008-111, Electrical Eng. and Computer Science Dept., Univ. of California, Berkeley, Sept. 2008.
[34] A.E.H. Love, A Treatise on the Mathematical Theory of Elasticity. Dover Publications, 1944.
[35] D. Mumford, "Elastica in Computer Vision," Algebric Geometry and Its Applications, B. Chandrajit, ed., Springer-Verlag, 1994.
[36] B. O'Neill, Semi-Riemannian Geometry with Applications to Relativity. Academic Press, 1983.
[37] S.E. Palmer, Vision Science: Photons to Phenomenology. The MIT Press, 1999.
[38] J. Petitot, "Neurogeometry of v1 and Kanizsa Contours," Axiomathes, vol. 13, nos. 3/4, pp. 347-363, 2003.
[39] X. Ren, C. Fowlkes, and J. Malik, "Learning Probabilistic Models for Contour Completion in Natural Images," Int'l J. Computer Vision, vol. 77, nos. 1-3, pp. 47-63, 2008.
[40] D.L. Ringach and R. Shapley, "Spatial and Temporal Properties of Illusory Contours and Amodal Boundary Completion," Vision Research, vol. 36, no. 19, pp. 3037-3050, 1996.
[41] K.S. Rockland and J.S. Lund, "Widespread Periodic Intrinsic Connections in the Tree Shrew Visual Corte," Science, vol. 215, no. 19, pp. 1532-1534, 1982.
[42] W.S. Rutkowski, "Shape Completion," Computer Vision, Graphics, and Image Processing, vol. 9, pp. 89-101, 1979.
[43] A. Sarti, R. Malladi, and J.A. Sethian, "Subjective Surfaces: A Method for Completing Missing Boundaries," Int'l J. Computer Vision, vol. 46, no. 32, pp. 201-221, 2002.
[44] I.J. Schoenberg, "Contributions to the Problem of Approximation of Equidistant Data by Analytic Functions," Quarterly of Applied Math., vol. 4, pp. 44-99, 1946.
[45] F. Schumann, "Beitrage zur Analyse der Gesichtswhrnehmungen," Zeitschrift für Psychologie, vol. 33, pp. 161-185, 1904.
[46] E. Sharon, A. Brandt, and R. Basri, "Completion Energies and Scale," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1117-1131, Oct. 2000.
[47] M. Singh, "Modal and Amodal Completion Generate Different Shapes," Psychological Science, vol. 15, no. 7, pp. 454-459, 2004.
[48] M. Singh and J.M. Fulvio, "Visual Extrapolation of Contour Geometry," Proc. Nat'l Academy of Sciences USA, vol. 102, pp. 939-944, 2005.
[49] M. Singh and D.D. Hoffman, "Completing Visual Contours: The Relationship between Relatability and Minimizing Inflections," Perception and Psychophisics, vol. 61, pp. 943-951, 1999.
[50] H. Takeichi, "The Effect of Curvature on Visual Interpolation," Perception, vol. 24, pp. 1011-1020, 1995.
[51] S. Ullman, "Filling in the Gaps: The Shape of Subjective Contours and a Model for Their Creation," Biological Cybernetics, vol. 25, pp. 1-6, 1976.
[52] R. von der Heydt, E. Peterhans, and G. Baumgartner, "Illusory Contours and Cortical Neuron Responses," Science, vol. 224, pp. 1260-1262, 1984.
[53] I. Weiss, "3D Shape Representation by Contours," Computer Vision, Graphics, and Image Processing, vol. 41, pp. 80-100, 1988.
[54] M. Wertheimer, "Untersuchungen zur Lehre von der Gestalt," Psychologische Forschung, vol. 4, pp. 301-350, 1923.
[55] S. Willard, General Topology. Addison-Wesley Publishing Company, 1970.
[56] L.R. Williams and D.W. Jacobs, "Stochastic Completion Fields: A Neural Model of Illusory Contour Shape and Salience," Neural Computation, vol. 9, no. 4, pp. 837-858, 1997.
25 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool