This Article 
 Bibliographic References 
 Add to: 
The Structure of Multiplicative Motions in Natural Imagery
July 2010 (vol. 32 no. 7)
pp. 1310-1316
Konstantinos G. Derpanis, York University, Toronto
Richard P. Wildes, York University, Toronto
A theoretical investigation of the frequency structure of multiplicative image motion signals is presented, e.g., as associated with translucency phenomena. Previous work has claimed that the multiplicative composition of visual signals generally results in the annihilation of oriented structure in the spectral domain. As a result, research has focused on multiplicative signals in highly specialized scenarios where highly structured spectral signatures are prevalent, or introduced a nonlinearity to transform the multiplicative image signal to an additive one. In contrast, in this paper, it is shown that oriented structure is present in multiplicative cases when natural domain constraints are taken into account. This analysis suggests that the various instances of naturally occurring multiple motion structures can be treated in a unified manner. As an example application of the developed theory, a multiple motion estimator previously proposed for translation, additive transparency, and occlusion is adapted to multiplicative image motions. This estimator is shown to yield superior performance over the alternative practice of introducing a nonlinear preprocessing step.

[1] E.H. Adelson and P. Anandan, "Ordinal Characteristics of Transparency," Proc. Am. Assoc. of Artificial Intelligence Workshop Qualitative Vision, pp. 77-81, 1990.
[2] E.H. Adelson and J.R. Bergen, "Spatiotemporal Energy Models for the Perception of Motion," J. Optical Soc. of Am. A, vol. 2, no. 2, pp. 284-299, Feb. 1985.
[3] I. Austvoll, "A Study of the Yosemite Sequence Used as a Test Sequence for Estimation of Optical Flow," Proc. Scandinavian Conf. Image Analysis, 2005.
[4] S. Baker, S. Roth, D. Scharstein, M.J. Black, J.P. Lewis, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[5] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, "Performance of Optical Flow Techniques," Int'l J. Computer Vision, vol. 12, no. 1, pp. 43-77, Feb. 1994.
[6] S.S. Beauchemin and J.L. Barron, "The Frequency Structure of 1D Occluding Image Signals," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 2, pp. 200-206, Feb. 2000.
[7] J.R. Bergen, P.J. Burt, R. Hingorani, and S. Peleg, "A Three-Frame Algorithm for Estimating Two-Component Image Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 9, pp. 886-896, Sept. 1992.
[8] R.N. Bracewell, The Fourier Transform and Its Applications. McGraw-Hill, 2000.
[9] P. Brodatz, Textures. Dover, 1966.
[10] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc. B, vol. 39, no. 1, pp. 1-38, 1977.
[11] K.G. Derpanis and R.P. Wildes, "Detecting Spatiotemporal Structure Boundaries: Beyond Motion Discontinuities," Proc. Asian Conf. Computer Vision, 2009.
[12] K.G. Derpanis and R.P. Wildes, "Early Spatiotemporal Grouping with a Distributed Oriented Energy Representation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[13] M. Fahle and T. Poggio, "Visual Hyperacuity: Spatio-Temporal Interpolation in Human Vision," Proc. Royal Soc. of London B, vol. 213, pp. 451-477, Nov. 1981.
[14] D.J. Fleet, Measurement of Image Velocity. Kluwer, 1992.
[15] D.J. Fleet and A.D. Jepson, "Computation of Component Image Velocity from Local Phase Information," Int'l J. Computer Vision, vol. 5, no. 1, pp. 77-104, Aug. 1990.
[16] D.J. Fleet and K. Langley, "Computational Analysis of Non-Fourier Motion," Vision Research, vol. 34, no. 22, pp. 3057-3079, Nov. 1994.
[17] F.J. Harris, "On the Use of Windows for Harmonic Analysis with the Discrete Fourier Transform," Proc. IEEE, vol. 66, no. 1, pp. 51-83, Jan. 1978.
[18] D.J. Heeger, "Model for the Extraction of Image Flow," J. Optical Soc. of Am. A, vol. 2, no. 2, pp. 1455-1471, Aug. 1987.
[19] B.K.P. Horn, Robot Vision. MIT Press, 1986.
[20] B.K.P. Horn and B.G. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, nos. 1-3, pp. 185-203, Aug. 1981.
[21] A.D. Jepson and M.J. Black, "Mixture Models for Optical Flow Computation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 760-761, 1993.
[22] D. Kersten, "Transparency and the Cooperative Computation of Scene Attributes," Computational Models of Visual Processing, M. Landy and J.A. Movshon, eds., pp. 209-228, MIT Press, 1991.
[23] K. Langley, "Computational Models of Coherent and Transparency Plaid Motion," Vision Research, vol. 39, pp. 87-108, Jan. 1998.
[24] C. Liu, W.T. Freeman, E.H. Adelson, and Y. Weiss, "Human-Assisted Motion Annotation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[25] P. Meer, D. Mintz, D.Y. Kim, and A. Rosenfeld, "Robust Regression Methods for Computer Vision: A Review," Int'l J. Computer Vision, vol. 6, no. 1, pp. 59-70, Apr. 1991.
[26] M. Shizawa and K. Mase, "Principle of Superposition: A Common Computational Framework for Analysis of Multiple Motion," Proc. Motion Workshop, pp. 164-172, 1991.
[27] E.P. Simoncelli and B. Olshausen, "Natural Image Statistics and Neural Representation," Ann. Rev. Neuroscience, vol. 24, pp. 1193-1216, May 2001.
[28] R.S. Szeliski, S. Avidan, and P. Anandan, "Layer Extraction from Multiple Images Containing Reflections and Transparency," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. I: 246-253, 2000.
[29] A.B. Watson and A.J. Ahumada, "Model of Human Visual-Motion Sensing," J. Optical Soc. of Am. A, vol. 2, no. 2, pp. 322-342, Feb. 1985.
[30] W. Yu, G. Sommer, S. Beauchemin, and K. Daniilidis, "Oriented Structure of the Occlusion Distortion: Is It Reliable?," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1286-1290, Sept. 2002.
[31] W. Yu, G. Sommer, and K. Daniilidis, "Multiple Motion Analysis: In Spatial or in Spectral Domain?" Computer Vision and Image Understanding, vol. 90, no. 2, pp. 129-152, May 2003.
[32] W.C. Yu, K. Daniilidis, S. Beauchemin, and G. Sommer, "Detection and Characterization of Multiple Motion Points," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. I: 171-177, 1999.

Index Terms:
Multiplicative motion, translucency, dynamic occlusion, pseudotransparency, non-Fourier motion, spectral analysis, optical flow, multiple motion.
Konstantinos G. Derpanis, Richard P. Wildes, "The Structure of Multiplicative Motions in Natural Imagery," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 7, pp. 1310-1316, July 2010, doi:10.1109/TPAMI.2010.64
Usage of this product signifies your acceptance of the Terms of Use.