This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Spacetime Texture Representation and Recognition Based on a Spatiotemporal Orientation Analysis
June 2012 (vol. 34 no. 6)
pp. 1193-1205
ASCII Text
x 
 
K. G. P. Derpanis, R. Wildes, "Spacetime Texture Representation and Recognition Based on a Spatiotemporal Orientation Analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 6, pp. 1193-1205, June, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2011.221,
author = {K. G. P. Derpanis and R. Wildes},
title = {Spacetime Texture Representation and Recognition Based on a Spatiotemporal Orientation Analysis},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {34},
number = {6},
issn = {0162-8828},
year = {2012},
pages = {1193-1205},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.221},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Spacetime Texture Representation and Recognition Based on a Spatiotemporal Orientation Analysis
IS - 6
SN - 0162-8828
SP1193
EP1205
EPD - 1193-1205
A1 - K. G. P. Derpanis,
A1 - R. Wildes,
PY - 2012
KW - stochastic processes
KW - image recognition
KW - image representation
KW - image sequences
KW - image texture
KW - state-of-the-art approach
KW - visual spacetime texture representation
KW - visual spacetime texture recognition
KW - spatiotemporal orientation analysis
KW - aggregate dynamic property
KW - local measurement
KW - spatiotemporal support
KW - image sequence
KW - stochastic dynamics
KW - aggregate region property
KW - image streaming
KW - associated recognition method
KW - spacetime orientation structure
KW - empirical evaluation
KW - original image data sets
KW - Vehicle dynamics
KW - Frequency domain analysis
KW - Dynamics
KW - Spatiotemporal phenomena
KW - Visualization
KW - Energy measurement
KW - Pattern recognition
KW - spatiotemporal orientation.
KW - Spacetime texture
KW - image motion
KW - dynamic texture
KW - temporal texture
KW - time-varying texture
KW - textured motion
KW - turbulent flow
KW - stochastic dynamics
KW - distributed representation
VL - 34
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
K. G. P. Derpanis, Dept. of Comput. Sci. & Eng., York Univ., Toronto, ON, Canada
R. Wildes, Dept. of Comput. Sci. & Eng., York Univ., Toronto, ON, Canada
This paper is concerned with the representation and recognition of the observed dynamics (i.e., excluding purely spatial appearance cues) of spacetime texture based on a spatiotemporal orientation analysis. The term “spacetime texture” is taken to refer to patterns in visual spacetime, (x,y,t), that primarily are characterized by the aggregate dynamic properties of elements or local measurements accumulated over a region of spatiotemporal support, rather than in terms of the dynamics of individual constituents. Examples include image sequences of natural processes that exhibit stochastic dynamics (e.g., fire, water, and windblown vegetation) as well as images of simpler dynamics when analyzed in terms of aggregate region properties (e.g., uniform motion of elements in imagery, such as pedestrians and vehicular traffic). Spacetime texture representation and recognition is important as it provides an early means of capturing the structure of an ensuing image stream in a meaningful fashion. Toward such ends, a novel approach to spacetime texture representation and an associated recognition method are described based on distributions (histograms) of spacetime orientation structure. Empirical evaluation on both standard and original image data sets shows the promise of the approach, including significant improvement over alternative state-of-the-art approaches in recognizing the same pattern from different viewpoints.

[1] D. Heeger and A. Pentland, "Seeing Structure through Chaos," Proc. Workshop Motion, pp. 131-136, 1986.
[2] R. Nelson and R. Polana, "Qualitative Recognition of Motion Using Temporal Texture," CVGIP: Image Understanding, vol. 56, no. 1, pp. 78-89, 1992.
[3] Z. Bar-Joseph, R. El-Yaniv, D. Lischinski, and M. Werman, "Texture Mixing and Texture Movie Synthesis Using Statistical Learning," IEEE Trans. Visualization and Computer Graphics, vol. 7, no. 2, pp. 120-135, Apr.-June 2001.
[4] P. Saisan, G. Doretto, Y. Wu, and S. Soatto, "Dynamic Texture Recognition," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 58-63, 2001.
[5] Y. Wang and S. Zhu, "Modeling Textured Motion: Particle, Wave and Sketch," Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 213-220, 2003.
[6] J. Bergen, "Theories of Visual Texture Perception," Proc. Vision and Visual Dysfunction, pp. 114-134, 1991.
[7] D. Chetverikov and R. Peteri, "A Brief Survey of Dynamic Texture Description and Recognition," Proc. Int'l Conf. Computer Recognition Systems, pp. 17-26, 2005.
[8] T. Kung and W. Richards, "Inferring 'Water' from Images," Natural Computation, W. Richards, ed., pp. 224-233, MIT Press, 1988.
[9] P. Bouthemy and R. Fablet, "Motion Characterization from Temporal Cooccurrences of Local Motion-Based Measures for Video Indexing," Proc. 14th Int'l Conf. Pattern Recognition, pp. 905-908, 1998.
[10] Z. Lu, W. Xie, J. Pei, and J. Huang, "Dynamic Texture Recognition by Spatio-Temporal Multiresolution Histograms," Proc. IEEE Workshop Motion Video Computing, pp. 241-246, 2005.
[11] R. Polana and R. Nelson, "Temporal Texture and Activity Recognition," Motion-Based Recognition, M. Shah and R. Jain, eds., Kluwer, 1997.
[12] G. Zhao and M. Pietikainen, "Dynamic Texture Recognition Using Local Binary Patterns with an Application to Facial Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 915-928, June 2007.
[13] M. Szummer and R. Picard, "Temporal Texture Modeling," Proc. Int'l Conf. Image Processing, pp. 823-826, 1996.
[14] G. Doretto, A. Chiuso, Y. Wu, and S. Soatto, "Dynamic Textures," Int'l J. Computer Vision, vol. 51, no. 2, pp. 91-109, 2003.
[15] A. Fitzgibbon, "Stochastic Rigidity: Image Registration for Nowhere-Static Scenes," Proc. Eighth IEEE Int'l Conf. Computer Vision, pp. 662-669, 2001.
[16] A. Chan and N. Vasconcelos, "Probabilistic Kernels for the Classification of Auto-Regressive Visual Processes," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 846-851, 2005.
[17] F. Woolfe and A. Fitzgibbon, "Shift-Invariant Dynamic Texture Recognition," Proc. Ninth European Conf. Computer Vision, pp. 549-562, 2006.
[18] A. Ravichandran, R. Chaudhry, and R. Vidal, "View-Invariant Dynamic Texture Recognition Using a Bag of Dynamical Systems," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2009.
[19] D. Heeger, "Model for the Extraction of Image Flow," J. Optical Soc. Am. A, vol. 2, no. 2, pp. 1455-1471, 1987.
[20] E. Simoncelli, "Distributed Analysis and Representation of Visual Motion," PhD dissertation, MIT, 1993.
[21] G. Granlund and H. Knutsson, Signal Processing for Computer Vision. Kluwer, 1995.
[22] O. Chomat and J. Crowley, "Probabilistic Recognition of Activity Using Local Appearance," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 104-109, 1999.
[23] P. Dollar, V. Rabaud, G. Cottrell, and S. Belongie, "Behavior Recognition via Sparse Spatio-Temporal Features," Proc. IEEE Second Int'l Workshop Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 65-72, 2005.
[24] K. Derpanis, M. Sizintsev, K. Cannons, and R. Wildes, "Efficient Action Spotting Based on a Spacetime Oriented Structure Representation," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2010.
[25] A. Zaharescu and R. Wildes, "Anomalous Behaviour Detection Using Spatiotemporal Oriented Energies Subset Inclusion Histogram Comparison and Event-Driven Processing," Proc. 11th European Conf. Computer Vision, 2010.
[26] R. Wildes and J. Bergen, "Qualitative Spatiotemporal Analysis Using an Oriented Energy Representation," Proc. Sixth European Conf. Computer Vision, pp. 768-784, 2000.
[27] K. Cannons, J. Gryn, and R. Wildes, "Visual Tracking using a Pixelwise Spatiotemporal Oriented Energy Representation," Proc. 11th European Conf. Computer Vision, 2010.
[28] M. Sizintsev and R. Wildes, "Spatiotemporal Stereo via Spatiotemporal Quadric Element (Stequel) Matching," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2009.
[29] K. Derpanis and R. Wildes, "Early Spatiotemporal Grouping with a Distributed Oriented Energy Representation," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2009.
[30] K. Derpanis and R. Wildes, "Detecting Spatiotemporal Structure Boundaries: Beyond Motion Discontinuities," Proc. Asian Conf. Computer Vision, 2009.
[31] M. Tuceryan and A. Jain, "Texture Analysis," Handbook of Pattern Recognition and Computer Vision, C. Chen, L. Pau, and P. Wang, eds., second ed., World Scientific Publishing, 1998.
[32] T. Leung and J. Malik, "Representing and Recognizing the Visual Appearance of Materials Using Three-Dimensional Textons," Int'l J. Computer Vision, vol. 43, no. 1, pp. 29-44, 2001.
[33] O. Cula and K. Dana, "3D Texture Recognition Using Bidirectional Feature Histograms," Int'l J. Computer Vision, vol. 59, no. 1, pp. 33-60, 2004.
[34] M. Varma and A. Zisserman, "A Statistical Approach to Texture Classification from Single Images," Int'l J. Computer Vision, vol. 62, nos. 1/2, pp. 61-81, 2005.
[35] D. Williams and R. Sekuler, "Coherent Global Motion Percepts from Stochastic Local Motions," Vision Research, vol. 24, no. 1, pp. 55-62, 1984.
[36] D. Williams, S. Tweten, and R. Sekuler, "Using Metamers to Explore Motion Perception," Vision Research, vol. 31, no. 2, pp. 275-286, 1991.
[37] S. Treue, K. Hol, and H. Rauber, "Seeing Multiple Directions of Motion—Physiology and Psychophysics," Nature Neuroscience, vol. 3, pp. 270-276, 2000.
[38] K. Derpanis and R. Wildes, "Dynamic Texture Recognition Based on Distributions of Spacetime Oriented Structure," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2010.
[39] M. Fahle and T. Poggio, "Visual Hyperacuity: Spatio-Temporal Interpolation in Human Vision," Proc. Royal Soc. London B, vol. 213, no. 1193, pp. 451-477, 1981.
[40] A. Watson and A. Ahumada, "A Look at Motion in the Frequency Domain," Proc. Motion Workshop, pp. 1-10, 1983.
[41] E. Adelson and J. Bergen, "Spatiotemporal Energy Models for the Perception of Motion," J. Optical Soc. Am. A, vol. 2, no. 2, pp. 284-299, 1985.
[42] K. Garg and S. Nayar, "Vision and Rain," Int'l J. Computer Vision, vol. 75, no. 1, pp. 3-27, 2007.
[43] P. Barnum, S. Narasimhan, and T. Kanade, "Analysis of Rain and Snow in Frequency Space," Int'l J. Computer Vision, vol. 86, nos. 2/3, pp. 256-274, 2010.
[44] M. Langer and R. Mann, "Optical Snow," Int'l J. Computer Vision, vol. 55, no. 1, pp. 55-71, 2003.
[45] K. Derpanis and R. Wildes, "Classification of Traffic Video Based on a Spatiotemporal Orientation Analysis," Proc. IEEE Applications of Computer Vision, pp. 606-613, 2011.
[46] D. Fleet, Measurement of Image Velocity. Kluwer, 1992.
[47] K. Derpanis and R. Wildes, "The Structure of Multiplicative Motions in Natural Imagery," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 7, pp. 1310-1316, July 2010.
[48] K. Derpanis and J. Gryn, "Three-Dimensional nth Derivative of Gaussian Separable Steerable Filters," Proc. IEEE Int'l Conf. Image Processing, pp. 553-556, 2005.
[49] R. Bracewell, The Fourier Transform and Its Applications. McGraw-Hill, 2000.
[50] W. Freeman and E. Adelson, "The Design and Use of Steerable Filters," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 9, pp. 891-906, Sept. 1991.
[51] Y. Rubner, C. Tomasi, and L. Guibas, "The Earth Mover's Distance as a Metric for Image Retrieval," Int'l J. Computer Vision, vol. 40,  no. 2, pp. 99-121, 2000.
[52] A. Bhattacharyya, "On a Measure of Divergence between Two Statistical Populations Defined by Their Probability Distribution," Bull. Calcutta Math. Soc., vol. 35, pp. 99-110, 1943.
[53] R. Duda, P. Hart, and D. Stork, Pattern Classification. Wiley, 2001.
[54] V. Vapnik, The Nature of Statistical Learning Theory. Springer, 1995.
[55] A. Chan and N. Vasconcelos, "Classifying Video with Kernel Dynamic Textures," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2007.
[56] E. Rosch and C. Mervis, "Family Resemblances: Studies in the Internal Structure of Categories," Cognitive Psychology, vol. 7, pp. 573-605, 1975.
[57] W. Yu, K. Daniilidis, S. Beauchemin, and G. Sommer, "Detection and Characterization of Multiple Motion Points," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 171-177, 1999.
[58] S. Beauchemin and J. Barron, "The Frequency Structure of 1D Occluding Image Signals," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 2, pp. 200-206, Feb. 2000.
[59] T. Lindeberg, "Linear Spatio-Temporal Scale-Space," Scale-Space, pp. 113-127, 1997.

Index Terms:
stochastic processes,image recognition,image representation,image sequences,image texture,state-of-the-art approach,visual spacetime texture representation,visual spacetime texture recognition,spatiotemporal orientation analysis,aggregate dynamic property,local measurement,spatiotemporal support,image sequence,stochastic dynamics,aggregate region property,image streaming,associated recognition method,spacetime orientation structure,empirical evaluation,original image data sets,Vehicle dynamics,Frequency domain analysis,Dynamics,Spatiotemporal phenomena,Visualization,Energy measurement,Pattern recognition,spatiotemporal orientation.,Spacetime texture,image motion,dynamic texture,temporal texture,time-varying texture,textured motion,turbulent flow,stochastic dynamics,distributed representation
Citation:
K. G. P. Derpanis, R. Wildes, "Spacetime Texture Representation and Recognition Based on a Spatiotemporal Orientation Analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 6, pp. 1193-1205, June 2012, doi:10.1109/TPAMI.2011.221
Usage of this product signifies your acceptance of the Terms of Use.