The Community for Technology Leaders
RSS Icon
Issue No.02 - Feb. (2013 vol.35)
pp: 450-462
O. Oreifej , Univ. of Central Florida, Orlando, FL, USA
Xin Li , Univ. of Central Florida, Orlando, FL, USA
M. Shah , Univ. of Central Florida, Orlando, FL, USA
Turbulence mitigation refers to the stabilization of videos with nonuniform deformations due to the influence of optical turbulence. Typical approaches for turbulence mitigation follow averaging or dewarping techniques. Although these methods can reduce the turbulence, they distort the independently moving objects, which can often be of great interest. In this paper, we address the novel problem of simultaneous turbulence mitigation and moving object detection. We propose a novel three-term low-rank matrix decomposition approach in which we decompose the turbulence sequence into three components: the background, the turbulence, and the object. We simplify this extremely difficult problem into a minimization of nuclear norm, Frobenius norm, and 21 norm. Our method is based on two observations: First, the turbulence causes dense and Gaussian noise and therefore can be captured by Frobenius norm, while the moving objects are sparse and thus can be captured by 21 norm. Second, since the object's motion is linear and intrinsically different from the Gaussian-like turbulence, a Gaussian-based turbulence model can be employed to enforce an additional constraint on the search space of the minimization. We demonstrate the robustness of our approach on challenging sequences which are significantly distorted with atmospheric turbulence and include extremely tiny moving objects.
Optimization, Mathematical model, Equations, Minimization, Matrix decomposition, Force, Object detection,restoring force, Three-term decomposition, turbulence mitigation, rank optimization, moving object detection, particle advection
O. Oreifej, Xin Li, M. Shah, "Simultaneous Video Stabilization and Moving Object Detection in Turbulence", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 2, pp. 450-462, Feb. 2013, doi:10.1109/TPAMI.2012.97
[1] M. Roggemann and B. Welsh, Imaging through Turbulence. CRC Press, 1996.
[2] D. Li, R.M. Mersereau, and S. Simske, "Atmospheric Turbulence-Degraded Image Restoration Using Principal Components Analysis," IEEE Geoscience and Remote Sensing Letters, vol. 4, no. 3, pp. 340-344, July 2007.
[3] J. Goodman, Statistical Optics. Wiley-Interscience, 2000.
[4] G.C.M. Aubailly, M. Vorontsov, and M. Valley, "Automated Video Enhancement from a Stream of Atmospherically-Distorted Images: The Lucky-Region Fusion Approach," Proc. SPIE, vol. 7463, pp. 74630C-74630C, 2009.
[5] X. Zhu and P. Milanfar, "Image Reconstruction from Videos Distorted by Atmospheric Turbulence," Proc. SPIE, vol. 7543, pp. 75430S-75430S, 2010.
[6] M. Shimizu, S. Yoshimura, M. Tanaka, and M. Okutomi, "Super-Resolution from Image Sequence under Influence of Hot-Air Optical Turbulence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[7] O. Oreifej, G. Shu, T. Pace, and M. Shah, "A Two-Stage Reconstruction Approach for Seeing through Water," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[8] Y. Yitzhaky, I. Dror, and N. Kopeika, "Restoration of Atmospherically Blurred Images According to Weather Predicted Atmospheric Modulation Transfer Function (MTF)," Optical Eng., vol. 36, pp. 3064-3072, 1997.
[9] M. Chang, C. Font, C. Gilbreath, E. Oh, E. Distefano, S. Restaino, C. Wilcox, and F. Santiago, "Comparing Horizontal Path C2N Measurements over 0.6 km in the Tropical Littoral Environment and in the Desert," Proc. SPIE, vol. 6551, p. 65510I, 2007.
[10] A. Efros, V. Isler, J. Shi, and M. Visontai, "Seeing through Water," Proc. Neural Information Processing Systems, 2004.
[11] A. Bjorck, Numerical Methods for Least Squares Problems. SIAM, 1996.
[12] S. Ali and M. Shah, "A Lagrangian Particle Dynamics Approach for Crowd Flow Segmentation and Stability Analysis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[13] E.J. Candes, X. Li, Y. Ma, and J. Wright, "Robust Principal Component Analysis?" Proc. IEEE Sensor Array and Multichannel Signal Workshop, 2009.
[14] H. Ji, C. Liu, and Y.X. Zuowei Shen, "Robust Video Denoising Using Low Rank Matrix Completion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[15] Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, "RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2233-2246, Nov. 2012.
[16] Y. Yu and D. Schuurmans, "Rank/Norm Regularization with Closed-Form Solutions: Application to Subspace Clustering," Proc. Conf. Uncertainty in Artificial Intelligence, 2011.
[17] S.H. Chan, R. Khoshabeh, K.B. Gibson, P.E. Gill, and T.Q. Nguyen, "An Augmented Lagrangian Method for Total Variation Video Restoration," IEEE Trans. Image Processing, vol. 20, no. 11, pp. 3097-3111, Nov. 2010.
[18] R. Cucchiara, C. Grana, M. Piccardi, and A. Prati, "Detecting Moving Objects, Ghosts, and Shadows in Video Streams," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 10, pp. 1337-1342, Oct. 2003.
[19] N.M. Oliver, B. Rosario, and A.P. Pentland, "A Bayesian Computer Vision System for Modeling Human Interactions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 831-843, Aug. 2000.
[20] C. Stauffer and W. Grimson, "Adaptive Background Mixture Models for Real-Time Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[21] A. Elgammal, D. Harwood, and L. Davis, "Non-Parametric Model for Background Subtraction," Proc. Sixth IEEE Int'l Conf. Computer Vision, 1999.
[22] Z. Zivkovic, "Improved Adaptive Gaussian Mixture Model for Background Subtraction," Proc. 17th Int'l Conf. Pattern Recognition, 2004.
[23] Y. Sheikh and M. Shah, "Bayesian Modeling of Dynamic Scenes for Object Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 11, pp. 1778-1792, Nov. 2005.
[24] X. Zhu and P. Milanfar, "Stabilizing and Deblurring Atmospheric Turbulence," Proc. IEEE Int'l Conf. Computational Photography, 2011.
[25] Y. Tian and S. Narasimhan, "A Globally Optimal Data-Driven Approach for Image Distortion Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[26] Y. Tian and S. Narsimhan, "Seeing through Water: Image Restoration Using Model-Based Tracking," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[27] A. Buades, B. Coll, and J. Morel, "A Non-Local Algorithm for Image Denoising," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[28] C. Liu and W. Freeman, "A High-Quality Video Denoising Algorithm Based on Reliable Motion Estimation," Proc. 11th European Conf. Computer Vision, 2010.
[29] Z. Wen, D. Fraser, A. Lambert, and H. Li, "Reconstruction of Underwater Image by Bispectrum," Proc. IEEE Int'l Conf. Image Processing, 2007.
[30] J. Goodman, Introduction to Fourier Optics. Roberts and Company Publishers, 2004.
[31] A. Donate, G. Dahme, and E. Ribeiro, "Classification of Textures Distorted by Water Waves," Proc. 18th Int'l Conf. Pattern Recognition, 2006.
[32] N. Joshi and M. Cohen, "Lucky Imaging for Multi-Image Denoising, Sharpening, and Haze Removal," Proc. IEEE Int'l Conf. Computational Photography, 2010.
[33] Z. Lin, M. Chen, L. Wu, and Y. Ma, "The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices," technical report, Univ. of Illinois at Urbana-Champaign, 2009.
[34] D.P. Bertsekas, Nonlinear Programming. Athena Scientific, 2004.
[35] J.-F. Cai, E.J. Candes, and Z. Shen, "A Singular Value Thresholding Algorithm for Matrix Completion," SIAM J. Optimization, vol. 20, no. 4, pp. 1956-1982, 2010.
[36] H. Branover and Y. Unge, Progress in Turbulence Research. AIAA, 1994.
[37] M. Isichenko and A. Gruzinov, "Isotopological Relaxation, Coherent Structures, and Gaussian Turbulence in Two-Dimensional (2-D) Magnetohydrodynamics (Mhd)," Physics of Plasmas, vol. 1, 1994.
[38] S. Wu, B. Moore, and M. Shah, "Chaotic Invariants of Lagrangian Particle Trajectories for Anomaly Detection in Crowded Scenes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[39] S. Wu, O. Oreifej, and M. Shah, "Action Recognition in Videos Acquired by a Moving Camera Using Motion Decomposition Of Lagrangian Particle Trajectories," Proc. 12th IEEE Int'l Conf. Computer Vision, 2011.
[40] A. Agarwal, S. Neghaban, and M. Wainwright, "Noisy Matrix Decomposition: Optimal Rates in High Dimensions," technical report,, Feb. 2011.
[41] M. Tao and X. Yuan, "Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations," SIAM J. Optimization, vol. 21, pp. 57-81, 2011.
[42] D. Gabay and B. Mercier, "A Dual Algorithm for the Solution of Nonlinear Variational Problems via Finite Element Approximation," Computers and Math. with Applciations, vol. 2, pp. 17-40, 1976.
[43] R. Glowinski and A. Marroco, "Sur l'Approximation, par Elements Finis d'Ordre Un, et la Resolution, par Penalisation-Dualite, d'une Classe de Problemes de Dirichlet Non Lineaires," Revue Francaise D'Automatique, Informatique et Recherche Operationelle, 1975.
[44] D. Goldfarb and S. Ma, "Fast Multiple Splitting Algorithms for Convex Optimization," preprint, 2009. available online
[45] B. He and X. Yuan, "Linearized Alternating Direction Method with Gaussian Back Substitutions for Separable Convex Programming," 2011/103192.html, 2011.
[46] Y. Zhang, "An Alternating Direction Algorithm for Nonnegative Matrix Factorization," technical report, Rice Univ., 2010.
74 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool