The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - October (2011 vol.33)
pp: 2051-2065
P. F. U. Gotardo , Dept. of Electr. & Comput. Eng., Ohio State Univ., Columbus, OH, USA
A. M. Martinez , Dept. of Electr. & Comput. Eng., Ohio State Univ., Columbus, OH, USA
ABSTRACT
We address the classical computer vision problems of rigid and nonrigid structure from motion (SFM) with occlusion. We assume that the columns of the input observation matrix W describe smooth 2D point trajectories over time. We then derive a family of efficient methods that estimate the column space of W using compact parameterizations in the Discrete Cosine Transform (DCT) domain. Our methods tolerate high percentages of missing data and incorporate new models for the smooth time trajectories of 2D-points, affine and weak-perspective cameras, and 3D deformable shape. We solve a rigid SFM problem by estimating the smooth time trajectory of a single camera moving around the structure of interest. By considering a weak-perspective camera model from the outset, we directly compute euclidean 3D shape reconstructions without requiring postprocessing steps such as euclidean upgrade and bundle adjustment. Our results on real SFM data sets with high percentages of missing data compared positively to those in the literature. In nonrigid SFM, we propose a novel 3D shape trajectory approach that solves for the deformable structure as the smooth time trajectory of a single point in a linear shape space. A key result shows that, compared to state-of-the-art algorithms, our nonrigid SFM method can better model complex articulated deformation with higher frequency DCT components while still maintaining the low-rank factorization constraint. Finally, we also offer an approach for nonrigid SFM when W is presented with missing data.
INDEX TERMS
shape recognition, cameras, computer graphics, computer vision, discrete cosine transforms, matrix decomposition, motion estimation, missing data, smooth time trajectories, 3D deformable shape, occlusion, computer vision, structure from motion, input observation matrix, compact parameterizations, discrete cosine transform, weak-perspective cameras, motion estimation, 3D shape reconstructions, 3D shape trajectory, low-rank factorization constraint, Cameras, Shape, Discrete cosine transforms, Three dimensional displays, Trajectory, Jacobian matrices, Periodic structures, shape trajectory., Structure from motion, matrix factorization, missing data, camera trajectory
CITATION
P. F. U. Gotardo, A. M. Martinez, "Computing Smooth Time Trajectories for Camera and Deformable Shape in Structure from Motion with Occlusion", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 10, pp. 2051-2065, October 2011, doi:10.1109/TPAMI.2011.50
REFERENCES
[1] I. Akhter, Y. Sheikh, and S. Khan, "In Defense of Orthonormality Constraints for Nonrigid Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1534-1541, 2009.
[2] I. Akhter, Y.A. Sheikh, S. Khan, and T. Kanade, "Nonrigid Structure from Motion in Trajectory Space," Proc. Neural Information Processing Systems, Dec. 2008.
[3] A. Bartoli, V. Gay-Bellile, U. Castellani, J. Peyras, S. Olsen, and P. Sayd, "Coarse-to-Fine Low-Rank Structure-from-Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, no. 1, pp. 1-8, 2008.
[4] D. Bertsekas, Nonlinear Programming. Athena Scientific Press, 1999.
[5] C. Bregler, A. Hertzmann, and H. Biermann, "Recovering Non-Rigid 3d Shape from Image Streams," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 690-696, 2000.
[6] A.M. Buchanan and A.W. Fitzgibbon, "Damped Newton Algorithms for Matrix Factorization with Missing Data," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 316-322, 2005.
[7] P. Chen and D. Suter, "Recovering the Missing Components in a Large Noisy Low-Rank Matrix: Application to SFM," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 8, pp. 1051-1063, Aug. 2004.
[8] P. Chen, "Optimization Algorithms on Subspaces: Revisiting Missing Data Problem in Low-Rank Matrix," Int'l J. Computer Vision, vol. 80, no. 1, pp. 125-142, 2008.
[9] F. de la Torre and M. Black, "Robust Principal Component Analysis for Computer Vision," Proc. Int'l Conf. Computer Vision, vol. 1, pp. 362-369, 2001.
[10] L. Ding and A. Martinez, "Modelling and Recognition of the Linguistic Components in American Sign Language," Image and Vision Computing, vol. 27, no. 12, pp. 1826-1844, 2009.
[11] D. Forsyth and J. Ponce, Computer Vision: A Modern Approach. Prentice Hall, 2003.
[12] J. Fortuna and A.M. Martinez, "Rigid Structure from Motion from a Blind Source Separation Perspective," Int'l J. Computer Vision, 2010.
[13] G.H. Golub and C.F. Van Loan, Matrix Computations. Johns Hopkins Studies in Math. Sciences, The Johns Hopkins Univ. Press, 1996.
[14] N. Guilbert, A. Bartoli, and A. Heyden, "Affine Approximation for Direct Batch Recovery of Euclidian Structure and Motion from Sparse Data," Int'l J. Computer Vision, vol. 69, no. 3, pp. 317-333, 2006.
[15] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2003.
[16] R. Hartley and F. Schaffalizky, "Powerfactorization: 3D Reconstruction with Missing or Uncertain Data," Proc. Australia-Japan Advanced Workshop Computer Vision, 2003.
[17] D. Jacobs, "Linear Fitting with Missing Data for Structure-from-Motion," Computer Vision and Image Understanding, vol. 82, no. 1, pp. 57-81, 2001.
[18] A. Jain, "A Sinusoidal Family of Unitary Transforms," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 1, no. 4, pp. 356-365, Oct. 1979.
[19] H. Jia and A.M. Martinez, "Low-Rank Matrix Fitting Based on Subspace Perturbation Analysis with Applications to Structure from Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 841-854, May 2009.
[20] J. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics, second ed. Wiley, 1999.
[21] M. Marques and J. Costeira, "Estimating 3D Shape from Degenerate Sequences with Missing Data," Computer Vision and Image Understanding, vol. 113, no. 2, pp. 261-272, 2009.
[22] D. Martinec and T. Pajdla, "Structure from Many Perspective Images with Occlusions," Proc. European Conf. Computer Vision, pp. 355-369, 2002.
[23] S.I. Olsen and A. Bartoli, "Implicit Non-Rigid Structure-from-Motion with Priors," J. Math. Imaging Vision, vol. 31, nos. 2/3, pp. 233-244, 2008.
[24] M. Paladini, A. Del Bue, M. Stošić, M. Dodig, J. Xavier, and L. Agapito, "Factorization for Non-Rigid and Articulated Structure Using Metric Projections," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2898-2905, 2009.
[25] M. Pollefeys, F. Verbiest, and L. Van Gool, "Surviving Dominant Planes in Uncalibrated Structure and Motion Recovery," Proc. European Conf. Computer Vision, vol. 2, pp. 837-851, 2002.
[26] V. Rabaud and S. Belongie, "Rethinking Nonrigid Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 1-8, 2008.
[27] J. Shi and C. Tomasi, "Good Features to Track," Proc. IEEE Conf. Computer Vision and Pattern Recognition, no. 1, pp. 593-600, 1994.
[28] J.-P. Tardif, A. Bartoli, M. Trudeau, N. Guilbert, and S. Roy, "Algorithms for Batch Matrix Factorization with Application to Structure-from-Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, no. 1, pp. 1-8, 2007.
[29] C. Tomasi and T. Kanade, "Shape and Motion from Image Streams under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, Nov. 1992.
[30] L. Torresani, A. Hertzmann, and C. Bregler, "Nonrigid Structure-from-Motion: Estimating Shape and Motion with Hierarchical Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 878-892, May 2008.
[31] T. Wiberg, "Computation of Principal Components When Data Is Missing," Proc. Second Symp. Computational Statistics, pp. 229-236, 1976.
[32] J. Xiao, J. Chai, and T. Kanade, "A Closed Form Solution to Non-Rigid Shape and Motion Recovery," Int'l J. Computer Vision, vol. 67, no. 2, pp. 233-246, 2006.
[33] J. Xiao and T. Kanade, "Non-Rigid Shape and Motion Recovery: Degenerate Deformations," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 668-675, 2004.
[34] J. Yan and M. Pollefeys, "A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery from Video," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 865-877, May 2008.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool