The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2008 vol.30)
pp: 1300-1307
Yilei Xu , University of California, Riverside, Riverside
Amit Roy-Chowdhury , University of California, Riverside, Riverside
ABSTRACT
In this paper, we show how to estimate, accurately and efficiently, the 3D motion of a rigid object and time-varying lighting in a dynamic scene. This is achieved in an inverse compositional tracking framework with a novel warping function that involves a 2D → 3D → 2D transformation. This also allows us to extend traditional two frame inverse compositional tracking to a sequence of frames, leading to even higher computational savings. We prove the theoretical convergence of this method and show that it leads to significant reduction in computational burden. Experimental analysis on multiple video sequences shows impressive speed-up over existing methods while retaining a high level of accuracy.
INDEX TERMS
Motion, Video analysis
CITATION
Yilei Xu, Amit Roy-Chowdhury, "Inverse Compositional Estimation of 3D Pose And Lighting in Dynamic Scenes", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 7, pp. 1300-1307, July 2008, doi:10.1109/TPAMI.2008.81
REFERENCES
[1] G.D. Hager and P.N. Belhumeur, “Efficient Region Tracking with Parametric Models of Geometry and Illumination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp. 1025-1039, Oct. 1998.
[2] D. Freedman and M. Turek, “Illumination-Invariant Tracking via Graph Cuts,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[3] H. Jin, P. Favaro, and S. Soatto, “Real-Time Feature Tracking and Outlier Rejection with Changes in Illumination,” Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[4] Y. Xu and A. Roy-Chowdhury, “Integrating Motion, Illumination and Structure in Video Sequences, with Applications in Illumination-Invariant Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 793-807, May 2007.
[5] R. Basri and D.W. Jacobs, “Lambertian Reflectance and Linear Subspaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[6] R. Ramamoorthi and P. Hanrahan, “On the Relationship between Radiance and Irradiance: Determining the Illumination from Images of a Convex Lambertian Object,” J. Optical Soc. Am. A, vol. 18, no. 10, Oct. 2001.
[7] S. Baker and I. Matthews, “Lucas-Kanade 20 Years On: A Unifying Framework,” Int'l J. Computer Vision, vol. 56, no. 3, pp. 221-255, Mar. 2004.
[8] B.D. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision (DARPA),” Proc. DARPA Image Understanding Workshop, Apr. 1981.
[9] H.-Y. Shum and R. Szeliski, “Construction of Panoramic Image Mosaics with Global and Local Alignment,” Int'l J. Computer Vision, vol. 16, no. 1, pp. 63-84, 2000.
[10] I. Matthews and S. Baker, “Active Appearance Models Revisited,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 135-164, Nov. 2004.
[11] S. Romdhani and T. Vetter, “Efficient, Robust and Accurate Fitting of a 3D Morphable Model,” Proc. 10th IEEE Int'l Conf. Computer Vision, 2003.
[12] A. Bartoli, “Groupwise Geometric and Photometric Direct Image Registration,” Proc. Seventh British Machine Vision Conf., Sept. 2006.
[13] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[14] P. Eisert and B. Girod, “Illumination-Compensated Motion Estimation for Analysis Synthesis Coding,” 3D Image Analysis and Synthesis, pp. 61-66, 1996.
[15] M. Gouiffes, C. Collewet, C. Fernandez-Maloigne, and A. Trémeau, “Feature Points Tracking: Robustness to Specular Highlights and Lighting Changes,” Proc. Ninth European Conf. Computer Vision, May 2006.
[16] G. Finlayson, S. Hordley, and M. Drew, Removing Shadows from Images, 2002.
[17] D. Pizarro and A. Bartoli, “Shadow Resistant Direct Image Registration,” Proc. 15th Scandinavian Conf. Image Analysis, June 2007.
[18] J. Shi and C. Tomasi, “Good Features to Track,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994.
[19] A. Kale and C. Jaynes, “A Joint Illumination and Shape Model for Visual Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 602-609, 2006.
[20] L.D. Lathauwer, B.D. Moor, and J. Vandewalle, “A Multillinear Singular Value Decomposition,” SIAM J. Matrix Analysis and Applications, vol. 21, no. 4, pp. 1253-1278, 2000.
[21] R. Szeliski and S.B. Kang, “Recovering 3D Shape and Motion from Image Streams Using Non-Linear Least Squares,” J. Visual Comm. and Image Representation, vol. 5, no. 1, pp. 10-28, 1994.
[22] R. Ramamoorthi, “Analytic PCA Construction for Theoretical Analysis of Lighting Variability in Images of a Lambertian Object,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, 2002.
40 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool