The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2013 vol.35)
pp: 1674-1689
S. Taheri , Dept. of Electr. & Comput. Eng., Univ. of Maryland, College Park, MD, USA
A. C. Sankaranarayanan , Dept. of Electr. & Comput. Eng., Rice Univ., Houston, TX, USA
R. Chellappa , Dept. of Electr. & Comput. Eng., Univ. of Maryland, College Park, MD, USA
ABSTRACT
The albedo of a Lambertian object is a surface property that contributes to an object's appearance under changing illumination. As a signature independent of illumination, the albedo is useful for object recognition. Single image-based albedo estimation algorithms suffer due to shadows and non-Lambertian effects of the image. In this paper, we propose a sequential algorithm to estimate the albedo from a sequence of images of a known 3D object in varying poses and illumination conditions. We first show that by knowing/estimating the pose of the object at each frame of a sequence, the object's albedo can be efficiently estimated using a Kalman filter. We then extend this for the case of unknown pose by simultaneously tracking the pose as well as updating the albedo through a Rao-Blackwellized particle filter (RBPF). More specifically, the albedo is marginalized from the posterior distribution and estimated analytically using the Kalman filter, while the pose parameters are estimated using importance sampling and by minimizing the projection error of the face onto its spherical harmonic subspace, which results in an illumination-insensitive pose tracking algorithm. Illustrations and experiments are provided to validate the effectiveness of the approach using various synthetic and real sequences followed by applications to unconstrained, video-based face recognition.
INDEX TERMS
Face, Lighting, Estimation, Shape, Harmonic analysis, Kalman filters, Solid modeling,intrinsic image statistics, Albedo, pose tracking, spherical harmonics, sequential algorithm, Kalman filter, Rao-Blackwellized particle filter
CITATION
S. Taheri, A. C. Sankaranarayanan, R. Chellappa, "Joint Albedo Estimation and Pose Tracking from Video", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1674-1689, July 2013, doi:10.1109/TPAMI.2012.249
REFERENCES
[1] S. Biswas, G. Aggarwal, and R. Chellappa, "Robust Estimation of Albedo for Illumination-Invariant Matching and Shape Recovery," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 884-899, May 2009.
[2] R. Ramamoorthi, "Modeling Illumination Variation with Spherical Harmonics," Face Processing: Advanced Modeling Methods, 2006.
[3] B.K.P. Horn and M.J. Brooks, Shape from Shading. MIT Press, 1989.
[4] R. Zhang, P. sing Tsai, J.E. Cryer, and M. Shah, "Shape from Shading: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 690-706, Aug. 1999.
[5] L. Zhang and D. Samaras, "Face Recognition from a Single Training Image under Arbitrary Unknown Lighting Using Spherical Harmonics," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 351-363, Mar. 2006.
[6] S. Zhou, G. Aggarwal, R. Chellappa, and D. Jacobs, "Appearance Characterization of Linear Lambertian Objects, Generalized Photometric Stereo and Illumination-Invariant Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 2, pp. 230-245, Feb. 2007.
[7] A. Lakdawalla and A. Hertzmann, "Shape from Video: Dense Shape, Texture, Motion and Lighting from Monocular Image Streams," Proc. Photometric Analysis for Computer Vision, 2007.
[8] Y. Xu and A. Roy-Chowdhury, "Integrating Motion, Illumination, and Structure in Video with Applications in Illumination-Invariant Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 793-806, May 2007.
[9] R. Basri and D. Jacobs, "Lambertian Reflectance and Linear Subspaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[10] Z. Wen, Z. Liu, and T. Huang, "Face Relighting with Radiance Environment Maps," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, 2003.
[11] Y. Xu, A. Roy-Chowdhury, and K. Patel, "Pose and Illumination Invariant Face Recognition in Video," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[12] A. Barmpoutis, R. Kumar, B.C. Vemuri, and A. Banerjee, "Beyond the Lambertian Assumption: A Generative Model for ABRDF Fields of Faces Using Anti-Symmetric Tensor Splines," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[13] T. Weyrich, W. Matusik, H. Pfister, B. Bickel, C. Donner, C. Tu, J. Mcandless, J. Lee, A. Ngan, H. Wann, and J.M. Gross, "Analysis of Human Faces Using a Measurement-Based Skin Reflectance Model," ACM Trans. Graphics, vol. 25, no. 3, pp. 1013-1024, 2006.
[14] Y. Wang, Z. Liu, G. Hua, Z. Wen, Z. Zhang, and D. Samaras, "Face Re-Lighting from a Single Image under Harsh Lighting Conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[15] N. Joshi and D.J. Kriegman, "Shape from Varying Illumination and Viewpoint," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[16] C.H. Esteban, G. Vogiatzis, and R. Cipolla, "Multiview Photometric Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 3, pp. 548-554, Mar. 2008.
[17] J. Lim, J. Ho, M. hsuan Yang, and D. Kriegman, "Passive Photometric Stereo from Motion," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[18] L. Zhang, B. Curless, A. Hertzmann, and S.M. Seitz, "Shape and Motion under Varying Illumination: Unifying Structure from Motion, Photometric Stereo, and multiView Stereo," Proc. IEEE Int'l Conf. Comp. Vision, 2003.
[19] D. Simakov, D. Frolova, and R. Basri, "Dense Shape Reconstruction of a Moving Object under Arbitrary, Unknown Lighting," Proc. IEEE Int'l Conf. Comp. Vision, pp. 1202-1209, 2003.
[20] R.J. Woodham, "Photometric Method for Determining Surface Orientation from Multiple Images," Optical Eng., vol. 19, no. 1, pp. 139-144, 1980.
[21] K. Ikeuchi, "Determining Surface Orientation of Specular Surfaces by Using the Photometric Stereo Method," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 6, pp. 661-669, Nov. 1981.
[22] W.Y. Zhao and R. Chellappa, "Symmetric Shape-from-Shading Using Self-Ratio Image," Int'l J. Computer Vision, vol. 45, no. 1, pp. 55-75, 2001.
[23] J. Atick, P. Griffin, and A. Redlich, "Statistical Approach to SFS: Reconstruction of 3D Face Surfaces from Single 2D Images," Neural Computation, vol. 8, pp. 1321-1340, 1996.
[24] R. Dovgard and R. Basri, "Statistical Symmetric Shape from Shading for 3D Structure Recovery of Faces," Proc. European Conf. Computer Vision, 2004.
[25] W.A.P. Smith and E.R. Hancock, "Recovering Facial Shape Using a Statistical Model of Surface Normal Direction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 1914-1930, Dec. 2006.
[26] B.K.P. Horn, "Determining Lightness from an Image," Computer Graphics and Image Processing, vol. 3, pp. 277-299, 1974.
[27] E.H. Land and J.J. McCann, "Lightness and Retinex Theory," J. Optical Soc. Am., vol. 61, no. 1, pp. 1-11, 1971.
[28] M.F. Tappen, E.H. Adelson, and W.T. Freeman, "Estimating Intrinsic Component Images Using nonLinear Regression," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[29] V. Blanz and T. Vetter, "Face Recognition Based on Fitting a 3D Morphable Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, pp. 1063-1074, Sept. 2003.
[30] Y. Wang, L. Zhang, Z. Liu, G. Hua, Z. Wen, Z. Zhang, and D. Samaras, "Face Relighting from a Single Image under Arbitrary Unknown Lighting Conditions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 11, pp. 1968-1984, Nov. 2009.
[31] S. Biswas and R. Chellappa, "Pose-Robust Albedo Estimation from a Single Image," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2683-2690, 2010.
[32] T. Yu, N. Xu, and N. Ahuja, "Recovering Shape and Reflectance Model of Non-Lambertian Objects from Multiple Views," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[33] G. Aggarwal, A. Veeraraghavan, and R. Chellappa, "3D Facial Pose Tracking in Uncalibrated Videos," Proc. First Int'l Conf. Pattern Recognition and Machine Intelligence, 2005.
[34] S. Ba and J. Odobez, "A Probabilistic Head Pose Tracking Evaluation in Single and Multiple Camera Setups," Proc. Classification of Events, Activities and Relationship Evaluation and Workshop, 2007.
[35] M.L. Cascia, S. Sclaroff, and V. Athitsos, "Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 4, pp. 322-336, Apr. 2000.
[36] T.K. Marks, J.R. Hershey, and J.R. Movellan, "Tracking Motion, Deformation, and Texture Using Conditionally Gaussian Processes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 2, pp. 348-363, Feb. 2010.
[37] E. Murphy-Chutorian and M. Trivedi, "Head Pose Estimation in Computer Vision: A Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 4, pp. 607-626, Apr. 2008.
[38] Y. Tanabe, T. Inui, and Y. Onodera, Group Theory and Its Applications in Physics. Springer, 1990.
[39] Z. Yue, W. Zhao, and R. Chellappa, "Pose-Encoded Spherical Harmonics for Face Recognition and Synthesis Using a Single Image," EURASIP J. Advances in Signal Processing, vol. 2008, 2008.
[40] V. Blanz and T. Vetter, "A Morphable Model for the Synthesis of 3D Faces," Proc. ACM Siggraph, 1999.
[41] P. Paysan, R. Knothe, B. Amberg, S. Romdhani, and T. Vetter, "A 3D Face Model for Pose and Illumination Invariant Face Recognition," Proc. Sixth IEEE Int'l Conf. Advanced Video and Signal Based Surveillance, 2009.
[42] D.F. DeMenthon and L.S. Davis, "Model-Based Object Pose in 25 Lines of Code," Int'l J. Computer Vision, vol. 15, nos. 1/2, pp. 123-141, 1995.
[43] R.B. Altman, "A Probabilistic Algorithm for Calculating Structure: Borrowing from Simulated Annealing," technical report, Stanford Univ., 1990.
[44] R.B. Altman and J.F. Brinkley, "Probabilistic Constraint Satisfaction with Structural Models," Proc. Symp. Computer Applications in Medical Care, 1993.
[45] P.S. Maybeck, Stochastic Models Estimation and Control. Academic Press, 1979.
[46] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[47] S. Ba and J. Odobez, "A Rao-Blackwellized Mixed State Particle Filter for Head Pose Tracking in Meetings," Proc. ACM ICMI Workshop Multimodal Multiparty Meeting Processing, 2005.
[48] A.C. Sankaranarayanan, A. Srivastava, and R. Chellappa, "Algorithmic and Architectural Optimizations for Computationally Efficient Particle Filtering," IEEE Trans. Image Processing, vol. 17, no. 5, pp. 737-748, May 2008.
[49] J. Shi and C. Tomasi, "Good Features to Track," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994.
[50] B. Anderson and J. Moore, Optimal Filtering. Prentice Hall, 1979.
[51] S. Hong, S.S. Chin, P.M. Djurić, and M. Bolić, "Design and Implementation of Flexible Resampling Mechanism for High-Speed Parallel Particle Filters," J. VLSI Signal Processing, vol. 44, nos. 1/2, pp. 47-62, 2006.
105 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool