This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Framework for 3D Model-Based Visual Tracking Using a GPU-Accelerated Particle Filter
January 2012 (vol. 18 no. 1)
pp. 68-80
J. Anthony Brown, McMaster University, Hamilton
David W. Capson, McMaster University, Hamilton
A novel framework for acceleration of particle filtering approaches to 3D model-based, markerless visual tracking in monocular video is described. Specifically, we present a methodology for partitioning and mapping the computationally expensive weight-update stage of a particle filter to a graphics processing unit (GPU) to achieve particle- and pixel-level parallelism. Nvidia CUDA and Direct3D are employed to harness the massively parallel computational power of modern GPUs for simulation (3D model rendering) and evaluation (segmentation, feature extraction, and weight calculation) of hundreds of particles at high speeds. The proposed framework addresses the computational intensity that is intrinsic to all particle filter approaches, including those that have been modified to minimize the number of particles required for a particular task. Performance and tracking quality results for rigid object and articulated hand tracking experiments demonstrate markerless, model-based visual tracking on consumer-grade graphics hardware with pixel-level accuracy up to 95 percent at 60+ frames per second. The framework accelerates particle evaluation up to 49 times over a comparable CPU-only implementation, providing an increased particle count while maintaining real-time frame rates.

[1] V. Lepetit and P. Fua, “Monocular Model-Based 3D Tracking of Rigid Objects: A Survey,” Foundations and Trends in Computer Graphics and Vision, vol. 1, pp. 1-89, 2005.
[2] J.V. Candy, Bayesian Signal Processing: Classical, Modern and Particle Filtering Methods. John Wiley and Sons, Inc., 2009.
[3] B. Ristic, S. Arulampalam, and N. Gordon, Beyond the Kalman Filter: Particle Filters for Tracking Applications. Artech House, 2004.
[4] Sequential Monte Carlo Methods in Practice, A. Doucet, N. De Freitas, and N. Gordon, eds. Springer, 2001.
[5] H. Tamimi, H. Andreasson, A. Treptow, T. Duckett, and A. Zell, “Localization of Mobile Robots with Omnidirectional Vision Using Particle Filter and Iterative SIFT,” Proc. Robotics and Autonomous Systems, vol. 54, no. 9, pp. 758-765, 2006.
[6] S. Thrun, W. Burgard, and D. Fox, Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, Sept. 2005.
[7] D. Fox, W. Burgard, F. Dellaert, and S. Thrun, “Monte Carlo Localization: Efficient Position Estimation for Mobile Robots,” Proc. the Nat'l Conf. Artificial Intelligence (AAAI '99), pp. 343-349, 1999.
[8] F. Gustafsson, F. Gunnarsson, N. Bergman, U. Forssell, J. Jansson, R. Karlsson, and P.J. Nordlund, “Particle Filters for Positioning, Navigation, and Tracking,” IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 425-437, Feb. 2002.
[9] M. Isard and A. Blake, “CONDENSATION - Conditional Density Propagation for Visual Tracking,” Int'l J. Computer Vision, vol. 29, pp. 5-28, 1998.
[10] S.K. Zhou, R. Chellappa, and B. Moghaddam, “Visual Tracking and Recognition Using Appearance-Adaptive Models in Particle Filters,” IEEE Trans. Image Processing, vol. 13, no. 11, pp. 1491-1506, Nov. 2004.
[11] W.-Y. Chang, C.-S. Chen, and Y.-D. Jian, “Visual Tracking in High-Dimensional State Space by Appearance-guided Particle Filtering,” IEEE Trans. Image Processing, vol. 17, no. 7, pp. 1154-1167, July 2008.
[12] M. Bray, E. Koller-Meier, and L. Van Gool, “Smart Particle Filtering for High-Dimensional Tracking,” Computer Vision and Image Understanding, vol. 106, no. 1, pp. 116-129, 2007.
[13] Y. Wu, J. Lin, and T. Huang, “Analyzing and Capturing Articulated Hand Motion in Image Sequences,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1910-1922, Dec. 2005.
[14] J. Cui and Z. Sun, “Visual Hand Motion Capture for Guiding a Dexterous Hand,” Proc. Sixth IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 729-734, May 2004.
[15] J. Fung and S. Mann, “Using Graphics Devices in Reverse: GPU-Based Image Processing and Computer Vision,” Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 9-12, 2008.
[16] W.M. Hwu, C. Rodrigues, S. Ryoo, and J. Stratton, “Compute Unified Device Architecture Application Suitability,” Computing in Science and Eng., vol. 11, no. 3, pp. 16-26, 2009.
[17] A. Coates, P. Baumstarck, Q. Le, and A. Ng, “Scalable Learning for Object Detection with GPU Hardware,” Proc. IEEE/RSJ Int'l Conf. Intelligent Robots and Systems (IROS '09), pp. 4287-4293, Oct. 2009.
[18] J.P. Walters, V. Balu, S. Kompalli, and V. Chaudhary, “Evaluating the Use of GPUs in Liver Image Segmentation and HMMER Database Searches,” Proc. IEEE Int'l Symp. Parallel and Distributed Processing, pp. 1-12, 2009.
[19] O. Kutter, R. Shams, and N. Navab, “Visualization and GPU-Accelerated Simulation of Medical Ultrasound from CT Images,” Computer Methods and Programs in Biomedicine, vol. 94, pp. 250-266, June 2009.
[20] V.W. Lee, C. Kim, J. Chhugani, M. Deisher, D. Kim, A.D. Nguyen, N. Satish, M. Smelyanskiy, S. Chennupaty, P. Hammarlund, R. Singhal, and P. Dubey, “Debunking the 100x GPU versus CPU Myth: An Evaluation of Throughput Computing on CPU and GPU,” SIGARCH Computer Architecture News, vol. 38, no. 3, pp. 451-460, 2010.
[21] S. Ryoo, C.I. Rodrigues, S.S. Baghsorkhi, S.S. Stone, D.B. Kirk, and W.-m.W. Hwu, “Optimization Principles and Application Performance Evaluation of a Multithreaded GPU Using CUDA,” Proc. 13th ACM SIGPLAN Symp. Principles and Practice of Parallel Programming, pp. 73-82, 2008.
[22] T. Schoenemann and D. Cremers, “A Combinatorial Solution for Model-Based Image Segmentation and Real-Time Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 7, pp. 1153-1164, July 2010.
[23] M. Breitenstein, D. Kuettel, T. Weise, L. Van Gool, and H. Pfister, “Real-Time Face Pose Estimation from Single Range Images,” CVPR '08: Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[24] S. Lee, G.J. Kim, and S. Choi, “Real-Time Tracking of Visually Attended Objects in Virtual Environments and Its Application to LOD,” IEEE Trans. Visual Computer Graphics, vol. 15, no. 1, pp. 6-19, Jan./Feb. 2009.
[25] J. Ferreira, J. Lobo, and J. Dias, “Bayesian Real-Time Perception Algorithms on GPU,” J. Real-Time Image Processing, Feb. 2010.
[26] O. Mateo Lozano and K. Otsuka, “Real-Time Visual Tracker by Stream Processing,” J. Signal Processing Systems, vol. 57, pp. 285-295, 2009, doi: 10.1007/s11265-008-0250-2.
[27] K.-Y. Liu, L. Tang, S.-Q. Li, L. Wang, and W. Liu, “Parallel Particle Filter Algorithm in Face Tracking,” ICME '09: Proc. IEEE Int'l Conf. Multimedia and Expo, pp. 1813-1816, 2009.
[28] R. Cabido, D. Concha, J.J. Pantrigo, and A.S. Montemayor, “High Speed Articulated Object Tracking Using GPUs: A Particle Filter Approach,” ISPAN '09: Proc. 10th Int'l Symp. Pervasive Systems, Algorithms, and Networks, pp. 757-762, 2009.
[29] R. Cabido, A. Montemayor, J. Pantrigo, and B. Payne, “Multiscale and Local Search Methods for Real Time Region Tracking with Particle Filters: Local Search Driven by Adaptive Scale Estimation on GPUs,” Machine Vision and Applications, vol. 21, pp. 43-58, 2009.
[30] Z. Pezzementi, S. Voros, and G.D. Hager, “Articulated Object Tracking by Rendering Consistent Appearance Parts,” ICRA '09: Proc. Int'l Conf. Robotics and Automation, pp. 1225-1232, 2009.
[31] I. Park, M. Germann, M. Breitenstein, and H. Pfister, “Fast and Automatic Object Pose Estimation for Range Images on the GPU,” Machine Vision and Applications, vol. 21, pp. 749-766, 2010.
[32] G. Klein and D. Murray, “Full-3D Edge Tracking with a Particle Filter,” Proc. British Machine Vision Conf. (BMVC '06), vol. 3, pp. 1119-1128, Sept. 2006.
[33] M. Pupilli and A. Calway, “Real-Time Camera Tracking Using a Particle Filter,” Proc. British Machine Vision Conf. (BMVC '05), Sept. 2005.
[34] E. Murphy-Chutorian and M. Trivedi, “Particle Filtering with Rendered Models: A Two Pass Approach to Multi-Object 3D Tracking with the GPU,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition Workshops (CVPRW '08), pp. 1-8, June 2008.
[35] C. Lenz, G. Panin, and A. Knoll, “A GPU-Accelerated Particle Filter with Pixel-Level Likelihood,” VMV '08: Proc. Int'l Workshop Vision, Modeling and Visualization, Oct. 2008.
[36] D.B. Kirk, W. Mei, and W. Hwu, Programming Massively Parallel Processors: A Hands-on Approach. Morgan Kaufmann Publishers, 2010.
[37] J.A. Brown and D.W. Capson, “GPU-Accelerated 3-D Model-Based Tracking,” J. Physics: Conf. Series Proc. the High Performance Computing Symp. (HPCS '10), vol. 256, p. 012004, June 2010.
[38] A. Sherrod, Ultimate Game Programming with DirectX. Charles River Media, 2006.
[39] N. Gordon, D. Salmond, and A. Smith, “Novel Approach to Nonlinear/Non-Gaussian Bayesian State Estimation,” IEE Proc. F Radar and Signal Processing, vol. 140, no. 2, pp. 107-113, Apr. 1993.
[40] M.S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A Tutorial on Particle Filters for Online Nonlinear/Non-Gaussian Bayesian Tracking,” IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 174-188, Feb. 2002.
[41] A.C. Sankaranarayanan, A. Srivastava, and R. Chellappa, “Algorithmic and Architectural Optimizations for Computationally Efficient Particle Filtering,” IEEE Trans. Image Processing, vol. 17, no. 5, pp. 737-748, May 2008.
[42] E. Maggio, F. Smerladi, and A. Cavallaro, “Adaptive Multifeature Tracking in Aparticle Filtering Framework,” IEEE Trans. Circuits and Systems for Video Technology, vol. 17, no. 10, pp. 1348-1359, Oct. 2007.
[43] H. Wang, D. Suter, K. Schindler, and C. Shen, “Adaptive Object Tracking Based on an Effective Appearance Filter,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1661-1667, Sept. 2007.
[44] M. Harris, “Optimizing Parallel Reduction in Cuda,” technical report, NVIDIA Corporation, 2007.
[45] K. Smith, D. Gatica-Perez, J. Odobez, and S. Ba, “Evaluating Multi-Object Tracking,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition - Workshops, pp. 36-36, June 2005.
[46] F. Bashir and F. Porikli, “Performance Evaluation of Object Detection and Tracking Systems,” Proc. IEEE Int'l Workshop Performance Evaluation of Tracking and Surveillance (PETS '06), June 2006.
[47] T. Roosendaal and S. Selleri, The Official Blender 2.3 Guide. Springer, 2009.
[48] B.C. Brandao, J. Wainer, and S.K. Goldenstein, “Subspace Hierarchical Particle Filter,” Proc. 19th Brazilian Symp. Computer Graphics and Image Processing, pp. 194-204, 2006.
[49] B. Stenger, A. Thayananthan, P.H. Torr, and R. Cipolla, “Model-Based Hand Tracking Using a Hierarchical Bayesian Filter,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1372-1384, Sept. 2006.

Index Terms:
Edge and feature detection, graphics processors, Monte Carlo simulation, tracking, vision I/O.
Citation:
J. Anthony Brown, David W. Capson, "A Framework for 3D Model-Based Visual Tracking Using a GPU-Accelerated Particle Filter," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 1, pp. 68-80, Jan. 2012, doi:10.1109/TVCG.2011.34
Usage of this product signifies your acceptance of the Terms of Use.