| | This Article | |
| |
| |
| | Share | |
| |
| |
| | Bibliographic References | |
| |
| |
| | Add to: | |
| |
 Digg
 Furl
 Spurl
 Blink
 Simpy
 Google
 Del.icio.us
 Y!MyWeb
| |
| | Search | |
| |
| |
| | |
Alignment of Continuous Video onto 3D Point Clouds
August 2005 (vol. 27 no. 8)
pp. 1305-1318
We propose a general framework for aligning continuous (oblique) video onto 3D sensor data. We align a point cloud computed from the video onto the point cloud directly obtained from a 3D sensor. This is in contrast to existing techniques where the 2D images are aligned to a 3D model derived from the 3D sensor data. Using point clouds enables the alignment for scenes full of objects that are difficult to model; for example, trees. To compute 3D point clouds from video, motion stereo is used along with a state-of-the-art algorithm for camera pose estimation. Our experiments with real data demonstrate the advantages of the proposed registration algorithm for texturing models in large-scale semiurban environments. The capability to align video before a 3D model is built from the 3D sensor data offers new practical opportunities for 3D modeling. We introduce a novel modeling-through-registration approach that fuses 3D information from both the 3D sensor and the video. Initial experiments with real data illustrate the potential of the proposed approach.
[1] B.D.O. Anderson and J.B. Moore, Optimal Filtering. Prentice-Hall, 1979.
[2] P.J. Besl and N.D. McKay, “A Method for Registration of 3D Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, pp. 239-256, 1992.
[3] R.J. Campbell and P.J. Flynn, “Survey of Free-Form Object Representation and Recognition Techniques,” Computer Vision and Image Understanding, vol. 81, pp. 166-210, 2001.
[4] Y. Chen and G. Medioni, “Object Modeling by Registration of Multiple Range Images,” Image Vision Computing, vol. 10, pp. 145-155, 1992.
[5] C. Chen, Y. Hung, and J. Cheng, ”RANSAC-Based DARCES: A New Approach to Fast Automatic Registration of Partially Overlapping Range Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 11, Nov. 1999.
[6] P.E. Debevec, C.J. Taylor, and J. Malik, “Modeling and Rendering Architecture from Photograph: A Hybrid Geometry- and Image-Based Approach,” Proc. SIGGRAPH '96, pp. 11-20, 1996.
[7] O.D. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, 1993.
[8] M. Fischler and R. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Application to Image Analysis and Automated Cartography,” Comm. ACM, vol. 24, pp. 381-395, 1981.
[9] C. Frueh and A. Zakhor, “Constructing 3D City Models by Merging Ground-Based and Airborne Views,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2003.
[10] G. Godin, M. Rioux, and R. Baribeau, “Three-Dimensional Registration Using Range and Intensity Information,” Proc. SPIE Videometrics III, vol. 2350, pp. 279-290, 1994.
[11] S. Granger and X. Pennec, “Multi-Scale EM-ICP: A Fast and Robust Approach for Surface Registration,” Proc. European Conf. Computer Vision, 2002.
[12] C. Brenner, N. Haala, and D. Fritsch, “Towards Fully Automated 3D City Model Generation,” Proc. Workshop Automatic Extraction of Man-Made Objects from Aerial and Space Images III, 2001.
[13] N. Haala and C. Brenner, “Generation of 3D City Models from Airborne Laser Scanning Data,” Proc. EARSEL Workshop LIDAR Remote Sensing on Land and Sea, pp. 105-112, 1997.
[14] C. Harris and M. Stephens, “A Combined Corner and Edge Detector,” Proc. Fourth Alvey Vision Conf., pp. 147-151, 1998.
[15] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[16] M. Harville, A. Rahimi, T. Darrell, G. Gordon, and J. Woodfill, “3D Pose Tracking with Linear Depth and Brightness Constraints,” Proc. Int'l Conf. Computer Vision, 1999.
[17] B.K.P. Horn, “Closed-Form Solution of Absolute Orientation Using Unit Quaternions,” J. Optical Soc. Am. A, vol. 4, pp. 629-642, 1987.
[18] S. Hsu, S. Samarasekera, R. Kumar, and H. Sawhney, “Pose Estimation, Model Refinement, and Enhanced Visualization Using Video,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2000.
[19] M. Irani and P. Anandan, “Parallax Geometry of Pairs of Points for 3D Scene Analysis,” Proc. European Conf. Computer Vision, 1996.
[20] R. Kumar, P. Anandan, and K. Hanna, “Direct Recovery of Shape from Multiple Views: A Parallax Based Approach,” Proc. Int'l Conf. Pattern Recognition, 1994.
[21] D. Lowe, “Fitting Parameterized Three-Dimensional Models to Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, pp. 441-450, 1991.
[22] T. Masuda, K. Sakaue, and N. Yokoya, “Registration and Integration of Multiple Range Images for 3D Model Construction,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1996.
[23] L. Matthies, T. Kanade, and R. Szeliski, “Kalman Filter-Based Algorithms for Estimating Depth from Image Sequences,” Int'l J. Computer Vision, no. 3, pp. 209-236, 1989.
[24] D.M. Mount and S. Arya, “ANN: A Library for Approximate Nearest Neighbor Searching,” Proc. Center for Geometric Computing Second Ann. Workshop Computational Geometry, 1997.
[25] D. Nister, “An Efficient Solution to the Five-Point Relative Pose Problem,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, pp. 756-770, 2004.
[26] D. Nister, “Reconstruction from Uncalibrated Sequences with a Hiearchy of Trifocal Tensors,” Proc. European Conf. Computer Vision, 2000.
[27] U. Neumann et al., “Augmented Virtual Environments (AVE): Dynamic Fusion of Imagery and 3D Models,” Proc. Virtual Reality Conf., 2003.
[28] J. Rodriguez and J.K. Aggarwal, “Matching Aerial Images to 3D Terrain Maps,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, pp. 1138-1149, 1990.
[29] S. Rusinkiewicz and M. Levoy, “Efficient Variants of the ICP Algorithm,” Proc. Third Int'l Conf. 3D Digital Imaging and Modeling, 2001.
[30] H. Sawhney, A. Arpa, R. Kumar, S. Samarasekera, M. Aggarwal, H. Hsu, D. Nister, and K. Hanna, “Video Flashlights— Real Time Rendering of Multiple Videos for Immersive Model Visualization,” Proc. Eurographics Workshop Rendering Techniques, pp. 157-168, 2002.
[31] D. Simon, M. Hebert, and T. Kanade, “Real-Time 3-D Pose Estimation Using a High-Speed Range Sensor,” Proc. IEEE Int'l Conf. Robotics and Automation, vol. 3, pp. 2235-2241, 1994.
[32] V. Sequiera, K. Ng, E. Wolfart, J. Concalves, and D. Hogg, “Automated Reconstruction of 3D Models from Real Environments,” ISPRS J. Photogrammetry Remote Sensing, vol. 54, pp. 1-22, 1999.
[33] I. Stamos and P. Allen, “Geometry and Texture Recovery of Scenes of Large Scale,” Computer Vision and Image Understanding, vol. 88, pp. 94-118, 2002.
[34] I. Stamos and P. Allen, “Automatic Registration of 2-D with 3-D Imagery in Urban Environments,” Proc. Int'l Conf. Computer Vision, 2001.
[35] H. Tao, H. Sawhney, and R. Kumar, “A Global Matching Framework for Stereo Computation,” Proc. Int'l Conf. Computer Vision, 2001.
[36] T.Y. Tian, C. Tomas, and D. Heeger, “Comparison of Approaches to Egomotion Computation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1996.
[37] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, “Bundle Adjustment— A Modern Synthesis,” Proc. Vision Algorithms '99, B. Triggs, A. Zisserman, and R. Szeliski, eds., pp. 298-372, 2000.
[38] P. Viola and W. Wells, “Alignment by Maximization of Mutual Information,” Int'l J. Computer Vision, vol. 24, no. 2, pp. 137-154, 1997.
[39] Z. Zhang, “Iterative Point Matching for Registration of Free-Form Curves and Surfaces,” Int'l J. Computer Vision, vol. 13, pp. 119-152, 1994.
Index Terms:
Index Terms- Alignment, pose estimation, motion stereo, range data, sensor fusion, 3D model and visualization.
Citation:
Wenyi Zhao, David Nister, Steve Hsu, "Alignment of Continuous Video onto 3D Point Clouds," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1305-1318, Aug. 2005, doi:10.1109/TPAMI.2005.152
