This Article 
 Bibliographic References 
 Add to: 
Multi-Primitive Hierarchical (MPH) Stereo Analysis
March 1994 (vol. 16 no. 3)
pp. 227-240

This paper develops and demonstrates a new computational framework for an accurate, robust, and efficient stereo approach. In multi-primitive hierarchical (MPH) computational model, stereo analysis is performed in multiple stages, incorporating multiple primitives, utilizing a hierarchical control strategy. The MPH stereo system consists of three integrated subsystems: region-based analysis module; linear edge segment-based analysis module; and edgel-based stereo analysis module. Results of stereo analysis at higher levels of the hierarchy are used for guidance at the lower levels. The MPH stereo system does not overly rely on one type of primitive and therefore will reliably work on a wide range of scenes. The MPH stereo analysis results in the generation of several disparity maps of multiple abstraction. Disparity maps generated at each level can be fused to obtain an accurate and fine resolution disparity map. The MPH approach also provides the capability to selectively analyze image regions with varying detail. This provides the means for adaptively extracting range information of only sufficient resolution. Thus, a stereo system that utilizes primitives of different abstraction and a multilevel hierarchical computational strategy will be superior to a single-level, single-primitive system. Extensive experimentation is carried out on a wide array of scenes of varying complexity from two application domains to systematically evaluate the validity and performance of the MPH framework. The MPH stereo system is able to analyze images in most cases with 85%/spl sim/100% matching accuracy in under a minute of processing time and yield depth values typically within /spl plusmn/2% of the actual depth.

[1] J. P. Frisby,Seeing. Oxford, UK: Oxford University, 1979.
[2] W. Richards, "Stereopsis with and without monocular contours,"Vision Research, vol. 17, pp. 967-969, 1977.
[3] H. S. Lim and T. O. Binford, "Stereo correspondence: A hierarchical approach," inProc. Image Understanding Workshop, pp. 234-241, Feb. 1987.
[4] H. S. Lim and T. O. Binford, "Structural correspondence in stereo vision," inProc. Image Understanding Workshop, vol. 2, pp. 794-808, Apr. 1988.
[5] D. B. Gennery, "Stereo-camera calibration," inProc. ARPA Image Understanding Workshop, pp. 101-107, Nov. 1979.
[6] K. L. Boyer, D. M. Wuescher, and S. Sarkar, "Dynamic edge warping: An experimental system for recovering disparity maps in weakly constrained systems,"IEEE Trans. Syst.. Man, Cybernetics, vol. 21, pp. 143-158, Jan./Feb. 1991.
[7] S. T. Barnard and M. A. Fischler, "Computational stereo,"Comput. Surveys, vol. 14, no. 4, pp. 553-572, 1982.
[8] D. Marr and T. Poggio, "A computational theory of human stereo vision,"Proc. Royal Soc. London, vol. B 204, pp. 301-328, 1979.
[9] B. Julesz, "Binocular depth perception of computer-generated patterns,"Bell Syst. Tech. J., vol. 39, pp. 1125-1162, 1960.
[10] D. B. Gennery, "A stereo vision system for an autonomous vehicle," inProc. 5th Int. Joint Conf. Artificial Intelligence, vol. 2, pp. 576-582, Aug. 1977.
[11] H. P. Moravec, "The Stanford cart and the CMU rover,"Proc. IEEE, vol. 71, pp. 872-884, July 1983.
[12] H. K. Nishihara, "Practical real-time imaging stereo matcher,"Opt. Eng., vol. 23, pp. 536-545, Sept. 1984.
[13] W. E. L. Grimson, "Computational experiments with a feature based stereo algorithm,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-7, pp. 17-34, Jan. 1985.
[14] Y. C. Kim and J. K. Aggarwal, "Positioning three dimensional objects using stereo images,"IEEE J. Robotics Automat., vol. RA-3, no. 6, pp. 361-373, Aug. 1987.
[15] R. D. Arnold, "Automated stereo perception," Ph.D. thesis, Stanford Univ., Stanford, CA, Mar. 1983; Tech. Rep. AIM-351 and STAN-CS- 83-961.
[16] H. H. Baker and T. O. Binford, "Depth from edge and intensity based stereo," inProc. 7th Int. Joint Conf. AI, pp. 631-636, 1981.
[17] N. Ayache and B. Faverjon, "Efficient registration of stereo images by matching graph descriptions of edge segments,"Int. J. Comput. Vision, pp. 107-131, 1987.
[18] R. Horaud and T. Skordas, "Stereo correspondence through feature grouping and maximal cliques,"IEEE Trans. Pattern Anal. Machine Intell., vol. 11, pp. 1168-1180, Nov. 1989.
[19] G. Medioni and R. Nevatia, "Segment-based stereo matching,"Comput. Vision, Graphics, and Image Processing, vol. 31, pp. 2-18, July 1985.
[20] D. Sherman and S. Peleg, "Stereo by incremental matching of contours,"IEEE Trans. Pattern Anal. Machine Intell., vol. 12, pp. 1102-1106, 1990.
[21] R. C. K. Chung and R. Nevatia, "Use of monocular groupings and occlusion analysis in a hierarchical stereo system," inProc. IEEE Conf. Comput. Vision Pattern Recognition, Maui, HI, 1991, pp. 50-56.
[22] W. Hoff and N. Ahuja, "Surface from stereo: Integrating feature matching, disparity estimation, and contour detection,"IEEE Trans. Pattern Anal. Machine Intell., vol. 11, pp. 121-136, Feb. 1989.
[23] W. E. L. Grimson,From Images to Surfaces: A Computational Study of the Human Early visual System. Cambridge, MA: MIT Press, 1981.
[24] R. Mohan, G. Medioni, and R. Nevatia, "Stereo error detection, correction, and evaluation,"IEEE Trans. Pattern Anal. Machine Intell., vol. 11, pp. 113-120, Feb. 1989.
[25] J. E. W. Mayhew and J. P. Frisby, "Psychophysical and computational studies towards a theory of human stereopsis,"Artificial Intell., vol. 17, pp. 349-385, 1981.
[26] Y. Ohta and T. Kanade, "Stereo by intra- and inter-scanline search using dynamic programming,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-7, pp. 139-154, Mar. 1985.
[27] P. Burt and B. Julesz, "A disparity gradient limit for binocular fusion,"Science, vol. 208, pp. 615-617, May 1980.
[28] D. Terzopoulos, "The computation of visible-surface representations,"IEEE Trans. Pattern Anal. Machine Intell., vol. 10, pp. 417-437, July 1988.
[29] D. Marr and T. Poggio, "Cooperative computational of stereo disparity,"Science, vol. 194, pp. 283-287, 1976.
[30] S. T. Barnard and W. B. Thompson, "Disparity analysis of images,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-2, pp. 333-340, July 1980.
[31] H. P. Moravec, "Towards automatic visual obstacle avoidance," inProc. 5th Int. Joint Conf. Artificial Intelligence, vol. 2, Aug. 1977.
[32] S. B. Marapane and M. M. Trivedi, "Region-based stereo analysis for robotic applications,"IEEE Trans. Syst., Man, Cybernetics, vol. 19, pp. 1447-1464, Nov./Dec. 1989. Special issue on computer vision.
[33] K. L. Boyer and A. C. Kak, "Structural stereopsis for 3-D vision,"IEEE Trans. Pattern Anal. Machine Intell., vol. 10, pp. 144-166, Mar. 1988.
[34] T. Pang, R. M. Haralick, and L. G. Shapiro, "Matching topographic structures in stereo vision,"Pattern Recognition Lett., vol. 9, pp. 127-136, Apr. 1989.
[35] S. D. Cochran and G. Medioni, "Accurate surface description from binocular stereo, inProc. DARPA Image Understanding Workshop(Palo Alto, CA), May 1989, pp. 857-869.
[36] U. R. Dhond and J. K. Aggarwal, "Structure from stereo--A review,"IEEE Trans. Syst. Man Cybern., vol. 19, no. 6, pp. 1489-1510, Nov. 1989.
[37] H. K. Nishihara and T. Poggio, "Stereo vision for robotics," inFirst Int. Symp. Robotics Research. New York: IEEE Press, 1983, pp. 489-505.
[38] S. B. Marapane, "Computational framework for multi-primitive hierarchical stereo analysis," Ph.D. dissertation, Dept. of Electrical and Comput. Eng., University of Tennessee, Knoxville, TN, 1991.
[39] N. M. Vaidya and K. L. Boyer, "Stereopsis and image registration from extended edge features in the absence of camera pose information," inProc. IEEE Conf. Comput. Vision Pattern Recognition, Lahaina, HI, 1991, pp. 76-82.
[40] M. Watanabe and Y. Ohta, "Cooperative integration of multiple stereo algorithms," inProc. 3rd Int. Conf. Computer Vision, Osaka, Japan, 1990, pp. 476-480.
[41] S. B. Marapane and M. M. Trivedi, "Extracting depth by binocular stereo in a robot vision system," inProc. Conf. Applications Digital Image Processing XI, SPIE, 1988.
[42] L. Vinet, P. T. Sander, L. Cohen, and A. Gagalowick, "Hierarchical region based stereo matching," inProc. Comput. Vision Pattern Recognition Conf., 1989.
[43] G. Xu, H. Kondo, and S. Tsuji, "A region-based stereo algorithm," inProc. 11th Int. Joint Conf. Artificial Intelligence, vol. 2, 1989.
[44] S. Randriamasy and A. Gagalowicz, "Region based stereo matching oriented image processing," inProc. IEEE Conf. Comput. Vision Pattern Recognition, 1991, pp. 736-737.
[45] M. M. Trivedi and C. Chen, "Sensor-driven intelligent robotics," inAdvances in Computers, vol. 32, M. Yovits, Ed. New York: Academic, 1991, pp. 105-148.
[46] S. B. Marapane and M. M. Trivedi, "An active vision system for depth extraction using multi-primitive hierarchical stereo analysis and multiple depth cues," inProc. Sensor Fusion Aerospace Applications Conf., SPIE, Apr. 1993.

Index Terms:
stereo image processing; image segmentation; edge detection; hierarchical systems; multi-primitive hierarchical stereo analysis; hierarchical control strategy; region-based analysis module; linear edge segment-based analysis module; edgel-based stereo analysis module; disparity maps; multiple abstraction; image regions
S.B. Marapane, M.M. Trivedi, "Multi-Primitive Hierarchical (MPH) Stereo Analysis," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 3, pp. 227-240, March 1994, doi:10.1109/34.276122
Usage of this product signifies your acceptance of the Terms of Use.