This Article 
 Bibliographic References 
 Add to: 
Image Interpretation Using Multiple Sensing Modalities
August 1992 (vol. 14 no. 8)
pp. 840-847

The AIMS (automatic interpretation using multiple sensors) system, which uses registered laser radar and thermal imagers, is discussed. Its objective is to detect and recognize man-made objects at kilometer range in outdoor scenes. The multisensor fusion approach is applied to four sensing modalities (range, intensity, velocity, and thermal) to improve both image segmentation and interpretation. Low-level attributes of image segments (regions) are computed by the segmentation modules and then converted to the KEE format. The knowledge-based interpretation modules are constructed using KEE and Lisp. AIMS applies forward chaining in a bottom-up fashion to derive object-level interpretations from databases generated by the low-level processing modules. The efficiency of the interpretaton process is enhanced by transferring nonsymbolic processing tasks to a concurrent service manager (program). A parallel implementation of the interpretation module is reported. Experimental results using real data are presented.

[1] C. G. Bachman,Laser Radar Systems and Techniques. Dedham, MA: Artech House, 1979.
[2] C. Chu and J. K. Aggarwal, "Interpretation of laser radar images by a knowledge-based system,"J. Machine Vision Applications, no. 4, pp. 145-163, 1991.
[3] D. M. McKeown, Jr., W. A. Harvey, and J. McDermott, "Rule-based interpretation of aerial imagery,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-7, no. 5, pp. 570-585, Sept. 1985.
[4] J. A. Mulder, A. K. Mackworth, and W. S. Havens, "Knowledge structuring and constraint satisfaction: the Mapsee approach,"IEEE Trans. Patt. Anal. Machine Intell., vol. 10, no. 9, pp. 866-879, Nov. 1988.
[5] Y. Ohta,Knowledge-Based Interpretation of Outdoor Natural Color Scenes. Boston: Pitman, 1985.
[6] B. A. Draperet al., "Tools and experiments in the knowledge-directed interpretation of road scenes," inProc. DARPA Image Understanding Workshop, 1987, pp. 179-193.
[7] A. K. Jain and R. Hoffman, "Evidence-based recognition of 3-D objects,"IEEE Trans. Patt. Anal. Machine Intell., vol. 10, no. 6, pp. 783-802, Nov. 1988.
[8] T. J. Fan, G. Medioni, and R. Nevatia, "3D object recognition using surface descriptions," inProc. DARPA Image Understanding Workshop, 1988, pp. 383-397.
[9] D. E. Dudgeon, J. G. Verly, and R. L. Delanoy, "An experimental target recognition system for laser radar imagery," inProc. DARPA Image Understanding Workshop, 1989, pp. 479-506.
[10] N. Nandhakumar and J. K. Aggarwal, "Integrated analysis of thermal and visual images for scene interpretation,"IEEE Trans. Patt. Anal. Machine Intell., vol. 10, no. 4, pp. 469-481, July 1988.
[11] B. Bhanu and R. D. Holben, "Model-based segmentation of FLIR images,"IEEE Trans. Aerospace Electron. Syst., vol. 26, no. 1, pp. 2-11, Jan. 1990.
[12] C. W. Tong, S. K. Rogers, J. P. Mills, and M. K. Kabrisky, "Multisensor data fusion of laser radar and forward looking infrared (FLIR) for target segmentation and enhancement," inProc. SPIE, May 1987, vol. 782, pp. 10-19.
[13] C. Chu and J. K. Aggarwal, "The integration of region and edge-based segmentation," inProc. Third Int. Conf. Comput. Vision(Osaka, Japan), Dec. 4-7, 1990, pp. 117-120.
[14] C. Chu, N. Nandhakumar, and J. K. Aggarwal, "Image segmentation using laser radar data,"Patt. Recogn., vol. 23, no. 6, pp. 569-581, 1990.
[15] C. Chu and A. C. Bovik, "Visual surface reconstruction using minimax approximation,"Patt. Recogn., vol. 21, no. 4, pp. 303-312, 1988.
[16] F. P. Incropera and D. P. De Witt,Fundamentals of Heat Transfer. New York: Wiley, 1981.
[17] J. Burns, A. Hanson, and E. Riseman, "Extracting straight lines,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-8, no. 4, pp. 425-455, July 1986.
[18] R. Belknap, E. Riseman, and A. Hanson, "The information fusion problem and rule-based hypotheses applied to complex aggregations of image events," inProc. DARPA Image Understanding Workshop, 1985, pp. 279-292.
[19] T. Pavlidis and Y. T. Liow, "Integrating region growing and edge detection,"IEEE Trans. Patt. Anal. Machine Intell., vol. 12, no. 3, pp. 225-233, Mar. 1990.
[20] J. F. Haddon and J. F. Boyce, "Image segmentation by unifying region and boundary information,"IEEE Trans. Patt. Anal. Machine Intell., vol. 12, no. 3, pp. 929-948, Oct. 1990.
[21] W. Clancey, "Heuristic Classification,"Artificial Intelligence, Vol. 27, 1985, pp. 289-350.
[22] C. Chu and J. K. Aggarwal, "Multisensor image interpretation using laser radar and thermal images," inProc. Seventh Conf. Artificial Intell. Applications, (Miami Beach, FL), Feb. 24-28, 1991, pp. 190-196.
[23] A. Gupta, C. L. Forgy, D. Kalp, A. Newell, and M. Tambe, "Parallel OPS5 on the encore multimax," inProc. Int. Conf. Parallel Processing, 1988, pp. 271-280.
[24] D. E. Shaw, "On the range of applicability of an artificial intelligence machine," inParallel Computation and Computers for Artificial Intelligence(J. S. Kowlick, Ed.). Boston: Kluwer, 1987.

Index Terms:
image interpretation; multiple sensing modalities; AIMS; laser radar; thermal imagers; multisensor fusion; image segmentation; KEE format; knowledge-based interpretation modules; Lisp; forward chaining; concurrent service manager; computer vision; computerised pattern recognition; infrared imaging; knowledge based systems; optical radar; remote sensing by laser beam
C.C. Chu, J.K. Aggarwal, "Image Interpretation Using Multiple Sensing Modalities," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 8, pp. 840-847, Aug. 1992, doi:10.1109/34.149595
Usage of this product signifies your acceptance of the Terms of Use.