In this paper, we present a system addressing autonomous recognition of ship types in infrared images. Firstly, segmentation is implemented after the target region is automatically found based on detection of salient features of the target. Feature extraction is then accomplished as the moment functions for both the target boundary and the solid silhouette are used as the featureset. Lastly, the classification method based on Support Vector Machines (SVMs) is adopted in the recognition stage, as the training sets are obtained through projections of three-dimensional ship models designed by investigators of Naval Postgraduate School. The system was implemented and experimentally validated using both simulated three-dimensional ship model images and real images derived from video of an AN/AAS-44V ForwardLooking Infrared(FLIR) sensor. Moreover, our proposed system is general and can be generalized for other similar pattern recognition applications.