In this paper, we present a method that detects lesions in two-dimensional (2D) cross-sectional brain images. Use of the morphological watershed segmentation technique localizes shape variation in the gray level distribution of brain images and, in turn, identifies the regions with abnormal shape and/or texture structure. The detected brain areas are then subjected to a model-based inverse filtering to determine their physiological characteristics whether they are lesions or other types of anomalies. The proposed algorithm was tested on different images of "The Whole Brain Atlas" database [13]. The experimental results have produced 90% classification accuracy in processing 10 arbitrary images, representing different kinds of brain lesion.