We present a segmentation technique for image sequences using Self Organizing Feature Maps (SOFM). Our goal is to develop a method, which can identify homogeneous regions in a frame to represent higher-level objects for content-based manipulation of image sequences. The proposed scheme extracts pixel-based multiple features, such as motion and textures, and then, different weights are applied to each feature component based on motion confidence measures. These multiple feature space is transformed to one dimensional label space by using the SOFM. The over-segmentation neural network outputs are merged in order to generate desired segmentation resolution. Our experimental results show the validity of the proposed scheme.