In this paper, we propose a novel OOI (Object-of-Interest) segmentation algorithm from natural images that is based on human attention and semantic region merging. To do this, we segment an image into regions and merge them as a semantic object. Then, we create an Attention Window based on saliency map and saliency points from an image. Within the AW, a Support Vector Machine is used to select the salient regions, which are then clustered into the OOI using the proposed region merging. Unlike other algorithms, the proposed method allows multiple OOIs to be segmented according to the saliency map. Experiments with the algorithm on more than 300 natural images have shown results close to human perception.