An efficient method to detect and extract color objects from a cluttered scene based on statistical and spatial color similarity is proposed. Color region adjacency graphs (RAG) and six 1-D histograms corresponding to the RGB and HIS color spaces are used to represent models and scenes. A histogram intersection (HI) strategy is applied to a similarity measure of statistical color distribution between them and the RAGs are exploited to guide the search for the interesting object regions at which a global maximal value of histogram intersection is available. The color spatial relationships among the RAGs are also used to check the matching result to avoid the false positive identifications, which may be caused by a normal HI method. This strategy of combining RAG and HI makes the detection robust and precise. The experiments conducted have shown that known color objects in a complex scene can be accurately identified and extracted from the background.