Automatic identifying weave pattern is an important research area for computer aided design of textile products. Based on the scanned surface images, we have developed a semi-automatic identification system of weave patterns for double-layer weft woven necktie fabric. With manual partitioning, the complicated necktie fabric pattern is regarded as combination of several single-layer weave patterns. Image processing techniques are employed to segment the fabric color image, such as color clustering, morphological operation, etc. A novel algorithm to correct warp displacements is proposed to perform yarns segmentation more precisely. Finally, the integrated method to identify the double-layer woven fabric, which combines automatic identifying single-layer weave with human-computer interaction, is illustrated. The experimental results show the practicability and efficiency of our system.