This study presents a new automatic and fast approach to design optimized Gabor filters for textile flaw detection applications. Using a semi-supervised approach solves the defect detection problem. The aim is to automatically discriminate between “known” non-defective background textures and “unknown” defective textures. The parameters of the optimal 2-D Gabor filters are derived by constrained minimization of a Fisher cost function. Such optimized Gabor filters are capable of detecting both, structural and tonal defects. This adaptable approach can detect a large variety of flaw types, while at the same time, accounting for their changing appearance in different texture backgrounds. When applied to a large database of textile fabrics, accurate detection with a low false alarm rate was achieved.