In this paper we propose a Genetic-Based Inverse Voting Hough Transform (GBIVHT) method to detect buried crack defects in engineering structures. The method is applied to B-scan images obtained according to the ultrasonic Time Of Flight Diffraction technique. In these image representations of the ultrasound data, crack defects are characterized by multiple arcs of diffraction that can be approximated by a parabolic model. Thus, the crack defect detection problem in non-destructive inspection of engineering structures is transformed into a parabola detection and localization on B-scan images. In the proposed GBIVHT method, the local peak detection problem of conventional HT is converted into a parameter optimization problem that operates directly on the B-scan images. The optimization task is done using the well-known Genetic Algorithms. Our main goals are an accurate detection of the parabolas while circumventing the computational complexity and huge storage problem tied to conventional HT.