This paper proposes improvements in the Yamazaki method for the simultaneous optimization of Multilayer Perceptron network weights and architectures. The main objective is to propose a set of modifications, with respective validations, aimed at creating a more efficient optimization process. The optimization hybrid algorithm is based on the simulated annealing and tabu search algorithms as well as the Yamazaki method. The modifications are carried out in the implementation criteria, such as the neighbor generation mechanism and cooling schedule. One of the main points of this paper is the creation of a new neighbor generation mechanism aimed at increasing the search space. Experimental results, using two different data sets (wine and gases), show that the optimization hybrid algorithm obtained the lowest value of the average classification error and obtained a substantial gain in execution time (reduced by an average of 46.72%).