A harmony search is modified to optimize typical multi-mode resource constrained project scheduling problem, which aims to minimize the project make span. In order to inherit elitist solution and maintain population's diversity based on distance simultaneously, we add a selection operator when the harmony memory is initialized and modify the replacement operator. A new solution can be improvised by three options, i.e. usage of harmony memory, pitch adjusting and randomization. The first option is adjusted to select solutions from harmony memory based on average objective value, and the second is only to justify mode selection. Finally, the harmony search is compared with the genetic algorithm which has better performance. The numerical experiments based on benchmark test sets of 20 and 30 activities with three modes show that harmony search can obtain high-quality and robust solutions.