We present a novel feature-based non-rigid image registration algorithm using a small number of automatically extracted points and their associated local salient region features. Our automatic registration is a hybrid approach co-optimizing point-based and image-based terms. Motivated by the paradigm of the TPS-RPM algorithm [6], we develop the RHDM (Robust Hybrid Deformable Matching) algorithm by alternatively optimizing correspondences and transformations for registration. The local salient region features and the geometric features, together with the softassign and deterministic annealing techniques, are used for solving correspondences. Thin-plate splines are used for generating a smooth non-rigid spatial transformation. Our algorithm is built to be extremely robust to feature extraction errors. A new dynamic outlier rejection mechanism is described for rejecting outliers and generating accurate spatial mappings. A local refinement technique is used for correcting non-exactly matched correspondences arising from image noise and irregular deformations. In contrast with the TPS-RPM algorithm, which can handle only outliers in one point set, our algorithm is able to handle a considerable number of outliers in both point sets. The experimental results demonstrate the robustness and accuracy of our algorithm.