A robot system controlled by human thoughts is introduced in this paper. Aiming at the recognition problem of electroencephalogram (EEG) signals in the system, we present a novel pattern recognition method. The method combines the genetic algorithm (GA) with the support vector machine (SVM). It includes two techniques. One is that the feature selection and model parameters of the SVM are optimized synchronously, which constitutes a hybrid optimization. The other is that the hybrid optimization is realized by using the GA. The method is used to classify three types of EEG signals in the system. The experiment results show that this method can yield significantly higher classification accuracy than ones obtained with individual optimizations.