Recently machine learning-based intrusion detection approaches have been subjected to extensive researches because they can detect both misuse and anomaly. In this paper, we use an unsupervised learning method for anomaly detection. This is done by introducing a new kind of kernel function, a simple form of P-kernel, to one-class SVM. Test sand comparison this method with standard SVM and several other existing machine learning algorithms shows that the approach proposed in this paper yielded highly accurate.