We present a new network anomaly detection system using dissimilarity-based one-class support vector machine( DSVMC). we transform the raw data into a dissimilarity space using Dissimilarity Representations (DR). DR describe objects by their dissimilarities to a set of target class. DSVMC are constructed on these DR. We propose a framework of anomaly detection using DSVMC. A new strategy of prototype selection has been proposed to obtain better DR. We not only offer a better approach in strategy to describe to distribution of large training dataset but also reduce the computational cost of prototype selection largely. In order to deploy the ADS in real-time detection application, we use Kernel Primary Component Analysis (KPCA) to reduce the dimension of transformed data. Evaluation has been made among traditional one-class classifiers, the dissimilarity-based one class SVM classifier without optimization of DR (WSVMC) and our DSVMC on KDDCUP’ 99 dataset. The results show that DSVMC can achieve high detection rate than WSVMC and more robust performance than traditional one-class classifiers.