Issue No. 03 - March (2012 vol. 34)
M. Wohlmayr , Dept. of Electr. Eng., Graz Univ. of Technol., Graz, Austria
F. Pernkopf , Dept. of Electr. Eng., Graz Univ. of Technol., Graz, Austria
S. Tschiatschek , Dept. of Electr. Eng., Graz Univ. of Technol., Graz, Austria
We present a maximum margin parameter learning algorithm for Bayesian network classifiers using a conjugate gradient (CG) method for optimization. In contrast to previous approaches, we maintain the normalization constraints on the parameters of the Bayesian network during optimization, i.e., the probabilistic interpretation of the model is not lost. This enables us to handle missing features in discriminatively optimized Bayesian networks. In experiments, we compare the classification performance of maximum margin parameter learning to conditional likelihood and maximum likelihood learning approaches. Discriminative parameter learning significantly outperforms generative maximum likelihood estimation for naive Bayes and tree augmented naive Bayes structures on all considered data sets. Furthermore, maximizing the margin dominates the conditional likelihood approach in terms of classification performance in most cases. We provide results for a recently proposed maximum margin optimization approach based on convex relaxation . While the classification results are highly similar, our CG-based optimization is computationally up to orders of magnitude faster. Margin-optimized Bayesian network classifiers achieve classification performance comparable to support vector machines (SVMs) using fewer parameters. Moreover, we show that unanticipated missing feature values during classification can be easily processed by discriminatively optimized Bayesian network classifiers, a case where discriminative classifiers usually require mechanisms to complete unknown feature values in the data first.
pattern classification, belief networks, conjugate gradient methods, convex programming, feature extraction, learning (artificial intelligence), maximum likelihood estimation, margin-optimized Bayesian network classifiers, maximum margin Bayesian network classifiers, maximum margin parameter learning algorithm, conjugate gradient method, normalization constraints, probabilistic interpretation, missing feature handling, conditional likelihood learning, maximum likelihood learning, discriminative parameter learning, maximum margin optimization approach, convex relaxation, CG-based optimization, Bayesian methods, Optimization, Niobium, Fasteners, Random variables, Training, Algorithm design and analysis, convex relaxation., Bayesian network classifier, discriminative learning, discriminative classifiers, large margin training, missing features
M. Wohlmayr, F. Pernkopf and S. Tschiatschek, "Maximum Margin Bayesian Network Classifiers," in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 34, no. , pp. 521-532, 2012.