This Article 
 Bibliographic References 
 Add to: 
Large Margin Gaussian Mixture Models with Differential Privacy
July-Aug. 2012 (vol. 9 no. 4)
pp. 463-469
Manas A. Pathak, Carnegie Mellon University, Pittsburgh
Bhiksha Raj, Carnegie Mellon University, Pittsburgh
As increasing amounts of sensitive personal information is aggregated into data repositories, it has become important to develop mechanisms for processing the data without revealing information about individual data instances. The differential privacy model provides a framework for the development and theoretical analysis of such mechanisms. In this paper, we propose an algorithm for learning a discriminatively trained multiclass Gaussian mixture model-based classifier that preserves differential privacy using a large margin loss function with a perturbed regularization term. We present a theoretical upper bound on the excess risk of the classifier introduced by the perturbation.

[1] M. Pathak and B. Raj , “Large Margin Multiclass Gaussian Classification with Differential Privacy,” Proc. ECML/PKDD Workshop Privacy and Security Issues in Data Mining and Machine Learning, 2010.
[2] C. Dwork , “Differential Privacy,” Proc. Int'l Colloquium Automata, Languages and Programming, 2006.
[3] K. Chaudhuri and C. Monteleoni , “Privacy-Preserving Logistic Regression,” Proc. Neural Information Processing Systems (NIPS), pp. 289-296, 2008.
[4] G. McLachlan and D. Peel , Finite Mixture Models, Wiley Series in Probability and Statistics. Wiley-Interscience, 2000.
[5] F. Sha and L.K. Saul , “Large Margin Gaussian Mixture Modeling for Phonetic Classification and Recognition,” Proc. IEEE Int'l Conf. Acoustics, Speech and Signal Processing (ICASSP), pp. 265-268, 2006.
[6] I. Dinur and K. Nissim , “Revealing Information while Preserving Privacy,” Proc. Symp. Principles of Database Systems, 2003.
[7] C. Dwork and K. Nissim , “Privacy-Preserving Datamining on Vertically Partitioned Databases,” Proc. 24th Ann. Int'l Cryptology Conf. (CRYPTO), 2004.
[8] A. Blum , C. Dwork , F. McSherry , and K. Nissim , “Practical Privacy: The suLQ Framework,” Proc. Symp. Principles of Database Systems, 2005.
[9] B. Barak , K. Chaudhuri , C. Dwork , S. Kale , F. McSherry , and K. Talwar , “Privacy, Accuracy, and Consistency Too: A Holistic Solution to Contingency Table Release,” Proc. Symp. Principles of Database Systems, pp. 273-282, 2007.
[10] S.P. Kasiviswanathan , H.K. Lee , K. Nissim , S. Raskhodnikova , and A. Smith , “What Can We Learn Privately?,” Proc. IEEE Symp. Foundations of Computer Science (FOCS), pp. 531-540, 2008.
[11] G. Jagannathan , K. Pillaipakkamnatt , and R.N. Wright , “A Practical Differentially Private Random Decision Tree Classifier,” Proc. ICDM Workshop Privacy Aspects of Data Mining, pp. 114-121, 2009.
[12] F. Sha and L.K. Saul , “Large Margin Hidden Markov Models for Automatic Speech Recognition,” Proc. Neural Information Processing Systems (NIPS), pp. 1249-1256, 2007.
[13] P.C. Mahalanobis , “On the Generalised Distance in Statistics,” Proc. the Nat'l Inst. of Sciences of India, vol. 2, pp. 49-55, 1936.
[14] L. Vandenberghe and S. Boyd , “Semidefinite Programming,” SIAM Rev., vol. 38, pp. 49-95, 1996.
[15] O. Chapelle , “Training a Support Vector Machine in the Primal,” Neural Computation, vol. 19, no. 5, pp. 1155-1178, 2007.
[16] K. Chaudhuri , C. Monteleoni , and A.D. Sarwate , “Differentially Private Empirical Risk Minimization,” J. Machine Learning Research, vol. 12, pp. 1069-1109, 2011.
[17] K. Sridharan , S. Shalev-Shwartz , and N. Srebro , “Fast Rates for Regularized Objectives,” Proc. Neural Information Processing Systems (NIPS), pp. 1545-1552, 2008.
[18] M. Grant and S. Boyd , “CVX: Matlab Software for Disciplined Convex Programming, Version 1.21,” http://cvxr.comcvx , 2010.
[19] A. Frank and A. Asuncion , “UCI Machine Learning Repository,” http://archive.ics.uci.eduml , 2010.

Index Terms:
Differential privacy, machine learning.
Manas A. Pathak, Bhiksha Raj, "Large Margin Gaussian Mixture Models with Differential Privacy," IEEE Transactions on Dependable and Secure Computing, vol. 9, no. 4, pp. 463-469, July-Aug. 2012, doi:10.1109/TDSC.2012.27
Usage of this product signifies your acceptance of the Terms of Use.