2013 IEEE 54th Annual Symposium on Foundations of Computer Science (2006)

Berkeley, California

Oct. 21, 2006 to Oct. 24, 2006

ISSN: 0272-5428

ISBN: 0-7695-2720-5

pp: 563-574

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/FOCS.2006.51

Parikshit Gopalan , Georgia Tech., USA

Subhash Khot , Georgia Tech., USA

Ashok Kumar Ponnuswami , Georgia Tech., USA

Vitaly Feldman , Harvard University, USA

ABSTRACT

We address well-studied problems concerning the learnability of parities and halfspaces in the presence of classification noise. <p>Learning of parities under the uniform distribution with random classification noise, also called the noisy parity problem is a famous open problem in computational learning. We reduce a number of basic problems regarding learning under the uniform distribution to learning of noisy parities. We show that under the uniform distribution, learning parities with adversarial classification noise reduces to learning parities with random classification noise. Together with the parity learning algorithm of Blum et al. [5], this gives the first nontrivial algorithm for learning parities with adversarial noise. We show that learning of DNF expressions reduces to learning noisy parities of just logarithmic number of variables. We show that learning of k-juntas reduces to learning noisy parities of k variables. These reductions work even in the presence of random classification noise in the original DNF or junta.</p> <p> We then consider the problem of learning halfspaces over \mathbb{Q}^n with adversarial noise or finding a halfspace that maximizes the agreement rate with a given set of examples. We prove an essentially optimal hardness factor of 2-\in, improving the factor of \frac{{85}} {{84}} - \in due to Bshouty and Burroughs [8].</p> <p>Finally, we show that majorities of halfspaces are hard to PAC-learn using any representation, based on the cryptographic assumption underlying the Ajtai-Dwork cryptosystem.</p>

INDEX TERMS

null

CITATION

Parikshit Gopalan,
Subhash Khot,
Ashok Kumar Ponnuswami,
Vitaly Feldman,
"New Results for Learning Noisy Parities and Halfspaces",

*2013 IEEE 54th Annual Symposium on Foundations of Computer Science*, vol. 00, no. , pp. 563-574, 2006, doi:10.1109/FOCS.2006.51SEARCH