Publication 2011 Issue No. 12 - December Abstract - Causal Inference on Discrete Data Using Additive Noise Models
 This Article Share Bibliographic References Add to: Digg Furl Spurl Blink Simpy Google Del.icio.us Y!MyWeb Search Similar Articles Articles by Jonas Peters Articles by Dominik Janzing Articles by Bernhard Schölkopf
Causal Inference on Discrete Data Using Additive Noise Models
December 2011 (vol. 33 no. 12)
pp. 2436-2450
 ASCII Text x Jonas Peters, Dominik Janzing, Bernhard Schölkopf, "Causal Inference on Discrete Data Using Additive Noise Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2436-2450, December, 2011.
 BibTex x @article{ 10.1109/TPAMI.2011.71,author = {Jonas Peters and Dominik Janzing and Bernhard Schölkopf},title = {Causal Inference on Discrete Data Using Additive Noise Models},journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},volume = {33},number = {12},issn = {0162-8828},year = {2011},pages = {2436-2450},doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.71},publisher = {IEEE Computer Society},address = {Los Alamitos, CA, USA},}
 RefWorks Procite/RefMan/Endnote x TY - JOURJO - IEEE Transactions on Pattern Analysis and Machine IntelligenceTI - Causal Inference on Discrete Data Using Additive Noise ModelsIS - 12SN - 0162-8828SP2436EP2450EPD - 2436-2450A1 - Jonas Peters, A1 - Dominik Janzing, A1 - Bernhard Schölkopf, PY - 2011KW - Causal inferenceKW - regressionKW - graphical models.VL - 33JA - IEEE Transactions on Pattern Analysis and Machine IntelligenceER -
Jonas Peters, Max Planck Institute for Biological Cybernetics, Tübingen
Dominik Janzing, Max Planck Institute for Biological Cybernetics, Tübingen
Bernhard Schölkopf, Max Planck Institute for Biological Cybernetics, Tübingen
Inferring the causal structure of a set of random variables from a finite sample of the joint distribution is an important problem in science. The case of two random variables is particularly challenging since no (conditional) independences can be exploited. Recent methods that are based on additive noise models suggest the following principle: Whenever the joint distribution {\bf P}^{(X,Y)} admits such a model in one direction, e.g., Y=f(X)+N, N \perp\kern-6pt \perp X, but does not admit the reversed model X=g(Y)+\tilde{N}, \tilde{N} \perp\kern-6pt \perp Y, one infers the former direction to be causal (i.e., X\rightarrow Y). Up to now, these approaches only dealt with continuous variables. In many situations, however, the variables of interest are discrete or even have only finitely many states. In this work, we extend the notion of additive noise models to these cases. We prove that it almost never occurs that additive noise models can be fit in both directions. We further propose an efficient algorithm that is able to perform this way of causal inference on finite samples of discrete variables. We show that the algorithm works on both synthetic and real data sets.

[1] A. Agresti, Categorical Data Analysis, second ed. Wiley-Interscience, 2002.
[2] R. Armann and I. Bülthoff, in preparation, https://webdav. tuebingen.mpg.decause-effect /, 2010.
[3] A. Asuncion, and D. Newman, UCI Machine Learning Repository, http://archive.ics.uci.eduml/, 2007.
[4] W.G. Cochran, "Some Methods for Strengthening the Common $\chi^2$ Tests," Biometrics, vol. 10, pp. 417-451, 1954.
[5] J. Czerniak, and H. Zarzycki, "Application of Rough Sets in the Presumptive Diagnosis of Urinary System Diseases," Artificial Intelligence and Security in Computing Systems, pp. 41-51, Kluwer Academic Publishers, 2003.
[6] D. Heckerman, C. Meek, and G. Cooper, "A Bayesian Approach to Causal Discovery," Computation, Causation, and Discovery, C. Glymour and G. Cooper, eds., pp. 141-165, MIT Press, 1999.
[7] P. Hoyer, D. Janzing, J. Mooij, J. Peters, and B. Schölkopf, "Nonlinear Causal Discovery with Additive Noise Models," Proc. Neural Information Processing Systems, pp. 689-696, 2009.
[8] D. Janzing, J. Peters, J.M. Mooij, and B. Schölkopf, "Identifying Confounders Using Additive Noise Models," Proc. 25th Conf. Uncertainty in Artificial Intelligence, pp. 249-257, 2009.
[9] D. Janzing and B. Steudel, "Justifying Additive-Noise-Model Based Causal Discovery via Algorithmic Information Theory," Open Systems and Information Dynamics, vol. 17, pp. 189-212, 2010.
[10] Y. Kano and S. Shimizu, "Causal Inference Using Nonnormality," Proc. Int'l Symp. Science of Modeling, the 30th Anniversary of the Information Criterion, pp. 261-270, 2003.
[11] J. Mooij, D. Janzing, J. Peters, and B. Schölkopf, "Regression by Dependence Minimization and Its Application to Causal Inference," Proc. 26th Int'l Conf. Machine Learning, pp. 745-752, 2009.
[12] W. Nash, T. Sellers, S. Talbot, A. Cawthorn, and W. Ford, "The Population Biology of Abalone (Haliotis Species) in Tasmania," Technical Report No. 48 (ISSN 1034-3288), I. Blacklip Abalone (H. rubra) from the North Coast and Islands of Bass Strait, Sea Fisheries Division, 1994.
[13] J. Pearl, Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, 2000.
[14] J. Peters, D. Janzing, A. Gretton, and B. Schölkopf, "Detecting the Direction of Causal Time Series," Proc. 26th Int'l Conf. Machine Learning, pp. 801-808, 2009.
[15] J. Peters, D. Janzing, and B. Schölkopf, "Identifying Cause and Effect on Discrete Data Using Additive Noise Models," Proc. 13th Int'l Conf. Artificial Intelligence and Statistics, vol. 9, pp. 597-604, 2010.
[16] S. Shimizu, P.O. Hoyer, A. Hyvärinen, and A.J. Kerminen, "A Linear Non-Gaussian Acyclic Model for Causal Discovery," J. Machine Learning Research, vol. 7, pp. 2003-2030, 2006.
[17] P. Spirtes, C. Glymour, and R. Scheines, Causation, Prediction, and Search, second ed. MIT Press, 2000.
[18] X. Sun, D. Janzing, and B. Schölkopf, "Causal Inference by Choosing Graphs with Most Plausible Markov Kernels," Proc. Ninth Int'l Symp. Artificial Intelligence and Math., pp. 1-11, 2006.
[19] X. Sun, D. Janzing, and B. Schölkopf, "Causal Reasoning by Evaluating the Complexity of Conditional Densities with Kernel Methods," Neurocomputing, vol. 71, pp. 1248-1256, 2008.
[20] R. Tillman, A. Gretton, and P. Spirtes, "Nonlinear Directed Acyclic Structure Learning with Weakly Additive Noise Models," Proc. Neural Information Processing Systems, Dec. 2009.
[21] T. Verma and J. Pearl, "Equivalence and Synthesis of Causal Models," Proc. Sixth Conf. Uncertainty in Artificial Intelligence, pp. 255-270, 1991.
[22] K. Zhang and A. Hyvarinen, "On the Identifiability of the Post-Nonlinear Causal Model," Proc. 25th Conf. Uncertainty in Artificial Intelligence, pp. 647-655, 2009.

Index Terms:
Causal inference, regression, graphical models.
Citation:
Jonas Peters, Dominik Janzing, Bernhard Schölkopf, "Causal Inference on Discrete Data Using Additive Noise Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2436-2450, Dec. 2011, doi:10.1109/TPAMI.2011.71