CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2011 vol.33 Issue No.12 - December

Subscribe

Issue No.12 - December (2011 vol.33)

pp: 2436-2450

Jonas Peters , Max Planck Institute for Biological Cybernetics, Tübingen

Dominik Janzing , Max Planck Institute for Biological Cybernetics, Tübingen

Bernhard Schölkopf , Max Planck Institute for Biological Cybernetics, Tübingen

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.71

ABSTRACT

Inferring the causal structure of a set of random variables from a finite sample of the joint distribution is an important problem in science. The case of two random variables is particularly challenging since no (conditional) independences can be exploited. Recent methods that are based on additive noise models suggest the following principle: Whenever the joint distribution {\bf P}^{(X,Y)} admits such a model in one direction, e.g., Y=f(X)+N, N \perp\kern-6pt \perp X, but does not admit the reversed model X=g(Y)+\tilde{N}, \tilde{N} \perp\kern-6pt \perp Y, one infers the former direction to be causal (i.e., X\rightarrow Y). Up to now, these approaches only dealt with continuous variables. In many situations, however, the variables of interest are discrete or even have only finitely many states. In this work, we extend the notion of additive noise models to these cases. We prove that it almost never occurs that additive noise models can be fit in both directions. We further propose an efficient algorithm that is able to perform this way of causal inference on finite samples of discrete variables. We show that the algorithm works on both synthetic and real data sets.

INDEX TERMS

Causal inference, regression, graphical models.

CITATION

Jonas Peters, Dominik Janzing, Bernhard Schölkopf, "Causal Inference on Discrete Data Using Additive Noise Models",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.33, no. 12, pp. 2436-2450, December 2011, doi:10.1109/TPAMI.2011.71REFERENCES

- [1] A. Agresti,
Categorical Data Analysis, second ed. Wiley-Interscience, 2002.- [2] R. Armann and I. Bülthoff, in preparation, https://webdav. tuebingen.mpg.decause-effect /, 2010.
- [3] A. Asuncion, and D. Newman, UCI Machine Learning Repository, http://archive.ics.uci.eduml/, 2007.
- [4] W.G. Cochran, "Some Methods for Strengthening the Common $\chi^2$ Tests,"
Biometrics, vol. 10, pp. 417-451, 1954.- [5] J. Czerniak, and H. Zarzycki, "Application of Rough Sets in the Presumptive Diagnosis of Urinary System Diseases,"
Artificial Intelligence and Security in Computing Systems, pp. 41-51, Kluwer Academic Publishers, 2003.- [6] D. Heckerman, C. Meek, and G. Cooper, "A Bayesian Approach to Causal Discovery,"
Computation, Causation, and Discovery, C. Glymour and G. Cooper, eds., pp. 141-165, MIT Press, 1999.- [7] P. Hoyer, D. Janzing, J. Mooij, J. Peters, and B. Schölkopf, "Nonlinear Causal Discovery with Additive Noise Models,"
Proc. Neural Information Processing Systems, pp. 689-696, 2009.- [8] D. Janzing, J. Peters, J.M. Mooij, and B. Schölkopf, "Identifying Confounders Using Additive Noise Models,"
Proc. 25th Conf. Uncertainty in Artificial Intelligence, pp. 249-257, 2009.- [9] D. Janzing and B. Steudel, "Justifying Additive-Noise-Model Based Causal Discovery via Algorithmic Information Theory,"
Open Systems and Information Dynamics, vol. 17, pp. 189-212, 2010.- [10] Y. Kano and S. Shimizu, "Causal Inference Using Nonnormality,"
Proc. Int'l Symp. Science of Modeling, the 30th Anniversary of the Information Criterion, pp. 261-270, 2003.- [11] J. Mooij, D. Janzing, J. Peters, and B. Schölkopf, "Regression by Dependence Minimization and Its Application to Causal Inference,"
Proc. 26th Int'l Conf. Machine Learning, pp. 745-752, 2009.- [12] W. Nash, T. Sellers, S. Talbot, A. Cawthorn, and W. Ford, "The Population Biology of Abalone (Haliotis Species) in Tasmania," Technical Report No. 48 (ISSN 1034-3288), I. Blacklip Abalone (H. rubra) from the North Coast and Islands of Bass Strait, Sea Fisheries Division, 1994.
- [13] J. Pearl,
Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, 2000.- [14] J. Peters, D. Janzing, A. Gretton, and B. Schölkopf, "Detecting the Direction of Causal Time Series,"
Proc. 26th Int'l Conf. Machine Learning, pp. 801-808, 2009.- [15] J. Peters, D. Janzing, and B. Schölkopf, "Identifying Cause and Effect on Discrete Data Using Additive Noise Models,"
Proc. 13th Int'l Conf. Artificial Intelligence and Statistics, vol. 9, pp. 597-604, 2010.- [16] S. Shimizu, P.O. Hoyer, A. Hyvärinen, and A.J. Kerminen, "A Linear Non-Gaussian Acyclic Model for Causal Discovery,"
J. Machine Learning Research, vol. 7, pp. 2003-2030, 2006.- [17] P. Spirtes, C. Glymour, and R. Scheines,
Causation, Prediction, and Search, second ed. MIT Press, 2000.- [18] X. Sun, D. Janzing, and B. Schölkopf, "Causal Inference by Choosing Graphs with Most Plausible Markov Kernels,"
Proc. Ninth Int'l Symp. Artificial Intelligence and Math., pp. 1-11, 2006.- [19] X. Sun, D. Janzing, and B. Schölkopf, "Causal Reasoning by Evaluating the Complexity of Conditional Densities with Kernel Methods,"
Neurocomputing, vol. 71, pp. 1248-1256, 2008.- [20] R. Tillman, A. Gretton, and P. Spirtes, "Nonlinear Directed Acyclic Structure Learning with Weakly Additive Noise Models,"
Proc. Neural Information Processing Systems, Dec. 2009.- [21] T. Verma and J. Pearl, "Equivalence and Synthesis of Causal Models,"
Proc. Sixth Conf. Uncertainty in Artificial Intelligence, pp. 255-270, 1991.- [22] K. Zhang and A. Hyvarinen, "On the Identifiability of the Post-Nonlinear Causal Model,"
Proc. 25th Conf. Uncertainty in Artificial Intelligence, pp. 647-655, 2009. |