The Community for Technology Leaders
Subscribe
Issue No.07 - July (2012 vol.23)
pp: 1169-1177
Yinglong Xia , IBM T.J. Watson Research Center, Yorktown Heights
Viktor K. Prasanna , University of Southern California, Los Angeles
ABSTRACT
Evidence propagation is a major step in exact inference, a key problem in exploring probabilistic graphical models. In this paper, we propose a novel approach for parallelizing evidence propagation in junction trees on clusters. Our proposed method explores structural parallelism in a given junction tree. We decompose a junction tree into a set of subtrees, each consisting of one or multiple leaf-root paths in the junction tree. In evidence propagation, we first perform evidence collection in these subtrees concurrently. Then, the partially updated subtrees exchange data for junction tree merging, so that all the cliques in the junction tree can be fully updated for evidence collection. Finally, evidence distribution is performed in all the subtrees to complete evidence propagation. Since merging subtrees requires communication across processors, we propose a technique called bitmap partitioning to explore the tradeoff between bandwidth utilization efficiency and the overhead due to the startup latency of message passing. We implemented the proposed method using Message Passing Interface (MPI) on a state-of-the-art Myrinet cluster consisting of 128 processors. Compared with a baseline method, our technique results in improved scalability.
INDEX TERMS
Junction tree, exact inference, decomposition, message passing.
CITATION
Yinglong Xia, Viktor K. Prasanna, "Distributed Evidence Propagation in Junction Trees on Clusters", IEEE Transactions on Parallel & Distributed Systems, vol.23, no. 7, pp. 1169-1177, July 2012, doi:10.1109/TPDS.2011.278
REFERENCES
 [1] D.A. Bader, "High-Performance Algorithm Engineering for Large-Scale Graph Problems and Computational Biology," Proc. Fourth Int'l Workshop Efficient and Experimental Algorithms, pp. 16-21, 2005. [2] K.W. Cameron, R. Ge, and X.-H. Sun, "$\log_{n}{P}$ and $\log_{3}{P}$ : Accurate Analytical Models of Point-to-Point Communication in Distributed Systems," IEEE Trans. Computers, vol. 56, no. 3, pp. 314-327, Mar. 2007. [3] D. Heckerman, "Bayesian Networks for Data Mining," Data Mining and Knowledge Discovery, vol. 1, pp. 79-119, 1997. [4] T. Ito, T. Uno, X. Zhou, and T. Nishizeki, "Partitioning a Weighted Tree to Subtrees of Almost Uniform Size," Proc. 19th Int'l Symp. Algorithms and Computation, pp. 196-207, 2008. [5] A.V. Kozlov and J.P. Singh, "A Parallel Lauritzen-Spiegelhalter Algorithm for Probabilistic Inference," Proc. Supercomputing, pp. 320-329, 1994. [6] S.L. Lauritzen and D.J. Spiegelhalter, "Local Computation with Probabilities and Graphical Structures and Their Application to Expert Systems," J. Royal Statistical Soc. B, vol. 50, pp. 157-224, 1988. [7] M. Lin, I. Lebedev, and J. Wawrzynek, "High-Throughput Bayesian Computing Machine with Reconfigurable Hardware," Proc. 18th Ann. ACM/SIGDA Int'l Symp. Field Programmable Gate Arrays, pp. 73-82, 2010. [8] B. Middleton, M. Shwe, D. Heckerman, H. Lehmann, and G. Cooper, "Probabilistic Diagnosis Using a Reformulation of the INTERNIST-1/QMR Knowledge Base," Methods of Information Medicine, vol. 30, pp. 241-255, 1991. [9] D. Pennock, "Logarithmic Time Parallel Bayesian Inference," Proc. 14th Ann. Conf. Uncertainty in Artificial Intelligence, pp. 431-438, 1998. [10] S.J. Russell and P. Norvig, Artificial Intelligence: A Modern Approach, second ed. Prentice Hall, 2002. [11] R.D. Shachter, S.K. Andersen, and P. Szolovits, "Global Conditioning for Probabilistic Inference in Belief Networks," Proc. 10th Conf. Uncertainty in Artificial Intelligence, pp. 514-522, 1994. [12] M. Taufer, M. Crowley, D.J. Price, A.A. Chien, and C.L. BrooksIII, "Study of a Highly Accurate and Fast Protein-Ligand Docking Method Based on Molecular Dynamics: Research Articles," IEEE Third Int'l Workshop High Performance Computational Biology, vol. 17, no. 14, pp. 1627-1641, Dec. 2005. [13] USC Center for High-Performance Computing and Communications, http://www.usc.eduhpcc/, 2012. [14] Y. Xia, X. Feng, and V.K. Prasanna, "Parallel Evidence Propagation on Multicore Processors," Proc. PaCT '09: 10th Int'l Conf. Parallel Computing Technologies, pp. 377-391, 2009. [15] Y. Xia and V.K. Prasanna, "Node Level Primitives for Parallel Exact Inference," Proc. 19th Int'l Symp. Computer Architecture and High Performance Computing, pp. 221-228, Oct. 2007. [16] Y. Xia and V.K. Prasanna, "Junction Tree Decomposition for Parallel Exact Inference," Proc. IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS), 2008. [17] Y. Xia and V.K. Prasanna, "Distributed Evidence Propagation in Junction Trees," Proc. 22nd Int'l Symp. Computer Architecture and High Performance Computing, Oct. 2010. [18] Y. Xia and V.K. Prasanna, "Scalable Node-Level Computation Kernels for Parallel Exact Inference," IEEE Trans. Computer, vol. 59, no. 1, pp. 103-115, Jan. 2010. [19] H. Zha, X. He, C. Ding, H. Simon, and M. Gu, "Bipartite Graph Partitioning and Data Clustering," Proc. 10th Int'l Conf. Information and Knowledge Management, pp. 25-32, 2001. [20] A.Y. Zomaya, Parallel Computing for Bioinformatics and Computational Biology: Models, Enabling Technologies, and Case Studies. Wiley, 2006.