We present a novel approach for learning parameters of a Bayesian network from distributed heterogeneous dataset. In this case, the whole dataset is distributed in several sites and each site contains observations for a different subset of features. The new method uses the collective learning approach proposed in our earlier work and substantially reduces the computational and transmission overhead. Theoretical analysis is given and experimental results are provided to illustrate the accuracy and efficiency of our method.