It was recently proposed the use of Bayesian networks for object tracking. Bayesian networks allow to model the interaction among detected trajectories, in order to obtain a reliable object identification in the presence of occlusions. However, the architecture of the Bayesian network has been defined using simple heuristic rules which fail in many cases. This paper addresses the above problem and presents a new method to estimate the network architecture from the video sequences using supervised learning techniques. Experimental results are presented showing that significant performance gains (increase of accuracy and decrease of complexity) are achieved by the proposed methods.