Tensor fields specifically, matrix valued data sets, have recently attracted increased attention in the fields of image processing, computer vision, visualization and medical imaging. In this paper, we present a novel definition of tensor "distance" grounded in concepts from information theory and incorporate it in the segmentation of tensor-valued images. In some applications, a symmetric positive definite (SPD) tensor at each point of a tensor valued image can be interpreted as the covariance matrix of a local Gaussian distribution. Thus, a natural measure of dissimilarity between SPD tensors would be the KL divergence or its relative. We propose the square root of the J-divergence (symmetrized KL) between two Gaussian distributions corresponding to the tensors being compared that leads to a novel closed form expression. Unlike the traditional Frobenius norm-based tensor distance, our "distance" is affine invariant, a desirable property in many applications. We then incorporate this new tensor "distance" in a region based active contour model for bimodal tensor field segmentation and show its application to the segmentation of diffusion tensor magnetic resonance images (DT-MRI) as well as for the texture segmentation problem in computer vision. Synthetic and real data experiments are shown to depict the performance of the proposed model.