Least squares minimization of the differential epipolar constraint is a fast and efficient technique to estimate structure and motion for pair of views. Previous work in this area showed how unbiased and consistent estimates could be obtained minimizing the squared errors. However, it implicitly assumes that the errors along the x and y directions are identical and uncorrelated. This is rarely the case for real data, due to the aperture problem. Instead, one should minimize the covariance weighted squared error. Moreover, when dense sequences are acquired, further robustness can be achieved by integrating the reconstruction of structure over time. This paper has two main contributions: (i) we show that the minimization of the weighted squared errors (i.e. Maximum-Likelihood estimate) outperforms the more traditional approach of un-weighted least squares, (ii) we show how structure estimation can be integrated over time in a multi-view approach that drastically improves estimates.