We introduce an efficient method for recovering the camera parameters automatically from the cast shadows of two 3D points observed over time. Compared to previous related work, our method has less restrictions in the sense that object-to-shadow correspondences do not have to be available in the image. We demonstrate how the horizon line may be recovered from only shadow points, and how the camera intrinsic and extrinsic parameters are determined using the pole-polar relationship and minimizing the algebraic distance of the principal point. The approach is fully validated on both synthetic and real data, and tested against various sources of error. We finally present an application to metrology from shadows only - i.e. when the object is not visible in the image.