We propose a method for auto-calibrating a compound-type omnidirectional camera (system). This camera produces spherical panoramic image through stitching some individual images captured by several modular cameras, each of which covers a different field of view of the full viewing sphere. By auto-calibration we mean that our algorithm works online in a self-calibration manner: it receives as input image point correspondences only, and outputs all the camera parameters. The calibration task consists of lens-distortion-removal, intrinsic-calibration, and relative-orientation. The focus of the paper is an algorithm for automatically removing lens distortion. By exploiting planar projective invariants, our algorithm is capable of estimating centre-of-distortion and distortion parameters efficiently without using any nonlinear iteration.