In many image and computer vision applications, shadows interfere with fundamental tasks such as moving objects segmentation and tracking. In this paper, a novel method is proposed to detect the moving cast shadows in the scene. The normalized coefficients of orthogonal transform of image block are proved to be illumination invariant and are used to classify moving shadows and foreground objects. Five kinds of orthogonal transform: DCT, DFT, Haar Transform, SVD and Hadamard Transform, are utilized in our work to detect moving cast shadows. Experimental results show that the proposed method succeeds in detecting moving cast shadows within indoor and outdoor environments.