A generalized Mumford-Shah model for color image restoration and edge detection is established in this paper. First the color images are consideedr as the manifold based on the "surface based method", then a new physical quantity in the form of vector product, which describes the differences of the gradient between different channels, is introduced into the regularization term of the "object", then a generalized energy functional is proposed. Finally, we proposed a fast PDE?s numerical iterative algorithm utilizing the steepest descent method and half-point scheme. Experimental results show the proposed model has good ability to overcoming the bad phenomena of color fluctuations along edges caused in the straightforward extension of the original Mumford-Shah model in vectorial cases.