We consider the problem of clustering data lying on multiple subspaces of unknown and possibly different dimensions. We show that one can represent the subspaces with a set of polynomials whose derivatives at a data point give normal vectors to the subspace associated with the data point. Since the polynomials can be estimated linearly from data, subspace clustering is reduced to classifying one point per subspace. We do so by choosing points in the data set that minimize a distance function. A basis for the complement of each subspace is then recovered by applying standard PCA to the set of derivatives (normal vectors) at those points. The final result is a new GPCA algorithm for subspace clustering based on simple linear and polynomial algebra. Our experiments show that our method outperforms existing algebraic algorithms based on polynomial factorization and provides a good initialization to iterative techniques such as K-subspace and EM. We also present applications of GPCA on computer vision problems such as vanishing point detection, face clustering, and news video segmentation.