Parameterized circuit models are desired at various VLSI design stages to account for the increasing process-induced performance variations. However, the large number of process variation sources encountered in modern VLSI technologies often lead to overly complex parameterized models whose generation as well as application is computationally expensive. In this paper, we address this challenge by proposing a general VLSI parameter dimension reduction technique that can produce more compact parameterized models in a compressed set of variation variables. Unlike the widely used principle component analysis (PCA), our new approach is based upon the powerful reduced rank regression (RRR) theory and can lead to a much greater reduction of the parameter space due to the consideration of design-specific structural information. The application of our parameter reduction technique is demonstrated under the context of digital circuit timing simulation and analog macromodeling. Our experimental results have indicated an up to 10X reduction of the parameter space. The application of these compact parameterized models is also outlined under the context of system-level analysis.