We discuss the relationship between the discriminative training of Gaussian models and the maximum entropy framework for log-linear models. Observing that linear transforms leave the distributions resulting from the log-linear model unchanged, we derive a discriminative linear feature reduction technique from the maximum entropy approach and compare it to the well-known linear discriminant analysis. From experiments on different corpora we observe that the new technique performs better than linear discriminant analysis if the dimensionality of the feature space is large with respect to the number of classes.