This paper presents a simple but robust visual tracking algorithm based on representing the appearances of objects using affine warps of learned linear subspaces of the image space. The tracker adaptively updates this subspace while tracking by finding a linear subspace that best approximates the observations made in the previous frames. Instead of the traditional L^2-reconstruction error norm which leads to subsapce estimation using PCA or SVD, we argue that a variant of it, the uniform L^2-reconstruction error norm, is the right one for tracking. Under this framework, we provide a simple and a computationally inexpensive algorithm for finding a subspace whose uniform L^2-reconstruction error norm for a given collection of data samples is below some threshold, and a simple tracking algorithm is an immediate consequence. We show experimental results on a variety of image sequences of people and man-made objects moving under challenging imaging conditions, which include drastic illumination variation, partial occlusion and extreme pose variation.