Automatic 3D model acquisition and 3D tracking of simple objects under motion using a single camera is often difficult due to the sparsity of information from which to establish the model. We have developed an automatic scheme that first computes a simple pointalistic Euclidean model of the object and then enriches this model using hyper-patches. These hyper-patches contain information on both the orientation and intensity pattern variation of roughly planar patches on an object. This information allows both the spatial and intensity distortions of the projected patch to be modelled accurately under 3D object motion. We show that hyper-patches not only can be computed automatically during model acquisition from a monocular image sequence, but that they are also extremely appropriate for the task of visual tracking.