This paper presents a unified approach to adaptive target detection and tracking. The unifying concept is "coherent motion energy", a measure of the extent to which a single motion dominates local spatiotemporal structure. There are three major components to the approach. First, a multiresolution analysis of coherent motion energy is used to detect salient dynamic targets. Second, a robust affine transformation estimator is used to recover frame-to-frame target motion across regions of interest defined by coherent motion. Third, a method of template adaptation based on coherent motion weighted goodness of match is used to drive automatic template update. Empirical evaluation of the approach shows the contribution of the various components and documents strong performance of the integrated whole.