The task of vision based people tracking is a major research problem in the context of surveillance applications or human behavior estimation, but it has had only minimal impact on (Ubiquitous) Augmented Reality applications thus far. Deploying stationary infrastructural cameras within indoor environments for the purpose of Augmented Reality could provide a users' devices with additional functionality that a small device and mobile sensors cannot provide to its user. Therefore people tracking could be expected to become an ubiquitously available infrastructural element in buildings since surveillance cameras are widely used. The use for scenarios indoors or close to buildings is obvious. We present and discuss several different ways where people tracking in real-time could influence the fields of Augmented Reality and further vision based applications.