This paper presents a new method of detecting and predicting motion tracking failures with applications in human motion and gait analysis. We define a tracking failure as an event and describe its temporal characteristics using a hidden Markov model (HMM). This stochastic model is trained using previous examples of tracking failures. We derive vector observations for the HMM using the noise covariance matrices characterizing a tracked, 3-D structural model of the human body. We show a causal relationship between the conditional output probability of the HMM, as transformed using a logarithmic mapping function, and impending tracking failures. Results are illustrated on several multi-view sequences of complex human motion.