Water and ice clouds and other aerosols will significantly impact the performance of air and space based hyperspectral remote sensing systems. The effects are seen both in terms of optically thin layers intervening in the sensor line-of-sight and in the impact of cloud shadowed illumination on observed materials. While contamination is expected more often than not, many experiments and algorithms either explicitly or implicitly are biased toward ideal conditions. In this paper we discuss a range of findings regarding the expectation of cloud and aerosol contamination, the automated recognition of that contamination, approaches to minimizing the deleterious effects on remotely sensed signals and the expected impact on material identification performance. Because of a relative paucity of data, simulation plays an important role in these studies; the paper includes a review of current capabilities to replicate cloud impacts through modeling.