Conventional vision systems are designed to perform in clear weather. However, any outdoor vision system is incomplete without mechanisms that guarantee satisfactory performance under poor weather conditions. It is known that the atmosphere can significantly alter light energy reaching an observer. Therefore, atmospheric scattering models must be used to make vision systems robust in bad weather. In this paper, we develop a geometric framework for analyzing the chromatic effects of atmospheric scattering. First, we study a simple color model for atmospheric scattering and verify it for fog and haze. Then, based on the physics of scattering, we derive several geometric constraints on scene color changes, caused by varying atmospheric conditions. Finally, using these constraints we develop algorithms for computing fog or haze color, depth segmentation, extracting three-dimensional structure, and recovering “true” scene colors, from two or more images taken under different but unknown weather conditions.