The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2013 vol.35)
pp: 2526-2538
N. Jacobs , Dept. of Comput. Sci., Univ. of Kentucky, Lexington, KY, USA
A. Abrams , Dept. of Comput. Sci., Univ. of Kentucky, Lexington, KY, USA
R. Pless , Dept. of Comput. Sci., Washington Univ. in St. Louis, St. Louis, MO, USA
ABSTRACT
We describe algorithms that use cloud shadows as a form of stochastically structured light to support 3D scene geometry estimation. Taking video captured from a static outdoor camera as input, we use the relationship of the time series of intensity values between pairs of pixels as the primary input to our algorithms. We describe two cues that relate the 3D distance between a pair of points to the pair of intensity time series. The first cue results from the fact that two pixels that are nearby in the world are more likely to be under a cloud at the same time than two distant points. We describe methods for using this cue to estimate focal length and scene structure. The second cue is based on the motion of cloud shadows across the scene; this cue results in a set of linear constraints on scene structure. These constraints have an inherent ambiguity, which we show how to overcome by combining the cloud motion cue with the spatial cue. We evaluate our method on several time lapses of real outdoor scenes.
INDEX TERMS
Correlation, Clouds, Cameras, Delay, Time series analysis, Geometry, Satellites,clouds, Time lapse, depth map, nonmetric multidimensional scaling, image formation, shape from shadows
CITATION
N. Jacobs, A. Abrams, R. Pless, "Two Cloud-Based Cues for Estimating Scene Structure and Camera Calibration", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 10, pp. 2526-2538, Oct. 2013, doi:10.1109/TPAMI.2013.55
REFERENCES
[1] G.J. Burton and I.R. Moorhead, "Color and Spatial Structure in Natural Scenes," Applied Optics, vol. 26, pp. 157-170, 1987.
[2] V.A. Billock, "Neural Acclimation to 1/F Spatial Frequency Spectra in Natural Images Transduced by the Human Visual System," Physica D: Nonlinear Phenomena, vol. 137, pp. 379-391, 2000.
[3] C.-H. Sun and L.R. Thorne, "Inferring Spatial Cloud Statistics from Limited Field-of-View, Zenith Observations," Proc. Atmospheric Radiation Measurements Science Team Meeting, 2000.
[4] L.R. Thorne, K. Buch, C.-H. Sun, and C. Diegert, "Data and Image Fusion for Geometrical Cloud Characterization," Technical Report SAND97-9252, Sandia Nat'l Laboratories, 1997.
[5] C. Stauffer and W.E.L. Grimson, "Adaptive Background Mixture Models for Real-Time Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[6] A. Mittal, A. Monnet, and N. Paragios, "Scene Modeling and Change Detection in Dynamic Scenes: A Subspace Approach," Computer Vision and Image Understanding, vol. 113, pp. 63-79, 2009.
[7] A. Prati, I. Mikic, M.M. Trivedi, and R. Cucchiara, "Detecting Moving Shadows: Algorithms and Evaluation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 918-923, July 2003.
[8] T. Horprasert, D. Harwood, and L. Davis, "A Statistical Approach for Real-Time Robust Background Subtraction and Shadow Detection," Proc. IEEE Int'l Conf. Ccomputer Vision FRAME-RATE Workshop, 1999.
[9] P.M. Dare, "Shadow Analysis in High-Resolution Satellite Imagery of Urban Areas," Photogrammetric Eng. and Remote Sensing, vol. 71, pp. 169-177, 2005.
[10] S.J. Koppal and S.G. Narasimhan, "Appearance Derivatives for Isonormal Clustering of Scenes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 8, pp. 1375-1385, Aug. 2009.
[11] K. Sunkavalli, W. Matusik, H. Pfister, and S. Rusinkiewicz, "Factored Time-Lapse Video," ACM Trans. Graphics, vol. 26, article 101, 2007.
[12] S.J. Kim, J.-M. Frahm, and M. Pollefeys, "Radiometric Calibration with Illumination Change for Outdoor Scene Analysis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[13] K. Sunkavalli, F. Romeiro, W. Matusik, T. Zickler, and H. Pfister, "What Do Color Changes Reveal about an Outdoor Scene?" Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[14] N. Jacobs, N. Roman, and R. Pless, "Consistent Temporal Variations in Many Outdoor Scenes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[15] N. Jacobs, S. Satkin, N. Roman, R. Speyer, and R. Pless, "Geolocating Static Cameras," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[16] J.-F. Lalonde, A.A. Efros, and S.G. Narasimhan, "Webcam Clip Art: Appearance and Illuminant Transfer from Time-Lapse Sequences," ACM Trans. Graphics, vol. 28, article 131, 2009.
[17] S. Lovejoy and D. Schertzer, "Generalized Scale Invariance in the Atmosphere," Water Resources Research, 1985.
[18] I. Borg and P.J.F. Groenen, Modern Multidimensional Scaling: Theory and Applications, second ed. Springer, 2005.
[19] J.-F. Lalonde, S.G. Narasimhan, and A.A. Efros, "What Does the Sky Tell Us about the Camera?" Proc. European Conf. Computer Vision, 2008.
[20] N. Jacobs, B. Bies, and R. Pless, "Using Cloud Shadows to Infer Scene Structure and Camera Calibration," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[21] J. Kenney and E. Keeping, Mathematics of Statistics, vol. 3. Van Nostrand, 1954.
[22] J. Kruskal, "Nonmetric Multidimensional Scaling: A Numerical Method," Psychometrika, vol. 29, pp. 115-129, 1964.
[23] http://cs.uky.edu/jacobs/projectsshape-from-clouds /, 2013.
58 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool