CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2011 vol.33 Issue No.04 - April

Subscribe

Issue No.04 - April (2011 vol.33)

pp: 671-686

Ashok Veeraraghavan , Mitsubishi Electric Research Labs, Cambridge

Dikpal Reddy , University of Maryland, College Park

Ramesh Raskar , Massachusettes Institute of Technology, Cambridge

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2010.87

ABSTRACT

We show that, via temporal modulation, one can observe and capture a high-speed periodic video well beyond the abilities of a low-frame-rate camera. By strobing the exposure with unique sequences within the integration time of each frame, we take coded projections of dynamic events. From a sequence of such frames, we reconstruct a high-speed video of the high-frequency periodic process. Strobing is used in entertainment, medical imaging, and industrial inspection to generate lower beat frequencies. But this is limited to scenes with a detectable single dominant frequency and requires high-intensity lighting. In this paper, we address the problem of sub-Nyquist sampling of periodic signals and show designs to capture and reconstruct such signals. The key result is that for such signals, the Nyquist rate constraint can be imposed on the strobe rate rather than the sensor rate. The technique is based on intentional aliasing of the frequency components of the periodic signal while the reconstruction algorithm exploits recent advances in sparse representations and compressive sensing. We exploit the sparsity of periodic signals in the Fourier domain to develop reconstruction algorithms that are inspired by compressive sensing.

INDEX TERMS

Computational imaging, high-speed imaging, compressive sensing, compressive video sensing, stroboscopy.

CITATION

Ashok Veeraraghavan, Dikpal Reddy, Ramesh Raskar, "Coded Strobing Photography: Compressive Sensing of High Speed Periodic Videos",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.33, no. 4, pp. 671-686, April 2011, doi:10.1109/TPAMI.2010.87REFERENCES

- [1] S. Baker and T. Kanade, "Limits on Super-Resolution and How to Break Them,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1167-1183, Sept. 2002.- [2] R. Baraniuk, "Compressive Sensing,"
IEEE Signal Processing Magazine, vol. 24, no. 4, pp. 118-121, July 2007.- [3] R.G. Baraniuk, V. Cevher, M.F. Duarte, and C. Hegde, "Model-Based Compressive Sensing,"
IEEE Trans. Information Theory, to appear. - [4] B. Bascle, A. Blake, and A. Zisserman, "Motion Deblurring and Super-Resolution from an Image Sequence,"
Proc. Fourth European Conf. Computer Vision, vol. II, pp. 573-582, 1996.- [5] S. Belongie and J. Wills, "Structure from Periodic Motion,"
Proc. Workshop Spatial Coherence for Visual Motion Analysis, 2004.- [6] M. Ben-Ezra and S. Nayar, "Motion-Based Motion Deblurring,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 689-698, June 2004.- [7] I. Bilinskis and A. Mikelson,
Randomized Signal Processing. Prentice-Hall, Inc., 1992.- [8] E. Candes, "Compressive Sampling,"
Proc. Int'l Congress of Mathematicians, vol. 3, pp. 1433-1452, 2006.- [9] E. Candes and J. Romberg, "Sparsity and Incoherence in Compressive Sampling,"
Inverse Problems, vol. 23, no. 3, p. 969, 2007.- [10] E. Candes and T. Tao, "Decoding by Linear Programming,"
IEEE Trans. Information Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005.- [11] Checkline, "Industrial Stroboscopes," http://www.checkline. comstroboscopes/, 2008.
- [12] D. Donoho, "Compressed Sensing,"
IEEE Trans. Information Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.- [13] H. Edgerton, "Rapatronic Photographs," pp. 1951-1963, http://simplethinking.com/homerapatronic_photographs.htm , 2010.
- [14] W. Freeman, T. Jones, and E. Pasztor, "Example-Based Super-Resolution,"
IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 56-65, Mar./Apr. 2002.- [15] H. Greenspan, S. Peled, G. Oz, and N. Kiryati, "MRI Inter-Slice Reconstruction Using Super-Resolution,"
Proc. Fourth Int'l Conf. Medical Image Computing and Computer-Assisted Intervention, pp. 1204-1206, 2001.- [16] M.G.L. Gustafsson, "Nonlinear Structured-Illumination Microscopy: Wide-Field Fluorescence Imaging with Theoretically Unlimited Resolution,"
Proc. Nat'l Academy of Sciences USA, vol. 102, no. 37, pp. 13081-13086, 2005.- [17] M. Irani and S. Peleg, "Improving Resolution by Image Registration,"
CVGIP: Graphical Models Image Processing, vol. 53, no. 3, pp. 231-239, 1991.- [18] J. Jia, Y.-W. Tai, T.-P. Wu, and C.-K. Tang, "Video Repairing under Variable Illumination Using Cyclic Motions,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 5, pp. 832-839, May 2006.- [19] I. Laptev, S.J. Belongie, P. Perez, and J. Wills, "Periodic Motion Detection and Segmentation via Approximate Sequence Alignment,"
Proc. 10th IEEE Int'l Conf. Computer Vision, vol. 1, pp. 816-823, 2005.- [20] H. Larsson, S. Hertegård, and B. Hammarberg, "Vocal Fold Vibrations: High-Speed Imaging, Kymography, and Acoustic Analysis: A Preliminary Report,"
The Laryngoscope, vol. 110, no. 12, p. 2117, 2000.- [21] A. Levin, P. Sand, T. Cho, F. Durand, and W. Freeman, "Motion-Invariant Photography,"
Proc. ACM SIGGRAPH, vol. 8, 2008.- [22] Z. Lin and H.-Y. Shum, "Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 83-97, Jan. 2004.- [23] F. Marvasti,
Nonuniform Sampling: Theory and Practice. Kluwer Academic/Plenum Publishers, 2001.- [24] P. Mergell, H. Herzel, and I. Titze, "Irregular Vocal-Fold Vibration—High-Speed Observation and Modeling,"
J. Acoustical Soc. of Am., vol. 108, pp. 2996-3002, 2000.- [25] E. Muybridge,
Animals in Motion, first ed. Dover Publications, Chapman and Hall 1899, 1957.- [26] D. Needell and J. Tropp, "Cosamp: Iterative Signal Recovery from Incomplete and Inaccurate Samples,"
Applied and Computational Harmonic Analysis, vol. 26, no. 3, pp. 301-321, 2009.- [27] A.V. Oppenheim, R.W. Schafer, and J.R. Buck,
Discrete-Time Signal Processing. Prentice-Hall, 1999.- [28] Pointgrey Research, "PGR IEEE-1394 Digital Camera Register Reference," 2006.
- [29] R. Raskar, A. Agrawal, and J. Tumblin, "Coded Exposure Photography: Motion Deblurring Using Fluttered Shutter,"
ACM Trans. Graphics, vol. 25, no. 3, pp. 795-804, 2006.- [30] G. Schade, M. Hess, F. Müller, T. Kirchhoff, M. Ludwigs, R. Hillman, and J. Kobler, "Physical and Technical Elements of Short-Interval, Color-Filtered Double Strobe Flash-Stroboscopy,"
HNO, vol. 50, no. 12, pp. 1079-1083, Dec. 2002.- [31] S. Seitz and C. Dyer, "View-Invariant Analysis of Cyclic Motion,"
Int'l J. Computer Vision, vol. 25, no. 3, pp. 231-251, 1997.- [32] H.S. Shaw and D.D. Deliyski, "Mucosal Wave: A Normophonic Study across Visualization Techniques,"
J. Voice, vol. 22, no. 1, pp. 23-33, Jan. 2008.- [33] E. Shechtman, Y. Caspi, and M. Irani, "Increasing Space-Time Resolution in Video,"
Proc. Seventh European Conf. Computer Vision-Part I, pp. 753-768, 2002.- [34] C. Theobalt, I. Albrecht, J. Haber, M. Magnor, and H.-P. Seidel, "Pitching a Baseball: Tracking High-Speed Motion with Multi-Exposure Images,"
Proc. ACM SIGGRAPH, pp. 540-547, 2004.- [35] J.A. Tropp, J.N. Laska, M.F. Duarte, J.K. Romberg, and R.G. Baraniuk, "Beyond Nyquist: Efficient Sampling of Sparse Bandlimited Signals,"
IEEE Trans. Information Theory, vol. 56, no. 1, pp. 520-544, Jan. 2010.- [36] B. Wandell, P. Catrysse, J. DiCarlo, D. Yang, and A.E. Gamal, "Multiple Capture Single Image Architecture with a CMOS Sensor,"
Proc. Int'l Symp. Multispectral Imaging and Color Reproduction for Digital Archives, pp. 11-17, 1999.- [37] B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy, "High Performance Imaging Using Large Camera Arrays,"
Proc. ACM SIGGRAPH, pp. 765-776, 2005. |