The Community for Technology Leaders
RSS Icon
Issue No.12 - Dec. (2013 vol.35)
pp: 2982-2996
Sanjeev J. Koppal , Harvard Univ., Cambridge, MA, USA
Ioannis Gkioulekas , Harvard Univ., Cambridge, MA, USA
Hyunsung Park , Harvard Univ., Cambridge, MA, USA
Kenneth B. Crozier , Harvard Univ., Cambridge, MA, USA
Todd Zickler , Harvard Univ., Cambridge, MA, USA
Achieving computer vision on microscale devices is a challenge. On these platforms, the power and mass constraints are severe enough for even the most common computations (matrix manipulations, convolution, etc.) to be difficult. This paper proposes and analyzes a class of miniature vision sensors that can help overcome these constraints. These sensors reduce power requirements through template-based optical convolution, and they enable a wide field-of-view within a small form through a refractive optical design. We describe the tradeoffs between the field-of-view, volume, and mass of these sensors and we provide analytic tools to navigate the design space. We demonstrate milliscale prototypes for computer vision tasks such as locating edges, tracking targets, and detecting faces. Finally, we utilize photolithographic fabrication tools to further miniaturize the optical designs and demonstrate fiducial detection onboard a small autonomous air vehicle.
Optical sensors, Optical imaging, Optical refraction, Photodetectors, Optical attenuators,micro/nano robotics, Computational sensors, micro/nano computer vision, optical templates, optical computing
Sanjeev J. Koppal, Ioannis Gkioulekas, Travis Young, Hyunsung Park, Kenneth B. Crozier, Geoffrey L. Barrows, Todd Zickler, "Toward Wide-Angle Microvision Sensors", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 12, pp. 2982-2996, Dec. 2013, doi:10.1109/TPAMI.2013.22
[1] Zeemax Optical Software, http:/, 2010.
[2] F. Ababsa and M. Mallem, "A Robust Circular Fiducial Detection Technique and Real-Time 3D Camera Tracking," J. Multimedia, vol. 3, pp. 34-41, 2008.
[3] E. Altug, J. Ostrowski, and R. Mahony, "Control of a Quadrotor Helicopter Using Visual Feedback," Proc. IEEE Int'l Conf. Robotics Automation, 2002.
[4] Arduino and Pro, "Arduino Website," http:/, 2012.
[5] S. Avidan, "Support Vector Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[6] H. Bagherinia and R. Manduchi, "A Theory of Color Barcodes," Proc. IEEE Color and Photometry in Computer Vision Workshop, 2011.
[7] G. Barrows, J. Chahl, and Y. Srinivasan, "Biomimetic Visual Sensing and Flight Control," Proc. Bristol UAV Conf., 2002.
[8] M.J. Black and A.D. Jepson, "EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation," Int'l J. Computer Vision, vol. 26, pp. 63-84, 1998.
[9] N. Borrelli, Microoptics Technology: Fabrication and Applications of Lens Arrays and Devices. CRC Press, 1999.
[10] V. Brajovic and T. Kanade, "Computational Sensor for Visual Tracking with Attention," IEEE J. Solid State Circuits, vol. 33, no. 8, pp. 1199-1207, Aug. 1998.
[11] CentEye and Inc, "Centeye Website," http:/, 2012.
[12] A. Chandrakasan, N. Verma, J. Kwong, D. Daly, N. Ickes, D. Finchelstein, and B. Calhoun, "Micropower Wireless Sensors," Proc. NSTI Nanotechnology Conf., 2006.
[13] V. Chari and P. Sturm, "Multi-View Geometry of the Refractive Plane," Proc. British Machine Vision Conf., 2009.
[14] Y. Cho and U. Neumann, "Multi-Ring Color Fiducial Systems for Scalable Fiducial Tracking Augmented Reality," Proc. IEEE Virtual Reality Ann. Int'l Symp., 1998.
[15] D. Claus and A. Fitzgibbon, "Visual Marker Detection and Decoding in AR Systems: A Comparative Study," Proc. Int'l Symp. Mixed Augmented Reality, 2002.
[16] Collection, Flying Insects and Robots. Springer, 2009.
[17] O. Cossairt, D. Miau, and S. Nayar, "A Scaling Law for Computational Imaging with Spherical Optics," J. Optical Soc. Am. A, vol. 28, pp. 2540-2553, 2011.
[18] M. Duarte, M. Davenport, D. Takhar, J. Laska, T. Sun, K. Kelly, and R. Baraniuk, "Single-Pixel Imaging via Compressive Sampling," IEEE Signal Processing Magazine, vol. 25, no. 2, pp. 83-91, Mar. 2008.
[19] M. Edge and I. Turner, The Underwater Photographer. Focal Press, 1999.
[20] J.H. Elder and S.W. Zucker, "Local Scale Control for Edge Detection and Blur Estimation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 7, pp. 699-716, July 1998.
[21] C. Farabet, C. Poulet, and Y. LeCun, "An FPGA-Based Stream Processor for Embedded Real-Time Vision with Convolutional Networks," Proc. Fifth IEEE Workshop Embedded Computer Vision, 2009.
[22] A. Fluegel http:/, 2007.
[23] I. Gkioulekas and T. Zickler, "Dimensionality Reduction Using the Sparse Linear Model," Proc. Neural Information Processing Systems, 2011.
[24] J.W. Goodman, Introduction to Fourier Optics. McGraw-Hill, 1968.
[25] B. Gyselinckx, C. Van Hoof, J. Ryckaert, R. Yazicioglu, P. Fiorini, and V. Leonov, "Human++: Autonomous Wireless Sensors for Body Area Networks," Proc. IEEE Custom Integrated Circuits Conf., pp. 13-19, 2006.
[26] H.P. Herzig, Micro-Optics: Elements, Systems and Applications. Taylor and Francis, 1999.
[27] S. Hiura, A. Mohan, and R. Raskar, "Krill-Eye: Superposition Compound Eye for Wide-Angle Imaging Via Grin Lenses," Proc. Workshop Omni Directional Vision, 2009.
[28] K. Jeong, J. Kim, and L. Lee, "Biologically Inspired Artificial Compound Eyes," Science, vol. 312, pp. 557-561, 2006.
[29] M. Karpelson, G. Wei, and R.J. Wood, "Milligram-Scale High-Voltage Power Electronics for Piezoelectric Microrobots," Proc. IEEE Int'l Conf. Robotics Automation, 2009.
[30] H. Kato and M. Billinghurst, "Marker Tracking and hmd Calibration for a Video-Based Augmented Reality Conferencing System," Proc. Second IEEE ACM Int'l Workshop Augmented Reality, 1999.
[31] H. Ko, G. Shin, S. Wang, M. Stoykovich, J. Lee, D. Kim, J. Ha, Y. Huang, K. Hwang, and J. Rogers, "Curvilinear Electronics Formed Using Silicon Membrane Circuits and Elastomeric Transfer Elements," Small, vol. 5, pp. 2703-2709, 2009.
[32] S.J. Koppal, "Toward Micro Vision Sensors Website," http://www.koppal.commicrovisionsensors.html , 2012.
[33] G. Krishnan and S.K. Nayar, "Towards a True Spherical Camera," Proc. SPIE, 2009.
[34] N. Kumar, A.C. Berg, P. Belhumeur, and S.K. Nayar, "Attribute and Simile Classifiers for Face Verification," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[35] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient Based Learning Applied to Document Recognition," Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998.
[36] A. Levin, R. Fergus, and B. Freeman, "Image and Depth from a Conventional Camera with a Coded Aperture," Proc. ACM Siggraph, 2007.
[37] D. Marr and E. Hildreth, "Theory of Edge Detection," Proc. Royal Soc. London, vol. 207, pp. 187-217, 1980.
[38] K. Mielenz, "On the Diffraction Limit for Lensless Imaging," J. Research of the NIST, vol. 104, pp. 479-485, 1999.
[39] K. Miyamoto, "Fish Eye Lens," J. Optical Soc. Am., vol. 54, pp. 1060-1061, 1964.
[40] A. Mohan, G. Woo, S. Hiura, Q. Smithwick, and R. Raskar, "Bokode: Imperceptible Visual Tags for Camera Based Interaction from a Distance," Proc. ACM Siggraph, 2009.
[41] S.K. Nayar, V. Branzoi, and T.E. Boult, "Programmable Imaging: Towards a Flexible Camera," Int'l J. Computer Vision, vol. 70, pp. 7-22, 2006.
[42] R. Ng, "Fourier Slice Photography," ACM Trans. Graphics, vol. 24, pp. 735-744, 2005.
[43] E. Olson, "AprilTag: A Robust and Flexible Visual Fiducial System," Proc. IEEE Int'l Conf. Robotics and Automation, 2011.
[44] M. O'Toole and K. Kutulakos, "Optical Computing for Fast Light Transport Analysis," Pro. ACM Siggraph Asia, 2010.
[45] F. Pedrotti and L. Pedrotti, Introduction to Optics. Benjamin Cummings, 2006.
[46] C. Raghavendra, K. Sivalingam, and T. Znati, Wireless Sensor Networks. Springer, 2004.
[47] J. Rekimoto and Y. Ayatsuka, "CyberCode: Designing Augmented Reality Environments with Visual Tags," Proc. ACM Designing Augmented Reality Environments, 2000.
[48] J. Sattar, E. Bourque, P. Giguere, and G. Dudek, "Fourier Tags: Smoothly Degradable Fiducial Markers for Use in Human-Robot Interaction," Proc. Fourth Canadian Conf. Computer and Robot Vision, 2007.
[49] S. Shalev-Shwartz, Y. Wexler, and A. Shashua, "Shareboost: Efficient Multiclass Learning with Feature Sharing," Proc. Neural Information Processing Systems, 2011.
[50] E. Steltz and R. Fearing, "Dynamometer Power Output Measurements of Miniature Piezoelectric Actuators," IEEE/ACM Trans. Mechatronics, vol. 14, no. 1, pp. 1-10, Feb. 2009.
[51] R. Swaminathan, M. Grossberg, and S. Nayar, "Caustics of Catadioptric Cameras," Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001.
[52] J. Tanida, T. Kumagai, K. Yamada, S. Miyatake, K. Ishida, T. Morimoto, N. Kondou, D. Miyazaki, and Y. Ichioka, "Thin Observation Module by Bound Optics (TOMBO): Concept and Experimental Verification," Applied Optics, vol. 40, pp. 1806-1813, 2001.
[53] A. Torralba, K. Murphy, and W. Freeman, "Sharing Visual Features for Multiclass and Multiview Object Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 854-869, May 2007.
[54] T. Treibitz, Y. Schechner, and H. Singh, "Flat Refractive Geometry," Proc. IEEE Conf. Computer Vision Pattern Recognition, 2008.
[55] S. Uttam, N. Goodman, M. Neifeld, C. Kim, R. John, J. Kim, and D. Brady, "Optically Multiplexed Imaging with Superposition Space Tracking," Optics Express, vol. 17, pp. 1691-1713, 2009.
[56] A. Veeraraghavan, R. Raskar, A. Agrawal, A. Mohan, and J. Tumblin, "Dappled Photography: Mask Enhanced Cameras for Heterodyned Light Fields and Coded Aperture Refocusing," Proc. ACM Siggraph, 2007.
[57] P.A. Viola and M.J. Jones, "Robust Real-Time Face Detection," Int'l J. Computer Vision, vol. 57, pp. 137-154, 2004.
[58] R. Volker, M. Eisner, and K. Weible, "Miniaturized Imaging Systems," Microelectronic Eng., vol. 67/68, pp. 461-472, 2003.
[59] A. Wilhelm, B. Surgenor, and J. Pharoah, "Evaluation of a Micro Fuel Cell as Applied to a Mobile Robot," Proc. IEEE Int'l Conf. Mechatronics Automation, 2005.
[60] W. Wolf, B. Ozer, and T. Lv, "Smart Cameras as Embedded Systems," Computer, vol. 35, no. 9, pp. 48-53, Sept. 2002.
[61] R.W. Wood, Physical Optics. Macmillan, 1911.
[62] F. Yu and S. Jutamulia, Optical Pattern Recognition. Cambridge Univ. Press, 1998.
[63] X. Zhang, S. Fronz, and N. Navab, "Visual Marker Detection and Decoding in AR Systems: A Comparative Study," Proc. First Int'l Symp. Mixed and Augmented Reality, 2002.
[64] A. Zomet and S. Nayar, "Lensless Imaging with a Controllable Aperture," Proc. IEEE Conf. Computer Vision Pattern Recognition, 2006.
88 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool