This Article 
 Bibliographic References 
 Add to: 
Penrose Pixels for Super-Resolution
July 2011 (vol. 33 no. 7)
pp. 1370-1383
Moshe Ben-Ezra, Microsoft Research Asia, Beijing
Zhouchen Lin, Microsoft Research Asia, Beijing
Bennett Wilburn, Refocus Imaging
Wei Zhang, The Chinese University of Hong Kong, Hong Kong
We present a novel approach to reconstruction-based super-resolution that uses aperiodic pixel tilings, such as a Penrose tiling or a biological retina, for improved performance. To this aim, we develop a new variant of the well-known error back projection super-resolution algorithm that makes use of the exact detector model in its back projection operator for better accuracy. Pixels in our model can vary in shape and size, and there may be gaps between adjacent pixels. The algorithm applies equally well to periodic or aperiodic pixel tilings. We present analysis and extensive tests using synthetic and real images to show that our approach using aperiodic layouts substantially outperforms existing reconstruction-based algorithms for regular pixel arrays. We close with a discussion of the feasibility of manufacturing CMOS or CCD chips with pixels arranged in Penrose tilings.

[1] Hexagonal Image Processing Survey, HexagonLiterature.html, 2011.
[2] Omnivision Technologies, http:/, 2011.
[3] STMicroelectronics and Soitec Join Forces to Develop Next-Generation Technology for CMOS Image Sensors, news/year2009t2379.htm, 2009.
[4], 2011.
[5] index.html, 2011.
[6] , 2011.
[7] S. Baker and T. Kanade, "Limits on Super-Resolution and How to Break Them," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp. 1167-1183, Sept. 2002.
[8] D.F. Barbe, Charge-Coupled Devices. Springer-Verlag, 1980.
[9] M. Ben-Ezra, Z. Lin, and B. Wilburn, "Penrose Pixels: Super-Resolution in the Detector Layout Domain," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.
[10] M. Ben-Ezra, A. Zomet, and S. Nayar, "Video Super-Resolution Using Controlled Subpixel Detector Shifts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 977-987, June 2005.
[11] S. Borman and R. Stevenson, "Spatial Resolution Enhancement of Low-Resolution Image Sequences: A Comprehensive Review with Directions for Future Research," technical report, Univ. of Notre Dame, 1998.
[12] T. Chen, P. Catrysse, A.E. Gamal, and B. Wandell, "How Small Should Pixel Size Be?" Proc. SPIE, pp. 451-459, 2000.
[13] D. Daly, Microlens Arrays. Taylor and Francis Inc., 2001.
[14] M. Elad and A. Feuer, "Restoration of Single Super-Resolution Image from Several Blurred, Noisy and Down-Sampled Measured Images," IEEE Trans. Image Processing, vol. 6, no. 12, pp. 1646-1658, Dec. 1997.
[15] S. Farsiu, D. Robinson, M. Elad, and P. Milanfar, "Advances and Challenges in Super-Resolution," Int'l J. Imaging Systems and Technology, vol. 14, no. 2, pp. 47-57, 2004.
[16] R. Fergus, A. Torralba, and W. Freeman, "Random Lens Imaging," MIT Computer Science and Artificial Intelligence Laboratory TR, vol. 58, p. 1, 2006.
[17] W. Freeman and E. Pasztor, "Learning Low-Level Vision," Proc. IEEE Int'l Conf. Computer Vision, pp. 1182-1189, 1999.
[18] G.H. Golub and C.F.V. Loan, Matrix Computations, third ed. The John Hopkins Univ. Press, 1996.
[19] B. Grunbaum and G. Shephard, Tilings and Patterns. Freeman, 1987.
[20] R. Hardie, "A Fast Image Super-Resolution Algorithm Using an Adaptive Wiener Filter," IEEE Trans. Image Processing, vol. 16, no. 12, pp. 2953-2964, Dec. 2007.
[21] R. Hardie, K. Barnard, and E. Amstrong, "Joint Map Registration and High-Resolution Image Estimation Using a Sequence of Undersampled Images," IEEE Trans. Image Processing, vol. 6, no. 12, pp. 1621-1633, Dec. 1997.
[22] R.A. Horn and C.R. Johnson, Matrix Analysis, vol. 1. Cambridge Univ. Press, 1985.
[23] M. Irani and S. Peleg, "Improving Resolution by Image Restoration," Computer Vision, Graphics, and Image Processing, vol. 53, pp. 231-239, 1991.
[24] C. Lin, H. Yang, and C. Chao, "Hexagonal Microlens Array Modeling and Fabrication Using a Thermal Reflow Process," J. Micromechanics and Microeng., vol. 107, pp. 775-781, 2003.
[25] Z. Lin and H.-Y. Shum, "Fundamental Limits of Reconstruction-Based Superresolution Algorithms under Local Translation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 83-97, Jan. 2004.
[26] L. Middleton and J. Sivaswamy, "Edge Detection in a Hexagonal-Image Processing Framework," Image and Vision Computing, vol. 19, no. 14, pp. 1071-1081, 2001.
[27] S. Park, M. Park, and M. Kang, "Super-Resolution Image Reconstruction: A Technical Overview," IEEE Signal Processing Magazine, vol. 20, no. 3, pp. 21-36, May 2003.
[28] A.J. Patti, M.I. Sezan, and A.M. Tekalp, "Superresolution Video Reconstruction with Arbitrary Sampling Lattices and Nonzero Aperture Time," IEEE Trans. Image Processing, vol. 6, no. 8, pp. 1064-1076, Aug. 1997.
[29] J.B. Pendry, "Negative Refraction Makes a Perfect Lens," Physical Rev. Letters, vol. 18, no. 85, pp. 3966-3969, 2000.
[30] L. Pickup, D. Capel, S. Roberts, and A. Zisserman, "Bayesian Image Super-Resolution, Continued," Advances in Neural Information Processing Systems, vol. 19, pp. 1089-1096, 2007.
[31] I. Shcherback and O. Yadid-Pecht, "Photoresponse Analysis and Pixel Shape Optimization for CMOS Active Pixel Sensors," IEEE Trans. Electron Devices, vol. 50, no. 1, pp. 12-18, Jan. 2003.
[32] T. Sibley and S. Wagon, "Rhombic Penrose Tilings Can Be 3-Colored," The Am. Math. Monthly, vol. 107, no. 3, pp. 251-252, 2000.
[33] D. Takhar, J. Laska, M. Wakin, M. Duarte, D. Baron, S. Sarvotham, K. Kelly, and R. Baraniuk, "A New Compressive Imaging Camera Architecture Using Optical-Domain Compression," Proc. IS&T/SPIE Computational Imaging IV, 2006.
[34] M. Tappen, B. Russell, and W. Freeman, "Exploiting the Sparse Derivative Prior for Super-Resolution and Image Demosaicing," Proc. Third Int'l Workshop Statistical and Computational Theories of Vision, 2003.
[35] W.-Y. Zhao and H.S. Sawhney, "Is Super-Resolution with Optical Flow Feasible," Proc. European Conf. Computer Vision, vol. 1, pp. 599-613, 2002.

Index Terms:
Super-resolution, Penrose tiling, CMOS sensor, CCD sensor.
Moshe Ben-Ezra, Zhouchen Lin, Bennett Wilburn, Wei Zhang, "Penrose Pixels for Super-Resolution," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 33, no. 7, pp. 1370-1383, July 2011, doi:10.1109/TPAMI.2010.213
Usage of this product signifies your acceptance of the Terms of Use.