Subscribe

Issue No.04 - April (2011 vol.17)

pp: 487-499

Pradeep Sen , University of New Mexico, Albuquerque

Soheil Darabi , University of New Mexico, Albuquerque

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TVCG.2010.46

ABSTRACT

Recently, there has been growing interest in compressed sensing (CS), the new theory that shows how a small set of linear measurements can be used to reconstruct a signal if it is sparse in a transform domain. Although CS has been applied to many problems in other fields, in computer graphics, it has only been used so far to accelerate the acquisition of light transport. In this paper, we propose a novel application of compressed sensing by using it to accelerate ray-traced rendering in a manner that exploits the sparsity of the final image in the wavelet basis. To do this, we raytrace only a subset of the pixel samples in the spatial domain and use a simple, greedy CS-based algorithm to estimate the wavelet transform of the image during rendering. Since the energy of the image is concentrated more compactly in the wavelet domain, less samples are required for a result of given quality than with conventional spatial-domain rendering. By taking the inverse wavelet transform of the result, we compute an accurate reconstruction of the desired final image. Our results show that our framework can achieve high-quality images with approximately 75 percent of the pixel samples using a nonadaptive sampling scheme. In addition, we also perform better than other algorithms that might be used to fill in the missing pixel data, such as interpolation or inpainting. Furthermore, since the algorithm works in image space, it is completely independent of scene complexity.

INDEX TERMS

Rendering, ray tracing, sampling and reconstruction, compressed sensing.

CITATION

Pradeep Sen, Soheil Darabi, "Compressive Rendering: A Rendering Application of Compressed Sensing",

*IEEE Transactions on Visualization & Computer Graphics*, vol.17, no. 4, pp. 487-499, April 2011, doi:10.1109/TVCG.2010.46REFERENCES

- [1] T. Whitted, "An Improved Illumination Model for Shaded Display,"
Comm. ACM, vol. 23, no. 6, pp. 343-349, 1980.- [2] R.L. Cook, L. Carpenter, and E. Catmull, "The REYES Image Rendering Architecture,"
SIGGRAPH Computer Graphics, vol. 21, no. 4, pp. 95-102, 1987.- [3] M. Segal and K. Akeley, "The OpenGL Graphics System: A Specification, v. 1.0," technical report, Silicon Graphics, 1992.
- [4] M. Bolin and G. Meyer, "A Frequency Based Ray Tracer,"
Proc. ACM SIGGRAPH '95, pp. 409-418, 1995.- [5] M. Pharr and G. Humphreys,
Physically Based Rendering: From Theory to Implementation. Morgan Kaufmann Publishers, Inc., 2004.- [6] J. Gu, S. Nayar, E. Grinspun, P. Belhumeur, and R. Ramamoorthi, "Compressive Structured Light for Recovering Inhomogeneous Participating Media,"
Proc. European Conf. Computer Vision (ECCV), Oct. 2008.- [7] P. Peers, D. Mahajan, B. Lamond, A. Ghosh, W. Matusik, R. Ramamoorthi, and P. Debevec, "Compressive Light Transport Sensing,"
ACM Trans. Graphics, vol. 28, no. 1, pp. 1-18, 2009.- [8] P. Sen and S. Darabi, "Compressive Dual Photography,"
Computer Graphics Forum, vol. 28, no. 2, pp. 609-618, 2009.- [9] H. Feichtinger, K. Gröchenig, and T. Strohmer, "Efficient Numerical Methods in Non-Uniform Sampling Theory,"
Numerical Math., vol. 69, pp. 423-440, 1995.- [10] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, "Image Inpainting,"
Proc. ACM SIGGRAPH '00, pp. 417-424, 2000.- [11] P. Schröder, "Wavelets in Computer Graphics,"
Proc. IEEE, vol. 84, no. 4, pp. 615-625, Apr. 1996.- [12] P. Schröder and W. Sweldens, "Wavelets in Computer Graphics,"
Proc. ACM SIGGRAPH '96, no. 13, 1996.- [13] E.J. Stollnitz, T.D. Derose, and D.H. Salesin,
Wavelets for Computer Graphics: Theory and Applications. Morgan Kaufmann Publishers, Inc., 1996.- [14] S. Mallat,
A Wavelet Tour of Signal Processing, second ed. Academic Press, 1999.- [15] P. Hanrahan, D. Salzman, and L. Aupperle, "A Rapid Hierarchical Radiosity Algorithm,"
SIGGRAPH Computer Graphics, vol. 25, no. 4, pp. 197-206, 1991.- [16] S.J. Gortler, P. Schröder, M.F. Cohen, and P. Hanrahan, "Wavelet Radiosity,"
Proc. ACM SIGGRAPH '93, pp. 221-230, 1993.- [17] D. Lischinski, F. Tampieri, and D.P. Greenberg, "Combining Hierarchical Radiosity and Discontinuity Meshing,"
Proc. ACM SIGGRAPH, pp. 199-208, 1993.- [18] P. Schröder, S.J. Gortler, M.F. Cohen, and P. Hanrahan, "Wavelet Projections for Radiosity,"
Computer Graphics Forum, vol. 13, no. 2, pp. 141-151, 1994.- [19] R. Ramamoorthi and P. Hanrahan, "An Efficient Representation for Irradiance Environment Maps,"
Proc. ACM SIGGRAPH, 2001.- [20] P.-P. Sloan, J. Kautz, and J. Snyder, "Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments,"
Proc. ACM SIGGRAPH, pp. 527-536, 2002.- [21] R. Ng, R. Ramamoorthi, and P. Hanrahan, "All-Frequency Shadows Using Non-Linear Wavelet Lighting Approximation,"
ACM Trans. Graphics, vol. 22, no. 3, pp. 376-381, 2003.- [22] R. Ng, R. Ramamoorthi, and P. Hanrahan, "Triple Product Wavelet Integrals for All-Frequency Relighting,"
Proc. ACM SIGGRAPH, pp. 477-487, 2004.- [23] T. Malzbender, "Fourier Volume Rendering,"
ACM Trans. Graphics, vol. 12, no. 3, pp. 233-250, 1993.- [24] T. Totsuka and M. Levoy, "Frequency Domain Volume Rendering,"
Proc. ACM SIGGRAPH, pp. 271-278, 1993.- [25] M.H. Gross, L. Lippert, R. Dittrich, and S. Häring, "Two Methods for Wavelet-Based Volume Rendering,"
Computers and Graphics, vol. 21, no. 2, pp. 237-252, 1997.- [26] D.P. Mitchell, "Generating Antialiased Images at Low Sampling Densities,"
Proc. ACM SIGGRAPH, pp. 65-72, 1987.- [27] B. Walter, G. Drettakis, and S. Parker, "Interactive Rendering Using the Render Cache,"
Proc. 10th Eurographics Workshop Rendering, D. Lischinski and G. Larson, eds., pp. 235-246, June 1999.- [28] P. Tole, F. Pellacini, B. Walter, and D. Greenberg, "Interactive Global Illumination in Dynamic Scenes,"
ACM Trans. Graphics, vol. 21, no. 3, pp. 537-546, 2002.- [29] F. Pighin, D. Lischinski, and D. Salesin, "Progressive Previewing of Ray-Traced Images Using Image Plane Disconinuity Meshing,"
Proc. Eurographics Workshop Rendering '97, pp. 115-125, 1997.- [30] B. Guo, "Progressive Radiance Evaluation Using Directional Coherence Maps,"
Proc. ACM SIGGRAPH '98, pp. 255-266, 1998.- [31] K. Bala, B. Walter, and D.P. Greenberg, "Combining Edges and Points for Interactive High-Quality Rendering,"
ACM Trans. Graphics, vol. 22, no. 3, pp. 631-640, 2003.- [32] P. Sen, M. Cammarano, and P. Hanrahan, "Shadow Silhouette Maps,"
ACM Trans. Graphics, vol. 22, no. 3, pp. 521-526, 2003.- [33] P. Sen, "Silhouette Maps for Improved Texture Magnification,"
Proc. ACM SIGGRAPH/EUROGRAPHICS Conf. Graphics Hardware (HWWS '04), pp. 65-73, 2004.- [34] T. Hachisuka, W. Jarosz, R.P. Weistroffer, K. Dale, G. Humphreys, M. Zwicker, and H.W. Jensen, "Multidimensional Adaptive Sampling and Reconstruction for Ray Tracings,"
ACM Trans. Graphics, vol. 27, no. 3, pp. 1-10, 2008.- [35] S. Masnou and J.-M. Morel, "Level Lines Based Disocclusion,"
Proc. IEEE Int'l Conf. Image Processing (ICIP), pp. 259-263, Oct. 1998.- [36] F. Marvasti,
Nonuniform Sampling: Theory and Practice. Kluwer Academic Publishers, 2001.- [37] F. Marvasti, C. Liu, and G. Adams, "Analysis and Recovery of Multidimensional Signals from Irregular Samples Using Nonlinear and Iterative Techniques,"
Signal Processing, vol. 36, no. 1, pp. 13-30, 1994.- [38] E.J. Candès, J. Romberg, and T. Tao, "Robust Uncertainty Principles: Exact Signal Reconstruction from Highly Incomplete Frequency Information,"
IEEE Trans. Information Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006.- [39] D.L. Donoho, "Compressed Sensing,"
IEEE Trans. Information Theory, vol. 52, no. 4, pp. 1289-1306, Apr. 2006.- [40] "Rice University Compressive Sensing Resources Website," http://www.dsp.ece.rice.educs/, 2009.
- [41] E.J. Candès, M. Rudelson, T. Tao, and R. Vershynin, "Error Correction via Linear Programming,"
Proc. IEEE Symp. Foundations of Computer Science, pp. 295-308, 2005.- [42] J.A. Tropp and A.C. Gilbert, "Signal Recovery from Random Measurements via Orthogonal Matching Pursuit,"
IEEE Trans. Information Theory, vol. 53, no. 12, pp. 4655-4666, Dec. 2007.- [43] D. Needell and R. Vershynin, "Uniform Uncertainty Principle and Signal Recovery via Regularized Orthogonal Matching Pursuit,"
Foundations of Computational Math., vol. 9, no. 3, pp. 317-334, June 2009.- [44] E.J. Candès and T. Tao, "Near Optimal Signal Recovery from Random Projections: Universal Encoding Strategies?"
IEEE Trans. Information Theory, vol. 52, no. 12, pp. 5406-5425, Dec. 2006.- [45] D.L. Donoho and X. Huo, "Uncertainty Principles and Ideal Atomic Decomposition,"
IEEE Trans. Information Theory, vol. 47, no. 7, pp. 2845-2862, Nov. 2001.- [46] R.C. Gonzalez and R.E. Woods,
Digital Image Processing. Addison-Wesley Longman Publishing Co., Inc., 2001.- [47] R.L. Cook, "Stochastic Sampling in Computer Graphics,"
ACM Trans. Graphics, vol. 5, no. 1, pp. 51-72, 1986.- [48] "LuxRender," http:/www.luxrender.net/, 2010.
- [49] D. Dunbar and G. Humphreys, "A Spatial Data Structure for Fast Poisson-Disk Sample Generation,"
ACM Trans. Graphics, vol. 25, no. 3, pp. 503-508, 2006.- [50] "Intel Math Kernel Library," http:/www.intel.com/, 2009.
- [51] "Stanford Systems Optimization Laboratory," http://www. stanford.edu/group/SOL/ softwarelsqr.html, 2009.
- [52] Y. Tsaig and D.L. Donoho, "Extensions of Compressed Sensing,"
Signal Processing, vol. 86, no. 3, pp. 549-571, 2006.- [53] D.S. Taubman and M.W. Marcellin,
JPEG 2000: Image Compression Fundamentals, Standards and Practice. Springer, 2001.- [54] E. Alper and S. Mavinkurve, "Image Inpainting," http://www.eecs.harvard.edu/sanjayinpainting /, 2002.
- [55] J. Painter and K. Sloan, "Antialiased Ray Tracing by Adaptive Progressive Refinement,"
SIGGRAPH Computer Graphics, vol. 23, no. 3, pp. 281-288, 1989.- [56]
Cloudy with a Chance of Meatballs, Sony Pictures Animation, http:/www.cloudywithachanceofmeatballs.com /, 2009. |