This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Energy Aware Computing through Probabilistic Switching: A Study of Limits
September 2005 (vol. 54 no. 9)
pp. 1123-1137
The main result in this paper establishes the energy savings derived by using probabilistic AND as well as NOT gates constructed from an idealized switch that produces a probabilistic bit (pbit). A probabilistic switch produces the desired value as an output that is 0 or 1 with probability p, represented as a pbit, and, hence, can produce the wrong output value with a probability of (1-p). In contrast with a probabilistic switch, a conventional deterministic switch produces a bit whose value is always correct. Our switch-based gate constructions are a particular case of a systematic methodology developed here for building energy-aware networks for computing, using pbits. Interesting examples of such networks include AND, OR, and NOT gates (or, as functions, Boolean conjunction, disjunction, and negation, respectively). To quantify the energy savings, novel measures of "technology independent” energy complexity are also introduced here—these measures parallel conventional machine-independent notions of computational complexity such as the algorithm's running time and space. Networks of switches can be related to Turing machines and to Boolean circuits, both of which are widely known and well-understood models of computation. Our gate and network constructions lend substance to the following thesis (established for the first time by this author [1], [2], [3]): The mathematical technique referred to as randomization yielding probabilistic algorithms results in energy savings through a physical interpretation based on statistical thermodynamics and, hence, can serve as a basis for energy-aware computing. While the estimates of the energy saved through pbit--based probabilistic computing switches and networks developed here rely on the constructs and thermodynamic models due to Boltzmann, Gibbs, and Planck, this work has also led to the innovation of probabilistic CMOS-based devices and computing frameworks. Thus, for completeness, the relationship between the physical models on which this work is based and the electrical domain of CMOS-based switching will be discussed.

[1] K.V. Palem, “Thermodynamics of Randomized Computing, a Discipline for Energy Aware Algorithm Design and Analysis,” Technical Report GIT-CC-02-56, Georgia Inst. of Tech nology, Nov. 2002.
[2] K.V. Palem, “Energy Aware Computation: From Algorithms and Thermodynamics to Randomized (Semiconductor) Devices,” Technical Report GIT-CC-03-10, Georgia Inst. of Tech nology, Feb. 2003.
[3] K.V. Palem, “Energy Aware Computing through Randomized Switching,” Technical Report GIT-CC-03-16, Georgia Inst. of Tech nology, May 2003.
[4] J.T. Schwartz, “Fast Probabilistic Algorithms for Verification of Polynomial Identities,” J. ACM, vol. 27, pp. 701-717, 1980.
[5] M.O. Rabin, “Probabilistic Algorithms,” Algorithms and Complexity, New Directions and Recent Trends, J.F. Traub, ed., pp. 29-39, 1976.
[6] G.J. Chaitin and J.T. Schwartz, “A Note on Monte Carlo Primality Tests and Algorithmic Information Theory,” Comm. Pure and Applied Math., vol. 31, pp. 521-527, 1978.
[7] R. Balian, From Microphysics to Macrophysics, vols. 1, 2. Springer-Verlag, 1991.
[8] U.V. Vazirani and V.V. Vazirani, “Efficient and Secure Pseudo-Random Number Generation (Extended Abstract),” Proc. Ann. Symp. Foundations of Computer Science, pp. 458-463, 1984.
[9] L. Szilard, “On the Decrease of Entropy in a Thermodynamic System by the Intervention of Intelligent Beings,” Maxwell's Demon: Why Warmth Disperses and Time Passes, H. Leff and E. Rex, 1998.
[10] N. Pippenger and M.J. Fischer, “Relations among Complexity Measures,” J. ACM, vol. 26, pp. 361-381, Apr. 1979.
[11] N. Pippenger, “On Simultaneous Resource Bounds (Preliminary Version),” Proc. 20th Ann. Symp. Foundations of Computer Science, pp. 307-311, 1979.
[12] C.H. Papadimitriou, Computational Complexity. Addison-Wesley, 1994.
[13] M. Sipser, Introduction to the Theory of Computation. PWS Publishing Company, 1997.
[14] A.A. Razborov, “A Lower Bound on the Monote Complexity of the Logical Permanent,” Mat. Zametki, vol. 37, no. 6,English translation in Math. Notes of the Academy of Sciences of the USSR, 1985.
[15] A.A. Razborov, “Lower Bounds on the Monote Complexity of Some Boolean Functions,” Dokl. Akad. Nauk SSSR, English translation in Soviet Math. Dokl., vol. 31, pp. 798-801, 1985.
[16] K. Palem, “Proof as Experiment: Computation from a Thermodynamic Perspective,” Proc. Int'l Symp. Verification: Theory and Practice, June 2003.
[17] S. Carnot, Reflections on the Motive Power of Fire (Reflexions sur la Puissance Motrice du Feu et sur les Machines Propres a Developper Cette Puissance), 1824.
[18] R. Clausius, “Uber die Bewegende Kraft der Warme,” Annalen der Physik, 1850.
[19] J.C. Maxwell, “Illustrations of the Dynamical Theory of Gases,” Phil. Mag., vol. 19, pp. 19-32, 1860.
[20] J.C. Maxwell, “On the Dynamical Theory of Gases,” Philosophical Trans. Royal Soc. London, vol. 157, pp. 49-88, 1867.
[21] L. Boltzmann and S.G. Brush, Lectures on Gas Theory, English translation by S.G. Brush. Dover Publications, 1995.
[22] J.W. Gibbs, Elementary Principles in Statistical Mechanics. New York: Scribner, 1902.
[23] M. Planck, Treatise on Thermodynamics. Dover Publications, 1922.
[24] J. von Neumann, Fourth University of Illinois Lecture in Theory of Self-Reproducing Automata, A. W. Burks, ed. Univ. of Illinois Press, 1966.
[25] R. Landauer, “Irreversibility and Heat Generation in the Computing Process,” IBM J. Research and Development, vol. 3, pp. 183-191, July 1961.
[26] C.H. Bennett, “Logical Reversibility of Computation,” IBM J. Research and Development, vol. 17, pp. 525-532, Nov. 1973.
[27] E. Fredkin and T. Toffoli, “Conservative Logic,” Proc. Physics of Computation Conf., pp. 219-253, 1982.
[28] J.D. Meindl, “Low Power Microelectronics: Retrospect and Prospect,” Proc. IEEE, pp. 619-635, Apr. 1995.
[29] J.D. Meindl and J.A. Davis, “The Fundamental Limit on Binary Switching Energy for Terescale Integration (TSI),” IEEE. J. Solid-State Circuits, pp. 1515-1516, Oct. 2000.
[30] K.-U. Stein, “Noise-Induced Error Rate as Limiting Factor for Energy per Operation in Digital ICS,” IEEE J. Solid-State Circuits, vol. 31, no. 5, 1977.
[31] M.O. Rabin and D.S. Scott, “Finite Automata and Their Decision Problems,” IBM J. Research and Development, vol. 3, no. 2, pp. 115-125, 1959.
[32] M.O. Rabin, “Probabilistic Automata,” Information and Control, vol. 6, pp. 230-245, 1963.
[33] R.M. Karp, “Probabilistic Analysis of Partitioning Algorithms for the Traveling-Salesman Problem in the Plane,” Math. Operations Research, vol. 2, no. 3, pp. 209-224, Aug. 1977.
[34] J. Gill, “Computational Complexity of Probabilistic Turing Machines,” SIAM J. Computing, vol. 6, no. 4, pp. 675-695, 1977.
[35] R. Feynman, Feynman Lectures on Computation. Addison-Wesley, 1996.
[36] Z. Kohavi, Switching and Finite Automata Theory. New York: McGraw-Hill, 1970.
[37] Z. Manna, Mathematical Theory of Computation. New York: McGraw-Hill, 1974.
[38] K. Palem, S. Cheemalavagu, and P. Korkmaz, “The Physical Representation of Probabilistic Bits (pbits) and the Energy Consumption of Randomized Switching,” CREST technical report, June 2003.
[39] C. Cheemalavagu, P. Korkmaz, and K. Palem, “Ultra Low-Energy Computing via Probabilistic Algorithms and Devices: CMOS Device Primitives and the Energy-Probability Relationship,” Proc. 2004 Int'l Conf. Solid State Devices and Materials, Sept. 2004.
[40] K.V. Palem, L.N. Chakrapani, B.E.S. Akgul, P. Korkmaz, and B. Seshasayee, “Ultra Energy-Performance Efficient Embedded SoC Architectures Based on Probabilistic CMOS (PCMOS),” Int'l Conf. Computers, Architectures, and Synthesis for Embedded Systems (CASES) 2005, submitted for publication.
[41] A. Sinha and A. Chandrakasan, “Jouletrack: A Web Based Tool for Software Energy Profiling,” Proc. 38th Conf. Design Automation, 2001.
[42] M.S. Gupta, “Fluctuations and Dissipation in Elementary One-Bit Information Storage System,” Int'l J. Theoretical Physics, vol. 21, nos. 3/4, pp. 275-282, Apr. 1982.
[43] K. Palem, S. Cheemalavagu, and P. Korkmaz, “Introverted Switches as a Basis for Minimizing Leakage in CMOS,” US patent invention disclosure, in preparation.
[44] V. Tiwari, S. Malik, and P. Ashar, “Guarded Evaluation: Pushing Power Management in Logic Synthesis/Design,” IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, vol. 17, Oct. 1998.
[45] A.J. Martin, M. Nystrom, and C.G. Wong, “Tutorial: Asynchronous Microprocessor Design,” Proc. 35th Int'l Symp. Microarchitecture, Nov. 2003.
[46] B. Kish, “End of Moore's Law: Thermal (Noise) Death of Integration in Micro and Nano Electronics,” Physics Letters A, vol. 305, 2002.
[47] K.V. Palem, S. Cheemalavagu, P. Korkmaz, and B.E.S. Akgul, “Probabilistic and Introverted Switching to Conserve Energy in a Digital System,” US Patent application, 27 Apr. 2005.

Index Terms:
Index Terms- Energy-aware systems, low-power design, probabilistic computation.
Citation:
Krishna V. Palem, "Energy Aware Computing through Probabilistic Switching: A Study of Limits," IEEE Transactions on Computers, vol. 54, no. 9, pp. 1123-1137, Sept. 2005, doi:10.1109/TC.2005.145
Usage of this product signifies your acceptance of the Terms of Use.