This Article 
 Bibliographic References 
 Add to: 
Memory MISER: Improving Main Memory Energy Efficiency in Servers
March 2009 (vol. 58 no. 3)
pp. 336-350
Matthew E. Tolentino, Virginia Polytechnic and State University, Blacksburg
Joseph Turner, Virginia Polytechnic and State University, Blacksburg
Kirk W. Cameron, Virginia Polytechnic and State University, Blacksburg
Main memory power in volume and mid-range servers is growing as a fraction of total system power. The resulting energy consumption increases system cost and the heat produced reduces reliability. Emergent memory technology will provide systems with the ability to dynamically turn-on (online) and turn-off (offline) memory devices at runtime. This technology, coupled with slack in memory demand, offers the potential for significant energy savings in servers. However, to gain general acceptance in the server community, power-aware techniques must maintain performance and scale to thousands of memory devices. We propose a Memory Management Infra-Structure for Energy Reduction (Memory MISER) that is transparent, performance-neutral, and scalable. Memory MISER provides: 1) a prototype Linux kernel that manages memory at device granularity, and 2) a userspace daemon that tracks systemic memory demand and implements energy- and performance-constrained device controller policies. Experiments on an 8-node cluster of servers show our Memory MISER conserves memory energy up to 56.8% with no performance degradation for scientific codes that utilized the entire cluster. For multi-user workloads, we achieved memory energy savings of up to 67.94% with no performance degradation. Normalizing to total system energy consumption, our power-aware memory approach reduced energy between 18.81% and 39.02%.

[1] K.J. Astrom and B. Wittenmark, Adaptive Control. Addison-Wesley, 1995.
[2] D.H. Bailey, “Performance of Future High-End Computers,” DOE Mission Computing Conf., 2003.
[3] R. Bianchini and R. Rajamony, “Power and Energy Management for Server Systems,” IEEE Computer, vol. 37, pp. 68-74, 2004.
[4] K.W. Cameron, X. Feng, and R. Ge, “Performance- and Energy-Conscious Distributed DVS Scheduling for Scientific Applications on Power-Aware Clusters,” Proc. IEEE/ACM Supercomputing (SC), 2005.
[5] K.W. Cameron, R. Ge, and X. Feng, “High-Performance, Power-Aware Distributed Computing for Scientific Applications,” IEEE Computer, vol. 38, 2005.
[6] V. Delaluz, M. Kandemir, N. Vijaykrishnan, A. Sivasubramaniam, and M.J. Irwin, “Hardware and Software Techniques for Controlling DRAM Power Modes,” IEEE Trans. Computers, vol. 50, no.11 pp.1154-1173, Nov. 2001.
[7] S. Chandra and A. Vahdat, “Application-Specific Network Management for Energy-Aware Streaming of Popular Multimedia Formats,” Proc. USENIX, 2002.
[8] G. Chen, K. Malkowski, M. Kandemir, and P. Raghavan, “Reducing Power with Performance Constraints for Parallel Sparse Applications,” Proc. First Workshop High-Performance, Power-Aware Computing (HP-PAC), 2005.
[9] V. Delaluz, M. Kandemir, N. Vijaykrishnan, A. Sivasubramaniam, and M.J. Irwin, “DRAM Energy Management Using Software and Hardware Directed Power Mode Control,” Proc. Seventh Int'l Symp. High Performance Computer Architecture (HPCA), 2001.
[10] V. Delaluz, A. Sivasubramaniam, M. Kandemir, N. Vijaykrishnan, and M.J. Irwin, “Scheduler Based DRAM Energy Management,” Proc. 39th Design Automation Conf. (DAC), 2002.
[11] The Eco nomist, Computing Going Green, Mar. 2007.
[12] X. Fan, C.S. Ellis, and A.R. Lebeck, “Memory Controller Policies for DRAM Power Management,” Proc. Int'l Symp. Low-Power Electronics and Design (ISPLED), 2001.
[13] W. Feng and C. Hsu, “Green Destiny and Its Evolving Parts,” Proc. 19th Int'l Supercomputer Conf., 2004.
[14] X. Feng, R. Ge, and K.W. Cameron, “ARGUS: Supercomputing in 1/10 Cubic Meter,” Proc. Parallel and Distributed Computing and Networks (PDCN), 2005.
[15] V. Freeh and D.K. Lowenthal, “Using Multiple Energy Gears in MPI Programs on a Power-Scalable Cluster,” Proc. 10th ACM Symp. Principles and Practice of Parallel Programming (PPOPP), 2005.
[16] R. Ge, X. Feng, and K.W. Cameron, “Improvement of Power-Performance Efficiency for High-End Computing,” Proc. 19th IEEE/ACM Int'l Parallel and Distributed Processing Symp. (IPDPS), 2005.
[17] J. Haas and P. Vogt, “Fully-Buffered DIMM Technology Moves Enterprise Platforms to the Next Level,” Technology@Intel Magazine, vol. 3, 2005.
[18] G. Grohoski, “Niagra 2: A Highly-Threaded Server-On-A-Chip,” Proc. HotChips '06, vol. 18, 2006.
[19] J.L. Henning, “SPEC CPU2006 Memory Footprint,” Computer Architecture News, vol. 35, no. 1, pp. 84-89, 2007.
[20] C.V. Hollot, V. Misra, D. Towsley, and W.-B. Gong, “A Control-Theoretic Analysis of RED,” Proc. IEEE INFOCOM, 2001.
[21] H. Huang, P. Pillai, and K. Shin, “Design and Implementation of Power-Aware Virtual Memory,” Proc. Fourth Int'l Workshop Power-Aware Computer Systems (PACS), 2005.
[22] H. Huang, K. Shin, C. Lefurgy, K. Rajamani, T. Keller, E.V. Hensbergen, and F. Rawson, “Cooperative Software-Hardware Power Management for Main Memory,” Proc. USENIX Ann. Technical Conf., 2003.
[23] C.-H. Hsu and U. Kremer, Compiler-Directed Dynamic Voltage Scaling for Memory-Bound Applications. Dept. Computer Science, Rutgers Univ., 2002.
[24] N. Kappiah, V.W. Freeh, D.K. Lowenthal, and F. Pan, “Exploiting Slack Time in Power-Aware, High-Performance Programs,” Proc. IEEE/ACM Supercomputing (SC '05), Nov. 2005.
[25] J.G. Koomey, Estimating Total Power Consumption by Servers in the US and the World, final report, 2007.
[26] A.R. Lebeck, X. Fan, H. Zeng, and C. Ellis, “Power Aware Page Allocation,” Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2000.
[27] C. Lefurgy, K. Rajamani, F. Rawson, W. Felter, M. Kistler, and T. Keller, “Energy Management for Commercial Servers,” IEEE Computer, vol. 36, pp. 39-48, 2003.
[28] X. Li, Z. Li, F. Danvid, Y. Zhou, and S. Kumar, “Performance Directed Energy Management for Main Memory and Disks,” Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2004.
[29] N. Magen, A. Kolodny, U. Weiser, and N. Shamir, “Interconnect-Power Dissipation in a Microprocessor,” Proc. Int'l Workshop System-Level Interconnect Prediction (SLIP), 2004.
[30] V. Misra, W.-B. Gong, and D. Towsley, “Fluid-Based Analysis of a Network of AQM Routers Supporting TCP Flows with an Application to RED,” Proc. ACM SIGCOMM, 2000.
[31] V. Pandey, W. Jiang, Y. Zhou, and R. Bianchini, “DMA-Aware Memory Energy Management,” Proc. Int'l Symp. High-Performance Computer Architecture (HPCA), 2006.
[32] Rambus Inc., RDRAM, http:/, 1999.
[33] K. Skadron, T. Abdelzaher, and M.R. Stan, “Control-Theoretic Techniques and Thermal-RC Modeling for Accurate and LocalizedDynamic Thermal Management,” Proc. Int'l Symp. High-Performance Computer Architecture (HPCA), 2002.
[34] M.E. Tolentino, J. Turner, and K.W. Cameron, “An Implementation of Page Allocation Shaping for Energy Efficiency,” Proc. Third Workshop High-Performance, Power-Aware Computing (HP-PAC), 2007.
[35] M.E. Tolentino, “Flexible Operating System Structure for Dynamic Memory Management,” Computer Science, vol. 88, MS thesis, Univ. of Washington, 2004.
[36] M.E. Tolentino, J. Turner, and K.W. Cameron, “Memory-MISER: A Performance-Constrained Runtime System for Power-Scalable Clusters,” Proc. ACM Int'l Conf. Computing Frontiers, 2007.
[37] Q. Wu, P. Juang, M. Martonosi, and D.W. Clark, “Formal Online Methods for Voltage/Frequency Control in Multiple Clock Domain Multiprocessors,” Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2004.
[38] Q. Zhu, Z. Chen, L. Tan, Y. Zhou, K. Keeton, and J. Wilkes, “Hibernator: Helping Disk Arrays Sleep Through the Winter,” Proc. 20th ACM Symp. Operating Systems Principles (SOSP), 2005.
[39] Q. Zhu and Y. Zhou, “Power Aware Storage Cache Management,” IEEE Trans. Computers, vol. 54, no. 5, May 2005.

Index Terms:
Energy-aware systems, Allocation/deallocation strategies, Main memory, Performance
Matthew E. Tolentino, Joseph Turner, Kirk W. Cameron, "Memory MISER: Improving Main Memory Energy Efficiency in Servers," IEEE Transactions on Computers, vol. 58, no. 3, pp. 336-350, March 2009, doi:10.1109/TC.2008.177
Usage of this product signifies your acceptance of the Terms of Use.