This Article 
 Bibliographic References 
 Add to: 
MIMO Power Control for High-Density Servers in an Enclosure
October 2010 (vol. 21 no. 10)
pp. 1412-1426
Xiaorui Wang, University of Tennessee, Knoxville
Ming Chen, University of Tennessee, Knoxville
Xing Fu, University of Tennessee, Knoxville
Power control is becoming a key challenge for effectively operating a modern data center. In addition to reducing operating costs, precisely controlling power consumption is an essential way to avoid system failures caused by power capacity overload or overheating due to increasing high server density. Control-theoretic techniques have recently shown a lot of promise for power management because of their better control performance and theoretical guarantees on control accuracy and system stability. However, existing work oversimplifies the problem by controlling a single server independently from others. As a result, at the enclosure level where multiple high-density servers are correlated by common workloads and share common power supplies, power cannot be shared to improve application performance. In this paper, we propose an enclosure-level power controller that shifts power among servers based on their performance needs, while controlling the total power of the enclosure to be lower than a constraint. Our controller features a rigorous design based on an optimal Multi-Input-Multi-Output (MIMO) control theory. We present detailed control problem formulation and transformation to a standard constrained least-squares problem, as well as stability analysis in the face of significant workload variations. We then conduct extensive experiments on a physical testbed to compare our controller with three state-of-the-art controllers: a heuristic-based MIMO control solution, a Single-Input-Single-Output (SISO) control solution, and an improved SISO controller with simple power shifting among servers. Our empirical results demonstrate that our controller outperforms all the three baselines by having more accurate power control and up to 11.8 percent better benchmark performance.

[1] X. Wang and M. Chen, "Cluster-Level Feedback Power Control for Performance Optimization," Proc. 14th IEEE Int'l Symp. High-Performance Computer Architecture (HPCA), 2008.
[2] R. Bianchini and R. Rajamony, "Power and Energy Management for Server Systems," Computer, vol. 37, no. 11, pp. 68-74, Nov. 2004.
[3] C. Patel, C. Bash, R. Sharma, M. Beitelmal, and R. Friedrich, "Smart Cooling of Data Centers," Proc. ASME Int'l Electronic Packaging Technical Conf. and Exhibition (InterPACK), July 2003.
[4] R.J. Minerick, V.W. Freeh, and P.M. Kogge, "Dynamic Power Management Using Feedback," Proc. Workshop Compilers and Operating Systems for Low Power (COLP), Sept. 2002.
[5] C. Lefurgy, X. Wang, and M. Ware, "Power Capping: A Prelude to Power Shifting," Cluster Computing, vol. 11, no. 2, pp. 183-195, 2008.
[6] V. Delaluz, M.T. Kandemir, N. Vijaykrishnan, A. Sivasubramaniam, and M.J. Irwin, "DRAM Energy Management Using Software and Hardware Directed Power Mode Control," Proc. Seventh Int'l Symp. High-Performance Computer Architecture (HPCA), 2001.
[7] W. Felter, K. Rajamani, T. Keller, and C. Rusu, "A Performance-Conserving Approach for Reducing Peak Power Consumption in Server Systems," Proc. 19th Ann. Int'l Conf. Supercomputing (ICS), 2005.
[8] S. Gurumurthi, A. Sivasubramaniam, M. Kandemir, and H. Franke, "DRPM: Dynamic Speed Control for Power Management in Server Class Disks," Proc. Int'l Symp. Computer Architecture (ISCA), 2003.
[9] K. Skadron, T. Abdelzaher, and M.R. Stan, "Control-Theoretic Techniques and Thermal-RC Modeling for Accurate and Localized Dynamic Thermal Management," Proc. Eighth Int'l Symp. High-Performance Computer Architecture (HPCA), 2002.
[10] Q. Wu, P. Juang, M. Martonosi, L.-S. Peh, and D.W. Clark, "Formal Control Techniques for Power-Performance Management," IEEE Micro, vol. 25, no. 5, pp. 52-62, Sept. 2005.
[11] J.L. Hellerstein, Y. Diao, S. Parekh, and D.M. Tilbury, Feedback Control of Computing Systems. John Wiley & Sons, 2004.
[12] Y. Diao, J.L. Hellerstein, S. Parekh, H. Shaikh, and M. Surendra, "Controlling Quality of Service in Multi-Tier Web Applications," Proc. 26th IEEE Int'l Conf. Distributed Computing Systems (ICDCS), 2006.
[13] P. Ranganathan, P. Leech, D. Irwin, and J.S. Chase, "Ensemble-Level Power Management for Dense Blade Servers," Proc. Int'l Symp. Computer Architecture (ISCA), 2006.
[14] P. Bohrer, E.N. Elnozahy, T. Keller, M. Kistler, C. Lefurgy, C. McDowell, and R. Rajamony, "The Case for Power Management in Web Servers," Power Aware Computing, Kluwer Academic Publishers, 2002.
[15] G.F. Franklin, J.D. Powell, and M. Workman, Digital Control of Dynamic Systems, third ed., Addison-Wesley, 1997.
[16] J.M. Maciejowski, Predictive Control with Constraints. Prentice Hall, 2002.
[17] P.E. Gill, W. Murray, and M.H. Wright, Practical Optimization. Academic Press, 1981.
[18] C. Lu, X. Wang, and X. Koutsoukos, "Feedback Utilization Control in Distributed Real-Time Systems with End-to-End Tasks," IEEE Trans. Parallel and Distributed Systems, vol. 16, no. 6, pp. 550-561, June 2005.
[19] F.L. Lewis and V.L. Syrmos, Optimal Control, second ed., John Wiley & Sons, 1995.
[20] P. Tondel, T. Johansen, and A. Bemporad, "An Algorithm for Multi-Parametric Quadratic Programming and Explicit MPC Solutions," Proc. IEEE Conf. Decision and Control, 2001.
[21] Electronic Educational Devices Inc., "Watts Up Pro Power Meter," http:/, 2010.
[22] Netperf: A Network Performance Benchmark, ed. B, Information Networks Division, Hewlett-Packard, Cupertino, CA, Mar. 1993.
[23] AMD, BIOS and Kernel Developer's Guide for AMD Athlon 64 and AMD Opteron Processors, Revision 3.30, white paper Publication 26094, Advanced Micro Devices, Inc., Feb. 2006.
[24] W. Kim, M. Gupta, G.-Y. Wei, and D. Brooks, "System Level Analysis of Fast, Per-Core dvfs Using On-Chip Switching Regulators," Proc. 14th Int'l Symp. High-Performance Computer Architecture (HPCA), 2008.
[25] R. Raghavendra, P. Ranganathan, V. Talwar, Z. Wang, and X. Zhu, "No Power Struggles: Coordinated Multi-Level Power Management for the Data Center," Proc. Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2008.
[26] X. Wang, M. Chen, C. Lefurgy, and T. Keller, "Ship: Scalable Hierarchical Power Control for Large-Scale Data Centers," Proc. 18th Int'l Conf. Parallel Architectures and Compilation Techniques (PACT), 2009.
[27] Innovative Computing Laboratory, Univ. of Tennessee, "HPL—A Portable Implementation of the High-Performance Linpack Benchmark for Distributed-Memory Computers," http://www.netlib. org/benchmarkhpl/, 2010.
[28] C. Lefurgy, K. Rajamani, F. Rawson, W. Felter, M. Kistler, and T.W. Keller, "Energy Management for Commercial Servers," Computer, vol. 36, no. 12, pp. 39-48, Dec. 2003.
[29] H. Zeng, C.S. Ellis, A.R. Lebeck, and A. Vahdat, "ECOSystem: Managing Energy as a First Class Operating System Resource," Proc. Tenth Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS X), 2002.
[30] Y.-H. Lu, L. Benini, and G.D. Micheli, "Operating-System Directed Power Reduction," Proc. Int'l Symp. Low Power Electronics and Design (ISLPED), 2000.
[31] X. Fan, W.-D. Weber, and L.A. Barroso, "Power Provisioning for a Warehouse-Sized Computer," Proc. Int'l Symp. Computer Architecture (ISCA), 2007.
[32] D. Brooks and M. Martonosi, "Dynamic Thermal Management for High-Performance Microprocessors," Proc. Seventh Int'l Symp. High-Performance Computer Architecture (HPCA), 2001.
[33] B. Diniz, D. Guedes, W. MeiraJr., and R. Bianchini, "Limiting the Power Consumption of Main Memory," Proc. Int'l Symp. Computer Architecture (ISCA), 2007.
[34] D. Meisner, B.T. Gold, and T.F. Wenisch, "Powernap: Eliminating Server Idle Power," Proc. 14th Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS), 2009.
[35] T.F. Abdelzaher, J. Stankovic, C. Lu, R. Zhang, and Y. Lu, "Feedback Performance Control in Software Services," IEEE Control Systems, vol. 23, no. 3, pp. 74-90, Jun. 2003.
[36] D.C. Steere, A. Goel, J. Gruenberg, D. McNamee, C. Pu, and J. Walpole, "A Feedback-Driven Proportion Allocator for Real-Rate Scheduling," Proc. Third Symp. Operating Systems Design and Implementation (OSDI), 1999.
[37] X. Wang, D. Jia, C. Lu, and X. Koutsoukos, "DEUCON: Decentralized End-to-End Utilization Control for Distributed Real-Time Systems," IEEE Trans. Parallel and Distributed Systems, vol. 18, no. 7, pp. 996-1009, July 2007.
[38] M. Karlsson, C.T. Karamanolis, and X. Zhu, "Triage: Performance Differentiation for Storage Systems Using Adaptive Control," ACM Trans. Storage, vol. 1, no. 4, pp. 457-480, 2005.
[39] S. Keshav, "A Control-Theoretic Approach to Flow Control," Proc. ACM SIGCOMM, 1991.
[40] M.E. Femal and V.W. Freeh, "Boosting Data Center Performance through Non-Uniform Power Allocation," Proc. Second Int'l Conf. Autonomic Computing (ICAC), 2005.
[41] Z. Wang, C. McCarthy, X. Zhu, P. Ranganathan, and V. Talwar, "Feedback Control Algorithms for Power Management of Servers," Proc. Third Int'l Workshop Feedback Control Implementation and Design in Computing Systems and Networks (FeBID), 2008.
[42] Y. Zhu and F. Mueller, "Feedback EDF Scheduling Exploiting Dynamic Voltage Scaling," Proc. Tenth IEEE Real-Time and Embedded Technology and Applications Symp. (RTAS), 2004.
[43] V. Sharma, A. Thomas, T. Abdelzaher, K. Skadron, and Z. Lu, "Power-Aware QoS Management in Web Servers," Proc. IEEE Int'l Real-Time Systems Symp. (RTSS), 2003.
[44] Y. Chen, A. Das, W. Qin, A. Sivasubramaniam, Q. Wang, and N. Gautam, "Managing Server Energy and Operational Costs in Hosting Centers," Proc. ACM SIGMETRICS, 2005.

Index Terms:
Power control, power capping, power management, servers, data centers, feedback control.
Xiaorui Wang, Ming Chen, Xing Fu, "MIMO Power Control for High-Density Servers in an Enclosure," IEEE Transactions on Parallel and Distributed Systems, vol. 21, no. 10, pp. 1412-1426, Oct. 2010, doi:10.1109/TPDS.2010.31
Usage of this product signifies your acceptance of the Terms of Use.