This Article 
 Bibliographic References 
 Add to: 
A CFD-Based Tool for Studying Temperature in Rack-Mounted Servers
August 2008 (vol. 57 no. 8)
pp. 1129-1142
Jeonghwan Choi, Penn State University, University Park
Youngjae Kim, Penn State University, University Park
Anand Sivasubramaniam, Penn State University, University Park
Jelena Srebric, Penn State University, University Park
Qian Wang, Penn State University, University Park
Joonwon Lee, Korea Advanced Institute of Science and Technology, Daejeon
Temperature-aware computing is becoming more important in design of computer systems as power densities are increasing and the implications of high operating temperatures result in higher failure rates of components and increased demand for cooling capability. Computer architects and system software designers need to understand the thermal consequences of their proposals, and develop techniques to lower operating temperatures to reduce both transient and permanent component failures. Recognizing the need for thermal modeling tools to support those researches, there has been recent work on modeling temperatures of processors at the micro-architectural level which can be easily understood and employed by computer architects for processor designs. However, there is a dearth of such tools in the academic/research community for undertaking architectural/systems studies beyond a processor - a server box, rack or even a machine room. In this paper we presents a detailed 3-dimensional Computational Fluid Dynamics based thermal modeling tool, called ThermoStat, for rack-mounted server systems. We conduct several experiments with this tool to show how different load conditions affect the thermal profile, and also illustrate how this tool can help design dynamic thermal management techniques. We propose reactive and proactive thermal management for rack mounted server and Isothermal workload distribution for rack.

[1] S. Gurumurthi, A. Sivasubramaniam, and V. Natarajan, “Disk Drive Roadmap from the Thermal Perspective: A Case for Dynamic Thermal Management,” Proc. 32nd IEEE Int'l Symp. Computer Architecture, pp. 38-49, June 2005.
[2] S. Charrap, P. Lu, and Y. He, “Thermal Stability of Recorded Information at High Densities,” IEEE Trans. Magnetics, vol. 33, no. 1, pp. 978-983, Jan. 1997.
[3] L. Yeh and R. Chy, Thermal Management of Microelectronic Equipment. Am. Soc. of Mechanical Eng., 2001.
[4] S. Gunther, F. Binns, D. Carmean, and J. Hall, “Managing the Impact of Increasing Microprocessor Power Consumption,” Intel Technology J., vol. 5, Feb. 2001.
[5] H.F. Hamann, A. Weger, J. Lacey, Z. Hu, P. Bose, E. Cohen, and J. Wakil, “Hotspot-Limited Microprocessors: Direct Temperature and Power Distribution Measurements,” IEEE J. Solid-State Circuits, vol. 42, pp. 56-65, Jan. 2007.
[6] K. Skadron, M. Stan, W. Huang, S. Velusamy, K. Sankaranarayanan, and D. Tarjan, “Temperature-Aware Microarchitecture,” Proc. 30th IEEE Int'l Symp. Computer Architecture, pp. 1-13, June 2003.
[7] A. Weissel and F. Bellosa, “Dynamic Thermal Management for Distributed Systems,” Proc. First Workshop Temperature-Aware Computer Systems, June 2004.
[8] Y. Li, K. Skadron, Z. Hu, and D. Brooks, “Performance, Energy, and Thermal Considerations for SMT and CMP Architectures,” Proc. 11th Int'l Symp. High-Performance Computer Architecture, pp.71-82, Feb. 2005.
[9] J. Srinivasan and S. Adve, “Predictive Dynamic Thermal Management for Multimedia Applications,” Proc. 17th Int'l Conf. Supercomputing, pp. 109-120, June 2003.
[10] J. Hasan, A. Jalote, T.N. Vijaykumar, and C. Brodle, “Heat Stroke: Power-Density-Based Denial of Service in SMT,” Proc. 11th Int'l Symp. High-Performance Computer Architecture, pp. 166-177, 2005.
[11] Y. Kim, S. Gurumurthi, and A. Sivasubramaniam, “Understanding the Performance-Temperature Interactions in Disk I/O of Server Workloads,” Proc. 12th Int'l Symp. High-Performance Computer Architecture, pp. 179-189, Feb. 2006.
[12] T. Heath, A.P. Centeno, P. George, Y. Jaluria, and R. Bianchini, “Mercury and Freon: Temperature Emulation and Management in Server Systems,” Proc. 12th Int'l Conf. Architectural Support for Programming Languages and Operating Systems, Oct. 2006.
[13] D. Brooks, V. Tiwari, and M. Martonosi, “Wattch: A Framework for Architectural-Level Power Analysis and Optimizations,” Proc. 27th Int'l Symp. Computer Architecture, pp. 83-94, June 2000.
[14] W. Chen, M. Dubois, and P. Stenstrom, “Integrating Complete-System and User-level Performance/Power Simulators: The SimWattch Approach,” Proc. IEEE Int'l Symp. Performance Analysis of Systems and Software, 2003.
[15] D. Kudithipudi, S. Petko, and E. John, “Cache Leakage Power Analysis in Embedded Applications,” Proc. 47th IEEE Midwest Symp. Circuits and Systems, pp. II517-II520, 2004.
[16] R. Bianchini and R. Rajamony, “Power and Energy Management for Server Systems,” Computer, vol. 37, no. 11, Nov. 2004.
[17] J.S. Chase, D.C. Anderson, P.N. Thakar, A.M. Vahdat, and R.P. Doyle, “Managing Energy and Server Resources in Hosting Centers,” Proc. 18th ACM Symp. Operating System Principles, Oct. 2001.
[18] F. Bellosa, S. Kellner, M. Waitz, and A. Weissel, “Event-Driven Energy Accounting of Dynamic Thermal Management,” Proc. Workshop Compilers and Operating Systems for Low Power, Sept. 2003.
[19] R. Huang and D. Chung, “Thermal Design of a Disk-Array System,” Proc. Eighth InterSociety Conf. Thermal and Thermomechanical Phenomena in Electronic Systems, pp. 106-112, May 2002.
[20] D. Brooks and M. Martonosi, “Dynamic Thermal Management for High-Performance Microprocessors,” Proc. Seventh Int'l Symp. High-Performance Computer Architecture, pp. 171-182, Jan. 2001.
[21] M. Gomaa, M.D. Powel, and T.N. Vijaykumar, “Heat-and-Run: Leveraging SMT and CMP to Manage Power Density through the Operating System,” Proc. 11th Int'l Conf. Architectural Support for Programming Languages and Operating Systems, pp. 260-270, 2004.
[22] L. Shang, L.-S. Peh, A. Kumar, and N. Jha, “Thermal Modeling, Characterization and Management of On-Chip Networks,” Proc. 37th IEEE Int'l Symp. Microarchitecture, pp. 67-78, Dec. 2004.
[23] S. Gurumurthi, A. Sivasubramaniam, M. Kandemir, and H. Franke, “DRPM: Dynamic Speed Control for Power Management in Server Class Disks,” Proc. 30th IEEE Int'l Symp. Computer Architecture, pp. 169-179, June 2003.
[24] E. Pinheiro and R. Bianchini, “Energy Conservation Techniques for Disk Array-Based Servers,” Proc. 18th IEEE Int'l Conf. Supercomputing (ICS '04), June 2004.
[25] Q. Zhu, F. David, C. Devraj, Z. Li, Y. Zhou, and P. Cao, “Reducing Energy Consumption of Disk Storage Using Power-Aware Cache Management,” Proc. 10th Int'l Symp. High-Performance Computer Architecture, 2004.
[26] N. Rasmussen, “Guidelines for Specification of Data Center Power Density,” APC white paper, 2004.
[27] R.K. Sharma, C.E. Bash, and C.D. Patel, “Dimensionless Parameters for Evaluation of Thermal Design and Performance of Large-Scale Data Centers,” Proc. Eighth ASME/AIAA Joint Thermophysics and Heat Transfer Conf., June 2002.
[28] C.E. Bash, C.D. Patel, and P.K. Sharma, “Efficient Thermal Management of Data Center: Immediate and Long-Term Research Needs,” Int'l J. Heat, Ventilating, Air-Conditioning and Refrigeration Research Needs, vol. 9, no. 2, pp. 137-152, 2003.
[29] P. Rodgers and V. Eveloy, “Prediction of Microelectronics Thermal Behavior in Electronic Equipment: Status, Challenges and Future Requirements,” Proc. Ninth Int'l Conf. Thermal and Mechanical Simulation and Experiments in Micro-Electronics and Micro-Systems, 2003.
[30] J.F. Karlsson and B. Moshfegh, “Investigation of Indoor Climate and Power Usage in a Data Center,” Energy and Buildings, vol. 37, pp. 1075-1083, 2005.
[31] C.D. Patel, R. Sharma, C.E. Bash, and A. Beitelmal, “Thermal Considerations in Cooling Large-Scale High Compute Density Data Centers,” Proc. Eighth Intersoc. Conf. Thermal and Thermomechanical Phenomena in Electronic Systems, May 2002.
[32] C.D. Patel, C.E. Bash, C. Belady, L. Stahl, and D. Sullivan, “Computational Fluid Dynamics Modeling of High Compute Density Data Centers to Assure System Inlet Air Specifications,” Proc. Pacific RIM/ASME Int'l Electronic Packaging Technical Conf. and Exhibition, July 2001.
[33] C.D. Patel, C.E. Bash, and M. Beitelmal, “Smart Cooling of Data Centers,” Proc. Pacific RIM/ASME Int'l Electronics Packaging Technical Conf. and Exhibition, July 2003.
[34] M.H. Beitelmal and C.D. Patel, “Thermo-Fluids Provisioning of a High Performance High Density Data Center,” Technical Report HPL 2004-146 (R.1), Hewlett-Packard Laboratories, 2004.
[35] C.D. Patel and A.J. Shah, “Cost Model for Planning, Development and Operation of a Data Center,” Technical Report HPL 2005-107 (R.1), Hewlett-Packard Laboratories, 2005.
[36] S.V. Patankar and K.C. Karki, “Distribution of Cooling Airflow in a Raised-Floor Data Center,” Proc. Am. Soc. Heating, Refrigerating, and Air-Conditioning Eng., 2004.
[37] J. Moore, J. Chase, P. Ranganathan, and R. Sharma, “Making Scheduling 'Cool': Temperature-Aware Workload Placement in Data Centers,” Proc. Usenix Ann. Technical Conf., Apr. 2005.
[38] J. Moore, R. Sharma, R. Shih, J. Chase, C.D. Patel, and P. Ranganathan, “Going Beyond CPUs: The Potential of Temperature-Aware Solutions for the Data Center,” Proc. First Workshop Temperature-Aware Computer Systems , May 2002.
[39] R. Sharma, C. Bash, C. Patel, R. Friedrich, and J. Chase, “Balance of Power: Dynamic Thermal Management for Internet Data Centers,” IEEE Internet Computing, vol. 9, no. 1, pp. 42-49, Jan./Feb. 2005.
[40] D. Agonafer, L. Gan-Li, and D.B. Spalding, “LVEL Turbulence Model for Conjugate Heat Transfer at Low Reynolds Numbers,” Am. Soc. Mechanical Eng., EEP, Application of CAE/CAD to Electronic Systems, 1996.
[41] K.K. Dhinsa, C.J. Bailey, and K.A. Pericleous, “Turbulence Modelling and Its Impact on CFD Predictions for Cooling of Electronic Components,” Proc. Ninth Intersoc. Conf. Thermal and Thermomechanical Phenomena in Electronic Systems, 2004.
[42] M.K. Patterson, X. Wei, and Y. Joshi, “Use of Computational Fluid Dynamics in the Design and Optimization of Microchannel Heat Exchangers for Microelectronics Cooling,” Proc. ASME Summer Heat Transfer Conf., 2005.
[43] G. Xiong, M. Lu, C.L. Chen, B.P. Wang, and D. Kehl, “Numerical Optimization of a Power Electronics Cooling Assembly,” Proc. 16th IEEE Applied Power Electronics Conf. and Exposition, 2001.
[44] J. Choi, Y. Kim, A. Sivasubramaniam, J. Srebric, Q. Wang, and J. Lee, “Modeling and Managing Thermal Proles of Rack-Mounted Servers with ThermoStat,” Proc. 13th Int'l Symp. High-Performance Computer Architecture, Feb. 2007.
[45] “Intel P4 Power Measure,” http:/, 2006.
[46] “Intel Xeon Processor,”, 2008.
[47] “Power Supply,” http:/, Summary of Rationale for Version 1.0 ENERGY STAR External Power Supply (EPS) Specification Sept. 2005, 2008.
[48] “Phoenics User Manual for Program Version 3.6.,” CHAM, http:/, 2008.
[49] “Thermal Sensor DS18B20,” http:/, 2008.
[50] “SPCR's Unique Heatsink Testing Methodology,” http://www. silentpcreview.comSections+index-req-printpage-artid-46.html , 2002.
[51] J. Srebric and Q. Chen, “An Example of Verification, Validation, and Reporting of Indoor Environment CFD Analyses,” ASHRAE Trans., vol. 108, no. 2, pp. 185-194, 2002.
[52] K. Dhinsa, C. Bailey, and K. Pericleous, “Investigation into the Performance of Turbulence Models for Fluid Flow and Heat Transfer Phenomena in Electronic Applications,” IEEE Trans. Components and Packaging Technologies, vol. 28, no. 4, pp. 686-699, Dec. 2005.
[53] Computer Systems Laboratory, Pennsylvania State Univ., http:/, 2008.
[54] M.J. Crippen et al., “BladeCenter Packaging, Power, and Cooling,” IBM J. Research and Development, vol. 49, no. 6, pp. 887-904, 2005.
[55] C.D. Patel, C.E. Bash, R. Sharma, A. Beitelmal, and C.G. Malone, “Smart Chip, System and Data Center Enabled by Advanced Flexible Cooling Resources,” Proc. 21st Ann. IEEE Semiconductor Thermal Measurement and Management Symp., pp. 78-85, 2005.
[56] S. Guggari, D. Agonafer, C. Belady, and L. Stahl, “A Hybrid Methodology for the Optimization of Data Center Room Layout,” Proc. Pacific Rim/ASME Int'l Electronic Packaging Technical Conf. and Exhibition, pp. 605-612, July 2003.

Index Terms:
Measurement, evaluation, modeling, simulation of multiple-processor systems, Energy-aware systems, Power Management
Jeonghwan Choi, Youngjae Kim, Anand Sivasubramaniam, Jelena Srebric, Qian Wang, Joonwon Lee, "A CFD-Based Tool for Studying Temperature in Rack-Mounted Servers," IEEE Transactions on Computers, vol. 57, no. 8, pp. 1129-1142, Aug. 2008, doi:10.1109/TC.2008.52
Usage of this product signifies your acceptance of the Terms of Use.